[rds-devel] Has anyone tried running RDS over 10GE / IWARP NICs ?
Richard Frank
richard.frank at oracle.com
Wed Apr 2 13:44:07 PDT 2008
Steve, we would like to get RDS running over the Chelsio 10GE NICs .
It looks like there is some sort of a problem - we are assuming it is
something simple - as RDS does not do much more than connect over an RC
and do send / recvs.
Can you help us move this forward - either by looking at this yourself
and or putting in contact with folks that can ?
Scott Weitzenkamp (sweitzen) wrote:
> Chelsio S310E-SR+ (see
> http://www.chelsio.com/products_10g_adapters.html), I'm sure they would
> be happy to sell you a few to use.
>
> Perhaps RDS does not use RDMA CM in a way that iWARP needs?
>
> Scott
>
>
>
>
>> -----Original Message-----
>> From: Richard Frank [mailto:richard.frank at oracle.com]
>> Sent: Wednesday, April 02, 2008 11:52 AM
>> To: Scott Weitzenkamp (sweitzen)
>> Cc: rds-devel at oss.oracle.com
>> Subject: Re: [rds-devel] Has anyone tried running RDS over
>> 10GE / IWARP NICs ?
>>
>> Who's cards are these - perhaps they will want to get RDS running and
>> help out ?
>>
>> Scott Weitzenkamp (sweitzen) wrote:
>>
>>> Sorry, but I doubt I will have time to rebuild RDS and try this.
>>>
>>> Scott
>>>
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Richard Frank [mailto:richard.frank at oracle.com]
>>>> Sent: Wednesday, April 02, 2008 11:23 AM
>>>> To: Scott Weitzenkamp (sweitzen)
>>>> Cc: rds-devel at oss.oracle.com
>>>> Subject: Re: [rds-devel] Has anyone tried running RDS over
>>>> 10GE / IWARP NICs ?
>>>>
>>>> OK - well - I guess we are not too surprised...
>>>>
>>>> Short of us getting some these NICs to sort this out ourselves..
>>>>
>>>> Can you enable rds debug in configure.mk and send us the dmesg +
>>>> /var/log/messages
>>>>
>>>> You will need to unload/load rds after building it with debug..
>>>>
>>>> Scott Weitzenkamp (sweitzen) wrote:
>>>>
>>>>
>>>>> Yes, it's an iWARP NIC, and the OFED 1.3 perftest
>>>>>
>>>>>
>>>> ib_rdma_lat program is
>>>>
>>>>
>>>>> working.
>>>>>
>>>>> Scott
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Richard Frank [mailto:richard.frank at oracle.com]
>>>>>> Sent: Wednesday, April 02, 2008 11:04 AM
>>>>>> To: Scott Weitzenkamp (sweitzen)
>>>>>> Cc: rds-devel at oss.oracle.com; [ofa_general]
>>>>>> Subject: Re: [rds-devel] Has anyone tried running RDS over
>>>>>> 10GE / IWARP NICs ?
>>>>>>
>>>>>> RDS does not run over regular 10G NICs - that appear as
>>>>>>
>>>>>>
>>>> simple NICS -
>>>>
>>>>
>>>>>> this was disabled in 1.3.
>>>>>>
>>>>>> For now we are interested in RDS over IWARP NICS -
>>>>>>
>> configured as
>>
>>>>>> accessible via the verbs interfaces.
>>>>>>
>>>>>> Richard Frank wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> is the rds driver loaded (modprobe rds)
>>>>>>>
>>>>>>> Scott Weitzenkamp (sweitzen) wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Does't appear to work with Chelsio and OFED 1.3:
>>>>>>>>
>>>>>>>> [root at svbu-qa2950-1 counters]# ethtool -i eth2
>>>>>>>> driver: cxgb3
>>>>>>>> version: 1.0-ofed
>>>>>>>> firmware-version: T 5.0.0 TP 1.1.0
>>>>>>>> bus-info: 0000:0b:00.0
>>>>>>>> [root at svbu-qa2950-1 counters]# ifconfig eth2
>>>>>>>> eth2 Link encap:Ethernet HWaddr 00:07:43:05:43:9F
>>>>>>>> inet addr:192.168.0.198 Bcast:192.168.0.255
>>>>>>>> Mask:255.255.255.0
>>>>>>>> inet6 addr: fe80::207:43ff:fe05:439f/64 Scope:Link
>>>>>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>>>>>>> RX packets:144770 errors:0 dropped:0
>>>>>>>>
>> overruns:0 frame:0
>>
>>>>>>>> TX packets:144781 errors:0 dropped:0
>>>>>>>>
>>>>>>>>
>>>> overruns:0 carrier:0
>>>>
>>>>
>>>>>>>> collisions:0 txqueuelen:1000
>>>>>>>> RX bytes:207891512 (198.2 MiB) TX bytes:9348152
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> (8.9 MiB)
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> Interrupt:169 Memory:fceff000-fcefffff
>>>>>>>>
>>>>>>>> [root at svbu-qa2950-1 counters]# rds-sink -s
>>>>>>>>
>>>>>>>>
>>>> 192.168.0.198:22222 -i 1
>>>>
>>>>
>>>>>>>> rds-sink: Unable to bind socket: Cannot assign
>>>>>>>>
>> requested address
>>
>>>>>>>> Scott Weitzenkamp
>>>>>>>> SQA and Release Manager
>>>>>>>> Data Center Access Engineering
>>>>>>>> Cisco Systems
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: rds-devel-bounces at oss.oracle.com
>>>>>>>>> [mailto:rds-devel-bounces at oss.oracle.com] On Behalf Of
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>> Richard Frank
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>> Sent: Wednesday, April 02, 2008 10:31 AM
>>>>>>>>> To: rds-devel at oss.oracle.com; [ofa_general]
>>>>>>>>> Subject: [rds-devel] Has anyone tried running RDS over 10GE /
>>>>>>>>> IWARP NICs ?
>>>>>>>>>
>>>>>>>>> We'd appreciate some feed back on your experience and would
>>>>>>>>> like to sort
>>>>>>>>> out any issues ASAP.
>>>>>>>>>
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rds-devel mailing list
>>>>>>>>> rds-devel at oss.oracle.com
>>>>>>>>> http://oss.oracle.com/mailman/listinfo/rds-devel
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rds-devel mailing list
>>>>>>> rds-devel at oss.oracle.com
>>>>>>> http://oss.oracle.com/mailman/listinfo/rds-devel
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
More information about the rds-devel
mailing list