[rds-devel] Has anyone tried running RDS over 10GE / IWARP NICs ?
Richard Frank
richard.frank at oracle.com
Wed Apr 2 16:14:15 PDT 2008
Great - we are happy to help out.
Olaf Kirch is the RDS owner - so if you want the bare truth - Olaf is
the first stop !
Best would be to post everything to rds-devel..
Rick
Steve Wise wrote:
> Hi Richard,
>
> My team here at OGC will be enabling RDS on chelsio very soon.
> Hopefully this month or next. Jon Mason on my team will be doing
> this. I've Cced him. Our goal is to release any changes needed in a
> ofed-1.3.x point release.
>
> Can we email you if we have questions/issues related to the RDS
> implementation?
>
>
>
> Steve.
>
>
> Richard Frank wrote:
>> Steve, we would like to get RDS running over the Chelsio 10GE NICs .
>>
>> It looks like there is some sort of a problem - we are assuming it is
>> something simple - as RDS does not do much more than connect over an
>> RC and do send / recvs.
>>
>> Can you help us move this forward - either by looking at this
>> yourself and or putting in contact with folks that can ?
>>
>>
>> Scott Weitzenkamp (sweitzen) wrote:
>>> Chelsio S310E-SR+ (see
>>> http://www.chelsio.com/products_10g_adapters.html), I'm sure they would
>>> be happy to sell you a few to use.
>>>
>>> Perhaps RDS does not use RDMA CM in a way that iWARP needs?
>>>
>>> Scott
>>>
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Richard Frank [mailto:richard.frank at oracle.com] Sent:
>>>> Wednesday, April 02, 2008 11:52 AM
>>>> To: Scott Weitzenkamp (sweitzen)
>>>> Cc: rds-devel at oss.oracle.com
>>>> Subject: Re: [rds-devel] Has anyone tried running RDS over 10GE /
>>>> IWARP NICs ?
>>>>
>>>> Who's cards are these - perhaps they will want to get RDS running
>>>> and help out ?
>>>>
>>>> Scott Weitzenkamp (sweitzen) wrote:
>>>>
>>>>> Sorry, but I doubt I will have time to rebuild RDS and try this.
>>>>>
>>>>> Scott
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Richard Frank [mailto:richard.frank at oracle.com] Sent:
>>>>>> Wednesday, April 02, 2008 11:23 AM
>>>>>> To: Scott Weitzenkamp (sweitzen)
>>>>>> Cc: rds-devel at oss.oracle.com
>>>>>> Subject: Re: [rds-devel] Has anyone tried running RDS over 10GE /
>>>>>> IWARP NICs ?
>>>>>>
>>>>>> OK - well - I guess we are not too surprised...
>>>>>>
>>>>>> Short of us getting some these NICs to sort this out ourselves..
>>>>>>
>>>>>> Can you enable rds debug in configure.mk and send us the dmesg +
>>>>>> /var/log/messages
>>>>>>
>>>>>> You will need to unload/load rds after building it with debug..
>>>>>>
>>>>>> Scott Weitzenkamp (sweitzen) wrote:
>>>>>>
>>>>>>> Yes, it's an iWARP NIC, and the OFED 1.3 perftest
>>>>>> ib_rdma_lat program is
>>>>>>
>>>>>>> working.
>>>>>>>
>>>>>>> Scott
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Richard Frank [mailto:richard.frank at oracle.com] Sent:
>>>>>>>> Wednesday, April 02, 2008 11:04 AM
>>>>>>>> To: Scott Weitzenkamp (sweitzen)
>>>>>>>> Cc: rds-devel at oss.oracle.com; [ofa_general]
>>>>>>>> Subject: Re: [rds-devel] Has anyone tried running RDS over 10GE
>>>>>>>> / IWARP NICs ?
>>>>>>>>
>>>>>>>> RDS does not run over regular 10G NICs - that appear as
>>>>>>>>
>>>>>> simple NICS -
>>>>>>>> this was disabled in 1.3.
>>>>>>>>
>>>>>>>> For now we are interested in RDS over IWARP NICS -
>>>> configured as
>>>>>>>> accessible via the verbs interfaces.
>>>>>>>>
>>>>>>>> Richard Frank wrote:
>>>>>>>>
>>>>>>>>> is the rds driver loaded (modprobe rds)
>>>>>>>>>
>>>>>>>>> Scott Weitzenkamp (sweitzen) wrote:
>>>>>>>>>
>>>>>>>>>> Does't appear to work with Chelsio and OFED 1.3:
>>>>>>>>>>
>>>>>>>>>> [root at svbu-qa2950-1 counters]# ethtool -i eth2
>>>>>>>>>> driver: cxgb3
>>>>>>>>>> version: 1.0-ofed
>>>>>>>>>> firmware-version: T 5.0.0 TP 1.1.0
>>>>>>>>>> bus-info: 0000:0b:00.0
>>>>>>>>>> [root at svbu-qa2950-1 counters]# ifconfig eth2
>>>>>>>>>> eth2 Link encap:Ethernet HWaddr 00:07:43:05:43:9F
>>>>>>>>>> inet addr:192.168.0.198 Bcast:192.168.0.255
>>>>>>>>>> Mask:255.255.255.0
>>>>>>>>>> inet6 addr: fe80::207:43ff:fe05:439f/64 Scope:Link
>>>>>>>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>>>>>>>>> RX packets:144770 errors:0 dropped:0
>>>> overruns:0 frame:0
>>>>
>>>>>>>>>> TX packets:144781 errors:0 dropped:0
>>>>>>>>>>
>>>>>> overruns:0 carrier:0
>>>>>>
>>>>>>>>>> collisions:0 txqueuelen:1000
>>>>>>>>>> RX bytes:207891512 (198.2 MiB) TX bytes:9348152
>>>>>>>>>>
>>>>>>>> (8.9 MiB)
>>>>>>>>
>>>>>>>>>> Interrupt:169 Memory:fceff000-fcefffff
>>>>>>>>>>
>>>>>>>>>> [root at svbu-qa2950-1 counters]# rds-sink -s
>>>>>>>>>>
>>>>>> 192.168.0.198:22222 -i 1
>>>>>>
>>>>>>>>>> rds-sink: Unable to bind socket: Cannot assign
>>>> requested address
>>>>
>>>>>>>>>> Scott Weitzenkamp
>>>>>>>>>> SQA and Release Manager
>>>>>>>>>> Data Center Access Engineering
>>>>>>>>>> Cisco Systems
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: rds-devel-bounces at oss.oracle.com
>>>>>>>>>>> [mailto:rds-devel-bounces at oss.oracle.com] On Behalf Of
>>>>>>>>>>>
>>>>>>>> Richard Frank
>>>>>>>>
>>>>>>>>>>> Sent: Wednesday, April 02, 2008 10:31 AM
>>>>>>>>>>> To: rds-devel at oss.oracle.com; [ofa_general]
>>>>>>>>>>> Subject: [rds-devel] Has anyone tried running RDS over 10GE
>>>>>>>>>>> / IWARP NICs ?
>>>>>>>>>>>
>>>>>>>>>>> We'd appreciate some feed back on your experience and would
>>>>>>>>>>> like to sort out any issues ASAP.
>>>>>>>>>>>
>>>>>>>>>>> Rick
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> rds-devel mailing list
>>>>>>>>>>> rds-devel at oss.oracle.com
>>>>>>>>>>> http://oss.oracle.com/mailman/listinfo/rds-devel
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rds-devel mailing list
>>>>>>>>> rds-devel at oss.oracle.com
>>>>>>>>> http://oss.oracle.com/mailman/listinfo/rds-devel
>>>>>>>>>
More information about the rds-devel
mailing list