[rds-devel] RDS issue

[Karun Sharma (Contractor - GS Labs)]

Hi Andy:

Thanks for the response. 

I doubt if this is IB limitation. I tried the same with SDP and it is working fine for 16M message size. That also eliminates the possibility of a bug with qperf tool. So I suspect RDS only.

[root at nsgib92 ~]# qperf 192.168.200.84 -v -m 16M sdp_bw
sdp_bw:
    bw              =   738 MB/sec
    msg_rate        =    44 /sec
    msg_size        =    16 MiB (16,777,216)
    send_cost       =   359 ms/GB
    recv_cost       =  1.36 sec/GB
    send_cpus_used  =  26.5 % cpus
    recv_cpus_used  =   100 % cpus
[root at nsgib92 ~]# qperf 192.168.200.84 -v -m 17M sdp_bw
sdp_bw:
    bw              =   740 MB/sec
    msg_rate        =  41.5 /sec
    msg_size        =    17 MiB (17,825,792)
    send_cost       =   203 ms/GB
    recv_cost       =  1.35 sec/GB
    send_cpus_used  =    15 % cpus
    recv_cpus_used  =   100 % cpus
[root at nsgib92 ~]# qperf 192.168.200.84 -v -m 20M sdp_bw
sdp_bw:
    bw              =   734 MB/sec
    msg_rate        =  34.8 /sec
    msg_size        =    20 MiB (20,971,520)
    send_cost       =   204 ms/GB
    recv_cost       =  1.36 sec/GB
    send_cpus_used  =  14.9 % cpus
    recv_cpus_used  =   100 % cpus
[root at nsgib92 ~]# 

Thanks
Karun

________________________________________
From: Andy Grover [andy.grover at oracle.com]
Sent: Saturday, May 23, 2009 5:09 AM
To: Karun Sharma (Contractor - GS Labs)
Cc: rds-devel at oss.oracle.com
Subject: Re: [rds-devel] RDS issue

Hi Karun,

There is a message size limit of 1MB, but this is not enforced in the
code. This is bad and should be fixed, but your 16MB issue thus is not
an RDS bug :)

It's kind of curious that 16MB fails... could this be an IB limitation,
I wonder?

Regards -- Andy

Karun Sharma (Contractor - GS Labs) wrote:
> Hello:
>
> The issue I am facing is while running qperf with -m option for
> testing RDS . "-m" is used to specify message size. What I observed
> is that till "15M", it works fine but if I increase the message size
> to "16M", the test gives me 0 bytes/sec result. (see below)
>
> I checked the internet and OFED mailing list, but couldn't find
> anything which limits the message size. Is there any limit on message
> size or is it a bug? Any views?
>
> Thanks Karun
>
> --------------------------------------------------------------------------------------------
>  [root at nsgib84 ~]# qperf 192.168.200.92 -v -m 15M rds_bw rds_bw: bw
> =   354 MB/sec msg_rate        =  22.4 /sec msg_size        =    15
> MiB (15,728,640) send_cost       =  2.29 sec/GB recv_cost       =
> 1.05 sec/GB send_cpus_used  =  82.6 % cpus recv_cpus_used  =    37 %
> cpus [root at nsgib84 ~]# qperf 192.168.200.92 -v -m 16M rds_bw rds_bw:
> bw             =    0 bytes/sec msg_size       =   16 MiB
> (16,777,216) loc_cpus_used  =  100 % cpus rem_cpus_used  =  0.5 %
> cpus [root at nsgib84 ~]#
> -----------------------------------------------------------------------------------------------
>  _______________________________________________ rds-devel mailing
> list rds-devel at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/rds-devel