[rds-devel] Re: [PATCH RFC RDS/IB] version 2. increase number of unsignaled work requests

[Richard Frank]

On Thu, 2007-01-25 at 11:25 +0200, Vladimir Sokolovsky wrote:
> 
> Although the code does not have interleaving fragments from different
> messages, it seems that multiple RDS messages can be in-flight
> simultaneously because rds_send_xmit() does not wait for completions:
> it
> continues to process the next message as soon as the transport layer
> (e.g., IB) reports that the messages "have been sent".
> With IB, it just means that the messages have been scheduled for
> transmission by the HW.
> 
> Did I miss something?

Pipelining sends is a good thing - and we probably need this to achieve
the performance level the ref RDS driver attained. Oracle can have many
thousands of messages in flight at a time from a single process - and
performance is dependent on the rds sends completing immediately.

We should try and send all that we can - as fast as we can - shoving
them down the wire (at least upto what we set max dtos for the qp at
creation time) - and keep the pipeline going as each dto completion
arrives. Of course we free the sent messages only after recving an ack.