[rds-devel] Re: [PATCH RFC RDS/IB] version 2. increase number of unsignaled work requests

[Richard Frank]

On Thu, 2007-01-25 at 11:25 +0200, Vladimir Sokolovsky wrote:
> 
> Although the code does not have interleaving fragments from different
> messages, it seems that multiple RDS messages can be in-flight
> simultaneously because rds_send_xmit() does not wait for completions:
> it
> continues to process the next message as soon as the transport layer
> (e.g., IB) reports that the messages "have been sent".
> With IB, it just means that the messages have been scheduled for
> transmission by the HW.
> 
> Did I miss something?

Another important consideration is RNR retry period. Assuming there is
no flow control in the RDS wire protocol (was none in the ref driver)
then we are dependent on fast RNR recovery (IB flow control). 

The ref RDS driver would set this retry period to be a minimum value -
so sender retries very quickly. This change had a large impact on thru-
put.