[rds-devel] Re: RDS IB transport software flow control?

Or Gerlitz ogerlitz at voltaire.com
Tue Nov 6 01:21:58 PST 2007


Richard Frank wrote:
> Yes - it relies on IB RNRs - reasoning was simply to keep the RDS wire 
> protocol to a minimum.

Thinking on the matter, on the one hand the current RDS/IB code does not 
have software flow control but it does do explicit ACK-ing on each message.

It might be a win/win to have software flow control protocol and which 
indirectly provide ACK-ing functionality, since IB RC keep the order of 
messages, if you know how many credits you have on the remote side, you 
know what was the last messages the accepted and the credit update is 
sent after the message placed into memory. The flow control info can be 
piggybacked on messages going the other direction and below some water 
mark, we would need a "credit-update" message. Its not so complex, the 
SDP flow control defined in the IB spec can be used as a starting point.

The win/win here, comes from not doing ACK on every packet as been done 
now (and I hear that ways are being searched re how to optimize this, eg 
with RDMA, etc), not doing ACK for protocol based on IB RC, something 
which creates doubts and confusion for users / reviewers / implementors 
and more important, not loading the IB network with RNR NAK messages 
where you have for some reason an imbalanced setup and not having the 
HCA to arm and act on RNR timers, etc.

Or.




More information about the rds-devel mailing list