[rds-devel] Re: RDS IB transport software flow control?
Or Gerlitz
ogerlitz at voltaire.com
Tue Nov 6 01:21:58 PST 2007
Richard Frank wrote:
> Yes - it relies on IB RNRs - reasoning was simply to keep the RDS wire
> protocol to a minimum.
Thinking on the matter, on the one hand the current RDS/IB code does not
have software flow control but it does do explicit ACK-ing on each message.
It might be a win/win to have software flow control protocol and which
indirectly provide ACK-ing functionality, since IB RC keep the order of
messages, if you know how many credits you have on the remote side, you
know what was the last messages the accepted and the credit update is
sent after the message placed into memory. The flow control info can be
piggybacked on messages going the other direction and below some water
mark, we would need a "credit-update" message. Its not so complex, the
SDP flow control defined in the IB spec can be used as a starting point.
The win/win here, comes from not doing ACK on every packet as been done
now (and I hear that ways are being searched re how to optimize this, eg
with RDMA, etc), not doing ACK for protocol based on IB RC, something
which creates doubts and confusion for users / reviewers / implementors
and more important, not loading the IB network with RNR NAK messages
where you have for some reason an imbalanced setup and not having the
HCA to arm and act on RNR timers, etc.
Or.
More information about the rds-devel
mailing list