[rds-devel] Re: RDS IB transport software flow control?

Or Gerlitz or.gerlitz at gmail.com
Thu Nov 8 15:15:20 PST 2007


On 11/7/07, Richard Frank <richard.frank at oracle.com> wrote:
>
>
> I'm pretty sure the code is not doing this - we should update the
> comments - and get our the design docs as you keep reminding us to do.


I don't understand the code to the extent of saying whether or not it is
done.


> One thing we might want to consider is - if the IB transport knows that
> we have a single HCA configured - then we can do away with the current
> HA ACKing  - as it is only needed for HA with multiple adapters ! This
> sounds a bit off - but consider all the single HCA boards out
> there....running the HA acking is pure overhead.


I would not take this path - of having different code running for the case
you have one hca with one port, one hca with two ports, two hcas, etc. Yet
again I think you raise here an arguments against acking and in favour of
flow control, since the latter covers the first and is needs in all schemes.


> >> A couple of additional optimizations for our existing flow control
> >> would be to 1) add srq support - this will reduce the possibility of
> >> RNRs 2) use an rdma write - vs - message for the ack to remove the
> >> requirement of having a recv buffer posted to handle the ack.
>
> > Can you elaborate a little on RDMA ACKs vs SEND ACKs? I guess you
> > don't mean to rdma-write-with-immediate since this also consumes a WR
> > at the receiver side. If it just rdma-write, how would the sender be
> > notified on the ACK reception, would it do polling-on-memory?
>

RDMA back the ack frame for an RC. And yes, on the send side - when a
> send completes, and or when the CQ is drained, or when a send is
> initiated - check the current ACK hwm for the RC and release send
> buffers which have been acked.


OK, thanks for the clarification. What is the status of 1 & 2 which you
mention here, are they under implementation?

Or.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/rds-devel/attachments/20071109/4eda23b6/attachment.html


More information about the rds-devel mailing list