[rds-devel] OK - so what is a common completion model / unified
poll in RDS anyway ?
richard.frank at oracle.com
Thu Nov 15 05:08:03 PST 2007
Olaf Kirch wrote:
> On Thursday 15 November 2007 13:19, Richard Frank wrote:
>> A lot of what's driving the choice of flags is driven by the desire to
>> use common code in our application for all socket implementations -
>> including UDP and RDS. This is where the pollout / pollin originates -
>> it's transparent - well 99.9% - to the app that it is running on RDS.
> Well, I'm talking about congestion updates here. I'm not talking about
> the uncongested case, where POLLIN and POLLOUT are absolutely fine, and
> the way RDS uses them is in synch with "normal" semantics.
> I'm only talking about how we signal congestion updates.
> Does Oracle exchange congestion information over UDP at all?
No it does not - that's the issue - so it's only expecting pollin and
>> If we want to change the flags - we can not do it in RDS v3 - due to our
>> requirement of not wanting to break RDS v2 applications.
> It doesn't have to break anything. When we receive a congestion update,
> we can continue to return POLLIN for some time - but along with another
> flag, such as POLLPRI.
OK - I see.
> And as for the RDMA operation, that's a new interface anyway, so you're not
> breaking interoperability with anything there.
Yes - your right.
> I really think we should either
> find something other than POLLIN for that, such as POLLSMG.
> Or should use a
> recvmsg based interface to tell the application about completed RDMA ops - then
> it would make sense to signal RDMA completions via POLLIN. For instance you
> can include a control message containing the most recently completed rdma
> operation, obviating the need for a separate interface.
Using POLLMSG and barrier might be simpler.
> With your proposal, if an application sees POLLIN, it would have to
> - call recvmsg
> - check for completed RDMA transfers
> - try to resend any messages that were previously refused by
> sendmsg because the destination was congested
> i.e. a single incoming message triggers a lot of unrelated activity.
Yea - it's ugly but simple.
> With my proposal, things would look like this:
> - POLLIN: call recvmsg
> - POLLIN|POLLPRI: call recvmsg; retry congested sends
> - POLLMSG: check for completed RDMA
This is much better - in the case of POLLMSG - the app could still use
the barrier to see which rdma completed...
More information about the rds-devel