[rds-devel] OK - so what is a common completion model / unified poll in RDS anyway ?

Richard Frank richard.frank at oracle.com
Thu Nov 15 05:08:03 PST 2007

Olaf Kirch wrote:
> On Thursday 15 November 2007 13:19, Richard Frank wrote:
>> A lot of what's driving the choice of flags is driven by the desire to 
>> use common code in our application for all socket implementations - 
>> including UDP and RDS.  This is where the pollout / pollin originates - 
>> it's transparent - well 99.9% - to the app that it is running on RDS.
> Well, I'm talking about congestion updates here. I'm not talking about
> the uncongested case, where POLLIN and POLLOUT are absolutely fine, and
> the way RDS uses them is in synch with "normal" semantics.
> I'm only talking about how we signal congestion updates.
> Does Oracle exchange congestion information over UDP at all?
No it does not - that's the issue - so it's only expecting pollin and 

>> If we want to change the flags - we can not do it in RDS v3 - due to our 
>> requirement of not wanting to break RDS v2 applications.
> It doesn't have to break anything. When we receive a congestion update,
> we can continue to return POLLIN for some time - but along with another
> flag, such as POLLPRI.
OK - I see.

> And as for the RDMA operation, that's a new interface anyway, so you're not
> breaking interoperability with anything there. 
Yes - your right.
> I really think we should either
> find something other than POLLIN for that, such as POLLSMG. 
> Or should use a
> recvmsg based interface to tell the application about completed RDMA ops - then
> it would make sense to signal RDMA completions via POLLIN. For instance you
> can include a control message containing the most recently completed rdma
> operation, obviating the need for a separate interface.
Using POLLMSG and barrier might be simpler.

> With your proposal, if an application sees POLLIN, it would have to
>  -	call recvmsg
>  -	check for completed RDMA transfers
>  -	try to resend any messages that were previously refused by
> 	sendmsg because the destination was congested
> i.e. a single incoming message triggers a lot of unrelated activity.
Yea - it's ugly but simple.
> With my proposal, things would look like this:
>  -	POLLIN: call recvmsg
>  -	POLLIN|POLLPRI: call recvmsg; retry congested sends
>  -	POLLMSG: check for completed RDMA
This is much better  - in the case of POLLMSG -  the app could still use 
the barrier to see which rdma completed...
> Olaf

More information about the rds-devel mailing list