[rds-devel] OK - so what is a common completion model / unified poll in RDS anyway ?

[Richard Frank]

Olaf Kirch wrote:
> On Thursday 15 November 2007 13:19, Richard Frank wrote:
>   
>> A lot of what's driving the choice of flags is driven by the desire to 
>> use common code in our application for all socket implementations - 
>> including UDP and RDS.  This is where the pollout / pollin originates - 
>> it's transparent - well 99.9% - to the app that it is running on RDS.
>>     
>
> Well, I'm talking about congestion updates here. I'm not talking about
> the uncongested case, where POLLIN and POLLOUT are absolutely fine, and
> the way RDS uses them is in synch with "normal" semantics.
>
> I'm only talking about how we signal congestion updates.
>
> Does Oracle exchange congestion information over UDP at all?
>
>   
No it does not - that's the issue - so it's only expecting pollin and 
pollout.

>> If we want to change the flags - we can not do it in RDS v3 - due to our 
>> requirement of not wanting to break RDS v2 applications.
>>     
>
> It doesn't have to break anything. When we receive a congestion update,
> we can continue to return POLLIN for some time - but along with another
> flag, such as POLLPRI.
>
>   
OK - I see.

> And as for the RDMA operation, that's a new interface anyway, so you're not
> breaking interoperability with anything there. 
Yes - your right.
> I really think we should either
> find something other than POLLIN for that, such as POLLSMG. 
OK
> Or should use a
> recvmsg based interface to tell the application about completed RDMA ops - then
> it would make sense to signal RDMA completions via POLLIN. For instance you
> can include a control message containing the most recently completed rdma
> operation, obviating the need for a separate interface.
>
>   
Using POLLMSG and barrier might be simpler.

> With your proposal, if an application sees POLLIN, it would have to
>
>  -	call recvmsg
>  -	check for completed RDMA transfers
>  -	try to resend any messages that were previously refused by
> 	sendmsg because the destination was congested
>
> i.e. a single incoming message triggers a lot of unrelated activity.
>
>   
Yea - it's ugly but simple.
> With my proposal, things would look like this:
>
>  -	POLLIN: call recvmsg
>  -	POLLIN|POLLPRI: call recvmsg; retry congested sends
>  -	POLLMSG: check for completed RDMA
>
>   
This is much better  - in the case of POLLMSG -  the app could still use 
the barrier to see which rdma completed...
> Olaf
>