[rds-devel] Re: RDS/RDMA protocol

[Richard Frank]

Or Gerlitz wrote:
> Richard Frank wrote:
>> Zach Brown wrote:
>
>>> You should be sending these mails to rds-devel.  There's a great chance
>>> that you'll want the people actively involved in RDS to respond.
>
>> Yes - we're getting hammered about not using rds-dev - let's try and 
>> move all our RDS discussion - that does not contain Oracle private 
>> information - into rds-dev.
>
> I did not CC rds-devel by mistake, sorry.
>
>> The message send is a compound operation - it includes a regular RDS 
>> send (the immediate data) and the RDMA data.
>>
>> The rdma operation is pushed down the wire and then the immediate 
>> data is pipelined right behind the RDMA.
>>
>> Keep in mind when the client requested the rdma - it sent along an 
>> identifier for the operation. When the rdma server responds - it 
>> encodes the client provided identifier in the immediate  data 
>> response message.
>>
>> So when the client recv's the immediate data message (response to 
>> request) - it gets back the id of the request which completed and it 
>> knows the rdma is complete due to ordering of the RC that both ops 
>> traveled over.
>
> Just to make sure I understand, the "immediate data message" you 
> mention here is at the app level, not RDS level, correct? for example 
> when you say "it encodes the client provided identifier" etc its left 
> to the app how to encode this?
>
> Or.
>
Yes, the app is sending the immediate data - along with the rdma 
operation via the single RDS send operation.

The immediate data send and rdma are an atomic operation - either both 
succeed or neither does.