[rds-devel] what is rdma immediate data and how is it used ?

[Or Gerlitz]

Richard Frank wrote:
> Or Gerlitz wrote:
>> Richard Frank wrote:
>>> Our planned use of immediate data (notification message of rdma 
>>> completion) keeps the immediate data (msg) separate from the rdma data.

>> Isn't this somehow non optimal for the case of rdma-write (disk read)?

> In the rdma write case (data to client) the immediate data identifies 
> the request that is completing.

Indeed, and with sending the immediate data along with (ie immediatly 
after, or even in post-send-list fashion, ie call ib_post_send with two 
work requests, the first of the rdma-write and the second of the 
immediate data send) the rdma-write, there will be no need as you noted 
to ask for completion on the rdma-write. Also assuming some 
credits/windowing scheme is used between the disk client and server, it 
will let the client to note this request is done more quickly and 
dequeue and submit another request.

> In the rdma read case (data from client), the rdma server  can 
> optionally send notification (immediate data).  The rdma server could - 
> for example - let the client know that the client buffer is no longer in 
> use - the data is now on the rdma server - and if it is in non-volatile 
> memory - it could indicate that the clients request is complete.  From 
> the RDS driver perspective, the immediate data rm (even if zero client 
> data) also provides a mechanism to auto free the key setup for the rdma 
> xfer (resource management). If moving the data from the rdma server to 
> persistent storage takes additional work - then when this work is 
> complete - a separate message is sent to the client indicating 
> completion of the request.

Yes, I understand that, thanks for the clarification

Or.