[rds-devel] IB/iWARP code separation

[Steve Wise]

Or Gerlitz wrote:
> On Thu, Nov 6, 2008 at 11:44 PM, Steve Wise <swise at opengridcomputing.com> wrote:
>
>   
>>> So Rick's thought about the impossibility to reclaim the buffer at the
>>> side initiating the rdma following the completion was wrong, I see.
>>>       
>
>   
>> He might have been talking about more than just reclaiming the buffer.
>>     
>
> In that context he mentioned two issues: current RDS design/code
> assumes for rdma completion to indicates that (1) buffer ownership is
> returned to client (2) rdma has been processed by remote HCA - and is
> pending delivery to remote host memory.
>
> >From your other reply I understand that the buffer can be reclaimed,
> and from Rick response I understand that he's thinking on submitting
> 0bread following rdma-write to address the 2nd requirement. So it
> seems we are done in that respect.
>
>   
>> Async events. In the iWARP RDMAP protocol, any ingress errors (like bad stag on an
>> incoming write or read response) result in a TERMINATE message sent back to
>> the peer (and then the connection torn down).  The peer who gets the
>> TERMINATE message will map that to an async event.  Looks like cxgb3 and amso1100 > map it to IB_EVENT_QP_REQ_ERR, and nes maps it to IB_EVENT_QP_FATAL.
>>     
>
> When the connection is torn down - is it somehow notified by the iw cm
> at either side? a possible solution would be to terminate the
> connection in the ULP (RDS) level when getting such async event, or
> maybe have the iw cm also register an async handler and generate
> disconnect callback on the CM ID associated with that QP.
>
>   
The cm id associated with the connection will indeed get DISCONNECTED 
events because the connection is torn down via the TERMINATE process.  
Both sides will get a DISCONNECTED event...

Also, both sides will get an async error on the QP.

Everyone gets error notification.  :)