[rds-devel] [PATCH RFC net-next 1/6] sock: MSG_PEEK support for sk_error_queue

[Sowmini Varadhan]

On (01/18/18 08:53), Eric Dumazet wrote:
> 
> The thing is : MSG_PEEK 'support' will also need SO_PEEK_OFF support.

sure, I'll drop the MSG_PEEK idea (which I wasnt very thrilled
about anyway)

> So lets properly design things, and not re-use legacy stuff that is
> proven to be not multi-thread ready and too complex.
> 
> If you want to design a new channel of communication, do it, and
> maintain it.

My instinct is to go with the fixed size ancillary data- which itself
allows 2 options:

1. cmsg_data has a sock_extended_err preamble
   with ee_origin = SO_EE_ORIGIN_ZEROCOPY_COOKIE (or similar),
   and the ee_data is an array of 32 bit cookies (can pack at most 8 
   32-bit cookies, if we want to pack this into an skb->cb)

   Using the sock_extended_err as preamble will allow this to be usable by
   existing tcp zcopy applications (they can use the ee_origin to find
   out if this a batch of cookies or the existing hi/lo values).

2. If we have the option of passing completion-notification up as ancillary 
   data on the pollin/recvmsg channel itself (instead of MSG_ERRQUEUE)
   we dont have to try to retain "backward compat" to the
   SO_EE_ORIGIN_ZEROCOPY API: we can just use a completely new data
   struct for the notification and potentially pack more cookies into
   48 bytes (RDS could be the first guinea pig for this- doesnt even
   have to be done across all protocol families on day-1).

I think the shmem channel suggestion would be an optional optimization
that can be added later- it may not even be necessary, since most
applications will likely be sending *and* receiving data, so passing up
cookies with recvmsg should be "good enough" to save syscall overhead
for the common case.

I can work #2, if there are no objections to it.

--Sowmini