[rds-devel] extra header abstraction

Andy Grover andy.grover at oracle.com
Fri May 21 11:13:20 PDT 2010


On 05/21/2010 04:30 AM, Dirk Kirsten wrote:
> I have another issue; this time it is about the extra header provided
> by RDS. I really like your abstraction of RDS and the underlying
> transport modules, it makes it really straightforward to program
> another transport module. Nevertheless, I have some stuff which I
> have to transmit for my transport module. This stuff is just 6 bytes
> long, so I though it would be a good idea to pack this into the extra
> header provided by RDS. Unfortunately I figured out that the extra
> header is kind of hard coded into the rds core. Why is this so or is
> this due to historic reasons?

Hi Dirk,

There hasn't been a need to handle dynamic exthdrs, so hardcoded has 
been the simplest way.

(BTW the rds core/transport api is not clean to the point that it's 
possible to add a new transport with zero core changes. The goal is 
modularization of functionality to reduce overall complexity, and to 
separate dependencies so that rds can be used over tcp without requiring 
the entire ib stack.)

> I think it would be more reasonable to put the handling of the extra
> header into the transport modules themself. Especially because you
> are using the extra header mainly for some credit based infiniband
> flow control, if I understood it correct. Because it is
> infiniband-specific wouldn't it be better to handle this in the IB
> transport module? My problem now is that I can not use the extra
> header, because I do not want to change your rds core code. Did I got
> the architecture of the extra header or did I missed something? It
> would be really nice to know if you plan to change anything regarding
> this topic.

The credit field is actually in the main part of the header. We 
currently use the exthdr for rdma-related fields.

I recommend changing the core to add the type and size of your new 
exthdr -- it should be relatively minimal. Note that there's an 
artificial restriction of one exthdr per message, so if you are also 
using rdma exthdrs, that's another change that will be needed.

Regards -- Andy



More information about the rds-devel mailing list