[rds-devel] Re: [PATCH] Implement our own RDMA MR pool

Or Gerlitz ogerlitz at voltaire.com
Thu Feb 7 01:37:44 PST 2008


Olaf Kirch wrote:
> On Wednesday 30 January 2008 16:06, Or Gerlitz wrote:

Hi Olaf,

I see this thread are somehow very important, so I want to try and 
address each of the problems you have pointed on... here goes for two of 
them:

>>>      -	smaller memory footprint - the fmr_pool would allocate lots
>>>      	of memory in advance (in particular, one full sized address
>>>     	vector for each FMR, which was never used anyway).

>> thanks for pointing this out, I sent Roland a patch yesterday that fixes 
>> that in the fmr_pool, that is no page_list is allocated if the user does 
>> not ask for caching

> Okay, good :)

So we agree this point is closed?

>>>      -	there is no separate thread for flushing MRs. If we're in
>>>      	IRQ context, we put a work item on the rds work queue.
>>>     	Otherwise, the flush code is executed in the calling context

>> what problem does this solve?

> It's less complicated. The ib_fmr pool uses a complicated mechanism for
> synchronizing the thread that does the invalidation with the thread(s)
> that are requesting it. The first problem I did run into was that this
> mechanism was buggy, and the calling thread would actually continue before
> any FMR had been invalidated at all.

I see, so is this solved by the two patches you and Roland came with? 
they were merged by Roland:

> commit a3cd7d9070be417a21905c997ee32d756d999b38
> Author: Olaf Kirch <olaf.kirch at oracle.com>
> Date:   Wed Jan 16 18:37:16 2008 +0100
> IB/fmr_pool: ib_fmr_pool_flush() should flush all dirty FMRs

> commit a656eb758fc6e6a42659ecf5ba34a5c5a2aeec17
> Author: Olaf Kirch <olaf.kirch at oracle.com>
> Date:   Wed Jan 16 18:36:27 2008 +0100
> IB/fmr_pool: Flush serial numbers can get out of sync

Or.





More information about the rds-devel mailing list