[rds-devel] The meaning of MR invalidation

[Olaf Kirch]

Hi Or,

thanks for your explanation!

On Thursday 14 February 2008 15:31, Or Gerlitz wrote:
> When there are multiple I/Os are running in parallel, and each being 
> served by different FMR --> different rkey --> different cache slots, 
> first, they all compete on the cache and second, since fmr_unmap does 
> SYNC_TPT which flushes the cache, one have to try and avoid calling 
> fmr_unmap when possible.
> 
> So basically, when each fmr is remapped n times, over time, you get less 
> SYNC_TPT calls compared to the case where each fmr is mapped once before 
> moved to the unmap queue. However, if you use enough FMRs such that you 
> don't call SYNC_TPT "too much" the use-once design should function quite 
> well compared to use-n design.
> 
> For example, a scheme where you have to serve 1000 1MB IOs/sec, and you 
> alloc 5k FMRs and once every 4 seconds you unmap 4K FMRs from a 
> background thread, might work quite good, but this has to be validated 
> ofcourse.

Yes, this needs some performance measurements.

One last question - when I remap a FMR, does that invalidate the
mapping associated with the previous R_Key?

> Now, if you are willing to go with that approach, it means that in case 
> core fmr pool API is enhanced such that you can --specify-- how many 
> times an fmr can be mapped before its queued for unmap, RDS should be 
> able to use this cache again and not have one of its own!

I would also like to have some control over when a flush can be
triggered asynchronously. Right now, I can call invalidate, but that
is synchronous and can't be called from atomic context.

Second, I need a way to hook into the FMR destroy - I need to keep
the memory pinned until I know the FMR has gone away, so I do want
to be called back when that happens.

Olaf
-- 
Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
okir at lst.de |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax