[rds-devel] The meaning of MR invalidation
Or Gerlitz
ogerlitz at voltaire.com
Sun Feb 17 23:17:57 PST 2008
>> From: Olaf Kirch
>> So there is a strong case to reuse FMRs. The last nagging worry I have
>> is, what happens when I remap a FMR - does that invalidate the old
>> mapping, or not?
Dror Goldenberg wrote:
> FMR remapping does not guarantee invalidation of the old mapping. The MR
> number changes when remapped, so it guaranteed that the new MR number
> makes the new mapping accessible. The old mapping may or may not exist
> until SYNC_TPT is submitted (and completes).
OK, that was my thinking as well, and just to clarify, my understanding
is that "MR number" refers to R_Key. So at the bottom line, through the
life time of an FMR there are synchronization points (SYNC_TPT) where
you know that from now and on old mapping possibly kept by a remote peer
are invalid.
However, the minute this synched-FMR is used again (mapped) one gets the
--same set-- of R_Keys before the synchronization, so unless some
explicit synchronization with the remote party is added, the other
synch does not worth much from the aspect you are interested in, at
least as I understand it, since the remote may still try to use the
old-from-your-perspective instance of the R_Key
I have brought the point of a need for some invalidation protocol
earlier over this thread, and gave the SCSI aborts as an example.
> From: Olaf Kirch [mailto:olaf.kirch at oracle.com]
> ... and the bad news is, if I change the code to use FMRs exactly once,
> I lose roughly 10% in performance. I ran with 1, 2, and 4 tasks, a wide
> range of message sizes and RDMA sizes from 64k to 1M. The degradation
> was pretty similar across all test runs.
So how many IOs (=FMR mappings) per second are needed to serve this load?
Or.
More information about the rds-devel
mailing list