[rds-devel] The meaning of MR invalidation
Or Gerlitz
ogerlitz at voltaire.com
Tue Feb 19 04:33:12 PST 2008
Dror Goldenberg wrote:
> I am not sure I understood what you meant in the paragraph below. Maybe
> more info or example will do.
Hi Dror,
My understanding (which I have validated by looking on the mthca driver
xxx_map_phys_fmr / xxx_fmr_unmap code) is that once an FMR is created
its being associated with a 32bits key, where each remap of the fmr
changes some of the lower bits of the key. At some point a set of fmrs
are unmapped where unmapping does two things, first, for each fmr it
resets the key to the initial value, and second it issues SYNC_TPT. Once
an fmr is now remapped, it would get the initial key it had and so on.
This means the ULP can not count on fmr umapping as the only mean for
invalidation of rkey held by remote party, since with remapping this fmr
,they get the same set of rkeys that were used before unmapping, and
hence the unmap operation is a synchronization point, no more.
ULPs like the SCSI based iser and srp, would not invalidate the mapping
until either SCSI response was received (so here there's trust on the
target that it will not attempt to use this rkey) or the initiator
issued SCSI abort and there was response on the abort or the target
declared as being non-responsive, etc.
The point I was trying to make with Olaf is that first, the RDS design
should count on the remote side when it sends a response on a mapping
and second, --if-- he wants to go an extra step, I am not sure that
counting on fmr unmapping and even destroying provides any real
advantage over doing nothing, as of all the reasons discussed above. So
what's needed here is some invalidation protocol where the client says
"I want to revoke this <transaction ID, rkey> pair" and the server says
"fine, I will not this pair any more", etc.
Or
> -----Original Message-----
> From: Or Gerlitz [mailto:ogerlitz at voltaire.com]
> However, the minute this synched-FMR is used again (mapped) one gets the
> --same set-- of R_Keys before the synchronization, so unless some
> explicit synchronization with the remote party is added, the other
> synch does not worth much from the aspect you are interested in, at
> least as I understand it, since the remote may still try to use the
> old-from-your-perspective instance of the R_Key
More information about the rds-devel
mailing list