[Tmem-devel] Implementing Transcendent Memory on Lguest

[Nitin Gupta]

Hi Dan,

On Tue, Feb 9, 2010 at 4:49 AM, Dan Magenheimer
<dan.magenheimer at oracle.com> wrote:
>> However, I could not understand your coherence issue: this vswap
>> driver simply sends swap page to hypervisor synchronously.
>
> I am asking if anything in the virtio "layer" buffers
> the data, making it appear to the client that the data
> has been copied from/to the hypervisor (and all of tmem's
> data structures updated in the hypervisor) but it really
> hasn't happened yet. If so, bad things can happen.
>

I think we can set "virtqueue" size to 1 *and* make it work
synchronously (this is what vswap does). This should avoid
any buffering problem you are concerned with.

On a side note, I think such simplistic behavior can result
in poor performance but handling swapping case always
brings surprises so maybe anything fancier -- multiple queues
having large sizes, working asynchronously -- may be difficult
to implement.

Thanks,
Nitin


>
>> -----Original Message-----
>> From: Nitin Gupta [mailto:ngupta at vflare.org]
>> Sent: Monday, February 08, 2010 11:30 AM
>> To: Dan Magenheimer
>> Cc: Gaurav Kukreja; tmem-devel at oss.oracle.com; rusty at rustcorp.com.au
>> Subject: Re: [Tmem-devel] Implementing Transcendent Memory on Lguest
>>
>> Hi Dan,
>>
>> On 02/08/2010 11:52 PM, Dan Magenheimer wrote:
>> >
>> > Nice!  I have been out of the office most of the last month so
>> > am still getting unburied. :-(
>> >
>> > Again admitting ignorance of virtio, I have two concerns, performance
>> > and coherence, both assuming that there is some temporary storage
>> > of the tmem data (e.g. in a "ring buffer" or equivalent).  Is
>> > there any such temporary storage, or does the data always get
>> > transferred directly to the hypervisor?
>> >
>>
>> This is a simple prototype for "swap to hypervisor" kind of thing.
>> Complete description of this driver is present in the mail I pointed
>> out.
>> For now, it sends every page that is swapped to /dev/vswap (visible
>> inside
>> guest) to host.
>>
>> > For example, for performance, a ring buffer would be an extra page
>> > copy; for coherence, data stored in a ring buffer might not get
>> > flushed by a tmem flush command.
>>
>> I do not have any performance data, so cannot comment on that.
>> However, I could not understand your coherence issue: this vswap
>> driver simply sends swap page to hypervisor synchronously.
>>
>> Nitin
>>
>>
>> >
>> >> -----Original Message-----
>> >> From: Nitin Gupta [mailto:ngupta at vflare.org]
>> >> Sent: Monday, February 08, 2010 11:04 AM
>> >> To: Dan Magenheimer
>> >> Cc: Gaurav Kukreja; tmem-devel at oss.oracle.com; rusty at rustcorp.com.au
>> >> Subject: Re: [Tmem-devel] Implementing Transcendent Memory on Lguest
>> >>
>> >> On 02/08/2010 11:04 PM, Dan Magenheimer wrote:
>> >>>
>> >>> I'm not very familiar with virtio but tmem requires synchronous
>> >>> data transfer between the "tmem server" (hypervisor) and the
>> >>> "tmem client" (guest).  I suspect that virtio assumes an
>> >>> asynchronous model.  If a virtio device can be configured
>> >>> as a synchronous device then your idea might work.
>> >>>
>> >>
>> >> virtio can be made to work synchronously. For virtio version of
>> >> "preswap", refer this:
>> >> http://oss.oracle.com/pipermail/tmem-devel/2010-January.txt
>> >>
>> >> It creates "vswap" device which swaps directly over hypervisor.
>> >> Pages are transferred to host using virtio in synchronous manner.
>> >>
>> >> (Links to code are at the end of above archived mail)
>> >>
>> >> Nitin
>> >>
>> >>
>> >>
>> >>>
>> >>> =============
>> >>>
>> >>> From: Gaurav Kukreja [mailto:mailme.gaurav at gmail.com]
>> >>> Sent: Sunday, February 07, 2010 11:11 PM
>> >>> To: tmem-devel at oss.oracle.com; rusty at rustcorp.com.au
>> >>> Subject: [Tmem-devel] Implementing Transcendent Memory on Lguest
>> >>>
>> >>> Hi,
>> >>>
>> >>> I have been working on trying to implement Transcendent Memory in
>> >> lguest.
>> >>>
>> >>> So far, Tmem on lguest is just working very rudimentarily. I am
>> >> currently working, on virtio balloon driver for the lguest, trying
>> to
>> >> get it to work.
>> >>>
>> >>> I was wondering, and maybe this is vague, whether is it possible to
>> >> create a implementation of Transcendent Memory, as a Virtio Device.
>> I
>> >> have read Rusty's paper on Virtio. I am yet to find out of the
>> >> technical issues regarding the same.
>> >>>
>> >>> Let us please evaluate, whether, there would be any benefits of
>> >> implementing, Transcendent Memory as a Virtio device. I dont know,
>> >> anything about KVM, but would it then be possible to port Tmem, to
>> KVM
>> >> easily?
>> >>>
>> >>> Regards,
>> >>>
>> >>
>> >
>>
>