[Tmem-users] Experiments with Tmem and trying to find out Tmem USAGE statistics

[Icarus Sonic]

Hello Dan,

Thank you for your information. It helps me a lot.

yes, my CPU is 32-bit only and I'm just looking at dom0.

After running some memory eater program, 'xm tmem-list 0 --a' show that
domain0 does use tmem. I've tried xentop + t. It told me statistics of how
many tmem operations occured.

Try to find a 64bit system for better testing tmem.

Regards,
Weijia

On Wed, May 12, 2010 at 10:09 PM, Dan Magenheimer <
dan.magenheimer at oracle.com> wrote:

>  Hi Icarus(?) –
>
>
>
> Tmem has very limited functionality on a 32-bit Xen/CPU because allocation
> from the Xen heap works differently.  As a result, I haven’t tested it on
> 32-bit in a long time, though it may work for some things.  Is your CPU
> really 32-bit-only?  Or are you just looking at your dom0?
>
>
>
> Also, tmem wasn’t really designed to work in dom0 though, again, it may
> work with some functionality limitations.  It is intended to run in PV
> guests.
>
>
>
> And, the linux-side tmem code is getting old... I need to update it and
> also provide better user documentation.
>
>
> To answer your questions:
>
>
>
> 1.        You should not see an increase in system time (in a PV guest)
> from tmem because nearly all the work done in tmem is done in the
> hypervisor.  So I suspect the increase you are seeing is due to other
> factors within the dom0 kernel.
>
> 2.       The output you provided shows that tmem is not really being used
> at all.  Assuming you are using Xen-4.0.0 tools, then xentop should also
> show summary numbers for tmem (see the new “t” keyboard command), but only
> if tmem is actually in use.  You can also use “xm tmem-list --all –long |
> xen-tmem-list-parse” to see more detailed information.
>
>
>
> Hope that helps!
>
> Dan
>
>
>
>
>
> *From:* Icarus Sonic [mailto:icarus.sonic at gmail.com]
> *Sent:* Wednesday, May 12, 2010 1:05 AM
> *To:* tmem-users at oss.oracle.com
>
> *Subject:* [Tmem-users] Experiments with Tmem and trying to find out Tmem
> USAGE statistics
>
>
>
> 1. Experiment data - does it conform to your expectation?
>
>
> Hi, I've installed on a Dell Optilex GX260(32bit x86 , 1G Memory) Xen-4.0.0
> and linux-2.6.18.8-xen kernel dom0 with tmem patched and PRECACHE/PRESWAP
> opened.
>
> I did the following experiments and found that tmem does work: I wrote a c
> program that touch each page of a 512M 'unsigned char' array, and measured
> its running time in dom0. I tried it with three different settings. The
> result shows that tmem works but introduce a little bit overhead. Experiment
> result is listed as followed:
>
> *Setting 1: tmem off(by booting xen without 'tmem' argument), dom0 with
> all available memory close to 1024GB.
> Setting 2: tmem off(by booting xen without 'tmem' argument), dom0 with only
> 256MB memory. xen argument: dom0_mem=256M
> Setting 3: tmem on, dom0 with only 256MB memory. xen argument: dom0_mem=256M
> tmem
>
> With each setting, I ran it three times. Run time is listed in
> [real],[user],[sys] format
>
> Results
> ==========================================
> Setting1: round1(1.503,0.025,1.466), round2(1.487,0.030,1.456),
> round3(1.486,0.039,1.444)*
> *Setting2: round1(16.591,0.036,2.779), round2(10.434,0.022,2.368)
> round3(13.262,0.033,2.305)
> Setting3: round1(8.739,0.035,4.616), round2(4.582,0.036,2.852),
> round3(4.604,0.043,3.592)
>
> *With setting 2,run time is 10x of that with setting 1, which is caused by
> swapping. Results in setting 3 demostrate that tmem alleviate swapping
> problem but introduced overhead.
>
>
> 2. Question: Where can i found how much tmem is used by a domain, and how
> much tmem is available?
> Tried xm tmem-list -a | xen-tmem-list-parse to see statistics of tmem
> usage, its output:
>
> total tmem ops=93859 (errors=91550) -- tmem pages avail=0
> datastructs: objs=0 (max=0) pgps=0 (max=0) nodes=0 (max=0)
> misc: failed_copies=0 alloc_failed=0 alloc_page_failed=0 low_mem=0
> evicted=0/0 relinq=0/0, max_evicts_per_relinq=0, flush_pools=0, eph_count=0,
> eph_max=0
> domid0:
> weight=0,cap=0,compress=0,frozen=0,total_cycles=61882224,succ_eph_gets=0,succ_pers_puts=0,succ_pers_gets=0,eph_count=0,max_eph=0,compression
> ratio=0% (samples=0,poor=0,nomem=0)
> domid0,id0[EP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
> gets=0/0(0%) flush=0/0 flobj=0/0
> domid0,id1[EP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
> gets=0/0(0%) flush=0/0 flobj=0/0
> domid0,id2[PP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
> gets=0/0(0%) flush=0/0 flobj=0/0
> failed get cycles: avg=642, max=18856, min=384, samples=91550
> flush cycles: avg=785, max=2812, min=668, samples=1966
> flush_obj cycles: avg=4623, max=12508, min=696, samples=334
>
> But it does not tell me how much tmem is used by a domain or how much tmem
> is available in hypervisor?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/tmem-users/attachments/20100513/4d5c624a/attachment.html