Hello Dan,<br><br>Thank you for your information. It helps me a lot.<br><br>yes, my CPU is 32-bit only and I'm just looking at dom0.<br><br>After running some memory eater program, 'xm tmem-list 0 --a' show that domain0 does use tmem. I've tried xentop + t. It told me statistics of how many tmem operations occured.<br>
<br>Try to find a 64bit system for better testing tmem.<br><br>Regards,<br>Weijia<br><br><div class="gmail_quote">On Wed, May 12, 2010 at 10:09 PM, Dan Magenheimer <span dir="ltr"><<a href="mailto:dan.magenheimer@oracle.com">dan.magenheimer@oracle.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US">
<div><div class="im">
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Hi Icarus(?) –</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Tmem has very limited functionality on a 32-bit Xen/CPU because allocation
from the Xen heap works differently. As a result, I haven’t tested
it on 32-bit in a long time, though it may work for some things. Is your
CPU really 32-bit-only? Or are you just looking at your dom0?</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Also, tmem wasn’t really designed to work in dom0 though,
again, it may work with some functionality limitations. It is intended to
run in PV guests.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">And, the linux-side tmem code is getting old... I need to update
it and also provide better user documentation.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"><br>
To answer your questions: </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"><span>1.<span style="font-family: "Times New Roman"; font-style: normal; font-variant: normal; font-weight: normal; font-size: 7pt; line-height: normal; font-size-adjust: none; font-stretch: normal;">
</span></span></span><span style="font-size: 11pt; color: rgb(31, 73, 125);"> You should not see an increase in system time (in a PV
guest) from tmem because nearly all the work done in tmem is done in the
hypervisor. So I suspect the increase you are seeing is due to other
factors within the dom0 kernel.</span></p>
</div><p><span style="font-size: 11pt; color: rgb(31, 73, 125);"><span>2.<span style="font-family: "Times New Roman"; font-style: normal; font-variant: normal; font-weight: normal; font-size: 7pt; line-height: normal; font-size-adjust: none; font-stretch: normal;">
</span></span></span><span style="font-size: 11pt; color: rgb(31, 73, 125);">The output you provided shows that tmem is not really being used
at all. Assuming you are using Xen-4.0.0 tools, then xentop should also
show summary numbers for tmem (see the new “t” keyboard command),
but only if tmem is actually in use. You can also use “xm tmem-list
--all –long | xen-tmem-list-parse” to see more detailed
information.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Hope that helps!</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Dan</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<div style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color blue; border-width: medium medium medium 1.5pt; padding: 0in 0in 0in 4pt;">
<div>
<div style="border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> Icarus Sonic
[mailto:<a href="mailto:icarus.sonic@gmail.com" target="_blank">icarus.sonic@gmail.com</a>] <br><div class="im">
<b>Sent:</b> Wednesday, May 12, 2010 1:05 AM<br>
</div><b>To:</b> <a href="mailto:tmem-users@oss.oracle.com" target="_blank">tmem-users@oss.oracle.com</a><div class="im"><br>
<b>Subject:</b> [Tmem-users] Experiments with Tmem and trying to find out Tmem
USAGE statistics</div></span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="margin-bottom: 12pt;">1. Experiment data - does it
conform to your expectation?</p><div><div></div><div class="h5"><br>
<br>
Hi, I've installed on a Dell Optilex GX260(32bit x86 , 1G Memory) Xen-4.0.0 and
linux-2.6.18.8-xen kernel dom0 with tmem patched and PRECACHE/PRESWAP opened.<br>
<br>
I did the following experiments and found that tmem does work: I wrote a c
program that touch each page of a 512M 'unsigned char' array, and measured its
running time in dom0. I tried it with three different settings. The result
shows that tmem works but introduce a little bit overhead. Experiment result is
listed as followed:<br>
<br>
<i>Setting 1: tmem off(by booting xen without 'tmem' argument), dom0 with all
available memory close to 1024GB.<br>
Setting 2: tmem off(by booting xen without 'tmem' argument), dom0 with only
256MB memory. xen argument: <b><span style="color: rgb(0, 102, 0);">dom0_mem=256M</span></b><br>
Setting 3: tmem on, dom0 with only 256MB memory. xen argument: <b><span style="color: rgb(0, 102, 0);">dom0_mem=256M tmem</span></b><br>
<br>
With each setting, I ran it three times. Run time is listed in
[real],[user],[sys] format<br>
<br>
Results <br>
==========================================<br>
Setting1: round1(1.503,0.025,1.466), round2(1.487,0.030,1.456),
round3(1.486,0.039,1.444)</i><br>
<i>Setting2: round1(16.591,0.036,2.779), round2(10.434,0.022,2.368)
round3(13.262,0.033,2.305)<br>
Setting3: round1(8.739,0.035,4.616), round2(4.582,0.036,2.852),
round3(4.604,0.043,3.592)<br>
<br>
</i>With setting 2,run time is 10x of that with setting 1, which is caused by swapping.
Results in setting 3 demostrate that tmem alleviate swapping problem but
introduced overhead. <br>
<br>
<br>
2. Question: Where can i found how much tmem is used by a domain, and how much
tmem is available?<br>
Tried xm tmem-list -a | xen-tmem-list-parse to see statistics of tmem usage,
its output:<br>
<br>
total tmem ops=93859 (errors=91550) -- tmem pages avail=0<br>
datastructs: objs=0 (max=0) pgps=0 (max=0) nodes=0 (max=0)<br>
misc: failed_copies=0 alloc_failed=0 alloc_page_failed=0 low_mem=0 evicted=0/0
relinq=0/0, max_evicts_per_relinq=0, flush_pools=0, eph_count=0, eph_max=0<br>
domid0:
weight=0,cap=0,compress=0,frozen=0,total_cycles=61882224,succ_eph_gets=0,succ_pers_puts=0,succ_pers_gets=0,eph_count=0,max_eph=0,compression
ratio=0% (samples=0,poor=0,nomem=0)<br>
domid0,id0[EP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
gets=0/0(0%) flush=0/0 flobj=0/0<br>
domid0,id1[EP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
gets=0/0(0%) flush=0/0 flobj=0/0<br>
domid0,id2[PP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
gets=0/0(0%) flush=0/0 flobj=0/0<br>
failed get cycles: avg=642, max=18856, min=384, samples=91550<br>
flush cycles: avg=785, max=2812, min=668, samples=1966<br>
flush_obj cycles: avg=4623, max=12508, min=696, samples=334<br>
<br>
But it does not tell me how much tmem is used by a domain or how much tmem is
available in hypervisor?<br>
<br>
</div></div>
</div>
</div>
</div>
</blockquote></div><br>