<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:441267545;
mso-list-type:hybrid;
mso-list-template-ids:-951927596 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi Icarus(?) –<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Tmem has very limited functionality on a 32-bit Xen/CPU because allocation
from the Xen heap works differently. As a result, I haven’t tested
it on 32-bit in a long time, though it may work for some things. Is your
CPU really 32-bit-only? Or are you just looking at your dom0?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Also, tmem wasn’t really designed to work in dom0 though,
again, it may work with some functionality limitations. It is intended to
run in PV guests.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>And, the linux-side tmem code is getting old... I need to update
it and also provide better user documentation.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><br>
To answer your questions: <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><span
style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> You should not see an increase in system time (in a PV
guest) from tmem because nearly all the work done in tmem is done in the
hypervisor. So I suspect the increase you are seeing is due to other
factors within the dom0 kernel.<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><span
style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>The output you provided shows that tmem is not really being used
at all. Assuming you are using Xen-4.0.0 tools, then xentop should also
show summary numbers for tmem (see the new “t” keyboard command),
but only if tmem is actually in use. You can also use “xm tmem-list
--all –long | xen-tmem-list-parse” to see more detailed
information.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hope that helps!<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Dan<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Icarus Sonic
[mailto:icarus.sonic@gmail.com] <br>
<b>Sent:</b> Wednesday, May 12, 2010 1:05 AM<br>
<b>To:</b> tmem-users@oss.oracle.com<br>
<b>Subject:</b> [Tmem-users] Experiments with Tmem and trying to find out Tmem
USAGE statistics<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'>1. Experiment data - does it
conform to your expectation?<br>
<br>
Hi, I've installed on a Dell Optilex GX260(32bit x86 , 1G Memory) Xen-4.0.0 and
linux-2.6.18.8-xen kernel dom0 with tmem patched and PRECACHE/PRESWAP opened.<br>
<br>
I did the following experiments and found that tmem does work: I wrote a c
program that touch each page of a 512M 'unsigned char' array, and measured its
running time in dom0. I tried it with three different settings. The result
shows that tmem works but introduce a little bit overhead. Experiment result is
listed as followed:<br>
<br>
<i>Setting 1: tmem off(by booting xen without 'tmem' argument), dom0 with all
available memory close to 1024GB.<br>
Setting 2: tmem off(by booting xen without 'tmem' argument), dom0 with only
256MB memory. xen argument: <b><span style='color:#006600'>dom0_mem=256M</span></b><br>
Setting 3: tmem on, dom0 with only 256MB memory. xen argument: <b><span
style='color:#006600'>dom0_mem=256M tmem</span></b><br>
<br>
With each setting, I ran it three times. Run time is listed in
[real],[user],[sys] format<br>
<br>
Results <br>
==========================================<br>
Setting1: round1(1.503,0.025,1.466), round2(1.487,0.030,1.456),
round3(1.486,0.039,1.444)</i><br>
<i>Setting2: round1(16.591,0.036,2.779), round2(10.434,0.022,2.368)
round3(13.262,0.033,2.305)<br>
Setting3: round1(8.739,0.035,4.616), round2(4.582,0.036,2.852),
round3(4.604,0.043,3.592)<br>
<br>
</i>With setting 2,run time is 10x of that with setting 1, which is caused by swapping.
Results in setting 3 demostrate that tmem alleviate swapping problem but
introduced overhead. <br>
<br>
<br>
2. Question: Where can i found how much tmem is used by a domain, and how much
tmem is available?<br>
Tried xm tmem-list -a | xen-tmem-list-parse to see statistics of tmem usage,
its output:<br>
<br>
total tmem ops=93859 (errors=91550) -- tmem pages avail=0<br>
datastructs: objs=0 (max=0) pgps=0 (max=0) nodes=0 (max=0)<br>
misc: failed_copies=0 alloc_failed=0 alloc_page_failed=0 low_mem=0 evicted=0/0
relinq=0/0, max_evicts_per_relinq=0, flush_pools=0, eph_count=0, eph_max=0<br>
domid0:
weight=0,cap=0,compress=0,frozen=0,total_cycles=61882224,succ_eph_gets=0,succ_pers_puts=0,succ_pers_gets=0,eph_count=0,max_eph=0,compression
ratio=0% (samples=0,poor=0,nomem=0)<br>
domid0,id0[EP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
gets=0/0(0%) flush=0/0 flobj=0/0<br>
domid0,id1[EP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
gets=0/0(0%) flush=0/0 flobj=0/0<br>
domid0,id2[PP]:pgp=0(max=0) obj=0(0) objnode=0(0) puts=0/0/0(dup=0/0)
gets=0/0(0%) flush=0/0 flobj=0/0<br>
failed get cycles: avg=642, max=18856, min=384, samples=91550<br>
flush cycles: avg=785, max=2812, min=668, samples=1966<br>
flush_obj cycles: avg=4623, max=12508, min=696, samples=334<br>
<br>
But it does not tell me how much tmem is used by a domain or how much tmem is
available in hypervisor?<br>
<br>
<o:p></o:p></p>
</div>
</div>
</body>
</html>