[Ocfs2-users] High on buffers and deep on swap
Eckenfels. Bernd
B.Eckenfels at seeburger.de
Tue Apr 10 11:11:34 PDT 2007
Hello Luis,
the large usage of cache by inodes is not an issue in the typical
database setup, because you have only a very limited number of files
(used) on your filesystems. Also the database usually grows initially
and then does not request much more additional memory in operations.
However you are right, Oracle with Data Files on ocfsv2 has no need to
cache the data blocks on the FS layer. But I gues sthis is not done
(memadvice, fcntl or similiar methods can be used to enforce "directio"
- i am not sure which one is used by Oracle). I think the original
problem reports with large usage have been due to non-db workloads.
Gruss
Bernd
________________________________
From: ocfs2-users-bounces at oss.oracle.com
[mailto:ocfs2-users-bounces at oss.oracle.com] On Behalf Of Luis Freitas
Sent: Tuesday, April 10, 2007 7:59 PM
To: Alexei_Roudnev; ocfs2-users at oss.oracle.com
Subject: Re: [Ocfs2-users] High on buffers and deep on swap
Alexei,
I am not a Linux kernel specialist, but generally memory on the
"buffer cache" is on a different "list" than memory "free", so the
kernel has to reclaim the buffers from the buffer "list" to release them
to the "free" "list" as memory pressure increases.
This has a processing cost. And on large database servers is
undesirable as the database does not use the buffer cache, so we have
the processing cost without any benefit.
Usually on a database environment I tune the kernel as to keep a
large amount of the memory on the "free" list, so that it be readly
available when spawning new database processes.
Regards,
Luis
Alexei_Roudnev <Alexei_Roudnev at exigengroup.com> wrote:
Suni, you do know it much better - is not is strange a little?
(so many ext3 inode chache objects, togetrher with 3 GB of cached disk
space).
But I dont see anything wrong below - big 'cached' memory means
only 'you have so many unused memory that system cached files in it' and
nothing more. Cached memory is in reality _free for immediate use_
memory (can be used by anyone immediately).
What ysstem is doing is _instead of freeing memory and filling
it by zerous, free unused memory and fill it in by buffer cache data, so
that if someone need these data he have them at once. If someone need
more memory, system reuse any of 'chached' memory without any delay
(because these are buffers which are already written to the disk or
which was never udpated).
Big 'cached' value means only _big free memory_.
----- O riginal Message -----
From: Luis Freitas <mailto:lfreitas34 at yahoo.com>
To: Alexei_Roudnev
<mailto:Alexei_Roudnev at exigengroup.com> ; Brian Sieler
<mailto:blsieler at gmail.com> ; ocfs2-users at oss.oracle.com
Sent: Monday, April 09, 2007 4:32 PM
Subject: Re: [Ocfs2-users] High on buffers and deep on
swap
Alexei,
How can I relate the information on slabtop to the
actual memory used by buffers?
I see this on slabtop:
Active / Total Objects (% used) : 603822 / 649643
(92.9%)
Active / Total Slabs (% used) : 47216 / 47216
(100.0%)
Active / Total Caches (% used) : 97 / 133 (72.9%)
Active / Total Size (% used) : 176601.08K /
181508.49K (97.3%)
Minimum / Average / Maximum Object : 0.01K / 0.28K /
128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE
NAME
202461 202451 99% 0.54K 28923 7 115692K
ext3_inode_cache
232206 232153 99% 0.15K 8931 26 35724K
dentry_cache
29974 26092 87% 0.27K 2141 14 8564K
radix_tree_node
68250 62400 91% 0.05K 910 75 3640K
buffer_head
855 855 100% 4.00K 855 1 3420K
pmd
647 647 100% 4.00K 647 1 2588K
size-4096
8595 7694 89% 0.25K 573 15 2292K
filp
20835 17331 83% 0.09K 463 45 1852K
vm_area_struct
780 767 98% 2.00K 390 2 1560K
size-2048
18849 7962 42% 0.06K 309 61 1236K
size-64
2440 632 25% 0.50K 305 8 1220K
size-512
2926 2889 98% 0.34K 266 11 1064K
inode_cache
256 256 100% 3.00K 128 2 1024K
biovec-(256)
600 592 98% 1.38K 120 5 960K
task_struct
515 512 99% 1.38K 103 5 824K
pirpIo
5084 2365 46% 0.12K 164 31 656K
size-128
The largest area is about 100Mb, but on free there are
over 3Gb on the "cached" column:
[oracle at br001sv0431 ~]$ free
total used free shared
buffers cached
Mem: 5190736 4461420 729316 0
141836 3265464
-/+ buffers/cache: 1054120 4136616
Swap: 2048248 0 2048248
[oracle at br001sv0431 ~]$
Regards,
Luis
Alexei_Roudnev <Alexei_Roudnev at exigengroup.com> wrote:
Did you run slabtop ? It can show unreleased
buffers in the system.
----- Original Message -----
From: Luis Freitas
<mailto:lfreitas34 at yahoo.com>
To: Alexei_Roudnev
<mailto:Alexei_Roudnev at exigengroup.com> ; Brian Sieler
<mailto:blsieler at gmail.com> ; ocfs2-users at oss.oracle.com
Sent: Monday, April 09, 2007 2:29 PM
Subject: Re: [Ocfs2-users] High on
buffers and deep on swap
Alexei,
Yes, it seems to have no effect, which
too is very strange. On 2.4 vm.freepages had a very easy to notice
effect.
There are other people having
problems with buffers not being released on the list and some of them
are forcing the kernel cache to be flushed with:
echo 3 > /proc/sys/vm/drop_caches
But I dont see this parameter on
RHAS 4.0.
Also, to be fair this seems to be a
generic VM issue, I see this on servers that are not running ocfs2 too.
And I only see this behavior on machines with more than 2Gb of memory.
Regards,
Luis
Regards,
Luis
Alexei_Roudnev
<Alexei_Roudnev at exigengroup.com> wrote:
Did you tried vm.swappiness parameter?
(/proc/sys/vm/swappiness)
-----
Original Message -----
From: Brian Sieler
<mailto:blsieler at gmail.com>
To: 'Luis Freitas'
<mailto:lfreitas34 at yahoo.com> ; ocfs2-users at oss.oracle.com
Sent: Sunday, April 08, 2007 10:52 PM
Subject: RE: [Ocfs2-users] High on
buffers and deep on swap
Luis, yes I am experiencing what appears
to be a similar problem you are describing. See my post from just a few
minutes ago on another thread.
I run a 2-node cluster with OCFS2/RAC on
2.6.9-34.0.2.ELsmp (RHEL 4.0) as well.
total used free
shared buffers cached
Mem: 4044496 4005516 38980
0 34108 2236636
-/+ buffers/cache: 1734772 2309724
Swap: 2097144 648244 1448900
If you've uncovered anything since
posting this message, please pass it along?
________________________________
From: ocfs2-users-bounces at oss.oracle.com
[mailto:ocfs2-users-bounces at oss.oracle.com] On Behalf Of Luis Freitas
Sent: Friday, February 23, 2007 5:32 PM
To: ocfs2-users at oss.oracle.com
Subject: [Ocfs2-users] High on buffers
and deep on swap
Hi,
This is a bit off topic, hope there
is not a problem.
Anyone out there experiencing high
swapping with the kernel retaining a large amount of buffers? This used
to be a problem on 2.4, and I usually changed /proc/sys/vm/freepages to
fix it. But on 2.6 this parameter no longer exists.
One of the servers here is holding
over 3.5Gb of cache even when using over 700Mb of swap, and free memory
is always low.
[oracle at br001sv0432 ~]$ free
total used free
shared buffers cached
Mem: 5190736 4810880 379856
0 143032 3583868
-/+ buffers/cache: 1083980 4106756
Swap: 2048248 723064 1325184
[oracle at br001sv0432 ~]$
I am tuning /proc/sys/vm/swappiness,
but this seems to have no effect at all. Changed from 60 to 10 and seems
to have no effect. The server runs Oracle RAC with OCFS2.
Regards,
Luis
________________________________
Food fight?
<http://answers.yahoo.com/dir/index;_ylc=X3oDMTFvbGNhMGE3BF9TAzM5NjU0NTE
wOARfcwMzOTY1NDUxMDMEc2VjA21haWxfdGFnbGluZQRzbGsDbWFpbF90YWcx?link=ask&s
id=396545367> Enjoy some healthy debate
in the Yahoo! Answers Food & Drink Q&A.
<http://answers.yahoo.com/dir/index;_ylc=X3oDMTFvbGNhMGE3BF9TAzM5NjU0NTE
wOARfcwMzOTY1NDUxMDMEc2VjA21haWxfdGFnbGluZQRzbGsDbWFpbF90YWcx?link=ask&s
id=396545367>
________________________________
_______________________________________________
Ocfs2-users mailing list
Ocfs2-users at oss.oracle.com
http://oss.oracle.com/mailman/listinfo/ocfs2-users
________________________________
Be a PS3 game guru.
Get your game face on with the latest
PS3 news and previews at Yahoo! Games.
<http://us.rd.yahoo.com/evt=49936/*http://videogames.yahoo.com>
________________________________
Finding fabulous fares is fun.
Let Yahoo! FareChase search your favorite travel sites
<http://farechase.yahoo.com/promo-generic-14795097;_ylc=X3oDMTFtNW45amVp
BF9TAzk3NDA3NTg5BF9zAzI3MTk0ODEEcG9zAzEEc2VjA21haWx0YWdsaW5lBHNsawNxMS0w
Nw--> to find flight and hotel bargains.
________________________________
Bored stiff? <http://us.rd.yahoo.com/evt=49935/*http://games.yahoo.com>
Loosen up...
Download and play hundreds of games for free
<http://us.rd.yahoo.com/evt=49935/*http://games.yahoo.com> on Yahoo!
Games.
SEEBURGER AG
Headquarters:
Edisonstraße 1
D-75015 Bretten
Tel.: 0 72 52/96-0
Fax: 0 72 52/96-2222
Internet: http://www.seeburger.de
e-mail: info at seeburger.de
Vorstand:
Bernd Seeburger, Axel Haas, Michael Kleeberg
Vorsitzender des Aufsichtsrats:
Dr. Franz Scherer
Handelsregister:
HRB 240708 Mannheim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20070410/e44677b2/attachment-0001.html
More information about the Ocfs2-users
mailing list