[Ocfs2-users] High on buffers and deep on swap

Tue Apr 10 10:58:47 PDT 2007

Alexei,

      I am not a Linux kernel specialist, but generally memory on the "buffer cache" is on a different "list" than memory "free", so the kernel has to reclaim the buffers from the buffer "list" to release them to the "free" "list" as memory pressure increases.

     This has a processing cost. And on large database servers is undesirable as the database does not use the buffer cache, so we have the processing cost without any benefit.

       Usually on a database environment I tune the kernel as to keep a large amount of the memory on the "free" list, so that it be readly available when spawning new database processes.

  Regards,
  Luis

Alexei_Roudnev <Alexei_Roudnev at exigengroup.com> wrote:
      Suni, you do know it much better - is not is strange a little? (so many ext3 inode chache objects, togetrher with 3 GB of cached disk space).

  But I dont see anything wrong below - big 'cached' memory means only 'you have so many unused memory that system cached files in it' and nothing more. Cached memory is in reality _free for immediate use_ memory (can be used by anyone immediately).

  What ysstem is doing is _instead of freeing memory and filling it by zerous, free unused memory and fill it in by buffer cache data, so that if someone need these data he have them at once. If someone need more memory, system reuse any of 'chached' memory without any delay (because these are buffers which are already written to the disk or which was never udpated).

  Big 'cached' value means only _big free memory_.

    ----- O  riginal Message ----- 
  From: Luis Freitas 
  To: Alexei_Roudnev ; Brian Sieler ; ocfs2-users at oss.oracle.com 
  Sent: Monday, April 09, 2007 4:32 PM
  Subject: Re: [Ocfs2-users] High on buffers and deep on swap

  Alexei,

     How can I relate the information on slabtop to the actual memory used by buffers?

     I see this on slabtop:

   Active / Total Objects (% used)    : 603822 / 649643 (92.9%)
 Active / Total Slabs (% used)      : 47216 / 47216 (100.0%)
 Active / Total Caches (% used)     : 97 / 133 (72.9%)
 Active / Total Size (% used)       : 176601.08K / 181508.49K (97.3%)
 Minimum / Average / Maximum Object : 0.01K / 0.28K / 128.00K
    OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
202461 202451  99%    0.54K  28923        7    115692K ext3_inode_cache
232206 232153  99%    0.15K   8931       26     35724K dentry_cache
 29974  26092  87%    0.27K   2141       14      8564K radix_tree_node
 68250  62400  91%    0.05K    910       75      3640K buffer_head
   855    855 100%    4.00K    855        1      3420K pmd
   647    647 100%    4.00K    647        1      2588K size-4096
  8595   7694  89%    0.25K    573       15      2292K filp
 20835  17331  83%    0.09K    463       45      1852K vm_area_struct
   780    767  98%    2.00K    390        2      1560K size-2048
 18849   7962  42%    0.06K    309       61      1236K size-64
  2440    632  25%    0.50K    305        8      1220K size-512
  2926   2889  98%    0.34K    266       11      1064K inode_cache
   256    256 100%    3.00K    128        2      1024K biovec-(256)
   600    592  98%    1.38K    120        5       960K task_struct
   515    512  99%    1.38K    103        5       824K pirpIo
  5084   2365  46%    0.12K    164       31       656K size-128

    The largest area is about 100Mb, but on free there are over 3Gb on the "cached" column:

  [oracle at br001sv0431 ~]$ free
             total       used       free     shared    buffers     cached
Mem:       5190736    4461420     729316          0     141836    3265464
-/+ buffers/cache:    1054120    4136616
Swap:      2048248          0    2048248
[oracle at br001sv0431 ~]$

  Regards,
  Luis

Alexei_Roudnev <Alexei_Roudnev at exigengroup.com> wrote:
      Did you run slabtop ? It can show unreleased buffers in the system.

    ----- Original Message ----- 
  From: Luis Freitas 
  To: Alexei_Roudnev ; Brian Sieler ; ocfs2-users at oss.oracle.com 
  Sent: Monday, April 09, 2007 2:29 PM
  Subject: Re: [Ocfs2-users] High on buffers and deep on swap

  Alexei,

    Yes, it seems to have no effect, which too is very strange. On 2.4 vm.freepages had a very easy to notice effect.

     There are other people having problems with buffers not being released on the list and some of them are forcing the kernel cache to be flushed with:

  echo 3 > /proc/sys/vm/drop_caches

      But I dont see this parameter on RHAS 4.0.

       Also, to be fair this seems to be a generic VM issue, I see this on servers that are not running ocfs2 too. And I only see this behavior on machines with more than 2Gb of memory.

  Regards,
  Luis

  Regards,
Luis

Alexei_Roudnev <Alexei_Roudnev at exigengroup.com> wrote:
      v\:* {   BEHAVIOR: url(#default#VML)  }  o\:* {   BEHAVIOR: url(#default#VML)  }  w\:* {   BEHAVIOR: url(#default#VML)  }  .shape {   BEHAVIOR: url(#default#VML)  }              Did you tried vm.swappiness parameter?

  (/proc/sys/vm/swappiness)

    ----- 
  Original Message ----- 
  From: Brian Sieler 
  To: 'Luis Freitas' ; ocfs2-users at oss.oracle.com 
  Sent: Sunday, April 08, 2007 10:52 PM
  Subject: RE: [Ocfs2-users] High on buffers and deep on swap

    Luis, yes I am experiencing what appears to be a similar problem you are describing. See my post from just a few minutes ago on another thread.

  I run a 2-node cluster with OCFS2/RAC on 2.6.9-34.0.2.ELsmp (RHEL 4.0) as well.

               total       used       free     shared    buffers     cached
  Mem:       4044496    4005516      38980          0      34108    2236636
  -/+ buffers/cache:    1734772    2309724
  Swap:      2097144     648244    1448900

  If you’ve uncovered anything since posting this message, please pass it along?

---------------------------------

  From: ocfs2-users-bounces at oss.oracle.com [mailto:ocfs2-users-bounces at oss.oracle.com] On Behalf Of Luis Freitas
Sent: Friday, February 23, 2007 5:32 PM
To: ocfs2-users at oss.oracle.com
Subject: [Ocfs2-users] High on buffers and deep on swap

    Hi,

       This is a bit off topic, hope there is not a problem.

       Anyone out there experiencing high swapping with the kernel retaining a large amount of buffers? This used to be a problem on 2.4, and I usually changed /proc/sys/vm/freepages to fix it. But on 2.6 this parameter no longer exists.

        One of the servers here is holding over 3.5Gb of cache even when using over 700Mb of swap, and free memory is always low.

    [oracle at br001sv0432 ~]$ free
             total       used       free     shared    buffers     cached
Mem:       5190736    4810880     379856          0     143032    3583868
-/+ buffers/cache:    1083980    4106756
Swap:      2048248     723064    1325184
[oracle at br001sv0432 ~]$

       I am tuning /proc/sys/vm/swappiness, but this seems to have no effect at all. Changed from 60 to 10 and seems to have no effect. The server runs Oracle RAC with OCFS2.

    Regards,

    Luis

---------------------------------

  Food fight? Enjoy some healthy debate
in the Yahoo! Answers Food & Drink Q&A.

---------------------------------

_______________________________________________
Ocfs2-users mailing list
Ocfs2-users at oss.oracle.com
http://oss.oracle.com/mailman/listinfo/ocfs2-users

---------------------------------
  Be a PS3 game guru.
Get your game face on with the latest PS3 news and previews at Yahoo! Games.

---------------------------------
  Finding fabulous fares is fun.
Let Yahoo! FareChase search your favorite travel sites to find flight and hotel bargains.

---------------------------------
Bored stiff? Loosen up...
Download and play hundreds of games for free on Yahoo! Games.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20070410/69c92965/attachment-0001.html