[Ocfs2-users] ocfs2 is still eating memory
Sunil Mushran
Sunil.Mushran at oracle.com
Mon Mar 5 13:46:53 PST 2007
Well, kswapd is supposed to flush the caches. As in, the vm
controls the lifetime of the inodes in the inode_cache not ocfs2.
All ocfs2 can do is free the memory associated with the inode when
asked to. And it does that when you manually flush the cache. Qs is
why the vm is not doing it on its own. (fwiw, you are on a beta kernel.)
John Lange wrote:
> With a large (12 tbyte) ocfs2 file system mounted, all I need to do is a
> "find ." and ocfs2 will slowly consume all ram until oom-killer is
> invoked.
>
> Previously this was thought to be related to the file system being
> exported via nfs but this has now been ruled out.
>
> The following are some stats; first section is after the "find ." had
> been running for about 10 minutes. It is far from out of memory at this
> point but I didn't have time this afternoon to run it until it died.
>
> Second section is right after issuing a
>
> sync ; echo 3 > /proc/sys/vm/drop_caches
>
> so you can see the memory get freed up.
>
> This is running a recent SUSE SP1 BRANCH kernel
>
> 2.6.16.37-SLES10_SP1_BRANCH_20070213192756-smp
>
> I have already been told on the bug tracker that this is not an ocfs2
> issue. I'm not trying to be a pain, I just wanted to repeat my findings
> so that ocfs2 list is aware that while it may not be an ocfs2 code
> problem it certainly does prevent you from using ocfs2 in production.
>
> If someone has a better place where these finding could be reported
> please let me know.
>
>
>
>
> ======= section 1 =========
>
> Mon Mar 5 15:15:01 CST 2007
> # vmstat
> procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
> r b swpd free buff cache si so bi bo in cs us sy id wa st
> 2 1 240 1563768 241264 47076 0 0 1 4 11 6 0 1 94 4 0
> # vmstat -s
> 2075084 total memory
> 511456 used memory
> 182952 active memory
> 223820 inactive memory
> 1563628 free memory
> 241264 buffer memory
> 47076 swap cache
> 2056304 total swap
> 240 used swap
> 2056064 free swap
> 612937 non-nice user cpu ticks
> 7186 nice user cpu ticks
> 2272575 system cpu ticks
> 293987671 idle cpu ticks
> 13392542 IO-wait cpu ticks
> 50629 IRQ cpu ticks
> 1667207 softirq cpu ticks
> 0 steal cpu ticks
> 88657366 pages paged in
> 1816763473 pages paged out
> 0 pages swapped in
> 61 pages swapped out
> 2268215696 interrupts
> 2382140383 CPU context switches
> 1171569943 boot time
> 109385 forks
> # vmstat -m
> Cache Num Total Size Pages
> rpc_buffers 8 8 2048 2
> rpc_tasks 8 15 256 15
> rpc_inode_cache 0 0 512 7
> ocfs2_lock 152 203 16 203
> ocfs2_inode_cache 29868 29868 896 4
> ocfs2_uptodate 6655 6780 32 113
> ocfs2_em_ent 29854 29854 64 59
> dlmfs_inode_cache 1 6 640 6
> dlm_mle_cache 10 10 384 10
> configfs_dir_cache 33 78 48 78
> fib6_nodes 7 113 32 113
> ip6_dst_cache 7 15 256 15
> ndisc_cache 1 15 256 15
> RAWv6 5 6 640 6
> UDPv6 3 6 640 6
> tw_sock_TCPv6 0 0 128 30
> request_sock_TCPv6 0 0 128 30
> TCPv6 10 12 1280 3
> ip_fib_alias 16 113 32 113
> ip_fib_hash 16 113 32 113
> dm_events 16 169 20 169
> Cache Num Total Size Pages
> dm_tio 3676 3857 16 203
> dm_io 3695 3887 20 169
> uhci_urb_priv 0 0 40 92
> ext3_inode_cache 950 1912 512 8
> ext3_xattr 0 0 48 78
> journal_handle 2 169 20 169
> journal_head 462 648 52 72
> revoke_table 6 254 12 254
> revoke_record 0 0 16 203
> qla2xxx_srbs 242 270 128 30
> scsi_cmd_cache 59 90 384 10
> sgpool-256 32 32 4096 1
> sgpool-128 32 32 2048 2
> sgpool-64 32 32 1024 4
> sgpool-32 32 32 512 8
> sgpool-16 32 45 256 15
> sgpool-8 156 210 128 30
> scsi_io_context 0 0 104 37
> UNIX 410 427 512 7
> ip_mrt_cache 0 0 128 30
> tcp_bind_bucket 12 203 16 203
> Cache Num Total Size Pages
> inet_peer_cache 89 118 64 59
> secpath_cache 0 0 128 30
> xfrm_dst_cache 0 0 384 10
> ip_dst_cache 178 285 256 15
> arp_cache 5 15 256 15
> RAW 3 7 512 7
> UDP 32 56 512 7
> tw_sock_TCP 0 0 128 30
> request_sock_TCP 0 0 64 59
> TCP 16 35 1152 7
> flow_cache 0 0 128 30
> cfq_ioc_pool 184 720 96 40
> cfq_pool 166 600 96 40
> crq_pool 181 312 48 78
> deadline_drq 0 0 52 72
> as_arq 0 0 64 59
> mqueue_inode_cache 1 6 640 6
> isofs_inode_cache 0 0 384 10
> minix_inode_cache 0 0 420 9
> hugetlbfs_inode_cache 1 11 356 11
> ext2_inode_cache 0 0 492 8
> Cache Num Total Size Pages
> ext2_xattr 0 0 48 78
> dnotify_cache 1 169 20 169
> dquot 0 0 128 30
> eventpoll_pwq 17 101 36 101
> eventpoll_epi 17 30 128 30
> inotify_event_cache 0 0 28 127
> inotify_watch_cache 40 184 40 92
> kioctx 0 0 256 15
> kiocb 0 0 128 30
> fasync_cache 1 203 16 203
> shmem_inode_cache 618 624 460 8
> posix_timers_cache 0 0 100 39
> uid_cache 7 59 64 59
> blkdev_ioc 88 254 28 127
> blkdev_queue 58 60 960 4
> blkdev_requests 204 264 176 22
> biovec-(256) 312 312 3072 2
> biovec-128 368 370 1536 5
> biovec-64 480 485 768 5
> biovec-16 480 495 256 15
> biovec-4 480 531 64 59
> Cache Num Total Size Pages
> biovec-1 609 5684 16 203
> bio 592 720 128 30
> sock_inode_cache 505 553 512 7
> skbuff_fclone_cache 73 80 384 10
> skbuff_head_cache 673 870 256 15
> file_lock_cache 8 126 92 42
> acpi_operand 634 828 40 92
> acpi_parse_ext 0 0 44 84
> acpi_parse 0 0 28 127
> acpi_state 0 0 48 78
> delayacct_cache 191 312 48 78
> taskstats_cache 32 32 236 16
> proc_inode_cache 59 140 372 10
> sigqueue 97 135 144 27
> radix_tree_node 87360 87360 276 14
> bdev_cache 56 56 512 7
> sysfs_dir_cache 4887 4968 40 92
> mnt_cache 29 60 128 30
> inode_cache 1071 1309 356 11
> dentry_cache 34133 34133 132 29
> filp 3048 4180 192 20
> Cache Num Total Size Pages
> names_cache 26 26 4096 1
> idr_layer_cache 203 232 136 29
> buffer_head 60840 60840 52 72
> mm_struct 120 171 448 9
> vm_area_struct 5831 8888 88 44
> fs_cache 120 236 64 59
> files_cache 121 198 448 9
> signal_cache 166 200 384 10
> sighand_cache 157 177 1344 3
> task_struct 182 230 1376 5
> anon_vma 2781 3048 12 254
> pgd 97 97 4096 1
> size-131072(DMA) 0 0 131072 1
> size-131072 0 0 131072 1
> size-65536(DMA) 0 0 65536 1
> size-65536 0 0 65536 1
> size-32768(DMA) 0 0 32768 1
> size-32768 3 3 32768 1
> size-16384(DMA) 0 0 16384 1
> size-16384 21 21 16384 1
> size-8192(DMA) 0 0 8192 1
> Cache Num Total Size Pages
> size-8192 175 175 8192 1
> size-4096(DMA) 0 0 4096 1
> size-4096 111 111 4096 1
> size-2048(DMA) 0 0 2048 2
> size-2048 682 708 2048 2
> size-1024(DMA) 0 0 1024 4
> size-1024 373 404 1024 4
> size-512(DMA) 1 8 512 8
> size-512 529 568 512 8
> size-256(DMA) 0 0 256 15
> size-256 30690 30690 256 15
> size-128(DMA) 0 0 128 30
> size-128 33360 33360 128 30
> size-64(DMA) 0 0 64 59
> size-32(DMA) 0 0 32 113
> size-64 4166 7080 64 59
> size-32 35030 35030 32 113
> kmem_cache 150 150 256 15
>
>
> ======== section 2 ========
>
> Mon Mar 5 15:15:28 CST 2007
> # vmstat
> procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
> r b swpd free buff cache si so bi bo in cs us sy id wa st
> 0 1 240 1873872 5460 42120 0 0 1 4 11 6 0 1 94 4 0
> # vmstat -s
> 2075084 total memory
> 201212 used memory
> 161028 active memory
> 4184 inactive memory
> 1873872 free memory
> 5460 buffer memory
> 42120 swap cache
> 2056304 total swap
> 240 used swap
> 2056064 free swap
> 612951 non-nice user cpu ticks
> 7186 nice user cpu ticks
> 2272690 system cpu ticks
> 293990233 idle cpu ticks
> 13395382 IO-wait cpu ticks
> 50630 IRQ cpu ticks
> 1667219 softirq cpu ticks
> 0 steal cpu ticks
> 88673230 pages paged in
> 1816764408 pages paged out
> 0 pages swapped in
> 61 pages swapped out
> 2268233528 interrupts
> 2382233674 CPU context switches
> 1171569943 boot time
> 109411 forks
> # vmstat -m
> Cache Num Total Size Pages
> rpc_buffers 8 8 2048 2
> rpc_tasks 8 15 256 15
> rpc_inode_cache 0 0 512 7
> ocfs2_lock 152 203 16 203
> ocfs2_inode_cache 452 452 896 4
> ocfs2_uptodate 6496 6780 32 113
> ocfs2_em_ent 523 1357 64 59
> dlmfs_inode_cache 1 6 640 6
> dlm_mle_cache 10 20 384 10
> configfs_dir_cache 33 78 48 78
> fib6_nodes 7 113 32 113
> ip6_dst_cache 7 15 256 15
> ndisc_cache 1 15 256 15
> RAWv6 5 6 640 6
> UDPv6 3 6 640 6
> tw_sock_TCPv6 0 0 128 30
> request_sock_TCPv6 0 0 128 30
> TCPv6 10 12 1280 3
> ip_fib_alias 16 113 32 113
> ip_fib_hash 16 113 32 113
> dm_events 16 169 20 169
> Cache Num Total Size Pages
> dm_tio 3718 3857 16 203
> dm_io 3663 3887 20 169
> uhci_urb_priv 0 0 40 92
> ext3_inode_cache 859 1904 512 8
> ext3_xattr 0 0 48 78
> journal_handle 18 169 20 169
> journal_head 127 648 52 72
> revoke_table 6 254 12 254
> revoke_record 0 0 16 203
> qla2xxx_srbs 211 270 128 30
> scsi_cmd_cache 86 90 384 10
> sgpool-256 32 32 4096 1
> sgpool-128 32 32 2048 2
> sgpool-64 32 32 1024 4
> sgpool-32 40 40 512 8
> sgpool-16 49 60 256 15
> sgpool-8 159 210 128 30
> scsi_io_context 0 0 104 37
> UNIX 394 427 512 7
> ip_mrt_cache 0 0 128 30
> tcp_bind_bucket 12 203 16 203
> Cache Num Total Size Pages
> inet_peer_cache 89 118 64 59
> secpath_cache 0 0 128 30
> xfrm_dst_cache 0 0 384 10
> ip_dst_cache 169 285 256 15
> arp_cache 6 15 256 15
> RAW 3 7 512 7
> UDP 32 56 512 7
> tw_sock_TCP 0 0 128 30
> request_sock_TCP 0 0 64 59
> TCP 16 35 1152 7
> flow_cache 0 0 128 30
> cfq_ioc_pool 261 720 96 40
> cfq_pool 243 600 96 40
> crq_pool 241 312 48 78
> deadline_drq 0 0 52 72
> as_arq 0 0 64 59
> mqueue_inode_cache 1 6 640 6
> isofs_inode_cache 0 0 384 10
> minix_inode_cache 0 0 420 9
> hugetlbfs_inode_cache 1 11 356 11
> ext2_inode_cache 0 0 492 8
> Cache Num Total Size Pages
> ext2_xattr 0 0 48 78
> dnotify_cache 1 169 20 169
> dquot 0 0 128 30
> eventpoll_pwq 1 101 36 101
> eventpoll_epi 1 30 128 30
> inotify_event_cache 0 127 28 127
> inotify_watch_cache 40 184 40 92
> kioctx 0 0 256 15
> kiocb 0 0 128 30
> fasync_cache 1 203 16 203
> shmem_inode_cache 618 624 460 8
> posix_timers_cache 0 0 100 39
> uid_cache 7 59 64 59
> blkdev_ioc 150 254 28 127
> blkdev_queue 58 60 960 4
> blkdev_requests 211 264 176 22
> biovec-(256) 312 312 3072 2
> biovec-128 368 370 1536 5
> biovec-64 485 485 768 5
> biovec-16 495 495 256 15
> biovec-4 531 531 64 59
> Cache Num Total Size Pages
> biovec-1 723 5684 16 203
> bio 654 750 128 30
> sock_inode_cache 473 553 512 7
> skbuff_fclone_cache 78 80 384 10
> skbuff_head_cache 673 870 256 15
> file_lock_cache 12 126 92 42
> acpi_operand 634 828 40 92
> acpi_parse_ext 0 0 44 84
> acpi_parse 0 0 28 127
> acpi_state 0 0 48 78
> delayacct_cache 236 312 48 78
> taskstats_cache 12 32 236 16
> proc_inode_cache 21 140 372 10
> sigqueue 93 135 144 27
> radix_tree_node 3225 9464 276 14
> bdev_cache 56 56 512 7
> sysfs_dir_cache 4887 4968 40 92
> mnt_cache 29 60 128 30
> inode_cache 1055 1309 356 11
> dentry_cache 3351 9541 132 29
> filp 3096 4180 192 20
> Cache Num Total Size Pages
> names_cache 26 26 4096 1
> idr_layer_cache 203 232 136 29
> buffer_head 1577 11592 52 72
> mm_struct 120 171 448 9
> vm_area_struct 5748 8888 88 44
> fs_cache 210 236 64 59
> files_cache 121 198 448 9
> signal_cache 150 200 384 10
> sighand_cache 157 177 1344 3
> task_struct 182 230 1376 5
> anon_vma 2751 3048 12 254
> pgd 96 96 4096 1
> size-131072(DMA) 0 0 131072 1
> size-131072 0 0 131072 1
> size-65536(DMA) 0 0 65536 1
> size-65536 0 0 65536 1
> size-32768(DMA) 0 0 32768 1
> size-32768 3 3 32768 1
> size-16384(DMA) 0 0 16384 1
> size-16384 21 21 16384 1
> size-8192(DMA) 0 0 8192 1
> Cache Num Total Size Pages
> size-8192 169 169 8192 1
> size-4096(DMA) 0 0 4096 1
> size-4096 109 109 4096 1
> size-2048(DMA) 0 0 2048 2
> size-2048 682 708 2048 2
> size-1024(DMA) 0 0 1024 4
> size-1024 373 404 1024 4
> size-512(DMA) 1 8 512 8
> size-512 491 560 512 8
> size-256(DMA) 0 0 256 15
> size-256 1560 2295 256 15
> size-128(DMA) 0 0 128 30
> size-128 3870 17580 128 30
> size-64(DMA) 0 0 64 59
> size-32(DMA) 0 0 32 113
> size-64 4118 7080 64 59
> size-32 5920 19097 32 113
> kmem_cache 150 150 256 15
>
>
> ================
>
>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
>
More information about the Ocfs2-users
mailing list