[Ocfs2-users] Space not reclaimed after deletion of files.
Guy
wyldfury at gmail.com
Tue Oct 27 07:47:59 PDT 2009
Hi,
I've got two machines connecting to an ocfs2 filesystem on an iSCSI
target provided by our hosting company (the iSCSI target is on an HP
SAN). After a bit of usage any deleted files disappear, but checking
the usage with "df" shows that space is not being reclaimed.
I've unmounted the partition from both nodes and run "fsck.ocfs2 -fy
/dev/sdb1" which found a whole lot of orphaned inodes which is
apparently fixed.
Remounting showed that the space still was not reclaimed. I did some
testing to see whether the information supplied by "df" was simply
incorrect, but once the partition showed as full I could not write
anything more to it.
After the weekend I tried to remount the full partition and had the
attached kernel errors splash across the screen. I had to power cycle
the server through the hosting company's interface to get it
responsive again. I then had to do a mkfs.ocfs2 to be able to mount
the partition without those kernel errors.
OS: Ubuntu 9.04 (Jaunty)
ocfs2: 1.5.0 (from "modinfo ocfs2")
ocfs2-tools: 1.3.9-0ubuntu3
Is there any more information I can provide that might help with this?
Thanks
Guy
--
Don't just do something...sit there!
-------------- next part --------------
stoat kernel: [ 244.947060] general protection fault: 0000 [#1] SMP
stoat kernel: [ 244.947060] general protection fault: 0000 [#1] SMP
stoat kernel: [ 244.947180] last sysfs file: /sys/fs/o2cb/interface_revision
stoat kernel: [ 244.947180] last sysfs file: /sys/fs/o2cb/interface_revision
stoat kernel: [ 244.949863] Stack:
stoat kernel: [ 244.949863] Stack:
stoat kernel: [ 244.950065] ffff88007185bc50 ffff88007f20c690 ffff88007185bc30 ffffffff802b3081
stoat kernel: [ 244.950065] ffff88007185bc50 ffff88007f20c690 ffff88007185bc30 ffffffff802b3081
stoat kernel: [ 244.950291] ffff88007185bcb0 ffffffff802b31e3 ffff88007185bc68 00000010802b3081
stoat kernel: [ 244.950291] ffff88007185bcb0 ffffffff802b31e3 ffff88007185bc68 00000010802b3081
stoat kernel: [ 244.950562] Call Trace:
stoat kernel: [ 244.950562] Call Trace:
stoat kernel: [ 244.950614] [<ffffffff802b3081>] mempool_alloc_slab+0x11/0x20
stoat kernel: [ 244.950614] [<ffffffff802b3081>] mempool_alloc_slab+0x11/0x20
stoat kernel: [ 244.950704] [<ffffffff802b31e3>] mempool_alloc+0x53/0x130
stoat kernel: [ 244.950704] [<ffffffff802b31e3>] mempool_alloc+0x53/0x130
stoat kernel: [ 244.950792] [<ffffffff802b31e3>] ? mempool_alloc+0x53/0x130
stoat kernel: [ 244.950792] [<ffffffff802b31e3>] ? mempool_alloc+0x53/0x130
stoat kernel: [ 244.950881] [<ffffffff803112fc>] bvec_alloc_bs+0x6c/0x140
stoat kernel: [ 244.950881] [<ffffffff803112fc>] bvec_alloc_bs+0x6c/0x140
stoat kernel: [ 244.950970] [<ffffffff80311429>] bio_alloc_bioset+0x59/0x100
stoat kernel: [ 244.950970] [<ffffffff80311429>] bio_alloc_bioset+0x59/0x100
stoat kernel: [ 244.951059] [<ffffffff80311580>] bio_alloc+0x10/0x20
stoat kernel: [ 244.951059] [<ffffffff80311580>] bio_alloc+0x10/0x20
stoat kernel: [ 244.951146] [<ffffffff8030c8d0>] submit_bh+0x70/0x130
stoat kernel: [ 244.951146] [<ffffffff8030c8d0>] submit_bh+0x70/0x130
stoat kernel: [ 244.951234] [<ffffffffa02fd8dd>] ocfs2_read_blocks+0x29d/0x790 [ocfs2]
stoat kernel: [ 244.951234] [<ffffffffa02fd8dd>] ocfs2_read_blocks+0x29d/0x790 [ocfs2]
stoat kernel: [ 244.951336] [<ffffffffa03373f8>] ? _ocfs2_get_system_file_inode+0xa8/0x180 [ocfs2]
stoat kernel: [ 244.951336] [<ffffffffa03373f8>] ? _ocfs2_get_system_file_inode+0xa8/0x180 [ocfs2]
stoat kernel: [ 244.951457] [<ffffffffa0319c4c>] ocfs2_read_journal_inode+0x9c/0x190 [ocfs2]
stoat kernel: [ 244.951457] [<ffffffffa0319c4c>] ocfs2_read_journal_inode+0x9c/0x190 [ocfs2]
stoat kernel: [ 244.951560] [<ffffffffa0319ddd>] ocfs2_check_journals_nolocks+0x9d/0x140 [ocfs2]
stoat kernel: [ 244.951560] [<ffffffffa0319ddd>] ocfs2_check_journals_nolocks+0x9d/0x140 [ocfs2]
stoat kernel: [ 244.951678] [<ffffffffa0308d68>] ? ocfs2_super_lock+0x188/0x350 [ocfs2]
stoat kernel: [ 244.951678] [<ffffffffa0308d68>] ? ocfs2_super_lock+0x188/0x350 [ocfs2]
stoat kernel: [ 244.951779] [<ffffffffa031c447>] __ocfs2_recovery_thread+0x397/0x600 [ocfs2]
stoat kernel: [ 244.951779] [<ffffffffa031c447>] __ocfs2_recovery_thread+0x397/0x600 [ocfs2]
stoat kernel: [ 244.951881] [<ffffffff80268920>] ? autoremove_wake_function+0x0/0x40
stoat kernel: [ 244.951881] [<ffffffff80268920>] ? autoremove_wake_function+0x0/0x40
stoat kernel: [ 244.951973] [<ffffffffa031c0b0>] ? __ocfs2_recovery_thread+0x0/0x600 [ocfs2]
stoat kernel: [ 244.951973] [<ffffffffa031c0b0>] ? __ocfs2_recovery_thread+0x0/0x600 [ocfs2]
stoat kernel: [ 244.952075] [<ffffffff802684b9>] kthread+0x49/0x90
stoat kernel: [ 244.952075] [<ffffffff802684b9>] kthread+0x49/0x90
stoat kernel: [ 244.952162] [<ffffffff80213979>] child_rip+0xa/0x11
stoat kernel: [ 244.952162] [<ffffffff80213979>] child_rip+0xa/0x11
stoat kernel: [ 244.952250] [<ffffffff80268470>] ? kthread+0x0/0x90
stoat kernel: [ 244.952250] [<ffffffff80268470>] ? kthread+0x0/0x90
stoat kernel: [ 244.952337] [<ffffffff8021396f>] ? child_rip+0x0/0x11
stoat kernel: [ 244.952337] [<ffffffff8021396f>] ? child_rip+0x0/0x11
stoat kernel: [ 244.952425] Code: fa 0f 1f 80 00 00 00 00 65 8b 04 25 24 00 00 00 48 98 49 8b 94 c5 e8 00 00 00 8b 7a 18 89 7d d4 48 8b 1a 48 85 db 74 47 8b 42 14 <48> 8b 04 c3 48 89 02 4c 89 e7 57 9d 66 0f 1f 44 00 00 66 45 85
stoat kernel: [ 244.952425] Code: fa 0f 1f 80 00 00 00 00 65 8b 04 25 24 00 00 00 48 98 49 8b 94 c5 e8 00 00 00 8b 7a 18 89 7d d4 48 8b 1a 48 85 db 74 47 8b 42 14 <48> 8b 04 c3 48 89 02 4c 89 e7 57 9d 66 0f 1f 44 00 00 66 45 85
stoat kernel: [ 244.966137] general protection fault: 0000 [#2] SMP
stoat kernel: [ 244.966137] general protection fault: 0000 [#2] SMP
stoat kernel: [ 244.966271] last sysfs file: /sys/fs/o2cb/interface_revision
stoat kernel: [ 244.966271] last sysfs file: /sys/fs/o2cb/interface_revision
stoat kernel: [ 244.968820] Stack:
stoat kernel: [ 244.968820] Stack:
stoat kernel: [ 244.969022] ffff880055d97ae8 ffff88007f20c690 ffff880055d97ac8 ffffffff802b3081
stoat kernel: [ 244.969022] ffff880055d97ae8 ffff88007f20c690 ffff880055d97ac8 ffffffff802b3081
stoat kernel: [ 244.969247] ffff880055d97b48 ffffffff802b31e3 ffff880055d97b00 00000010802b3081
stoat kernel: [ 244.969247] ffff880055d97b48 ffffffff802b31e3 ffff880055d97b00 00000010802b3081
stoat kernel: [ 244.969519] Call Trace:
stoat kernel: [ 244.969570] [<ffffffff802b3081>] mempool_alloc_slab+0x11/0x20
stoat kernel: [ 244.969659] [<ffffffff802b31e3>] mempool_alloc+0x53/0x130
stoat kernel: [ 244.969748] [<ffffffff802b31e3>] ? mempool_alloc+0x53/0x130
stoat kernel: [ 244.969836] [<ffffffff803112fc>] bvec_alloc_bs+0x6c/0x140
stoat kernel: [ 244.969925] [<ffffffff80311429>] bio_alloc_bioset+0x59/0x100
stoat kernel: [ 244.970014] [<ffffffff80311580>] bio_alloc+0x10/0x20
stoat kernel: [ 244.970102] [<ffffffff80313ea3>] dio_new_bio+0x63/0x130
stoat kernel: [ 244.970189] [<ffffffff8031428f>] dio_send_cur_page+0xaf/0xc0
stoat kernel: [ 244.970278] [<ffffffff80314b5c>] direct_io_worker+0x21c/0x390
stoat kernel: [ 244.970367] [<ffffffff80314f0f>] __blockdev_direct_IO+0x23f/0x380
stoat kernel: [ 244.970457] [<ffffffff80311f10>] ? blkdev_get_blocks+0x0/0xc0
stoat kernel: [ 244.970546] [<ffffffff803137d9>] blkdev_direct_IO+0x49/0x50
stoat kernel: [ 244.970634] [<ffffffff80311f10>] ? blkdev_get_blocks+0x0/0xc0
stoat kernel: [ 244.970723] [<ffffffff802b26e0>] generic_file_aio_read+0x1c0/0x1e0
stoat kernel: [ 244.970812] [<ffffffff8022f669>] ? default_spin_lock_flags+0x9/0x10
stoat kernel: [ 244.970903] [<ffffffff802e7911>] do_sync_read+0xf1/0x140
stoat kernel: [ 244.970992] [<ffffffff8069db3e>] ? do_page_fault+0x32e/0x780
stoat kernel: [ 244.971081] [<ffffffff80268920>] ? autoremove_wake_function+0x0/0x40
stoat kernel: [ 244.971173] [<ffffffff80312d48>] ? __blkdev_put+0x88/0x1d0
stoat kernel: [ 244.971261] [<ffffffff80317df8>] ? inotify_inode_queue_event+0xd8/0x100
stoat kernel: [ 244.971352] [<ffffffff803f6083>] ? apparmor_file_permission+0x23/0x30
stoat kernel: [ 244.971443] [<ffffffff803d0751>] ? security_file_permission+0x11/0x20
stoat kernel: [ 244.969519] Call Trace:
stoat kernel: [ 244.969570] [<ffffffff802b3081>] mempool_alloc_slab+0x11/0x20
stoat kernel: [ 244.969659] [<ffffffff802b31e3>] mempool_alloc+0x53/0x130
stoat kernel: [ 244.969748] [<ffffffff802b31e3>] ?
stoat kernel: [ 244.971536] [<ffffffff802e80f8>] vfs_read+0xc8/0x130
stoat kernel: [ 244.971623] [<ffffffff802e81fa>] sys_pread64+0x9a/0xa0
stoat kernel: [ 244.971711] [<ffffffff8021253a>] system_call_fastpath+0x16/0x1b
stoat kernel: [ 244.971800] Code: fa 0f 1f 80 00 00 00 00 65 8b 04 25 24 00 00 00 48 98 49 8b 94 c5 e8 00 00 00 8b 7a 18 89 7d d4 48 8b 1a 48 85 db 74 47 8b 42 14 <48> 8b 04 c3 48 89 02 4c 89 e7 57 9d 66 0f 1f 44 00 00 66 45 85
stoat kernel: [ 245.011365] general protection fault: 0000 [#3] SMP
stoat kernel: [ 245.011483] last sysfs file: /sys/fs/o2cb/interface_revision
stoat kernel: [ 245.014059] Stack:
stoat kernel: [ 245.014260] ffff88007d257980 ffff88007f20c690 ffff88007d257960 ffffffff802b3081
stoat kernel: [ 245.014485] ffff88007d2579e0 ffffffff802b31e3 ffff88007d257998 00000010802b3081
stoat kernel: [ 245.014757] Call Trace:
stoat kernel: [ 245.014809] [<ffffffff802b3081>] mempool_alloc_slab+0x11/0x20
stoat kernel: [ 245.014898] [<ffffffff802b31e3>] mempool_alloc+0x53/0x130
stoat kernel: [ 245.014986] [<ffffffff802b31e3>] ? mempool_alloc+0x53/0x130
stoat kernel: [ 245.015075] [<ffffffff803112fc>] bvec_alloc_bs+0x6c/0x140
stoat kernel: [ 245.015163] [<ffffffff80311429>] bio_alloc_bioset+0x59/0x100
stoat kernel: [ 245.015252] [<ffffffff80311580>] bio_alloc+0x10/0x20
stoat kernel: [ 245.015339] [<ffffffff8030c8d0>] submit_bh+0x70/0x130
stoat kernel: [ 245.015426] [<ffffffff8030f2b2>] __block_write_full_page+0x1d2/0x3b0
stoat kernel: [ 245.015517] [<ffffffff80351da0>] ? ext3_get_block+0x0/0x130
stoat kernel: [ 245.015606] [<ffffffff80351da0>] ? ext3_get_block+0x0/0x130
stoat kernel: [ 245.015695] [<ffffffff8030f57f>] block_write_full_page+0xef/0x110
stoat kernel: [ 245.015785] [<ffffffff80350123>] ext3_ordered_writepage+0xd3/0x1b0
stoat kernel: [ 245.015875] [<ffffffff802bfd9e>] ? __dec_zone_page_state+0x1e/0x20
stoat kernel: [ 245.016850] [<ffffffff802b7aa2>] __writepage+0x12/0x50
stoat kernel: [ 245.016938] [<ffffffff802b8c28>] write_cache_pages+0x228/0x480
stoat kernel: [ 245.017027] [<ffffffff802b7a90>] ? __writepage+0x0/0x50
stoat kernel: [ 245.017116] [<ffffffffa019f0ee>] ? nfs_commit_inode+0x8e/0xc0 [nfs]
stoat kernel: [ 245.017218] [<ffffffff802b8e9f>] generic_writepages+0x1f/0x30
stoat kernel: [ 245.017307] [<ffffffff802b8eed>] do_writepages+0x3d/0x50
stoat kernel: [ 245.017395] [<ffffffff80306f15>] __sync_single_inode+0x75/0x330
stoat kernel: [ 245.017485] [<ffffffff80307223>] __writeback_single_inode+0x53/0x1b0
stoat kernel: [ 245.017575] [<ffffffff80240d2b>] ? dequeue_entity+0x1b/0x250
stoat kernel: [ 245.017665] [<ffffffff80307904>] generic_sync_sb_inodes+0x314/0x4e0
stoat kernel: [ 245.017755] [<ffffffff80307caa>] writeback_inodes+0x5a/0x100
stoat kernel: [ 245.017844] [<ffffffff802b7e14>] wb_kupdate+0xa4/0x120
stoat kernel: [ 245.017932] [<ffffffff802b96c6>] __pdflush+0x136/0x220
stoat kernel: [ 245.018020] [<ffffffff802b9806>] pdflush+0x56/0x60
stoat kernel: [ 245.018106] [<ffffffff802b7d70>] ? wb_kupdate+0x0/0x120
stoat kernel: [ 245.018194] [<ffffffff802b97b0>] ? pdflush+0x0/0x60
stoat kernel: [ 245.018281] [<ffffffff802684b9>] kthread+0x49/0x90
stoat kernel: [ 245.018369] [<ffffffff80213979>] child_rip+0xa/0x11
stoat kernel: [ 245.018456] [<ffffffff80268470>] ? kthread+0x0/0x90
stoat kernel: [ 245.018543] [<ffffffff8021396f>] ? child_rip+0x0/0x11
stoat kernel: [ 245.018631] Code: fa 0f 1f 80 00 00 00 00 65 8b 04 25 24 00 00 00 48 98 49 8b 94 c5 e8 00 00 00 8b 7a 18 89 7d d4 48 8b 1a 48 85 db 74 47 8b 42 14 <48> 8b 04 c3 48 89 02 4c 89 e7 57 9d 66 0f 1f 44 00 00 66 45 85 mempool_alloc+0x53/0x130
stoat kernel: [ 244.969836] [<ffffffff803112fc>] bvec_alloc_bs+0x6c/0x140
stoat kernel: [ 244.969925] [<ffffffff80311429>] bio_alloc_bioset+0x59/0x100
stoat kernel: [ 244.970014] [<ffffffff80311580>] bio_alloc+0x10/0x20
stoat kernel: [ 244.970102] [<ffffffff80313ea3>] dio_new_bio+0x63/0x130
stoat kernel: [ 244.970189] [<ffffffff8031428f>] dio_send_cur_page+0xaf/0xc0
stoat kernel: [ 244.970278] [<ffffffff80314b5c>] direct_io_worker+0x21c/0x390
stoat kernel: [ 244.970367] [<ffffffff80314f0f>] __blockdev_direct_IO+0x23f/0x380
stoat kernel: [ 244.970457] [<ffffffff80311f10>] ? blkdev_get_blocks+0x0/0xc0
stoat kernel: [ 244.970546] [<ffffffff803137d9>] blkdev_direct_IO+0x49/0x50
stoat kernel: [ 244.970634] [<ffffffff80311f10>] ? blkdev_get_blocks+0x0/0xc0
stoat kernel: [ 244.970723] [<ffffffff802b26e0>] generic_file_aio_read+0x1c0/0x1e0
stoat kernel: [ 244.970812] [<ffffffff8022f669>] ? default_spin_lock_flags+0x9/0x10
stoat kernel: [ 244.970903] [<ffffffff802e7911>] do_sync_read+0xf1/0x140
stoat kernel: [ 244.970992] [<ffffffff8069db3e>] ? do_page_fault+0x32e/0x780
stoat kernel: [ 244.971081] [<ffffffff80268920>] ? autoremove_wake_function+0x0/0x40
stoat kernel: [ 244.971173] [<ffffffff80312d48>] ? __blkdev_put+0x88/0x1d0
stoat kernel: [ 244.971261] [<ffffffff80317df8>] ? inotify_inode_queue_event+0xd8/0x100
stoat kernel: [ 244.971352] [<ffffffff803f6083>] ? apparmor_file_permission+0x23/0x30
stoat kernel: [ 244.971443] [<ffffffff803d0751>] ? security_file_permission+0x11/0x20
stoat kernel: [ 244.971536] [<ffffffff802e80f8>] vfs_read+0xc8/0x130
stoat kernel: [ 244.971623] [<ffffffff802e81fa>] sys_pread64+0x9a/0xa0
stoat kernel: [ 244.971711] [<ffffffff8021253a>] system_call_fastpath+0x16/0x1b
stoat kernel: [ 244.971800] Code: fa 0f 1f 80 00 00 00 00 65 8b 04 25 24 00 00 00 48 98 49 8b 94 c5 e8 00 00 00 8b 7a 18 89 7d d4 48 8b 1a 48 85 db 74 47 8b 42 14 <48> 8b 04 c3 48 89 02 4c 89 e7 57 9d 66 0f 1f 44 00 00 66 45 85
stoat kernel: [ 245.011365] general protection fault: 0000 [#3] SMP
stoat kernel: [ 245.011483] last sysfs file: /sys/fs/o2cb/interface_revision
stoat kernel: [ 245.014059] Stack:
stoat kernel: [ 245.014260] ffff88007d257980 ffff88007f20c690 ffff88007d257960 ffffffff802b3081
stoat kernel: [ 245.014485] ffff88007d2579e0 ffffffff802b31e3 ffff88007d257998 00000010802b3081
stoat kernel: [ 245.014757] Call Trace:
stoat kernel: [ 245.014809] [<ffffffff802b3081>] mempool_alloc_slab+0x11/0x20
stoat kernel: [ 245.014898] [<ffffffff802b31e3>] mempool_alloc+0x53/0x130
stoat kernel: [ 245.014986] [<ffffffff802b31e3>] ? mempool_alloc+0x53/0x130
stoat kernel: [ 245.015075] [<ffffffff803112fc>] bvec_alloc_bs+0x6c/0x140
stoat kernel: [ 245.015163] [<ffffffff80311429>] bio_alloc_bioset+0x59/0x100
stoat kernel: [ 245.015252] [<ffffffff80311580>] bio_alloc+0x10/0x20
stoat kernel: [ 245.015339] [<ffffffff8030c8d0>] submit_bh+0x70/0x130
stoat kernel: [ 245.015426] [<ffffffff8030f2b2>] __block_write_full_page+0x1d2/0x3b0
stoat kernel: [ 245.015517] [<ffffffff80351da0>] ? ext3_get_block+0x0/0x130
stoat kernel: [ 245.015606] [<ffffffff80351da0>] ? ext3_get_block+0x0/0x130
stoat kernel: [ 245.015695] [<ffffffff8030f57f>] block_write_full_page+0xef/0x110
stoat kernel: [ 245.015785] [<ffffffff80350123>] ext3_ordered_writepage+0xd3/0x1b0
stoat kernel: [ 245.015875] [<ffffffff802bfd9e>] ? __dec_zone_page_state+0x1e/0x20
stoat kernel: [ 245.016850] [<ffffffff802b7aa2>] __writepage+0x12/0x50
stoat kernel: [ 245.016938] [<ffffffff802b8c28>] write_cache_pages+0x228/0x480
stoat kernel: [ 245.017027] [<ffffffff802b7a90>] ? __writepage+0x0/0x50
stoat kernel: [ 245.017116] [<ffffffffa019f0ee>] ? nfs_commit_inode+0x8e/0xc0 [nfs]
stoat kernel: [ 245.017218] [<ffffffff802b8e9f>] generic_writepages+0x1f/0x30
stoat kernel: [ 245.017307] [<ffffffff802b8eed>] do_writepages+0x3d/0x50
stoat kernel: [ 245.017395] [<ffffffff80306f15>] __sync_single_inode+0x75/0x330
stoat kernel: [ 245.017485] [<ffffffff80307223>] __writeback_single_inode+0x53/0x1b0
stoat kernel: [ 245.017575] [<ffffffff80240d2b>] ? dequeue_entity+0x1b/0x250
stoat kernel: [ 245.017665] [<ffffffff80307904>] generic_sync_sb_inodes+0x314/0x4e0
stoat kernel: [ 245.017755] [<ffffffff80307caa>] writeback_inodes+0x5a/0x100
stoat kernel: [ 245.017844] [<ffffffff802b7e14>] wb_kupdate+0xa4/0x120
stoat kernel: [ 245.017932] [<ffffffff802b96c6>] __pdflush+0x136/0x220
stoat kernel: [ 245.018020] [<ffffffff802b9806>] pdflush+0x56/0x60
stoat kernel: [ 245.018106] [<ffffffff802b7d70>] ? wb_kupdate+0x0/0x120
stoat kernel: [ 245.018194] [<ffffffff802b97b0>] ? pdflush+0x0/0x60
stoat kernel: [ 245.018281] [<ffffffff802684b9>] kthread+0x49/0x90
stoat kernel: [ 245.018369] [<ffffffff80213979>] child_rip+0xa/0x11
stoat kernel: [ 245.018456] [<ffffffff80268470>] ? kthread+0x0/0x90
stoat kernel: [ 245.018543] [<ffffffff8021396f>] ? child_rip+0x0/0x11
stoat kernel: [ 245.018631] Code: fa 0f 1f 80 00 00 00 00 65 8b 04 25 24 00 00 00 48 98 49 8b 94 c5 e8 00 00 00 8b 7a 18 89 7d d4 48 8b 1a 48 85 db 74 47 8b 42 14 <48> 8b 04 c3 48 89 02 4c 89 e7 57 9d 66 0f 1f 44 00 00 66 45 85
More information about the Ocfs2-users
mailing list