[Ocfs2-users] how do I remove ocfs2 orphaned files that are taking up diskspace ?
Tim Hughes
tim.hughes at tradefair.com
Tue Feb 24 11:08:00 PST 2009
I am looking for a little help with some orphaned files that are taking up diskspace.
I deleted a approximately 30 x 1GB mysql-bin-XXXXX.log files from a three node ocfs2 cluster. The files appeared removed from the filesystem but the results of a `df -h /var/lib/mysql` showed that no disk space has been cleared. A `du -sh /var/lib/mysql` on the other hand says that ~ 30GB was removed.
<snip>
[root at host1 mysql]# du -sh /var/lib/mysql
50G /var/lib/mysql
[root at host1 mysql]# df -h /var/lib/mysql/
Filesystem Size Used Avail Use% Mounted on
/dev/xvdb1 100G 83G 18G 83% /var/lib/mysql
[root at host1 mysql]#
<snip>
First thoughts were that mysql was holding the file descriptors open but `lsof` showed nothing. After trying a few other things we discovered that the files were orphans of ocfs2.
<snip>
[root@ host1 ~]# debugfs.ocfs2 -R "ls -l //orphan_dir:0001" /dev/xvdb1
13 drwxr-xr-x 2 0 0 4096 23-Feb-2009 21:54 .
6 drwxr-xr-x 18 0 0 4096 20-Aug-2008 15:54 ..
1064634 -rw-rw---- 0 100 101 1024 5-Sep-2008 14:14 0000000000103eba
1064635 -rw-rw---- 0 100 101 0 5-Sep-2008 14:14 0000000000103ebb
1064633 -rw-rw---- 0 100 101 8554 5-Sep-2008 14:14 0000000000103eb9
1064637 -rw-rw---- 0 100 101 114688 1-Oct-2008 12:33 0000000000103ebd
1064636 -rw-rw---- 0 100 101 8844 1-Oct-2008 11:37 0000000000103ebc
1193689 -rw-rw---- 0 100 101 1073742258 21-Jan-2009 15:18 00000000001236d9
1193690 -rw-rw---- 0 100 101 1073741942 21-Jan-2009 20:10 00000000001236da
1193691 -rw-rw---- 0 100 101 1073743434 22-Jan-2009 14:29 00000000001236db
1193692 -rw-rw---- 0 100 101 1073742303 22-Jan-2009 18:37 00000000001236dc
1193693 -rw-rw---- 0 100 101 1073741875 23-Jan-2009 12:59 00000000001236dd
1193694 -rw-rw---- 0 100 101 1073741973 23-Jan-2009 18:36 00000000001236de
1193695 -rw-rw---- 0 100 101 1073742198 26-Jan-2009 14:05 00000000001236df
1193696 -rw-rw---- 0 100 101 1073742221 26-Jan-2009 20:18 00000000001236e0
1193697 -rw-rw---- 0 100 101 1073742068 27-Jan-2009 14:46 00000000001236e1
1193698 -rw-rw---- 0 100 101 1005225645 28-Jan-2009 00:06 00000000001236e2
<snip>
We have shutdown mysql and tried unmounting and mounting the ocfs2 file system from the node where we deleted the files. This made no difference so we decided to take down the mysql cluster and unmounting the ocfs2 filesystem from all nodes just incase something was for some reason holding these files open that we couldn't find. This didn't clear the space and the orphans were still there.
Next we unmounted it all again and ran `fsck.ocfs2 /dev/xvdb1` on it with the following results which indicated there was nothing wrong.
<snip>
Checking OCFS2 filesystem in /dev/xvdb1:
label: db-store01
uuid: 74 c1 14 6c af ee 4e 29 84 e4 c1 7a a8 cc 96 eb
number of blocks: 26214047
bytes per block: 4096
number of clusters: 26214047
bytes per cluster: 4096
max slots: 16
/dev/xvdb1 is clean. It will be checked after 20 additional mounts.
<snip>
We are running :
ocfs2-tools-1.4.1-1.el5
ocfs2-2.6.18-92.1.10.el5xen-1.4.1-1.el5
with kernel 2.6.18-92.1.10.el5xen on RHEL5
How do we reclaim this space ?
Tim Hughes
www.tradefair.com
Tradefair | Level 2, Yellow Building | 1 Nicholas Road | London | W11 4AN
The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company.
More information about the Ocfs2-users
mailing list