<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Goncalo,<br>
<br>
run "lsof |grep deleted" on all nodes. If that lists any, it means some
process still has the file open. That file will not get cleaned till
the process exits or closes the file<br>
<br>
If that command doesn't list any, there is a way(in ocfs2-1.4.2) to
clean, but it needs an unmount/mount of all nodes but you can do it on
one node at a time. Do it once and see if the orphans are cleaned, if
not do it second time. Second time it should clean.<br>
<br>
thanks,<br>
--Srini<br>
<br>
Gonçalo Borges wrote:
<blockquote cite="mid4A6DD2D0.6000801@lip.pt" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Hi Karim...<br>
<br>
<br>
Running the commands (in ALL clients) to identify the application/node
associated with the orphan_dir does not provide me any output.<br>
<br>
root@fw01 ~]# for i in 07 08 09 10 11 12 21 22 23 24 25 26; do echo
"### core$i ###"; ssh core$i "find /proc -name fd -exec ls -l {} \; |
grep deleted; lsof | grep -i deleted"; done<br>
### core07 ###<br>
### core08 ###<br>
### core09 ###<br>
### core10 ###<br>
### core11 ###<br>
### core12 ###<br>
### core21 ###<br>
### core22 ###<br>
### core23 ###<br>
### core24 ###<br>
### core25 ###<br>
### core26 ###<br>
<br>
I've also tried "mount -o remount /site06", and several syncs, in all
clients, but without success.<br>
<br>
The orphan file continues there... :(<br>
<br>
Cheers<br>
Goncalo<br>
<br>
<br>
On 07/27/2009 04:33 PM, Karim Alkhayer wrote:
<blockquote
cite="mid:!&!AAAAAAAAAAAYAAAAAAAAAK4MJckmPrBLvG1VP+tkWavCgAAAEAAAALkKsCDkJDBNnvSSanZ5uzYBAAAAAA==@gmail.com"
type="cite">
<meta http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
h3
{mso-style-priority:9;
mso-style-link:"Heading 3 Char";
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:13.5pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
code
{mso-style-priority:99;
font-family:"Courier New";}
p.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing
{mso-style-priority:1;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.Heading3Char
{mso-style-name:"Heading 3 Char";
mso-style-priority:9;
mso-style-link:"Heading 3";
font-weight:bold;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="Section1">
<p class="MsoNormal"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Hi
Goncalo,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Here’re
some guidelines to rectify your issue:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><u><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Identify
cluster node and application associated with orphan_dir<o:p></o:p></span></u></b></p>
<p class="MsoNormal"><span
style="font-size: 10pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom: 12pt;"><a
moz-do-not-send="true" name="top"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Run the
following
command(s) on each cluster node to identify which node, application or
user
(holders) are associated with orphan_dir entries.<o:p></o:p></span></a></p>
<p class="MsoNormal"><code><span style="font-size: 10pt;"># find
/proc -name fd
-exec ls -l {} \; | grep deleted</span></code><span
style="font-size: 10pt; font-family: "Courier New";"><br>
<code> or</code><br>
<code># lsof | grep -i deleted</code></span><span
style="font-family: "Arial","sans-serif";"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom: 12pt;"><span
style="font-family: "Arial","sans-serif";"><br>
</span><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Next,
review the output of the above command(s) noting any that relate to the
OCFS2
filesystem in question.<br>
At this point, you should be able to determine the holding process id
(pid)</span><span
style="font-size: 10pt; font-family: "Arial","sans-serif";"><o:p></o:p></span></p>
<p class="MsoNormal"><a moz-do-not-send="true" name="aref4"></a><b><u><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Releasing
disk space associated with OCFS2
orphan directories</span></u></b><span
style="font-size: 10pt; font-family: "Arial","sans-serif";"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom: 12pt;"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">The above
step allows you to identify the pid
associated with orphaned files.<br>
If the holding process(es) can still be gracefully interacted with via
their
user interface, and you are certain that the process is safe to stop
without
adverse effect upon your environment, then shutdown the process(es) in
question. Once the process(es) close their open file descriptors,
orphaned
files will be deleted and the associated disk space made available.<br>
<br>
If the process(es) in question cannot be interacted with via their user
interface, or if you are certain the processes are no longer required,
then
kill the associated process(es) i.e. `kill <pid>`. If any
process(es) are
no longer communicable (i.e. zombie) or cannot be successfully killed,
a forced
unmount of the OCFS2 volume in question and/or reboot of the associated
cluster
node may be necessary in order to recover the disk space associated
with
orphaned files.<o:p></o:p></span></p>
<p class="MsoNoSpacing"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Let
us know how it goes!<o:p></o:p></span></p>
<p class="MsoNoSpacing"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";"><o:p> </o:p></span></p>
<p class="MsoNoSpacing"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Best
regards,<o:p></o:p></span></p>
<p class="MsoNoSpacing"><span
style="font-size: 10pt; font-family: "Arial","sans-serif";">Karim
Alkhayer<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<div>
<div
style="border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span
style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: windowtext;">From:</span></b><span
style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: windowtext;">
<a class="moz-txt-link-abbreviated"
href="mailto:ocfs2-users-bounces@oss.oracle.com">ocfs2-users-bounces@oss.oracle.com</a>
[<a class="moz-txt-link-freetext"
href="mailto:ocfs2-users-bounces@oss.oracle.com">mailto:ocfs2-users-bounces@oss.oracle.com</a>]
<b>On Behalf Of </b>Gonçalo
Borges<br>
<b>Sent:</b> Monday, July 27, 2009 4:35 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated"
href="mailto:ocfs2-users@oss.oracle.com">ocfs2-users@oss.oracle.com</a><br>
<b>Subject:</b> [Ocfs2-users] How to clean orphan metadata?<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi All...<br>
<br>
1) I have recently deleted a big 100GB file from an OCFS2 partition.
The
problem is that a "df" command still shows that partition with 142 GB
of used spaced when it should report ~42Gb of used space (look to <b>/site06)</b>:<br>
<br>
[root@core23 ~]# df -h<br>
Filesystem
Size Used Avail Use% Mounted on<br>
/dev/sda1
87G 2.4G 80G 3% /<br>
tmpfs
512M 0 512M 0% /dev/shm<br>
none
512M 104K 512M 1% /var/lib/xenstored<br>
/dev/mapper/iscsi04-lun1p1<br>
851G 63G 788G 8% /site04<br>
/dev/mapper/iscsi05-lun1p1<br>
851G 65G 787G 8% /site05<br>
/dev/mapper/iscsi06-lun2p1<br>
884G 100G 785G 12% /apoio06<br>
/dev/mapper/iscsi06-lun1p1<br>
<b>851G 142G 709G 17% /site06<br>
<br>
<br>
</b>2) Running "debugfs.ocfs2 /dev/mapper/iscsi06-lun1p1", I found
the following relevant file:<br>
<br>
debugfs: ls -l //orphan_dir:0001<br>
13
drwxr-xr-x 2 0
0 3896 27-Jul-2009
09:55 .<br>
6
drwxr-xr-x 18 0
0 4096 9-Jul-2009
12:24 ..<br>
524781
-rw-r--r-- 0 0
0 104857600000 24-Jul-2009 16:35 00000000000801ed<br>
<br>
<br>
3) I need to clean this metadata information, but I can not run
"fsck.ocfs2 -f" because this is a production filesystem being
accessed by 12 clients. To run "fsck.ocfs2 -f" I would have to
unmount the partition from all the clients, and this is not a solution
at the
time. The software I'm currently using is:<br>
<br>
[root@core09 log]# cat /etc/redhat-release <br>
Scientific Linux SL release 5.3 (Boron)<br>
<br>
[root@core09 log]# uname -a<br>
Linux core09.ncg.ingrid.pt 2.6.18-128.1.16.el5xen #1 SMP Tue Jun 30
07:06:24
EDT 2009 x86_64 x86_64 x86_64 GNU/Linux<br>
<br>
[root@core09 log]# rpm -qa | grep ocfs2<br>
ocfs2-2.6.18-128.1.16.el5xen-1.4.2-1.el5<br>
ocfs2-tools-1.4.2-1.el5<br>
ocfs2console-1.4.2-1.el5<br>
<br>
<br>
Is there a workaround for this?<br>
Cheers<br>
Goncalo<o:p></o:p></p>
</div>
</blockquote>
<br>
<pre wrap="">
<hr size="4" width="90%">
_______________________________________________
Ocfs2-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Ocfs2-users@oss.oracle.com">Ocfs2-users@oss.oracle.com</a>
<a class="moz-txt-link-freetext" href="http://oss.oracle.com/mailman/listinfo/ocfs2-users">http://oss.oracle.com/mailman/listinfo/ocfs2-users</a></pre>
</blockquote>
</body>
</html>