<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Lucida Console";
panose-1:2 11 6 9 4 5 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>We had a system crash while unmounting some volumes. It is a
6 node<o:p></o:p></p>
<p class=MsoNormal>cluster, hostnames are n1 -> n6. Info below, Any ideas ?<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Thanks in advance,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>charlie<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>ocfs2-tools-1.4.0-0.5<o:p></o:p></p>
<p class=MsoNormal>ocfs2console-1.4.0-0.5<o:p></o:p></p>
<p class=MsoNormal>Linux sr2600-1 2.6.16.60-0.34-smp #1 SMP Fri Jan 16 14:59:01
UTC 2009 x86_64 x86_64 x86_64 GNU/Linux<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>OCFS2 Node Manager 1.4.1-1-SLES Wed Jul 23 18:33:42 UTC 2008
(build f922955d99ef972235bd0c1fc236c5ddbb368611) <o:p></o:p></p>
<p class=MsoNormal>o2cb heartbeat: registered disk
mode <o:p></o:p></p>
<p class=MsoNormal>OCFS2 DLM 1.4.1-1-SLES Wed Jul 23 18:33:42 UTC 2008 (build
f922955d99ef972235bd0c1fc236c5ddbb368611) <o:p></o:p></p>
<p class=MsoNormal>OCFS2 DLMFS 1.4.1-1-SLES Wed Jul 23 18:33:42 UTC 2008 (build
f922955d99ef972235bd0c1fc236c5ddbb368611) <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>from /var/log/messages<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2_dlm: Node 3 leaves domain
27169AFBA10C425B91D386F55FC37AF5<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2_dlm: Nodes in domain
("27169AFBA10C425B91D386F55FC37AF5"): 1 2 4 <o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2_dlm: Node 4 leaves domain
27169AFBA10C425B91D386F55FC37AF5<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2_dlm: Nodes in domain
("27169AFBA10C425B91D386F55FC37AF5"): 1 2 <o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2: Unmounting device (253,8)
on (node 1)<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2_dlm: Node 5 leaves domain
65EAD2E63E7549B69335BCAD58C5A476<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:05 n2 kernel: ocfs2_dlm: Nodes in domain
("65EAD2E63E7549B69335BCAD58C5A476"): 0 1 2 3 4 <o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:06 n2 kernel: o2net: no longer connected to
node n6 (num 5) at 192.168.100.60:7777<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:08 n2 kernel: ocfs2_dlm: Node 0 leaves domain
65EAD2E63E7549B69335BCAD58C5A476<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:08 n2 kernel: ocfs2_dlm: Nodes in domain
("65EAD2E63E7549B69335BCAD58C5A476"): 1 2 3 4 <o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:09 n2 kernel: o2net: no longer connected to
node n1 (num 0) at 192.168.100.10:7777<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:09 n2 kernel: ocfs2: Unmounting device (253,9)
on (node 1)<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:13 n2 kernel: ocfs2: Unmounting device (253,6)
on (node 1)<o:p></o:p></p>
<p class=MsoNormal>Feb 2 15:04:18 n2 kernel: ocfs2: Unmounting device (253,7)
on (node 1)<o:p></o:p></p>
<p class=MsoNormal> Feb 2 15:07:22 n2 syslog-ng[5679]: syslog-ng version 1.6.8
starting <span style='font-family:Wingdings'>ß</span>------------
system restart<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>stack trace from core dump file:<o:p></o:p></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>PID: 14669
TASK: ffff8101289cb7b0 CPU: 1 COMMAND: "umount"<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #0
[ffff8101040ef810] machine_kexec at ffffffff8011c036<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #1
[ffff8101040ef8e0] crash_kexec at ffffffff80153ea9<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #2
[ffff8101040ef9a0] __die at ffffffff802ec510<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #3
[ffff8101040ef9e0] die at ffffffff8010c786<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #4
[ffff8101040efa10] do_invalid_op at ffffffff8010cd2d<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #5
[ffff8101040efad0] error_exit at ffffffff8010bc91<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
[exception RIP: dlm_add_lock_to_array+249]<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RIP: ffffffff8852750f RSP: ffff8101040efb88 RFLAGS: 00010286<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RAX: ffff810103897ac8 RBX: ffff810103e02ec0 RCX: 0000000000016a72<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RDX: 0000000000000000 RSI: 0000000000000292 RDI: ffffffff8035a99c<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RBP: ffff810104056000 R8: ffffffff8045a260 R9:
0000000000000001<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
R10: ffff810103897a80 R11: 0000000000000000 R12: ffff810104056080<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
R13: 0000000000000000 R14: 0000000000000001 R15: ffff810104056000<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #6
[ffff8101040efb80] dlm_add_lock_to_array at ffffffff8852750f<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #7
[ffff8101040efba0] dlm_send_one_lockres at ffffffff88527a55<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #8
[ffff8101040efc80] dlm_empty_lockres at ffffffff88520ae2<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'> #9
[ffff8101040efdb0] ocfs2_dlm_shutdown at ffffffff8856fc8a<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#10
[ffff8101040efdd0] ocfs2_dismount_volume at ffffffff8859416c<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#11
[ffff8101040efe40] ocfs2_put_super at ffffffff88594668<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#12
[ffff8101040efe50] generic_shutdown_super at ffffffff8018c305<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#13
[ffff8101040efe70] kill_block_super at ffffffff8018c3d6<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#14
[ffff8101040efe90] deactivate_super at ffffffff8018c4ac<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#15
[ffff8101040efeb0] sys_umount at ffffffff801a1376<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>#16 [ffff8101040eff80]
system_call at ffffffff8010adba<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RIP: 00002b4ba5ba0987 RSP: 00007fff050e3990 RFLAGS: 00010246<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RAX: 00000000000000a6 RBX: ffffffff8010adba RCX: 00007fff050e4140<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RDX: 000000000000006e RSI: 0000000000000001 RDI: 00000000005101e0<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
RBP: 00000000005101c0 R8: 000000000000006f R9:
fefeff344d544b2e<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000510250<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
R13: 00000000005101e0 R14: 0000000000510200 R15: 00007fff050e42e0<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-family:"Lucida Console"'>
ORIG_RAX: 00000000000000a6 CS: 0033 SS: 002b<o:p></o:p></span></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>