<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>Hi,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>When system booting getting error message “modprobe:
FATAL: Module ocfs2_stackglue not found” in message. Some nodes reboot
without any error message.<o:p></o:p></p>
<p class=MsoNormal>-------------------------------------------------<o:p></o:p></p>
<p class=MsoNormal>ul 27 10:02:19 alf3 kernel: ip_tables: (C) 2000-2006
Netfilter Core Team<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:19 alf3 kernel: Netfilter messages via NETLINK
v0.30.<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:19 alf3 kernel: ip_conntrack version 2.4 (8192
buckets, 65536 max) - 304 bytes per conntrack<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:19 alf3 kernel: e1000: eth0:
e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:20 alf3 setroubleshoot: [server.ERROR] cannot
start systen DBus service: Failed to connect to socket /var/run/db<o:p></o:p></p>
<p class=MsoNormal>us/system_bus_socket: No such file or directory<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:20 alf3 kernel: VMware memory control driver
initialized<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:20 alf3 kernel: e1000: eth0: e1000_set_tso: TSO
is Enabled<o:p></o:p></p>
<p class=MsoNormal><span style='background:yellow;mso-highlight:yellow'>Jul 27
10:02:21 alf3 modprobe: FATAL: Module ocfs2_stackglue not found.</span> <o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:21 alf3 kernel: OCFS2 Node Manager 1.4.2 Wed
Jul 1 19:55:44 PDT 2009 (build 0b9eb999c4d39c0d4b66219a2752cda6)<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:21 alf3 kernel: OCFS2 DLM 1.4.2 Wed Jul 1
19:55:44 PDT 2009 (build 0faae8d4263a8c594749be558d8d7edd)<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:21 alf3 kernel: OCFS2 DLMFS 1.4.2 Wed Jul
1 19:55:44 PDT 2009 (build 0faae8d4263a8c594749be558d8d7edd)<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:21 alf3 kernel: OCFS2 User DLM kernel interface
loaded<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:25 alf3 kernel: o2net: connected to node alf0
(num 0) at 172.25.29.10:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:25 alf3 kernel: o2net: connected to node alf2
(num 2) at 172.25.29.12:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:25 alf3 kernel: o2net: accepted connection from
node alf5 (num 5) at 172.25.29.15:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:26 alf3 kernel: o2net: accepted connection from
node alf4 (num 4) at 172.25.29.14:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:27 alf3 kernel: o2net: connected to node alf1
(num 1) at 172.25.29.11:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 27 10:02:31 alf3 kernel: OCFS2 1.4.2 Wed Jul 1
19:55:41 PDT 2009 (build 966fd2793489955b2271e7bb7e691088)<o:p></o:p></p>
<div style='mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.0pt;
padding:0in 0in 1.0pt 0in'>
<p class=MsoNormal style='border:none;padding:0in'>Jul 27 10:02:31 alf3 kernel:
ocfs2_dlm: Nodes in domain ("7BE7E9E2026A40F8801B56257D805C88"): 0 1
2 3 4 5<o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div style='mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.0pt;
padding:0in 0in 1.0pt 0in'>
<p class=MsoNormal style='border:none;padding:0in'>Kernel log from another node
alf1 for above node alf3 is like<o:p></o:p></p>
<p class=MsoNormal style='border:none;padding:0in'><o:p> </o:p></p>
</div>
<p class=MsoNormal>Jul 29 10:15:57 alf1 kernel: o2net: connection to node alf3
(num 3) at 172.25.29.13:7777 has been idle for 30.0 seconds, shut<o:p></o:p></p>
<p class=MsoNormal>ting it down.<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:15:57 alf1 kernel: (0,1):o2net_idle_timer:1506
here are some times that might help debug the situation: (tmr 124887<o:p></o:p></p>
<p class=MsoNormal>6927.861591 now 1248876957.858464 dr 1248876927.861556 adv
1248876927.861622:1248876927.861623 func (0ffa2aed:506) 1248876927<o:p></o:p></p>
<p class=MsoNormal>.861592:1248876927.861604)<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:15:57 alf1 kernel: o2net: no longer connected to
node alf3 (num 3) at 172.25.29.13:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:16:27 alf1 kernel:
(2600,1):o2net_connect_expired:1667 ERROR: no connection established with node
3 after 30.0 seco<o:p></o:p></p>
<p class=MsoNormal>nds, giving up and returning errors.<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:27 alf1 last message repeated 2 times<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:30 alf1 kernel:
(2618,0):ocfs2_dlm_eviction_cb:98 device (8,33): dlm has evicted node 3<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:32 alf1 kernel:
(2629,2):dlm_get_lock_resource:844 7BE7E9E2026A40F8801B56257D805C88:$RECOVERY:
at least one node<o:p></o:p></p>
<p class=MsoNormal> (3) to recover before lock mastery can begin<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:32 alf1 kernel:
(2629,2):dlm_get_lock_resource:878 7BE7E9E2026A40F8801B56257D805C88: recovery
map is not empty, <o:p></o:p></p>
<p class=MsoNormal>but must master $RECOVERY lock now<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:32 alf1 kernel: (2629,1):dlm_do_recovery:524
(2629) Node 1 is the Recovery Master for the Dead Node 3 for Domain<o:p></o:p></p>
<p class=MsoNormal> 7BE7E9E2026A40F8801B56257D805C88<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:34 alf1 kernel: o2net: accepted connection from
node alf3 (num 3) at 172.25.29.13:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:38 alf1 kernel: ocfs2_dlm: Node 3 joins domain
7BE7E9E2026A40F8801B56257D805C88<o:p></o:p></p>
<p class=MsoNormal>Jul 29 10:17:38 alf1 kernel: ocfs2_dlm: Nodes in domain
("7BE7E9E2026A40F8801B56257D805C88"): 1 2 3 4 5 <o:p></o:p></p>
<p class=MsoNormal>Jul 29 11:09:42 alf1 kernel: o2net: connected to node alf0
(num 0) at 172.25.29.10:7777<o:p></o:p></p>
<p class=MsoNormal>Jul 29 11:09:45 alf1 kernel: ocfs2_dlm: Node 0 joins domain
7BE7E9E2026A40F8801B56257D805C88<o:p></o:p></p>
<div style='mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.0pt;
padding:0in 0in 1.0pt 0in'>
<p class=MsoNormal style='border:none;padding:0in'>Jul 29 11:09:45 alf1 kernel:
ocfs2_dlm: Nodes in domain ("7BE7E9E2026A40F8801B56257D805C88"): 0 1
2 3 4 5<o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>OS = Red Hat 5.2 <o:p></o:p></p>
<p class=MsoNormal>[root@alf3 /]# uname -a<o:p></o:p></p>
<p class=MsoNormal>Linux alf3 2.6.18-128.1.16.el5 #1 SMP Fri Jun 26 10:53:31
EDT 2009 x86_64 x86_64 x86_64 GNU/Linux<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>[root@alf3 /]# rpm -qa | grep ocfs2<o:p></o:p></p>
<p class=MsoNormal>ocfs2-tools-1.4.2-1.el5<o:p></o:p></p>
<p class=MsoNormal>ocfs2-2.6.18-128.1.16.el5-1.4.2-1.el5<o:p></o:p></p>
<p class=MsoNormal>ocfs2console-1.4.2-1.el5<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Any help will be appreciated, OCFS2 cluster is not stable. Mounting
File system for file sharing with Alfresco.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Thanks<o:p></o:p></p>
<p class=MsoNormal>Raheel<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>