<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>
<!--
body
{line-height:1.5}
blockquote
{margin-top:0px;
margin-bottom:0px;
margin-left:0.5em}
body
{font-size:16pt;
font-family:微软雅黑;
color:rgb(0,0,0);
line-height:1.5}
-->
</style>
</head>
<body>
<div><span></span>
<div>
<div><span style="color:rgb(51,51,51); font-family:'Microsoft YaHei',Arial,Helvetica,sans-serif; font-size:24px; line-height:30px; widows:1">A similar problem is described</span><font color="#333333" face="\Microsoft YaHei\, Arial, Helvetica, sans-serif" size="5"><span style="line-height:30px; widows:1"> </span><span style="line-height:normal">below</span><span style="line-height:30px; widows:1">.</span></font></div>
<div><span style="color:rgb(51,51,51); font-family:'Microsoft YaHei, Arial',Helvetica,sans-serif; font-size:x-large; line-height:normal; background-color:window">There is a race window to triger BUG i</span><font color="#333333" face="\Microsoft YaHei, Arial\, Helvetica, sans-serif" size="5" style="background-color:window"><span style="line-height:normal">n </span><span style="line-height:24px">dlm_drop_lockres_ref</span><span style="line-height:normal">.</span></font></div>
<div style="widows:1"><font color="#333333" style="background-color:window"><span style="font-family:'Microsoft YaHei','Arial, Helvetica',sans-serif; font-size:x-large; line-height:normal">all nodes will hang in the futhure.</span></font><span style="line-height:normal; background-color:window; color:rgb(51,51,51); font-family:'Microsoft YaHei','Arial, Helvetica',sans-serif; font-size:x-large"> </span></div>
<div><br>
</div>
<div>Node 1 Node 3</div>
</div>
<div><span style="font-size:16pt; line-height:1.5"> </span><span style="line-height:1.5"> </span><span style="line-height:24px">m</span><span style="line-height:24px">ount.oc</span><span style="line-height:24px">fs2</span> vol1 and create node lock,
reboot</div>
<div><span style="font-size:16pt; line-height:1.5"> </span><span style=""> waiting for Node 3 Node 3 mount.ocfs2 vol1</span></div>
<div><span style=""> </span><span style="font-size:16pt; line-height:1.5"> </span><span style="font-size:16pt; line-height:1.5"> </span><span style="line-height:1.5"> </span><span style="line-height:1.5"> </span><span style="line-height:24px">fail to
mount vol1, do not </span><span style="line-height:24px">get lock on journal </span><span style="font-size:16pt; line-height:1.5; background-color:window"> </span><span style="font-size:16pt; background-color:window; line-height:24px">fail
to mount vol1,</span><span style="background-color:rgb(255,255,255)"><span style="line-height:24px"> </span><span style="line-height:24px">Local alloc hasn't been recovered!</span><span style="line-height:24px"> </span></span></div>
<div><span style=""> </span><span style="font-size:16pt; line-height:1.5"> </span><span style="font-size:16pt; line-height:1.5"> </span><span style="font-size:16pt; line-height:1.5">
</span><span style="line-height:1.5"> </span><span style="line-height:24px">dlm_drop_lockres_ref and lockres don't exsit, </span><span style="background-color:window; font-size:16px; line-height:24px"> </span></div>
<div><span style=""> </span><span style="font-size:16pt; line-height:1.5"> </span><span style="font-size:16pt; line-height:1.5"> </span><span style="font-size:16pt; line-height:1.5"> return Error -22 and triger BUG.</span></div>
<div><span style="font-size:16pt; line-height:1.5">I think </span><span style="font-size:16pt; line-height:24px">the BUG should be removed for the case. </span></div>
<div><span style="line-height:normal; font-size:16pt; background-color:window">But i can't say for sure what will
</span><span style="line-height:normal; font-size:16pt; background-color:window">come and remove the BU</span><span style="background-color:rgb(255,255,255)"><span style="line-height:normal">G? Thanks for </span><span style="line-height:18px; widows:1">your </span><span style="line-height:normal">reply
.</span></span></div>
<div><span style="line-height:normal; font-size:16pt; background-color:window"><br>
</span></div>
<div><span style="line-height:24px"><font size="3">dlm_drop_lockres_ref</font></span></div>
<div><font size="3">--- dlmmaster.c 2015-10-12 02:09:45.000000000 +0800 <br>
+++ /root/dlmmaster.c 2015-12-28 11:39:14.560429513 +0800 <br>
@@ -2275,7 +2275,6 @@ <br>
mlog(ML_ERROR, "%s: res %.*s, DEREF to node %u got %d\n", <br>
dlm->name, namelen, lockname, res->owner, r); <br>
dlm_print_one_lock_resource(res); <br>
- BUG(); <br>
} <br>
return ret; <br>
} </font><br>
</div>
<div><span style="font-size:16pt; line-height:1.5"><br>
</span></div>
<div><span style="font-size:16pt; line-height:1.5"><br>
</span></div>
<div>Node 3</div>
<div><font size="3">Dec 26 23:29:40 cvknode55 kernel: [ 7708.864231] (mount.ocfs2,6023,1):dlm_send_remote_lock_request:332 ERROR: E496D3D3799A46E6AC4251B4F7FBFFDF: res M0000000000000000000268e0ecb551, Error -92 send CREATE LOCK to node 3
<br>
Dec 26 23:29:40 cvknode55 kernel: [ 7708.968289] (mount.ocfs2,6023,1):dlm_send_remote_lock_request:332 ERROR: E496D3D3799A46E6AC4251B4F7FBFFDF: res M0000000000000000000268e0ecb551, Error -92 send CREATE LOCK to node</font></div>
<div style="font-size:16px"><span style="">Dec 26 23:29:40 cvknode55 kernel: [ 7709.066019] o2dlm: Node 3 joins domain E496D3D3799A46E6AC4251B4F7FBFFDF ( 1 3 ) 2 nodes <br>
</span></div>
<div style="font-size:16px"><span style="">Dec 26 23:29:40 cvknode55 kernel: [ 7709.072297] (mount.ocfs2,6023,1):__ocfs2_cluster_lock:1486 ERROR: DLM error -22 while calling ocfs2_dlm_lock on resource M0000000000000000000268e0ecb551 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.072302] (mount.ocfs2,6023,1):ocfs2_inode_lock_full_nested:2333 ERROR: status = -22 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.072305] (mount.ocfs2,6023,1):ocfs2_journal_init:860 ERROR: Could not get lock on journal! <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.072308] (mount.ocfs2,6023,1):ocfs2_check_volume:2433 ERROR: Could not initialize journal! <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.072311] (mount.ocfs2,6023,1):ocfs2_check_volume:2510 ERROR: status = -22 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.072314] (mount.ocfs2,6023,1):ocfs2_mount_volume:1889 ERROR: status = -22 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212472] (dlm_thread,6313,2):dlm_drop_lockres_ref:2316 ERROR: E496D3D3799A46E6AC4251B4F7FBFFDF: res M0000000000000000000268e0ecb551, DEREF to node 3 got -22 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212479] lockres: M0000000000000000000268e0ecb551, owner=3, state=64 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212480] last used: 4296818545, refcnt: 3, on purge list: yes <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212481] on dirty list: no, on reco list: no, migrating pending: no <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212482] inflight locks: 0, asts reserved: 0 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212483] refmap nodes: [ ], inflight=0 <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212484] res lvb: <br>
Dec 26 23:29:40 cvknode55 kernel: [ 7709.212485] granted queue: <br>
</span></div>
<div><span style=""><span style="font-size:16px">Dec 26 23:29:40 cvknode55 kernel: [ 7709.212486] converting queue: </span><br style="font-size:16px">
<span style="font-size:16px">Dec 26 23:29:40 cvknode55 kernel: [ 7709.212487] blocked queue: </span><br style="font-size:16px">
<span style="font-size:16px">Dec 26 23:29:40 cvknode55 kernel: [ 7709.212509] ------------[ cut here ]------------ </span><br style="font-size:16px">
<span style="font-size:16px">Dec 26 23:29:40 cvknode55 kernel: [ 7709.212511] Kernel BUG at ffffffffa02f4471 [verbose debug info unavailable] </span></span></div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>Node 1</div>
<div><font size="3">Dec 26 23:31:07 cvknode21 kernel: [ 153.221008] Sleep 5 seconds for live map build up. <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.225039] o2dlm: Joining domain E496D3D3799A46E6AC4251B4F7FBFFDF ( 1 3 ) 2 nodes </font><br>
</div>
<div><font size="3">Dec 26 23:31:12 cvknode21 kernel: [ 158.231096] (kworker/u65:3,502,8):dlm_create_lock_handler:513 ERROR: dlm status = DLM_IVLOCKID <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.303089] JBD2: Ignoring recovery information on journal <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369080] (mount.ocfs2,6151,2):ocfs2_load_local_alloc:354 ERROR: Local alloc hasn't been recovered! <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369080] found = 70, set = 70, taken = 256, off = 161793 <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369080] umount left unclean filesystem. run ocfs2.fsck -f <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369090] (mount.ocfs2,6151,2):ocfs2_load_local_alloc:371 ERROR: status = -22 <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369093] (mount.ocfs2,6151,2):ocfs2_check_volume:2481 ERROR: status = -22 <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369096] (mount.ocfs2,6151,2):ocfs2_check_volume:2510 ERROR: status = -22 <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.369099] (mount.ocfs2,6151,2):ocfs2_mount_volume:1889 ERROR: status = -22 <br>
Dec 26 23:31:12 cvknode21 kernel: [ 158.371208] (kworker/u65:3,502,8):dlm_deref_lockres_handler:2361 ERROR: E496D3D3799A46E6AC4251B4F7FBFFDF:M0000000000000000000268e0ecb551: bad lockres name </font></div>
</div>
<hr color="#b5c4df" size="1" align="left" style="width:210px; height:1px">
<div><span>
<div style="margin:10px; font-family:verdana; font-size:10pt">
<div>zhangguanghui</div>
</div>
</span></div>
<blockquote style="margin-top:0px; margin-bottom:0px; margin-left:0.5em">
<div> </div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0cm 0cm 0cm">
<div style="padding-right:8px; padding-left:8px; font-size:12px; font-family:tahoma; color:#000000; background:#efefef; padding-bottom:8px; padding-top:8px">
<div><b>From:</b> <a href="mailto:ocfs2-users-bounces@oss.oracle.com">ocfs2-users-bounces@oss.oracle.com</a></div>
<div><b>Date:</b> 2015-12-22 21:47</div>
<div><b>To:</b> <a href="mailto:joseph.qi@huawei.com">Joseph Qi</a></div>
<div><b>CC:</b> <a href="mailto:sivakumar@zohocorp.com">Siva Sokkumuthu</a>; <a href="mailto:ocfs2-users@oss.oracle.com">
ocfs2-users@oss.oracle.com</a></div>
<div><b>Subject:</b> Re: [Ocfs2-users] Ocfs2 clients hang</div>
</div>
</div>
<div>
<div class="FoxDiv20151228105922990948">
<div style="font-size:10pt">
<div id="">
<div>Hi Joseph,<br>
</div>
<div><br>
</div>
<div> We are facing ocfs2 server hang problem frequently and suddenly 4 nodes going to hang stat expect 1 node. After reboot everything is come to normal, this behavior happend many times. Do we have any debug and fix for this issue.<br>
</div>
<div><br>
</div>
<div>Regards<br>
</div>
<div>Prabu</div>
<div><br>
</div>
</div>
<div class="zmail_extra">
<div id="1">
<div><br>
</div>
<div>---- On Tue, 22 Dec 2015 16:30:52 +0530 <b>Joseph Qi <joseph.qi@huawei.com></b>wrote ----
<br>
</div>
</div>
<div><br>
</div>
<blockquote style="border-left-width:1px; border-left-style:solid; border-left-color:rgb(204,204,204); padding-left:6px; margin:0px 0px 0px 5px">
<div>
<div>Hi Prabu, <br>
</div>
<div>From the log you provided, I can only see that node 5 disconnected with <br>
</div>
<div>node 2, 3, 1 and 4. It seemed that something wrong happened on the four <br>
</div>
<div>nodes, and node 5 did recovery for them. After that, the four nodes <br>
</div>
<div>joined again. <br>
</div>
<div><br>
</div>
<div>On 2015/12/22 16:23, gjprabu wrote: <br>
</div>
<div>> Hi, <br>
</div>
<div>> <br>
</div>
<div>> Anybody please help me on this issue. <br>
</div>
<div>> <br>
</div>
<div>> Regards <br>
</div>
<div>> Prabu <br>
</div>
<div>> <br>
</div>
<div>> ---- On Mon, 21 Dec 2015 15:16:49 +0530 *gjprabu <<a href="mailto:gjprabu@zohocorp.com" target="_blank">gjprabu@zohocorp.com</a>>*wrote ----
<br>
</div>
<div>> <br>
</div>
<div>> Dear Team, <br>
</div>
<div>> <br>
</div>
<div>> Ocfs2 clients are getting hang often and unusable. Please find the logs. Kindly provide the solution, it will be highly appreciated.
<br>
</div>
<div>> <br>
</div>
<div>> <br>
</div>
<div>> [3659684.042530] o2dlm: Node 4 joins domain A895BC216BE641A8A7E20AA89D57E051 ( 1 2 3 4 5 ) 5 nodes
<br>
</div>
<div>> <br>
</div>
<div>> [3992993.101490] (kworker/u192:1,63211,24):dlm_create_lock_handler:515 ERROR: dlm status = DLM_IVLOCKID
<br>
</div>
<div>> [3993002.193285] (kworker/u192:1,63211,24):dlm_deref_lockres_handler:2267 ERROR: A895BC216BE641A8A7E20AA89D57E051:M0000000000000062d2dcd000000000: bad lockres name
<br>
</div>
<div>> [3993032.457220] (kworker/u192:0,67418,11):dlm_do_assert_master:1680 ERROR: Error -112 when sending message 502 (key 0xc3460ae7) to node 2
<br>
</div>
<div>> [3993062.547989] (kworker/u192:0,67418,11):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 2
<br>
</div>
<div>> [3993064.860776] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 2
<br>
</div>
<div>> [3993064.860804] o2cb: o2dlm has evicted node 2 from domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993073.280062] o2dlm: Begin recovery on domain A895BC216BE641A8A7E20AA89D57E051 for node 2
<br>
</div>
<div>> [3993094.623695] (dlm_thread,46268,8):dlm_send_proxy_ast_msg:484 ERROR: A895BC216BE641A8A7E20AA89D57E051: res S000000000000000000000200000000, error -112 send AST to node 4
<br>
</div>
<div>> [3993094.624281] (dlm_thread,46268,8):dlm_flush_asts:605 ERROR: status = -112
<br>
</div>
<div>> [3993094.687668] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -112 when sending message 502 (key 0xc3460ae7) to node 3
<br>
</div>
<div>> [3993094.815662] (dlm_reco_thread,46269,7):dlm_do_master_requery:1666 ERROR: Error -112 when sending message 514 (key 0xc3460ae7) to node 1
<br>
</div>
<div>> [3993094.816118] (dlm_reco_thread,46269,7):dlm_pre_master_reco_lockres:2166 ERROR: status = -112
<br>
</div>
<div>> [3993124.778525] (dlm_reco_thread,46269,7):dlm_do_master_requery:1666 ERROR: Error -107 when sending message 514 (key 0xc3460ae7) to node 3
<br>
</div>
<div>> [3993124.779032] (dlm_reco_thread,46269,7):dlm_pre_master_reco_lockres:2166 ERROR: status = -107
<br>
</div>
<div>> [3993133.332516] o2cb: o2dlm has evicted node 3 from domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993139.915122] o2cb: o2dlm has evicted node 1 from domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993147.071956] o2cb: o2dlm has evicted node 4 from domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993147.071968] (dlm_reco_thread,46269,7):dlm_do_master_requery:1666 ERROR: Error -107 when sending message 514 (key 0xc3460ae7) to node 4
<br>
</div>
<div>> [3993147.071975] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 4
<br>
</div>
<div>> [3993147.071997] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 4
<br>
</div>
<div>> [3993147.072001] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 4
<br>
</div>
<div>> [3993147.072005] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 4
<br>
</div>
<div>> [3993147.072009] (kworker/u192:0,67418,15):dlm_do_assert_master:1680 ERROR: Error -107 when sending message 502 (key 0xc3460ae7) to node 4
<br>
</div>
<div>> [3993147.075019] (dlm_reco_thread,46269,7):dlm_pre_master_reco_lockres:2166 ERROR: status = -107
<br>
</div>
<div>> [3993147.075353] (dlm_reco_thread,46269,7):dlm_do_master_request:1347 ERROR: link to 1 went down!
<br>
</div>
<div>> [3993147.075701] (dlm_reco_thread,46269,7):dlm_get_lock_resource:932 ERROR: status = -107
<br>
</div>
<div>> [3993147.076001] (dlm_reco_thread,46269,7):dlm_do_master_request:1347 ERROR: link to 3 went down!
<br>
</div>
<div>> [3993147.076329] (dlm_reco_thread,46269,7):dlm_get_lock_resource:932 ERROR: status = -107
<br>
</div>
<div>> [3993147.076634] (dlm_reco_thread,46269,7):dlm_do_master_request:1347 ERROR: link to 4 went down!
<br>
</div>
<div>> [3993147.076968] (dlm_reco_thread,46269,7):dlm_get_lock_resource:932 ERROR: status = -107
<br>
</div>
<div>> [3993147.077275] (dlm_reco_thread,46269,7):dlm_restart_lock_mastery:1236 ERROR: node down! 1
<br>
</div>
<div>> [3993147.077591] (dlm_reco_thread,46269,7):dlm_restart_lock_mastery:1229 node 3 up while restarting
<br>
</div>
<div>> [3993147.077594] (dlm_reco_thread,46269,7):dlm_wait_for_lock_mastery:1053 ERROR: status = -11
<br>
</div>
<div>> [3993155.171570] (dlm_reco_thread,46269,7):dlm_do_master_request:1347 ERROR: link to 3 went down!
<br>
</div>
<div>> [3993155.171874] (dlm_reco_thread,46269,7):dlm_get_lock_resource:932 ERROR: status = -107
<br>
</div>
<div>> [3993155.172150] (dlm_reco_thread,46269,7):dlm_do_master_request:1347 ERROR: link to 4 went down!
<br>
</div>
<div>> [3993155.172446] (dlm_reco_thread,46269,7):dlm_get_lock_resource:932 ERROR: status = -107
<br>
</div>
<div>> [3993155.172719] (dlm_reco_thread,46269,7):dlm_restart_lock_mastery:1236 ERROR: node down! 3
<br>
</div>
<div>> [3993155.173001] (dlm_reco_thread,46269,7):dlm_restart_lock_mastery:1229 node 4 up while restarting
<br>
</div>
<div>> [3993155.173003] (dlm_reco_thread,46269,7):dlm_wait_for_lock_mastery:1053 ERROR: status = -11
<br>
</div>
<div>> [3993155.173283] (dlm_reco_thread,46269,7):dlm_do_master_request:1347 ERROR: link to 4 went down!
<br>
</div>
<div>> [3993155.173581] (dlm_reco_thread,46269,7):dlm_get_lock_resource:932 ERROR: status = -107
<br>
</div>
<div>> [3993155.173858] (dlm_reco_thread,46269,7):dlm_restart_lock_mastery:1236 ERROR: node down! 4
<br>
</div>
<div>> [3993155.174135] (dlm_reco_thread,46269,7):dlm_wait_for_lock_mastery:1053 ERROR: status = -11
<br>
</div>
<div>> [3993155.174458] o2dlm: Node 5 (me) is the Recovery Master for the dead node 2 in domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993158.361220] o2dlm: End recovery on domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993158.361228] o2dlm: Begin recovery on domain A895BC216BE641A8A7E20AA89D57E051 for node 1
<br>
</div>
<div>> [3993158.361305] o2dlm: Node 5 (me) is the Recovery Master for the dead node 1 in domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993161.833543] o2dlm: End recovery on domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993161.833551] o2dlm: Begin recovery on domain A895BC216BE641A8A7E20AA89D57E051 for node 3
<br>
</div>
<div>> [3993161.833620] o2dlm: Node 5 (me) is the Recovery Master for the dead node 3 in domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993165.188817] o2dlm: End recovery on domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993165.188826] o2dlm: Begin recovery on domain A895BC216BE641A8A7E20AA89D57E051 for node 4
<br>
</div>
<div>> [3993165.188907] o2dlm: Node 5 (me) is the Recovery Master for the dead node 4 in domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> [3993168.551610] o2dlm: End recovery on domain A895BC216BE641A8A7E20AA89D57E051
<br>
</div>
<div>> <br>
</div>
<div>> [3996486.869628] o2dlm: Node 4 joins domain A895BC216BE641A8A7E20AA89D57E051 ( 4 5 ) 2 nodes
<br>
</div>
<div>> [3996778.703664] o2dlm: Node 4 leaves domain A895BC216BE641A8A7E20AA89D57E051 ( 5 ) 1 nodes
<br>
</div>
<div>> [3997012.295536] o2dlm: Node 2 joins domain A895BC216BE641A8A7E20AA89D57E051 ( 2 5 ) 2 nodes
<br>
</div>
<div>> [3997099.498157] o2dlm: Node 4 joins domain A895BC216BE641A8A7E20AA89D57E051 ( 2 4 5 ) 3 nodes
<br>
</div>
<div>> [3997783.633140] o2dlm: Node 1 joins domain A895BC216BE641A8A7E20AA89D57E051 ( 1 2 4 5 ) 4 nodes
<br>
</div>
<div>> [3997864.039868] o2dlm: Node 3 joins domain A895BC216BE641A8A7E20AA89D57E051 ( 1 2 3 4 5 ) 5 nodes
<br>
</div>
<div>> <br>
</div>
<div>> Regards <br>
</div>
<div>> Prabu <br>
</div>
<div>> ** <br>
</div>
<div>> <br>
</div>
<div>> <br>
</div>
<div>> <br>
</div>
<div>> <br>
</div>
<div>> <br>
</div>
<div>> _______________________________________________ <br>
</div>
<div>> Ocfs2-users mailing list <br>
</div>
<div>> <a href="mailto:Ocfs2-users@oss.oracle.com" target="_blank">Ocfs2-users@oss.oracle.com</a>
<br>
</div>
<div>> <a href="https://oss.oracle.com/mailman/listinfo/ocfs2-users" target="_blank">
https://oss.oracle.com/mailman/listinfo/ocfs2-users</a> <br>
</div>
<div>> <br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<span style="font-size:7.5pt; font-family:华文细黑; color:gray"><span lang="EN-US">-------------------------------------------------------------------------------------------------------------------------------------<br>
</span>本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出<span lang="EN-US"><br>
</span>的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、<span lang="EN-US"><br>
</span>或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本<span lang="EN-US"><br>
</span>邮件！<span lang="EN-US"><br>
</span></span><span lang="EN-US" style="font-size:7.5pt; font-family:"Arial","sans-serif"; color:gray">This e-mail and its attachments contain confidential information from H3C, which is
<br>
intended only for the person or entity whose address is listed above. Any use of the
<br>
information contained herein in any way (including, but not limited to, total or partial
<br>
disclosure, reproduction, or dissemination) by persons other than the intended <br>
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
<br>
by phone or email immediately and delete it!</span>
</body>
</html>