<div>Sunil,</div> <div> </div> <div> First I want to make clear that I do think that Oracle Cluster File System provides a great value for Oracle Linux customers and I do know that one has to pay top dollar for equivalent functionality on other platforms, for example Veritas Storage Foundation, and others offered by IBM and HP.</div> <div> </div> <div> But, the Linux platform is the only one where there are two independent clusterwares running (O2CB and CRS). On all the other platforms, as far as I know, when there is a second clusterware on the machine, CRS acts as a client to it. Use of a uncertified clusterware stack independently and concurrently with CRS is not even allowed on other platforms.</div> <div> </div> <div> This is kind of funny because both o2cb and crs are Oracle products. </div> <div><BR>Regards,</div> <div>Luis Freitas</div>
<div><BR><B><I>Sunil Mushran <Sunil.Mushran@oracle.com></I></B> wrote:</div> <BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">Fencing is not a fs operation but a cluster operation. The fs is only a <BR>client<BR>of the cluster stack.<BR><BR>Alexei_Roudnev wrote:<BR>> It all depends of the usage scenario.<BR>><BR>> Tipical usage is, for example:<BR>><BR>> (1) Shared application home. Writes happens once / week during maintanance,<BR>> otehr time files are opened for reading only. Few logfiles<BR>> can be redirected if required.<BR>><BR>> So, when server see a problems, it HAD NOT any pending IO for a 3 days - so<BR>> what the purpose of reboot? It 100% knows that NO ANY IO<BR>> is pending, and other nodes have not any IO pending as well.<BR>><BR>> (2) Backup storage for the RAC. FS is not opened 90% of the time. At night,<BR>> one node opens it and creates a few files. Other
node have not any pending<BR>> IO on this FS. Fencing passive node (which dont run any backup) is not<BR>> useful because it HAD NOT ANY PENDING IO for a few hours.<BR>><BR>> (3) WEB server. 10 nodes, 1 only makes updates. The same - most nodes have<BR>> not any pending IO.<BR>><BR>> Of course there is always a risk of FS corruption in the clusters. Any layer<BR>> can keep pending IO forever (I saw Linux kernel keeping it for 10 minutes).<BR>> Problem is that in such cases software fencing can't help as well because<BR>> node is half-dead and can't detect it's own status.<BR>><BR>> So, the key point here is not in _fence for each ap-chi_ but _keep system<BR>> without pending writes as long as possible and make clean transition between<BR>> active write/active read / passive states. Then you can avoid self-fencing<BR>> in 90% cases (because of server wil be in passive or active reads state). I<BR>> mounT FS but don't cd into
it, or just CD but dont read - passive status. I<BR>> read file - active read for 1 minute, tbhnen flush buffers so that it is in<BR>> passive mode again. I began top write - switch system to write mode. I did<BR>> not write blocks for 1 minute - flush everything, wait 1 more minute and<BR>> switch to passive mode.<BR>><BR>><BR>><BR>><BR>> ----- Original Message ----- <BR>> From: "Sunil Mushran" <SUNIL.MUSHRAN@ORACLE.COM><BR>> To: "David Miller" <SYSLOG@D.SPARKS.NET><BR>> Cc: <OCFS2-USERS@OSS.ORACLE.COM><BR>> Sent: Monday, April 09, 2007 3:18 PM<BR>> Subject: Re: [Ocfs2-users] Catatonic nodes under SLES10<BR>><BR>><BR>> <BR>>> For io fencing to be graceful, one requires better hardware. Read<BR>>> <BR>> expensive.<BR>> <BR>>> As in, switches where one can choke off all the ios to the storage from<BR>>> a specific<BR>>> node.<BR>>><BR>>> Read the following for a discussion on
force umounts. In short, not<BR>>> possible as yet.<BR>>> http://lwn.net/Articles/192632/<BR>>><BR>>> Readonly does not work wrt to io fencing. As in, ro only stops any new<BR>>> userspace<BR>>> writes but cannot stop pending writes. And writes could be lodged in any<BR>>> io layer.<BR>>> A reboot is the cheapest way to avoid corruption. (While a reboot is<BR>>> painful, it is<BR>>> much less painful than a corrupted fs.)<BR>>><BR>>> With 1.2.5 you should be able to increase the network timeouts and<BR>>> hopefully avoid<BR>>> the problem.<BR>>><BR>>> David Miller wrote:<BR>>> <BR>>>> Alexei_Roudnev wrote:<BR>>>> <BR>>>>> Did you checked<BR>>>>><BR>>>>> /proc/sys/kernel/panic /proc/sys/kernel/panic_on_oops<BR>>>>><BR>>>>> system variables?<BR>>>>><BR>>>>> <BR>>>>
No. Maybe I'm missing something here.<BR>>>><BR>>>> Are you saying that a panic/freeze/reboot is the expected/desirable<BR>>>> behavior? That nothing more graceful could be done, like to just<BR>>>> dismount the ocfs2 file systems, or force them to a read-only mount or<BR>>>> something like that? We have to reload the kernel?<BR>>>><BR>>>> Thanks,<BR>>>><BR>>>> --- David<BR>>>><BR>>>> <BR>>>>> ----- Original Message ----- From: "David Miller" <SYSLOG@D.SPARKS.NET><BR>>>>> To: <OCFS2-USERS@OSS.ORACLE.COM><BR>>>>> Sent: Monday, April 02, 2007 9:01 AM<BR>>>>> Subject: [Ocfs2-users] Catatonic nodes under SLES10<BR>>>>><BR>>>>> <BR>>>> [snip]<BR>>>><BR>>>> <BR>>>>> Both servers will be connected to a dual-host external RAID system.<BR>>>>> I've setup ocfs2 on
a couple of test systems and everything appears<BR>>>>> to work fine.<BR>>>>><BR>>>>> Until, that is, one of the systems loses network connectivity.<BR>>>>><BR>>>>> When the systems can't talk to each other anymore, but the disk<BR>>>>> heartbeat is still alive, the high numbered node goes catatonic.<BR>>>>> Under SLES 9 it fenced itself off with a kernel panic; under 10 it<BR>>>>> simply stops responding to network or console. A power cycling is<BR>>>>> required to bring it back up.<BR>>>>><BR>>>>> The desired behavior would be for the higher numbered node to lose<BR>>>>> access to the ocfs2 file system(s). I don't really care whether it<BR>>>>> would simply timeout ala stale NFS mounts, or immediately error like<BR>>>>> access to non-existent
files.<BR>>>>><BR>>>>><BR>>>>> <BR>>>> _______________________________________________<BR>>>> Ocfs2-users mailing list<BR>>>> Ocfs2-users@oss.oracle.com<BR>>>> http://oss.oracle.com/mailman/listinfo/ocfs2-users<BR>>>> <BR>>> _______________________________________________<BR>>> Ocfs2-users mailing list<BR>>> Ocfs2-users@oss.oracle.com<BR>>> http://oss.oracle.com/mailman/listinfo/ocfs2-users<BR>>><BR>>> <BR>><BR>> <BR><BR>_______________________________________________<BR>Ocfs2-users mailing list<BR>Ocfs2-users@oss.oracle.com<BR>http://oss.oracle.com/mailman/listinfo/ocfs2-users<BR></BLOCKQUOTE><BR><p> 
<hr size=1><a href="http://us.rd.yahoo.com/evt=49938/*http://tools.search.yahoo.com/toolbar/features/mail/">Never miss an email again!<br>Yahoo! Toolbar</a> alerts you the instant new Mail arrives.<a href="
http://us.rd.yahoo.com/evt=49937/*http://tools.search.yahoo.com/toolbar/features/mail/"> Check it out.</a>