[Ocfs2-users] Fwd: Returned mail: see transcript for details

[Anjan Chakraborty]

Luis,
  Let me understanfd your suggestion. Are you suggesting:
  $ORA_CRS_HOME -- without datavolume,nointr
  $ORACLE_HOME   -- without datavolume,nointr
  But place OCR, Voting Disks & future datafiles with datavolume,nointr
   
  Did I understand correctly?
  I have also gone back to OCFS2 1.2.5-6.
  Okay, if this is the suggestion -- I will surely try.
  Thanks.
  Anjan

Luis Freitas <lfreitas34 at yahoo.com> wrote:
  
  Anjan,

    The two installs I did with x86_64 were with ASM, with kernel versions that today are very old, some years ago. Here I am using x86 (even as the hardware is x86_64 capable) due to a managment decision not to go to x86_64. 

     Anyone else on the list?

     About the mount points, the CRS home is a ORACLE_HOME, so you shouldnt mount it with datavolume. If you want to keep the CRS ORACLE_HOME and the RDBMS ORACLE_HOME each on one filesystem you should have at least a third partition for the cluster registry, voting disks and datafiles, mounted with the datavolume option, and mount those two without it. 

    If you need to get this working fast, I think you should try a older OCFS2 module version. If not keep working with Mark and the other Oracle people here on the list as they can diagnose these errors and get it fixed.

    What is the firewire disk you are using?

Regards,
Luis


Anjan Chakraborty <anjan.chakraborty at yahoo.com> wrote:  

Luis Freitas <lfreitas34 at yahoo.com> wrote:  Luis,
As far as I know, it's not recommended to mount ORACLE_HOME with datavolume,nointr but that's not the case with CRS. So, I have different mount options for ORA_CRS_HOME (home/oracle/ocrvotcrs) & ORACLE_HOME (home/oracle/orasys). If I am wrong, please correct me. Also, I don't have any datafile yet because I don't have the basic thing -- Oracle binary under ORACLE_HOME. Once I have that, I will have datafiles & mount will contain datavolume,nointr.
Do you know anybody has successfully implemented OCFS2 1.2.7-1 on Red Hat4.5 (kernel 2.6.9-55.EL) under 86_64 architecture? 
What exactly you have?
Thanks for your continuous effort to help me.
I hope somebody else will also start joining this discussion & help me.
I am really stuck -- can't advance at all -- need help desperately.
Anjan


Anjan,

   Well it is only a sugestion.

   About those mount options, perhaps the Oracle people can give you a feedback on that also. But I noticed that you have two mount points, and seems to be installing CRS binaries on one, and RDBMS binaries on the other. So, where are datafiles? 

    You shouldnt be installing the CRS home on /home/oracle/ocrvotcrs as it has the datavolume option. It is not clear if you are trying this.

    The filesystems where you have binaries should not have the datavolume option, and the filesystems with datafiles, voting disks and the cluster registry need to have the datavolume option.

Regards,
Luis

Anjan Chakraborty <anjan.chakraborty at yahoo.com> wrote:    Mark,
  Thanks to both of you for trying to help me.
  I have alreaddy communicated to Luis that I have to have OCFS2 because my software needs Clustering technology that is absent in EXT3. Moreover, both CRS & RDBMS homes should also be on shared/clustered system -- so, OCFS2 is the only choice.
  If you can help me to understand/resolve this issue, I will really appreciate that.
  Please note that my FireWire shared drive works perfectly when I use RAW but as soon as I am trying to use OCFS2, all the problems started.
  Thanks.
  Anjan 

Mark Fasheh <mark.fasheh at oracle.com> wrote:
  Luis,
Thanks for helping here. I have one comment regarding moving the
shared home to ext3. If Anjan's setup is having issues like this, moving his
oracle home to a local disk would only hide problems which take longer to
reproduce for the crs and data files partitions. What needs to happen is
that the root cause of his problems is discovered and fixed.

Keep in mind, I'm not saying someone couldn't use ext3 for their oracle
install - it's an excellent choice for running a non-shared oracle home.
It's just not a good shared disk diagnostic tool ;)

Once again, thanks for stepping up and lending a hand.
--Mark

On Tue, Nov 27, 2007 at 08:50:28AM -0800, Luis Freitas wrote:
> Anjan,
> 
> You dont need to share the database binaries, only the CRS and the
> datafiles. You can do it to save disk space, but it is not mandatory. The CRS
> and datafiles are much less stressfull to the filesystem structures as there is
> a reduced number of large files, although they usually have a heavy i/o load,
> and stress the disk subsystem and the locking algorithms.
> 
> So you can have two separated ext3 filesystems located at the same place on
> each server, and one or more ocfs2 shared filesystems for the CRS and the
> database datafiles. The Oracle installer takes care of copying the binaries
> between the servers during the installation.
> 
> It might be usefull to try a lower version like 1.2.6, as you are using the
> latest version available. I am using 1.2.4-2 here with RH 4.0 and kernel
> 2.6.9-42 and it seems rather stable, only needed to increase the timeouts. (But
> I dont have the oracle_home shared.)
> 
> Also you might have a hardware problem somewhere on the SAN. And I still
> have to check those mount options you sent...
> 
> One detail. I dont know if the Centos distro includes the OCFS2 module. Are
> you using the modules downloaded from the oss.oracle.com site for the
> equivalent RH 4.0 kernel, or modules built by Centos? If using CENTOS modules
> you might get better results by changing to the Oracle built modules for the
> equivalent RH 4.0 kernel.
> 
> Regards,
> Luis
> 
> Anjan Chakraborty wrote:
> 
> Luis,
> I am intending to use CRS/RAC that needs a Cluster File System. How does
> EXT3 falls into that area?
> Thanks.
> Anjan
> 
> Thanks a lot for the response. Here is what I am doing:
> 
> 1.
> mkfs.ocfs2 -b 4K -C 32K -N 4 -L ocrvotcrs /dev/sdb3 -- for CRS
> mkfs.ocfs2 -b 4K -C 32K -N 4 -L orasys /dev/sdb4 -- for RDBMS
> 
> 2. Then mounting using /etc/fstab:
> /dev/sdb3 /home/oracle/ocrvotcrs ocfs2 _netdev,datavolume,nointr
> 0 0
> /dev/sdb4 /home/oracle/orasys ocfs2 _netdev 0 0
> If you find anything wrong here, can you please tell what to do?
> It's a non-production system & so I can experiment with whatever you
> suggest and won't held you responsible for that.
> Thanks.
> Anjan
> 
> 
> Luis Freitas wrote:
> 
> Anjan,
> 
> Are you installing the binaries on OCSF2 too? How are you mounting
> the filesystem?
> 
> You might want to try using ext3 for the binaries and OCF2 only for
> datafiles and archives, until you get this fixed.
> 
> Regards,
> Luis
> 
> Anjan Chakraborty wrote:
> 
> Hi,
> I sent an email to Mark Fisheh of Oracle Corp. & posted this issue
> at OTN under Linux thread this morning. I hope that someone among
> you might have experienced this and can help. On that basis, I am
> sending this to you too. I am stuck & will really appreciate if you
> can shed some light on this.
> Thanks.
> Anjan
> ***********************************************************************************************************
> I have a 2 node CentOS 4.5 86_64 system (kernel 2.6.9-55.EL). On
> this I installed Oracle OCFS2 1.2.7-1 (with exact kernel matching).
> After this I installed Oracle CRS 10.2.0.1 and that installation
> went fine. Then I tried to install Oracle RDBMS 10.2.0.1 and all
> the problems started from there. The /var/log/messages file got
> filled up with messages (giving some to avoid confusion):
> ocfs2_read_locked_inode: .. : ERROR: Invalid dinode #0 signature =
> ocfs2_lookup: .. : ERROR: Unable to create inode ....
> 
> Then OUI gave several error messages, e.g.
> .... Invalid stored block length on file ...../em/em.war followed
> by I/O error in file
> Errors in invoking to files ins_rdbms.mk and ins_ldap.mk
> 
> Then /var/log/messages gave:
> OCFS2: ERROR (device ....): ocfs2_extend_file: Dinode # ...... has
> bad signature O' # I ....
> And the installation failed & CRS died. And the machines reboot.
> I ran fsck.ocfs2 -n /dev/...., it came clean.
> I have tested this several timnes & always same thing happening.
> If I use RAW partitions, everything works fine. So, the problem may
> be in the OCFS2 & OS/Oracle -- but, not sure how to bypass this.
> I have to have OCFS2 -- can't use RAW for various reasons.
> Can somebody please help me to resolve this?
> Thanks.
> ***********************************************************************************************************
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Be a better pen pal. Text or chat with friends inside Yahoo! Mail.
> See how._______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
> 
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Be a better pen pal. Text or chat with friends inside Yahoo! Mail. See
> how.
> 
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Never miss a thing. Make Yahoo your homepage.
> 
> 
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Get easy, one-click access to your favorites. Make Yahoo! your homepage.

> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
--
Mark Fasheh
Senior Software Developer, Oracle
mark.fasheh at oracle.com

  
  
---------------------------------
  Get easy, one-click access to your favorites. Make Yahoo! your homepage. 
  
  
---------------------------------
  Be a better pen pal. Text or chat with friends inside Yahoo! Mail. See how.
  
  
---------------------------------
  Be a better sports nut! Let your teams follow you with Yahoo Mobile. Try it now.
  
  
---------------------------------
  Never miss a thing. Make Yahoo your homepage. 
    
---------------------------------
  Be a better pen pal. Text or chat with friends inside Yahoo! Mail. See how.Anjan,

    The two installs I did with x86_64 were with ASM, with kernel versions that today are very old, some years ago. Here I am using x86 (even as the hardware is x86_64 capable) due to a managment decision not to go to x86_64. 

     Anyone else on the list?

     About the mount points, the CRS home is a ORACLE_HOME, so you shouldnt mount it with datavolume. If you want to keep the CRS ORACLE_HOME and the RDBMS ORACLE_HOME each on one filesystem you should have at least a third partition for the cluster registry, voting disks and datafiles, mounted with the datavolume option, and mount those two without it. 

    If you need to get this working fast, I think you should try a older OCFS2 module version. If not keep working with Mark and the other Oracle people here on the list as they can diagnose these errors and get it fixed.

    What is the firewire disk you are using?

Regards,
Luis


Anjan Chakraborty <anjan.chakraborty at yahoo.com> wrote:  

Luis Freitas <lfreitas34 at yahoo.com> wrote:  Luis,
As far as I know, it's not recommended to mount ORACLE_HOME with datavolume,nointr but that's not the case with CRS. So, I have different mount options for ORA_CRS_HOME (home/oracle/ocrvotcrs) & ORACLE_HOME (home/oracle/orasys). If I am wrong, please correct me. Also, I don't have any datafile yet because I don't have the basic thing -- Oracle binary under ORACLE_HOME. Once I have that, I will have datafiles & mount will contain datavolume,nointr.
Do you know anybody has successfully implemented OCFS2 1.2.7-1 on Red Hat4.5 (kernel 2.6.9-55.EL) under 86_64 architecture? 
What exactly you have?
Thanks for your continuous effort to help me.
I hope somebody else will also start joining this discussion & help me.
I am really stuck -- can't advance at all -- need help desperately.
Anjan


Anjan,

   Well it is only a sugestion.

   About those mount options, perhaps the Oracle people can give you a feedback on that also. But I noticed that you have two mount points, and seems to be installing CRS binaries on one, and RDBMS binaries on the other. So, where are datafiles? 

    You shouldnt be installing the CRS home on /home/oracle/ocrvotcrs as it has the datavolume option. It is not clear if you are trying this.

    The filesystems where you have binaries should not have the datavolume option, and the filesystems with datafiles, voting disks and the cluster registry need to have the datavolume option.

Regards,
Luis

Anjan Chakraborty <anjan.chakraborty at yahoo.com> wrote:    Mark,
  Thanks to both of you for trying to help me.
  I have alreaddy communicated to Luis that I have to have OCFS2 because my software needs Clustering technology that is absent in EXT3. Moreover, both CRS & RDBMS homes should also be on shared/clustered system -- so, OCFS2 is the only choice.
  If you can help me to understand/resolve this issue, I will really appreciate that.
  Please note that my FireWire shared drive works perfectly when I use RAW but as soon as I am trying to use OCFS2, all the problems started.
  Thanks.
  Anjan 

Mark Fasheh <mark.fasheh at oracle.com> wrote:
  Luis,
Thanks for helping here. I have one comment regarding moving the
shared home to ext3. If Anjan's setup is having issues like this, moving his
oracle home to a local disk would only hide problems which take longer to
reproduce for the crs and data files partitions. What needs to happen is
that the root cause of his problems is discovered and fixed.

Keep in mind, I'm not saying someone couldn't use ext3 for their oracle
install - it's an excellent choice for running a non-shared oracle home.
It's just not a good shared disk diagnostic tool ;)

Once again, thanks for stepping up and lending a hand.
--Mark

On Tue, Nov 27, 2007 at 08:50:28AM -0800, Luis Freitas wrote:
> Anjan,
> 
> You dont need to share the database binaries, only the CRS and the
> datafiles. You can do it to save disk space, but it is not mandatory. The CRS
> and datafiles are much less stressfull to the filesystem structures as there is
> a reduced number of large files, although they usually have a heavy i/o load,
> and stress the disk subsystem and the locking algorithms.
> 
> So you can have two separated ext3 filesystems located at the same place on
> each server, and one or more ocfs2 shared filesystems for the CRS and the
> database datafiles. The Oracle installer takes care of copying the binaries
> between the servers during the installation.
> 
> It might be usefull to try a lower version like 1.2.6, as you are using the
> latest version available. I am using 1.2.4-2 here with RH 4.0 and kernel
> 2.6.9-42 and it seems rather stable, only needed to increase the timeouts. (But
> I dont have the oracle_home shared.)
> 
> Also you might have a hardware problem somewhere on the SAN. And I still
> have to check those mount options you sent...
> 
> One detail. I dont know if the Centos distro includes the OCFS2 module. Are
> you using the modules downloaded from the oss.oracle.com site for the
> equivalent RH 4.0 kernel, or modules built by Centos? If using CENTOS modules
> you might get better results by changing to the Oracle built modules for the
> equivalent RH 4.0 kernel.
> 
> Regards,
> Luis
> 
> Anjan Chakraborty wrote:
> 
> Luis,
> I am intending to use CRS/RAC that needs a Cluster File System. How does
> EXT3 falls into that area?
> Thanks.
> Anjan
> 
> Thanks a lot for the response. Here is what I am doing:
> 
> 1.
> mkfs.ocfs2 -b 4K -C 32K -N 4 -L ocrvotcrs /dev/sdb3 -- for CRS
> mkfs.ocfs2 -b 4K -C 32K -N 4 -L orasys /dev/sdb4 -- for RDBMS
> 
> 2. Then mounting using /etc/fstab:
> /dev/sdb3 /home/oracle/ocrvotcrs ocfs2 _netdev,datavolume,nointr
> 0 0
> /dev/sdb4 /home/oracle/orasys ocfs2 _netdev 0 0
> If you find anything wrong here, can you please tell what to do?
> It's a non-production system & so I can experiment with whatever you
> suggest and won't held you responsible for that.
> Thanks.
> Anjan
> 
> 
> Luis Freitas wrote:
> 
> Anjan,
> 
> Are you installing the binaries on OCSF2 too? How are you mounting
> the filesystem?
> 
> You might want to try using ext3 for the binaries and OCF2 only for
> datafiles and archives, until you get this fixed.
> 
> Regards,
> Luis
> 
> Anjan Chakraborty wrote:
> 
> Hi,
> I sent an email to Mark Fisheh of Oracle Corp. & posted this issue
> at OTN under Linux thread this morning. I hope that someone among
> you might have experienced this and can help. On that basis, I am
> sending this to you too. I am stuck & will really appreciate if you
> can shed some light on this.
> Thanks.
> Anjan
> ***********************************************************************************************************
> I have a 2 node CentOS 4.5 86_64 system (kernel 2.6.9-55.EL). On
> this I installed Oracle OCFS2 1.2.7-1 (with exact kernel matching).
> After this I installed Oracle CRS 10.2.0.1 and that installation
> went fine. Then I tried to install Oracle RDBMS 10.2.0.1 and all
> the problems started from there. The /var/log/messages file got
> filled up with messages (giving some to avoid confusion):
> ocfs2_read_locked_inode: .. : ERROR: Invalid dinode #0 signature =
> ocfs2_lookup: .. : ERROR: Unable to create inode ....
> 
> Then OUI gave several error messages, e.g.
> .... Invalid stored block length on file ...../em/em.war followed
> by I/O error in file
> Errors in invoking to files ins_rdbms.mk and ins_ldap.mk
> 
> Then /var/log/messages gave:
> OCFS2: ERROR (device ....): ocfs2_extend_file: Dinode # ...... has
> bad signature O' # I ....
> And the installation failed & CRS died. And the machines reboot.
> I ran fsck.ocfs2 -n /dev/...., it came clean.
> I have tested this several timnes & always same thing happening.
> If I use RAW partitions, everything works fine. So, the problem may
> be in the OCFS2 & OS/Oracle -- but, not sure how to bypass this.
> I have to have OCFS2 -- can't use RAW for various reasons.
> Can somebody please help me to resolve this?
> Thanks.
> ***********************************************************************************************************
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Be a better pen pal. Text or chat with friends inside Yahoo! Mail.
> See how._______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
> 
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Be a better pen pal. Text or chat with friends inside Yahoo! Mail. See
> how.
> 
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Never miss a thing. Make Yahoo your homepage.
> 
> 
> 
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> Get easy, one-click access to your favorites. Make Yahoo! your homepage.

> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
--
Mark Fasheh
Senior Software Developer, Oracle
mark.fasheh at oracle.com

  
  
---------------------------------
  Get easy, one-click access to your favorites. Make Yahoo! your homepage. 
  
  
---------------------------------
  Be a better pen pal. Text or chat with friends inside Yahoo! Mail. See how.
  
  
---------------------------------
  Be a better sports nut! Let your teams follow you with Yahoo Mobile. Try it now.
    
---------------------------------
  Never miss a thing. Make Yahoo your homepage. _______________________________________________
Ocfs2-users mailing list
Ocfs2-users at oss.oracle.com
http://oss.oracle.com/mailman/listinfo/ocfs2-users

       
---------------------------------
Be a better sports nut! Let your teams follow you with Yahoo Mobile. Try it now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20071128/750bc25e/attachment-0001.html