[Oracleasm-users] Shifting from OCFS2 to ASM

[Joel Becker]

On Wed, Jun 17, 2009 at 03:44:04AM +0300, Karim Alkhayer wrote:
> We're considering ASM with Oracle Clusterware 10.2.0.4 as a replacement to
> OCFS2 in order to maintain a more reliable RAC. We've been hitting stability
> issues with our 2 node RAC running IA64 SLES9 SP3 and Oracle 10.1 and OCFS2
> 1.2.1 - As the upgrade options are limited, ASM seems to be a better
> alternative and been recommended by Oracle support as well.

	Wearing my ocfs2 hat, I have to wonder if we've heard from you
about these stability issues.  1.2 with 2 nodes should be very stable,
and just as viable as ASM.
	Now putting on my oracleasm hat, because using ASM or ocfs2 or
both (both is quite usable too) is up to you in the end.

> 1.     The primary RAC is hosting 5 databases with around 500 GB divided
> into a few disk groups with RAID 6 hosted in two SANs; these disk groups are
> composed of disks of 36GB with 15K RPM and 146GB with 10K RPM. I'd like to
> know if we need to break the RAID6. If so, what is the best recommended RAID
> type? 

[ Caveat - I don't work on ASM, this mailing list is for oracleasm, the
  Linux ASMLib.  You may want to contact Oracle support to get more
  direct communication with the ASM folks. ]

	You don't need to break the raid6.  The question is whether you
want to.  I assume that you don't mix your disks - that some raid6
groups are 15K RPM disks and other raid6 groups are 10K RPM disks.  I
further assume that they are on some sort of smart storage with a cache.
	I would start by keeping the raid6 LUNs.  When you give them
to ASM, make sure that they are aligned.  ASM stripes on 1MB boundaries.
So make sure that the partitions you create on these LNUs start at an
appropriate stripe boundary on the array.  That way, ASM's stripes line
up with the array's stripes.  Get this wrong and the performance
suffers.
	If you have a good raid6 environment, you probably want to use
'external redundancy' when creating your diskgroups.  That's telling ASM
not to handle redundancy because you trust your storage to do it.
	See how the performance is (I expect it to be fine).  If there
is a problem, the ASM team and Oracle support can help you figure it out
better.

> 2.     We've got consistent shared LUN listing across the cluster, an
> example is /dev/mapper/2003013840ad20008 (we're using this method to create
> and mount OCFS2 volumes); is it possible to maintain the same way of
> discovery? There is a note in "Configuring Oracle ASMLib on Multipath Disks"
> saying that dm prefix must be used for the scan order and exclude. Do we
> still need ASMLib, and can we use the device mapper instead of dm prefix?

	Here I can help.  dm is 'device mapper'.  It's just that the
current scan code relies on /proc/partitions, which displays 'dm', not
'mapper/'.  The mapper/ names are just aliases for the dm devices.
	Are your dm devices just multipaths?  Or do you do other things?
I'm assuming multipath, as ASM will do the LVM for you :-)
	You can, if you like, just use regular paths and no ASMLib for
your ASM devices.  Make your diskstring "/dev/mapper/*" or something
like it.  This will use POSIX I/O ops on your disks instead of ASMLib.
However, you lose the benefits of ASMLib.
	If you want to use ASMLib, you can easily make it work with your
multipath devices.  First, if you are on RHEL5, comment out the
following line in /etc/udev/rules.d/50-udev.rules:

#KERNEL=="dm-[0-9]*", ACTION=="add", OPTIONS+="ignore_device"

Next, set 'dm' in your SCANORDER as specified in the document on
multipath configuration.  This will detect your device mapper multipath
devices.  Tada!
	If you have any furthur problems, I have fancier scanning code
in the works, and you could try it out if you like.

Joel

-- 

"The nearest approach to immortality on Earth is a government
 bureau."
	- James F. Byrnes

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127