[Ocfs2-users] Missing something basic...
Sunil Mushran
Sunil.Mushran at oracle.com
Fri Oct 19 09:31:10 PDT 2007
Actually GFS(2) is also a shared disk clustered filesystem.
Lustre is a good example of a distributed fs.
paul fretter (TOC) wrote:
> I had similar confusion myself when first looking for a suitable cluster
> FS. I'm not an expert at this, so forgive me if my language appears
> simplistic.
>
> There seemed to be 2 basic species:
> - There are those which aggregate local storage LUNs from each host into
> a single contiguous 'virtual' device, e.g. RedHat GFS etc. Bear in mind
> the4se 'local' LUNS could be a local disk, or a dedicated LUN on a SAN.
>
> - Then there are those which expect all hosts to have direct (shared)
> access to the same LUN.
>
> OCFS2 falls into the latter category.
>
> For disk redundancy you could use a shared disk shelf (e.g. IBM DS4xxx),
> create a RAID(1,4 or 5) set in the hardware and present it as a single
> shared-access LUN. All your nodes then have a connection to the LUN
> (e.g. by fibrechannel). In effect it is a small SAN! OCFS2 is then the
> glue which manages the file locking and metadata so that nodes don't try
> to write to the same blocks at the same time.
>
> By creating a "RAID" using multiple OCFS2 volumes you are bringing the
> role of device level redundancy work into the OS, which is effectively
> software RAID and is not conducive to high performance or reliability.
>
> Then for high availability of the storage, a good way might be to create
> a duplicate shared device and let the hardware perform mirroring for you
> (e.g. over fibrechannel, infiniband, iSCSI) etc, and also let the
> hardware do the failover for you.
>
> So, by using a SAN with hardware RAID and hardware mirroring to a second
> SAN device (with hardware RAID), you can achieve resilience and high
> availability, leaving OCFS2 blissfully unaware.
>
> Hope this helps.
>
> Kind regards
> Paul Fretter
>
>
>> -----Original Message-----
>> From: ocfs2-users-bounces at oss.oracle.com [mailto:ocfs2-users-
>> bounces at oss.oracle.com] On Behalf Of Randy Ramsdell
>> Sent: 18 October 2007 14:00
>> Cc: ocfs2-users at oss.oracle.com
>> Subject: Re: [Ocfs2-users] Missing something basic...
>>
>> Benjamin Smith wrote:
>>
>>> I'm stumped. I'm doing some research on clustered file systems to be
>>>
>> deployed
>>
>>> over winter break, and am testing on spare machines first.
>>>
>>> I have two identically configured computers, each with a 10 GB
>>> partition, /dev/hda2. I intend to combine these two LAN/RAID1 style
>>>
>> to
>>
>>> represent 10 GB of redundant cluster storage, so that if either
>>>
>> machine
>>
>>> fails, computing can resume with reasonable efficiency.
>>>
>>> These machines are called "cluster1" and "cluster2", and are
>>>
>> currently on a
>>
>>> local Gb LAN. They are running CentOS 4.4 (recompile of RHEL 4.4)
>>>
>> I've set up
>>
>>> SSH RSA keys so that I can ssh directly from either to the other
>>>
>> without
>>
>>> passwords, though I use a non-standard port, defined in ssh_config
>>>
>> and
>>
>>> sshd_config.
>>>
>>> I've installed the RPMs without incident. I've set up a cluster
>>>
>> called "ocfs2"
>>
>>> with nodes "cluster1" and "cluster2", with the corresponding LAN IP
>>> addresses. I've confirmed that configuration changes populate to
>>>
>> cluster2
>>
>>> when I push the appropriate button in the X11 ocfs2console on
>>>
>> cluster1. I've
>>
>>> checked the firewall(s) to allow inbound TCP to port 7777
>>>
> connections
>
>> on both
>>
>>> machines, and verified this with nmap. I've also tried turning off
>>>
>> iptables
>>
>>> completely. On cluster1, I've formatted and mounted partition
>>>
>> "oracle"
>>
>>> to /meda/cluster using the ocfs2console and I can r/w to this
>>>
>> partition with
>>
>>> other applications. There's about a 5-second delay when
>>>
>> mounting/unmounting,
>>
>>> and the FAQ reflects that this is normal. SELinux is completely off.
>>>
>>> Questions:
>>>
>>> 1) How do I get this "oracle" partition to show/mount on host
>>>
>> cluster2, and
>>
>>> subsequent systems added to the cluster? Should I be expecting a
>>>
>> /dev/* block
>>
>>> device to mount, or is there some other program I should be using,
>>>
>> similar to
>>
>>> smbmount?
>>>
>>>
>> As the previous post states, you need a shared storage. A quick and
>> easy
>> way to do this is to install iscsi-target on another system (target1)
>> and then use open-iscsi to log into the target you just created. So
>> have a third system that create the shared target. Then on cluster1
>> log
>> into the target to create the ocfs2 cluster FS. At this point, you can
>> mount this target on cluster1. On cluster2, log into the target and
>> mount as you would normally. Of course you will need the correct
>> cluster
>> set up. Now you have two systems mounting the shared storage and both
>> r/w.
>>
>> Note: You may be able to do this with just two systems. Use cluster1
>> as
>> the iscsi target system and ocfs2. On cluster1 install iscsi-target
>> software and log into the volume share from cluster1 itself. Cluster2
>> would just log in to the the target as normal.
>>
>>
>>> 2) How do I get this /dev/hda2 (aka "oracle") on cluster1 to combine
>>>
>> (RAID1
>>
>>> style) with /dev/hda2 on cluster2, so that if either host goes down
>>>
> I
>
>> still
>>
>>> have a complete FS to work from? Am I mis-understanding the
>>>
> abilities
>
>> and
>>
>>> intentions of OCFS2? Do I need to do something with NBD, GNBD, ENDB,
>>>
>> or
>>
>>> similar? If so, what's the "recommended" approach?
>>>
>>>
>>>
>> Yes you are misunderstanding how ocfs2 works. To use raid for the
>> described purposes, you must use it on the target1 system mentioned
>> above. On target1, raid two drives or two partitions and then use this
>> array as the target volume you export to cluster1 and cluster2. This
>> way
>> you have a raid array for data protection and ocfs2 for service
>> integrity.
>>
>>> Thanks,
>>>
>>> -Ben
>>>
>>>
>>>
>> _______________________________________________
>> Ocfs2-users mailing list
>> Ocfs2-users at oss.oracle.com
>> http://oss.oracle.com/mailman/listinfo/ocfs2-users
>>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
>
More information about the Ocfs2-users
mailing list