We have never used RDM in the past due to backup reasons etc and VM admins not having to deal the SAN admins. Do you think this would resolve the issue?<div> </div><div><br><br><div class="gmail_quote">On Tue, Dec 14, 2010 at 3:25 PM, Sunil Mushran <span dir="ltr"><<a href="mailto:sunil.mushran@oracle.com">sunil.mushran@oracle.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div bgcolor="#ffffff" text="#000000">
I meant repeats 60 secs at a stretch. If not, as it seems so, then
the messages<br>
should be only annoying.<br>
<br>
VMFS uses SCSI Reservation to perform disk based locking. See if
they have<br>
some logging in ESX that shows when a VMFS performs
reserve/unreserve<br>
on a SCSI device. You'll have to look at the logs of all nodes. As
in, that log<br>
will be on a different node than that that got this error.<br>
<br>
BTW, any reason you are not using RDM.<div><div></div><div class="h5"><br>
<br>
On 12/14/2010 12:51 PM, brad hancock wrote:
<blockquote type="cite">The issue does repeat.
<div>
<div><br>
</div>
<div>I looked through the vsphere 4.1, and the host logs and
didn't see anything weird that corresponds with these times. </div>
<div><br>
</div>
<div>What is a reservation conflict? Can this issue cause the
nodes to see different data?</div>
<div><br>
</div>
<div><br>
</div>
<div>Dec 14 07:37:52 mdcvmsmes02 kernel: [351952.113847] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 07:37:52 mdcvmsmes02 kernel: [351952.113859] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 07:37:52 mdcvmsmes02 kernel: [351952.113868]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 07:37:52 mdcvmsmes02 kernel: [351952.114134]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 07:37:52 mdcvmsmes02 kernel: [351952.114379]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.233764] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.233775] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.233855]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.234112]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.234365]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.234789] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.234793] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.234796]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.235033]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 07:51:01 mdcvmsmes02 kernel: [352762.235273]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 09:23:15 mdcvmsmes02 kernel: [358423.734356] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 09:23:15 mdcvmsmes02 kernel: [358423.734366] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 09:23:15 mdcvmsmes02 kernel: [358423.734370]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 09:23:15 mdcvmsmes02 kernel: [358423.734620]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 09:23:15 mdcvmsmes02 kernel: [358423.734882]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.184302] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.184312] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.184316]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.184565]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.184809]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.188045] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.188045] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.188045]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.188045]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 10:25:27 mdcvmsmes02 kernel: [362254.188045]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062] sd
1:0:0:0: reservation conflict</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062] sd
1:0:0:0: [sdb] Result: hostbyte=DID_OK
driverbyte=DRIVER_OK,SUGGEST_OK</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062]
end_request: I/O error, dev sdb, sector 1735</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062]
(0,0):o2hb_bio_end_io:225 ERROR: IO Error -5</div>
<div>Dec 14 10:33:08 mdcvmsmes02 kernel: [362727.621062]
(1882,0):o2hb_do_disk_heartbeat:753 ERROR: status = -5</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<br>
<div class="gmail_quote">On Tue, Dec 14, 2010 at 11:38 AM, Sunil
Mushran <span dir="ltr"><<a href="mailto:sunil.mushran@oracle.com" target="_blank">sunil.mushran@oracle.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div bgcolor="#ffffff" text="#000000">
<div>
<div>sd 1:0:0:0: reservation conflict<br>
<br>
</div>
That's the cause of the error in the guest. You'll have
to track the error<br>
to ESX's management domain. See the logs.<br>
<br>
Does this error come repeatedly? This error is only a
problem for o2hb<br>
if it continues for the next 60 secs. Else it can be
ignored.
<div>
<div><br>
<br>
On 12/14/2010 07:20 AM, brad hancock wrote:</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>The issue is starting to come up
again. Both machines are logging the error a couple
of minutes apart from each other.
<div><br>
<div>sd 1:0:0:0: reservation conflict</div>
<div>Dec 13 16:40:07 mdcvmsmes01 kernel:
[295051.378262] sd 1:0:0:0: [sdb] Result:
hostbyte=DID_OK d</div>
<div>Dec 13 16:40:07 mdcvmsmes01 kernel:
[295051.378347] end_request: I/O error, dev sdb,
sector 173</div>
<div>Dec 13 16:40:07 mdcvmsmes01 kernel:
[295051.378694] (0,1):o2hb_bio_end_io:225 ERROR:
IO Error -</div>
<div>Dec 13 16:40:07 mdcvmsmes01 kernel:
[295051.379055]
(1897,1):o2hb_do_disk_heartbeat:753 ERROR:</div>
</div>
<div><br>
</div>
<div>Should I open a bug report? Who with, VMware or
Oracle?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<div class="gmail_quote">On Sun, Dec 12, 2010 at
9:25 AM, brad hancock <span dir="ltr"><<a href="mailto:braddhancock@gmail.com" target="_blank">braddhancock@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">Kevin,
<div>I modified the VMFS virtual disk to
Independent, and I haven't seen the issue
since the change Friday morning. I noticed
this didn't work for you. I will continue to
watch it and let the list know. The issue I
saw after several weeks was the data was not
in sync. Two nodes saw different data on the
same OCFS2 drive. </div>
<div><br>
</div>
<div>We have Vsphere 4.1, and HP EVA 3000 SAN.
</div>
<div><br>
</div>
<div>Thanks,</div>
<div>
<div>
<div><br>
</div>
<div><br>
<br>
<div class="gmail_quote">On Sat, Dec 11,
2010 at 10:41 AM, <span dir="ltr"><<a href="mailto:kevin@utahsysadmin.com" target="_blank">kevin@utahsysadmin.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div>On Fri, 10 Dec 2010 06:26:06
-0800, <a href="mailto:ocfs2-users-request@oracle.com" target="_blank">ocfs2-users-request@oracle.com</a>
wrote:<br>
><br>
> My setup has the SCSI
controller set to Physical so the
guest can be on<br>
> different hosts, but I do not
have the disk setup as
Independent. I am<br>
> going<br>
> to change that setting in
VMware and see if it makes a
difference.<br>
><br>
</div>
<div>> > [2037805.922718]
end_request: I/O error, dev sdb,
sector 1735<br>
> > [2037805.922974]
(0,0):o2hb_bio_end_io:225 ERROR:
IO Error -5<br>
> > [2037805.923370]
(27506,0):o2hb_do_disk_heartbeat:753
ERROR: status =<br>
-5<br>
<br>
</div>
Brad,<br>
<br>
I have had the same issue for over a
year on ESX 3.5 as well as on
vSphere<br>
4.0. I have not tried yet on 4.1.
The error occurs when I put the
shared<br>
disk on either SATA or FC LUNs on
our SAN. It also doesn't matter if
the<br>
virtual machines are on the same
physical host or not (with
independent<br>
disks). The only problem that has
come from it is the occasional
reboot of<br>
one of the VMs, which for me is
tolerable. I keep hoping to upgrade
to a<br>
new SAN thinking that might fix it.
The vSphere 4.0 release IOPS<br>
capability is higher than the SAN
(it's 5 years old) so I didn't think
it<br>
was VMware's fault. If you have
fairly new hardware, maybe there is
a real<br>
bug somewhere. I don't get I/O
errors in any of my other
implementations<br>
on this SAN. I sent a post like
yours to the list when I first built
it,<br>
but never opened a bug report with
either OCFS or VMware. If you
create a<br>
bug report I could add information
from my implementation as well. (I<br>
actually have two of these setups
and they both have the same errors.)<br>
<br>
Of course, if you find a solution,
please post that as well.<br>
<br>
Thanks,<br>
Kevin<br>
<br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<pre><fieldset></fieldset>
_______________________________________________
Ocfs2-users mailing list
<a href="mailto:Ocfs2-users@oss.oracle.com" target="_blank">Ocfs2-users@oss.oracle.com</a>
<a href="http://oss.oracle.com/mailman/listinfo/ocfs2-users" target="_blank">http://oss.oracle.com/mailman/listinfo/ocfs2-users</a></pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>