[Ocfs2-users] heartbeat write timeout

[Stephan A. Rickauer]

Sunil Mushran wrote:
> Are you seeing timeouts with elevator=deadline?

Yes, I can confirm that I have timeouts also with the deadline scheduler.

> We only test with the default value and have not seen any
> disk hb timeouts on either 2G fc or gige iscsi. And these
> are heavy db loads.

Let me tell you how my test setup looks like: Two AMD Opteron 2.8 GhZ
machines with 2GB RAM each are connected using Broadcom NetXtreme
BCM5704 Gigabit Ethernet NIC's over Dell PowerConnect 5324 Gigabit
switches to a Gigabit iSCSI SATA storage device (Transtec Provigo)
serving up to 8 TB disk space.

I've run load test using bonnie++ over a couple of days now which shows
read/write performance with OCFS2 of around 85MB/s. Testing on RH AS4 I
can confirm reproducable crashes with cfq scheduler AND heartbeart
default 7 after a couple of minutes of heavy load operation.

After changing the scheduler as mentioned in the FAQ and reported on
this list (thanks, guys) I _again_ had hb timeout related crashes after
a few minutes of load. After changing the HEARTBEAT_THRESHOLD to 30 as
mentioned by Gavin (thanks) my bonnie++ tests runs fine for more than 12
hours now. I'd call that heavy load, too. ;)

However, the story seems to be slightly different on SuSE 10.1b6: Though
a change of the scheduler and the adjustment of HEARTBEAT_THRESHOLD
improved stability I still have crashes after a couple of hours (instead
of 'minutes' before). However, OCFS2 seems to be way faster on SuSE (can
someone confirm that?) than on Red Hat, which leads me to the assumption
another raise of the HEARTBEAT_THRESHOLD value might fix the problem again.

> When the hb thread panics, it dumps messages indicating
> the times it took to perform the tasks. Could you share
> those messages?

Actually, I have not seen those messages. Give me a couple of minutes
and I will reproduce the crash to post the numbers here.

Thanks for your help!

-- 

 Stephan A. Rickauer

 -----------------------------------------------------------
 Institut für Neuroinformatik          Tel: +41 44 635 30 50
 Universität / ETH Zürich              Sek: +41 44 635 30 52
 Winterthurerstrasse 190               Fax: +41 44 635 30 53
 CH-8057 Zürich                        Web:  www.ini.ethz.ch

 RSA public key: https://www.ini.ethz.ch/~stephan/pubkey.asc
 -----------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 890 bytes
Desc: OpenPGP digital signature
Url : http://oss.oracle.com/pipermail/ocfs2-users/attachments/20060331/88dda96d/signature.bin