[Ocfs2-users] Tracking down hangs

Andrew Robert Nicols andrew.nicols at luns.net.uk
Thu Jun 3 03:12:49 PDT 2010


We're using a storage solution involving two SunFire X4500 servers using
DRBD to replicate a 15TB partition across the network with ocfs2 on top.
We're sharing the partition from one server over NFS and the other is
mounted read-only at present.  The DBRD backing store is software RAID 60
on 40 disks.

We've been seeing periodic issues whereby our NFS clients (Debian Lenny)
are very slow to perform simple operations such as cat a 4 character file,
or perform an ls.

This affects all of the NFS clients at the same time and typically lasts
from between a few seconds to maybe 2 minutes. Operation then continues as
normal and service resumes. We've also seen this affecting the read-only
server which has the ocfs2 partition mounted.

We've been having trouble trying to find out the cause of the issues. but
can reliably reproduce such failures as follows:

On each host, check for cats taking longer than 1 second:
while true; do time cat /srv/healthchecks/smallfile > /dev/null; done 2>&1 | awk '/m[1-9]/ {print strftime(), $_}'

To actually reproduce the failure, we then run a dd on the filestore:
dd if=/dev/zero of=/srv/test/dd-test-`date +%s` bs=1M count=1000 && echo "Syncing" && time sync

At the time that the sync finishes, all of the NFS clients and the
read-only server show that it took some time to return the cat of an
unrelated file - usually the same amount of time it took to run the sync.

What's the best place to start looking for the cause of these hangs? I've
attached the dmesg output which includes some call traces for hung threads. 
I have stat_sysdir output though I suspect it's not so relevant. A
scanlocks output doesn't reveal any busy locks that I can see (unless I'm
not hitting it at the right time or misreading the output).

For the DRBD replication there's a pair of bonded GBit NICs dedicated to
the job. The other two GBit bonded NICs in the boxes are being used for NFS
and o2net/ocfs communication. We don't believe that the network is at
fault.

We're using Debian Lenny with the stock AMD64 kernel out of the Lenny
repository - 2.6.26-2-amd64.
We're using ocfs2 tools version 1.4.4 which we have packaged for Debian ourselves.

The ocfs2 version reported in /sys/module/ocfs2/version is 1.5.0.

Here's the current o2cb configuration:

root at thumper5:#/srv/test# /etc/init.d/o2cb status
Driver for "configfs": Loaded
Filesystem "configfs": Mounted
Stack glue driver: Loaded
Stack plugin "o2cb": Loaded
Driver for "ocfs2_dlmfs": Loaded
Filesystem "ocfs2_dlmfs": Mounted
Checking O2CB cluster thumperpool: Online
Heartbeat dead threshold = 61
  Network idle timeout: 60000
  Network keepalive delay: 2000
  Network reconnect delay: 2000
Checking O2CB heartbeat: Active

We also tried a Heartbeat dead threshold of 31 with a Network idle timeout
of 30000 to the same effect.

Any assistance would be very much appreciated,

Andrew

-- 
Systems Developer

e: andrew.nicols at luns.net.uk
im: a.nicols at jabber.lancs.ac.uk
t: +44 (0)1524 5 10147

Lancaster University Network Services is a limited company registered in
England and Wales. Registered number: 04311892. Registered office:
University House, Lancaster University, Lancaster, LA1 4YW
-------------- next part --------------
33 mmio m1048576 at 0xfd700000 port 0xfd726000 irq 38
[   14.104093] ata20: SATA max UDMA/133 mmio m1048576 at 0xfd700000 port 0xfd728000 irq 38
[   14.104093] ata21: SATA max UDMA/133 mmio m1048576 at 0xfd700000 port 0xfd732000 irq 38
[   14.104093] ata22: SATA max UDMA/133 mmio m1048576 at 0xfd700000 port 0xfd734000 irq 38
[   14.104093] ata23: SATA max UDMA/133 mmio m1048576 at 0xfd700000 port 0xfd736000 irq 38
[   14.104093] ata24: SATA max UDMA/133 mmio m1048576 at 0xfd700000 port 0xfd738000 irq 38
[   14.104093] ACPI: PCI Interrupt 0000:03:00.0[D] -> GSI 19 (level, low) -> IRQ 19
[   14.104093] ohci_hcd 0000:03:00.0: OHCI Host Controller
[   14.104093] ohci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 1
[   14.104093] ohci_hcd 0000:03:00.0: irq 19, io mem 0xfd1fe000
[   14.634168] usb usb1: configuration #1 chosen from 1 choice
[   14.634197] hub 1-0:1.0: USB hub found
[   14.634207] hub 1-0:1.0: 3 ports detected
[   14.756186] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
[   14.756188] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   14.756191] usb usb1: Product: OHCI Host Controller
[   14.756192] usb usb1: Manufacturer: Linux 2.6.26-2-amd64 ohci_hcd
[   14.756195] usb usb1: SerialNumber: 0000:03:00.0
[   14.760094] ACPI: PCI Interrupt 0000:03:00.1[D] -> GSI 19 (level, low) -> IRQ 19
[   14.760094] ohci_hcd 0000:03:00.1: OHCI Host Controller
[   14.760094] ohci_hcd 0000:03:00.1: new USB bus registered, assigned bus number 2
[   14.760094] ohci_hcd 0000:03:00.1: irq 19, io mem 0xfd1fd000
[   14.830349] usb usb2: configuration #1 chosen from 1 choice
[   14.830375] hub 2-0:1.0: USB hub found
[   14.830384] hub 2-0:1.0: 3 ports detected
[   14.964228] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
[   14.964231] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   14.964233] usb usb2: Product: OHCI Host Controller
[   14.964235] usb usb2: Manufacturer: Linux 2.6.26-2-amd64 ohci_hcd
[   14.964237] usb usb2: SerialNumber: 0000:03:00.1
[   14.964317] ACPI: PCI Interrupt 0000:03:04.0[A] -> GSI 17 (level, low) -> IRQ 17
[   14.964325] ohci_hcd 0000:03:04.0: OHCI Host Controller
[   14.964366] ohci_hcd 0000:03:04.0: new USB bus registered, assigned bus number 3
[   14.964383] ohci_hcd 0000:03:04.0: irq 17, io mem 0xfd1fc000
[   15.051204] usb usb3: configuration #1 chosen from 1 choice
[   15.051228] hub 3-0:1.0: USB hub found
[   15.051236] hub 3-0:1.0: 3 ports detected
[   15.107086] usb 1-1: new full speed USB device using ohci_hcd and address 2
[   15.119952] ata17: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   15.151896] ata17.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPU78F, K2AOAJ0A, max UDMA/133
[   15.151898] ata17.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   15.151905] ata17.00: max_sectors limited to 256 for NCQ
[   15.153868] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
[   15.153870] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   15.153872] usb usb3: Product: OHCI Host Controller
[   15.153874] usb usb3: Manufacturer: Linux 2.6.26-2-amd64 ohci_hcd
[   15.153875] usb usb3: SerialNumber: 0000:03:04.0
[   15.153930] ACPI: PCI Interrupt 0000:03:04.1[B] -> GSI 18 (level, low) -> IRQ 18
[   15.153938] ohci_hcd 0000:03:04.1: OHCI Host Controller
[   15.153980] ohci_hcd 0000:03:04.1: new USB bus registered, assigned bus number 4
[   15.153995] ohci_hcd 0000:03:04.1: irq 18, io mem 0xfd1fb000
[   15.181318] ata17.00: max_sectors limited to 256 for NCQ
[   15.181320] ata17.00: configured for UDMA/133
[   15.241133] usb usb4: configuration #1 chosen from 1 choice
[   15.241158] hub 4-0:1.0: USB hub found
[   15.241166] hub 4-0:1.0: 2 ports detected
[   15.316113] usb 1-1: configuration #1 chosen from 1 choice
[   15.335451] usb 1-1: New USB device found, idVendor=046b, idProduct=ff10
[   15.335454] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   15.335456] usb 1-1: Product: Virtual Keyboard and Mouse
[   15.335458] usb 1-1: Manufacturer: American Megatrends Inc.
[   15.351030] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
[   15.351033] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   15.351035] usb usb4: Product: OHCI Host Controller
[   15.351037] usb usb4: Manufacturer: Linux 2.6.26-2-amd64 ohci_hcd
[   15.351039] usb usb4: SerialNumber: 0000:03:04.1
[   15.358952] ACPI: PCI Interrupt 0000:03:04.2[C] -> GSI 19 (level, low) -> IRQ 19
[   15.358952] ehci_hcd 0000:03:04.2: EHCI Host Controller
[   15.358952] ehci_hcd 0000:03:04.2: new USB bus registered, assigned bus number 5
[   15.358952] ehci_hcd 0000:03:04.2: irq 19, io mem 0xfd1fac00
[   15.374942] ehci_hcd 0000:03:04.2: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
[   15.374992] usb usb5: configuration #1 chosen from 1 choice
[   15.375016] hub 5-0:1.0: USB hub found
[   15.375023] hub 5-0:1.0: 5 ports detected
[   15.518363] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
[   15.518365] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   15.518368] usb usb5: Product: EHCI Host Controller
[   15.518370] usb usb5: Manufacturer: Linux 2.6.26-2-amd64 ehci_hcd
[   15.518372] usb usb5: SerialNumber: 0000:03:04.2
[   15.727477] ata18: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   15.790062] ata18.00: ATA-7: HITACHI HDS7250SASUN500G 0648KP59ZF, K2AOAJ0A, max UDMA/133
[   15.790065] ata18.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   15.790072] ata18.00: max_sectors limited to 256 for NCQ
[   15.814008] ata18.00: max_sectors limited to 256 for NCQ
[   15.814010] ata18.00: configured for UDMA/133
[   15.837918] usb 5-5: new high speed USB device using ehci_hcd and address 2
[   15.991509] usb 5-5: configuration #1 chosen from 1 choice
[   15.991723] hub 5-5:1.0: USB hub found
[   15.991844] hub 5-5:1.0: 4 ports detected
[   16.117519] usb 5-5: New USB device found, idVendor=04b4, idProduct=6560
[   16.117522] usb 5-5: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[   16.373941] ata19: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   16.405891] ata19.00: ATA-7: HITACHI HDS7250SASUN500G 0648KP66GH, K2AOAJ0A, max UDMA/133
[   16.405894] ata19.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   16.405901] ata19.00: max_sectors limited to 256 for NCQ
[   16.429824] ata19.00: max_sectors limited to 256 for NCQ
[   16.429826] ata19.00: configured for UDMA/133
[   16.904869] ata20: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   16.936788] ata20.00: ATA-7: HITACHI HDS7250SASUN500G 0647KND5RH, K2AOAJ0A, max UDMA/133
[   16.936790] ata20.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   16.936797] ata20.00: max_sectors limited to 256 for NCQ
[   16.960733] ata20.00: max_sectors limited to 256 for NCQ
[   16.960735] ata20.00: configured for UDMA/133
[   17.464022] ata21: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   17.495960] ata21.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPME3F, K2AOAJ0A, max UDMA/133
[   17.495963] ata21.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   17.495970] ata21.00: max_sectors limited to 256 for NCQ
[   17.519906] ata21.00: max_sectors limited to 256 for NCQ
[   17.519908] ata21.00: configured for UDMA/133
[   18.221278] ata22: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   18.501146] ata22.00: ATA-7: HITACHI HDS7250SASUN500G 0648KMTE5H, K2AOAJ0A, max UDMA/133
[   18.501149] ata22.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   18.501156] ata22.00: max_sectors limited to 256 for NCQ
[   18.525092] ata22.00: max_sectors limited to 256 for NCQ
[   18.525094] ata22.00: configured for UDMA/133
[   19.001687] ata23: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   19.033625] ata23.00: ATA-7: HITACHI HDS7250SASUN500G 0648KM1W7F, K2AOAJ0A, max UDMA/133
[   19.033628] ata23.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   19.033635] ata23.00: max_sectors limited to 256 for NCQ
[   19.057570] ata23.00: max_sectors limited to 256 for NCQ
[   19.057572] ata23.00: configured for UDMA/133
[   19.533692] ata24: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   19.565625] ata24.00: ATA-7: HITACHI HDS7250SASUN500G 0648KMZY0F, K2AOAJ0A, max UDMA/133
[   19.565628] ata24.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   19.565636] ata24.00: max_sectors limited to 256 for NCQ
[   19.589571] ata24.00: max_sectors limited to 256 for NCQ
[   19.589573] ata24.00: configured for UDMA/133
[   19.591331] scsi 16:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 17:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 18:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 19:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 20:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 21:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 22:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] scsi 23:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   19.591331] ACPI: PCI Interrupt 0000:06:01.0[A] -> GSI 46 (level, low) -> IRQ 46
[   19.591331] sata_mv 0000:06:01.0: Gen-II 32 slots 8 ports SCSI mode IRQ via INTx
[   19.591331] scsi24 : sata_mv
[   19.591331] scsi25 : sata_mv
[   19.591331] scsi26 : sata_mv
[   19.591331] scsi27 : sata_mv
[   19.591331] scsi28 : sata_mv
[   19.591331] scsi29 : sata_mv
[   19.591331] scsi30 : sata_mv
[   19.591331] scsi31 : sata_mv
[   19.591331] ata25: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd922000 irq 46
[   19.591331] ata26: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd924000 irq 46
[   19.591331] ata27: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd926000 irq 46
[   19.591331] ata28: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd928000 irq 46
[   19.591331] ata29: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd932000 irq 46
[   19.591331] ata30: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd934000 irq 46
[   19.591331] ata31: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd936000 irq 46
[   19.591331] ata32: SATA max UDMA/133 mmio m1048576 at 0xfd900000 port 0xfd938000 irq 46
[   20.537828] ata25: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   20.553806] ata25.00: ATA-7: HITACHI HDS7250SASUN500G 0647KLEY3H, K2AOAJ0A, max UDMA/133
[   20.553808] ata25.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   20.553815] ata25.00: max_sectors limited to 256 for NCQ
[   20.577751] ata25.00: max_sectors limited to 256 for NCQ
[   20.577753] ata25.00: configured for UDMA/133
[   21.501146] ata26: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   21.533160] ata26.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPU6RF, K2AOAJ0A, max UDMA/133
[   21.533163] ata26.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   21.533170] ata26.00: max_sectors limited to 256 for NCQ
[   21.557164] ata26.00: max_sectors limited to 256 for NCQ
[   21.557166] ata26.00: configured for UDMA/133
[   22.035477] ata27: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   22.242982] ata27.00: ATA-7: HITACHI HDS7250SASUN500G 0648KM1VNF, K2AOAJ0A, max UDMA/133
[   22.242984] ata27.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   22.242991] ata27.00: max_sectors limited to 256 for NCQ
[   22.266928] ata27.00: max_sectors limited to 256 for NCQ
[   22.266930] ata27.00: configured for UDMA/133
[   23.618730] ata28: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   23.650668] ata28.00: ATA-7: HITACHI HDS7250SASUN500G 0648KM1WLF, K2AOAJ0A, max UDMA/133
[   23.650672] ata28.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   23.650679] ata28.00: max_sectors limited to 256 for NCQ
[   23.674614] ata28.00: max_sectors limited to 256 for NCQ
[   23.674616] ata28.00: configured for UDMA/133
[   24.564391] ata29: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   24.580368] ata29.00: ATA-7: HITACHI HDS7250SASUN500G 0648KMPDRF, K2AOAJ0A, max UDMA/133
[   24.580371] ata29.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   24.580378] ata29.00: max_sectors limited to 256 for NCQ
[   24.604314] ata29.00: max_sectors limited to 256 for NCQ
[   24.604317] ata29.00: configured for UDMA/133
[   25.080916] ata30: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   25.112925] ata30.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPU2DF, K2AOAJ0A, max UDMA/133
[   25.112928] ata30.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   25.112935] ata30.00: max_sectors limited to 256 for NCQ
[   25.136929] ata30.00: max_sectors limited to 256 for NCQ
[   25.136931] ata30.00: configured for UDMA/133
[   25.615242] ata31: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   25.647180] ata31.00: ATA-7: HITACHI HDS7250SASUN500G 0648KNZX6F, K2AOAJ0A, max UDMA/133
[   25.647183] ata31.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   25.647190] ata31.00: max_sectors limited to 256 for NCQ
[   25.671126] ata31.00: max_sectors limited to 256 for NCQ
[   25.671128] ata31.00: configured for UDMA/133
[   26.443694] ata32: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   26.475632] ata32.00: ATA-7: HITACHI HDS7250SASUN500G 0647KLP8DF, K2AOAJ0A, max UDMA/133
[   26.475635] ata32.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   26.475642] ata32.00: max_sectors limited to 256 for NCQ
[   26.993056] ata32.00: max_sectors limited to 256 for NCQ
[   26.993058] ata32.00: configured for UDMA/133
[   26.993013] scsi 24:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 25:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 26:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 27:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 28:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 29:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 30:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993013] scsi 31:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   26.993509] ACPI: PCI Interrupt 0000:0a:01.0[A] -> GSI 68 (level, low) -> IRQ 68
[   26.995314] ACPI: PCI Interrupt 0000:07:01.0[A] -> GSI 52 (level, low) -> IRQ 52
[   26.993509] sata_mv 0000:0a:01.0: Gen-II 32 slots 8 ports SCSI mode IRQ via INTx
[   26.993509] scsi32 : sata_mv
[   26.993509] scsi33 : sata_mv
[   26.993509] scsi34 : sata_mv
[   26.993509] scsi35 : sata_mv
[   26.993509] scsi36 : sata_mv
[   26.993509] scsi37 : sata_mv
[   26.993509] scsi38 : sata_mv
[   26.993510] scsi39 : sata_mv
[   26.993510] ata33: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe322000 irq 68
[   26.993510] ata34: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe324000 irq 68
[   26.993510] ata35: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe326000 irq 68
[   26.993510] ata36: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe328000 irq 68
[   26.993510] ata37: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe332000 irq 68
[   26.993510] ata38: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe334000 irq 68
[   26.993510] ata39: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe336000 irq 68
[   26.993510] ata40: SATA max UDMA/133 mmio m1048576 at 0xfe300000 port 0xfe338000 irq 68
[   27.739967] e1000: 0000:07:01.0: e1000_probe: (PCI-X:133MHz:64-bit) 00:14:4f:21:19:f8
[   27.778383] e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
[   27.778399] ACPI: PCI Interrupt 0000:07:01.1[B] -> GSI 53 (level, low) -> IRQ 53
[   27.964222] ata33: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   27.996239] ata33.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPS0PF, K2AOAJ0A, max UDMA/133
[   27.996241] ata33.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   27.996249] ata33.00: max_sectors limited to 256 for NCQ
[   28.020242] ata33.00: max_sectors limited to 256 for NCQ
[   28.020245] ata33.00: configured for UDMA/133
[   28.050982] e1000: 0000:07:01.1: e1000_probe: (PCI-X:133MHz:64-bit) 00:14:4f:21:19:f9
[   28.090885] e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
[   28.090900] ACPI: PCI Interrupt 0000:08:01.0[A] -> GSI 61 (level, low) -> IRQ 61
[   28.365821] e1000: 0000:08:01.0: e1000_probe: (PCI-X:133MHz:64-bit) 00:14:4f:21:19:fa
[   28.405731] e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
[   28.405747] ACPI: PCI Interrupt 0000:08:01.1[B] -> GSI 62 (level, low) -> IRQ 62
[   28.761212] ata34: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   28.793230] ata34.00: ATA-7: HITACHI HDS7250SASUN500G 0648KP5VXF, K2AOAJ0A, max UDMA/133
[   28.793233] ata34.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   28.793240] ata34.00: max_sectors limited to 256 for NCQ
[   28.817234] ata34.00: max_sectors limited to 256 for NCQ
[   28.817237] ata34.00: configured for UDMA/133
[   28.918110] e1000: 0000:08:01.1: e1000_probe: (PCI-X:133MHz:64-bit) 00:14:4f:21:19:fb
[   28.954548] e1000: eth3: e1000_probe: Intel(R) PRO/1000 Network Connection
[   29.487735] ata35: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   29.527744] ata35.00: ATA-7: HITACHI HDS7250SASUN500G 0647KLP4HF, K2AOAJ0A, max UDMA/133
[   29.527747] ata35.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   29.527754] ata35.00: max_sectors limited to 256 for NCQ
[   29.551748] ata35.00: max_sectors limited to 256 for NCQ
[   29.551750] ata35.00: configured for UDMA/133
[   30.134814] ata36: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   30.166752] ata36.00: ATA-7: HITACHI HDS7250SASUN500G 0647KLJEDG, K2AOAJ0A, max UDMA/133
[   30.166755] ata36.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   30.166762] ata36.00: max_sectors limited to 256 for NCQ
[   30.425899] ata36.00: max_sectors limited to 256 for NCQ
[   30.425902] ata36.00: configured for UDMA/133
[   31.239689] ata37: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   31.271697] ata37.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPTHPF, K2AOAJ0A, max UDMA/133
[   31.271700] ata37.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   31.271708] ata37.00: max_sectors limited to 256 for NCQ
[   31.295702] ata37.00: max_sectors limited to 256 for NCQ
[   31.295704] ata37.00: configured for UDMA/133
[   31.774016] ata38: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   31.805955] ata38.00: ATA-7: HITACHI HDS7250SASUN500G 0648KKZZBH, K2AOAJ0A, max UDMA/133
[   31.805958] ata38.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   31.805965] ata38.00: max_sectors limited to 256 for NCQ
[   31.829900] ata38.00: max_sectors limited to 256 for NCQ
[   31.829902] ata38.00: configured for UDMA/133
[   32.787462] ata39: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   32.827380] ata39.00: ATA-7: HITACHI HDS7250SASUN500G 0647KLLMYF, K2AOAJ0A, max UDMA/133
[   32.827383] ata39.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   32.827390] ata39.00: max_sectors limited to 256 for NCQ
[   32.851326] ata39.00: max_sectors limited to 256 for NCQ
[   32.851328] ata39.00: configured for UDMA/133
[   33.700405] ata40: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   33.732343] ata40.00: ATA-7: HITACHI HDS7250SASUN500G 0648KMGNLF, K2AOAJ0A, max UDMA/133
[   33.732346] ata40.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   33.732353] ata40.00: max_sectors limited to 256 for NCQ
[   33.756289] ata40.00: max_sectors limited to 256 for NCQ
[   33.756291] ata40.00: configured for UDMA/133
[   34.151517] scsi 32:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 33:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 34:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 35:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 36:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 37:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 38:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.151517] scsi 39:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   34.152676] ACPI: PCI Interrupt 0000:0b:01.0[A] -> GSI 76 (level, low) -> IRQ 76
[   34.152804] sata_mv 0000:0b:01.0: Gen-II 32 slots 8 ports SCSI mode IRQ via INTx
[   34.156505] scsi40 : sata_mv
[   34.156505] scsi41 : sata_mv
[   34.156505] scsi42 : sata_mv
[   34.156505] scsi43 : sata_mv
[   34.156505] scsi44 : sata_mv
[   34.156505] scsi45 : sata_mv
[   34.156505] scsi46 : sata_mv
[   34.156505] scsi47 : sata_mv
[   34.156505] ata41: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe522000 irq 76
[   34.156505] ata42: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe524000 irq 76
[   34.156505] ata43: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe526000 irq 76
[   34.156505] ata44: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe528000 irq 76
[   34.156505] ata45: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe532000 irq 76
[   34.156505] ata46: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe534000 irq 76
[   34.156505] ata47: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe536000 irq 76
[   34.156505] ata48: SATA max UDMA/133 mmio m1048576 at 0xfe500000 port 0xfe538000 irq 76
[   34.729538] ata41: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   34.761545] ata41.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPLSYF, K2AOAJ0A, max UDMA/133
[   34.761548] ata41.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   34.761556] ata41.00: max_sectors limited to 256 for NCQ
[   34.785550] ata41.00: max_sectors limited to 256 for NCQ
[   34.785552] ata41.00: configured for UDMA/133
[   35.263864] ata42: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   35.295801] ata42.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPS56F, K2AOAJ0A, max UDMA/133
[   35.295804] ata42.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   35.295811] ata42.00: max_sectors limited to 256 for NCQ
[   35.319746] ata42.00: max_sectors limited to 256 for NCQ
[   35.319748] ata42.00: configured for UDMA/133
[   36.380845] ata43: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   36.412782] ata43.00: ATA-7: HITACHI HDS7250SASUN500G 0648KNMRHH, K2AOAJ0A, max UDMA/133
[   36.412786] ata43.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   36.412793] ata43.00: max_sectors limited to 256 for NCQ
[   36.436728] ata43.00: max_sectors limited to 256 for NCQ
[   36.436730] ata43.00: configured for UDMA/133
[   37.404243] ata44: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   37.544634] ata44.00: ATA-7: HITACHI HDS7250SASUN500G 0648KK4WLF, K2AOAJ0A, max UDMA/133
[   37.544637] ata44.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   37.544644] ata44.00: max_sectors limited to 256 for NCQ
[   37.568577] ata44.00: max_sectors limited to 256 for NCQ
[   37.568580] ata44.00: configured for UDMA/133
[   38.147537] ata45: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   38.179475] ata45.00: ATA-7: HITACHI HDS7250SASUN500G 0648KMLYPF, K2AOAJ0A, max UDMA/133
[   38.179477] ata45.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   38.179484] ata45.00: max_sectors limited to 256 for NCQ
[   38.203421] ata45.00: max_sectors limited to 256 for NCQ
[   38.203423] ata45.00: configured for UDMA/133
[   38.679537] ata46: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   38.711475] ata46.00: ATA-7: HITACHI HDS7250SASUN500G 0648KPU4HF, K2AOAJ0A, max UDMA/133
[   38.711478] ata46.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   38.711486] ata46.00: max_sectors limited to 256 for NCQ
[   38.735420] ata46.00: max_sectors limited to 256 for NCQ
[   38.735423] ata46.00: configured for UDMA/133
[   39.211537] ata47: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   39.243475] ata47.00: ATA-7: HITACHI HDS7250SASUN500G 0649KN20RH, K2AOAV0A, max UDMA/133
[   39.243478] ata47.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   39.243486] ata47.00: max_sectors limited to 256 for NCQ
[   39.267420] ata47.00: max_sectors limited to 256 for NCQ
[   39.267423] ata47.00: configured for UDMA/133
[   40.120811] ata48: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   40.152829] ata48.00: ATA-7: HITACHI HDS7250SASUN500G 0647KN74GH, K2AOAJ0A, max UDMA/133
[   40.152832] ata48.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   40.152839] ata48.00: max_sectors limited to 256 for NCQ
[   40.176834] ata48.00: max_sectors limited to 256 for NCQ
[   40.176836] ata48.00: configured for UDMA/133
[   40.451893] scsi 40:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 41:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 42:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 43:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 44:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 45:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 46:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.451893] scsi 47:0:0:0: Direct-Access     ATA      HITACHI HDS7250S K2AO PQ: 0 ANSI: 5
[   40.587947] Uniform Multi-Platform E-IDE driver
[   40.587947] ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
[   40.596009] Driver 'sd' needs updating - please use bus_type methods
[   40.596130] sd 0:0:0:0: [sda] 976773168 512-byte hardware sectors (500108 MB)
[   40.596147] sd 0:0:0:0: [sda] Write Protect is off
[   40.596150] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   40.596175] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.596243] sd 0:0:0:0: [sda] 976773168 512-byte hardware sectors (500108 MB)
[   40.597264] sd 0:0:0:0: [sda] Write Protect is off
[   40.597266] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   40.597290] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.597294]  sda:<6>AMD8111: 0000:00:07.1 (rev 03) UDMA133 controller
[   40.597522] AMD8111: IDE controller (0x1022:0x7469 rev 0x03) at  PCI slot 0000:00:07.1
[   40.597534] AMD8111: not 100% native mode: will probe irqs later
[   40.597543] AMD8111: IDE port disabled
[   40.597547]     ide0: BM-DMA at 0x1000-0x1007
[   40.597553] Probing IDE interface ide0...
[   40.614723]  unknown partition table
[   40.614723] sd 0:0:0:0: [sda] Attached SCSI disk
[   40.614723] sd 1:0:0:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[   40.614723] sd 1:0:0:0: [sdb] Write Protect is off
[   40.614723] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   40.614723] sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.614723] sd 1:0:0:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[   40.614723] sd 1:0:0:0: [sdb] Write Protect is off
[   40.614723] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   40.614723] sd 1:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.614723]  sdb: unknown partition table
[   40.637952] sd 1:0:0:0: [sdb] Attached SCSI disk
[   40.638048] sd 2:0:0:0: [sdc] 976773168 512-byte hardware sectors (500108 MB)
[   40.638069] sd 2:0:0:0: [sdc] Write Protect is off
[   40.638072] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[   40.638106] sd 2:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.638169] sd 2:0:0:0: [sdc] 976773168 512-byte hardware sectors (500108 MB)
[   40.638187] sd 2:0:0:0: [sdc] Write Protect is off
[   40.638189] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[   40.638222] sd 2:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.638226]  sdc: unknown partition table
[   40.652250] sd 2:0:0:0: [sdc] Attached SCSI disk
[   40.652302] sd 3:0:0:0: [sdd] 976773168 512-byte hardware sectors (500108 MB)
[   40.652302] sd 3:0:0:0: [sdd] Write Protect is off
[   40.652302] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[   40.652302] sd 3:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.652302] sd 3:0:0:0: [sdd] 976773168 512-byte hardware sectors (500108 MB)
[   40.652302] sd 3:0:0:0: [sdd] Write Protect is off
[   40.652302] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[   40.652302] sd 3:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.652302]  sdd: unknown partition table
[   40.665576] sd 3:0:0:0: [sdd] Attached SCSI disk
[   40.665665] sd 4:0:0:0: [sde] 976773168 512-byte hardware sectors (500108 MB)
[   40.665681] sd 4:0:0:0: [sde] Write Protect is off
[   40.665684] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
[   40.665708] sd 4:0:0:0: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.665759] sd 4:0:0:0: [sde] 976773168 512-byte hardware sectors (500108 MB)
[   40.665772] sd 4:0:0:0: [sde] Write Protect is off
[   40.665774] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
[   40.665797] sd 4:0:0:0: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.665800]  sde: unknown partition table
[   40.680675] sd 4:0:0:0: [sde] Attached SCSI disk
[   40.680753] sd 5:0:0:0: [sdf] 976773168 512-byte hardware sectors (500108 MB)
[   40.680773] sd 5:0:0:0: [sdf] Write Protect is off
[   40.680775] sd 5:0:0:0: [sdf] Mode Sense: 00 3a 00 00
[   40.680808] sd 5:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.680866] sd 5:0:0:0: [sdf] 976773168 512-byte hardware sectors (500108 MB)
[   40.680884] sd 5:0:0:0: [sdf] Write Protect is off
[   40.680887] sd 5:0:0:0: [sdf] Mode Sense: 00 3a 00 00
[   40.680920] sd 5:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.680923]  sdf: unknown partition table
[   40.695625] sd 5:0:0:0: [sdf] Attached SCSI disk
[   40.695625] sd 6:0:0:0: [sdg] 976773168 512-byte hardware sectors (500108 MB)
[   40.695625] sd 6:0:0:0: [sdg] Write Protect is off
[   40.695625] sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00
[   40.695625] sd 6:0:0:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.695625] sd 6:0:0:0: [sdg] 976773168 512-byte hardware sectors (500108 MB)
[   40.695625] sd 6:0:0:0: [sdg] Write Protect is off
[   40.695625] sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00
[   40.695625] sd 6:0:0:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.695625]  sdg:<6>usbcore: registered new interface driver hiddev
[   40.722196] input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input1
[   40.724049]  unknown partition table
[   40.724049] sd 6:0:0:0: [sdg] Attached SCSI disk
[   40.724049] sd 7:0:0:0: [sdh] 976773168 512-byte hardware sectors (500108 MB)
[   40.724049] sd 7:0:0:0: [sdh] Write Protect is off
[   40.724049] sd 7:0:0:0: [sdh] Mode Sense: 00 3a 00 00
[   40.724049] sd 7:0:0:0: [sdh] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.724049] sd 7:0:0:0: [sdh] 976773168 512-byte hardware sectors (500108 MB)
[   40.724049] sd 7:0:0:0: [sdh] Write Protect is off
[   40.724049] sd 7:0:0:0: [sdh] Mode Sense: 00 3a 00 00
[   40.724049] sd 7:0:0:0: [sdh] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.724049]  sdh: unknown partition table
[   40.736691] sd 7:0:0:0: [sdh] Attached SCSI disk
[   40.736691] sd 8:0:0:0: [sdi] 976773168 512-byte hardware sectors (500108 MB)
[   40.736691] sd 8:0:0:0: [sdi] Write Protect is off
[   40.736691] sd 8:0:0:0: [sdi] Mode Sense: 00 3a 00 00
[   40.736691] sd 8:0:0:0: [sdi] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.736691] sd 8:0:0:0: [sdi] 976773168 512-byte hardware sectors (500108 MB)
[   40.736691] sd 8:0:0:0: [sdi] Write Protect is off
[   40.736691] sd 8:0:0:0: [sdi] Mode Sense: 00 3a 00 00
[   40.736691] sd 8:0:0:0: [sdi] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.736691]  sdi:<6>input,hidraw0: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:03:00.0-1
[   40.754855] input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input2
[   40.756503]  unknown partition table
[   40.756600] sd 8:0:0:0: [sdi] Attached SCSI disk
[   40.756678] sd 9:0:0:0: [sdj] 976773168 512-byte hardware sectors (500108 MB)
[   40.756693] sd 9:0:0:0: [sdj] Write Protect is off
[   40.756696] sd 9:0:0:0: [sdj] Mode Sense: 00 3a 00 00
[   40.756721] sd 9:0:0:0: [sdj] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.756776] sd 9:0:0:0: [sdj] 976773168 512-byte hardware sectors (500108 MB)
[   40.756789] sd 9:0:0:0: [sdj] Write Protect is off
[   40.756791] sd 9:0:0:0: [sdj] Mode Sense: 00 3a 00 00
[   40.756814] sd 9:0:0:0: [sdj] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.756817]  sdj: unknown partition table
[   40.771867] sd 9:0:0:0: [sdj] Attached SCSI disk
[   40.771921] sd 10:0:0:0: [sdk] 976773168 512-byte hardware sectors (500108 MB)
[   40.771935] sd 10:0:0:0: [sdk] Write Protect is off
[   40.771937] sd 10:0:0:0: [sdk] Mode Sense: 00 3a 00 00
[   40.771960] sd 10:0:0:0: [sdk] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.772003] sd 10:0:0:0: [sdk] 976773168 512-byte hardware sectors (500108 MB)
[   40.772017] sd 10:0:0:0: [sdk] Write Protect is off
[   40.772019] sd 10:0:0:0: [sdk] Mode Sense: 00 3a 00 00
[   40.772042] sd 10:0:0:0: [sdk] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.772045]  sdk:<6>input,hidraw1: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:03:00.0-1
[   40.780863] usbcore: registered new interface driver usbhid
[   40.780866] usbhid: v2.6:USB HID core driver
[   40.790161]  unknown partition table
[   40.790161] sd 10:0:0:0: [sdk] Attached SCSI disk
[   40.790161] sd 11:0:0:0: [sdl] 976773168 512-byte hardware sectors (500108 MB)
[   40.790161] sd 11:0:0:0: [sdl] Write Protect is off
[   40.790161] sd 11:0:0:0: [sdl] Mode Sense: 00 3a 00 00
[   40.790161] sd 11:0:0:0: [sdl] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.790161] sd 11:0:0:0: [sdl] 976773168 512-byte hardware sectors (500108 MB)
[   40.790161] sd 11:0:0:0: [sdl] Write Protect is off
[   40.790161] sd 11:0:0:0: [sdl] Mode Sense: 00 3a 00 00
[   40.790161] sd 11:0:0:0: [sdl] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.790161]  sdl: unknown partition table
[   40.821011] sd 11:0:0:0: [sdl] Attached SCSI disk
[   40.822067] sd 12:0:0:0: [sdm] 976773168 512-byte hardware sectors (500108 MB)
[   40.822067] sd 12:0:0:0: [sdm] Write Protect is off
[   40.822067] sd 12:0:0:0: [sdm] Mode Sense: 00 3a 00 00
[   40.822067] sd 12:0:0:0: [sdm] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.822067] sd 12:0:0:0: [sdm] 976773168 512-byte hardware sectors (500108 MB)
[   40.822067] sd 12:0:0:0: [sdm] Write Protect is off
[   40.822067] sd 12:0:0:0: [sdm] Mode Sense: 00 3a 00 00
[   40.822067] sd 12:0:0:0: [sdm] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.822067]  sdm: unknown partition table
[   40.838825] sd 12:0:0:0: [sdm] Attached SCSI disk
[   40.838825] sd 13:0:0:0: [sdn] 976773168 512-byte hardware sectors (500108 MB)
[   40.838825] sd 13:0:0:0: [sdn] Write Protect is off
[   40.838825] sd 13:0:0:0: [sdn] Mode Sense: 00 3a 00 00
[   40.838825] sd 13:0:0:0: [sdn] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.838825] sd 13:0:0:0: [sdn] 976773168 512-byte hardware sectors (500108 MB)
[   40.838825] sd 13:0:0:0: [sdn] Write Protect is off
[   40.838825] sd 13:0:0:0: [sdn] Mode Sense: 00 3a 00 00
[   40.838825] sd 13:0:0:0: [sdn] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.838825]  sdn: unknown partition table
[   40.855628] sd 13:0:0:0: [sdn] Attached SCSI disk
[   40.855703] sd 14:0:0:0: [sdo] 976773168 512-byte hardware sectors (500108 MB)
[   40.855723] sd 14:0:0:0: [sdo] Write Protect is off
[   40.855725] sd 14:0:0:0: [sdo] Mode Sense: 00 3a 00 00
[   40.855760] sd 14:0:0:0: [sdo] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.855814] sd 14:0:0:0: [sdo] 976773168 512-byte hardware sectors (500108 MB)
[   40.855833] sd 14:0:0:0: [sdo] Write Protect is off
[   40.855835] sd 14:0:0:0: [sdo] Mode Sense: 00 3a 00 00
[   40.855869] sd 14:0:0:0: [sdo] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.855872]  sdo: unknown partition table
[   40.870555] sd 14:0:0:0: [sdo] Attached SCSI disk
[   40.870624] sd 15:0:0:0: [sdp] 976773168 512-byte hardware sectors (500108 MB)
[   40.870643] sd 15:0:0:0: [sdp] Write Protect is off
[   40.870645] sd 15:0:0:0: [sdp] Mode Sense: 00 3a 00 00
[   40.870679] sd 15:0:0:0: [sdp] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.870735] sd 15:0:0:0: [sdp] 976773168 512-byte hardware sectors (500108 MB)
[   40.870754] sd 15:0:0:0: [sdp] Write Protect is off
[   40.870756] sd 15:0:0:0: [sdp] Mode Sense: 00 3a 00 00
[   40.870789] sd 15:0:0:0: [sdp] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.870792]  sdp: unknown partition table
[   40.889217] sd 15:0:0:0: [sdp] Attached SCSI disk
[   40.889290] sd 16:0:0:0: [sdq] 976773168 512-byte hardware sectors (500108 MB)
[   40.889310] sd 16:0:0:0: [sdq] Write Protect is off
[   40.889312] sd 16:0:0:0: [sdq] Mode Sense: 00 3a 00 00
[   40.889346] sd 16:0:0:0: [sdq] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.889402] sd 16:0:0:0: [sdq] 976773168 512-byte hardware sectors (500108 MB)
[   40.889421] sd 16:0:0:0: [sdq] Write Protect is off
[   40.889423] sd 16:0:0:0: [sdq] Mode Sense: 00 3a 00 00
[   40.889457] sd 16:0:0:0: [sdq] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.889460]  sdq: unknown partition table
[   40.902634] sd 16:0:0:0: [sdq] Attached SCSI disk
[   40.902709] sd 17:0:0:0: [sdr] 976773168 512-byte hardware sectors (500108 MB)
[   40.902728] sd 17:0:0:0: [sdr] Write Protect is off
[   40.902730] sd 17:0:0:0: [sdr] Mode Sense: 00 3a 00 00
[   40.902764] sd 17:0:0:0: [sdr] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.902819] sd 17:0:0:0: [sdr] 976773168 512-byte hardware sectors (500108 MB)
[   40.902837] sd 17:0:0:0: [sdr] Write Protect is off
[   40.902840] sd 17:0:0:0: [sdr] Mode Sense: 00 3a 00 00
[   40.902874] sd 17:0:0:0: [sdr] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.902876]  sdr: unknown partition table
[   40.918872] sd 17:0:0:0: [sdr] Attached SCSI disk
[   40.918948] sd 18:0:0:0: [sds] 976773168 512-byte hardware sectors (500108 MB)
[   40.918967] sd 18:0:0:0: [sds] Write Protect is off
[   40.918969] sd 18:0:0:0: [sds] Mode Sense: 00 3a 00 00
[   40.919002] sd 18:0:0:0: [sds] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.919060] sd 18:0:0:0: [sds] 976773168 512-byte hardware sectors (500108 MB)
[   40.919078] sd 18:0:0:0: [sds] Write Protect is off
[   40.919080] sd 18:0:0:0: [sds] Mode Sense: 00 3a 00 00
[   40.919113] sd 18:0:0:0: [sds] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.919116]  sds: unknown partition table
[   40.934264] sd 18:0:0:0: [sds] Attached SCSI disk
[   40.934334] sd 19:0:0:0: [sdt] 976773168 512-byte hardware sectors (500108 MB)
[   40.934353] sd 19:0:0:0: [sdt] Write Protect is off
[   40.934355] sd 19:0:0:0: [sdt] Mode Sense: 00 3a 00 00
[   40.934389] sd 19:0:0:0: [sdt] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.934445] sd 19:0:0:0: [sdt] 976773168 512-byte hardware sectors (500108 MB)
[   40.934463] sd 19:0:0:0: [sdt] Write Protect is off
[   40.934465] sd 19:0:0:0: [sdt] Mode Sense: 00 3a 00 00
[   40.934498] sd 19:0:0:0: [sdt] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.934501]  sdt: unknown partition table
[   40.953916] sd 19:0:0:0: [sdt] Attached SCSI disk
[   40.953987] sd 20:0:0:0: [sdu] 976773168 512-byte hardware sectors (500108 MB)
[   40.954006] sd 20:0:0:0: [sdu] Write Protect is off
[   40.954009] sd 20:0:0:0: [sdu] Mode Sense: 00 3a 00 00
[   40.954043] sd 20:0:0:0: [sdu] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.954097] sd 20:0:0:0: [sdu] 976773168 512-byte hardware sectors (500108 MB)
[   40.954115] sd 20:0:0:0: [sdu] Write Protect is off
[   40.954118] sd 20:0:0:0: [sdu] Mode Sense: 00 3a 00 00
[   40.954151] sd 20:0:0:0: [sdu] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.954154]  sdu: unknown partition table
[   40.971725] sd 20:0:0:0: [sdu] Attached SCSI disk
[   40.971725] sd 21:0:0:0: [sdv] 976773168 512-byte hardware sectors (500108 MB)
[   40.971725] sd 21:0:0:0: [sdv] Write Protect is off
[   40.971725] sd 21:0:0:0: [sdv] Mode Sense: 00 3a 00 00
[   40.971725] sd 21:0:0:0: [sdv] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.971725] sd 21:0:0:0: [sdv] 976773168 512-byte hardware sectors (500108 MB)
[   40.971725] sd 21:0:0:0: [sdv] Write Protect is off
[   40.971725] sd 21:0:0:0: [sdv] Mode Sense: 00 3a 00 00
[   40.971725] sd 21:0:0:0: [sdv] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.971725]  sdv: unknown partition table
[   40.984162] sd 21:0:0:0: [sdv] Attached SCSI disk
[   40.984234] sd 22:0:0:0: [sdw] 976773168 512-byte hardware sectors (500108 MB)
[   40.984253] sd 22:0:0:0: [sdw] Write Protect is off
[   40.984255] sd 22:0:0:0: [sdw] Mode Sense: 00 3a 00 00
[   40.984289] sd 22:0:0:0: [sdw] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.984343] sd 22:0:0:0: [sdw] 976773168 512-byte hardware sectors (500108 MB)
[   40.984362] sd 22:0:0:0: [sdw] Write Protect is off
[   40.984364] sd 22:0:0:0: [sdw] Mode Sense: 00 3a 00 00
[   40.984398] sd 22:0:0:0: [sdw] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   40.984400]  sdw: unknown partition table
[   41.002772] sd 22:0:0:0: [sdw] Attached SCSI disk
[   41.002772] sd 23:0:0:0: [sdx] 976773168 512-byte hardware sectors (500108 MB)
[   41.002772] sd 23:0:0:0: [sdx] Write Protect is off
[   41.002772] sd 23:0:0:0: [sdx] Mode Sense: 00 3a 00 00
[   41.002772] sd 23:0:0:0: [sdx] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.002772] sd 23:0:0:0: [sdx] 976773168 512-byte hardware sectors (500108 MB)
[   41.002772] sd 23:0:0:0: [sdx] Write Protect is off
[   41.002772] sd 23:0:0:0: [sdx] Mode Sense: 00 3a 00 00
[   41.002773] sd 23:0:0:0: [sdx] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.002773]  sdx: unknown partition table
[   41.016990] sd 23:0:0:0: [sdx] Attached SCSI disk
[   41.017063] sd 24:0:0:0: [sdy] 976773168 512-byte hardware sectors (500108 MB)
[   41.017082] sd 24:0:0:0: [sdy] Write Protect is off
[   41.017084] sd 24:0:0:0: [sdy] Mode Sense: 00 3a 00 00
[   41.017117] sd 24:0:0:0: [sdy] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.017169] sd 24:0:0:0: [sdy] 976773168 512-byte hardware sectors (500108 MB)
[   41.017187] sd 24:0:0:0: [sdy] Write Protect is off
[   41.017189] sd 24:0:0:0: [sdy] Mode Sense: 00 3a 00 00
[   41.017222] sd 24:0:0:0: [sdy] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.017225]  sdy: sdy1 sdy2
[   41.027003] sd 24:0:0:0: [sdy] Attached SCSI disk
[   41.027079] sd 25:0:0:0: [sdz] 976773168 512-byte hardware sectors (500108 MB)
[   41.027098] sd 25:0:0:0: [sdz] Write Protect is off
[   41.027100] sd 25:0:0:0: [sdz] Mode Sense: 00 3a 00 00
[   41.027133] sd 25:0:0:0: [sdz] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.027187] sd 25:0:0:0: [sdz] 976773168 512-byte hardware sectors (500108 MB)
[   41.027204] sd 25:0:0:0: [sdz] Write Protect is off
[   41.027206] sd 25:0:0:0: [sdz] Mode Sense: 00 3a 00 00
[   41.027239] sd 25:0:0:0: [sdz] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.027241]  sdz: unknown partition table
[   41.050772] sd 25:0:0:0: [sdz] Attached SCSI disk
[   41.050772] sd 26:0:0:0: [sdaa] 976773168 512-byte hardware sectors (500108 MB)
[   41.050772] sd 26:0:0:0: [sdaa] Write Protect is off
[   41.050772] sd 26:0:0:0: [sdaa] Mode Sense: 00 3a 00 00
[   41.050772] sd 26:0:0:0: [sdaa] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.050772] sd 26:0:0:0: [sdaa] 976773168 512-byte hardware sectors (500108 MB)
[   41.050772] sd 26:0:0:0: [sdaa] Write Protect is off
[   41.050772] sd 26:0:0:0: [sdaa] Mode Sense: 00 3a 00 00
[   41.050772] sd 26:0:0:0: [sdaa] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.050772]  sdaa: unknown partition table
[   41.063084] sd 26:0:0:0: [sdaa] Attached SCSI disk
[   41.063160] sd 27:0:0:0: [sdab] 976773168 512-byte hardware sectors (500108 MB)
[   41.063179] sd 27:0:0:0: [sdab] Write Protect is off
[   41.063181] sd 27:0:0:0: [sdab] Mode Sense: 00 3a 00 00
[   41.063215] sd 27:0:0:0: [sdab] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.063272] sd 27:0:0:0: [sdab] 976773168 512-byte hardware sectors (500108 MB)
[   41.063290] sd 27:0:0:0: [sdab] Write Protect is off
[   41.063292] sd 27:0:0:0: [sdab] Mode Sense: 00 3a 00 00
[   41.063325] sd 27:0:0:0: [sdab] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.063328]  sdab: unknown partition table
[   41.085728] sd 27:0:0:0: [sdab] Attached SCSI disk
[   41.085802] sd 28:0:0:0: [sdac] 976773168 512-byte hardware sectors (500108 MB)
[   41.085820] sd 28:0:0:0: [sdac] Write Protect is off
[   41.085823] sd 28:0:0:0: [sdac] Mode Sense: 00 3a 00 00
[   41.085857] sd 28:0:0:0: [sdac] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.085910] sd 28:0:0:0: [sdac] 976773168 512-byte hardware sectors (500108 MB)
[   41.085928] sd 28:0:0:0: [sdac] Write Protect is off
[   41.085931] sd 28:0:0:0: [sdac] Mode Sense: 00 3a 00 00
[   41.085964] sd 28:0:0:0: [sdac] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.085967]  sdac: sdac1 sdac2
[   41.095545] sd 28:0:0:0: [sdac] Attached SCSI disk
[   41.095619] sd 29:0:0:0: [sdad] 976773168 512-byte hardware sectors (500108 MB)
[   41.095638] sd 29:0:0:0: [sdad] Write Protect is off
[   41.095640] sd 29:0:0:0: [sdad] Mode Sense: 00 3a 00 00
[   41.095674] sd 29:0:0:0: [sdad] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.095728] sd 29:0:0:0: [sdad] 976773168 512-byte hardware sectors (500108 MB)
[   41.095746] sd 29:0:0:0: [sdad] Write Protect is off
[   41.095748] sd 29:0:0:0: [sdad] Mode Sense: 00 3a 00 00
[   41.095781] sd 29:0:0:0: [sdad] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.095783]  sdad: unknown partition table
[   41.113856] sd 29:0:0:0: [sdad] Attached SCSI disk
[   41.113856] sd 30:0:0:0: [sdae] 976773168 512-byte hardware sectors (500108 MB)
[   41.113856] sd 30:0:0:0: [sdae] Write Protect is off
[   41.113856] sd 30:0:0:0: [sdae] Mode Sense: 00 3a 00 00
[   41.113856] sd 30:0:0:0: [sdae] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.113900] sd 30:0:0:0: [sdae] 976773168 512-byte hardware sectors (500108 MB)
[   41.113918] sd 30:0:0:0: [sdae] Write Protect is off
[   41.113920] sd 30:0:0:0: [sdae] Mode Sense: 00 3a 00 00
[   41.113954] sd 30:0:0:0: [sdae] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.113957]  sdae: unknown partition table
[   41.128578] sd 30:0:0:0: [sdae] Attached SCSI disk
[   41.128650] sd 31:0:0:0: [sdaf] 976773168 512-byte hardware sectors (500108 MB)
[   41.128670] sd 31:0:0:0: [sdaf] Write Protect is off
[   41.128672] sd 31:0:0:0: [sdaf] Mode Sense: 00 3a 00 00
[   41.128706] sd 31:0:0:0: [sdaf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.128762] sd 31:0:0:0: [sdaf] 976773168 512-byte hardware sectors (500108 MB)
[   41.128781] sd 31:0:0:0: [sdaf] Write Protect is off
[   41.128783] sd 31:0:0:0: [sdaf] Mode Sense: 00 3a 00 00
[   41.128816] sd 31:0:0:0: [sdaf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.128819]  sdaf: unknown partition table
[   41.140771] sd 31:0:0:0: [sdaf] Attached SCSI disk
[   41.140848] sd 32:0:0:0: [sdag] 976773168 512-byte hardware sectors (500108 MB)
[   41.140867] sd 32:0:0:0: [sdag] Write Protect is off
[   41.140869] sd 32:0:0:0: [sdag] Mode Sense: 00 3a 00 00
[   41.140902] sd 32:0:0:0: [sdag] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.140956] sd 32:0:0:0: [sdag] 976773168 512-byte hardware sectors (500108 MB)
[   41.140975] sd 32:0:0:0: [sdag] Write Protect is off
[   41.140977] sd 32:0:0:0: [sdag] Mode Sense: 00 3a 00 00
[   41.141010] sd 32:0:0:0: [sdag] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.141013]  sdag: unknown partition table
[   41.157650] sd 32:0:0:0: [sdag] Attached SCSI disk
[   41.157650] sd 33:0:0:0: [sdah] 976773168 512-byte hardware sectors (500108 MB)
[   41.157650] sd 33:0:0:0: [sdah] Write Protect is off
[   41.157650] sd 33:0:0:0: [sdah] Mode Sense: 00 3a 00 00
[   41.157650] sd 33:0:0:0: [sdah] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.157650] sd 33:0:0:0: [sdah] 976773168 512-byte hardware sectors (500108 MB)
[   41.157650] sd 33:0:0:0: [sdah] Write Protect is off
[   41.157650] sd 33:0:0:0: [sdah] Mode Sense: 00 3a 00 00
[   41.157650] sd 33:0:0:0: [sdah] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.157650]  sdah: unknown partition table
[   41.179375] sd 33:0:0:0: [sdah] Attached SCSI disk
[   41.179375] sd 34:0:0:0: [sdai] 976773168 512-byte hardware sectors (500108 MB)
[   41.179375] sd 34:0:0:0: [sdai] Write Protect is off
[   41.179375] sd 34:0:0:0: [sdai] Mode Sense: 00 3a 00 00
[   41.179375] sd 34:0:0:0: [sdai] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.179375] sd 34:0:0:0: [sdai] 976773168 512-byte hardware sectors (500108 MB)
[   41.179375] sd 34:0:0:0: [sdai] Write Protect is off
[   41.179375] sd 34:0:0:0: [sdai] Mode Sense: 00 3a 00 00
[   41.179375] sd 34:0:0:0: [sdai] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.179375]  sdai: unknown partition table
[   41.191524] sd 34:0:0:0: [sdai] Attached SCSI disk
[   41.191524] sd 35:0:0:0: [sdaj] 976773168 512-byte hardware sectors (500108 MB)
[   41.191524] sd 35:0:0:0: [sdaj] Write Protect is off
[   41.191524] sd 35:0:0:0: [sdaj] Mode Sense: 00 3a 00 00
[   41.191524] sd 35:0:0:0: [sdaj] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.191524] sd 35:0:0:0: [sdaj] 976773168 512-byte hardware sectors (500108 MB)
[   41.191524] sd 35:0:0:0: [sdaj] Write Protect is off
[   41.191524] sd 35:0:0:0: [sdaj] Mode Sense: 00 3a 00 00
[   41.191524] sd 35:0:0:0: [sdaj] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.191524]  sdaj:ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
[   41.209428]  unknown partition table
[   41.209428] sd 35:0:0:0: [sdaj] Attached SCSI disk
[   41.209428] sd 36:0:0:0: [sdak] 976773168 512-byte hardware sectors (500108 MB)
[   41.209428] sd 36:0:0:0: [sdak] Write Protect is off
[   41.209428] sd 36:0:0:0: [sdak] Mode Sense: 00 3a 00 00
[   41.209428] sd 36:0:0:0: [sdak] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.209428] sd 36:0:0:0: [sdak] 976773168 512-byte hardware sectors (500108 MB)
[   41.209428] sd 36:0:0:0: [sdak] Write Protect is off
[   41.209428] sd 36:0:0:0: [sdak] Mode Sense: 00 3a 00 00
[   41.209428] sd 36:0:0:0: [sdak] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.209428]  sdak: unknown partition table
[   41.225428] sd 36:0:0:0: [sdak] Attached SCSI disk
[   41.225428] sd 37:0:0:0: [sdal] 976773168 512-byte hardware sectors (500108 MB)
[   41.225428] sd 37:0:0:0: [sdal] Write Protect is off
[   41.225428] sd 37:0:0:0: [sdal] Mode Sense: 00 3a 00 00
[   41.225428] sd 37:0:0:0: [sdal] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.225428] sd 37:0:0:0: [sdal] 976773168 512-byte hardware sectors (500108 MB)
[   41.225428] sd 37:0:0:0: [sdal] Write Protect is off
[   41.225428] sd 37:0:0:0: [sdal] Mode Sense: 00 3a 00 00
[   41.225428] sd 37:0:0:0: [sdal] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.225428]  sdal: unknown partition table
[   41.241428] sd 37:0:0:0: [sdal] Attached SCSI disk
[   41.241428] sd 38:0:0:0: [sdam] 976773168 512-byte hardware sectors (500108 MB)
[   41.241428] sd 38:0:0:0: [sdam] Write Protect is off
[   41.241428] sd 38:0:0:0: [sdam] Mode Sense: 00 3a 00 00
[   41.241428] sd 38:0:0:0: [sdam] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.241428] sd 38:0:0:0: [sdam] 976773168 512-byte hardware sectors (500108 MB)
[   41.241428] sd 38:0:0:0: [sdam] Write Protect is off
[   41.241428] sd 38:0:0:0: [sdam] Mode Sense: 00 3a 00 00
[   41.241428] sd 38:0:0:0: [sdam] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.241428]  sdam: unknown partition table
[   41.260541] sd 38:0:0:0: [sdam] Attached SCSI disk
[   41.260541] sd 39:0:0:0: [sdan] 976773168 512-byte hardware sectors (500108 MB)
[   41.260541] sd 39:0:0:0: [sdan] Write Protect is off
[   41.260541] sd 39:0:0:0: [sdan] Mode Sense: 00 3a 00 00
[   41.260541] sd 39:0:0:0: [sdan] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.260541] sd 39:0:0:0: [sdan] 976773168 512-byte hardware sectors (500108 MB)
[   41.260541] sd 39:0:0:0: [sdan] Write Protect is off
[   41.260541] sd 39:0:0:0: [sdan] Mode Sense: 00 3a 00 00
[   41.260541] sd 39:0:0:0: [sdan] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.260541]  sdan: unknown partition table
[   41.272540] sd 39:0:0:0: [sdan] Attached SCSI disk
[   41.272540] sd 40:0:0:0: [sdao] 976773168 512-byte hardware sectors (500108 MB)
[   41.272540] sd 40:0:0:0: [sdao] Write Protect is off
[   41.272540] sd 40:0:0:0: [sdao] Mode Sense: 00 3a 00 00
[   41.272540] sd 40:0:0:0: [sdao] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.272541] sd 40:0:0:0: [sdao] 976773168 512-byte hardware sectors (500108 MB)
[   41.272541] sd 40:0:0:0: [sdao] Write Protect is off
[   41.272541] sd 40:0:0:0: [sdao] Mode Sense: 00 3a 00 00
[   41.272541] sd 40:0:0:0: [sdao] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.272541]  sdao: unknown partition table
[   41.283966] sd 40:0:0:0: [sdao] Attached SCSI disk
[   41.284023] sd 41:0:0:0: [sdap] 976773168 512-byte hardware sectors (500108 MB)
[   41.284037] sd 41:0:0:0: [sdap] Write Protect is off
[   41.284039] sd 41:0:0:0: [sdap] Mode Sense: 00 3a 00 00
[   41.284063] sd 41:0:0:0: [sdap] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.284106] sd 41:0:0:0: [sdap] 976773168 512-byte hardware sectors (500108 MB)
[   41.284119] sd 41:0:0:0: [sdap] Write Protect is off
[   41.284121] sd 41:0:0:0: [sdap] Mode Sense: 00 3a 00 00
[   41.284145] sd 41:0:0:0: [sdap] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.284147]  sdap: unknown partition table
[   41.300541] sd 41:0:0:0: [sdap] Attached SCSI disk
[   41.300541] sd 42:0:0:0: [sdaq] 976773168 512-byte hardware sectors (500108 MB)
[   41.300541] sd 42:0:0:0: [sdaq] Write Protect is off
[   41.300541] sd 42:0:0:0: [sdaq] Mode Sense: 00 3a 00 00
[   41.300541] sd 42:0:0:0: [sdaq] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.300541] sd 42:0:0:0: [sdaq] 976773168 512-byte hardware sectors (500108 MB)
[   41.300541] sd 42:0:0:0: [sdaq] Write Protect is off
[   41.300541] sd 42:0:0:0: [sdaq] Mode Sense: 00 3a 00 00
[   41.300541] sd 42:0:0:0: [sdaq] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.300541]  sdaq: unknown partition table
[   41.317568] sd 42:0:0:0: [sdaq] Attached SCSI disk
[   41.317568] sd 43:0:0:0: [sdar] 976773168 512-byte hardware sectors (500108 MB)
[   41.317568] sd 43:0:0:0: [sdar] Write Protect is off
[   41.317568] sd 43:0:0:0: [sdar] Mode Sense: 00 3a 00 00
[   41.317568] sd 43:0:0:0: [sdar] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.317568] sd 43:0:0:0: [sdar] 976773168 512-byte hardware sectors (500108 MB)
[   41.317568] sd 43:0:0:0: [sdar] Write Protect is off
[   41.317568] sd 43:0:0:0: [sdar] Mode Sense: 00 3a 00 00
[   41.317568] sd 43:0:0:0: [sdar] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.317568]  sdar: unknown partition table
[   41.333569] sd 43:0:0:0: [sdar] Attached SCSI disk
[   41.333569] sd 44:0:0:0: [sdas] 976773168 512-byte hardware sectors (500108 MB)
[   41.333569] sd 44:0:0:0: [sdas] Write Protect is off
[   41.333569] sd 44:0:0:0: [sdas] Mode Sense: 00 3a 00 00
[   41.333569] sd 44:0:0:0: [sdas] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.333569] sd 44:0:0:0: [sdas] 976773168 512-byte hardware sectors (500108 MB)
[   41.333569] sd 44:0:0:0: [sdas] Write Protect is off
[   41.333569] sd 44:0:0:0: [sdas] Mode Sense: 00 3a 00 00
[   41.333569] sd 44:0:0:0: [sdas] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.333569]  sdas: unknown partition table
[   41.346492] sd 44:0:0:0: [sdas] Attached SCSI disk
[   41.346492] sd 45:0:0:0: [sdat] 976773168 512-byte hardware sectors (500108 MB)
[   41.346492] sd 45:0:0:0: [sdat] Write Protect is off
[   41.346492] sd 45:0:0:0: [sdat] Mode Sense: 00 3a 00 00
[   41.346492] sd 45:0:0:0: [sdat] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.346492] sd 45:0:0:0: [sdat] 976773168 512-byte hardware sectors (500108 MB)
[   41.346492] sd 45:0:0:0: [sdat] Write Protect is off
[   41.346492] sd 45:0:0:0: [sdat] Mode Sense: 00 3a 00 00
[   41.346492] sd 45:0:0:0: [sdat] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.346492]  sdat: unknown partition table
[   41.364552] sd 45:0:0:0: [sdat] Attached SCSI disk
[   41.364552] sd 46:0:0:0: [sdau] 976773168 512-byte hardware sectors (500108 MB)
[   41.364552] sd 46:0:0:0: [sdau] Write Protect is off
[   41.364552] sd 46:0:0:0: [sdau] Mode Sense: 00 3a 00 00
[   41.364552] sd 46:0:0:0: [sdau] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.364552] sd 46:0:0:0: [sdau] 976773168 512-byte hardware sectors (500108 MB)
[   41.364552] sd 46:0:0:0: [sdau] Write Protect is off
[   41.364552] sd 46:0:0:0: [sdau] Mode Sense: 00 3a 00 00
[   41.364552] sd 46:0:0:0: [sdau] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.364552]  sdau: unknown partition table
[   41.376616] sd 46:0:0:0: [sdau] Attached SCSI disk
[   41.376616] sd 47:0:0:0: [sdav] 976773168 512-byte hardware sectors (500108 MB)
[   41.376617] sd 47:0:0:0: [sdav] Write Protect is off
[   41.376617] sd 47:0:0:0: [sdav] Mode Sense: 00 3a 00 00
[   41.376617] sd 47:0:0:0: [sdav] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.376617] sd 47:0:0:0: [sdav] 976773168 512-byte hardware sectors (500108 MB)
[   41.376617] sd 47:0:0:0: [sdav] Write Protect is off
[   41.376617] sd 47:0:0:0: [sdav] Mode Sense: 00 3a 00 00
[   41.376617] sd 47:0:0:0: [sdav] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[   41.376617]  sdav: unknown partition table
[   41.397513] sd 47:0:0:0: [sdav] Attached SCSI disk
[   41.690918] md: raid0 personality registered for level 0
[   41.694858] md: raid1 personality registered for level 1
[   41.698827] xor: automatically using best checksumming function: generic_sse
[   41.717553]    generic_sse:  8305.000 MB/sec
[   41.717553] xor: using function: generic_sse (8305.000 MB/sec)
[   41.717553] async_tx: api initialized (async)
[   41.791254] raid6: int64x1   2362 MB/s
[   41.859254] raid6: int64x2   3225 MB/s
[   41.927254] raid6: int64x4   3101 MB/s
[   41.996232] raid6: int64x8   2122 MB/s
[   42.064232] raid6: sse2x1    3066 MB/s
[   42.132232] raid6: sse2x2    4207 MB/s
[   42.200232] raid6: sse2x4    4609 MB/s
[   42.200232] raid6: using algorithm sse2x4 (4609 MB/s)
[   42.200232] md: raid6 personality registered for level 6
[   42.200232] md: raid5 personality registered for level 5
[   42.200232] md: raid4 personality registered for level 4
[   42.202959] md: md0 stopped.
[   42.215329] md: bind<sdac1>
[   42.215329] md: bind<sdy1>
[   42.224176] raid1: raid set md0 active with 2 out of 2 mirrors
[   42.224176] md: md1 stopped.
[   42.235126] md: bind<sdac2>
[   42.235259] md: bind<sdy2>
[   42.245065] raid1: raid set md1 active with 2 out of 2 mirrors
[   42.245065] md: md2 stopped.
[   42.255758] md: bind<sdx>
[   42.255758] md: bind<sdav>
[   42.255758] md: bind<sdan>
[   42.255758] md: bind<sdp>
[   42.255843] md: bind<sdh>
[   42.255843] md: bind<sdu>
[   42.255843] md: bind<sdas>
[   42.255843] md: bind<sdak>
[   42.255843] md: bind<sde>
[   42.255843] md: bind<sdt>
[   42.255843] md: bind<sdaa>
[   42.255843] md: bind<sdaf>
[   42.263771] raid5: device sdaf operational as raid disk 0
[   42.263771] raid5: device sde operational as raid disk 9
[   42.263771] raid5: device sdak operational as raid disk 8
[   42.263771] raid5: device sdas operational as raid disk 7
[   42.263771] raid5: device sdu operational as raid disk 6
[   42.263771] raid5: device sdh operational as raid disk 5
[   42.263771] raid5: device sdp operational as raid disk 4
[   42.263771] raid5: device sdan operational as raid disk 3
[   42.263771] raid5: device sdav operational as raid disk 2
[   42.263771] raid5: device sdx operational as raid disk 1
[   42.263771] raid5: allocated 10574kB for md2
[   42.263771] raid5: raid level 6 set md2 active with 10 out of 10 devices, algorithm 2
[   42.263771] RAID5 conf printout:
[   42.263771]  --- rd:10 wd:10
[   42.263771]  disk 0, o:1, dev:sdaf
[   42.263771]  disk 1, o:1, dev:sdx
[   42.263771]  disk 2, o:1, dev:sdav
[   42.263771]  disk 3, o:1, dev:sdan
[   42.263771]  disk 4, o:1, dev:sdp
[   42.263771]  disk 5, o:1, dev:sdh
[   42.263771]  disk 6, o:1, dev:sdu
[   42.263771]  disk 7, o:1, dev:sdas
[   42.263771]  disk 8, o:1, dev:sdak
[   42.263771]  disk 9, o:1, dev:sde
[   42.263771] md: md3 stopped.
[   42.280938] md: bind<sdar>
[   42.280938] md: bind<sdaj>
[   42.280938] md: bind<sdl>
[   42.280938] md: bind<sdd>
[   42.280938] md: bind<sdq>
[   42.280938] md: bind<sdao>
[   42.280938] md: bind<sdag>
[   42.282001] md: bind<sdi>
[   42.282001] md: bind<sda>
[   42.282001] md: bind<sdaq>
[   42.282001] md: bind<sdz>
[   42.282001] md: bind<sdab>
[   42.289461] raid5: device sdab operational as raid disk 0
[   42.289461] raid5: device sda operational as raid disk 9
[   42.289461] raid5: device sdi operational as raid disk 8
[   42.289461] raid5: device sdag operational as raid disk 7
[   42.289461] raid5: device sdao operational as raid disk 6
[   42.289461] raid5: device sdq operational as raid disk 5
[   42.289461] raid5: device sdd operational as raid disk 4
[   42.289461] raid5: device sdl operational as raid disk 3
[   42.289461] raid5: device sdaj operational as raid disk 2
[   42.289461] raid5: device sdar operational as raid disk 1
[   42.289980] raid5: allocated 10574kB for md3
[   42.289982] raid5: raid level 6 set md3 active with 10 out of 10 devices, algorithm 2
[   42.289984] RAID5 conf printout:
[   42.289986]  --- rd:10 wd:10
[   42.289987]  disk 0, o:1, dev:sdab
[   42.289989]  disk 1, o:1, dev:sdar
[   42.289990]  disk 2, o:1, dev:sdaj
[   42.289992]  disk 3, o:1, dev:sdl
[   42.289993]  disk 4, o:1, dev:sdd
[   42.289995]  disk 5, o:1, dev:sdq
[   42.289996]  disk 6, o:1, dev:sdao
[   42.289998]  disk 7, o:1, dev:sdag
[   42.289999]  disk 8, o:1, dev:sdi
[   42.290000]  disk 9, o:1, dev:sda
[   42.290238] md: md4 stopped.
[   42.298857] md: bind<sds>
[   42.298857] md: bind<sdw>
[   42.300905] md: bind<sdau>
[   42.302853] md: bind<sdai>
[   42.302853] md: bind<sdam>
[   42.302853] md: bind<sdk>
[   42.302853] md: bind<sdo>
[   42.302853] md: bind<sdc>
[   42.302853] md: bind<sdg>
[   42.302853] md: bind<sdah>
[   42.302853] md: bind<sdae>
[   42.311762] raid5: device sdae operational as raid disk 0
[   42.311766] raid5: device sdg operational as raid disk 9
[   42.311768] raid5: device sdc operational as raid disk 8
[   42.311770] raid5: device sdo operational as raid disk 7
[   42.311771] raid5: device sdk operational as raid disk 6
[   42.311773] raid5: device sdam operational as raid disk 5
[   42.311775] raid5: device sdai operational as raid disk 4
[   42.311777] raid5: device sdau operational as raid disk 3
[   42.311779] raid5: device sdw operational as raid disk 2
[   42.311780] raid5: device sds operational as raid disk 1
[   42.311898] raid5: allocated 10574kB for md4
[   42.311898] raid5: raid level 6 set md4 active with 10 out of 10 devices, algorithm 2
[   42.311898] RAID5 conf printout:
[   42.311898]  --- rd:10 wd:10
[   42.311898]  disk 0, o:1, dev:sdae
[   42.311898]  disk 1, o:1, dev:sds
[   42.311898]  disk 2, o:1, dev:sdw
[   42.311898]  disk 3, o:1, dev:sdau
[   42.311898]  disk 4, o:1, dev:sdai
[   42.311898]  disk 5, o:1, dev:sdam
[   42.311898]  disk 6, o:1, dev:sdk
[   42.311898]  disk 7, o:1, dev:sdo
[   42.311898]  disk 8, o:1, dev:sdc
[   42.311898]  disk 9, o:1, dev:sdg
[   42.311898] md: md5 stopped.
[   42.319899] md: bind<sdr>
[   42.319899] md: bind<sdv>
[   42.319899] md: bind<sdap>
[   42.319899] md: bind<sdat>
[   42.319899] md: bind<sdal>
[   42.319899] md: bind<sdj>
[   42.319899] md: bind<sdn>
[   42.319899] md: bind<sdb>
[   42.320004] md: bind<sdf>
[   42.320142] md: bind<sdm>
[   42.320259] md: bind<sdad>
[   42.328236] raid5: device sdad operational as raid disk 0
[   42.328236] raid5: device sdf operational as raid disk 9
[   42.328236] raid5: device sdb operational as raid disk 8
[   42.328236] raid5: device sdn operational as raid disk 7
[   42.328236] raid5: device sdj operational as raid disk 6
[   42.328236] raid5: device sdal operational as raid disk 5
[   42.328236] raid5: device sdat operational as raid disk 4
[   42.328236] raid5: device sdap operational as raid disk 3
[   42.328236] raid5: device sdv operational as raid disk 2
[   42.328236] raid5: device sdr operational as raid disk 1
[   42.328898] raid5: allocated 10574kB for md5
[   42.328900] raid5: raid level 6 set md5 active with 10 out of 10 devices, algorithm 2
[   42.328903] RAID5 conf printout:
[   42.328904]  --- rd:10 wd:10
[   42.328906]  disk 0, o:1, dev:sdad
[   42.328907]  disk 1, o:1, dev:sdr
[   42.328909]  disk 2, o:1, dev:sdv
[   42.328910]  disk 3, o:1, dev:sdap
[   42.328912]  disk 4, o:1, dev:sdat
[   42.328914]  disk 5, o:1, dev:sdal
[   42.328915]  disk 6, o:1, dev:sdj
[   42.328917]  disk 7, o:1, dev:sdn
[   42.328918]  disk 8, o:1, dev:sdb
[   42.328920]  disk 9, o:1, dev:sdf
[   42.331005] md: md6 stopped.
[   42.333395] md: bind<md3>
[   42.333522] md: bind<md4>
[   42.333636] md: bind<md5>
[   42.333757] md: bind<md2>
[   42.343643] md6: setting max_sectors to 128, segment boundary to 32767
[   42.343646] raid0: looking at md2
[   42.343648] raid0:   comparing md2(3907091904) with md2(3907091904)
[   42.343650] raid0:   END
[   42.343651] raid0:   ==> UNIQUE
[   42.343652] raid0: 1 zones
[   42.343653] raid0: looking at md5
[   42.343654] raid0:   comparing md5(3907091904) with md2(3907091904)
[   42.343656] raid0:   EQUAL
[   42.343658] raid0: looking at md4
[   42.343659] raid0:   comparing md4(3907091904) with md2(3907091904)
[   42.343661] raid0:   EQUAL
[   42.343662] raid0: looking at md3
[   42.343664] raid0:   comparing md3(3907091904) with md2(3907091904)
[   42.343666] raid0:   EQUAL
[   42.343667] raid0: FINAL 1 zones
[   42.343669] raid0: done.
[   42.343671] raid0 : md_size is 15628367616 blocks.
[   42.343672] raid0 : conf->hash_spacing is 15628367616 blocks.
[   42.343674] raid0 : nb_zone is 2.
[   42.343675] raid0 : Allocating 16 bytes for hash.
[   42.405967] device-mapper: uevent: version 1.0.3
[   42.405967] device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel at redhat.com
[   42.529020] PM: Starting manual resume from disk
[   42.575814] kjournald starting.  Commit interval 5 seconds
[   42.575814] EXT3-fs: mounted filesystem with ordered data mode.
[   43.898887] udevd version 125 started
[   44.700705] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[   44.713033] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   44.713033] shpchp: shpc_init: cannot reserve MMIO region
[   44.782957] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   44.782959] shpchp: shpc_init: cannot reserve MMIO region
[   44.847543] shpchp: HPC vendor_id 1022 device_id 7460 ss_vid 0 ss_did 0
[   44.847545] shpchp: shpc_init: cannot reserve MMIO region
[   44.912135] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   44.912137] shpchp: shpc_init: cannot reserve MMIO region
[   44.976718] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   44.976720] shpchp: shpc_init: cannot reserve MMIO region
[   45.044722] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   45.044724] shpchp: shpc_init: cannot reserve MMIO region
[   45.109141] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   45.109143] shpchp: shpc_init: cannot reserve MMIO region
[   45.175087] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   45.175094] shpchp: shpc_init: cannot reserve MMIO region
[   45.242131] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   45.242131] shpchp: shpc_init: cannot reserve MMIO region
[   45.285103] input: Power Button (FF) as /class/input/input3
[   45.345377] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   45.345379] shpchp: shpc_init: cannot reserve MMIO region
[   45.409993] shpchp: HPC vendor_id 1022 device_id 7458 ss_vid 0 ss_did 0
[   45.409996] shpchp: shpc_init: cannot reserve MMIO region
[   45.474631] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[   45.527086] ACPI: Power Button (FF) [PWRF]
[   45.527179] input: Power Button (CM) as /class/input/input4
[   45.601111] ACPI: Power Button (CM) [PWRB]
[   45.603765] AMD768 RNG detected
[   45.731030] input: PC Speaker as /class/input/input5
[   45.883243] Error: Driver 'pcspkr' is already registered, aborting...
[   46.957695] EXT3 FS on dm-0, internal journal
[   47.897513] loop: module loaded
[   48.824634] kjournald starting.  Commit interval 5 seconds
[   48.836634] EXT3 FS on md0, internal journal
[   48.836634] EXT3-fs: mounted filesystem with ordered data mode.
[   48.884924] kjournald starting.  Commit interval 5 seconds
[   48.888924] EXT3 FS on dm-1, internal journal
[   48.888924] EXT3-fs: mounted filesystem with ordered data mode.
[   48.925400] kjournald starting.  Commit interval 5 seconds
[   48.937872] EXT3 FS on dm-2, internal journal
[   48.937872] EXT3-fs: mounted filesystem with ordered data mode.
[   48.979065] kjournald starting.  Commit interval 5 seconds
[   48.992284] EXT3 FS on dm-3, internal journal
[   48.992284] EXT3-fs: mounted filesystem with ordered data mode.
[   49.017474] Adding 3903480k swap on /dev/mapper/vg0-swap.  Priority:-1 extents:1 across:3903480k
[   49.565391] ip_tables: (C) 2000-2006 Netfilter Core Team
[   49.671327] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[   49.937235] Ethernet Channel Bonding Driver: v3.2.5 (March 21, 2008)
[   49.937235] bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
[   49.937739] bonding: bond0: setting mode to 802.3ad (4).
[   49.937739] bonding: bond0: Setting MII monitoring interval to 100.
[   49.937739] bonding: bond0: Setting up delay to 200.
[   49.937739] bonding: bond0: Setting down delay to 200.
[   49.978685] bonding: bond0: enslaving eth0 as a backup interface with a down link.
[   49.980917] e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   50.014686] bonding: bond0: enslaving eth1 as a backup interface with a down link.
[   50.017352] e1000: eth1: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   50.043091] bonding: bond0: link status up for interface eth0, enabling it in 200 ms.
[   50.043097] bonding: bond0: link status up for interface eth1, enabling it in 200 ms.
[   50.160966] bonding: bond1 is being created...
[   50.160966] bonding: bond1: setting mode to 802.3ad (4).
[   50.160966] bonding: bond1: Setting MII monitoring interval to 100.
[   50.160966] bonding: bond1: Setting up delay to 200.
[   50.160966] bonding: bond1: Setting down delay to 200.
[   50.199147] bonding: bond1: enslaving eth2 as a backup interface with a down link.
[   50.199522] e1000: eth2: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   50.240928] bonding: bond1: enslaving eth3 as a backup interface with a down link.
[   50.241298] e1000: eth3: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   50.254468] bonding: bond0: link status definitely up for interface eth0.
[   50.254475] bonding: bond0: link status definitely up for interface eth1.
[   50.266433] bonding: bond1: link status up for interface eth2, enabling it in 200 ms.
[   50.266439] bonding: bond1: link status up for interface eth3, enabling it in 200 ms.
[   50.361893] NET: Registered protocol family 10
[   50.363153] lo: Disabled Privacy Extensions
[   50.366133] ADDRCONF(NETDEV_UP): bond0: link is not ready
[   50.366798] ADDRCONF(NETDEV_UP): bond1: link is not ready
[   50.428789] drbd: initialised. Version: 8.3.2 (api:88/proto:86-90)
[   50.428789] drbd: GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by phil at fat-tyre, 2009-07-03 15:35:39
[   50.428789] drbd: registered as block device major 147
[   50.428789] drbd: minor_table @ 0xffff8103f94a53c0
[   50.445230] block drbd0: Starting worker thread (from cqueue [4578])
[   50.445230] block drbd0: disk( Diskless -> Attaching ) 
[   50.457458] block drbd0: Found 64 transactions (3958 active extents) in activity log.
[   50.457458] block drbd0: Method to ensure write ordering: barrier
[   50.457458] block drbd0: Backing device's merge_bvec_fn() = ffffffffa012106e
[   50.457458] block drbd0: max_segment_size ( = BIO size ) = 0
[   50.457458] block drbd0: Adjusting my ra_pages to backing device's (32 -> 128)
[   50.457458] block drbd0: drbd_bm_resize called with capacity == 31255781280
[   50.479664] bonding: bond1: link status definitely up for interface eth2.
[   50.479671] bonding: bond1: link status definitely up for interface eth3.
[   50.814074] block drbd0: resync bitmap: bits=3906972660 words=61046448
[   50.814080] block drbd0: size = 15 TB (15627890640 KB)
[   52.436496] block drbd0: recounting of set bits took additional 88 jiffies
[   52.436496] block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
[   52.436496] block drbd0: disk( Attaching -> UpToDate ) 
[   52.436496] block drbd0: Barriers not supported on meta data device - disabling
[   52.602890] block drbd0: conn( StandAlone -> Unconnected ) 
[   52.602890] block drbd0: Starting receiver thread (from drbd0_worker [4588])
[   52.602890] block drbd0: receiver (re)started
[   52.602890] block drbd0: conn( Unconnected -> WFConnection ) 
[   58.483980] ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
[   58.568092] ADDRCONF(NETDEV_CHANGE): bond1: link becomes ready
[   58.949609] block drbd0: Handshake successful: Agreed network protocol version 90
[   58.950101] block drbd0: Peer authenticated using 20 bytes of 'sha1' HMAC
[   58.950108] block drbd0: conn( WFConnection -> WFReportParams ) 
[   58.952094] block drbd0: Starting asender thread (from drbd0_receiver [4622])
[   58.952094] block drbd0: data-integrity-alg: <not-used>
[   58.952094] block drbd0: drbd_sync_handshake:
[   58.952094] block drbd0: self 19E7AFE5788793A0:0000000000000000:8E425F0092DD9387:498BBCEDBE36C105 bits:0 flags:0
[   58.952094] block drbd0: peer 90A6FA0792E86F85:19E7AFE5788793A1:8E425F0092DD9387:498BBCEDBE36C105 bits:1 flags:0
[   58.952094] block drbd0: uuid_compare()=-1 by rule 5
[   59.296421] block drbd0: peer( Unknown -> Primary ) conn( WFReportParams -> WFBitMapT ) pdsk( DUnknown -> UpToDate ) 
[   59.323287] block drbd0: role( Secondary -> Primary ) 
[   59.500412] OCFS2 Node Manager 1.5.0
[   59.510351] OCFS2 DLM 1.5.0
[   59.514759] ocfs2: Registered cluster interface o2cb
[   59.532362] OCFS2 DLMFS 1.5.0
[   59.532362] OCFS2 User DLM kernel interface loaded
[   69.585323] block drbd0: conn( WFBitMapT -> WFSyncUUID ) 
[   70.918492] block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0
[   70.926471] block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0)
[   70.926471] block drbd0: conn( WFSyncUUID -> SyncTarget ) disk( UpToDate -> Inconsistent ) 
[   70.926471] block drbd0: Began resync as SyncTarget (will sync 4 KB [1 bits set]).
[   71.182461] bond0: no IPv6 routers present
[   71.207187] block drbd0: Method to ensure write ordering: flush
[   71.306401] block drbd0: local disk flush failed with status -95
[   71.378103] block drbd0: Method to ensure write ordering: drain
[   71.401907] bond1: no IPv6 routers present
[   71.433898] block drbd0: Resync done (total 1 sec; paused 0 sec; 4 K/sec)
[   71.433898] block drbd0: conn( SyncTarget -> Connected ) disk( Inconsistent -> UpToDate ) 
[   71.829539] block drbd0: helper command: /sbin/drbdadm after-resync-target minor-0
[   71.833424] block drbd0: helper command: /sbin/drbdadm after-resync-target minor-0 exit code 0 (0x0)
[   73.558584] o2net: connected to node thumper4 (num 0) at 172.16.8.114:7777
[   77.966776] OCFS2 1.5.0
[   77.990990] ocfs2_dlm: Nodes in domain ("695ED53AFE184119A1E92D4093982E0C"): 0 1 
[   78.148976] kjournald starting.  Commit interval 5 seconds
[   78.148977] ocfs2: Mounting device (147,0) on (node 1, slot 0) with ordered data mode.
[   79.026259] RPC: Registered udp transport module.
[   79.026259] RPC: Registered tcp transport module.
[   79.138326] Installing knfsd (copyright (C) 1996 okir at monad.swb.de).
[   81.510784] ipmi message handler version 39.2
[   81.525214] IPMI System Interface driver.
[   81.525214] ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0
[   81.731745] ipmi: Found new BMC (man_id: 0x00002a,  prod_id: 0x4701, dev_id: 0x20)
[   81.731752] IPMI kcs interface initialized
[   81.745978] ipmi device interface
[   82.153853] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   82.163055] NFSD: starting 90-second grace period
[  143.730030] warning: `ntpd' uses 32-bit capabilities (legacy support in use)
[79915.382875] INFO: task nfsd:5458 blocked for more than 120 seconds.
[79915.629365] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79915.815100] nfsd          D 0000000000000000     0  5458      2
[79915.815104]  ffff8103db04b9e8 0000000000000046 0000000000000000 00000000ffffffff
[79915.815108]  ffff8103fd1707d0 ffff8103ff08c6d0 ffff8103fd170a58 00000003db04ba90
[79915.815111]  ffffffff00000001 0000000000000000 00000000ffffffff 0000000000000000
[79915.815113] Call Trace:
[79915.815139]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79915.815145]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79915.815167]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79915.815171]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79915.815177]  [<ffffffff802acb89>] ifind+0x44/0x88
[79915.815181]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79915.815193]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79915.815207]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79915.815212]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79915.815215]  [<ffffffff802ac911>] iput+0x27/0x60
[79915.815222]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79915.815231]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79915.815235]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79915.815248]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79915.815258]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79915.815270]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79915.815291]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79915.815296]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79915.815307]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79915.815318]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79915.815332]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79915.815344]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79915.815373]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79915.815378]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79915.815387]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79915.815396]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79915.815401]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79915.815405]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79915.815414]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79915.815425]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79915.815428]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79915.815431] 
[79915.815432] INFO: task nfsd:5459 blocked for more than 120 seconds.
[79915.967243] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79916.171904] nfsd          D 0000000000000000     0  5459      2
[79916.171906]  ffff8103dbc839e8 0000000000000046 ffff81034eafabc0 00000000ffffffff
[79916.171910]  ffff8103ff2f31a0 ffff8103fcd90810 ffff8103ff2f3428 0000000301045880
[79916.171913]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79916.171916] Call Trace:
[79916.171927]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79916.171933]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79916.171945]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79916.171949]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79916.171954]  [<ffffffff802acb89>] ifind+0x44/0x88
[79916.171959]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79916.171970]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79916.171984]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79916.171989]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79916.171992]  [<ffffffff802ac911>] iput+0x27/0x60
[79916.171999]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79916.172009]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79916.172013]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79916.172027]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79916.172038]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79916.172050]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79916.172064]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79916.172069]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79916.172081]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79916.172092]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79916.172106]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79916.172118]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79916.172135]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79916.172141]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79916.172150]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.172159]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79916.172164]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79916.172168]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79916.172177]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.172186]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.172197]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.172200]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79916.172203] 
[79916.172205] INFO: task nfsd:5460 blocked for more than 120 seconds.
[79916.291381] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79916.493562] nfsd          D 0000000000000000     0  5460      2
[79916.493565]  ffff8103db8179e8 0000000000000046 ffff810271a6d880 00000000ffffffff
[79916.493569]  ffff8103fcfaca50 ffff8103fc9c8710 ffff8103fcfaccd8 0000000307228880
[79916.493572]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79916.493575] Call Trace:
[79916.493586]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79916.493592]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79916.493603]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79916.493607]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79916.493613]  [<ffffffff802acb89>] ifind+0x44/0x88
[79916.493617]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79916.493629]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79916.493643]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79916.493648]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79916.493651]  [<ffffffff802ac911>] iput+0x27/0x60
[79916.493658]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79916.493667]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79916.493672]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79916.493685]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79916.493696]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79916.493708]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79916.493722]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79916.493728]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79916.493739]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79916.493750]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79916.493764]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79916.493776]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79916.493793]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79916.493799]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79916.493808]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.493817]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79916.493822]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79916.493826]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79916.493835]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.493844]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.493855]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.493858]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79916.493861] 
[79916.493862] INFO: task nfsd:5461 blocked for more than 120 seconds.
[79916.579108] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79916.717563] nfsd          D 0000000000000000     0  5461      2
[79916.717566]  ffff8103dbd0b9e8 0000000000000046 ffff810271a6d880 00000000ffffffff
[79916.717569]  ffff8103fc9c8710 ffff8103fcd28380 ffff8103fc9c8998 0000000307228880
[79916.717573]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79916.717576] Call Trace:
[79916.717586]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79916.717592]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79916.717604]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79916.717608]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79916.717614]  [<ffffffff802acb89>] ifind+0x44/0x88
[79916.717619]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79916.717630]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79916.717644]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79916.717649]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79916.717652]  [<ffffffff802ac911>] iput+0x27/0x60
[79916.717659]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79916.717668]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79916.717673]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79916.717686]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79916.717697]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79916.717709]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79916.717723]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79916.717729]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79916.717740]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79916.717752]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79916.717766]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79916.717778]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79916.717794]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79916.717800]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79916.717809]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.717819]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79916.717823]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79916.717827]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79916.717837]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.717846]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.717856]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.717859]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79916.717862] 
[79916.717864] INFO: task nfsd:5462 blocked for more than 120 seconds.
[79916.807124] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79916.900845] nfsd          D 0000000000000001     0  5462      2
[79916.901153]  ffff8103db9456a0 0000000000000046 0000000000000000 0000000000001000
[79916.901156]  ffff8103ff288380 ffff8103fd550a90 ffff8103ff288608 00000003802ba8a9
[79916.901160]  ffff8102a6de1cf8 ffff81029d824b58 ffff810057107338 ffffffffa03ca3ef
[79916.901163] Call Trace:
[79916.901181]  [<ffffffffa03ca3ef>] :ocfs2:ocfs2_buffer_cached+0x94/0x110
[79916.901187]  [<ffffffff8042a439>] __down_read+0x87/0xa1
[79916.901202]  [<ffffffffa03b59a1>] :ocfs2:ocfs2_start_trans+0xd3/0x1b5
[79916.901214]  [<ffffffffa0398047>] :ocfs2:ocfs2_write_begin_nolock+0xa83/0xfc8
[79916.901235]  [<ffffffffa0399d63>] :ocfs2:ocfs2_write_begin+0x112/0x1cf
[79916.901243]  [<ffffffff80271e7e>] generic_file_buffered_write+0x14c/0x630
[79916.901555]  [<ffffffff80238b41>] current_fs_time+0x1e/0x24
[79916.901560]  [<ffffffff80270c4a>] remove_suid+0x18/0x4b
[79916.901564]  [<ffffffff802adb7a>] file_update_time+0xb2/0xf2
[79916.901570]  [<ffffffff802726a1>] __generic_file_aio_write_nolock+0x33f/0x3a9
[79916.901581]  [<ffffffffa039f1f6>] :ocfs2:ocfs2_cluster_unlock+0x21d/0x289
[79916.901592]  [<ffffffff80272800>] generic_file_aio_write_nolock+0x34/0x80
[79916.901605]  [<ffffffffa03adadb>] :ocfs2:ocfs2_file_aio_write+0x425/0x562
[79916.901609]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79916.901617]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79916.901628]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79916.901642]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79916.901647]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79916.901650]  [<ffffffff802ac911>] iput+0x27/0x60
[79916.901657]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79916.901666]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79916.901671]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79916.901684]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79916.901695]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79916.901707]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79916.901721]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79916.901727]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79916.901738]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79916.901750]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79916.901764]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79916.901776]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79916.901792]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79916.901798]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79916.901808]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.901817]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79916.901822]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79916.901825]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79916.901847]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.901856]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.901867]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79916.901870]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79916.901873] 
[79916.901874] INFO: task nfsd:5463 blocked for more than 120 seconds.
[79916.976223] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79917.088925] nfsd          D 0000000000000000     0  5463      2
[79917.088929]  ffff8103db1619e8 0000000000000046 ffff81034eafa5c0 00000000ffffffff
[79917.088932]  ffff8103fd550a90 ffff8101fe4cd3f0 ffff8103fd550d18 0000000370d5fcb8
[79917.088936]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79917.088939] Call Trace:
[79917.088954]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79917.088960]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79917.088979]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79917.088984]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79917.088989]  [<ffffffff802acb89>] ifind+0x44/0x88
[79917.088994]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79917.089005]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79917.089019]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79917.089024]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79917.089028]  [<ffffffff802ac911>] iput+0x27/0x60
[79917.089035]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79917.089044]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79917.089049]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79917.089062]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79917.089073]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79917.089085]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79917.089104]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79917.089110]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79917.089121]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79917.089133]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79917.089147]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79917.089160]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79917.089186]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79917.089191]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79917.089201]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.089210]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79917.089215]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79917.089219]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79917.089228]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.089237]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.089247]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.089250]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79917.089253] 
[79917.089255] INFO: task nfsd:5464 blocked for more than 120 seconds.
[79917.177776] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79917.302240] nfsd          D 0000000000000000     0  5464      2
[79917.302243]  ffff8103da81d9e8 0000000000000046 ffff810271a6d680 00000000ffffffff
[79917.302247]  ffff8103fcdf6200 ffff8103ff2f31a0 ffff8103fcdf6488 0000000307228880
[79917.302250]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79917.302253] Call Trace:
[79917.302264]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79917.302270]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79917.302281]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79917.302285]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79917.302291]  [<ffffffff802acb89>] ifind+0x44/0x88
[79917.302296]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79917.302307]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79917.302321]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79917.302326]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79917.302329]  [<ffffffff802ac911>] iput+0x27/0x60
[79917.302336]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79917.302345]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79917.302350]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79917.302363]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79917.302374]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79917.302386]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79917.302400]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79917.302406]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79917.302417]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79917.302428]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79917.302443]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79917.302455]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79917.302471]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79917.302477]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79917.302486]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.302495]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79917.302500]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79917.302504]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79917.302513]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.302522]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.302532]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.302536]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79917.302538] 
[79917.302540] INFO: task nfsd:5465 blocked for more than 120 seconds.
[79917.381716] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79917.597162] nfsd          D 0000000000000000     0  5465      2
[79917.597165]  ffff8103da9399e8 0000000000000046 ffff81034eafabc0 00000000ffffffff
[79917.597168]  ffff8103fcd90810 ffff8103fcdf6f20 ffff8103fcd90a98 000000038022889d
[79917.597172]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79917.597175] Call Trace:
[79917.597185]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79917.597192]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79917.597203]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79917.597207]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79917.597213]  [<ffffffff802acb89>] ifind+0x44/0x88
[79917.597217]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79917.597229]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79917.597242]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79917.597247]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79917.597251]  [<ffffffff802ac911>] iput+0x27/0x60
[79917.597258]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79917.597267]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79917.597272]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79917.597277]  [<ffffffff802960b1>] cache_alloc_refill+0x111/0x20c
[79917.597288]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79917.597299]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79917.597311]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79917.597325]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79917.597330]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79917.597342]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79917.597353]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79917.597367]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79917.597379]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79917.597396]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79917.597401]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79917.597411]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.597420]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79917.597425]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79917.597428]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79917.597438]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.597447]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.597457]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.597460]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79917.597463] 
[79917.597465] INFO: task nfsd:5466 blocked for more than 120 seconds.
[79917.720630] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79917.882508] nfsd          D 0000000000000000     0  5466      2
[79917.882511]  ffff8103daa359e8 0000000000000046 ffff810271a6d880 00000000ffffffff
[79917.882515]  ffff8103fcd28380 ffff8103fcdf6200 ffff8103fcd28608 0000000307228880
[79917.882518]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79917.882521] Call Trace:
[79917.882532]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79917.882538]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79917.882550]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79917.882554]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79917.882559]  [<ffffffff802acb89>] ifind+0x44/0x88
[79917.882564]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79917.882575]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79917.882589]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79917.882594]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79917.882597]  [<ffffffff802ac911>] iput+0x27/0x60
[79917.882604]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79917.882613]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79917.882618]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79917.882631]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79917.882642]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79917.882655]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79917.882668]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79917.882674]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79917.882685]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79917.882697]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79917.882711]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79917.882723]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79917.882739]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79917.882745]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79917.882755]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.882764]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79917.882768]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79917.882772]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79917.882781]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.882791]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.882801]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79917.882804]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79917.882807] 
[79917.882809] INFO: task nfsd:5467 blocked for more than 120 seconds.
[79918.049924] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79918.196403] nfsd          D 0000000000000000     0  5467      2
[79918.196406]  ffff8103dab319e8 0000000000000046 ffff810271a6d680 00000000ffffffff
[79918.196409]  ffff8103ff2890a0 ffff8103fcfaca50 ffff8103ff289328 0000000307228880
[79918.196413]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79918.196416] Call Trace:
[79918.196426]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79918.196432]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79918.196444]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79918.196448]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79918.196454]  [<ffffffff802acb89>] ifind+0x44/0x88
[79918.196458]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79918.196469]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79918.196483]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79918.196488]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79918.196492]  [<ffffffff802ac911>] iput+0x27/0x60
[79918.196498]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79918.196508]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79918.196513]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79918.196526]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79918.196537]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79918.196549]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79918.196563]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79918.196568]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79918.196580]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79918.196591]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79918.196606]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79918.196618]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79918.196634]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79918.196640]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79918.196649]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.196658]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79918.196663]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79918.196667]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79918.196676]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.196685]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.196695]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.196699]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79918.196701] 
[79918.196703] INFO: task nfsd:5468 blocked for more than 120 seconds.
[79918.316628] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79918.482345] nfsd          D 0000000000000001     0  5468      2
[79918.482348]  ffff8103da44b6a0 0000000000000046 0000000000000000 0000000000001000
[79918.482351]  ffff8103ff334580 ffff8101fc989630 ffff8103ff334808 00000003802ba8a9
[79918.482355]  ffff8102b0789950 ffff8102a6de1cf8 ffff81029d824b58 ffffffffa03ca3ef
[79918.482358] Call Trace:
[79918.482375]  [<ffffffffa03ca3ef>] :ocfs2:ocfs2_buffer_cached+0x94/0x110
[79918.482381]  [<ffffffff8042a439>] __down_read+0x87/0xa1
[79918.482395]  [<ffffffffa03b59a1>] :ocfs2:ocfs2_start_trans+0xd3/0x1b5
[79918.482407]  [<ffffffffa0397966>] :ocfs2:ocfs2_write_begin_nolock+0x3a2/0xfc8
[79918.486349]  [<ffffffff8042a31c>] __down_write_nested+0x12/0xa1
[79918.486362]  [<ffffffffa0399d63>] :ocfs2:ocfs2_write_begin+0x112/0x1cf
[79918.486368]  [<ffffffff8024ac06>] getnstimeofday+0x39/0x98
[79918.486373]  [<ffffffff80271e7e>] generic_file_buffered_write+0x14c/0x630
[79918.486377]  [<ffffffff80248d47>] ktime_get+0xc/0x41
[79918.486385]  [<ffffffff80238b41>] current_fs_time+0x1e/0x24
[79918.486389]  [<ffffffff80270c4a>] remove_suid+0x18/0x4b
[79918.486394]  [<ffffffff802adb7a>] file_update_time+0xb2/0xf2
[79918.486399]  [<ffffffff802726a1>] __generic_file_aio_write_nolock+0x33f/0x3a9
[79918.486410]  [<ffffffffa039f1f6>] :ocfs2:ocfs2_cluster_unlock+0x21d/0x289
[79918.486417]  [<ffffffff80272800>] generic_file_aio_write_nolock+0x34/0x80
[79918.486417]  [<ffffffffa03adadb>] :ocfs2:ocfs2_file_aio_write+0x425/0x562
[79918.486417]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79918.486417]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79918.486417]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79918.486417]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79918.486417]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79918.486417]  [<ffffffff802ac911>] iput+0x27/0x60
[79918.486417]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79918.486417]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79918.486417]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79918.486417]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79918.486417]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79918.486417]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79918.486417]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79918.486417]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79918.486417]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79918.486417]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79918.486417]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79918.486417]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79918.486417]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79918.486417]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79918.486417]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.486417]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79918.486417]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79918.486417]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79918.486417]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.486417]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.486417]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.486417]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79918.486417] 
[79918.486417] INFO: task nfsd:5469 blocked for more than 120 seconds.
[79918.609649] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79918.755370] nfsd          D 0000000000000000     0  5469      2
[79918.755373]  ffff8103da547c90 0000000000000046 0000000000000000 ffffffff80295cdc
[79918.755376]  ffff8103fcdf6890 ffff8103ff08c6d0 ffff8103fcdf6b18 0000000300008850
[79918.755380]  ffff810000021e80 0000000000000000 00000000ffffffff 0000000000000000
[79918.755383] Call Trace:
[79918.755386]  [<ffffffff80295cdc>] fallback_alloc+0xe7/0x1e1
[79918.755400]  [<ffffffffa0182a2d>] :jbd:start_this_handle+0x11b/0x304
[79918.755405]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79918.755409]  [<ffffffff80278dfa>] pagevec_lookup_tag+0x1a/0x21
[79918.755418]  [<ffffffffa0182cb1>] :jbd:journal_start+0x9b/0xd2
[79918.755426]  [<ffffffffa0182cf6>] :jbd:journal_force_commit+0xe/0x26
[79918.755439]  [<ffffffffa03ab201>] :ocfs2:ocfs2_sync_file+0xb8/0x11a
[79918.755451]  [<ffffffffa0483983>] :nfsd:nfsd_sync_dir+0x39/0x52
[79918.755462]  [<ffffffffa048589b>] :nfsd:nfsd_rename+0x2e4/0x35c
[79918.755477]  [<ffffffffa048b9d7>] :nfsd:nfsd3_proc_rename+0x12c/0x13c
[79918.755488]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79918.755504]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79918.755510]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79918.755519]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.755528]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79918.755533]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79918.755537]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79918.755546]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.755555]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.755566]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79918.755569]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79918.755572] 
[79918.755573] INFO: task nfsd:5470 blocked for more than 120 seconds.
[79918.886556] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79919.044288] nfsd          D 0000000000000001     0  5470      2
[79919.044291]  ffff8103da663c20 0000000000000046 ffff81027ca5f568 ffffe2000602e8a8
[79919.044294]  ffff8103ff254300 ffff8103ff288380 ffff8103ff254588 000000037ca5f678
[79919.044297]  0000000000000000 ffff8103da663c50 ffff81027ca5f678 ffffffff80272e84
[79919.044300] Call Trace:
[79919.044307]  [<ffffffff80272e84>] find_get_pages+0x2f/0x6d
[79919.044313]  [<ffffffff8042a439>] __down_read+0x87/0xa1
[79919.044328]  [<ffffffffa03b59a1>] :ocfs2:ocfs2_start_trans+0xd3/0x1b5
[79919.044340]  [<ffffffffa03937b6>] :ocfs2:ocfs2_truncate_inline+0x7f/0x2a9
[79919.044356]  [<ffffffffa03abc2f>] :ocfs2:ocfs2_setattr+0x751/0x147f
[79919.044366]  [<ffffffff802ae06b>] notify_change+0x174/0x301
[79919.044380]  [<ffffffffa0484a90>] :nfsd:nfsd_setattr+0x35a/0x3fc
[79919.044393]  [<ffffffffa048b287>] :nfsd:nfsd3_proc_setattr+0x8d/0x99
[79919.044403]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79919.044419]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79919.044425]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79919.044435]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.044444]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79919.044448]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79919.044452]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79919.044461]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.044470]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.044480]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.044484]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79919.044486] 
[79919.044488] INFO: task nfsd:5471 blocked for more than 120 seconds.
[79919.169844] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79919.343561] nfsd          D 0000000000000000     0  5471      2
[79919.343564]  ffff8103da75f9e8 0000000000000046 ffff81034eafa5c0 00000000ffffffff
[79919.343568]  ffff8103ff3a8ee0 ffff8103ff2890a0 ffff8103ff3a9168 0000000307228880
[79919.343571]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79919.343574] Call Trace:
[79919.343585]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79919.343591]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79919.343603]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79919.343606]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79919.343612]  [<ffffffff802acb89>] ifind+0x44/0x88
[79919.343617]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79919.343628]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79919.343642]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79919.343647]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79919.343650]  [<ffffffff802ac911>] iput+0x27/0x60
[79919.343657]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79919.343666]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79919.343671]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79919.343684]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79919.343695]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79919.343707]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79919.343721]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79919.343727]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79919.343738]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79919.343750]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79919.343764]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79919.343776]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79919.343792]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79919.343798]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79919.343808]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.343817]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79919.343821]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79919.343825]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79919.343834]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.343844]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.343854]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.343857]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79919.343860] 
[79919.343862] INFO: task nfsd:5472 blocked for more than 120 seconds.
[79919.471040] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79919.616743] nfsd          D 0000000000000000     0  5472      2
[79919.616746]  ffff8103da05b9e8 0000000000000046 ffff81034eafa5c0 00000000ffffffff
[79919.616750]  ffff8103fcdf6f20 ffff8103fcfad770 ffff8103fcdf71a8 0000000307228880
[79919.616753]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79919.616756] Call Trace:
[79919.616767]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79919.616773]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79919.616785]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79919.616788]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79919.616794]  [<ffffffff802acb89>] ifind+0x44/0x88
[79919.616799]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79919.616810]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79919.616824]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79919.616829]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79919.616832]  [<ffffffff802ac911>] iput+0x27/0x60
[79919.616839]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79919.616848]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79919.616853]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79919.616866]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79919.616877]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79919.616889]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79919.616903]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79919.616909]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79919.616920]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79919.616932]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79919.616946]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79919.616958]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79919.616974]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79919.616980]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79919.620752]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.620761]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79919.620766]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79919.620770]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79919.620779]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.620788]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.620798]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.620801]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79919.620804] 
[79919.620806] INFO: task nfsd:5473 blocked for more than 120 seconds.
[79919.771749] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79919.925454] nfsd          D 0000000000000000     0  5473      2
[79919.925457]  ffff8103da1759e8 0000000000000046 ffff81034eafa9c0 00000000ffffffff
[79919.925460]  ffff8103fcfad770 ffff8103fd1707d0 ffff8103fcfad9f8 0000000307228880
[79919.925464]  ffffffffdbc87b40 3137343630396237 7fffffffffffffff 000000000000003b
[79919.925467] Call Trace:
[79919.925477]  [<ffffffff8042978d>] __mutex_lock_slowpath+0x64/0x9b
[79919.925483]  [<ffffffff804295f2>] mutex_lock+0xa/0xb
[79919.925495]  [<ffffffffa03ad83d>] :ocfs2:ocfs2_file_aio_write+0x187/0x562
[79919.925499]  [<ffffffff802aca5a>] find_inode+0x28/0x6d
[79919.925505]  [<ffffffff802acb89>] ifind+0x44/0x88
[79919.925509]  [<ffffffff802ad42c>] iget5_locked+0x46/0x176
[79919.925521]  [<ffffffffa03afd91>] :ocfs2:ocfs2_init_locked_inode+0x0/0xca
[79919.925534]  [<ffffffffa03ad6b6>] :ocfs2:ocfs2_file_aio_write+0x0/0x562
[79919.925539]  [<ffffffff8029ad73>] do_sync_readv_writev+0xc0/0x107
[79919.925543]  [<ffffffff802ac911>] iput+0x27/0x60
[79919.925550]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79919.925559]  [<ffffffff8029ac09>] rw_copy_check_uvector+0x6c/0xdc
[79919.925564]  [<ffffffff8029b426>] do_readv_writev+0xb2/0x18b
[79919.925577]  [<ffffffffa03a0dcb>] :ocfs2:ocfs2_build_lock_name+0xba/0x16a
[79919.925588]  [<ffffffffa039ed6a>] :ocfs2:ocfs2_add_lockres_tracking+0x62/0x8e
[79919.925600]  [<ffffffffa03a515e>] :ocfs2:ocfs2_file_lock_res_init+0x5e/0x6d
[79919.925614]  [<ffffffffa0483d90>] :nfsd:nfsd_vfs_write+0xe9/0x2dd
[79919.925619]  [<ffffffff802999f9>] __dentry_open+0x154/0x238
[79919.925631]  [<ffffffffa048441d>] :nfsd:nfsd_open+0x141/0x175
[79919.925642]  [<ffffffffa0484655>] :nfsd:nfsd_write+0xc5/0xe2
[79919.925657]  [<ffffffffa048b5fc>] :nfsd:nfsd3_proc_write+0xe6/0x105
[79919.925668]  [<ffffffffa0480245>] :nfsd:nfsd_dispatch+0xde/0x1b6
[79919.925685]  [<ffffffffa03f25c5>] :sunrpc:svc_process+0x408/0x6e7
[79919.925691]  [<ffffffff8042a3c4>] __down_read+0x12/0xa1
[79919.925700]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.925709]  [<ffffffffa0480810>] :nfsd:nfsd+0x194/0x2a4
[79919.925714]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79919.925718]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79919.925727]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.925736]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.925746]  [<ffffffffa048067c>] :nfsd:nfsd+0x0/0x2a4
[79919.925749]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79919.925752] 
[79921.827928] INFO: task ocfs2_wq:4727 blocked for more than 120 seconds.
[79921.963970] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[79922.158466] ocfs2_wq      D 0000000000000000     0  4727      2
[79922.158469]  ffff8101fe547d60 0000000000000046 0000000000000000 ffffffffa039a6a6
[79922.158473]  ffff8101fdc84240 ffff8103ff08c6d0 ffff8101fdc844c8 00000003a03a00f4
[79922.158476]  ffff8101fe547e40 0000000000000000 00000000ffffffff 0000000000000000
[79922.158479] Call Trace:
[79922.158491]  [<ffffffffa039a6a6>] :ocfs2:ocfs2_read_blocks+0x59c/0x682
[79922.158501]  [<ffffffff8042a439>] __down_read+0x87/0xa1
[79922.158515]  [<ffffffffa03b59a1>] :ocfs2:ocfs2_start_trans+0xd3/0x1b5
[79922.158525]  [<ffffffffa0390190>] :ocfs2:ocfs2_do_insert_extent+0x41/0xce1
[79922.158535]  [<ffffffffa0390190>] :ocfs2:ocfs2_do_insert_extent+0x41/0xce1
[79922.158546]  [<ffffffffa038abc6>] :ocfs2:__ocfs2_flush_truncate_log+0x279/0x7bb
[79922.158549]  [<ffffffff8020a857>] __switch_to+0x96/0x35e
[79922.158554]  [<ffffffff8022efd6>] hrtick_set+0x88/0xf7
[79922.158566]  [<ffffffffa039154e>] :ocfs2:ocfs2_truncate_log_worker+0x0/0x149
[79922.158577]  [<ffffffffa038b130>] :ocfs2:ocfs2_flush_truncate_log+0x28/0x39
[79922.158588]  [<ffffffffa03915ad>] :ocfs2:ocfs2_truncate_log_worker+0x5f/0x149
[79922.158598]  [<ffffffffa039154e>] :ocfs2:ocfs2_truncate_log_worker+0x0/0x149
[79922.158602]  [<ffffffff80243070>] run_workqueue+0x82/0x111
[79922.158606]  [<ffffffff8024393d>] worker_thread+0xd5/0xe0
[79922.158610]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[79922.158615]  [<ffffffff80243868>] worker_thread+0x0/0xe0
[79922.158617]  [<ffffffff8024604b>] kthread+0x47/0x74
[79922.158620]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[79922.158624]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[79922.158632]  [<ffffffff80246004>] kthread+0x0/0x74
[79922.158635]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[79922.158638] 
[160264.757099] device eth1 entered promiscuous mode
[160269.188513] device eth1 left promiscuous mode
[160282.882880] device eth0 entered promiscuous mode
[160434.218961] device eth0 left promiscuous mode
[160453.793073] device bond0 entered promiscuous mode
[160453.793079] device eth0 entered promiscuous mode
[160453.793187] device eth1 entered promiscuous mode
[164454.419241] ocfs2_dlm: Node 0 leaves domain 695ED53AFE184119A1E92D4093982E0C
[164454.419241] ocfs2_dlm: Nodes in domain ("695ED53AFE184119A1E92D4093982E0C"): 1 
[164456.134175] o2net: no longer connected to node thumper4 (num 0) at 172.16.8.114:7777
[165409.924817] nfsd: last server has exited
[165409.924817] nfsd: unexporting all filesystems
[165409.924817] KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (157)
[165451.031883] ocfs2: Unmounting device (147,0) on (node 1)
[165452.834386] ocfs2: Unregistered cluster interface o2cb
[165452.962792] OCFS2 Node Manager 1.5.0
[165452.971113] OCFS2 DLM 1.5.0
[165452.972031] ocfs2: Registered cluster interface o2cb
[165452.992838] OCFS2 DLMFS 1.5.0
[165452.993295] OCFS2 User DLM kernel interface loaded
[165459.491689] OCFS2 1.5.0
[165459.520468] ocfs2_dlm: Nodes in domain ("695ED53AFE184119A1E92D4093982E0C"): 1 
[165459.657302] kjournald starting.  Commit interval 5 seconds
[165459.665231] ocfs2: Mounting device (147,0) on (node 1, slot 0) with ordered data mode.
[165459.757507] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[165459.770677] NFSD: starting 90-second grace period
[165567.324782] o2net: connected to node thumper4 (num 0) at 172.16.8.114:7777
[165571.421892] ocfs2_dlm: Node 0 joins domain 695ED53AFE184119A1E92D4093982E0C
[165571.421897] ocfs2_dlm: Nodes in domain ("695ED53AFE184119A1E92D4093982E0C"): 0 1 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://oss.oracle.com/pipermail/ocfs2-users/attachments/20100603/9eb0ea1b/attachment-0001.bin 


More information about the Ocfs2-users mailing list