fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
...
fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
Starting 3 threads
Threads: 3
Threads: 3
Threads: 3: [mmm] [10.7% done] [ 11602/  9324 kb/s] [eta 00m:25s]
Threads: 3: [mmm] [14.3% done] [  6027/ 11380 kb/s] [eta 00m:24s]
Threads: 3: [mmm] [18.5% done] [  6152/ 11664 kb/s] [eta 00m:22s]
Threads: 3: [mmm] [21.4% done] [ 11640/  5892 kb/s] [eta 00m:22s]
Threads: 3: [mmm] [23.3% done] [  3712/ 12632 kb/s] [eta 00m:23s]
Threads: 3: [mmm] [26.7% done] [ 11360/  5236 kb/s] [eta 00m:22s]
Threads: 3: [mmm] [30.0% done] [  9688/  5488 kb/s] [eta 00m:21s]
Threads: 3: [mmm] [32.3% done] [  5019/  8850 kb/s] [eta 00m:21s]
Threads: 3: [mmm] [34.4% done] [ 10866/  4529 kb/s] [eta 00m:21s]
Threads: 3: [mmm] [37.5% done] [ 11736/  2080 kb/s] [eta 00m:20s]
Threads: 3: [mmm] [40.6% done] [ 10668/  2588 kb/s] [eta 00m:19s]
Threads: 3: [mmm] [43.8% done] [  5468/  6388 kb/s] [eta 00m:18s]
Threads: 3: [mmm] [46.9% done] [  5272/  6931 kb/s] [eta 00m:17s]
Threads: 3: [mmm] [50.0% done] [  4140/  7906 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [51.5% done] [  4636/  4728 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [52.9% done] [  3692/  5048 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [55.9% done] [  5088/  2936 kb/s] [eta 00m:15s]
Threads: 3: [mmm] [58.8% done] [  5012/  3632 kb/s] [eta 00m:14s]
Bad verify header a000c900
Bad verify header 9555c840
Bad verify header 5071ea20
Bad verify header 1f592a20
fio: pid=3884, err=1/file:io_u.c:496, error=Operation not permitted
Threads: 2: [mm_] [60.0% done] [  3328/  3760 kb/s] [eta 00m:14s]
Threads: 1: [_m_] [64.7% done] [  3160/  4584 kb/s] [eta 00m:12s]
fio: (groupid=0): err= 0: pid=3882
  read : io=50,648KiB, bw=2,382KiB/s, runt= 21773msec
    slat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    clat (msec): min=    5, max=  659, avg=114.66, stdev=128.84
    bw (KiB/s) : min=   90, max=11114, per=49.78%, avg=3780.36, stdev=3129.47
  write: io=51,752KiB, bw=2,465KiB/s, runt= 21491msec
    slat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    clat (msec): min=    0, max=  687, avg=93.59, stdev=123.74
    bw (KiB/s) : min=    0, max= 9136, per=55.92%, avg=3485.20, stdev=2684.46
  cpu          : usr=94.54%, sys=0.90%, ctx=284
  IO depths    : 1=0.1%, 2=0.6%, 4=11.7%, 8=87.6%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=5.5%, 4=2.2%, 8=4.2%, 16=13.8%, 32=20.7%, 64=16.4%, 128=7.2%
     lat (msec): 256=11.7%, 512=17.4%, 1024=0.7%, >=2048=0.0%
fio: (groupid=0): err= 0: pid=3883
  read : io=54,148KiB, bw=2,436KiB/s, runt= 22757msec
    slat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    clat (msec): min=    3, max=  826, avg=122.02, stdev=146.71
    bw (KiB/s) : min=   53, max= 7405, per=36.08%, avg=2739.61, stdev=1740.81
  write: io=48,252KiB, bw=2,172KiB/s, runt= 22745msec
    slat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    clat (msec): min=    0, max=  779, avg=93.68, stdev=127.53
    bw (KiB/s) : min=    0, max= 6414, per=41.17%, avg=2565.92, stdev=2094.48
  cpu          : usr=97.05%, sys=0.72%, ctx=783
  IO depths    : 1=0.2%, 2=0.7%, 4=14.7%, 8=84.5%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=4.1%, 4=1.5%, 8=2.8%, 16=11.4%, 32=26.6%, 64=20.5%, 128=4.5%
     lat (msec): 256=6.3%, 512=21.3%, 1024=0.8%, >=2048=0.0%
fio: (groupid=0): err= 1 (file:io_u.c:496, error=Operation not permitted): pid=3884
  read : io=63,972KiB, bw=3,175KiB/s, runt= 20632msec
    slat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    clat (msec): min=    4, max=  676, avg=92.78, stdev=115.53
    bw (KiB/s) : min=  400, max=12499, per=52.11%, avg=3957.46, stdev=2864.36
  write: io=38,428KiB, bw=1,948KiB/s, runt= 20195msec
    slat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    clat (msec): min=    0, max=  547, avg=107.25, stdev=135.99
    bw (KiB/s) : min=    0, max=10651, per=55.92%, avg=3485.11, stdev=3299.38
  cpu          : usr=97.04%, sys=0.58%, ctx=284
  IO depths    : 1=0.1%, 2=0.3%, 4=5.9%, 8=93.7%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=2.3%, 4=1.7%, 8=5.9%, 16=14.6%, 32=24.7%, 64=17.5%, 128=5.7%
     lat (msec): 256=9.0%, 512=18.0%, 1024=0.5%, >=2048=0.0%

Run status group 0 (all jobs):
   READ: io=165MiB, aggrb=7,594KiB/s, minb=2,382KiB/s, maxb=3,175KiB/s, mint=20632msec, maxt=22757msec
  WRITE: io=135MiB, aggrb=6,232KiB/s, minb=1,948KiB/s, maxb=2,465KiB/s, mint=20632msec, maxt=22757msec

Disk stats (read/write):
  sda: ios=3095/2596, merge=568/511, ticks=367612/299404, in_queue=667016, util=98.94%
Command exited with non-zero status 1
62.73user 0.50system 0:23.21elapsed 272%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2440minor)pagefaults 0swaps