fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
...
fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
Starting 3 threads
Threads: 3
Threads: 3
Threads: 3: [mmm] [15.0% done] [ 12552/ 13940 kb/s] [eta 00m:17s]
Threads: 3: [mmm] [19.0% done] [ 11176/ 11656 kb/s] [eta 00m:17s]
Threads: 3: [mmm] [25.0% done] [ 14812/  9040 kb/s] [eta 00m:15s]
Threads: 3: [mmm] [30.0% done] [ 14600/  7308 kb/s] [eta 00m:14s]
Threads: 3: [mmm] [33.3% done] [ 11072/  9212 kb/s] [eta 00m:14s]
Threads: 3: [mmm] [38.1% done] [ 11156/  8872 kb/s] [eta 00m:13s]
Threads: 3: [mmm] [42.9% done] [ 15180/  3160 kb/s] [eta 00m:12s]
Threads: 3: [mmm] [47.6% done] [ 13932/  3524 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [52.4% done] [ 10044/  7268 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [54.5% done] [ 11068/  3256 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [56.5% done] [  9344/  4548 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [60.9% done] [  7928/  4316 kb/s] [eta 00m:09s]
Threads: 3: [mmm] [62.5% done] [  7828/  3688 kb/s] [eta 00m:09s]
Threads: 3: [mmm] [66.7% done] [  9012/  2208 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [68.0% done] [  9288/  3476 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [72.0% done] [ 23892/     0 kb/s] [eta 00m:07s]
Threads: 3: [VVV] [79.2% done] [ 22816/     0 kb/s] [eta 00m:05s]
Threads: 2: [_VV] [83.3% done] [ 18968/     0 kb/s] [eta 00m:04s]
Threads: 2: [_VV] [91.3% done] [ 22212/     0 kb/s] [eta 00m:02s]
fio: (groupid=0): err= 0: pid=22685
  read : io=100MiB, bw=5,235KiB/s, runt= 20027msec
    slat (msec): min=    0, max= 3016, avg=277.33, stdev=709.47
    clat (msec): min=    0, max=  417, avg=80.63, stdev=63.77
    bw (KiB/s) : min=  123, max=10534, per=38.68%, avg=5488.17, stdev=2454.76
  write: io=18,992KiB, bw=1,137KiB/s, runt= 17100msec
    slat (msec): min=    0, max=    3, avg= 0.12, stdev= 0.50
    clat (msec): min=    0, max=  734, avg=78.85, stdev=111.38
    bw (KiB/s) : min=    0, max=10378, per=51.79%, avg=3351.80, stdev=3586.03
  cpu          : usr=4.27%, sys=0.24%, ctx=1773
  IO depths    : 1=0.2%, 2=0.4%, 4=6.9%, 8=92.5%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=2.0%, 4=0.8%, 10=2.4%, 20=9.3%, 50=26.9%, 100=30.2%
     lat (msec): 250=24.3%, 500=3.8%, 750=0.2%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=22686
  read : io=100MiB, bw=4,743KiB/s, runt= 22105msec
    slat (msec): min=    0, max= 5466, avg=1278.46, stdev=1817.67
    clat (msec): min=    4, max=  332, avg=70.11, stdev=62.12
    bw (KiB/s) : min=  102, max=15491, per=37.18%, avg=5275.23, stdev=4015.97
  write: io=46,076KiB, bw=2,821KiB/s, runt= 16720msec
    slat (msec): min=    0, max=    7, avg= 0.16, stdev= 0.74
    clat (msec): min=    0, max=  916, avg=91.59, stdev=100.58
    bw (KiB/s) : min=    0, max= 9663, per=52.89%, avg=3422.92, stdev=3045.24
  cpu          : usr=4.36%, sys=0.27%, ctx=1756
  IO depths    : 1=0.1%, 2=0.2%, 4=15.4%, 8=84.2%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=1.1%, 4=0.3%, 10=3.8%, 20=13.8%, 50=26.5%, 100=29.4%
     lat (msec): 250=21.5%, 500=3.4%, 750=0.1%, 1000=0.1%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=22687
  read : io=100MiB, bw=4,729KiB/s, runt= 22171msec
    slat (msec): min=    0, max= 5220, avg=1310.17, stdev=1760.14
    clat (msec): min=    1, max=  519, avg=71.16, stdev=67.40
    bw (KiB/s) : min=   52, max=11198, per=35.28%, avg=5005.46, stdev=3366.84
  write: io=43,016KiB, bw=2,576KiB/s, runt= 17098msec
    slat (msec): min=    0, max=    3, avg= 0.06, stdev= 0.27
    clat (msec): min=    0, max=  792, avg=82.60, stdev=92.83
    bw (KiB/s) : min=    0, max= 8040, per=48.66%, avg=3149.12, stdev=2679.04
  cpu          : usr=4.56%, sys=0.27%, ctx=1775
  IO depths    : 1=0.1%, 2=0.2%, 4=17.2%, 8=82.5%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=1.4%, 4=0.6%, 10=5.9%, 20=14.1%, 50=26.6%, 100=24.9%
     lat (msec): 250=22.5%, 500=3.6%, 750=0.2%, 1000=0.0%, >=2000=0.0%

Run status group 0 (all jobs):
   READ: io=300MiB, aggrb=14,188KiB/s, minb=4,729KiB/s, maxb=5,235KiB/s, mint=20027msec, maxt=22171msec
  WRITE: io=106MiB, aggrb=6,472KiB/s, minb=1,137KiB/s, maxb=2,821KiB/s, mint=20027msec, maxt=22171msec

Disk stats (read/write):
  sda: ios=4477/1641, merge=1430/446, ticks=334104/153700, in_queue=487816, util=99.46%
2.89user 0.17system 0:22.42elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2390minor)pagefaults 0swaps