fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
...
fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
Starting 3 threads
Threads: 3
Threads: 3
Threads: 3: [mmm] [15.0% done] [ 12172/ 11500 kb/s] [eta 00m:17s]
Threads: 3: [mmm] [20.0% done] [ 18840/  5440 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [23.8% done] [ 11992/ 10352 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [28.6% done] [ 16528/  5168 kb/s] [eta 00m:15s]
Threads: 3: [mmm] [33.3% done] [ 16000/  5996 kb/s] [eta 00m:14s]
Threads: 3: [mmm] [38.1% done] [ 16332/  2444 kb/s] [eta 00m:13s]
Threads: 3: [mmm] [42.9% done] [ 11900/  7648 kb/s] [eta 00m:12s]
Threads: 3: [mmm] [47.6% done] [ 14936/  2252 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [50.0% done] [ 11664/  4940 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [54.5% done] [   552/ 14416 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [56.5% done] [  7824/  5472 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [60.9% done] [  9744/  2924 kb/s] [eta 00m:09s]
Threads: 3: [mmm] [62.5% done] [ 11528/   168 kb/s] [eta 00m:09s]
Threads: 3: [mmm] [64.0% done] [  6120/  4428 kb/s] [eta 00m:09s]
Threads: 3: [VVV] [68.0% done] [  9484/   920 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [72.0% done] [ 21952/     0 kb/s] [eta 00m:07s]
Threads: 3: [VVV] [76.0% done] [ 20772/     0 kb/s] [eta 00m:06s]
Threads: 3: [VVV] [80.0% done] [ 20256/     0 kb/s] [eta 00m:05s]
Threads: 2: [EVV] [84.0% done] [ 18876/     0 kb/s] [eta 00m:04s]
fio: (groupid=0): err= 0: pid=24988
  read : io=100MiB, bw=4,941KiB/s, runt= 21220msec
    slat (msec): min=    0, max= 5429, avg=977.44, stdev=1682.57
    clat (msec): min=    0, max=  521, avg=75.47, stdev=65.76
    bw (KiB/s) : min=   56, max=11932, per=38.24%, avg=5530.62, stdev=2998.81
  write: io=24,192KiB, bw=1,490KiB/s, runt= 16622msec
    slat (msec): min=    0, max=    2, avg= 0.03, stdev= 0.18
    clat (msec): min=    0, max=  837, avg=80.52, stdev=107.01
    bw (KiB/s) : min=    0, max= 8055, per=55.50%, avg=3010.00, stdev=2890.00
  cpu          : usr=3.90%, sys=0.38%, ctx=1798
  IO depths    : 1=0.1%, 2=0.2%, 4=10.1%, 8=89.6%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=0.8%, 4=0.8%, 10=3.7%, 20=11.8%, 50=29.1%, 100=29.0%
     lat (msec): 250=21.3%, 500=2.9%, 750=0.3%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=24989
  read : io=100MiB, bw=4,821KiB/s, runt= 21750msec
    slat (msec): min=    0, max= 4539, avg=776.64, stdev=1335.88
    clat (msec): min=    3, max=  515, avg=75.43, stdev=69.55
    bw (KiB/s) : min=  102, max=11223, per=38.00%, avg=5496.32, stdev=2849.56
  write: io=29,984KiB, bw=1,784KiB/s, runt= 17207msec
    slat (msec): min=    0, max=    6, avg= 0.10, stdev= 0.50
    clat (msec): min=    0, max=  908, avg=88.78, stdev=92.58
    bw (KiB/s) : min=    0, max= 7397, per=58.82%, avg=3189.67, stdev=2619.69
  cpu          : usr=4.30%, sys=0.28%, ctx=1780
  IO depths    : 1=0.1%, 2=0.2%, 4=13.2%, 8=86.5%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=0.6%, 4=0.5%, 10=3.4%, 20=13.9%, 50=26.5%, 100=27.6%
     lat (msec): 250=24.3%, 500=2.9%, 750=0.2%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=24990
  read : io=100MiB, bw=4,839KiB/s, runt= 21665msec
    slat (msec): min=    0, max= 4765, avg=731.27, stdev=1371.19
    clat (msec): min=    0, max=  438, avg=72.38, stdev=64.24
    bw (KiB/s) : min=   56, max= 9704, per=36.46%, avg=5272.50, stdev=3018.77
  write: io=36,960KiB, bw=2,241KiB/s, runt= 16888msec
    slat (msec): min=    0, max=    3, avg= 0.08, stdev= 0.36
    clat (msec): min=    0, max=  833, avg=97.82, stdev=105.07
    bw (KiB/s) : min=    0, max= 8467, per=74.91%, avg=4062.24, stdev=2840.05
  cpu          : usr=4.32%, sys=0.26%, ctx=1795
  IO depths    : 1=0.1%, 2=0.3%, 4=11.6%, 8=87.9%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=0.4%, 4=0.3%, 10=3.0%, 20=13.3%, 50=28.6%, 100=28.3%
     lat (msec): 250=22.2%, 500=3.5%, 750=0.2%, 1000=0.0%, >=2000=0.0%

Run status group 0 (all jobs):
   READ: io=300MiB, aggrb=14,463KiB/s, minb=4,821KiB/s, maxb=4,941KiB/s, mint=21220msec, maxt=21750msec
  WRITE: io=91,136KiB, aggrb=5,423KiB/s, minb=1,490KiB/s, maxb=2,241KiB/s, mint=21220msec, maxt=21750msec

Disk stats (read/write):
  sda: ios=4620/1466, merge=1475/455, ticks=344688/169528, in_queue=514200, util=99.43%
2.77user 0.20system 0:21.97elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2379minor)pagefaults 0swaps