fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
...
fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
Starting 3 threads
Threads: 3
Threads: 3
Threads: 3: [mmm] [14.3% done] [  7736/ 18241 kb/s] [eta 00m:18s]
Threads: 3: [mmm] [19.0% done] [ 15530/  8196 kb/s] [eta 00m:17s]
Threads: 3: [mmm] [23.8% done] [ 11950/ 10333 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [28.6% done] [ 14700/  7118 kb/s] [eta 00m:15s]
Threads: 3: [mmm] [33.3% done] [ 13988/  6768 kb/s] [eta 00m:14s]
Threads: 3: [mmm] [38.1% done] [ 14701/  4945 kb/s] [eta 00m:13s]
Threads: 3: [mmm] [42.9% done] [  8740/  9321 kb/s] [eta 00m:12s]
Threads: 3: [mmm] [45.5% done] [ 10630/  5762 kb/s] [eta 00m:12s]
Threads: 3: [mmm] [50.0% done] [  8439/  7094 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [52.2% done] [  8476/  6054 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [56.5% done] [  6513/  7489 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [58.3% done] [  8427/  4295 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [60.0% done] [  6972/  4477 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [61.5% done] [  8106/  2629 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [68.0% done] [ 10776/  1007 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [72.0% done] [ 21028/   203 kb/s] [eta 00m:07s]
Threads: 3: [VVV] [76.0% done] [ 24283/     0 kb/s] [eta 00m:06s]
Threads: 3: [VVV] [83.3% done] [ 22645/     0 kb/s] [eta 00m:04s]
Threads: 3: [VVV] [87.5% done] [ 22044/     0 kb/s] [eta 00m:03s]
Threads: 2: [_VV] [91.7% done] [ 22775/     0 kb/s] [eta 00m:02s]
fio: (groupid=0): err= 0: pid=23063
  read : io=100MiB, bw=4,906KiB/s, runt= 21373msec
    slat (msec): min=    0, max= 4162, avg=507.32, stdev=1074.47
    clat (msec): min=    3, max=  548, avg=74.73, stdev=68.84
    bw (KiB/s) : min=  107, max=10783, per=36.92%, avg=5069.94, stdev=2647.92
  write: io=32,692KiB, bw=1,948KiB/s, runt= 17177msec
    slat (msec): min=    0, max=    3, avg= 0.08, stdev= 0.32
    clat (msec): min=    0, max=  808, avg=108.89, stdev=129.79
    bw (KiB/s) : min=    0, max= 8134, per=47.24%, avg=3587.65, stdev=3074.56
  cpu          : usr=4.44%, sys=0.30%, ctx=1791
  IO depths    : 1=0.1%, 2=0.2%, 4=9.6%, 8=90.1%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=1.4%, 4=0.3%, 10=3.4%, 20=12.6%, 50=26.1%, 100=30.4%
     lat (msec): 250=20.3%, 500=4.8%, 750=0.3%, 1000=0.2%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=23064
  read : io=100MiB, bw=4,666KiB/s, runt= 22470msec
    slat (msec): min=    0, max= 6533, avg=1640.46, stdev=2216.72
    clat (msec): min=    0, max=  411, avg=67.25, stdev=65.96
    bw (KiB/s) : min=   40, max=13827, per=39.55%, avg=5432.11, stdev=3922.11
  write: io=46,588KiB, bw=2,991KiB/s, runt= 15949msec
    slat (msec): min=    0, max=   14, avg= 0.13, stdev= 0.85
    clat (msec): min=    0, max=  827, avg=88.18, stdev=103.05
    bw (KiB/s) : min=    0, max=11225, per=51.37%, avg=3901.83, stdev=2928.91
  cpu          : usr=4.50%, sys=0.50%, ctx=1806
  IO depths    : 1=0.1%, 2=0.3%, 4=16.2%, 8=83.4%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=2.5%, 4=0.8%, 10=4.3%, 20=14.2%, 50=27.8%, 100=28.6%
     lat (msec): 250=17.3%, 500=4.0%, 750=0.4%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=23065
  read : io=100MiB, bw=4,578KiB/s, runt= 22904msec
    slat (msec): min=    0, max= 6318, avg=1853.54, stdev=2255.41
    clat (msec): min=    0, max=  600, avg=66.81, stdev=71.79
    bw (KiB/s) : min=   30, max=11815, per=34.88%, avg=4790.61, stdev=3298.80
  write: io=48,136KiB, bw=2,896KiB/s, runt= 17017msec
    slat (msec): min=    0, max=    2, avg= 0.04, stdev= 0.24
    clat (msec): min=    0, max=  833, avg=88.05, stdev=109.02
    bw (KiB/s) : min=    0, max=15080, per=43.41%, avg=3297.15, stdev=3273.50
  cpu          : usr=4.77%, sys=0.33%, ctx=1868
  IO depths    : 1=0.2%, 2=0.5%, 4=15.4%, 8=84.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=2.6%, 4=0.7%, 10=6.6%, 20=16.4%, 50=26.7%, 100=23.8%
     lat (msec): 250=18.3%, 500=4.2%, 750=0.5%, 1000=0.1%, >=2000=0.0%

Run status group 0 (all jobs):
   READ: io=300MiB, aggrb=13,734KiB/s, minb=4,578KiB/s, maxb=4,906KiB/s, mint=21373msec, maxt=22904msec
  WRITE: io=124MiB, aggrb=7,595KiB/s, minb=1,948KiB/s, maxb=2,991KiB/s, mint=21373msec, maxt=22904msec

Disk stats (read/write):
  sda: ios=4461/1890, merge=1506/379, ticks=306044/309608, in_queue=615640, util=99.40%
3.11user 0.27system 0:23.15elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2398minor)pagefaults 0swaps