fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
...
fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
Starting 3 threads
Threads: 3
Threads: 3
Threads: 3: [mmm] [14.3% done] [ 20620/  4588 kb/s] [eta 00m:18s]
Threads: 3: [mmm] [19.0% done] [ 10088/ 13124 kb/s] [eta 00m:17s]
Threads: 3: [mmm] [23.8% done] [ 17204/  5924 kb/s] [eta 00m:16s]
Threads: 3: [mmm] [28.6% done] [ 14292/  8752 kb/s] [eta 00m:15s]
Threads: 3: [mmm] [33.3% done] [ 11060/  8988 kb/s] [eta 00m:14s]
Threads: 3: [mmm] [38.1% done] [ 15104/  4628 kb/s] [eta 00m:13s]
Threads: 3: [mmm] [42.9% done] [  9780/  8908 kb/s] [eta 00m:12s]
Threads: 3: [mmm] [45.5% done] [  1892/ 14180 kb/s] [eta 00m:12s]
Threads: 3: [mmm] [50.0% done] [  9372/  7840 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [52.2% done] [ 11784/  1132 kb/s] [eta 00m:11s]
Threads: 3: [mmm] [56.5% done] [  9460/  3744 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [58.3% done] [ 10608/  1944 kb/s] [eta 00m:10s]
Threads: 3: [mmm] [62.5% done] [  7284/  4940 kb/s] [eta 00m:09s]
Threads: 3: [mmm] [64.0% done] [  3636/  6392 kb/s] [eta 00m:09s]
Threads: 3: [mmm] [65.4% done] [  7384/  2772 kb/s] [eta 00m:09s]
Threads: 3: [VVV] [72.0% done] [ 18400/    72 kb/s] [eta 00m:07s]
Threads: 3: [VVV] [76.0% done] [ 21220/     0 kb/s] [eta 00m:06s]
Threads: 3: [VVV] [80.0% done] [ 20088/     0 kb/s] [eta 00m:05s]
Threads: 3: [VVV] [84.0% done] [ 20880/     0 kb/s] [eta 00m:04s]
Threads: 2: [VVE] [88.0% done] [ 20780/     0 kb/s] [eta 00m:03s]
fio: (groupid=0): err= 0: pid=2261
  read : io=100MiB, bw=4,547KiB/s, runt= 23060msec
    slat (msec): min=    0, max= 5495, avg=1404.57, stdev=1911.86
    clat (msec): min=    3, max=  564, avg=66.96, stdev=67.19
    bw (KiB/s) : min=   11, max=18610, per=37.06%, avg=5055.17, stdev=3842.37
  write: io=45,840KiB, bw=2,670KiB/s, runt= 17575msec
    slat (msec): min=    0, max=    9, avg= 0.09, stdev= 0.59
    clat (msec): min=    0, max=  844, avg=88.14, stdev=93.59
    bw (KiB/s) : min=    0, max= 8380, per=51.27%, avg=3365.12, stdev=2767.33
  cpu          : usr=3.95%, sys=0.29%, ctx=1888
  IO depths    : 1=0.1%, 2=0.2%, 4=17.0%, 8=82.7%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=1.2%, 4=0.5%, 10=5.9%, 20=16.5%, 50=25.5%, 100=26.0%
     lat (msec): 250=21.0%, 500=3.0%, 750=0.3%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=2262
  read : io=100MiB, bw=4,600KiB/s, runt= 22795msec
    slat (msec): min=    0, max= 6225, avg=1314.30, stdev=1952.78
    clat (msec): min=    0, max=  476, avg=75.36, stdev=68.36
    bw (KiB/s) : min=   71, max=10885, per=37.37%, avg=5098.03, stdev=3015.18
  write: io=34,520KiB, bw=2,127KiB/s, runt= 16616msec
    slat (msec): min=    0, max=    6, avg= 0.09, stdev= 0.52
    clat (msec): min=    0, max=  796, avg=77.40, stdev=79.85
    bw (KiB/s) : min=    0, max=10424, per=51.06%, avg=3351.11, stdev=3002.30
  cpu          : usr=4.18%, sys=0.39%, ctx=1820
  IO depths    : 1=0.1%, 2=0.4%, 4=14.3%, 8=85.2%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=1.1%, 4=0.6%, 10=3.5%, 20=12.6%, 50=28.2%, 100=28.3%
     lat (msec): 250=22.8%, 500=2.6%, 750=0.1%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=2263
  read : io=100MiB, bw=4,706KiB/s, runt= 22278msec
    slat (msec): min=    0, max= 5169, avg=884.49, stdev=1513.94
    clat (msec): min=    1, max=  537, avg=74.73, stdev=69.26
    bw (KiB/s) : min=   28, max=10644, per=38.22%, avg=5214.24, stdev=3034.68
  write: io=32,296KiB, bw=1,935KiB/s, runt= 17090msec
    slat (msec): min=    0, max=    4, avg= 0.05, stdev= 0.29
    clat (msec): min=    0, max=  835, avg=91.28, stdev=95.22
    bw (KiB/s) : min=    0, max= 7230, per=52.26%, avg=3429.61, stdev=2609.79
  cpu          : usr=3.97%, sys=0.34%, ctx=1788
  IO depths    : 1=0.1%, 2=0.3%, 4=13.0%, 8=86.6%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=0.9%, 4=0.5%, 10=2.9%, 20=12.8%, 50=28.7%, 100=27.6%
     lat (msec): 250=22.9%, 500=3.2%, 750=0.3%, 1000=0.0%, >=2000=0.0%

Run status group 0 (all jobs):
   READ: io=300MiB, aggrb=13,641KiB/s, minb=4,547KiB/s, maxb=4,706KiB/s, mint=22278msec, maxt=23060msec
  WRITE: io=110MiB, aggrb=6,563KiB/s, minb=1,935KiB/s, maxb=2,670KiB/s, mint=22278msec, maxt=23060msec

Disk stats (read/write):
  sda: ios=4529/1874, merge=1609/499, ticks=329704/176248, in_queue=505952, util=99.37%
2.83user 0.33system 0:23.36elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2375minor)pagefaults 0swaps