fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
...
fio: (g=0): rw=randrw, bs=4K-256K/4K-256K, ioengine=libaio, iodepth=8
Starting 3 threads
Threads: 3
Threads: 3
Threads: 3: [mFm] [7.7% done] [  7777/ 40496 kb/s] [eta 00m:36s]
Threads: 3: [mFm] [8.3% done] [  1483/ 19403 kb/s] [eta 00m:44s]
Threads: 3: [FFm] [8.3% done] [     0/ 23420 kb/s] [eta 00m:55s]
Threads: 3: [FFm] [10.0% done] [     0/     0 kb/s] [eta 00m:54s]
Threads: 3: [FFm] [11.7% done] [     0/     0 kb/s] [eta 00m:53s]
Threads: 3: [FFm] [13.3% done] [     0/     0 kb/s] [eta 00m:52s]
Threads: 3: [FFm] [15.0% done] [     0/     0 kb/s] [eta 00m:51s]
Threads: 3: [FFF] [100.0% done] [  1145/ 89990 kb/s] [eta 00m:00s]
Threads: 3: [FFF] [100.0% done] [     0/     0 kb/s] [eta 00m:00s]
Threads: 3: [FFF] [100.0% done] [     0/     0 kb/s] [eta 00m:00s]
Threads: 3: [FFF] [100.0% done] [     0/     0 kb/s] [eta 00m:00s]
Threads: 3: [FFF] [100.0% done] [     0/     0 kb/s] [eta 00m:00s]
Threads: 3: [FFF] [100.0% done] [     0/     0 kb/s] [eta 00m:00s]
Threads: 3: [VVV] [66.7% done] [  1840/     0 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [68.0% done] [ 19155/     0 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [69.2% done] [ 36441/     0 kb/s] [eta 00m:08s]
Threads: 3: [VVV] [67.9% done] [ 28456/     0 kb/s] [eta 00m:09s]
Threads: 3: [VVV] [74.1% done] [ 31134/     0 kb/s] [eta 00m:07s]
Threads: 3: [VVV] [80.8% done] [ 29143/     0 kb/s] [eta 00m:05s]
Threads: 3: [VVV] [84.6% done] [ 29009/     0 kb/s] [eta 00m:04s]
Threads: 3: [VVV] [92.0% done] [ 35027/     0 kb/s] [eta 00m:02s]
Threads: 1: [VEE] [100.0% done] [ 33239/     0 kb/s] [eta 00m:00s]
fio: (groupid=0): err= 0: pid=17760
  read : io=100MiB, bw=4,171KiB/s, runt= 25134msec
    slat (msec): min=  102, max=21031, avg=17945.03, stdev=4722.90
    clat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    bw (KiB/s) : min=    0, max=38123, per=68.18%, avg=8532.28, stdev=10107.37
  write: io=95,512KiB, bw=6,235KiB/s, runt= 15684msec
    slat (msec): min=    0, max=   16, avg= 1.88, stdev= 2.26
    clat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    bw (KiB/s) : min=    0, max=14718, per=40.35%, avg=7359.00, stdev=10407.20
  cpu          : usr=4.84%, sys=1.48%, ctx=1010
  IO depths    : 1=12.6%, 2=25.0%, 4=49.9%, 8=12.5%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=99.9%, 4=0.0%, 10=0.0%, 20=0.0%, 50=0.0%, 100=0.0%
     lat (msec): 250=0.0%, 500=0.0%, 750=0.0%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=17761
  read : io=100MiB, bw=4,458KiB/s, runt= 23518msec
    slat (msec): min=  123, max=21032, avg=16601.86, stdev=3885.60
    clat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    bw (KiB/s) : min=   18, max=22136, per=80.50%, avg=10074.50, stdev=6128.90
  write: io=95,744KiB, bw=6,257KiB/s, runt= 15667msec
    slat (msec): min=    0, max=  148, avg= 1.74, stdev= 4.01
    clat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    bw (KiB/s) : min=    0, max=88600, per=208.01%, avg=37936.33, stdev=45650.62
  cpu          : usr=5.27%, sys=1.50%, ctx=943
  IO depths    : 1=12.6%, 2=25.0%, 4=50.0%, 8=12.5%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=99.9%, 4=0.0%, 10=0.0%, 20=0.0%, 50=0.0%, 100=0.0%
     lat (msec): 250=0.0%, 500=0.0%, 750=0.0%, 1000=0.0%, >=2000=0.0%
fio: (groupid=0): err= 0: pid=17762
  read : io=100MiB, bw=4,363KiB/s, runt= 24030msec
    slat (msec): min=  175, max=14262, avg=9976.81, stdev=3200.01
    clat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    bw (KiB/s) : min=   18, max=19928, per=66.44%, avg=8315.11, stdev=6460.48
  write: io=88,584KiB, bw=5,773KiB/s, runt= 15712msec
    slat (msec): min=    0, max=   10, avg= 1.53, stdev= 1.70
    clat (msec): min=    0, max=    0, avg= 0.00, stdev= 0.00
    bw (KiB/s) : min=    0, max=117692, per=322.66%, avg=58846.00, stdev=83220.81
  cpu          : usr=4.71%, sys=1.25%, ctx=1094
  IO depths    : 1=12.6%, 2=25.0%, 4=50.0%, 8=12.4%, 16=0.0%, 32=0.0%, >=64=0.0%
     lat (msec): 2=99.9%, 4=0.0%, 10=0.0%, 20=0.0%, 50=0.0%, 100=0.0%
     lat (msec): 250=0.0%, 500=0.0%, 750=0.0%, 1000=0.0%, >=2000=0.0%

Run status group 0 (all jobs):
   READ: io=300MiB, aggrb=12,515KiB/s, minb=4,171KiB/s, maxb=4,458KiB/s, mint=23518msec, maxt=25134msec
  WRITE: io=273MiB, aggrb=18,238KiB/s, minb=5,773KiB/s, maxb=6,257KiB/s, mint=23518msec, maxt=25134msec

Disk stats (read/write):
  sda: ios=3397/2637, merge=1264/67208, ticks=94368/1786676, in_queue=1881140, util=97.95%
3.64user 1.10system 0:25.87elapsed 18%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2430minor)pagefaults 0swaps