windows fio

C:\Users\user\Documents\fio>fio-3.25-x86-windows.exe --name=4Gread --rw=read --direct=1 --iodepth=1 --blocksize=1024kb --size=4g --ioengine=windowsaio --filename=z:z:z --numjobs=12 -group_reporting   --runtime=600

fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4Gread: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
...
fio-3.25
Starting 12 threads
Jobs: 12 (f=33): [R(3),f(1),R(2),f(1),R(1),f(1),R(2),f(1)][100.0%][r=529MiB/s][r=528 IOPS][eta 00m:00s]
4Gread: (groupid=0, jobs=12): err= 0: pid=14768: Thu Oct 6 16:45:34 2022
  read: IOPS=533, BW=534MiB/s (560MB/s)(48.0GiB/92064msec)
    slat (usec): min=26, max=1064, avg=55.06, stdev=14.71
    clat (usec): min=2422, max=38626, avg=22393.81, stdev=710.96
     lat (usec): min=2466, max=38664, avg=22448.86, stdev=711.18
    clat percentiles (usec):
     |  1.00th=[22152],  5.00th=[22152], 10.00th=[22414], 20.00th=[22414],
     | 30.00th=[22414], 40.00th=[22414], 50.00th=[22414], 60.00th=[22414],
     | 70.00th=[22414], 80.00th=[22414], 90.00th=[22414], 95.00th=[22414],
     | 99.00th=[22676], 99.50th=[26870], 99.90th=[30540], 99.95th=[32113],
     | 99.99th=[36963]
   bw (  KiB/s): min=487416, max=572007, per=100.00%, avg=547385.48, stdev=1013.73, samples=2160
   iops        : min=  468, max=  552, avg=530.09, stdev= 1.04, samples=2160
  lat (msec)   : 4=0.01%, 10=0.03%, 20=0.23%, 50=99.73%
  cpu          : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=49152,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=534MiB/s (560MB/s), 534MiB/s-534MiB/s (560MB/s-560MB/s), io=48.0GiB (51.5GB), run=92064-92064msec

C:\Users\user\Documents\fio>fio-3.25-x86-windows.exe --name=4Gread --rw=write --direct=1 --iodepth=1 --blocksize=1024kb --size=4g --ioengine=windowsaio --filename=z:z --numjobs=4 -group_reporting   --runtime=600

fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4Gread: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=windowsaio, iodepth=1
...
fio-3.25
Starting 4 threads
Jobs: 4 (f=8): [W(4)][100.0%][w=319MiB/s][w=319 IOPS][eta 00m:00s]
4Gread: (groupid=0, jobs=4): err= 0: pid=2576: Thu Oct 6 16:46:36 2022
  write: IOPS=406, BW=407MiB/s (426MB/s)(16.0GiB/40301msec); 0 zone resets
    slat (usec): min=57, max=5236, avg=121.98, stdev=90.38
    clat (usec): min=1968, max=75228, avg=9711.80, stdev=4444.87
     lat (usec): min=2361, max=75335, avg=9833.78, stdev=4446.57
    clat percentiles (usec):
     |  1.00th=[ 7635],  5.00th=[ 7701], 10.00th=[ 7701], 20.00th=[ 7701],
     | 30.00th=[ 7767], 40.00th=[ 7767], 50.00th=[ 7832], 60.00th=[ 8029],
     | 70.00th=[ 8586], 80.00th=[ 9896], 90.00th=[15008], 95.00th=[20317],
     | 99.00th=[27657], 99.50th=[30016], 99.90th=[38536], 99.95th=[45351],
     | 99.99th=[74974]
   bw (  KiB/s): min=269576, max=522076, per=100.00%, avg=416422.44, stdev=14059.10, samples=316
   iops        : min=  260, max=  506, avg=404.61, stdev=13.75, samples=316
  lat (msec)   : 2=0.01%, 4=0.01%, 10=80.60%, 20=14.18%, 50=5.16%
  lat (msec)   : 100=0.05%
  cpu          : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,16384,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=407MiB/s (426MB/s), 407MiB/s-407MiB/s (426MB/s-426MB/s), io=16.0GiB (17.2GB), run=40301-40301msec