sequential_buffered: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 sequential_nonbuffered: (g=1): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 rand_buffered: (g=2): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 rand_nonbuffered: (g=3): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.39 Starting 4 processes sequential_buffered: (groupid=0, jobs=1): err= 0: pid=225892: Sun Apr 13 16:32:18 2025 read: IOPS=364k, BW=1422MiB/s (1491MB/s)(64.0MiB/45msec) clat (nsec): min=414, max=244928, avg=2552.07, stdev=17447.08 lat (nsec): min=434, max=244948, avg=2573.40, stdev=17447.28 clat percentiles (nsec): | 1.00th=[ 426], 5.00th=[ 434], 10.00th=[ 438], 20.00th=[ 446], | 30.00th=[ 450], 40.00th=[ 454], 50.00th=[ 462], 60.00th=[ 474], | 70.00th=[ 490], 80.00th=[ 506], 90.00th=[ 556], 95.00th=[ 652], | 99.00th=[118272], 99.50th=[154624], 99.90th=[211968], 99.95th=[224256], | 99.99th=[238592] lat (nsec) : 500=77.34%, 750=19.23%, 1000=1.57% lat (usec) : 2=0.21%, 4=0.06%, 10=0.01%, 20=0.01%, 50=0.12% lat (usec) : 100=0.31%, 250=1.15% cpu : usr=6.82%, sys=34.09%, ctx=246, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=16384,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 sequential_nonbuffered: (groupid=1, jobs=1): err= 0: pid=225893: Sun Apr 13 16:32:18 2025 read: IOPS=35.0k, BW=137MiB/s (143MB/s)(64.0MiB/468msec) clat (usec): min=10, max=310, avg=28.03, stdev=24.76 lat (usec): min=10, max=310, avg=28.07, stdev=24.76 clat percentiles (usec): | 1.00th=[ 11], 5.00th=[ 13], 10.00th=[ 13], 20.00th=[ 13], | 30.00th=[ 13], 40.00th=[ 13], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 25], 80.00th=[ 64], 90.00th=[ 68], 95.00th=[ 73], | 99.00th=[ 85], 99.50th=[ 122], 99.90th=[ 182], 99.95th=[ 194], | 99.99th=[ 239] lat (usec) : 20=65.66%, 50=11.58%, 100=22.11%, 250=0.65%, 500=0.01% cpu : usr=2.36%, sys=5.35%, ctx=16388, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=16384,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 rand_buffered: (groupid=2, jobs=1): err= 0: pid=225899: Sun Apr 13 16:32:18 2025 read: IOPS=14.8k, BW=57.9MiB/s (60.7MB/s)(64.0MiB/1105msec) clat (usec): min=11, max=740, avg=67.02, stdev=18.23 lat (usec): min=11, max=740, avg=67.06, stdev=18.23 clat percentiles (usec): | 1.00th=[ 12], 5.00th=[ 64], 10.00th=[ 65], 20.00th=[ 67], | 30.00th=[ 67], 40.00th=[ 67], 50.00th=[ 68], 60.00th=[ 68], | 70.00th=[ 72], 80.00th=[ 73], 90.00th=[ 74], 95.00th=[ 75], | 99.00th=[ 92], 99.50th=[ 106], 99.90th=[ 351], 99.95th=[ 429], | 99.99th=[ 586] bw ( KiB/s): min=54936, max=58928, per=95.99%, avg=56932.00, stdev=2822.77, samples=2 iops : min=13734, max=14732, avg=14233.00, stdev=705.69, samples=2 lat (usec) : 20=4.00%, 50=0.02%, 100=95.34%, 250=0.51%, 500=0.12% lat (usec) : 750=0.02% cpu : usr=0.91%, sys=3.71%, ctx=16386, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=16384,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 rand_nonbuffered: (groupid=3, jobs=1): err= 0: pid=225901: Sun Apr 13 16:32:18 2025 read: IOPS=15.1k, BW=59.1MiB/s (62.0MB/s)(64.0MiB/1082msec) clat (usec): min=10, max=161, avg=65.44, stdev=11.95 lat (usec): min=10, max=161, avg=65.48, stdev=11.95 clat percentiles (usec): | 1.00th=[ 11], 5.00th=[ 63], 10.00th=[ 64], 20.00th=[ 65], | 30.00th=[ 65], 40.00th=[ 66], 50.00th=[ 67], 60.00th=[ 68], | 70.00th=[ 71], 80.00th=[ 72], 90.00th=[ 73], 95.00th=[ 74], | 99.00th=[ 86], 99.50th=[ 94], 99.90th=[ 112], 99.95th=[ 115], | 99.99th=[ 153] bw ( KiB/s): min=60152, max=60984, per=100.00%, avg=60568.00, stdev=588.31, samples=2 iops : min=15038, max=15246, avg=15142.00, stdev=147.08, samples=2 lat (usec) : 20=4.01%, 50=0.02%, 100=95.66%, 250=0.31% cpu : usr=1.20%, sys=2.41%, ctx=16388, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=16384,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=1422MiB/s (1491MB/s), 1422MiB/s-1422MiB/s (1491MB/s-1491MB/s), io=64.0MiB (67.1MB), run=45-45msec Run status group 1 (all jobs): READ: bw=137MiB/s (143MB/s), 137MiB/s-137MiB/s (143MB/s-143MB/s), io=64.0MiB (67.1MB), run=468-468msec Run status group 2 (all jobs): READ: bw=57.9MiB/s (60.7MB/s), 57.9MiB/s-57.9MiB/s (60.7MB/s-60.7MB/s), io=64.0MiB (67.1MB), run=1105-1105msec Run status group 3 (all jobs): READ: bw=59.1MiB/s (62.0MB/s), 59.1MiB/s-59.1MiB/s (62.0MB/s-62.0MB/s), io=64.0MiB (67.1MB), run=1082-1082msec Disk stats (read/write): nvme1n1: ios=60308/601, sectors=612976/18856, merge=67/21, ticks=3076/2672, in_queue=5750, util=75.21%