Benchmark JuiceFS at AWS 2

Deployed JuiceFS to self-managed K3s cluster with mountOptions

- cache-size=0

to disable any cache.

Use t3a.medium to do the test.

Sequential Read 512M 1 Job

# fio --name=sequential-read --directory=. --rw=read --refill_buffers --bs=4M --size=512M

sequential-read: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
fio-3.7
Starting 1 process
sequential-read: Laying out IO file (1 file / 512MiB)
Jobs: 1 (f=1)
sequential-read: (groupid=0, jobs=1): err= 0: pid=181: Fri Dec 23 04:35:19 2022
read: IOPS=60, BW=241MiB/s (252MB/s)(512MiB/2127msec)
clat (usec): min=3663, max=98850, avg=16572.69, stdev=21702.62
lat (usec): min=3664, max=98851, avg=16576.32, stdev=21702.17
clat percentiles (usec):
| 1.00th=[ 3687], 5.00th=[ 3982], 10.00th=[ 4047], 20.00th=[ 4490],
| 30.00th=[ 4817], 40.00th=[ 5211], 50.00th=[ 6325], 60.00th=[ 8291],
| 70.00th=[10290], 80.00th=[19268], 90.00th=[59507], 95.00th=[67634],
| 99.00th=[84411], 99.50th=[99091], 99.90th=[99091], 99.95th=[99091],
| 99.99th=[99091]
bw ( KiB/s): min=196608, max=294912, per=96.34%, avg=237465.75, stdev=45953.47, samples=4
iops : min= 48, max= 72, avg=57.75, stdev=11.44, samples=4
lat (msec) : 4=5.47%, 10=64.06%, 20=10.94%, 50=7.03%, 100=12.50%
cpu : usr=0.00%, sys=10.87%, ctx=2506, majf=0, minf=1033
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=128,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: bw=241MiB/s (252MB/s), 241MiB/s-241MiB/s (252MB/s-252MB/s), io=512MiB (537MB), run=2127-2127msec

Sequential Write 512M 1 Job

# fio --name=sequential-write --directory=. --rw=write --refill_buffers --bs=4M --size=512M

equential-write: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [W(1)][-.-%][r=0KiB/s,w=187MiB/s][r=0,w=46 IOPS][eta 00m:00s]
sequential-read: (groupid=0, jobs=1): err= 0: pid=187: Fri Dec 23 04:35:46 2022
write: IOPS=48, BW=194MiB/s (203MB/s)(512MiB/2641msec)
clat (usec): min=3425, max=60858, avg=15901.72, stdev=13752.35
lat (usec): min=3426, max=60859, avg=15903.28, stdev=13752.38
clat percentiles (usec):
| 1.00th=[ 3425], 5.00th=[ 3654], 10.00th=[ 3851], 20.00th=[ 4359],
| 30.00th=[ 5211], 40.00th=[ 6652], 50.00th=[11207], 60.00th=[17433],
| 70.00th=[20841], 80.00th=[25297], 90.00th=[33162], 95.00th=[44827],
| 99.00th=[57410], 99.50th=[61080], 99.90th=[61080], 99.95th=[61080],
| 99.99th=[61080]
bw ( KiB/s): min=180224, max=204800, per=98.17%, avg=194891.00, stdev=8957.65, samples=5
iops : min= 44, max= 50, avg=47.40, stdev= 2.19, samples=5
lat (msec) : 4=13.28%, 10=35.16%, 20=17.97%, 50=29.69%, 100=3.91%
cpu : usr=5.80%, sys=14.62%, ctx=4628, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,128,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: bw=194MiB/s (203MB/s), 194MiB/s-194MiB/s (203MB/s-203MB/s), io=512MiB (537MB), run=2641-2641msec

The performance is quite promising.