mbox series

[0/2] brd: small improvements

Message ID 20230801022929.8972-1-kch@nvidia.com (mailing list archive)
Headers show
Series brd: small improvements | expand

Message

Chaitanya Kulkarni Aug. 1, 2023, 2:29 a.m. UTC 
Hi,

Small improvements for brd to remove deprecated APIs and add pr_fmt
macro to remove the repetative brd prefix in the pr_xxx() string.

I've ran few fio read/write/verify tests on this, they seems to pass.

-ck

Chaitanya Kulkarni (2):
  brd: use memcpy_to|from_page() in copy_to|from_brd()
  brd: remove redundent prefix with pr_fmt macro

 drivers/block/brd.c | 37 +++++++++++++++----------------------
 1 file changed, 15 insertions(+), 22 deletions(-)

linux-block # sh test-brd-memcpy-perf.sh 
+ ./compile_brd.sh
+ umount /mnt/brd
umount: /mnt/brd: not mounted.
+ dmesg -c
+ modprobe -r brd
+ lsmod
+ grep brd
++ nproc
+ make -j 48 M=drivers/block modules
+ HOST=drivers/block/brd.ko
++ uname -r
+ HOST_DEST=/lib/modules/6.5.0-rc3lblk+/kernel/drivers/block/
+ cp drivers/block/brd.ko /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//
+ ls -lrth /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
-rw-r--r--. 1 root root 378K Jul 31 19:13 /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
+ dmesg -c
+ lsmod
+ grep brd
+ modprobe brd rd_nr=1 rd_size=20971520
+ sleep 1
+ dmesg -c
[12164.622572] brd: module loaded
+ test_perf
+ git log --oneline -2
3dbc8dd05802 (HEAD -> for-next) brd: remove redundent prefix with pr_fmt macro
5644af5fe8d1 brd: use memcpy_to|from_page() in copy_to|from_brd()
+ ./compile_brd.sh
+ umount /mnt/brd
umount: /mnt/brd: not mounted.
+ dmesg -c
+ modprobe -r brd
+ lsmod
+ grep brd
++ nproc
+ make -j 48 M=drivers/block modules
+ HOST=drivers/block/brd.ko
++ uname -r
+ HOST_DEST=/lib/modules/6.5.0-rc3lblk+/kernel/drivers/block/
+ cp drivers/block/brd.ko /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//
+ ls -lrth /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
-rw-r--r--. 1 root root 378K Jul 31 19:13 /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
+ dmesg -c
[12165.683470] brd: module unloaded
+ lsmod
+ grep brd
+ modprobe brd rd_nr=1 rd_size=20971520
+ sleep 1
+ fio fio/randwrite.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDWRITE: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [w(48)][100.0%][w=26.5GiB/s][w=6948k IOPS][eta 00m:00s]
RANDWRITE: (groupid=0, jobs=48): err= 0: pid=6654: Mon Jul 31 19:14:50 2023
  write: IOPS=6580k, BW=25.1GiB/s (27.0GB/s)(1506GiB/60002msec); 0 zone resets
    slat (nsec): min=992, max=48184k, avg=6291.87, stdev=44125.08
    clat (nsec): min=741, max=48194k, avg=7820.21, stdev=48267.24
     lat (usec): min=2, max=48215, avg=14.11, stdev=65.65
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    5], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    6], 40.00th=[    7], 50.00th=[    7], 60.00th=[    7],
     | 70.00th=[    8], 80.00th=[    8], 90.00th=[    9], 95.00th=[   10],
     | 99.00th=[   26], 99.50th=[   64], 99.90th=[  137], 99.95th=[  404],
     | 99.99th=[  594]
   bw (  MiB/s): min= 4967, max=30851, per=100.00%, avg=25716.15, stdev=107.71, samples=5712
   iops        : min=1271674, max=7898038, avg=6583332.27, stdev=27573.66, samples=5712
  lat (nsec)   : 750=0.01%, 1000=0.01%
  lat (usec)   : 2=0.01%, 4=0.93%, 10=94.93%, 20=3.07%, 50=0.34%
  lat (usec)   : 100=0.58%, 250=0.07%, 500=0.07%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=12.86%, sys=84.29%, ctx=26503, majf=0, minf=627
  IO depths    : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,394811720,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
  WRITE: bw=25.1GiB/s (27.0GB/s), 25.1GiB/s-25.1GiB/s (27.0GB/s-27.0GB/s), io=1506GiB (1617GB), run=60002-60002msec

Disk stats (read/write):
  ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ for i in 1 2 3
+ fio fio/randread.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDREAD: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [r(48)][100.0%][r=36.4GiB/s][r=9542k IOPS][eta 00m:00s]
RANDREAD: (groupid=0, jobs=48): err= 0: pid=6704: Mon Jul 31 19:15:50 2023
  read: IOPS=9635k, BW=36.8GiB/s (39.5GB/s)(2205GiB/60002msec)
    slat (nsec): min=1342, max=64089k, avg=4179.67, stdev=20589.24
    clat (nsec): min=771, max=64095k, avg=5428.32, stdev=23628.88
     lat (usec): min=2, max=64107, avg= 9.61, stdev=31.37
    clat percentiles (nsec):
     |  1.00th=[ 3440],  5.00th=[ 3728], 10.00th=[ 3920], 20.00th=[ 4192],
     | 30.00th=[ 4384], 40.00th=[ 4640], 50.00th=[ 4896], 60.00th=[ 5216],
     | 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7456],
     | 99.00th=[ 9536], 99.50th=[34048], 99.90th=[41216], 99.95th=[43264],
     | 99.99th=[66048]
   bw (  MiB/s): min=28759, max=39419, per=100.00%, avg=37689.01, stdev=32.63, samples=5712
   iops        : min=7362346, max=10091438, avg=9648387.52, stdev=8354.07, samples=5712
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 4=12.70%, 10=86.45%, 20=0.17%, 50=0.66%
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  lat (msec)   : 100=0.01%
  cpu          : usr=14.07%, sys=84.64%, ctx=27355, majf=0, minf=732
  IO depths    : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=578147226,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
   READ: bw=36.8GiB/s (39.5GB/s), 36.8GiB/s-36.8GiB/s (39.5GB/s-39.5GB/s), io=2205GiB (2368GB), run=60002-60002msec

Disk stats (read/write):
  ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ for i in 1 2 3
+ fio fio/randread.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDREAD: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [r(48)][100.0%][r=36.5GiB/s][r=9568k IOPS][eta 00m:00s]
RANDREAD: (groupid=0, jobs=48): err= 0: pid=6772: Mon Jul 31 19:16:50 2023
  read: IOPS=9544k, BW=36.4GiB/s (39.1GB/s)(2185GiB/60005msec)
    slat (nsec): min=1292, max=52199k, avg=4216.56, stdev=30223.82
    clat (nsec): min=761, max=52205k, avg=5481.37, stdev=34604.93
     lat (usec): min=2, max=52218, avg= 9.70, stdev=46.06
    clat percentiles (nsec):
     |  1.00th=[ 3440],  5.00th=[ 3728], 10.00th=[ 3920], 20.00th=[ 4192],
     | 30.00th=[ 4384], 40.00th=[ 4640], 50.00th=[ 4896], 60.00th=[ 5216],
     | 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7392],
     | 99.00th=[ 9408], 99.50th=[35584], 99.90th=[41216], 99.95th=[42752],
     | 99.99th=[80384]
   bw (  MiB/s): min=25868, max=40243, per=100.00%, avg=37331.79, stdev=46.77, samples=5712
   iops        : min=6622274, max=10302360, avg=9556939.51, stdev=11972.54, samples=5712
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 4=12.55%, 10=86.62%, 20=0.15%, 50=0.66%
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  lat (msec)   : 100=0.01%
  cpu          : usr=12.19%, sys=85.45%, ctx=25680, majf=0, minf=686
  IO depths    : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=572715634,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
   READ: bw=36.4GiB/s (39.1GB/s), 36.4GiB/s-36.4GiB/s (39.1GB/s-39.1GB/s), io=2185GiB (2346GB), run=60005-60005msec

Disk stats (read/write):
  ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ for i in 1 2 3
+ fio fio/randread.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDREAD: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [r(48)][100.0%][r=36.4GiB/s][r=9549k IOPS][eta 00m:00s]
RANDREAD: (groupid=0, jobs=48): err= 0: pid=6823: Mon Jul 31 19:17:51 2023
  read: IOPS=9544k, BW=36.4GiB/s (39.1GB/s)(2185GiB/60003msec)
    slat (nsec): min=1332, max=40081k, avg=4213.58, stdev=30581.29
    clat (nsec): min=761, max=40087k, avg=5479.64, stdev=34518.58
     lat (usec): min=2, max=40100, avg= 9.69, stdev=46.18
    clat percentiles (nsec):
     |  1.00th=[ 3472],  5.00th=[ 3760], 10.00th=[ 3952], 20.00th=[ 4192],
     | 30.00th=[ 4384], 40.00th=[ 4640], 50.00th=[ 4896], 60.00th=[ 5216],
     | 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7392],
     | 99.00th=[ 9408], 99.50th=[35584], 99.90th=[40704], 99.95th=[42240],
     | 99.99th=[82432]
   bw (  MiB/s): min=25167, max=40648, per=100.00%, avg=37327.34, stdev=47.56, samples=5712
   iops        : min=6442770, max=10406041, avg=9555798.60, stdev=12175.34, samples=5712
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 4=12.35%, 10=86.82%, 20=0.16%, 50=0.65%
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=12.36%, sys=85.26%, ctx=25612, majf=0, minf=774
  IO depths    : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=572691650,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
   READ: bw=36.4GiB/s (39.1GB/s), 36.4GiB/s-36.4GiB/s (39.1GB/s-39.1GB/s), io=2185GiB (2346GB), run=60003-60003msec

Disk stats (read/write):
  ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ fio fio/verify.fio --filename=/dev/ram0 --ioengine=io_uring --size=10G
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16
fio-3.34
Starting 1 process
Jobs: 1 (f=1): [V(1)][77.8%][r=1829MiB/s][r=468k IOPS][eta 00m:04s]                   
write-and-verify: (groupid=0, jobs=1): err= 0: pid=6876: Mon Jul 31 19:18:06 2023
  read: IOPS=468k, BW=1827MiB/s (1916MB/s)(6472MiB/3542msec)
    slat (nsec): min=1022, max=96523, avg=1313.44, stdev=369.23
    clat (nsec): min=711, max=128955, avg=32147.10, stdev=2033.64
     lat (nsec): min=1974, max=130207, avg=33460.54, stdev=2081.94
    clat percentiles (nsec):
     |  1.00th=[30592],  5.00th=[31104], 10.00th=[31104], 20.00th=[31360],
     | 30.00th=[31360], 40.00th=[31616], 50.00th=[31616], 60.00th=[31872],
     | 70.00th=[32128], 80.00th=[32128], 90.00th=[32640], 95.00th=[34560],
     | 99.00th=[40192], 99.50th=[43264], 99.90th=[51456], 99.95th=[55552],
     | 99.99th=[84480]
  write: IOPS=237k, BW=926MiB/s (971MB/s)(10.0GiB/11061msec); 0 zone resets
    slat (nsec): min=1954, max=71735, avg=3897.72, stdev=938.16
    clat (nsec): min=430, max=136489, avg=63372.98, stdev=5430.72
     lat (usec): min=4, max=142, avg=67.27, stdev= 5.70
    clat percentiles (usec):
     |  1.00th=[   48],  5.00th=[   55], 10.00th=[   58], 20.00th=[   61],
     | 30.00th=[   62], 40.00th=[   63], 50.00th=[   64], 60.00th=[   65],
     | 70.00th=[   66], 80.00th=[   67], 90.00th=[   70], 95.00th=[   73],
     | 99.00th=[   80], 99.50th=[   83], 99.90th=[   91], 99.95th=[   99],
     | 99.99th=[  118]
   bw (  KiB/s): min=93464, max=1144224, per=96.18%, avg=911805.22, stdev=187568.47, samples=23
   iops        : min=23366, max=286056, avg=227951.30, stdev=46892.12, samples=23
  lat (nsec)   : 500=0.01%, 750=0.01%
  lat (usec)   : 4=0.01%, 10=0.01%, 20=0.01%, 50=39.72%, 100=60.25%
  lat (usec)   : 250=0.03%
  cpu          : usr=55.59%, sys=44.35%, ctx=20, majf=0, minf=45314
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=1656716,2621440,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1827MiB/s (1916MB/s), 1827MiB/s-1827MiB/s (1916MB/s-1916MB/s), io=6472MiB (6786MB), run=3542-3542msec
  WRITE: bw=926MiB/s (971MB/s), 926MiB/s-926MiB/s (971MB/s-971MB/s), io=10.0GiB (10.7GB), run=11061-11061msec

Disk stats (read/write):
  ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ modprobe -r brd
+ modprobe -r brd
+ set +x