| Message ID | cover.1727368214.git.anand.jain@oracle.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | raid1 balancing methods | expand |
在 2024/9/27 17:55, Anand Jain 写道: > The RAID1-balancing methods helps distribute read I/O across devices, and > this patch introduces three balancing methods: rotation, latency, and > devid. These methods are enabled under the `CONFIG_BTRFS_DEBUG` config > option and are on top of the previously added > `/sys/fs/btrfs/<UUID>/read_policy` interface to configure the desired > RAID1 read balancing method. I am currently testing this on 6.12-rc1 with policy rotation, seems good for now. Would be better if policy can be set in mount options. HAN Yuwei > > I've tested these patches using fio and filesystem defragmentation > workloads on a two-device RAID1 setup (with both data and metadata > mirrored across identical devices). I tracked device read counts by > extracting stats from `/sys/devices/<..>/stat` for each device. Below is > a summary of the results, with each result the average of three > iterations. > > A typical generic random rw workload: > > $ fio --filename=/btrfs/foo --size=10Gi --direct=1 --rw=randrw --bs=4k \ > --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based \ > --group_reporting --name=iops-test-job --eta-newline=1 > > | | | | Read I/O count | > | | Read | Write | devid1 | devid2 | > |---------|------------|------------|--------|--------| > | pid | 29.4MiB/s | 29.5MiB/s | 456548 | 447975 | > | rotation| 29.3MiB/s | 29.3MiB/s | 450105 | 450055 | > | latency | 21.9MiB/s | 21.9MiB/s | 672387 | 0 | > | devid:1 | 22.0MiB/s | 22.0MiB/s | 674788 | 0 | > > Defragmentation with compression workload: > > $ xfs_io -f -d -c 'pwrite -S 0xab 0 1G' /btrfs/foo > $ sync > $ echo 3 > /proc/sys/vm/drop_caches > $ btrfs filesystem defrag -f -c /btrfs/foo > > | | Time | Read I/O Count | > | | Real | devid1 | devid2 | > |---------|-------|--------|--------| > | pid | 21.61s| 3810 | 0 | > | rotation| 11.55s| 1905 | 1905 | > | latency | 20.99s| 0 | 3810 | > | devid:2 | 21.41s| 0 | 3810 | > > . The PID-based balancing method works well for the generic random rw fio > workload. > . The rotation method is ideal when you want to keep both devices active, > and it boosts performance in sequential defragmentation scenarios. > . The latency-based method work well when we have mixed device types or > when one device experiences intermittent I/O failures the latency > increases and it automatically picks the other device for further Read > IOs. > . The devid method is a more hands-on approach, useful for diagnosing and > testing RAID1 mirror synchronizations. > > Anand Jain (3): > btrfs: introduce RAID1 round-robin read balancing > btrfs: use the path with the lowest latency for RAID1 reads > btrfs: add RAID1 preferred read device feature > > fs/btrfs/sysfs.c | 94 ++++++++++++++++++++++++++++++------- > fs/btrfs/volumes.c | 113 +++++++++++++++++++++++++++++++++++++++++++++ > fs/btrfs/volumes.h | 14 ++++++ > 3 files changed, 205 insertions(+), 16 deletions(-) >
On 4/10/24 4:14 pm, Yuwei Han wrote: > > 在 2024/9/27 17:55, Anand Jain 写道: >> The RAID1-balancing methods helps distribute read I/O across devices, and >> this patch introduces three balancing methods: rotation, latency, and >> devid. These methods are enabled under the `CONFIG_BTRFS_DEBUG` config >> option and are on top of the previously added >> `/sys/fs/btrfs/<UUID>/read_policy` interface to configure the desired >> RAID1 read balancing method. > I am currently testing this on 6.12-rc1 with policy rotation, seems good > for now. Thanks for testing and reviewing. > Would be better if policy can be set in mount options. I think it is a good idea. However, we should also consolidate our sysfs knobs, mount options, and btrfs properties all together where applicable and make it easy to use. V2 is in the ML. Thanks, -Anand > > HAN Yuwei >> >> I've tested these patches using fio and filesystem defragmentation >> workloads on a two-device RAID1 setup (with both data and metadata >> mirrored across identical devices). I tracked device read counts by >> extracting stats from `/sys/devices/<..>/stat` for each device. Below is >> a summary of the results, with each result the average of three >> iterations. >> >> A typical generic random rw workload: >> >> $ fio --filename=/btrfs/foo --size=10Gi --direct=1 --rw=randrw --bs=4k \ >> --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 -- >> time_based \ >> --group_reporting --name=iops-test-job --eta-newline=1 >> >> | | | | Read I/O count | >> | | Read | Write | devid1 | devid2 | >> |---------|------------|------------|--------|--------| >> | pid | 29.4MiB/s | 29.5MiB/s | 456548 | 447975 | >> | rotation| 29.3MiB/s | 29.3MiB/s | 450105 | 450055 | >> | latency | 21.9MiB/s | 21.9MiB/s | 672387 | 0 | >> | devid:1 | 22.0MiB/s | 22.0MiB/s | 674788 | 0 | >> >> Defragmentation with compression workload: >> >> $ xfs_io -f -d -c 'pwrite -S 0xab 0 1G' /btrfs/foo >> $ sync >> $ echo 3 > /proc/sys/vm/drop_caches >> $ btrfs filesystem defrag -f -c /btrfs/foo >> >> | | Time | Read I/O Count | >> | | Real | devid1 | devid2 | >> |---------|-------|--------|--------| >> | pid | 21.61s| 3810 | 0 | >> | rotation| 11.55s| 1905 | 1905 | >> | latency | 20.99s| 0 | 3810 | >> | devid:2 | 21.41s| 0 | 3810 | >> >> . The PID-based balancing method works well for the generic random rw fio >> workload. >> . The rotation method is ideal when you want to keep both devices active, >> and it boosts performance in sequential defragmentation scenarios. >> . The latency-based method work well when we have mixed device types or >> when one device experiences intermittent I/O failures the latency >> increases and it automatically picks the other device for further Read >> IOs. >> . The devid method is a more hands-on approach, useful for diagnosing and >> testing RAID1 mirror synchronizations. >> >> Anand Jain (3): >> btrfs: introduce RAID1 round-robin read balancing >> btrfs: use the path with the lowest latency for RAID1 reads >> btrfs: add RAID1 preferred read device feature >> >> fs/btrfs/sysfs.c | 94 ++++++++++++++++++++++++++++++------- >> fs/btrfs/volumes.c | 113 +++++++++++++++++++++++++++++++++++++++++++++ >> fs/btrfs/volumes.h | 14 ++++++ >> 3 files changed, 205 insertions(+), 16 deletions(-) >> >