[RFC,0/2] btrfs: make extent buffer memory continuous

Message

Qu Wenruo July 25, 2023, 2:57 a.m. UTC

[REPO]
https://github.com/adam900710/linux/tree/eb_page_cleanups

This includes the submitted extent buffer accessors cleanup as
the dependency.

[BACKGROUND]
We have a lot of extent buffer code addressing the cross-page accesses, on
the other hand, other filesystems like XFS is mapping its xfs_buf into
kernel virtual address space, so that they can access the content of
xfs_buf without bothering the page boundaries.

[OBJECTIVE]
This patchset is mostly learning from the xfs_buf, to greatly simplify
the extent buffer accessors.

Now all the extent buffer accessors are turned into wrappers of
memcpy()/memcmp()/memmove().

For now, it can pass test cases from btrfs test case without new
regressions.

[RFC]
But I still want to get more feedbacks on this topic, since it's
changing the very core of btrfs extent buffer.

Furthermore, this change may not be 32bit systems friendly, as kernel
virtual address space is only 128MiB for 32bit systems, not sure if it's
going to cause any regression on 32bit systems.

[TODO]
- Benchmarks
  I'm not 100% sure if this going to cause any performance change.
  In theory, we off-load the cross-page handling to hardware MMU, which
  should improve performance, but we spend more time initializing the
  extent buffer.

- More tests on 32bit and 64bit systems

Qu Wenruo (2):
  btrfs: map uncontinuous extent buffer pages into virtual address space
  btrfs: utilize the physically/virtually continuous extent buffer
    memory

 fs/btrfs/disk-io.c   |  18 +--
 fs/btrfs/extent_io.c | 303 ++++++++++++++-----------------------------
 fs/btrfs/extent_io.h |  17 +++
 3 files changed, 119 insertions(+), 219 deletions(-)

Comments

Qu Wenruo July 26, 2023, 9:16 a.m. UTC | #1

On 2023/7/25 10:57, Qu Wenruo wrote:
> [REPO]
> https://github.com/adam900710/linux/tree/eb_page_cleanups
>
> This includes the submitted extent buffer accessors cleanup as
> the dependency.
>
> [BACKGROUND]
> We have a lot of extent buffer code addressing the cross-page accesses, on
> the other hand, other filesystems like XFS is mapping its xfs_buf into
> kernel virtual address space, so that they can access the content of
> xfs_buf without bothering the page boundaries.
>
> [OBJECTIVE]
> This patchset is mostly learning from the xfs_buf, to greatly simplify
> the extent buffer accessors.
>
> Now all the extent buffer accessors are turned into wrappers of
> memcpy()/memcmp()/memmove().
>
> For now, it can pass test cases from btrfs test case without new
> regressions.
>
> [RFC]
> But I still want to get more feedbacks on this topic, since it's
> changing the very core of btrfs extent buffer.
>
> Furthermore, this change may not be 32bit systems friendly, as kernel
> virtual address space is only 128MiB for 32bit systems, not sure if it's
> going to cause any regression on 32bit systems.
>
> [TODO]
> - Benchmarks
>    I'm not 100% sure if this going to cause any performance change.
>    In theory, we off-load the cross-page handling to hardware MMU, which
>    should improve performance, but we spend more time initializing the
>    extent buffer.

I tried an fio run with the following parameters on a PCIE3 NVME device:

fio -rw=randrw --size=8g \
	--bsrange=512b-64k --bs_unaligned \
	--ioengine=libaio --fsync=1024 \
	--filename=$mnt/job --name=job1 --name=job2 --runtime=300s

This would result heavy enough metadata workload for btrfs, and the new
patchset did get a small improvement on both throughput and latency:

Baseline:
    READ: bw=33.0MiB/s (34.6MB/s), 16.5MiB/s-16.6MiB/s
(17.3MB/s-17.4MB/s), io=8136MiB (8531MB), run=245999-246658msec
   WRITE: bw=33.0MiB/s (34.6MB/s), 16.5MiB/s-16.5MiB/s
(17.3MB/s-17.3MB/s), io=8144MiB (8539MB), run=245999-246658msec

Patched:
    READ: bw=33.0MiB/s (34.6MB/s), 16.5MiB/s-16.6MiB/s
(17.3MB/s-17.4MB/s), io=8136MiB (8531MB), run=245999-246658msec
   WRITE: bw=33.0MiB/s (34.6MB/s), 16.5MiB/s-16.5MiB/s
(17.3MB/s-17.3MB/s), io=8144MiB (8539MB), run=245999-246658msec

The throughput and latency both got around 2.6%.

Thanks,
Qu
>
> - More tests on 32bit and 64bit systems
>
> Qu Wenruo (2):
>    btrfs: map uncontinuous extent buffer pages into virtual address space
>    btrfs: utilize the physically/virtually continuous extent buffer
>      memory
>
>   fs/btrfs/disk-io.c   |  18 +--
>   fs/btrfs/extent_io.c | 303 ++++++++++++++-----------------------------
>   fs/btrfs/extent_io.h |  17 +++
>   3 files changed, 119 insertions(+), 219 deletions(-)
>