mbox series

[RFC,0/3] Parallelizing filesystem writeback

Message ID 20250212103634.448437-1-kundan.kumar@samsung.com (mailing list archive)
Headers show
Series Parallelizing filesystem writeback | expand

Message

Kundan Kumar Feb. 12, 2025, 10:36 a.m. UTC 
This RFC is an attempt to share a POC for our LSFMM'25[0] proposal.
We initially prepared it based on the percpu writeback context design.
But based on feedback received initially, we moved towards a more
FS-geometry aware design. The patches are still clumsy, and there is
softlockup issue while running fstests as well(mentioned below).
Yet, we would like to gather feedback which will help in ironing
out the infra.

Kernel tree:
https://github.com/SamsungDS/linux/tree/priv/pw/rfc

Design
==========================
- A new "wb_ctx" structure is introduced that represents a writeback
  context/thread.
  A new "wb_ctx_list" field is introduced in bdi_writeback that describes
  a wb_ctx array, which gets initialized during wb_init.

- Writeback handling is modified to use "wb_ctx_list[0]" writeback
  context, so it continues to remain single threaded.

- In xfs, during AG initialization, we try to assign a different writeback
  context to each AG from the wb_ctx_list.

- During writeback, a "get_wb_ctx" handler is introduced in
  super_operations to fetch the writeback context from the inode, which
  gets used for writeback handling. Currently this handler is implemented
  only by xfs, so writeback handling is parallelized (per AG) for xfs
  only. It can be easily extended to other filesystems.

Discussion
==========================
- Design for API that would be used for deciding the number of writeback
  contexts by FS (suggested by Dave).

TBDs
==========================
- Need to explore cgroup related writeback handling.

- Add a list_lock field in struct wb_ctx for protecting pctx_b_* lists.
  Currently there is a "bdi_writeback->list_lock" for protecting per
  writeback context list.

- Ability to create writeback contexts dynamically. Currently, NR_WB_CTX
  number of writeback contexts are present in bdi_writeback and get
  initialized during wb_init.

Testing
==========================
- Encountered a softlockup issue while running fstests (test generic/051).
  Investigating the same.

[0] https://lore.kernel.org/all/20250129102627.161448-1-kundan.kumar@samsung.com/


Kundan Kumar (3):
  writeback: add parallel writeback infrastructure
  fs: modify writeback infra to work with wb_ctx
  xfs: use the parallel writeback infra per AG

 fs/fs-writeback.c                | 507 ++++++++++++++++++++-----------
 fs/xfs/libxfs/xfs_ag.c           |  16 +
 fs/xfs/libxfs/xfs_ag.h           |   1 +
 fs/xfs/xfs_super.c               |  20 ++
 include/linux/backing-dev-defs.h |  67 +++-
 include/linux/backing-dev.h      |  34 ++-
 include/linux/fs.h               |   1 +
 mm/backing-dev.c                 |  68 +++--
 mm/page-writeback.c              |  82 ++---
 9 files changed, 551 insertions(+), 245 deletions(-)


base-commit: 57d357a7663c4025c68ae07974b679d84e2d5d1a