[PATCHSET,v3,0/2] xfs: speed up parallel workqueues

Message

Darrick J. Wong Jan. 28, 2021, 6:03 a.m. UTC

Hi all,

After some discussion on IRC with Dave, we came to the conclusion that
our background workqueue behavior could use some tweaking.  Kernel
worker threads that scan the filesystem and/or run their own
transactions more closely fit the definition of an unbound workqueue --
the work items can take a long time, they don't have much in common with
the submitter thread, the submitter isn't waiting hotly for a response,
and we the process scheduler should deal with scheduling them.
Furthermore, we don't want to place artificial limits on workqueue
scaling because that can leave unused capacity while we're blocking
(pwork is currently used for mount time quotacheck).

Therefore, we switch pwork to use an unbound workqueue, and now we let
the workqueue code figure out the relevant concurrency.  We expose the
pwork workqueue via sysfs, and document the interesting knobs in the
administrator's guide.

v2: expose all the workqueues via sysfs
v3: only expose those workqueues that exist for kernel threads that
    create their own transactions, and update the admin guide.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=workqueue-speedups-5.12
---
 Documentation/admin-guide/xfs.rst |   38 +++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_iwalk.c                |    5 +----
 fs/xfs/xfs_log.c                  |    5 +++--
 fs/xfs/xfs_mru_cache.c            |    2 +-
 fs/xfs/xfs_pwork.c                |   25 +++++-------------------
 fs/xfs/xfs_pwork.h                |    4 +---
 fs/xfs/xfs_super.c                |   23 ++++++++++++++--------
 fs/xfs/xfs_super.h                |    6 ++++++
 8 files changed, 69 insertions(+), 39 deletions(-)