[GIT,PULL,22/23] xfs_scrub: vectorize kernel calls

Message ID	172230460022.1455085.11627065701892282147.stg-ugh@frogsfrogsfrogs (mailing list archive)
State	Accepted, archived
Headers	show Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12D0133E8 for <linux-xfs@vger.kernel.org>; Tue, 30 Jul 2024 02:46:03 +0000 (UTC) Date: Mon, 29 Jul 2024 19:46:02 -0700 Subject: [GIT PULL 22/23] xfs_scrub: vectorize kernel calls From: "Darrick J. Wong" <djwong@kernel.org> To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <172230460022.1455085.11627065701892282147.stg-ugh@frogsfrogsfrogs> Precedence: bulk MIME-Version: 1.0 In-Reply-To: <20240730013626.GF6352@frogsfrogsfrogs> References: <20240730013626.GF6352@frogsfrogsfrogs> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit
Series	[GIT,PULL,01/23] libxfs: fixes for 6.9 \| expand [GIT,PULL,01/23] libxfs: fixes for 6.9 [GIT,PULL,02/23] libxfs: sync with 6.10 [GIT,PULL,03/23] xfsprogs: atomic file updates [GIT,PULL,04/23] xfsprogs: set and validate dir/attr block owners [GIT,PULL,05/23] xfsprogs: inode-related repair fixes [GIT,PULL,06/23] xfs_scrub: fixes to the repair code [GIT,PULL,07/23] xfs_scrub: improve warnings about difficult repairs [GIT,PULL,08/23] xfs_scrub: track data dependencies for repairs [GIT,PULL,09/23] xfs_scrub: use scrub_item to track check progress [GIT,PULL,10/23] xfs_scrub: improve scheduling of repair items [GIT,PULL,11/23] xfs_scrub: detect deceptive filename extensions [GIT,PULL,12/23] xfs_scrub: move fstrim to a separate phase [GIT,PULL,13/23] xfs_scrub: use free space histograms to reduce fstrim runtime [GIT,PULL,14/23] xfs_scrub: tighten security of systemd services [GIT,PULL,15/23] xfs_scrub_all: automatic media scan service [GIT,PULL,16/23] xfs_scrub_all: improve systemd handling [GIT,PULL,17/23] xfsprogs: improve extended attribute validation [GIT,PULL,18/23] xfsprogs: Parent Pointers [GIT,PULL,19/23] xfsprogs: scrubbing for parent pointers [GIT,PULL,20/23] xfsprogs: offline repair for parent pointers [GIT,PULL,21/23] xfsprogs: detect and correct directory tree problems [GIT,PULL,22/23] xfs_scrub: vectorize kernel calls [GIT,PULL,23/23] xfs_repair: fixes for kernel 6.10

Message ID

172230460022.1455085.11627065701892282147.stg-ugh@frogsfrogsfrogs (mailing list archive)

State

Accepted, archived

Headers

Date: Mon, 29 Jul 2024 19:46:02 -0700
Subject: [GIT PULL 22/23] xfs_scrub: vectorize kernel calls
From: "Darrick J. Wong" <djwong@kernel.org>
To: cem@kernel.org, djwong@kernel.org
Cc: hch@lst.de, linux-xfs@vger.kernel.org
Message-ID: 
 <172230460022.1455085.11627065701892282147.stg-ugh@frogsfrogsfrogs>
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <20240730013626.GF6352@frogsfrogsfrogs>
References: <20240730013626.GF6352@frogsfrogsfrogs>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Series

[GIT,PULL,01/23] libxfs: fixes for 6.9 | expand

Message

Darrick J. Wong July 30, 2024, 2:46 a.m. UTC

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 5a30504f0c60e10dc0cecd201c5afc18083fd0ac:

xfs_scrub: defer phase5 file scans if dirloop fails (2024-07-29 17:01:13 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/vectorized-scrub-6.10_2024-07-29

for you to fetch changes up to df914edeeb1e1919831192951d657cfc73b46418:

xfs_scrub: try spot repairs of metadata items to make scrub progress (2024-07-29 17:01:13 -0700)

----------------------------------------------------------------
xfs_scrub: vectorize kernel calls [v30.9 22/28]

Create a vectorized version of the metadata scrub and repair ioctl, and
adapt xfs_scrub to use that.  This is an experiment to measure overhead
and to try refactoring xfs_scrub.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (10):
man: document vectored scrub mode
libfrog: support vectored scrub
xfs_io: support vectored scrub
xfs_scrub: split the scrub epilogue code into a separate function
xfs_scrub: split the repair epilogue code into a separate function
xfs_scrub: convert scrub and repair epilogues to use xfs_scrub_vec
xfs_scrub: vectorize scrub calls
xfs_scrub: vectorize repair calls
xfs_scrub: use scrub barriers to reduce kernel calls
xfs_scrub: try spot repairs of metadata items to make scrub progress

io/scrub.c                           | 368 +++++++++++++++++++++++++++++++----
libfrog/fsgeom.h                     |   6 +
libfrog/scrub.c                      | 137 +++++++++++++
libfrog/scrub.h                      |  35 ++++
man/man2/ioctl_xfs_scrubv_metadata.2 | 171 ++++++++++++++++
man/man8/xfs_io.8                    |  51 +++++
scrub/phase1.c                       |   2 +
scrub/phase2.c                       |  93 +++++++--
scrub/phase3.c                       |  84 ++++++--
scrub/repair.c                       | 355 ++++++++++++++++++++-------------
scrub/scrub.c                        | 360 +++++++++++++++++++++++++---------
scrub/scrub.h                        |  19 ++
scrub/scrub_private.h                |  55 ++++--
scrub/xfs_scrub.c                    |   1 +
14 files changed, 1431 insertions(+), 306 deletions(-)
create mode 100644 man/man2/ioctl_xfs_scrubv_metadata.2

[GIT,PULL,22/23] xfs_scrub: vectorize kernel calls

Pull-request

Message