[GIT,PULL,12/23] xfs_scrub: move fstrim to a separate phase

Message ID	172230458966.1455085.4399793624957809063.stg-ugh@frogsfrogsfrogs (mailing list archive)
State	Accepted, archived
Headers	show Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9221146588 for <linux-xfs@vger.kernel.org>; Tue, 30 Jul 2024 02:43:26 +0000 (UTC) Date: Mon, 29 Jul 2024 19:43:25 -0700 Subject: [GIT PULL 12/23] xfs_scrub: move fstrim to a separate phase From: "Darrick J. Wong" <djwong@kernel.org> To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <172230458966.1455085.4399793624957809063.stg-ugh@frogsfrogsfrogs> Precedence: bulk MIME-Version: 1.0 In-Reply-To: <20240730013626.GF6352@frogsfrogsfrogs> References: <20240730013626.GF6352@frogsfrogsfrogs> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit
Series	[GIT,PULL,01/23] libxfs: fixes for 6.9 \| expand [GIT,PULL,01/23] libxfs: fixes for 6.9 [GIT,PULL,02/23] libxfs: sync with 6.10 [GIT,PULL,03/23] xfsprogs: atomic file updates [GIT,PULL,04/23] xfsprogs: set and validate dir/attr block owners [GIT,PULL,05/23] xfsprogs: inode-related repair fixes [GIT,PULL,06/23] xfs_scrub: fixes to the repair code [GIT,PULL,07/23] xfs_scrub: improve warnings about difficult repairs [GIT,PULL,08/23] xfs_scrub: track data dependencies for repairs [GIT,PULL,09/23] xfs_scrub: use scrub_item to track check progress [GIT,PULL,10/23] xfs_scrub: improve scheduling of repair items [GIT,PULL,11/23] xfs_scrub: detect deceptive filename extensions [GIT,PULL,12/23] xfs_scrub: move fstrim to a separate phase [GIT,PULL,13/23] xfs_scrub: use free space histograms to reduce fstrim runtime [GIT,PULL,14/23] xfs_scrub: tighten security of systemd services [GIT,PULL,15/23] xfs_scrub_all: automatic media scan service [GIT,PULL,16/23] xfs_scrub_all: improve systemd handling [GIT,PULL,17/23] xfsprogs: improve extended attribute validation [GIT,PULL,18/23] xfsprogs: Parent Pointers [GIT,PULL,19/23] xfsprogs: scrubbing for parent pointers [GIT,PULL,20/23] xfsprogs: offline repair for parent pointers [GIT,PULL,21/23] xfsprogs: detect and correct directory tree problems [GIT,PULL,22/23] xfs_scrub: vectorize kernel calls [GIT,PULL,23/23] xfs_repair: fixes for kernel 6.10

Message ID

172230458966.1455085.4399793624957809063.stg-ugh@frogsfrogsfrogs (mailing list archive)

State

Accepted, archived

Headers

Date: Mon, 29 Jul 2024 19:43:25 -0700
Subject: [GIT PULL 12/23] xfs_scrub: move fstrim to a separate phase
From: "Darrick J. Wong" <djwong@kernel.org>
To: cem@kernel.org, djwong@kernel.org
Cc: hch@lst.de, linux-xfs@vger.kernel.org
Message-ID: <172230458966.1455085.4399793624957809063.stg-ugh@frogsfrogsfrogs>
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <20240730013626.GF6352@frogsfrogsfrogs>
References: <20240730013626.GF6352@frogsfrogsfrogs>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Series

[GIT,PULL,01/23] libxfs: fixes for 6.9 | expand

Message

Darrick J. Wong July 30, 2024, 2:43 a.m. UTC

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 746ee95b71649b4ae515893ffa3bbe7b5e815d0d:

xfs_scrub: dump unicode points (2024-07-29 17:01:09 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-fstrim-phase-6.10_2024-07-29

for you to fetch changes up to 5ccdd24dc9987b50333332b7381ff1a305e67ef7:

xfs_scrub: improve responsiveness while trimming the filesystem (2024-07-29 17:01:09 -0700)

----------------------------------------------------------------
xfs_scrub: move fstrim to a separate phase [v30.9 12/28]

Back when I originally designed xfs_scrub, all filesystem metadata
checks were complete by the end of phase 3, and phase 4 was where all
the metadata repairs occurred.  On the grounds that the filesystem
should be fully consistent by then, I made a call to FITRIM at the end
of phase 4 to discard empty space in the filesystem.

Unfortunately, that's no longer the case -- summary counters, link
counts, and quota counters are not checked until phase 7.  It's not safe
to instruct the storage to unmap "empty" areas if we don't know where
those empty areas are, so we need to create a phase 8 to trim the fs.
While we're at it, make it more obvious that fstrim only gets to run if
there are no unfixed corruptions and no other runtime errors have
occurred.

Finally, reduce the latency impacts on the rest of the system by
breaking up the fstrim work into a loop that targets only 16GB per call.
This enables better progress reporting for interactive runs and cgroup
based resource constraints for background runs.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (7):
xfs_scrub: move FITRIM to phase 8
xfs_scrub: ignore phase 8 if the user disabled fstrim
xfs_scrub: collapse trim_filesystem
xfs_scrub: fix the work estimation for phase 8
xfs_scrub: report FITRIM errors properly
xfs_scrub: don't call FITRIM after runtime errors
xfs_scrub: improve responsiveness while trimming the filesystem

scrub/Makefile    |   1 +
scrub/phase4.c    |  30 ++---------
scrub/phase8.c    | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
scrub/vfs.c       |  22 +++++---
scrub/vfs.h       |   2 +-
scrub/xfs_scrub.c |  11 +++-
scrub/xfs_scrub.h |   3 ++
7 files changed, 183 insertions(+), 37 deletions(-)
create mode 100644 scrub/phase8.c

[GIT,PULL,12/23] xfs_scrub: move fstrim to a separate phase

Pull-request

Message