[GIT,PULL,15/23] xfs_scrub_all: automatic media scan service

Message ID	172230459290.1455085.13159985441266119133.stg-ugh@frogsfrogsfrogs (mailing list archive)
State	Accepted, archived
Headers	show Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA30DF60 for <linux-xfs@vger.kernel.org>; Tue, 30 Jul 2024 02:44:13 +0000 (UTC) Date: Mon, 29 Jul 2024 19:44:12 -0700 Subject: [GIT PULL 15/23] xfs_scrub_all: automatic media scan service From: "Darrick J. Wong" <djwong@kernel.org> To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <172230459290.1455085.13159985441266119133.stg-ugh@frogsfrogsfrogs> Precedence: bulk MIME-Version: 1.0 In-Reply-To: <20240730013626.GF6352@frogsfrogsfrogs> References: <20240730013626.GF6352@frogsfrogsfrogs> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit
Series	[GIT,PULL,01/23] libxfs: fixes for 6.9 \| expand [GIT,PULL,01/23] libxfs: fixes for 6.9 [GIT,PULL,02/23] libxfs: sync with 6.10 [GIT,PULL,03/23] xfsprogs: atomic file updates [GIT,PULL,04/23] xfsprogs: set and validate dir/attr block owners [GIT,PULL,05/23] xfsprogs: inode-related repair fixes [GIT,PULL,06/23] xfs_scrub: fixes to the repair code [GIT,PULL,07/23] xfs_scrub: improve warnings about difficult repairs [GIT,PULL,08/23] xfs_scrub: track data dependencies for repairs [GIT,PULL,09/23] xfs_scrub: use scrub_item to track check progress [GIT,PULL,10/23] xfs_scrub: improve scheduling of repair items [GIT,PULL,11/23] xfs_scrub: detect deceptive filename extensions [GIT,PULL,12/23] xfs_scrub: move fstrim to a separate phase [GIT,PULL,13/23] xfs_scrub: use free space histograms to reduce fstrim runtime [GIT,PULL,14/23] xfs_scrub: tighten security of systemd services [GIT,PULL,15/23] xfs_scrub_all: automatic media scan service [GIT,PULL,16/23] xfs_scrub_all: improve systemd handling [GIT,PULL,17/23] xfsprogs: improve extended attribute validation [GIT,PULL,18/23] xfsprogs: Parent Pointers [GIT,PULL,19/23] xfsprogs: scrubbing for parent pointers [GIT,PULL,20/23] xfsprogs: offline repair for parent pointers [GIT,PULL,21/23] xfsprogs: detect and correct directory tree problems [GIT,PULL,22/23] xfs_scrub: vectorize kernel calls [GIT,PULL,23/23] xfs_repair: fixes for kernel 6.10

Message ID

172230459290.1455085.13159985441266119133.stg-ugh@frogsfrogsfrogs (mailing list archive)

State

Accepted, archived

Headers

Date: Mon, 29 Jul 2024 19:44:12 -0700
Subject: [GIT PULL 15/23] xfs_scrub_all: automatic media scan service
From: "Darrick J. Wong" <djwong@kernel.org>
To: cem@kernel.org, djwong@kernel.org
Cc: hch@lst.de, linux-xfs@vger.kernel.org
Message-ID: 
 <172230459290.1455085.13159985441266119133.stg-ugh@frogsfrogsfrogs>
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <20240730013626.GF6352@frogsfrogsfrogs>
References: <20240730013626.GF6352@frogsfrogsfrogs>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Series

[GIT,PULL,01/23] libxfs: fixes for 6.9 | expand

Message

Darrick J. Wong July 30, 2024, 2:44 a.m. UTC

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 50411335572120153cc84d54213cd5ca9dd11b14:

xfs_scrub_all: tighten up the security on the background systemd service (2024-07-29 17:01:10 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-media-scan-service-6.10_2024-07-29

for you to fetch changes up to e040916f649f71fb6a695dfcc6f6c9f576c5c4db:

xfs_scrub_all: failure reporting for the xfs_scrub_all job (2024-07-29 17:01:10 -0700)

----------------------------------------------------------------
xfs_scrub_all: automatic media scan service [v30.9 15/28]

Now that we've completed the online fsck functionality, there are a few
things that could be improved in the automatic service.  Specifically,
we would like to perform a more intensive metadata + media scan once per
month, to give the user confidence that the filesystem isn't losing data
silently.  To accomplish this, enhance xfs_scrub_all to be able to
trigger media scans.  Next, add a duplicate set of system services that
start the media scans automatically.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (6):
xfs_scrub_all: only use the xfs_scrub@ systemd services in service mode
xfs_scrub_all: remove journalctl background process
xfs_scrub_all: support metadata+media scans of all filesystems
xfs_scrub_all: enable periodic file data scrubs automatically
xfs_scrub_all: trigger automatic media scans once per month
xfs_scrub_all: failure reporting for the xfs_scrub_all job

debian/rules                                     |   3 +-
include/builddefs.in                             |   3 +
man/man8/Makefile                                |   7 +-
man/man8/{xfs_scrub_all.8 => xfs_scrub_all.8.in} |  20 +++-
scrub/Makefile                                   |  21 +++-
scrub/xfs_scrub@.service.in                      |   2 +-
scrub/xfs_scrub_all.cron.in                      |   2 +-
scrub/xfs_scrub_all.in                           | 126 ++++++++++++++++++-----
scrub/xfs_scrub_all.service.in                   |   9 +-
scrub/xfs_scrub_all_fail.service.in              |  71 +++++++++++++
scrub/xfs_scrub_fail.in                          |  46 +++++++--
scrub/xfs_scrub_fail@.service.in                 |   2 +-
scrub/xfs_scrub_media@.service.in                | 100 ++++++++++++++++++
scrub/xfs_scrub_media_fail@.service.in           |  76 ++++++++++++++
14 files changed, 443 insertions(+), 45 deletions(-)
rename man/man8/{xfs_scrub_all.8 => xfs_scrub_all.8.in} (59%)
create mode 100644 scrub/xfs_scrub_all_fail.service.in
create mode 100644 scrub/xfs_scrub_media@.service.in
create mode 100644 scrub/xfs_scrub_media_fail@.service.in

[GIT,PULL,15/23] xfs_scrub_all: automatic media scan service

Pull-request

Message