From patchwork Sun Dec 31 22:40:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507929 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05B53C13B for ; Sun, 31 Dec 2023 22:40:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UDjzgUf2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6FE37C433C8; Sun, 31 Dec 2023 22:40:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062437; bh=7vnpE72P4aocWzYVh8lKypZ5Ar16UHGET8u4pVhNU+s=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=UDjzgUf2ue2UC0mApwISM97/g1IY1p530YI2eOdty1DIlc/nGgldVgRuQpimw+Yia 0W3hazGcdF9isj/p7tTduWTLp6/dpuhVsbLfN5qfH1aw7j92DRLPpUzoe6gK6zxbzb BCrQHuKYY/18Mwxid8weVqOtTWHen23QBh2aCRJCeb2VWgaTQEW7U1cuTWMhX9rNpZ q0lSpn9A5tcvzs9v9Ua9pmLCB/hk/JFwcZpcvIgfqUE78PPpbw48foTPjH/1Ixm1nQ tLZWtOqQdsEXRi4LzNWWldJC1O+3aPF7IT2jtq/9rmVw6IGbiVDY2ziExWqx28yROQ 3TnEUjNdKDHlQ== Date: Sun, 31 Dec 2023 14:40:36 -0800 Subject: [PATCH 1/9] xfs_scrub: track repair items by principal, not by individual repairs From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999459.1797790.17309089356090705010.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Create a new structure to track scrub and repair state by principal filesystem object (e.g. ag number or inode number/generation) so that we can more easily examine and ensure that we satisfy repair order dependencies. This transposition will eventually enable bulk scrub operations and will also save a lot of memory if a given object needs a lot of work. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/phase1.c | 4 ++ scrub/phase2.c | 14 ++++++-- scrub/phase3.c | 19 ++++++----- scrub/phase4.c | 6 ++-- scrub/phase5.c | 5 ++- scrub/phase7.c | 4 ++ scrub/scrub.c | 68 ++++++++++++++++++++++++++++++++-------- scrub/scrub.h | 83 +++++++++++++++++++++++++++++++++++++++++++++---- scrub/scrub_private.h | 19 +++++++++++ 9 files changed, 185 insertions(+), 37 deletions(-) diff --git a/scrub/phase1.c b/scrub/phase1.c index a61e154a84a..9920f29a693 100644 --- a/scrub/phase1.c +++ b/scrub/phase1.c @@ -52,6 +52,7 @@ static int report_to_kernel( struct scrub_ctx *ctx) { + struct scrub_item sri; struct action_list alist; int ret; @@ -60,8 +61,9 @@ report_to_kernel( ctx->warnings_found) return 0; + scrub_item_init_fs(&sri); action_list_init(&alist); - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_HEALTHY, 0, &alist); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_HEALTHY, 0, &alist, &sri); if (ret) return ret; diff --git a/scrub/phase2.c b/scrub/phase2.c index 3e88c969b43..518923d6628 100644 --- a/scrub/phase2.c +++ b/scrub/phase2.c @@ -57,6 +57,7 @@ scan_ag_metadata( xfs_agnumber_t agno, void *arg) { + struct scrub_item sri; struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; struct action_list alist; @@ -68,6 +69,7 @@ scan_ag_metadata( if (sctl->aborted) return; + scrub_item_init_ag(&sri, agno); action_list_init(&alist); action_list_init(&immediate_alist); snprintf(descr, DESCR_BUFSZ, _("AG %u"), agno); @@ -76,7 +78,7 @@ scan_ag_metadata( * First we scrub and fix the AG headers, because we need * them to work well enough to check the AG btrees. */ - ret = scrub_ag_headers(ctx, agno, &alist); + ret = scrub_ag_headers(ctx, agno, &alist, &sri); if (ret) goto err; @@ -86,7 +88,7 @@ scan_ag_metadata( goto err; /* Now scrub the AG btrees. */ - ret = scrub_ag_metadata(ctx, agno, &alist); + ret = scrub_ag_metadata(ctx, agno, &alist, &sri); if (ret) goto err; @@ -120,6 +122,7 @@ scan_fs_metadata( xfs_agnumber_t type, void *arg) { + struct scrub_item sri; struct action_list alist; struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; @@ -129,8 +132,9 @@ scan_fs_metadata( if (sctl->aborted) goto out; + scrub_item_init_fs(&sri); action_list_init(&alist); - ret = scrub_fs_metadata(ctx, type, &alist); + ret = scrub_fs_metadata(ctx, type, &alist, &sri); if (ret) { sctl->aborted = true; goto out; @@ -162,6 +166,7 @@ phase2_func( .rbm_done = false, }; struct action_list alist; + struct scrub_item sri; const struct xfrog_scrub_descr *sc = xfrog_scrubbers; xfs_agnumber_t agno; unsigned int type; @@ -183,8 +188,9 @@ phase2_func( * upgrades) off of the sb 0 scrubber (which currently does nothing). * If errors occur, this function will log them and return nonzero. */ + scrub_item_init_ag(&sri, 0); action_list_init(&alist); - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_SB, 0, &alist); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_SB, 0, &alist, &sri); if (ret) goto out_wq; ret = action_list_process(ctx, -1, &alist, diff --git a/scrub/phase3.c b/scrub/phase3.c index b03b55250a3..642b8406e5b 100644 --- a/scrub/phase3.c +++ b/scrub/phase3.c @@ -105,12 +105,14 @@ scrub_inode( void *arg) { struct action_list alist; + struct scrub_item sri; struct scrub_inode_ctx *ictx = arg; struct ptcounter *icount = ictx->icount; xfs_agnumber_t agno; int fd = -1; int error; + scrub_item_init_file(&sri, bstat); action_list_init(&alist); agno = cvt_ino_to_agno(&ctx->mnt, bstat->bs_ino); background_sleep(); @@ -143,7 +145,7 @@ scrub_inode( fd = scrub_open_handle(handle); /* Scrub the inode. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_INODE, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_INODE, &alist, &sri); if (error) goto out; @@ -152,13 +154,13 @@ scrub_inode( goto out; /* Scrub all block mappings. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTD, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTD, &alist, &sri); if (error) goto out; - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTA, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTA, &alist, &sri); if (error) goto out; - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTC, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTC, &alist, &sri); if (error) goto out; @@ -179,24 +181,25 @@ scrub_inode( if (S_ISLNK(bstat->bs_mode) || !bstat->bs_mode) { /* Check symlink contents. */ error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_SYMLINK, - &alist); + &alist, &sri); if (error) goto out; } if (S_ISDIR(bstat->bs_mode) || !bstat->bs_mode) { /* Check the directory entries. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_DIR, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_DIR, &alist, + &sri); if (error) goto out; } /* Check all the extended attributes. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_XATTR, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_XATTR, &alist, &sri); if (error) goto out; /* Check parent pointers. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_PARENT, &alist); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_PARENT, &alist, &sri); if (error) goto out; diff --git a/scrub/phase4.c b/scrub/phase4.c index d01dc89f44f..1c4aab996ab 100644 --- a/scrub/phase4.c +++ b/scrub/phase4.c @@ -130,6 +130,7 @@ phase4_func( { struct xfs_fsop_geom fsgeom; struct action_list alist; + struct scrub_item sri; int ret; if (!have_action_items(ctx)) @@ -142,8 +143,9 @@ phase4_func( * chance that repairs of primary metadata fail due to secondary * metadata. If repairs fails, we'll come back during phase 7. */ + scrub_item_init_fs(&sri); action_list_init(&alist); - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_FSCOUNTERS, 0, &alist); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_FSCOUNTERS, 0, &alist, &sri); if (ret) return ret; @@ -159,7 +161,7 @@ phase4_func( if (fsgeom.sick & XFS_FSOP_GEOM_SICK_QUOTACHECK) { ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_QUOTACHECK, 0, - &alist); + &alist, &sri); if (ret) return ret; } diff --git a/scrub/phase5.c b/scrub/phase5.c index 68d35cd5852..ace6c3a9843 100644 --- a/scrub/phase5.c +++ b/scrub/phase5.c @@ -385,6 +385,7 @@ check_fs_label( } struct fs_scan_item { + struct scrub_item sri; struct action_list alist; bool *abortedp; unsigned int scrub_type; @@ -412,7 +413,8 @@ fs_scan_worker( nanosleep(&tv, NULL); } - ret = scrub_meta_type(ctx, item->scrub_type, 0, &item->alist); + ret = scrub_meta_type(ctx, item->scrub_type, 0, &item->alist, + &item->sri); if (ret) { str_liberror(ctx, ret, _("checking fs scan metadata")); *item->abortedp = true; @@ -450,6 +452,7 @@ queue_fs_scan( str_liberror(ctx, ret, _("setting up fs scan")); return ret; } + scrub_item_init_fs(&item->sri); action_list_init(&item->alist); item->scrub_type = scrub_type; item->abortedp = abortedp; diff --git a/scrub/phase7.c b/scrub/phase7.c index 820a68f99a4..314a886b091 100644 --- a/scrub/phase7.c +++ b/scrub/phase7.c @@ -99,6 +99,7 @@ phase7_func( struct scrub_ctx *ctx) { struct summary_counts totalcount = {0}; + struct scrub_item sri; struct action_list alist; struct ptvar *ptvar; unsigned long long used_data; @@ -117,8 +118,9 @@ phase7_func( int error; /* Check and fix the summary metadata. */ + scrub_item_init_fs(&sri); action_list_init(&alist); - error = scrub_summary_metadata(ctx, &alist); + error = scrub_summary_metadata(ctx, &alist, &sri); if (error) return error; error = action_list_process(ctx, -1, &alist, diff --git a/scrub/scrub.c b/scrub/scrub.c index 6e857c79dfb..e242e38ed0c 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -264,7 +264,8 @@ scrub_meta_type( struct scrub_ctx *ctx, unsigned int type, xfs_agnumber_t agno, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { struct xfs_scrub_metadata meta = { .sm_type = type, @@ -283,11 +284,13 @@ scrub_meta_type( case CHECK_ABORT: return ECANCELED; case CHECK_REPAIR: + scrub_item_save_state(sri, type, meta.sm_flags); ret = scrub_save_repair(ctx, alist, &meta); if (ret) return ret; fallthrough; case CHECK_DONE: + scrub_item_clean_state(sri, type); return 0; default: /* CHECK_RETRY should never happen. */ @@ -305,7 +308,8 @@ scrub_group( struct scrub_ctx *ctx, enum xfrog_scrub_group group, xfs_agnumber_t agno, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { const struct xfrog_scrub_descr *sc; unsigned int type; @@ -317,7 +321,7 @@ scrub_group( if (sc->group != group) continue; - ret = scrub_meta_type(ctx, type, agno, alist); + ret = scrub_meta_type(ctx, type, agno, alist, sri); if (ret) return ret; } @@ -330,9 +334,10 @@ int scrub_ag_headers( struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { - return scrub_group(ctx, XFROG_SCRUB_GROUP_AGHEADER, agno, alist); + return scrub_group(ctx, XFROG_SCRUB_GROUP_AGHEADER, agno, alist, sri); } /* Scrub each AG's metadata btrees. */ @@ -340,9 +345,10 @@ int scrub_ag_metadata( struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { - return scrub_group(ctx, XFROG_SCRUB_GROUP_PERAG, agno, alist); + return scrub_group(ctx, XFROG_SCRUB_GROUP_PERAG, agno, alist, sri); } /* Scrub whole-filesystem metadata. */ @@ -350,20 +356,22 @@ int scrub_fs_metadata( struct scrub_ctx *ctx, unsigned int type, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { ASSERT(xfrog_scrubbers[type].group == XFROG_SCRUB_GROUP_FS); - return scrub_meta_type(ctx, type, 0, alist); + return scrub_meta_type(ctx, type, 0, alist, sri); } /* Scrub all FS summary metadata. */ int scrub_summary_metadata( struct scrub_ctx *ctx, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { - return scrub_group(ctx, XFROG_SCRUB_GROUP_SUMMARY, 0, alist); + return scrub_group(ctx, XFROG_SCRUB_GROUP_SUMMARY, 0, alist, sri); } /* How many items do we have to check? */ @@ -425,7 +433,8 @@ scrub_file( int fd, const struct xfs_bulkstat *bstat, unsigned int type, - struct action_list *alist) + struct action_list *alist, + struct scrub_item *sri) { struct xfs_scrub_metadata meta = {0}; struct xfs_fd xfd; @@ -454,12 +463,45 @@ scrub_file( fix = xfs_check_metadata(ctx, xfdp, &meta, true); if (fix == CHECK_ABORT) return ECANCELED; - if (fix == CHECK_DONE) + if (fix == CHECK_DONE) { + scrub_item_clean_state(sri, type); return 0; + } + scrub_item_save_state(sri, type, meta.sm_flags); return scrub_save_repair(ctx, alist, &meta); } +/* Dump a scrub item for debugging purposes. */ +void +scrub_item_dump( + struct scrub_item *sri, + unsigned int group_mask, + const char *tag) +{ + unsigned int i; + + if (group_mask == 0) + group_mask = -1U; + + printf("DUMP SCRUB ITEM FOR %s\n", tag); + if (sri->sri_ino != -1ULL) + printf("ino 0x%llx gen %u\n", (unsigned long long)sri->sri_ino, + sri->sri_gen); + if (sri->sri_agno != -1U) + printf("agno %u\n", sri->sri_agno); + + foreach_scrub_type(i) { + unsigned int g = 1U << xfrog_scrubbers[i].group; + + if (g & group_mask) + printf("[%u]: type '%s' state 0x%x\n", i, + xfrog_scrubbers[i].name, + sri->sri_state[i]); + } + fflush(stdout); +} + /* * Test the availability of a kernel scrub command. If errors occur (or the * scrub ioctl is rejected) the errors will be logged and this function will diff --git a/scrub/scrub.h b/scrub/scrub.h index 98819a25b62..21ea4147e0f 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -16,17 +16,85 @@ enum check_outcome { struct action_item; +/* + * These flags record the metadata object state that the kernel returned. + * We want to remember if the object was corrupt, if the cross-referencing + * revealed inconsistencies (xcorrupt), if the cross referencing itself failed + * (xfail) or if the object is correct but could be optimised (preen). + */ +#define SCRUB_ITEM_CORRUPT (XFS_SCRUB_OFLAG_CORRUPT) /* (1 << 1) */ +#define SCRUB_ITEM_PREEN (XFS_SCRUB_OFLAG_PREEN) /* (1 << 2) */ +#define SCRUB_ITEM_XFAIL (XFS_SCRUB_OFLAG_XFAIL) /* (1 << 3) */ +#define SCRUB_ITEM_XCORRUPT (XFS_SCRUB_OFLAG_XCORRUPT) /* (1 << 4) */ + +/* All of the state flags that we need to prioritize repair work. */ +#define SCRUB_ITEM_REPAIR_ANY (SCRUB_ITEM_CORRUPT | \ + SCRUB_ITEM_PREEN | \ + SCRUB_ITEM_XFAIL | \ + SCRUB_ITEM_XCORRUPT) + +struct scrub_item { + /* + * Information we need to call the scrub and repair ioctls. Per-AG + * items should set the ino/gen fields to -1; per-inode items should + * set sri_agno to -1; and per-fs items should set all three fields to + * -1. Or use the macros below. + */ + __u64 sri_ino; + __u32 sri_gen; + __u32 sri_agno; + + /* Scrub item state flags, one for each XFS_SCRUB_TYPE. */ + __u8 sri_state[XFS_SCRUB_TYPE_NR]; +}; + +#define foreach_scrub_type(loopvar) \ + for ((loopvar) = 0; (loopvar) < XFS_SCRUB_TYPE_NR; (loopvar)++) + +static inline void +scrub_item_init_ag(struct scrub_item *sri, xfs_agnumber_t agno) +{ + memset(sri, 0, sizeof(*sri)); + sri->sri_agno = agno; + sri->sri_ino = -1ULL; + sri->sri_gen = -1U; +} + +static inline void +scrub_item_init_fs(struct scrub_item *sri) +{ + memset(sri, 0, sizeof(*sri)); + sri->sri_agno = -1U; + sri->sri_ino = -1ULL; + sri->sri_gen = -1U; +} + +static inline void +scrub_item_init_file(struct scrub_item *sri, const struct xfs_bulkstat *bstat) +{ + memset(sri, 0, sizeof(*sri)); + sri->sri_agno = -1U; + sri->sri_ino = bstat->bs_ino; + sri->sri_gen = bstat->bs_gen; +} + +void scrub_item_dump(struct scrub_item *sri, unsigned int group_mask, + const char *tag); + void scrub_report_preen_triggers(struct scrub_ctx *ctx); int scrub_ag_headers(struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist); + struct action_list *alist, struct scrub_item *sri); int scrub_ag_metadata(struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist); + struct action_list *alist, struct scrub_item *sri); int scrub_fs_metadata(struct scrub_ctx *ctx, unsigned int scrub_type, - struct action_list *alist); -int scrub_iscan_metadata(struct scrub_ctx *ctx, struct action_list *alist); -int scrub_summary_metadata(struct scrub_ctx *ctx, struct action_list *alist); + struct action_list *alist, struct scrub_item *sri); +int scrub_iscan_metadata(struct scrub_ctx *ctx, struct action_list *alist, + struct scrub_item *sri); +int scrub_summary_metadata(struct scrub_ctx *ctx, struct action_list *alist, + struct scrub_item *sri); int scrub_meta_type(struct scrub_ctx *ctx, unsigned int type, - xfs_agnumber_t agno, struct action_list *alist); + xfs_agnumber_t agno, struct action_list *alist, + struct scrub_item *sri); bool can_scrub_fs_metadata(struct scrub_ctx *ctx); bool can_scrub_inode(struct scrub_ctx *ctx); @@ -39,7 +107,8 @@ bool can_repair(struct scrub_ctx *ctx); bool can_force_rebuild(struct scrub_ctx *ctx); int scrub_file(struct scrub_ctx *ctx, int fd, const struct xfs_bulkstat *bstat, - unsigned int type, struct action_list *alist); + unsigned int type, struct action_list *alist, + struct scrub_item *sri); /* Repair parameters are the scrub inputs and retry count. */ struct action_item { diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index a24d485a286..090efb54c0a 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -52,4 +52,23 @@ static inline bool needs_repair(struct xfs_scrub_metadata *sm) void scrub_warn_incomplete_scrub(struct scrub_ctx *ctx, struct descr *dsc, struct xfs_scrub_metadata *meta); +/* Scrub item functions */ + +static inline void +scrub_item_save_state( + struct scrub_item *sri, + unsigned int scrub_type, + unsigned int scrub_flags) +{ + sri->sri_state[scrub_type] = scrub_flags & SCRUB_ITEM_REPAIR_ANY; +} + +static inline void +scrub_item_clean_state( + struct scrub_item *sri, + unsigned int scrub_type) +{ + sri->sri_state[scrub_type] = 0; +} + #endif /* XFS_SCRUB_SCRUB_PRIVATE_H_ */ From patchwork Sun Dec 31 22:40:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507930 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9B64C127 for ; Sun, 31 Dec 2023 22:40:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GnSWTBUa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13A68C433C8; Sun, 31 Dec 2023 22:40:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062453; bh=s02EnlVfrr146cQ5lboSnMNd7fNRQtQ+ZHExKtxa2iA=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=GnSWTBUaryf3HZpvu9sHFNi8TIdeYv7hJN/Mc7S+NrlqX3jx38TRV+qqJwr5GA8OR 1OSi3Z7LrbjlM/6W88dM/FgBNpFPneR4w4YniqAGkTY62OMzYonh9BKc0fb4qQoxGV szF9cWtxCx4E99VUJdbFUCsmOQF/R9PkSmvqGWT23d/1uDQhb89w1HiSZxH+FX6FED D/9MnYunrhbetH7DeLAEbmH8193KCKkqDnF5+iu+gaztIcQmYC2v39akALal1HnlXd 5hnhSMKcsboTlqBzM0Usf7NMRoBo5KArYuwZnEkTS4IJHMh4BuXXE7SSnCKXlAovRJ yoYp6x5jvLRJA== Date: Sun, 31 Dec 2023 14:40:52 -0800 Subject: [PATCH 2/9] xfs_scrub: use repair_item to direct repair activities From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999472.1797790.14608691085754775710.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Now that the new scrub_item tracks the state of any filesystem object needing any kind of repair, use it to drive filesystem repairs and updates to the in-kernel health status when repair finishes. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/phase1.c | 2 scrub/phase2.c | 24 ++-- scrub/phase3.c | 57 ++++---- scrub/phase4.c | 7 - scrub/phase5.c | 2 scrub/phase7.c | 3 scrub/repair.c | 381 +++++++++++++++++++++++++++++++------------------------- scrub/repair.h | 45 +++++-- scrub/scrub.c | 44 ------ scrub/scrub.h | 12 -- 10 files changed, 298 insertions(+), 279 deletions(-) diff --git a/scrub/phase1.c b/scrub/phase1.c index 9920f29a693..b1bbc694e64 100644 --- a/scrub/phase1.c +++ b/scrub/phase1.c @@ -71,7 +71,7 @@ report_to_kernel( * Complain if we cannot fail the clean bill of health, unless we're * just testing repairs. */ - if (action_list_length(&alist) > 0 && + if (repair_item_count_needsrepair(&sri) != 0 && !debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) { str_info(ctx, _("Couldn't upload clean bill of health."), NULL); action_list_discard(&alist); diff --git a/scrub/phase2.c b/scrub/phase2.c index 518923d6628..26ce5818030 100644 --- a/scrub/phase2.c +++ b/scrub/phase2.c @@ -58,6 +58,7 @@ scan_ag_metadata( void *arg) { struct scrub_item sri; + struct scrub_item fix_now; struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; struct action_list alist; @@ -83,7 +84,7 @@ scan_ag_metadata( goto err; /* Repair header damage. */ - ret = action_list_process_or_defer(ctx, agno, &alist); + ret = repair_item_corruption(ctx, &sri); if (ret) goto err; @@ -99,17 +100,19 @@ scan_ag_metadata( * the inobt from rmapbt data, but if the rmapbt is broken even * at this early phase then we are sunk. */ - difficulty = action_list_difficulty(&alist); - action_list_find_mustfix(&alist, &immediate_alist); + difficulty = repair_item_difficulty(&sri); + repair_item_mustfix(&sri, &fix_now); warn_repair_difficulties(ctx, difficulty, descr); /* Repair (inode) btree damage. */ - ret = action_list_process_or_defer(ctx, agno, &immediate_alist); + ret = repair_item_corruption(ctx, &fix_now); if (ret) goto err; /* Everything else gets fixed during phase 4. */ - action_list_defer(ctx, agno, &alist); + ret = repair_item_defer(ctx, &sri); + if (ret) + goto err; return; err: sctl->aborted = true; @@ -141,10 +144,14 @@ scan_fs_metadata( } /* Complain about metadata corruptions that might not be fixable. */ - difficulty = action_list_difficulty(&alist); + difficulty = repair_item_difficulty(&sri); warn_repair_difficulties(ctx, difficulty, xfrog_scrubbers[type].descr); - action_list_defer(ctx, 0, &alist); + ret = repair_item_defer(ctx, &sri); + if (ret) { + sctl->aborted = true; + goto out; + } out: if (type == XFS_SCRUB_TYPE_RTBITMAP) { @@ -193,8 +200,7 @@ phase2_func( ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_SB, 0, &alist, &sri); if (ret) goto out_wq; - ret = action_list_process(ctx, -1, &alist, - XRM_FINAL_WARNING | XRM_NOPROGRESS); + ret = repair_item_completely(ctx, &sri); if (ret) goto out_wq; diff --git a/scrub/phase3.c b/scrub/phase3.c index 642b8406e5b..e602d8c7ec4 100644 --- a/scrub/phase3.c +++ b/scrub/phase3.c @@ -55,45 +55,48 @@ report_close_error( * Defer all the repairs until phase 4, being careful about locking since the * inode scrub threads are not per-AG. */ -static void +static int defer_inode_repair( - struct scrub_inode_ctx *ictx, - xfs_agnumber_t agno, - struct action_list *alist) + struct scrub_inode_ctx *ictx, + const struct xfs_bulkstat *bstat, + struct scrub_item *sri) { - if (alist->nr == 0) - return; + struct action_item *aitem = NULL; + xfs_agnumber_t agno; + int ret; + ret = repair_item_to_action_item(ictx->ctx, sri, &aitem); + if (ret || !aitem) + return ret; + + agno = cvt_ino_to_agno(&ictx->ctx->mnt, bstat->bs_ino); pthread_mutex_lock(&ictx->locks[agno]); - action_list_defer(ictx->ctx, agno, alist); + action_list_add(&ictx->ctx->action_lists[agno], aitem); pthread_mutex_unlock(&ictx->locks[agno]); + return 0; } -/* Run repair actions now and defer unfinished items for later. */ +/* Run repair actions now and leave unfinished items for later. */ static int try_inode_repair( - struct scrub_inode_ctx *ictx, - int fd, - xfs_agnumber_t agno, - struct action_list *alist) + struct scrub_inode_ctx *ictx, + struct scrub_item *sri, + int fd, + const struct xfs_bulkstat *bstat) { - int ret; - /* * If at the start of phase 3 we already had ag/rt metadata repairs * queued up for phase 4, leave the action list untouched so that file - * metadata repairs will be deferred in scan order until phase 4. + * metadata repairs will be deferred until phase 4. */ if (ictx->always_defer_repairs) return 0; - ret = action_list_process(ictx->ctx, fd, alist, - XRM_REPAIR_ONLY | XRM_NOPROGRESS); - if (ret) - return ret; - - defer_inode_repair(ictx, agno, alist); - return 0; + /* + * Try to repair the file metadata. Unfixed metadata will remain in + * the scrub item state to be queued as a single action item. + */ + return repair_file_corruption(ictx->ctx, sri, fd); } /* Verify the contents, xattrs, and extent maps of an inode. */ @@ -108,13 +111,11 @@ scrub_inode( struct scrub_item sri; struct scrub_inode_ctx *ictx = arg; struct ptcounter *icount = ictx->icount; - xfs_agnumber_t agno; int fd = -1; int error; scrub_item_init_file(&sri, bstat); action_list_init(&alist); - agno = cvt_ino_to_agno(&ctx->mnt, bstat->bs_ino); background_sleep(); /* @@ -149,7 +150,7 @@ scrub_inode( if (error) goto out; - error = try_inode_repair(ictx, fd, agno, &alist); + error = try_inode_repair(ictx, &sri, fd, bstat); if (error) goto out; @@ -164,7 +165,7 @@ scrub_inode( if (error) goto out; - error = try_inode_repair(ictx, fd, agno, &alist); + error = try_inode_repair(ictx, &sri, fd, bstat); if (error) goto out; @@ -204,7 +205,7 @@ scrub_inode( goto out; /* Try to repair the file while it's open. */ - error = try_inode_repair(ictx, fd, agno, &alist); + error = try_inode_repair(ictx, &sri, fd, bstat); if (error) goto out; @@ -221,7 +222,7 @@ scrub_inode( progress_add(1); if (!error && !ictx->aborted) - defer_inode_repair(ictx, agno, &alist); + error = defer_inode_repair(ictx, bstat, &sri); if (fd >= 0) { int err2; diff --git a/scrub/phase4.c b/scrub/phase4.c index 1c4aab996ab..98518635b2b 100644 --- a/scrub/phase4.c +++ b/scrub/phase4.c @@ -40,7 +40,7 @@ repair_ag( /* Repair anything broken until we fail to make progress. */ do { - ret = action_list_process(ctx, -1, alist, flags); + ret = action_list_process(ctx, alist, flags); if (ret) { *aborted = true; return; @@ -55,7 +55,7 @@ repair_ag( /* Try once more, but this time complain if we can't fix things. */ flags |= XRM_FINAL_WARNING; - ret = action_list_process(ctx, -1, alist, flags); + ret = action_list_process(ctx, alist, flags); if (ret) *aborted = true; } @@ -167,8 +167,7 @@ phase4_func( } /* Repair counters before starting on the rest. */ - ret = action_list_process(ctx, -1, &alist, - XRM_REPAIR_ONLY | XRM_NOPROGRESS); + ret = repair_item_corruption(ctx, &sri); if (ret) return ret; action_list_discard(&alist); diff --git a/scrub/phase5.c b/scrub/phase5.c index ace6c3a9843..79bfea8f6b5 100644 --- a/scrub/phase5.c +++ b/scrub/phase5.c @@ -421,7 +421,7 @@ fs_scan_worker( goto out; } - ret = action_list_process(ctx, ctx->mnt.fd, &item->alist, + ret = action_list_process(ctx, &item->alist, XRM_FINAL_WARNING | XRM_NOPROGRESS); if (ret) { str_liberror(ctx, ret, _("repairing fs scan metadata")); diff --git a/scrub/phase7.c b/scrub/phase7.c index 314a886b091..404bfb82243 100644 --- a/scrub/phase7.c +++ b/scrub/phase7.c @@ -123,8 +123,7 @@ phase7_func( error = scrub_summary_metadata(ctx, &alist, &sri); if (error) return error; - error = action_list_process(ctx, -1, &alist, - XRM_FINAL_WARNING | XRM_NOPROGRESS); + error = repair_item_completely(ctx, &sri); if (error) return error; diff --git a/scrub/repair.c b/scrub/repair.c index 30817d268d6..6e09c592ed4 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -27,7 +27,8 @@ static enum check_outcome xfs_repair_metadata( struct scrub_ctx *ctx, struct xfs_fd *xfdp, - struct action_item *aitem, + unsigned int scrub_type, + struct scrub_item *sri, unsigned int repair_flags) { struct xfs_scrub_metadata meta = { 0 }; @@ -35,20 +36,20 @@ xfs_repair_metadata( DEFINE_DESCR(dsc, ctx, format_scrub_descr); int error; - assert(aitem->type < XFS_SCRUB_TYPE_NR); + assert(scrub_type < XFS_SCRUB_TYPE_NR); assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); - meta.sm_type = aitem->type; - meta.sm_flags = aitem->flags | XFS_SCRUB_IFLAG_REPAIR; + meta.sm_type = scrub_type; + meta.sm_flags = XFS_SCRUB_IFLAG_REPAIR; if (use_force_rebuild) meta.sm_flags |= XFS_SCRUB_IFLAG_FORCE_REBUILD; - switch (xfrog_scrubbers[aitem->type].group) { + switch (xfrog_scrubbers[scrub_type].group) { case XFROG_SCRUB_GROUP_AGHEADER: case XFROG_SCRUB_GROUP_PERAG: - meta.sm_agno = aitem->agno; + meta.sm_agno = sri->sri_agno; break; case XFROG_SCRUB_GROUP_INODE: - meta.sm_ino = aitem->ino; - meta.sm_gen = aitem->gen; + meta.sm_ino = sri->sri_ino; + meta.sm_gen = sri->sri_gen; break; default: break; @@ -58,9 +59,10 @@ xfs_repair_metadata( return CHECK_RETRY; memcpy(&oldm, &meta, sizeof(oldm)); + oldm.sm_flags = sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY; descr_set(&dsc, &oldm); - if (needs_repair(&meta)) + if (needs_repair(&oldm)) str_info(ctx, descr_render(&dsc), _("Attempting repair.")); else if (debug || verbose) str_info(ctx, descr_render(&dsc), @@ -92,8 +94,10 @@ _("Filesystem is shut down, aborting.")); * it done and move on. */ if (is_unoptimized(&oldm) || - debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) + debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) { + scrub_item_clean_state(sri, scrub_type); return CHECK_DONE; + } /* * If we're in no-complain mode, requeue the check for * later. It's possible that an error in another @@ -109,6 +113,7 @@ _("Filesystem is shut down, aborting.")); /* Kernel doesn't know how to repair this? */ str_corrupt(ctx, descr_render(&dsc), _("Don't know how to fix; offline repair required.")); + scrub_item_clean_state(sri, scrub_type); return CHECK_DONE; case EROFS: /* Read-only filesystem, can't fix. */ @@ -118,23 +123,28 @@ _("Read-only filesystem; cannot make changes.")); return CHECK_ABORT; case ENOENT: /* Metadata not present, just skip it. */ + scrub_item_clean_state(sri, scrub_type); return CHECK_DONE; case ENOMEM: case ENOSPC: /* Don't care if preen fails due to low resources. */ - if (is_unoptimized(&oldm) && !needs_repair(&oldm)) + if (is_unoptimized(&oldm) && !needs_repair(&oldm)) { + scrub_item_clean_state(sri, scrub_type); return CHECK_DONE; + } fallthrough; default: /* - * Operational error. If the caller doesn't want us - * to complain about repair failures, tell the caller - * to requeue the repair for later and don't say a - * thing. Otherwise, print error and bail out. + * Operational error. If the caller doesn't want us to + * complain about repair failures, tell the caller to requeue + * the repair for later and don't say a thing. Otherwise, + * print an error, mark the item clean because we're done with + * trying to repair it, and bail out. */ if (!(repair_flags & XRM_FINAL_WARNING)) return CHECK_RETRY; str_liberror(ctx, error, descr_render(&dsc)); + scrub_item_clean_state(sri, scrub_type); return CHECK_DONE; } @@ -186,12 +196,13 @@ _("Repair unsuccessful; offline repair required.")); record_preen(ctx, descr_render(&dsc), _("Optimization successful.")); } + + scrub_item_clean_state(sri, scrub_type); return CHECK_DONE; } /* * Prioritize action items in order of how long we can wait. - * 0 = do it now, 10000 = do it later. * * To minimize the amount of repair work, we want to prioritize metadata * objects by perceived corruptness. If CORRUPT is set, the fields are @@ -207,104 +218,34 @@ _("Repair unsuccessful; offline repair required.")); * in order. */ -/* Sort action items in severity order. */ -static int -PRIO( - const struct action_item *aitem, - int order) -{ - if (aitem->flags & XFS_SCRUB_OFLAG_CORRUPT) - return order; - else if (aitem->flags & XFS_SCRUB_OFLAG_XCORRUPT) - return 100 + order; - else if (aitem->flags & XFS_SCRUB_OFLAG_XFAIL) - return 200 + order; - else if (aitem->flags & XFS_SCRUB_OFLAG_PREEN) - return 300 + order; - abort(); -} - -/* Sort the repair items in dependency order. */ -static int -xfs_action_item_priority( - const struct action_item *aitem) -{ - switch (aitem->type) { - case XFS_SCRUB_TYPE_SB: - case XFS_SCRUB_TYPE_AGF: - case XFS_SCRUB_TYPE_AGFL: - case XFS_SCRUB_TYPE_AGI: - case XFS_SCRUB_TYPE_BNOBT: - case XFS_SCRUB_TYPE_CNTBT: - case XFS_SCRUB_TYPE_INOBT: - case XFS_SCRUB_TYPE_FINOBT: - case XFS_SCRUB_TYPE_REFCNTBT: - case XFS_SCRUB_TYPE_RMAPBT: - case XFS_SCRUB_TYPE_INODE: - case XFS_SCRUB_TYPE_BMBTD: - case XFS_SCRUB_TYPE_BMBTA: - case XFS_SCRUB_TYPE_BMBTC: - return PRIO(aitem, aitem->type - 1); - case XFS_SCRUB_TYPE_DIR: - case XFS_SCRUB_TYPE_XATTR: - case XFS_SCRUB_TYPE_SYMLINK: - case XFS_SCRUB_TYPE_PARENT: - return PRIO(aitem, XFS_SCRUB_TYPE_DIR); - case XFS_SCRUB_TYPE_RTBITMAP: - case XFS_SCRUB_TYPE_RTSUM: - return PRIO(aitem, XFS_SCRUB_TYPE_RTBITMAP); - case XFS_SCRUB_TYPE_UQUOTA: - case XFS_SCRUB_TYPE_GQUOTA: - case XFS_SCRUB_TYPE_PQUOTA: - return PRIO(aitem, XFS_SCRUB_TYPE_UQUOTA); - case XFS_SCRUB_TYPE_QUOTACHECK: - /* This should always go after [UGP]QUOTA no matter what. */ - return PRIO(aitem, aitem->type); - case XFS_SCRUB_TYPE_FSCOUNTERS: - /* This should always go after AG headers no matter what. */ - return PRIO(aitem, INT_MAX); - } - abort(); -} - -/* Make sure that btrees get repaired before headers. */ -static int -xfs_action_item_compare( - void *priv, - const struct list_head *a, - const struct list_head *b) -{ - const struct action_item *ra; - const struct action_item *rb; - - ra = container_of(a, struct action_item, list); - rb = container_of(b, struct action_item, list); - - return xfs_action_item_priority(ra) - xfs_action_item_priority(rb); -} +struct action_item { + struct list_head list; + struct scrub_item sri; +}; /* * Figure out which AG metadata must be fixed before we can move on * to the inode scan. */ void -action_list_find_mustfix( - struct action_list *alist, - struct action_list *immediate_alist) +repair_item_mustfix( + struct scrub_item *sri, + struct scrub_item *fix_now) { - struct action_item *n; - struct action_item *aitem; + unsigned int scrub_type; - list_for_each_entry_safe(aitem, n, &alist->list, list) { - if (!(aitem->flags & XFS_SCRUB_OFLAG_CORRUPT)) + assert(sri->sri_agno != -1U); + scrub_item_init_ag(fix_now, sri->sri_agno); + + foreach_scrub_type(scrub_type) { + if (!(sri->sri_state[scrub_type] & SCRUB_ITEM_CORRUPT)) continue; - switch (aitem->type) { + + switch (scrub_type) { case XFS_SCRUB_TYPE_AGI: case XFS_SCRUB_TYPE_FINOBT: case XFS_SCRUB_TYPE_INOBT: - alist->nr--; - list_move_tail(&aitem->list, &immediate_alist->list); - immediate_alist->nr++; + fix_now->sri_state[scrub_type] |= SCRUB_ITEM_CORRUPT; break; } } @@ -312,19 +253,19 @@ action_list_find_mustfix( /* Determine if primary or secondary metadata are inconsistent. */ unsigned int -action_list_difficulty( - const struct action_list *alist) +repair_item_difficulty( + const struct scrub_item *sri) { - struct action_item *aitem, *n; - unsigned int ret = 0; + unsigned int scrub_type; + unsigned int ret = 0; - list_for_each_entry_safe(aitem, n, &alist->list, list) { - if (!(aitem->flags & (XFS_SCRUB_OFLAG_CORRUPT | - XFS_SCRUB_OFLAG_XCORRUPT | - XFS_SCRUB_OFLAG_XFAIL))) + foreach_scrub_type(scrub_type) { + if (!(sri->sri_state[scrub_type] & (XFS_SCRUB_OFLAG_CORRUPT | + XFS_SCRUB_OFLAG_XCORRUPT | + XFS_SCRUB_OFLAG_XFAIL))) continue; - switch (aitem->type) { + switch (scrub_type) { case XFS_SCRUB_TYPE_RMAPBT: ret |= REPAIR_DIFFICULTY_SECONDARY; break; @@ -404,13 +345,19 @@ action_list_init( alist->sorted = false; } -/* Number of repairs in this list. */ +/* Number of pending repairs in this list. */ unsigned long long action_list_length( struct action_list *alist) { - return alist->nr; -}; + struct action_item *aitem; + unsigned long long ret = 0; + + list_for_each_entry(aitem, &alist->list, list) + ret += repair_item_count_needsrepair(&aitem->sri); + + return ret; +} /* Add to the list of repairs. */ void @@ -423,60 +370,78 @@ action_list_add( alist->sorted = false; } -/* Splice two repair lists. */ -void -action_list_splice( - struct action_list *dest, - struct action_list *src) -{ - if (src->nr == 0) - return; - - list_splice_tail_init(&src->list, &dest->list); - dest->nr += src->nr; - src->nr = 0; - dest->sorted = false; -} - /* Repair everything on this list. */ int action_list_process( struct scrub_ctx *ctx, - int fd, struct action_list *alist, unsigned int repair_flags) +{ + struct action_item *aitem; + struct action_item *n; + int ret; + + list_for_each_entry_safe(aitem, n, &alist->list, list) { + if (scrub_excessive_errors(ctx)) + return ECANCELED; + + ret = repair_item(ctx, &aitem->sri, repair_flags); + if (ret) + break; + + if (repair_item_count_needsrepair(&aitem->sri) == 0) { + list_del(&aitem->list); + free(aitem); + } + } + + return ret; +} + +/* + * For a given filesystem object, perform all repairs of a given class + * (corrupt, xcorrupt, xfail, preen) if the repair item says it's needed. + */ +static int +repair_item_class( + struct scrub_ctx *ctx, + struct scrub_item *sri, + int override_fd, + uint8_t repair_mask, + unsigned int flags) { struct xfs_fd xfd; struct xfs_fd *xfdp = &ctx->mnt; - struct action_item *aitem; - struct action_item *n; - enum check_outcome fix; + unsigned int scrub_type; + + if (ctx->mode < SCRUB_MODE_REPAIR) + return 0; /* * If the caller passed us a file descriptor for a scrub, use it * instead of scrub-by-handle because this enables the kernel to skip * costly inode btree lookups. */ - if (fd >= 0) { + if (override_fd >= 0) { memcpy(&xfd, xfdp, sizeof(xfd)); - xfd.fd = fd; + xfd.fd = override_fd; xfdp = &xfd; } - if (!alist->sorted) { - list_sort(NULL, &alist->list, xfs_action_item_compare); - alist->sorted = true; - } + foreach_scrub_type(scrub_type) { + enum check_outcome fix; - list_for_each_entry_safe(aitem, n, &alist->list, list) { - fix = xfs_repair_metadata(ctx, xfdp, aitem, repair_flags); + if (scrub_excessive_errors(ctx)) + return ECANCELED; + + if (!(sri->sri_state[scrub_type] & repair_mask)) + continue; + + fix = xfs_repair_metadata(ctx, xfdp, scrub_type, sri, flags); switch (fix) { case CHECK_DONE: - if (!(repair_flags & XRM_NOPROGRESS)) + if (!(flags & XRM_NOPROGRESS)) progress_add(1); - alist->nr--; - list_del(&aitem->list); - free(aitem); continue; case CHECK_ABORT: return ECANCELED; @@ -487,37 +452,113 @@ action_list_process( } } - if (scrub_excessive_errors(ctx)) - return ECANCELED; + return 0; +} + +/* + * Repair all parts (i.e. scrub types) of this filesystem object for which + * corruption has been observed directly. Other types of repair work (fixing + * cross referencing problems and preening) are deferred. + * + * This function should only be called to perform spot repairs of fs objects + * during phase 2 and 3 while we still have open handles to those objects. + */ +int +repair_item_corruption( + struct scrub_ctx *ctx, + struct scrub_item *sri) +{ + return repair_item_class(ctx, sri, -1, SCRUB_ITEM_CORRUPT, + XRM_REPAIR_ONLY | XRM_NOPROGRESS); +} + +/* Repair all parts of this file, similar to repair_item_corruption. */ +int +repair_file_corruption( + struct scrub_ctx *ctx, + struct scrub_item *sri, + int override_fd) +{ + return repair_item_class(ctx, sri, override_fd, SCRUB_ITEM_CORRUPT, + XRM_REPAIR_ONLY | XRM_NOPROGRESS); +} + +/* + * Repair everything in this filesystem object that needs it. This includes + * cross-referencing and preening. + */ +int +repair_item( + struct scrub_ctx *ctx, + struct scrub_item *sri, + unsigned int flags) +{ + int ret; + + ret = repair_item_class(ctx, sri, -1, SCRUB_ITEM_CORRUPT, flags); + if (ret) + return ret; + + ret = repair_item_class(ctx, sri, -1, SCRUB_ITEM_XCORRUPT, flags); + if (ret) + return ret; + + ret = repair_item_class(ctx, sri, -1, SCRUB_ITEM_XFAIL, flags); + if (ret) + return ret; + + return repair_item_class(ctx, sri, -1, SCRUB_ITEM_PREEN, flags); +} + +/* Create an action item around a scrub item that needs repairs. */ +int +repair_item_to_action_item( + struct scrub_ctx *ctx, + const struct scrub_item *sri, + struct action_item **aitemp) +{ + struct action_item *aitem; + + if (repair_item_count_needsrepair(sri) == 0) + return 0; + + aitem = malloc(sizeof(struct action_item)); + if (!aitem) { + int error = errno; + + str_liberror(ctx, error, _("creating repair action item")); + return error; + } + + INIT_LIST_HEAD(&aitem->list); + memcpy(&aitem->sri, sri, sizeof(struct scrub_item)); + + *aitemp = aitem; return 0; } /* Defer all the repairs until phase 4. */ -void -action_list_defer( - struct scrub_ctx *ctx, - xfs_agnumber_t agno, - struct action_list *alist) +int +repair_item_defer( + struct scrub_ctx *ctx, + const struct scrub_item *sri) { + struct action_item *aitem = NULL; + unsigned int agno; + int error; + + error = repair_item_to_action_item(ctx, sri, &aitem); + if (error || !aitem) + return error; + + if (sri->sri_agno != -1U) + agno = sri->sri_agno; + else if (sri->sri_ino != -1ULL && sri->sri_gen != -1U) + agno = cvt_ino_to_agno(&ctx->mnt, sri->sri_ino); + else + agno = 0; ASSERT(agno < ctx->mnt.fsgeom.agcount); - action_list_splice(&ctx->action_lists[agno], alist); -} - -/* Run actions now and defer unfinished items for later. */ -int -action_list_process_or_defer( - struct scrub_ctx *ctx, - xfs_agnumber_t agno, - struct action_list *alist) -{ - int ret; - - ret = action_list_process(ctx, -1, alist, - XRM_REPAIR_ONLY | XRM_NOPROGRESS); - if (ret) - return ret; - - action_list_defer(ctx, agno, alist); + action_list_add(&ctx->action_lists[agno], aitem); return 0; } diff --git a/scrub/repair.h b/scrub/repair.h index b61bd29c860..463a3f9bfef 100644 --- a/scrub/repair.h +++ b/scrub/repair.h @@ -12,6 +12,8 @@ struct action_list { bool sorted; }; +struct action_item; + int action_lists_alloc(size_t nr, struct action_list **listsp); void action_lists_free(struct action_list **listsp); @@ -25,16 +27,14 @@ static inline bool action_list_empty(const struct action_list *alist) unsigned long long action_list_length(struct action_list *alist); void action_list_add(struct action_list *dest, struct action_item *item); void action_list_discard(struct action_list *alist); -void action_list_splice(struct action_list *dest, struct action_list *src); -void action_list_find_mustfix(struct action_list *actions, - struct action_list *immediate_alist); +void repair_item_mustfix(struct scrub_item *sri, struct scrub_item *fix_now); /* Primary metadata is corrupt */ #define REPAIR_DIFFICULTY_PRIMARY (1U << 0) /* Secondary metadata is corrupt */ #define REPAIR_DIFFICULTY_SECONDARY (1U << 1) -unsigned int action_list_difficulty(const struct action_list *actions); +unsigned int repair_item_difficulty(const struct scrub_item *sri); /* * Only ask the kernel to repair this object if the kernel directly told us it @@ -49,11 +49,36 @@ unsigned int action_list_difficulty(const struct action_list *actions); /* Don't call progress_add after repairing an item. */ #define XRM_NOPROGRESS (1U << 2) -int action_list_process(struct scrub_ctx *ctx, int fd, - struct action_list *alist, unsigned int repair_flags); -void action_list_defer(struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist); -int action_list_process_or_defer(struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist); +int action_list_process(struct scrub_ctx *ctx, struct action_list *alist, + unsigned int repair_flags); +int repair_item_corruption(struct scrub_ctx *ctx, struct scrub_item *sri); +int repair_file_corruption(struct scrub_ctx *ctx, struct scrub_item *sri, + int override_fd); +int repair_item(struct scrub_ctx *ctx, struct scrub_item *sri, + unsigned int repair_flags); +int repair_item_to_action_item(struct scrub_ctx *ctx, + const struct scrub_item *sri, struct action_item **aitemp); +int repair_item_defer(struct scrub_ctx *ctx, const struct scrub_item *sri); + +static inline unsigned int +repair_item_count_needsrepair( + const struct scrub_item *sri) +{ + unsigned int scrub_type; + unsigned int nr = 0; + + foreach_scrub_type(scrub_type) + if (sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY) + nr++; + return nr; +} + +static inline int +repair_item_completely( + struct scrub_ctx *ctx, + struct scrub_item *sri) +{ + return repair_item(ctx, sri, XRM_FINAL_WARNING | XRM_NOPROGRESS); +} #endif /* XFS_SCRUB_REPAIR_H_ */ diff --git a/scrub/scrub.c b/scrub/scrub.c index e242e38ed0c..54f397fb92a 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -217,42 +217,6 @@ _("Optimizations of %s are possible."), _(xfrog_scrubbers[i].descr)); } } -/* Save a scrub context for later repairs. */ -static int -scrub_save_repair( - struct scrub_ctx *ctx, - struct action_list *alist, - struct xfs_scrub_metadata *meta) -{ - struct action_item *aitem; - - /* Schedule this item for later repairs. */ - aitem = malloc(sizeof(struct action_item)); - if (!aitem) { - str_errno(ctx, _("adding item to repair list")); - return errno; - } - - memset(aitem, 0, sizeof(*aitem)); - aitem->type = meta->sm_type; - aitem->flags = meta->sm_flags; - switch (xfrog_scrubbers[meta->sm_type].group) { - case XFROG_SCRUB_GROUP_AGHEADER: - case XFROG_SCRUB_GROUP_PERAG: - aitem->agno = meta->sm_agno; - break; - case XFROG_SCRUB_GROUP_INODE: - aitem->ino = meta->sm_ino; - aitem->gen = meta->sm_gen; - break; - default: - break; - } - - action_list_add(alist, aitem); - return 0; -} - /* * Scrub a single XFS_SCRUB_TYPE_*, saving corruption reports for later. * @@ -272,7 +236,6 @@ scrub_meta_type( .sm_agno = agno, }; enum check_outcome fix; - int ret; background_sleep(); @@ -285,10 +248,7 @@ scrub_meta_type( return ECANCELED; case CHECK_REPAIR: scrub_item_save_state(sri, type, meta.sm_flags); - ret = scrub_save_repair(ctx, alist, &meta); - if (ret) - return ret; - fallthrough; + return 0; case CHECK_DONE: scrub_item_clean_state(sri, type); return 0; @@ -469,7 +429,7 @@ scrub_file( } scrub_item_save_state(sri, type, meta.sm_flags); - return scrub_save_repair(ctx, alist, &meta); + return 0; } /* Dump a scrub item for debugging purposes. */ diff --git a/scrub/scrub.h b/scrub/scrub.h index 21ea4147e0f..0d6825a5a95 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -14,8 +14,6 @@ enum check_outcome { CHECK_RETRY, /* repair failed, try again later */ }; -struct action_item; - /* * These flags record the metadata object state that the kernel returned. * We want to remember if the object was corrupt, if the cross-referencing @@ -110,14 +108,4 @@ int scrub_file(struct scrub_ctx *ctx, int fd, const struct xfs_bulkstat *bstat, unsigned int type, struct action_list *alist, struct scrub_item *sri); -/* Repair parameters are the scrub inputs and retry count. */ -struct action_item { - struct list_head list; - __u64 ino; - __u32 type; - __u32 flags; - __u32 gen; - __u32 agno; -}; - #endif /* XFS_SCRUB_SCRUB_H_ */ From patchwork Sun Dec 31 22:41:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507931 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E89EC13B for ; Sun, 31 Dec 2023 22:41:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QnqhxwQU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CC7DFC433C8; Sun, 31 Dec 2023 22:41:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062468; bh=KFUjHXkILOUcB+JpmoFjlA/ESaDFnjjlridN0fApeSk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=QnqhxwQU0ihZFYv4YMyBhZsjVBIXO4bvDo+vEv3zqCnCFld9JP5PqHrOPzjOBM0b+ JC3UKFcRA7v47W/6f6cl5wQXMR4+Aji9V0EYqBZQInDXtE6kqo0xLHvWxZ+1dKspbi 2IkCHRViQrESM/VvNSgM3VUfbv1UA1zqW1H0voKsD0nHApjFcSzCAYo7Sl0G8ROOw6 Uk/JzoPVXQFyQSoS05gIBb7F0QrtIP8u8RQKP9Wk40DLJ7+qE3ECzZbrwAwQ9oDmZP 0pJK7EzOWWEx5iQCV7T/mKxcbRuXnCdNnaGh7OjzUwvJ+2qMChgaiy7/zluqVDioCj xbRNGnir6z+yg== Date: Sun, 31 Dec 2023 14:41:08 -0800 Subject: [PATCH 3/9] xfs_scrub: remove action lists from phaseX code From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999486.1797790.16655448538358333691.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Now that we track repair schedules by filesystem object (and not individual repairs) we can get rid of all the onstack list heads and whatnot in the phaseX code. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/phase1.c | 5 +---- scrub/phase2.c | 16 ++++------------ scrub/phase3.c | 19 ++++++++----------- scrub/phase4.c | 8 ++------ scrub/phase5.c | 8 ++------ scrub/phase7.c | 4 +--- scrub/scrub.c | 37 ++++++++++++++++++++----------------- scrub/scrub.h | 16 +++++----------- 8 files changed, 43 insertions(+), 70 deletions(-) diff --git a/scrub/phase1.c b/scrub/phase1.c index b1bbc694e64..1e56f9fb1ee 100644 --- a/scrub/phase1.c +++ b/scrub/phase1.c @@ -53,7 +53,6 @@ report_to_kernel( struct scrub_ctx *ctx) { struct scrub_item sri; - struct action_list alist; int ret; if (!ctx->scrub_setup_succeeded || ctx->corruptions_found || @@ -62,8 +61,7 @@ report_to_kernel( return 0; scrub_item_init_fs(&sri); - action_list_init(&alist); - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_HEALTHY, 0, &alist, &sri); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_HEALTHY, &sri); if (ret) return ret; @@ -74,7 +72,6 @@ report_to_kernel( if (repair_item_count_needsrepair(&sri) != 0 && !debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) { str_info(ctx, _("Couldn't upload clean bill of health."), NULL); - action_list_discard(&alist); } return 0; diff --git a/scrub/phase2.c b/scrub/phase2.c index 26ce5818030..4d4552d8477 100644 --- a/scrub/phase2.c +++ b/scrub/phase2.c @@ -61,8 +61,6 @@ scan_ag_metadata( struct scrub_item fix_now; struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; - struct action_list alist; - struct action_list immediate_alist; char descr[DESCR_BUFSZ]; unsigned int difficulty; int ret; @@ -71,15 +69,13 @@ scan_ag_metadata( return; scrub_item_init_ag(&sri, agno); - action_list_init(&alist); - action_list_init(&immediate_alist); snprintf(descr, DESCR_BUFSZ, _("AG %u"), agno); /* * First we scrub and fix the AG headers, because we need * them to work well enough to check the AG btrees. */ - ret = scrub_ag_headers(ctx, agno, &alist, &sri); + ret = scrub_ag_headers(ctx, &sri); if (ret) goto err; @@ -89,7 +85,7 @@ scan_ag_metadata( goto err; /* Now scrub the AG btrees. */ - ret = scrub_ag_metadata(ctx, agno, &alist, &sri); + ret = scrub_ag_metadata(ctx, &sri); if (ret) goto err; @@ -126,7 +122,6 @@ scan_fs_metadata( void *arg) { struct scrub_item sri; - struct action_list alist; struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; unsigned int difficulty; @@ -136,8 +131,7 @@ scan_fs_metadata( goto out; scrub_item_init_fs(&sri); - action_list_init(&alist); - ret = scrub_fs_metadata(ctx, type, &alist, &sri); + ret = scrub_fs_metadata(ctx, type, &sri); if (ret) { sctl->aborted = true; goto out; @@ -172,7 +166,6 @@ phase2_func( .aborted = false, .rbm_done = false, }; - struct action_list alist; struct scrub_item sri; const struct xfrog_scrub_descr *sc = xfrog_scrubbers; xfs_agnumber_t agno; @@ -196,8 +189,7 @@ phase2_func( * If errors occur, this function will log them and return nonzero. */ scrub_item_init_ag(&sri, 0); - action_list_init(&alist); - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_SB, 0, &alist, &sri); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_SB, &sri); if (ret) goto out_wq; ret = repair_item_completely(ctx, &sri); diff --git a/scrub/phase3.c b/scrub/phase3.c index e602d8c7ec4..fa2eef4dea1 100644 --- a/scrub/phase3.c +++ b/scrub/phase3.c @@ -107,7 +107,6 @@ scrub_inode( struct xfs_bulkstat *bstat, void *arg) { - struct action_list alist; struct scrub_item sri; struct scrub_inode_ctx *ictx = arg; struct ptcounter *icount = ictx->icount; @@ -115,7 +114,6 @@ scrub_inode( int error; scrub_item_init_file(&sri, bstat); - action_list_init(&alist); background_sleep(); /* @@ -146,7 +144,7 @@ scrub_inode( fd = scrub_open_handle(handle); /* Scrub the inode. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_INODE, &alist, &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_INODE, &sri); if (error) goto out; @@ -155,13 +153,13 @@ scrub_inode( goto out; /* Scrub all block mappings. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTD, &alist, &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTD, &sri); if (error) goto out; - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTA, &alist, &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTA, &sri); if (error) goto out; - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTC, &alist, &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_BMBTC, &sri); if (error) goto out; @@ -182,25 +180,24 @@ scrub_inode( if (S_ISLNK(bstat->bs_mode) || !bstat->bs_mode) { /* Check symlink contents. */ error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_SYMLINK, - &alist, &sri); + &sri); if (error) goto out; } if (S_ISDIR(bstat->bs_mode) || !bstat->bs_mode) { /* Check the directory entries. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_DIR, &alist, - &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_DIR, &sri); if (error) goto out; } /* Check all the extended attributes. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_XATTR, &alist, &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_XATTR, &sri); if (error) goto out; /* Check parent pointers. */ - error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_PARENT, &alist, &sri); + error = scrub_file(ctx, fd, bstat, XFS_SCRUB_TYPE_PARENT, &sri); if (error) goto out; diff --git a/scrub/phase4.c b/scrub/phase4.c index 98518635b2b..230c559f07f 100644 --- a/scrub/phase4.c +++ b/scrub/phase4.c @@ -129,7 +129,6 @@ phase4_func( struct scrub_ctx *ctx) { struct xfs_fsop_geom fsgeom; - struct action_list alist; struct scrub_item sri; int ret; @@ -144,8 +143,7 @@ phase4_func( * metadata. If repairs fails, we'll come back during phase 7. */ scrub_item_init_fs(&sri); - action_list_init(&alist); - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_FSCOUNTERS, 0, &alist, &sri); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_FSCOUNTERS, &sri); if (ret) return ret; @@ -160,8 +158,7 @@ phase4_func( return ret; if (fsgeom.sick & XFS_FSOP_GEOM_SICK_QUOTACHECK) { - ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_QUOTACHECK, 0, - &alist, &sri); + ret = scrub_meta_type(ctx, XFS_SCRUB_TYPE_QUOTACHECK, &sri); if (ret) return ret; } @@ -170,7 +167,6 @@ phase4_func( ret = repair_item_corruption(ctx, &sri); if (ret) return ret; - action_list_discard(&alist); ret = repair_everything(ctx); if (ret) diff --git a/scrub/phase5.c b/scrub/phase5.c index 79bfea8f6b5..6c9a518db4d 100644 --- a/scrub/phase5.c +++ b/scrub/phase5.c @@ -386,7 +386,6 @@ check_fs_label( struct fs_scan_item { struct scrub_item sri; - struct action_list alist; bool *abortedp; unsigned int scrub_type; }; @@ -413,16 +412,14 @@ fs_scan_worker( nanosleep(&tv, NULL); } - ret = scrub_meta_type(ctx, item->scrub_type, 0, &item->alist, - &item->sri); + ret = scrub_meta_type(ctx, item->scrub_type, &item->sri); if (ret) { str_liberror(ctx, ret, _("checking fs scan metadata")); *item->abortedp = true; goto out; } - ret = action_list_process(ctx, &item->alist, - XRM_FINAL_WARNING | XRM_NOPROGRESS); + ret = repair_item_completely(ctx, &item->sri); if (ret) { str_liberror(ctx, ret, _("repairing fs scan metadata")); *item->abortedp = true; @@ -453,7 +450,6 @@ queue_fs_scan( return ret; } scrub_item_init_fs(&item->sri); - action_list_init(&item->alist); item->scrub_type = scrub_type; item->abortedp = abortedp; diff --git a/scrub/phase7.c b/scrub/phase7.c index 404bfb82243..02da6b42beb 100644 --- a/scrub/phase7.c +++ b/scrub/phase7.c @@ -100,7 +100,6 @@ phase7_func( { struct summary_counts totalcount = {0}; struct scrub_item sri; - struct action_list alist; struct ptvar *ptvar; unsigned long long used_data; unsigned long long used_rt; @@ -119,8 +118,7 @@ phase7_func( /* Check and fix the summary metadata. */ scrub_item_init_fs(&sri); - action_list_init(&alist); - error = scrub_summary_metadata(ctx, &alist, &sri); + error = scrub_summary_metadata(ctx, &sri); if (error) return error; error = repair_item_completely(ctx, &sri); diff --git a/scrub/scrub.c b/scrub/scrub.c index 54f397fb92a..ca3eea42ece 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -219,6 +219,7 @@ _("Optimizations of %s are possible."), _(xfrog_scrubbers[i].descr)); /* * Scrub a single XFS_SCRUB_TYPE_*, saving corruption reports for later. + * Do not call this function to repair file metadata. * * Returns 0 for success. If errors occur, this function will log them and * return a positive error code. @@ -227,18 +228,29 @@ int scrub_meta_type( struct scrub_ctx *ctx, unsigned int type, - xfs_agnumber_t agno, - struct action_list *alist, struct scrub_item *sri) { struct xfs_scrub_metadata meta = { .sm_type = type, - .sm_agno = agno, }; enum check_outcome fix; background_sleep(); + switch (xfrog_scrubbers[type].group) { + case XFROG_SCRUB_GROUP_AGHEADER: + case XFROG_SCRUB_GROUP_PERAG: + meta.sm_agno = sri->sri_agno; + break; + case XFROG_SCRUB_GROUP_FS: + case XFROG_SCRUB_GROUP_SUMMARY: + case XFROG_SCRUB_GROUP_NONE: + break; + default: + assert(0); + break; + } + /* Check the item. */ fix = xfs_check_metadata(ctx, &ctx->mnt, &meta, false); progress_add(1); @@ -267,8 +279,6 @@ static bool scrub_group( struct scrub_ctx *ctx, enum xfrog_scrub_group group, - xfs_agnumber_t agno, - struct action_list *alist, struct scrub_item *sri) { const struct xfrog_scrub_descr *sc; @@ -281,7 +291,7 @@ scrub_group( if (sc->group != group) continue; - ret = scrub_meta_type(ctx, type, agno, alist, sri); + ret = scrub_meta_type(ctx, type, sri); if (ret) return ret; } @@ -293,22 +303,18 @@ scrub_group( int scrub_ag_headers( struct scrub_ctx *ctx, - xfs_agnumber_t agno, - struct action_list *alist, struct scrub_item *sri) { - return scrub_group(ctx, XFROG_SCRUB_GROUP_AGHEADER, agno, alist, sri); + return scrub_group(ctx, XFROG_SCRUB_GROUP_AGHEADER, sri); } /* Scrub each AG's metadata btrees. */ int scrub_ag_metadata( struct scrub_ctx *ctx, - xfs_agnumber_t agno, - struct action_list *alist, struct scrub_item *sri) { - return scrub_group(ctx, XFROG_SCRUB_GROUP_PERAG, agno, alist, sri); + return scrub_group(ctx, XFROG_SCRUB_GROUP_PERAG, sri); } /* Scrub whole-filesystem metadata. */ @@ -316,22 +322,20 @@ int scrub_fs_metadata( struct scrub_ctx *ctx, unsigned int type, - struct action_list *alist, struct scrub_item *sri) { ASSERT(xfrog_scrubbers[type].group == XFROG_SCRUB_GROUP_FS); - return scrub_meta_type(ctx, type, 0, alist, sri); + return scrub_meta_type(ctx, type, sri); } /* Scrub all FS summary metadata. */ int scrub_summary_metadata( struct scrub_ctx *ctx, - struct action_list *alist, struct scrub_item *sri) { - return scrub_group(ctx, XFROG_SCRUB_GROUP_SUMMARY, 0, alist, sri); + return scrub_group(ctx, XFROG_SCRUB_GROUP_SUMMARY, sri); } /* How many items do we have to check? */ @@ -393,7 +397,6 @@ scrub_file( int fd, const struct xfs_bulkstat *bstat, unsigned int type, - struct action_list *alist, struct scrub_item *sri) { struct xfs_scrub_metadata meta = {0}; diff --git a/scrub/scrub.h b/scrub/scrub.h index 0d6825a5a95..b2e91efac70 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -80,18 +80,13 @@ void scrub_item_dump(struct scrub_item *sri, unsigned int group_mask, const char *tag); void scrub_report_preen_triggers(struct scrub_ctx *ctx); -int scrub_ag_headers(struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist, struct scrub_item *sri); -int scrub_ag_metadata(struct scrub_ctx *ctx, xfs_agnumber_t agno, - struct action_list *alist, struct scrub_item *sri); +int scrub_ag_headers(struct scrub_ctx *ctx, struct scrub_item *sri); +int scrub_ag_metadata(struct scrub_ctx *ctx, struct scrub_item *sri); int scrub_fs_metadata(struct scrub_ctx *ctx, unsigned int scrub_type, - struct action_list *alist, struct scrub_item *sri); -int scrub_iscan_metadata(struct scrub_ctx *ctx, struct action_list *alist, - struct scrub_item *sri); -int scrub_summary_metadata(struct scrub_ctx *ctx, struct action_list *alist, struct scrub_item *sri); +int scrub_iscan_metadata(struct scrub_ctx *ctx, struct scrub_item *sri); +int scrub_summary_metadata(struct scrub_ctx *ctx, struct scrub_item *sri); int scrub_meta_type(struct scrub_ctx *ctx, unsigned int type, - xfs_agnumber_t agno, struct action_list *alist, struct scrub_item *sri); bool can_scrub_fs_metadata(struct scrub_ctx *ctx); @@ -105,7 +100,6 @@ bool can_repair(struct scrub_ctx *ctx); bool can_force_rebuild(struct scrub_ctx *ctx); int scrub_file(struct scrub_ctx *ctx, int fd, const struct xfs_bulkstat *bstat, - unsigned int type, struct action_list *alist, - struct scrub_item *sri); + unsigned int type, struct scrub_item *sri); #endif /* XFS_SCRUB_SCRUB_H_ */ From patchwork Sun Dec 31 22:41:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507932 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90E11C129 for ; Sun, 31 Dec 2023 22:41:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L3ogzQS+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B4F3C433C7; Sun, 31 Dec 2023 22:41:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062484; bh=/zOO9hlp/QwysXdyL5i3ozIfutgrORhvOmhAGiyOgQQ=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=L3ogzQS+ED+E9dLKiJFTwcCYeERwZC+tSzd+ND/6F+Yr+FBjiHK2G+IiVbsJ+vrbL a5aku8Am06//g+scjlG73lvB4Op9iTLQoamraBpz9NqGQ1BXpSN/T9aSknXodMXKX9 vdtV1vtawSyKQuWbm0FCdd6FtImWFU2HsdJQp/hBSWJVR06L1aSh9qIBKSXMJD5gvl Gp5FW38jF4mRYQOiAwaIX0+1NEZW43gjmUOw4cuhoiabkYQTiMCrD4oOyf2CgSlnhD sPSYehmom0rGTjptBfAs5ZorUgxpOD+Hj1/21s5tqsURY6ebFzV7nKAlTOfpJL1EAR rYNrgkEZQGBaA== Date: Sun, 31 Dec 2023 14:41:24 -0800 Subject: [PATCH 4/9] xfs_scrub: remove scrub_metadata_file From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999499.1797790.4054299004240774717.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Collapse this function with scrub_meta_type. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/phase2.c | 2 +- scrub/scrub.c | 12 ------------ scrub/scrub.h | 2 -- 3 files changed, 1 insertion(+), 15 deletions(-) diff --git a/scrub/phase2.c b/scrub/phase2.c index 4d4552d8477..4d90291ed14 100644 --- a/scrub/phase2.c +++ b/scrub/phase2.c @@ -131,7 +131,7 @@ scan_fs_metadata( goto out; scrub_item_init_fs(&sri); - ret = scrub_fs_metadata(ctx, type, &sri); + ret = scrub_meta_type(ctx, type, &sri); if (ret) { sctl->aborted = true; goto out; diff --git a/scrub/scrub.c b/scrub/scrub.c index ca3eea42ece..5c14ed2092e 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -317,18 +317,6 @@ scrub_ag_metadata( return scrub_group(ctx, XFROG_SCRUB_GROUP_PERAG, sri); } -/* Scrub whole-filesystem metadata. */ -int -scrub_fs_metadata( - struct scrub_ctx *ctx, - unsigned int type, - struct scrub_item *sri) -{ - ASSERT(xfrog_scrubbers[type].group == XFROG_SCRUB_GROUP_FS); - - return scrub_meta_type(ctx, type, sri); -} - /* Scrub all FS summary metadata. */ int scrub_summary_metadata( diff --git a/scrub/scrub.h b/scrub/scrub.h index b2e91efac70..874e1fe1319 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -82,8 +82,6 @@ void scrub_item_dump(struct scrub_item *sri, unsigned int group_mask, void scrub_report_preen_triggers(struct scrub_ctx *ctx); int scrub_ag_headers(struct scrub_ctx *ctx, struct scrub_item *sri); int scrub_ag_metadata(struct scrub_ctx *ctx, struct scrub_item *sri); -int scrub_fs_metadata(struct scrub_ctx *ctx, unsigned int scrub_type, - struct scrub_item *sri); int scrub_iscan_metadata(struct scrub_ctx *ctx, struct scrub_item *sri); int scrub_summary_metadata(struct scrub_ctx *ctx, struct scrub_item *sri); int scrub_meta_type(struct scrub_ctx *ctx, unsigned int type, From patchwork Sun Dec 31 22:41:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507933 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BBCCD512 for ; Sun, 31 Dec 2023 22:41:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BOHPSfvh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1A3C5C433C8; Sun, 31 Dec 2023 22:41:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062500; bh=ZEpHer0TNuXDO9XH3Wyvi0fpZgxrprmgPhjXvMD29+0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=BOHPSfvhu/ec9hvK1c6bvnvTqRnyGM8K8dpBI9qUjz59GXeDoEtI+xpXIpCtyguoj lUO9u682qFZvSWSjxTbwc91aGH4sEXfHUaePqWo2YcYmt4P5ULSuT45KGuxZwPedX0 s0FVRecMITf/1EQIrisVzdlgcSIggmXtms5R7/E/foxGy0iubKdQpeVnws6ALHLRAN dYlJtwNsa1NCCpP9WK82SEZsbmQPI5yDw+RrN0RvQaQDACrAqbLeXXxmxQZL9BIMdu Ai2lsW201fd9wSLjFDziRD53xmAKAWZC0bb5aIvbpPqvvMruviy1xn8TXuB9jne7lt TkNfApUIAnHJQ== Date: Sun, 31 Dec 2023 14:41:39 -0800 Subject: [PATCH 5/9] xfs_scrub: boost the repair priority of dependencies of damaged items From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999513.1797790.8882968878143930752.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong In XFS, certain types of metadata objects depend on the correctness of lower level metadata objects. For example, directory blocks are stored in the data fork of directory files, which means that any issues with the inode core and the data fork should be dealt with before we try to repair a directory. xfs_scrub prioritises repairs by the severity of what the kernel scrub function reports -- anything directly observed to be corrupt get repaired first, then anything that had trouble with cross referencing, and finally anything that was correct but could be further optimised. Returning to the above example, if a directory data fork mapping offset is off by a bit flip, scrub will mark that as failing cross referencing, but it'll mark the directory as corrupt. Repair should check out the mapping problem before it tackles the directory. Do this by embedding a dependency table and using it to boost the priority of the repair_item fields as needed. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- libfrog/scrub.c | 1 scrub/repair.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++- scrub/scrub.h | 12 ++++++ scrub/scrub_private.h | 8 ++++ 4 files changed, 117 insertions(+), 3 deletions(-) diff --git a/libfrog/scrub.c b/libfrog/scrub.c index 1df2965fe2d..baaa4b4d940 100644 --- a/libfrog/scrub.c +++ b/libfrog/scrub.c @@ -150,6 +150,7 @@ const struct xfrog_scrub_descr xfrog_scrubbers[XFS_SCRUB_TYPE_NR] = { .group = XFROG_SCRUB_GROUP_NONE, }, }; +#undef DEP /* Invoke the scrub ioctl. Returns zero or negative error code. */ int diff --git a/scrub/repair.c b/scrub/repair.c index 6e09c592ed4..5f13f3c7a5f 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -22,6 +22,29 @@ /* General repair routines. */ +/* + * Bitmap showing the correctness dependencies between scrub types for repairs. + * There are no edges between AG btrees and AG headers because we can't mount + * the filesystem if the btree root pointers in the AG headers are wrong. + * Dependencies cannot cross scrub groups. + */ +#define DEP(x) (1U << (x)) +static const unsigned int repair_deps[XFS_SCRUB_TYPE_NR] = { + [XFS_SCRUB_TYPE_BMBTD] = DEP(XFS_SCRUB_TYPE_INODE), + [XFS_SCRUB_TYPE_BMBTA] = DEP(XFS_SCRUB_TYPE_INODE), + [XFS_SCRUB_TYPE_BMBTC] = DEP(XFS_SCRUB_TYPE_INODE), + [XFS_SCRUB_TYPE_DIR] = DEP(XFS_SCRUB_TYPE_BMBTD), + [XFS_SCRUB_TYPE_XATTR] = DEP(XFS_SCRUB_TYPE_BMBTA), + [XFS_SCRUB_TYPE_SYMLINK] = DEP(XFS_SCRUB_TYPE_BMBTD), + [XFS_SCRUB_TYPE_PARENT] = DEP(XFS_SCRUB_TYPE_DIR) | + DEP(XFS_SCRUB_TYPE_XATTR), + [XFS_SCRUB_TYPE_QUOTACHECK] = DEP(XFS_SCRUB_TYPE_UQUOTA) | + DEP(XFS_SCRUB_TYPE_GQUOTA) | + DEP(XFS_SCRUB_TYPE_PQUOTA), + [XFS_SCRUB_TYPE_RTSUM] = DEP(XFS_SCRUB_TYPE_RTBITMAP), +}; +#undef DEP + /* Repair some metadata. */ static enum check_outcome xfs_repair_metadata( @@ -34,8 +57,16 @@ xfs_repair_metadata( struct xfs_scrub_metadata meta = { 0 }; struct xfs_scrub_metadata oldm; DEFINE_DESCR(dsc, ctx, format_scrub_descr); + bool repair_only; int error; + /* + * If the caller boosted the priority of this scrub type on behalf of a + * higher level repair by setting IFLAG_REPAIR, turn off REPAIR_ONLY. + */ + repair_only = (repair_flags & XRM_REPAIR_ONLY) && + scrub_item_type_boosted(sri, scrub_type); + assert(scrub_type < XFS_SCRUB_TYPE_NR); assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); meta.sm_type = scrub_type; @@ -55,7 +86,7 @@ xfs_repair_metadata( break; } - if (!is_corrupt(&meta) && (repair_flags & XRM_REPAIR_ONLY)) + if (!is_corrupt(&meta) && repair_only) return CHECK_RETRY; memcpy(&oldm, &meta, sizeof(oldm)); @@ -223,6 +254,60 @@ struct action_item { struct scrub_item sri; }; +/* + * The operation of higher level metadata objects depends on the correctness of + * lower level metadata objects. This means that if X depends on Y, we must + * investigate and correct all the observed issues with Y before we try to make + * a correction to X. For all scheduled repair activity on X, boost the + * priority of repairs on all the Ys to ensure this correctness. + */ +static void +repair_item_boost_priorities( + struct scrub_item *sri) +{ + unsigned int scrub_type; + + foreach_scrub_type(scrub_type) { + unsigned int dep_mask = repair_deps[scrub_type]; + unsigned int b; + + if (repair_item_count_needsrepair(sri) == 0 || !dep_mask) + continue; + + /* + * Check if the repairs for this scrub type depend on any other + * scrub types that have been flagged with cross-referencing + * errors and are not already tagged for the highest priority + * repair (SCRUB_ITEM_CORRUPT). If so, boost the priority of + * that scrub type (via SCRUB_ITEM_BOOST_REPAIR) so that any + * problems with the dependencies will (hopefully) be fixed + * before we start repairs on this scrub type. + * + * So far in the history of xfs_scrub we have maintained that + * lower numbered scrub types do not depend on higher numbered + * scrub types, so we need only process the bit mask once. + */ + for (b = 0; b < XFS_SCRUB_TYPE_NR; b++, dep_mask >>= 1) { + if (!dep_mask) + break; + if (!(dep_mask & 1)) + continue; + if (!(sri->sri_state[b] & SCRUB_ITEM_REPAIR_XREF)) + continue; + if (sri->sri_state[b] & SCRUB_ITEM_CORRUPT) + continue; + sri->sri_state[b] |= SCRUB_ITEM_BOOST_REPAIR; + } + } +} + +/* + * These are the scrub item state bits that must be copied when scheduling + * a (per-AG) scrub type for immediate repairs. The original state tracking + * bits are left untouched to force a rescan in phase 4. + */ +#define MUSTFIX_STATES (SCRUB_ITEM_CORRUPT | \ + SCRUB_ITEM_BOOST_REPAIR) /* * Figure out which AG metadata must be fixed before we can move on * to the inode scan. @@ -235,17 +320,21 @@ repair_item_mustfix( unsigned int scrub_type; assert(sri->sri_agno != -1U); + repair_item_boost_priorities(sri); scrub_item_init_ag(fix_now, sri->sri_agno); foreach_scrub_type(scrub_type) { - if (!(sri->sri_state[scrub_type] & SCRUB_ITEM_CORRUPT)) + unsigned int state; + + state = sri->sri_state[scrub_type] & MUSTFIX_STATES; + if (!state) continue; switch (scrub_type) { case XFS_SCRUB_TYPE_AGI: case XFS_SCRUB_TYPE_FINOBT: case XFS_SCRUB_TYPE_INOBT: - fix_now->sri_state[scrub_type] |= SCRUB_ITEM_CORRUPT; + fix_now->sri_state[scrub_type] = state; break; } } @@ -479,6 +568,8 @@ repair_file_corruption( struct scrub_item *sri, int override_fd) { + repair_item_boost_priorities(sri); + return repair_item_class(ctx, sri, override_fd, SCRUB_ITEM_CORRUPT, XRM_REPAIR_ONLY | XRM_NOPROGRESS); } @@ -495,6 +586,8 @@ repair_item( { int ret; + repair_item_boost_priorities(sri); + ret = repair_item_class(ctx, sri, -1, SCRUB_ITEM_CORRUPT, flags); if (ret) return ret; diff --git a/scrub/scrub.h b/scrub/scrub.h index 874e1fe1319..f22a952629e 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -14,6 +14,14 @@ enum check_outcome { CHECK_RETRY, /* repair failed, try again later */ }; +/* + * This flag boosts the repair priority of a scrub item when a dependent scrub + * item is scheduled for repair. Use a separate flag to preserve the + * corruption state that we got from the kernel. Priority boost is cleared the + * next time xfs_repair_metadata is called. + */ +#define SCRUB_ITEM_BOOST_REPAIR (1 << 0) + /* * These flags record the metadata object state that the kernel returned. * We want to remember if the object was corrupt, if the cross-referencing @@ -31,6 +39,10 @@ enum check_outcome { SCRUB_ITEM_XFAIL | \ SCRUB_ITEM_XCORRUPT) +/* Cross-referencing failures only. */ +#define SCRUB_ITEM_REPAIR_XREF (SCRUB_ITEM_XFAIL | \ + SCRUB_ITEM_XCORRUPT) + struct scrub_item { /* * Information we need to call the scrub and repair ioctls. Per-AG diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index 090efb54c0a..08b9130cbc9 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -71,4 +71,12 @@ scrub_item_clean_state( sri->sri_state[scrub_type] = 0; } +static inline bool +scrub_item_type_boosted( + struct scrub_item *sri, + unsigned int scrub_type) +{ + return sri->sri_state[scrub_type] & SCRUB_ITEM_BOOST_REPAIR; +} + #endif /* XFS_SCRUB_SCRUB_PRIVATE_H_ */ From patchwork Sun Dec 31 22:41:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507934 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32B4BC14F for ; Sun, 31 Dec 2023 22:41:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Z9BgRlUS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B82CEC433C8; Sun, 31 Dec 2023 22:41:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062515; bh=at5XH/D9HBsNLQIsAWvBBgk1yrb6E/PRH61VbJhy02Q=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Z9BgRlUSuwGZj20+dIvtU9kNW14nvDC5pkH8PUgU4m1x3/QdrpxGtQM+KbX5kCukI b4TZ4ptxzeKFYN8k+6zmqoHkWpd6h0J2biIjPAF9Suryo7jICBDpu67DIQ55iKyPzp +rXIapcY2KUKoDj61XSE9ZAD0Muzjj94oYWYb/FCUDwiYSt0sgurUgnET+ypXpKr8d nr0vti5Wo3eQqH9J2IpCC6IWV44JtWYIuhdWFzF7WK8wDhGlWYDyWQuCQ1PuCDppRx +g8jq2S62sqrZo3ozhfn2ZiSt1jI9AIZ/bdNUIymo+Phy2Q0MRB94C4XwL4DT8oPs5 ILUAsNVSZ14ng== Date: Sun, 31 Dec 2023 14:41:55 -0800 Subject: [PATCH 6/9] xfs_scrub: clean up repair_item_difficulty a little From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999526.1797790.2646087585739839329.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Document the flags handling in repair_item_difficulty. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/repair.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/scrub/repair.c b/scrub/repair.c index 5f13f3c7a5f..d4521f50c68 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -340,6 +340,15 @@ repair_item_mustfix( } } +/* + * These scrub item states correspond to metadata that is inconsistent in some + * way and must be repaired. If too many metadata objects share these states, + * this can make repairs difficult. + */ +#define HARDREPAIR_STATES (SCRUB_ITEM_CORRUPT | \ + SCRUB_ITEM_XCORRUPT | \ + SCRUB_ITEM_XFAIL) + /* Determine if primary or secondary metadata are inconsistent. */ unsigned int repair_item_difficulty( @@ -349,9 +358,10 @@ repair_item_difficulty( unsigned int ret = 0; foreach_scrub_type(scrub_type) { - if (!(sri->sri_state[scrub_type] & (XFS_SCRUB_OFLAG_CORRUPT | - XFS_SCRUB_OFLAG_XCORRUPT | - XFS_SCRUB_OFLAG_XFAIL))) + unsigned int state; + + state = sri->sri_state[scrub_type] & HARDREPAIR_STATES; + if (!state) continue; switch (scrub_type) { From patchwork Sun Dec 31 22:42:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507935 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBD2EC127 for ; Sun, 31 Dec 2023 22:42:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HX97UcpO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3EDAEC433C8; Sun, 31 Dec 2023 22:42:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062531; bh=ZboCjy4sx/Qd+c78g9jqWIeeEajYNyKKNrM6M/IR0U4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=HX97UcpOrrJV6pPp7XcTmSjl4woGfT7yTnlv0RumSZahIEzjOFsQH4+sT6yKh1Ymb TIFCHSpv3si0b8UchoNkeAo2VQHNjzpbiPv+d8zJqJ0gqYyrbHqemwmIfzDwp05sY0 gQU8cVgcTRJdxr1WmkX+DWCV39FGdGl112lhGTeFiA9qq+f9eyTFojvd1xHT1Y1Whl Les6qBDt1+ndlq7X7gKatGWt2d9rvtr/MmkWw2Qi1PQ+b0Tig6dhBRZK1ZYACDhdXk dN1g3uKQtmxmgQAeUNCn5MEJlLEDlrRY9tw90ha4AyhR8aEx1iOOWcyErDi72ekRwj OTB4qzsDhBYJg== Date: Sun, 31 Dec 2023 14:42:10 -0800 Subject: [PATCH 7/9] xfs_scrub: check dependencies of a scrub type before repairing From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999539.1797790.7472502131864497541.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Now that we have a map of a scrub type to its dependent scrub types, use this information to avoid trying to fix higher level metadata before the lower levels have passed. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/repair.c | 32 ++++++++++++++++++++++++++++++++ scrub/scrub.h | 5 +++++ 2 files changed, 37 insertions(+) diff --git a/scrub/repair.c b/scrub/repair.c index d4521f50c68..9b4b5d01626 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -497,6 +497,29 @@ action_list_process( return ret; } +/* Decide if the dependent scrub types of the given scrub type are ok. */ +static bool +repair_item_dependencies_ok( + const struct scrub_item *sri, + unsigned int scrub_type) +{ + unsigned int dep_mask = repair_deps[scrub_type]; + unsigned int b; + + for (b = 0; dep_mask && b < XFS_SCRUB_TYPE_NR; b++, dep_mask >>= 1) { + if (!(dep_mask & 1)) + continue; + /* + * If this lower level object also needs repair, we can't fix + * the higher level item. + */ + if (sri->sri_state[b] & SCRUB_ITEM_NEEDSREPAIR) + return false; + } + + return true; +} + /* * For a given filesystem object, perform all repairs of a given class * (corrupt, xcorrupt, xfail, preen) if the repair item says it's needed. @@ -536,6 +559,15 @@ repair_item_class( if (!(sri->sri_state[scrub_type] & repair_mask)) continue; + /* + * Don't try to repair higher level items if their lower-level + * dependencies haven't been verified, unless this is our last + * chance to fix things without complaint. + */ + if (!(flags & XRM_FINAL_WARNING) && + !repair_item_dependencies_ok(sri, scrub_type)) + continue; + fix = xfs_repair_metadata(ctx, xfdp, scrub_type, sri, flags); switch (fix) { case CHECK_DONE: diff --git a/scrub/scrub.h b/scrub/scrub.h index f22a952629e..3ae0bfd2952 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -43,6 +43,11 @@ enum check_outcome { #define SCRUB_ITEM_REPAIR_XREF (SCRUB_ITEM_XFAIL | \ SCRUB_ITEM_XCORRUPT) +/* Mask of bits signalling that a piece of metadata requires attention. */ +#define SCRUB_ITEM_NEEDSREPAIR (SCRUB_ITEM_CORRUPT | \ + SCRUB_ITEM_XFAIL | \ + SCRUB_ITEM_XCORRUPT) + struct scrub_item { /* * Information we need to call the scrub and repair ioctls. Per-AG From patchwork Sun Dec 31 22:42:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507936 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2087AC147 for ; Sun, 31 Dec 2023 22:42:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KusABDnu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E2CD8C433C8; Sun, 31 Dec 2023 22:42:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062547; bh=G99nYKnKxgolGdbJcvfVjtSZBCLd2GfV8FhjfZhjWGk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=KusABDnuewOPGwIbfQkPqkhg/ZAebDpTssJYJZp98gEem5C3iQX3jw/JnmiTfpaju qyL8fI6eL4yco0aSMjVikw15THzS7bP04fs2mXP0Qxx1HWa58mlPMqpj8Ha0F5F8IP ygZZfqerZrclSusW0XCK/U+jAAwT23xJ3MkH/XueVxOqqFdCZGGeg87mOQGAAFYv/m Dee+6CP6VZM0Q83bLJDtm9fdiN7RJsUZ5DfsPt0LZjCBIHro0QD67gG0unS7HfDnCa Euyji7/0yLH9XgiIEEESS9AeAn+lo4Px76ZDg7g3yvfbwVeZRgRqFddDxoeRxlC5Ha w0zukII4fGxcA== Date: Sun, 31 Dec 2023 14:42:26 -0800 Subject: [PATCH 8/9] xfs_scrub: retry incomplete repairs From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999552.1797790.13157454061191356913.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong If a repair says it didn't do anything on account of not being able to complete a scan of the metadata, retry the repair a few times; if even that doesn't work, we can delay it to phase 4. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/repair.c | 15 ++++++++++++++- scrub/scrub.c | 3 +-- scrub/scrub_private.h | 10 ++++++++++ 3 files changed, 25 insertions(+), 3 deletions(-) diff --git a/scrub/repair.c b/scrub/repair.c index 9b4b5d01626..2b863bb4195 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -58,6 +58,7 @@ xfs_repair_metadata( struct xfs_scrub_metadata oldm; DEFINE_DESCR(dsc, ctx, format_scrub_descr); bool repair_only; + unsigned int tries = 0; int error; /* @@ -99,6 +100,7 @@ xfs_repair_metadata( str_info(ctx, descr_render(&dsc), _("Attempting optimization.")); +retry: error = -xfrog_scrub_metadata(xfdp, &meta); switch (error) { case 0: @@ -179,9 +181,20 @@ _("Read-only filesystem; cannot make changes.")); return CHECK_DONE; } + /* + * If the kernel says the repair was incomplete or that there was a + * cross-referencing discrepancy but no obvious corruption, we'll try + * the repair again, just in case the fs was busy. Only retry so many + * times. + */ + if (want_retry(&meta) && tries < 10) { + tries++; + goto retry; + } + if (repair_flags & XRM_FINAL_WARNING) scrub_warn_incomplete_scrub(ctx, &dsc, &meta); - if (needs_repair(&meta)) { + if (needs_repair(&meta) || is_incomplete(&meta)) { /* * Still broken; if we've been told not to complain then we * just requeue this and try again later. Otherwise we diff --git a/scrub/scrub.c b/scrub/scrub.c index 5c14ed2092e..5fc549f9728 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -137,8 +137,7 @@ _("Filesystem is shut down, aborting.")); * we'll try the scan again, just in case the fs was busy. * Only retry so many times. */ - if (tries < 10 && (is_incomplete(meta) || - (xref_disagrees(meta) && !is_corrupt(meta)))) { + if (want_retry(meta) && tries < 10) { tries++; goto retry; } diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index 08b9130cbc9..53372e1f322 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -49,6 +49,16 @@ static inline bool needs_repair(struct xfs_scrub_metadata *sm) return is_corrupt(sm) || xref_disagrees(sm); } +/* + * We want to retry an operation if the kernel says it couldn't complete the + * scan/repair; or if there were cross-referencing problems but the object was + * not obviously corrupt. + */ +static inline bool want_retry(struct xfs_scrub_metadata *sm) +{ + return is_incomplete(sm) || (xref_disagrees(sm) && !is_corrupt(sm)); +} + void scrub_warn_incomplete_scrub(struct scrub_ctx *ctx, struct descr *dsc, struct xfs_scrub_metadata *meta); From patchwork Sun Dec 31 22:42:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507937 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE588C13B for ; Sun, 31 Dec 2023 22:42:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="X0o9Yc/e" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7D103C433C8; Sun, 31 Dec 2023 22:42:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704062562; bh=2FkJ3HvnyNmiDp35GmWSvYVVayTVtwQ6IhctgNtAXac=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=X0o9Yc/evHjzIbCCSn867AE3H02HKl5YPozPZyJjRVh8A0Ds4ticqn1ugctvYD2VD CHr1URXu8p5Qcz8B7a3tG3wTJiab0I5pPBcwUevTSvO/SuZKaRkXLGkIzBeA8kGgrc eVxT8LSXKER8yY9fNvanc42xHLZQuFdYJBH5BgX4AsUz5idRxWoJlM+XZK7XmDFOhn bFwB6ZvM2q926I21qqdr6KN6UQQToYZmKnr844rWc9lSn6SrbOWOrYzGk+8ovG5oX7 nF4kun90u/Q+HZuWPBp6guw1EkegOKlIWzsYv4VmpTrcMn6HrpBZbR5H2s08va04d5 SSc5LHljhqZmQ== Date: Sun, 31 Dec 2023 14:42:42 -0800 Subject: [PATCH 9/9] xfs_scrub: remove unused action_list fields From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404999565.1797790.5249264048650632926.stgit@frogsfrogsfrogs> In-Reply-To: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> References: <170404999439.1797790.8016278650267736019.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Remove some fields since we don't need them anymore. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- scrub/repair.c | 5 ----- scrub/repair.h | 2 -- 2 files changed, 7 deletions(-) diff --git a/scrub/repair.c b/scrub/repair.c index 2b863bb4195..a3a8fb311d0 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -432,7 +432,6 @@ action_list_discard( struct action_item *n; list_for_each_entry_safe(aitem, n, &alist->list, list) { - alist->nr--; list_del(&aitem->list); free(aitem); } @@ -453,8 +452,6 @@ action_list_init( struct action_list *alist) { INIT_LIST_HEAD(&alist->list); - alist->nr = 0; - alist->sorted = false; } /* Number of pending repairs in this list. */ @@ -478,8 +475,6 @@ action_list_add( struct action_item *aitem) { list_add_tail(&aitem->list, &alist->list); - alist->nr++; - alist->sorted = false; } /* Repair everything on this list. */ diff --git a/scrub/repair.h b/scrub/repair.h index 463a3f9bfef..a38cdd5e6df 100644 --- a/scrub/repair.h +++ b/scrub/repair.h @@ -8,8 +8,6 @@ struct action_list { struct list_head list; - unsigned long long nr; - bool sorted; }; struct action_item;