From patchwork Sun Dec 31 22:10:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507813 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15F69C127 for ; Sun, 31 Dec 2023 22:10:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QkNx0HKu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB810C433C8; Sun, 31 Dec 2023 22:10:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704060638; bh=pJkQ4EBOjTt2FeYaolhIlsXJ8GwZKXwo3NFqTegQHEE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=QkNx0HKurUxAz3hXRuOxQGtd2NB0FeDCVksT2EpKabsCqdMYjbf9t0t37RO2OLDVY R2HNWy8NAofCYnfgv2fIZm3RoP45XErES6x9a9+dAq2UeWqEPXQkHVJgBGH4vkd0VG l81Jg6XD2W2CsgDWjFAJ4PHFOAs5ORQq5LhCQkghgL4lLrxOBvsmQdjPJP2XTGmBYL AelMNV6ZP/RNO+A8urSBNS/LSSkcwAROJuW4ufiylzF2EzKhG2xNV5qUl9laKMOTSn vCKb/3j0dpTjEoO1usYANy2VCqTKEISouAsKBT+BrmCCu4JPcLLVb8WrBye9D+zRwz uN8jXBFmqPXkw== Date: Sun, 31 Dec 2023 14:10:38 -0800 Subject: [PATCH 1/3] xfs: report health of inode link counts From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404991587.1793944.15546271111816175118.stgit@frogsfrogsfrogs> In-Reply-To: <170404991573.1793944.10238192046951704393.stgit@frogsfrogsfrogs> References: <170404991573.1793944.10238192046951704393.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Report on the health of the inode link counts. Signed-off-by: Darrick J. Wong --- libxfs/xfs_fs.h | 1 + libxfs/xfs_health.h | 4 +++- spaceman/health.c | 4 ++++ 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/libxfs/xfs_fs.h b/libxfs/xfs_fs.h index 07acbed9235..f10d0aa0e33 100644 --- a/libxfs/xfs_fs.h +++ b/libxfs/xfs_fs.h @@ -196,6 +196,7 @@ struct xfs_fsop_geom { #define XFS_FSOP_GEOM_SICK_RT_BITMAP (1 << 4) /* realtime bitmap */ #define XFS_FSOP_GEOM_SICK_RT_SUMMARY (1 << 5) /* realtime summary */ #define XFS_FSOP_GEOM_SICK_QUOTACHECK (1 << 6) /* quota counts */ +#define XFS_FSOP_GEOM_SICK_NLINKS (1 << 7) /* inode link counts */ /* Output for XFS_FS_COUNTS */ typedef struct xfs_fsop_counts { diff --git a/libxfs/xfs_health.h b/libxfs/xfs_health.h index 5626e53b3f0..2bfe2dc404a 100644 --- a/libxfs/xfs_health.h +++ b/libxfs/xfs_health.h @@ -42,6 +42,7 @@ struct xfs_fsop_geom; #define XFS_SICK_FS_GQUOTA (1 << 2) /* group quota */ #define XFS_SICK_FS_PQUOTA (1 << 3) /* project quota */ #define XFS_SICK_FS_QUOTACHECK (1 << 4) /* quota counts */ +#define XFS_SICK_FS_NLINKS (1 << 5) /* inode link counts */ /* Observable health issues for realtime volume metadata. */ #define XFS_SICK_RT_BITMAP (1 << 0) /* realtime bitmap */ @@ -79,7 +80,8 @@ struct xfs_fsop_geom; XFS_SICK_FS_UQUOTA | \ XFS_SICK_FS_GQUOTA | \ XFS_SICK_FS_PQUOTA | \ - XFS_SICK_FS_QUOTACHECK) + XFS_SICK_FS_QUOTACHECK | \ + XFS_SICK_FS_NLINKS) #define XFS_SICK_RT_PRIMARY (XFS_SICK_RT_BITMAP | \ XFS_SICK_RT_SUMMARY) diff --git a/spaceman/health.c b/spaceman/health.c index 3318f9d1a7f..88b12c0b0ea 100644 --- a/spaceman/health.c +++ b/spaceman/health.c @@ -76,6 +76,10 @@ static const struct flag_map fs_flags[] = { .mask = XFS_FSOP_GEOM_SICK_QUOTACHECK, .descr = "quota counts", }, + { + .mask = XFS_FSOP_GEOM_SICK_NLINKS, + .descr = "inode link counts", + }, {0}, }; From patchwork Sun Dec 31 22:10:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507814 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F01AEC13B for ; Sun, 31 Dec 2023 22:10:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fxOuwyVT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C70BC433C8; Sun, 31 Dec 2023 22:10:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704060654; bh=SlY9SlVgzxiMeq92njfRQSzrBtavHUWPIpccILoxJNQ=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=fxOuwyVTUmRhNwcVKz9oEwnMsPAeNOErf6SK8CeYZnXE8F1EJ/GL/fJE17oQ+tGL1 enFg63bL0c8gKCg69T+rU9TsqMAAF3MSva4wOTij71hEMbmRxMNsMmGvQ7xG7UfzTH wtGxL9hQXe1vSQYnpRHg80YqVT8a+VzTsWan/K/1Dj1PJor7i+vMTIowS8jGz7nWzl I2O1Nq5TXBRTeRHRFyh/ASJuyhe53DiN62/svXfQrgQih02XSzHBaKsM01xc9Uiaxx 5z5G0SwxiiP3+reD7LpsT+l9y5UULPFG3ClrUKtB+acN1XRGL4LlJ4rwmzmz/x62qI 3nCPna/RWCzgg== Date: Sun, 31 Dec 2023 14:10:54 -0800 Subject: [PATCH 2/3] xfs: teach scrub to check file nlinks From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404991600.1793944.13700260385207472315.stgit@frogsfrogsfrogs> In-Reply-To: <170404991573.1793944.10238192046951704393.stgit@frogsfrogsfrogs> References: <170404991573.1793944.10238192046951704393.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Create the necessary scrub code to walk the filesystem's directory tree so that we can compute file link counts. Similar to quotacheck, we create an incore shadow array of link count information and then we walk the filesystem a second time to compare the link counts. We need live updates to keep the information up to date during the lengthy scan, so this scrubber remains disabled until the next patch. Signed-off-by: Darrick J. Wong --- libfrog/scrub.c | 5 +++++ libxfs/xfs_fs.h | 3 ++- man/man2/ioctl_xfs_scrub_metadata.2 | 4 ++++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/libfrog/scrub.c b/libfrog/scrub.c index 53c47bc2b5d..b6b8ae042c4 100644 --- a/libfrog/scrub.c +++ b/libfrog/scrub.c @@ -139,6 +139,11 @@ const struct xfrog_scrub_descr xfrog_scrubbers[XFS_SCRUB_TYPE_NR] = { .descr = "quota counters", .group = XFROG_SCRUB_GROUP_ISCAN, }, + [XFS_SCRUB_TYPE_NLINKS] = { + .name = "nlinks", + .descr = "inode link counts", + .group = XFROG_SCRUB_GROUP_ISCAN, + }, }; /* Invoke the scrub ioctl. Returns zero or negative error code. */ diff --git a/libxfs/xfs_fs.h b/libxfs/xfs_fs.h index f10d0aa0e33..515cd27d3b3 100644 --- a/libxfs/xfs_fs.h +++ b/libxfs/xfs_fs.h @@ -712,9 +712,10 @@ struct xfs_scrub_metadata { #define XFS_SCRUB_TYPE_PQUOTA 23 /* project quotas */ #define XFS_SCRUB_TYPE_FSCOUNTERS 24 /* fs summary counters */ #define XFS_SCRUB_TYPE_QUOTACHECK 25 /* quota counters */ +#define XFS_SCRUB_TYPE_NLINKS 26 /* inode link counts */ /* Number of scrub subcommands. */ -#define XFS_SCRUB_TYPE_NR 26 +#define XFS_SCRUB_TYPE_NR 27 /* i: Repair this metadata. */ #define XFS_SCRUB_IFLAG_REPAIR (1u << 0) diff --git a/man/man2/ioctl_xfs_scrub_metadata.2 b/man/man2/ioctl_xfs_scrub_metadata.2 index 046e3e3657b..8e8bb72fb3b 100644 --- a/man/man2/ioctl_xfs_scrub_metadata.2 +++ b/man/man2/ioctl_xfs_scrub_metadata.2 @@ -164,6 +164,10 @@ Examine all user, group, or project quota records for corruption. .B XFS_SCRUB_TYPE_FSCOUNTERS Examine all filesystem summary counters (free blocks, inode count, free inode count) for errors. + +.TP +.B XFS_SCRUB_TYPE_NLINKS +Scan all inodes in the filesystem to verify each file's link count. .RE .PD 1 From patchwork Sun Dec 31 22:11:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13507815 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C840C12B for ; Sun, 31 Dec 2023 22:11:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V34LL8NG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2138DC433C7; Sun, 31 Dec 2023 22:11:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704060670; bh=1hrr0K5B3oXhe/BP8Hy7Ga4eVdiV30x1zsxF3eMrtHc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=V34LL8NGWgeklRF86SzZ0catDfpXQSKxaCJTBV6eqwZlofWeOJWKJ1ZaG95E++IYZ SH3axX7UE5zwzUtiw97a0itLLqFavktt22xSXVsmacKVHt9NLJqGXoeA4wauqpGqoC ZDgRXtDnkffFh+G4TW9aHPdiq/J7OqGdjej3zM4hjOrhSGpqnYQwFvvq3RYhJ59SMa Af0WZ37UKbpkiwuudjnmLz/pDGFq5mvTk4k9CO2I7l25IB9itwjW2kfUXbo5j3D6Iu HBvC0DngDJeRYv78Y2bXNdnWdi6j45kBD7hTb3wDreVB8nfS0LlBSXh+9yF39IAm33 cnIAezSe+8qEQ== Date: Sun, 31 Dec 2023 14:11:09 -0800 Subject: [PATCH 3/3] xfs_scrub: use multiple threads to run in-kernel metadata scrubs that scan inodes From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170404991614.1793944.11604736668794289069.stgit@frogsfrogsfrogs> In-Reply-To: <170404991573.1793944.10238192046951704393.stgit@frogsfrogsfrogs> References: <170404991573.1793944.10238192046951704393.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Instead of running the inode link count and quotacheck scanners in serial, run them in parallel, with a slight delay to stagger the work to reduce inode resource contention. Signed-off-by: Darrick J. Wong --- scrub/phase5.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++++++------ scrub/scrub.c | 18 +++---- scrub/scrub.h | 1 3 files changed, 145 insertions(+), 24 deletions(-) diff --git a/scrub/phase5.c b/scrub/phase5.c index 0a91e4f0640..b4c635d3452 100644 --- a/scrub/phase5.c +++ b/scrub/phase5.c @@ -384,26 +384,146 @@ check_fs_label( return error; } -/* Check directory connectivity. */ -int -phase5_func( - struct scrub_ctx *ctx) -{ +typedef int (*fs_scan_item_fn)(struct scrub_ctx *, struct action_list *); + +struct fs_scan_item { struct action_list alist; - bool aborted = false; + bool *abortedp; + fs_scan_item_fn scrub_fn; +}; + +/* Run one full-fs scan scrubber in this thread. */ +static void +fs_scan_worker( + struct workqueue *wq, + xfs_agnumber_t nr, + void *arg) +{ + struct timespec tv; + struct fs_scan_item *item = arg; + struct scrub_ctx *ctx = wq->wq_ctx; int ret; /* - * Check and fix anything that requires a full inode scan. We do this - * after we've checked all inodes and repaired anything that could get - * in the way of a scan. + * Delay each successive fs scan by a second so that the threads are + * less likely to contend on the inobt and inode buffers. */ - action_list_init(&alist); - ret = scrub_iscan_metadata(ctx, &alist); - if (ret) - return ret; - ret = action_list_process(ctx, ctx->mnt.fd, &alist, + if (nr) { + tv.tv_sec = nr; + tv.tv_nsec = 0; + nanosleep(&tv, NULL); + } + + ret = item->scrub_fn(ctx, &item->alist); + if (ret) { + str_liberror(ctx, ret, _("checking fs scan metadata")); + *item->abortedp = true; + goto out; + } + + ret = action_list_process(ctx, ctx->mnt.fd, &item->alist, ALP_COMPLAIN_IF_UNFIXED | ALP_NOPROGRESS); + if (ret) { + str_liberror(ctx, ret, _("repairing fs scan metadata")); + *item->abortedp = true; + goto out; + } + +out: + free(item); + return; +} + +/* Queue one full-fs scan scrubber. */ +static int +queue_fs_scan( + struct workqueue *wq, + bool *abortedp, + xfs_agnumber_t nr, + fs_scan_item_fn scrub_fn) +{ + struct fs_scan_item *item; + struct scrub_ctx *ctx = wq->wq_ctx; + int ret; + + item = malloc(sizeof(struct fs_scan_item)); + if (!item) { + ret = ENOMEM; + str_liberror(ctx, ret, _("setting up fs scan")); + return ret; + } + action_list_init(&item->alist); + item->scrub_fn = scrub_fn; + item->abortedp = abortedp; + + ret = -workqueue_add(wq, fs_scan_worker, nr, item); + if (ret) + str_liberror(ctx, ret, _("queuing fs scan work")); + + return ret; +} + +/* Run multiple full-fs scan scrubbers at the same time. */ +static int +run_kernel_fs_scan_scrubbers( + struct scrub_ctx *ctx) +{ + struct workqueue wq_fs_scan; + unsigned int nr_threads = scrub_nproc_workqueue(ctx); + xfs_agnumber_t nr = 0; + bool aborted = false; + int ret, ret2; + + ret = -workqueue_create(&wq_fs_scan, (struct xfs_mount *)ctx, + nr_threads); + if (ret) { + str_liberror(ctx, ret, _("setting up fs scan workqueue")); + return ret; + } + + /* + * The nlinks scanner is much faster than quotacheck because it only + * walks directories, so we start it first. + */ + ret = queue_fs_scan(&wq_fs_scan, &aborted, nr, scrub_nlinks); + if (ret) + goto wait; + + if (nr_threads > 1) + nr++; + + ret = queue_fs_scan(&wq_fs_scan, &aborted, nr, scrub_quotacheck); + if (ret) + goto wait; + +wait: + ret2 = -workqueue_terminate(&wq_fs_scan); + if (ret2) { + str_liberror(ctx, ret2, _("joining fs scan workqueue")); + if (!ret) + ret = ret2; + } + if (aborted && !ret) + ret = ECANCELED; + + workqueue_destroy(&wq_fs_scan); + return ret; +} + +/* Check directory connectivity. */ +int +phase5_func( + struct scrub_ctx *ctx) +{ + bool aborted = false; + int ret; + + /* + * Check and fix anything that requires a full filesystem scan. We do + * this after we've checked all inodes and repaired anything that could + * get in the way of a scan. + */ + ret = run_kernel_fs_scan_scrubbers(ctx); if (ret) return ret; @@ -436,7 +556,7 @@ phase5_estimate( int *rshift) { *items = scrub_estimate_iscan_work(ctx); - *nr_threads = scrub_nproc(ctx); + *nr_threads = scrub_nproc(ctx) * 2; *rshift = 0; return 0; } diff --git a/scrub/scrub.c b/scrub/scrub.c index a22633a8115..b7ec54c16a4 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -422,15 +422,6 @@ scrub_summary_metadata( return scrub_group(ctx, XFROG_SCRUB_GROUP_SUMMARY, 0, alist); } -/* Scrub all metadata requiring a full inode scan. */ -int -scrub_iscan_metadata( - struct scrub_ctx *ctx, - struct action_list *alist) -{ - return scrub_group(ctx, XFROG_SCRUB_GROUP_ISCAN, 0, alist); -} - /* Scrub /only/ the superblock summary counters. */ int scrub_fs_counters( @@ -449,6 +440,15 @@ scrub_quotacheck( return scrub_meta_type(ctx, XFS_SCRUB_TYPE_QUOTACHECK, 0, alist); } +/* Scrub /only/ the file link counters. */ +int +scrub_nlinks( + struct scrub_ctx *ctx, + struct action_list *alist) +{ + return scrub_meta_type(ctx, XFS_SCRUB_TYPE_NLINKS, 0, alist); +} + /* How many items do we have to check? */ unsigned int scrub_estimate_ag_work( diff --git a/scrub/scrub.h b/scrub/scrub.h index 927f86de9ec..5e3f40bf1f4 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -28,6 +28,7 @@ int scrub_iscan_metadata(struct scrub_ctx *ctx, struct action_list *alist); int scrub_summary_metadata(struct scrub_ctx *ctx, struct action_list *alist); int scrub_fs_counters(struct scrub_ctx *ctx, struct action_list *alist); int scrub_quotacheck(struct scrub_ctx *ctx, struct action_list *alist); +int scrub_nlinks(struct scrub_ctx *ctx, struct action_list *alist); bool can_scrub_fs_metadata(struct scrub_ctx *ctx); bool can_scrub_inode(struct scrub_ctx *ctx);