From patchwork Tue Dec 31 23:53:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924091 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5289029415 for ; Tue, 31 Dec 2024 23:53:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689207; cv=none; b=KqxnzrqNC3tOxJxk8pMRWqUOKtWI4zvFiy7jEjnNASGwqmyjCIp+0Dz7Bt0S/kfan9HdyjzrSj6wjkceY+Pje1hh+if+/UvA01H287YNxN7aWJ1yARSWFwubFR0uVeSuw9HIXcyqAPwt4EYADNUi/5FuSvrT9hksRdB2VlxIlpk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689207; c=relaxed/simple; bh=w60tYK2jzdoYZUyA6gHs9YUSfTF6fop1OxP4IVuqBH0=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rWqoNdMGH1MuyQfvOQGk4pl6NDZNJiku9ugc3HHRmp7MZ/e2FKQuSzz41hTorijmseDuNbh0NafYfm9IjjstvW5KIrw3MY29XrasOyRFFgRZW9hedboAUKClV8+HSdE6Jk3NFaJo9t1tZTAQuXcUXb2cPpgd6aS3KucJEwJ55Yg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=X4XvWT/0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="X4XvWT/0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 303A1C4CED7; Tue, 31 Dec 2024 23:53:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689207; bh=w60tYK2jzdoYZUyA6gHs9YUSfTF6fop1OxP4IVuqBH0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=X4XvWT/0yaJ0oJiSstFmcsX7eytKT6qmtULCw4ux3EyN7QBXA8x5Qbcem8eNdQMFo 3xSB112B6sBBoF1egA90bYfNbP1GjzWsY/iGB5WClEg0teOLNnwA4FMfvEG0HcFQxK KBkSsf1taWWghP5c6kYfFEz/onga4ttSKOUU5kk8KOeDRKT6+chHQsZRDVjwngKFTC HMpi5SxHRVC0b1jauHaOqfDJuNtpgeTgvGV14Wk3jRNdXZ3k4ybULNLEYTAKHo5p1M RgAb43dTGpBSkUq7m0q9YZCd7+I+0s18O4c8mths8GCGEuVkQpYm/Jl/3aG2beUoDM pNxKDQ7oCabKQ== Date: Tue, 31 Dec 2024 15:53:26 -0800 Subject: [PATCH 01/10] xfs_repair: allow sysadmins to add free inode btree indexes From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779151.2710949.17310084377052885141.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow the sysadmin to use xfs_repair to upgrade an existing filesystem to support the free inode btree. Signed-off-by: "Darrick J. Wong" --- man/man8/xfs_admin.8 | 7 +++++++ repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 28 +++++++++++++++++++++++++++- repair/xfs_repair.c | 11 +++++++++++ 5 files changed, 47 insertions(+), 1 deletion(-) diff --git a/man/man8/xfs_admin.8 b/man/man8/xfs_admin.8 index 63f8ee90307b30..e07fc3ddb3fb82 100644 --- a/man/man8/xfs_admin.8 +++ b/man/man8/xfs_admin.8 @@ -163,6 +163,13 @@ .SH OPTIONS extended attributes, symbolic links, and realtime free space metadata. The filesystem cannot be downgraded after this feature is enabled. Once enabled, the filesystem will not be mountable by older kernels. +.TP 0.4i +.B finobt +Track free inodes through a separate free inode btree index to speed up inode +allocation on old filesystems. +This upgrade can fail if any AG has less than 1% free space remaining. +The filesystem cannot be downgraded after this feature is enabled. +This feature was added to Linux 3.16. .RE .TP .BI \-U " uuid" diff --git a/repair/globals.c b/repair/globals.c index 143b4a8beb53f4..f13497c3121d6b 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -53,6 +53,7 @@ bool add_inobtcount; /* add inode btree counts to AGI */ bool add_bigtime; /* add support for timestamps up to 2486 */ bool add_nrext64; bool add_exchrange; /* add file content exchange support */ +bool add_finobt; /* add free inode btrees */ /* misc status variables */ diff --git a/repair/globals.h b/repair/globals.h index 8bb9bbaeca4fb0..c5b27d9a60cf2e 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -94,6 +94,7 @@ extern bool add_inobtcount; /* add inode btree counts to AGI */ extern bool add_bigtime; /* add support for timestamps up to 2486 */ extern bool add_nrext64; extern bool add_exchrange; /* add file content exchange support */ +extern bool add_finobt; /* add free inode btrees */ /* misc status variables */ diff --git a/repair/phase2.c b/repair/phase2.c index 71576f5806e473..1bb7cd19025be7 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -123,7 +123,7 @@ set_inobtcount( exit(0); } - if (!xfs_has_finobt(mp)) { + if (!xfs_has_finobt(mp) && !add_finobt) { printf( _("Inode btree count feature requires free inode btree.\n")); exit(0); @@ -212,6 +212,28 @@ set_exchrange( return true; } +static bool +set_finobt( + struct xfs_mount *mp, + struct xfs_sb *new_sb) +{ + if (xfs_has_finobt(mp)) { + printf(_("Filesystem already supports free inode btrees.\n")); + exit(0); + } + + if (!xfs_has_crc(mp)) { + printf( + _("Free inode btree feature only supported on V5 filesystems.\n")); + exit(0); + } + + printf(_("Adding free inode btrees to filesystem.\n")); + new_sb->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_FINOBT; + new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR; + return true; +} + struct check_state { struct xfs_sb sb; uint64_t features; @@ -378,6 +400,8 @@ need_check_fs_free_space( struct xfs_mount *mp, const struct check_state *old) { + if (xfs_has_finobt(mp) && !(old->features & XFS_FEAT_FINOBT)) + return true; return false; } @@ -455,6 +479,8 @@ upgrade_filesystem( dirty |= set_nrext64(mp, &new_sb); if (add_exchrange) dirty |= set_exchrange(mp, &new_sb); + if (add_finobt) + dirty |= set_finobt(mp, &new_sb); if (!dirty) return; diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index 7bf75c09b94542..d8f92b52b66f3a 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -71,6 +71,7 @@ enum c_opt_nums { CONVERT_BIGTIME, CONVERT_NREXT64, CONVERT_EXCHRANGE, + CONVERT_FINOBT, C_MAX_OPTS, }; @@ -80,6 +81,7 @@ static char *c_opts[] = { [CONVERT_BIGTIME] = "bigtime", [CONVERT_NREXT64] = "nrext64", [CONVERT_EXCHRANGE] = "exchange", + [CONVERT_FINOBT] = "finobt", [C_MAX_OPTS] = NULL, }; @@ -372,6 +374,15 @@ process_args(int argc, char **argv) _("-c exchange only supports upgrades\n")); add_exchrange = true; break; + case CONVERT_FINOBT: + if (!val) + do_abort( + _("-c finobt requires a parameter\n")); + if (strtol(val, NULL, 0) != 1) + do_abort( + _("-c finobt only supports upgrades\n")); + add_finobt = true; + break; default: unknown('c', val); break; From patchwork Tue Dec 31 23:53:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924092 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA7C1B0425 for ; Tue, 31 Dec 2024 23:53:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689223; cv=none; b=YV9iMvEfeHypk3JLu3hguXc3sYEoHElPgZ/eKuNS50tZsJRKlcGhxsj6cyECjOk0rGyyefjailU18xhFYv3sIKVMPDNTx56vifQBniJj6XGBcuJFnp0RGh4Y3mt0lwaW+HgxTlb2oeZgIWsAQ9tIpGDFDWiT3nQlXULcDiCS3KE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689223; c=relaxed/simple; bh=tWCZS2jtY1yIgQCNdLu6jQtUt61/w2hguawMdZ3QOtU=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OttkEm1+mRIqg0liLOBH+9f0iUsq4roV3miXHScDfUUZUhCzt+kDlTWLiiG1/zFlLLnXe3YUZGYYmR4hTD8qFH4tT/W/dNTp7POK0F90P3npwBcnhX7GxflFNREnWuzsNRLNon8FvSrlKh5lucKkl2+01KLh6ZHABWqHAKoa0mo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ca9E2A/2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ca9E2A/2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C75A4C4CED2; Tue, 31 Dec 2024 23:53:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689222; bh=tWCZS2jtY1yIgQCNdLu6jQtUt61/w2hguawMdZ3QOtU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Ca9E2A/23cPXBHZvJR+Y9y4R9ehmXoPUwcc3eZ5kwX6QDOAhoEyQ4AM/d0saTCMrU brSeiGxMLJTKWYIPTKXYRWC0Q931uiRxf+AxHUZUAX0aD8s8g8HlJAsOx5fRluU596 v0tIgnYnCXycW57QX2PDdxEE9D6jNurHsxehxg+T1gf7nCT8Jvnlla9cW90YGyZo54 X5+77kApiUsUM9/4f1fmyvEAUpjp4i4Sv9ao+RoMSLr2bNJEcDWIU/6351r0/dzUXo Qal2CZ5eu+OTi5KxlqR27uBcVljG2r10ugCMnAyEmNanz7lEMUuHAjfRlh1NDWoHuZ 1iQjybdILv+Jg== Date: Tue, 31 Dec 2024 15:53:42 -0800 Subject: [PATCH 02/10] xfs_repair: allow sysadmins to add reflink From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779164.2710949.2907165533692467121.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow the sysadmin to use xfs_repair to upgrade an existing filesystem to support the reference count btree, and therefore reflink. Signed-off-by: "Darrick J. Wong" --- man/man8/xfs_admin.8 | 6 ++++++ repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 33 ++++++++++++++++++++++++++++++++- repair/rmap.c | 6 +++--- repair/xfs_repair.c | 11 +++++++++++ 6 files changed, 54 insertions(+), 4 deletions(-) diff --git a/man/man8/xfs_admin.8 b/man/man8/xfs_admin.8 index e07fc3ddb3fb82..3a9175c9f018e5 100644 --- a/man/man8/xfs_admin.8 +++ b/man/man8/xfs_admin.8 @@ -170,6 +170,12 @@ .SH OPTIONS This upgrade can fail if any AG has less than 1% free space remaining. The filesystem cannot be downgraded after this feature is enabled. This feature was added to Linux 3.16. +.TP 0.4i +.B reflink +Enable sharing of file data blocks. +This upgrade can fail if any AG has less than 2% free space remaining. +The filesystem cannot be downgraded after this feature is enabled. +This feature was added to Linux 4.9. .RE .TP .BI \-U " uuid" diff --git a/repair/globals.c b/repair/globals.c index f13497c3121d6b..cf4421e34dec84 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -54,6 +54,7 @@ bool add_bigtime; /* add support for timestamps up to 2486 */ bool add_nrext64; bool add_exchrange; /* add file content exchange support */ bool add_finobt; /* add free inode btrees */ +bool add_reflink; /* add reference count btrees */ /* misc status variables */ diff --git a/repair/globals.h b/repair/globals.h index c5b27d9a60cf2e..efbb8db79bc080 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -95,6 +95,7 @@ extern bool add_bigtime; /* add support for timestamps up to 2486 */ extern bool add_nrext64; extern bool add_exchrange; /* add file content exchange support */ extern bool add_finobt; /* add free inode btrees */ +extern bool add_reflink; /* add reference count btrees */ /* misc status variables */ diff --git a/repair/phase2.c b/repair/phase2.c index 1bb7cd19025be7..9cd841f8d05fc6 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -200,7 +200,7 @@ set_exchrange( exit(0); } - if (!xfs_has_reflink(mp)) { + if (!xfs_has_reflink(mp) && !add_reflink) { printf( _("File exchange-range feature cannot be added without reflink.\n")); exit(0); @@ -234,6 +234,33 @@ set_finobt( return true; } +static bool +set_reflink( + struct xfs_mount *mp, + struct xfs_sb *new_sb) +{ + if (xfs_has_reflink(mp)) { + printf(_("Filesystem already supports reflink.\n")); + exit(0); + } + + if (!xfs_has_crc(mp)) { + printf( + _("Reflink feature only supported on V5 filesystems.\n")); + exit(0); + } + + if (xfs_has_realtime(mp)) { + printf(_("Reflink feature not supported with realtime.\n")); + exit(0); + } + + printf(_("Adding reflink support to filesystem.\n")); + new_sb->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_REFLINK; + new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR; + return true; +} + struct check_state { struct xfs_sb sb; uint64_t features; @@ -402,6 +429,8 @@ need_check_fs_free_space( { if (xfs_has_finobt(mp) && !(old->features & XFS_FEAT_FINOBT)) return true; + if (xfs_has_reflink(mp) && !(old->features & XFS_FEAT_REFLINK)) + return true; return false; } @@ -481,6 +510,8 @@ upgrade_filesystem( dirty |= set_exchrange(mp, &new_sb); if (add_finobt) dirty |= set_finobt(mp, &new_sb); + if (add_reflink) + dirty |= set_reflink(mp, &new_sb); if (!dirty) return; diff --git a/repair/rmap.c b/repair/rmap.c index 97510dd875911a..91f864351f6013 100644 --- a/repair/rmap.c +++ b/repair/rmap.c @@ -68,7 +68,7 @@ bool rmap_needs_work( struct xfs_mount *mp) { - return xfs_has_reflink(mp) || + return xfs_has_reflink(mp) || add_reflink || xfs_has_rmapbt(mp); } @@ -1800,7 +1800,7 @@ check_refcounts( struct xfs_perag *pag = NULL; int error; - if (!xfs_has_reflink(mp)) + if (!xfs_has_reflink(mp) || add_reflink) return; if (refcbt_suspect) { if (no_modify && agno == 0) @@ -1859,7 +1859,7 @@ check_rtrefcounts( struct xfs_inode *ip = NULL; int error; - if (!xfs_has_reflink(mp)) + if (!xfs_has_reflink(mp) || add_reflink) return; if (refcbt_suspect) { if (no_modify && rgno == 0) diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index d8f92b52b66f3a..e436dc2ef736d6 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -72,6 +72,7 @@ enum c_opt_nums { CONVERT_NREXT64, CONVERT_EXCHRANGE, CONVERT_FINOBT, + CONVERT_REFLINK, C_MAX_OPTS, }; @@ -82,6 +83,7 @@ static char *c_opts[] = { [CONVERT_NREXT64] = "nrext64", [CONVERT_EXCHRANGE] = "exchange", [CONVERT_FINOBT] = "finobt", + [CONVERT_REFLINK] = "reflink", [C_MAX_OPTS] = NULL, }; @@ -383,6 +385,15 @@ process_args(int argc, char **argv) _("-c finobt only supports upgrades\n")); add_finobt = true; break; + case CONVERT_REFLINK: + if (!val) + do_abort( + _("-c reflink requires a parameter\n")); + if (strtol(val, NULL, 0) != 1) + do_abort( + _("-c reflink only supports upgrades\n")); + add_reflink = true; + break; default: unknown('c', val); break; From patchwork Tue Dec 31 23:53:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924093 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90ED229415 for ; Tue, 31 Dec 2024 23:53:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689238; cv=none; b=rJbvoYZwaVVUk7bPMO7fmhY7G43/S54spckQTQglxRlOmkazl4BhXs2kwVSG+uUB0nMlJEBKBNAyaLUYEItQEDdzdtkObFGepMGs/Y+J1G03aZeYydCK7eUORSiKtdXYGXIjqzws7C+Nwl9c18h856LYX6XmI4A9tZI53aiqIP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689238; c=relaxed/simple; bh=hwj1M6YNC4bPowm3o9GHTAOpb3z1TWp0D3wj4VWAKj4=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EjFnzUbPnCZceVdEUFtxFZ/qKjhhrMArkhI6z7prbJDUUVhghULG/FsmvvhBwAr4oOWCKMnyZN1a4xPY8We8kvisEySuy/zZGj98M3L15Jw/YsVVl1rr9yeAoUIeeLnJcRWMYxJQME3HXknZTeFO+ZShIqs6AX0mvlNCUqJeH94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HN7+mZ9i; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HN7+mZ9i" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D8C1C4CED2; Tue, 31 Dec 2024 23:53:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689238; bh=hwj1M6YNC4bPowm3o9GHTAOpb3z1TWp0D3wj4VWAKj4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=HN7+mZ9iJ0W9Ybro/lGGJaXPEFk9MvEE1u+p0yczRU6s3LWYCWjjWL2vneVo8uWhu 3X94VVIZn7aAEeUggEAEtWjbpMLVShYN7DWeOocWW843FOSBGVeiK8IRkUQm5yRJtj qVAQnjzE0MuJ5pxJNNdeiA9CjgCbd6Jqy6flMkVmqNf2hUnQZlGsE7/K3jtmapcY3b YQzP51edOirIuSfteQUUpd3d1ls5McOKaO1/RhIqFxWorn4oqf555WSsvmBu2Hjeo0 ieRlbI5SPs1TE934qMQZ20TGM6b/tJuWqY6CBjrwYVtZrxSHtbqOyG1XSx1457hLNJ Xv9sb9y/gx1Bg== Date: Tue, 31 Dec 2024 15:53:57 -0800 Subject: [PATCH 03/10] xfs_repair: allow sysadmins to add reverse mapping indexes From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779179.2710949.12292387414126353408.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow the sysadmin to use xfs_repair to upgrade an existing filesystem to support the reverse mapping btree index. This is needed for online fsck. Signed-off-by: "Darrick J. Wong" --- man/man8/xfs_admin.8 | 8 ++++++++ repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 38 ++++++++++++++++++++++++++++++++++++++ repair/rmap.c | 6 +++--- repair/xfs_repair.c | 11 +++++++++++ 6 files changed, 62 insertions(+), 3 deletions(-) diff --git a/man/man8/xfs_admin.8 b/man/man8/xfs_admin.8 index 3a9175c9f018e5..74a400dcfeb557 100644 --- a/man/man8/xfs_admin.8 +++ b/man/man8/xfs_admin.8 @@ -176,6 +176,14 @@ .SH OPTIONS This upgrade can fail if any AG has less than 2% free space remaining. The filesystem cannot be downgraded after this feature is enabled. This feature was added to Linux 4.9. +.TP 0.4i +.B rmapbt +Store an index of the owners of on-disk blocks. +This enables much stronger cross-referencing of various metadata structures +and online repairs to space usage metadata. +The filesystem cannot be downgraded after this feature is enabled. +This upgrade can fail if any AG has less than 5% free space remaining. +This feature was added to Linux 4.8. .RE .TP .BI \-U " uuid" diff --git a/repair/globals.c b/repair/globals.c index cf4421e34dec84..dd7c422bb922e4 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -55,6 +55,7 @@ bool add_nrext64; bool add_exchrange; /* add file content exchange support */ bool add_finobt; /* add free inode btrees */ bool add_reflink; /* add reference count btrees */ +bool add_rmapbt; /* add reverse mapping btrees */ /* misc status variables */ diff --git a/repair/globals.h b/repair/globals.h index efbb8db79bc080..d8c2aae23d8f0a 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -96,6 +96,7 @@ extern bool add_nrext64; extern bool add_exchrange; /* add file content exchange support */ extern bool add_finobt; /* add free inode btrees */ extern bool add_reflink; /* add reference count btrees */ +extern bool add_rmapbt; /* add reverse mapping btrees */ /* misc status variables */ diff --git a/repair/phase2.c b/repair/phase2.c index 9cd841f8d05fc6..9dd37e7fc5c111 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -261,6 +261,40 @@ set_reflink( return true; } +static bool +set_rmapbt( + struct xfs_mount *mp, + struct xfs_sb *new_sb) +{ + if (xfs_has_rmapbt(mp)) { + printf(_("Filesystem already supports reverse mapping btrees.\n")); + exit(0); + } + + if (!xfs_has_crc(mp)) { + printf( + _("Reverse mapping btree feature only supported on V5 filesystems.\n")); + exit(0); + } + + if (xfs_has_realtime(mp)) { + printf( + _("Reverse mapping btree feature not supported with realtime.\n")); + exit(0); + } + + if (xfs_has_reflink(mp) && !add_reflink) { + printf( + _("Reverse mapping btrees cannot be added when reflink is enabled.\n")); + exit(0); + } + + printf(_("Adding reverse mapping btrees to filesystem.\n")); + new_sb->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_RMAPBT; + new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR; + return true; +} + struct check_state { struct xfs_sb sb; uint64_t features; @@ -431,6 +465,8 @@ need_check_fs_free_space( return true; if (xfs_has_reflink(mp) && !(old->features & XFS_FEAT_REFLINK)) return true; + if (xfs_has_rmapbt(mp) && !(old->features & XFS_FEAT_RMAPBT)) + return true; return false; } @@ -512,6 +548,8 @@ upgrade_filesystem( dirty |= set_finobt(mp, &new_sb); if (add_reflink) dirty |= set_reflink(mp, &new_sb); + if (add_rmapbt) + dirty |= set_rmapbt(mp, &new_sb); if (!dirty) return; diff --git a/repair/rmap.c b/repair/rmap.c index 91f864351f6013..f1f837d33ea4f4 100644 --- a/repair/rmap.c +++ b/repair/rmap.c @@ -69,7 +69,7 @@ rmap_needs_work( struct xfs_mount *mp) { return xfs_has_reflink(mp) || add_reflink || - xfs_has_rmapbt(mp); + xfs_has_rmapbt(mp) || add_rmapbt; } static inline bool rmaps_has_observations(const struct xfs_ag_rmap *ag_rmap) @@ -1339,7 +1339,7 @@ rmaps_verify_btree( struct xfs_perag *pag = NULL; int error; - if (!xfs_has_rmapbt(mp)) + if (!xfs_has_rmapbt(mp) || add_rmapbt) return; if (rmapbt_suspect) { if (no_modify && agno == 0) @@ -1398,7 +1398,7 @@ rtrmaps_verify_btree( struct xfs_inode *ip = NULL; int error; - if (!xfs_has_rmapbt(mp)) + if (!xfs_has_rmapbt(mp) || add_rmapbt) return; if (rmapbt_suspect) { if (no_modify && rgno == 0) diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index e436dc2ef736d6..ca72c65f9d772a 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -73,6 +73,7 @@ enum c_opt_nums { CONVERT_EXCHRANGE, CONVERT_FINOBT, CONVERT_REFLINK, + CONVERT_RMAPBT, C_MAX_OPTS, }; @@ -84,6 +85,7 @@ static char *c_opts[] = { [CONVERT_EXCHRANGE] = "exchange", [CONVERT_FINOBT] = "finobt", [CONVERT_REFLINK] = "reflink", + [CONVERT_RMAPBT] = "rmapbt", [C_MAX_OPTS] = NULL, }; @@ -394,6 +396,15 @@ process_args(int argc, char **argv) _("-c reflink only supports upgrades\n")); add_reflink = true; break; + case CONVERT_RMAPBT: + if (!val) + do_abort( + _("-c rmapbt requires a parameter\n")); + if (strtol(val, NULL, 0) != 1) + do_abort( + _("-c rmapbt only supports upgrades\n")); + add_rmapbt = true; + break; default: unknown('c', val); break; From patchwork Tue Dec 31 23:54:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924094 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3609C29415 for ; Tue, 31 Dec 2024 23:54:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689254; cv=none; b=hOBZ1MmYjJSg39m4nZ1tKfKthuXBwcCSJyAPeRmlfJqKTitXPlRPvHqajnYJ4Aj3lji8cauAlyt//Z0QMBJSC6KsEww0HXLWO0cyxMCfKJiCJAxALBh5KkSHQvlqmWw3DU2DrgJMhlRn7ol9iGTqu/9a5F8Z4LcpDFIj5kOaDwE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689254; c=relaxed/simple; bh=d6K66dHoTlG5bT0Hs4rnmYy8y184dxMN3jdboHGmU+Q=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=A9OJX7fMI1iHIAUvW2TDxMy3Ozc+GIyykvtVUjHV+AP/kn6+4QhLUHjtzz51jAde8APxzAfb1kfkivvo1BHr9Syg6Fjpqb+YC+ncU/N3SVmrtGyQL58XaCPex85bGuPspZ+1kgTz/JSjvQE3IEby+c675bJvGbsnrtz0p7dW5R8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eCVz72Jy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eCVz72Jy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10E18C4CED2; Tue, 31 Dec 2024 23:54:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689254; bh=d6K66dHoTlG5bT0Hs4rnmYy8y184dxMN3jdboHGmU+Q=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=eCVz72JyVgnP8pXSc4w5DCIh/w2Mt2JhTCjk6dOBUjBSqxRbwkRgs7eX9uyJ585VT kCxgGkzP5fd9GbuVLLOQRp/KspKk3h/jLjgH/vJqxLNX7QHQ7XExVkH+l89QA2JV+T NciY4BSxgIdyNdU6tgei0evL9Tdj1GJuDbJz7MKsfECKSdiBWumDsbS8Bj3E/B3g5G SLY3frapSLvzl58TxJNRqoqYJE7wBpcJRApiArL8imEAwgHm/jEq5ZbgolklXeUQfA 5cdMNsSAH/EZWDBu354sCYXPogPLp6vGpa1mGnDaXAVZgRpsE3zbNcKiLMcQjKJrPX p9tlypUSrAIag== Date: Tue, 31 Dec 2024 15:54:13 -0800 Subject: [PATCH 04/10] xfs_repair: upgrade an existing filesystem to have parent pointers From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779195.2710949.139844435210923982.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Upgrade an existing filesystem to have parent pointers. Signed-off-by: "Darrick J. Wong" --- man/man8/xfs_admin.8 | 8 ++++++++ repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 39 +++++++++++++++++++++++++++++++++++++++ repair/pptr.c | 15 ++++++++++++++- repair/xfs_repair.c | 11 +++++++++++ 6 files changed, 74 insertions(+), 1 deletion(-) diff --git a/man/man8/xfs_admin.8 b/man/man8/xfs_admin.8 index 74a400dcfeb557..a25e599e5f8e2c 100644 --- a/man/man8/xfs_admin.8 +++ b/man/man8/xfs_admin.8 @@ -184,6 +184,14 @@ .SH OPTIONS The filesystem cannot be downgraded after this feature is enabled. This upgrade can fail if any AG has less than 5% free space remaining. This feature was added to Linux 4.8. +.TP 0.4i +.B parent +Store in each child file a mirror a pointing back to the parent directory. +This enables much stronger cross-referencing and online repairs of the +directory tree. +The filesystem cannot be downgraded after this feature is enabled. +This upgrade can fail if the filesystem has less than 25% free space remaining. +This feature is not upstream yet. .RE .TP .BI \-U " uuid" diff --git a/repair/globals.c b/repair/globals.c index dd7c422bb922e4..320fcf6cfd701e 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -56,6 +56,7 @@ bool add_exchrange; /* add file content exchange support */ bool add_finobt; /* add free inode btrees */ bool add_reflink; /* add reference count btrees */ bool add_rmapbt; /* add reverse mapping btrees */ +bool add_parent; /* add parent pointers */ /* misc status variables */ diff --git a/repair/globals.h b/repair/globals.h index d8c2aae23d8f0a..77d5d110048713 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -97,6 +97,7 @@ extern bool add_exchrange; /* add file content exchange support */ extern bool add_finobt; /* add free inode btrees */ extern bool add_reflink; /* add reference count btrees */ extern bool add_rmapbt; /* add reverse mapping btrees */ +extern bool add_parent; /* add parent pointers */ /* misc status variables */ diff --git a/repair/phase2.c b/repair/phase2.c index 9dd37e7fc5c111..763cffdfe9d8d2 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -295,6 +295,28 @@ set_rmapbt( return true; } +static bool +set_parent( + struct xfs_mount *mp, + struct xfs_sb *new_sb) +{ + if (xfs_has_parent(mp)) { + printf(_("Filesystem already supports parent pointers.\n")); + exit(0); + } + + if (!xfs_has_crc(mp)) { + printf( + _("Parent pointer feature only supported on V5 filesystems.\n")); + exit(0); + } + + printf(_("Adding parent pointers to filesystem.\n")); + new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_PARENT; + new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR; + return true; +} + struct check_state { struct xfs_sb sb; uint64_t features; @@ -435,6 +457,19 @@ check_fs_free_space( libxfs_trans_cancel(tp); } + /* + * If we're adding parent pointers, we need at least 25% free since + * scanning the entire filesystem to guesstimate the overhead is + * prohibitively expensive. + */ + if (xfs_has_parent(mp) && !(old->features & XFS_FEAT_PARENT)) { + if (mp->m_sb.sb_fdblocks < mp->m_sb.sb_dblocks / 4) { + printf( + _("Filesystem does not have enough space to add parent pointers.\n")); + exit(1); + } + } + /* * Would the post-upgrade filesystem have enough free space on the data * device after making per-AG reservations? @@ -467,6 +502,8 @@ need_check_fs_free_space( return true; if (xfs_has_rmapbt(mp) && !(old->features & XFS_FEAT_RMAPBT)) return true; + if (xfs_has_parent(mp) && !(old->features & XFS_FEAT_PARENT)) + return true; return false; } @@ -550,6 +587,8 @@ upgrade_filesystem( dirty |= set_reflink(mp, &new_sb); if (add_rmapbt) dirty |= set_rmapbt(mp, &new_sb); + if (add_parent) + dirty |= set_parent(mp, &new_sb); if (!dirty) return; diff --git a/repair/pptr.c b/repair/pptr.c index ac0a9c618bc87d..a8156e55f1fdfc 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -793,7 +793,7 @@ add_missing_parent_ptr( ag_pptr->namelen, name); return; - } else { + } else if (!add_parent) { do_warn( _("adding missing ino %llu parent pointer (ino %llu gen 0x%x name '%.*s')\n"), (unsigned long long)ip->i_ino, @@ -801,6 +801,19 @@ add_missing_parent_ptr( ag_pptr->parent_gen, ag_pptr->namelen, name); + } else { + static bool warned = false; + static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; + + if (!warned) { + pthread_mutex_lock(&lock); + if (!warned) { + do_warn( + _("setting parent pointers to upgrade filesystem\n")); + warned = true; + } + pthread_mutex_unlock(&lock); + } } error = add_file_pptr(ip, ag_pptr, name); diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index ca72c65f9d772a..189665a07d6892 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -74,6 +74,7 @@ enum c_opt_nums { CONVERT_FINOBT, CONVERT_REFLINK, CONVERT_RMAPBT, + CONVERT_PARENT, C_MAX_OPTS, }; @@ -86,6 +87,7 @@ static char *c_opts[] = { [CONVERT_FINOBT] = "finobt", [CONVERT_REFLINK] = "reflink", [CONVERT_RMAPBT] = "rmapbt", + [CONVERT_PARENT] = "parent", [C_MAX_OPTS] = NULL, }; @@ -405,6 +407,15 @@ process_args(int argc, char **argv) _("-c rmapbt only supports upgrades\n")); add_rmapbt = true; break; + case CONVERT_PARENT: + if (!val) + do_abort( + _("-c parent requires a parameter\n")); + if (strtol(val, NULL, 0) != 1) + do_abort( + _("-c parent only supports upgrades\n")); + add_parent = true; + break; default: unknown('c', val); break; From patchwork Tue Dec 31 23:54:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924095 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D645329415 for ; Tue, 31 Dec 2024 23:54:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689269; cv=none; b=hQW6b7WZvXFRBPZ+oGymy5XogQHUYjdWLZTGof5v0NSg6fZJKS2WUevZ2YFAZ640A0t1O48alAOVPKD1W3YgqAIM6wcC/TCCdS5tyWfNK6K1aaSN18oYXo2FgeIatfGa2m/A/r+JS+U0UppsUJkAlFLvgVOfxniRcHPI2P+se6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689269; c=relaxed/simple; bh=Z2k1uFxxqCk0yRK44uvnyAC0PsAJ+INECkDn8iiYlKI=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sipgAFtZHIrDO2eMYI4GEuw5pbZdI95DpxozRYNVanPUoSyjwaR3S9bSRSW3woakG1TJGRuabSsOdy16qmoGAHNYXnY3Y18Kp4L9SSyxqpfPKQxk+IKJOd9hFMAd+KOsEn2V+HRj7mj3Qvoa45ydx2QxTEljSmbwCAmMivaMdL8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MVk5HNvH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MVk5HNvH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AD06BC4CED2; Tue, 31 Dec 2024 23:54:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689269; bh=Z2k1uFxxqCk0yRK44uvnyAC0PsAJ+INECkDn8iiYlKI=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=MVk5HNvH7f2Shi44XN2AcFJ2X+GEFcA1Lh5ModnE445RbRyddJwoOxm+WDIOqwtVS Xmg0NJ/ypTB76pQTWp0no9/NBGXlkdpAlL8rfSTaQIDOOCj1tewEMCuM6Shnk/g8tD 8kGFpKF1TWco+oyXxztcGT/GxSsCxIJzTWeWF6pE6ephNJ8fuXSer8fTrjcRuAYXqr ceE5o++rf7rNjAIEFGJ5v/n1TEBLrRor6Ciiv+MXKhFO7swmqOubcn78uFfNcg+Pui tzNy+qW9UJLzAJy/sGf+xrARDfItbV0fp5NTJTHh2mkz18SRc6KOnl9L4wFuJnsci5 MLQ7wdKU6v3Lg== Date: Tue, 31 Dec 2024 15:54:29 -0800 Subject: [PATCH 05/10] xfs_repair: allow sysadmins to add metadata directories From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779210.2710949.14166489432103862989.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow the sysadmin to use xfs_repair to upgrade an existing filesystem to support metadata directories. This will be needed to upgrade filesystems to support realtime rmap and reflink. Signed-off-by: "Darrick J. Wong" --- man/man8/xfs_admin.8 | 8 ++++++ repair/dino_chunks.c | 6 ++++ repair/dinode.c | 5 +++- repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++ repair/phase4.c | 5 +++- repair/protos.h | 6 ++++ repair/xfs_repair.c | 11 ++++++++ 9 files changed, 109 insertions(+), 3 deletions(-) diff --git a/man/man8/xfs_admin.8 b/man/man8/xfs_admin.8 index a25e599e5f8e2c..e55dee6070e460 100644 --- a/man/man8/xfs_admin.8 +++ b/man/man8/xfs_admin.8 @@ -191,6 +191,14 @@ .SH OPTIONS directory tree. The filesystem cannot be downgraded after this feature is enabled. This upgrade can fail if the filesystem has less than 25% free space remaining. +.TP 0.4i +.B metadir +Create a directory tree of metadata inodes instead of storing them all in the +superblock. +This is required for reverse mapping btrees and reflink support on the realtime +device. +The filesystem cannot be downgraded after this feature is enabled. +This upgrade can fail if any AG has less than 5% free space remaining. This feature is not upstream yet. .RE .TP diff --git a/repair/dino_chunks.c b/repair/dino_chunks.c index 250985ec264ead..120c490b1d8324 100644 --- a/repair/dino_chunks.c +++ b/repair/dino_chunks.c @@ -955,7 +955,11 @@ process_inode_chunk( } if (status) { - if (mp->m_sb.sb_rootino == ino) { + if (wipe_pre_metadir_file(ino)) { + if (!ino_discovery) + do_warn( + _("wiping pre-metadir metadata inode %"PRIu64".\n"), ino); + } else if (mp->m_sb.sb_rootino == ino) { need_root_inode = 1; if (!no_modify) { diff --git a/repair/dinode.c b/repair/dinode.c index 0c559c40808588..42c7e9fa5cc5e7 100644 --- a/repair/dinode.c +++ b/repair/dinode.c @@ -3068,6 +3068,9 @@ process_dinode_int( ASSERT(uncertain == 0 || verify_mode != 0); ASSERT(ino_bpp != NULL || verify_mode != 0); + if (wipe_pre_metadir_file(lino)) + goto clear_bad_out; + /* * This is the only valid point to check the CRC; after this we may have * made changes which invalidate it, and the CRC is only updated again @@ -3278,7 +3281,7 @@ _("bad (negative) size %" PRId64 " on inode %" PRIu64 "\n"), if (flags & XFS_DIFLAG_NEWRTBM) { /* must be a rt bitmap inode */ if (lino != mp->m_sb.sb_rbmino) { - if (!uncertain) { + if (!uncertain && !add_metadir) { do_warn( _("inode %" PRIu64 " not rt bitmap\n"), lino); diff --git a/repair/globals.c b/repair/globals.c index 320fcf6cfd701e..603fea73da1654 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -57,6 +57,7 @@ bool add_finobt; /* add free inode btrees */ bool add_reflink; /* add reference count btrees */ bool add_rmapbt; /* add reverse mapping btrees */ bool add_parent; /* add parent pointers */ +bool add_metadir; /* add metadata directory tree */ /* misc status variables */ diff --git a/repair/globals.h b/repair/globals.h index 77d5d110048713..9211e5e2432c9a 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -98,6 +98,7 @@ extern bool add_finobt; /* add free inode btrees */ extern bool add_reflink; /* add reference count btrees */ extern bool add_rmapbt; /* add reverse mapping btrees */ extern bool add_parent; /* add parent pointers */ +extern bool add_metadir; /* add metadata directory tree */ /* misc status variables */ diff --git a/repair/phase2.c b/repair/phase2.c index 763cffdfe9d8d2..35f4c19de0555c 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -317,6 +317,71 @@ set_parent( return true; } +static xfs_ino_t doomed_rbmino = NULLFSINO; +static xfs_ino_t doomed_rsumino = NULLFSINO; +static xfs_ino_t doomed_uquotino = NULLFSINO; +static xfs_ino_t doomed_gquotino = NULLFSINO; +static xfs_ino_t doomed_pquotino = NULLFSINO; + +bool +wipe_pre_metadir_file( + xfs_ino_t ino) +{ + if (ino == doomed_rbmino || + ino == doomed_rsumino || + ino == doomed_uquotino || + ino == doomed_gquotino || + ino == doomed_pquotino) + return true; + return false; +} + +static bool +set_metadir( + struct xfs_mount *mp, + struct xfs_sb *new_sb) +{ + if (xfs_has_metadir(mp)) { + printf(_("Filesystem already supports metadata directory trees.\n")); + exit(0); + } + + if (!xfs_has_crc(mp)) { + printf( + _("Metadata directory trees only supported on V5 filesystems.\n")); + exit(0); + } + + printf(_("Adding metadata directory trees to filesystem.\n")); + new_sb->sb_features_incompat |= (XFS_SB_FEAT_INCOMPAT_METADIR | + XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR); + + /* Blow out all the old metadata inodes; we'll rebuild in phase6. */ + new_sb->sb_metadirino = new_sb->sb_rootino + 1; + doomed_rbmino = mp->m_sb.sb_rbmino; + doomed_rsumino = mp->m_sb.sb_rsumino; + doomed_uquotino = mp->m_sb.sb_uquotino; + doomed_gquotino = mp->m_sb.sb_gquotino; + doomed_pquotino = mp->m_sb.sb_pquotino; + + new_sb->sb_rbmino = new_sb->sb_metadirino + 1; + new_sb->sb_rsumino = new_sb->sb_rbmino + 1; + new_sb->sb_uquotino = NULLFSINO; + new_sb->sb_gquotino = NULLFSINO; + new_sb->sb_pquotino = NULLFSINO; + + /* Indicate that we need a rebuild. */ + need_metadir_inode = 1; + need_rbmino = 1; + need_rsumino = 1; + have_uquotino = 0; + have_gquotino = 0; + have_pquotino = 0; + quotacheck_skip(); + + return true; +} + struct check_state { struct xfs_sb sb; uint64_t features; @@ -504,6 +569,8 @@ need_check_fs_free_space( return true; if (xfs_has_parent(mp) && !(old->features & XFS_FEAT_PARENT)) return true; + if (xfs_has_metadir(mp) && !(old->features & XFS_FEAT_METADIR)) + return true; return false; } @@ -589,6 +656,8 @@ upgrade_filesystem( dirty |= set_rmapbt(mp, &new_sb); if (add_parent) dirty |= set_parent(mp, &new_sb); + if (add_metadir) + dirty |= set_metadir(mp, &new_sb); if (!dirty) return; diff --git a/repair/phase4.c b/repair/phase4.c index b752b4c871ea83..6d3c7857c6c343 100644 --- a/repair/phase4.c +++ b/repair/phase4.c @@ -431,7 +431,10 @@ phase4(xfs_mount_t *mp) if (xfs_has_metadir(mp) && (is_inode_free(irec, 1) || !inode_isadir(irec, 1))) { need_metadir_inode = true; - if (no_modify) + if (add_metadir) + do_warn( + _("metadata directory root inode needs to be initialized\n")); + else if (no_modify) do_warn( _("metadata directory root inode would be lost\n")); else diff --git a/repair/protos.h b/repair/protos.h index e2f39f1d6e8aa3..ce171f3dd87cb6 100644 --- a/repair/protos.h +++ b/repair/protos.h @@ -3,6 +3,8 @@ * Copyright (c) 2000-2001,2005 Silicon Graphics, Inc. * All Rights Reserved. */ +#ifndef __XFS_REPAIR_PROTOS_H__ +#define __XFS_REPAIR_PROTOS_H__ void xfs_init(struct libxfs_init *args); @@ -45,3 +47,7 @@ void phase7(struct xfs_mount *, int); int verify_set_agheader(struct xfs_mount *, struct xfs_buf *, struct xfs_sb *, struct xfs_agf *, struct xfs_agi *, xfs_agnumber_t); + +bool wipe_pre_metadir_file(xfs_ino_t ino); + +#endif /* __XFS_REPAIR_PROTOS_H__ */ diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index 189665a07d6892..d4101f7d2297d7 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -75,6 +75,7 @@ enum c_opt_nums { CONVERT_REFLINK, CONVERT_RMAPBT, CONVERT_PARENT, + CONVERT_METADIR, C_MAX_OPTS, }; @@ -88,6 +89,7 @@ static char *c_opts[] = { [CONVERT_REFLINK] = "reflink", [CONVERT_RMAPBT] = "rmapbt", [CONVERT_PARENT] = "parent", + [CONVERT_METADIR] = "metadir", [C_MAX_OPTS] = NULL, }; @@ -416,6 +418,15 @@ process_args(int argc, char **argv) _("-c parent only supports upgrades\n")); add_parent = true; break; + case CONVERT_METADIR: + if (!val) + do_abort( + _("-c metadir requires a parameter\n")); + if (strtol(val, NULL, 0) != 1) + do_abort( + _("-c metadir only supports upgrades\n")); + add_metadir = true; + break; default: unknown('c', val); break; From patchwork Tue Dec 31 23:54:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924096 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F29729415 for ; Tue, 31 Dec 2024 23:54:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689285; cv=none; b=eMB0Qh37CsB4iWeq0lgyPv1eyV/d9kJFFA1LHQr3K+9cNJ91pAVOpmuHSceYbB9Zj31q3FkCL/gRgGWxtIGRJHUfpEazFgDIplaTmo5lXHQzPmcItnt7NgxZxH4oTRaLw28SneNzi9frEOQViquipLHATX6Zl6BIH3IGpV38ixc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689285; c=relaxed/simple; bh=zagRP1YqMtS5/TO+furciXYwNbtsyuUT/1tAN175e7c=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Av4NBL4WwdrYiI69m/cQUeKWIx/R5D/9U4sFQON61FPrAKNYqZiPHfuz5EXQfxe8PFqjKDqeJ8z62wltY6LEvCpEi/TVB6+XZwPvU6bLwKNSZ/GC67W+1BAh6NdIyKMt7rys1WsJ63YE102ymOV75OYIAQfP3OFCen8/CFFHmKI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ApXEJ7bH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ApXEJ7bH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E896C4CED2; Tue, 31 Dec 2024 23:54:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689285; bh=zagRP1YqMtS5/TO+furciXYwNbtsyuUT/1tAN175e7c=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ApXEJ7bHrUWH0c7geEsTEjO4H0oHxdc6xKBxf0yVIc0oSefovRXlWqi2Fmneo1nHR 9Jd9D6WalOAwAZKQ486UkDA4XqadO/8MNmp4QJqSCFalWicZkqMnMf3NcABqQ69o8k 54OdXiuC4VBLsRzPhCaSv4ijwDPcYfRojTcNGq8TS33wSqiyY/MoeYy70vVpE+RseB Ff+CyKwd37CDHb+tVUUQ5dZEAnG/epE4UYFkCayyTxyYbdSZ4AygxT3oBfdLcoUqEu uwE8VPAnJo39yT94tjYs/NhTc4WnWfFGcpKxT7wGj5GdoshSJ6nRz7E6jKBot8Vwjs /pq/TT8M4rHTQ== Date: Tue, 31 Dec 2024 15:54:44 -0800 Subject: [PATCH 06/10] xfs_repair: upgrade filesystems to support rtgroups when adding metadir From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779226.2710949.17487947341302189597.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Amend the metadir upgrade code to initialize the rtgroups related fields in the superblock. This obviously means that we can't upgrade metadir to a filesystem with an existing rt section. Signed-off-by: "Darrick J. Wong" --- repair/phase2.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/repair/phase2.c b/repair/phase2.c index 35f4c19de0555c..fa6ea91711557c 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -341,6 +341,9 @@ set_metadir( struct xfs_mount *mp, struct xfs_sb *new_sb) { + struct xfs_rtgroup *rtg; + unsigned int rgsize; + if (xfs_has_metadir(mp)) { printf(_("Filesystem already supports metadata directory trees.\n")); exit(0); @@ -352,6 +355,15 @@ set_metadir( exit(0); } + if (xfs_has_realtime(mp)) { + printf( + _("Realtime groups cannot be added to an existing realtime section.\n")); + exit(0); + } + + if (!xfs_has_exchange_range(mp)) + set_exchrange(mp, new_sb); + printf(_("Adding metadata directory trees to filesystem.\n")); new_sb->sb_features_incompat |= (XFS_SB_FEAT_INCOMPAT_METADIR | XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR); @@ -364,21 +376,35 @@ set_metadir( doomed_gquotino = mp->m_sb.sb_gquotino; doomed_pquotino = mp->m_sb.sb_pquotino; - new_sb->sb_rbmino = new_sb->sb_metadirino + 1; - new_sb->sb_rsumino = new_sb->sb_rbmino + 1; + new_sb->sb_rbmino = NULLFSINO; + new_sb->sb_rsumino = NULLFSINO; new_sb->sb_uquotino = NULLFSINO; new_sb->sb_gquotino = NULLFSINO; new_sb->sb_pquotino = NULLFSINO; + rgsize = XFS_B_TO_FSBT(mp, 1ULL << 40); /* 1TB */ + rgsize -= rgsize % new_sb->sb_rextsize; + new_sb->sb_rgextents = rgsize; + new_sb->sb_rgcount = 0; + new_sb->sb_rgblklog = libxfs_compute_rgblklog(new_sb->sb_rgextents, + new_sb->sb_rextsize); /* Indicate that we need a rebuild. */ need_metadir_inode = 1; need_rbmino = 1; need_rsumino = 1; - have_uquotino = 0; - have_gquotino = 0; - have_pquotino = 0; + clear_quota_inode(XFS_DQTYPE_USER); + clear_quota_inode(XFS_DQTYPE_GROUP); + clear_quota_inode(XFS_DQTYPE_PROJ); quotacheck_skip(); + /* Dump incore rt freespace inodes. */ + rtg = libxfs_rtgroup_grab(mp, 0); + if (rtg) { + libxfs_rtginode_irele(&rtg->rtg_inodes[XFS_RTGI_BITMAP]); + libxfs_rtginode_irele(&rtg->rtg_inodes[XFS_RTGI_SUMMARY]); + libxfs_rtgroup_rele(rtg); + } + return true; } From patchwork Tue Dec 31 23:55:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924097 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17A0729415 for ; Tue, 31 Dec 2024 23:55:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689301; cv=none; b=DNHDH7DqKvE+wMCIT+c8joLQs9mSy/x/uXfvq8s9/6XdSBjzkcrsDszV88z1YOTaSdHM3Wk7lXm4exMYP+kll3lLe2PCMnTjTY+xosv7RzhOYfRPN4yLB5zOOVSiAs9TiM6E8N13gXQzOB4MktpRu3sRWo4lEl0EpUKIwmgeyf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689301; c=relaxed/simple; bh=zP900MfPC9RDVyofFdZKwn5kMJFhtC3z8GcG9ZVg0X8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ZSoa6sRfwaiRJhlfebxl/QvIi10li2vuRP6R36BCDXbtY76nbFzOW5VTet9Sj/lKoElgliQFak4F5nMS1xbxnm2G1J9dYv8RLAySKEkqXTXxfDbhdulJVR7exPGvukRbMKfHqAyredv2nzw2niGOF6HjnAm0K52/Wfl2KJWkR1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ccfs8qfJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ccfs8qfJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7A07C4CED2; Tue, 31 Dec 2024 23:55:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689301; bh=zP900MfPC9RDVyofFdZKwn5kMJFhtC3z8GcG9ZVg0X8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ccfs8qfJeZSdYtCVE/TOZuCFsu98Y+xjI4pF3WTNA4QGnalaSCwK7AeiGZ89b3/ip RD5V+MLVhKTarQ0fRzR/mGIzgaXzR2xZWjZL1nqV7++rWCo5+RjR4MNOZVXqOX2jpI 232zD/uT8cjDh7xVoHSZ1tjbrwKu0w4/o23fQuVNIjHA5PnpVnpcskiyigT/x0FwbX 9R0Pfw6/6Y5w85FVMH5DB/GXLtdFhb8FtVHZMaMiKGaAXlxqWbNlyKS9UNLn5tSPjF qc8Vqc4HSda4ErUWymdpBHseQrJ2FPOtp6ELTZjyua3R0vIXxeHlLZHIJ8zEcQNR5N cR9Qz/CYLUQMg== Date: Tue, 31 Dec 2024 15:55:00 -0800 Subject: [PATCH 07/10] xfs_repair: allow sysadmins to add realtime reverse mapping indexes From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779241.2710949.14047199401963688785.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow the sysadmin to use xfs_repair to upgrade an existing filesystem to support the reverse mapping btree index for realtime volumes. This is needed for online fsck. Signed-off-by: "Darrick J. Wong" --- libxfs/libxfs_api_defs.h | 1 + repair/phase2.c | 64 ++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 60 insertions(+), 5 deletions(-) diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h index 76f55515bb41f7..2502a7736d1670 100644 --- a/libxfs/libxfs_api_defs.h +++ b/libxfs/libxfs_api_defs.h @@ -78,6 +78,7 @@ #define xfs_btree_bload libxfs_btree_bload #define xfs_btree_bload_compute_geometry libxfs_btree_bload_compute_geometry #define xfs_btree_calc_size libxfs_btree_calc_size +#define xfs_btree_compute_maxlevels libxfs_btree_compute_maxlevels #define xfs_btree_decrement libxfs_btree_decrement #define xfs_btree_del_cursor libxfs_btree_del_cursor #define xfs_btree_delete libxfs_btree_delete diff --git a/repair/phase2.c b/repair/phase2.c index fa6ea91711557c..b1288bf3dd90cd 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -277,9 +277,8 @@ set_rmapbt( exit(0); } - if (xfs_has_realtime(mp)) { - printf( - _("Reverse mapping btree feature not supported with realtime.\n")); + if (xfs_has_realtime(mp) && !xfs_has_rtgroups(mp)) { + printf(_("Reverse mapping btree requires realtime groups.\n")); exit(0); } @@ -292,6 +291,7 @@ set_rmapbt( printf(_("Adding reverse mapping btrees to filesystem.\n")); new_sb->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_RMAPBT; new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR; + return true; } @@ -466,6 +466,37 @@ check_free_space( return avail > GIGABYTES(10, mp->m_sb.sb_blocklog); } +/* + * Reserve space to handle rt rmap btree expansion. + * + * If the rmap inode for this group already exists, we assume that we're adding + * some other feature. Note that we have not validated the metadata directory + * tree, so we must perform the lookup by hand and abort the upgrade if there + * are errors. Otherwise, the amount of space needed to handle a new maximally + * sized rmap btree is added to @new_resv. + */ +static int +reserve_rtrmap_inode( + struct xfs_rtgroup *rtg, + xfs_rfsblock_t *new_resv) +{ + struct xfs_mount *mp = rtg_mount(rtg); + struct xfs_inode *ip = rtg_rmap(rtg); + xfs_filblks_t ask; + + if (!xfs_has_rtrmapbt(mp)) + return 0; + + ask = libxfs_rtrmapbt_calc_reserves(mp); + + /* failed to load the rtdir inode? */ + if (!ip) { + *new_resv += ask; + return 0; + } + return -libxfs_metafile_resv_init(ip, ask); +} + static void check_fs_free_space( struct xfs_mount *mp, @@ -473,6 +504,8 @@ check_fs_free_space( struct xfs_sb *new_sb) { struct xfs_perag *pag = NULL; + struct xfs_rtgroup *rtg = NULL; + xfs_rfsblock_t new_resv = 0; int error; /* Make sure we have enough space for per-AG reservations. */ @@ -548,6 +581,21 @@ check_fs_free_space( libxfs_trans_cancel(tp); } + /* Realtime metadata btree inodes */ + while ((rtg = xfs_rtgroup_next(mp, rtg))) { + error = reserve_rtrmap_inode(rtg, &new_resv); + if (error == ENOSPC) { + printf( +_("Not enough free space would remain for rtgroup %u rmap inode.\n"), + rtg_rgno(rtg)); + exit(0); + } + if (error) + do_error( +_("Error %d while checking rtgroup %u rmap inode space reservation.\n"), + error, rtg_rgno(rtg)); + } + /* * If we're adding parent pointers, we need at least 25% free since * scanning the entire filesystem to guesstimate the overhead is @@ -563,13 +611,19 @@ check_fs_free_space( /* * Would the post-upgrade filesystem have enough free space on the data - * device after making per-AG reservations? + * device after making per-AG reservations and reserving rt metadata + * inode blocks? */ - if (!check_free_space(mp, mp->m_sb.sb_fdblocks, mp->m_sb.sb_dblocks)) { + if (new_resv > mp->m_sb.sb_fdblocks || + !check_free_space(mp, mp->m_sb.sb_fdblocks, mp->m_sb.sb_dblocks)) { printf(_("Filesystem will be low on space after upgrade.\n")); exit(1); } + /* Unreserve the realtime metadata reservations. */ + while ((rtg = xfs_rtgroup_next(mp, rtg))) + libxfs_metafile_resv_free(rtg_rmap(rtg)); + /* * Release the per-AG reservations and mark the per-AG structure as * uninitialized so that we don't trip over stale cached counters From patchwork Tue Dec 31 23:55:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924098 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A49BD29415 for ; Tue, 31 Dec 2024 23:55:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689316; cv=none; b=KMRDzvyP3C+hDloLtIPjC4gXvVEKII8EqVPRi2gIGmjTAu/hW9fbyBUkFfiGZSkkMxgyq+lxJsUS4JFA8fPvMR0GspDd4/zs9Gmmho2KkK+zU1oImyyYFIRlD0aUjuH8NPLroNE25O9f8daAj1SMZ2q1OI8sWmJCx/X6WUApjtY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689316; c=relaxed/simple; bh=FRG0NimpfRHKaQrMjA/o5w9Rxapr1kOU8LtkPwiUoxE=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GvevE66SXY4jbGeP4LipIEjOgR6S3P31Pjm2mrK4w4VQsXXn2Qq31yQtBz+lAC9m8gtP4eNVDlXUwesVcUEzlhrfdhLhNZkeWovB3RpnVve3KMeK5QMKiZAH6ZIQQJOewQsV4zirSSSy1zbI/Mg3LLvkKxhf1xZ/lN5DKJKyuHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PC2uBfS0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PC2uBfS0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81974C4CED2; Tue, 31 Dec 2024 23:55:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689316; bh=FRG0NimpfRHKaQrMjA/o5w9Rxapr1kOU8LtkPwiUoxE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=PC2uBfS0MOIjJOBDbAEPUhZsTfy9jSBhoayitAaMUDvuVDTJK0zWZNvbsH76wJXPk gRRty3LmaAGvlVM9yHTL3x3HFK6m13QvBIsIBx+ZkUnhYigDn0TgjuK+347iJdfHE7 0krvsshcPoNeYGmiyRN19Cc4pNKBC8wx+xUWk4cHfa2Au2bwJTPxDS2BKDWMTsv+n/ wjaQx+0VjKrw8IZP0jvnulQyiuDcg3h/uNcE+Sdr77b3QFi8uVp/sEixTmwN2VluxW JNGXStaT18n/k/3z8iERFQ2eRTFz5jW8ExtdS+hGTxidx4/XQxfCUBU8OFYxA1/nWy 2RYpJQppCzDmg== Date: Tue, 31 Dec 2024 15:55:16 -0800 Subject: [PATCH 08/10] xfs_repair: allow sysadmins to add realtime reflink From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779257.2710949.13339646892376161679.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow the sysadmin to use xfs_repair to upgrade an existing filesystem to support the realtime reference count btree, and therefore reflink on realtime volumes. Signed-off-by: "Darrick J. Wong" --- repair/phase2.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 50 insertions(+), 3 deletions(-) diff --git a/repair/phase2.c b/repair/phase2.c index b1288bf3dd90cd..8dc936b572196e 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -250,14 +250,15 @@ set_reflink( exit(0); } - if (xfs_has_realtime(mp)) { - printf(_("Reflink feature not supported with realtime.\n")); + if (xfs_has_realtime(mp) && !xfs_has_rtgroups(mp)) { + printf(_("Reference count btree requires realtime groups.\n")); exit(0); } printf(_("Adding reflink support to filesystem.\n")); new_sb->sb_features_ro_compat |= XFS_SB_FEAT_RO_COMPAT_REFLINK; new_sb->sb_features_incompat |= XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR; + return true; } @@ -497,6 +498,38 @@ reserve_rtrmap_inode( return -libxfs_metafile_resv_init(ip, ask); } +/* + * Reserve space to handle rt refcount btree expansion. + * + * If the refcount inode for this group already exists, we assume that we're + * adding some other feature. Note that we have not validated the metadata + * directory tree, so we must perform the lookup by hand and abort the upgrade + * if there are errors. If the inode does not exist, the amount of space + * needed to handle a new maximally sized refcount btree is added to @new_resv. + */ +static int +reserve_rtrefcount_inode( + struct xfs_rtgroup *rtg, + xfs_rfsblock_t *new_resv) +{ + struct xfs_mount *mp = rtg_mount(rtg); + struct xfs_inode *ip = rtg_refcount(rtg); + xfs_filblks_t ask; + + if (!xfs_has_rtreflink(mp)) + return 0; + + ask = libxfs_rtrefcountbt_calc_reserves(mp); + + /* failed to load the rtdir inode? */ + if (!ip) { + *new_resv += ask; + return 0; + } + + return -libxfs_metafile_resv_init(ip, ask); +} + static void check_fs_free_space( struct xfs_mount *mp, @@ -594,6 +627,18 @@ _("Not enough free space would remain for rtgroup %u rmap inode.\n"), do_error( _("Error %d while checking rtgroup %u rmap inode space reservation.\n"), error, rtg_rgno(rtg)); + + error = reserve_rtrefcount_inode(rtg, &new_resv); + if (error == ENOSPC) { + printf( +_("Not enough free space would remain for rtgroup %u refcount inode.\n"), + rtg_rgno(rtg)); + exit(0); + } + if (error) + do_error( +_("Error %d while checking rtgroup %u refcount inode space reservation.\n"), + error, rtg_rgno(rtg)); } /* @@ -621,8 +666,10 @@ _("Error %d while checking rtgroup %u rmap inode space reservation.\n"), } /* Unreserve the realtime metadata reservations. */ - while ((rtg = xfs_rtgroup_next(mp, rtg))) + while ((rtg = xfs_rtgroup_next(mp, rtg))) { libxfs_metafile_resv_free(rtg_rmap(rtg)); + libxfs_metafile_resv_free(rtg_refcount(rtg)); + } /* * Release the per-AG reservations and mark the per-AG structure as From patchwork Tue Dec 31 23:55:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924099 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FAC71B0421 for ; Tue, 31 Dec 2024 23:55:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689332; cv=none; b=DzKwsRvs1HMnOQkXv/7pgMfwVZHRPyn6Fdrumol3c81h88z3/CMXAG4z/Kej4uqUoQn0VpxI4N15zklnMcS3JXLs4oxxn5wOBldRpy902o5x4voZwkYv2kgpQOwITwCsuFofmco+D2u8F2ak8WxdDrabwjJSykr87/DZZewhhsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689332; c=relaxed/simple; bh=yzfBUlqjSaOPjaCG0NSDV5KSqH6DRLWFFY3aTTMXITU=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hm9Li93YrblDvYSInBxaI48TybzOeOtTiDcF3GRLGKYldhx5u9TT8aiITzVvfnY9tJBMn5O28g4K3nFIIG1TKi8JUnwXSgf+WumPr9ERljHNBwrdEjvB6iFP3fammggEdWyTHZMhMoqCf6kQm3FjNLTTrYxva59mGBswwy6fT4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pXcO71bh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pXcO71bh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2B3B9C4CED2; Tue, 31 Dec 2024 23:55:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689332; bh=yzfBUlqjSaOPjaCG0NSDV5KSqH6DRLWFFY3aTTMXITU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=pXcO71bhZgvQ27CDCqp+WuB5hsbp5zYREE2To5zPKp5OR4+h1HNl45YjE/mAjiv5l ngo4EtLPTYvleSlfQU37yPva/BOIhGK7JwtdDA4fM4A7iQc0YreCHGqiXr871MfpAo sQNUUHl30tFXNmOq1Q2KxmUTX16VY9zkDyWWd98xO5dgy3pwUHwy20oks/lJaHXMst S7c0tA9ryoN4CuGhTiY/xScrP+hnyQotPIKfzZbfxc2CV9ZavJ14grO2Xo0nE2pLNZ u3hbd49yVWYoGqRbkhI7XdX19wjlkrwKqVfsk4thiHYH2+cS8xwSHozf/uhwLU6H25 SfBVT6Jqjrz5Q== Date: Tue, 31 Dec 2024 15:55:31 -0800 Subject: [PATCH 09/10] xfs_repair: skip free space checks when upgrading From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779272.2710949.2287968490226708363.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Add a debug knob to disable the free space checks when upgrading a system. This is extremely risky and will cause severe tire damage!!! Signed-off-by: "Darrick J. Wong" --- repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 2 ++ repair/xfs_repair.c | 11 +++++++++++ 4 files changed, 15 insertions(+) diff --git a/repair/globals.c b/repair/globals.c index 603fea73da1654..fe9f9ac5914bb0 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -48,6 +48,7 @@ char *rt_name; /* Name of realtime device */ int rt_spec; /* Realtime dev specified as option */ int convert_lazy_count; /* Convert lazy-count mode on/off */ int lazy_count; /* What to set if to if converting */ +bool skip_freesp_check_on_upgrade; /* do not enable */ bool features_changed; /* did we change superblock feature bits? */ bool add_inobtcount; /* add inode btree counts to AGI */ bool add_bigtime; /* add support for timestamps up to 2486 */ diff --git a/repair/globals.h b/repair/globals.h index 9211e5e2432c9a..c660971080f7e4 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -89,6 +89,7 @@ extern char *rt_name; /* Name of realtime device */ extern int rt_spec; /* Realtime dev specified as option */ extern int convert_lazy_count; /* Convert lazy-count mode on/off */ extern int lazy_count; /* What to set if to if converting */ +extern bool skip_freesp_check_on_upgrade; /* do not enable */ extern bool features_changed; /* did we change superblock feature bits? */ extern bool add_inobtcount; /* add inode btree counts to AGI */ extern bool add_bigtime; /* add support for timestamps up to 2486 */ diff --git a/repair/phase2.c b/repair/phase2.c index 8dc936b572196e..780294d24c9900 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -688,6 +688,8 @@ need_check_fs_free_space( struct xfs_mount *mp, const struct check_state *old) { + if (skip_freesp_check_on_upgrade) + return false; if (xfs_has_finobt(mp) && !(old->features & XFS_FEAT_FINOBT)) return true; if (xfs_has_reflink(mp) && !(old->features & XFS_FEAT_REFLINK)) diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index d4101f7d2297d7..55e417201b34f7 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -46,6 +46,7 @@ enum o_opt_nums { BLOAD_LEAF_SLACK, BLOAD_NODE_SLACK, NOQUOTA, + SKIP_FREESP_CHECK, O_MAX_OPTS, }; @@ -59,6 +60,7 @@ static char *o_opts[] = { [BLOAD_LEAF_SLACK] = "debug_bload_leaf_slack", [BLOAD_NODE_SLACK] = "debug_bload_node_slack", [NOQUOTA] = "noquota", + [SKIP_FREESP_CHECK] = "debug_skip_freesp_check_on_upgrade", [O_MAX_OPTS] = NULL, }; @@ -323,6 +325,15 @@ process_args(int argc, char **argv) case NOQUOTA: quotacheck_skip(); break; + case SKIP_FREESP_CHECK: + if (!val) + do_abort( + _("-o debug_skip_freesp_check_on_upgrade requires a parameter\n")); + skip_freesp_check_on_upgrade = (int)strtol(val, NULL, 0); + if (skip_freesp_check_on_upgrade) + do_log( + _("WARNING: Allowing filesystem upgrades to proceed without free space check. THIS MAY DESTROY YOUR FILESYSTEM!!!\n")); + break; default: unknown('o', val); break; From patchwork Tue Dec 31 23:55:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13924100 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2A7D29415 for ; Tue, 31 Dec 2024 23:55:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689348; cv=none; b=FgqLcZhLbkoob3tCc+dK+jQf3eeBjbonxnWMOEQVqaTDkwUuRU+8YTZ1Ta3IePrwU2KBF72JKHutbSViDHlV8MdM/VwclGUdmhA/VsUt70SpNq4IxOODXAJmIJT4ty2gIl/7vfLZl9uf1xJJ4Xal3OZcfkZPX20EAZtRTERewGg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735689348; c=relaxed/simple; bh=iW0VSLmxwouGLalyJTPjrXf3Oly3hMgfOoGGOwu7Ac4=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sE8/HRJ+EGmJMpaU2J0BQzx7g3+JM79USfEwFGuZ4hDLv3EvfkRXrqua6DWbqSRDqjsO625/WUVvjQYkJ4qNFglwfeBp8q5vATSt9wQPSOLI0B4xSVavCsQlq8l9PARTpstGBVTl+x43kVbLGo2QHj7M0P7kSKQIcmxVtOPT7uk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=N77OdRyT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="N77OdRyT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA56BC4CED2; Tue, 31 Dec 2024 23:55:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735689347; bh=iW0VSLmxwouGLalyJTPjrXf3Oly3hMgfOoGGOwu7Ac4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=N77OdRyTv3wnn41dD56tZbC/J8MSXOrTBHYgRupnHFwTb8nTk6MgHlK7OodJpAGnJ qcvBFMtHrm5s7iodyko8OiVyWkX+7BuDpjGg3y2LAu8ATPkLrZu1wndvxiWctkR5bD 5rm7FLCDAJ2aqj9G+c7tM+QdRIAXlakaAB6bI1A9fQYyoiM42zJEuQXep5Y2BSnRSK MmE2AFv7LXrrc370AB/P7SoAnzMZJQLfcxUu5dUJ47D9C5bBJ/Wgu/S+oNKN29rDOJ EY+cf4wOKk4v/f70qMT5PZc0QN/tLPJMZi+79dWBE8tdNKUuJl6EflLtlvub9+T7s0 zM0zK1LyWw+gA== Date: Tue, 31 Dec 2024 15:55:47 -0800 Subject: [PATCH 10/10] xfs_repair: allow adding rmapbt to reflink filesystems From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <173568779287.2710949.3018758018747416020.stgit@frogsfrogsfrogs> In-Reply-To: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> References: <173568779121.2710949.16873326283859979950.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong New debugging knob so that I can upgrade a filesystem to have rmap btrees even if reflink was already enabled. We cannot easily precompute the space requirements, so this is dangerous. Signed-off-by: "Darrick J. Wong" --- repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 3 ++- repair/xfs_repair.c | 11 +++++++++++ 4 files changed, 15 insertions(+), 1 deletion(-) diff --git a/repair/globals.c b/repair/globals.c index fe9f9ac5914bb0..f4f1d317917183 100644 --- a/repair/globals.c +++ b/repair/globals.c @@ -49,6 +49,7 @@ int rt_spec; /* Realtime dev specified as option */ int convert_lazy_count; /* Convert lazy-count mode on/off */ int lazy_count; /* What to set if to if converting */ bool skip_freesp_check_on_upgrade; /* do not enable */ +bool allow_rmapbt_upgrade_with_reflink; /* add rmapbt when reflink already on */ bool features_changed; /* did we change superblock feature bits? */ bool add_inobtcount; /* add inode btree counts to AGI */ bool add_bigtime; /* add support for timestamps up to 2486 */ diff --git a/repair/globals.h b/repair/globals.h index c660971080f7e4..febbbbcc81f931 100644 --- a/repair/globals.h +++ b/repair/globals.h @@ -90,6 +90,7 @@ extern int rt_spec; /* Realtime dev specified as option */ extern int convert_lazy_count; /* Convert lazy-count mode on/off */ extern int lazy_count; /* What to set if to if converting */ extern bool skip_freesp_check_on_upgrade; /* do not enable */ +extern bool allow_rmapbt_upgrade_with_reflink; /* add rmapbt when reflink already on */ extern bool features_changed; /* did we change superblock feature bits? */ extern bool add_inobtcount; /* add inode btree counts to AGI */ extern bool add_bigtime; /* add support for timestamps up to 2486 */ diff --git a/repair/phase2.c b/repair/phase2.c index 780294d24c9900..29a406f69ca3a1 100644 --- a/repair/phase2.c +++ b/repair/phase2.c @@ -283,7 +283,8 @@ set_rmapbt( exit(0); } - if (xfs_has_reflink(mp) && !add_reflink) { + if (xfs_has_reflink(mp) && !add_reflink && + !allow_rmapbt_upgrade_with_reflink) { printf( _("Reverse mapping btrees cannot be added when reflink is enabled.\n")); exit(0); diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c index 55e417201b34f7..4cff11d81d6bcb 100644 --- a/repair/xfs_repair.c +++ b/repair/xfs_repair.c @@ -47,6 +47,7 @@ enum o_opt_nums { BLOAD_NODE_SLACK, NOQUOTA, SKIP_FREESP_CHECK, + ALLOW_RMAPBT_UPGRADE_WITH_REFLINK, O_MAX_OPTS, }; @@ -61,6 +62,7 @@ static char *o_opts[] = { [BLOAD_NODE_SLACK] = "debug_bload_node_slack", [NOQUOTA] = "noquota", [SKIP_FREESP_CHECK] = "debug_skip_freesp_check_on_upgrade", + [ALLOW_RMAPBT_UPGRADE_WITH_REFLINK] = "debug_allow_rmapbt_upgrade_with_reflink", [O_MAX_OPTS] = NULL, }; @@ -334,6 +336,15 @@ process_args(int argc, char **argv) do_log( _("WARNING: Allowing filesystem upgrades to proceed without free space check. THIS MAY DESTROY YOUR FILESYSTEM!!!\n")); break; + case ALLOW_RMAPBT_UPGRADE_WITH_REFLINK: + if (!val) + do_abort( + _("-o debug_allow_rmapbt_upgrade_with_reflink requires a parameter\n")); + allow_rmapbt_upgrade_with_reflink = (int)strtol(val, NULL, 0); + if (allow_rmapbt_upgrade_with_reflink) + do_log( + _("WARNING: Allowing filesystem upgrade to rmapbt when reflink enabled. THIS MAY DESTROY YOUR FILESYSTEM!!!\n")); + break; default: unknown('o', val); break;