From patchwork Fri Dec 30 22:19:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13085669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E3FAC4332F for ; Sat, 31 Dec 2022 02:29:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235911AbiLaC3F (ORCPT ); Fri, 30 Dec 2022 21:29:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231466AbiLaC3D (ORCPT ); Fri, 30 Dec 2022 21:29:03 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2595D1C921 for ; Fri, 30 Dec 2022 18:29:03 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C327EB81E6D for ; Sat, 31 Dec 2022 02:29:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 868A0C433D2; Sat, 31 Dec 2022 02:29:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1672453740; bh=MvTTqZihwbHiEUTypuNXwMvBlapOLO0upWwetK3wHU0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=vJ87fvXe5I9cUMinQPKPfUYh6SIdDQGR84g8TUo2sy7/Qu+BAQGcM9Bi+BTO+volD UYmm3sTAjAR5AwwLhzdL8cHb+fWI2e2VHUnwJFEZ84vCYNUm1RyhRuLbsYhscU1ww8 xbrX3Q/rg9ebg4dgN9swgW0cYwVAuyPjp0BS8iII6h82GcwC97cnN71dhZpmixuZEX gKNOuZ5M4aY4FZYF/XvRnaRiRN6HZf52eggyA9EoxGqZ33qrXKpMIQluqJoB702dGV lfwmwFLNyOn5lz+3mDzZR309kvbLI14OOvs+NEbr12IpdWoY9z+PMD4C0JQYseL/ay v3s2ICSdTpouw== Subject: [PATCH 1/5] xfs_db: allow selecting logdev blocks From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Date: Fri, 30 Dec 2022 14:19:39 -0800 Message-ID: <167243877995.730695.10725248450234022103.stgit@magnolia> In-Reply-To: <167243877981.730695.7761889719607533776.stgit@magnolia> References: <167243877981.730695.7761889719607533776.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Make it so that xfs_db can examine blocks on an external log device. Signed-off-by: Darrick J. Wong --- db/io.c | 18 ++++++++++++++++++ db/io.h | 2 ++ 2 files changed, 20 insertions(+) diff --git a/db/io.c b/db/io.c index 00eb5e98dc2..8e3b32d9551 100644 --- a/db/io.c +++ b/db/io.c @@ -660,6 +660,24 @@ set_rt_cur( return 0; } +int +set_log_cur( + const typ_t *type, + xfs_daddr_t blknum, + int len, + int ring_flag, + bbmap_t *bbmap) +{ + if (!mp->m_logdev_targp->bt_bdev || + mp->m_logdev_targp->bt_bdev == mp->m_ddev_targp->bt_bdev) { + printf(_("external log device not loaded, use -l.\n")); + return ENODEV; + } + + __set_cur(mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap); + return 0; +} + void set_iocur_type( const typ_t *type) diff --git a/db/io.h b/db/io.h index 1a37ee78c72..b3d8123d548 100644 --- a/db/io.h +++ b/db/io.h @@ -49,6 +49,8 @@ extern void push_cur_and_set_type(void); extern void write_cur(void); extern void set_cur(const struct typ *type, xfs_daddr_t blknum, int len, int ring_add, bbmap_t *bbmap); +extern int set_log_cur(const struct typ *type, xfs_daddr_t blknum, + int len, int ring_add, bbmap_t *bbmap); extern int set_rt_cur(const struct typ *type, xfs_daddr_t blknum, int len, int ring_add, bbmap_t *bbmap); extern void ring_add(void); From patchwork Fri Dec 30 22:19:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13085670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8903C4332F for ; Sat, 31 Dec 2022 02:29:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236037AbiLaC3V (ORCPT ); Fri, 30 Dec 2022 21:29:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231466AbiLaC3R (ORCPT ); Fri, 30 Dec 2022 21:29:17 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 135131C921 for ; Fri, 30 Dec 2022 18:29:17 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A647861CD4 for ; Sat, 31 Dec 2022 02:29:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1139EC433D2; Sat, 31 Dec 2022 02:29:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1672453756; bh=ZrUN8IeOBx9+YyI/ttkm6jck4D9A/2umUbgUAuB1b30=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=LdVGcB1lcfWfTLsXHGRyn6za55L0xchsVp9GMNgvApzQiWhjlOJYcZW2knqVUDKmR FbYnOjK8VMW3hWZdnJ4TmoHU+3YFn1ASV8QYUQgoJFp6bvaIf2S7x1wTfqKKz5slPD oL/HwgOX4IWeG0OGOUl2fvRHCD0NHylgElpdY8ZnC/UGOWo8UgVIB2yUrqnlzYY/ct oxpZupgoqULofUPg3t8IGfmFGTpvBmYYIOMi2dGPGzPmpG+xjQUFHJeKzQkjJvNKb9 vZfXn8keoIEU0UvAlJcFRcZWJq54Vc0UcSxAAfBeFuHXnlcT/lclOtRL73OrXABjDN qu5sxi+cFLaJQ== Subject: [PATCH 2/5] xfs_db: allow setting current address to log blocks From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Date: Fri, 30 Dec 2022 14:19:40 -0800 Message-ID: <167243878009.730695.2275996586942542508.stgit@magnolia> In-Reply-To: <167243877981.730695.7761889719607533776.stgit@magnolia> References: <167243877981.730695.7761889719607533776.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Add commands so that users can target blocks on an external log device. Signed-off-by: Darrick J. Wong --- db/block.c | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++- man/man8/xfs_db.8 | 17 +++++++++ 2 files changed, 119 insertions(+), 1 deletion(-) diff --git a/db/block.c b/db/block.c index 1afe201d0b2..98df0ce10ac 100644 --- a/db/block.c +++ b/db/block.c @@ -29,6 +29,8 @@ static int rtblock_f(int argc, char **argv); static void rtblock_help(void); static int rtextent_f(int argc, char **argv); static void rtextent_help(void); +static int logblock_f(int argc, char **argv); +static void logblock_help(void); static void print_rawdata(void *data, int len); static const cmdinfo_t ablock_cmd = @@ -49,6 +51,9 @@ static const cmdinfo_t rtblock_cmd = static const cmdinfo_t rtextent_cmd = { "rtextent", "rtx", rtextent_f, 0, 1, 1, N_("[rtxno]"), N_("set address to rtextent value"), rtextent_help }; +static const cmdinfo_t logblock_cmd = + { "logblock", "lsb", logblock_f, 0, 1, 1, N_("[logbno]"), + N_("set address to logblock value"), logblock_help }; static void ablock_help(void) @@ -116,6 +121,7 @@ block_init(void) add_command(&fsblock_cmd); add_command(&rtblock_cmd); add_command(&rtextent_cmd); + add_command(&logblock_cmd); } static void @@ -132,6 +138,7 @@ daddr_help(void) enum daddr_target { DT_DATA, DT_RT, + DT_LOG, }; static int @@ -145,18 +152,27 @@ daddr_f( xfs_rfsblock_t max_daddrs = mp->m_sb.sb_dblocks; enum daddr_target tgt = DT_DATA; - while ((c = getopt(argc, argv, "r")) != -1) { + while ((c = getopt(argc, argv, "rl")) != -1) { switch (c) { case 'r': tgt = DT_RT; max_daddrs = mp->m_sb.sb_rblocks; break; + case 'l': + tgt = DT_LOG; + max_daddrs = mp->m_sb.sb_logblocks; + break; default: daddr_help(); return 0; } } + if (tgt == DT_LOG && mp->m_sb.sb_logstart > 0) { + dbprintf(_("filesystem has internal log\n")); + return 0; + } + if (optind == argc) { xfs_daddr_t daddr = iocur_top->off >> BBSHIFT; @@ -191,6 +207,9 @@ daddr_f( case DT_RT: set_rt_cur(&typtab[TYP_DATA], d, 1, DB_RING_ADD, NULL); break; + case DT_LOG: + set_log_cur(&typtab[TYP_DATA], d, 1, DB_RING_ADD, NULL); + break; } return 0; } @@ -408,6 +427,88 @@ rtextent_f( return 0; } +static void +logblock_help(void) +{ + dbprintf(_( +"\n Example:\n" +"\n" +" 'logblock 1023' - sets the file position to the 1023rd log block.\n" +" The external log device or the block offset within the internal log will be\n" +" chosen as appropriate.\n" +)); +} + +static int +logblock_f( + int argc, + char **argv) +{ + xfs_fsblock_t logblock; + char *p; + + if (argc == 1) { + if (mp->m_sb.sb_logstart > 0 && iocur_is_ddev(iocur_top)) { + logblock = XFS_DADDR_TO_FSB(mp, + iocur_top->off >> BBSHIFT); + + if (logblock < mp->m_sb.sb_logstart || + logblock >= mp->m_sb.sb_logstart + + mp->m_sb.sb_logblocks) { + dbprintf( + _("current address not within internal log\n")); + return 0; + } + + dbprintf(_("current logblock is %lld\n"), + logblock - mp->m_sb.sb_logstart); + return 0; + } + + if (mp->m_sb.sb_logstart == 0 && + iocur_is_extlogdev(iocur_top)) { + logblock = XFS_BB_TO_FSB(mp, + iocur_top->off >> BBSHIFT); + + if (logblock >= mp->m_sb.sb_logblocks) { + dbprintf( + _("current address not within external log\n")); + return 0; + } + + dbprintf(_("current logblock is %lld\n"), logblock); + return 0; + } + + dbprintf(_("current address does not point to log\n")); + return 0; + } + + logblock = strtoull(argv[1], &p, 0); + if (*p != '\0') { + dbprintf(_("bad logblock %s\n"), argv[1]); + return 0; + } + + if (logblock >= mp->m_sb.sb_logblocks) { + dbprintf(_("bad logblock %s\n"), argv[1]); + return 0; + } + + ASSERT(typtab[TYP_DATA].typnm == TYP_DATA); + + if (mp->m_sb.sb_logstart) { + logblock += mp->m_sb.sb_logstart; + set_cur(&typtab[TYP_DATA], XFS_FSB_TO_DADDR(mp, logblock), + blkbb, DB_RING_ADD, NULL); + } else { + set_log_cur(&typtab[TYP_DATA], XFS_FSB_TO_BB(mp, logblock), + blkbb, DB_RING_ADD, NULL); + } + + return 0; +} + void print_block( const field_t *fields, diff --git a/man/man8/xfs_db.8 b/man/man8/xfs_db.8 index 1246eb6327c..9c1ce5d79cf 100644 --- a/man/man8/xfs_db.8 +++ b/man/man8/xfs_db.8 @@ -600,6 +600,9 @@ The type is set to .B data (uninterpreted). +If an address and the +.B \-l +option are specified, the current address is set to the external log device. If an address and the .B \-r option are specified, the current address is set to the realtime device. @@ -839,6 +842,20 @@ Start logging output to .IR filename , stop logging, or print the current logging status. .TP +.BI "logblock [" logbno ] +Set current address to the log block value given by +.IR logbno . +If no value for +.I logbno +is given the current address is printed, expressed as an fsb. +The type is set to +.B data +(uninterpreted). +If the filesystem has an external log, then the address will be within the log +device. +If the filesystem has an internal log, then the address will be within the +internal log. +.TP .BI "logformat [\-c " cycle "] [\-s " sunit "]" Reformats the log to the specified log cycle and log stripe unit. This has the effect of clearing the log destructively. From patchwork Fri Dec 30 22:19:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13085671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9E8BC4332F for ; Sat, 31 Dec 2022 02:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231466AbiLaC3g (ORCPT ); Fri, 30 Dec 2022 21:29:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236141AbiLaC3d (ORCPT ); Fri, 30 Dec 2022 21:29:33 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA7CD1CB19 for ; Fri, 30 Dec 2022 18:29:32 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 392A361CFE for ; Sat, 31 Dec 2022 02:29:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9C7D5C433D2; Sat, 31 Dec 2022 02:29:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1672453771; bh=nEAtQNa7OQkQABY+p9/cW81hxe3Ae4yyhUolspb077Q=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=is8WFG9Jxo7T3jYcmyOZkwebfg83GhrqhDa94u/uhiIgY9PfqKgYKTAFiJMTQEerT +8mWlRsLiMc3PY/JfxMMaXemJfS6HpjBV6wCLksKHFhZTwsWONWoEiSRBcBWuaLCuL egZG9ly5bgjaJzyB5DpD0Ontrl7XV5/9Xw+w20UbwiyuT4wGpDlSzTCVwbHbUXuAoA YqCRelMMZ7oUlduBOiowjDwAwTRf8lmMf4xzo6kXYvozPKhndHmR9Zh37ojYfcQtam PtNPrX91g6Iux/F6C3VdmDYFcej9QPHLSe4Ar52XAkZl1hAz0/DamnjpT2OMvbWgTb uyCrHw+BhFQ7g== Subject: [PATCH 3/5] xfs_db: metadump external log devices From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Date: Fri, 30 Dec 2022 14:19:40 -0800 Message-ID: <167243878022.730695.17197730934160899346.stgit@magnolia> In-Reply-To: <167243877981.730695.7761889719607533776.stgit@magnolia> References: <167243877981.730695.7761889719607533776.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Teach the metadump command to dump the contents of an external log to the metadump file. Older mdrestore programs aren't going to recognize the new metablock info flag, change the magic number before adding new information flags to signal that the metablock is describing blocks on either an external log device or a realtime device. Realtime support isn't needed now, but it will be for realtime groups. Signed-off-by: Darrick J. Wong --- db/metadump.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++- db/xfs_metadump.sh | 5 +- include/xfs_metadump.h | 3 + man/man8/xfs_metadump.8 | 13 +++++- 4 files changed, 111 insertions(+), 8 deletions(-) diff --git a/db/metadump.c b/db/metadump.c index 996c97ca6a2..f337493d505 100644 --- a/db/metadump.c +++ b/db/metadump.c @@ -3002,6 +3002,74 @@ _("Could not discern log; image will contain unobfuscated metadata in log.")); return !write_buf(iocur_top); } +static int +copy_external_log(void) +{ + struct xlog log; + int dirty; + xfs_daddr_t logstart; + int logblocks; + int logversion; + int cycle = XLOG_INIT_CYCLE; + int error; + + if (show_progress) + print_progress("Copying external log"); + + push_cur(); + error = set_log_cur(&typtab[TYP_LOG], + XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart), + mp->m_sb.sb_logblocks * blkbb, DB_RING_IGN, NULL); + if (error) + return 0; + if (iocur_top->data == NULL) { + pop_cur(); + print_warning("cannot read external log"); + return !stop_on_read_error; + } + + /* If not obfuscating or zeroing, just copy the log as it is */ + if (!obfuscate && !zero_stale_data) + goto done; + + dirty = xlog_is_dirty(mp, &log, &x, 0); + + switch (dirty) { + case 0: + /* clear out a clean log */ + if (show_progress) + print_progress("Zeroing clean log"); + + logstart = XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart); + logblocks = XFS_FSB_TO_BB(mp, mp->m_sb.sb_logblocks); + logversion = xfs_has_logv2(mp) ? 2 : 1; + if (xfs_has_crc(mp)) + cycle = log.l_curr_cycle + 1; + + libxfs_log_clear(NULL, iocur_top->data, logstart, logblocks, + &mp->m_sb.sb_uuid, logversion, + mp->m_sb.sb_logsunit, XLOG_FMT, cycle, true); + break; + case 1: + /* keep the dirty log */ + if (obfuscate) + print_warning( +_("Warning: log recovery of an obfuscated metadata image can leak " +"unobfuscated metadata and/or cause image corruption. If possible, " +"please mount the filesystem to clean the log, or disable obfuscation.")); + break; + case -1: + /* log detection error */ + if (obfuscate) + print_warning( +_("Could not discern log; image will contain unobfuscated metadata in log.")); + break; + } + +done: + return !write_buf(iocur_top); +} + static int metadump_f( int argc, @@ -3012,6 +3080,7 @@ metadump_f( int start_iocur_sp; int outfd = -1; int ret; + bool copy_external = false; char *p; exitcode = 1; @@ -3035,7 +3104,7 @@ metadump_f( return 0; } - while ((c = getopt(argc, argv, "aegm:ow")) != EOF) { + while ((c = getopt(argc, argv, "aegm:owx")) != EOF) { switch (c) { case 'a': zero_stale_data = 0; @@ -3060,6 +3129,9 @@ metadump_f( case 'w': show_warnings = 1; break; + case 'x': + copy_external = true; + break; default: print_warning("bad option for metadump command"); return 0; @@ -3071,13 +3143,23 @@ metadump_f( return 0; } + /* + * Use the old format if there are no external devices with metadata to + * dump. + */ + if (mp->m_sb.sb_logstart != 0) + copy_external = false; + metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE); if (metablock == NULL) { print_warning("memory allocation failure"); return 0; } metablock->mb_blocklog = BBSHIFT; - metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC); + if (copy_external) + metablock->mb_magic = cpu_to_be32(XFS_MDX_MAGIC); + else + metablock->mb_magic = cpu_to_be32(XFS_MD_MAGIC); /* Set flags about state of metadump */ metablock->mb_info = XFS_METADUMP_INFO_FLAGS; @@ -3165,7 +3247,7 @@ metadump_f( exitcode = 0; - for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) { + for (agno = 0; !exitcode && agno < mp->m_sb.sb_agcount; agno++) { if (!scan_ag(agno)) { exitcode = 1; break; @@ -3184,6 +3266,16 @@ metadump_f( if (!exitcode) exitcode = write_index() < 0; + /* write the external log, if desired */ + if (!exitcode && mp->m_sb.sb_logstart == 0 && copy_external) { + metablock->mb_info |= XFS_METADUMP_LOGDEV; + + if (!copy_external_log()) + exitcode = 1; + if (!exitcode) + exitcode = write_index() < 0; + } + if (progress_since_warning) fputc('\n', stdout_metadump ? stderr : stdout); diff --git a/db/xfs_metadump.sh b/db/xfs_metadump.sh index 9852a5bc2b0..06bfc4e7bd4 100755 --- a/db/xfs_metadump.sh +++ b/db/xfs_metadump.sh @@ -6,9 +6,9 @@ OPTS=" " DBOPTS=" " -USAGE="Usage: xfs_metadump [-aefFogwV] [-m max_extents] [-l logdev] source target" +USAGE="Usage: xfs_metadump [-aefFgoVwx] [-m max_extents] [-l logdev] source target" -while getopts "aefgl:m:owFV" c +while getopts "aefgl:m:owFVx" c do case $c in a) OPTS=$OPTS"-a ";; @@ -24,6 +24,7 @@ do status=$? exit $status ;; + x) OPTS=$OPTS"-x ";; \?) echo $USAGE 1>&2 exit 2 ;; diff --git a/include/xfs_metadump.h b/include/xfs_metadump.h index fbd9902327b..2373b0d8b50 100644 --- a/include/xfs_metadump.h +++ b/include/xfs_metadump.h @@ -8,6 +8,7 @@ #define _XFS_METADUMP_H_ #define XFS_MD_MAGIC 0x5846534d /* 'XFSM' */ +#define XFS_MDX_MAGIC 0x584d4458 /* 'XMDX' */ typedef struct xfs_metablock { __be32 mb_magic; @@ -22,5 +23,7 @@ typedef struct xfs_metablock { #define XFS_METADUMP_OBFUSCATED (1 << 1) #define XFS_METADUMP_FULLBLOCKS (1 << 2) #define XFS_METADUMP_DIRTYLOG (1 << 3) +#define XFS_METADUMP_LOGDEV (1 << 4) /* targets external log device */ +#define XFS_METADUMP_RTDEV (1 << 5) /* targets realtime volume */ #endif /* _XFS_METADUMP_H_ */ diff --git a/man/man8/xfs_metadump.8 b/man/man8/xfs_metadump.8 index c0e79d77993..b940cb084b5 100644 --- a/man/man8/xfs_metadump.8 +++ b/man/man8/xfs_metadump.8 @@ -4,7 +4,7 @@ xfs_metadump \- copy XFS filesystem metadata to a file .SH SYNOPSIS .B xfs_metadump [ -.B \-aefFgow +.B \-aefFgowx ] [ .B \-m .I max_extents @@ -123,8 +123,10 @@ is stdout. .TP .BI \-l " logdev" For filesystems which use an external log, this specifies the device where the -external log resides. The external log is not copied, only internal logs are -copied. +external log resides. +To record the contents of the external log in the dump, the +.B \-x +option must also be specified. .TP .B \-m Set the maximum size of an allowed metadata extent. Extremely large metadata @@ -138,6 +140,11 @@ Disables obfuscation of file names and extended attributes. Prints warnings of inconsistent metadata encountered to stderr. Bad metadata is still copied. .TP +.B \-x +Dump the external log device, if present. +The metadump file will not be compatible with older versions of +.BR xfs_mdrestore (1). +.TP .B \-V Prints the version number and exits. .SH DIAGNOSTICS From patchwork Fri Dec 30 22:19:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13085672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9B30C4332F for ; Sat, 31 Dec 2022 02:29:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236149AbiLaC3v (ORCPT ); Fri, 30 Dec 2022 21:29:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236141AbiLaC3v (ORCPT ); Fri, 30 Dec 2022 21:29:51 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2C6F1CB17 for ; Fri, 30 Dec 2022 18:29:49 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5E367B81E72 for ; Sat, 31 Dec 2022 02:29:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1EE80C433EF; Sat, 31 Dec 2022 02:29:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1672453787; bh=c+dlL9gCIxOVHKYd7GxKSXIdJB4iLg98Gp4SUnZq2Po=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=kV1HJlysTJ9n0uLhYLanKx6ewkbzNSOCb0IbFBUIbFPdc2nySdGNfnnuXkO11XYCS gtSFGxIeQLisYBECrJZTYmKGxaDm+aXOlMVnhTuC0FRqEn/nlyXl1UqkyVgbYgSomj cFoEHvSxHLw/JfrEe9FJzpmR91qRAoQunC2ewXVtuNUxXll7DMc1OfqUF6O+zvejvk 54JwrZF+zlGTDyOaCEXkQOvumy0xo5n8VLjN2d1c/NapfhXuM62ps/R06KVrFtRAzJ CwCSzrw0hZ1INAHEkQTJSRDRW8t5deSWUQ1HfPXcby/35feuRREUDJAArbVPtnkbrT /d3QXg5QNvo/Q== Subject: [PATCH 4/5] xfs_mdrestore: restore log contents to external log devices From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Date: Fri, 30 Dec 2022 14:19:40 -0800 Message-ID: <167243878035.730695.15058686240875755491.stgit@magnolia> In-Reply-To: <167243877981.730695.7761889719607533776.stgit@magnolia> References: <167243877981.730695.7761889719607533776.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Support restoring log data to an external log device, if the dumped filesystem had one. Signed-off-by: Darrick J. Wong --- man/man8/xfs_mdrestore.8 | 8 ++ mdrestore/xfs_mdrestore.c | 176 +++++++++++++++++++++++++++++++-------------- 2 files changed, 131 insertions(+), 53 deletions(-) diff --git a/man/man8/xfs_mdrestore.8 b/man/man8/xfs_mdrestore.8 index 72f3b297787..4626b98e749 100644 --- a/man/man8/xfs_mdrestore.8 +++ b/man/man8/xfs_mdrestore.8 @@ -5,12 +5,17 @@ xfs_mdrestore \- restores an XFS metadump image to a filesystem image .B xfs_mdrestore [ .B \-gi +] [ +.B \-l logdev ] .I source .I target .br .B xfs_mdrestore .B \-i +[ +.B \-l logdev +] .I source .br .B xfs_mdrestore \-V @@ -43,6 +48,9 @@ can be destroyed. .B \-g Shows restore progress on stdout. .TP +.B \-l +Restore log contents to this external log device. +.TP .B \-i Shows metadump information on stdout. If no .I target diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c index 9f8cbe98cd6..4318fac9008 100644 --- a/mdrestore/xfs_mdrestore.c +++ b/mdrestore/xfs_mdrestore.c @@ -38,6 +38,67 @@ print_progress(const char *fmt, ...) progress_since_warning = 1; } +extern int platform_check_ismounted(char *, char *, struct stat *, int); + +static int +open_device( + char *path, + int *is_target_file) +{ + struct stat statbuf; + int open_flags = O_RDWR; + int dst_fd; + + *is_target_file = 0; + if (stat(path, &statbuf) < 0) { + /* ok, assume it's a file and create it */ + open_flags |= O_CREAT; + *is_target_file = 1; + } else if (S_ISREG(statbuf.st_mode)) { + open_flags |= O_TRUNC; + *is_target_file = 1; + } else { + /* + * check to make sure a filesystem isn't mounted on the device + */ + if (platform_check_ismounted(path, NULL, &statbuf, 0)) + fatal("a filesystem is mounted on target device \"%s\"," + " cannot restore to a mounted filesystem.\n", + path); + } + + dst_fd = open(path, open_flags, 0644); + if (dst_fd < 0) + fatal("couldn't open target \"%s\"\n", path); + + return dst_fd; +} + +static void +check_dev( + int dst_fd, + int is_target_file, + unsigned long long bytes) +{ + if (is_target_file) { + /* ensure regular files are correctly sized */ + + if (ftruncate(dst_fd, bytes)) + fatal("cannot set filesystem image size: %s\n", + strerror(errno)); + } else { + /* ensure device is sufficiently large enough */ + + char *lb[XFS_MAX_SECTORSIZE] = { NULL }; + off64_t off; + + off = bytes - sizeof(lb); + if (pwrite(dst_fd, lb, sizeof(lb), off) < 0) + fatal("failed to write last block, is target too " + "small? (error: %s)\n", strerror(errno)); + } +} + /* * perform_restore() -- do the actual work to restore the metadump * @@ -53,7 +114,8 @@ perform_restore( FILE *src_f, int dst_fd, int is_target_file, - const struct xfs_metablock *mbp) + const struct xfs_metablock *mbp, + char *log_path) { struct xfs_metablock *metablock; /* header + index + blocks */ __be64 *block_index; @@ -64,6 +126,10 @@ perform_restore( int mb_count; xfs_sb_t sb; int64_t bytes_read; + int log_fd = -1; + bool is_mdx; + + is_mdx = mbp->mb_magic == cpu_to_be32(XFS_MDX_MAGIC); block_size = 1 << mbp->mb_blocklog; max_indices = (block_size - sizeof(xfs_metablock_t)) / sizeof(__be64); @@ -76,6 +142,7 @@ perform_restore( if (mb_count == 0 || mb_count > max_indices) fatal("bad block count: %u\n", mb_count); + memcpy(metablock, mbp, sizeof(struct xfs_metablock)); block_index = (__be64 *)((char *)metablock + sizeof(xfs_metablock_t)); block_buffer = (char *)metablock + block_size; @@ -106,32 +173,43 @@ perform_restore( ((struct xfs_dsb*)block_buffer)->sb_inprogress = 1; - if (is_target_file) { - /* ensure regular files are correctly sized */ - - if (ftruncate(dst_fd, sb.sb_dblocks * sb.sb_blocksize)) - fatal("cannot set filesystem image size: %s\n", - strerror(errno)); - } else { - /* ensure device is sufficiently large enough */ - - char *lb[XFS_MAX_SECTORSIZE] = { NULL }; - off64_t off; - - off = sb.sb_dblocks * sb.sb_blocksize - sizeof(lb); - if (pwrite(dst_fd, lb, sizeof(lb), off) < 0) - fatal("failed to write last block, is target too " - "small? (error: %s)\n", strerror(errno)); - } + check_dev(dst_fd, is_target_file, sb.sb_dblocks * sb.sb_blocksize); bytes_read = 0; for (;;) { + int write_fd = dst_fd; + + if (metablock->mb_magic != mbp->mb_magic) + fatal("magic value 0x%x wrong, expected 0x%x\n", + metablock->mb_magic, mbp->mb_magic); + + if (metablock->mb_info & XFS_METADUMP_LOGDEV) { + int log_is_file; + + if (!is_mdx) + fatal("logdev set on an old style metadump?\n"); + if (log_fd == -1) { + if (!log_path) + fatal( + "metadump has log contents but -l was not specified?\n"); + log_fd = open_device(log_path, &log_is_file); + check_dev(log_fd, log_is_file, + sb.sb_logblocks * sb.sb_blocksize); + } + write_fd = log_fd; + } + if (metablock->mb_info & XFS_METADUMP_RTDEV) { + if (!is_mdx) + fatal("rtdev set on an old style metadump?\n"); + fatal("rtdev not supported\n"); + } + if (show_progress && (bytes_read & ((1 << 20) - 1)) == 0) print_progress("%lld MB read", bytes_read >> 20); for (cur_index = 0; cur_index < mb_count; cur_index++) { - if (pwrite(dst_fd, &block_buffer[cur_index << + if (pwrite(write_fd, &block_buffer[cur_index << mbp->mb_blocklog], block_size, be64_to_cpu(block_index[cur_index]) << BBSHIFT) < 0) @@ -139,11 +217,20 @@ perform_restore( be64_to_cpu(block_index[cur_index]) << BBSHIFT, strerror(errno)); } - if (mb_count < max_indices) - break; + if (is_mdx) { + size_t nr = fread(metablock, block_size, 1, src_f); - if (fread(metablock, block_size, 1, src_f) != 1) - fatal("error reading from metadump file\n"); + if (nr == 0) + break; + if (nr != 1) + fatal("error reading from extended metadump file\n"); + } else { + if (mb_count < max_indices) + break; + + if (fread(metablock, block_size, 1, src_f) != 1) + fatal("error reading from metadump file\n"); + } mb_count = be16_to_cpu(metablock->mb_count); if (mb_count == 0) @@ -170,38 +257,41 @@ perform_restore( if (pwrite(dst_fd, block_buffer, sb.sb_sectsize, 0) < 0) fatal("error writing primary superblock: %s\n", strerror(errno)); + if (log_fd >= 0) + close(log_fd); + free(metablock); } static void usage(void) { - fprintf(stderr, "Usage: %s [-V] [-g] [-i] source target\n", progname); + fprintf(stderr, "Usage: %s [-V] [-g] [-i] [-l logdev] source target\n", progname); exit(1); } -extern int platform_check_ismounted(char *, char *, struct stat *, int); - int main( int argc, char **argv) { + char *log_path = NULL; FILE *src_f; int dst_fd; int c; - int open_flags; - struct stat statbuf; int is_target_file; struct xfs_metablock mb; progname = basename(argv[0]); - while ((c = getopt(argc, argv, "giV")) != EOF) { + while ((c = getopt(argc, argv, "gl:iV")) != EOF) { switch (c) { case 'g': show_progress = 1; break; + case 'l': + log_path = optarg; + break; case 'i': show_info = 1; break; @@ -238,7 +328,8 @@ main( if (fread(&mb, sizeof(mb), 1, src_f) != 1) fatal("error reading from metadump file\n"); - if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC)) + if (mb.mb_magic != cpu_to_be32(XFS_MD_MAGIC) && + mb.mb_magic != cpu_to_be32(XFS_MDX_MAGIC)) fatal("specified file is not a metadata dump\n"); if (show_info) { @@ -260,30 +351,9 @@ main( optind++; /* check and open target */ - open_flags = O_RDWR; - is_target_file = 0; - if (stat(argv[optind], &statbuf) < 0) { - /* ok, assume it's a file and create it */ - open_flags |= O_CREAT; - is_target_file = 1; - } else if (S_ISREG(statbuf.st_mode)) { - open_flags |= O_TRUNC; - is_target_file = 1; - } else { - /* - * check to make sure a filesystem isn't mounted on the device - */ - if (platform_check_ismounted(argv[optind], NULL, &statbuf, 0)) - fatal("a filesystem is mounted on target device \"%s\"," - " cannot restore to a mounted filesystem.\n", - argv[optind]); - } + dst_fd = open_device(argv[optind], &is_target_file); - dst_fd = open(argv[optind], open_flags, 0644); - if (dst_fd < 0) - fatal("couldn't open target \"%s\"\n", argv[optind]); - - perform_restore(src_f, dst_fd, is_target_file, &mb); + perform_restore(src_f, dst_fd, is_target_file, &mb, log_path); close(dst_fd); if (src_f != stdin) From patchwork Fri Dec 30 22:19:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13085673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58635C4332F for ; Sat, 31 Dec 2022 02:30:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236156AbiLaCaG (ORCPT ); Fri, 30 Dec 2022 21:30:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236141AbiLaCaE (ORCPT ); Fri, 30 Dec 2022 21:30:04 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEB931CB17 for ; Fri, 30 Dec 2022 18:30:03 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4A38E61CFE for ; Sat, 31 Dec 2022 02:30:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9EFF0C433EF; Sat, 31 Dec 2022 02:30:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1672453802; bh=adiCzSPihOW6N6ACa8PfPRYlmQgnxd3/zAXfe1mQV2o=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=PCE3YjYbYcLK9NDYewdY7ORA8m8VSRbF3oyiUWaD1r4K9gZ0qW0OZxn2Bf5BLf2bE kngoiqWU+E3LpxSZxkHrRdBnXycIgP/pkXII+oSHsy8gx9l1VBxkjBZdpbDpzis1vY XHRxTl/2dOYzVgqmzEqq1wYBEt5BINM4qu9hN+zQSaxzbDC+GcQq3EMBbhDwiLw35a PWCeB8O7F9K9ewzdeFLHgu5qZfbKvPJmWFueYq6cq34dkj4KX7WFzy1C+s/MSGSmWd BMx2EEL06RhVsxMgg3QNUVRZXqyqgbc6V4hSh8qmEOivhqF6wbZi8KWs5YPvj2S6nE +LDMMYfpdZuBQ== Subject: [PATCH 5/5] xfs_mdrestore: fix missed progress reporting From: "Darrick J. Wong" To: djwong@kernel.org, cem@kernel.org Cc: linux-xfs@vger.kernel.org Date: Fri, 30 Dec 2022 14:19:40 -0800 Message-ID: <167243878049.730695.4353400122351023727.stgit@magnolia> In-Reply-To: <167243877981.730695.7761889719607533776.stgit@magnolia> References: <167243877981.730695.7761889719607533776.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Currently, the progress reporting only triggers when the number of bytes read is exactly a multiple of a megabyte. This isn't always guaranteed, since AG headers can be 512 bytes in size. Fix the algorithm by recording the number of megabytes we've reported as being read, and emit a new report any time the bytes_read count, once converted to megabytes, doesn't match. Signed-off-by: Darrick J. Wong --- mdrestore/xfs_mdrestore.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c index 4318fac9008..672010bcc6e 100644 --- a/mdrestore/xfs_mdrestore.c +++ b/mdrestore/xfs_mdrestore.c @@ -126,6 +126,7 @@ perform_restore( int mb_count; xfs_sb_t sb; int64_t bytes_read; + int64_t mb_read = 0; int log_fd = -1; bool is_mdx; @@ -205,8 +206,14 @@ perform_restore( fatal("rtdev not supported\n"); } - if (show_progress && (bytes_read & ((1 << 20) - 1)) == 0) - print_progress("%lld MB read", bytes_read >> 20); + if (show_progress) { + int64_t mb_now = bytes_read >> 20; + + if (mb_now != mb_read) { + print_progress("%lld MB read", mb_now); + mb_read = mb_now; + } + } for (cur_index = 0; cur_index < mb_count; cur_index++) { if (pwrite(write_fd, &block_buffer[cur_index << @@ -245,6 +252,9 @@ perform_restore( bytes_read += block_size + (mb_count << mbp->mb_blocklog); } + if (show_progress && bytes_read > (mb_read << 20)) + print_progress("%lld MB read", mb_read + 1); + if (progress_since_warning) putchar('\n');