From patchwork Tue Jun 6 22:29:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13269791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E69AC7EE2F for ; Tue, 6 Jun 2023 22:29:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239864AbjFFW3d (ORCPT ); Tue, 6 Jun 2023 18:29:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239862AbjFFW3a (ORCPT ); Tue, 6 Jun 2023 18:29:30 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBD7E1717; Tue, 6 Jun 2023 15:29:29 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8871562D9A; Tue, 6 Jun 2023 22:29:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E0AA9C433D2; Tue, 6 Jun 2023 22:29:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686090569; bh=0eqc4eS96XfhmaM65cuwLkQ+u2jlhnVKaAqHvfso1I8=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=t4QLybElDJjKdLnQqGRX3KjzdWFtcrXCkDUvITSM07B8LNBxVL2CvYkXZtpVBe2eh 8Y+OrnSY4MEZIlQQQfYC6Cwtl7K+7Sb3XuNyIbxVibX6rlBpYaxxBU46As6768JCvY ZK7yMyD1mvYW9RieDNWGqbiDtyiOrOYHDPnqcDlQg+yzlnVt2ui3SC5wJRuAFzFsz1 M40mAhcVCUqnIKTaEDgY4CfmgnB6pW2Saq07QXlk+IqdcznWiy4k1boxkies82KDJA x35T0tpg5juIgrI0khwb4MmIONw7KuhOw2vqNMKMLAC24iUqhTlBCC7vRo/huDZ8Ew +r/4PkQ6KCNjw== Subject: [PATCH 1/3] fuzzy: disallow post-test online rebuilds when testing online fsck From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Jun 2023 15:29:28 -0700 Message-ID: <168609056848.2592490.8544982472802991550.stgit@frogsfrogsfrogs> In-Reply-To: <168609056295.2592490.1272515528324889317.stgit@frogsfrogsfrogs> References: <168609056295.2592490.1272515528324889317.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong If we're testing the online fsck code or running fuzz tests of the filesystem, don't let the post-test checks rebuild the filesystem metadata, because this is redundant with the test and will disturb the metadata if the tools fail. Signed-off-by: Darrick J. Wong --- common/fuzzy | 7 ++++++- common/rc | 2 +- common/xfs | 12 +++++++++++- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/common/fuzzy b/common/fuzzy index a78a354142..7228158034 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -721,6 +721,9 @@ _scratch_xfs_fuzz_metadata() { # Always capture full core dumps from crashing tools ulimit -c unlimited + _xfs_skip_online_rebuild + _xfs_skip_offline_rebuild + echo "${fields}" | while read field; do echo "${verbs}" | while read fuzzverb; do __scratch_xfs_fuzz_mdrestore @@ -1356,6 +1359,9 @@ _scratch_xfs_stress_scrub() { rm -f "$tmp.scrub" fi + _xfs_skip_online_rebuild + _xfs_skip_offline_rebuild + local start="$(date +%s)" local end if [ -n "$SOAK_DURATION" ]; then @@ -1429,7 +1435,6 @@ __scratch_xfs_stress_setup_force_rebuild() { # and wait for 30*TIME_FACTOR seconds to see if the filesystem goes down. # Same requirements and arguments as _scratch_xfs_stress_scrub. _scratch_xfs_stress_online_repair() { - touch "$RESULT_DIR/.skip_orebuild" # no need to test online rebuild __scratch_xfs_stress_setup_force_rebuild XFS_SCRUB_FORCE_REPAIR=1 _scratch_xfs_stress_scrub "$@" } diff --git a/common/rc b/common/rc index 37074371d7..c2ed40d768 100644 --- a/common/rc +++ b/common/rc @@ -1749,7 +1749,7 @@ _require_scratch_nocheck() exit 1 fi fi - rm -f ${RESULT_DIR}/require_scratch "$RESULT_DIR/.skip_orebuild" + rm -f ${RESULT_DIR}/require_scratch "$RESULT_DIR/.skip_orebuild" "$RESULT_DIR/.skip_rebuild" } # we need the scratch device and it needs to not be an lvm device diff --git a/common/xfs b/common/xfs index 137ac9dbbe..d85acd9572 100644 --- a/common/xfs +++ b/common/xfs @@ -721,6 +721,16 @@ _scratch_xfs_mdrestore() _xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$@" } +# Do not use xfs_repair (offline fsck) to rebuild the filesystem +_xfs_skip_offline_rebuild() { + touch "$RESULT_DIR/.skip_rebuild" +} + +# Do not use xfs_scrub (online fsck) to rebuild the filesystem +_xfs_skip_online_rebuild() { + touch "$RESULT_DIR/.skip_orebuild" +} + # run xfs_check and friends on a FS. _check_xfs_filesystem() { @@ -849,7 +859,7 @@ _check_xfs_filesystem() fi # Optionally test the index rebuilding behavior. - if [ -n "$TEST_XFS_REPAIR_REBUILD" ]; then + if [ -n "$TEST_XFS_REPAIR_REBUILD" ] && [ ! -e "$RESULT_DIR/.skip_rebuild" ]; then rebuild_ok=1 $XFS_REPAIR_PROG $extra_options $extra_log_options $extra_rt_options $device >$tmp.repair 2>&1 if [ $? -ne 0 ]; then From patchwork Tue Jun 6 22:29:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13269792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F180AC77B7A for ; Tue, 6 Jun 2023 22:29:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239870AbjFFW3g (ORCPT ); Tue, 6 Jun 2023 18:29:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239862AbjFFW3g (ORCPT ); Tue, 6 Jun 2023 18:29:36 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CC2A1717; Tue, 6 Jun 2023 15:29:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 18D4C6388E; Tue, 6 Jun 2023 22:29:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 74363C433D2; Tue, 6 Jun 2023 22:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686090574; bh=VLBWifqvMXkHC5Bk8nQPtw5uVk0zWt5F7qclcEA/EPk=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=A2bcDNSh2GkRmhAKcFQih6F6hkrgG4WnvdOwUgT1MSD7XanKDLlJ7d7QkOmL4ovKn VO3X8bcL1s1hZ3kG54xeY+9ERJz8aOanW+9BZUNNsgoxqyjd954U3zqNnnRSyG3vVn 6V0UtJAGj66z8DrJaanNPhDtvUuSlLc0OvFR0pkpZcAYx80zemyiaG+2jaxBHpmmiD pcKPIUL+8V7aimN2DN3+x/u9+rWfXnf8BzXZvFLELX56IaZSgMY4ZJ2mJO+J+ga0Xg HoYb1R1Jo7ak3eG8AZXFbqMiUqWzDtuloIqx9+hk8NY1LEHg0EWfjlztV0f8JS8JA9 TOhw/1ky7MZ3Q== Subject: [PATCH 2/3] xfs/503: don't rebuild the fs metadata when testing metadump From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Jun 2023 15:29:34 -0700 Message-ID: <168609057408.2592490.14528335416861268945.stgit@frogsfrogsfrogs> In-Reply-To: <168609056295.2592490.1272515528324889317.stgit@frogsfrogsfrogs> References: <168609056295.2592490.1272515528324889317.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong This test exercises metadump with the standard populate image. There's no need to test rebuilding the entire fs every step of the way since we're just going to metadump over it. Signed-off-by: Darrick J. Wong --- tests/xfs/503 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/xfs/503 b/tests/xfs/503 index c786b04ccd..f5710ece35 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -32,6 +32,8 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_xfs_copy _require_scratch_nocheck _require_populate_commands +_xfs_skip_online_rebuild +_xfs_skip_offline_rebuild echo "Format and populate" _scratch_populate_cached nofill > $seqres.full 2>&1 From patchwork Tue Jun 6 22:29:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13269793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48F1EC7EE2F for ; Tue, 6 Jun 2023 22:29:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239872AbjFFW3o (ORCPT ); Tue, 6 Jun 2023 18:29:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239862AbjFFW3m (ORCPT ); Tue, 6 Jun 2023 18:29:42 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 257361717; Tue, 6 Jun 2023 15:29:41 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AEA936387C; Tue, 6 Jun 2023 22:29:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13EC6C433EF; Tue, 6 Jun 2023 22:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686090580; bh=fEm/Idf75DCZarblOvbO8t1Xx4/xc9Xee6F/imxJwIU=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=FvfvPJ7YH80Sq7U4BAKjVI6gXViI/fAtswokgFdbuRrv9NLFMtTFrtMhxzmqfb2xH /442zC8wKGoAV5PHx+wIQvAH1Uk97H3uhvm3h8hcYz64xWNYNgVHJKXEQQGzjQ+O5q 596Y5CGzQP9duViqvW7e4D4KqzC5G0JNnWPzyeUaq4HaCdk+8fGn8IU50glB/E2dKH KORS4rldytRUDIhs0RDYF32dMYngUiiGMQbHdLG8ZLHPTq3mrQZUzAHca/VyBQA8XA kahyT8R/XSHl3SrVeXQaArkRe13RwWDwGNYXhitosoMp9F5M2qiircby0JKG7Wna3j G6rtz6+4YC95w== Subject: [PATCH 3/3] common/xfs: compress online repair rebuild output by default From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Jun 2023 15:29:39 -0700 Message-ID: <168609057963.2592490.3833577466322610879.stgit@frogsfrogsfrogs> In-Reply-To: <168609056295.2592490.1272515528324889317.stgit@frogsfrogsfrogs> References: <168609056295.2592490.1272515528324889317.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Force-repairing the filesystem after a test can fill up /tmp with quite a lot of logging message. We don't have a better place to stash that output in case the scrub fails and we need to analyze it later, so compress it with gzip and only decompress it later. Signed-off-by: Darrick J. Wong --- common/xfs | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/common/xfs b/common/xfs index d85acd9572..c7671f8f9d 100644 --- a/common/xfs +++ b/common/xfs @@ -910,25 +910,28 @@ _check_xfs_filesystem() # repairs considerably when the directory tree is very large. find $mntpt &>/dev/null & - XFS_SCRUB_FORCE_REPAIR=1 "$XFS_SCRUB_PROG" -v -d $mntpt > $tmp.scrub 2>&1 - if [ $? -ne 0 ]; then - if grep -q 'No space left on device' $tmp.scrub; then + XFS_SCRUB_FORCE_REPAIR=1 "$XFS_SCRUB_PROG" -v -d $mntpt 2>&1 | gzip > $tmp.scrub.gz + ret=$? + if [ $ret -ne 0 ]; then + if zgrep -q 'No space left on device' $tmp.scrub.gz; then # It's not an error if the fs does not have # enough space to complete a repair. We will # check everything, though. - echo "*** XFS_SCRUB_FORCE_REPAIR=1 xfs_scrub -v -d ran out of space ***" >> $seqres.full - cat $tmp.scrub >> $seqres.full + echo "*** XFS_SCRUB_FORCE_REPAIR=1 xfs_scrub -v -d ran out of space ret=$ret ***" >> $seqres.full + echo "See $seqres.scrubout.gz for details." >> $seqres.full + mv $tmp.scrub.gz $seqres.scrubout.gz echo "*** end xfs_scrub output" >> $seqres.full else _log_err "_check_xfs_filesystem: filesystem on $device failed scrub orebuild" - echo "*** XFS_SCRUB_FORCE_REPAIR=1 xfs_scrub -v -d output ***" >> $seqres.full - cat $tmp.scrub >> $seqres.full + echo "*** XFS_SCRUB_FORCE_REPAIR=1 xfs_scrub -v -d output ret=$ret ***" >> $seqres.full + echo "See $seqres.scrubout.gz for details." >> $seqres.full + mv $tmp.scrub.gz $seqres.scrubout.gz echo "*** end xfs_scrub output" >> $seqres.full ok=0 orebuild_ok=0 fi fi - rm -f $tmp.scrub + rm -f $tmp.scrub.gz # Clear force_repair because xfs_scrub could have set it $XFS_IO_PROG -x -c 'inject noerror' "$mntpt" >> $seqres.full