From patchwork Wed Feb 7 02:18:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548035 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B77D4DDAD; Wed, 7 Feb 2024 02:18:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272320; cv=none; b=I0LU/3dampwyUQQqR5yRcnbOOlfVcd1+iVHR73FMWRjy/SI0+nbKjDbG0Mm+MorjPNHo9jgBKbQnphnyQtaF+TEEZGg5GiOih7352wUg2vO/0sz58j3H3aJ3mVWHkSYYI6Uzq5oVicNsGd4Te4Vb6qT7tWMjAj9+5yg5pvmYZvw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272320; c=relaxed/simple; bh=fs+/6X504QXUZiSdF82CtLw9I4//7aorGeisV9NNaoc=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QjSx5MBM0Ey877l+JcJ/lsdvfTXTnpMt0zMGwka1ow7i2rZpqgssnq0TW4MuMQMlmcwfwXRaD76jlRErSLHQ9SSDC7iZN9yl8NkmEGnl8Gqbsk1MA9uk5FdyE4ZBHl2rvJ/MeB8sUk69p+KeMt+V16TakczLc5niDm3qyUxfB9Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pjkUwKFm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pjkUwKFm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2600BC433C7; Wed, 7 Feb 2024 02:18:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272320; bh=fs+/6X504QXUZiSdF82CtLw9I4//7aorGeisV9NNaoc=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=pjkUwKFmlCOwkJt+N+xl6sCbpaJfvlGXcKQi0Fk0bOpyQD+JK3lfy8c9aH7eV2r39 aE3JcprgS+qixtCvzU0/pVBFtH3/nclmNJNASnnHhd3Y2Qw6L/3/UomM5XJGKkmNVH agF1T+cA4X1llxh5oS8fgdrXQC8RVEWi8OmcpLO3soGKwJ9RKwkqQrswa7p8G3PHBX Ra+0+KMz8DC+7Rbq1MxI/9tLwRmgZ+1ontZ3106YYaCM6iK9+iK00krojCZ4+MQ2/z qS0gI/ioBas9tj6S1a5IQLn3Ql0Ca7siTpkPrzJJrNJSZeul+Y6C1Z7Ff2zTTwdQN3 rAOUIBIycV/rw== Subject: [PATCH 01/10] generic/256: constrain runtime with TIME_FACTOR From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Andrey Albershteyn , Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:18:39 -0800 Message-ID: <170727231958.3726171.771566828109380255.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong This test runs 500 iterations of a "fill the fs and try to punch" test. Hole punching can be particularly slow if, say, the filesystem is mounted with -odiscard and the DISCARD operation takes a very long time. In extreme cases, I can see test runtimes of 4+ hours. Constrain the runtime of _test_full_fs_punch by establishing a deadline of (30 seconds * TIME_FACTOR) and breaking out of the for loop if the test goes beyond the time budget. This keeps the runtime within the customary 30 seconds. Signed-off-by: "Darrick J. Wong" Reviewed-by: Andrey Albershteyn Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- tests/generic/256 | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/generic/256 b/tests/generic/256 index 808a730f3a..ea6cc2938a 100755 --- a/tests/generic/256 +++ b/tests/generic/256 @@ -44,6 +44,8 @@ _test_full_fs_punch() local file_len=$(( $(( $hole_len + $hole_interval )) * $iterations )) local path=`dirname $file_name` local hole_offset=0 + local start_time + local stop_time if [ $# -ne 5 ] then @@ -57,6 +59,9 @@ _test_full_fs_punch() -c "fsync" $file_name &> /dev/null chmod 666 $file_name + start_time="$(date +%s)" + stop_time=$(( start_time + (30 * TIME_FACTOR) )) + # All files are created as a non root user to prevent reserved blocks # from being consumed. _fill_fs $(( 1024 * 1024 * 1024 )) $path/fill $block_size 1 \ @@ -64,6 +69,8 @@ _test_full_fs_punch() for (( i=0; i<$iterations; i++ )) do + test "$(date +%s)" -ge "$stop_time" && break + # This part must not be done as root in order to # test that reserved blocks are used when needed _user_do "$XFS_IO_PROG -f -c \"fpunch $hole_offset $hole_len\" $file_name" From patchwork Wed Feb 7 02:18:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548036 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EAE8DDAD; Wed, 7 Feb 2024 02:18:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272326; cv=none; b=u1LOyt6pXS4MWwqg1UsPa8vYg3quv3N/Mmc9JVAfreHc2JUnOr6k4yCm6p2Bg1+nqyYLh+YYafK5kaeCDGQHQ7uCgp8y4FwS1lMYzHYsBeieFM/D/YZMV1uqHT+7PRUa9/jt9bFae7lFx5aHkZYzEmv87wV+ILYfvzgHQYQjeBk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272326; c=relaxed/simple; bh=TJ5FeDirYnRdm5/usk5qnsXnux3D16SRIp1x05YuE90=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=b+XenuGKUin70Z3Qng6zzWPcjkg9+NicObTo3sfD742A8mYJSEl6KuA3a581mbYkkBwTaqmWSlkzQ2RHSYqKCJD02l/9threJbdpClNgFmSXC25Vs3jhPLy/AANZ4UtyNbpFCnS7XXADR1m2ghykI3mNSQNmUlutUmFnsOt9Z2g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VDGb5viY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VDGb5viY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4687C433F1; Wed, 7 Feb 2024 02:18:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272325; bh=TJ5FeDirYnRdm5/usk5qnsXnux3D16SRIp1x05YuE90=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=VDGb5viYxjpfjyH/G8z5HcYKpVTHNOw8WSriEhUECjWNHMDalK4DoFK2Z7Z6oNuQm lB3kT6gzOeWDyf1WXtCluk7RaONoZdpehpB8rZclkE5JQAe+bHwYDaqFJRyNO5IWIt tmoDkUHNi4NlhFRf0/DwNKKOw8gfZkCCk2KQjoDwkyXnDmbX7/FLw2BV3EhfIDcl+K K9S3HujZkRXh9MKpB45chG4gx75HvF6A+rMYktVaPULxbcP5FCppytvT+hoMYb8YR5 AO6WlaoP53DbN6ZNLsiGqrrbJotyCip6HEkakNY35avFNdkTfY5ZE7UxwAVBUtTQIV iXP9o+mVpf/uw== Subject: [PATCH 02/10] common/xfs: simplify maximum metadump format detection From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:18:45 -0800 Message-ID: <170727232530.3726171.13484518243714744011.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong xfs_metadump (aka the wrapper around xfs_db -c metadump) advertises the -v switch to turn on v2 format in its help screen. There's no need to fire up xfs_db on the scratch device which will load the AGs and take much longer. While we're at it, reduce the amount of boilerplate in the test files by changing the function to emit the max version supported. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- common/xfs | 10 +++++++--- tests/xfs/129 | 3 +-- tests/xfs/234 | 3 +-- tests/xfs/253 | 3 +-- tests/xfs/291 | 3 +-- tests/xfs/432 | 3 +-- tests/xfs/503 | 3 +-- tests/xfs/605 | 3 +-- 8 files changed, 14 insertions(+), 17 deletions(-) diff --git a/common/xfs b/common/xfs index 11cfd79562..248ccefda3 100644 --- a/common/xfs +++ b/common/xfs @@ -713,10 +713,14 @@ _xfs_mdrestore() { $XFS_MDRESTORE_PROG $options "${metadump}" "${device}" } -_scratch_metadump_v2_supported() +# What is the maximum metadump file format supported by xfs_metadump? +_xfs_metadump_max_version() { - $XFS_DB_PROG -c "help metadump" $SCRATCH_DEV | \ - grep -q "Metadump version to be used" + if $XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-v version'; then + echo 2 + else + echo 1 + fi } # Snapshot the metadata on the scratch device diff --git a/tests/xfs/129 b/tests/xfs/129 index 8a817b416c..cdac2349df 100755 --- a/tests/xfs/129 +++ b/tests/xfs/129 @@ -106,8 +106,7 @@ verify_metadump_v2() _scratch_mkfs >/dev/null 2>&1 -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) _scratch_mount diff --git a/tests/xfs/234 b/tests/xfs/234 index c9bdb674ab..f4f8af6d3a 100755 --- a/tests/xfs/234 +++ b/tests/xfs/234 @@ -106,8 +106,7 @@ verify_metadump_v2() _scratch_mkfs >/dev/null 2>&1 -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) _scratch_mount diff --git a/tests/xfs/253 b/tests/xfs/253 index 8e18ddb83a..3b567999d8 100755 --- a/tests/xfs/253 +++ b/tests/xfs/253 @@ -233,8 +233,7 @@ cd $here _scratch_unmount -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) verify_metadump_v1 $max_md_version diff --git a/tests/xfs/291 b/tests/xfs/291 index 33193eb78e..1433140821 100755 --- a/tests/xfs/291 +++ b/tests/xfs/291 @@ -92,8 +92,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed" # Yes they can! Now... # Can xfs_metadump cope with this monster? -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) for md_version in $(seq 1 $max_md_version); do version="" diff --git a/tests/xfs/432 b/tests/xfs/432 index a215d3ce2e..7e402aa88f 100755 --- a/tests/xfs/432 +++ b/tests/xfs/432 @@ -87,8 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks" echo "Try to metadump, restore and check restored metadump image" -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) for md_version in $(seq 1 $max_md_version); do version="" diff --git a/tests/xfs/503 b/tests/xfs/503 index a1479eb613..8643c3d483 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -54,8 +54,7 @@ check_restored_metadump_image() _destroy_loop_device $loop_dev } -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) echo "metadump and mdrestore" for md_version in $(seq 1 $max_md_version); do diff --git a/tests/xfs/605 b/tests/xfs/605 index 5cbf5d2550..f2cd7aba98 100755 --- a/tests/xfs/605 +++ b/tests/xfs/605 @@ -44,8 +44,7 @@ testfile=${SCRATCH_MNT}/testfile echo "Format filesystem on scratch device" _scratch_mkfs >> $seqres.full 2>&1 -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) external_log=0 if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then From patchwork Wed Feb 7 02:18:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548037 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E06EDDB3; Wed, 7 Feb 2024 02:18:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272332; cv=none; b=iWunw7lC1ifWkqNgtygHraCcYGtzTwpinL/gtIkTYaEKXzksQvDQmhfIL4G9WEo3/T7WP/RLI6CQJyQVTeov5kA23jc35kBgag9PF+C/dokvFDcEUS7SPzvL1r5SZFhFSN7Afq5d5OmhVhepYdm7iU7YAglcummWag7X+jFpWNc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272332; c=relaxed/simple; bh=iGY7XRQ1s4xARR/pMbr9fB7mn/vLdaqW1moTfoSqxIw=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uVY6nxd/X5rHZ0sRW1sZr6Ed0wDWYqwa/1W5kNnkvnKmo5IDp0nzOy1OtSLv1B6EzHJlZQ8bjc9tb5lQ462faVgDaWh/lq0YQ5CcVETX9b3MjkAqLSn/8E5w84WJmBjm6+Lo7qPMStRDry9BXQ/WZWy7lO3ikUFpUXC28K0axMo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VQA7eCXq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VQA7eCXq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B801C433F1; Wed, 7 Feb 2024 02:18:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272331; bh=iGY7XRQ1s4xARR/pMbr9fB7mn/vLdaqW1moTfoSqxIw=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=VQA7eCXqk9PDQj4YfBCV2k66D+pCw2rBy+sgYivORQJ/CHv298xwsMPakI/8ef0Wa GJP48LSPiYDSZtHMDJn8Tiuu45R4YkhblE8+GTTNZ18feQ094tBV04CqkG7azDTwq4 QBkRq17V62Yvse4CUTxK27pGoTgRJi/+FhrjzTGJMfpGLO2gLE8R3Ca4o1w/Mc7u1/ CVhLnJUoICB9C70OZZINqJnCjxDKFye2pBiZtHjpKWbtop1qNmMv0kowD1volE4UAS TG3X4m47+r048E/cl20ksKmDfnWtF8GSA4FZgbZ6ZmO4QUTiDuFOcsS/dn9GdcGMQH pdRoPgKjRNUUA== Subject: [PATCH 03/10] common/populate: always metadump full metadata blocks From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:18:51 -0800 Message-ID: <170727233097.3726171.8150749815178513838.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Commit e443cadcea pushed the -a and -o options to the _scratch_xfs_metadump callsites. Unfortunately, it missed the _xfs_metadump callsite in common/populate, so fix that now. Fixes: e443cadcea ("common/xfs: Do not append -a and -o options to metadump") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- common/populate | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/common/populate b/common/populate index cfbfd88a7f..33f2db8d4a 100644 --- a/common/populate +++ b/common/populate @@ -1029,7 +1029,7 @@ _scratch_populate_cached() { logdev=$SCRATCH_LOGDEV _xfs_metadump "$POPULATE_METADUMP" "$SCRATCH_DEV" "$logdev" \ - compress + compress -a -o ;; "ext2"|"ext3"|"ext4") _scratch_ext4_populate $@ From patchwork Wed Feb 7 02:18:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548038 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE722DF58; Wed, 7 Feb 2024 02:18:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272337; cv=none; b=L1dbAgvIaEuBjbp6wImoHLpPBf559pK5xXAllIII9O/kHTlaEXu3bCbLV8ikb7p3OS5gytwV9nOD3geZ37JsNQY44SgCag1+OyDQEsGigaUCK9W7jy3YOLD6CSA7H2XdLaAUUdGmsF/UIjWdZEqYhql4mY3kJDbtTcvFPAF+dKA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272337; c=relaxed/simple; bh=1IOiy2rQ0ddq1L/rGnXUtiLoMFwslmoiw31g8QGGKsk=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rylpMJejnNfAdIPKnvEWbyhxmfkdd3c6UzjTq8F6ESZtVfBsu/tKHOFuDpMpdH5Ilezs+SvZswdJ9dCx+lGh6NEbJsVohGMUCgDTnb5s19lcFXrLcQJZc0jm/SBA4MJZrNUvn3uGIuOzudjfLKAEmrt4F/q7KyrY1pURrKy/04Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qyKjFh7K; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qyKjFh7K" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42E25C433F1; Wed, 7 Feb 2024 02:18:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272337; bh=1IOiy2rQ0ddq1L/rGnXUtiLoMFwslmoiw31g8QGGKsk=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=qyKjFh7Ksm3TCyMRPppinkwkafyLMh47aUiK+JQ4Uy4wdPoxTGGWLGX0vZq/leHbO fU8ySJYpoawh/W+uqCU+S1iiCOqL3w4ZCZj7bRYaGo84zkB7IiaGxfhFwvAD4/+g7C yUwsZowawlQEu5hXVrjFAyz1rst5frHWjF2uw/Au4hRF4MeifAekbSgwh8Z/xa8iNb eUqkyizqJbsTBrFUnWMf1gjTmPSBPids4oSIPSMAvIh5HjGQmWVRCb9MfT4QWASw+J /tLvrFF5jGANcjnC9JeoaARQ9GTIWXBqlUpdPY/hqw91mjhSXtbXUwAI844+t/g+IX tM+Iz2PQf7R8Q== Subject: [PATCH 04/10] xfs/336: fix omitted -a and -o in metadump call From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:18:56 -0800 Message-ID: <170727233668.3726171.7285378887952888883.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Commit e443cadcea reworked _xfs_metadump to require that all callers always pass the arguments they want -- no more defaulting to "-a -o". Unfortunately, a few got missed. Fix some of them now; the rest will get cleaned up in the next patch. Fixes: e443cadcea ("common/xfs: Do not append -a and -o options to metadump") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- tests/xfs/284 | 4 ++-- tests/xfs/336 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/xfs/284 b/tests/xfs/284 index 58f330035e..443c375759 100755 --- a/tests/xfs/284 +++ b/tests/xfs/284 @@ -42,12 +42,12 @@ COPY_FILE="${TEST_DIR}/${seq}_copyfile" # xfs_metadump should refuse to dump a mounted device _scratch_mkfs >> $seqres.full 2>&1 _scratch_mount -_scratch_xfs_metadump $METADUMP_FILE 2>&1 | filter_mounted +_scratch_xfs_metadump $METADUMP_FILE -a -o 2>&1 | filter_mounted _scratch_unmount # Test restore to a mounted device # xfs_mdrestore should refuse to restore to a mounted device -_scratch_xfs_metadump $METADUMP_FILE +_scratch_xfs_metadump $METADUMP_FILE -a -o _scratch_mount _scratch_xfs_mdrestore $METADUMP_FILE 2>&1 | filter_mounted _scratch_unmount diff --git a/tests/xfs/336 b/tests/xfs/336 index 43b3790cbb..3c30f1a40b 100755 --- a/tests/xfs/336 +++ b/tests/xfs/336 @@ -62,7 +62,7 @@ _scratch_cycle_mount echo "Create metadump file" _scratch_unmount -_scratch_xfs_metadump $metadump_file -a +_scratch_xfs_metadump $metadump_file -a -o # Now restore the obfuscated one back and take a look around echo "Restore metadump" From patchwork Wed Feb 7 02:19:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548039 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AE79EEDA; Wed, 7 Feb 2024 02:19:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272343; cv=none; b=Y3cfE/eiUBF5cSUiUQBrcqZ7BkMvFKMY7f2X/kYLl87iMQXT/s8QV8DR2K6RPMV+Nks2exgVbZRZYhxFr3XKvbdconXhIxqVJRTSWgOLWBzO7WS/UwtjGVfLH1+7cSv4ELVET2weocgg/FgNALwbrEc4BXXPMvllBi8ifzH8KgE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272343; c=relaxed/simple; bh=y1EObu8pcl4d3kDj3a+brhLV/+h+Y/fU/fKXSPrhB+A=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BKnf05WhJMQt1CWhR+74UC3r1ws65b1BBGH8QEjo3X3E427whsNb6V29ZHC0FldLMKbM2iXqEzdhAg6IXV54uUxUjY3AlVYZCCgYLFojK38ZN4GEWu4HQBS2q+fVbUYGWUuE/Sipk7bLJx9DDoz4zXoZoBbJS7UlmU8C6XYyjJA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CXewGE6H; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CXewGE6H" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E290BC433C7; Wed, 7 Feb 2024 02:19:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272343; bh=y1EObu8pcl4d3kDj3a+brhLV/+h+Y/fU/fKXSPrhB+A=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=CXewGE6HfM1B2zXs6bcOJL5FJVVDYEZdNrksQhmLq/M9XugNeM7Ek+ftIZD65Dko5 rKSybNDjTkA2avlKhtYwodCdoQ4sk1ZHAFe66cDk9v4g7KVU0K6ug8HjSPMKO4LCUX IrAsjKrUKsuaeXxGgdOjI0V2HGPBWVfvvYVjv5+A+Js1kXOqdcwKBWyj/w8sGPfENy bTgDySgiFscAJP+SiI8rcP/rjS/BewMBw8nmnpEs67wLyqMNDf24zbzPt8HMKUOsnH 0HQBoPcBMTRFr/8WHhZNJsELkK8RLEh8/P4fqLXRlUeq9fUzbMWa6x9bSta9TN2xzf t44YeXoUPgoGg== Subject: [PATCH 05/10] common: refactor metadump v1 and v2 tests, version 2 From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:19:02 -0800 Message-ID: <170727234241.3726171.5377809483090058891.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291, xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly duplicate copies of the same code. While we're at it, fix the fsck so that it includes xfs_scrub. [v2] After the first version of this patch was committed to fstests for-next, Zorro reported that the cleanup function in common/xfs_metadump_tests zapped one of his test machines because of a well known shell variable expansion + globbing footgun. This can trigger when running fstests on older configurations where a test adds _cleanup_verify_metadump to the local _cleanup function but exits before calling _setup_verify_metadump to set XFS_METADUMP_IMG to a non-empty value. Redesign the cleanup function to check for non-empty values of XFS_METADUMP_{FILE,IMG} before proceeding with the rm. Change the globbed parameter of "rm -f $XFS_METADUMP_IMG*" to a for loop so that if the glob does not match any files, the loop variable will be set to a path that does not resolve anywhere. The for-next branch was reverted to v2024.01.14, hence this patch is being resubmitted with the fix inline instead of as a separate fix patch. Longer term maybe we ought to set -u or something. Or figure out how to make the root directory readonly. Reported-by: Zorro Lang Link: https://lore.kernel.org/fstests/20240205060016.7fgiyafbnrvf5chj@dell-per750-06-vm-08.rhts.eng.pek2.redhat.com/ Signed-off-by: "Darrick J. Wong" --- common/metadump | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ common/rc | 10 ---- common/xfs | 14 ++++++ tests/xfs/129 | 90 ++---------------------------------- tests/xfs/234 | 91 ++----------------------------------- tests/xfs/253 | 89 ++---------------------------------- tests/xfs/291 | 31 ++++-------- tests/xfs/432 | 30 ++---------- tests/xfs/503 | 60 +++--------------------- tests/xfs/605 | 84 ++-------------------------------- 10 files changed, 195 insertions(+), 441 deletions(-) create mode 100644 common/metadump diff --git a/common/metadump b/common/metadump new file mode 100644 index 0000000000..4b576f045e --- /dev/null +++ b/common/metadump @@ -0,0 +1,137 @@ +##/bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2024 Oracle. All Rights Reserved. +# +# Filesystem metadata dump testing functions. +# + +# Set up environment variables for a metadump test. Requires the test and +# scratch devices. Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION. +_xfs_setup_verify_metadump() +{ + XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump" + XFS_METADUMP_IMG="$TEST_DIR/${seq}_image" + MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)" + + rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"* +} + +_xfs_cleanup_verify_metadump() +{ + local img + + _scratch_unmount &>> $seqres.full + + test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" + + if [ -n "$XFS_METADUMP_IMG" ]; then + losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do + losetup -d "$ldev" + done + + # Don't call rm directly with a globbed argument here to avoid + # issues issues with variable expansions. + for img in "$XFS_METADUMP_IMG"*; do + test -e "$img" && rm -f "$img" + done + fi +} + +# Create a metadump in v1 format, restore it to fs image files, then mount the +# images and fsck them. +_xfs_verify_metadump_v1() +{ + local metadump_args="$1" + local extra_test="$2" + + local metadump_file="$XFS_METADUMP_FILE" + local version="" + local data_img="$XFS_METADUMP_IMG.data" + local data_loop + + # Force v1 if we detect v2 support + if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then + version="-v 1" + fi + + # Capture metadump, which creates metadump_file + _scratch_xfs_metadump $metadump_file $metadump_args $version + + # Restore metadump, which creates data_img + SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file + + # Create loopdev for data device so we can mount the fs + data_loop=$(_create_loop_device $data_img) + + # Mount fs, run an extra test, fsck, and unmount + SCRATCH_DEV=$data_loop _scratch_mount + if [ -n "$extra_test" ]; then + SCRATCH_DEV=$data_loop $extra_test + fi + SCRATCH_DEV=$data_loop _check_xfs_scratch_fs + SCRATCH_DEV=$data_loop _scratch_unmount + + # Tear down what we created + _destroy_loop_device $data_loop + rm -f $data_img +} + +# Create a metadump in v2 format, restore it to fs image files, then mount the +# images and fsck them. +_xfs_verify_metadump_v2() +{ + local metadump_args="$1" + local extra_test="$2" + + local metadump_file="$XFS_METADUMP_FILE" + local version="-v 2" + local data_img="$XFS_METADUMP_IMG.data" + local data_loop + local log_img="" + local log_loop + + # Capture metadump, which creates metadump_file + _scratch_xfs_metadump $metadump_file $metadump_args $version + + # + # Metadump v2 files can contain contents dumped from an external log + # device. Use a temporary file to hold the log device contents restored + # from such a metadump file. + test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log" + + # Restore metadump, which creates data_img and log_img + SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \ + _scratch_xfs_mdrestore $metadump_file + + # Create loopdev for data device so we can mount the fs + data_loop=$(_create_loop_device $data_img) + + # Create loopdev for log device if we recovered anything + test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + + # Mount fs, run an extra test, fsck, and unmount + SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + if [ -n "$extra_test" ]; then + SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + fi + SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs + SCRATCH_DEV=$data_loop _scratch_unmount + + # Tear down what we created + if [ -b "$log_loop" ]; then + _destroy_loop_device $log_loop + rm -f $log_img + fi + _destroy_loop_device $data_loop + rm -f $data_img +} + +# Verify both metadump formats if possible +_xfs_verify_metadumps() +{ + _xfs_verify_metadump_v1 "$@" + + if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then + _xfs_verify_metadump_v2 "$@" + fi +} diff --git a/common/rc b/common/rc index 524ffa02aa..0b69f7f54f 100644 --- a/common/rc +++ b/common/rc @@ -3320,15 +3320,7 @@ _check_scratch_fs() case $FSTYP in xfs) - local scratch_log="none" - local scratch_rt="none" - [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \ - scratch_log="$SCRATCH_LOGDEV" - - [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \ - scratch_rt="$SCRATCH_RTDEV" - - _check_xfs_filesystem $device $scratch_log $scratch_rt + _check_xfs_scratch_fs $device ;; udf) _check_udf_filesystem $device $udf_fsize diff --git a/common/xfs b/common/xfs index 248ccefda3..6a48960a7f 100644 --- a/common/xfs +++ b/common/xfs @@ -1035,6 +1035,20 @@ _check_xfs_test_fs() return $? } +_check_xfs_scratch_fs() +{ + local device="${1:-$SCRATCH_DEV}" + local scratch_log="none" + local scratch_rt="none" + [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \ + scratch_log="$SCRATCH_LOGDEV" + + [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \ + scratch_rt="$SCRATCH_RTDEV" + + _check_xfs_filesystem $device $scratch_log $scratch_rt +} + # modeled after _scratch_xfs_repair _test_xfs_repair() { diff --git a/tests/xfs/129 b/tests/xfs/129 index cdac2349df..ec1a2ff3da 100755 --- a/tests/xfs/129 +++ b/tests/xfs/129 @@ -16,98 +16,23 @@ _cleanup() { cd / _scratch_unmount > /dev/null 2>&1 - [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev - rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \ - $TEST_DIR/log-image + _xfs_cleanup_verify_metadump + rm -rf $tmp.* $testdir } # Import common functions. . ./common/filter . ./common/reflink +. ./common/metadump # real QA test starts here _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_loop _require_scratch_reflink - -metadump_file=$TEST_DIR/${seq}_metadump - -verify_metadump_v1() -{ - local max_version=$1 - local version="" - - if [[ $max_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $metadump_file $version - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - SCRATCH_DEV=$datadev _scratch_unmount - - logdev=$SCRATCH_LOGDEV - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - version="-v 2" - - _scratch_xfs_metadump $metadump_file $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev=${SCRATCH_LOGDEV} - if [[ -s $TEST_DIR/log-image ]]; then - logdev=$(_create_loop_device $TEST_DIR/log-image) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - if [[ -s $TEST_DIR/log-image ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $TEST_DIR/log-image - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} +_xfs_setup_verify_metadump _scratch_mkfs >/dev/null 2>&1 - -max_md_version=$(_xfs_metadump_max_version) - _scratch_mount testdir=$SCRATCH_MNT/test-$seq @@ -127,12 +52,7 @@ done _scratch_unmount echo "Create metadump file, restore it and check restored fs" - -verify_metadump_v1 $max_md_version - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_xfs_verify_metadumps # success, all done status=0 diff --git a/tests/xfs/234 b/tests/xfs/234 index f4f8af6d3a..6fdea42d21 100755 --- a/tests/xfs/234 +++ b/tests/xfs/234 @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump _cleanup() { cd / - _scratch_unmount > /dev/null 2>&1 - [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev - rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \ - $TEST_DIR/log-image + _xfs_cleanup_verify_metadump + rm -rf $tmp.* $testdir } # Import common functions. . ./common/filter +. ./common/metadump # real QA test starts here _supported_fs xfs @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_loop _require_xfs_scratch_rmapbt _require_xfs_io_command "fpunch" - -metadump_file=$TEST_DIR/${seq}_metadump - -verify_metadump_v1() -{ - local max_version=$1 - local version="" - - if [[ $max_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $metadump_file $version - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - SCRATCH_DEV=$datadev _scratch_unmount - - logdev=$SCRATCH_LOGDEV - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - version="-v 2" - - _scratch_xfs_metadump $metadump_file $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev=${SCRATCH_LOGDEV} - if [[ -s $TEST_DIR/log-image ]]; then - logdev=$(_create_loop_device $TEST_DIR/log-image) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - if [[ -s $TEST_DIR/log-image ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $TEST_DIR/log-image - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} +_xfs_setup_verify_metadump _scratch_mkfs >/dev/null 2>&1 - -max_md_version=$(_xfs_metadump_max_version) - _scratch_mount testdir=$SCRATCH_MNT/test-$seq @@ -127,12 +51,7 @@ done _scratch_unmount echo "Create metadump file, restore it and check restored fs" - -verify_metadump_v1 $max_md_version - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_xfs_verify_metadumps # success, all done status=0 diff --git a/tests/xfs/253 b/tests/xfs/253 index 3b567999d8..18c58eb8d5 100755 --- a/tests/xfs/253 +++ b/tests/xfs/253 @@ -26,23 +26,21 @@ _cleanup() cd / rm -f $tmp.* rm -rf "${OUTPUT_DIR}" - rm -f "${METADUMP_FILE}" - [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev + _xfs_cleanup_verify_metadump } # Import common functions. . ./common/filter +. ./common/metadump _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_test _require_scratch +_xfs_setup_verify_metadump # real QA test starts here OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}" -METADUMP_FILE="${TEST_DIR}/${seq}_metadump" ORPHANAGE="lost+found" _supported_fs xfs @@ -52,24 +50,7 @@ function create_file() { touch $(printf "$@") } -verify_metadump_v1() -{ - local max_version=$1 - local version="" - - if [[ $max_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $METADUMP_FILE $version - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \ - _scratch_xfs_mdrestore $METADUMP_FILE - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - +extra_test() { cd "${SCRATCH_MNT}" # Get a listing of all the files after obfuscation @@ -78,60 +59,6 @@ verify_metadump_v1() ls -R | od -c >> $seqres.full cd / - - SCRATCH_DEV=$datadev _scratch_unmount - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - version="-v 2" - - _scratch_xfs_metadump $METADUMP_FILE $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $METADUMP_FILE - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev=${SCRATCH_LOGDEV} - if [[ -s $TEST_DIR/log-image ]]; then - logdev=$(_create_loop_device $TEST_DIR/log-image) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - - cd "${SCRATCH_MNT}" - - # Get a listing of all the files after obfuscation - echo "Metadump v2" >> $seqres.full - ls -R >> $seqres.full - ls -R | od -c >> $seqres.full - - cd / - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - if [[ -s $TEST_DIR/log-image ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $TEST_DIR/log-image - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image } echo "Disciplyne of silence is goed." @@ -233,13 +160,7 @@ cd $here _scratch_unmount -max_md_version=$(_xfs_metadump_max_version) - -verify_metadump_v1 $max_md_version - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_xfs_verify_metadumps '' extra_test # Finally, re-make the filesystem since to ensure we don't # leave a directory with duplicate entries lying around. diff --git a/tests/xfs/291 b/tests/xfs/291 index 1433140821..2bd94d7b9b 100755 --- a/tests/xfs/291 +++ b/tests/xfs/291 @@ -9,11 +9,21 @@ . ./common/preamble _begin_fstest auto repair metadump +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* + _xfs_cleanup_verify_metadump +} + # Import common functions. . ./common/filter +. ./common/metadump _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" +_xfs_setup_verify_metadump # real QA test starts here _require_scratch @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed" # Yes they can! Now... # Can xfs_metadump cope with this monster? -max_md_version=$(_xfs_metadump_max_version) - -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $tmp.metadump -a -o $version || \ - _fail "xfs_metadump failed" - - slogdev=$SCRATCH_LOGDEV - if [[ -z $version || $version == "-v 1" ]]; then - slogdev="" - fi - SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \ - $tmp.metadump || _fail "xfs_mdrestore failed" - SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \ - _fail "xfs_repair of metadump failed" -done +_xfs_verify_metadumps '-a -o' # Yes it can; success, all done status=0 diff --git a/tests/xfs/432 b/tests/xfs/432 index 7e402aa88f..4eae92e75b 100755 --- a/tests/xfs/432 +++ b/tests/xfs/432 @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump _cleanup() { cd / - rm -f "$tmp".* $metadump_file $metadump_img + rm -f "$tmp".* + _xfs_cleanup_verify_metadump } # Import common functions. . ./common/filter +. ./common/metadump # real QA test starts here _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_scratch +_xfs_setup_verify_metadump rm -f "$seqres.full" @@ -54,9 +57,6 @@ echo "Format and mount" _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1 _scratch_mount >> "$seqres.full" 2>&1 -metadump_file="$TEST_DIR/meta-$seq" -metadump_img="$TEST_DIR/img-$seq" -rm -f $metadump_file $metadump_img testdir="$SCRATCH_MNT/test-$seq" max_fname_len=255 blksz=$(_get_block_size $SCRATCH_MNT) @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks" echo "Try to metadump, restore and check restored metadump image" -max_md_version=$(_xfs_metadump_max_version) - -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a -o -w $version - - slogdev=$SCRATCH_LOGDEV - if [[ -z $version || $version == "-v 1" ]]; then - slogdev="" - fi - - SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \ - echo "xfs_repair on restored fs returned $?" -done +_xfs_verify_metadumps '-a -o -w' # success, all done status=0 diff --git a/tests/xfs/503 b/tests/xfs/503 index 8643c3d483..854cc74bbe 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -17,11 +17,13 @@ _cleanup() { cd / rm -rf $tmp.* $testdir + _xfs_cleanup_verify_metadump } # Import common functions. . ./common/filter . ./common/populate +. ./common/metadump testdir=$TEST_DIR/test-$seq @@ -35,6 +37,7 @@ _require_scratch_nocheck _require_populate_commands _xfs_skip_online_rebuild _xfs_skip_offline_rebuild +_xfs_setup_verify_metadump echo "Format and populate" _scratch_populate_cached nofill > $seqres.full 2>&1 @@ -43,66 +46,17 @@ mkdir -p $testdir metadump_file=$testdir/scratch.md copy_file=$testdir/copy.img -check_restored_metadump_image() -{ - local image=$1 - - loop_dev=$(_create_loop_device $image) - SCRATCH_DEV=$loop_dev _scratch_mount - SCRATCH_DEV=$loop_dev _check_scratch_fs - SCRATCH_DEV=$loop_dev _scratch_unmount - _destroy_loop_device $loop_dev -} - -max_md_version=$(_xfs_metadump_max_version) - echo "metadump and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_xfs_verify_metadumps '-a -o' echo "metadump a and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_xfs_verify_metadumps '-a' echo "metadump g and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -g $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_xfs_verify_metadumps '-g' >> $seqres.full echo "metadump ag and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_xfs_verify_metadumps '-a -g' >> $seqres.full echo copy $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full diff --git a/tests/xfs/605 b/tests/xfs/605 index f2cd7aba98..13cf065495 100755 --- a/tests/xfs/605 +++ b/tests/xfs/605 @@ -15,17 +15,13 @@ _cleanup() { cd / rm -r -f $tmp.* - _scratch_unmount > /dev/null 2>&1 - [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev - rm -r -f $metadump_file $TEST_DIR/data-image \ - $TEST_DIR/log-image + _xfs_cleanup_verify_metadump } # Import common functions. . ./common/dmflakey . ./common/inject +. ./common/metadump # real QA test starts here _supported_fs xfs @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin _require_dm_target flakey _require_xfs_io_command "pwrite" _require_test_program "punch-alternating" +_xfs_setup_verify_metadump -metadump_file=${TEST_DIR}/${seq}.md testfile=${SCRATCH_MNT}/testfile echo "Format filesystem on scratch device" _scratch_mkfs >> $seqres.full 2>&1 -max_md_version=$(_xfs_metadump_max_version) - external_log=0 if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then external_log=1 fi -if [[ $max_md_version == 1 && $external_log == 1 ]]; then +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then _notrun "metadump v1 does not support external log device" fi -verify_metadump_v1() -{ - local version="" - if [[ $max_md_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $metadump_file -a -o $version - - SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - SCRATCH_DEV=$datadev _check_scratch_fs - SCRATCH_DEV=$datadev _scratch_unmount - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - local version="-v 2" - - _scratch_xfs_metadump $metadump_file -a -o $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev="" - if [[ -s $slogdev ]]; then - logdev=$(_create_loop_device $slogdev) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - if [[ -s $logdev ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $slogdev - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - echo "Initialize and mount filesystem on flakey device" _init_flakey _load_flakey_table $FLAKEY_ALLOW_WRITES @@ -160,14 +93,7 @@ echo -n "Filesystem has a " _print_logstate echo "Create metadump file, restore it and check restored fs" - -if [[ $external_log == 0 ]]; then - verify_metadump_v1 $max_md_version -fi - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_xfs_verify_metadumps '-a -o' # Mount the fs to replay the contents from the dirty log. _scratch_mount From patchwork Wed Feb 7 02:19:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548040 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CD5FF9C3; Wed, 7 Feb 2024 02:19:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272349; cv=none; b=li12hQXxGt5MJUJf57paU+AuL6EKZE2nzKHm6/i7mORLMvP6w/X8VuavI5ImI/LcB0V/94CmNknXKK+8oH3fIo1K9ZBkV15twiCK53SI/EYOqE2QLT4r/m39M90kXbiwDWUAWyPT0BF0zOkyTxyo2vdXec7D3h5e8Zvc2FBne7A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272349; c=relaxed/simple; bh=lyqMSNXvolgxaaMKuPV1joOmxLSzc6JKNIqxchBCgM4=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iSkZ5S5lpip0Xi99nrZ2o0B4HhfihA7Ki5GOazlt8+YBfr/qzMx0NxVegPDo6D+XFr2VIMriSVgjeUcHFz4ngL8YhgRjYSi+sfmZXN4VglNWjL3Kehd6pOa84kTMlFSShLi7Qbf8oWhagm3XlKnlRcx3GJo1lEsZQBs3FG98TGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ujx7QAFk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ujx7QAFk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9A6ECC433F1; Wed, 7 Feb 2024 02:19:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272348; bh=lyqMSNXvolgxaaMKuPV1joOmxLSzc6JKNIqxchBCgM4=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ujx7QAFk+QuNlHamNhGZFwE/WzPWWIVWpKz54QRi3s85poMIzRPR3fIb9sM8xe+fV XyALonYhhGKCKrLzt3d/mM7XXu2JgBe+Rjv3Fp8aID0x5IRs/flEjwC82TTZ/n7U8C ryMYF4Df2R63mHKUC8RsZ4dC+7rznlSDaKhthucZXri4l5+rC58XFl1AAQCY/KApY8 jk9AEzeOaREujgPtrdoiKIjnz6OIcMSt/Cc4CXtDw4CmXiPVaBW0/NN/H1HlSY0bI2 orfVCghSlSHVcRuiVTH0ColKPZZYnOPqhhp27yktzPapYEKA5lxyP1KpkPYH+a0IxA aIdUWdM9i2xFg== Subject: [PATCH 06/10] xfs/{129,234,253,605}: disable metadump v1 testing with external devices From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:19:08 -0800 Message-ID: <170727234812.3726171.2841992060887445453.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The metadump v1 format does not support capturing content from log devices or realtime devices. Hence it does not make sense to test these scenarios. Create predicates to decide if we want to test a particular metadump format, then convert existing tests to check formats explicitly. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- common/metadump | 25 ++++++++++++++++++++----- tests/xfs/605 | 9 --------- 2 files changed, 20 insertions(+), 14 deletions(-) diff --git a/common/metadump b/common/metadump index 4b576f045e..3373edfe92 100644 --- a/common/metadump +++ b/common/metadump @@ -37,6 +37,24 @@ _xfs_cleanup_verify_metadump() fi } +# Can xfs_metadump snapshot the fs metadata to a v1 metadump file? +_scratch_xfs_can_metadump_v1() +{ + # metadump v1 does not support log devices + [ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_LOGDEV" ] && return 1 + + # metadump v1 does not support realtime devices + [ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_RTDEV" ] && return 1 + + return 0 +} + +# Can xfs_metadump snapshot the fs metadata to a v2 metadump file? +_scratch_xfs_can_metadump_v2() +{ + test "$MAX_XFS_METADUMP_VERSION" -ge 2 +} + # Create a metadump in v1 format, restore it to fs image files, then mount the # images and fsck them. _xfs_verify_metadump_v1() @@ -129,9 +147,6 @@ _xfs_verify_metadump_v2() # Verify both metadump formats if possible _xfs_verify_metadumps() { - _xfs_verify_metadump_v1 "$@" - - if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then - _xfs_verify_metadump_v2 "$@" - fi + _scratch_xfs_can_metadump_v1 && _xfs_verify_metadump_v1 "$@" + _scratch_xfs_can_metadump_v2 && _xfs_verify_metadump_v2 "$@" } diff --git a/tests/xfs/605 b/tests/xfs/605 index 13cf065495..78458c7665 100755 --- a/tests/xfs/605 +++ b/tests/xfs/605 @@ -40,15 +40,6 @@ testfile=${SCRATCH_MNT}/testfile echo "Format filesystem on scratch device" _scratch_mkfs >> $seqres.full 2>&1 -external_log=0 -if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then - external_log=1 -fi - -if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then - _notrun "metadump v1 does not support external log device" -fi - echo "Initialize and mount filesystem on flakey device" _init_flakey _load_flakey_table $FLAKEY_ALLOW_WRITES From patchwork Wed Feb 7 02:19:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548041 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD7FBFBE9; Wed, 7 Feb 2024 02:19:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272354; cv=none; b=qJ80vSIdT5wi4JQfbWy+O+hhypvYwHFzf7sa0StAdr4pxWLK+i7+n87IHMPHTkiYvvAKGgGkixGTCUik2WcOawX9L52VIIp5QFp9Dw4aXjG98wCJqRH9l+Nh7Mp2BA4MZAER76QAglkrihSPImdf7c8tKKfvFvS0R87NNoeOYtk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272354; c=relaxed/simple; bh=hmfvZMG8kVPbA5v7sUSUoQ3wTAvS9cmUq87RVw+eUJA=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WI/XccGV/JfXGpDgLw35wPVHFh3cdxEs38CF0EZA9Jjwrs6LAmQRdf5IYVyO99WWR0gmOM9U4shYRxFp0kjPnwrfr0pHO6HNn7gygCJzHh9rS+kVMXRDkFHSN13zXLQIwZeFSaNC6cKIdVuc0h/dCNLrW1rTRVHbT4rnfkJR8lY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iP5gq5pP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iP5gq5pP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3FB6FC433C7; Wed, 7 Feb 2024 02:19:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272354; bh=hmfvZMG8kVPbA5v7sUSUoQ3wTAvS9cmUq87RVw+eUJA=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=iP5gq5pPlwFSbkf8XKUb3LmK18jw0us1IWuc3G3qJIw5y1UbFd4h7IDuCAkVWbMRi kOcwHhGcTuGgmFRSRMAibQCZYD2xE30Kk1oggXQg25Kw/7WkX52og4zFdOasXSBCgv aNyD8PigB/auy0GhPe8lK2BHbKXqsQ+RD1yUHaguDSINuofoq9QiV8IwU5BuZVADlb 9d8LcmSSJaiCRHdHMAUN3ENCaeKurKGgXleyjq97UATaYLCEkCp7hSQuWFAIkBYch2 jTNfPiiBLAHRx1k5RrAvUL8A7mEdwubMIs9oqK3s+oY+YrxpnDI7ROVHfXl1RJg0ZE 9eAprhOo6DOaw== Subject: [PATCH 07/10] xfs/503: test metadump obfuscation, not progressbars From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:19:13 -0800 Message-ID: <170727235379.3726171.14857033108645144507.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The -g switch to xfs_metadump turns on progress reporting, but nothing in this test actually checks that it works. The -o switch turns off obfuscation, which is much more critical to support teams. Change this test to check -o and -ao instead of -g or -ag. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- tests/xfs/503 | 10 +++++----- tests/xfs/503.out | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/xfs/503 b/tests/xfs/503 index 854cc74bbe..07d243bc06 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -47,16 +47,16 @@ metadump_file=$testdir/scratch.md copy_file=$testdir/copy.img echo "metadump and mdrestore" -_xfs_verify_metadumps '-a -o' +_xfs_verify_metadumps echo "metadump a and mdrestore" _xfs_verify_metadumps '-a' -echo "metadump g and mdrestore" -_xfs_verify_metadumps '-g' >> $seqres.full +echo "metadump o and mdrestore" +_xfs_verify_metadumps '-o' -echo "metadump ag and mdrestore" -_xfs_verify_metadumps '-a -g' >> $seqres.full +echo "metadump ao and mdrestore" +_xfs_verify_metadumps '-a -o' echo copy $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full diff --git a/tests/xfs/503.out b/tests/xfs/503.out index 496f2516e4..5e7488456d 100644 --- a/tests/xfs/503.out +++ b/tests/xfs/503.out @@ -2,7 +2,7 @@ QA output created by 503 Format and populate metadump and mdrestore metadump a and mdrestore -metadump g and mdrestore -metadump ag and mdrestore +metadump o and mdrestore +metadump ao and mdrestore copy recopy From patchwork Wed Feb 7 02:19:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548042 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61138FBEB; Wed, 7 Feb 2024 02:19:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272360; cv=none; b=VR0yG8UCZ5VFcK2qlSA4upWjDHVSRc0mU1pr5dvpvCl90nykhDDbs29OXGkfbNxYLrIykkggTTXJbKMpguYtAAhAzmhzXe5kHuohCEuSLTPxfx/oDnYytKunc5CqFlpz+GFAFJvxScuSxztQImUp3GRs4E9VgKuRLEtGcPDRDvw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272360; c=relaxed/simple; bh=Qmkuef+jv6iUjSWKplTunZ9HuLxmGKijMrmARJaVADE=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=es5yXz7GhawCjCxTsq77tQa5FIvl3qRwC4lAIfq/Rev4FN6RPuX9upi284KPCh9ma5vSMwymhNycBpxR8/Xednwvj8/qn5P2XEXBSjbtSmtWdv+ynCNqKE/Mkx8/yvTpKgNoBoO/9nQuVNoCvHlfvm7knHXZXydYv8CpcSwK6o8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UAjA+R6f; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UAjA+R6f" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CF479C433F1; Wed, 7 Feb 2024 02:19:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272359; bh=Qmkuef+jv6iUjSWKplTunZ9HuLxmGKijMrmARJaVADE=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=UAjA+R6fJoqroDB8G75A05o7fXv3IpZru6US/hCUNLsdW+NYtOAH07onkkRGbucF7 kxf6YGDU5D++tkON5I+neV4w3KgNWXJFrhGC5rVAgZPeTmF6jrma4yckhniP5HJ43V OtRO+KPbZ8Qv+COAxY90UJXpU0LhJeu1qtAnu///lGP0TRgyLMVcWu6DpaSybInHne Px9kF1baVDEyTNKFTmbQ2blike6REreRhlbxxoZGQxm+M6efHsT1cnVajV3i4ErC7O bWHGe4ne060UEQwmvftiCxP5CQDuLGLnjdHBfPsOYB68KRdp9LdzixGRVrlDnVg1wR 19Qm/2Ash5ARw== Subject: [PATCH 08/10] xfs/503: split copy and metadump into two tests From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:19:19 -0800 Message-ID: <170727235942.3726171.4745513175828645904.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong This test examines the behavior of xfs_copy and xfs_metadump. Metadump now supports capturing external log contents, but copy does not. Split the test into two to improve coverage on multidevice filesystems. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- tests/xfs/503 | 17 +++-------------- tests/xfs/503.out | 2 -- tests/xfs/601 | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/601.out | 4 ++++ 4 files changed, 61 insertions(+), 16 deletions(-) create mode 100755 tests/xfs/601 create mode 100755 tests/xfs/601.out diff --git a/tests/xfs/503 b/tests/xfs/503 index 07d243bc06..01cff7b08d 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -4,11 +4,11 @@ # # FS QA Test No. 503 # -# Populate a XFS filesystem and ensure that metadump, mdrestore, and copy -# all work properly. +# Populate a XFS filesystem and ensure that metadump and mdrestore all work +# properly. # . ./common/preamble -_begin_fstest auto copy metadump +_begin_fstest auto metadump _register_cleanup "_cleanup" BUS @@ -32,7 +32,6 @@ _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_loop -_require_xfs_copy _require_scratch_nocheck _require_populate_commands _xfs_skip_online_rebuild @@ -58,16 +57,6 @@ _xfs_verify_metadumps '-o' echo "metadump ao and mdrestore" _xfs_verify_metadumps '-a -o' -echo copy -$XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full -_check_scratch_fs $copy_file - -echo recopy -$XFS_COPY_PROG $copy_file $SCRATCH_DEV >> $seqres.full -_scratch_mount -_check_scratch_fs -_scratch_unmount - # success, all done status=0 exit diff --git a/tests/xfs/503.out b/tests/xfs/503.out index 5e7488456d..7f3d3a5f24 100644 --- a/tests/xfs/503.out +++ b/tests/xfs/503.out @@ -4,5 +4,3 @@ metadump and mdrestore metadump a and mdrestore metadump o and mdrestore metadump ao and mdrestore -copy -recopy diff --git a/tests/xfs/601 b/tests/xfs/601 new file mode 100755 index 0000000000..e1e94ca8ff --- /dev/null +++ b/tests/xfs/601 @@ -0,0 +1,54 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2019 Oracle, Inc. All Rights Reserved. +# +# FS QA Test No. 601 +# +# Populate a XFS filesystem and ensure that xfs_copy works properly. +# +. ./common/preamble +_begin_fstest auto copy + +_register_cleanup "_cleanup" BUS + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -rf $tmp.* $testdir +} + +# Import common functions. +. ./common/filter +. ./common/populate + +testdir=$TEST_DIR/test-$seq + +# real QA test starts here +_supported_fs xfs + +_require_xfs_copy +_require_scratch_nocheck +_require_populate_commands +_xfs_skip_online_rebuild +_xfs_skip_offline_rebuild + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +mkdir -p $testdir +copy_file=$testdir/copy.img + +echo copy +$XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full +_check_scratch_fs $copy_file + +echo recopy +$XFS_COPY_PROG $copy_file $SCRATCH_DEV >> $seqres.full +_scratch_mount +_check_scratch_fs +_scratch_unmount + +# success, all done +status=0 +exit diff --git a/tests/xfs/601.out b/tests/xfs/601.out new file mode 100755 index 0000000000..0cbe0c0d09 --- /dev/null +++ b/tests/xfs/601.out @@ -0,0 +1,4 @@ +QA output created by 601 +Format and populate +copy +recopy From patchwork Wed Feb 7 02:19:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548043 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC6A5FBE9; Wed, 7 Feb 2024 02:19:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272365; cv=none; b=l7NYoWMaH5fuGiZ1GOaQF1ZGxl2D/RgqkT8OV+9/LxnNYs34s5XFQGCaqYqugC3Kxg/ku/QnDW4HJ2DDB5Wuzhkt4XW38Wt9+c77rcQD1+xvi/2B5LajOiD8IKXjuMfFMMtTEAxkRyvA5ErnjrQfiJjQ1Y/EcYLAT5YOebKoRnQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272365; c=relaxed/simple; bh=JY0an+aV7JvYhVNzONkthuKt4vhY7UOlu6g94nyVTr0=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hGaz11I0FoIrVLup1TFHUEYwa8NURmnZbIFkSzrNshQTYdPE9QTV9Fhd5TSdhY4STCyWUp2VVxr8GQp6oMJNnzTHEKx7CnNplo0aRnzGVyXZujkXk8IlV/H01ihaCuLc6e5A6l3xYJ+yUWxScPclp/asOxEd+8uZMDFlxqxOA2E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ai1z8DXa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ai1z8DXa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 78908C43390; Wed, 7 Feb 2024 02:19:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272365; bh=JY0an+aV7JvYhVNzONkthuKt4vhY7UOlu6g94nyVTr0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ai1z8DXaZINJqPpyww4geaCZrMhkEnzCp8WIVvweIVVLdlELvXN+6eUCovseQGrhY 78UG5kiJQFEnA2xL4sFMonoFdeyacISA45slmvpMaLAdHx9VmQxqWPwMqeDEW0E0Vs CC9qO9E5aa3XsmRvxetaMOJmVOCfRmne8upDZ2prQHdeF4LfLxhjCIcyK0KwbTs8LG 7Xnh4JAzNoqQYI4N8GTPmaCE/FswFaQ75qiWZm0p4nkawyxKlNXdNlGhStoXkLKEJM OQ3lwYKdsBr/RfYhtxFN2qLUjkMQnLUm++JakuLSp/EKquhR9HAllkwVnVV3CfmYNB IPpJQsG2l7zKA== Subject: [PATCH 09/10] common/xfs: only pass -l in _xfs_mdrestore for v2 metadumps From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:19:25 -0800 Message-ID: <170727236500.3726171.13197353676937666826.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong fstests has a weird history with external log devices -- prior to the introduction of metadump v2, a dump/restore cycle would leave an external log unaltered, and most tests worked just fine. Were those tests ignorant? Or did they pass intentionally? Either way, we don't want to pass -l to xfs_mdrestore just because we have an external log, because that switch is new and causes regressions when testing with xfsprogs from before 6.5. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- common/xfs | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/common/xfs b/common/xfs index 6a48960a7f..65b509691b 100644 --- a/common/xfs +++ b/common/xfs @@ -689,12 +689,25 @@ _xfs_metadump() { return $res } +# What is the version of this metadump file? +_xfs_metadumpfile_version() { + local file="$1" + local magic + + magic="$($XFS_IO_PROG -c 'pread -q -v 0 4' "$file")" + case "$magic" in + "00000000: 58 4d 44 32 XMD2") echo 2;; + "00000000: 58 46 53 4d XFSM") echo 1;; + esac +} + _xfs_mdrestore() { local metadump="$1" local device="$2" local logdev="$3" shift; shift; shift local options="$@" + local dumpfile_ver # If we're configured for compressed dumps and there isn't already an # uncompressed dump, see if we can use DUMP_COMPRESSOR to decompress @@ -705,8 +718,18 @@ _xfs_mdrestore() { done fi test -r "$metadump" || return 1 + dumpfile_ver="$(_xfs_metadumpfile_version "$metadump")" - if [ "$logdev" != "none" ]; then + if [ "$logdev" != "none" ] && [[ $dumpfile_ver > 1 ]]; then + # metadump and mdrestore began capturing and restoring the + # contents of external log devices with the addition of the + # metadump v2 format. Hence it only makes sense to specify -l + # here if the dump file itself is in v2 format. + # + # With a v1 metadump, the log device is not changed by the dump + # and restore process. Historically, fstests either didn't + # notice or _notrun themselves when external logs were in use. + # Don't break that for people testing with xfsprogs < 6.5. options="$options -l $logdev" fi From patchwork Wed Feb 7 02:19:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13548044 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B334AFC0E; Wed, 7 Feb 2024 02:19:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272371; cv=none; b=D8u/UnhqIqJTWMEmNG5lmGW+ZCJUhnkWCM5UaX8tMHcdW6UQHxjj14B/6eYjE9wZAu3XEewh1e9dUkBT/gM9DEUsk7KdjHELxFvnfY2pWi89RewhMpoPgk3laPwW1LLMqofXK2e7nY/FWdRx+ET9oqaXtksRdVh1gMFsaYmc8MI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707272371; c=relaxed/simple; bh=UAlpnsmgl0Qg1lH8S9OO69u9rlBwquRjVoJ8Ib/QKUc=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bzMIxaFEh398DVWmO/XzR7U3DTkeShydaRjGkM+bCgfErpu8degln+ZwZsci9acy85p+EEs2dQZPyPEVy6CZnq3qfwAbwHAOlZ3AHE2Uk7ugyTKiwzzRlv//OjDlyTEuYNyriocYvoFp/LlSykTb9YiyrSfSXPG3OHaq51ycNus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ifGQgVHg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ifGQgVHg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A3B3C433C7; Wed, 7 Feb 2024 02:19:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707272371; bh=UAlpnsmgl0Qg1lH8S9OO69u9rlBwquRjVoJ8Ib/QKUc=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ifGQgVHgERpA8ZARfB7Fvub2txrQG/h7uuakGPPXsAMRcm5hPW2O7J7lDWbFBODk5 FVOiXeC6QOzMDxbtVGCT8PYlmDdfVMIsrcXeiswgd2mi4JyNf5wMWPSwtuVLOoNdRx 5AiG6WNspKh504ekiUIvhv/I17fn05KNMk26m/qVXHlrtolhGnHyLLfDAZkC5WauZS IlH/x8uRWfO+bdv1nB6L/t1cVIuCybwQ8l/OBwyTFOoM99QJrJsD692/wPmXCQ/Fxl XVoXimuJK2aT5wyp9lJ8uQPab0ezuO9Bpo1ROrcsXlz619ZxakoM+v9Xn2khY2f3l2 +MIfpV1JU06gg== Subject: [PATCH 10/10] xfs/122: fix for xfs_attr_shortform removal in 6.8 From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: Andrey Albershteyn , Christoph Hellwig , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 06 Feb 2024 18:19:30 -0800 Message-ID: <170727237066.3726171.11659068396490963689.stgit@frogsfrogsfrogs> In-Reply-To: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> References: <170727231361.3726171.14834727104549554832.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The xfs_attr_shortform struct (with multiple flexarrays) was removed in 6.8. Check the two surviving structures (the attr sf header and entry) instead. Signed-off-by: "Darrick J. Wong" Reviewed-by: Andrey Albershteyn Reviewed-by: Christoph Hellwig Signed-off-by: Zorro Lang --- tests/xfs/122.out | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index 89f7b735b0..067a0ec76b 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -62,6 +62,8 @@ sizeof(struct xfs_agfl) = 36 sizeof(struct xfs_attr3_leaf_hdr) = 80 sizeof(struct xfs_attr3_leafblock) = 88 sizeof(struct xfs_attr3_rmt_hdr) = 56 +sizeof(struct xfs_attr_sf_entry) = 3 +sizeof(struct xfs_attr_sf_hdr) = 4 sizeof(struct xfs_attr_shortform) = 8 sizeof(struct xfs_attrd_log_format) = 16 sizeof(struct xfs_attri_log_format) = 40