From patchwork Thu Jan 25 19:04:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531522 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EFDA135405; Thu, 25 Jan 2024 19:04:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209455; cv=none; b=HrYSTBQ7fth+p4/TlR+rAzWp4mNZYo6EuTM0+KJztlA82zZh/7dOvphR9XjbQyhsftpvf+ULzCPRCCz3auq3UhBcT8BEuJHymH/Nfi/oAdpXZAErCT+n6t1z+BzfM1XccImrDhD5XAaRgnPfp0JV8J39anrjUn/k+SKeK+dRZBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209455; c=relaxed/simple; bh=5AokXJeh3keA1gMhAeE9qrT5mnwtCxg8tC62AA/W9z0=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YXuCZ5UcSYqhdEqs3bHjarxSKMwQ0Gf9OGpbHJiDzVPQhmroJ9ecX4tHUIKnPWHiMgstW0qyddPwpV6wZUupDKFteq/VHumhcZzlS4erokd/44M8B1fWXLH1YH3uZog3jDKyT+tatcVFxTvXrEvkxEwkLKmdnhpMYDOkU87uh+E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=K69CorOZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="K69CorOZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8761DC433F1; Thu, 25 Jan 2024 19:04:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209454; bh=5AokXJeh3keA1gMhAeE9qrT5mnwtCxg8tC62AA/W9z0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=K69CorOZsYoloRi/nBItTR1XuxCMQ1kmaaTsrn912pUjzEf+K1HkVJZOBwneLNMw0 yeymByi9KO9gPEgneA6p+B43io3co+RaecoECxaUvZNCtFXvLJrIR1qVkN4Zm1Al47 wmIAVqpxKvkk9yjvK9qB+cICTFKzT1CBcjfGVDlobhtNKAQmUnCA+nJNrr0iMf02SY rJEy4mROwLNn2W/eDQwXCp229i6vrMVa5DhxdWor2Flj5EMmhk4dRkC9+5CqxKpF7R ICS4pR3p6iA7Ay+NyfZja+i0vaETIEBv1YXImx/EqRjlHKvn+UAsEAJp7BK7jK4sPZ 2T52Q/1UM6DMg== Date: Thu, 25 Jan 2024 11:04:14 -0800 Subject: [PATCH 01/10] generic/256: constrain runtime with TIME_FACTOR From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924382.3283496.6995781268514337077.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong This test runs 500 iterations of a "fill the fs and try to punch" test. Hole punching can be particularly slow if, say, the filesystem is mounted with -odiscard and the DISCARD operation takes a very long time. In extreme cases, I can see test runtimes of 4+ hours. Constrain the runtime of _test_full_fs_punch by establishing a deadline of (30 seconds * TIME_FACTOR) and breaking out of the for loop if the test goes beyond the time budget. This keeps the runtime within the customary 30 seconds. Signed-off-by: Darrick J. Wong Reviewed-by: Andrey Albershteyn Reviewed-by: Christoph Hellwig --- tests/generic/256 | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/generic/256 b/tests/generic/256 index 808a730f3a..ea6cc2938a 100755 --- a/tests/generic/256 +++ b/tests/generic/256 @@ -44,6 +44,8 @@ _test_full_fs_punch() local file_len=$(( $(( $hole_len + $hole_interval )) * $iterations )) local path=`dirname $file_name` local hole_offset=0 + local start_time + local stop_time if [ $# -ne 5 ] then @@ -57,6 +59,9 @@ _test_full_fs_punch() -c "fsync" $file_name &> /dev/null chmod 666 $file_name + start_time="$(date +%s)" + stop_time=$(( start_time + (30 * TIME_FACTOR) )) + # All files are created as a non root user to prevent reserved blocks # from being consumed. _fill_fs $(( 1024 * 1024 * 1024 )) $path/fill $block_size 1 \ @@ -64,6 +69,8 @@ _test_full_fs_punch() for (( i=0; i<$iterations; i++ )) do + test "$(date +%s)" -ge "$stop_time" && break + # This part must not be done as root in order to # test that reserved blocks are used when needed _user_do "$XFS_IO_PROG -f -c \"fpunch $hole_offset $hole_len\" $file_name" From patchwork Thu Jan 25 19:04:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531523 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C05545C0B; Thu, 25 Jan 2024 19:04:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209470; cv=none; b=ut1tyfdrzxLj9G5W3fHoPODQM3t7XJFqITROMB9uNBCZgqXA5sxNl6NYDwv04X4uxH1iinBzIwf/5lPtx2gqlJRyad6RzPhS+4J5ARtBF1/ld3RQa9v6lK1eAa9sknpN4jCuXdqf+mx+wO9OQnqNpyLuRrtmD/26YxMXD4aNu/s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209470; c=relaxed/simple; bh=JI8ILibsFbTQ0GK01vlYUcf86OvXk2DrI0rW/4FMV6M=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tmQH9eFYxEg6MX8fq/0SX9C3Xa3Rm/yRyIP9ySBzxfNj8HkVIoMth2UXpgFx1fxa/lbfx6JzO1psvpjCGHPkSwzb6e6GMAFdF/BSFoA92N55X7GW6j/a/1c90IzQz4sf0npJ7/2vfHI0Fl3jZ2JEqmYJnOiZiW5x8IA3YLQqDOk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=S7jl0Xpo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="S7jl0Xpo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29E17C433F1; Thu, 25 Jan 2024 19:04:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209470; bh=JI8ILibsFbTQ0GK01vlYUcf86OvXk2DrI0rW/4FMV6M=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=S7jl0XponCT3Lh9dUsxVEGukqar4UrK6E0BvxJJLFii0tD6kNTUmscLsefylRkUkM qCcuSCb/dcF0rh9VsJ7ioQPFM+8igqjpOevnzb//OGw42kdYunFYo9rRLTpm2JQI5o wjIvEu0WpxQbpDN+WtF+x3KHHdMNERU1Lac/nO1kGDWJKf+ybMpnaZBhOBQb6esq4M m6XFtYR1OdjrvQCdvj6v4Lty1QZ8US30BHJnb/QlBNtbDh+VmzFxWL1NfsxwOhGxj3 5bjnnE8ow6LE4AeIDLxZXKfoX39PRmRjPSelCCv/tbvXCjscykHPdGD3einf1b5jq2 SRh0hUBC2oQHQ== Date: Thu, 25 Jan 2024 11:04:29 -0800 Subject: [PATCH 02/10] common/xfs: simplify maximum metadump format detection From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924396.3283496.16690906003840940544.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong xfs_metadump (aka the wrapper around xfs_db -c metadump) advertises the -v switch to turn on v2 format in its help screen. There's no need to fire up xfs_db on the scratch device which will load the AGs and take much longer. While we're at it, reduce the amount of boilerplate in the test files by changing the function to emit the max version supported. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- common/xfs | 10 +++++++--- tests/xfs/129 | 3 +-- tests/xfs/234 | 3 +-- tests/xfs/253 | 3 +-- tests/xfs/291 | 3 +-- tests/xfs/432 | 3 +-- tests/xfs/503 | 3 +-- tests/xfs/605 | 3 +-- 8 files changed, 14 insertions(+), 17 deletions(-) diff --git a/common/xfs b/common/xfs index 11cfd79562..248ccefda3 100644 --- a/common/xfs +++ b/common/xfs @@ -713,10 +713,14 @@ _xfs_mdrestore() { $XFS_MDRESTORE_PROG $options "${metadump}" "${device}" } -_scratch_metadump_v2_supported() +# What is the maximum metadump file format supported by xfs_metadump? +_xfs_metadump_max_version() { - $XFS_DB_PROG -c "help metadump" $SCRATCH_DEV | \ - grep -q "Metadump version to be used" + if $XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-v version'; then + echo 2 + else + echo 1 + fi } # Snapshot the metadata on the scratch device diff --git a/tests/xfs/129 b/tests/xfs/129 index 8a817b416c..cdac2349df 100755 --- a/tests/xfs/129 +++ b/tests/xfs/129 @@ -106,8 +106,7 @@ verify_metadump_v2() _scratch_mkfs >/dev/null 2>&1 -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) _scratch_mount diff --git a/tests/xfs/234 b/tests/xfs/234 index c9bdb674ab..f4f8af6d3a 100755 --- a/tests/xfs/234 +++ b/tests/xfs/234 @@ -106,8 +106,7 @@ verify_metadump_v2() _scratch_mkfs >/dev/null 2>&1 -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) _scratch_mount diff --git a/tests/xfs/253 b/tests/xfs/253 index 8e18ddb83a..3b567999d8 100755 --- a/tests/xfs/253 +++ b/tests/xfs/253 @@ -233,8 +233,7 @@ cd $here _scratch_unmount -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) verify_metadump_v1 $max_md_version diff --git a/tests/xfs/291 b/tests/xfs/291 index 33193eb78e..1433140821 100755 --- a/tests/xfs/291 +++ b/tests/xfs/291 @@ -92,8 +92,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed" # Yes they can! Now... # Can xfs_metadump cope with this monster? -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) for md_version in $(seq 1 $max_md_version); do version="" diff --git a/tests/xfs/432 b/tests/xfs/432 index a215d3ce2e..7e402aa88f 100755 --- a/tests/xfs/432 +++ b/tests/xfs/432 @@ -87,8 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks" echo "Try to metadump, restore and check restored metadump image" -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) for md_version in $(seq 1 $max_md_version); do version="" diff --git a/tests/xfs/503 b/tests/xfs/503 index a1479eb613..8643c3d483 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -54,8 +54,7 @@ check_restored_metadump_image() _destroy_loop_device $loop_dev } -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) echo "metadump and mdrestore" for md_version in $(seq 1 $max_md_version); do diff --git a/tests/xfs/605 b/tests/xfs/605 index 5cbf5d2550..f2cd7aba98 100755 --- a/tests/xfs/605 +++ b/tests/xfs/605 @@ -44,8 +44,7 @@ testfile=${SCRATCH_MNT}/testfile echo "Format filesystem on scratch device" _scratch_mkfs >> $seqres.full 2>&1 -max_md_version=1 -_scratch_metadump_v2_supported && max_md_version=2 +max_md_version=$(_xfs_metadump_max_version) external_log=0 if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then From patchwork Thu Jan 25 19:04:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531524 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A1F879C7; Thu, 25 Jan 2024 19:04:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209486; cv=none; b=bv0d7mix7jmdiqPX3qZLICD390+y2d+ClGLtrU1pNhMJokLzO3NmBr90nX7hQAFwfHcvejycEvJKRAXvRnvzTs37ul9E/uXhICdYitmz8Xd+SsJe9jyhR7ChvMqsTKi05gbcWCPwQQX3LPmSmyEx3xcrxQBKMrCy0YnbzCQfA80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209486; c=relaxed/simple; bh=hN6jaGKYHn0Ll9Q+qOYQuEkgXYuuEgKo9T6VXUeZgog=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fZ/QrPR61D53IgWYdJESMOyJ3xkhw9ZdKzbiGMQvPlwKwJxsoUBQ3paISG2b7KX+x8U5ONiXmffz5nkM1xKnWFLzv/iUuMVekqxc9UotjjQz0oCg5Dbo7Tyvyutdkvt2ebcZ5NzXJRPhGJP2bekHo0CHiqSrj8ccIuKgqgmY8So= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cEhy6uiP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cEhy6uiP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4643C433F1; Thu, 25 Jan 2024 19:04:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209485; bh=hN6jaGKYHn0Ll9Q+qOYQuEkgXYuuEgKo9T6VXUeZgog=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=cEhy6uiPS7ZLSTs6X7OOD+s1U+6VD4jXMKCJcOeztdzwsquMeL0nK+QjAFNwFzNq5 wK2/paNl3u6DDO6PnRkF7G04XipL1g5bEh5itKVkBgqY3cqECxNWHke1OEjo+f9KRG S3OAaQZJnXWLsB9uYCKSVx0xNdkJUP9sW8gtI9M19eFiVvJ6e5+lhq0PJo3QTadtVa WMUi6BxHg4qIpU8G6y/4v5ApvjoN1TqOwOU3/v9opp8UKIAy35ihTdzIwcbNjvp+Yr yvoYMVoueVD9EXCzRaBCh1CjEIbBjPFJ+4yEWxPgunJ3TPwQaIdp0PceHAhcjD8X4A zV6y5hMAXTVfQ== Date: Thu, 25 Jan 2024 11:04:45 -0800 Subject: [PATCH 03/10] common/populate: always metadump full metadata blocks From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924408.3283496.10108828998772840672.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Commit e443cadcea pushed the -a and -o options to the _scratch_xfs_metadump callsites. Unfortunately, it missed the _xfs_metadump callsite in common/populate, so fix that now. Fixes: e443cadcea ("common/xfs: Do not append -a and -o options to metadump") Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- common/populate | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/common/populate b/common/populate index cfbfd88a7f..33f2db8d4a 100644 --- a/common/populate +++ b/common/populate @@ -1029,7 +1029,7 @@ _scratch_populate_cached() { logdev=$SCRATCH_LOGDEV _xfs_metadump "$POPULATE_METADUMP" "$SCRATCH_DEV" "$logdev" \ - compress + compress -a -o ;; "ext2"|"ext3"|"ext4") _scratch_ext4_populate $@ From patchwork Thu Jan 25 19:05:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531525 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19051130E52; Thu, 25 Jan 2024 19:05:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209502; cv=none; b=ZokPGvQvHEbUmEvfJyGI/d3266fW7faI+XjXzgcttVTeheZsKnU/UumBf9JfXXfBz7pwO8E89znUHVQwMsxOqHoD1mqbad22B1kEPlp+STfmJ4blHP2rf9WQAPnk9rWdeMw2ElCYHiMN1mEtjIo+vPR4fKG1PxL5tqsFpe1uxHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209502; c=relaxed/simple; bh=YIo9Fz1LaXEbfE5IIpbp5yfR31HNfAz9ZnlfHPqmXuk=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=m+XxbTiQqtfxv/7Iq5R7nojW3zesSRBuzWiUtkmSz96F57mM2agiD1Dlb2ZRXRXkduP4aBcMpTAZnlxGV+vUBIh9XCpGjYrHKy/oFK21sy3q/SQGmHY0745UHxK3a20vqulQBZrj0VMS++9w9fkJ5aeJdRH0XdA5mW+RVG6qQEU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YPospv4f; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YPospv4f" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8560CC43394; Thu, 25 Jan 2024 19:05:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209501; bh=YIo9Fz1LaXEbfE5IIpbp5yfR31HNfAz9ZnlfHPqmXuk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=YPospv4fGm2CXkNHrUiCKxt4zSXzELYlYfRbJ/CRqRgnZmogJNU+2oNfkf3xEfsTN WPU71kHstKSXvRn/Y5xZC+qZInlpUDt2KLsmmLg7T++XKwg05zpHg1bcgeFFDoBRdK CMXhjIBQHrsH8JUKq4nbSNQqWHqUUFlUR7VeCXShhQsaF8h41X09lXVFnx61z1AMSb SXYu8X9xDh4G471g50iZkpZLTfvcjNjbkJsxloLp5e6Poar+FeqKKqzasrQl3x1Xru vGpcJzJ7WkQVb+4Mk3zo6gFONT3kMmVEyDKQIUINyUiiKcxqHXM9M8Zn7+1ML6xkfG rY887Rjd0znZA== Date: Thu, 25 Jan 2024 11:05:01 -0800 Subject: [PATCH 04/10] xfs/336: fix omitted -a and -o in metadump call From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924420.3283496.6869276450141660418.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Commit e443cadcea reworked _xfs_metadump to require that all callers always pass the arguments they want -- no more defaulting to "-a -o". Unfortunately, a few got missed. Fix some of them now; the rest will get cleaned up in the next patch. Fixes: e443cadcea ("common/xfs: Do not append -a and -o options to metadump") Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- tests/xfs/284 | 4 ++-- tests/xfs/336 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/xfs/284 b/tests/xfs/284 index 58f330035e..443c375759 100755 --- a/tests/xfs/284 +++ b/tests/xfs/284 @@ -42,12 +42,12 @@ COPY_FILE="${TEST_DIR}/${seq}_copyfile" # xfs_metadump should refuse to dump a mounted device _scratch_mkfs >> $seqres.full 2>&1 _scratch_mount -_scratch_xfs_metadump $METADUMP_FILE 2>&1 | filter_mounted +_scratch_xfs_metadump $METADUMP_FILE -a -o 2>&1 | filter_mounted _scratch_unmount # Test restore to a mounted device # xfs_mdrestore should refuse to restore to a mounted device -_scratch_xfs_metadump $METADUMP_FILE +_scratch_xfs_metadump $METADUMP_FILE -a -o _scratch_mount _scratch_xfs_mdrestore $METADUMP_FILE 2>&1 | filter_mounted _scratch_unmount diff --git a/tests/xfs/336 b/tests/xfs/336 index 43b3790cbb..3c30f1a40b 100755 --- a/tests/xfs/336 +++ b/tests/xfs/336 @@ -62,7 +62,7 @@ _scratch_cycle_mount echo "Create metadump file" _scratch_unmount -_scratch_xfs_metadump $metadump_file -a +_scratch_xfs_metadump $metadump_file -a -o # Now restore the obfuscated one back and take a look around echo "Restore metadump" From patchwork Thu Jan 25 19:05:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531526 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B414B537E3; Thu, 25 Jan 2024 19:05:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209517; cv=none; b=V40EH7I1/hp76KsRqV41Gtko8fC5cG3twr9YduQW0q98KF4Fy7AV1fA6e5m8RJpQKs7AANhWEVcU/LiAgk7IHtgHQPdW1r9oVkIJ1Did9iU6fOQaGy0crjIFJJd+vEiaZ7rHDfPobfLOargp9eVSYYVdZ7IcdShJdRsyTDgcyhQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209517; c=relaxed/simple; bh=SfBPMXQFjztWWUpNysYwj0v9XXS+IXeUveTi0SOOIl4=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ElVkq/A8WO6FUtIkt+G/b9Z3wVL+UxDr+Los2DGt/10YpjYlIzEgq9iR8VVaOZ/UO6enR9WtqVOalnn7kavGJwiUbuYXQi+am6IwiuwpqQal+NBG/NZca2luJpNz0046Z50prRM0eR1D/Ky5bOLm0wvPlaayS2daXbjD+3LKvi8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HPnRdwHe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HPnRdwHe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C622C433F1; Thu, 25 Jan 2024 19:05:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209517; bh=SfBPMXQFjztWWUpNysYwj0v9XXS+IXeUveTi0SOOIl4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=HPnRdwHeotd6r316F8i7PEW+lxTIdukZIpPKQA2urUPWn6XqguaX2QVNWSjIsWnKL yQpDmZYuAjxfI9OBAeB1yDkI02egsiv30wKnl7mkaFjH6iXXXsR3fn63WvX4awyKU3 3HgkyxTDJbYAzCqekUJbwtBn22hV7t4vXcmzA7MbnzcJt5iIJ/QCEQqRoGYGc+mZ2F L6sjxs+04MxIaHK6tWQ5UrdV4sXtrJySJX64ADBnOvUIbfj5K+o2eeFgJmJeBL++eI STjsxtojHyNwlNjTIXWbqf5p4UajgyhswkWRmfiWfWkEbDT/EMJ8qzFefP2+SX6Cet WkkB6r/XrOoBA== Date: Thu, 25 Jan 2024 11:05:16 -0800 Subject: [PATCH 05/10] common: refactor metadump v1 and v2 tests From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924435.3283496.2022458241568622607.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291, xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly duplicate copies of the same code. While we're at it, fix the fsck so that it includes xfs_scrub. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- common/rc | 10 ---- common/xfs | 14 +++++ common/xfs_metadump_tests | 123 +++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/129 | 90 ++------------------------------- tests/xfs/234 | 91 ++------------------------------- tests/xfs/253 | 89 ++------------------------------- tests/xfs/291 | 31 ++++------- tests/xfs/432 | 30 ++--------- tests/xfs/503 | 60 +++------------------- tests/xfs/605 | 84 ++----------------------------- 10 files changed, 181 insertions(+), 441 deletions(-) create mode 100644 common/xfs_metadump_tests diff --git a/common/rc b/common/rc index 524ffa02aa..0b69f7f54f 100644 --- a/common/rc +++ b/common/rc @@ -3320,15 +3320,7 @@ _check_scratch_fs() case $FSTYP in xfs) - local scratch_log="none" - local scratch_rt="none" - [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \ - scratch_log="$SCRATCH_LOGDEV" - - [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \ - scratch_rt="$SCRATCH_RTDEV" - - _check_xfs_filesystem $device $scratch_log $scratch_rt + _check_xfs_scratch_fs $device ;; udf) _check_udf_filesystem $device $udf_fsize diff --git a/common/xfs b/common/xfs index 248ccefda3..6a48960a7f 100644 --- a/common/xfs +++ b/common/xfs @@ -1035,6 +1035,20 @@ _check_xfs_test_fs() return $? } +_check_xfs_scratch_fs() +{ + local device="${1:-$SCRATCH_DEV}" + local scratch_log="none" + local scratch_rt="none" + [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \ + scratch_log="$SCRATCH_LOGDEV" + + [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \ + scratch_rt="$SCRATCH_RTDEV" + + _check_xfs_filesystem $device $scratch_log $scratch_rt +} + # modeled after _scratch_xfs_repair _test_xfs_repair() { diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests new file mode 100644 index 0000000000..dd3dec1fb4 --- /dev/null +++ b/common/xfs_metadump_tests @@ -0,0 +1,123 @@ +# +# XFS specific metadump testing functions. +# + +# Set up environment variables for a metadump test. Requires the test and +# scratch devices. Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION. +_setup_verify_metadump() +{ + XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump" + XFS_METADUMP_IMG="$TEST_DIR/${seq}_image" + MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)" + + rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"* +} + +_cleanup_verify_metadump() +{ + _scratch_unmount &>> $seqres.full + + losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do + losetup -d "$ldev" + done + rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"* +} + +# Create a metadump in v1 format, restore it to fs image files, then mount the +# images and fsck them. +_verify_metadump_v1() +{ + local metadump_args="$1" + local extra_test="$2" + + local metadump_file="$XFS_METADUMP_FILE" + local version="" + local data_img="$XFS_METADUMP_IMG.data" + local data_loop + + # Force v1 if we detect v2 support + if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then + version="-v 1" + fi + + # Capture metadump, which creates metadump_file + _scratch_xfs_metadump $metadump_file $metadump_args $version + + # Restore metadump, which creates data_img + SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file + + # Create loopdev for data device so we can mount the fs + data_loop=$(_create_loop_device $data_img) + + # Mount fs, run an extra test, fsck, and unmount + SCRATCH_DEV=$data_loop _scratch_mount + if [ -n "$extra_test" ]; then + SCRATCH_DEV=$data_loop $extra_test + fi + SCRATCH_DEV=$data_loop _check_xfs_scratch_fs + SCRATCH_DEV=$data_loop _scratch_unmount + + # Tear down what we created + _destroy_loop_device $data_loop + rm -f $data_img +} + +# Create a metadump in v2 format, restore it to fs image files, then mount the +# images and fsck them. +_verify_metadump_v2() +{ + local metadump_args="$1" + local extra_test="$2" + + local metadump_file="$XFS_METADUMP_FILE" + local version="-v 2" + local data_img="$XFS_METADUMP_IMG.data" + local data_loop + local log_img="" + local log_loop + + # Capture metadump, which creates metadump_file + _scratch_xfs_metadump $metadump_file $metadump_args $version + + # + # Metadump v2 files can contain contents dumped from an external log + # device. Use a temporary file to hold the log device contents restored + # from such a metadump file. + test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log" + + # Restore metadump, which creates data_img and log_img + SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \ + _scratch_xfs_mdrestore $metadump_file + + # Create loopdev for data device so we can mount the fs + data_loop=$(_create_loop_device $data_img) + + # Create loopdev for log device if we recovered anything + test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + + # Mount fs, run an extra test, fsck, and unmount + SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + if [ -n "$extra_test" ]; then + SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + fi + SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs + SCRATCH_DEV=$data_loop _scratch_unmount + + # Tear down what we created + if [ -b "$log_loop" ]; then + _destroy_loop_device $log_loop + rm -f $log_img + fi + _destroy_loop_device $data_loop + rm -f $data_img +} + +# Verify both metadump formats if possible +_verify_metadumps() +{ + _verify_metadump_v1 "$@" + + if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then + _verify_metadump_v2 "$@" + fi +} diff --git a/tests/xfs/129 b/tests/xfs/129 index cdac2349df..c3a9bcefee 100755 --- a/tests/xfs/129 +++ b/tests/xfs/129 @@ -16,98 +16,23 @@ _cleanup() { cd / _scratch_unmount > /dev/null 2>&1 - [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev - rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \ - $TEST_DIR/log-image + _cleanup_verify_metadump + rm -rf $tmp.* $testdir } # Import common functions. . ./common/filter . ./common/reflink +. ./common/xfs_metadump_tests # real QA test starts here _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_loop _require_scratch_reflink - -metadump_file=$TEST_DIR/${seq}_metadump - -verify_metadump_v1() -{ - local max_version=$1 - local version="" - - if [[ $max_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $metadump_file $version - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - SCRATCH_DEV=$datadev _scratch_unmount - - logdev=$SCRATCH_LOGDEV - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - version="-v 2" - - _scratch_xfs_metadump $metadump_file $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev=${SCRATCH_LOGDEV} - if [[ -s $TEST_DIR/log-image ]]; then - logdev=$(_create_loop_device $TEST_DIR/log-image) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - if [[ -s $TEST_DIR/log-image ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $TEST_DIR/log-image - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} +_setup_verify_metadump _scratch_mkfs >/dev/null 2>&1 - -max_md_version=$(_xfs_metadump_max_version) - _scratch_mount testdir=$SCRATCH_MNT/test-$seq @@ -127,12 +52,7 @@ done _scratch_unmount echo "Create metadump file, restore it and check restored fs" - -verify_metadump_v1 $max_md_version - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_verify_metadumps # success, all done status=0 diff --git a/tests/xfs/234 b/tests/xfs/234 index f4f8af6d3a..8f808c7507 100755 --- a/tests/xfs/234 +++ b/tests/xfs/234 @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump _cleanup() { cd / - _scratch_unmount > /dev/null 2>&1 - [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev - rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \ - $TEST_DIR/log-image + _cleanup_verify_metadump + rm -rf $tmp.* $testdir } # Import common functions. . ./common/filter +. ./common/xfs_metadump_tests # real QA test starts here _supported_fs xfs @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_loop _require_xfs_scratch_rmapbt _require_xfs_io_command "fpunch" - -metadump_file=$TEST_DIR/${seq}_metadump - -verify_metadump_v1() -{ - local max_version=$1 - local version="" - - if [[ $max_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $metadump_file $version - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - SCRATCH_DEV=$datadev _scratch_unmount - - logdev=$SCRATCH_LOGDEV - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - version="-v 2" - - _scratch_xfs_metadump $metadump_file $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev=${SCRATCH_LOGDEV} - if [[ -s $TEST_DIR/log-image ]]; then - logdev=$(_create_loop_device $TEST_DIR/log-image) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - [[ -z $logdev ]] && logdev=none - _check_xfs_filesystem $datadev $logdev none - - if [[ -s $TEST_DIR/log-image ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $TEST_DIR/log-image - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} +_setup_verify_metadump _scratch_mkfs >/dev/null 2>&1 - -max_md_version=$(_xfs_metadump_max_version) - _scratch_mount testdir=$SCRATCH_MNT/test-$seq @@ -127,12 +51,7 @@ done _scratch_unmount echo "Create metadump file, restore it and check restored fs" - -verify_metadump_v1 $max_md_version - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_verify_metadumps # success, all done status=0 diff --git a/tests/xfs/253 b/tests/xfs/253 index 3b567999d8..6623c435e5 100755 --- a/tests/xfs/253 +++ b/tests/xfs/253 @@ -26,23 +26,21 @@ _cleanup() cd / rm -f $tmp.* rm -rf "${OUTPUT_DIR}" - rm -f "${METADUMP_FILE}" - [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev + _cleanup_verify_metadump } # Import common functions. . ./common/filter +. ./common/xfs_metadump_tests _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_test _require_scratch +_setup_verify_metadump # real QA test starts here OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}" -METADUMP_FILE="${TEST_DIR}/${seq}_metadump" ORPHANAGE="lost+found" _supported_fs xfs @@ -52,24 +50,7 @@ function create_file() { touch $(printf "$@") } -verify_metadump_v1() -{ - local max_version=$1 - local version="" - - if [[ $max_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $METADUMP_FILE $version - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \ - _scratch_xfs_mdrestore $METADUMP_FILE - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - +extra_test() { cd "${SCRATCH_MNT}" # Get a listing of all the files after obfuscation @@ -78,60 +59,6 @@ verify_metadump_v1() ls -R | od -c >> $seqres.full cd / - - SCRATCH_DEV=$datadev _scratch_unmount - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - version="-v 2" - - _scratch_xfs_metadump $METADUMP_FILE $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $METADUMP_FILE - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev=${SCRATCH_LOGDEV} - if [[ -s $TEST_DIR/log-image ]]; then - logdev=$(_create_loop_device $TEST_DIR/log-image) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - - cd "${SCRATCH_MNT}" - - # Get a listing of all the files after obfuscation - echo "Metadump v2" >> $seqres.full - ls -R >> $seqres.full - ls -R | od -c >> $seqres.full - - cd / - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - if [[ -s $TEST_DIR/log-image ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $TEST_DIR/log-image - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image } echo "Disciplyne of silence is goed." @@ -233,13 +160,7 @@ cd $here _scratch_unmount -max_md_version=$(_xfs_metadump_max_version) - -verify_metadump_v1 $max_md_version - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_verify_metadumps '' extra_test # Finally, re-make the filesystem since to ensure we don't # leave a directory with duplicate entries lying around. diff --git a/tests/xfs/291 b/tests/xfs/291 index 1433140821..c475d89ad9 100755 --- a/tests/xfs/291 +++ b/tests/xfs/291 @@ -9,11 +9,21 @@ . ./common/preamble _begin_fstest auto repair metadump +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* + _cleanup_verify_metadump +} + # Import common functions. . ./common/filter +. ./common/xfs_metadump_tests _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" +_setup_verify_metadump # real QA test starts here _require_scratch @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed" # Yes they can! Now... # Can xfs_metadump cope with this monster? -max_md_version=$(_xfs_metadump_max_version) - -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $tmp.metadump -a -o $version || \ - _fail "xfs_metadump failed" - - slogdev=$SCRATCH_LOGDEV - if [[ -z $version || $version == "-v 1" ]]; then - slogdev="" - fi - SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \ - $tmp.metadump || _fail "xfs_mdrestore failed" - SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \ - _fail "xfs_repair of metadump failed" -done +_verify_metadumps '-a -o' # Yes it can; success, all done status=0 diff --git a/tests/xfs/432 b/tests/xfs/432 index 7e402aa88f..579e1b556a 100755 --- a/tests/xfs/432 +++ b/tests/xfs/432 @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump _cleanup() { cd / - rm -f "$tmp".* $metadump_file $metadump_img + rm -f "$tmp".* + _cleanup_verify_metadump } # Import common functions. . ./common/filter +. ./common/xfs_metadump_tests # real QA test starts here _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_scratch +_setup_verify_metadump rm -f "$seqres.full" @@ -54,9 +57,6 @@ echo "Format and mount" _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1 _scratch_mount >> "$seqres.full" 2>&1 -metadump_file="$TEST_DIR/meta-$seq" -metadump_img="$TEST_DIR/img-$seq" -rm -f $metadump_file $metadump_img testdir="$SCRATCH_MNT/test-$seq" max_fname_len=255 blksz=$(_get_block_size $SCRATCH_MNT) @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks" echo "Try to metadump, restore and check restored metadump image" -max_md_version=$(_xfs_metadump_max_version) - -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a -o -w $version - - slogdev=$SCRATCH_LOGDEV - if [[ -z $version || $version == "-v 1" ]]; then - slogdev="" - fi - - SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \ - echo "xfs_repair on restored fs returned $?" -done +_verify_metadumps '-a -o -w' # success, all done status=0 diff --git a/tests/xfs/503 b/tests/xfs/503 index 8643c3d483..ff6b344a9c 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -17,11 +17,13 @@ _cleanup() { cd / rm -rf $tmp.* $testdir + _cleanup_verify_metadump } # Import common functions. . ./common/filter . ./common/populate +. ./common/xfs_metadump_tests testdir=$TEST_DIR/test-$seq @@ -35,6 +37,7 @@ _require_scratch_nocheck _require_populate_commands _xfs_skip_online_rebuild _xfs_skip_offline_rebuild +_setup_verify_metadump echo "Format and populate" _scratch_populate_cached nofill > $seqres.full 2>&1 @@ -43,66 +46,17 @@ mkdir -p $testdir metadump_file=$testdir/scratch.md copy_file=$testdir/copy.img -check_restored_metadump_image() -{ - local image=$1 - - loop_dev=$(_create_loop_device $image) - SCRATCH_DEV=$loop_dev _scratch_mount - SCRATCH_DEV=$loop_dev _check_scratch_fs - SCRATCH_DEV=$loop_dev _scratch_unmount - _destroy_loop_device $loop_dev -} - -max_md_version=$(_xfs_metadump_max_version) - echo "metadump and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_verify_metadumps '-a -o' echo "metadump a and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_verify_metadumps '-a' echo "metadump g and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -g $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_verify_metadumps '-g' >> $seqres.full echo "metadump ag and mdrestore" -for md_version in $(seq 1 $max_md_version); do - version="" - if [[ $max_md_version == 2 ]]; then - version="-v $md_version" - fi - - _scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full - SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file - check_restored_metadump_image $TEST_DIR/image -done +_verify_metadumps '-a -g' >> $seqres.full echo copy $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full diff --git a/tests/xfs/605 b/tests/xfs/605 index f2cd7aba98..af917f0f32 100755 --- a/tests/xfs/605 +++ b/tests/xfs/605 @@ -15,17 +15,13 @@ _cleanup() { cd / rm -r -f $tmp.* - _scratch_unmount > /dev/null 2>&1 - [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \ - _destroy_loop_device $logdev - [[ -n $datadev ]] && _destroy_loop_device $datadev - rm -r -f $metadump_file $TEST_DIR/data-image \ - $TEST_DIR/log-image + _cleanup_verify_metadump } # Import common functions. . ./common/dmflakey . ./common/inject +. ./common/xfs_metadump_tests # real QA test starts here _supported_fs xfs @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin _require_dm_target flakey _require_xfs_io_command "pwrite" _require_test_program "punch-alternating" +_setup_verify_metadump -metadump_file=${TEST_DIR}/${seq}.md testfile=${SCRATCH_MNT}/testfile echo "Format filesystem on scratch device" _scratch_mkfs >> $seqres.full 2>&1 -max_md_version=$(_xfs_metadump_max_version) - external_log=0 if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then external_log=1 fi -if [[ $max_md_version == 1 && $external_log == 1 ]]; then +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then _notrun "metadump v1 does not support external log device" fi -verify_metadump_v1() -{ - local version="" - if [[ $max_md_version == 2 ]]; then - version="-v 1" - fi - - _scratch_xfs_metadump $metadump_file -a -o $version - - SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - SCRATCH_DEV=$datadev _scratch_mount - SCRATCH_DEV=$datadev _check_scratch_fs - SCRATCH_DEV=$datadev _scratch_unmount - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - -verify_metadump_v2() -{ - local version="-v 2" - - _scratch_xfs_metadump $metadump_file -a -o $version - - # Metadump v2 files can contain contents dumped from an external log - # device. Use a temporary file to hold the log device contents restored - # from such a metadump file. - slogdev="" - if [[ -n $SCRATCH_LOGDEV ]]; then - slogdev=$TEST_DIR/log-image - fi - - SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \ - _scratch_xfs_mdrestore $metadump_file - - datadev=$(_create_loop_device $TEST_DIR/data-image) - - logdev="" - if [[ -s $slogdev ]]; then - logdev=$(_create_loop_device $slogdev) - fi - - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs - SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount - - if [[ -s $logdev ]]; then - _destroy_loop_device $logdev - logdev="" - rm -f $slogdev - fi - - _destroy_loop_device $datadev - datadev="" - rm -f $TEST_DIR/data-image -} - echo "Initialize and mount filesystem on flakey device" _init_flakey _load_flakey_table $FLAKEY_ALLOW_WRITES @@ -160,14 +93,7 @@ echo -n "Filesystem has a " _print_logstate echo "Create metadump file, restore it and check restored fs" - -if [[ $external_log == 0 ]]; then - verify_metadump_v1 $max_md_version -fi - -if [[ $max_md_version == 2 ]]; then - verify_metadump_v2 -fi +_verify_metadumps '-a -o' # Mount the fs to replay the contents from the dirty log. _scratch_mount From patchwork Thu Jan 25 19:05:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531527 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 208E8537E3; Thu, 25 Jan 2024 19:05:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209533; cv=none; b=QMqsgDArRJ7b1Ymx1Xks+gYsGGu8v4rYT14bxPjcFCIkjwd5NLLeUQURTSyVTCUsvkUg1XhSYQ6eDMQTIVdVYzpNZ8KJi8ky3175oty5O4GdVk3uQbE474WG7uInx3ijjsEPtHwlZEx0WnjeeFPRYXu8YcURDNXIuCBVUBs2zUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209533; c=relaxed/simple; bh=ZCFloK9HuTCJHwMc9UDEL6A7rdoZQY8gSpEeIR0fe5c=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DhP8T42AZU6Iibc8FywhTkBbrlPMP43zrv/AlBlcgdZ+xmH/myJe8s8rjp7xvDA1tMVdq8yWmTQGFae5zgmm5qnWm3+sIkhjZr2/kQvV8BZ/8vu/okvhLIBg3xYSnqkQk2QUvjbyJ0XUyZhoIcG9WP7c5pwxSzorlMN2UJKvwqY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=n5AbLLwV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="n5AbLLwV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E0728C433F1; Thu, 25 Jan 2024 19:05:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209533; bh=ZCFloK9HuTCJHwMc9UDEL6A7rdoZQY8gSpEeIR0fe5c=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=n5AbLLwVXh58GvO7VcoyqM9dgQurvTq9n4WowhcDqd/bSMtOkYooJH4C3eR3mls8c kgwy69Xf0wbitg4lTz6jB/d5HJiH2GoZu80rfseCUULOWajA+olsVH2hqXpTjZ6rQ6 U0X3wASdHvas3vfNb/28fAvCNT7WQ7NWVCtlRj7iWGnwC7B4dQ4CARrKwUumwQ2hON BIYsV4NYmnHhCSdZu3epohxflu1XVZnPWMl1yoemE6DL1e2rXvdYQZZLQJD452YAWz OrG0DGzMzS+0rcMSDxJHAh9ICEQQdw4k0iCFiAhzefz66SAPRtYX3kPlNCDez7OWd0 U4MBnZtEUiAsA== Date: Thu, 25 Jan 2024 11:05:32 -0800 Subject: [PATCH 06/10] xfs/{129,234,253,605}: disable metadump v1 testing with external devices From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924449.3283496.4305194198701650108.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The metadump v1 format does not support capturing content from log devices or realtime devices. Hence it does not make sense to test these scenarios. Create predicates to decide if we want to test a particular metadump format, then convert existing tests to check formats explicitly. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- common/xfs_metadump_tests | 25 ++++++++++++++++++++----- tests/xfs/605 | 9 --------- 2 files changed, 20 insertions(+), 14 deletions(-) diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests index dd3dec1fb4..fdc3a1fb10 100644 --- a/common/xfs_metadump_tests +++ b/common/xfs_metadump_tests @@ -23,6 +23,24 @@ _cleanup_verify_metadump() rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"* } +# Can xfs_metadump snapshot the fs metadata to a v1 metadump file? +_scratch_xfs_can_metadump_v1() +{ + # metadump v1 does not support log devices + [ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_LOGDEV" ] && return 1 + + # metadump v1 does not support realtime devices + [ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_RTDEV" ] && return 1 + + return 0 +} + +# Can xfs_metadump snapshot the fs metadata to a v2 metadump file? +_scratch_xfs_can_metadump_v2() +{ + test "$MAX_XFS_METADUMP_VERSION" -ge 2 +} + # Create a metadump in v1 format, restore it to fs image files, then mount the # images and fsck them. _verify_metadump_v1() @@ -115,9 +133,6 @@ _verify_metadump_v2() # Verify both metadump formats if possible _verify_metadumps() { - _verify_metadump_v1 "$@" - - if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then - _verify_metadump_v2 "$@" - fi + _scratch_xfs_can_metadump_v1 && _verify_metadump_v1 "$@" + _scratch_xfs_can_metadump_v2 && _verify_metadump_v2 "$@" } diff --git a/tests/xfs/605 b/tests/xfs/605 index af917f0f32..4b6ffcb2b4 100755 --- a/tests/xfs/605 +++ b/tests/xfs/605 @@ -40,15 +40,6 @@ testfile=${SCRATCH_MNT}/testfile echo "Format filesystem on scratch device" _scratch_mkfs >> $seqres.full 2>&1 -external_log=0 -if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then - external_log=1 -fi - -if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then - _notrun "metadump v1 does not support external log device" -fi - echo "Initialize and mount filesystem on flakey device" _init_flakey _load_flakey_table $FLAKEY_ALLOW_WRITES From patchwork Thu Jan 25 19:05:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531528 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41469130E25; Thu, 25 Jan 2024 19:05:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209549; cv=none; b=NjI56CzrrqrNlGilwuHfGRQ2kdhGkGaBikapEKQ572GcfK6VOxcYbC5WcRFWkh5Xa7IBy5Ogodkczq3iuC5hP9w3MNih9ozwwAI3QMZ8Mvkn6FyjWVpcEfd2A/Y+SZ6iofuT4NoBUHzvi5Qfik2d53qKHvBG8nY2Xc+ObYpLWWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209549; c=relaxed/simple; bh=XrdHFp1d6U1J9PVsu0/jiWqxSkzmBrGOy8c/7VH3fjM=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pMpXO1DJpUG3JzKJ5Z1cZcyQ1s+qgQUZ6SXY5zdXR3CKjo3QW4rCTZ7fhXQdcnlmIq1mUzh21v8MEmet0ujWVIixPUm48ApheugQ/o5YQzoISbyImDoYP8a9JXs5LGHUbbhq4AqjHRQF+tDAApzev0x1cCAWcczyMjrHdaaWaWg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HdnK9FcV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HdnK9FcV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 943BBC433C7; Thu, 25 Jan 2024 19:05:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209548; bh=XrdHFp1d6U1J9PVsu0/jiWqxSkzmBrGOy8c/7VH3fjM=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=HdnK9FcVstq3UvKd6p8+5VJrG7vJsvTyIx7qUldBWDjKTsWN99wrmYNvB5ol+nSCO rIutk7ljGb88RCkLjZgbbo2izZ46cmOShgo/L141LX//umHugxgvsVSDcOlx4bsD1C /5iZ9XBgogLKVSgtHY4XM6Xj71qQbfms4foorx7D3UT6getxz/lWKHQA53tdtmXVfc Twk2Qxb3va1MnoCV9JiEkSiVjCgSJTC/otOoMIaNsXNOH5KLPrq7zMWs8KKfYxM/we IJgnb8WC1Mkzfvd5PLARkcBpmNuokzcYYkmpwa72lWLKBz6O0YJTbn1Z5cq+Ni3Zkp 2TF7aD6YccwQw== Date: Thu, 25 Jan 2024 11:05:48 -0800 Subject: [PATCH 07/10] xfs/503: test metadump obfuscation, not progressbars From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924464.3283496.16784489077157560763.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The -g switch to xfs_metadump turns on progress reporting, but nothing in this test actually checks that it works. The -o switch turns off obfuscation, which is much more critical to support teams. Change this test to check -o and -ao instead of -g or -ag. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- tests/xfs/503 | 10 +++++----- tests/xfs/503.out | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/xfs/503 b/tests/xfs/503 index ff6b344a9c..79bbbdd26d 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -47,16 +47,16 @@ metadump_file=$testdir/scratch.md copy_file=$testdir/copy.img echo "metadump and mdrestore" -_verify_metadumps '-a -o' +_verify_metadumps echo "metadump a and mdrestore" _verify_metadumps '-a' -echo "metadump g and mdrestore" -_verify_metadumps '-g' >> $seqres.full +echo "metadump o and mdrestore" +_verify_metadumps '-o' -echo "metadump ag and mdrestore" -_verify_metadumps '-a -g' >> $seqres.full +echo "metadump ao and mdrestore" +_verify_metadumps '-a -o' echo copy $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full diff --git a/tests/xfs/503.out b/tests/xfs/503.out index 496f2516e4..5e7488456d 100644 --- a/tests/xfs/503.out +++ b/tests/xfs/503.out @@ -2,7 +2,7 @@ QA output created by 503 Format and populate metadump and mdrestore metadump a and mdrestore -metadump g and mdrestore -metadump ag and mdrestore +metadump o and mdrestore +metadump ao and mdrestore copy recopy From patchwork Thu Jan 25 19:06:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531529 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 867506A005; Thu, 25 Jan 2024 19:06:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209564; cv=none; b=t6Rpq5klaAEWxpfA/XW2pNBoiqX1Hd3hmiwZL623fQuEGci8C08G+j17D8702zfx0iKqYV8omkLYpHp7f3JnPqopW8hRRXWRQEEc6L1YrHH2qGJ3k3sDFPZ0IX5ZhcU8zBAzE7xxVh/frD6SOn6qGhLGQlaFTws2DbXCG24CQMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209564; c=relaxed/simple; bh=1Zri91UrAXgUr22zpBTj1DjMFMXJP3oYQ3aTzoErS3E=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FMWPtwF0FY3Y/boEkcF6w3Y6Lk11i580GbrAktIaluYa00fF4sNtE0jPzC5/3qWJM5go/ibLqz9VJwlh0j+ii8jdEvlIJ/NApvhL/fM+OrPOxd5NfaIFbDaYQtSc9HiXCJnjWt5lCL5mHaz2ffRmHQ3/k7DLqcfOv7xm/HuroBo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y1kKQM+i; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y1kKQM+i" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4DF4FC433C7; Thu, 25 Jan 2024 19:06:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209564; bh=1Zri91UrAXgUr22zpBTj1DjMFMXJP3oYQ3aTzoErS3E=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Y1kKQM+i9IcQ6BaN64cib2MbAxA//1D6uVD2c7TDUbxRUxaReAtoOVu+9frHbtuMW GntJ+gQcuyTjd4rktr5r6rVaAELfl+KzBLlh5+FDvPUHQnCEmgrwKa81BQf9xsrtb8 nRqB6ewIOMOPQODzZsjf5oceQFK0hU2FXbO4rVfWPcgtaeH2o/QEo0OrlxWlyZe6I3 0PP/cXSioDnTQwWDbUiawluw9wgeDolcoeVjWynAceBywl/UGdPiYQ/dJdxdW2J+9n JLbURqknYhXmwVIdR+kRuUYoWkWWiYlE4753Ri6aB7cQF/k5q07dJV0v7By2q6mSn6 WQHD1rrzcIx5w== Date: Thu, 25 Jan 2024 11:06:03 -0800 Subject: [PATCH 08/10] xfs/503: split copy and metadump into two tests From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924478.3283496.11965906815443674241.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong This test examines the behavior of xfs_copy and xfs_metadump. Metadump now supports capturing external log contents, but copy does not. Split the test into two to improve coverage on multidevice filesystems. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- tests/xfs/1876 | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1876.out | 4 ++++ tests/xfs/503 | 17 +++------------- tests/xfs/503.out | 2 -- 4 files changed, 61 insertions(+), 16 deletions(-) create mode 100755 tests/xfs/1876 create mode 100755 tests/xfs/1876.out diff --git a/tests/xfs/1876 b/tests/xfs/1876 new file mode 100755 index 0000000000..feeb82fca0 --- /dev/null +++ b/tests/xfs/1876 @@ -0,0 +1,54 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2019 Oracle, Inc. All Rights Reserved. +# +# FS QA Test No. 1876 +# +# Populate a XFS filesystem and ensure that xfs_copy works properly. +# +. ./common/preamble +_begin_fstest auto copy + +_register_cleanup "_cleanup" BUS + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -rf $tmp.* $testdir +} + +# Import common functions. +. ./common/filter +. ./common/populate + +testdir=$TEST_DIR/test-$seq + +# real QA test starts here +_supported_fs xfs + +_require_xfs_copy +_require_scratch_nocheck +_require_populate_commands +_xfs_skip_online_rebuild +_xfs_skip_offline_rebuild + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +mkdir -p $testdir +copy_file=$testdir/copy.img + +echo copy +$XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full +_check_scratch_fs $copy_file + +echo recopy +$XFS_COPY_PROG $copy_file $SCRATCH_DEV >> $seqres.full +_scratch_mount +_check_scratch_fs +_scratch_unmount + +# success, all done +status=0 +exit diff --git a/tests/xfs/1876.out b/tests/xfs/1876.out new file mode 100755 index 0000000000..0bf8e156c0 --- /dev/null +++ b/tests/xfs/1876.out @@ -0,0 +1,4 @@ +QA output created by 1876 +Format and populate +copy +recopy diff --git a/tests/xfs/503 b/tests/xfs/503 index 79bbbdd26d..606fcbdcc4 100755 --- a/tests/xfs/503 +++ b/tests/xfs/503 @@ -4,11 +4,11 @@ # # FS QA Test No. 503 # -# Populate a XFS filesystem and ensure that metadump, mdrestore, and copy -# all work properly. +# Populate a XFS filesystem and ensure that metadump and mdrestore all work +# properly. # . ./common/preamble -_begin_fstest auto copy metadump +_begin_fstest auto metadump _register_cleanup "_cleanup" BUS @@ -32,7 +32,6 @@ _supported_fs xfs _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore" _require_loop -_require_xfs_copy _require_scratch_nocheck _require_populate_commands _xfs_skip_online_rebuild @@ -58,16 +57,6 @@ _verify_metadumps '-o' echo "metadump ao and mdrestore" _verify_metadumps '-a -o' -echo copy -$XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full -_check_scratch_fs $copy_file - -echo recopy -$XFS_COPY_PROG $copy_file $SCRATCH_DEV >> $seqres.full -_scratch_mount -_check_scratch_fs -_scratch_unmount - # success, all done status=0 exit diff --git a/tests/xfs/503.out b/tests/xfs/503.out index 5e7488456d..7f3d3a5f24 100644 --- a/tests/xfs/503.out +++ b/tests/xfs/503.out @@ -4,5 +4,3 @@ metadump and mdrestore metadump a and mdrestore metadump o and mdrestore metadump ao and mdrestore -copy -recopy From patchwork Thu Jan 25 19:06:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531530 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 806E76A005; Thu, 25 Jan 2024 19:06:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209580; cv=none; b=jVygJ1GqYDbRSmViL0gvdPWhdwanVlb/M+3p6PYnMWuwdaQ5QHUn1l6pCZCKfTKFKfaAe5HF+S0HlsHbpDLfeKJXsm4haESw64lSYuc3emwXDuE62KTFMb9A2jMHt6cXHX+fwtTiWPOUcXnub9YEUS6fmV589I++AuHAHM4/NOs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209580; c=relaxed/simple; bh=lujOX5jDxrf+xAdwjPmZfVAFc2qNESt/SPyuOqHDCI0=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TBkvoIDl9CryclD5LxvDihmSKuf6JqWgtX/3D+32gByN0LfXM2bYxKeeIya817L5Xq+32C79LoEFWbg1UP4q6wTn8wxX+2RSsKPrNR9r1aIIvWLYA0UhJTqPcIji6csvNEdQpqk/iKT1nvyMkucfXWYQyANFmDkQENkdY9Joycg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=d8gL+SvJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="d8gL+SvJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02D42C433F1; Thu, 25 Jan 2024 19:06:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209580; bh=lujOX5jDxrf+xAdwjPmZfVAFc2qNESt/SPyuOqHDCI0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=d8gL+SvJUU+dLSlL8n8mX7X4vQisI5o9F34qkugVBU5STM+F39/9pRbQXM/rw8w7N aTdRTEgLsM8JvlkgeSjMN/0HQ7ZiFhztLoCeopskvrVad3uuSoKvcK2ZHXumIYeRJx uUHKUwGdtJarDbpwQq+1S8asPWOyMMUulzz9n4PdqrcpzVdjGGj5g1BB9CnDpnHmx2 Dc1GxS2wP/STxJwijBWosS7O7Ml0EfZ+G4xmzmvJIGwZ7awfwZWrk8hof+eKTDKRIH u0CcJp8k3k14AemaZ6Enb2tXdiBDApWORVHaFjjPA90RhQ+JnqDHf44CUm8R6uJfSb af8+oIWv72Njg== Date: Thu, 25 Jan 2024 11:06:19 -0800 Subject: [PATCH 09/10] common/xfs: only pass -l in _xfs_mdrestore for v2 metadumps From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924493.3283496.11650772421388432291.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong fstests has a weird history with external log devices -- prior to the introduction of metadump v2, a dump/restore cycle would leave an external log unaltered, and most tests worked just fine. Were those tests ignorant? Or did they pass intentionally? Either way, we don't want to pass -l to xfs_mdrestore just because we have an external log, because that switch is new and causes regressions when testing with xfsprogs from before 6.5. Signed-off-by: Darrick J. Wong Reviewed-by: Christoph Hellwig --- common/xfs | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/common/xfs b/common/xfs index 6a48960a7f..65b509691b 100644 --- a/common/xfs +++ b/common/xfs @@ -689,12 +689,25 @@ _xfs_metadump() { return $res } +# What is the version of this metadump file? +_xfs_metadumpfile_version() { + local file="$1" + local magic + + magic="$($XFS_IO_PROG -c 'pread -q -v 0 4' "$file")" + case "$magic" in + "00000000: 58 4d 44 32 XMD2") echo 2;; + "00000000: 58 46 53 4d XFSM") echo 1;; + esac +} + _xfs_mdrestore() { local metadump="$1" local device="$2" local logdev="$3" shift; shift; shift local options="$@" + local dumpfile_ver # If we're configured for compressed dumps and there isn't already an # uncompressed dump, see if we can use DUMP_COMPRESSOR to decompress @@ -705,8 +718,18 @@ _xfs_mdrestore() { done fi test -r "$metadump" || return 1 + dumpfile_ver="$(_xfs_metadumpfile_version "$metadump")" - if [ "$logdev" != "none" ]; then + if [ "$logdev" != "none" ] && [[ $dumpfile_ver > 1 ]]; then + # metadump and mdrestore began capturing and restoring the + # contents of external log devices with the addition of the + # metadump v2 format. Hence it only makes sense to specify -l + # here if the dump file itself is in v2 format. + # + # With a v1 metadump, the log device is not changed by the dump + # and restore process. Historically, fstests either didn't + # notice or _notrun themselves when external logs were in use. + # Don't break that for people testing with xfsprogs < 6.5. options="$options -l $logdev" fi From patchwork Thu Jan 25 19:06:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13531531 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 523EC6A005; Thu, 25 Jan 2024 19:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209596; cv=none; b=hELdXL6E1pQmAKiS9eUOCI+uqjdPNEq+YLvjkOCc4r4bNUpBTtr9hHLo+iXxFAWO5Fl51HWwPSgmBRBYvBD5YlQQiQbN7DjvfaIbMjQZcmFGK/it83k7I92l35RbeS2tp9EwUA5xbNBWzf4WOlmd/FiG2KoT3WTnJW+Z0sc3YGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706209596; c=relaxed/simple; bh=sLM2uLqDZXHu+g0PvRoJd31ZQQVQUao6m22FUiAOJZk=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tXQDqrauo8ulr/1QvMlx2k0xe+J4CjLxdF7KVdfY6r/I/ds33Boy8OadIMwcGxDxYyY8AifclMjeQry+CeyGRl7NZwbVyTp67VvovVYKS4nz7ZmnGuVjW0F2mKiYp47MVVT4Ay+PHvFPLMuNXJUzufBMcQyYf/TZBow7yLJQdEM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oUZRbtUX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oUZRbtUX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB83FC433C7; Thu, 25 Jan 2024 19:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706209595; bh=sLM2uLqDZXHu+g0PvRoJd31ZQQVQUao6m22FUiAOJZk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=oUZRbtUX35jt4sZNA7VPOFMypyDxBFIBFdW0Ap1FF8RmXgEiM/lgBO7h8hFJZLBnO ouXdCSUUpWSSa9jl8/ULpsmaTizGWTSSTK1BxaWJMipkH8Ep33K43Pz2dTEYXZ01Wi sNEoDgGIt3NdUfOCoTKeb6HN1jlLUnMyMM4GHal5lmeFlvWR1GNVw7AljZbgIShgS0 MCGBw5YvqgmyaauykF7TXNAkuNFADX6Or+AcPPG0ZrlaWDcaZig+5dCDeUBGC8LNmc 99UGvrsSc3zRfhgSoAveLeVs1zVNRgkzZ60SMEDTorYA5NBjUP7XQHoYJZv+4izlF8 lL9dv/zVBTmlw== Date: Thu, 25 Jan 2024 11:06:35 -0800 Subject: [PATCH 10/10] xfs/122: fix for xfs_attr_shortform removal in 6.8 From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170620924507.3283496.17636943697618850238.stgit@frogsfrogsfrogs> In-Reply-To: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> References: <170620924356.3283496.1996184171093691313.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The xfs_attr_shortform struct (with multiple flexarrays) was removed in 6.8. Check the two surviving structures (the attr sf header and entry) instead. Signed-off-by: Darrick J. Wong Reviewed-by: Andrey Albershteyn Reviewed-by: Christoph Hellwig --- tests/xfs/122.out | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index 89f7b735b0..067a0ec76b 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -62,6 +62,8 @@ sizeof(struct xfs_agfl) = 36 sizeof(struct xfs_attr3_leaf_hdr) = 80 sizeof(struct xfs_attr3_leafblock) = 88 sizeof(struct xfs_attr3_rmt_hdr) = 56 +sizeof(struct xfs_attr_sf_entry) = 3 +sizeof(struct xfs_attr_sf_hdr) = 4 sizeof(struct xfs_attr_shortform) = 8 sizeof(struct xfs_attrd_log_format) = 16 sizeof(struct xfs_attri_log_format) = 40