From patchwork Wed Dec 27 14:01:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508557 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B54EC7F9; Mon, 1 Jan 2024 01:01:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TkYG3HND" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D4A7C433C8; Mon, 1 Jan 2024 01:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070899; bh=P1eEzr0BokPFcfCFa/BXeWCAtFOYWxo7JQ0feEnXhY8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=TkYG3HND9iWUJFpN4iytta16UrTrA6Dr2UCt8U7yl3F0hDzevShP3C5KrzBoZdrX2 LnjAtCN8hp83ojDQBZNZs6pMQGxnCFxIYRVLQHmahB6HirKfhmgHSzc2LxH3u89wH7 carvhScT+oUtGstGi+ROF44XirzUcoPei2jqmyvK1fynsGKhP9wfMbGPCTrxIRbj+w AV2fVzbqCxhd2ouCTrMqdiFsYQ0G7mxh6rqziV3Q60p4VBmlwKjFwqZ0aBEOAX26BF jRL4becGKD4W4Pi9O720bWzoO1nqoUs4G9wMnXDYef3sleA/0wgMEmB/D147GYIopd A1LJs0cV5FZKA== Date: Sun, 31 Dec 2023 17:01:38 +9900 Subject: [PATCH 1/9] xfs/122: update fields for realtime reflink From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032029.1827358.6809381410014348048.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Add a few more ondisk structures for realtime reflink. Signed-off-by: Darrick J. Wong --- tests/xfs/122.out | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index f5621cac12..9c2d2fffb4 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -120,6 +120,9 @@ sizeof(struct xfs_rmap_key) = 20 sizeof(struct xfs_rmap_rec) = 24 sizeof(struct xfs_rtbuf_blkinfo) = 48 sizeof(struct xfs_rtgroup_geometry) = 128 +sizeof(struct xfs_rtrefcount_key) = 4 +sizeof(struct xfs_rtrefcount_rec) = 12 +sizeof(struct xfs_rtrefcount_root) = 4 sizeof(struct xfs_rtrmap_key) = 20 sizeof(struct xfs_rtrmap_rec) = 24 sizeof(struct xfs_rtrmap_root) = 4 From patchwork Wed Dec 27 14:01:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508558 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01BCF7EE; Mon, 1 Jan 2024 01:01:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZJdwqIa0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C581FC433C7; Mon, 1 Jan 2024 01:01:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070914; bh=jKlY8tEzhfFHB9ElAuOkFtT99EZtNEOcbHZvrOBYPJ8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ZJdwqIa0W9U+0cZOn0EQlFH7NjTNWDQgEilU662z8d5hCpnTzjyjsD3SKH01YeTAy 3+9ETiuSGL2g4TokYcy3UQQO5dq4JoUQ/gWCevUnHDV7uD/UkceKyP9XJ25DMbAoXt DbNNj4s2xyL+VmDoUgsrDWbD2ooYcl8ajw428YVrOjvcHMbf0/Gj6sdWhb7A6Qr+Wl Y5hEA0NHbLTvpTdxOAyA4JthKPpliUTol/J1F54FNWSSiPQJbMO7hYtYFgZiXAZ105 Vj9GS/UVdOJUsLZsb0IA1/2bGrUM1otucFwuB8vSZrB4CHeJN3r/cYqAutN4n7Ewih rkOu5eAHf2p6A== Date: Sun, 31 Dec 2023 17:01:54 +9900 Subject: [PATCH 2/9] common/populate: create realtime refcount btree From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032043.1827358.8872006071415407711.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Populate a realtime refcount btree when we're creating a sample fs. Signed-off-by: Darrick J. Wong --- common/populate | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/common/populate b/common/populate index 84ff9304e3..1e51eedddc 100644 --- a/common/populate +++ b/common/populate @@ -438,16 +438,30 @@ _scratch_xfs_populate() { local dir="${SCRATCH_MNT}/INOBT" __populate_create_dir "${dir}" "${nr}" true --file-pct 100 - # Reverse-mapping btree + is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")" is_rmapbt="$(_xfs_has_feature "$SCRATCH_MNT" rmapbt -v)" + is_reflink="$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)" + + # Reverse-mapping btree if [ $is_rmapbt -gt 0 ]; then echo "+ rmapbt btree" nr="$((blksz * 2 / 24))" __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RMAPBT" fi + # Realtime Reference-count btree comes before the rtrmapbt so that + # the refcount entries are created in rtgroup 0. + if [ $is_reflink -gt 0 ] && [ $is_rt -gt 0 ]; then + echo "+ rtreflink btree" + rt_blksz=$(_xfs_get_rtextsize "$SCRATCH_MNT") + nr="$((rt_blksz * 2 / 12))" + $XFS_IO_PROG -R -f -c 'truncate 0' "${SCRATCH_MNT}/RTREFCOUNTBT" + __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RTREFCOUNTBT" + $XFS_IO_PROG -R -f -c 'truncate 0' "${SCRATCH_MNT}/RTREFCOUNTBT2" + cp --reflink=always "${SCRATCH_MNT}/RTREFCOUNTBT" "${SCRATCH_MNT}/RTREFCOUNTBT2" + fi + # Realtime Reverse-mapping btree - is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")" if [ $is_rmapbt -gt 0 ] && [ $is_rt -gt 0 ]; then echo "+ rtrmapbt btree" nr="$((blksz * 2 / 24))" @@ -456,7 +470,6 @@ _scratch_xfs_populate() { fi # Reference-count btree - is_reflink="$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)" if [ $is_reflink -gt 0 ]; then echo "+ reflink btree" nr="$((blksz * 2 / 12))" @@ -512,6 +525,7 @@ _scratch_xfs_populate() { __populate_fragment_file "${SCRATCH_MNT}/RMAPBT" __populate_fragment_file "${SCRATCH_MNT}/RTRMAPBT" __populate_fragment_file "${SCRATCH_MNT}/REFCOUNTBT" + __populate_fragment_file "${SCRATCH_MNT}/RTREFCOUNTBT" umount "${SCRATCH_MNT}" } @@ -753,6 +767,10 @@ __populate_check_xfs_rgbtree_height() { path_format="/realtime/%u.rmap" bt_prefix="u3.rtrmapbt" ;; + "refcnt") + path_format="/realtime/%u.refcount" + bt_prefix="u3.rtrefcbt" + ;; *) _fail "Don't know about rt btree ${bt_type}" ;; @@ -827,6 +845,8 @@ _scratch_xfs_populate_check() { test $is_reflink -ne 0 && __populate_check_xfs_agbtree_height "refcnt" test $is_rmapbt -ne 0 && test $is_rt -gt 0 && \ __populate_check_xfs_rgbtree_height "rmap" + test $is_reflink -ne 0 && test $is_rt -gt 0 && \ + __populate_check_xfs_rgbtree_height "refcnt" } # Check data fork format of ext4 file From patchwork Wed Dec 27 14:02:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508559 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85B167ED; Mon, 1 Jan 2024 01:02:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RwK1p1G4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 581A8C433C7; Mon, 1 Jan 2024 01:02:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070930; bh=LaZlLLEP2L/fWf6Rk4ZkhLdi5hc9Fr0Bj0OQjIb0WsM=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=RwK1p1G4gDJILqFxETivP/0lW1obfrpyxudKZ3kThDB8uzQAY1yCf7YMsHoXeAbrl E3zsVd0XLPGe3ur0XNcEV6yPqDsWP7acnAKSY+t5rBshGb2wnJQAEx7JgEspOZNtFR XCWcCOn1vqcmJmB4BBnVOBiZT/mPcc20OPEyQ31m4pFm4wYB5HEx1JMVVGzBCN3Ce6 Jx170o0vNkTtEUsetovSg95awdWPG0L9W1zyFKTn6VnA7Y58dmdSP5htsAzmA5t7BV 3owI63ANGYeKz6INBB8N/mJuyvRIR39zYgyV2p78xQAFTdicz/tSGMJZeTLha8nYgO v9zfpP0/dxHsg== Date: Sun, 31 Dec 2023 17:02:09 +9900 Subject: [PATCH 3/9] xfs: create fuzz tests for the realtime refcount btree From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032055.1827358.9623496768181487478.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Create fuzz tests for the realtime refcount btree record and key/ptr blocks. Signed-off-by: Darrick J. Wong --- common/xfs | 4 ++++ tests/xfs/1538 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/xfs/1538.out | 4 ++++ tests/xfs/1539 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/xfs/1539.out | 4 ++++ tests/xfs/1540 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/xfs/1540.out | 4 ++++ tests/xfs/1541 | 42 ++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1541.out | 4 ++++ tests/xfs/1542 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/xfs/1542.out | 4 ++++ tests/xfs/1543 | 40 ++++++++++++++++++++++++++++++++++++++++ tests/xfs/1543.out | 4 ++++ tests/xfs/1544 | 40 ++++++++++++++++++++++++++++++++++++++++ tests/xfs/1544.out | 4 ++++ tests/xfs/1545 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/xfs/1545.out | 4 ++++ 17 files changed, 363 insertions(+) create mode 100755 tests/xfs/1538 create mode 100644 tests/xfs/1538.out create mode 100755 tests/xfs/1539 create mode 100644 tests/xfs/1539.out create mode 100755 tests/xfs/1540 create mode 100644 tests/xfs/1540.out create mode 100755 tests/xfs/1541 create mode 100644 tests/xfs/1541.out create mode 100755 tests/xfs/1542 create mode 100644 tests/xfs/1542.out create mode 100755 tests/xfs/1543 create mode 100644 tests/xfs/1543.out create mode 100755 tests/xfs/1544 create mode 100644 tests/xfs/1544.out create mode 100755 tests/xfs/1545 create mode 100644 tests/xfs/1545.out diff --git a/common/xfs b/common/xfs index b8dd8d4a40..aab04bfb18 100644 --- a/common/xfs +++ b/common/xfs @@ -1929,6 +1929,10 @@ _scratch_xfs_find_rgbtree_height() { path_format="/realtime/%u.rmap" bt_prefix="u3.rtrmapbt" ;; + "refcnt") + path_format="/realtime/%u.refcount" + bt_prefix="u3.rtrefcbt" + ;; *) _fail "Don't know about rt btree ${bt_type}" ;; diff --git a/tests/xfs/1538 b/tests/xfs/1538 new file mode 100755 index 0000000000..2bd630fdf1 --- /dev/null +++ b/tests/xfs/1538 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1538 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt record field. +# Use xfs_scrub to fix the corruption. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_online_repair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" +inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path") + +echo "Fuzz rtrefcountbt recs" +_scratch_xfs_fuzz_metadata '' 'online' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full +echo "Done fuzzing rtrefcountbt recs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1538.out b/tests/xfs/1538.out new file mode 100644 index 0000000000..968cfd6ef9 --- /dev/null +++ b/tests/xfs/1538.out @@ -0,0 +1,4 @@ +QA output created by 1538 +Format and populate +Fuzz rtrefcountbt recs +Done fuzzing rtrefcountbt recs diff --git a/tests/xfs/1539 b/tests/xfs/1539 new file mode 100755 index 0000000000..8086943211 --- /dev/null +++ b/tests/xfs/1539 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1539 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt record field. +# Use xfs_repair to fix the corruption. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_repair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" +inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path") + +echo "Fuzz rtrefcountbt recs" +_scratch_xfs_fuzz_metadata '' 'offline' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full +echo "Done fuzzing rtrefcountbt recs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1539.out b/tests/xfs/1539.out new file mode 100644 index 0000000000..aa3a963dc2 --- /dev/null +++ b/tests/xfs/1539.out @@ -0,0 +1,4 @@ +QA output created by 1539 +Format and populate +Fuzz rtrefcountbt recs +Done fuzzing rtrefcountbt recs diff --git a/tests/xfs/1540 b/tests/xfs/1540 new file mode 100755 index 0000000000..ab8dee2a02 --- /dev/null +++ b/tests/xfs/1540 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1540 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt record field. +# Do not fix the filesystem, to test metadata verifiers. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_norepair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" +inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path") + +echo "Fuzz rtrefcountbt recs" +_scratch_xfs_fuzz_metadata '' 'none' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full +echo "Done fuzzing rtrefcountbt recs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1540.out b/tests/xfs/1540.out new file mode 100644 index 0000000000..37f3311837 --- /dev/null +++ b/tests/xfs/1540.out @@ -0,0 +1,4 @@ +QA output created by 1540 +Format and populate +Fuzz rtrefcountbt recs +Done fuzzing rtrefcountbt recs diff --git a/tests/xfs/1541 b/tests/xfs/1541 new file mode 100755 index 0000000000..18312456f4 --- /dev/null +++ b/tests/xfs/1541 @@ -0,0 +1,42 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1541 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt record field. +# Try online repair and, if necessary, offline repair, +# to test the most likely usage pattern. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_bothrepair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" +inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path") + +echo "Fuzz rtrefcountbt recs" +_scratch_xfs_fuzz_metadata '' 'both' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full +echo "Done fuzzing rtrefcountbt recs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1541.out b/tests/xfs/1541.out new file mode 100644 index 0000000000..35a9b73471 --- /dev/null +++ b/tests/xfs/1541.out @@ -0,0 +1,4 @@ +QA output created by 1541 +Format and populate +Fuzz rtrefcountbt recs +Done fuzzing rtrefcountbt recs diff --git a/tests/xfs/1542 b/tests/xfs/1542 new file mode 100755 index 0000000000..9246a31a31 --- /dev/null +++ b/tests/xfs/1542 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1542 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field. +# Use xfs_scrub to fix the corruption. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_online_repair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" +inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path") + +echo "Fuzz rtrefcountbt keyptrs" +_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'online' "path -m $path" >> $seqres.full +echo "Done fuzzing rtrefcountbt keyptrs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1542.out b/tests/xfs/1542.out new file mode 100644 index 0000000000..55d820b4b1 --- /dev/null +++ b/tests/xfs/1542.out @@ -0,0 +1,4 @@ +QA output created by 1542 +Format and populate +Fuzz rtrefcountbt keyptrs +Done fuzzing rtrefcountbt keyptrs diff --git a/tests/xfs/1543 b/tests/xfs/1543 new file mode 100755 index 0000000000..38afae106a --- /dev/null +++ b/tests/xfs/1543 @@ -0,0 +1,40 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1543 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field. +# Use xfs_repair to fix the corruption. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_repair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" + +echo "Fuzz rtrefcountbt keyptrs" +_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'offline' "path -m $path" >> $seqres.full +echo "Done fuzzing rtrefcountbt keyptrs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1543.out b/tests/xfs/1543.out new file mode 100644 index 0000000000..e7afa10744 --- /dev/null +++ b/tests/xfs/1543.out @@ -0,0 +1,4 @@ +QA output created by 1543 +Format and populate +Fuzz rtrefcountbt keyptrs +Done fuzzing rtrefcountbt keyptrs diff --git a/tests/xfs/1544 b/tests/xfs/1544 new file mode 100755 index 0000000000..86b11b1955 --- /dev/null +++ b/tests/xfs/1544 @@ -0,0 +1,40 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1544 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field. +# Do not fix the filesystem, to test metadata verifiers. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_norepair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" + +echo "Fuzz rtrefcountbt keyptrs" +_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'none' "path -m $path" >> $seqres.full +echo "Done fuzzing rtrefcountbt keyptrs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1544.out b/tests/xfs/1544.out new file mode 100644 index 0000000000..b39532c160 --- /dev/null +++ b/tests/xfs/1544.out @@ -0,0 +1,4 @@ +QA output created by 1544 +Format and populate +Fuzz rtrefcountbt keyptrs +Done fuzzing rtrefcountbt keyptrs diff --git a/tests/xfs/1545 b/tests/xfs/1545 new file mode 100755 index 0000000000..1dbe03506b --- /dev/null +++ b/tests/xfs/1545 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1545 +# +# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field. +# Try online repair and, if necessary, offline repair, +# to test the most likely usage pattern. + +. ./common/preamble +_begin_fstest dangerous_fuzzers dangerous_bothrepair realtime + +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/populate +. ./common/fuzzy +. ./common/reflink + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch_reflink +_require_scratch_xfs_fuzz_fields +_disable_dmesg_check + +echo "Format and populate" +_scratch_populate_cached nofill > $seqres.full 2>&1 + +path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \ + _fail "could not find two-level rtrefcountbt" + +echo "Fuzz rtrefcountbt keyptrs" +_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'both' "path -m $path" >> $seqres.full +echo "Done fuzzing rtrefcountbt keyptrs" + +# success, all done +status=0 +exit diff --git a/tests/xfs/1545.out b/tests/xfs/1545.out new file mode 100644 index 0000000000..982a0d64df --- /dev/null +++ b/tests/xfs/1545.out @@ -0,0 +1,4 @@ +QA output created by 1545 +Format and populate +Fuzz rtrefcountbt keyptrs +Done fuzzing rtrefcountbt keyptrs From patchwork Wed Dec 27 14:02:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508560 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F0257EE; Mon, 1 Jan 2024 01:02:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Fv3IyO+W" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3CEAC433C7; Mon, 1 Jan 2024 01:02:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070946; bh=H7LrtpjWIjCPTWP+1OQCQlLph2koDrat4UkLQ1WobcU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Fv3IyO+WDtpGZHqujGIsfOCJ6hsT6vlHVG3nFKbvGazvVO5f3h5gyYncKzjeQbIXK LdhMarQEjkXdo1fgJqN6y6Axsc6p9RIuJc+MQb2qHmg7MxfFL79NJoNbUoJKhcZHhu g1RWWgwW4s23pPrCUt1HKSRoB9IvCfCJ6GLzbqtdNY9XrB1EQfOo2X1JTWHKX9/0SV Hhyf3oSCGG7n7I0jPRA9w9w7T5R2DXWQaNtK/n4jco3/35PTTNUYLU3w8XsQyT3Z7h 0iNUbqmCn4EwGYTLn/5lHT7oxM9qCOShRU78GEpMmDRIuk79qZs6cmjZF/YiSdh24i 59hZe5DeptnSg== Date: Sun, 31 Dec 2023 17:02:25 +9900 Subject: [PATCH 4/9] xfs/27[24]: adapt for checking files on the realtime volume From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032069.1827358.7647888380084910379.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Adapt both tests to behave properly if the two files being tested are on the realtime volume. Signed-off-by: Darrick J. Wong --- tests/xfs/272 | 40 +++++++++++++++++++++++++------------ tests/xfs/274 | 62 ++++++++++++++++++++++++++++++++++++++++----------------- 2 files changed, 70 insertions(+), 32 deletions(-) diff --git a/tests/xfs/272 b/tests/xfs/272 index d5f3a74177..edc5b16967 100755 --- a/tests/xfs/272 +++ b/tests/xfs/272 @@ -40,26 +40,40 @@ $here/src/punch-alternating $SCRATCH_MNT/urk >> $seqres.full ino=$(stat -c '%i' $SCRATCH_MNT/urk) echo "Get fsmap" | tee -a $seqres.full -$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT | tr '[]()' ' ' > $TEST_DIR/fsmap +cat $TEST_DIR/fsmap >> $seqres.full echo "Get bmap" | tee -a $seqres.full -$XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/urk >> $seqres.full $XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/urk | grep '^[[:space:]]*[0-9]*:' | grep -v 'hole' | tr '[]()' ' ' > $TEST_DIR/bmap +cat $TEST_DIR/bmap >> $seqres.full echo "Check bmap and fsmap" | tee -a $seqres.full -cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do - qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total}(| [01]*)$" - echo "${qstr}" >> $seqres.full - grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full - found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) - test $found -eq 1 || echo "Unexpected output for offset ${offrange}." -done +if $XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT/urk | grep -q realtime; then + # file on rt volume + cat $TEST_DIR/bmap | while read ext offrange colon rtblockrange total crap; do + qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${rtblockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${total}(| [01]*)$" + echo "${qstr}" >> $seqres.full + grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full + found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) + test $found -eq 1 || echo "Unexpected output for offset ${offrange}." + done -echo "Check device field of FS metadata and regular file" -data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}') -rt_dev=$(grep "${ino}[[:space:]]*[0-9]*\.\.[0-9]*" $TEST_DIR/fsmap | head -n 1 | awk '{print $2}') -test "${data_dev}" = "${rt_dev}" || echo "data ${data_dev} realtime ${rt_dev}?" + echo "Check device field of FS metadata and regular file" +else + # file on data volume + cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do + qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total}(| [01]*)$" + echo "${qstr}" >> $seqres.full + grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full + found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) + test $found -eq 1 || echo "Unexpected output for offset ${offrange}." + done + + echo "Check device field of FS metadata and regular file" + data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}') + rt_dev=$(grep "${ino}[[:space:]]*[0-9]*\.\.[0-9]*" $TEST_DIR/fsmap | head -n 1 | awk '{print $2}') + test "${data_dev}" = "${rt_dev}" || echo "data ${data_dev} realtime ${rt_dev}?" +fi # success, all done status=0 diff --git a/tests/xfs/274 b/tests/xfs/274 index cd483d77bc..f16b49fc1c 100755 --- a/tests/xfs/274 +++ b/tests/xfs/274 @@ -40,34 +40,58 @@ _cp_reflink $SCRATCH_MNT/f1 $SCRATCH_MNT/f2 ino=$(stat -c '%i' $SCRATCH_MNT/f1) echo "Get fsmap" | tee -a $seqres.full -$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT | tr '[]()' ' ' > $TEST_DIR/fsmap +cat $TEST_DIR/fsmap >> $seqres.full echo "Get f1 bmap" | tee -a $seqres.full -$XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f1 >> $seqres.full $XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f1 | grep '^[[:space:]]*[0-9]*:' | grep -v 'hole' | tr '[]()' ' ' > $TEST_DIR/bmap +cat $TEST_DIR/bmap >> $seqres.full -echo "Check f1 bmap and fsmap" | tee -a $seqres.full -cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do - qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 010[01]{4}$" - echo "${qstr}" >> $seqres.full - grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full - found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) - test $found -eq 1 || echo "Unexpected output for offset ${offrange}." -done +if _xfs_is_realtime_file $SCRATCH_MNT/f1 && ! _xfs_has_feature $SCRATCH_MNT rtgroups; then + # file on rt volume + echo "Check f1 bmap and fsmap" | tee -a $seqres.full + cat $TEST_DIR/bmap | while read ext offrange colon rtblockrange total crap; do + qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${rtblockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${total} 010[01]{4}$" + echo "${qstr}" >> $seqres.full + grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full + found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) + test $found -eq 1 || echo "Unexpected output for offset ${offrange}." + done +else + # file on data volume + echo "Check f1 bmap and fsmap" | tee -a $seqres.full + cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do + qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 010[01]{4}$" + echo "${qstr}" >> $seqres.full + grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full + found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) + test $found -eq 1 || echo "Unexpected output for offset ${offrange}." + done +fi echo "Get f2 bmap" | tee -a $seqres.full -$XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f2 >> $seqres.full $XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f2 | grep '^[[:space:]]*[0-9]*:' | grep -v 'hole' | tr '[]()' ' ' > $TEST_DIR/bmap +cat $TEST_DIR/bmap >> $seqres.full -echo "Check f2 bmap and fsmap" | tee -a $seqres.full -cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do - qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 010[01]{4}$" - echo "${qstr}" >> $seqres.full - grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full - found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) - test $found -eq 1 || echo "Unexpected output for offset ${offrange}." -done +if _xfs_is_realtime_file $SCRATCH_MNT/f2 && ! _xfs_has_feature $SCRATCH_MNT rtgroups; then + echo "Check f2 bmap and fsmap" | tee -a $seqres.full + cat $TEST_DIR/bmap | while read ext offrange colon rtblockrange total crap; do + qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${rtblockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${total} 010[01]{4}$" + echo "${qstr}" >> $seqres.full + grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full + found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) + test $found -eq 1 || echo "Unexpected output for offset ${offrange}." + done +else + echo "Check f2 bmap and fsmap" | tee -a $seqres.full + cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do + qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 010[01]{4}$" + echo "${qstr}" >> $seqres.full + grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full + found=$(grep -E -c "${qstr}" $TEST_DIR/fsmap) + test $found -eq 1 || echo "Unexpected output for offset ${offrange}." + done +fi # success, all done status=0 From patchwork Wed Dec 27 14:02:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508580 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07BF97EF; Mon, 1 Jan 2024 01:02:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NHNru/oX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87030C433C8; Mon, 1 Jan 2024 01:02:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070961; bh=zgzSX+sXh/oB3KgXb9RGzPP8BblpJQN7rbqn9NFxIrI=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=NHNru/oXxOnGvtU4PUS9sDufycyJfCVeJJ4Tl1Zac4WOB44FSX/ikOZafenHN9jUk XLmNJon/iO6eibdkay4l2fbMcoxR2hDC6GAiWZrM1iJLSp22XR2z2RKmFdR1FPcpt8 iSzZSNrOftPKQjKgP2HhG/ihnPGkWCW9BsnUoG0luHyG0KCiioF7vDpUjnou4f9PCL /s+P4SgxpGECP/eUfD2ru7CRjNyUdDdPgeRkGgcLI6OujtHS31WZMmUyjNZ+xEcrCe SyP+PRDvtaTxFND9gpxPnuHZiH0SNPuA3BE40q2OS9YtzJO/s2pK5GpUHIxmff0tNP o8+dEQQiz5b3w== Date: Sun, 31 Dec 2023 17:02:41 +9900 Subject: [PATCH 5/9] xfs: race fsstress with realtime refcount btree scrub and repair From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032082.1827358.7127035550813883108.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Race checking and rebuilding realtime refcount btrees with fsstress. Signed-off-by: Darrick J. Wong --- tests/xfs/1818 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1818.out | 2 ++ tests/xfs/1819 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1819.out | 2 ++ 4 files changed, 90 insertions(+) create mode 100755 tests/xfs/1818 create mode 100644 tests/xfs/1818.out create mode 100755 tests/xfs/1819 create mode 100644 tests/xfs/1819.out diff --git a/tests/xfs/1818 b/tests/xfs/1818 new file mode 100755 index 0000000000..674fff25fa --- /dev/null +++ b/tests/xfs/1818 @@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1818 +# +# Race fsstress and rt refcount btree scrub for a while to see if we crash or +# livelock. +# +. ./common/preamble +_begin_fstest scrub dangerous_fsstress_scrub + +_cleanup() { + _scratch_xfs_stress_scrub_cleanup &> /dev/null + cd / + rm -r -f $tmp.* +} +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/fuzzy +. ./common/inject +. ./common/xfs + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch +_require_xfs_stress_scrub + +_scratch_mkfs > "$seqres.full" 2>&1 +_scratch_mount +_require_xfs_has_feature "$SCRATCH_MNT" realtime +_require_xfs_has_feature "$SCRATCH_MNT" reflink +_xfs_force_bdev realtime $SCRATCH_MNT + +_scratch_xfs_stress_scrub -s "scrub rtrefcountbt %rgno%" + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/xfs/1818.out b/tests/xfs/1818.out new file mode 100644 index 0000000000..700d301da1 --- /dev/null +++ b/tests/xfs/1818.out @@ -0,0 +1,2 @@ +QA output created by 1818 +Silence is golden diff --git a/tests/xfs/1819 b/tests/xfs/1819 new file mode 100755 index 0000000000..e27a29271f --- /dev/null +++ b/tests/xfs/1819 @@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022-2024 Oracle. All Rights Reserved. +# +# FS QA Test No. 1819 +# +# Race fsstress and rt refcount btree scrub for a while to see if we crash or +# livelock. +# +. ./common/preamble +_begin_fstest online_repair dangerous_fsstress_repair + +_cleanup() { + _scratch_xfs_stress_scrub_cleanup &> /dev/null + cd / + rm -r -f $tmp.* +} +_register_cleanup "_cleanup" BUS + +# Import common functions. +. ./common/filter +. ./common/fuzzy +. ./common/inject +. ./common/xfs + +# real QA test starts here +_supported_fs xfs +_require_realtime +_require_scratch +_require_xfs_stress_online_repair + +_scratch_mkfs > "$seqres.full" 2>&1 +_scratch_mount +_require_xfs_has_feature "$SCRATCH_MNT" realtime +_require_xfs_has_feature "$SCRATCH_MNT" reflink +_xfs_force_bdev realtime $SCRATCH_MNT + +_scratch_xfs_stress_online_repair -s "repair rtrefcountbt %rgno%" + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/xfs/1819.out b/tests/xfs/1819.out new file mode 100644 index 0000000000..041e17ab61 --- /dev/null +++ b/tests/xfs/1819.out @@ -0,0 +1,2 @@ +QA output created by 1819 +Silence is golden From patchwork Wed Dec 27 14:02:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508581 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69C517EE; Mon, 1 Jan 2024 01:02:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aJA6QKAy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C07BC433C7; Mon, 1 Jan 2024 01:02:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070977; bh=ryjV8Jrw4OGG3FzNh3otD97RHKFVzbHCCZkGrrqFVx8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=aJA6QKAy3u/A5G+DAS6srTTWvlONw3wvF69oDqTM2/+fayYiSBKqL0yDszIWI4+U+ 4ID2Ei/TZf8Y8Uo8DrUuGHSE0AN1nSrqUPsjuZCXGkDwDopu3GV0/9YJd/phCgMZyk 6QX0xMqZhcRv7DCzf2XfUzEdwnKMrpQ5Yv0OvmCMPTWpw1NZbWWf1vO0XBEc6O4YHb r/t7/mS1ORtp85bdjhk9SR7qk+igBJDKpWGxOWvr4d62MBc6/HmB7nltOkloIe0TCm ZgYrLh29zV+ehM3eunrgXCjEuRz/59zg+MTo3b/UAxgUiU5x12gSylIeS1tmTnCZqc pUR5iY/TQCIBg== Date: Sun, 31 Dec 2023 17:02:56 +9900 Subject: [PATCH 6/9] xfs: remove xfs/131 now that we allow reflink on realtime volumes From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032095.1827358.13900239085409160451.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Remove this test, since we now support reflink on the rt volume. Signed-off-by: Darrick J. Wong --- tests/xfs/131 | 48 ------------------------------------------------ tests/xfs/131.out | 5 ----- 2 files changed, 53 deletions(-) delete mode 100755 tests/xfs/131 delete mode 100644 tests/xfs/131.out diff --git a/tests/xfs/131 b/tests/xfs/131 deleted file mode 100755 index 879e2dc6e8..0000000000 --- a/tests/xfs/131 +++ /dev/null @@ -1,48 +0,0 @@ -#! /bin/bash -# SPDX-License-Identifier: GPL-2.0 -# Copyright (c) 2015, Oracle and/or its affiliates. All Rights Reserved. -# -# FS QA Test No. 131 -# -# Ensure that we can't reflink realtime files. -# -. ./common/preamble -_begin_fstest auto quick clone realtime - -# Override the default cleanup function. -_cleanup() -{ - cd / - umount $SCRATCH_MNT > /dev/null 2>&1 - rm -rf $tmp.* $testdir $metadump_file -} - -# Import common functions. -. ./common/filter -. ./common/reflink - -# real QA test starts here -_supported_fs xfs -_require_realtime -_require_scratch_reflink -_require_cp_reflink - -echo "Format and mount scratch device" -_scratch_mkfs >> $seqres.full -_scratch_mount - -testdir=$SCRATCH_MNT/test-$seq -mkdir $testdir - -echo "Create the original file blocks" -blksz=65536 -$XFS_IO_PROG -R -f -c "truncate $blksz" $testdir/file1 - -echo "Reflink every block" -_cp_reflink $testdir/file1 $testdir/file2 2>&1 | _filter_scratch - -test -s $testdir/file2 && _fail "Should not be able to reflink a realtime file." - -# success, all done -status=0 -exit diff --git a/tests/xfs/131.out b/tests/xfs/131.out deleted file mode 100644 index 3c0186f0c7..0000000000 --- a/tests/xfs/131.out +++ /dev/null @@ -1,5 +0,0 @@ -QA output created by 131 -Format and mount scratch device -Create the original file blocks -Reflink every block -cp: failed to clone 'SCRATCH_MNT/test-131/file2' from 'SCRATCH_MNT/test-131/file1': Invalid argument From patchwork Wed Dec 27 14:03:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508582 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E6727F9; Mon, 1 Jan 2024 01:03:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gSJr19cm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D00D2C433C8; Mon, 1 Jan 2024 01:03:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704070992; bh=NsIOAnrzdlE+bWwF2Rc+7zi1weKTWm0JBf8hj+ENiS4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=gSJr19cmlat1JvkEdanqOu8A6oEy9KgwDYs3kaAtP6yK8BksYQowljBUGo4uSjylC /7/S04yBbGKELmzztwk+1sL3ZguFRa7/pdjAz9xdmw+1evA31wKKsSiufXPqSzzCUJ aNp2bntM2Ro1ZacgFR0YZDL5jUQplcEOt69VixenkEAi7/1vj6m69Bu2WQox1HX1/0 XiSVlFHfgVKGL6LyIyJ51xEgHJH/hbbto6YHoePmucV7cgLQQQawt7HL7b9WG3DO3v +rMHxtghlitm04GOKK1vdxmvkAqIl3WQoNkxH2tGE0iW+/nKjyF/qCAoMTN/Xi7QG9 9dsSg4ShMvbbQ== Date: Sun, 31 Dec 2023 17:03:12 +9900 Subject: [PATCH 7/9] xfs/856: add rtreflink upgrade to test matrix From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032108.1827358.4212368779476095141.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Add realtime reflink to the features that this test will try to upgrade. Signed-off-by: Darrick J. Wong --- tests/xfs/1856 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/xfs/1856 b/tests/xfs/1856 index 8453d9bff2..4126c28e5d 100755 --- a/tests/xfs/1856 +++ b/tests/xfs/1856 @@ -205,12 +205,13 @@ post_exercise() # upgrade don't spread failure to the rest of the tests. FEATURES=() if rt_configured; then - # rmap wasn't added to rt devices until after metadir and rtgroups + # reflink & rmap weren't added to rt devices until after metadir and rtgroups check_repair_upgrade finobt && FEATURES+=("finobt") check_repair_upgrade inobtcount && FEATURES+=("inobtcount") check_repair_upgrade bigtime && FEATURES+=("bigtime") check_repair_upgrade metadir && FEATURES+=("metadir") supports_rtgroups && check_repair_upgrade rmapbt && FEATURES+=("rmapbt") + supports_rtgroups && check_repair_upgrade reflink && FEATURES+=("reflink") else check_repair_upgrade finobt && FEATURES+=("finobt") check_repair_upgrade rmapbt && FEATURES+=("rmapbt") From patchwork Wed Dec 27 14:03:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508583 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 314017F9; Mon, 1 Jan 2024 01:03:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fYNRdl7N" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7DBB0C433C8; Mon, 1 Jan 2024 01:03:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704071008; bh=v5t/VCu9Eh7EbvI3dy4porQrhQuQOlHnWICHHOqNeCo=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=fYNRdl7NPb4pV/jwI4mYXAGYpxNYeVToyjTvuSD0Xnu3hcES+uBSzHnzvMlHeH9u7 i9rWK8hJmpovDo4N94up2XlWwwE78pvH1fMgZ9aGxRekvYyjfYl7g9rQ8I4WDj8FKa cj4NXZo+5lT/jkxutoYP7/TfyUE5Eu7utjOenjIeRKa9tqexcg9VZmDP2CHAd+yEYT NY86SUfbvvBsJeLk6vgWKaqULXn6BZRJA6ksfSezbsQRHZu2wen7vlyDpOY+NRavHp Gd42jsHBDfU8Va10jZTw3kQ1aQUBoCLC3e3hIcPTCf+zTVf1gE8IMV06QWahLOenJL WiGKT9lYcwDnQ== Date: Sun, 31 Dec 2023 17:03:28 +9900 Subject: [PATCH 8/9] generic/331,xfs/240: support files that skip delayed allocation From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032121.1827358.9242931540348227468.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The goal of this test is to ensure that log recovery finishes a copy on write operation in the event of temporary media errors. It's important that the test observe some sort of IO error once we switch the scratch device to fail all IOs, but regrettably the test encoded the specific behavior of XFS and btrfs when the test was written -- the aio write to the page cache doesn't have to touch the disk and succeeds, and the fdatasync flushes things to disk and hits the IO error. However, this is not how things work on the XFS realtime device. There is no delalloc on realtime, so the aio write allocates an unwritten extent to stage the write. The allocation fails due to EIO, so it's the write call that fails. Therefore, all we need to do is to detect an IO error at any point between the write and the fdatasync call to be satisfied that the test does what we want to do. Signed-off-by: Darrick J. Wong --- tests/generic/331 | 12 ++++++++++-- tests/generic/331.out | 2 +- tests/xfs/240 | 13 +++++++++++-- tests/xfs/240.out | 2 +- 4 files changed, 23 insertions(+), 6 deletions(-) diff --git a/tests/generic/331 b/tests/generic/331 index 492abedf76..8c665ce4fc 100755 --- a/tests/generic/331 +++ b/tests/generic/331 @@ -59,9 +59,17 @@ echo "CoW and unmount" $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full sync + +# If the filesystem supports delalloc, then the fdatasync will report an IO +# error. If the write goes directly to disk, then aiocp will return nonzero. +unset write_failed _dmerror_load_error_table -$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full -$XFS_IO_PROG -c "fdatasync" $testdir/file2 +$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 &>> $seqres.full || \ + write_failed=1 +$XFS_IO_PROG -c "fdatasync" $testdir/file2 2>&1 | grep -q 'Input.output error' && \ + write_failed=1 +test -n $write_failed && echo "write failed" + _dmerror_load_working_table _dmerror_unmount _dmerror_mount diff --git a/tests/generic/331.out b/tests/generic/331.out index adbf841d00..d8ccea704b 100644 --- a/tests/generic/331.out +++ b/tests/generic/331.out @@ -5,7 +5,7 @@ Compare files 1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-331/file1 1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-331/file2 CoW and unmount -fdatasync: Input/output error +write failed Compare files 1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-331/file1 d94b0ab13385aba594411c174b1cc13c SCRATCH_MNT/test-331/file2 diff --git a/tests/xfs/240 b/tests/xfs/240 index a65c270d23..cabe309201 100755 --- a/tests/xfs/240 +++ b/tests/xfs/240 @@ -66,8 +66,17 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full sync _dmerror_load_error_table -$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full -$XFS_IO_PROG -c "fdatasync" $testdir/file2 + +# If the filesystem supports delalloc, then the fdatasync will report an IO +# error. If the write goes directly to disk, then aiocp will return nonzero. +unset write_failed +_dmerror_load_error_table +$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 &>> $seqres.full || \ + write_failed=1 +$XFS_IO_PROG -c "fdatasync" $testdir/file2 2>&1 | grep -q 'Input.output error' && \ + write_failed=1 +test -n $write_failed && echo "write failed" + _dmerror_load_working_table _dmerror_unmount _dmerror_mount diff --git a/tests/xfs/240.out b/tests/xfs/240.out index 1a22e8a389..00bb116e5c 100644 --- a/tests/xfs/240.out +++ b/tests/xfs/240.out @@ -5,7 +5,7 @@ Compare files 1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-240/file1 1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-240/file2 CoW and unmount -fdatasync: Input/output error +write failed Compare files 1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-240/file1 d94b0ab13385aba594411c174b1cc13c SCRATCH_MNT/test-240/file2 From patchwork Wed Dec 27 14:03:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508584 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A50C67F9; Mon, 1 Jan 2024 01:03:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SgoC9bx9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A3EBC433C7; Mon, 1 Jan 2024 01:03:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704071024; bh=Jn8EPPLiIwzwsA/m8ZGWz3YKKTg8B+NLLSD/2m7i0ZE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=SgoC9bx9lac+zvF0oQIwizJmZJuKQlGE/fa2u2WG3iCz4bll+6dEVzmuzhH45809A XVMAyq1N5YpJhus66Vihv4nC5wNgLqgHLw+pKxeLF1/XiLXWKntJdSWyGbP0+btbFk Bbc3Hin3gXBGgBAljdyuBetvslPICrKHTGiRCA+wu52nTcMF3iMOdBsDu6Pf8Hs4+U K/4LtOIi2xFnZ8CyDi8TeqMHTLXEIkD1Jh9jdFb6tVBY+xkwiDUoDgj/3f4S3xUDNf moK8ARDtzUU9qD9i4n080t1RMQB5jFY6awDWObBQ6ZLs74hE4MtRKYzQxb+LD5Q5J4 3C+j4iGdzFCGg== Date: Sun, 31 Dec 2023 17:03:43 +9900 Subject: [PATCH 9/9] common/xfs: fix _xfs_get_file_block_size when rtinherit is set and no rt section From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: guan@eryu.me, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Message-ID: <170405032134.1827358.4148547989863979616.stgit@frogsfrogsfrogs> In-Reply-To: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> References: <170405032011.1827358.11723561661069109569.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong It's possible for the sysadmin to set rtinherit on the directory tree even if there isn't a realtime section attached to the filesystem. When this is the case, the realtime flag is /not/ passed to new files, and file data is written to the data device. The file allocation unit for the file is the fs blocksize, and it is not correct to use the rt extent. fstests can be fooled into doing the incorrect thing if test runner puts '-d rtinherit=1 -r extsize=28k' into MKFS_OPTIONS without configuring a realtime device. This causes many tests to do the wrong thing because they think they must operate on units of 28k (and not 4k). Fix this. Signed-off-by: Darrick J. Wong --- common/xfs | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/common/xfs b/common/xfs index aab04bfb18..69a0eb620c 100644 --- a/common/xfs +++ b/common/xfs @@ -208,6 +208,8 @@ _xfs_get_file_block_size() { local path="$1" + # If rtinherit or realtime are not set on the path, then all files + # will be created on the data device. if ! ($XFS_IO_PROG -c "stat -v" "$path" 2>&1 | grep -E -q '(rt-inherit|realtime)'); then _get_block_size "$path" return @@ -218,6 +220,15 @@ _xfs_get_file_block_size() while ! $XFS_INFO_PROG "$path" &>/dev/null && [ "$path" != "/" ]; do path="$(dirname "$path")" done + + # If there's no realtime section, the rtinherit and rextsize settings + # are irrelevant -- all files are created on the data device. + if $XFS_INFO_PROG "$path" | grep -q 'realtime =none'; then + _get_block_size "$path" + return + fi + + # Otherwise, report the rt extent size. _xfs_get_rtextsize "$path" }