From patchwork Tue Dec 5 12:45:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480206 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="LMacoShu" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAC68134; Tue, 5 Dec 2023 04:45:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780313; x=1733316313; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=vH5ll84B3/EfTeO+vugCYebs+7bGkX44oA4L317bX/A=; b=LMacoShuTtz5FBK2G0W3OR0KkJKDUv+4fS4jOd19e+7FXe53wto/8e9H TMm1qmsVPqvT98QqmxpR4jazJH86sy84pYOC7JllVNisU88/mTILJ4rYZ huw4iGekpBTRTuqSZILFs1N+aehfOKmW5rlu+SbqJrKFRwQbGci/ov+Xn nsJrg0Rmf05+BjZm6wLgOYa6e2IN0IxrHismgFP6Y0HUaUq9C0cbgj5as JHWnkS++lYCPXO3+PajA648JlzdTBOYXaJ1v+fmdfsKwUmZ+IdebWYWZd 16Ooo9W3/GUIWo92ij8g7hFDbQpHVrE0nW7/TZsrYukP8pBxGNp3Fx8lW Q==; X-CSE-ConnectionGUID: x/gPJsyfQRip3p1F1OnSkA== X-CSE-MsgGUID: j18GS1zGSOKDvpVhJVKoHw== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043894" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:13 +0800 IronPort-SDR: OywhGj5uSwVoK4QfqJsOkjUfGrdvAPy/vQXnga2lk5ncoPGJfPICYAD1DuoCyyM4A1frBaPbi0 DcJwMAdoCVqQ== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:15 -0800 IronPort-SDR: Wx9i/wOC1NxVPBcCAM52diWwXFXkg7XhD/PQ/J8q63kIZjvjvCgKctgO1genTVe2rlrx3dsiqm heW4AWxy0tOQ== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:12 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:07 -0800 Subject: [PATCH v2 1/7] fstests: doc: add new raid-stripe-tree group Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-1-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=705; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=vH5ll84B3/EfTeO+vugCYebs+7bGkX44oA4L317bX/A=; b=bW19jpSa3QzFgSKhThZ5FToPrIPW7A3X/PzhJnFGlJjYz0qRTxxo4/BplIfvq29Jc+Uq/29bu 87k2F9HUyRMCA8Og1KZsmjtDkuCn5b41/Fb9Su19FMFDMe/DA5UmkE/ X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a new test group for testing the raid-stripe-tree feature of btrfs with fstests. Signed-off-by: Johannes Thumshirn --- doc/group-names.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/group-names.txt b/doc/group-names.txt index c3dcca375537..9c1624868518 100644 --- a/doc/group-names.txt +++ b/doc/group-names.txt @@ -94,6 +94,7 @@ punch fallocate FALLOC_FL_PUNCH_HOLE qgroup btrfs qgroup feature quota filesystem usage quotas raid btrfs RAID +raid-stripe-tree btrfs raid-stripe-tree feature read_repair btrfs error correction on read failure realtime XFS realtime volumes recoveryloop crash recovery loops From patchwork Tue Dec 5 12:45:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480208 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="JDAiRiga" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A778F182; Tue, 5 Dec 2023 04:45:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780314; x=1733316314; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=0vFVxyKpqAp7DKayL6Ik1UXOmUX6REN6nOAs9orTBSo=; b=JDAiRigat1yeRiG5UADGXaQVxg+v1/X516z+eQhlO2doOzP5rtRhiGC7 qrQ6XDSXQu0c1DxBlevFdwgk/t11/kvGXNmm1WwIKHbr1ceXsQMuB4FmM H9uS7EYcDMmm/jGy+96qVSM3t9JkmWHhtMDfqmDrkkB1XYfdxl+KX6tdt p8itqno62LIEUZXar1ZET/cqBdmbT0NHcX5Qyp0OW9RTBQ/YueHoaa+fK 3/HVSAtEMYplQOmaBXtRdPitzhHNlJ9BrgFS24vH19uUazmezA4C8kPcG AqtLNerP9p1vQa52+uRtS1kqfEjRftqx8Ko0ngxj2Dyw4cFFcE9tnITh8 A==; X-CSE-ConnectionGUID: WONirUX2TkqgVzSxYIlbqQ== X-CSE-MsgGUID: Y9SD2CFiTIqH1YS+mcASbQ== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043900" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:14 +0800 IronPort-SDR: hlHx45imY1XvsjuQkfuX37DRPDh7JlI5xR6GFVpvJtWkCINSpaR4NBalPgdg9MtXnddUxtRJx/ w8MDoCxCWXTw== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:16 -0800 IronPort-SDR: slw6nzBIrgX6LfaWZxF4WK7kfwUM3q8qNvVJhYSUVAJCl+tzmFbAvrqNERgXc8QH0CnLN922et 7Mom5CouuCEw== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:14 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:08 -0800 Subject: [PATCH v2 2/7] common: add filter for btrfs raid-stripe dump Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-2-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=1491; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=0vFVxyKpqAp7DKayL6Ik1UXOmUX6REN6nOAs9orTBSo=; b=VYEja1VcmxM92+/Ij8DR34Fgfz+1MTEV1MMQYWVQla4qf6RwEw2Gu9hjuTQzoxaBta8Wo1W9/ 2g8+avg02PHDI5LwkIm8r5sIy51Qk+NrFd+QLBiploXtMgTM/11ke7c X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Signed-off-by: Johannes Thumshirn --- common/filter.btrfs | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/common/filter.btrfs b/common/filter.btrfs index 02c6b92dfa94..2003ba7b7015 100644 --- a/common/filter.btrfs +++ b/common/filter.btrfs @@ -125,5 +125,19 @@ _filter_btrfs_cloner_error() sed -e "s/\(clone failed:\) Operation not supported/\1 Invalid argument/g" } +# filter output of "btrfs inspect-internal dump-tree -t raid-stripe" +_filter_stripe_tree() +{ + sed -E -e "s/leaf [0-9]+ items [0-9]+ free space [0-9]+ generation [0-9]+ owner RAID_STRIPE_TREE/leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE/" \ + -e "s/leaf [0-9]+ flags 0x1\(WRITTEN\) backref revision 1/leaf XXXXXXXXX flags 0x1\(WRITTEN\) backref revision 1/" \ + -e "s/checksum stored [0-9a-f]+/checksum stored /" \ + -e "s/checksum calced [0-9a-f]+/checksum calced /" \ + -e "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}//" \ + -e "s/item ([0-9]+) key \([0-9]+ RAID_STRIPE ([0-9]+)\) itemoff [0-9]+ itemsize ([0-9]+)/item \1 key \(XXXXXX RAID_STRIPE \2\) itemoff XXXXX itemsize \3/" \ + -e "s/stripe ([0-9]+) devid ([0-9]+) physical [0-9]+/stripe \1 devid \2 physical XXXXXXXXX/" \ + -e "s/total bytes [0-9]+/total bytes XXXXXXXX/" \ + -e "s/bytes used [0-9]+/bytes used XXXXXX/" +} + # make sure this script returns success /bin/true From patchwork Tue Dec 5 12:45:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480209 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="IoxOC4du" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 057AEC6; Tue, 5 Dec 2023 04:45:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780315; x=1733316315; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=+rFy2WgK7PfNOEketIUxl3nuhKURPGlzKuzEf8gm7zA=; b=IoxOC4du7PWbR9U5/GOB+hsOXpbbryG8jBJpVoPKFCxF2KHGSfOHazZN FwJ1mswD5w+vNjAiT66JvhPiETPoAzVaWC4ORBi+kuEZbGxF6nLWyBc2o aYmsdRlhhYYiciKPaeAqBcdvec8fkS/+vC4/MciD/O5+GZW0T0Jdj/Lmx DZZyd5timcloErXwJP6q2TGp79ovWcwtPv4578jCCfETkDgx/rGmDiBd8 yMJP3bwHTVkePRxzIhfncyHrb0pJl1gp6zmd+bhSS+5DgGR8pQNCIZMa6 q+i/SdfXcJJ8aeZuMu3du2yOlQsE20RzvcggQ6Tb767mwJwZWNlW2WMAm A==; X-CSE-ConnectionGUID: stgFyXKHQqa8a7pc+mRKcA== X-CSE-MsgGUID: dfYt3BBEQKKQ9ROZhuNWFg== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043913" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:15 +0800 IronPort-SDR: PVZLSJb9cWpy0xaqjKbEpvrmpAlJhU+U5a5RsTacBDW6HvC4d9YE+3xBS800dd2WYEwzuSM7o7 BRNVCSHUqyCw== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:17 -0800 IronPort-SDR: G7Xd3/Tq+Oi6BmbC7yfZdmnWNFqdLL8QxJ044EPlFVz3+PGzn/S1zdCdWlU8bzegchhrlv+JBX 50unHLwOZc9A== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:15 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:09 -0800 Subject: [PATCH v2 3/7] btrfs: add fstest for stripe-tree metadata with 4k write Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-3-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=4077; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=+rFy2WgK7PfNOEketIUxl3nuhKURPGlzKuzEf8gm7zA=; b=oT6NLepm8j5rPRenRCPyeT3nPUAPonm3fCx3TLXPWlIsRowazhn/qmMIspJstOyySsne9ONHH FH72yWczYvQBroJlQx8tYyRXt9eTjZEgonVc2+gIDHW81EeS+8gsLHb X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Test a simple 4k write on all RAID profiles currently supported with the raid-stripe-tree. Signed-off-by: Johannes Thumshirn --- tests/btrfs/302 | 53 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/302.out | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 111 insertions(+) diff --git a/tests/btrfs/302 b/tests/btrfs/302 new file mode 100755 index 000000000000..5d32ca8ba92f --- /dev/null +++ b/tests/btrfs/302 @@ -0,0 +1,53 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 302 +# +# Test on-disk layout of RAID Stripe Tree Metadata writing 4k to a new file on +# a pristine file system. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_4k_write() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + $XFS_IO_PROG -fc "pwrite 0 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test basic 4k write =" +test_4k_write raid0 2 +test_4k_write raid1 2 +test_4k_write raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/302.out b/tests/btrfs/302.out new file mode 100644 index 000000000000..149630e69501 --- /dev/null +++ b/tests/btrfs/302.out @@ -0,0 +1,58 @@ +QA output created by 302 += Test basic 4k write = +==== Testing raid0 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 12:45:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480210 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="JsIy9b+j" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10CF910F; Tue, 5 Dec 2023 04:45:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780316; x=1733316316; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=kgKvbSZKh/K3z/aAEwo5FZa/vZnvGjn2gcGXhWOMP9k=; b=JsIy9b+j01iX0giqFX7ZsiqFUA/aWqbHwKifx5KVv6UHj2lEEzIUFxz3 zSNTVU08vmK8k+yRu/kHg1S0iFg966UB2jjLy+XcUU41G/kA6OzdaFZPJ qAC9EkdyDOAvsFPrBnlhHl9MMcddSdMvL89cXfP8PFG4o5kY+FlwY+RiS fk6AslAvI375ofXi5lLjEwBBr89WXY+rXUZQbPrsZptNvu63Pyhg+paeV wKK3HWAx0qphhBzIgWOhCwk5AP6xF7RWkFNLqn6LCeBCmw0jfhLyWuQLb SOvraEkMculY0eYxrfNj1mZKhJ4OUguyujsrbo9Hjg8Lxl85cpsXFjvjZ w==; X-CSE-ConnectionGUID: QIdcn4jkSkWOpGFyD4HA2A== X-CSE-MsgGUID: yM5A6yLhT66w4uIIwEGm2g== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043918" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:16 +0800 IronPort-SDR: qAZO5J7KHWbmklP17wqudcazG9bJT1QIge49VQaZOfwQ/uNncZ3odJH299Mm3oiOZobdfGJIqZ xRRVyfyT93bQ== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:18 -0800 IronPort-SDR: lcAkGO+bY8hTFLpmA1jLm4b8JYHepYkEsyHC5kEBeDFYfZC+RlnsDB7zp/U3njeA/Yfauw2aBs jc6Ow/YLDqWQ== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:16 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:10 -0800 Subject: [PATCH v2 4/7] btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-4-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=5580; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=kgKvbSZKh/K3z/aAEwo5FZa/vZnvGjn2gcGXhWOMP9k=; b=h6XpaY0iVO+Crg44FNOyGzWkTT0BLA7kj8NeGAu1sN4+cV0o+weYwZ0KH/keVHVkDmYfHRY5k 74xD7fJgTRhAuH53hqEnyTOGZrQIMSA64YqmLpTa3P+m7ObwFkRX5ik X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test-case writing 8k to a raid-stripe-tree formatted filesystem with one stripe pre-filled to 60k so the 8k are split into a 4k write finishing stripe 1 and a 4k write starting the next stripe. Signed-off-by: Johannes Thumshirn --- tests/btrfs/303 | 58 +++++++++++++++++++++++++++++++++++++ tests/btrfs/303.out | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 140 insertions(+) diff --git a/tests/btrfs/303 b/tests/btrfs/303 new file mode 100755 index 000000000000..9f3a5df16423 --- /dev/null +++ b/tests/btrfs/303 @@ -0,0 +1,58 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 303 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 8k to a new file +# with a filesystem prepropulated, so that 4k of the write are written to the +# 1st stripe and 4k start a new stripe. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_8k_new_stripe() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + # Fill the first stripe up to 64k - 4k + $XFS_IO_PROG -fc "pwrite 0 60k" -c fsync "$SCRATCH_MNT/bar" | _filter_xfs_io + + # The actual 8k write + $XFS_IO_PROG -fc "pwrite 0 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 8k write to a new file so that 4k start a new stripe =" +test_8k_new_stripe raid0 2 +test_8k_new_stripe raid1 2 +test_8k_new_stripe raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/303.out b/tests/btrfs/303.out new file mode 100644 index 000000000000..1259d0f4a587 --- /dev/null +++ b/tests/btrfs/303.out @@ -0,0 +1,82 @@ +QA output created by 303 += Test 8k write to a new file so that 4k start a new stripe = +==== Testing raid0 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 12:45:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480211 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="CGb7SdVg" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0205124; Tue, 5 Dec 2023 04:45:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780317; x=1733316317; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=TsibjB9rL/e2eyCJwTCPlgSHmlSJ69QQAWOIpWJkQ6E=; b=CGb7SdVgu1yAjYLeca3dAXGm+GHFgzm/ZN9YDZFAJhNfaOaBXrdBhwJO TzqpiayKlL0MJTF5mKwojA9qisB1oLINuagcaT+GaTiuz+36lzw2Z3i5i nUs6iy0isohg7j1p0YozbRSF1XMSmXIFyRRa8uKPZdBpXPfQb5+MwApa2 4gR3xWSnblaG9RSgPRxZSmNbHEh7ePj2TZ0QJ2stTFAwlnWtxZ82bzgKk FYU6+dOxxnlXBdehuetIBWuKZdmmGlIqMrJQ9wAvkfsF4SqHXNLK8qekT MfvEJYcTgR9GqpQnykZxKsntY7Qtd0bTIuZgXlUySWk5QS+HuI6dUT7aG w==; X-CSE-ConnectionGUID: +zG4AehDR82nvi29j6o8cg== X-CSE-MsgGUID: 9ZLdY8qkRL6zwXRx3cRKYg== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043921" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:17 +0800 IronPort-SDR: hvQYp2+ctS/z812rPTAHIWVs0kOlRjbKCh0LVZjuTmk1CZ0EkkeN2r8/MfiV2AqWuFetzO3Gbn 5Q2TJ1fw8EzA== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:19 -0800 IronPort-SDR: EKSq3DcERl2qwyobXOG/pcRv5y0BChtlcT2ecvI8Sp2cQJWONbGZVTaxaOhr8T8fEselPKENEH JqFlBd7jUowA== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:17 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:11 -0800 Subject: [PATCH v2 5/7] btrfs: add fstest for writing to a file at an offset with RST Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-5-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=5182; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=TsibjB9rL/e2eyCJwTCPlgSHmlSJ69QQAWOIpWJkQ6E=; b=8vDUOn/ofX+a1DjpkwWrVeSphFG7nnqbTzseOzBNpUkOKNwJbc7i2Audr300iSPvq4oBH6dSr HouhodDHK13DPjtzpZLzj7TY7VDBpyE+rAqtVrGEYwtfI2kWD99flhH X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a fstest writing 4k at offset 64k to a file with one RAID tripe already pre-filled for a raid-stripe-tree formatted file system. Signed-off-by: Johannes Thumshirn --- tests/btrfs/304 | 56 +++++++++++++++++++++++++++++++++++++++ tests/btrfs/304.out | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+) diff --git a/tests/btrfs/304 b/tests/btrfs/304 new file mode 100755 index 000000000000..a95cfbac1e1b --- /dev/null +++ b/tests/btrfs/304 @@ -0,0 +1,56 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 304 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 4k to an emppty +# file at offset 64k with one stripe pre-filled on an otherwise pristine +# filesystem. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_4k_write_64koff() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + # precondition one stripe + $XFS_IO_PROG -fc "pwrite 0 64k" "$SCRATCH_MNT/bar" | _filter_xfs_io + + $XFS_IO_PROG -fc "pwrite 64k 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} +echo "= Test 4k write to an empty file at offset 64k with one stripe prefilled =" +test_4k_write_64koff raid0 2 +test_4k_write_64koff raid1 2 +test_4k_write_64koff raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/304.out b/tests/btrfs/304.out new file mode 100644 index 000000000000..e94deec1ab21 --- /dev/null +++ b/tests/btrfs/304.out @@ -0,0 +1,75 @@ +QA output created by 304 += Test 4k write to an empty file at offset 64k with one stripe prefilled = +==== Testing raid0 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 12:45:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480212 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="lHqeDWtv" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2627120; Tue, 5 Dec 2023 04:45:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780318; x=1733316318; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=VwXVUK59GoSFm7hUfF/gj25JjQ0QqDyn97ZRS5tO/QU=; b=lHqeDWtvpJXyoGojUItXnjQk1ca1/DcWveqgssG3bQTJHgYi9UOQNlKA gxTTm0ZJSJKagmwVnXrCzFY6+KMVQoYT7WsemaO9tSmmX7qewqWvN/Mwt 2kWxxxuZha4X9psGw7HV1v26KWvVoi0oXmQ5+IxU4lnLFJzQ4lVaZj+aF 2CBNO+CNv6sj09YT0UfVliX8y+GoFr5W9no2QqCaSqnU4U0RwaECyo7B5 LGYzaVtd5llv6jeCHOykBqsEh/ZZ8mMmWdCf7X91jAndMFukXNJJ+rQf5 RewHHwSQeAU3RL9CepoP/9NetZHOME8lt4+eci/33NgLuIg5kxzsjG4uO g==; X-CSE-ConnectionGUID: 2x2k0RJIRmiJ1w8R7Bmqqw== X-CSE-MsgGUID: kNfw3ApYQ1KuoeUZnCRf7Q== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043927" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:18 +0800 IronPort-SDR: tYYCt49v/bAhGQGtya9CmrCZxFCqZEtQaZCn0+lq5/dVdSyi1/YNTWs/ZUJtP3edCjrkURgtFD Upy02mqAuvdA== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:20 -0800 IronPort-SDR: ibGTaZuSdryJcTFhqb6nyQp+i8jtsQm5achtRzBLTIgy6wm8u8nKLQIEL2Cpre/eRjioivd2Q0 wA4W3N0YB4Bw== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:18 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:12 -0800 Subject: [PATCH v2 6/7] btrfs: add fstests to write 128k to a RST filesystem Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-6-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=4423; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=VwXVUK59GoSFm7hUfF/gj25JjQ0QqDyn97ZRS5tO/QU=; b=PH35EttU+8i8T8WO3/buW6VxhyZxM+yGR4eLutcq/HMjD+SvyNev4IRwrDATzin4kU7IW/r8o HR1fvq9M2/0DwlWNMVv1/gB9q9aGHKVxk4XHw5Bdyzzh0oYxoWtoZ8Z X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test writing 128k to a file on an empty filesystem formatted with a raid-stripe-tree. Signed-off-by: Johannes Thumshirn --- tests/btrfs/305 | 53 +++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/305.out | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 118 insertions(+) diff --git a/tests/btrfs/305 b/tests/btrfs/305 new file mode 100755 index 000000000000..362ae1966f4f --- /dev/null +++ b/tests/btrfs/305 @@ -0,0 +1,53 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 305 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to a new +# file on a pristine filesystem +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_128k_write() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + $XFS_IO_PROG -fc "pwrite 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 128k write to empty file =" +test_128k_write raid0 2 +test_128k_write raid1 2 +test_128k_write raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/305.out b/tests/btrfs/305.out new file mode 100644 index 000000000000..0dbd3beca77f --- /dev/null +++ b/tests/btrfs/305.out @@ -0,0 +1,65 @@ +QA output created by 305 += Test 128k write to empty file = +==== Testing raid0 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 12:45:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480213 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="kw9Kga7P" Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBA2B127; Tue, 5 Dec 2023 04:45:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701780319; x=1733316319; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=z7vEX4gmj/jJp0gY4rCnLAfsht5j7h66X3R43XmqB5A=; b=kw9Kga7PrYXVpuOFuCdwR+MgPPLkhOsu6jxY6JpoYfCDE+b2onWsg9kx 74IQAu1OXoHDmDmmrl0E5234b5mysCr+S48rN0PH7hkqZNrsWk7RkCq00 5xkvDgv8giht57Aiu+PvNpNRPUZoBzflRVjg3alAQEzio6EnPbxRJSPE3 +rtiyjDKolweGXRwLrppR7JdFVFN4XzYS9oszV2rK5joQde+NR7PB/4U+ 9JAS+fr0hoKB++vYtCtoquSvg0BiJd2D1sG/Bwwn5hjYeaDIFEVUOfcTf wwld7WCzFUw79krPb21HvIKX5YywolyqIcgtJKvuX8Qq4jDVdXb1q5DuU w==; X-CSE-ConnectionGUID: RQUWkVyJRA+Yl5MxnoghHg== X-CSE-MsgGUID: vHquBdZqS3mhzcZ++37GtQ== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="4043932" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 05 Dec 2023 20:45:19 +0800 IronPort-SDR: iAu2dKQh9n6JsBTfb4GTx302KtADAjbUNOzt9flpdRK/nqQTSAa7mFHs5gLe7AHJZRcjsDU6aO 6gG2Gje7ynTg== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 03:56:21 -0800 IronPort-SDR: ExkdAK+/WggCk8/tZV5lwNGz6HkHXwDOCASTfD+uS3QDUDuEJkFYDo8b/n0mVG3RP2cOZ6sV94 QUQBWr/Zhg0g== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 04:45:19 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 04:45:13 -0800 Subject: [PATCH v2 7/7] btrfs: add fstest for overwriting a file partially with RST Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v2-7-25f80eea345b@wdc.com> References: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> In-Reply-To: <20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701780310; l=6665; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=z7vEX4gmj/jJp0gY4rCnLAfsht5j7h66X3R43XmqB5A=; b=eG6cncQta+1fAJR04TIMoeFAv7cjAQKKrYMAtBE9dxI9Ltr1QHc+G3Nf7JYOOeZTjsG4fiEzS v2kOu4XHKDbDhyEdMO/AU49W6iZ77N3K3HrlhTnH70aAf9Nkb7f7bYI X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test writing 128k to an empty file with one stripe already pre-filled on-disk. Then overwrite a portion of the file in the middle. Signed-off-by: Johannes Thumshirn --- tests/btrfs/306 | 56 +++++++++++++++++++++++++++ tests/btrfs/306.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 162 insertions(+) diff --git a/tests/btrfs/306 b/tests/btrfs/306 new file mode 100755 index 000000000000..f2708bbcd2a5 --- /dev/null +++ b/tests/btrfs/306 @@ -0,0 +1,56 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 306 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to an empty +# file on a filesystem that has one stripe already pre-filled. Afterwards +# overwrite a portion of the file. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_128k_write_overwrite() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile + _scratch_mount + + $XFS_IO_PROG -fc "pwrite -W 0 32k" "$SCRATCH_MNT/bar" | _filter_xfs_io + $XFS_IO_PROG -fc "pwrite -W 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io + $XFS_IO_PROG -fc "pwrite -W 64k 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 128k write to empty file with 1st stripe partially prefilled then overwrite =" +test_128k_write_overwrite raid0 2 +test_128k_write_overwrite raid1 2 +test_128k_write_overwrite raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/306.out b/tests/btrfs/306.out new file mode 100644 index 000000000000..52f3d68f77ff --- /dev/null +++ b/tests/btrfs/306.out @@ -0,0 +1,106 @@ +QA output created by 306 += Test 128k write to empty file with 1st stripe partially prefilled then overwrite = +==== Testing raid0 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX + item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX + item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid