From patchwork Tue Dec 5 17:47:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480532 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="nCIu8GRo" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D421188; Tue, 5 Dec 2023 09:47:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798426; x=1733334426; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=vH5ll84B3/EfTeO+vugCYebs+7bGkX44oA4L317bX/A=; b=nCIu8GRomJ3eieRMZo6xKaioYJeZiYVgLAcmbyIEFLFkntZWBxdEfXHL aVuNOaxARhBSi9reYQjZ2c1caZgzywJfN2flIzMXWry7B5FzwkMIHDRBn MOxYxRrBbo/4rsnjxbV2FG/vcBVGc/4ku42WSgNLQVIhEXA6cODT+vbSf 586YxxFwGqzHmKyE7en3rf7N15qxKp4nFA+duEW0ioDe7/onJreoANwAj w/leZm7dYVjdgOXmUstRqp1HtF+qNtKd6lhcwh3AHI8di82R7Us9NatO2 laCFK1BE2LrFP7/Bbg5vL9cu+GipLGlv39UEflaNYyoVOlRiNCfbPRMGL Q==; X-CSE-ConnectionGUID: ke6ADZ/jSyOhwTWgG16JJg== X-CSE-MsgGUID: 953kDhoJThWu3LJ9DR5TNQ== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944948" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:05 +0800 IronPort-SDR: SAQAzY1FxeKYvusdod0kqdYQFu6CcpvkJ/2e//o9e2PxhyxZ2XyYzuQZXEWDECCGXrssvCGzpm W9o0VuCGC5Gg== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:26 -0800 IronPort-SDR: 9LBT7cG5JOftD4csJkISptlahmJfamTb4iPQXiU4jLkdxMf5IHNkGYHf4Bh1VXgqmyqxUCkyd3 S5y8Lh/MNYQA== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:05 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:01 -0800 Subject: [PATCH v3 1/7] fstests: doc: add new raid-stripe-tree group Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-1-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=705; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=vH5ll84B3/EfTeO+vugCYebs+7bGkX44oA4L317bX/A=; b=aYw3M4v9D5N57gF5kMcAuHfk1hyiMC7CH233pVS9f+2Opc5rMgJhSZIjaeTIOuDtoiyUwVavo iV6+iqZOV1kDRsFVF6xByAqtsc/gfg+xxnc+jqK9L34dcP406fcT3Dn X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a new test group for testing the raid-stripe-tree feature of btrfs with fstests. Signed-off-by: Johannes Thumshirn --- doc/group-names.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/group-names.txt b/doc/group-names.txt index c3dcca375537..9c1624868518 100644 --- a/doc/group-names.txt +++ b/doc/group-names.txt @@ -94,6 +94,7 @@ punch fallocate FALLOC_FL_PUNCH_HOLE qgroup btrfs qgroup feature quota filesystem usage quotas raid btrfs RAID +raid-stripe-tree btrfs raid-stripe-tree feature read_repair btrfs error correction on read failure realtime XFS realtime volumes recoveryloop crash recovery loops From patchwork Tue Dec 5 17:47:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480533 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="aMlShtK6" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 371A318D; Tue, 5 Dec 2023 09:47:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798427; x=1733334427; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=0vFVxyKpqAp7DKayL6Ik1UXOmUX6REN6nOAs9orTBSo=; b=aMlShtK6if3ijreij7xCKua+6vhxhfVo0PSW1t7AIPQXBSjHpfQ/s6r2 41ifJ+nn8qw0KKRgwum26lZXuMf88SEXWS+tbvk3rU59cX5QoEiAOXUdm s2TuaAEpBZkUXU3bRzPXZzyTyOG2Rmj2ua5rF+/Y7SKx3tuPZGi2+sBX2 BtYGWmEOLZSjPRzp/zweiKZn08SAFsG9RerythiJOEjyNxRrM8CEAF9Au 2zBlE2HzoP21dmWNLpD3NoC/59/RuHxRwXiay/l+KAxqpsb/n+LUanM8l TFgdz+QX9m6n/+c36ZYK/0OQ/F6nUXZu6ObdZR2NFewUlD9ETE2iY0MST w==; X-CSE-ConnectionGUID: z0DAkijnRXWAe+vedJ2nYQ== X-CSE-MsgGUID: 0wRV5txVTq62qcuhLa5+RQ== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944954" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:06 +0800 IronPort-SDR: AJypIuObM0/1S9Zf2x3Vn2Jo5J7VgsFUWp+ey36HjdIvpXIzO/Q42naXBV5o6S1P25CBYb9+gF FvrRaw52/29w== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:27 -0800 IronPort-SDR: 7Wa70RChXDSa7JQ9fxwtNePvxkDHB9RX/1XVfTyz+iMgMamiuXgPaxqZM5aql+l5C/GG7WL03/ JP+xxsuYE2Ug== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:06 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:02 -0800 Subject: [PATCH v3 2/7] common: add filter for btrfs raid-stripe dump Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-2-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=1491; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=0vFVxyKpqAp7DKayL6Ik1UXOmUX6REN6nOAs9orTBSo=; b=lL6AwdIzdDtveXDi9fYNMdOoukqqYhHlgpOEMKxXXvp7ZSxg7g6NMT+5ElVDw+4+nnNlviGzx r2bos5fiPWyDWbUOB636csHSpXqs85Vv0eR4xwcJBSz/FIoIzTBh8cE X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Signed-off-by: Johannes Thumshirn --- common/filter.btrfs | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/common/filter.btrfs b/common/filter.btrfs index 02c6b92dfa94..2003ba7b7015 100644 --- a/common/filter.btrfs +++ b/common/filter.btrfs @@ -125,5 +125,19 @@ _filter_btrfs_cloner_error() sed -e "s/\(clone failed:\) Operation not supported/\1 Invalid argument/g" } +# filter output of "btrfs inspect-internal dump-tree -t raid-stripe" +_filter_stripe_tree() +{ + sed -E -e "s/leaf [0-9]+ items [0-9]+ free space [0-9]+ generation [0-9]+ owner RAID_STRIPE_TREE/leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE/" \ + -e "s/leaf [0-9]+ flags 0x1\(WRITTEN\) backref revision 1/leaf XXXXXXXXX flags 0x1\(WRITTEN\) backref revision 1/" \ + -e "s/checksum stored [0-9a-f]+/checksum stored /" \ + -e "s/checksum calced [0-9a-f]+/checksum calced /" \ + -e "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}//" \ + -e "s/item ([0-9]+) key \([0-9]+ RAID_STRIPE ([0-9]+)\) itemoff [0-9]+ itemsize ([0-9]+)/item \1 key \(XXXXXX RAID_STRIPE \2\) itemoff XXXXX itemsize \3/" \ + -e "s/stripe ([0-9]+) devid ([0-9]+) physical [0-9]+/stripe \1 devid \2 physical XXXXXXXXX/" \ + -e "s/total bytes [0-9]+/total bytes XXXXXXXX/" \ + -e "s/bytes used [0-9]+/bytes used XXXXXX/" +} + # make sure this script returns success /bin/true From patchwork Tue Dec 5 17:47:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480534 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="M/H+HRGM" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC89818F; Tue, 5 Dec 2023 09:47:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798427; x=1733334427; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Y9EXjlti+5WseWTI6MzBguBOleH+YIzjaLKCtaO5Rz8=; b=M/H+HRGMs5TA6SWwDADoVLvtfvJtuwABmAs+KtxiKHw/TSNPP/swQWUg A64vWSjJERvSqzeBfcTaAEaIUOw4TjmkslwxO1onIjMd73gOwgyTkEhGR K0mJcT4zAmeA9WNwKqMDSoS3wb7FPQW3hsC/hkMreYB2RpeC9v74UQ7vh rM9o2OziaKmqci4lQW1MH273IExUXy0j//gOvmGMiTJbmyQIQR6Mca8wE rH61v/zKU5enVWk2Bx+hHcy/xMM45yxpZ7SHPHjyzK4jtwhBuO8U/DA3V 80JuiUEAIl/M5Vapollwueuxhc5NPa/+ca7YmTurVOndsvaN07FfPwvBN g==; X-CSE-ConnectionGUID: MQ4quvSWSxGeCkiqqVvidg== X-CSE-MsgGUID: ADL03WQhTE+VLr9fNBqffQ== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944957" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:07 +0800 IronPort-SDR: Xyb05o5Lr+UKpblc8jhoUy2zgoCq3mSmVFFNIpRrazkePOmHuu2M4VoMSHgt0Qh+hIQdLDbTaT RILwEbLMaYYg== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:28 -0800 IronPort-SDR: AlgkmIeZDJ89L6jDY8lyy0jDc4SUhR92soxvFKUDEgxZ9kLcSUAgNZFPlCGTo0SJ24anklK0P1 sayc+0xEVgrQ== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:07 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:03 -0800 Subject: [PATCH v3 3/7] btrfs: add fstest for stripe-tree metadata with 4k write Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-3-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=4097; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=Y9EXjlti+5WseWTI6MzBguBOleH+YIzjaLKCtaO5Rz8=; b=NEpjW8MFDnaz31YVk59FtG+zXwJNgGkF5s5qUpJX8rQVClCOz6vb/RntkleooA6YlIiVpoLWG QYkOVtQFLShCZ3N7pJyG6b3K3mX6aR+SBWhPIYgKvu5+F9kF+LoYc/M X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Test a simple 4k write on all RAID profiles currently supported with the raid-stripe-tree. Signed-off-by: Johannes Thumshirn --- tests/btrfs/304 | 53 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/304.out | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 111 insertions(+) diff --git a/tests/btrfs/304 b/tests/btrfs/304 new file mode 100755 index 000000000000..b773168ca3c0 --- /dev/null +++ b/tests/btrfs/304 @@ -0,0 +1,53 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 302 +# +# Test on-disk layout of RAID Stripe Tree Metadata writing 4k to a new file on +# a pristine file system. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_4k_write() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree + _scratch_mount + + $XFS_IO_PROG -fc "pwrite 0 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test basic 4k write =" +test_4k_write raid0 2 +test_4k_write raid1 2 +test_4k_write raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/304.out b/tests/btrfs/304.out new file mode 100644 index 000000000000..48036efbf0cf --- /dev/null +++ b/tests/btrfs/304.out @@ -0,0 +1,58 @@ +QA output created by 304 += Test basic 4k write = +==== Testing raid0 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5fed275e7617a806f94c173746a2a723 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 17:47:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480535 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="ODZIjeHt" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CC2A122; Tue, 5 Dec 2023 09:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798428; x=1733334428; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=t25s7OpZGFDlhQFDfE919RsW2ZGCcg2o6K96+AfSesU=; b=ODZIjeHtBnDLKImdM0CXCHGgDkDBkCc1fwjh2T4M65zabtbtZQqOlrmk gro0nN9Yu9Qlpch+Imxm1RlEEFGQi0b9gg41ogP6xvYo7f5d5ee78wpvu 6mMpG8Z/7F0nry+08NjtgO5EEj3OXT8gJf+RPDAJ2FRrWVqzbG7/EetEP owd6NMQNy0IRMFMKwnbYbugja0+AqmrUH4F4TSdAUN+OSMalxmBLlmCt6 OHONJEnHHpRWzbUwy31m6J1Krg+5lO7d7HfjMbAVf+tRo2i55ml/re+vK EMiAfzS26wejtBWs6+sZHGaLSaBJPWAxmJhJ8GqaQc31+BEKMz6Z0QF9C Q==; X-CSE-ConnectionGUID: EBiPorKQR8SQxahd+8fRQw== X-CSE-MsgGUID: tNVPaMLKTLWCp/WU2rnuSA== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944962" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:08 +0800 IronPort-SDR: MNknaOVLKWoyJP8ZTn8ytxOLY+aK7vFzXf/ZNx1fiJvQ5JjLj1lEnwQvbT171OpVCKQF+sxiyu Mss+LKhlCibg== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:29 -0800 IronPort-SDR: 9+NspR7ZgmWWft8woZORUC+qd5Mwm1zM6rwHUK28cYucZZ2h3qWKz+C2a2gA56Smvc5ZsK6ybm +52V7VzIrt9g== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:08 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:04 -0800 Subject: [PATCH v3 4/7] btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-4-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=5600; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=t25s7OpZGFDlhQFDfE919RsW2ZGCcg2o6K96+AfSesU=; b=hc1J/3860CXff3qv4CID4HClxQTPMSlXm/CnK9It+zt3g9sgJOUwnMuRLoQ8oc3md/RsKID7v nR44/rqHoXQDiuLwBlwIp42VHbkmqWQcbsKrgCXLLVsGAcFQjP6nbvY X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test-case writing 8k to a raid-stripe-tree formatted filesystem with one stripe pre-filled to 60k so the 8k are split into a 4k write finishing stripe 1 and a 4k write starting the next stripe. Signed-off-by: Johannes Thumshirn --- tests/btrfs/305 | 58 +++++++++++++++++++++++++++++++++++++ tests/btrfs/305.out | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 140 insertions(+) diff --git a/tests/btrfs/305 b/tests/btrfs/305 new file mode 100755 index 000000000000..db449167a58e --- /dev/null +++ b/tests/btrfs/305 @@ -0,0 +1,58 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 303 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 8k to a new file +# with a filesystem prepropulated, so that 4k of the write are written to the +# 1st stripe and 4k start a new stripe. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_8k_new_stripe() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree + _scratch_mount + + # Fill the first stripe up to 64k - 4k + $XFS_IO_PROG -fc "pwrite 0 60k" -c fsync "$SCRATCH_MNT/bar" | _filter_xfs_io + + # The actual 8k write + $XFS_IO_PROG -fc "pwrite 0 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 8k write to a new file so that 4k start a new stripe =" +test_8k_new_stripe raid0 2 +test_8k_new_stripe raid1 2 +test_8k_new_stripe raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/305.out b/tests/btrfs/305.out new file mode 100644 index 000000000000..7460501ef40b --- /dev/null +++ b/tests/btrfs/305.out @@ -0,0 +1,82 @@ +QA output created by 305 += Test 8k write to a new file so that 4k start a new stripe = +==== Testing raid0 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 61440/61440 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +9d3940adb41dd525e008a847e01b15f4 SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 61440) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 17:47:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480536 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="XOpgxvWu" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C7F3188; Tue, 5 Dec 2023 09:47:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798429; x=1733334429; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=4gvMjZyYXF4pEgxkZppRwNbOOMIgqNyuJZ6mJK51WJg=; b=XOpgxvWueLJ+ixA4dbS9LiMwp1et267FKvdAau/3WIFuFHIZPb/haWlE V0b20Dpjkp2brXmgmCHLSScUCCG+vULQGnD58o26Cy3DOUdZ5vsvmXpqK rb4ccngo1i0xWmuGi+oDWTjuQzM77m9YEMOp4MhDYMFZCpfgqgmgtQ/mF wBbfwHM86iklV4NV//FSSJchT6CSYYggmTr2aUjWQKWe2fr52DA00bPA1 nFZ+UdBQBMP0R6IXm05Vsc9WQLF4q1zsSvoHIBp0d2qMlj1OdlKzTQbwX cJgmK/VK2PhkKC+6mVYiAZeP0NgocFvZ29R/XW6GE6Y/UQtcFKjjRLIU7 g==; X-CSE-ConnectionGUID: o0K5u6BmTi2p4J0uVzKSYQ== X-CSE-MsgGUID: LzZVqUGpRH2x7zR9JBa7SQ== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944965" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:09 +0800 IronPort-SDR: zPXPJiQ6QtROQS/MYhK6vtVnfxB9TT3Ih1p4JluysGvzQegYAOFFpgfWWUAdip1tDscnukCbSl dCBPYYaxNZQw== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:30 -0800 IronPort-SDR: qBUxn3yuXhAtNGdaTlLCQmn3LzcJHj/vi4ooosieZCS78Aj+fIgAKDIg10oCDsEFHZdil1MJuq j35JQEAJOZVQ== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:09 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:05 -0800 Subject: [PATCH v3 5/7] btrfs: add fstest for writing to a file at an offset with RST Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-5-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=5202; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=4gvMjZyYXF4pEgxkZppRwNbOOMIgqNyuJZ6mJK51WJg=; b=X+PicZqoUHo/JjFjsfVwhc0ly5Z6deGONm2MEyG0sHGnhF3kAGr8QosPYbT0EbE2zl/zo/TJH WzmNpBRXcnMDmOwQP4ugckofcX/giwItiQVANMFd9Axv3mZizQsUSbz X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a fstest writing 4k at offset 64k to a file with one RAID tripe already pre-filled for a raid-stripe-tree formatted file system. Signed-off-by: Johannes Thumshirn --- tests/btrfs/306 | 56 +++++++++++++++++++++++++++++++++++++++ tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+) diff --git a/tests/btrfs/306 b/tests/btrfs/306 new file mode 100755 index 000000000000..55808a11cf85 --- /dev/null +++ b/tests/btrfs/306 @@ -0,0 +1,56 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 304 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 4k to an emppty +# file at offset 64k with one stripe pre-filled on an otherwise pristine +# filesystem. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_4k_write_64koff() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree + _scratch_mount + + # precondition one stripe + $XFS_IO_PROG -fc "pwrite 0 64k" "$SCRATCH_MNT/bar" | _filter_xfs_io + + $XFS_IO_PROG -fc "pwrite 64k 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} +echo "= Test 4k write to an empty file at offset 64k with one stripe prefilled =" +test_4k_write_64koff raid0 2 +test_4k_write_64koff raid1 2 +test_4k_write_64koff raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/306.out b/tests/btrfs/306.out new file mode 100644 index 000000000000..8031cddb79f7 --- /dev/null +++ b/tests/btrfs/306.out @@ -0,0 +1,75 @@ +QA output created by 306 += Test 4k write to an empty file at offset 64k with one stripe prefilled = +==== Testing raid0 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +381b0e7d72cb4f75286fe2b445e8d92a SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 4096) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 17:47:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480537 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="PgaqY65X" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A379196; Tue, 5 Dec 2023 09:47:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798430; x=1733334430; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=cInMmF2jJcXYU/g90qsDljttGMCC4VXVQJQZsR4f0CA=; b=PgaqY65XRkNmXSbmPRXPU4rkshayFaxiE06NQKOCGLGo44zznlGwk6IS f33pvLyOWHLdslmpB9v889hnbAwUb8/i9OKfM+36m+wr8NgcxgT1XujMM nspeuRhuq3dXPqjcPBSh0C9Fy4G48OfEoW7a3TPXn5LxxYIcVwGzRqK9W /w+maW1lV+rGIAXeh4FFe/6qpbeUfczWJr+XbC+rS3BXRc5M0vQ/7sqRq amScxyA/5uSpLnGadvfaXd0YKvcK5phuXEPxztXsC0sS/G+GwNMxD9kzd b9/vdrlS4fd1JlKT6SDuxt1gbFeAIQBLeSoP4qP1oRXh9g3D1sH+eEi73 w==; X-CSE-ConnectionGUID: OyG4LH/pR62+ob5U2OTXiA== X-CSE-MsgGUID: dsZDMgGXQyWW7fgTR+I/gg== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944972" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:10 +0800 IronPort-SDR: sQStszRFJ1+XMTbf8tlgadwc3zKMO8ef8mymuerdhL85OO01aWmKwyutlYSyMpu2TzCEHq+K6l QHLdqeicmS3Q== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:31 -0800 IronPort-SDR: eK/KJDOiDpOOGI29V+0jsQRLl+K7fZupHKPcYZuAxrWCXHEXINntZw0zlr2c8fulMgXRKJBG+b fgZjbcBbvHbg== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:10 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:06 -0800 Subject: [PATCH v3 6/7] btrfs: add fstests to write 128k to a RST filesystem Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-6-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=4443; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=cInMmF2jJcXYU/g90qsDljttGMCC4VXVQJQZsR4f0CA=; b=+Q/CXkHQj9L4xO2Na71ourhG0XAD307Eff/Bf9rwTxKJduDNm76qmCiR/9rzDDPPG9BqsgynR eI7/jvbch3ABL3bFKKtfeqfY9rfNd3xoB/1ld3REW6wjzfvQIQPAkYl X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test writing 128k to a file on an empty filesystem formatted with a raid-stripe-tree. Signed-off-by: Johannes Thumshirn --- tests/btrfs/307 | 53 +++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/307.out | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 118 insertions(+) diff --git a/tests/btrfs/307 b/tests/btrfs/307 new file mode 100755 index 000000000000..f5d5eb7ceee4 --- /dev/null +++ b/tests/btrfs/307 @@ -0,0 +1,53 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 305 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to a new +# file on a pristine filesystem +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_128k_write() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree + _scratch_mount + + $XFS_IO_PROG -fc "pwrite 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 128k write to empty file =" +test_128k_write raid0 2 +test_128k_write raid1 2 +test_128k_write raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/307.out b/tests/btrfs/307.out new file mode 100644 index 000000000000..b39f3162bff6 --- /dev/null +++ b/tests/btrfs/307.out @@ -0,0 +1,65 @@ +QA output created by 307 += Test 128k write to empty file = +==== Testing raid0 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid From patchwork Tue Dec 5 17:47:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13480538 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="qLnEPYWl" Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 722C018F; Tue, 5 Dec 2023 09:47:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1701798431; x=1733334431; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=L7RwUz52ypeOOfgAxBgi1jSj/I8ZxrfHoMionymVtig=; b=qLnEPYWlBgM3BpkzhQeeZPnvrrNqCsa6EUp2n8af5Veo3FEq+KxtI02T N8oqtMic6ZvssXkx+vaYbR3JEkXbOw31u31bQwWIVIelI7oghlblYkh5g /Ho8nBfz9qyOEmL8NWARlwuSQJd96rLFzMYKUCDxvkqE2b0E8dLI46yfg kzzECQPfqVODZP/mH9RnAwc735X3h2j7isVR9vuzj+sFB4vclkSOCBa1u aQfjjLdncC1PcYQ5b2M/BBfSqXrxXZJNAZ9kEswuBqOOhOnXhSvd8gRB7 xALKiWZ9qIkz7yp/g5s6TJcdolIgqyT1JGjF24HNxXuQPJRC6QGKgE5uX w==; X-CSE-ConnectionGUID: c/wA5/g3T4+GyZ7M1wmzvA== X-CSE-MsgGUID: VzaNV6N4Sq26EytLAFhCPg== X-IronPort-AV: E=Sophos;i="6.04,252,1695657600"; d="scan'208";a="3944975" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2023 01:47:11 +0800 IronPort-SDR: Ic6XV94VD/jWOaiWqBqZlxAJP1TDOcoxQFzF9m+W/yP2r91mGRrTwZp8VzkEWxEK9RIQ+HL9rx WIw2/zYpCZWA== Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 05 Dec 2023 08:52:32 -0800 IronPort-SDR: i2esY4esalCewl0RQdr/+lg3aQvtEWiqz3x86Tn2ELnTbfHu/vu+PEZzUgCSHtLsrtl0jLKR+k d5TGqZBnUjDA== WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 05 Dec 2023 09:47:11 -0800 From: Johannes Thumshirn Date: Tue, 05 Dec 2023 09:47:07 -0800 Subject: [PATCH v3 7/7] btrfs: add fstest for overwriting a file partially with RST Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231205-btrfs-raid-v3-7-0e857a5439a2@wdc.com> References: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> In-Reply-To: <20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com> To: Anand Jain , Zorro Lang Cc: Filipe Manana , fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1701798423; l=6685; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=L7RwUz52ypeOOfgAxBgi1jSj/I8ZxrfHoMionymVtig=; b=JwraXZiezt39foJdf7Hl/ffiCrI6NIzmcSo7J8F8b6L5PIn8b+r8AC8KDvn4/3sMRZmG7wIhh M+9XnbQN/UbBB8AlP+8kX2e0Kj7PsJv/LOK9qJM7gcujoQeHYdedtqV X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Add a test writing 128k to an empty file with one stripe already pre-filled on-disk. Then overwrite a portion of the file in the middle. Signed-off-by: Johannes Thumshirn --- tests/btrfs/308 | 56 +++++++++++++++++++++++++++ tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 162 insertions(+) diff --git a/tests/btrfs/308 b/tests/btrfs/308 new file mode 100755 index 000000000000..e57e32bc7fa2 --- /dev/null +++ b/tests/btrfs/308 @@ -0,0 +1,56 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Western Digital Cooperation. All Rights Reserved. +# +# FS QA Test 306 +# +# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to an empty +# file on a filesystem that has one stripe already pre-filled. Afterwards +# overwrite a portion of the file. +# +. ./common/preamble +_begin_fstest auto quick raid remount volume raid-stripe-tree + +. ./common/filter +. ./common/filter.btrfs + +_supported_fs btrfs +_require_btrfs_command inspect-internal dump-tree +_require_btrfs_mkfs_feature "raid-stripe-tree" +_require_scratch_dev_pool 4 +_require_btrfs_fs_feature "raid_stripe_tree" + +test_128k_write_overwrite() +{ + local profile=$1 + local ndevs=$2 + + _scratch_dev_pool_get $ndevs + + echo "==== Testing $profile ====" + _scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree + _scratch_mount + + $XFS_IO_PROG -fc "pwrite -W 0 32k" "$SCRATCH_MNT/bar" | _filter_xfs_io + $XFS_IO_PROG -fc "pwrite -W 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io + $XFS_IO_PROG -fc "pwrite -W 64k 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io + + _scratch_cycle_mount + md5sum "$SCRATCH_MNT/foo" | _filter_scratch + + _scratch_unmount + + $BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\ + _filter_btrfs_version | _filter_stripe_tree + + _scratch_dev_pool_put +} + +echo "= Test 128k write to empty file with 1st stripe partially prefilled then overwrite =" +test_128k_write_overwrite raid0 2 +test_128k_write_overwrite raid1 2 +test_128k_write_overwrite raid10 4 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/308.out b/tests/btrfs/308.out new file mode 100644 index 000000000000..c93a3a43b248 --- /dev/null +++ b/tests/btrfs/308.out @@ -0,0 +1,106 @@ +QA output created by 308 += Test 128k write to empty file with 1st stripe partially prefilled then overwrite = +==== Testing raid0 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 2 physical XXXXXXXXX + item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX + item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 24 + encoding: RAID0 + stripe 0 devid 1 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid1 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 131072) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID1 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid +==== Testing raid10 ==== +wrote 32768/32768 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 65536 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +d48858312a922db7eb86377f638dbc9f SCRATCH_MNT/foo + +raid stripe tree key (RAID_STRIPE_TREE ROOT_ITEM 0) +leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE +leaf XXXXXXXXX flags 0x1(WRITTEN) backref revision 1 +checksum stored +checksum calced +fs uuid +chunk uuid + item 0 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 1 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 2 key (XXXXXX RAID_STRIPE 65536) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 3 physical XXXXXXXXX + stripe 1 devid 4 physical XXXXXXXXX + item 3 key (XXXXXX RAID_STRIPE 32768) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX + item 4 key (XXXXXX RAID_STRIPE 8192) itemoff XXXXX itemsize 40 + encoding: RAID10 + stripe 0 devid 1 physical XXXXXXXXX + stripe 1 devid 2 physical XXXXXXXXX +total bytes XXXXXXXX +bytes used XXXXXX +uuid