From patchwork Mon Sep 11 13:22:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13380236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ADBECA0EC6 for ; Mon, 11 Sep 2023 22:08:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356185AbjIKWDI (ORCPT ); Mon, 11 Sep 2023 18:03:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237934AbjIKNXR (ORCPT ); Mon, 11 Sep 2023 09:23:17 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FFA9CD7 for ; Mon, 11 Sep 2023 06:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694438592; x=1725974592; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=v5sjPWXNw95QgO2j1uFYVXOa4VWaWPbh/DcihNdSso4=; b=JviwefIlxeuz2sFvd84VPSK1Nj9GCNeQU/u6KXYr6k9iNeIH2tLCqNxe l/XAPnagKE9i82OwQxk9w9m7YM1bsuHuQY+dx12U86iSYmPjm1eVnGCZy 9lJ2HlOInoyqDKv/P1YP5zby1TwiB4Q1mXTM+ax1ZU8YlFO7P99cJCm0s 1WQ25ZiwGGmH4iSb6W/NSxwRUlCynVo2ZZlqbS1GH1coLE1cTJcLGOeVn LVYEi0SF7Tt62ApbUnbPLphJQAKSiGeQEQA7qJELSbGXFFM3VigJiObdD j/5KIIVxRBfkJPWq7UIf20J78B+BR7AryVUu+p9VQY4FRs33LwaIfivb5 Q==; X-CSE-ConnectionGUID: Yj0R/gFURUGsBiEtI3OTZg== X-CSE-MsgGUID: WFoBy5DlQTyY1Ns/bU2w0Q== X-IronPort-AV: E=Sophos;i="6.02,244,1688400000"; d="scan'208";a="248143279" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Sep 2023 21:23:11 +0800 IronPort-SDR: Lk13+n1PHeCZRwcvrPBe+YNB7gqOVqTXKYvU+KtGOhw2bxg6TZk4ATZtpzHjwI4iYfWL2+aCse KVE1zLc3vyw6mN5izIkthTsiLENAjwdqHOUtR3M9JQ4nvl1dy+uLPT4bGkEXvsKBLZ5cnErox3 vtijKctjXyAI/JpXpsJ0DXopIiXPyIfpTRQcij+JOj25d8X2JKFnZImrUlBdAzgKofL/JYYTRK ZQ0yvDjsLjNKpMnw+Q3Gtj0aUwWb8mbai2G70MUiwJ3LI2sctM+iGWh8A+CzWiYxyGmakDTGTi mPc= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 11 Sep 2023 05:30:17 -0700 IronPort-SDR: cAibKOUD4oBNL+IxvCZZIHyDTQzw3mX7zgrGLeM++AqoT8kqC+z2zr1VbyVFa7s3D96gR2ON90 w//xwv9ZEmM60TJ2xP7HoIT9jbQMuu0PZaPKXRHpPq2SmlMCavEYKO+ueVBU3e0JQQMVcaJjwc O4qDkBmwfu2jh1ZhrF2oKrfFqkoKj/ONSCrzZGzOWyLvFK6OMdQmwyTrkTGptkxVwnaHcUQYH4 k8yIGhHqoqTyl9CLdUKePThKT3M8DNpFO1m53A0uswg2dEJptx9cEjt662ZyGSJHEu5UnqLybk CvU= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 11 Sep 2023 06:23:11 -0700 From: Johannes Thumshirn To: David Sterba Cc: Johannes Thumshirn , linux-btrfs@vger.kernel.org Subject: [PATCH 1/6] btrfs-progs: add raid-stripe-tree definitions Date: Mon, 11 Sep 2023 06:22:57 -0700 Message-ID: <20230911-raid-stripe-tree-v1-1-c8337f7444b5@wdc.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> References: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> MIME-Version: 1.0 X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694438542; l=3337; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=v5sjPWXNw95QgO2j1uFYVXOa4VWaWPbh/DcihNdSso4=; b=zmx0WYmHngfI2tBUw1kKTDVzGndQ6IqYwoEnw5cXj+2tLpQOjPhPiJwM9ol/lUKnHk+1JLAL6 EL4Q7r222pPDe1OE1uM6rmwA6u9KYAeFwkfMhzCKDnJJ2LihilQUycC X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add the definitions for the on-disk format of the RAID stripe tree. Signed-off-by: Johannes Thumshirn --- kernel-shared/accessors.h | 37 +++++++++++++++++++++++++++++++++++++ kernel-shared/uapi/btrfs_tree.h | 25 +++++++++++++++++++++++++ 2 files changed, 62 insertions(+) diff --git a/kernel-shared/accessors.h b/kernel-shared/accessors.h index 625acfbe8ca7..80635f31c90b 100644 --- a/kernel-shared/accessors.h +++ b/kernel-shared/accessors.h @@ -270,6 +270,43 @@ BTRFS_SETGET_FUNCS(free_space_extent_count, struct btrfs_free_space_info, extent_count, 32); BTRFS_SETGET_FUNCS(free_space_flags, struct btrfs_free_space_info, flags, 32); +/* struct btrfs_stripe_extent */ +BTRFS_SETGET_FUNCS(stripe_extent_encoding, struct btrfs_stripe_extent, encoding, 8); +BTRFS_SETGET_FUNCS(raid_stride_devid, struct btrfs_raid_stride, devid, 64); +BTRFS_SETGET_FUNCS(raid_stride_offset, struct btrfs_raid_stride, offset, 64); +BTRFS_SETGET_FUNCS(raid_stride_length, struct btrfs_raid_stride, length, 64); + +static inline struct btrfs_raid_stride *btrfs_raid_stride_nr( + struct btrfs_stripe_extent *dps, + int nr) +{ + unsigned long offset = (unsigned long)dps; + offset += offsetof(struct btrfs_stripe_extent, strides); + offset += nr * sizeof(struct btrfs_raid_stride); + return (struct btrfs_raid_stride *)offset; +} + +static inline u64 btrfs_raid_stride_devid_nr(struct extent_buffer *eb, + struct btrfs_stripe_extent *dps, + int nr) +{ + return btrfs_raid_stride_devid(eb, btrfs_raid_stride_nr(dps, nr)); +} + +static inline u64 btrfs_raid_stride_offset_nr(struct extent_buffer *eb, + struct btrfs_stripe_extent *dps, + int nr) +{ + return btrfs_raid_stride_offset(eb, btrfs_raid_stride_nr(dps, nr)); +} + +static inline u64 btrfs_raid_stride_length_nr(struct extent_buffer *eb, + struct btrfs_stripe_extent *dps, + int nr) +{ + return btrfs_raid_stride_length(eb, btrfs_raid_stride_nr(dps, nr)); +} + /* struct btrfs_inode_ref */ BTRFS_SETGET_FUNCS(inode_ref_name_len, struct btrfs_inode_ref, name_len, 16); BTRFS_SETGET_FUNCS(inode_ref_index, struct btrfs_inode_ref, index, 64); diff --git a/kernel-shared/uapi/btrfs_tree.h b/kernel-shared/uapi/btrfs_tree.h index ad555e7055ab..e89246a3fdbe 100644 --- a/kernel-shared/uapi/btrfs_tree.h +++ b/kernel-shared/uapi/btrfs_tree.h @@ -699,6 +699,31 @@ struct btrfs_super_block { __u8 padding[565]; } __attribute__ ((__packed__)); +struct btrfs_raid_stride { + /* btrfs device-id this raid extent lives on */ + __le64 devid; + /* offset from the devextent start */ + __le64 offset; + /* length of the stride on disk */ + __le64 length; +} __attribute__ ((__packed__)); + +#define BTRFS_STRIPE_DUP 0 +#define BTRFS_STRIPE_RAID0 1 +#define BTRFS_STRIPE_RAID1 2 +#define BTRFS_STRIPE_RAID1C3 3 +#define BTRFS_STRIPE_RAID1C4 4 +#define BTRFS_STRIPE_RAID5 5 +#define BTRFS_STRIPE_RAID6 6 +#define BTRFS_STRIPE_RAID10 7 + +struct btrfs_stripe_extent { + u8 encoding; + u8 reserved[7]; + /* array of raid strides this stripe is comprised of */ + struct btrfs_raid_stride strides; +} __attribute__ ((__packed__)); + #define BTRFS_FREE_SPACE_EXTENT 1 #define BTRFS_FREE_SPACE_BITMAP 2 From patchwork Mon Sep 11 13:22:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13380234 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BF6FCA0EC8 for ; Mon, 11 Sep 2023 22:07:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356133AbjIKWDC (ORCPT ); Mon, 11 Sep 2023 18:03:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237936AbjIKNXR (ORCPT ); Mon, 11 Sep 2023 09:23:17 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAA3712A for ; Mon, 11 Sep 2023 06:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694438593; x=1725974593; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KKDM20AA24mnFWIBaq+OM7WQs6kIS7ubq0Wq11kxM7o=; b=c0rf7XQa+57oynd8jr8961F7uA7ZeqoMDApsXGW78z85DY1tHSccEjmy NcndVqhW1kL8Vp4qTL3JTFR22RtNLXKcr02elIyZrgUjN6ePyLFGPunB3 zQ0Q7L86giaGvcJbsCe9SU/AxTOBwnWUwrqnzam7XqWvEbNWaTe2nWo+k SKzmUXDpObFCFsj5ZtQIXntf9eFqKhEzVdIfahjgCAdvBTbrP00w1elFF xiWzFXK7jfklfRvxxzUZivTQMmUxGwW31bGeBQnAx4HPDmjHfGljW43fJ PojYLfS2aQJeVh10gfmVD05l3+qQwDldEBM3aPasLwG/SsLvm3+tn8y+o A==; X-CSE-ConnectionGUID: UrfpZVrzQ5qx7OVs8YZWkQ== X-CSE-MsgGUID: dPmwOBzGS2umbhWlHiCwLg== X-IronPort-AV: E=Sophos;i="6.02,244,1688400000"; d="scan'208";a="248143280" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Sep 2023 21:23:12 +0800 IronPort-SDR: 6JgWW04Wn+2CogMIhOz030zT7JPbLzz1dN9vH79eBfjakbO0s77IpJrFcY6RL5X5nEDugdleSO WxB6NnQ1Fhs2MreffQfBlxEmJuQwe9DDjY9HgXbHyozspM2RMVgUGkhlcJyQ7BzlTDY/hhw0rf D87mtM8i2SwXWw/62b3/szsdbZjPZtmxeP0cAiEfH+h5WRHrMJx0gY4RvL3wRGF7QGcWQmDVzu 5HaJjCHbEHga+H+kcHRK9gx/d/DXc3Qi9AFDAWLCjh6cXO5zEe3jXxBeegXAlAUomFHg6tT2Tu hIA= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 11 Sep 2023 05:30:18 -0700 IronPort-SDR: MO8rPAJvMvrfjzQEMuTrLwo6AlMoYrwIWLnPJ3J8nIbVw6Uo2aYULYdpmPVf2iY/2cyUMeyhpz HTzpwkIn2wo+hBibvfOzd3x8TwKmabcn4CpSwTMI29fu3nHdeCzVcVQJ0CHbnxvIH5D2a3QjfG 8nERUsw/+YiR94Lcfz5tk28wVw6O6y0cpenO3AufxRy1XItruL5P2D7EglfR6j1Ta5pidSummv NQfotfZjVwxzSDtUDucW2E6c/8GRNYQxFjjyTxNguvpxoy7TNhNmLb1LTnq1mT4GEERIlPSinl PvY= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 11 Sep 2023 06:23:12 -0700 From: Johannes Thumshirn To: David Sterba Cc: Johannes Thumshirn , linux-btrfs@vger.kernel.org Subject: [PATCH 2/6] btrfs-progs: read fs with stripe tree from disk Date: Mon, 11 Sep 2023 06:22:58 -0700 Message-ID: <20230911-raid-stripe-tree-v1-2-c8337f7444b5@wdc.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> References: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> MIME-Version: 1.0 X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694438542; l=5277; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=KKDM20AA24mnFWIBaq+OM7WQs6kIS7ubq0Wq11kxM7o=; b=xYvN9Dd2Zf2B822BZGPhWR1JuPnOpU40gSA34DH6mES3ZNhb2vKsg2wGc1VGgk8BApPBRHl3r 9QX6HOil2xNBQM3DlQ8kw8m3kuJq2u2/wgFf/2GPuGpOlgMY2t6vymr X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When encountering a filesystem formatted with the raid stripe tree feature, read it from disk. Also add the incompat declaration to the tree printer. Signed-off-by: Johannes Thumshirn --- kernel-shared/ctree.h | 1 + kernel-shared/disk-io.c | 28 +++++++++++++++++++++++++++- kernel-shared/print-tree.c | 1 + kernel-shared/uapi/btrfs.h | 1 + kernel-shared/uapi/btrfs_tree.h | 3 +++ 5 files changed, 33 insertions(+), 1 deletion(-) diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 5d3392ae82a6..035358436d8f 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -298,6 +298,7 @@ struct btrfs_fs_info { struct btrfs_root *quota_root; struct btrfs_root *uuid_root; struct btrfs_root *block_group_root; + struct btrfs_root *stripe_root; struct rb_root global_roots_tree; struct rb_root fs_root_tree; diff --git a/kernel-shared/disk-io.c b/kernel-shared/disk-io.c index 442d3af8bc01..3ab32f7ad910 100644 --- a/kernel-shared/disk-io.c +++ b/kernel-shared/disk-io.c @@ -790,6 +790,10 @@ struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info, return fs_info->block_group_root ? fs_info->block_group_root : ERR_PTR(-ENOENT); + if (location->objectid == BTRFS_RAID_STRIPE_TREE_OBJECTID) + return fs_info->stripe_root ? fs_info->stripe_root : + ERR_PTR(-ENOENT); + BUG_ON(location->objectid == BTRFS_TREE_RELOC_OBJECTID); node = rb_search(&fs_info->fs_root_tree, (void *)&objectid, @@ -822,6 +826,9 @@ void btrfs_free_fs_info(struct btrfs_fs_info *fs_info) if (fs_info->quota_root) free(fs_info->quota_root); + if (fs_info->stripe_root) + free(fs_info->stripe_root); + free_global_roots_tree(&fs_info->global_roots_tree); free(fs_info->tree_root); free(fs_info->chunk_root); @@ -846,12 +853,14 @@ struct btrfs_fs_info *btrfs_new_fs_info(int writable, u64 sb_bytenr) fs_info->dev_root = calloc(1, sizeof(struct btrfs_root)); fs_info->quota_root = calloc(1, sizeof(struct btrfs_root)); fs_info->uuid_root = calloc(1, sizeof(struct btrfs_root)); + fs_info->stripe_root = calloc(1, sizeof(struct btrfs_root)); fs_info->block_group_root = calloc(1, sizeof(struct btrfs_root)); fs_info->super_copy = calloc(1, BTRFS_SUPER_INFO_SIZE); if (!fs_info->tree_root || !fs_info->chunk_root || !fs_info->dev_root || !fs_info->quota_root || !fs_info->uuid_root || - !fs_info->block_group_root || !fs_info->super_copy) + !fs_info->block_group_root || !fs_info->super_copy || + !fs_info->stripe_root) goto free_all; extent_buffer_init_cache(fs_info); @@ -1260,6 +1269,21 @@ int btrfs_setup_all_roots(struct btrfs_fs_info *fs_info, u64 root_tree_bytenr, return -EIO; } +#if EXPERIMENTAL + if (btrfs_fs_incompat(fs_info, RAID_STRIPE_TREE)) { + ret = btrfs_find_and_setup_root(root, fs_info, + BTRFS_RAID_STRIPE_TREE_OBJECTID, + fs_info->stripe_root); + if (ret) { + free(fs_info->stripe_root); + fs_info->stripe_root = NULL; + } else { + set_bit(BTRFS_ROOT_TRACK_DIRTY, + &fs_info->stripe_root->state); + } + } +#endif + if (maybe_load_block_groups(fs_info, flags)) { ret = btrfs_read_block_groups(fs_info); /* @@ -1317,6 +1341,8 @@ void btrfs_release_all_roots(struct btrfs_fs_info *fs_info) free_extent_buffer(fs_info->chunk_root->node); if (fs_info->uuid_root) free_extent_buffer(fs_info->uuid_root->node); + if (fs_info->stripe_root) + free_extent_buffer(fs_info->stripe_root->node); } static void free_map_lookup(struct cache_extent *ce) diff --git a/kernel-shared/print-tree.c b/kernel-shared/print-tree.c index 0f7f7b72f96a..3eff82b364ef 100644 --- a/kernel-shared/print-tree.c +++ b/kernel-shared/print-tree.c @@ -1708,6 +1708,7 @@ static struct readable_flag_entry incompat_flags_array[] = { DEF_INCOMPAT_FLAG_ENTRY(RAID1C34), DEF_INCOMPAT_FLAG_ENTRY(ZONED), DEF_INCOMPAT_FLAG_ENTRY(EXTENT_TREE_V2), + DEF_INCOMPAT_FLAG_ENTRY(RAID_STRIPE_TREE), }; static const int incompat_flags_num = sizeof(incompat_flags_array) / sizeof(struct readable_flag_entry); diff --git a/kernel-shared/uapi/btrfs.h b/kernel-shared/uapi/btrfs.h index 85b04f89a2a9..c750b7aa921f 100644 --- a/kernel-shared/uapi/btrfs.h +++ b/kernel-shared/uapi/btrfs.h @@ -356,6 +356,7 @@ _static_assert(sizeof(struct btrfs_ioctl_fs_info_args) == 1024); #define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11) #define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12) #define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) +#define BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE (1ULL << 14) struct btrfs_ioctl_feature_flags { __u64 compat_flags; diff --git a/kernel-shared/uapi/btrfs_tree.h b/kernel-shared/uapi/btrfs_tree.h index e89246a3fdbe..f4a5fb14b4f0 100644 --- a/kernel-shared/uapi/btrfs_tree.h +++ b/kernel-shared/uapi/btrfs_tree.h @@ -73,6 +73,9 @@ /* Holds the block group items for extent tree v2. */ #define BTRFS_BLOCK_GROUP_TREE_OBJECTID 11ULL +/* holds raid stripe entries */ +#define BTRFS_RAID_STRIPE_TREE_OBJECTID 12ULL + /* device stats in the device tree */ #define BTRFS_DEV_STATS_OBJECTID 0ULL From patchwork Mon Sep 11 13:22:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13380226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E646CA0ECB for ; Mon, 11 Sep 2023 22:07:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355929AbjIKWC1 (ORCPT ); Mon, 11 Sep 2023 18:02:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237939AbjIKNXS (ORCPT ); Mon, 11 Sep 2023 09:23:18 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62F75193 for ; Mon, 11 Sep 2023 06:23:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694438594; x=1725974594; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SDYGdOJH0EumI768Ubnvw2FIaQ00d/j26jaN2tdsPUk=; b=Wef4Me+M+nm+zAIBTZO0SVx+pdzBQNFDpMg4H3ysutd4MFDvHVb05nKg uiD2QgP5FEDmGxdTuywzkXu6Blns6GxY1BzgUvnUbIwrqL1xc7TaJ2IAU tiaSnOHmmWFB/yYEmv4piXI7kgvG/tvzMp+uPEYukS24lex/j8cY4GdnJ cCsUHZBuRqIsbt5nCIcC/00sFk3YNyBCNK7gsBTu+3cJn6npDVNkRoHxG JjQIvml+XItbn99kQJbDXzV9QsHYMYFoY4NhIWHqRAnGlzWbgTMDSHybH 53dPcyJUNvyK1MzqNE64X8TwhXrXVZeVwSUQN0QkZs8pK9E+FLEQ2IuI3 w==; X-CSE-ConnectionGUID: y5v9epF7Qkyy8RloYOpX2A== X-CSE-MsgGUID: 1XpEZX+iQpiv+9pFXySPNw== X-IronPort-AV: E=Sophos;i="6.02,244,1688400000"; d="scan'208";a="248143281" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Sep 2023 21:23:13 +0800 IronPort-SDR: /RbPAkl7yPgNouhQn3Q+mOiOUP1aRzLeWC1LWO83bhfM9bj6f78JdPl5kJp2ZUMcOOb9myYvqW hPlfpq+cvvX2yRQm83QY88x32iobDPpwDO6JQbDCGNoL0fAQPxm7bceHo5qK/vv07uRg2L1D01 lEqDB+lxKQqRjJJnPX1LmyTflajZ5lesd4LYodZbYXvdlNMCTqw4ufC/KwIGdySbOMvtEgE9Nl stSzNf2eHG5nOJPeBWdQNOWBtkZH7iwQT8iy1fshL6G8FZn0xOPJx8nfOIqV+/RhQ2wRCs1PGn RZQ= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 11 Sep 2023 05:30:18 -0700 IronPort-SDR: 7aEmYPPO/PKLbAQV9KMVXdUR28cWisBzkIV9z3xhQaoLLwHdBLPx+DIXNlVYjVX7rN2SbeSidJ emJzhd0DvO5uQflAo9yn6+nldvi3FbIw10NImZm4MqsOVrORwm2xMjpGIKGEHTL/g9xGgDugLX 9jba2UAtZrxPgeoTAd6JmExwwl5+7pOEq5lyjoVQM+RhMDs3twtQUxddZRbhpiyoemEvg2G9iY mLLOaDp+EjQcbT63AmjPDzqL0xWX5eK8lb2HoARCbZXXR9a34R9mCTTQSvF8CsMe3bazved8xS 63w= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 11 Sep 2023 06:23:13 -0700 From: Johannes Thumshirn To: David Sterba Cc: Johannes Thumshirn , linux-btrfs@vger.kernel.org Subject: [PATCH 3/6] btrfs-progs: add dump tree support for the raid stripe tree Date: Mon, 11 Sep 2023 06:22:59 -0700 Message-ID: <20230911-raid-stripe-tree-v1-3-c8337f7444b5@wdc.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> References: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> MIME-Version: 1.0 X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694438542; l=4472; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=SDYGdOJH0EumI768Ubnvw2FIaQ00d/j26jaN2tdsPUk=; b=K7hEWrPZdjPnwnnm/GiPTntGKPdSOf6+TDQ+ry/l9UVyiot0Fr53EZwhVd4IjJ6/czzCxy1Mq xkcfMernbyIDF4pI30pvsxEGp6+5Sm9D8zMvbzeS+45DmUTgkAHjQrx X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add support for the RAID stripe tree to btrfs inspect-internal dump-tree. Signed-off-by: Johannes Thumshirn --- cmds/inspect-dump-tree.c | 5 +++++ kernel-shared/ctree.h | 3 +++ kernel-shared/print-tree.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 60 insertions(+) diff --git a/cmds/inspect-dump-tree.c b/cmds/inspect-dump-tree.c index bfc0fff148dd..328ecd76c8a0 100644 --- a/cmds/inspect-dump-tree.c +++ b/cmds/inspect-dump-tree.c @@ -170,6 +170,7 @@ static u64 treeid_from_string(const char *str, const char **end) { "TREE_RELOC", BTRFS_TREE_RELOC_OBJECTID }, { "DATA_RELOC", BTRFS_DATA_RELOC_TREE_OBJECTID }, { "BLOCK_GROUP_TREE", BTRFS_BLOCK_GROUP_TREE_OBJECTID }, + { "RAID_STRIPE", BTRFS_RAID_STRIPE_TREE_OBJECTID }, }; if (strncasecmp("BTRFS_", str, strlen("BTRFS_")) == 0) @@ -729,6 +730,10 @@ again: if (!skip) pr_verbose(LOG_DEFAULT, "block group"); break; + case BTRFS_RAID_STRIPE_TREE_OBJECTID: + if (!skip) + printf("raid stripe"); + break; default: if (!skip) { pr_verbose(LOG_DEFAULT, "file"); diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 035358436d8f..26ab0b6aea17 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -650,6 +650,9 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info) #define BTRFS_QGROUP_LIMIT_KEY 244 #define BTRFS_QGROUP_RELATION_KEY 246 + +#define BTRFS_RAID_STRIPE_KEY 247 + /* * Obsolete name, see BTRFS_TEMPORARY_ITEM_KEY. */ diff --git a/kernel-shared/print-tree.c b/kernel-shared/print-tree.c index 3eff82b364ef..d719ebaebb40 100644 --- a/kernel-shared/print-tree.c +++ b/kernel-shared/print-tree.c @@ -637,6 +637,51 @@ static void print_free_space_header(struct extent_buffer *leaf, int slot) (unsigned long long)btrfs_free_space_bitmaps(leaf, header)); } +struct raid_encoding_map { + u8 encoding; + char name[16]; +}; + +static const struct raid_encoding_map raid_map[] = { + { BTRFS_STRIPE_DUP, "DUP" }, + { BTRFS_STRIPE_RAID0, "RAID0" }, + { BTRFS_STRIPE_RAID1, "RAID1" }, + { BTRFS_STRIPE_RAID1C3, "RAID1C3" }, + { BTRFS_STRIPE_RAID1C4, "RAID1C4" }, + { BTRFS_STRIPE_RAID5, "RAID5" }, + { BTRFS_STRIPE_RAID6, "RAID6" }, + { BTRFS_STRIPE_RAID10, "RAID10" } +}; + +static const char *stripe_encoding_name(u8 encoding) +{ + for (int i = 0; i < ARRAY_SIZE(raid_map); i++) { + if (raid_map[i].encoding == encoding) + return raid_map[i].name; + } + + return "UNKNOWN"; +} + +static void print_raid_stripe_key(struct extent_buffer *eb, + u32 item_size, + struct btrfs_stripe_extent *stripe) +{ + int num_stripes; + u8 encoding = btrfs_stripe_extent_encoding(eb, stripe); + int i; + + num_stripes = (item_size - offsetof(struct btrfs_stripe_extent, strides)) / + sizeof(struct btrfs_raid_stride); + + printf("\t\t\tencoding: %s\n", stripe_encoding_name(encoding)); + for (i = 0; i < num_stripes; i++) + printf("\t\t\tstripe %d devid %llu physical %llu length %llu\n", i, + (unsigned long long)btrfs_raid_stride_devid_nr(eb, stripe, i), + (unsigned long long)btrfs_raid_stride_offset_nr(eb, stripe, i), + (unsigned long long)btrfs_raid_stride_length_nr(eb, stripe, i)); +} + void print_key_type(FILE *stream, u64 objectid, u8 type) { static const char* key_to_str[256] = { @@ -681,6 +726,7 @@ void print_key_type(FILE *stream, u64 objectid, u8 type) [BTRFS_PERSISTENT_ITEM_KEY] = "PERSISTENT_ITEM", [BTRFS_UUID_KEY_SUBVOL] = "UUID_KEY_SUBVOL", [BTRFS_UUID_KEY_RECEIVED_SUBVOL] = "UUID_KEY_RECEIVED_SUBVOL", + [BTRFS_RAID_STRIPE_KEY] = "RAID_STRIPE_KEY", }; if (type == 0 && objectid == BTRFS_FREE_SPACE_OBJECTID) { @@ -793,6 +839,9 @@ void print_objectid(FILE *stream, u64 objectid, u8 type) case BTRFS_CSUM_CHANGE_OBJECTID: fprintf(stream, "CSUM_CHANGE"); break; + case BTRFS_RAID_STRIPE_TREE_OBJECTID: + fprintf(stream, "RAID_STRIPE_TREE"); + break; case (u64)-1: fprintf(stream, "-1"); break; @@ -1469,6 +1518,9 @@ void btrfs_print_leaf(struct extent_buffer *eb, unsigned int mode) case BTRFS_TEMPORARY_ITEM_KEY: print_temporary_item(eb, ptr, objectid, offset); break; + case BTRFS_RAID_STRIPE_KEY: + print_raid_stripe_key(eb, item_size, ptr); + break; }; fflush(stdout); } From patchwork Mon Sep 11 13:23:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13380215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A6A3CA0ECE for ; Mon, 11 Sep 2023 22:06:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355573AbjIKWBG (ORCPT ); Mon, 11 Sep 2023 18:01:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237941AbjIKNXT (ORCPT ); Mon, 11 Sep 2023 09:23:19 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2155612A for ; Mon, 11 Sep 2023 06:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694438595; x=1725974595; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=H022AhqxQLH8oJ/IhL9gK9CUAeyKynw0SLTJOhcuOHQ=; b=YilrwGy5AlUpmOxGxVMDPCg5FoylLJqk2YoNFnTAqfzcNgoNFC4orejf HiXZM37sQgQj6n4dy7YGQFJ3ehXxEznl2KXzwxtF0UlZSRqGQnCT78VW+ NvNOoSoYQVdeOEIApTj6YPMHaeLmKUdyw2WJYVd9TMGf/NlAfJ9CA6lZx vJo+W9TNDUOD19j3A2E7Tq8UkWr7z1sCFDY8uguyhfgsOE9zeXKBZ1KSt SHOEDFfqOohFFmKYcrpxoYnFq2UfhNgV50a7YdCpl69rgrNqIIEYulEM4 WloKy2f38sEi2ZuhPdJHxzBJqhdI8QgekUxDebYQn01y4CXBkM56rLOc7 A==; X-CSE-ConnectionGUID: 8ES1wyiBRRGwLWNFUoUwww== X-CSE-MsgGUID: OHUcN68/TdyRNJePVtl7FQ== X-IronPort-AV: E=Sophos;i="6.02,244,1688400000"; d="scan'208";a="248143283" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Sep 2023 21:23:14 +0800 IronPort-SDR: 6DD1LbTZDt0YQB/csjrBy5moV0FrM5Ce5Ab1rVbuMlM+WjrhvVKXZpdvTZlS/TDmU1uf7rbuwg B9TSuEa4//11sDNO6axZ7hYtAIrGCT+fWasseomEWjtiCavUUXaN/j9VWsN7taj5XBd5rXg6v9 MUFgfTgxJuaz1+SyarY7D8p6E4Z2Hd8npHMZrPI/R343zY1AHGlOdoF/dltxASWzCO0UbkfX/1 Rgw5ZLImApekPdoKjsNCvgBRCENJIc37XDZ03oY2KrKitg3m7RbwD3d1bzYDUkogBhGHGfg7/p Ebs= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 11 Sep 2023 05:30:19 -0700 IronPort-SDR: My0ZbToCJmvcAZDxNYlrQPbivIoV+mITsZaki3rPiNzcYIRXItz1xdhbguw/nFPdPojhxz7P4f f2TERQyq8ZWoJiALKCy41yjh938GK+0lAb8NfEqPRLMaxKHDiH+6/MQli9Mcogf3sQBTcYmArI cvpjlVzhpokCWcaaK4JmWIfRvkEQiwPc3dvlnzCcunjx+cGIJPWtG0C4mZDqZc2w3WgMTOWNfy KPYI3WwSBFBec0fKiw4kmfrh3EiaC8HpS+wSnjpmXxZfAUOFXpYBaCZD598PkJYgDZjeN62zcY +K0= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 11 Sep 2023 06:23:13 -0700 From: Johannes Thumshirn To: David Sterba Cc: Johannes Thumshirn , linux-btrfs@vger.kernel.org Subject: [PATCH 4/6] btrfs-progs: allow zoned RAID Date: Mon, 11 Sep 2023 06:23:00 -0700 Message-ID: <20230911-raid-stripe-tree-v1-4-c8337f7444b5@wdc.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> References: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> MIME-Version: 1.0 X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694438542; l=7572; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=H022AhqxQLH8oJ/IhL9gK9CUAeyKynw0SLTJOhcuOHQ=; b=Ye/UjLLY5aX1r6TuR1tcvcESvHzT9JgMZvwugj85EGD/X3aXSpKD3RcuNbZ42G6p/Qqe1GerI KLgMRiBO8/KA2ubdAesgYJ4LHXW4mujAcOzW5lfCIpObnHElXTBrMZ2 X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree is used. Signed-off-by: Johannes Thumshirn --- common/fsfeatures.c | 8 ++++++ kernel-shared/ctree.h | 3 +- kernel-shared/zoned.c | 34 ++++++++++++++++++++-- kernel-shared/zoned.h | 4 +-- mkfs/main.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++--- 5 files changed, 118 insertions(+), 10 deletions(-) diff --git a/common/fsfeatures.c b/common/fsfeatures.c index 00658fa5159f..2658f5072af4 100644 --- a/common/fsfeatures.c +++ b/common/fsfeatures.c @@ -189,6 +189,14 @@ static const struct btrfs_feature mkfs_features[] = { VERSION_NULL(safe), VERSION_NULL(default), .desc = "new extent tree format" + } , { + .name = "raid-stripe-tree", + .incompat_flag = BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE, + .sysfs_name = NULL, + VERSION_NULL(compat), + VERSION_NULL(safe), + VERSION_NULL(default), + .desc = "raid stripe tree" }, #endif /* Keep this one last */ diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 26ab0b6aea17..e6322a77d3c8 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -102,7 +102,8 @@ static inline u32 __BTRFS_LEAF_DATA_SIZE(u32 nodesize) BTRFS_FEATURE_INCOMPAT_RAID1C34 | \ BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \ BTRFS_FEATURE_INCOMPAT_ZONED | \ - BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2) + BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 | \ + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE) #else #define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF | \ diff --git a/kernel-shared/zoned.c b/kernel-shared/zoned.c index d187c5763406..d8fad4319e44 100644 --- a/kernel-shared/zoned.c +++ b/kernel-shared/zoned.c @@ -737,7 +737,7 @@ out: return ret; } -bool zoned_profile_supported(u64 map_type) +bool zoned_profile_supported(u64 map_type, bool rst) { bool data = (map_type & BTRFS_BLOCK_GROUP_DATA); u64 flags = (map_type & BTRFS_BLOCK_GROUP_PROFILE_MASK); @@ -746,9 +746,37 @@ bool zoned_profile_supported(u64 map_type) if (flags == 0) return true; - /* We can support DUP on metadata */ +#if EXPERIMENTAL + if (data) { + if ((flags & BTRFS_BLOCK_GROUP_DUP) && rst) + return true; + /* Data RAID1 needs a raid-stripe-tree */ + if ((flags & BTRFS_BLOCK_GROUP_RAID1_MASK) && rst) + return true; + /* Data RAID0 needs a raid-stripe-tree */ + if ((flags & BTRFS_BLOCK_GROUP_RAID0) && rst) + return true; + /* Data RAID10 needs a raid-stripe-tree */ + if ((flags & BTRFS_BLOCK_GROUP_RAID10) && rst) + return true; + } else { + /* We can support DUP on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_DUP) + return true; + /* We can support RAID1 on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_RAID1_MASK) + return true; + /* We can support RAID0 on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_RAID0) + return true; + /* We can support RAID10 on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_RAID10) + return true; + } +#else if (!data && (flags & BTRFS_BLOCK_GROUP_DUP)) return true; +#endif /* All other profiles are not supported yet */ return false; @@ -863,7 +891,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, } } - if (!zoned_profile_supported(map->type)) { + if (!zoned_profile_supported(map->type, !!fs_info->stripe_root)) { error("zoned: profile %s not yet supported", btrfs_group_profile_str(map->type)); ret = -EINVAL; diff --git a/kernel-shared/zoned.h b/kernel-shared/zoned.h index 9e4162cf25c5..6efc60281bc9 100644 --- a/kernel-shared/zoned.h +++ b/kernel-shared/zoned.h @@ -133,7 +133,7 @@ static inline bool btrfs_dev_is_empty_zone(struct btrfs_device *device, u64 pos) return zinfo->zones[zno].cond == BLK_ZONE_COND_EMPTY; } -bool zoned_profile_supported(u64 map_type); +bool zoned_profile_supported(u64 map_type, bool rst); int btrfs_reset_dev_zone(int fd, struct blk_zone *zone); u64 btrfs_find_allocatable_zones(struct btrfs_device *device, u64 hole_start, u64 hole_end, u64 num_bytes); @@ -214,7 +214,7 @@ static inline int btrfs_wipe_temporary_sb(struct btrfs_fs_devices *fs_devices) return 0; } -static inline bool zoned_profile_supported(u64 map_type) +static inline bool zoned_profile_supported(u64 map_type, bool rst) { return false; } diff --git a/mkfs/main.c b/mkfs/main.c index 7acd39ec6531..7d07ba1e7001 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -962,6 +962,38 @@ fail: return ret; } +static int setup_raid_stripe_tree_root(struct btrfs_fs_info *fs_info) +{ + struct btrfs_trans_handle *trans; + struct btrfs_root *stripe_root; + struct btrfs_key key = { + .objectid = BTRFS_RAID_STRIPE_TREE_OBJECTID, + .type = BTRFS_ROOT_ITEM_KEY, + }; + int ret; + + trans = btrfs_start_transaction(fs_info->tree_root, 0); + if (IS_ERR(trans)) + return PTR_ERR(trans); + + stripe_root = btrfs_create_tree(trans, fs_info, &key); + if (IS_ERR(stripe_root)) { + ret = PTR_ERR(stripe_root); + goto abort; + } + fs_info->stripe_root = stripe_root; + add_root_to_dirty_list(stripe_root); + + ret = btrfs_commit_transaction(trans, fs_info->tree_root); + if (ret) + return ret; + + return 0; +abort: + btrfs_abort_transaction(trans, ret); + return ret; +} + /* Thread callback for device preparation */ static void *prepare_one_device(void *ctx) { @@ -1472,10 +1504,39 @@ int BOX_MAIN(mkfs)(int argc, char **argv) if (ret) goto error; - if (opt_zoned && (!zoned_profile_supported(BTRFS_BLOCK_GROUP_METADATA | metadata_profile) || - !zoned_profile_supported(BTRFS_BLOCK_GROUP_DATA | data_profile))) { - error("zoned mode does not yet support RAID/DUP profiles, please specify '-d single -m single' manually"); - goto error; +#if EXPERIMENTAL + if (opt_zoned && device_count) { + switch (data_profile & BTRFS_BLOCK_GROUP_PROFILE_MASK) { + case BTRFS_BLOCK_GROUP_DUP: + case BTRFS_BLOCK_GROUP_RAID1: + case BTRFS_BLOCK_GROUP_RAID1C3: + case BTRFS_BLOCK_GROUP_RAID1C4: + case BTRFS_BLOCK_GROUP_RAID0: + case BTRFS_BLOCK_GROUP_RAID10: + features.incompat_flags |= + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE; + break; + default: + break; + } + } +#endif + + if (opt_zoned) { + u64 metadata = BTRFS_BLOCK_GROUP_METADATA | metadata_profile; + u64 data = BTRFS_BLOCK_GROUP_DATA | data_profile; +#if EXPERIMENTAL + bool rst = features.incompat_flags & + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE; +#else + bool rst = false; +#endif + + if (!zoned_profile_supported(metadata, rst) || + !zoned_profile_supported(data, rst)) { + error("zoned mode does not yet support RAID/DUP profiles, please specify '-d single -m single' manually"); + goto error; + } } t_prepare = calloc(device_count, sizeof(*t_prepare)); @@ -1585,6 +1646,15 @@ int BOX_MAIN(mkfs)(int argc, char **argv) goto error; } + if (features.incompat_flags & BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE) { + ret = setup_raid_stripe_tree_root(fs_info); + if (ret < 0) { + error("failed to initialize raid-stripe-tree: %d (%m)", + ret); + goto out; + } + } + trans = btrfs_start_transaction(root, 1); if (IS_ERR(trans)) { errno = -PTR_ERR(trans); @@ -1750,6 +1820,7 @@ raid_groups: goto out; } } + if (bconf.verbose) { char features_buf[BTRFS_FEATURE_STRING_BUF_SIZE]; From patchwork Mon Sep 11 13:23:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13380230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8C7BCA0EC7 for ; Mon, 11 Sep 2023 22:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355911AbjIKWCX (ORCPT ); Mon, 11 Sep 2023 18:02:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237944AbjIKNXU (ORCPT ); Mon, 11 Sep 2023 09:23:20 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E4F3193 for ; Mon, 11 Sep 2023 06:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694438596; x=1725974596; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iAd9M2zZ4NMSY/nI1P/jXW61ZlJl9tSSeDR7DnVrWhc=; b=Mqr1P/v0c/mMad3dZqqK49AWyG2LSeumqgyFs/FKFpTlPIeOGn3UwTPN xlGQdxKlklX75ADC3IFHEOMvaUltZzRONuUFN4prNOHV8y+SxiSPBFc8O 5xV+lED6Q1ELkSDtyigcTcqQr7RlZ18jcLZ6vFmpkYsiTjDBsE7peQfLd Mq3SvzVPivOrSdeUYG3tZm16bM7r/Bp/NIvklbyf53Qr1y9aLC1tKsxLL vlU5Hj0J0U/QYN/4p3boCmWzcd/d5HLdl9Ma/9pupsw97Cz5fMV9xsOT2 vwv7CLF0iFfYkVc7KcnwXv+Kjw0JvljH3kz6tuqkmE7C5Necm3IJdfa1J g==; X-CSE-ConnectionGUID: GtBT3eW2TM+pQtpuZk0hcQ== X-CSE-MsgGUID: b0P7lbMWQ4aoyj8Y1Y/W/g== X-IronPort-AV: E=Sophos;i="6.02,244,1688400000"; d="scan'208";a="248143284" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Sep 2023 21:23:15 +0800 IronPort-SDR: Jwn8A35XwoeWg6g3+fn5mBnV86qNErP+pL4Ps+sNMdHCLfNLiSQbPDrrAoiNXIi5A91vpqgZTQ +VU7vXJj7WYJB6Kb6T+Zp9GV7SIVYg+prQF1LWoyHEpOVqoivgPfww5ySeIgYVFC0HUpXBTw70 RS2/iUg5I/e6cYpTENF9AyD6Jcmb4vs7sbRcVkS5CqcwEwLcBwkpYnBAybcFp1yoF1vlTHlcdE 68QjaeAPN1DF/rV/HiL1EiBxjShtJ0lho8lXHFUVN3dEhfLHrvhE7SnhTfYdkeBdJOwA/QNwxH jWk= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 11 Sep 2023 05:30:20 -0700 IronPort-SDR: NmnmhoSykl/C2vcMNk5MLfrtysK3dDrZ3xNhqsMS/npCnjKRRsuONg2A+Xh5J3mciCjNiw3U7x 9FmQoFJ2Qwylf8NjDI57Qh8pHeKCJS7WYOde+w5bnKLhddW1XKsPB0Y157n3oIHqIpdiibcOLq 9NEZljO+RYS9kum6qkcX3RYkh9ZqTIsjgVt5CXyUY1bQmHdBDNS+ErImu4mHDQjt8pmpo0Mwaw WJSq3JWIFO7/lN1epJv7GgJnQNl0R55NZr1GfERt4uxLw3IjNg5OCSvEcysNeYcY8s83LP8iTv h7o= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 11 Sep 2023 06:23:14 -0700 From: Johannes Thumshirn To: David Sterba Cc: Johannes Thumshirn , linux-btrfs@vger.kernel.org Subject: [PATCH 5/6] btrfs-progs: load zone info for all zoned devices Date: Mon, 11 Sep 2023 06:23:01 -0700 Message-ID: <20230911-raid-stripe-tree-v1-5-c8337f7444b5@wdc.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> References: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> MIME-Version: 1.0 X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694438542; l=531; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=iAd9M2zZ4NMSY/nI1P/jXW61ZlJl9tSSeDR7DnVrWhc=; b=04y44cpQK3oYE3EFlzzlwqz36ZP8wHKt14qkihxMtPhI0ruuroL5vvDLsRXl+arHhsm/DLdbE TSCHS4P0WxKC99S1dLdgD1uA0EmO3KV6FQOTSWgy3IA3IHFcy6v9Hbm X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Signed-off-by: Johannes Thumshirn --- mkfs/main.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mkfs/main.c b/mkfs/main.c index 7d07ba1e7001..8a27704ac29e 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -1732,6 +1732,10 @@ int BOX_MAIN(mkfs)(int argc, char **argv) } } + + if (opt_zoned) + btrfs_get_dev_zone_info_all_devices(fs_info); + raid_groups: ret = create_raid_groups(trans, root, data_profile, metadata_profile, mixed, &allocation); From patchwork Mon Sep 11 13:23:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13380231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94048CA0ECB for ; Mon, 11 Sep 2023 22:07:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356044AbjIKWCr (ORCPT ); Mon, 11 Sep 2023 18:02:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237942AbjIKNXU (ORCPT ); Mon, 11 Sep 2023 09:23:20 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 444E2CD7 for ; Mon, 11 Sep 2023 06:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694438596; x=1725974596; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zApmCeqyy3SfHYiANMzUPLxS2p9rjxha3fR6HA0U5Bc=; b=AhOxesRnj9bhDL3oZgzIc3xgiADKfQTJ+4Asg6+8MNmOJmgKN9R7UEpQ kek7uUeSrXiwwWaLXVoJMQLmg+7WIbaygaBrDHh/R2zlEDumGCqgXsvBI IMyNW1iBY3kECb+qsHRRU7y3Lf3LJPlZcrblSxvBQy6TWL+pCDrQJf1nW LtINccU3igbV2uOb4QY5kNxt5AR0mxAnZ9X2+FJ2LjZezSGmfKgvoccGO Mp4+kaXPgylbL1zJ7FDaCGpBseUOpoh2OMS/yGshBQGmcPz2LRAOb18Z+ /tCmIm5y8i5VC3ALgUL+uEL8YtFB6TvFRKPxAp6Qs2dc1ITrsLT+QDVIB Q==; X-CSE-ConnectionGUID: YsJv9saxQCirmCnV7m52UQ== X-CSE-MsgGUID: nJtvIU8oSPOvDWqPP/tpdQ== X-IronPort-AV: E=Sophos;i="6.02,244,1688400000"; d="scan'208";a="248143285" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Sep 2023 21:23:15 +0800 IronPort-SDR: BPJc67i2ybptmxj5GfA2lAmeT5NRz946asZNynxVV+NROdLYg2Ig6cRKRWFWU7muaegu9haDRU mJfR7IFfGGuiCCpK53Z0bMTLKB7bGHVzYIX755FSETTCoD8zqsQGM3xiv6JTIPjKmQNmZbSNIN TB9fY4eVqQjY1ZjNdCA3kQyVIylomgq+iRprTGfYgoe4w0iz+4JmqzUt1itNwnO1A5Pp+ANkv5 rLWIjcTCNhOBdGVruQ+eqThtLwyeS9L2DvUsrePnv/f/xTBYUO4zDvyzw62CP3qoB6BA/Nh90w po4= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 11 Sep 2023 05:30:21 -0700 IronPort-SDR: y/d6LHwrN/xp1wcd8LFYJT39BeEAI4SItIc7uDRL7BtCzEjRKDZk8qPeMgucbFJGR+Ik8JUgWI MRDtiP6Patob290pElqm2UUTSAz+bAL9iWuX0QPAQC0SXPnzKK84dSW6UVK3Q/sTNIOr8NRH6+ DYyiwLp5WxieiIEz6rFIbvSUoclw8/Eus74VEBOizPpr6FPwJQOmhBtVXc7dPpegVhRk2l3ahX 2ITRsLq/5biyyjzezEFrTc+cQzDCHvM96TzEPlzVuIWDRpa5jNsR527i7Qu8FFqqvxYHIrAkVY /Gg= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 11 Sep 2023 06:23:15 -0700 From: Johannes Thumshirn To: David Sterba Cc: Johannes Thumshirn , linux-btrfs@vger.kernel.org Subject: [PATCH 6/6] btrfs-progs: read stripe tree when mapping blocks Date: Mon, 11 Sep 2023 06:23:02 -0700 Message-ID: <20230911-raid-stripe-tree-v1-6-c8337f7444b5@wdc.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> References: <20230911-raid-stripe-tree-v1-0-c8337f7444b5@wdc.com> MIME-Version: 1.0 X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694438542; l=3680; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=zApmCeqyy3SfHYiANMzUPLxS2p9rjxha3fR6HA0U5Bc=; b=GulItj5X+0XtrBgZehtIsXYfOpPTJHya2oSII2x5INWmrD59gDpDqwA4ShbitLPVWGJiPdwmm cBU96kUrpgxAoXjG+c1ZRXeiysG2fxEmoFw0vJsLMw3O68/4I5pVobD X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Signed-off-by: Johannes Thumshirn --- kernel-shared/volumes.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 113 insertions(+), 3 deletions(-) diff --git a/kernel-shared/volumes.c b/kernel-shared/volumes.c index 95d5930b95d8..2081f7db088f 100644 --- a/kernel-shared/volumes.c +++ b/kernel-shared/volumes.c @@ -1796,6 +1796,105 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, multi_ret, mirror_num, raid_map_ret); } +static bool btrfs_need_stripe_tree_update(struct btrfs_fs_info *fs_info, + u64 map_type) +{ +#if EXPERIMENTAL + const bool is_data = (map_type & BTRFS_BLOCK_GROUP_DATA); + + if (!btrfs_fs_incompat(fs_info, RAID_STRIPE_TREE)) + return false; + + if (!fs_info->stripe_root) + return false; + + if (!is_data) + return false; + + if (map_type & BTRFS_BLOCK_GROUP_DUP) + return true; + + if (map_type & BTRFS_BLOCK_GROUP_RAID1_MASK) + return true; + + if (map_type & BTRFS_BLOCK_GROUP_RAID0) + return true; + + if (map_type & BTRFS_BLOCK_GROUP_RAID10) + return true; + +#endif + return false; +} + +static int btrfs_stripe_tree_logical_to_physical(struct btrfs_fs_info *fs_info, + u64 logical, + struct btrfs_bio_stripe *stripe) +{ + struct btrfs_root *root = fs_info->stripe_root; + struct btrfs_path path = { 0 }; + struct btrfs_key key; + struct extent_buffer *leaf; + int slot; + int ret; + + key.objectid = logical; + key.type = BTRFS_RAID_STRIPE_KEY; + key.offset = 0; + + ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0); + if (ret < 0) + return ret; + + while (1) { + struct btrfs_key found_key; + struct btrfs_stripe_extent *extent; + int num_stripes; + u32 item_size; + int i; + + leaf = path.nodes[0]; + slot = path.slots[0]; + + if (slot >= btrfs_header_nritems(leaf)) { + ret = btrfs_next_leaf(root, &path); + if (ret == 0) + continue; + if (ret < 0) + goto error; + break; + } + + btrfs_item_key_to_cpu(leaf, &found_key, slot); + + if (found_key.type != BTRFS_RAID_STRIPE_KEY) + goto next; + + extent = btrfs_item_ptr(leaf, slot, + struct btrfs_stripe_extent); + item_size = btrfs_item_size(leaf, slot); + num_stripes = (item_size - + offsetof(struct btrfs_stripe_extent, strides)) / + sizeof(struct btrfs_raid_stride); + + for (i = 0; i < num_stripes; i++) { + if (stripe->dev->devid != + btrfs_raid_stride_devid_nr(leaf, extent, i)) + continue; + stripe->physical = btrfs_raid_stride_offset_nr(leaf, extent, i); + btrfs_release_path(&path); + return 0; + } + +next: + path.slots[0]++; + } + + btrfs_release_path(&path); +error: + return ret; +} + int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, u64 logical, u64 *length, u64 *type, struct btrfs_multi_bio **multi_ret, int mirror_num, @@ -1988,10 +2087,21 @@ again: BUG_ON(stripe_index >= map->num_stripes); for (i = 0; i < multi->num_stripes; i++) { - multi->stripes[i].physical = - map->stripes[stripe_index].physical + stripe_offset + - stripe_nr * map->stripe_len; multi->stripes[i].dev = map->stripes[stripe_index].dev; + + if (stripes_allocated && + btrfs_need_stripe_tree_update(fs_info, map->type)) { + int ret; + + ret = btrfs_stripe_tree_logical_to_physical(fs_info, logical, + &multi->stripes[i]); + if (ret) + return ret; + } else { + multi->stripes[i].physical = + map->stripes[stripe_index].physical + + stripe_offset + stripe_nr * map->stripe_len; + } stripe_index++; } *multi_ret = multi;