From patchwork Thu Sep 14 16:05:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5612BEEAA54 for ; Thu, 14 Sep 2023 16:05:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241457AbjINQFt (ORCPT ); Thu, 14 Sep 2023 12:05:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241312AbjINQFr (ORCPT ); Thu, 14 Sep 2023 12:05:47 -0400 Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CA8E1BF8 for ; Thu, 14 Sep 2023 09:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707543; x=1726243543; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Rt+TRcnIncRdW3sqGWFr+5uNWeiiI61Ww4Ly4jFNlIc=; b=MdMJdP2T01UJmp7ZOMt7gTDdEJH6RHn5ZgeDBjXpXH9lUoDyRLOy7W/h eFmJCtiNEjqrz2WlM/vppZCiqxTIQgawMWcZT7tcrUao69eVLy4S4+qsy Exmc86ReTLzaqhzCsjl+VaQLW/zmEof2uTHeYsdx3sHbU2t0G5bRbSfxA XCxJhOCFEp6M8xK4Ja+mquZtt+WsrsTdpUMU78x1jHT6hOblgjaw8aYSc VI6bc9XfGc6L4zfn0tj7HOKKZ7Wmey7rl17efNHaf7s+haWyOvRmH8Skh f0mYqKHvkJoLrhn4XTqOXL9r4YMYpr+v3bur8CyRxV1BLk9yKfldf3077 w==; X-CSE-ConnectionGUID: ncS1mGDbS+G+qGXj4nCitA== X-CSE-MsgGUID: qSAXSWVrTbqommcTb6umUA== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="242196079" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:05:42 +0800 IronPort-SDR: h5Kpt7tbHtx4vradJ6kBSCzmGAkstr83vb39cAhaDClooWDVrinYXL/QNnbbU+KvKSGHAeOjpT zCqAgtExpYbBB9347oq8dvl6xevPEwbx39CkivQAponDSGiZ7s2mDkmltRwwvOlP1SFKa/deHR bRxVimb1ozhz0l9ooGuGTAUIqSf7KlnnWrjAGCYEoFN7wESHXWYvw5G1/ltv2jEQ4sEQJ1VnY/ AIm2AJkqMDnR3cbn3j5DJxZ3bhkhOF+j4zRhF2dWoBJj3tVdqm1DTYWKiKqLU9KeIOQNXMCUCB Vug= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:18:24 -0700 IronPort-SDR: HkL0aInPNONvwHM3lPMNZXk1LBCIIzgcyuXQVL2bF47lRBa3nQUPSN2/Cs8nmg65US5ukEIWVD tfKZjTXTTjegouxpdT2+EWQ73UvHR/mxnG+FsujXs48WpV87RBbxN3o/91IuilYbzadFoPX+cS hZH1y67OhpgAvk2mmRVf3F1NDbMUkKattHByA2SU3sTwQtAOFCHOMJP8hIdCBT9/WUHkEdgq9M kE6q3blVe+hvGJu9EGOU8tehdgByxNSXUltVV6MflTULdFbu8VripkYwLFK8velKSFPKqPs5iv LYg= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:05:42 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:05:32 -0700 Subject: [PATCH v4 1/6] btrfs-progs: add raid-stripe-tree definitions MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v4-1-c921c15ec052@wdc.com> References: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> To: David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707540; l=3412; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=Rt+TRcnIncRdW3sqGWFr+5uNWeiiI61Ww4Ly4jFNlIc=; b=FGcSy8fZTEU3nXbr4POiJXmtcRARfydGYJXRUuhvU/Mq7rd8WosqVB7ygADhAmsq6WBYpSGOx MyQWyyo3FwMC9GsjhwHdKDa1RgyM5Mqpei7b7VBBQ786l4dUlpWR5iS X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add the definitions for the on-disk format of the RAID stripe tree. Signed-off-by: Johannes Thumshirn --- kernel-shared/accessors.h | 37 +++++++++++++++++++++++++++++++++++++ kernel-shared/uapi/btrfs_tree.h | 26 ++++++++++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/kernel-shared/accessors.h b/kernel-shared/accessors.h index 625acfbe8ca7..80635f31c90b 100644 --- a/kernel-shared/accessors.h +++ b/kernel-shared/accessors.h @@ -270,6 +270,43 @@ BTRFS_SETGET_FUNCS(free_space_extent_count, struct btrfs_free_space_info, extent_count, 32); BTRFS_SETGET_FUNCS(free_space_flags, struct btrfs_free_space_info, flags, 32); +/* struct btrfs_stripe_extent */ +BTRFS_SETGET_FUNCS(stripe_extent_encoding, struct btrfs_stripe_extent, encoding, 8); +BTRFS_SETGET_FUNCS(raid_stride_devid, struct btrfs_raid_stride, devid, 64); +BTRFS_SETGET_FUNCS(raid_stride_offset, struct btrfs_raid_stride, offset, 64); +BTRFS_SETGET_FUNCS(raid_stride_length, struct btrfs_raid_stride, length, 64); + +static inline struct btrfs_raid_stride *btrfs_raid_stride_nr( + struct btrfs_stripe_extent *dps, + int nr) +{ + unsigned long offset = (unsigned long)dps; + offset += offsetof(struct btrfs_stripe_extent, strides); + offset += nr * sizeof(struct btrfs_raid_stride); + return (struct btrfs_raid_stride *)offset; +} + +static inline u64 btrfs_raid_stride_devid_nr(struct extent_buffer *eb, + struct btrfs_stripe_extent *dps, + int nr) +{ + return btrfs_raid_stride_devid(eb, btrfs_raid_stride_nr(dps, nr)); +} + +static inline u64 btrfs_raid_stride_offset_nr(struct extent_buffer *eb, + struct btrfs_stripe_extent *dps, + int nr) +{ + return btrfs_raid_stride_offset(eb, btrfs_raid_stride_nr(dps, nr)); +} + +static inline u64 btrfs_raid_stride_length_nr(struct extent_buffer *eb, + struct btrfs_stripe_extent *dps, + int nr) +{ + return btrfs_raid_stride_length(eb, btrfs_raid_stride_nr(dps, nr)); +} + /* struct btrfs_inode_ref */ BTRFS_SETGET_FUNCS(inode_ref_name_len, struct btrfs_inode_ref, name_len, 16); BTRFS_SETGET_FUNCS(inode_ref_index, struct btrfs_inode_ref, index, 64); diff --git a/kernel-shared/uapi/btrfs_tree.h b/kernel-shared/uapi/btrfs_tree.h index ad555e7055ab..73411a8697ce 100644 --- a/kernel-shared/uapi/btrfs_tree.h +++ b/kernel-shared/uapi/btrfs_tree.h @@ -699,6 +699,32 @@ struct btrfs_super_block { __u8 padding[565]; } __attribute__ ((__packed__)); +struct btrfs_raid_stride { + /* btrfs device-id this raid extent lives on */ + __le64 devid; + /* offset from the devextent start */ + __le64 offset; + /* length of the stride on disk */ + __le64 length; +} __attribute__ ((__packed__)); + +/* The stripe_extent::encoding, 1:1 mapping of enum btrfs_raid_types */ +#define BTRFS_STRIPE_RAID0 1 +#define BTRFS_STRIPE_RAID1 2 +#define BTRFS_STRIPE_DUP 3 +#define BTRFS_STRIPE_RAID10 4 +#define BTRFS_STRIPE_RAID5 5 +#define BTRFS_STRIPE_RAID6 6 +#define BTRFS_STRIPE_RAID1C3 7 +#define BTRFS_STRIPE_RAID1C4 8 + +struct btrfs_stripe_extent { + u8 encoding; + u8 reserved[7]; + /* array of raid strides this stripe is comprised of */ + struct btrfs_raid_stride strides; +} __attribute__ ((__packed__)); + #define BTRFS_FREE_SPACE_EXTENT 1 #define BTRFS_FREE_SPACE_BITMAP 2 From patchwork Thu Sep 14 16:05:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD26AEEAA53 for ; Thu, 14 Sep 2023 16:05:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241456AbjINQFt (ORCPT ); Thu, 14 Sep 2023 12:05:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240667AbjINQFs (ORCPT ); Thu, 14 Sep 2023 12:05:48 -0400 Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 042361BDD for ; Thu, 14 Sep 2023 09:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707544; x=1726243544; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=8l6KkYM203DFih2M1SwkGYZVANo7hCju5ir8jwjt/Io=; b=Ss5ff0DdAbpb22WOKTpw5GbeHZ3R8oWnT1ANgly60aKOS+nFYMZfDSCu UTyWXTnLgNqeN1QwbQaH7UMlaW4Ww4E4Z0LFo4log72Hm0r7y2oi5QFm8 IivSbt15uf0JdstIrZ6kifdIv0SyBzzsF4xZyM5lIeHG3cC1vJaHgj7rx r3s322hV6WYBA/vuqLp2Am/gx+qJ7vkg7gM/3S2g1fIlBvLB44q/Av4J8 NMLHoT5FH2SUJSTcEOsLkVAraZygtLujh6l+E5Gw3WL1/HctBGAeF6tFC bljLxDtybqrj0u52JOhelPgjhJ+7k+fHYrdxlW+jkWrw589sa1055lWl+ g==; X-CSE-ConnectionGUID: QkHT+gIIQX2eR96KblqWCA== X-CSE-MsgGUID: 9e8ye/yISo28qw/xlgFEcA== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="242196082" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:05:42 +0800 IronPort-SDR: f5xeKcma3OJ95KXDukHkIjfmkDN/lJ/YzHbym1XIqDKPLYZ3f7SXEQhZthRr/9AvIt3pj51G3e WqKuDxg49F9lC23NXaN585c1g14cmQFCo9Z8BuYiBut13pmc/v2M3d0yDGBxtzWYf2EOGUybhf sFVZnIO9truSW7z5u2JLQwx0DfbkjaPVY+Me+8+P5siCjAJqYcSSJk6jPNWiCR3SvxR9N5wfJn +7TUEJFgIpIA+49i+gCt3r9bBY6z8trHN/GyMI+mK3YXpQuUQvZc/+7LDodZNhsVTyWg50szXV d2g= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:18:25 -0700 IronPort-SDR: +uHHR4y7Cae5QU772a4/1WH2PvLsKr31XBa3K+ANIyorKuAUQw8CsiUlDmcMS+o2nMAnjRd+mI 5ABNvJK3EUxdS5ntD1WwzZQm6kSFsCZoLx2GtoURHA0jrv7oKGaJ8F6BDTCFH4YB4tBQuWWg9g ZASK9I8MYIxb7cjmQlictH3sm/3Lc05+YUiztCfRY29qcwhMndnGslkHdJQVNp5APkDvcT40Tp QLg/gDFN9M3PrwN2hyjAhYYHRW/LqiJT7h3x9uYh0Hwtd258vrPfj9QoGxagb/60GUXDyOGyM9 hJ4= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:05:43 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:05:33 -0700 Subject: [PATCH v4 2/6] btrfs-progs: read fs with stripe tree from disk MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v4-2-c921c15ec052@wdc.com> References: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> To: David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707540; l=5277; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=8l6KkYM203DFih2M1SwkGYZVANo7hCju5ir8jwjt/Io=; b=Q3WwvKrYEBGpTH2x5RK20S6jEZe0Uox0ogXmYYeRw8nRk3MGB8IM4YIgjS0zSBMNMO5tndItI XeiqD3YeV2dDenuaM6VJehZuetX7s4TYsSQNNBquL5l+fRy6XYkOHo3 X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When encountering a filesystem formatted with the raid stripe tree feature, read it from disk. Also add the incompat declaration to the tree printer. Signed-off-by: Johannes Thumshirn --- kernel-shared/ctree.h | 1 + kernel-shared/disk-io.c | 28 +++++++++++++++++++++++++++- kernel-shared/print-tree.c | 1 + kernel-shared/uapi/btrfs.h | 1 + kernel-shared/uapi/btrfs_tree.h | 3 +++ 5 files changed, 33 insertions(+), 1 deletion(-) diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 5d3392ae82a6..035358436d8f 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -298,6 +298,7 @@ struct btrfs_fs_info { struct btrfs_root *quota_root; struct btrfs_root *uuid_root; struct btrfs_root *block_group_root; + struct btrfs_root *stripe_root; struct rb_root global_roots_tree; struct rb_root fs_root_tree; diff --git a/kernel-shared/disk-io.c b/kernel-shared/disk-io.c index 442d3af8bc01..3ab32f7ad910 100644 --- a/kernel-shared/disk-io.c +++ b/kernel-shared/disk-io.c @@ -790,6 +790,10 @@ struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info, return fs_info->block_group_root ? fs_info->block_group_root : ERR_PTR(-ENOENT); + if (location->objectid == BTRFS_RAID_STRIPE_TREE_OBJECTID) + return fs_info->stripe_root ? fs_info->stripe_root : + ERR_PTR(-ENOENT); + BUG_ON(location->objectid == BTRFS_TREE_RELOC_OBJECTID); node = rb_search(&fs_info->fs_root_tree, (void *)&objectid, @@ -822,6 +826,9 @@ void btrfs_free_fs_info(struct btrfs_fs_info *fs_info) if (fs_info->quota_root) free(fs_info->quota_root); + if (fs_info->stripe_root) + free(fs_info->stripe_root); + free_global_roots_tree(&fs_info->global_roots_tree); free(fs_info->tree_root); free(fs_info->chunk_root); @@ -846,12 +853,14 @@ struct btrfs_fs_info *btrfs_new_fs_info(int writable, u64 sb_bytenr) fs_info->dev_root = calloc(1, sizeof(struct btrfs_root)); fs_info->quota_root = calloc(1, sizeof(struct btrfs_root)); fs_info->uuid_root = calloc(1, sizeof(struct btrfs_root)); + fs_info->stripe_root = calloc(1, sizeof(struct btrfs_root)); fs_info->block_group_root = calloc(1, sizeof(struct btrfs_root)); fs_info->super_copy = calloc(1, BTRFS_SUPER_INFO_SIZE); if (!fs_info->tree_root || !fs_info->chunk_root || !fs_info->dev_root || !fs_info->quota_root || !fs_info->uuid_root || - !fs_info->block_group_root || !fs_info->super_copy) + !fs_info->block_group_root || !fs_info->super_copy || + !fs_info->stripe_root) goto free_all; extent_buffer_init_cache(fs_info); @@ -1260,6 +1269,21 @@ int btrfs_setup_all_roots(struct btrfs_fs_info *fs_info, u64 root_tree_bytenr, return -EIO; } +#if EXPERIMENTAL + if (btrfs_fs_incompat(fs_info, RAID_STRIPE_TREE)) { + ret = btrfs_find_and_setup_root(root, fs_info, + BTRFS_RAID_STRIPE_TREE_OBJECTID, + fs_info->stripe_root); + if (ret) { + free(fs_info->stripe_root); + fs_info->stripe_root = NULL; + } else { + set_bit(BTRFS_ROOT_TRACK_DIRTY, + &fs_info->stripe_root->state); + } + } +#endif + if (maybe_load_block_groups(fs_info, flags)) { ret = btrfs_read_block_groups(fs_info); /* @@ -1317,6 +1341,8 @@ void btrfs_release_all_roots(struct btrfs_fs_info *fs_info) free_extent_buffer(fs_info->chunk_root->node); if (fs_info->uuid_root) free_extent_buffer(fs_info->uuid_root->node); + if (fs_info->stripe_root) + free_extent_buffer(fs_info->stripe_root->node); } static void free_map_lookup(struct cache_extent *ce) diff --git a/kernel-shared/print-tree.c b/kernel-shared/print-tree.c index 0f7f7b72f96a..3eff82b364ef 100644 --- a/kernel-shared/print-tree.c +++ b/kernel-shared/print-tree.c @@ -1708,6 +1708,7 @@ static struct readable_flag_entry incompat_flags_array[] = { DEF_INCOMPAT_FLAG_ENTRY(RAID1C34), DEF_INCOMPAT_FLAG_ENTRY(ZONED), DEF_INCOMPAT_FLAG_ENTRY(EXTENT_TREE_V2), + DEF_INCOMPAT_FLAG_ENTRY(RAID_STRIPE_TREE), }; static const int incompat_flags_num = sizeof(incompat_flags_array) / sizeof(struct readable_flag_entry); diff --git a/kernel-shared/uapi/btrfs.h b/kernel-shared/uapi/btrfs.h index 85b04f89a2a9..c750b7aa921f 100644 --- a/kernel-shared/uapi/btrfs.h +++ b/kernel-shared/uapi/btrfs.h @@ -356,6 +356,7 @@ _static_assert(sizeof(struct btrfs_ioctl_fs_info_args) == 1024); #define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11) #define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12) #define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) +#define BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE (1ULL << 14) struct btrfs_ioctl_feature_flags { __u64 compat_flags; diff --git a/kernel-shared/uapi/btrfs_tree.h b/kernel-shared/uapi/btrfs_tree.h index 73411a8697ce..b76b1660f38b 100644 --- a/kernel-shared/uapi/btrfs_tree.h +++ b/kernel-shared/uapi/btrfs_tree.h @@ -73,6 +73,9 @@ /* Holds the block group items for extent tree v2. */ #define BTRFS_BLOCK_GROUP_TREE_OBJECTID 11ULL +/* holds raid stripe entries */ +#define BTRFS_RAID_STRIPE_TREE_OBJECTID 12ULL + /* device stats in the device tree */ #define BTRFS_DEV_STATS_OBJECTID 0ULL From patchwork Thu Sep 14 16:05:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 942B0EEAA52 for ; Thu, 14 Sep 2023 16:05:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241479AbjINQFu (ORCPT ); Thu, 14 Sep 2023 12:05:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241405AbjINQFs (ORCPT ); Thu, 14 Sep 2023 12:05:48 -0400 Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19D701BE1 for ; Thu, 14 Sep 2023 09:05:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707544; x=1726243544; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=nf0OYXIzMCrF6lIqT8+3WhVCgtORwFCZMCaG8O8IGqE=; b=oQu/zxfmXja8CLcbc1Xf/8C9itgl+E+tb+7XqrUNgzVAJh4tGY2P/jSR f8A8U1NnVQLynhs4qiTRVvj8wSwaplFKA2qMTnIlE4cnk1U/SGVkovcWi VlSELdaDFPM1jAGpcyIxNEVIdG66QWnn1mlth3nizwGaYiqrrJ03Qz5wi BOwiQQ66+UgokazpIeUrY9Zvhtxe97gS/GVeRFdlKOcOqfUm90a8DArzK sPY78ZoczpN/dJUXpoLj+wEcu41eXCZluXm65RIA0sS+/iKzSmsD9vjiu 9E7PThv5DEKwMe4vRJw84sWGPfYbxGfx3CeJbN1jRcLo6f2q+GurqsI4W A==; X-CSE-ConnectionGUID: K8nHfR9EQRebxGexSKxs1g== X-CSE-MsgGUID: v9AYeDDqQ9qet5Bpn0nDRA== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="242196086" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:05:43 +0800 IronPort-SDR: 6A5rVGBtheKBrFbZYLQHl2I9ooaQDDn0DcV9bGMNveGAcBxIm1+cs9ZzwaciLvX87G8CHHf+fC MZh3BS0ZMiYNTH5oDs+anpAY6o1gj3MjDtI6oXx3tAgGaGeX+Hbu88Jm4dvZjO8FChl98QRLaF tR6gL+t6W83aOlaRmv6/PyBReWL98Dn4vNbFjcQ015lv49iQftDloc6HkEiweZjCBEjes9OlXF /4O14x8xRLgFVsZ9xdty+sdA4TaYVktLoaRzhOwFhs8EJpdo7xDQuyUyjOREtE6jJeerPHZc8Z kDI= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:18:26 -0700 IronPort-SDR: 7tPxsckFkHk5YlwHSBTkOCvO3MGZaUNvXmeZA8zIvcAkdqCyOrJrwnMMabYsZ4bL3suK//vJ4y pxCGal9BshFIkQ6dXTnODn0pY98AwFeG1dTS1tbDSA8YjJObhX+hJjb44C/JFCXFPwksM0cd1f dDHuNVFDEcAZ+buLLVurqPUeirDPrVF7e8vFh8r1Ckiq9AuwviJUL37eG2zFSyH1PofwgmUKCi Agbe8x//5R4VyzlsmPF9KobSl7xtS+Ma25pM4psl9FtjCzhZ25eIDWV3Wn6ujAnYsMEbeer4cu dQY= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:05:44 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:05:34 -0700 Subject: [PATCH v4 3/6] btrfs-progs: add dump tree support for the raid stripe tree MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v4-3-c921c15ec052@wdc.com> References: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> To: David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707540; l=4898; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=nf0OYXIzMCrF6lIqT8+3WhVCgtORwFCZMCaG8O8IGqE=; b=iSPpIkFpk4UBoQtvAtLaOjQZ5m0pWKZ395psn2llgKotTlpxso5j1oNJDCx10qFAzHB6tdK/i c8O1M6Fmjt1BChLCNhSsPeCjOJjrXC186Cr3w2FD00vqI++IEGkAykU X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add support for the RAID stripe tree to btrfs inspect-internal dump-tree. Signed-off-by: Johannes Thumshirn --- cmds/inspect-dump-tree.c | 5 +++++ kernel-shared/ctree.h | 5 +++++ kernel-shared/print-tree.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+) diff --git a/cmds/inspect-dump-tree.c b/cmds/inspect-dump-tree.c index bfc0fff148dd..328ecd76c8a0 100644 --- a/cmds/inspect-dump-tree.c +++ b/cmds/inspect-dump-tree.c @@ -170,6 +170,7 @@ static u64 treeid_from_string(const char *str, const char **end) { "TREE_RELOC", BTRFS_TREE_RELOC_OBJECTID }, { "DATA_RELOC", BTRFS_DATA_RELOC_TREE_OBJECTID }, { "BLOCK_GROUP_TREE", BTRFS_BLOCK_GROUP_TREE_OBJECTID }, + { "RAID_STRIPE", BTRFS_RAID_STRIPE_TREE_OBJECTID }, }; if (strncasecmp("BTRFS_", str, strlen("BTRFS_")) == 0) @@ -729,6 +730,10 @@ again: if (!skip) pr_verbose(LOG_DEFAULT, "block group"); break; + case BTRFS_RAID_STRIPE_TREE_OBJECTID: + if (!skip) + printf("raid stripe"); + break; default: if (!skip) { pr_verbose(LOG_DEFAULT, "file"); diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 035358436d8f..de09c15ca0eb 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -556,6 +556,7 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info) */ #define BTRFS_EXTENT_DATA_KEY 108 + /* * csum items have the checksums for data in the extents */ @@ -640,6 +641,8 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info) #define BTRFS_DEV_ITEM_KEY 216 #define BTRFS_CHUNK_ITEM_KEY 228 +#define BTRFS_RAID_STRIPE_KEY 230 + #define BTRFS_BALANCE_ITEM_KEY 248 /* @@ -650,6 +653,8 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info) #define BTRFS_QGROUP_LIMIT_KEY 244 #define BTRFS_QGROUP_RELATION_KEY 246 + + /* * Obsolete name, see BTRFS_TEMPORARY_ITEM_KEY. */ diff --git a/kernel-shared/print-tree.c b/kernel-shared/print-tree.c index 3eff82b364ef..d719ebaebb40 100644 --- a/kernel-shared/print-tree.c +++ b/kernel-shared/print-tree.c @@ -637,6 +637,51 @@ static void print_free_space_header(struct extent_buffer *leaf, int slot) (unsigned long long)btrfs_free_space_bitmaps(leaf, header)); } +struct raid_encoding_map { + u8 encoding; + char name[16]; +}; + +static const struct raid_encoding_map raid_map[] = { + { BTRFS_STRIPE_DUP, "DUP" }, + { BTRFS_STRIPE_RAID0, "RAID0" }, + { BTRFS_STRIPE_RAID1, "RAID1" }, + { BTRFS_STRIPE_RAID1C3, "RAID1C3" }, + { BTRFS_STRIPE_RAID1C4, "RAID1C4" }, + { BTRFS_STRIPE_RAID5, "RAID5" }, + { BTRFS_STRIPE_RAID6, "RAID6" }, + { BTRFS_STRIPE_RAID10, "RAID10" } +}; + +static const char *stripe_encoding_name(u8 encoding) +{ + for (int i = 0; i < ARRAY_SIZE(raid_map); i++) { + if (raid_map[i].encoding == encoding) + return raid_map[i].name; + } + + return "UNKNOWN"; +} + +static void print_raid_stripe_key(struct extent_buffer *eb, + u32 item_size, + struct btrfs_stripe_extent *stripe) +{ + int num_stripes; + u8 encoding = btrfs_stripe_extent_encoding(eb, stripe); + int i; + + num_stripes = (item_size - offsetof(struct btrfs_stripe_extent, strides)) / + sizeof(struct btrfs_raid_stride); + + printf("\t\t\tencoding: %s\n", stripe_encoding_name(encoding)); + for (i = 0; i < num_stripes; i++) + printf("\t\t\tstripe %d devid %llu physical %llu length %llu\n", i, + (unsigned long long)btrfs_raid_stride_devid_nr(eb, stripe, i), + (unsigned long long)btrfs_raid_stride_offset_nr(eb, stripe, i), + (unsigned long long)btrfs_raid_stride_length_nr(eb, stripe, i)); +} + void print_key_type(FILE *stream, u64 objectid, u8 type) { static const char* key_to_str[256] = { @@ -681,6 +726,7 @@ void print_key_type(FILE *stream, u64 objectid, u8 type) [BTRFS_PERSISTENT_ITEM_KEY] = "PERSISTENT_ITEM", [BTRFS_UUID_KEY_SUBVOL] = "UUID_KEY_SUBVOL", [BTRFS_UUID_KEY_RECEIVED_SUBVOL] = "UUID_KEY_RECEIVED_SUBVOL", + [BTRFS_RAID_STRIPE_KEY] = "RAID_STRIPE_KEY", }; if (type == 0 && objectid == BTRFS_FREE_SPACE_OBJECTID) { @@ -793,6 +839,9 @@ void print_objectid(FILE *stream, u64 objectid, u8 type) case BTRFS_CSUM_CHANGE_OBJECTID: fprintf(stream, "CSUM_CHANGE"); break; + case BTRFS_RAID_STRIPE_TREE_OBJECTID: + fprintf(stream, "RAID_STRIPE_TREE"); + break; case (u64)-1: fprintf(stream, "-1"); break; @@ -1469,6 +1518,9 @@ void btrfs_print_leaf(struct extent_buffer *eb, unsigned int mode) case BTRFS_TEMPORARY_ITEM_KEY: print_temporary_item(eb, ptr, objectid, offset); break; + case BTRFS_RAID_STRIPE_KEY: + print_raid_stripe_key(eb, item_size, ptr); + break; }; fflush(stdout); } From patchwork Thu Sep 14 16:05:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5FECEEAA55 for ; Thu, 14 Sep 2023 16:05:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241481AbjINQFv (ORCPT ); Thu, 14 Sep 2023 12:05:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241300AbjINQFt (ORCPT ); Thu, 14 Sep 2023 12:05:49 -0400 Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B6CD1BE1 for ; Thu, 14 Sep 2023 09:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707545; x=1726243545; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Gzghtc64t6MDPgziAXVJWilOZTxzWpOTUJKusY4KU64=; b=hHx2f+Sf0HqNgr1KJ5qcH4ht7Xi1k+KqvrwKC6SR1kN12lneSHN1x6MZ m7NaISiCWW6g9gWsncJ5Y1R7xWwnGzxBNE0KPLQzumqVoBVKd9H7tmm5c 9vbbw/ksdhmUto5DPrR5ZsmS7JcyZLMyHP09gqS5v4rf88o9R6Pvsb/ca 9tjojtNYhKmSg57jOyfuLbWPqFFigQessXF88ug382leGM4QbMPGioGbn kmHwrjLflD55WNyjI+A/oAlANphzRn4oXD9i4qgpu/0tcMCnO5CL0mDlx 67eCQfABK020T44iK7gzWbYNGDwFNH3kdtDQ9pjTTfE/Mw8MeOnqVpUXZ w==; X-CSE-ConnectionGUID: 0PrTYKppS7SdDb2Se3+t0Q== X-CSE-MsgGUID: bNU7rFJsSyiJopAC6qDg5w== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="242196091" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:05:44 +0800 IronPort-SDR: i4QsQGcdzg0DrHYlj+8CxIBtY74QSTSKBo17WxOuPjIl8hGStowHyOvxR3wqAHWiaSc1zc2ITG ol0d3dBwm4zEMWrb9dGCSXexj2pALU+oX93a5yk+0fxgIUN2su7Rm6aiXbzV/kkV5YTcRdxAhJ 2b8a5QWtMrJj2LE9JVnhF3yaIdczgD+OQI6yKUoU4Iio4OyKK9FgnrHUqLmBThxBG6QE0yrdj3 MPPGZZLeAzLJHyMsjcFaF6+WsU6N/Heh6hSpJYXxSPcxVDvdUsTp3NWdbuzRZOv5+KzdVsxr1d 9p8= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:18:27 -0700 IronPort-SDR: Kk6h0yfhJNNgBwAbhKwoIW3mGOApNWoRdt1WkCqez0JDw4BetZFthmKiwb5JTBCOXQCQJiSdak /twHPXBLQ3il5zqtGMXJepd/dweWtXllDLTrwIXpmbjBLeJb1UZ3oN79aIK9575aJzIT/7aJG4 +jvQWIY8nG33ZxIN/DDU4KBxEli5foqS14WxjaxIrhtNi+xpzTFoWaSp/43WyZdMe0Es1VOSbH z246Z5ArRSg2IgnJ5CTPk4XHStyWb9RQLwOQoIGILsENN++yxc1V9+Rsc5/oV4QNNYKPjX6ZoJ p54= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:05:45 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:05:35 -0700 Subject: [PATCH v4 4/6] btrfs-progs: allow zoned RAID MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v4-4-c921c15ec052@wdc.com> References: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> To: David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707540; l=7572; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=Gzghtc64t6MDPgziAXVJWilOZTxzWpOTUJKusY4KU64=; b=0TisWmauFK81TzBnJkyP6dPV0AteeGKl57g7JkV+v9OiyFmXuhZSseI96qUCEy1TWbJ5I5nFr YC+XfKTo4qkAmSIqQbX8Jes4kfK2+ZxFB050PiowUI5/KbJE8bxj5hx X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree is used. Signed-off-by: Johannes Thumshirn --- common/fsfeatures.c | 8 ++++++ kernel-shared/ctree.h | 3 +- kernel-shared/zoned.c | 34 ++++++++++++++++++++-- kernel-shared/zoned.h | 4 +-- mkfs/main.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++--- 5 files changed, 118 insertions(+), 10 deletions(-) diff --git a/common/fsfeatures.c b/common/fsfeatures.c index 00658fa5159f..2658f5072af4 100644 --- a/common/fsfeatures.c +++ b/common/fsfeatures.c @@ -189,6 +189,14 @@ static const struct btrfs_feature mkfs_features[] = { VERSION_NULL(safe), VERSION_NULL(default), .desc = "new extent tree format" + } , { + .name = "raid-stripe-tree", + .incompat_flag = BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE, + .sysfs_name = NULL, + VERSION_NULL(compat), + VERSION_NULL(safe), + VERSION_NULL(default), + .desc = "raid stripe tree" }, #endif /* Keep this one last */ diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index de09c15ca0eb..f6ee467adaab 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -102,7 +102,8 @@ static inline u32 __BTRFS_LEAF_DATA_SIZE(u32 nodesize) BTRFS_FEATURE_INCOMPAT_RAID1C34 | \ BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \ BTRFS_FEATURE_INCOMPAT_ZONED | \ - BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2) + BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 | \ + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE) #else #define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF | \ diff --git a/kernel-shared/zoned.c b/kernel-shared/zoned.c index d187c5763406..d8fad4319e44 100644 --- a/kernel-shared/zoned.c +++ b/kernel-shared/zoned.c @@ -737,7 +737,7 @@ out: return ret; } -bool zoned_profile_supported(u64 map_type) +bool zoned_profile_supported(u64 map_type, bool rst) { bool data = (map_type & BTRFS_BLOCK_GROUP_DATA); u64 flags = (map_type & BTRFS_BLOCK_GROUP_PROFILE_MASK); @@ -746,9 +746,37 @@ bool zoned_profile_supported(u64 map_type) if (flags == 0) return true; - /* We can support DUP on metadata */ +#if EXPERIMENTAL + if (data) { + if ((flags & BTRFS_BLOCK_GROUP_DUP) && rst) + return true; + /* Data RAID1 needs a raid-stripe-tree */ + if ((flags & BTRFS_BLOCK_GROUP_RAID1_MASK) && rst) + return true; + /* Data RAID0 needs a raid-stripe-tree */ + if ((flags & BTRFS_BLOCK_GROUP_RAID0) && rst) + return true; + /* Data RAID10 needs a raid-stripe-tree */ + if ((flags & BTRFS_BLOCK_GROUP_RAID10) && rst) + return true; + } else { + /* We can support DUP on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_DUP) + return true; + /* We can support RAID1 on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_RAID1_MASK) + return true; + /* We can support RAID0 on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_RAID0) + return true; + /* We can support RAID10 on metadata/system */ + if (flags & BTRFS_BLOCK_GROUP_RAID10) + return true; + } +#else if (!data && (flags & BTRFS_BLOCK_GROUP_DUP)) return true; +#endif /* All other profiles are not supported yet */ return false; @@ -863,7 +891,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, } } - if (!zoned_profile_supported(map->type)) { + if (!zoned_profile_supported(map->type, !!fs_info->stripe_root)) { error("zoned: profile %s not yet supported", btrfs_group_profile_str(map->type)); ret = -EINVAL; diff --git a/kernel-shared/zoned.h b/kernel-shared/zoned.h index 9e4162cf25c5..6efc60281bc9 100644 --- a/kernel-shared/zoned.h +++ b/kernel-shared/zoned.h @@ -133,7 +133,7 @@ static inline bool btrfs_dev_is_empty_zone(struct btrfs_device *device, u64 pos) return zinfo->zones[zno].cond == BLK_ZONE_COND_EMPTY; } -bool zoned_profile_supported(u64 map_type); +bool zoned_profile_supported(u64 map_type, bool rst); int btrfs_reset_dev_zone(int fd, struct blk_zone *zone); u64 btrfs_find_allocatable_zones(struct btrfs_device *device, u64 hole_start, u64 hole_end, u64 num_bytes); @@ -214,7 +214,7 @@ static inline int btrfs_wipe_temporary_sb(struct btrfs_fs_devices *fs_devices) return 0; } -static inline bool zoned_profile_supported(u64 map_type) +static inline bool zoned_profile_supported(u64 map_type, bool rst) { return false; } diff --git a/mkfs/main.c b/mkfs/main.c index 7acd39ec6531..7d07ba1e7001 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -962,6 +962,38 @@ fail: return ret; } +static int setup_raid_stripe_tree_root(struct btrfs_fs_info *fs_info) +{ + struct btrfs_trans_handle *trans; + struct btrfs_root *stripe_root; + struct btrfs_key key = { + .objectid = BTRFS_RAID_STRIPE_TREE_OBJECTID, + .type = BTRFS_ROOT_ITEM_KEY, + }; + int ret; + + trans = btrfs_start_transaction(fs_info->tree_root, 0); + if (IS_ERR(trans)) + return PTR_ERR(trans); + + stripe_root = btrfs_create_tree(trans, fs_info, &key); + if (IS_ERR(stripe_root)) { + ret = PTR_ERR(stripe_root); + goto abort; + } + fs_info->stripe_root = stripe_root; + add_root_to_dirty_list(stripe_root); + + ret = btrfs_commit_transaction(trans, fs_info->tree_root); + if (ret) + return ret; + + return 0; +abort: + btrfs_abort_transaction(trans, ret); + return ret; +} + /* Thread callback for device preparation */ static void *prepare_one_device(void *ctx) { @@ -1472,10 +1504,39 @@ int BOX_MAIN(mkfs)(int argc, char **argv) if (ret) goto error; - if (opt_zoned && (!zoned_profile_supported(BTRFS_BLOCK_GROUP_METADATA | metadata_profile) || - !zoned_profile_supported(BTRFS_BLOCK_GROUP_DATA | data_profile))) { - error("zoned mode does not yet support RAID/DUP profiles, please specify '-d single -m single' manually"); - goto error; +#if EXPERIMENTAL + if (opt_zoned && device_count) { + switch (data_profile & BTRFS_BLOCK_GROUP_PROFILE_MASK) { + case BTRFS_BLOCK_GROUP_DUP: + case BTRFS_BLOCK_GROUP_RAID1: + case BTRFS_BLOCK_GROUP_RAID1C3: + case BTRFS_BLOCK_GROUP_RAID1C4: + case BTRFS_BLOCK_GROUP_RAID0: + case BTRFS_BLOCK_GROUP_RAID10: + features.incompat_flags |= + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE; + break; + default: + break; + } + } +#endif + + if (opt_zoned) { + u64 metadata = BTRFS_BLOCK_GROUP_METADATA | metadata_profile; + u64 data = BTRFS_BLOCK_GROUP_DATA | data_profile; +#if EXPERIMENTAL + bool rst = features.incompat_flags & + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE; +#else + bool rst = false; +#endif + + if (!zoned_profile_supported(metadata, rst) || + !zoned_profile_supported(data, rst)) { + error("zoned mode does not yet support RAID/DUP profiles, please specify '-d single -m single' manually"); + goto error; + } } t_prepare = calloc(device_count, sizeof(*t_prepare)); @@ -1585,6 +1646,15 @@ int BOX_MAIN(mkfs)(int argc, char **argv) goto error; } + if (features.incompat_flags & BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE) { + ret = setup_raid_stripe_tree_root(fs_info); + if (ret < 0) { + error("failed to initialize raid-stripe-tree: %d (%m)", + ret); + goto out; + } + } + trans = btrfs_start_transaction(root, 1); if (IS_ERR(trans)) { errno = -PTR_ERR(trans); @@ -1750,6 +1820,7 @@ raid_groups: goto out; } } + if (bconf.verbose) { char features_buf[BTRFS_FEATURE_STRING_BUF_SIZE]; From patchwork Thu Sep 14 16:05:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6A37EEAA56 for ; Thu, 14 Sep 2023 16:05:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241506AbjINQFw (ORCPT ); Thu, 14 Sep 2023 12:05:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55958 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241467AbjINQFt (ORCPT ); Thu, 14 Sep 2023 12:05:49 -0400 Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6237B1BEF for ; Thu, 14 Sep 2023 09:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707545; x=1726243545; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=iAd9M2zZ4NMSY/nI1P/jXW61ZlJl9tSSeDR7DnVrWhc=; b=drz6dub/G+jVHmiLer7K+Sqyw8lrarLB6UsDmP8OD8kujghF6Gh5gmz5 iTU7rDemvwVfgAgw9V4vnMbMDo778d2cKLvFwM5Y405PzeSzet2Wuv2yC YkeDz2u5eNC/t2/3bghm7fXQ8LhLSaV7cMeBE2KkWdLuxUFUDw9M7MdM3 66UEje1vwFmQMtihfKfMh9GdH6lzK+6vQKMcQx6QEXVgEIfgfyfckE/98 /cALFde4U6V39VLk8oe0PW4nCzqACUbBvayq5T4ouoRw9y9JoyC1x6dfp 0ki6kjSV3FyZbxUV+Uj4xoFuF8TqXK/vxdOppJ3vNB9cquay7f9fAg2HH Q==; X-CSE-ConnectionGUID: cCQOkBDsQxy4dBqOWc2m7Q== X-CSE-MsgGUID: hsqoD+SYTm+7OVYMD4eHIg== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="242196092" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:05:45 +0800 IronPort-SDR: DFYmggBN0iIrB8Z6NkWWSyWdWJZthdB5cBF08v2LIA6+NJySL8exD+CO4bp3Ac3BwaNY+NUBzI apAhZcWIlyGOhe/u1HpxubAH5Glx79J3ypIA2Q8hak/CBcoUB+3TpHIktkfQHYOqHNlBQ1GgR0 CDHUYvh8Zj8nNksoUKm7zB6h5RS+sWnQsWuAXXCJ9U1bNb280fh68IsoERR2pi7TTDperjRqFh Xse+qNxu3k/wehGnG9MGCYEpiy/qIq+DiBB/3sHxvqJfYqefTbbBCw5VSyYPtaeZuWeEv6SW9N /DE= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:18:27 -0700 IronPort-SDR: 7rzxLi1OLqmJ2D9LUs5JGPmpve64EL71tYXTE4KG18occFFeqE8rnteapk4vWEGBvrkcfdkg9u 2/zcgqT158ZK/MSwiB601edqtEbqWF2S9ae5c3ctShjTLMTxCUUgGPQjXYQPrSMVgT7ie5tOCx hzhYrfFHUK6IKWinCpZDVSgqt2WSZkuoWOTX3HycDqFbSbZ5ADOI8+y9Q9A5KuT1vhHDPsId2w yFkjMhTsNS9BrfM+QFq0l1hqmS9oo+XmTeiitBB7FaEnmhDqM6TR9MZCjSl4/fbiF89GyVEX3M U4s= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:05:45 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:05:36 -0700 Subject: [PATCH v4 5/6] btrfs-progs: load zone info for all zoned devices MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v4-5-c921c15ec052@wdc.com> References: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> To: David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707540; l=531; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=iAd9M2zZ4NMSY/nI1P/jXW61ZlJl9tSSeDR7DnVrWhc=; b=ajH9JFSR8hW7dHqJpRbxl57FFuVkKSFnQUaBlWhmdpChZ2YexzbthvL7yKY9PUZ6dtPBkd/Db nlGh4qqaIv2C+g7mUZImUHBCWr0YPJjCniGgDvD7MnfpEvdLE6NP2J7 X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Signed-off-by: Johannes Thumshirn --- mkfs/main.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mkfs/main.c b/mkfs/main.c index 7d07ba1e7001..8a27704ac29e 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -1732,6 +1732,10 @@ int BOX_MAIN(mkfs)(int argc, char **argv) } } + + if (opt_zoned) + btrfs_get_dev_zone_info_all_devices(fs_info); + raid_groups: ret = create_raid_groups(trans, root, data_profile, metadata_profile, mixed, &allocation); From patchwork Thu Sep 14 16:05:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A188CEEAA54 for ; Thu, 14 Sep 2023 16:05:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241500AbjINQFx (ORCPT ); Thu, 14 Sep 2023 12:05:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241469AbjINQFu (ORCPT ); Thu, 14 Sep 2023 12:05:50 -0400 Received: from esa4.hgst.iphmx.com (esa4.hgst.iphmx.com [216.71.154.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D76201BDD for ; Thu, 14 Sep 2023 09:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707545; x=1726243545; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=zApmCeqyy3SfHYiANMzUPLxS2p9rjxha3fR6HA0U5Bc=; b=J6dNZ6gjveH/SLKfuDmjZrMp7cvMQGzUrlePwOH6sKPRKC4HklyPBCHk DL9VyegE+N42KjOY+c4XNHm94QpM0gYAVr7mi5EcsHdipWBCc42rf79d+ k4CC30kZcqljmuaKFzcz3jMEaT1m1lyXcxWI8gHo6dR/GxpgjtgX4rfbO n8CHW037F5GwbMQev/A9ZeVOcZ1MQQ9ZFTpDpdXGPXAEGJ6Ffem4r8BJr Tm4p13pDOcuqKcpUxJUfcqwEV3GuHsye6pzWdsJlPTft4h36XljXYCu/Q FybxGS4taUi/mgdh27XIkFe0brrr8rHfPMn+EiKvMjF3exkCApt+hwey2 A==; X-CSE-ConnectionGUID: g8EtlwefRVKgza/XDWPPEQ== X-CSE-MsgGUID: oRZicmpbT2+cWOKi03fKlA== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="242196094" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:05:45 +0800 IronPort-SDR: Skx1s6jW8r373EiJlOAvWwyoNIrD65eyeIPZ/N6Zxu71p1kI/zk/MeSp9/cT+aoWskrYulcNBg 5bcMXnhUc6ag3oYELkb/uSDReORyYKe7fzyNGpLT3YbpLxYdyG/ZcsnPXRBa3WowrpycP7GjG4 aomeQ1GbGk64KqhFOFWpDQcapbjNejkARfTJqqz1H38c9XqjbH8+PCkrN4ZKiuC/6usdYwDypp Dfr2olWBVRcIIyJnShWZ+4cgAquLOpm93WVgsB3NmIxupCVYxYh4y86JHV8jjI5oPo3zTdo7rA poY= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:18:28 -0700 IronPort-SDR: 69iFPuve+Xn+LfDduX4MaOY2XahvPTUNkvzuEb6cx/YGr8Paq5g2YQ4RghWjHELC0Cn1LC4Z9Q 0Hh384GupQt8LAuaYaxXXXKcz+5wP0JYp+bAj5jOD3u5e481yGGBLILRgBr/vnXvMyxIhhHYdZ gyOGYdu89BhKMAzAJDfdXsfessXG2OqKnFqRYOOFLQIbm6m2kA+6MmsSu/M1QZmGbSxLKyz2kZ cfGRyAtiBp5QmsTENKoyVeTquceVDvGDx479aAcopetEaETZ59jMu9gSOQgusfWJ4JZqjPM5Ks YYY= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:05:46 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:05:37 -0700 Subject: [PATCH v4 6/6] btrfs-progs: read stripe tree when mapping blocks MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v4-6-c921c15ec052@wdc.com> References: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v4-0-c921c15ec052@wdc.com> To: David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707540; l=3680; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=zApmCeqyy3SfHYiANMzUPLxS2p9rjxha3fR6HA0U5Bc=; b=jLrWyk9NsnDv2EftcMZ/NaxnZNBf5bg/Td5HTBO711oF569qZG+qdOhAPyOJfaBzFphdt6jCj Ub9yDAYYZLaBVyfe4GRRvusPCAXDxBHFZiF62Wz49knsSoFZkkYJulQ X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Signed-off-by: Johannes Thumshirn --- kernel-shared/volumes.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 113 insertions(+), 3 deletions(-) diff --git a/kernel-shared/volumes.c b/kernel-shared/volumes.c index 95d5930b95d8..2081f7db088f 100644 --- a/kernel-shared/volumes.c +++ b/kernel-shared/volumes.c @@ -1796,6 +1796,105 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, multi_ret, mirror_num, raid_map_ret); } +static bool btrfs_need_stripe_tree_update(struct btrfs_fs_info *fs_info, + u64 map_type) +{ +#if EXPERIMENTAL + const bool is_data = (map_type & BTRFS_BLOCK_GROUP_DATA); + + if (!btrfs_fs_incompat(fs_info, RAID_STRIPE_TREE)) + return false; + + if (!fs_info->stripe_root) + return false; + + if (!is_data) + return false; + + if (map_type & BTRFS_BLOCK_GROUP_DUP) + return true; + + if (map_type & BTRFS_BLOCK_GROUP_RAID1_MASK) + return true; + + if (map_type & BTRFS_BLOCK_GROUP_RAID0) + return true; + + if (map_type & BTRFS_BLOCK_GROUP_RAID10) + return true; + +#endif + return false; +} + +static int btrfs_stripe_tree_logical_to_physical(struct btrfs_fs_info *fs_info, + u64 logical, + struct btrfs_bio_stripe *stripe) +{ + struct btrfs_root *root = fs_info->stripe_root; + struct btrfs_path path = { 0 }; + struct btrfs_key key; + struct extent_buffer *leaf; + int slot; + int ret; + + key.objectid = logical; + key.type = BTRFS_RAID_STRIPE_KEY; + key.offset = 0; + + ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0); + if (ret < 0) + return ret; + + while (1) { + struct btrfs_key found_key; + struct btrfs_stripe_extent *extent; + int num_stripes; + u32 item_size; + int i; + + leaf = path.nodes[0]; + slot = path.slots[0]; + + if (slot >= btrfs_header_nritems(leaf)) { + ret = btrfs_next_leaf(root, &path); + if (ret == 0) + continue; + if (ret < 0) + goto error; + break; + } + + btrfs_item_key_to_cpu(leaf, &found_key, slot); + + if (found_key.type != BTRFS_RAID_STRIPE_KEY) + goto next; + + extent = btrfs_item_ptr(leaf, slot, + struct btrfs_stripe_extent); + item_size = btrfs_item_size(leaf, slot); + num_stripes = (item_size - + offsetof(struct btrfs_stripe_extent, strides)) / + sizeof(struct btrfs_raid_stride); + + for (i = 0; i < num_stripes; i++) { + if (stripe->dev->devid != + btrfs_raid_stride_devid_nr(leaf, extent, i)) + continue; + stripe->physical = btrfs_raid_stride_offset_nr(leaf, extent, i); + btrfs_release_path(&path); + return 0; + } + +next: + path.slots[0]++; + } + + btrfs_release_path(&path); +error: + return ret; +} + int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, u64 logical, u64 *length, u64 *type, struct btrfs_multi_bio **multi_ret, int mirror_num, @@ -1988,10 +2087,21 @@ again: BUG_ON(stripe_index >= map->num_stripes); for (i = 0; i < multi->num_stripes; i++) { - multi->stripes[i].physical = - map->stripes[stripe_index].physical + stripe_offset + - stripe_nr * map->stripe_len; multi->stripes[i].dev = map->stripes[stripe_index].dev; + + if (stripes_allocated && + btrfs_need_stripe_tree_update(fs_info, map->type)) { + int ret; + + ret = btrfs_stripe_tree_logical_to_physical(fs_info, logical, + &multi->stripes[i]); + if (ret) + return ret; + } else { + multi->stripes[i].physical = + map->stripes[stripe_index].physical + + stripe_offset + stripe_nr * map->stripe_len; + } stripe_index++; } *multi_ret = multi;