From patchwork Thu Sep 14 16:06:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3C06EEAA53 for ; Thu, 14 Sep 2023 16:07:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241641AbjINQHP (ORCPT ); Thu, 14 Sep 2023 12:07:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241822AbjINQHK (ORCPT ); Thu, 14 Sep 2023 12:07:10 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1FC22114; Thu, 14 Sep 2023 09:07:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707624; x=1726243624; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=yig7ZYZ9K8+Q6gaqvbyj//qu2/U7MXP1hMerBHoL1fQ=; b=UZuuydUp0IS88mC3nyNJZ3buCGDEbyddC9lS7SKEvasojAAnIPpASXIx iYyMjYWWeyXSklCskGgGfg+7dcSVrVT9MMXZQVMujKyR5Q86vTwT6p1vL JcY16DsRNUYOeQCwS3AU01XsK/rywKpFBg9RY4PCzwtXFhtntXlF4FlIv T7R0K60LI3pdJU7suhVQF58IupRTbepOCfU7E6Wl4QQkhSUPHPbVImu7/ qUe5rYw5lfHJDByEgK3zPzm8VwddbF2ifw92aHwBoFJdp/av3Z0+xLKfN 5FldMk6rGorZvuc64NyIslgtM0+he3hsEaHCUnLoJKIbE+cE7ewcx9RGB g==; X-CSE-ConnectionGUID: Oq6kqQ9xTHqTkLidpu+IHw== X-CSE-MsgGUID: 6b046WrKR6SGOepNmxysbg== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490527" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:04 +0800 IronPort-SDR: d8sZR2K5C4RiTHJxWFO03PXgNqLIyAbaIfs1BGzhHQpxO90mj0ZT6h93Oxucz4cR9MaDYDZYL1 EopFsfqwStvBshvZTBR4BqU+KOhqXNzEqf+opXXWhx+GBEwLn92pvUWL4zmNXj37QCB9E7c3+p a9Sw9B83xDNCYkjxws1KkUQal58/BWjxTdjXKH39tzytVhmo/PIHntsMk3aTAA53MMrRmaMUtz oXN3+Xh7cWxtipernJEK0rXXk2PRVwkk4PdNyDHGu9rbPe44iDSeQ2d+LxAp0fDNYxr0mAQItk XvQ= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:05 -0700 IronPort-SDR: d4BGto01G7yzNL8AbiuA/U+KaKuQ+zjOs9i9moHzhV9pOOh/0LkBd3FIgbOtxEpyqbya2mx8HB gnbqY1aZExcaokfwqDHecWBHUhUht/hUzAa6tJCZTSfovAvBV51T0DIfujpDnGEYYU/P+pmGdS EZW1Gaz8NSL0EABd9ADKpdZDofZ47yurdt2sNd4VCugyTBjF2eBgAq8Fm0gWYiIkPrQ0fjUx98 +ihBS17KxqEtJK6unhPIEGLGxNSNRKdGDGVXbi48ZgM0BqUpMPJypaUhDsXiNlFGiVVI2vTe5R TqI= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:04 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:06:56 -0700 Subject: [PATCH v9 01/11] btrfs: add raid stripe tree definitions MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-1-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=4184; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=yig7ZYZ9K8+Q6gaqvbyj//qu2/U7MXP1hMerBHoL1fQ=; b=T2aI+H2ePlmrUB4XIWgfPXkzlHXUBMZ0knml0Vm/qH4n46fEcOOEKJ8TnRfcxlFIbyLARA4d3 u/+FMs15Y4PAxnbpGtXHk80f+rIxztIDl3stCsIEXzVB4oFq3TVKZ2V X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add definitions for the raid stripe tree. This tree will hold information about the on-disk layout of the stripes in a RAID set. Each stripe extent has a 1:1 relationship with an on-disk extent item and is doing the logical to per-drive physical address translation for the extent item in question. Signed-off-by: Johannes Thumshirn --- fs/btrfs/accessors.h | 10 ++++++++++ fs/btrfs/locking.c | 1 + include/uapi/linux/btrfs_tree.h | 31 +++++++++++++++++++++++++++++++ 3 files changed, 42 insertions(+) diff --git a/fs/btrfs/accessors.h b/fs/btrfs/accessors.h index f958eccff477..977ff160a024 100644 --- a/fs/btrfs/accessors.h +++ b/fs/btrfs/accessors.h @@ -306,6 +306,16 @@ BTRFS_SETGET_FUNCS(timespec_nsec, struct btrfs_timespec, nsec, 32); BTRFS_SETGET_STACK_FUNCS(stack_timespec_sec, struct btrfs_timespec, sec, 64); BTRFS_SETGET_STACK_FUNCS(stack_timespec_nsec, struct btrfs_timespec, nsec, 32); +BTRFS_SETGET_FUNCS(stripe_extent_encoding, struct btrfs_stripe_extent, encoding, 8); +BTRFS_SETGET_FUNCS(raid_stride_devid, struct btrfs_raid_stride, devid, 64); +BTRFS_SETGET_FUNCS(raid_stride_physical, struct btrfs_raid_stride, physical, 64); +BTRFS_SETGET_FUNCS(raid_stride_length, struct btrfs_raid_stride, length, 64); +BTRFS_SETGET_STACK_FUNCS(stack_stripe_extent_encoding, + struct btrfs_stripe_extent, encoding, 8); +BTRFS_SETGET_STACK_FUNCS(stack_raid_stride_devid, struct btrfs_raid_stride, devid, 64); +BTRFS_SETGET_STACK_FUNCS(stack_raid_stride_physical, struct btrfs_raid_stride, physical, 64); +BTRFS_SETGET_STACK_FUNCS(stack_raid_stride_length, struct btrfs_raid_stride, length, 64); + /* struct btrfs_dev_extent */ BTRFS_SETGET_FUNCS(dev_extent_chunk_tree, struct btrfs_dev_extent, chunk_tree, 64); BTRFS_SETGET_FUNCS(dev_extent_chunk_objectid, struct btrfs_dev_extent, diff --git a/fs/btrfs/locking.c b/fs/btrfs/locking.c index 6ac4fd8cc8dc..74d8e2003f58 100644 --- a/fs/btrfs/locking.c +++ b/fs/btrfs/locking.c @@ -74,6 +74,7 @@ static struct btrfs_lockdep_keyset { { .id = BTRFS_UUID_TREE_OBJECTID, DEFINE_NAME("uuid") }, { .id = BTRFS_FREE_SPACE_TREE_OBJECTID, DEFINE_NAME("free-space") }, { .id = BTRFS_BLOCK_GROUP_TREE_OBJECTID, DEFINE_NAME("block-group") }, + { .id = BTRFS_RAID_STRIPE_TREE_OBJECTID, DEFINE_NAME("raid-stripe") }, { .id = 0, DEFINE_NAME("tree") }, }; diff --git a/include/uapi/linux/btrfs_tree.h b/include/uapi/linux/btrfs_tree.h index fc3c32186d7e..6d9c43416b6e 100644 --- a/include/uapi/linux/btrfs_tree.h +++ b/include/uapi/linux/btrfs_tree.h @@ -73,6 +73,9 @@ /* Holds the block group items for extent tree v2. */ #define BTRFS_BLOCK_GROUP_TREE_OBJECTID 11ULL +/* Tracks RAID stripes in block groups. */ +#define BTRFS_RAID_STRIPE_TREE_OBJECTID 12ULL + /* device stats in the device tree */ #define BTRFS_DEV_STATS_OBJECTID 0ULL @@ -261,6 +264,8 @@ #define BTRFS_DEV_ITEM_KEY 216 #define BTRFS_CHUNK_ITEM_KEY 228 +#define BTRFS_RAID_STRIPE_KEY 230 + /* * Records the overall state of the qgroups. * There's only one instance of this key present, @@ -719,6 +724,32 @@ struct btrfs_free_space_header { __le64 num_bitmaps; } __attribute__ ((__packed__)); +struct btrfs_raid_stride { + /* The btrfs device-id this raid extent lives on */ + __le64 devid; + /* The physical location on disk */ + __le64 physical; + /* The length of stride on this disk */ + __le64 length; +} __attribute__ ((__packed__)); + +/* The stripe_extent::encoding, 1:1 mapping of enum btrfs_raid_types */ +#define BTRFS_STRIPE_RAID0 1 +#define BTRFS_STRIPE_RAID1 2 +#define BTRFS_STRIPE_DUP 3 +#define BTRFS_STRIPE_RAID10 4 +#define BTRFS_STRIPE_RAID5 5 +#define BTRFS_STRIPE_RAID6 6 +#define BTRFS_STRIPE_RAID1C3 7 +#define BTRFS_STRIPE_RAID1C4 8 + +struct btrfs_stripe_extent { + __u8 encoding; + __u8 reserved[7]; + /* An array of raid strides this stripe is composed of */ + struct btrfs_raid_stride strides[]; +} __attribute__ ((__packed__)); + #define BTRFS_HEADER_FLAG_WRITTEN (1ULL << 0) #define BTRFS_HEADER_FLAG_RELOC (1ULL << 1) From patchwork Thu Sep 14 16:06:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F679EEAA55 for ; Thu, 14 Sep 2023 16:07:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241841AbjINQHQ (ORCPT ); Thu, 14 Sep 2023 12:07:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241649AbjINQHK (ORCPT ); Thu, 14 Sep 2023 12:07:10 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37E9D2103; Thu, 14 Sep 2023 09:07:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707626; x=1726243626; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=P0037+7VztgRtReqF7DKCInw0cIljZg5YluFFhKagsE=; b=UpkuOuQwHt7+Yp/67834g5I37mtkpBAXy8w1ZKvlrHUwsCqi8Wu4HP2C FrIJrJvvFeVJQjz8tUKyNcF/l1f3QzfJ6ICxz45YE+zDad2MvarflBYeV eO1Z+EvbMuTvF6wQH0BuqR6XxoQtlWL+hm1e1LJEdKUQf2u7L9zjzXLXs sc0DPYgS7cLNBkuN8m7AYT5EqL/CjDiT85LIE1fvrzMzvgO0bqH6laDSs t1fwLlUSsOriQNogo7HKUZvr0MdJ4pjgzAifFg8BSCr+lPonNrhGUBNA4 przCx7VAlNULpke566Wvsh2nRPEHyBGAWrxab295UvRcJad648uvWt60G A==; X-CSE-ConnectionGUID: DN8DTrugSyeAiHUycKttLA== X-CSE-MsgGUID: On6SvCYvQlKQ3XG4hfkbdw== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490531" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:05 +0800 IronPort-SDR: BIHPiiDaHGgUA33UBJVNBEoERiH6ha8+ndwPX7LY906jcBcGieyUvEaqJRVFKawCMoSeMxith8 UfqCiYnqEwIQpo6/syIymtSweP5MhlgNYb+6kqA5RdjNLBcM9E+yyG88/BAP8Zu3pUe7pOsbPK B6J1+ruEZRlMsNureIAtIJBq/n9rFpEZXaV5qc8UT+9iGXj/c1zNzHUmi76zYKE1sfqhmcet+K YJw/Fy4fGmWIE3jm635Izjc2MXQJCHi9usOmwJu4wK0rcNbELborXguYrTXhFKl7QlMCDCffmQ 4xM= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:07 -0700 IronPort-SDR: ZzxviuFIUU1rJcLWzc+De6L1N1jn09FSEstz7Zr5tx3N5/iKc6vBwGul9jrHNpUjt51L1b/nbr zKt7InzVyoExvqefpqeLfMsPkdRxpRvG3PEeREMdU/AFLdo7MCPDCdgBbLiiqpBDisNWs85XF4 qVVzi8MYnAwwy3SuhC8xkQUq2pCEAJGH1+490RiAzuAHO5/pp3lNgKTdD1LGZO6isNB+WKdd5Z oG/zUp3vi3Y7TEN0LtpdCz6eAAUIKMSgiNZ2k1pzTM4GYfET6FIYJdWXJZOyQPBJEYt09gmYzc UPU= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:05 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:06:57 -0700 Subject: [PATCH v9 02/11] btrfs: read raid-stripe-tree from disk MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-2-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=4080; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=P0037+7VztgRtReqF7DKCInw0cIljZg5YluFFhKagsE=; b=17pb+HbVzgmZYVQ3kaCrrsx6cvMdRdxbn0TPQ5TOwUvv94+n/Nxf5I3USW0x9C0vFnc5ciXye Qebxnp7RcHVBcLZ7l0QmO5cPqyGdj7yyS2e2LYsZqxIn+XIQBcJ/9wg X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org If we find a raid-stripe-tree on mount, read it from disk. Signed-off-by: Johannes Thumshirn --- fs/btrfs/block-rsv.c | 6 ++++++ fs/btrfs/disk-io.c | 18 ++++++++++++++++++ fs/btrfs/fs.h | 1 + include/uapi/linux/btrfs.h | 1 + 4 files changed, 26 insertions(+) diff --git a/fs/btrfs/block-rsv.c b/fs/btrfs/block-rsv.c index 6a8f9629bbbd..ceb5f586a2d5 100644 --- a/fs/btrfs/block-rsv.c +++ b/fs/btrfs/block-rsv.c @@ -356,6 +356,11 @@ void btrfs_update_global_block_rsv(struct btrfs_fs_info *fs_info) min_items++; } + if (btrfs_fs_incompat(fs_info, RAID_STRIPE_TREE)) { + num_bytes += btrfs_root_used(&fs_info->stripe_root->root_item); + min_items++; + } + /* * But we also want to reserve enough space so we can do the fallback * global reserve for an unlink, which is an additional @@ -407,6 +412,7 @@ void btrfs_init_root_block_rsv(struct btrfs_root *root) case BTRFS_EXTENT_TREE_OBJECTID: case BTRFS_FREE_SPACE_TREE_OBJECTID: case BTRFS_BLOCK_GROUP_TREE_OBJECTID: + case BTRFS_RAID_STRIPE_TREE_OBJECTID: root->block_rsv = &fs_info->delayed_refs_rsv; break; case BTRFS_ROOT_TREE_OBJECTID: diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 163f37ad1b27..dc577b3c53f6 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1179,6 +1179,8 @@ static struct btrfs_root *btrfs_get_global_root(struct btrfs_fs_info *fs_info, return btrfs_grab_root(fs_info->block_group_root); case BTRFS_FREE_SPACE_TREE_OBJECTID: return btrfs_grab_root(btrfs_global_root(fs_info, &key)); + case BTRFS_RAID_STRIPE_TREE_OBJECTID: + return btrfs_grab_root(fs_info->stripe_root); default: return NULL; } @@ -1259,6 +1261,7 @@ void btrfs_free_fs_info(struct btrfs_fs_info *fs_info) btrfs_put_root(fs_info->fs_root); btrfs_put_root(fs_info->data_reloc_root); btrfs_put_root(fs_info->block_group_root); + btrfs_put_root(fs_info->stripe_root); btrfs_check_leaked_roots(fs_info); btrfs_extent_buffer_leak_debug_check(fs_info); kfree(fs_info->super_copy); @@ -1804,6 +1807,7 @@ static void free_root_pointers(struct btrfs_fs_info *info, bool free_chunk_root) free_root_extent_buffers(info->fs_root); free_root_extent_buffers(info->data_reloc_root); free_root_extent_buffers(info->block_group_root); + free_root_extent_buffers(info->stripe_root); if (free_chunk_root) free_root_extent_buffers(info->chunk_root); } @@ -2280,6 +2284,20 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info) fs_info->uuid_root = root; } + if (btrfs_fs_incompat(fs_info, RAID_STRIPE_TREE)) { + location.objectid = BTRFS_RAID_STRIPE_TREE_OBJECTID; + root = btrfs_read_tree_root(tree_root, &location); + if (IS_ERR(root)) { + if (!btrfs_test_opt(fs_info, IGNOREBADROOTS)) { + ret = PTR_ERR(root); + goto out; + } + } else { + set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state); + fs_info->stripe_root = root; + } + } + return 0; out: btrfs_warn(fs_info, "failed to read root (objectid=%llu): %d", diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h index d84a390336fc..5c7778e8b5ed 100644 --- a/fs/btrfs/fs.h +++ b/fs/btrfs/fs.h @@ -367,6 +367,7 @@ struct btrfs_fs_info { struct btrfs_root *uuid_root; struct btrfs_root *data_reloc_root; struct btrfs_root *block_group_root; + struct btrfs_root *stripe_root; /* The log root tree is a directory of all the other log roots */ struct btrfs_root *log_root_tree; diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index dbb8b96da50d..b9a1d9af8ae8 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -333,6 +333,7 @@ struct btrfs_ioctl_fs_info_args { #define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11) #define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12) #define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) +#define BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE (1ULL << 14) struct btrfs_ioctl_feature_flags { __u64 compat_flags; From patchwork Thu Sep 14 16:06:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FCFEEEAA54 for ; Thu, 14 Sep 2023 16:07:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241843AbjINQHS (ORCPT ); Thu, 14 Sep 2023 12:07:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241568AbjINQHL (ORCPT ); Thu, 14 Sep 2023 12:07:11 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCF321BEF; Thu, 14 Sep 2023 09:07:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707627; x=1726243627; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=jgfukhRUFwWAsbAZGn7KE/ZGcErBq6JwKqAEs9DqdKI=; b=MDWnQ35lin+IJOn2eYkQ1wzimbYzfDvcyZk1hX9pTr7zR20DrSpGfSH6 liQDVEah1fgEWI84POYc40LluOVy38jXWzy1DoSWQPh5CaLXN1fJr5X2P POQKZtAvYzEMXwRP40a12i/blvA35VKSEwJEoIfN9vOr1mmBEAYPAuxPx LH3t4yciliDdlDvRXJ2PWEbMSWGbD1ngD0tcFXvtwEnAQumjOESG0pzqH /wT7yZmv+de84KUPHXmRN55GQvB2ffe31DmcyRAjDeHmRkjuZemFdNIR7 eZnZLKeMI85S7T7WRWnLfR5YliWO+G10RUm1utWoq2YDiNCs8b7FYj5kU A==; X-CSE-ConnectionGUID: ZqMRwC2mS7iZa3TeX02M2w== X-CSE-MsgGUID: VhGShNMRSQOI0fQYWffHDA== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490535" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:07 +0800 IronPort-SDR: ePbn0KJ22BuvvQ+WIibzL4yrvQc72q752OfVJCBAxudEeIYIsK3+PzQ0xBufOIqOIpMIo/vnxO XQJ8jh9/jbkDEZG2GKZNyzLrsQY+wrUQ+UYsXRSv6m0dWL7IIQJ2oWJyxpXfxoNBedM+aknJed eCqw8gt3WDG2uE90V8BKewBsbszeFjKkMiNqjdQO1R0cI/o5IhsulqICof52vj6Qo7URoAAHjS Aa+asai7rFf0HuIgV0BKDxWtZdyeXBXxx+ZNguHrCuUhu6PtfY+MBT8bg8PtTAfmyb8CXsLvIE +nE= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:08 -0700 IronPort-SDR: K/U/nq0IUD4/VLMh+zjdowKKdQOkKF/Xw9BrMLwO/aaqIMVbfsDYHzC2hxw3soZoQvPqMH+Tqr NLhWXDpFgTamqzem4BjU9efThE6zqSpW/jKO9QoJZcyw1relIXn4s/07Cv6ycMNZG/76Zl2adV 81WvVhnNMjxbFFUZqTZjhbgILiLQLnJroCzbKg8rmwGAqcWJckJ72KXxJQzoB9ljzW4Isia1q2 ecVVkW8xJGFl5qnBnXcTLNuIqmQrLQKpfSF7FjgVj2ihGMJBCdupX+QNdYZdbrbRYKYd14/yY/ hpk= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:07 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:06:58 -0700 Subject: [PATCH v9 03/11] btrfs: add support for inserting raid stripe extents MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-3-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=16439; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=jgfukhRUFwWAsbAZGn7KE/ZGcErBq6JwKqAEs9DqdKI=; b=pH9C0Nh2cW6m8ObF9jJWr635e+pwDxvYoahaBkIMunycwgSQITfthizAKTKh/UkI/vyNl5/2L J9YhuWKJaD0A7L8h5mJ+qQWe4B9nGUWdqDzz7sBP/EiY9gRgpBCjaGp X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add support for inserting stripe extents into the raid stripe tree on completion of every write that needs an extra logical-to-physical translation when using RAID. Inserting the stripe extents happens after the data I/O has completed, this is done to a) support zone-append and b) rule out the possibility of a RAID-write-hole. Signed-off-by: Johannes Thumshirn --- fs/btrfs/Makefile | 2 +- fs/btrfs/bio.c | 23 +++++ fs/btrfs/extent-tree.c | 1 + fs/btrfs/inode.c | 8 +- fs/btrfs/ordered-data.c | 1 + fs/btrfs/ordered-data.h | 2 + fs/btrfs/raid-stripe-tree.c | 245 ++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/raid-stripe-tree.h | 34 ++++++ fs/btrfs/volumes.c | 4 +- fs/btrfs/volumes.h | 15 +-- 10 files changed, 326 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/Makefile b/fs/btrfs/Makefile index c57d80729d4f..525af975f61c 100644 --- a/fs/btrfs/Makefile +++ b/fs/btrfs/Makefile @@ -33,7 +33,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \ uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \ block-rsv.o delalloc-space.o block-group.o discard.o reflink.o \ subpage.o tree-mod-log.o extent-io-tree.o fs.o messages.o bio.o \ - lru_cache.o + lru_cache.o raid-stripe-tree.o btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o btrfs-$(CONFIG_BTRFS_FS_REF_VERIFY) += ref-verify.o diff --git a/fs/btrfs/bio.c b/fs/btrfs/bio.c index 31ff36990404..ddbe6f8d4ea2 100644 --- a/fs/btrfs/bio.c +++ b/fs/btrfs/bio.c @@ -14,6 +14,7 @@ #include "rcu-string.h" #include "zoned.h" #include "file-item.h" +#include "raid-stripe-tree.h" static struct bio_set btrfs_bioset; static struct bio_set btrfs_clone_bioset; @@ -415,6 +416,9 @@ static void btrfs_orig_write_end_io(struct bio *bio) else bio->bi_status = BLK_STS_OK; + if (bio_op(bio) == REQ_OP_ZONE_APPEND && !bio->bi_status) + stripe->physical = bio->bi_iter.bi_sector << SECTOR_SHIFT; + btrfs_orig_bbio_end_io(bbio); btrfs_put_bioc(bioc); } @@ -426,6 +430,8 @@ static void btrfs_clone_write_end_io(struct bio *bio) if (bio->bi_status) { atomic_inc(&stripe->bioc->error); btrfs_log_dev_io_error(bio, stripe->dev); + } else if (bio_op(bio) == REQ_OP_ZONE_APPEND) { + stripe->physical = bio->bi_iter.bi_sector << SECTOR_SHIFT; } /* Pass on control to the original bio this one was cloned from */ @@ -487,6 +493,7 @@ static void btrfs_submit_mirrored_bio(struct btrfs_io_context *bioc, int dev_nr) bio->bi_private = &bioc->stripes[dev_nr]; bio->bi_iter.bi_sector = bioc->stripes[dev_nr].physical >> SECTOR_SHIFT; bioc->stripes[dev_nr].bioc = bioc; + bioc->size = bio->bi_iter.bi_size; btrfs_submit_dev_bio(bioc->stripes[dev_nr].dev, bio); } @@ -496,6 +503,8 @@ static void __btrfs_submit_bio(struct bio *bio, struct btrfs_io_context *bioc, if (!bioc) { /* Single mirror read/write fast path. */ btrfs_bio(bio)->mirror_num = mirror_num; + if (bio_op(bio) != REQ_OP_READ) + btrfs_bio(bio)->orig_physical = smap->physical; bio->bi_iter.bi_sector = smap->physical >> SECTOR_SHIFT; if (bio_op(bio) != REQ_OP_READ) btrfs_bio(bio)->orig_physical = smap->physical; @@ -688,6 +697,20 @@ static bool btrfs_submit_chunk(struct btrfs_bio *bbio, int mirror_num) bio->bi_opf |= REQ_OP_ZONE_APPEND; } + if (is_data_bbio(bbio) && bioc && + btrfs_need_stripe_tree_update(bioc->fs_info, + bioc->map_type)) { + /* + * No locking for the list update, as we only add to + * the list in the I/O submission path, and list + * iteration only happens in the completion path, + * which can't happen until after the last submission. + */ + btrfs_get_bioc(bioc); + list_add_tail(&bioc->ordered_entry, + &bbio->ordered->bioc_list); + } + /* * Csum items for reloc roots have already been cloned at this * point, so they are handled as part of the no-checksum case. diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index cb12bfb047e7..959d7449ea0d 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -42,6 +42,7 @@ #include "file-item.h" #include "orphan.h" #include "tree-checker.h" +#include "raid-stripe-tree.h" #undef SCRAMBLE_DELAYED_REFS diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index e02a5ba5b533..b5e0ed3a36f7 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -71,6 +71,7 @@ #include "super.h" #include "orphan.h" #include "backref.h" +#include "raid-stripe-tree.h" struct btrfs_iget_args { u64 ino; @@ -3091,6 +3092,10 @@ int btrfs_finish_one_ordered(struct btrfs_ordered_extent *ordered_extent) trans->block_rsv = &inode->block_rsv; + ret = btrfs_insert_raid_extent(trans, ordered_extent); + if (ret) + goto out; + if (test_bit(BTRFS_ORDERED_COMPRESSED, &ordered_extent->flags)) compress_type = ordered_extent->compress_type; if (test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags)) { @@ -3224,7 +3229,8 @@ int btrfs_finish_one_ordered(struct btrfs_ordered_extent *ordered_extent) int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered) { if (btrfs_is_zoned(btrfs_sb(ordered->inode->i_sb)) && - !test_bit(BTRFS_ORDERED_IOERR, &ordered->flags)) + !test_bit(BTRFS_ORDERED_IOERR, &ordered->flags) && + list_empty(&ordered->bioc_list)) btrfs_finish_ordered_zoned(ordered); return btrfs_finish_one_ordered(ordered); } diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c index 345c449d588c..55c7d5543265 100644 --- a/fs/btrfs/ordered-data.c +++ b/fs/btrfs/ordered-data.c @@ -191,6 +191,7 @@ static struct btrfs_ordered_extent *alloc_ordered_extent( INIT_LIST_HEAD(&entry->log_list); INIT_LIST_HEAD(&entry->root_extent_list); INIT_LIST_HEAD(&entry->work_list); + INIT_LIST_HEAD(&entry->bioc_list); init_completion(&entry->completion); /* diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs/ordered-data.h index 173bd5c5df26..1c51ac57e5df 100644 --- a/fs/btrfs/ordered-data.h +++ b/fs/btrfs/ordered-data.h @@ -151,6 +151,8 @@ struct btrfs_ordered_extent { struct completion completion; struct btrfs_work flush_work; struct list_head work_list; + + struct list_head bioc_list; }; static inline void diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c new file mode 100644 index 000000000000..7cdcc45a8796 --- /dev/null +++ b/fs/btrfs/raid-stripe-tree.c @@ -0,0 +1,245 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2023 Western Digital Corporation or its affiliates. + */ + +#include + +#include "ctree.h" +#include "fs.h" +#include "accessors.h" +#include "transaction.h" +#include "disk-io.h" +#include "raid-stripe-tree.h" +#include "volumes.h" +#include "misc.h" +#include "print-tree.h" + +static int btrfs_insert_one_raid_extent(struct btrfs_trans_handle *trans, + int num_stripes, + struct btrfs_io_context *bioc) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_key stripe_key; + struct btrfs_root *stripe_root = fs_info->stripe_root; + u8 encoding = btrfs_bg_flags_to_raid_index(bioc->map_type); + struct btrfs_stripe_extent *stripe_extent; + const size_t item_size = struct_size(stripe_extent, strides, num_stripes); + int ret; + + stripe_extent = kzalloc(item_size, GFP_NOFS); + if (!stripe_extent) { + btrfs_abort_transaction(trans, -ENOMEM); + btrfs_end_transaction(trans); + return -ENOMEM; + } + + btrfs_set_stack_stripe_extent_encoding(stripe_extent, encoding); + for (int i = 0; i < num_stripes; i++) { + u64 devid = bioc->stripes[i].dev->devid; + u64 physical = bioc->stripes[i].physical; + u64 length = bioc->stripes[i].length; + struct btrfs_raid_stride *raid_stride = + &stripe_extent->strides[i]; + + if (length == 0) + length = bioc->size; + + btrfs_set_stack_raid_stride_devid(raid_stride, devid); + btrfs_set_stack_raid_stride_physical(raid_stride, physical); + btrfs_set_stack_raid_stride_length(raid_stride, length); + } + + stripe_key.objectid = bioc->logical; + stripe_key.type = BTRFS_RAID_STRIPE_KEY; + stripe_key.offset = bioc->size; + + ret = btrfs_insert_item(trans, stripe_root, &stripe_key, stripe_extent, + item_size); + if (ret) + btrfs_abort_transaction(trans, ret); + + kfree(stripe_extent); + + return ret; +} + +static int btrfs_insert_mirrored_raid_extents(struct btrfs_trans_handle *trans, + struct btrfs_ordered_extent *ordered, + u64 map_type) +{ + int num_stripes = btrfs_bg_type_to_factor(map_type); + struct btrfs_io_context *bioc; + int ret; + + list_for_each_entry(bioc, &ordered->bioc_list, ordered_entry) { + ret = btrfs_insert_one_raid_extent(trans, num_stripes, bioc); + if (ret) + return ret; + } + + return 0; +} + +static int btrfs_insert_striped_mirrored_raid_extents( + struct btrfs_trans_handle *trans, + struct btrfs_ordered_extent *ordered, + u64 map_type) +{ + struct btrfs_io_context *bioc; + struct btrfs_io_context *rbioc; + const int nstripes = list_count_nodes(&ordered->bioc_list); + const int index = btrfs_bg_flags_to_raid_index(map_type); + const int substripes = btrfs_raid_array[index].sub_stripes; + const int max_stripes = + trans->fs_info->fs_devices->rw_devices / substripes; + int left = nstripes; + int i; + int ret = 0; + u64 stripe_end; + u64 prev_end; + + if (nstripes == 1) + return btrfs_insert_mirrored_raid_extents(trans, ordered, map_type); + + rbioc = kzalloc(struct_size(rbioc, stripes, nstripes * substripes), + GFP_NOFS); + if (!rbioc) + return -ENOMEM; + + rbioc->map_type = map_type; + rbioc->logical = list_first_entry(&ordered->bioc_list, typeof(*rbioc), + ordered_entry)->logical; + + stripe_end = rbioc->logical; + prev_end = stripe_end; + i = 0; + list_for_each_entry(bioc, &ordered->bioc_list, ordered_entry) { + + rbioc->size += bioc->size; + for (int j = 0; j < substripes; j++) { + int stripe = i + j; + rbioc->stripes[stripe].dev = bioc->stripes[j].dev; + rbioc->stripes[stripe].physical = bioc->stripes[j].physical; + rbioc->stripes[stripe].length = bioc->size; + } + + stripe_end += rbioc->size; + if (i >= nstripes || + (stripe_end - prev_end >= max_stripes * BTRFS_STRIPE_LEN)) { + ret = btrfs_insert_one_raid_extent(trans, + nstripes * substripes, + rbioc); + if (ret) + goto out; + + left -= nstripes; + i = 0; + rbioc->logical += rbioc->size; + rbioc->size = 0; + } else { + i += substripes; + prev_end = stripe_end; + } + } + + if (left) { + bioc = list_prev_entry(bioc, ordered_entry); + ret = btrfs_insert_one_raid_extent(trans, substripes, bioc); + } + +out: + kfree(rbioc); + return ret; +} + +static int btrfs_insert_striped_raid_extents(struct btrfs_trans_handle *trans, + struct btrfs_ordered_extent *ordered, + u64 map_type) +{ + struct btrfs_io_context *bioc; + struct btrfs_io_context *rbioc; + const int nstripes = list_count_nodes(&ordered->bioc_list); + int i; + int ret = 0; + + rbioc = kzalloc(struct_size(rbioc, stripes, nstripes), GFP_NOFS); + if (!rbioc) + return -ENOMEM; + rbioc->map_type = map_type; + rbioc->logical = list_first_entry(&ordered->bioc_list, typeof(*rbioc), + ordered_entry)->logical; + + i = 0; + list_for_each_entry(bioc, &ordered->bioc_list, ordered_entry) { + rbioc->size += bioc->size; + rbioc->stripes[i].dev = bioc->stripes[0].dev; + rbioc->stripes[i].physical = bioc->stripes[0].physical; + rbioc->stripes[i].length = bioc->size; + + if (i == nstripes - 1) { + ret = btrfs_insert_one_raid_extent(trans, nstripes, rbioc); + if (ret) + goto out; + + i = 0; + rbioc->logical += rbioc->size; + rbioc->size = 0; + } else { + i++; + } + } + + if (i && i < nstripes - 1) + ret = btrfs_insert_one_raid_extent(trans, i, rbioc); + +out: + kfree(rbioc); + return ret; +} + +int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans, + struct btrfs_ordered_extent *ordered_extent) +{ + struct btrfs_io_context *bioc; + u64 map_type; + int ret; + + if (!trans->fs_info->stripe_root) + return 0; + + map_type = list_first_entry(&ordered_extent->bioc_list, typeof(*bioc), + ordered_entry)->map_type; + + switch (map_type & BTRFS_BLOCK_GROUP_PROFILE_MASK) { + case BTRFS_BLOCK_GROUP_DUP: + case BTRFS_BLOCK_GROUP_RAID1: + case BTRFS_BLOCK_GROUP_RAID1C3: + case BTRFS_BLOCK_GROUP_RAID1C4: + ret = btrfs_insert_mirrored_raid_extents(trans, ordered_extent, + map_type); + break; + case BTRFS_BLOCK_GROUP_RAID0: + ret = btrfs_insert_striped_raid_extents(trans, ordered_extent, + map_type); + break; + case BTRFS_BLOCK_GROUP_RAID10: + ret = btrfs_insert_striped_mirrored_raid_extents(trans, ordered_extent, map_type); + break; + default: + btrfs_err(trans->fs_info, "unknown block-group profile %lld", + map_type & BTRFS_BLOCK_GROUP_PROFILE_MASK); + ASSERT(0); + ret = -EINVAL; + break; + } + + while (!list_empty(&ordered_extent->bioc_list)) { + bioc = list_first_entry(&ordered_extent->bioc_list, + typeof(*bioc), ordered_entry); + list_del(&bioc->ordered_entry); + btrfs_put_bioc(bioc); + } + + return ret; +} diff --git a/fs/btrfs/raid-stripe-tree.h b/fs/btrfs/raid-stripe-tree.h new file mode 100644 index 000000000000..884f0e99d5e8 --- /dev/null +++ b/fs/btrfs/raid-stripe-tree.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2023 Western Digital Corporation or its affiliates. + */ + +#ifndef BTRFS_RAID_STRIPE_TREE_H +#define BTRFS_RAID_STRIPE_TREE_H + +struct btrfs_io_context; +struct btrfs_io_stripe; +struct btrfs_ordered_extent; +struct btrfs_trans_handle; + +int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans, + struct btrfs_ordered_extent *ordered_extent); + +static inline bool btrfs_need_stripe_tree_update(struct btrfs_fs_info *fs_info, + u64 map_type) +{ + u64 type = map_type & BTRFS_BLOCK_GROUP_TYPE_MASK; + u64 profile = map_type & BTRFS_BLOCK_GROUP_PROFILE_MASK; + + if (!fs_info->stripe_root) + return false; + + if (type != BTRFS_BLOCK_GROUP_DATA) + return false; + + if (profile & BTRFS_BLOCK_GROUP_RAID1_MASK) + return true; + + return false; +} +#endif diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index a1eae8b5b412..c2bac87912c7 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -5984,6 +5984,7 @@ static int find_live_mirror(struct btrfs_fs_info *fs_info, } static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_info, + u64 logical, u16 total_stripes) { struct btrfs_io_context *bioc; @@ -6003,6 +6004,7 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_ bioc->fs_info = fs_info; bioc->replace_stripe_src = -1; bioc->full_stripe_logical = (u64)-1; + bioc->logical = logical; return bioc; } @@ -6537,7 +6539,7 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, goto out; } - bioc = alloc_btrfs_io_context(fs_info, num_alloc_stripes); + bioc = alloc_btrfs_io_context(fs_info, logical, num_alloc_stripes); if (!bioc) { ret = -ENOMEM; goto out; diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 26397adc8706..2043aff6e966 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -390,12 +390,11 @@ struct btrfs_fs_devices { struct btrfs_io_stripe { struct btrfs_device *dev; - union { - /* Block mapping */ - u64 physical; - /* For the endio handler */ - struct btrfs_io_context *bioc; - }; + /* Block mapping */ + u64 physical; + u64 length; + /* For the endio handler */ + struct btrfs_io_context *bioc; }; struct btrfs_discard_stripe { @@ -428,6 +427,10 @@ struct btrfs_io_context { atomic_t error; u16 max_errors; + u64 logical; + u64 size; + struct list_head ordered_entry; + /* * The total number of stripes, including the extra duplicated * stripe for replace. From patchwork Thu Sep 14 16:06:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65545EEAA56 for ; Thu, 14 Sep 2023 16:07:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241850AbjINQHU (ORCPT ); Thu, 14 Sep 2023 12:07:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241717AbjINQHN (ORCPT ); Thu, 14 Sep 2023 12:07:13 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BB471FD5; Thu, 14 Sep 2023 09:07:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707629; x=1726243629; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=wt2EjSnLJ8D+t3XufPg5cUvMG9P+h96vVSNkrRx35uY=; b=TzEtfM/PlVPeJdHNFjMkI+h1XGFu5zH2XGMNXZGzq2S0aNWbe2cI0ey1 WtE+ZHSjRcc7MVq55biEo+lLIXjjhceO1vqZq84HIkTYieq6fZ+n6deow xfAvRqPaRoCYixOBL+not9TbwtCJrAXbP2aB7wLgM+gK8MCME/r6zBU3o NMZ6PO/9m2HLLBFj7jVfyTKYhfU3ju9Vs4sz+PhscYD9dgc9CNm9d8tIG fBjgoy5wu9HG2Za9fuMDW1GsFQk1qQijdfTV+QV3PxBLAYuUA68Jw6edu lOKWMJikQnApHxOQJ9wUtNgXQfgv53rWNFqfEwmH4zMHW2UKafxkIk7oS Q==; X-CSE-ConnectionGUID: shcK7XbSTlyoIH/HOi24FQ== X-CSE-MsgGUID: V1is4AJkS+SPYDSzl1ivsg== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490539" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:09 +0800 IronPort-SDR: 0MBxTIVMWkBibFUnPT2+WsV5BGB+TN6SLiLeUNHaqOnnm7dtNXr7PCaZcT1OmHzgevu+bnbbDF Dv/aM+w7GKeEJw6Je/OdEDwEs7qvyQFQJeRxJ+kPcBR1D3Z+KQn3rj+E5sL5w5P6CzLd2SRBIp +hOJu4CcYhvdKaXE3THicALzKQUAMErUXjR8TUiw1x3LO9rdZBYNkaAThidvnxGTiYso7gI2Wh Qae2O5rahUeuYXwQUahLUOe3TUsnA0wK1MZLJt4GxhkY/Ken9+DWh5dRaBm3fCXQg/j8p+nNFt FKc= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:10 -0700 IronPort-SDR: NzMtAD0aBLJu4UVoVwnSplUvtXjWV5h/9Eskc4ze8PukHf4LcjXEG0nViwwNYu3dF7+M/YzreL 2S0S41NuQ4tZ+ZkD5JkXohR9XZDZ1RbZdIMwhq9p4ZcfhYHOefpjH+XXvg3SbTrpTl0WUkDUkf uvVzg5MbOvQztuFqwqP8jhuFS3+O/H6Kj02O3rUii5G7X64TgIG3dWErSLF9o70mQqdwDHIFxD gz+rqtT0SvwSak1hqxTxT5WiaY/CrUFSE5dHYwk4MWUTUOOGQ470l1AxNJ23qFJUO9VxBPDcTQ iwU= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:08 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:06:59 -0700 Subject: [PATCH v9 04/11] btrfs: delete stripe extent on extent deletion MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-4-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=3151; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=wt2EjSnLJ8D+t3XufPg5cUvMG9P+h96vVSNkrRx35uY=; b=RtVjLsmQ9nmV1wv+PDIz8Cgkox8svVSkLIsVsv5ioDUxNpakVSWI+lka5XC5QuDZKXNLciTcf htX8Evb+aAoBv2qUa/n7NE1YXTZe2noQaaaTqSajUJeQ7TNR3MuCTkF X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org As each stripe extent is tied to an extent item, delete the stripe extent once the corresponding extent item is deleted. Signed-off-by: Johannes Thumshirn --- fs/btrfs/extent-tree.c | 6 +++++ fs/btrfs/raid-stripe-tree.c | 60 +++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/raid-stripe-tree.h | 2 ++ 3 files changed, 68 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 959d7449ea0d..27859c7773ce 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2860,6 +2860,12 @@ static int do_free_extent_accounting(struct btrfs_trans_handle *trans, btrfs_abort_transaction(trans, ret); return ret; } + + ret = btrfs_delete_raid_extent(trans, bytenr, num_bytes); + if (ret) { + btrfs_abort_transaction(trans, ret); + return ret; + } } ret = add_to_free_space_tree(trans, bytenr, num_bytes); diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c index 7cdcc45a8796..517bc08803f1 100644 --- a/fs/btrfs/raid-stripe-tree.c +++ b/fs/btrfs/raid-stripe-tree.c @@ -15,6 +15,66 @@ #include "misc.h" #include "print-tree.h" +int btrfs_delete_raid_extent(struct btrfs_trans_handle *trans, u64 start, + u64 length) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_root *stripe_root = fs_info->stripe_root; + struct btrfs_path *path; + struct btrfs_key key; + struct extent_buffer *leaf; + u64 found_start; + u64 found_end; + u64 end = start + length; + int slot; + int ret; + + if (!stripe_root) + return 0; + + path = btrfs_alloc_path(); + if (!path) + return -ENOMEM; + + while (1) { + + key.objectid = start; + key.type = BTRFS_RAID_STRIPE_KEY; + key.offset = length; + + ret = btrfs_search_slot(trans, stripe_root, &key, path, -1, 1); + if (ret < 0) + break; + if (ret > 0) { + ret = 0; + if (path->slots[0] == 0) + break; + path->slots[0]--; + } + + leaf = path->nodes[0]; + slot = path->slots[0]; + btrfs_item_key_to_cpu(leaf, &key, slot); + found_start = key.objectid; + found_end = found_start + key.offset; + + /* That stripe ends before we start, we're done */ + if (found_end <= start) + break; + + ASSERT(found_start >= start && found_end <= end); + ret = btrfs_del_item(trans, stripe_root, path); + if (ret) + break; + + btrfs_release_path(path); + } + + btrfs_free_path(path); + return ret; + +} + static int btrfs_insert_one_raid_extent(struct btrfs_trans_handle *trans, int num_stripes, struct btrfs_io_context *bioc) diff --git a/fs/btrfs/raid-stripe-tree.h b/fs/btrfs/raid-stripe-tree.h index 884f0e99d5e8..b3a127c997c8 100644 --- a/fs/btrfs/raid-stripe-tree.h +++ b/fs/btrfs/raid-stripe-tree.h @@ -11,6 +11,8 @@ struct btrfs_io_stripe; struct btrfs_ordered_extent; struct btrfs_trans_handle; +int btrfs_delete_raid_extent(struct btrfs_trans_handle *trans, u64 start, + u64 length); int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans, struct btrfs_ordered_extent *ordered_extent); From patchwork Thu Sep 14 16:07:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAB37EEAA52 for ; Thu, 14 Sep 2023 16:07:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241920AbjINQHf (ORCPT ); Thu, 14 Sep 2023 12:07:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241832AbjINQHP (ORCPT ); Thu, 14 Sep 2023 12:07:15 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A761C1FF6; Thu, 14 Sep 2023 09:07:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707631; x=1726243631; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=Yq+4Hu/kdBSP8VJdQwG78yDoI9jI2RynIMJjvJLIy94=; b=A/QLWn8PtwmLHxQfzqWyjUfSsjmr7FBeM59QB/RzgurtvMHfsq1avHJ1 G5lNyRV9HlSQhTOyi2rzoCFaeRHCDK8F4cwJeESOJxC98Xo8HlRzk8giW Nzp2eGNmX2aBXw3iH5JEa8QlWw11uA7WGP8r60PVE6cvgLVabirpevpzl G5wvBt2zHJyjSpcGYP5cSZB1178HZTaIxlPFbQ2zCVhqkxl7lO3TgIxcB sQaZMFOT1c8TKzesbBcF0t1jtyM9brMvRZbQb1uloONKBsBZL/MfJaWhc MVxoZSEXc2q8ktwrqIwNuSB59BpOxgrprc1i0DHTPcflaNWgH8fnuwoTE w==; X-CSE-ConnectionGUID: Iut/BNnCRaeqxqH+Gu7spg== X-CSE-MsgGUID: X2G1SwEQTrq8AMIgJJs+Ow== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490542" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:11 +0800 IronPort-SDR: Ju45+fAcGm3z7wBNcpujAQMTNjKtDbFoIy2SoopXKMwQeYLuNrXVwvAFgjP781R3CWotgHVoCA 1Y51Y2iTqQrbYsb+0NatG0DoxyASL67d5BaYIys6KTY3bfeR8nR9TJ9aMkJL+INt1ApI+8Wg4Z XUpLlvq2TtTbJutgrHXZMXkhiNycFgOUulMiglURmf7+DmEGN10sgVF147QYVGUnD8BsCVqDuu ZlFKWifWUv8VK3URfBB3+ZMfNXocKtEVgS/gX0sRAiT51knFJWuwMMqkW3XTPFiRWr8N+2o2m6 LEE= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:12 -0700 IronPort-SDR: 7KFfHeqB3pVLAGnO5MoHZqYGW8JZiY0jQCyzFiARJKPxWy4i7NlUqYnY8vFtDTNtnGms3TQKms Aa+BSNhVPBhNe7EpBfvBYTZ9Fstics5R5VzbKJgeu7IQspbpe6sTLT3D95NQJGoPqrbIhqp6Dv df352WZlI8mK3kxCvg8uNHWmr/BVRP3Bh4v9ymaIeBkmZv6mln+eJoiOVV3t0wwrLIHIJPyzk4 x9tGyA6UihyCTLFsS94O00HKxLVOQ1C8d5NOgVRJgGWFQF4b3xf50Ky7tcllRGUPS0FvVp0/sf rmY= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:10 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:00 -0700 Subject: [PATCH v9 05/11] btrfs: lookup physical address from stripe extent MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-5-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=7981; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=Yq+4Hu/kdBSP8VJdQwG78yDoI9jI2RynIMJjvJLIy94=; b=YMHja8RXU4rXaUWJY+fd8fQ3bW8/YarmpaFn2P4+kjg1CzAsGNBxT+tL3YZIqGZPBQf9Z2GAE 0VBaKGHYDRbDT38AizsnZGnWeW5IQRGMT2OcSfiQaU2P62emlpq4QJq X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Lookup the physical address from the raid stripe tree when a read on an RAID volume formatted with the raid stripe tree was attempted. Signed-off-by: Johannes Thumshirn --- fs/btrfs/raid-stripe-tree.c | 130 ++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/raid-stripe-tree.h | 11 ++++ fs/btrfs/volumes.c | 37 ++++++++++--- 3 files changed, 169 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c index 517bc08803f1..697a6e1fd255 100644 --- a/fs/btrfs/raid-stripe-tree.c +++ b/fs/btrfs/raid-stripe-tree.c @@ -303,3 +303,133 @@ int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans, return ret; } + +int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info, + u64 logical, u64 *length, u64 map_type, + u32 stripe_index, + struct btrfs_io_stripe *stripe) +{ + struct btrfs_root *stripe_root = fs_info->stripe_root; + struct btrfs_stripe_extent *stripe_extent; + struct btrfs_key stripe_key; + struct btrfs_key found_key; + struct btrfs_path *path; + struct extent_buffer *leaf; + const u64 end = logical + *length; + int num_stripes; + u8 encoding; + u64 offset; + u64 found_logical; + u64 found_length; + u64 found_end; + int slot; + int ret; + int i; + + stripe_key.objectid = logical; + stripe_key.type = BTRFS_RAID_STRIPE_KEY; + stripe_key.offset = 0; + + path = btrfs_alloc_path(); + if (!path) + return -ENOMEM; + + ret = btrfs_search_slot(NULL, stripe_root, &stripe_key, path, 0, 0); + if (ret < 0) + goto free_path; + if (ret) { + if (path->slots[0] != 0) + path->slots[0]--; + } + + + while (1) { + leaf = path->nodes[0]; + slot = path->slots[0]; + + btrfs_item_key_to_cpu(leaf, &found_key, slot); + found_logical = found_key.objectid; + found_length = found_key.offset; + found_end = found_logical + found_length; + + if (found_logical > end) { + ret = -ENOENT; + goto out; + } + + if (in_range(logical, found_logical, found_length)) + break; + + ret = btrfs_next_item(stripe_root, path); + if (ret) + goto out; + } + + offset = logical - found_logical; + + /* + * If we have a logically contiguous, but physically noncontinuous + * range, we need to split the bio. Record the length after which we + * must split the bio. + */ + if (end > found_end) + *length -= end - found_end; + + num_stripes = btrfs_num_raid_stripes(btrfs_item_size(leaf, slot)); + stripe_extent = btrfs_item_ptr(leaf, slot, struct btrfs_stripe_extent); + encoding = btrfs_stripe_extent_encoding(leaf, stripe_extent); + + if (encoding != btrfs_bg_flags_to_raid_index(map_type)) { + ret = -EUCLEAN; + btrfs_handle_fs_error(fs_info, ret, + "on-disk stripe encoding %d doesn't match RAID index %d", + encoding, + btrfs_bg_flags_to_raid_index(map_type)); + goto out; + } + + for (i = 0; i < num_stripes; i++) { + struct btrfs_raid_stride *stride = &stripe_extent->strides[i]; + u64 devid = btrfs_raid_stride_devid(leaf, stride); + u64 len = btrfs_raid_stride_length(leaf, stride); + u64 physical = btrfs_raid_stride_physical(leaf, stride); + + if (offset >= len) { + offset -= len; + + if (offset >= BTRFS_STRIPE_LEN) + continue; + } + + if (devid != stripe->dev->devid) + continue; + + if ((map_type & BTRFS_BLOCK_GROUP_DUP) && stripe_index != i) + continue; + + stripe->physical = physical + offset; + + ret = 0; + goto free_path; + } + + /* + * If we're here, we haven't found the requested devid in the stripe. + */ + ret = -ENOENT; +out: + if (ret > 0) + ret = -ENOENT; + if (ret && ret != -EIO) { + if (IS_ENABLED(CONFIG_BTRFS_DEBUG)) + btrfs_print_tree(leaf, 1); + btrfs_err(fs_info, + "cannot find raid-stripe for logical [%llu, %llu] devid %llu, profile %s", + logical, logical + *length, stripe->dev->devid, + btrfs_bg_type_to_raid_name(map_type)); + } +free_path: + btrfs_free_path(path); + + return ret; +} diff --git a/fs/btrfs/raid-stripe-tree.h b/fs/btrfs/raid-stripe-tree.h index b3a127c997c8..5d9629a815c1 100644 --- a/fs/btrfs/raid-stripe-tree.h +++ b/fs/btrfs/raid-stripe-tree.h @@ -13,6 +13,10 @@ struct btrfs_trans_handle; int btrfs_delete_raid_extent(struct btrfs_trans_handle *trans, u64 start, u64 length); +int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info, + u64 logical, u64 *length, u64 map_type, + u32 stripe_index, + struct btrfs_io_stripe *stripe); int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans, struct btrfs_ordered_extent *ordered_extent); @@ -33,4 +37,11 @@ static inline bool btrfs_need_stripe_tree_update(struct btrfs_fs_info *fs_info, return false; } + +static inline int btrfs_num_raid_stripes(u32 item_size) +{ + return (item_size - offsetof(struct btrfs_stripe_extent, strides)) / + sizeof(struct btrfs_raid_stride); +} + #endif diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index c2bac87912c7..2326dbcf85f6 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -35,6 +35,7 @@ #include "relocation.h" #include "scrub.h" #include "super.h" +#include "raid-stripe-tree.h" #define BTRFS_BLOCK_GROUP_STRIPE_MASK (BTRFS_BLOCK_GROUP_RAID0 | \ BTRFS_BLOCK_GROUP_RAID10 | \ @@ -6309,12 +6310,22 @@ static u64 btrfs_max_io_len(struct map_lookup *map, enum btrfs_map_op op, return U64_MAX; } -static void set_io_stripe(struct btrfs_io_stripe *dst, const struct map_lookup *map, - u32 stripe_index, u64 stripe_offset, u32 stripe_nr) +static int set_io_stripe(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, + u64 logical, u64 *length, struct btrfs_io_stripe *dst, + struct map_lookup *map, u32 stripe_index, + u64 stripe_offset, u64 stripe_nr) { dst->dev = map->stripes[stripe_index].dev; + + if (op == BTRFS_MAP_READ && + btrfs_need_stripe_tree_update(fs_info, map->type)) + return btrfs_get_raid_extent_offset(fs_info, logical, length, + map->type, stripe_index, + dst); + dst->physical = map->stripes[stripe_index].physical + stripe_offset + btrfs_stripe_nr_to_offset(stripe_nr); + return 0; } /* @@ -6531,11 +6542,11 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, */ if (smap && num_alloc_stripes == 1 && !((map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) && mirror_num > 1)) { - set_io_stripe(smap, map, stripe_index, stripe_offset, stripe_nr); + ret = set_io_stripe(fs_info, op, logical, length, smap, map, + stripe_index, stripe_offset, stripe_nr); if (mirror_num_ret) *mirror_num_ret = mirror_num; *bioc_ret = NULL; - ret = 0; goto out; } @@ -6566,21 +6577,29 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, bioc->full_stripe_logical = em->start + btrfs_stripe_nr_to_offset(stripe_nr * data_stripes); for (i = 0; i < num_stripes; i++) - set_io_stripe(&bioc->stripes[i], map, - (i + stripe_nr) % num_stripes, - stripe_offset, stripe_nr); + ret = set_io_stripe(fs_info, op, logical, length, + &bioc->stripes[i], map, + (i + stripe_nr) % num_stripes, + stripe_offset, stripe_nr); } else { /* * For all other non-RAID56 profiles, just copy the target * stripe into the bioc. */ for (i = 0; i < num_stripes; i++) { - set_io_stripe(&bioc->stripes[i], map, stripe_index, - stripe_offset, stripe_nr); + ret = set_io_stripe(fs_info, op, logical, length, + &bioc->stripes[i], map, stripe_index, + stripe_offset, stripe_nr); stripe_index++; } } + if (ret) { + *bioc_ret = NULL; + btrfs_put_bioc(bioc); + goto out; + } + if (op != BTRFS_MAP_READ) max_errors = btrfs_chunk_max_errors(map); From patchwork Thu Sep 14 16:07:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0AB1EEAA53 for ; Thu, 14 Sep 2023 16:07:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241670AbjINQHe (ORCPT ); Thu, 14 Sep 2023 12:07:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241581AbjINQHQ (ORCPT ); Thu, 14 Sep 2023 12:07:16 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B9061BDD; Thu, 14 Sep 2023 09:07:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707632; x=1726243632; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=QzXsAiN8EWNcEnSNzBlMi5nDNJGLElkV/zgsPabPAFk=; b=XgBoEToxR4enrl2IRlYv4XWVL1uMZU90IQ3GXQ0jJ+un9KpQxZdRtpWU 3n5cYdwp/FDvZnI+o6aNxcz5iutgjayRBPKEFBAwJ0Gf/QNeME/u/32Xh t1Yj/XSqvs9rBs+mv+tLtugmMkU+WvRzAtfgwBdALHohYlEAUX+Gc03h0 31TUt+286RLj5zpwzyoney6tl/J4BUs66t4xqzi1R/fDffHKlBY+/VKeZ goPaf85uHlQgwYZ+AcLaA3Gaqk5Fbvj2fowW/3xPf2115Hq6AA/EkYXsc B4XE6MJVyZaa3+iscQxe2hyEYbJjhSXl0vmrJDOoRX+urvwr/X6TdEjfW A==; X-CSE-ConnectionGUID: UUgP3R+eQ4eMq4oF2gywXg== X-CSE-MsgGUID: mGNLmd34SMWv39U4JO23YQ== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490543" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:11 +0800 IronPort-SDR: 32v1Uifsi9+m/eZCZmCDlM1oO3T2N2V3wFI160VNgGt6o2A7ik0n7CLv9qtcSCJJubIB/K1rIB l7EA1ac/mFyHaMlMy/RPPLJNreM9ktjvKyK5tBeV5IS29gPLFsL/oIwm9BVpH5aLZLPjPlQpd+ kqaCvT8KU37Oage8ZEvB8oJw1aXs1jm28sBdD3K28XR/SvUZi426bCkq27vONNC20lJpqx8WGZ xufNnlUoaefCegD+ybiVRVQ5X4wfnLQ9aj0WZqdTgIcnp6v43PMA4EUYHMKT1+Nq1Adnm6iIem rlQ= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:13 -0700 IronPort-SDR: CHBOVx2lN4ewmH41rtQPbi1A1xy2WUYm04VkovTYO5ROAiNoknymasgP0RaizDozipFIC9HO1j Crg/AndAC/n+GRpvfuC5PKIpfXsaODoqcsb5+ON0TCPoyQl59a6N8tzbkLH15cZnPjnji5679V itPaEtCLDW+9j8eWxhFQBwglFtFSRACW7Zpw6oOZrsK1RudwQJgIVzMZVzqJlpq/7+E3dyxmwV PFQuddyKQaG0zxXmt99ois7sAPcUWGHLpdYLA56eYbuSy29EKMyAoZA8FMi0mlFkHu1OS9Z4oN mos= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:11 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:01 -0700 Subject: [PATCH v9 06/11] btrfs: implement RST version of scrub MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-6-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=4850; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=QzXsAiN8EWNcEnSNzBlMi5nDNJGLElkV/zgsPabPAFk=; b=SzYPDugH1Fsgd4op95knUECcvGBvlmxEKaO7005yZ3K/1ol/AAvJFtBTYnhsXCTlCQV5Rsr3q nKrMxARCMDaA0k1FDmGI6uTeu+Fqxz2S087vKtyPeX83DeOv84hNZpX X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org A filesystem that uses the RAID stripe tree for logical to physical address translation can't use the regular scrub path, that reads all stripes and then checks if a sector is unused afterwards. When using the RAID stripe tree, this will result in lookup errors, as the stripe tree doesn't know the requested logical addresses. Instead, look up stripes that are backed by the extent bitmap. Signed-off-by: Johannes Thumshirn --- fs/btrfs/bio.c | 2 ++ fs/btrfs/raid-stripe-tree.c | 8 ++++++- fs/btrfs/scrub.c | 53 +++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/volumes.h | 1 + 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/bio.c b/fs/btrfs/bio.c index ddbe6f8d4ea2..bdb6e3effdbb 100644 --- a/fs/btrfs/bio.c +++ b/fs/btrfs/bio.c @@ -663,6 +663,8 @@ static bool btrfs_submit_chunk(struct btrfs_bio *bbio, int mirror_num) blk_status_t ret; int error; + smap.is_scrub = !bbio->inode; + btrfs_bio_counter_inc_blocked(fs_info); error = btrfs_map_block(fs_info, btrfs_op(bio), logical, &map_length, &bioc, &smap, &mirror_num, 1); diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c index 697a6e1fd255..63bf62c33436 100644 --- a/fs/btrfs/raid-stripe-tree.c +++ b/fs/btrfs/raid-stripe-tree.c @@ -334,6 +334,11 @@ int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info, if (!path) return -ENOMEM; + if (stripe->is_scrub) { + path->skip_locking = 1; + path->search_commit_root = 1; + } + ret = btrfs_search_slot(NULL, stripe_root, &stripe_key, path, 0, 0); if (ret < 0) goto free_path; @@ -420,7 +425,8 @@ int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info, out: if (ret > 0) ret = -ENOENT; - if (ret && ret != -EIO) { + if (ret && ret != -EIO && !stripe->is_scrub) { + if (IS_ENABLED(CONFIG_BTRFS_DEBUG)) btrfs_print_tree(leaf, 1); btrfs_err(fs_info, diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index f16220ce5fba..42948b66d4be 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -23,6 +23,7 @@ #include "accessors.h" #include "file-item.h" #include "scrub.h" +#include "raid-stripe-tree.h" /* * This is only the first step towards a full-features scrub. It reads all @@ -1634,6 +1635,53 @@ static void scrub_reset_stripe(struct scrub_stripe *stripe) } } +static void scrub_submit_extent_sector_read(struct scrub_ctx *sctx, + struct scrub_stripe *stripe) +{ + struct btrfs_fs_info *fs_info = stripe->bg->fs_info; + struct btrfs_bio *bbio = NULL; + int mirror = stripe->mirror_num; + int i; + + atomic_inc(&stripe->pending_io); + + for_each_set_bit(i, &stripe->extent_sector_bitmap, stripe->nr_sectors) { + struct page *page = scrub_stripe_get_page(stripe, i); + unsigned int pgoff = scrub_stripe_get_page_offset(stripe, i); + + /* The current sector cannot be merged, submit the bio. */ + if (bbio && + ((i > 0 && !test_bit(i - 1, &stripe->extent_sector_bitmap)) || + bbio->bio.bi_iter.bi_size >= BTRFS_STRIPE_LEN)) { + ASSERT(bbio->bio.bi_iter.bi_size); + atomic_inc(&stripe->pending_io); + btrfs_submit_bio(bbio, mirror); + bbio = NULL; + } + + if (!bbio) { + bbio = btrfs_bio_alloc(stripe->nr_sectors, REQ_OP_READ, + fs_info, scrub_read_endio, stripe); + bbio->bio.bi_iter.bi_sector = (stripe->logical + + (i << fs_info->sectorsize_bits)) >> SECTOR_SHIFT; + } + + __bio_add_page(&bbio->bio, page, fs_info->sectorsize, pgoff); + } + + if (bbio) { + ASSERT(bbio->bio.bi_iter.bi_size); + atomic_inc(&stripe->pending_io); + btrfs_submit_bio(bbio, mirror); + } + + if (atomic_dec_and_test(&stripe->pending_io)) { + wake_up(&stripe->io_wait); + INIT_WORK(&stripe->work, scrub_stripe_read_repair_worker); + queue_work(stripe->bg->fs_info->scrub_workers, &stripe->work); + } +} + static void scrub_submit_initial_read(struct scrub_ctx *sctx, struct scrub_stripe *stripe) { @@ -1645,6 +1693,11 @@ static void scrub_submit_initial_read(struct scrub_ctx *sctx, ASSERT(stripe->mirror_num > 0); ASSERT(test_bit(SCRUB_STRIPE_FLAG_INITIALIZED, &stripe->state)); + if (btrfs_need_stripe_tree_update(fs_info, stripe->bg->flags)) { + scrub_submit_extent_sector_read(sctx, stripe); + return; + } + bbio = btrfs_bio_alloc(SCRUB_STRIPE_PAGES, REQ_OP_READ, fs_info, scrub_read_endio, stripe); diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 2043aff6e966..067859de8f4c 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -393,6 +393,7 @@ struct btrfs_io_stripe { /* Block mapping */ u64 physical; u64 length; + bool is_scrub; /* For the endio handler */ struct btrfs_io_context *bioc; }; From patchwork Thu Sep 14 16:07:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBC81EEAA56 for ; Thu, 14 Sep 2023 16:07:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241925AbjINQHg (ORCPT ); Thu, 14 Sep 2023 12:07:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241842AbjINQHR (ORCPT ); Thu, 14 Sep 2023 12:07:17 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF0C71FC0; Thu, 14 Sep 2023 09:07:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707633; x=1726243633; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=d42cqIbxUEKquSGYYk3w7AaeNWr6XkbxX7jX+1lrcSo=; b=jmbVSmtJtHnOCE8OJ/jJIZIpeQA1y7J+rmnkzVDJDkR2ul4vkMx8p1bj +gVqr4fFXnA3WmfhHYdrEkHPXTX2XP8STblejG1TcS0QWOtt17nYkU0s8 jaIJEPLom/MzPj86iZySvkzl47/E55U6io2UHKbmdANnbMq+YgFkPcjYx skVqXVfN+p1vv4LlF2ohegDCRS3k93PohFV520pzsw+e8E/lQd46RJQJk Xt21rdB5LhTdViJflWoU/scU2cOQW8MP5B9AL16f03u5UyMnJqly2y9Xg VsBEz02gJEgQP4sm5QDR5L5QBEOyAWZteMcM98nq+vOaj+E/4FFOdXPOv A==; X-CSE-ConnectionGUID: 0uOTb0vlSEKK3KzsROHjqQ== X-CSE-MsgGUID: A/zWuJGlQEKHl+e3YSgVqA== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490548" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:13 +0800 IronPort-SDR: tE8UV3isZTCeH8JBePxSF48I6xApbZ1v5lROxiXoV2WOJP/aiaWfa5CqJfPIyziyZ3el4PE1Ay 7ugOjKT2wfWKIuO9liiCJgKyAMheujaGRsGDVCLqWUNigyNB9Lsxm3wzXPFWuuCMvJHxfHW5Am tX4FY+CCW5DDNbEgT1yFiUpJEZLIpJ6srA6AQAuVG6YVwdfqBAJNpwUHZ2KPsukKS34wwCJbLY pEw+Y8pqDq3MQKDYZi6Ij/S0F+Fcs/m4oUX0srj1zKm8E/nkvq1tnG4MTFkFe1lSdKhT8hk4Y6 vyA= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:14 -0700 IronPort-SDR: +CvEQs5BsLVIGqrJTcYE4ynxuB6mAsOW32+/ktK2G+qnaqNoOsaHOCA1meZtH54DhD6PD7ua2y LxE80qHHk9E5wJ9udvAndm7GJibMTl3flaUMJhhG5GzI0udbWToFcw7wLpXuuVmmf6B7Ims6Cc qqr+t3O0YLJGBAfXchgnhc7O1VYxvKSFqBeL+axmn5NdQwNuGajdUtVgn3CF4TbYM9viKO0adv fvcoUk4RnNMdH5T8Qt7HR/jkbtyTa/e8IEbyda12y71VCsXkcN9WYxfPF1cDW9jUWON9FrAS2K 8oQ= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:13 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:02 -0700 Subject: [PATCH v9 07/11] btrfs: zoned: allow zoned RAID MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-7-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=7309; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=d42cqIbxUEKquSGYYk3w7AaeNWr6XkbxX7jX+1lrcSo=; b=4/dBOqQnXE+SB9BAc5TTUm8Hj74OPgfEK2eZ0jSSAYeQEGSnsoYpo5DiQkBY1JNdG6XLmyNsX xALtVkIJA/PCryIF+haERYDDbxiG6ErT2y5g3+Zd4xoSRQBYR+fVcKd X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When we have a raid-stripe-tree, we can do RAID0/1/10 on zoned devices for data block-groups. For meta-data block-groups, we don't actually need anything special, as all meta-data I/O is protected by the btrfs_zoned_meta_io_lock() already. Signed-off-by: Johannes Thumshirn --- fs/btrfs/raid-stripe-tree.h | 7 ++- fs/btrfs/volumes.c | 2 + fs/btrfs/zoned.c | 144 ++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 148 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/raid-stripe-tree.h b/fs/btrfs/raid-stripe-tree.h index 5d9629a815c1..f31292ab9030 100644 --- a/fs/btrfs/raid-stripe-tree.h +++ b/fs/btrfs/raid-stripe-tree.h @@ -6,6 +6,11 @@ #ifndef BTRFS_RAID_STRIPE_TREE_H #define BTRFS_RAID_STRIPE_TREE_H +#define BTRFS_RST_SUPP_BLOCK_GROUP_MASK (BTRFS_BLOCK_GROUP_DUP |\ + BTRFS_BLOCK_GROUP_RAID1_MASK |\ + BTRFS_BLOCK_GROUP_RAID0 |\ + BTRFS_BLOCK_GROUP_RAID10) + struct btrfs_io_context; struct btrfs_io_stripe; struct btrfs_ordered_extent; @@ -32,7 +37,7 @@ static inline bool btrfs_need_stripe_tree_update(struct btrfs_fs_info *fs_info, if (type != BTRFS_BLOCK_GROUP_DATA) return false; - if (profile & BTRFS_BLOCK_GROUP_RAID1_MASK) + if (profile & BTRFS_RST_SUPP_BLOCK_GROUP_MASK) return true; return false; diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 2326dbcf85f6..dc311e38eb11 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -6541,6 +6541,8 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, * I/O context structure. */ if (smap && num_alloc_stripes == 1 && + !(btrfs_need_stripe_tree_update(fs_info, map->type) && + op != BTRFS_MAP_READ) && !((map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) && mirror_num > 1)) { ret = set_io_stripe(fs_info, op, logical, length, smap, map, stripe_index, stripe_offset, stripe_nr); diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c index d05510cb2cb2..ce2846c944d2 100644 --- a/fs/btrfs/zoned.c +++ b/fs/btrfs/zoned.c @@ -1397,9 +1397,11 @@ static int btrfs_load_block_group_dup(struct btrfs_block_group *bg, struct zone_info *zone_info, unsigned long *active) { - if (map->type & BTRFS_BLOCK_GROUP_DATA) { - btrfs_err(bg->fs_info, - "zoned: profile DUP not yet supported on data bg"); + struct btrfs_fs_info *fs_info = bg->fs_info; + + if (map->type & BTRFS_BLOCK_GROUP_DATA && + !fs_info->stripe_root) { + btrfs_err(fs_info, "zoned: data DUP profile needs stripe_root"); return -EINVAL; } @@ -1433,6 +1435,133 @@ static int btrfs_load_block_group_dup(struct btrfs_block_group *bg, return 0; } +static int btrfs_load_block_group_raid1(struct btrfs_block_group *bg, + struct map_lookup *map, + struct zone_info *zone_info, + unsigned long *active) +{ + struct btrfs_fs_info *fs_info = bg->fs_info; + int i; + + if (map->type & BTRFS_BLOCK_GROUP_DATA && + !fs_info->stripe_root) { + btrfs_err(fs_info, "zoned: data %s needs stripe_root", + btrfs_bg_type_to_raid_name(map->type)); + return -EINVAL; + + } + + for (i = 0; i < map->num_stripes; i++) { + if (zone_info[i].alloc_offset == WP_MISSING_DEV || + zone_info[i].alloc_offset == WP_CONVENTIONAL) + continue; + + if ((zone_info[0].alloc_offset != zone_info[i].alloc_offset) && + !btrfs_test_opt(fs_info, DEGRADED)) { + btrfs_err(fs_info, + "zoned: write pointer offset mismatch of zones in %s profile", + btrfs_bg_type_to_raid_name(map->type)); + return -EIO; + } + if (test_bit(0, active) != test_bit(i, active)) { + if (!btrfs_test_opt(fs_info, DEGRADED) && + !btrfs_zone_activate(bg)) { + return -EIO; + } + } else { + if (test_bit(0, active)) + set_bit(BLOCK_GROUP_FLAG_ZONE_IS_ACTIVE, + &bg->runtime_flags); + } + /* + * In case a device is missing we have a cap of 0, so don't + * use it. + */ + bg->zone_capacity = min_not_zero(zone_info[0].capacity, + zone_info[1].capacity); + } + + if (zone_info[0].alloc_offset != WP_MISSING_DEV) + bg->alloc_offset = zone_info[0].alloc_offset; + else + bg->alloc_offset = zone_info[i - 1].alloc_offset; + + return 0; +} + +static int btrfs_load_block_group_raid0(struct btrfs_block_group *bg, + struct map_lookup *map, + struct zone_info *zone_info, + unsigned long *active) +{ + struct btrfs_fs_info *fs_info = bg->fs_info; + + if (map->type & BTRFS_BLOCK_GROUP_DATA && + !fs_info->stripe_root) { + btrfs_err(fs_info, "zoned: data %s needs stripe_root", + btrfs_bg_type_to_raid_name(map->type)); + return -EINVAL; + + } + + for (int i = 0; i < map->num_stripes; i++) { + if (zone_info[i].alloc_offset == WP_MISSING_DEV || + zone_info[i].alloc_offset == WP_CONVENTIONAL) + continue; + + if (test_bit(0, active) != test_bit(i, active)) { + if (!btrfs_zone_activate(bg)) + return -EIO; + } else { + if (test_bit(0, active)) + set_bit(BLOCK_GROUP_FLAG_ZONE_IS_ACTIVE, + &bg->runtime_flags); + } + bg->zone_capacity += zone_info[i].capacity; + bg->alloc_offset += zone_info[i].alloc_offset; + } + + return 0; +} + +static int btrfs_load_block_group_raid10(struct btrfs_block_group *bg, + struct map_lookup *map, + struct zone_info *zone_info, + unsigned long *active) +{ + struct btrfs_fs_info *fs_info = bg->fs_info; + + if (map->type & BTRFS_BLOCK_GROUP_DATA && + !fs_info->stripe_root) { + btrfs_err(fs_info, "zoned: data %s needs stripe_root", + btrfs_bg_type_to_raid_name(map->type)); + return -EINVAL; + + } + + for (int i = 0; i < map->num_stripes; i++) { + if (zone_info[i].alloc_offset == WP_MISSING_DEV || + zone_info[i].alloc_offset == WP_CONVENTIONAL) + continue; + + if (test_bit(0, active) != test_bit(i, active)) { + if (!btrfs_zone_activate(bg)) + return -EIO; + } else { + if (test_bit(0, active)) + set_bit(BLOCK_GROUP_FLAG_ZONE_IS_ACTIVE, + &bg->runtime_flags); + } + + if ((i % map->sub_stripes) == 0) { + bg->zone_capacity += zone_info[i].capacity; + bg->alloc_offset += zone_info[i].alloc_offset; + } + } + + return 0; +} + int btrfs_load_block_group_zone_info(struct btrfs_block_group *cache, bool new) { struct btrfs_fs_info *fs_info = cache->fs_info; @@ -1525,11 +1654,18 @@ int btrfs_load_block_group_zone_info(struct btrfs_block_group *cache, bool new) ret = btrfs_load_block_group_dup(cache, map, zone_info, active); break; case BTRFS_BLOCK_GROUP_RAID1: + case BTRFS_BLOCK_GROUP_RAID1C3: + case BTRFS_BLOCK_GROUP_RAID1C4: + ret = btrfs_load_block_group_raid1(cache, map, zone_info, active); + break; case BTRFS_BLOCK_GROUP_RAID0: + ret = btrfs_load_block_group_raid0(cache, map, zone_info, active); + break; case BTRFS_BLOCK_GROUP_RAID10: + ret = btrfs_load_block_group_raid10(cache, map, zone_info, active); + break; case BTRFS_BLOCK_GROUP_RAID5: case BTRFS_BLOCK_GROUP_RAID6: - /* non-single profiles are not supported yet */ default: btrfs_err(fs_info, "zoned: profile %s not yet supported", btrfs_bg_type_to_raid_name(map->type)); From patchwork Thu Sep 14 16:07:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E079EEAA53 for ; Thu, 14 Sep 2023 16:07:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241933AbjINQHh (ORCPT ); Thu, 14 Sep 2023 12:07:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241822AbjINQHT (ORCPT ); Thu, 14 Sep 2023 12:07:19 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 004D01FE3; Thu, 14 Sep 2023 09:07:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707634; x=1726243634; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=jHNTpGHD7Us3Cugii4qHZugLDNfI6sOo7DVvwa2B5jU=; b=krnD1OJKD1oc1lU+v0jK8YwnpT0KWECS/FC/mOFWEbCqIBjImch42QH5 aedrUBuLI0OBQB+QdzQ8Lbscj+VF/WTTbw50cHjBNqa8asuQWeRaL8ukj yS1WR6G2fAMtSt8MvI8keodGYWhkUGLyEA2wqLlnJVM4NzJ2/9zqI4M8n 3xi0RD0eJ3ZBvmNGiOXAP+hf5niveHbzsRkPBO43yorTkpuCoFA8QHtil jtONdGaelRZ+SwF0e2lTgQ0zONHJi0KfevfeWku5bCHfuA6E1g3fpyhEE opbx6tZS4vjUv1FbZBCVxnDDzosmxYAxr6hszLkwiAoTdfWWX3YUPmR4l w==; X-CSE-ConnectionGUID: AWVtofuaRzmNv1bRWUvpVQ== X-CSE-MsgGUID: /kI59yT5RpmwY20c+rFdGQ== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490550" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:14 +0800 IronPort-SDR: Y5rtLsaA8OFY2MiVaBk7s2cdV+Vtpr8+trUm2/1pv2NAYV55iaB/uT11b17yz2R6+OI4NiYAyv g5aSa8H5t2sTgcTnLqRG6eIV2e8NGOgk0uRrcb7k8RZ+9FVgQq/f8gYtMONgL+voxM2TILDjQJ JwEWAy1gEa1jshuXJw5IWn+UnNIw/EijO8KMNrkKqQP4O3jfNA+XHiKgBn5e7kiLrI9UHCCLrI Rn15+91X/oCMu8pFgZfwgSBMI/f7ZLzvwj1MlvFfMg0uLnyJyY2994OxA7BBhsmPTf3FRROji3 1Ks= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:16 -0700 IronPort-SDR: PFmf0p6E6VfOBTvkuxvfAhTf5QNWbwqsBldqBAhQZ6xtPyAObXFwAay1ZJKzEnYxf0sHavSYZY wniyfwDn2gk7pODMMRfeUxDpsrORaMLLAEiZkGEaXdSThkgW37Jyh9pqcHWotHenmfhvpAj1Ir c1i3bdYwIPgwYT+stgbsN4yK34YhJPLJXtIrwZrZwxFpuszqobPWIz1lhUxvMvTRVo6TrEXMem FTSDy1/7ImM2moMjVcs5Ljv1QIy0R47KEP1rAwna7VZJpvhTC/JU5LczJNi6mZ70fmMWxxOqd/ 32M= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:14 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:03 -0700 Subject: [PATCH v9 08/11] btrfs: add raid stripe tree pretty printer MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-8-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=2269; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=jHNTpGHD7Us3Cugii4qHZugLDNfI6sOo7DVvwa2B5jU=; b=kVcUV/kcn+5SUTEkumOm+TYwd5JbpcYGm96YEMSItELyx4qG0FUvausBCjeugt6Fny3e22rpL Go0jdc/fEp+D7lkt0CBgLNRRp7HoIhvuKIjMZ8+lY38wmou9b3aQ4sk X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Decode raid-stripe-tree entries on btrfs_print_tree(). Signed-off-by: Johannes Thumshirn --- fs/btrfs/print-tree.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/fs/btrfs/print-tree.c b/fs/btrfs/print-tree.c index 0c93439e929f..f3f487fc6400 100644 --- a/fs/btrfs/print-tree.c +++ b/fs/btrfs/print-tree.c @@ -9,6 +9,8 @@ #include "print-tree.h" #include "accessors.h" #include "tree-checker.h" +#include "volumes.h" +#include "raid-stripe-tree.h" struct root_name_map { u64 id; @@ -28,6 +30,7 @@ static const struct root_name_map root_map[] = { { BTRFS_FREE_SPACE_TREE_OBJECTID, "FREE_SPACE_TREE" }, { BTRFS_BLOCK_GROUP_TREE_OBJECTID, "BLOCK_GROUP_TREE" }, { BTRFS_DATA_RELOC_TREE_OBJECTID, "DATA_RELOC_TREE" }, + { BTRFS_RAID_STRIPE_TREE_OBJECTID, "RAID_STRIPE_TREE" }, }; const char *btrfs_root_name(const struct btrfs_key *key, char *buf) @@ -189,6 +192,24 @@ static void print_uuid_item(const struct extent_buffer *l, unsigned long offset, } } +static void print_raid_stripe_key(const struct extent_buffer *eb, u32 item_size, + struct btrfs_stripe_extent *stripe) +{ + const int num_stripes = btrfs_num_raid_stripes(item_size); + const u8 encoding = btrfs_stripe_extent_encoding(eb, stripe); + int i; + + pr_info("\t\t\tencoding: %s\n", + (encoding && encoding < BTRFS_NR_RAID_TYPES) ? + btrfs_raid_array[encoding].raid_name : "unknown"); + + for (i = 0; i < num_stripes; i++) + pr_info("\t\t\tstride %d devid %llu physical %llu length %llu\n", + i, btrfs_raid_stride_devid(eb, &stripe->strides[i]), + btrfs_raid_stride_physical(eb, &stripe->strides[i]), + btrfs_raid_stride_length(eb, &stripe->strides[i])); +} + /* * Helper to output refs and locking status of extent buffer. Useful to debug * race condition related problems. @@ -349,6 +370,11 @@ void btrfs_print_leaf(const struct extent_buffer *l) print_uuid_item(l, btrfs_item_ptr_offset(l, i), btrfs_item_size(l, i)); break; + case BTRFS_RAID_STRIPE_KEY: + print_raid_stripe_key(l, btrfs_item_size(l, i), + btrfs_item_ptr(l, i, + struct btrfs_stripe_extent)); + break; } } } From patchwork Thu Sep 14 16:07:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2195EEAA52 for ; Thu, 14 Sep 2023 16:07:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241794AbjINQHj (ORCPT ); Thu, 14 Sep 2023 12:07:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241852AbjINQHU (ORCPT ); Thu, 14 Sep 2023 12:07:20 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58CEF1FFC; Thu, 14 Sep 2023 09:07:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707636; x=1726243636; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=D+BzdF5oP8uW6qZ5ynzhWFTIk+tOOC9drG8BemLcBr8=; b=FkmCiYs7ZQL5tJuTDHq3ncCQrXPjjzu6MG+ixXAl98jgq1sndvX6S/ua WB4vxT7q5vWi++fG+iaBQhK44tESYw6Hq/FuSO69cFin5AzQFGEUWcljh +xDehtMmpUAXEIQry1ck7Zxfb4uaclQswkFnzxuQxCOu5c9OM6Jkrxf1m Pf2/+xCQ5tKzYRVSQzmHFrUsTE3ZuFhZmQ4g6f2bGZ0/E3Tsc7+vEqOGe 7C8hv6ECS11XLWKGzTeQAQ3ezeVqWGQolNnf8tt2SCZzaXRKvh68/lD+Y 2VG5N0UMLmKwYSDtZ5iOAQlLHOMtl7dl8cJH9MD//EqZvEMqq20uZNts6 g==; X-CSE-ConnectionGUID: rJBrTymTTcaM/UiAnmdt/g== X-CSE-MsgGUID: ESow74hIRhuMDGqnZl3+Zw== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490553" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:16 +0800 IronPort-SDR: 5G4AWbt3pQNhr4i2mGZ3W7mXiYM8uofLbDKCL/jS5YUFsCcGadLKBD5s06W8wQ3p9fn44pGidE R/TUL4XCyOVvYM/ZkdNt5amK9Y6W6gZH8hK0jTRwPqQ0xPJ+0UcH6aPmrNzkO6geUPedKXbdgQ fFOEx6R2S53kBV8ZevpgrCTNi/WpO6o8kM0Qb0AcKI3rdcSbSCxADkhu3Dlv/ZNzFAKTD+uF8l Jjzk7yYYfzjHucynoAtMoHa7CZ7y2pyF10owjMIbqTeSLlmAotHTr0rf3qCqcXYSgrVeb9focB T30= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:17 -0700 IronPort-SDR: N1WXyOUfqrjmicfm0H5/OJBvzXOsz3EdCOpOOwZfZdLO8njeyNysLUf/CO+TaInLtZ7wIup3d6 R0qxzOVMak2IWRvpWIDmRnVBUrGCDqUoW9b8sgY+GjE6Sunny/8gU6V7WhwaYAlHBOToBYu79+ wVQbk9mVxJPo4WuQwpy4sSjKZvR1lQRHNzfrBu86TYBsaWQJdOcA6afBV/WUDl1JWpo28Z+tR5 ihEOI7Hy87cAMY+CFi2a0J4UJ9pWZ80Pa1f18i27BNYHCIBKpa1wLu0yv3rHjih+xUn3lzrTPD UMU= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:16 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:04 -0700 Subject: [PATCH v9 09/11] btrfs: announce presence of raid-stripe-tree in sysfs MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-9-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=1119; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=D+BzdF5oP8uW6qZ5ynzhWFTIk+tOOC9drG8BemLcBr8=; b=YPAqYzFyF2l2Wc3Vu/AtB90BNmJELAGsgqlNfc/E/C220KQQTvthliF/KHwDAwFykdjPPV9k2 4RMCO4lcraaCCJzt1hTOc/DBEKeof2MdvAD1+3NO9fy6eD/CicHlnFm X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org If a filesystem with a raid-stripe-tree is mounted, show the RST feature in sysfs. Reviewed-by: Josef Bacik Signed-off-by: Johannes Thumshirn --- fs/btrfs/sysfs.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index b1d1ac25237b..1bab3d7d251e 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs/sysfs.c @@ -297,6 +297,8 @@ BTRFS_FEAT_ATTR_INCOMPAT(zoned, ZONED); #ifdef CONFIG_BTRFS_DEBUG /* Remove once support for extent tree v2 is feature complete */ BTRFS_FEAT_ATTR_INCOMPAT(extent_tree_v2, EXTENT_TREE_V2); +/* Remove once support for raid stripe tree is feature complete */ +BTRFS_FEAT_ATTR_INCOMPAT(raid_stripe_tree, RAID_STRIPE_TREE); #endif #ifdef CONFIG_FS_VERITY BTRFS_FEAT_ATTR_COMPAT_RO(verity, VERITY); @@ -327,6 +329,7 @@ static struct attribute *btrfs_supported_feature_attrs[] = { #endif #ifdef CONFIG_BTRFS_DEBUG BTRFS_FEAT_ATTR_PTR(extent_tree_v2), + BTRFS_FEAT_ATTR_PTR(raid_stripe_tree), #endif #ifdef CONFIG_FS_VERITY BTRFS_FEAT_ATTR_PTR(verity), From patchwork Thu Sep 14 16:07:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 922A0EEAA55 for ; Thu, 14 Sep 2023 16:07:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241777AbjINQHk (ORCPT ); Thu, 14 Sep 2023 12:07:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241860AbjINQHV (ORCPT ); Thu, 14 Sep 2023 12:07:21 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBBD91FFD; Thu, 14 Sep 2023 09:07:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707637; x=1726243637; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=dAFGj2lSFggsGCAxlAi8drVBv+UlMJSnIu/xYz5RsQY=; b=OlSncPK6IwuAJqqRusaKBmItI6MEK89PelvbS+D1HXVBKi7vglXjNtmP XFg7YZY91x73fUZKu1yJGyeROOkHeXlg3rmPpPyuTdxN8SlLQGIHnCe38 xcuNVFF4stK19Empq1YTxp1/8C5x59Bw30n3H9k6CBAYfGwo23kxnP/aO p756NthR+7BSrQTaDbb6yw4MtP3oCyw2eYbr5f+eCo/uc6Nxpxy2E3L/P v5zFROTdAorS7i0Z0soGK8q+VFsSRNlWHkEZBfljpquPd+gJNI23vmhmj oKrbg7dn7SwbpKIu49ffmjte5y8tWa1ehmSREyy7NnsvqrXfX5CT3ynnJ w==; X-CSE-ConnectionGUID: FmfcZvvmSD+76dB2jk3zew== X-CSE-MsgGUID: L71ry+ZZQ0KW29JWL0M17A== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490559" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:17 +0800 IronPort-SDR: 3AsZ+kunBFYzKfyu3zzBus9gbMJl9wU5cEdEkTrcVLEY45OfxnyjEPTk3Q26ni/op+s05fuG9k vIEEeB5ybaPqLwV7MZfyHVcz3UFrTS3C0RQ9r7zw0HTlfgOHg2UOZ48IUJKk2tJa24/x9Zayya tawBJAx8o1oGnf0vjywYw90M8gg769KjbyPGlOcslpoYPdQB58GhFFvOFXVb8hYAUirrxUj0Te IOHxkHSGZYK5w4M7yEVzsZJqe/27EAvhaZA5sdaVw+gLkeMu6SuJKNJBO5wVs6YJU7Ckvqydqd iTc= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:18 -0700 IronPort-SDR: gqmI05Gt11sOtVS4hVMypzk0BmiyPmIS1c2fqlI5Lk7a8exO5LUHbaHpe4QrcKi0IkPv8t1uXI wR7sUdWa+YrGOTKq9l0/ujNJtIDjdc50iDa2I88VqAOVQ6BPuXX9+7f7o+yHD4mcOQvg1OArmg 5wLuqlCOW9Uo7VKPejqiaiK/2yNBuia3lqKwQ3Mt5Cj3+8AF38ZDhU73cHERkAQgdlQ8uYHe9c fWiy9nLmmmiMyhhf4zscuZHQGhgFbPvG+igbrRkMbBi27pRTqzVwWPt7TgI+U9PTef1e4QkvLr BiQ= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:17 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:05 -0700 Subject: [PATCH v9 10/11] btrfs: add trace events for RST MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-10-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=3855; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=dAFGj2lSFggsGCAxlAi8drVBv+UlMJSnIu/xYz5RsQY=; b=co+EasJda8IZv4EJ9EvHsTRizSP9zKL9+eZ6w2RoigsWuNiOAR9LVIw2Q4R8ZZC/iqO0y3r/c 1Ke0AyETENDD10rDl5cPeiBaPq5ayEuBcnIq2SDXmcZjB4vf8iJ60uu X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add trace events for raid-stripe-tree operations. Signed-off-by: Johannes Thumshirn --- fs/btrfs/raid-stripe-tree.c | 8 +++++ include/trace/events/btrfs.h | 75 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c index 63bf62c33436..ee4155377bf9 100644 --- a/fs/btrfs/raid-stripe-tree.c +++ b/fs/btrfs/raid-stripe-tree.c @@ -62,6 +62,9 @@ int btrfs_delete_raid_extent(struct btrfs_trans_handle *trans, u64 start, if (found_end <= start) break; + trace_btrfs_raid_extent_delete(fs_info, start, end, + found_start, found_end); + ASSERT(found_start >= start && found_end <= end); ret = btrfs_del_item(trans, stripe_root, path); if (ret) @@ -94,6 +97,8 @@ static int btrfs_insert_one_raid_extent(struct btrfs_trans_handle *trans, return -ENOMEM; } + trace_btrfs_insert_one_raid_extent(fs_info, bioc->logical, bioc->size, + num_stripes); btrfs_set_stack_stripe_extent_encoding(stripe_extent, encoding); for (int i = 0; i < num_stripes; i++) { u64 devid = bioc->stripes[i].dev->devid; @@ -414,6 +419,9 @@ int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info, stripe->physical = physical + offset; + trace_btrfs_get_raid_extent_offset(fs_info, logical, *length, + stripe->physical, devid); + ret = 0; goto free_path; } diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index b2db2c2f1c57..fcf246f84547 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -2497,6 +2497,81 @@ DEFINE_EVENT(btrfs_raid56_bio, raid56_write, TP_ARGS(rbio, bio, trace_info) ); +TRACE_EVENT(btrfs_insert_one_raid_extent, + + TP_PROTO(const struct btrfs_fs_info *fs_info, u64 logical, u64 length, + int num_stripes), + + TP_ARGS(fs_info, logical, length, num_stripes), + + TP_STRUCT__entry_btrfs( + __field( u64, logical ) + __field( u64, length ) + __field( int, num_stripes ) + ), + + TP_fast_assign_btrfs(fs_info, + __entry->logical = logical; + __entry->length = length; + __entry->num_stripes = num_stripes; + ), + + TP_printk_btrfs("logical=%llu length=%llu num_stripes=%d", + __entry->logical, __entry->length, + __entry->num_stripes) +); + +TRACE_EVENT(btrfs_raid_extent_delete, + + TP_PROTO(const struct btrfs_fs_info *fs_info, u64 start, u64 end, + u64 found_start, u64 found_end), + + TP_ARGS(fs_info, start, end, found_start, found_end), + + TP_STRUCT__entry_btrfs( + __field( u64, start ) + __field( u64, end ) + __field( u64, found_start ) + __field( u64, found_end ) + ), + + TP_fast_assign_btrfs(fs_info, + __entry->start = start; + __entry->end = end; + __entry->found_start = found_start; + __entry->found_end = found_end; + ), + + TP_printk_btrfs("start=%llu end=%llu found_start=%llu found_end=%llu", + __entry->start, __entry->end, __entry->found_start, + __entry->found_end) +); + +TRACE_EVENT(btrfs_get_raid_extent_offset, + + TP_PROTO(const struct btrfs_fs_info *fs_info, u64 logical, u64 length, + u64 physical, u64 devid), + + TP_ARGS(fs_info, logical, length, physical, devid), + + TP_STRUCT__entry_btrfs( + __field( u64, logical ) + __field( u64, length ) + __field( u64, physical ) + __field( u64, devid ) + ), + + TP_fast_assign_btrfs(fs_info, + __entry->logical = logical; + __entry->length = length; + __entry->physical = physical; + __entry->devid = devid; + ), + + TP_printk_btrfs("logical=%llu length=%llu physical=%llu devid=%llu", + __entry->logical, __entry->length, __entry->physical, + __entry->devid) +); #endif /* _TRACE_BTRFS_H */ /* This part must be outside protection */ From patchwork Thu Sep 14 16:07:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 13385699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54EEBEEAA52 for ; Thu, 14 Sep 2023 16:07:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241885AbjINQHv (ORCPT ); Thu, 14 Sep 2023 12:07:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241877AbjINQH0 (ORCPT ); Thu, 14 Sep 2023 12:07:26 -0400 Received: from esa3.hgst.iphmx.com (esa3.hgst.iphmx.com [216.71.153.141]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29D1E2109; Thu, 14 Sep 2023 09:07:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1694707642; x=1726243642; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=/fWmE+uB6+cLnKgUKo0l86vqhneVO3Sjo3PEyQRxI/Y=; b=Ff26f5aOS43NVuQODJamUwVSmfk4lNW9GIr15krnxTU2h8MSJNfZjp1G BjXzaHx9YSaJuDYzhfD8eyeB+tSVAf1RLfkE8IiXq6hXwmYoWqyVPQ7y3 6wFHDws+2m7+RjVQEJImPWBx6QT9vo5ClzJps7RsLxX168c7Pug3/FOI4 kyWHWfy4ou3CNGtW5+mOoQxqNZWa0Fy3NzJh2n5cCaSY4uzGOSC/9oQxW wRLFrlAk0aeFswzmoK/ejNNVYrEQUNCr0OtY2FoNnBva6Hs8DLifO7o+n fD46hVI9H7LPVemPzIqaykOUwGukpzwHurfwnicgEXbnTYoLs2bKqcZYT w==; X-CSE-ConnectionGUID: ijtyTw22T9yyJRVRUCVDXA== X-CSE-MsgGUID: iiPjG7IARIyMSibIPPhIcw== X-IronPort-AV: E=Sophos;i="6.02,146,1688400000"; d="scan'208";a="248490573" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 15 Sep 2023 00:07:21 +0800 IronPort-SDR: RaJZJs312+8yF5+dZ5R1vtNzP1DPHx9TiM6zZt/3dBUPkbYK87SEwFqTP+fKASwfjYQ0XmQYsy F13Ekxa19Z9rjcYHOJuCjVWuQsqGvfjkJdiD4BfzH3f9DlC27TyCAGKU2cCf69Ws/wOb8AOGXm m5bDkmqEY/zJV615Ay56hzk3fCvizkHTzhzNllD/063at+ww3jGPxtEKy5D/Nh6Eyqizr7Gvcl cBlZCxG37/ETPLVNXrwBXr8crP6dPV+RTQiRSRMZoJ9hl7aQvVoNeX1pQGyVm7/lGjBTE+H//3 TSg= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 14 Sep 2023 08:14:23 -0700 IronPort-SDR: 75PiryJ87YXpE26JXhHlYbdzH+7UKP6OZCuyxhnI28FBl10PVmgVNVXMBSsEjiwIFvdZ1UXkv/ yhf6boxZgiRRpWfvUbhfa1cxRMX+NMyFNluo9RnE6dOCciLDwqhqO3Qnb4gydeRwopxWP5K4AW 1Q3DD+cNmG7olLNiPtq1UqzydH0Mm6IJvGIEqN7HwYoULG4dmOD0t4hZ/GhGKgqrtX4j5SVQwt KvxLOgYlRX0QKj10LG0rPmBqm96VEwkO10G+N6ZOxR5gmo1N+dEHrZUXsaRctRwD1brSMpcQ3n fPs= WDCIronportException: Internal Received: from unknown (HELO redsun91.ssa.fujisawa.hgst.com) ([10.149.66.6]) by uls-op-cesaip01.wdc.com with ESMTP; 14 Sep 2023 09:07:18 -0700 From: Johannes Thumshirn Date: Thu, 14 Sep 2023 09:07:06 -0700 Subject: [PATCH v9 11/11] btrfs: add raid-stripe-tree to features enabled with debug MIME-Version: 1.0 Message-Id: <20230914-raid-stripe-tree-v9-11-15d423829637@wdc.com> References: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> In-Reply-To: <20230914-raid-stripe-tree-v9-0-15d423829637@wdc.com> To: Chris Mason , Josef Bacik , David Sterba Cc: Christoph Hellwig , Naohiro Aota , Qu Wenruo , Damien Le Moal , linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Thumshirn X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1694707621; l=775; i=johannes.thumshirn@wdc.com; s=20230613; h=from:subject:message-id; bh=/fWmE+uB6+cLnKgUKo0l86vqhneVO3Sjo3PEyQRxI/Y=; b=/C47RMNPoCTeCb2qkXbWbQIU0ZEwwdrRMtpSq8JkjzqOmqd+FSgvX1kpR/NcWZaczD7R6lQcI OciV2JTShDiCHTpq4QBCb9vyC1n2THbrm2wUMG33IjUwmYaZZd4Soj9 X-Developer-Key: i=johannes.thumshirn@wdc.com; a=ed25519; pk=TGmHKs78FdPi+QhrViEvjKIGwReUGCfa+3LEnGoR2KM= Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Until the RAID stripe tree code is well enough tested and feature complete, "hide" it behind CONFIG_BTRFS_DEBUG so only people who want to use it are actually using it. Reviewed-by: Josef Bacik Signed-off-by: Johannes Thumshirn --- fs/btrfs/fs.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h index 5c7778e8b5ed..0f5894e2bdeb 100644 --- a/fs/btrfs/fs.h +++ b/fs/btrfs/fs.h @@ -223,7 +223,8 @@ enum { */ #define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_SUPP_STABLE | \ - BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2) + BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 |\ + BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE) #else