From patchwork Mon Aug 23 20:14:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95EE4C4320A for ; Mon, 23 Aug 2021 20:15:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 749FA60F21 for ; Mon, 23 Aug 2021 20:15:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232408AbhHWUPn (ORCPT ); Mon, 23 Aug 2021 16:15:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232404AbhHWUPm (ORCPT ); Mon, 23 Aug 2021 16:15:42 -0400 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69A85C061575 for ; Mon, 23 Aug 2021 13:14:59 -0700 (PDT) Received: by mail-qk1-x735.google.com with SMTP id t4so6223583qkb.9 for ; Mon, 23 Aug 2021 13:14:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=D+2jZRy7/aZMd3WdFd2lIGfMuqdFaSIxMjPxzmvi5UE=; b=VsS9YF7PQkQipxoYKjs+BuM4yI1VcT7R2IQv7Sh9sh12GcHl8QI1TJDCHUWOCWklpG Ov5sgkVqyFIp5LO51au3iE5L4MWhh7HIanXso0Lm1liMuFLpGlzEHVBZOghFetdVBbae tsqgnYRF7tyHv7FnKZeTEP7dS+u+ZpR11rL7AOj9py0fno9GCMRbmfzpJjl26j+d8S0A AHAsz4aFRTSVMMfWMFdJd2FGhQqxAurDxGC1G6cssnV4lfmuE5rnHclBVuvwCNOn9RAc OlgBBj+oqcnQl6Dba9hZ0KYicjXoUZJrx4MNHpscXnvhwYW0Gr/fhAsi6xjs+r1zm5VF cJqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D+2jZRy7/aZMd3WdFd2lIGfMuqdFaSIxMjPxzmvi5UE=; b=Ys9CwHAbgrsn4aZHfguABGmQmoboCzg5bzfZCMOXVPFIok89yKVceLj3zaN2re8xWb pN2QEvvzRVLB86Cb13ztX/EdJoHiLXab64pi+XIA4jdI3/jF3j71hS9099OmQD6Op7z1 vkRWLuqy5InfIyQZVHSbLyfZk3qep1db4NxowxOk0izI0qyZl8E8TxqTJQxjybYhFJ4f 2YQUz29r71qrWQfrO9wLSqI+Ny0vOGf3bV2v1XCR8w+TVWIgqCiedK+9u4Kt8liFz3EM +7VXmWYCCJa7yHB7laLfQLey5dg1ygWAcHTJ+gBzQS23itAdx8Mp9NouQQATJRz1V1xr ABoQ== X-Gm-Message-State: AOAM530ibFfEVBfGDxq5HYkOZXnT/gfDbsSQi/BwUrFkBIq8WdwPxJWy Fx6dPZ8mpzq1lPZCBaRMBFM0H3/AiFMnmg== X-Google-Smtp-Source: ABdhPJxivbI6ciYjpKM/7x6WIUuwRD7BFrlTboss++D+U33Qo13yY/fv7tGaeHzbUeDi0VYKEdfo2g== X-Received: by 2002:a37:84c3:: with SMTP id g186mr22225847qkd.276.1629749698214; Mon, 23 Aug 2021 13:14:58 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id h24sm1548374qtp.63.2021.08.23.13.14.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:14:57 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 01/10] btrfs-progs: mkfs: use an associative array for init blocks Date: Mon, 23 Aug 2021 16:14:46 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Extent tree v2 will not create an extent tree or csum tree initially, and it will create a block group tree. To handle this we want to rework the initial mkfs step to take an array of the blocks we want to create and use the array to keep track of which blocks we need to create. Signed-off-by: Josef Bacik --- mkfs/common.c | 53 ++++++++++++++++++++++++++++++++------------------- mkfs/common.h | 10 ++++++++++ 2 files changed, 43 insertions(+), 20 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index 2c041224..35ee4bff 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -31,16 +31,18 @@ #include "mkfs/common.h" static u64 reference_root_table[] = { - [1] = BTRFS_ROOT_TREE_OBJECTID, - [2] = BTRFS_EXTENT_TREE_OBJECTID, - [3] = BTRFS_CHUNK_TREE_OBJECTID, - [4] = BTRFS_DEV_TREE_OBJECTID, - [5] = BTRFS_FS_TREE_OBJECTID, - [6] = BTRFS_CSUM_TREE_OBJECTID, + [MKFS_ROOT_TREE] = BTRFS_ROOT_TREE_OBJECTID, + [MKFS_EXTENT_TREE] = BTRFS_EXTENT_TREE_OBJECTID, + [MKFS_CHUNK_TREE] = BTRFS_CHUNK_TREE_OBJECTID, + [MKFS_DEV_TREE] = BTRFS_DEV_TREE_OBJECTID, + [MKFS_FS_TREE] = BTRFS_FS_TREE_OBJECTID, + [MKFS_CSUM_TREE] = BTRFS_CSUM_TREE_OBJECTID, }; static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, - struct extent_buffer *buf) + struct extent_buffer *buf, + const enum btrfs_mkfs_block *blocks, + int blocks_nr) { struct btrfs_root_item root_item; struct btrfs_inode_item *inode_item; @@ -49,6 +51,7 @@ static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, u32 itemoff; int ret = 0; int blk; + int i; u8 uuid[BTRFS_UUID_SIZE]; memset(buf->data + sizeof(struct btrfs_header), 0, @@ -71,7 +74,8 @@ static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, btrfs_set_disk_key_offset(&disk_key, 0); itemoff = __BTRFS_LEAF_DATA_SIZE(cfg->nodesize) - sizeof(root_item); - for (blk = 0; blk < MKFS_BLOCK_COUNT; blk++) { + for (i = 0; i < blocks_nr; i++) { + blk = blocks[i]; if (blk == MKFS_SUPER_BLOCK || blk == MKFS_ROOT_TREE || blk == MKFS_CHUNK_TREE) continue; @@ -145,10 +149,13 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) struct btrfs_chunk *chunk; struct btrfs_dev_item *dev_item; struct btrfs_dev_extent *dev_extent; + const enum btrfs_mkfs_block *blocks = extent_tree_v1_blocks; u8 chunk_tree_uuid[BTRFS_UUID_SIZE]; u8 *ptr; int i; int ret; + int blocks_nr = ARRAY_SIZE(extent_tree_v1_blocks); + int blk; u32 itemoff; u32 nritems = 0; u64 first_free; @@ -195,7 +202,10 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) uuid_generate(chunk_tree_uuid); cfg->blocks[MKFS_SUPER_BLOCK] = BTRFS_SUPER_INFO_OFFSET; - for (i = 1; i < MKFS_BLOCK_COUNT; i++) { + for (i = 0; i < blocks_nr; i++) { + blk = blocks[i]; + if (blk == MKFS_SUPER_BLOCK) + continue; cfg->blocks[i] = system_group_offset + cfg->nodesize * (i - 1); } @@ -236,7 +246,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) btrfs_header_chunk_tree_uuid(buf), BTRFS_UUID_SIZE); - ret = btrfs_create_tree_root(fd, cfg, buf); + ret = btrfs_create_tree_root(fd, cfg, buf, blocks, blocks_nr); if (ret < 0) goto out; @@ -245,30 +255,33 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) cfg->nodesize - sizeof(struct btrfs_header)); nritems = 0; itemoff = __BTRFS_LEAF_DATA_SIZE(cfg->nodesize); - for (i = 1; i < MKFS_BLOCK_COUNT; i++) { + for (i = 0; i < blocks_nr; i++) { + blk = blocks[i]; + if (blk == MKFS_SUPER_BLOCK) + continue; item_size = sizeof(struct btrfs_extent_item); if (!skinny_metadata) item_size += sizeof(struct btrfs_tree_block_info); - if (cfg->blocks[i] < first_free) { + if (cfg->blocks[blk] < first_free) { error("block[%d] below first free: %llu < %llu", - i, (unsigned long long)cfg->blocks[i], + i, (unsigned long long)cfg->blocks[blk], (unsigned long long)first_free); ret = -EINVAL; goto out; } - if (cfg->blocks[i] < cfg->blocks[i - 1]) { + if (cfg->blocks[blk] < cfg->blocks[blocks[i - 1]]) { error("blocks %d and %d in reverse order: %llu < %llu", - i, i - 1, - (unsigned long long)cfg->blocks[i], - (unsigned long long)cfg->blocks[i - 1]); + blk, blocks[i - 1], + (unsigned long long)cfg->blocks[blk], + (unsigned long long)cfg->blocks[blocks[i - 1]]); ret = -EINVAL; goto out; } /* create extent item */ itemoff -= item_size; - btrfs_set_disk_key_objectid(&disk_key, cfg->blocks[i]); + btrfs_set_disk_key_objectid(&disk_key, cfg->blocks[blk]); if (skinny_metadata) { btrfs_set_disk_key_type(&disk_key, BTRFS_METADATA_ITEM_KEY); @@ -292,8 +305,8 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) nritems++; /* create extent ref */ - ref_root = reference_root_table[i]; - btrfs_set_disk_key_objectid(&disk_key, cfg->blocks[i]); + ref_root = reference_root_table[blk]; + btrfs_set_disk_key_objectid(&disk_key, cfg->blocks[blk]); btrfs_set_disk_key_offset(&disk_key, ref_root); btrfs_set_disk_key_type(&disk_key, BTRFS_TREE_BLOCK_REF_KEY); btrfs_set_item_key(buf, &disk_key, nritems); diff --git a/mkfs/common.h b/mkfs/common.h index ea87c3ca..378da6bd 100644 --- a/mkfs/common.h +++ b/mkfs/common.h @@ -55,6 +55,16 @@ enum btrfs_mkfs_block { MKFS_BLOCK_COUNT }; +static const enum btrfs_mkfs_block extent_tree_v1_blocks[] = { + MKFS_SUPER_BLOCK, + MKFS_ROOT_TREE, + MKFS_EXTENT_TREE, + MKFS_CHUNK_TREE, + MKFS_DEV_TREE, + MKFS_FS_TREE, + MKFS_CSUM_TREE, +}; + struct btrfs_mkfs_config { /* Label of the new filesystem */ const char *label; From patchwork Mon Aug 23 20:14:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1522C432BE for ; Mon, 23 Aug 2021 20:15:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A64E9613DA for ; Mon, 23 Aug 2021 20:15:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232422AbhHWUPo (ORCPT ); Mon, 23 Aug 2021 16:15:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232412AbhHWUPo (ORCPT ); Mon, 23 Aug 2021 16:15:44 -0400 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CB30C061575 for ; Mon, 23 Aug 2021 13:15:01 -0700 (PDT) Received: by mail-qk1-x733.google.com with SMTP id a66so1847223qkc.1 for ; Mon, 23 Aug 2021 13:15:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=OUTnjDt2fPnKZkbPtE3c5yUpcclsZaL7w/4Ket3lhAM=; b=FWUSrmZew5PEAleOHtkeOSdWLlUHpVLKE4OvhLiNm8ztaHoIkpTyK7QqJtA8e3BFDd z/uGxAGecUhWoEmmIM105v3NyviPKpP6luFm76WnDqMrxaWkr6L2Hi6aaOTNDmHAfHI4 dCAdXbJPkw7mWz2rBNJMrqtCk/ExzdDU5kDFM9sZwwbvofXANC9s/4z5qqFYg2eYhHxg ktFFXVSkUpyXyQ5aXqko+U/Xfp3e3fpnTcJNJPxQKDpgSYB7Wp2ZCTMGkIm+6cVgkQE+ DKAOzlNDTjbY13Iqmgx7WKSbPbWWf6b4msvr/cNQbMW4l00w+JcHhbG2iYCRjcphCxHx nfVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OUTnjDt2fPnKZkbPtE3c5yUpcclsZaL7w/4Ket3lhAM=; b=LM4uzEDnfAlnjxL0W+qwn+YkNP2V1YPC3t7oFTjsmwNeRxX6bo6RP99v5jgRrfLKKA fiVNzNwhOdJrnel9bAxdDX5lOauSBsM8Ya4HSBiM6w4NWsCPFISSrLuQae4lHElrioBA /u82FqmliI8eD+HOAMr5t0q0bRfhIxh3NFBPdtOurq9rXWR46LD3+zVEJ+YtrVnVPH1e hMmywYy894mZbBdcZzT8m8+rPHp0yau52tG1qgrbc+M7Cid09FgReJKrtD8vRfVfino3 4b2qqyYoUZtzO1U6DYBXRPbdnAMUbwBbkcgm9oKMi2Xwyw12bfLh//rsjTFpZdyaxg+F pvDg== X-Gm-Message-State: AOAM530hHVrLMstRFAG6XifGwMDQjeS8iYKIpktOVRaV5wIf5Lu0LVIT 6z8QKxYNery7Z40FI293JTamVTBrhkEm7w== X-Google-Smtp-Source: ABdhPJwDIz+w4hO38IGNi55/OdvNH/4R52Jojzq9o9EBMOx8OiUACZpsqbfkuZf2NoSCFhzbVkoTHg== X-Received: by 2002:a05:620a:1495:: with SMTP id w21mr22801483qkj.443.1629749699878; Mon, 23 Aug 2021 13:14:59 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id n189sm9193188qka.69.2021.08.23.13.14.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:14:59 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 02/10] btrfs-progs: mkfs: get rid of MKFS_SUPER_BLOCK Date: Mon, 23 Aug 2021 16:14:47 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We use these block's in order to keep track of which blocks need to be added to the extent tree and where their roots need to be written. However we skip MKFS_SUPER_BLOCK for all of these helpers, and we don't actually need to keep track of the specific block we allocated because it is always BTRFS_SUPER_INFO_OFFSET. Remove this enum as we don't need it. Signed-off-by: Josef Bacik --- mkfs/common.c | 17 ++++++----------- mkfs/common.h | 2 -- 2 files changed, 6 insertions(+), 13 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index 35ee4bff..ee9ad390 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -76,8 +76,7 @@ static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, for (i = 0; i < blocks_nr; i++) { blk = blocks[i]; - if (blk == MKFS_SUPER_BLOCK || blk == MKFS_ROOT_TREE - || blk == MKFS_CHUNK_TREE) + if (blk == MKFS_ROOT_TREE || blk == MKFS_CHUNK_TREE) continue; btrfs_set_root_bytenr(&root_item, cfg->blocks[blk]); @@ -201,15 +200,12 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) uuid_generate(super.dev_item.uuid); uuid_generate(chunk_tree_uuid); - cfg->blocks[MKFS_SUPER_BLOCK] = BTRFS_SUPER_INFO_OFFSET; for (i = 0; i < blocks_nr; i++) { blk = blocks[i]; - if (blk == MKFS_SUPER_BLOCK) - continue; - cfg->blocks[i] = system_group_offset + cfg->nodesize * (i - 1); + cfg->blocks[blk] = system_group_offset + cfg->nodesize * i; } - btrfs_set_super_bytenr(&super, cfg->blocks[MKFS_SUPER_BLOCK]); + btrfs_set_super_bytenr(&super, BTRFS_SUPER_INFO_OFFSET); btrfs_set_super_num_devices(&super, 1); btrfs_set_super_magic(&super, BTRFS_MAGIC_TEMPORARY); btrfs_set_super_generation(&super, 1); @@ -257,8 +253,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) itemoff = __BTRFS_LEAF_DATA_SIZE(cfg->nodesize); for (i = 0; i < blocks_nr; i++) { blk = blocks[i]; - if (blk == MKFS_SUPER_BLOCK) - continue; + item_size = sizeof(struct btrfs_extent_item); if (!skinny_metadata) item_size += sizeof(struct btrfs_tree_block_info); @@ -270,7 +265,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) ret = -EINVAL; goto out; } - if (cfg->blocks[blk] < cfg->blocks[blocks[i - 1]]) { + if (i && cfg->blocks[blk] < cfg->blocks[blocks[i - 1]]) { error("blocks %d and %d in reverse order: %llu < %llu", blk, blocks[i - 1], (unsigned long long)cfg->blocks[blk], @@ -487,7 +482,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) buf->len = BTRFS_SUPER_INFO_SIZE; csum_tree_block_size(buf, btrfs_csum_type_size(cfg->csum_type), 0, cfg->csum_type); - ret = sbwrite(fd, buf->data, cfg->blocks[MKFS_SUPER_BLOCK]); + ret = sbwrite(fd, buf->data, BTRFS_SUPER_INFO_OFFSET); if (ret != BTRFS_SUPER_INFO_SIZE) { ret = (ret < 0 ? -errno : -EIO); goto out; diff --git a/mkfs/common.h b/mkfs/common.h index 378da6bd..f2d28057 100644 --- a/mkfs/common.h +++ b/mkfs/common.h @@ -45,7 +45,6 @@ struct btrfs_root; * Tree root blocks created during mkfs */ enum btrfs_mkfs_block { - MKFS_SUPER_BLOCK = 0, MKFS_ROOT_TREE, MKFS_EXTENT_TREE, MKFS_CHUNK_TREE, @@ -56,7 +55,6 @@ enum btrfs_mkfs_block { }; static const enum btrfs_mkfs_block extent_tree_v1_blocks[] = { - MKFS_SUPER_BLOCK, MKFS_ROOT_TREE, MKFS_EXTENT_TREE, MKFS_CHUNK_TREE, From patchwork Mon Aug 23 20:14:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6743CC4338F for ; Mon, 23 Aug 2021 20:15:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F28460F21 for ; Mon, 23 Aug 2021 20:15:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232446AbhHWUPq (ORCPT ); Mon, 23 Aug 2021 16:15:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232412AbhHWUPp (ORCPT ); Mon, 23 Aug 2021 16:15:45 -0400 Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5264C061575 for ; Mon, 23 Aug 2021 13:15:02 -0700 (PDT) Received: by mail-qk1-x72c.google.com with SMTP id e14so20607509qkg.3 for ; Mon, 23 Aug 2021 13:15:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5V+LAn/KXmEOAmsXEneXBpak+MuAs9x8XxDh5+4wpvU=; b=zrCPwUXTkl4prIIFL7MMUK7cowzpp2j6BjedVG8PRTIJ741vv5l6/KFkaUkbtaIQ4O 6QVKqNQ9aBgr0WXk0HtJGCSulLFF6q0KB9Zjh0i7Fk8E44+d8KUvcLuapoZRRfy3MFAj 6Q6r4Q7sDq9SQgv+zIGF/bskXZAm9TE09MIN0kNKWwqhMfz7+M2wxb5syFW1WIi0KxGE OFpcoGyf/zTvdxNQ2zCfNLgacZD8JHSQXZRLg87T1bM6cDcpVExcO7/rcERkL9cE26jW 3qOk1LCW5LfsP2AGmvL/EThXA9jwiavzGt+VmX+hJhQYidtNz3aTc3X1FkZoSSjKFSl0 OtFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5V+LAn/KXmEOAmsXEneXBpak+MuAs9x8XxDh5+4wpvU=; b=L2aegqs6cf7/dDxcWKEZ5DzplY2M5sF9rrrXXjWsqsY9dwlqIXh8j/JBpkFX7ziIDz Sw5RpsEfgU9dEA59WQvDCVMPZ1RKD63TtMd25Dnerz3AJpSGP0ST3S2JCJWFPpfS3SXh kPzlVNjHF699viRi3PAaB0Hsia/CmlrpYddPL+3Q1kt99I0cVrWt46H0qCqWm39mA3cT Jyvm1ioKtWTUvPfuI9ytnXqW639agQx/HcEfadddq+3aDCDdiq7H6E7mxhAuJVLSRpJN 1OKKRTRxMyvXwt2JBU2kAZ/1ECSWJPp274qmFH85G8QYIRHXolHmRehCCTkiI98agndb eXKA== X-Gm-Message-State: AOAM531QvweE+61tDg6TNg0Ow43GApKPTIe6Igd7IwJngC2G2T/7Dx7K 4o2gZFYfZwidqnF3aZNHgd9lZO07XY/huQ== X-Google-Smtp-Source: ABdhPJzJP4FWzXVnVF/81n9KKK3wGJQ7GLsHZvsPXJmLxcm4ZynnIJKt4i3GeiINJdLKS/nOw+pgxw== X-Received: by 2002:a37:809:: with SMTP id 9mr23004940qki.318.1629749701695; Mon, 23 Aug 2021 13:15:01 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id a24sm7561166qtj.43.2021.08.23.13.15.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:01 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Qu Wenruo Subject: [PATCH v2 03/10] btrfs-progs: mkfs: use blocks_nr to determine the super used bytes Date: Mon, 23 Aug 2021 16:14:48 -0400 Message-Id: <09b49b7d3c976f7aa34e3ff91f1b767c45e90812.1629749291.git.josef@toxicpanda.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We were setting the superblock's used bytes to a static number. However the number of blocks we have to write has the correct used size, so just add up the total number of blocks we're allocating as we determine their offsets. This value will be used later which is why I'm calculating it this way instead of doing the math to set the bytes_super specifically. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- mkfs/common.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mkfs/common.c b/mkfs/common.c index ee9ad390..8718969d 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -161,6 +161,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) u64 ref_root; u32 array_size; u32 item_size; + u64 total_used = 0; int skinny_metadata = !!(cfg->features & BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA); u64 num_bytes; @@ -203,6 +204,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) for (i = 0; i < blocks_nr; i++) { blk = blocks[i]; cfg->blocks[blk] = system_group_offset + cfg->nodesize * i; + total_used += cfg->nodesize; } btrfs_set_super_bytenr(&super, BTRFS_SUPER_INFO_OFFSET); @@ -212,7 +214,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) btrfs_set_super_root(&super, cfg->blocks[MKFS_ROOT_TREE]); btrfs_set_super_chunk_root(&super, cfg->blocks[MKFS_CHUNK_TREE]); btrfs_set_super_total_bytes(&super, num_bytes); - btrfs_set_super_bytes_used(&super, 6 * cfg->nodesize); + btrfs_set_super_bytes_used(&super, total_used); btrfs_set_super_sectorsize(&super, cfg->sectorsize); super.__unused_leafsize = cpu_to_le32(cfg->nodesize); btrfs_set_super_nodesize(&super, cfg->nodesize); From patchwork Mon Aug 23 20:14:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB8E9C432BE for ; Mon, 23 Aug 2021 20:15:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C5B6B613D0 for ; Mon, 23 Aug 2021 20:15:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232450AbhHWUPr (ORCPT ); Mon, 23 Aug 2021 16:15:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232412AbhHWUPr (ORCPT ); Mon, 23 Aug 2021 16:15:47 -0400 Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55E54C061575 for ; Mon, 23 Aug 2021 13:15:04 -0700 (PDT) Received: by mail-qk1-x729.google.com with SMTP id a10so11734861qka.12 for ; Mon, 23 Aug 2021 13:15:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c6J8br5T+/R5thnTvgKORFmCR7B/pEUhJ1PfQi2Zo/k=; b=HRWQCFhFvo/BuymJn6XXbl1RaKtedlmXKn59b0SS7ikmOqpPTi5Lpvl2kJgL8rFNjC h2ra3853i4wHaK/euoqvCDupHFzHKvoEgpyYbDATQ53fzBYUrS8yQTi8QsKyEqvL5jh4 yAHT42cBTmun7oKsOpJ6UegFJh2EcbWZmmJLNoGztalBntyGlX3MaHi7RKhqPqarbS+Z vYKQ9FN9O7mlFM1XeOW78KDvAyH37jSzoPehsDpRgpDmGswZIuW5ye2K0lGQPwB2/oYA 0uNJmPUkFyxdXfHw8gagJaZFvg3NLf95MKTgV2PkiDUHfPc29CGRt7z3EBFOmHMAHger B+iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c6J8br5T+/R5thnTvgKORFmCR7B/pEUhJ1PfQi2Zo/k=; b=cnZNqGA/7xeW1lcWCOtDEbozc2fMmAzCPKDOgw/1VSr1aE+Za61YJW4eD4CAyNaBh7 SFI25m/g90AYbGDLDF5qopWLnwF5Jbea7S5eyPLx4VN6Jq46+uMD4JQijoqW0FXiYUlO vJED2HONM0v8sji9Tsf9EObi5XbJvEfWTAlZfZ6NcjAjJQ7Nh8KMW9LtuEyXXFpJdWWC UmIDqgrJekwnW7QIpVeNUefRibenaAq3ym1coQthB6c11DxdXPfPilaXJgGaKlULfqS/ DYL+U7QBrzTuJkzT8uIBCyfMwYxmcEsa8P2GFCZcVoebFyY9jCtTjR0oOlsk+wWw6lv6 4kog== X-Gm-Message-State: AOAM530/FYHzM7GFI7wkstklja9mACA9MGwXRQ1whR3+Qtps6x9Ea9Bw 9yYAWgPe2xd9KYw/QtcyG2FN8skxAJfocQ== X-Google-Smtp-Source: ABdhPJwodLyc7PR22aT9nEl4vZS9Di47tP9Zs9FvslFgGa0M1yCZksU/PjPr7ly4UONOZvGB+my6Tg== X-Received: by 2002:a05:620a:b10:: with SMTP id t16mr23346715qkg.158.1629749703205; Mon, 23 Aug 2021 13:15:03 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id q14sm9086253qkl.44.2021.08.23.13.15.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:02 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Qu Wenruo Subject: [PATCH v2 04/10] btrfs-progs: mkfs: set nritems based on root items written Date: Mon, 23 Aug 2021 16:14:49 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org For the root tree we were just hard setting the nritems to 4, which will change when we move to extent tree v2. Instead set the nritems after we've added all the root items we need to the root tree. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- mkfs/common.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mkfs/common.c b/mkfs/common.c index 8718969d..339c5556 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -106,6 +106,8 @@ static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, itemoff -= sizeof(root_item); } + btrfs_set_header_nritems(buf, nritems); + /* generate checksum */ csum_tree_block_size(buf, btrfs_csum_type_size(cfg->csum_type), 0, cfg->csum_type); @@ -233,7 +235,6 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) memset(buf->data, 0, cfg->nodesize); buf->len = cfg->nodesize; btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_ROOT_TREE]); - btrfs_set_header_nritems(buf, 4); btrfs_set_header_generation(buf, 1); btrfs_set_header_backref_rev(buf, BTRFS_MIXED_BACKREF_REV); btrfs_set_header_owner(buf, BTRFS_ROOT_TREE_OBJECTID); From patchwork Mon Aug 23 20:14:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6645C432BE for ; Mon, 23 Aug 2021 20:15:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9194D613D3 for ; Mon, 23 Aug 2021 20:15:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232460AbhHWUPt (ORCPT ); Mon, 23 Aug 2021 16:15:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232412AbhHWUPt (ORCPT ); Mon, 23 Aug 2021 16:15:49 -0400 Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19554C061575 for ; Mon, 23 Aug 2021 13:15:06 -0700 (PDT) Received: by mail-qt1-x836.google.com with SMTP id g11so14892030qtk.5 for ; Mon, 23 Aug 2021 13:15:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3QDCulYuGjgRhd7CiWWolUZz+Qzo7UKc9pr9nZpMg9s=; b=SVROTRhFruoMJfltoWAwji0yrW2+S4jZcIooDrh3NwzAv0sIJrd8S0QUNBASbgUF5P MzP4NrcME6i31zknlDcotUnTX3gvSODmwF0U9kaL+dxwfa+T140y58ERNXEdxYC2JstH bvCpYXayU/Q/rh65dR+ry9kcMRdI/l6LhB/gTnNWZ/LXVX7pWwNoIoWBN5a+J25bP8TE e/K67k1APx767HmDUluUFDMcYSENVKMAfcnXlz4IPOwuOTa2nKloqndkAA46HiNLxDpc Njm8Km8PPPNi1uFFntCNHwL6KfjfxcQn7HDtDxlxrpNDFaHozKGaIsf+vdxBNsblm7Sw hk3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3QDCulYuGjgRhd7CiWWolUZz+Qzo7UKc9pr9nZpMg9s=; b=TZ+0TGSWm8pvms3yWvKDREJVfSni5YWFdYNsUxbaidg/aqKwtqjQakzA/rLPpKA5mW LcxvzCw8oSpmcNyZk6J238MPqvZiTqLLu5gCZsxLTQg8CVctFbkgu8xDGfaO14oJZkxN EwJhR8MVZT2nyoQDa2coZMwvrBWlHHuC1tey8VAPStuRUtKwljmjIJLEQhG9LlJBl8NQ 9fLGUwEOBwxKgm3Ki8vfShNqp38XKmR5aZN8LNTcKYXaCwUu5VKgscx223yu8bcxHKHT C6glnHijvuMuLvkUpJIHh5xe95HENA4PUtFU/LQPN3AnPzJv4Do7jf8c0PDldGSexg6q QRBQ== X-Gm-Message-State: AOAM531caNC69FBR+zknpBpWCweV2aYAKZAJyonLVJF/D9lCBJZOK4PX OffuHtDWNn6mBBMe+Y0RkDPeY+RwKH4AEA== X-Google-Smtp-Source: ABdhPJzJKRV3fc7z2r/CDu2lr4icusmY8FJxyMsbCvKGEHKV7HfyJHJUNRLWC0l/r+Ke8g0FFyT/0Q== X-Received: by 2002:ac8:73cb:: with SMTP id v11mr5711161qtp.292.1629749704894; Mon, 23 Aug 2021 13:15:04 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id w129sm9304069qkb.61.2021.08.23.13.15.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:04 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Qu Wenruo Subject: [PATCH v2 05/10] btrfs-progs: mkfs: add helper for writing empty tree nodes Date: Mon, 23 Aug 2021 16:14:50 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org With extent tree v2 we're going to be writing some more empty trees for the initial mkfs step, so take this common code and make it a helper. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- mkfs/common.c | 47 +++++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index 339c5556..a392a5b0 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -39,6 +39,25 @@ static u64 reference_root_table[] = { [MKFS_CSUM_TREE] = BTRFS_CSUM_TREE_OBJECTID, }; +static int btrfs_write_empty_tree(int fd, struct btrfs_mkfs_config *cfg, + struct extent_buffer *buf, u64 objectid, + u64 block) +{ + int ret; + + memset(buf->data + sizeof(struct btrfs_header), 0, + cfg->nodesize - sizeof(struct btrfs_header)); + btrfs_set_header_bytenr(buf, block); + btrfs_set_header_owner(buf, objectid); + btrfs_set_header_nritems(buf, 0); + csum_tree_block_size(buf, btrfs_csum_type_size(cfg->csum_type), 0, + cfg->csum_type); + ret = pwrite(fd, buf->data, cfg->nodesize, block); + if (ret != cfg->nodesize) + return ret < 0 ? -errno : -EIO; + return 0; +} + static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, struct extent_buffer *buf, const enum btrfs_mkfs_block *blocks, @@ -453,31 +472,15 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) } /* create the FS root */ - memset(buf->data + sizeof(struct btrfs_header), 0, - cfg->nodesize - sizeof(struct btrfs_header)); - btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_FS_TREE]); - btrfs_set_header_owner(buf, BTRFS_FS_TREE_OBJECTID); - btrfs_set_header_nritems(buf, 0); - csum_tree_block_size(buf, btrfs_csum_type_size(cfg->csum_type), 0, - cfg->csum_type); - ret = pwrite(fd, buf->data, cfg->nodesize, cfg->blocks[MKFS_FS_TREE]); - if (ret != cfg->nodesize) { - ret = (ret < 0 ? -errno : -EIO); + ret = btrfs_write_empty_tree(fd, cfg, buf, BTRFS_FS_TREE_OBJECTID, + cfg->blocks[MKFS_FS_TREE]); + if (ret) goto out; - } /* finally create the csum root */ - memset(buf->data + sizeof(struct btrfs_header), 0, - cfg->nodesize - sizeof(struct btrfs_header)); - btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_CSUM_TREE]); - btrfs_set_header_owner(buf, BTRFS_CSUM_TREE_OBJECTID); - btrfs_set_header_nritems(buf, 0); - csum_tree_block_size(buf, btrfs_csum_type_size(cfg->csum_type), 0, - cfg->csum_type); - ret = pwrite(fd, buf->data, cfg->nodesize, cfg->blocks[MKFS_CSUM_TREE]); - if (ret != cfg->nodesize) { - ret = (ret < 0 ? -errno : -EIO); + ret = btrfs_write_empty_tree(fd, cfg, buf, BTRFS_CSUM_TREE_OBJECTID, + cfg->blocks[MKFS_CSUM_TREE]); + if (ret) goto out; - } /* and write out the super block */ memset(buf->data, 0, BTRFS_SUPER_INFO_SIZE); From patchwork Mon Aug 23 20:14:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61079C4320A for ; Mon, 23 Aug 2021 20:15:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4BDEF613D0 for ; Mon, 23 Aug 2021 20:15:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232465AbhHWUPv (ORCPT ); Mon, 23 Aug 2021 16:15:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232412AbhHWUPu (ORCPT ); Mon, 23 Aug 2021 16:15:50 -0400 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4724C061575 for ; Mon, 23 Aug 2021 13:15:07 -0700 (PDT) Received: by mail-qk1-x735.google.com with SMTP id ay33so9108690qkb.10 for ; Mon, 23 Aug 2021 13:15:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=0tFf9JN+Jo8kT/tPISRImGoNIQDDhS8Strz5dXKSgkg=; b=DOa3R0mpv8Zn1D6/kCLvbvISmF8x9EfXItepQho5ykjWnH8Gnpp8BtGLba0gg8II1m 0WUN6ji3u9eQRNWg7uYMczlTtmwm5lzmGKPuOEGYwCam+kG3pjiYOodzzsN5aeIM7DgP 0DLwdGHSaEaZON6O+QvhTpjoEFqJ71eFPWCDnNTq8182+pbInD7bWrRqe5joaAD3UTyD H5ZCBuvsCnf0uEtMpVx+80KrDiJVYiq/6w8JxRr0VNQnMZD3HC4YtaGrosXMLHXMVLjb +Sm6FL0j10sZi4mwHdGRzbH1qQ4G0QLyJ7LOjfRDtRZqFfuU/nsWnY6AXCWKs9h8/Svj Z6Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0tFf9JN+Jo8kT/tPISRImGoNIQDDhS8Strz5dXKSgkg=; b=VVnl0v7wptJePs2zKT0Soiv3jtIqhoAhtvCd0BTku6a/c6lWGbMLHwti5duq713yuN kQ0ETkkkGsRez+7PvwC3K85MrWQ5tEg/WEGtssz+TCJp2jhq4WzkJkdm514eQU2p5+7n XYmA9GJlkkwrzyGQ++50UyHPl0QL6rAgpma3kxf2deXUerm3/FuLqhRwNkTCiIG3Z+M/ PoaRRLgSvgkVA/5uL3TP3R99buyTCaWglGlEwb95jWCDV4446dt6Sj2n5/ZjL59iCMor Md7PzcDxWiRaCBVAOgtc6ldsaAUQu5GjYzkxAFIus4B/T2o4Bm6UurW8NrG776nrwbeS QvQg== X-Gm-Message-State: AOAM533sJA0ljaIjObFYmBoVKqfJSVcbSNjKPmZUnDLoCPW2IEOkKfqz FgGFFWvbIDGr+O8iHdwG/3+yLbnpPq0BXQ== X-Google-Smtp-Source: ABdhPJziDC8bKyqIom/2u8fNU/aa2Ui6Q4UfDMpLWuUyBJZz80ucWbPEElF/kUrojyw14qt3GJmQ2w== X-Received: by 2002:a37:c94:: with SMTP id 142mr22925821qkm.78.1629749706459; Mon, 23 Aug 2021 13:15:06 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id x9sm9254507qko.125.2021.08.23.13.15.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:06 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 06/10] btrfs-progs: make sure track_dirty and ref_cows is set properly Date: Mon, 23 Aug 2021 16:14:51 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Adding support for the per-block group roots means we will be reading the roots directly in different places. Make sure we set ->track_dirty and ->ref_cows properly in the helper so we don't have to do this everywhere. Signed-off-by: Josef Bacik --- kernel-shared/disk-io.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel-shared/disk-io.c b/kernel-shared/disk-io.c index 84990a52..7f99fc8d 100644 --- a/kernel-shared/disk-io.c +++ b/kernel-shared/disk-io.c @@ -720,7 +720,10 @@ out: return ERR_PTR(-EIO); } insert: - root->ref_cows = 1; + if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) + root->track_dirty = 1; + if (is_fstree(root->root_key.objectid)) + root->ref_cows = 1; return root; } From patchwork Mon Aug 23 20:14:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8FB4C4338F for ; Mon, 23 Aug 2021 20:15:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C4679613D0 for ; Mon, 23 Aug 2021 20:15:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232494AbhHWUPz (ORCPT ); Mon, 23 Aug 2021 16:15:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232470AbhHWUPw (ORCPT ); Mon, 23 Aug 2021 16:15:52 -0400 Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [IPv6:2607:f8b0:4864:20::f2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66451C061757 for ; Mon, 23 Aug 2021 13:15:09 -0700 (PDT) Received: by mail-qv1-xf2d.google.com with SMTP id dt3so10429027qvb.6 for ; Mon, 23 Aug 2021 13:15:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=YGYGP5RQD+adbEB++MiprG5RWu+nlbYewivpOwHIuYI=; b=UgshwOXnD3ZC8THeXBgByu3GIzBA8vh3WhSemIFjZXeHWJuL5XYV4gJuNu76j5kZda TQugV++auYZImsPTBplD2t0uN0Ep7wqHwhcy2Gjuy7u0WeqNMS6WRJoXJmdJRIqhfxH4 PXjNPQ/mVHZxtWTY9Zb9jfvLhc795/9a0D//JWorFHOC5srFpg4unaLR8radSIc616bO tZWYvopA0IIGSyGuGR0S+Z/tajuB2jjFnWpsE95UmekFxHEzTAPiPwfpkHq5m4FqXP/U cZcq5q8oMNy9ly020VQFan1fhxnFO274r7cJwPcOaxWtkKGXBbFAOL448IFjYSvpo3sK mTlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YGYGP5RQD+adbEB++MiprG5RWu+nlbYewivpOwHIuYI=; b=OQAk2QEMoYr0TpYuiM4uWe6Gx9YyvW7fb8Y6HAW2kQVRJEzyJT7QWusQqhBt5nt2Kv +UpwOKUg5nEdAYbdHQoO7hT6/xlfw1tB0THYFrWPv+doF0cGQQBasEcdrfkXy7loW8X0 EcYJIcl7gXmpQ5ckt4s+8wTo7xV9z9VRCFvoOR7m+5YlG7e194rG+ckMopeRtLcSpL7y Md9DeiddgUdipzmtVDpK3X3i3NBZpxDFZdgmv75NNE9tHRI1nLy+qzScubREk71Mphi4 V/81a/Th3PyHsWNoe7v9MKavsvUSbGE9SodWeVwVsWQiu8j89Ki2cNaNgVapBcr2wt9q sueA== X-Gm-Message-State: AOAM533zry3S7xdFfgYpmXZlRty0clvRy+gmbohKqkLMgWW0hFtmKuVT hgwtLLGuzSzj2AinDGsqEmXmbeS/0aaCNg== X-Google-Smtp-Source: ABdhPJwKbrufqjW4NoLOqrPzY2eRQvKhVZdYpsFp6u+btfBM2Rc7neiaULdwuUZdLE/6X+8MKWT9xg== X-Received: by 2002:a05:6214:2408:: with SMTP id fv8mr35317961qvb.4.1629749708205; Mon, 23 Aug 2021 13:15:08 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id x25sm7242181qtj.77.2021.08.23.13.15.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:07 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 07/10] btrfs-progs: mkfs: add the block group item in make_btrfs() Date: Mon, 23 Aug 2021 16:14:52 -0400 Message-Id: <291a31ef4666c1f3b7fb265ec8dcecdd10943722.1629749291.git.josef@toxicpanda.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently we build a bare-bones file system in make_btrfs(), and then we load it up and fill in the rest of the file system after the fact. One thing we omit in make_btrfs() is the block group item for the temporary system chunk we allocate, because we just add it after we've opened the file system. However I want to be able to generate the free space tree at make_btrfs() time, because extent tree v2 will not have an extent tree that has every block allocated in the system. In order to do this I need to make sure that the free space tree entries are added on block group creation, which is annoying if we have to add this chunk after I've created a free space tree. So make future work simpler by simply adding our block group item at make_btrfs() time, this way I can do the right things with the free space tree in the generic make block group code without needing a special case for our temporary system chunk. Signed-off-by: Josef Bacik --- mkfs/common.c | 31 +++++++++++++++++++++++++++++++ mkfs/main.c | 9 ++------- 2 files changed, 33 insertions(+), 7 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index a392a5b0..9b5f96e3 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -188,6 +188,7 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) u64 num_bytes; u64 system_group_offset = BTRFS_BLOCK_RESERVED_1M_FOR_SUPER; u64 system_group_size = BTRFS_MKFS_SYSTEM_GROUP_SIZE; + bool add_block_group = true; if ((cfg->features & BTRFS_FEATURE_INCOMPAT_ZONED)) { system_group_offset = cfg->zone_size * BTRFS_NR_SB_LOG_ZONES; @@ -276,6 +277,36 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) for (i = 0; i < blocks_nr; i++) { blk = blocks[i]; + /* Add the block group item for our temporary chunk. */ + if (cfg->blocks[blk] > system_group_offset && + add_block_group) { + struct btrfs_block_group_item *bg_item; + + add_block_group = false; + + itemoff -= sizeof(*bg_item); + btrfs_set_disk_key_objectid(&disk_key, + system_group_offset); + btrfs_set_disk_key_offset(&disk_key, + system_group_size); + btrfs_set_disk_key_type(&disk_key, + BTRFS_BLOCK_GROUP_ITEM_KEY); + btrfs_set_item_key(buf, &disk_key, nritems); + btrfs_set_item_offset(buf, btrfs_item_nr(nritems), + itemoff); + btrfs_set_item_size(buf, btrfs_item_nr(nritems), + sizeof(*bg_item)); + + bg_item = btrfs_item_ptr(buf, nritems, + struct btrfs_block_group_item); + btrfs_set_block_group_used(buf, bg_item, total_used); + btrfs_set_block_group_flags(buf, bg_item, + BTRFS_BLOCK_GROUP_SYSTEM); + btrfs_set_block_group_chunk_objectid(buf, bg_item, + BTRFS_FIRST_CHUNK_TREE_OBJECTID); + nritems++; + } + item_size = sizeof(struct btrfs_extent_item); if (!skinny_metadata) item_size += sizeof(struct btrfs_tree_block_info); diff --git a/mkfs/main.c b/mkfs/main.c index eab93eb3..ea53e9c7 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -67,7 +67,6 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, struct btrfs_trans_handle *trans; struct btrfs_space_info *sinfo; u64 flags = BTRFS_BLOCK_GROUP_METADATA; - u64 bytes_used; u64 chunk_start = 0; u64 chunk_size = 0; u64 system_group_offset = BTRFS_BLOCK_RESERVED_1M_FOR_SUPER; @@ -90,16 +89,12 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, trans = btrfs_start_transaction(root, 1); BUG_ON(IS_ERR(trans)); - bytes_used = btrfs_super_bytes_used(fs_info->super_copy); root->fs_info->system_allocs = 1; /* - * First temporary system chunk must match the chunk layout - * created in make_btrfs(). + * We already created the block group item for our temporary system + * chunk in make_btrfs(), so account for the size here. */ - ret = btrfs_make_block_group(trans, fs_info, bytes_used, - BTRFS_BLOCK_GROUP_SYSTEM, - system_group_offset, system_group_size); allocation->system += system_group_size; if (ret) return ret; From patchwork Mon Aug 23 20:14:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A974AC432BE for ; Mon, 23 Aug 2021 20:15:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8DCD5613D3 for ; Mon, 23 Aug 2021 20:15:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232475AbhHWUP4 (ORCPT ); Mon, 23 Aug 2021 16:15:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbhHWUPz (ORCPT ); Mon, 23 Aug 2021 16:15:55 -0400 Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F421AC06175F for ; Mon, 23 Aug 2021 13:15:10 -0700 (PDT) Received: by mail-qk1-x730.google.com with SMTP id e14so20608165qkg.3 for ; Mon, 23 Aug 2021 13:15:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4RIjkuDeY3PlmnTFqIEBwtBoE99vXi4w6UmJB3UY2rg=; b=WVntmK3fUltCcwT7Nvv6auK9mmABs0rMfMlfI4r1HWsmPHBU6eS6p8CNZcsymFWJlX b9kLi2KjYTLIqR/CHPJTL+LvU/fcSPhfC9oufdvz04aPM8X1/tKbUgGTzuMl0rykE2oK P7g9wuGzxWeG4Dhj4emJaZHqm9k6JDLrDu1FZhHeNC+uOkoSP/UoUpKNPAVPURTKoWEw 9xXFuu7lm0MLq/eJ5e9e7zf08rkNo1YzjT5sQpgPOVatgyte3vZHQ2Dw699Az7vEQd6X 4G0vj/jwyV8IZa2+CBV9KgEGCjJaTZolTxd6igqw7IQ2Q0menckPJcUAJp/8FQLKQmiK zwKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4RIjkuDeY3PlmnTFqIEBwtBoE99vXi4w6UmJB3UY2rg=; b=QnsG8EhJwxvU4+TnIKN+EVk5L34FS9wDsrFj5FJxhmaYsJpMfd47Df0p703ccQWVC7 TqKZam5gQ0KmZOCCy4fff5726a+7Gnw938n2Fq+9T5mUGOcRk80SXBK1ddMTfQAuzTip 59vfxhXwx12lFYZ9uYbhrp4iVVa7i3Gc8uR7EqMeRL/OL7145OzFOoFnQeZOVWlyS13r 3eLv1odbrpLdUSkFCK2Kki18qR8vz6dhHZbCaEypo2VxaknYgd4hta8AtZ5hIbYoLI/D ENve7b5MBbJJ7+tYx1NE9+FXxNUtNI+k7//4mjz7MOLYVxcTBp10RuwrK08FINjujXyK ksSg== X-Gm-Message-State: AOAM531S2po54dGnxMNZk41A+bJ0N6czUjmNAKmqK64nK1PCbJ7V2BAl 0Gno9+ctk+lbff77T5qn3Ap+J5hGMbKOxw== X-Google-Smtp-Source: ABdhPJxGYMiqe0Ev8DaMq/nq31STlLhiJt6VGmZKogOXuBvO6IwdDyw1/saJDl5f4aCJ/L0vY7Loxg== X-Received: by 2002:a37:a13:: with SMTP id 19mr23764233qkk.285.1629749709767; Mon, 23 Aug 2021 13:15:09 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id c22sm860666qtc.89.2021.08.23.13.15.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:09 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 08/10] btrfs-progs: add add_block_group_free_space helper Date: Mon, 23 Aug 2021 16:14:53 -0400 Message-Id: <2baa0ddabfc39080848872a7ad4841482ef985e5.1629749291.git.josef@toxicpanda.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This exists in the kernel free-space-tree.c but not in progs. We need it to generate the free space items for new block groups, which is needed when we start creating the free space tree in make_btrfs(). Signed-off-by: Josef Bacik --- kernel-shared/extent-tree.c | 2 ++ kernel-shared/free-space-tree.c | 26 ++++++++++++++++++++++++++ kernel-shared/free-space-tree.h | 2 ++ 3 files changed, 30 insertions(+) diff --git a/kernel-shared/extent-tree.c b/kernel-shared/extent-tree.c index 056a9256..b5b43c10 100644 --- a/kernel-shared/extent-tree.c +++ b/kernel-shared/extent-tree.c @@ -2816,6 +2816,8 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, ret = btrfs_insert_item(trans, extent_root, &key, &bgi, sizeof(bgi)); BUG_ON(ret); + add_block_group_free_space(trans, cache); + return 0; } diff --git a/kernel-shared/free-space-tree.c b/kernel-shared/free-space-tree.c index 7f589dfe..1e127e31 100644 --- a/kernel-shared/free-space-tree.c +++ b/kernel-shared/free-space-tree.c @@ -986,6 +986,32 @@ out: return ret; } +int add_block_group_free_space(struct btrfs_trans_handle *trans, + struct btrfs_block_group *block_group) +{ + struct btrfs_path *path; + int ret; + + if (!btrfs_fs_compat_ro(trans->fs_info, FREE_SPACE_TREE)) + return 0; + + path = btrfs_alloc_path(); + if (!path) + return -ENOMEM; + + ret = add_new_free_space_info(trans, block_group, path); + if (ret) + goto out; + ret = __add_to_free_space_tree(trans, block_group, path, + block_group->start, + block_group->length); +out: + btrfs_free_path(path); + if (ret) + btrfs_abort_transaction(trans, ret); + return ret; +} + int populate_free_space_tree(struct btrfs_trans_handle *trans, struct btrfs_block_group *block_group) { diff --git a/kernel-shared/free-space-tree.h b/kernel-shared/free-space-tree.h index 3d32e167..4f6aa5fc 100644 --- a/kernel-shared/free-space-tree.h +++ b/kernel-shared/free-space-tree.h @@ -34,5 +34,7 @@ int add_to_free_space_tree(struct btrfs_trans_handle *trans, u64 start, int remove_from_free_space_tree(struct btrfs_trans_handle *trans, u64 start, u64 size); int btrfs_create_free_space_tree(struct btrfs_fs_info *info); +int add_block_group_free_space(struct btrfs_trans_handle *trans, + struct btrfs_block_group *block_group); #endif From patchwork Mon Aug 23 20:14:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 330A7C4320E for ; Mon, 23 Aug 2021 20:15:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1C142613D0 for ; Mon, 23 Aug 2021 20:15:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232486AbhHWUP4 (ORCPT ); Mon, 23 Aug 2021 16:15:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232412AbhHWUPz (ORCPT ); Mon, 23 Aug 2021 16:15:55 -0400 Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2A6EC061575 for ; Mon, 23 Aug 2021 13:15:12 -0700 (PDT) Received: by mail-qt1-x82c.google.com with SMTP id d9so14862051qty.12 for ; Mon, 23 Aug 2021 13:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=6swAtHPoQml+AuzvipsQDom1r8+viLfzrzDFrdFwk/U=; b=2OM0SVJ/tBhrXvaw0DkLrJqfu3789uR2gQ/b+eytyjMS+k5BPQ6YBiNlSa+6Kw7M4g lzLtiYeuOx0HcaeCvtJ8jj084LE0iuVZ8JRxCcDpMjpncg1RH8HVXmhnrgi2fBbTWvfF Ot5BatAbKcSVrBNSlOQaJxye9va/JFu3HHT0TJGDdSGhCbZ8nc3qAecRfb2HCiV98c+w 72RlsKmNk0ovi+0i1NnH9+jI3gNZTRmu+epI/QWnhhl6r84ucJzms3FglOuSyupC7kBy SA4t7XQt+dlBxKERNffow+rMdUOQQgytaUisXaNrOkK0Ck8yjxsSQKOoMDEmehF1DPk2 VLrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6swAtHPoQml+AuzvipsQDom1r8+viLfzrzDFrdFwk/U=; b=jh8ItediUx4Ayav5i5Ec3uNQ8UWV2K6UB0hlM1GPh8pZ5yQ4Ct+UuqzkO719pXOQls XiD6aasSzPgoWED6aldMsjlS/QLU0pr3/Jm++Q6zkFbWkjyDbGIDGw0LwP11aDhLRrbS p+vd0B7+3iIj6+iwm6NN2JYUR+zYZb4AJF8siqjGB9/m+wZ2r2sgsrL7Xj7OyXHufv5B CRN2+Xj3/kFrShy1ZkgVK8kJ3PAGMA/0yi9NP553G40+E/r2K/9X6jOxMsqEkti3v9cj YgM9YTq8vCccN1us1Ta1BlORYXwCeb9Y+rRTfeuX0MwWrOPfiD9OUrq+FT6YH9fWsewa xeQg== X-Gm-Message-State: AOAM530nEmWcPZ5esXmZItVbZ8nvyXXI0iD3we5P//ASPhZVA2L4T30J WKKCJc9zace5YqfKjiwAm//BJQHzx2fnMQ== X-Google-Smtp-Source: ABdhPJzpC/poI2xQZcSefonr7b9ccNq4Qh+GbswqlGcTRzjWuxMu/9jdzlshM/4uA8QXxffy5m7Keg== X-Received: by 2002:ac8:5756:: with SMTP id 22mr3625689qtx.389.1629749711299; Mon, 23 Aug 2021 13:15:11 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id k186sm9433090qkd.47.2021.08.23.13.15.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:10 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 09/10] btrfs-progs: mkfs: generate free space tree at make_btrfs() time Date: Mon, 23 Aug 2021 16:14:54 -0400 Message-Id: <99b0c0bd527103c7c5ade8215d33542f75db8e36.1629749291.git.josef@toxicpanda.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org With extent-tree-v2 we won't be able to cache block groups based on the extent tree, so we need to have a valid free space tree before we open the temporary file system to finish setting the file system up. Set up the basic free space entries for our temporary system chunk if we have the free space tree enabled and stop generating the tree after the fact. Signed-off-by: Josef Bacik --- mkfs/common.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++ mkfs/common.h | 9 +++++++ mkfs/main.c | 13 +++------- 3 files changed, 82 insertions(+), 9 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index 9b5f96e3..631e6d43 100644 --- a/mkfs/common.c +++ b/mkfs/common.c @@ -37,6 +37,7 @@ static u64 reference_root_table[] = { [MKFS_DEV_TREE] = BTRFS_DEV_TREE_OBJECTID, [MKFS_FS_TREE] = BTRFS_FS_TREE_OBJECTID, [MKFS_CSUM_TREE] = BTRFS_CSUM_TREE_OBJECTID, + [MKFS_FREE_SPACE_TREE] = BTRFS_FREE_SPACE_TREE_OBJECTID, }; static int btrfs_write_empty_tree(int fd, struct btrfs_mkfs_config *cfg, @@ -139,6 +140,55 @@ static int btrfs_create_tree_root(int fd, struct btrfs_mkfs_config *cfg, return ret; } +static int create_free_space_tree(int fd, struct btrfs_mkfs_config *cfg, + struct extent_buffer *buf, u64 group_start, + u64 group_size, u64 free_start) +{ + struct btrfs_free_space_info *info; + struct btrfs_disk_key disk_key; + enum btrfs_mkfs_block blk; + int itemoff = __BTRFS_LEAF_DATA_SIZE(cfg->nodesize); + int nritems = 0; + int i = 0; + int ret; + + memset(buf->data + sizeof(struct btrfs_header), 0, + cfg->nodesize - sizeof(struct btrfs_header)); + itemoff -= sizeof(*info); + + btrfs_set_disk_key_objectid(&disk_key, group_start); + btrfs_set_disk_key_offset(&disk_key, group_size); + btrfs_set_disk_key_type(&disk_key, BTRFS_FREE_SPACE_INFO_KEY); + btrfs_set_item_key(buf, &disk_key, nritems); + btrfs_set_item_offset(buf, btrfs_item_nr(nritems), itemoff); + btrfs_set_item_size(buf, btrfs_item_nr(nritems), sizeof(*info)); + + info = btrfs_item_ptr(buf, nritems, struct btrfs_free_space_info); + btrfs_set_free_space_extent_count(buf, info, 1); + btrfs_set_free_space_flags(buf, info, 0); + + nritems++; + btrfs_set_disk_key_objectid(&disk_key, free_start); + btrfs_set_disk_key_offset(&disk_key, group_start + group_size - + free_start); + btrfs_set_disk_key_type(&disk_key, BTRFS_FREE_SPACE_EXTENT_KEY); + btrfs_set_item_key(buf, &disk_key, nritems); + btrfs_set_item_offset(buf, btrfs_item_nr(nritems), itemoff); + btrfs_set_item_size(buf, btrfs_item_nr(nritems), 0); + + nritems++; + btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_FREE_SPACE_TREE]); + btrfs_set_header_owner(buf, BTRFS_FREE_SPACE_TREE_OBJECTID); + btrfs_set_header_nritems(buf, nritems); + csum_tree_block_size(buf, btrfs_csum_type_size(cfg->csum_type), 0, + cfg->csum_type); + ret = pwrite(fd, buf->data, cfg->nodesize, + cfg->blocks[MKFS_FREE_SPACE_TREE]); + if (ret != cfg->nodesize) + return ret < 0 ? -errno : -EIO; + return 0; +} + /* * @fs_uuid - if NULL, generates a UUID, returns back the new filesystem UUID * @@ -189,6 +239,12 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) u64 system_group_offset = BTRFS_BLOCK_RESERVED_1M_FOR_SUPER; u64 system_group_size = BTRFS_MKFS_SYSTEM_GROUP_SIZE; bool add_block_group = true; + bool free_space_tree = !!(cfg->runtime_features & + BTRFS_RUNTIME_FEATURE_FREE_SPACE_TREE); + + /* Don't include the free space tree in the blocks to process. */ + if (!free_space_tree) + blocks_nr--; if ((cfg->features & BTRFS_FEATURE_INCOMPAT_ZONED)) { system_group_offset = cfg->zone_size * BTRFS_NR_SB_LOG_ZONES; @@ -248,6 +304,11 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) else btrfs_set_super_cache_generation(&super, -1); btrfs_set_super_incompat_flags(&super, cfg->features); + if (free_space_tree) { + u64 ro_flags = BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE | + BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID; + btrfs_set_super_compat_ro_flags(&super, ro_flags); + } if (cfg->label) __strncpy_null(super.label, cfg->label, BTRFS_LABEL_SIZE - 1); @@ -513,6 +574,14 @@ int make_btrfs(int fd, struct btrfs_mkfs_config *cfg) if (ret) goto out; + if (free_space_tree) { + ret = create_free_space_tree(fd, cfg, buf, system_group_offset, + system_group_size, + system_group_offset + total_used); + if (ret) + goto out; + } + /* and write out the super block */ memset(buf->data, 0, BTRFS_SUPER_INFO_SIZE); memcpy(buf->data, &super, sizeof(super)); diff --git a/mkfs/common.h b/mkfs/common.h index f2d28057..f31b4ae4 100644 --- a/mkfs/common.h +++ b/mkfs/common.h @@ -51,6 +51,7 @@ enum btrfs_mkfs_block { MKFS_DEV_TREE, MKFS_FS_TREE, MKFS_CSUM_TREE, + MKFS_FREE_SPACE_TREE, MKFS_BLOCK_COUNT }; @@ -61,6 +62,12 @@ static const enum btrfs_mkfs_block extent_tree_v1_blocks[] = { MKFS_DEV_TREE, MKFS_FS_TREE, MKFS_CSUM_TREE, + + /* + * Since the free space tree is optional with v1 it must always be last + * in this array. + */ + MKFS_FREE_SPACE_TREE, }; struct btrfs_mkfs_config { @@ -72,6 +79,8 @@ struct btrfs_mkfs_config { u32 stripesize; /* Bitfield of incompat features, BTRFS_FEATURE_INCOMPAT_* */ u64 features; + /* Bitfield of BTRFS_RUNTIME_FEATURE_* */ + u64 runtime_features; /* Size of the filesystem in bytes */ u64 num_bytes; /* checksum algorithm to use */ diff --git a/mkfs/main.c b/mkfs/main.c index ea53e9c7..edfded1f 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -137,7 +137,6 @@ static int create_metadata_block_groups(struct btrfs_root *root, int mixed, root->fs_info->system_allocs = 0; ret = btrfs_commit_transaction(trans, root); - err: return ret; } @@ -254,7 +253,9 @@ static int recow_roots(struct btrfs_trans_handle *trans, ret = __recow_root(trans, info->csum_root); if (ret) return ret; - + ret = __recow_root(trans, info->free_space_root); + if (ret) + return ret; return 0; } @@ -1366,6 +1367,7 @@ int BOX_MAIN(mkfs)(int argc, char **argv) mkfs_cfg.sectorsize = sectorsize; mkfs_cfg.stripesize = stripesize; mkfs_cfg.features = features; + mkfs_cfg.runtime_features = runtime_features; mkfs_cfg.csum_type = csum_type; mkfs_cfg.zone_size = zone_size(file); @@ -1529,13 +1531,6 @@ raid_groups: goto out; } } - if (runtime_features & BTRFS_RUNTIME_FEATURE_FREE_SPACE_TREE) { - ret = btrfs_create_free_space_tree(fs_info); - if (ret < 0) { - error("failed to create free space tree: %d (%m)", ret); - goto out; - } - } if (verbose) { char features_buf[64]; From patchwork Mon Aug 23 20:14:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 12453567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 141A2C4320A for ; Mon, 23 Aug 2021 20:15:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F0F6E613DA for ; Mon, 23 Aug 2021 20:15:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232503AbhHWUP5 (ORCPT ); Mon, 23 Aug 2021 16:15:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbhHWUP4 (ORCPT ); Mon, 23 Aug 2021 16:15:56 -0400 Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FFE7C061575 for ; Mon, 23 Aug 2021 13:15:14 -0700 (PDT) Received: by mail-qv1-xf2a.google.com with SMTP id eh1so10426077qvb.11 for ; Mon, 23 Aug 2021 13:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=byX9w/Dz6OGdoxb/SrLuWvN50ehQZMZNsu4gsfldjFM=; b=XJZ0M8t1XEvxayFf/iM7062xtfOZtuZVy1YUmZCd4626pWAgh6X8CHuvv3e3hkLzLh E0VGEXcDp/RAhOWr6S74btJPhQAFhJFCiaVp6hcd8FVGxEGAHR83ryEDZOeayRxLU2mE /Ed6wTP0IoVjMYzHe6QKuWM5fPirrAu1mINzg5W8gU9Zb5Qgj3rLb0auL9OjBCd+qTig WQf+FEQ1oqwXOLYWrebpWyuXj+l3jcUZtPuq+gtQM4nHKW1NwcPt21w05YJqzQtJO099 q20x6EkxSy9ZiqlJOOHysimc2iVCZuj9JFGVp+F67csmcS4ukndEShRmGFsD82QDGGBf HTDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=byX9w/Dz6OGdoxb/SrLuWvN50ehQZMZNsu4gsfldjFM=; b=YfzU2VEI2sRUm/+VansIIAbQDPEnxApg2IiREA1vb+TQxfjQRTPkjd73A/sua3anQr 1CdtB6TFi8n5EcpQQ1XwectZzDuAezuEMcqFXRrSTgWL4ohramnoA3srgzkl5kBOMAlX aQKGvZVDREVPbJnT+8pxQMoZKkF69eYZJ5Lwok+nKJ4u89w99w+PQD3b9Al0KTWFGlLy RwYw2x5YpH4W0pAIgiUyc7HVfsG1U0uWoNKaAnJHiVtAtwxo0JhLSeRxo7bMvmS4KHv9 /9rcXd7JWs6EIZUipL3LrL0ce9JAyIG+WKOyQewi3MrQvFliWqYiI3zO9GXDzgCI/SJk ILDw== X-Gm-Message-State: AOAM532dxUTBQX0+tT02eixleijKnTUaAbQ6H8yBG9y/Sh5dubAisAxK 7L3Nv18XaW6L4RnG+qgE6R1XCkUfyJwtrA== X-Google-Smtp-Source: ABdhPJxDtySnX40kO0yBPNVnOx4tmy6uvzCbc0rhbQmfeV6fRBDMl+iptnh8+J1eE1xnCeou7jqu+g== X-Received: by 2002:a05:6214:4009:: with SMTP id kd9mr34811326qvb.40.1629749712861; Mon, 23 Aug 2021 13:15:12 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id g20sm9056887qki.73.2021.08.23.13.15.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 13:15:12 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v2 10/10] btrfs-progs: add the incompat flag for extent tree v2 Date: Mon, 23 Aug 2021 16:14:55 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org I will have a lot of preparatory patches to reduce the review pain of this large feature. In order to enable that work define the incompat flag. Once all of the work lands to support the feature there will be a patch to actually enable us to select it and manipulate file systems with that incompat flag set. Signed-off-by: Josef Bacik --- kernel-shared/ctree.h | 1 + mkfs/main.c | 8 +++++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 3cca6032..fb918aba 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -496,6 +496,7 @@ struct btrfs_super_block { #define BTRFS_FEATURE_INCOMPAT_METADATA_UUID (1ULL << 10) #define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11) #define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12) +#define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) #define BTRFS_FEATURE_COMPAT_SUPP 0ULL diff --git a/mkfs/main.c b/mkfs/main.c index edfded1f..7ea6910e 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -253,9 +253,11 @@ static int recow_roots(struct btrfs_trans_handle *trans, ret = __recow_root(trans, info->csum_root); if (ret) return ret; - ret = __recow_root(trans, info->free_space_root); - if (ret) - return ret; + if (info->free_space_root) { + ret = __recow_root(trans, info->free_space_root); + if (ret) + return ret; + } return 0; }