From patchwork Thu Aug 17 20:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 13356945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBEB6C5479A for ; Thu, 17 Aug 2023 20:58:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355088AbjHQU62 (ORCPT ); Thu, 17 Aug 2023 16:58:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355084AbjHQU5z (ORCPT ); Thu, 17 Aug 2023 16:57:55 -0400 Received: from mail-yb1-xb2e.google.com (mail-yb1-xb2e.google.com [IPv6:2607:f8b0:4864:20::b2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAA133583 for ; Thu, 17 Aug 2023 13:57:53 -0700 (PDT) Received: by mail-yb1-xb2e.google.com with SMTP id 3f1490d57ef6-d6fcffce486so255360276.3 for ; Thu, 17 Aug 2023 13:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20221208.gappssmtp.com; s=20221208; t=1692305873; x=1692910673; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=eXzvrSqu2NVVVGOJubdrgNOnpMI1Fh11yOes1nDnPZY=; b=KCOwhSyboLWFbkZt8OFP6htCdD0xDc3bu5MfVqNjlUJys4W2AyfBdV1pmaYQGiVBMJ 7V2prqoOP907wzPp8hmLHJhObKPBTQKgHpwmCqRDYWL9I1df12+/c31iBDIqKD1iwIV3 F2j4n5MSKoBviZc1Vj3obXaYV+sBTxFKjOFhgOGw+sSZNN7MQZ7gxvNI1O4T4ayG6JjP GOAYD3Yd/aXM9OqOWlxmOVPv5aQCKLdecCqSEvTj3EtgsuQhxj6BWESeesEK9M/xiGIk 8jJfBd/jIlPpKZuDwqovnRBIWUnKhameu3bEjophUoJK5IC791OMWdW62wGdBq6GY1IM JAog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692305873; x=1692910673; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eXzvrSqu2NVVVGOJubdrgNOnpMI1Fh11yOes1nDnPZY=; b=iRzzeGE7hfvvMsUA4XTGDm/+6Qt9o6I+POswlEOtwAbheHDF/n0ZNBdLVvyyrlHcsn DjkuUx8f0/v5pYAHFMvriuEqTVnKAY8bgyzgHXyguSljzl0Sp/YqS7VubGz37AX1FcYf zhl0yBOP8MQTiWlxGYtIZqJqyfw+aiOZzm206IEVVFcK48J8bJXKWrjtjp/pL8STmND3 gkBGU/tBjkEOFE/yPWzSCksBLqPPsOAX63u8OixiL7uH+r/avUVNXpL2a4O7k7p9KqaX Rq8et0B8my7i+NpLiL6cD6O4xSfOVn1pdxrLgSB07jwkUSe+iPI99vbHKr2I811MY9Ig nGuA== X-Gm-Message-State: AOJu0YzbGX7UpGQltZk2282nMt4O1d2odfrU0f4Cb/IFDvHmHi4dXpzX +yvrWHbOIXUlst9Tp0NBpUHQI9lf+UQr7Wg6iMPY9w== X-Google-Smtp-Source: AGHT+IHo5gEIew1kXCuT4a734xvBz8hQwkp+OYszXDYbOOnaGrFCwGAMcI9HP5HQFhZBqkDW0ZNvPw== X-Received: by 2002:a0d:df0b:0:b0:589:e802:af0b with SMTP id i11-20020a0ddf0b000000b00589e802af0bmr512366ywe.39.1692305872969; Thu, 17 Aug 2023 13:57:52 -0700 (PDT) Received: from localhost (cpe-76-182-20-124.nc.res.rr.com. [76.182.20.124]) by smtp.gmail.com with ESMTPSA id i127-20020a819185000000b0058c4e33b2d6sm100064ywg.90.2023.08.17.13.57.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Aug 2023 13:57:52 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/4] btrfs: fix incorrect splitting in btrfs_drop_extent_map_range Date: Thu, 17 Aug 2023 16:57:30 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org In production we were seeing a variety of WARN_ON()'s in the extent_map code, specifically in btrfs_drop_extent_map_range() when we have to call add_extent_mapping() for our second split. Consider the following extent map layout PINNED [0 16K) [32K, 48K) and then we call btrfs_drop_extent_map_range for [0, 36K), with skip_pinned == true. The initial loop will have start = 0 end = 36K len = 36K we will find the [0, 16k) extent, but since we are pinned we will skip it, which has this cod start = em_end; if (end != (u64)-1) len = start + len - em_end; em_end here is 16K, so now the values are start = 16K len = 16K + 36K - 16K = 36K len should instead be 20K. This is a problem when we find the next extent at [32K, 48K), we need to split this extent to leave [36K, 48k), however the code for the split looks like this split->start = start + len; split->len = em_end - (start + len); In this case we have em_end = 48K split->start = 16K + 36K //this should be 16K + 20K split->len = 48K - (16K + 36K) // this overflows as 16K + 36K is 52K and now we have an invalid extent_map in the tree that potentially overlaps other entries in the extent map. Even in the non-overlapping case we will have split->start set improperly, which will cause problems with any block related calculations. We don't actually need len in this loop, we can simply use end as our end point, and only adjust start up when we find a pinned extent we need to skip. Adjust the logic to do this, which keeps us from inserting an invalid extent map. We only skip_pinned in the relocation case, so this is relatively rare, except in the case where you are running relocation a lot, which can happen with auto relocation on. Fixes: 55ef68990029 ("Btrfs: Fix btrfs_drop_extent_cache for skip pinned case") Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/extent_map.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent_map.c b/fs/btrfs/extent_map.c index 0cdb3e86f29b..a6d8368ed0ed 100644 --- a/fs/btrfs/extent_map.c +++ b/fs/btrfs/extent_map.c @@ -760,8 +760,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end, if (skip_pinned && test_bit(EXTENT_FLAG_PINNED, &em->flags)) { start = em_end; - if (end != (u64)-1) - len = start + len - em_end; goto next; } @@ -829,8 +827,8 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end, if (!split) goto remove_em; } - split->start = start + len; - split->len = em_end - (start + len); + split->start = end; + split->len = em_end - end; split->block_start = em->block_start; split->flags = flags; split->compress_type = em->compress_type; From patchwork Thu Aug 17 20:57:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 13356946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 481BFC64ECB for ; Thu, 17 Aug 2023 20:58:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355092AbjHQU6a (ORCPT ); Thu, 17 Aug 2023 16:58:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355086AbjHQU54 (ORCPT ); Thu, 17 Aug 2023 16:57:56 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9AA630F6 for ; Thu, 17 Aug 2023 13:57:54 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-58cbdf3eecaso2689027b3.0 for ; Thu, 17 Aug 2023 13:57:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20221208.gappssmtp.com; s=20221208; t=1692305874; x=1692910674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=89qlipFYPHBP3W+xLufglLgFiGWCyiW+0/tCA16wQAA=; b=TUASaqeo16AxmCxJQbcuyg/omEPjrWwkeRpIeZL35gNm5/CptsnlUPGy4IsCC8wWcu B43kZxa9zhKKbZA8ABVn5WVrtGTqGOY5OR+fSKqeOnj641/yexrm4pnWN6/l09LrEErn yGlkusU9+rocZmZKkfSNNoU950O/UYnI872watHg4c9dr19okz/b7mYR3iEbcnyLbpKQ CV7sInUjuy/Ch00qhv0eZsRl/tSlIOx4Q9sgq3vU9lDbIiP4tfNeKNZnl4h1QqoSIj2b Z6NcPnzulCjDK8eY/kjGRiXI+rrUoCSmc/Ao8p5Zd/bVFiYgiqZrduilhOQFA91TjP9r hPTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692305874; x=1692910674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=89qlipFYPHBP3W+xLufglLgFiGWCyiW+0/tCA16wQAA=; b=KO28aQjM9H5UcRwGnV0ZHdzaMLOfWD45MQnfUeVPFKHQySk8hs4msMklOB6Dqkl+sq 0+URKv9hv+sw1URgDWPIoXgIzu3kcARV0k2r0aZjAqIdADPMGbLcSLI1/0zhefp0vHUy zMBQ62JO2U68vKDi2Y+MuI/W3eK6M4CRwxsUdIQS3k5CG0R9E/2dFE3FlxoA1yReJMqZ 4XsMH+umuSLdOFb803EeFnZ3wXqvFosq20512BeIu7L/HBt0vOiGUdWWeGHmlzJeGNFf 6UElAc2dDrWKWFMg21Olivd7Jh9/1dp3dQokPWKIvtg4KgBWOcgmB/v2EKR4lojqm1ma ge+w== X-Gm-Message-State: AOJu0YySYOXVLRaxmlRkkiCadtOj/pPp3WX+u2r14x5XzBG+QwYdzjWP z83OH1m9CK+IZZB7Lay7vm3xwbrhgUbfNloPgMN9BA== X-Google-Smtp-Source: AGHT+IHQP2rNWdzMfzakx/+2mtxPNq693T8By7xhaSd4leBmomrl/ilXqkXdcSfeEuawdgB0Gfa9tA== X-Received: by 2002:a81:658b:0:b0:565:ba4e:84fc with SMTP id z133-20020a81658b000000b00565ba4e84fcmr424724ywb.50.1692305874019; Thu, 17 Aug 2023 13:57:54 -0700 (PDT) Received: from localhost (cpe-76-182-20-124.nc.res.rr.com. [76.182.20.124]) by smtp.gmail.com with ESMTPSA id d3-20020a81ab43000000b00573898fb12bsm99823ywk.82.2023.08.17.13.57.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Aug 2023 13:57:53 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 2/4] btrfs: add extent_map tests for dropping with odd layouts Date: Thu, 17 Aug 2023 16:57:31 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org While investigating weird problems with the extent_map I wrote a self test testing the various edge cases of btrfs_drop_extent_map_range. This can split in different ways and behaves different in each case, so test the various edge cases to make sure everything is functioning properly. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/tests/extent-map-tests.c | 219 ++++++++++++++++++++++++++++++ 1 file changed, 219 insertions(+) diff --git a/fs/btrfs/tests/extent-map-tests.c b/fs/btrfs/tests/extent-map-tests.c index ed0f36ae5346..d5f5e48ab55c 100644 --- a/fs/btrfs/tests/extent-map-tests.c +++ b/fs/btrfs/tests/extent-map-tests.c @@ -6,6 +6,7 @@ #include #include "btrfs-tests.h" #include "../ctree.h" +#include "../btrfs_inode.h" #include "../volumes.h" #include "../disk-io.h" #include "../block-group.h" @@ -442,6 +443,219 @@ static int test_case_4(struct btrfs_fs_info *fs_info, return ret; } +static int add_compressed_extent(struct extent_map_tree *em_tree, + const u64 start, const u64 len, + const u64 block_start) +{ + struct extent_map *em; + int ret; + + em = alloc_extent_map(); + if (!em) { + test_std_err(TEST_ALLOC_EXTENT_MAP); + return -ENOMEM; + } + + em->start = start; + em->len = len; + em->block_start = block_start; + em->block_len = SZ_4K; + set_bit(EXTENT_FLAG_COMPRESSED, &em->flags); + write_lock(&em_tree->lock); + ret = add_extent_mapping(em_tree, em, 0); + write_unlock(&em_tree->lock); + free_extent_map(em); + if (ret < 0) { + test_err("cannot add extent map [%llu, %llu)", start, + start + len); + return ret; + } + + return 0; +} + +struct extent_range { + u64 start; + u64 len; +}; + +/* The valid states of the tree after every drop, as described below. */ +struct extent_range valid_ranges[][7] = { + { + { .start = 0, .len = SZ_8K }, /* [0, 8K) */ + { .start = SZ_4K * 3, .len = SZ_4K * 3}, /* [12k, 24k) */ + { .start = SZ_4K * 6, .len = SZ_4K * 3}, /* [24k, 36k) */ + { .start = SZ_32K + SZ_4K, .len = SZ_4K}, /* [36k, 40k) */ + { .start = SZ_4K * 10, .len = SZ_4K * 6}, /* [40k, 64k) */ + }, + { + { .start = 0, .len = SZ_8K }, /* [0, 8K) */ + { .start = SZ_4K * 5, .len = SZ_4K}, /* [20k, 24k) */ + { .start = SZ_4K * 6, .len = SZ_4K * 3}, /* [24k, 36k) */ + { .start = SZ_32K + SZ_4K, .len = SZ_4K}, /* [36k, 40k) */ + { .start = SZ_4K * 10, .len = SZ_4K * 6}, /* [40k, 64k) */ + }, + { + { .start = 0, .len = SZ_8K }, /* [0, 8K) */ + { .start = SZ_4K * 5, .len = SZ_4K}, /* [20k, 24k) */ + { .start = SZ_4K * 6, .len = SZ_4K}, /* [24k, 28k) */ + { .start = SZ_32K, .len = SZ_4K}, /* [32k, 36k) */ + { .start = SZ_32K + SZ_4K, .len = SZ_4K}, /* [36k, 40k) */ + { .start = SZ_4K * 10, .len = SZ_4K * 6}, /* [40k, 64k) */ + }, + { + { .start = 0, .len = SZ_8K}, /* [0, 8K) */ + { .start = SZ_4K * 5, .len = SZ_4K}, /* [20k, 24k) */ + { .start = SZ_4K * 6, .len = SZ_4K}, /* [24k, 28k) */ + } +}; + +static int validate_range(struct extent_map_tree *em_tree, int index) +{ + struct rb_node *n; + int i; + + for (i = 0, n = rb_first_cached(&em_tree->map); + valid_ranges[index][i].len && n; i++, n = rb_next(n)) { + struct extent_map *entry = rb_entry(n, struct extent_map, rb_node); + + if (entry->start != valid_ranges[index][i].start) { + test_err("mapping has start %llu expected %llu", + entry->start, valid_ranges[index][i].start); + return -EINVAL; + } + + if (entry->len != valid_ranges[index][i].len) { + test_err("mapping has len %llu expected %llu", + entry->len, valid_ranges[index][i].len); + return -EINVAL; + } + } + + /* + * We exited because we don't have any more entries in the extent_map + * but we still expect more valid entries. + */ + if (valid_ranges[index][i].len) { + test_err("missing an entry"); + return -EINVAL; + } + + /* We exited the loop but still have entries in the extent map. */ + if (n) { + test_err("we have a left over entry in the extent map we didn't expect"); + return -EINVAL; + } + + return 0; +} + +/* + * Test scenario: + * + * Test the various edge cases of btrfs_drop_extent_map_range, create the + * following ranges + * + * [0, 12k)[12k, 24k)[24k, 36k)[36k, 40k)[40k,64k) + * + * And then we'll drop: + * + * [8k, 12k) - test the single front split + * [12k, 20k) - test the single back split + * [28k, 32k) - test the double split + * [32k, 64k) - test whole em dropping + * + * They'll have the EXTENT_FLAG_COMPRESSED flag set to keep the em tree from + * merging the em's. + */ +static int test_case_5(void) +{ + struct extent_map_tree *em_tree; + struct inode *inode = NULL; + u64 start, end; + int ret; + + test_msg("Running btrfs_drop_extent_map_range tests"); + + inode = btrfs_new_test_inode(); + if (!inode) { + test_std_err(TEST_ALLOC_INODE); + return -ENOMEM; + } + + em_tree = &BTRFS_I(inode)->extent_tree; + + /* [0, 12k) */ + ret = add_compressed_extent(em_tree, 0, SZ_4K * 3, 0); + if (ret) { + test_err("cannot add extent range [0, 12K)"); + goto out; + } + + /* [12k, 24k) */ + ret = add_compressed_extent(em_tree, SZ_4K * 3, SZ_4K * 3, SZ_4K); + if (ret) { + test_err("cannot add extent range [12k, 24k)"); + goto out; + } + + /* [24k, 36k) */ + ret = add_compressed_extent(em_tree, SZ_4K * 6, SZ_4K * 3, SZ_8K); + if (ret) { + test_err("cannot add extent range [12k, 24k)"); + goto out; + } + + /* [36k, 40k) */ + ret = add_compressed_extent(em_tree, SZ_32K + SZ_4K, SZ_4K, SZ_4K * 3); + if (ret) { + test_err("cannot add extent range [12k, 24k)"); + goto out; + } + + /* [40k, 64k) */ + ret = add_compressed_extent(em_tree, SZ_4K * 10, SZ_4K * 6, SZ_16K); + if (ret) { + test_err("cannot add extent range [12k, 24k)"); + goto out; + } + + /* Drop [8k, 12k) */ + start = SZ_8K; + end = (3 * SZ_4K) - 1; + btrfs_drop_extent_map_range(BTRFS_I(inode), start, end, false); + ret = validate_range(&BTRFS_I(inode)->extent_tree, 0); + if (ret) + goto out; + + /* Drop [12k, 20k) */ + start = SZ_4K * 3; + end = SZ_16K + SZ_4K - 1; + btrfs_drop_extent_map_range(BTRFS_I(inode), start, end, false); + ret = validate_range(&BTRFS_I(inode)->extent_tree, 1); + if (ret) + goto out; + + /* Drop [28k, 32k) */ + start = SZ_32K - SZ_4K; + end = SZ_32K - 1; + btrfs_drop_extent_map_range(BTRFS_I(inode), start, end, false); + ret = validate_range(&BTRFS_I(inode)->extent_tree, 2); + if (ret) + goto out; + + /* Drop [32k, 64k) */ + start = SZ_32K; + end = SZ_64K - 1; + btrfs_drop_extent_map_range(BTRFS_I(inode), start, end, false); + ret = validate_range(&BTRFS_I(inode)->extent_tree, 3); + if (ret) + goto out; +out: + iput(inode); + return ret; +} + struct rmap_test_vector { u64 raid_type; u64 physical_start; @@ -619,6 +833,11 @@ int btrfs_test_extent_map(void) if (ret) goto out; ret = test_case_4(fs_info, em_tree); + if (ret) + goto out; + ret = test_case_5(); + if (ret) + goto out; test_msg("running rmap tests"); for (i = 0; i < ARRAY_SIZE(rmap_tests); i++) { From patchwork Thu Aug 17 20:57:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 13356944 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2960EC64EBC for ; Thu, 17 Aug 2023 20:58:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355091AbjHQU6a (ORCPT ); Thu, 17 Aug 2023 16:58:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355087AbjHQU55 (ORCPT ); Thu, 17 Aug 2023 16:57:57 -0400 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0775F3589 for ; Thu, 17 Aug 2023 13:57:55 -0700 (PDT) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-58d31f142eeso2725997b3.0 for ; Thu, 17 Aug 2023 13:57:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20221208.gappssmtp.com; s=20221208; t=1692305875; x=1692910675; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=EFsoj2bGHuawzfa+zUhKvIIj9PfIvsu28Gen0/mSx70=; b=U2tR5hn/LI1Xhws11RPv2sVVDeidRq12ZbaXBEsx2t0mayNOt41LzHC0IjF5YbOkyo pNUVC+ZEGaoIG3CgsCxUZphc93mX+iW8idZnyuNjV+MblXr3yFrWhZ/+1O1Izza9cvUK 2JlXP4NKIj0xh81sE8G9iOGwLd4bLsryPMN5XjsQdIebuP6gv+Agy7GjJeNZR70leqxy tmkeXHtZgA+SzgP1JztQo5wQjTTOo2OR5bTEjvuYOpwEErVE/vuWZv2+4IVT7jUcZeEW 3yjA0x+UCguKqir7XKofPEKDnUqNYWnOJRggqEuvzOGmuZ7t+quq2AZFSj48m205f2w0 lhdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692305875; x=1692910675; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EFsoj2bGHuawzfa+zUhKvIIj9PfIvsu28Gen0/mSx70=; b=V2nOw+cT+5d750pIyAoVu/NH+GGSvj0Q+kIc5nZItSn1ClyL2dVvkIjE/t0rMZ/wuB QohjbytA5815d07QASsMZ0W6/SzUiKrra8rfLZwn5pzAvZ+D8J+2IkpzlRcsYiM+YLuC dvA7xCnBtMj7u84UOZKAvpD1hEt+rMbGQPBPsYzEf7IBgfSdVuZekX0cY2+rFbpBvUX2 YPh46Cetnt6h4zdNg25gevczq7sxWXQyC8Eo37swA0jrwDr53nt84/U8yGojH5Up3GLU nWKD9SlMYrGs4azslXZ+S9MMs3ETxw3Zg2v/jZ7MWuWW7HztthelbQpmWSM4kMGVdtHZ MJJA== X-Gm-Message-State: AOJu0YwUytFYxn7zyhbdzN2qb2PQDYxi3/wBcjiQeIWByET/N0WeLWnv gmtMFBIDfnm0gB9vDNvm8kD+pinz9KIOldyubIQUpA== X-Google-Smtp-Source: AGHT+IGfeCFW8V6DfqAFPxL1xzAuz77MY1iJS3ET+uNl6DvD93rYAeAdjYko6ViH9qqiKLiAYBH+FA== X-Received: by 2002:a0d:ca82:0:b0:577:4387:197c with SMTP id m124-20020a0dca82000000b005774387197cmr508502ywd.16.1692305875073; Thu, 17 Aug 2023 13:57:55 -0700 (PDT) Received: from localhost (cpe-76-182-20-124.nc.res.rr.com. [76.182.20.124]) by smtp.gmail.com with ESMTPSA id j195-20020a8192cc000000b00576c727498dsm99318ywg.92.2023.08.17.13.57.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Aug 2023 13:57:54 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 3/4] btrfs: add a self test for btrfs_add_extent_mapping Date: Thu, 17 Aug 2023 16:57:32 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This helper is different from the normal add_extent_mapping in that it will stuff an em into a gap that exists between overlapping em's in the tree. It appeared there was a bug so I wrote a self test to validate it did the correct thing when it worked with two side by side ems. Thankfully it is correct, but more testing is better. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/tests/extent-map-tests.c | 57 +++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/fs/btrfs/tests/extent-map-tests.c b/fs/btrfs/tests/extent-map-tests.c index d5f5e48ab55c..18ab03f0d029 100644 --- a/fs/btrfs/tests/extent-map-tests.c +++ b/fs/btrfs/tests/extent-map-tests.c @@ -656,6 +656,60 @@ static int test_case_5(void) return ret; } +/* + * Test the btrfs_add_extent_mapping helper which will attempt to create an em + * for areas between two existing ems. Validate it doesn't do this when there + * are two unmerged em's side by side. + */ +static int test_case_6(struct btrfs_fs_info *fs_info, + struct extent_map_tree *em_tree) +{ + struct extent_map *em = NULL; + int ret; + + ret = add_compressed_extent(em_tree, 0, SZ_4K, 0); + if (ret) + goto out; + + ret = add_compressed_extent(em_tree, SZ_4K, SZ_4K, 0); + if (ret) + goto out; + + em = alloc_extent_map(); + if (!em) { + test_std_err(TEST_ALLOC_EXTENT_MAP); + return -ENOMEM; + } + + em->start = SZ_4K; + em->len = SZ_4K; + em->block_start = SZ_16K; + em->block_len = SZ_16K; + write_lock(&em_tree->lock); + ret = btrfs_add_extent_mapping(fs_info, em_tree, &em, 0, SZ_8K); + write_unlock(&em_tree->lock); + + if (ret != 0) { + test_err("got an error when adding our em: %d", ret); + goto out; + } + + ret = -EINVAL; + if (em->start != 0) { + test_err("unexpected em->start at %llu, wanted 0", em->start); + goto out; + } + if (em->len != SZ_4K) { + test_err("unexpected em->len %llu, expected 4K", em->len); + goto out; + } + ret = 0; +out: + free_extent_map(em); + free_extent_map_tree(em_tree); + return ret; +} + struct rmap_test_vector { u64 raid_type; u64 physical_start; @@ -836,6 +890,9 @@ int btrfs_test_extent_map(void) if (ret) goto out; ret = test_case_5(); + if (ret) + goto out; + ret = test_case_6(fs_info, em_tree); if (ret) goto out; From patchwork Thu Aug 17 20:57:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 13356948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FFE8C64EC0 for ; Thu, 17 Aug 2023 20:58:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355095AbjHQU6b (ORCPT ); Thu, 17 Aug 2023 16:58:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355093AbjHQU56 (ORCPT ); Thu, 17 Aug 2023 16:57:58 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F182E3589 for ; Thu, 17 Aug 2023 13:57:56 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-58c68c79befso2506137b3.3 for ; Thu, 17 Aug 2023 13:57:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20221208.gappssmtp.com; s=20221208; t=1692305876; x=1692910676; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=zjkIkgd6of4hrmXqZfl3fsYYxP4DFEWGRKCuP21YO+E=; b=nNs4YPAWLbGSvu5cl7e+RYDwfYcSyouPLErEFVfwifqnmCXXj51hwfm0Lw9FDgdpch FJxtuf679tbw+kHRKhwKg4bnC+IMYd3lG8a+fhc17pJx+sMZGPJGUmgyGOtGMW8YEDzG RfSpvdcAQr+66ppwwLXPpUgO8jyK6HqjjUvQXmTaNt2zethzxpO6yXylNYz5L+Ng4Dtl 7Kg12IxF47BzasqyEOlZa89Y8bZfftKfBT2hegvrP11Wg7WHvWzV4pdBYfnj1L9kvqbK XVNODo8t7zn+ccMIKvG5MBYacOnJM6CqINP4E2yeEmALsZEFUCqI1dW+KSMv6hobTtR1 6yrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692305876; x=1692910676; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zjkIkgd6of4hrmXqZfl3fsYYxP4DFEWGRKCuP21YO+E=; b=QP2fwIz9m5GIxb/3H6QIqe1LyjcIAFf1qbx2qkUIX6+d7+5YHAvc5OEGTjiy+pNEkr AHIqrl00U0yzYW9SDkPBhywVT+bRMLJm4Oh1nvplHiZ72j3JPQtAAvULbPxaQmjm2eiL bD0wPr107N6GDGVHCv1QkQ6b4roDZibgzs3Nk3ZRzA+aZwJU1BXS6WCXfRsrSzkWxLkh +GBfecyc1UGBQgtfnIa14hcCYdRypqVHTD9wWHEErnERAGthH4juJ06xA1TBv1l9nmcG sLAHrma6j2O7hnbaPyqnsb5qUn0XscExrhRil7UFUb8IaRmkMX7lQIBVyCs+8g3m1TOG RLzA== X-Gm-Message-State: AOJu0Yz5RMpSjY1gO7Vg/zHmHInnAGMGGKLN24Z8ksqnF9fLPkv3Ht7Y ZzVB1VlAonySWKUMXRMYFWee2HhjKw/GGagqIdoGjg== X-Google-Smtp-Source: AGHT+IFr8CC3JuN7BKnFaxPcx9MOVkbUZl710mjLmOj+KWsbnAjpGOe/q2+CX3v+ScEt1Y6W1QiIQQ== X-Received: by 2002:a81:83cb:0:b0:589:f4ec:4d51 with SMTP id t194-20020a8183cb000000b00589f4ec4d51mr587512ywf.3.1692305876035; Thu, 17 Aug 2023 13:57:56 -0700 (PDT) Received: from localhost (cpe-76-182-20-124.nc.res.rr.com. [76.182.20.124]) by smtp.gmail.com with ESMTPSA id q7-20020a0dce07000000b005772abf6234sm105901ywd.11.2023.08.17.13.57.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Aug 2023 13:57:55 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 4/4] btrfs: test invalid splitting when skipping pinned drop extent_map Date: Thu, 17 Aug 2023 16:57:33 -0400 Message-Id: X-Mailer: git-send-email 2.26.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This reproduces the bug fixed by "btrfs: fix incorrect splitting in btrfs_drop_extent_map_range", we were improperly calculating the range for the split extent. Add a test that exercises this scenario and validates that we get the correct resulting extent_maps in our tree. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/tests/extent-map-tests.c | 138 ++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) diff --git a/fs/btrfs/tests/extent-map-tests.c b/fs/btrfs/tests/extent-map-tests.c index 18ab03f0d029..06820a8b4d1f 100644 --- a/fs/btrfs/tests/extent-map-tests.c +++ b/fs/btrfs/tests/extent-map-tests.c @@ -710,6 +710,141 @@ static int test_case_6(struct btrfs_fs_info *fs_info, return ret; } +/* + * Regression test for btrfs_drop_extent_map_range. Calling with skip_pinned == + * true would mess up the start/end calculations and subsequent splits would be + * incorrect. + */ +static int test_case_7(void) +{ + struct extent_map_tree *em_tree; + struct extent_map *em = NULL; + struct inode *inode = NULL; + int ret; + + test_msg("Running btrfs_drop_extent_cache with pinned"); + + inode = btrfs_new_test_inode(); + if (!inode) { + test_std_err(TEST_ALLOC_INODE); + return -ENOMEM; + } + + em_tree = &BTRFS_I(inode)->extent_tree; + + em = alloc_extent_map(); + if (!em) { + test_std_err(TEST_ALLOC_EXTENT_MAP); + ret = -ENOMEM; + goto out; + } + + /* [0, 16K), pinned */ + em->start = 0; + em->len = SZ_16K; + em->block_start = 0; + em->block_len = SZ_4K; + set_bit(EXTENT_FLAG_PINNED, &em->flags); + write_lock(&em_tree->lock); + ret = add_extent_mapping(em_tree, em, 0); + write_unlock(&em_tree->lock); + if (ret < 0) { + test_err("couldn't add extent map"); + goto out; + } + free_extent_map(em); + + em = alloc_extent_map(); + if (!em) { + test_std_err(TEST_ALLOC_EXTENT_MAP); + ret = -ENOMEM; + goto out; + } + + /* [32K, 48K), not pinned */ + em->start = SZ_32K; + em->len = SZ_16K; + em->block_start = SZ_32K; + em->block_len = SZ_16K; + write_lock(&em_tree->lock); + ret = add_extent_mapping(em_tree, em, 0); + write_unlock(&em_tree->lock); + if (ret < 0) { + test_err("couldn't add extent map"); + goto out; + } + free_extent_map(em); + + /* + * Drop [0, 36K) This should skip the [0, 4K) extent and then split the + * [32K, 48K) extent. + */ + btrfs_drop_extent_map_range(BTRFS_I(inode), 0, (36 * SZ_1K) - 1, true); + + /* Make sure our extent maps look sane. */ + ret = -EINVAL; + + em = lookup_extent_mapping(em_tree, 0, SZ_16K); + if (!em) { + test_err("didn't find an em at 0 as expected"); + goto out; + } + + if (em->start != 0) { + test_err("em->start is %llu, expected 0", em->start); + goto out; + } + + if (em->len != SZ_16K) { + test_err("em->len is %llu, expected 16K", em->len); + goto out; + } + + free_extent_map(em); + + read_lock(&em_tree->lock); + em = lookup_extent_mapping(em_tree, SZ_16K, SZ_16K); + read_unlock(&em_tree->lock); + if (em) { + test_err("found an em when we weren't expecting one"); + goto out; + } + + read_lock(&em_tree->lock); + em = lookup_extent_mapping(em_tree, SZ_32K, SZ_16K); + read_unlock(&em_tree->lock); + if (!em) { + test_err("didn't find an em at 32K as expected"); + goto out; + } + + if (em->start != (36 * SZ_1K)) { + test_err("em->start is %llu, expected 36K", em->start); + goto out; + } + + if (em->len != (12 * SZ_1K)) { + test_err("em->len is %llu, expected 12K", em->len); + goto out; + } + + free_extent_map(em); + + read_lock(&em_tree->lock); + em = lookup_extent_mapping(em_tree, 48 * SZ_1K, (u64)-1); + read_unlock(&em_tree->lock); + if (em) { + test_err("found an unexpected em above 48K"); + goto out; + } + + ret = 0; +out: + free_extent_map(em); + iput(inode); + return ret; +} + struct rmap_test_vector { u64 raid_type; u64 physical_start; @@ -893,6 +1028,9 @@ int btrfs_test_extent_map(void) if (ret) goto out; ret = test_case_6(fs_info, em_tree); + if (ret) + goto out; + ret = test_case_7(); if (ret) goto out;