From patchwork Fri Sep 28 11:17:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0BA96CB for ; Fri, 28 Sep 2018 11:18:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E83112A23B for ; Fri, 28 Sep 2018 11:18:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC94C2B0C5; Fri, 28 Sep 2018 11:18:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 791DB2A23B for ; Fri, 28 Sep 2018 11:18:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729291AbeI1Rlo (ORCPT ); Fri, 28 Sep 2018 13:41:44 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:39798 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rlo (ORCPT ); Fri, 28 Sep 2018 13:41:44 -0400 Received: by mail-qk1-f193.google.com with SMTP id q5-v6so594343qki.6 for ; Fri, 28 Sep 2018 04:18:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=J1nvikXF93vBk79skKC0f+gf8yb5S1jfySvhFcqnzF4=; b=wWUsjHTVUEDPQWaWG3bewmxCA68ELsH9XNmeh40ax8g0ZaqTd97KgwN+HShQ418sB8 1RBWOmVA2e0jP1NoVpj7PBBDEJbKgAfcErLwO5vNHd9ij9WSIDCE830CPKpbVoJXFEVf yzpZ5zJ7pNYa9WK5PxKig/hOMG9nPY6+myBlOyQdyfwDWrDLtMOshIkjmluiAsGro48O eIxNj7mLZ7I7iTWFmH767+wKhDq+TSwwBhX9ecyDno3UEJLENlhtKfH0CMZUwg3ADYa1 GV1JZ449mAguIOtSnrxubqZlrZnj1M93aP0BTHgM9aK8CAXYs2YF9b//hUIisQEGrh6f +mEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=J1nvikXF93vBk79skKC0f+gf8yb5S1jfySvhFcqnzF4=; b=M3gutYRsfLMLL2rMLERrbyi0dknXSN0JOXt5g2K0b3j793QNClh9kOpVo8Km84zRgM S0wJcR8YfP/sU9ch4GazGJ4IY6Jx9WFzawbLy6UNt26hdtFlHjtRTPn6hGpIZNOBFLG0 VYEAxUCQqhYmif4Q6+kyVXjQLlwU/zKgQlT9BgxNFcpgXx/PrvoX96GDEkSK6WQHvXSo VZBPc2dq/4gEZJ33kePH5Pj7t2pfWSi9jMFaNDZ1z+sJr5JXAskrHChXFL7BPcaUBn3H M/ZXHWKt3BtSveukeJpGycLjr31c+G+XNRyfQTD5IYFBVlxPTYWTF8NPhyxipNK7vxw3 gY3w== X-Gm-Message-State: ABuFfojXLp/LoE1gxzi5HI6DEotCSNQsD5CzlLV48voYWChEyjcRqww7 ewLw3lLtB/fgfqwOXSPKZTMpfA== X-Google-Smtp-Source: ACcGV61bW8cet4Ah/UC0zzu1+TOj4IjOM5QUBA7xwkcwUM6HGjm1AATxRXdOJqKd9h/QW4MX0zqwhA== X-Received: by 2002:a37:9d82:: with SMTP id g124-v6mr11001895qke.19.1538133506926; Fri, 28 Sep 2018 04:18:26 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id j5-v6sm2856722qtb.34.2018.09.28.04.18.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:25 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 01/42] btrfs: add btrfs_delete_ref_head helper Date: Fri, 28 Sep 2018 07:17:40 -0400 Message-Id: <20180928111821.24376-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We do this dance in cleanup_ref_head and check_ref_cleanup, unify it into a helper and cleanup the calling functions. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/delayed-ref.c | 14 ++++++++++++++ fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/extent-tree.c | 22 +++------------------- 3 files changed, 19 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 62ff545ba1f7..3a9e4ac21794 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -393,6 +393,20 @@ btrfs_select_ref_head(struct btrfs_trans_handle *trans) return head; } +void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head) +{ + lockdep_assert_held(&delayed_refs->lock); + lockdep_assert_held(&head->lock); + + rb_erase(&head->href_node, &delayed_refs->href_root); + RB_CLEAR_NODE(&head->href_node); + atomic_dec(&delayed_refs->num_entries); + delayed_refs->num_heads--; + if (head->processing == 0) + delayed_refs->num_heads_ready--; +} + /* * Helper to insert the ref_node to the tail or merge with tail. * diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index d9f2a4ebd5db..7769177b489e 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -261,7 +261,8 @@ static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) { mutex_unlock(&head->mutex); } - +void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head); struct btrfs_delayed_ref_head * btrfs_select_ref_head(struct btrfs_trans_handle *trans); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f77226d8020a..d24a0de4a2e7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2492,12 +2492,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, spin_unlock(&delayed_refs->lock); return 1; } - delayed_refs->num_heads--; - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); + btrfs_delete_ref_head(delayed_refs, head); spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); - atomic_dec(&delayed_refs->num_entries); trace_run_delayed_ref_head(fs_info, head, 0); @@ -6984,22 +6981,9 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (!mutex_trylock(&head->mutex)) goto out; - /* - * at this point we have a head with no other entries. Go - * ahead and process it. - */ - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); - atomic_dec(&delayed_refs->num_entries); - - /* - * we don't take a ref on the node because we're removing it from the - * tree, so we just steal the ref the tree was holding. - */ - delayed_refs->num_heads--; - if (head->processing == 0) - delayed_refs->num_heads_ready--; + btrfs_delete_ref_head(delayed_refs, head); head->processing = 0; + spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); From patchwork Fri Sep 28 11:17:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619667 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 471DF15A7 for ; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 386952B061 for ; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2CCFD2B0C5; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1E3C2A23B for ; Fri, 28 Sep 2018 11:18:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729310AbeI1Rlr (ORCPT ); Fri, 28 Sep 2018 13:41:47 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:41521 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rlq (ORCPT ); Fri, 28 Sep 2018 13:41:46 -0400 Received: by mail-qk1-f194.google.com with SMTP id n3-v6so3553727qkn.8 for ; Fri, 28 Sep 2018 04:18:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/1YdvlrrEVlQm7E7WHAhXkpqC+TBu9F+Y13pNbn/ob8=; b=ZPXTzPaEV5RZSSoz70DOWlnUZQryPC+fWLI7dgyW6DKSQYeyjUeBn4ap7tr+wkCBw6 3yANyBWwMCGivBwVTV0KSP1KYfj//QTZU0FsNLLlwgU00kQcb+4Ro+ZUPVcEsMBLUETy KeoPs1OCeq4AtNPaD+kzt3eg2f+/bKHdVCBYcWYdh+rQUI+7hNFT7ndV6oHY5/WjPXTb WCrElZ8YgVYCEBqvy5GSYfR59HWK1EViy2VWwsbTyW1jDV+ffXoSMppRYGKrz9IVaEfu IG++T6FrGrQsZI+3PzZ0LHldS7bmVQVJ6jOu9Zb3mAdEBB7r94w5RB0k1PUMdr1CWKN2 An9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/1YdvlrrEVlQm7E7WHAhXkpqC+TBu9F+Y13pNbn/ob8=; b=XEKKJ8RX+8nA2rhBZ4DHQyY6tG5LjbUfNpqXvMraeRs9afnqBLcy8kr/HvivhmPwr7 T2DYL9LBED+qNEWiGtWZx5oAIda8iDirRZo8MPi5P2sVkq623ixpV59zXZQvwDVdkjvf bszsPVG8rA+uyXxh0VFdi9Q/LCb9zuEOjteUuAYuLsPtJNL3yrCwDd/dNabUT7qilqob 9m4Y0iMhntD/+etV9RNc9tUBBpZsSq4iRi4coOGeFh5Jh1L8Qv2eEyXdPYVorgOIq9Tc RAjDVUk3+eVQX0F/LYxBACI8BAEImNMp1LUFLhCkRZAlAxZ40gubqgg5ybN8gbWNz3gI zvdw== X-Gm-Message-State: ABuFfogkEIn45dTWIwPkFYJT1VxCDqos3bNSqTaYOIiCzvPQqFRRf3Iv 27k3kKZm/DbMAuMiQR/Umbj3Q745YSg= X-Google-Smtp-Source: ACcGV601YtEqtC5St3I08nwDTzyDonSSwUedHIE48N1SocE8zD/4CCdAic5kC004c8Je0f398+ZOXw== X-Received: by 2002:a37:f8c:: with SMTP id 12-v6mr11476719qkp.224.1538133508660; Fri, 28 Sep 2018 04:18:28 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u36-v6sm2926725qtc.46.2018.09.28.04.18.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:27 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 02/42] btrfs: add cleanup_ref_head_accounting helper Date: Fri, 28 Sep 2018 07:17:41 -0400 Message-Id: <20180928111821.24376-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We were missing some quota cleanups in check_ref_cleanup, so break the ref head accounting cleanup into a helper and call that from both check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that we don't screw up accounting in the future for other things that we add. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 67 +++++++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index d24a0de4a2e7..a44d55e36e11 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2461,6 +2461,41 @@ static int cleanup_extent_op(struct btrfs_trans_handle *trans, return ret ? ret : 1; } +static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + + if (head->total_ref_mod < 0) { + struct btrfs_space_info *space_info; + u64 flags; + + if (head->is_data) + flags = BTRFS_BLOCK_GROUP_DATA; + else if (head->is_system) + flags = BTRFS_BLOCK_GROUP_SYSTEM; + else + flags = BTRFS_BLOCK_GROUP_METADATA; + space_info = __find_space_info(fs_info, flags); + ASSERT(space_info); + percpu_counter_add_batch(&space_info->total_bytes_pinned, + -head->num_bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH); + + if (head->is_data) { + spin_lock(&delayed_refs->lock); + delayed_refs->pending_csums -= head->num_bytes; + spin_unlock(&delayed_refs->lock); + } + } + + /* Also free its reserved qgroup space */ + btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, + head->qgroup_reserved); +} + static int cleanup_ref_head(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head *head) { @@ -2496,31 +2531,6 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); - trace_run_delayed_ref_head(fs_info, head, 0); - - if (head->total_ref_mod < 0) { - struct btrfs_space_info *space_info; - u64 flags; - - if (head->is_data) - flags = BTRFS_BLOCK_GROUP_DATA; - else if (head->is_system) - flags = BTRFS_BLOCK_GROUP_SYSTEM; - else - flags = BTRFS_BLOCK_GROUP_METADATA; - space_info = __find_space_info(fs_info, flags); - ASSERT(space_info); - percpu_counter_add_batch(&space_info->total_bytes_pinned, - -head->num_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - - if (head->is_data) { - spin_lock(&delayed_refs->lock); - delayed_refs->pending_csums -= head->num_bytes; - spin_unlock(&delayed_refs->lock); - } - } - if (head->must_insert_reserved) { btrfs_pin_extent(fs_info, head->bytenr, head->num_bytes, 1); @@ -2530,9 +2540,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } - /* Also free its reserved qgroup space */ - btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, - head->qgroup_reserved); + cleanup_ref_head_accounting(trans, head); + + trace_run_delayed_ref_head(fs_info, head, 0); btrfs_delayed_ref_unlock(head); btrfs_put_delayed_ref_head(head); return 0; @@ -6991,6 +7001,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (head->must_insert_reserved) ret = 1; + cleanup_ref_head_accounting(trans, head); mutex_unlock(&head->mutex); btrfs_put_delayed_ref_head(head); return ret; From patchwork Fri Sep 28 11:17:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619669 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A32FC3CF1 for ; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A4662A23B for ; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8E7C42B0C5; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F54D2A23B for ; Fri, 28 Sep 2018 11:18:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729324AbeI1Rls (ORCPT ); Fri, 28 Sep 2018 13:41:48 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:46768 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rls (ORCPT ); Fri, 28 Sep 2018 13:41:48 -0400 Received: by mail-qt1-f193.google.com with SMTP id h22-v6so6088155qtr.13 for ; Fri, 28 Sep 2018 04:18:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rHTVMCxRLsTEVUkODyGqkPIz399o/p3gyRkwe0nkz4U=; b=P1OLigpHvXlqYIeD840SPmwBve5QmtOanrjYfubbm7OPn20+aHWRL3hEBOsL9dn0R1 UJxvVO5YNji3jVzxb1NuXSln2oaDjPNDpmt5PaI24YEDv1lGRls2Y0hRveya5rgqT38U RoqKTUu3s3MY5g6BkFCX6hjBbWZRuigtrAL8+sW59axo8vdzVCfFYwJjPZA61981l1Go STZZfZ5NEcmjkZ2L5JZOhQtNTESGlSJLx78+UfngGUZYLX+wlouf/oxvSDQt7uu4NrmK 0Dd2jnWyz/AC2NTJbfRs5jULi/HuwcZ2LkVlYUHe5IZxDQHJ9Qkw/dSePIiye4UnnqUU X2jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rHTVMCxRLsTEVUkODyGqkPIz399o/p3gyRkwe0nkz4U=; b=RijBZjSLeW8wrcivRlvQCHs2Ssgxz72xwfXvSitWOYI4X80PZKIGJsHlfXmQbz3UFL lmepI/1lRSp3JlipDb04wB3yLzROcpKSOl5kqtljCexaj+8wvoa4cvzI/wYM6mIqD6va efCoSsQns/Xmp9g8kBvimyKkYvk+wS3LdItvVmiSgnoT2a0j6yMvCTfy6paAwz1sRPmB RVP2XFoa7fhB67PooFsi79VGxZMVUxB0NjIFzNPgUHqNNohhwGBMPP/RmUSAXlWmwZbX bqisforUpRBP5tFkk7kKB3GwkL8HIf2rtC6eGZAS5K8auZ1kUPv6/4qz32qazoMuXvMU lqaQ== X-Gm-Message-State: ABuFfohlJm/53m/SZBrh4fS6qR957RE6Dw3mAf6+lCoQ060Kc9s8Kll2 66iJ1OjbkfT9oANuxihC/T9s7ZPp8c4= X-Google-Smtp-Source: ACcGV60/GtZD4u4eOyrgj7AypzhT53UhAskaQQ+SzVuBXmNLxily1GlY7T4/HImJNcxAsmKKUFxZlw== X-Received: by 2002:a0c:e205:: with SMTP id q5-v6mr11213749qvl.224.1538133510702; Fri, 28 Sep 2018 04:18:30 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s51-v6sm3212163qta.71.2018.09.28.04.18.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:29 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 03/42] btrfs: cleanup extent_op handling Date: Fri, 28 Sep 2018 07:17:42 -0400 Message-Id: <20180928111821.24376-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik The cleanup_extent_op function actually would run the extent_op if it needed running, which made the name sort of a misnomer. Change it to run_and_cleanup_extent_op, and move the actual cleanup work to cleanup_extent_op so it can be used by check_ref_cleanup() in order to unify the extent op handling. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index a44d55e36e11..98f36dfeccb0 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2442,19 +2442,33 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_ref btrfs_delayed_ref_unlock(head); } -static int cleanup_extent_op(struct btrfs_trans_handle *trans, - struct btrfs_delayed_ref_head *head) +static struct btrfs_delayed_extent_op * +cleanup_extent_op(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) { struct btrfs_delayed_extent_op *extent_op = head->extent_op; - int ret; if (!extent_op) - return 0; - head->extent_op = NULL; + return NULL; + if (head->must_insert_reserved) { + head->extent_op = NULL; btrfs_free_delayed_extent_op(extent_op); - return 0; + return NULL; } + return extent_op; +} + +static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) +{ + struct btrfs_delayed_extent_op *extent_op = + cleanup_extent_op(trans, head); + int ret; + + if (!extent_op) + return 0; + head->extent_op = NULL; spin_unlock(&head->lock); ret = run_delayed_extent_op(trans, head, extent_op); btrfs_free_delayed_extent_op(extent_op); @@ -2506,7 +2520,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, delayed_refs = &trans->transaction->delayed_refs; - ret = cleanup_extent_op(trans, head); + ret = run_and_cleanup_extent_op(trans, head); if (ret < 0) { unselect_delayed_ref_head(delayed_refs, head); btrfs_debug(fs_info, "run_delayed_extent_op returned %d", ret); @@ -6977,12 +6991,8 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (!RB_EMPTY_ROOT(&head->ref_tree)) goto out; - if (head->extent_op) { - if (!head->must_insert_reserved) - goto out; - btrfs_free_delayed_extent_op(head->extent_op); - head->extent_op = NULL; - } + if (cleanup_extent_op(trans, head) != NULL) + goto out; /* * waiting for the lock here would deadlock. If someone else has it From patchwork Fri Sep 28 11:17:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619671 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 80D266CB for ; Fri, 28 Sep 2018 11:18:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 789D32B061 for ; Fri, 28 Sep 2018 11:18:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D3E02B0CF; Fri, 28 Sep 2018 11:18:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A88C2B061 for ; Fri, 28 Sep 2018 11:18:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729332AbeI1Rlu (ORCPT ); Fri, 28 Sep 2018 13:41:50 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:45914 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rlu (ORCPT ); Fri, 28 Sep 2018 13:41:50 -0400 Received: by mail-qk1-f194.google.com with SMTP id q20-v6so3530868qkc.12 for ; Fri, 28 Sep 2018 04:18:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E8JF94eruCVpfwCQjNtPgc8hW7YmBCNIla9LHGUTiOs=; b=mXCmbHJZqUQzvctrq8r5Bph6itvgS3E9G9xxVuoewnEx7t2jMZMb0md+qMrNtwhVVf DPFDUYi5WNW/NB0ZrtVyrjU00pp6cOx4UT+bgn7da7+3IM1DjUSoXaoCMlof5mFQOTUN AoZz63j6A+PusoTpwFkVMi5tDpDq8IKK9++G+Q1lldJ/fMxkVZdgaNunnhZ2IaQ+BxrT AP236z7KgShHCnqOHG1omXrHUlVXdM1pCx7Vv+BJMy//pi/MFaONAET9z2oUJPPfC3hY gEo1G1q0A/K9Qh4uuLDq05/Aw6Qp6VAcNl2XuMj1x7maGFLEH0tJ3UhJQgjKrRICKFnc vRiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E8JF94eruCVpfwCQjNtPgc8hW7YmBCNIla9LHGUTiOs=; b=VuSwCS+ousvXadEUBYbIMtzHn35EWdR691cb0tuVenKrqYZKMYYZl35x3b3p2BYt7Q 3uSZbPcJ5q+XgaoRIMQVKdqzajJCqMwwdUK8xqsMweY8xJpNpwdsIXVRKm5I+EoemV/a Tl+U8Yc646FtPBdWCTK4s00ojTHq0Fiat2za/tRzaqGHEFPuhLoXsXxOpCFyCtq4p5Hj VEhEPXNBKzdlHENbEnztlNwScj8UvMLe/sXRV5mY2npXQQGa62go7/oZFaUQPFp9gv02 arszffnvZVWAzQMtb77X/kfK/lni6dbc5Uw0bd28hh58rlF7DSF6KpvMyvsOiLfNfDu0 URUQ== X-Gm-Message-State: ABuFfojh6LF5IMp8lwKjxWdfoX9jludQzdCvePaM2ieqe9BT+mpPCP3M d6ozlJ0f3nHErZIT4TF7v6d03w== X-Google-Smtp-Source: ACcGV63l43nAZd/Gn1ePZn8s8YKRkfmukaPHA/xKS7SJtfGisUPW/4p4G/xMpTUh5QSIfK5X1EAnTg== X-Received: by 2002:a37:31d5:: with SMTP id x204-v6mr10868830qkx.289.1538133512552; Fri, 28 Sep 2018 04:18:32 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 80-v6sm2482733qkj.75.2018.09.28.04.18.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:31 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 04/42] btrfs: only track ref_heads in delayed_ref_updates Date: Fri, 28 Sep 2018 07:17:43 -0400 Message-Id: <20180928111821.24376-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We use this number to figure out how many delayed refs to run, but __btrfs_run_delayed_refs really only checks every time we need a new delayed ref head, so we always run at least one ref head completely no matter what the number of items on it. Fix the accounting to only be adjusted when we add/remove a ref head. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 3a9e4ac21794..27f7dd4e3d52 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -234,8 +234,6 @@ static inline void drop_delayed_ref(struct btrfs_trans_handle *trans, ref->in_tree = 0; btrfs_put_delayed_ref(ref); atomic_dec(&delayed_refs->num_entries); - if (trans->delayed_ref_updates) - trans->delayed_ref_updates--; } static bool merge_ref(struct btrfs_trans_handle *trans, @@ -460,7 +458,6 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans, if (ref->action == BTRFS_ADD_DELAYED_REF) list_add_tail(&ref->add_list, &href->ref_add_list); atomic_inc(&root->num_entries); - trans->delayed_ref_updates++; spin_unlock(&href->lock); return ret; } From patchwork Fri Sep 28 11:17:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619673 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A4486CB for ; Fri, 28 Sep 2018 11:18:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12CE92B061 for ; Fri, 28 Sep 2018 11:18:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 079712B0CF; Fri, 28 Sep 2018 11:18:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD0B42B061 for ; Fri, 28 Sep 2018 11:18:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729347AbeI1Rlw (ORCPT ); Fri, 28 Sep 2018 13:41:52 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:36762 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rlw (ORCPT ); Fri, 28 Sep 2018 13:41:52 -0400 Received: by mail-qt1-f196.google.com with SMTP id e26-v6so6166408qtq.3 for ; Fri, 28 Sep 2018 04:18:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=+yz3n6lATRx6h1PNpaQrCmKxcZVsI4n1UZsZzxSFbKc=; b=bO5G6ngR++XHgU44+SKOdWJukr5NSQEZcLXI+x7Nml4y9xTrtAmfH1CpY+A5v3WOlv 0Lno1jdwYApAMdSYktk5a/ZLKzJ0JY8bkuzlEOchT24M26wZK6tVNLPbG2ZM+ycZJhvJ /BYHkieJpfKXZhqZ2ER93utgIRnbO7k+waFjw8x16IYSM5UIMIrVsUhb3DOSNC6kZSeH npfbELIyba6uOTvjEmKyNr0lTv40M/x4TP0S6f58hMwYBGU0lZu8dcK2Q6ttzS4zSZRR ZsqmkfiZ/0nmTiBIiFInFt6T6eFID330lF5y25Mr/QJJnyfkOX9nLdmstpvD9CsuyyOS +3LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+yz3n6lATRx6h1PNpaQrCmKxcZVsI4n1UZsZzxSFbKc=; b=C+Fdw8fS5OMaLAooh82atpnBDIAfL25m6gbAFaNx4TMqh53fYI9ewrXoRZ4Fu1tKlW sB066TLBMJW2YYMuLqy3e0CduHhhWgjiNbW3JWPiE5aZp7SfTP57DL4a9u9GLQ4KNnHf g17gmVtFe4p+4VwT6x0ZTHQHKKHg0lMxh5Ro4ZtnFhpSL8dekfbES/aae93OtAWc5lxo wt7Sv8peEAbyDrKRt6EGPy3EzF0oroo/Kteth9mq+aQD61G920GdF8x/XjrSsxp2RFwo dXsORBQlAfCuwk92vCBhAF8EBfz1NpXva9CoTHMPAzB7PWY+4jzwhtZRJdAJXHOaqYZy mgyQ== X-Gm-Message-State: ABuFfoiEcgvvJDnF4jsDGVtzzevj5dF+ptnlADgpCQ2kWwzdd8Y61QHY apapHe0+9GVRPNsUR9YpmYdsCQ== X-Google-Smtp-Source: ACcGV62FF7eZXKYHoRGNG7KrwXbNds6BUJlkfW1nPGZpSiRV3ubFsb0XXINuQg0rNR3BxkZl3Swkeg== X-Received: by 2002:aed:3c07:: with SMTP id t7-v6mr8950304qte.192.1538133514251; Fri, 28 Sep 2018 04:18:34 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f184-v6sm2610593qkc.23.2018.09.28.04.18.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:33 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 05/42] btrfs: only count ref heads run in __btrfs_run_delayed_refs Date: Fri, 28 Sep 2018 07:17:44 -0400 Message-Id: <20180928111821.24376-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We pick the number of ref's to run based on the number of ref heads, and only make the decision to stop once we've processed entire ref heads, so only count the ref heads we've run and bail once we've hit the number of ref heads we wanted to process. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 98f36dfeccb0..b32bd38390dd 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2592,6 +2592,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, spin_unlock(&delayed_refs->lock); break; } + count++; /* grab the lock that says we are going to process * all the refs for this head */ @@ -2605,7 +2606,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, */ if (ret == -EAGAIN) { locked_ref = NULL; - count++; continue; } } @@ -2633,7 +2633,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unselect_delayed_ref_head(delayed_refs, locked_ref); locked_ref = NULL; cond_resched(); - count++; continue; } @@ -2651,7 +2650,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, return ret; } locked_ref = NULL; - count++; continue; } @@ -2702,7 +2700,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, } btrfs_put_delayed_ref(ref); - count++; cond_resched(); } From patchwork Fri Sep 28 11:17:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619677 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 015FD3CF1 for ; Fri, 28 Sep 2018 11:18:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E830D2B061 for ; Fri, 28 Sep 2018 11:18:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC63F2B0F2; Fri, 28 Sep 2018 11:18:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B3E82B061 for ; Fri, 28 Sep 2018 11:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729385AbeI1Rlz (ORCPT ); Fri, 28 Sep 2018 13:41:55 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:35270 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rlz (ORCPT ); Fri, 28 Sep 2018 13:41:55 -0400 Received: by mail-qk1-f196.google.com with SMTP id a189-v6so3579815qkb.2 for ; Fri, 28 Sep 2018 04:18:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UMAxEiV+68Nd9dMH4VurZacgpPCSyaixMmaSB9avthc=; b=LZstdLslcA7zg5KtqN+XxEqOojhv/NQHnPxpj72MRu6jtnZuyMxCaAUpTJf3/hAu0T bxZGvyjI/76Ugm2zJHKIoDFcJxkN0x3VoN08zoY52ZzfWlq6pdlHHHPWgYBa5Zkgk6ha jXs3NwHOfqPybDEqGJTiG0G9YXoo6WgfJxwyPFKAEJNjICaXNXcl5xDlIyro/Y0Icbmj SdNwXovgiqlWZ49gOW3CyTCse6xr7yaBhdp37gy33OHL1hGIg6Ilszoy3M81X8F6Y0Hf haAB4D6UlvCtCzrEM/TSzmQY9MYocRdepY1AzE24ZllmrxdgTL5MC+MiWMUPtJ3gzQh+ dFsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UMAxEiV+68Nd9dMH4VurZacgpPCSyaixMmaSB9avthc=; b=TZByS/bw2qzjFYaMsTI+nhIL1UXHc1NxjKjckCUUjrbV1Xh/nAtVZVcip+kJf0NKOw Hu8fSXQwbEko3UpbTjK41pUGjCFfalc+V0UI8sLq7QWU95HG/IwvBuqhkselrMpN0DrZ dGyvwAYlndxjxxMv+xzRvLv7momtnm9KRjv4cj7+vrW+znVLAsBotQ1CKP3Aog+P27zM 0apFWGarxlAajiB7s1mLvznHp89VIIWcaarbA/4gVBU5GpL0ZPGgcO3AFWhwZ+XV63Fq t6QRFiSQYW3fibJnysXgrhdpxs3LwAw0etXKqXmIQhAf+Bml6St6DKJVtn9qYxJpm7DJ UvQg== X-Gm-Message-State: ABuFfoirRPZoXdvfG0Ml6hi2cG5vDHvpDaDsMHCxLaM223+icWImEsYS Gll7ZE0X8bJDqRjP8hBsS/A4TAsLDpg= X-Google-Smtp-Source: ACcGV60j2C6w9biBvKPoytSfQJt0OQRmEs6plJpQ2eCaGJfub0/uYifakjNOWpHZeGHhcXt1cPAr+Q== X-Received: by 2002:ae9:e004:: with SMTP id m4-v6mr11508246qkk.211.1538133516250; Fri, 28 Sep 2018 04:18:36 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id p5-v6sm2455620qkd.81.2018.09.28.04.18.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:35 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 06/42] btrfs: introduce delayed_refs_rsv Date: Fri, 28 Sep 2018 07:17:45 -0400 Message-Id: <20180928111821.24376-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong. So instead give them their own block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism. For now the delayed_refs_rsv will hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via the delayed_refs so that will stay there. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 66f1d3895bca..1a2c3b629af2 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -452,8 +452,9 @@ struct btrfs_space_info { #define BTRFS_BLOCK_RSV_TRANS 3 #define BTRFS_BLOCK_RSV_CHUNK 4 #define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8 struct btrfs_block_rsv { u64 size; @@ -794,6 +795,8 @@ struct btrfs_fs_info { struct btrfs_block_rsv chunk_block_rsv; /* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; + /* block reservation for delayed refs */ + struct btrfs_block_rsv delayed_refs_rsv; struct btrfs_block_rsv empty_block_rsv; @@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info, int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, - struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info); void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info, const u64 start); void btrfs_wait_block_group_reservations(struct btrfs_block_group_cache *bg); @@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum { enum btrfs_flush_state { FLUSH_DELAYED_ITEMS_NR = 1, FLUSH_DELAYED_ITEMS = 2, - FLUSH_DELALLOC = 3, - FLUSH_DELALLOC_WAIT = 4, - ALLOC_CHUNK = 5, - COMMIT_TRANS = 6, + FLUSH_DELAYED_REFS_NR = 3, + FLUSH_DELAYED_REFS = 4, + FLUSH_DELALLOC = 5, + FLUSH_DELALLOC_WAIT = 6, + ALLOC_CHUNK = 7, + COMMIT_TRANS = 8, }; int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes); @@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info, void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *block_rsv, u64 num_bytes); +void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr); +void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs(struct btrfs_fs_info *fs_info, + enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *src, + u64 num_bytes); int btrfs_inc_block_group_ro(struct btrfs_block_group_cache *cache); void btrfs_dec_block_group_ro(struct btrfs_block_group_cache *cache); void btrfs_put_block_group_cache(struct btrfs_fs_info *info); diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 27f7dd4e3d52..96ce087747b2 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans, * existing and update must have the same bytenr */ static noinline void -update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head *existing, struct btrfs_delayed_ref_head *update, int *old_ref_mod_ret) { + struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + struct btrfs_fs_info *fs_info = trans->fs_info; int old_ref_mod; BUG_ON(existing->is_data != update->is_data); @@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs, * versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { - if (existing->total_ref_mod >= 0 && old_ref_mod < 0) + u64 csum_items = + btrfs_csum_bytes_to_leaves(fs_info, + existing->num_bytes); + + if (existing->total_ref_mod >= 0 && old_ref_mod < 0) { delayed_refs->pending_csums -= existing->num_bytes; - if (existing->total_ref_mod < 0 && old_ref_mod >= 0) + btrfs_delayed_refs_rsv_release(fs_info, csum_items); + } + if (existing->total_ref_mod < 0 && old_ref_mod >= 0) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } } spin_unlock(&existing->lock); } @@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); - update_existing_head_ref(delayed_refs, existing, head_ref, + update_existing_head_ref(trans, existing, head_ref, old_ref_mod); /* * we've updated the existing ref, free the newly @@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, } else { if (old_ref_mod) *old_ref_mod = 0; - if (head_ref->is_data && head_ref->ref_mod < 0) + if (head_ref->is_data && head_ref->ref_mod < 0) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += + btrfs_csum_bytes_to_leaves(trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++; atomic_inc(&delayed_refs->num_entries); @@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret = insert_delayed_ref(trans, delayed_refs, head_ref, &ref->node); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); trace_add_delayed_tree_ref(fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ? @@ -866,6 +882,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, ret = insert_delayed_ref(trans, delayed_refs, head_ref, &ref->node); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); trace_add_delayed_data_ref(trans->fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ? @@ -903,6 +920,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, NULL, NULL, NULL); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); return 0; } diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5124c15705ce..377ad9c1cb17 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2692,6 +2692,9 @@ int open_ctree(struct super_block *sb, btrfs_init_block_rsv(&fs_info->empty_block_rsv, BTRFS_BLOCK_RSV_EMPTY); btrfs_init_block_rsv(&fs_info->delayed_block_rsv, BTRFS_BLOCK_RSV_DELOPS); + btrfs_init_block_rsv(&fs_info->delayed_refs_rsv, + BTRFS_BLOCK_RSV_DELREFS); + atomic_set(&fs_info->async_delalloc_pages, 0); atomic_set(&fs_info->defrag_running, 0); atomic_set(&fs_info->qgroup_op_seq, 0); @@ -4419,6 +4422,7 @@ void btrfs_cleanup_dirty_bgs(struct btrfs_transaction *cur_trans, spin_unlock(&cur_trans->dirty_bgs_lock); btrfs_put_block_group(cache); + btrfs_delayed_refs_rsv_release(fs_info, 1); spin_lock(&cur_trans->dirty_bgs_lock); } spin_unlock(&cur_trans->dirty_bgs_lock); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index b32bd38390dd..1213f573eea2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2481,6 +2481,7 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info = trans->fs_info; struct btrfs_delayed_ref_root *delayed_refs = &trans->transaction->delayed_refs; + int nr_items = 1; if (head->total_ref_mod < 0) { struct btrfs_space_info *space_info; @@ -2502,12 +2503,15 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, spin_lock(&delayed_refs->lock); delayed_refs->pending_csums -= head->num_bytes; spin_unlock(&delayed_refs->lock); + nr_items += btrfs_csum_bytes_to_leaves(fs_info, + head->num_bytes); } } /* Also free its reserved qgroup space */ btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, head->qgroup_reserved); + btrfs_delayed_refs_rsv_release(fs_info, nr_items); } static int cleanup_ref_head(struct btrfs_trans_handle *trans, @@ -2802,40 +2806,22 @@ u64 btrfs_csum_bytes_to_leaves(struct btrfs_fs_info *fs_info, u64 csum_bytes) return num_csums; } -int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, - struct btrfs_fs_info *fs_info) +bool btrfs_check_space_for_delayed_refs( struct btrfs_fs_info *fs_info) { - struct btrfs_block_rsv *global_rsv; - u64 num_heads = trans->transaction->delayed_refs.num_heads_ready; - u64 csum_bytes = trans->transaction->delayed_refs.pending_csums; - unsigned int num_dirty_bgs = trans->transaction->num_dirty_bgs; - u64 num_bytes, num_dirty_bgs_bytes; - int ret = 0; - - num_bytes = btrfs_calc_trans_metadata_size(fs_info, 1); - num_heads = heads_to_leaves(fs_info, num_heads); - if (num_heads > 1) - num_bytes += (num_heads - 1) * fs_info->nodesize; - num_bytes <<= 1; - num_bytes += btrfs_csum_bytes_to_leaves(fs_info, csum_bytes) * - fs_info->nodesize; - num_dirty_bgs_bytes = btrfs_calc_trans_metadata_size(fs_info, - num_dirty_bgs); - global_rsv = &fs_info->global_block_rsv; - - /* - * If we can't allocate any more chunks lets make sure we have _lots_ of - * wiggle room since running delayed refs can create more delayed refs. - */ - if (global_rsv->space_info->full) { - num_dirty_bgs_bytes <<= 1; - num_bytes <<= 1; - } + struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; + u64 reserved; + bool ret = false; spin_lock(&global_rsv->lock); - if (global_rsv->reserved <= num_bytes + num_dirty_bgs_bytes) - ret = 1; + reserved = global_rsv->reserved; spin_unlock(&global_rsv->lock); + + spin_lock(&delayed_refs_rsv->lock); + reserved += delayed_refs_rsv->reserved; + if (delayed_refs_rsv->size >= reserved) + ret = true; + spin_unlock(&delayed_refs_rsv->lock); return ret; } @@ -2855,7 +2841,7 @@ int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans, if (val >= NSEC_PER_SEC / 2) return 2; - return btrfs_check_space_for_delayed_refs(trans, fs_info); + return btrfs_check_space_for_delayed_refs(fs_info) ? 1 : 0; } struct async_delayed_refs { @@ -3610,6 +3596,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) */ mutex_lock(&trans->transaction->cache_write_mutex); while (!list_empty(&dirty)) { + bool drop_reserve = true; + cache = list_first_entry(&dirty, struct btrfs_block_group_cache, dirty_list); @@ -3682,6 +3670,7 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) list_add_tail(&cache->dirty_list, &cur_trans->dirty_bgs); btrfs_get_block_group(cache); + drop_reserve = false; } spin_unlock(&cur_trans->dirty_bgs_lock); } else if (ret) { @@ -3692,6 +3681,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) /* if its not on the io list, we need to put the block group */ if (should_put) btrfs_put_block_group(cache); + if (drop_reserve) + btrfs_delayed_refs_rsv_release(fs_info, 1); if (ret) break; @@ -3840,6 +3831,7 @@ int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans, /* if its not on the io list, we need to put the block group */ if (should_put) btrfs_put_block_group(cache); + btrfs_delayed_refs_rsv_release(fs_info, 1); spin_lock(&cur_trans->dirty_bgs_lock); } spin_unlock(&cur_trans->dirty_bgs_lock); @@ -4816,8 +4808,10 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *trans; u64 bytes; + u64 reclaim_bytes = 0; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) @@ -4850,12 +4844,16 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, return -ENOSPC; spin_lock(&delayed_rsv->lock); - if (delayed_rsv->size > bytes) - bytes = 0; - else - bytes -= delayed_rsv->size; + reclaim_bytes += delayed_rsv->reserved; spin_unlock(&delayed_rsv->lock); + spin_lock(&delayed_refs_rsv->lock); + reclaim_bytes += delayed_refs_rsv->reserved; + spin_unlock(&delayed_refs_rsv->lock); + if (reclaim_bytes >= bytes) + goto commit; + bytes -= reclaim_bytes; + if (__percpu_counter_compare(&space_info->total_bytes_pinned, bytes, BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) { @@ -4905,6 +4903,20 @@ static void flush_space(struct btrfs_fs_info *fs_info, shrink_delalloc(fs_info, num_bytes * 2, num_bytes, state == FLUSH_DELALLOC_WAIT); break; + case FLUSH_DELAYED_REFS_NR: + case FLUSH_DELAYED_REFS: + trans = btrfs_join_transaction(root); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + break; + } + if (state == FLUSH_DELAYED_REFS_NR) + nr = calc_reclaim_items_nr(fs_info, num_bytes); + else + nr = 0; + btrfs_run_delayed_refs(trans, nr); + btrfs_end_transaction(trans); + break; case ALLOC_CHUNK: trans = btrfs_join_transaction(root); if (IS_ERR(trans)) { @@ -5377,6 +5389,91 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info, return 0; } +/** + * btrfs_migrate_to_delayed_refs_rsv - transfer bytes to our delayed refs rsv. + * @fs_info - the fs info for our fs. + * @src - the source block rsv to transfer from. + * @num_bytes - the number of bytes to transfer. + * + * This transfers up to the num_bytes amount from the src rsv to the + * delayed_refs_rsv. Any extra bytes are returned to the space info. + */ +void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *src, + u64 num_bytes) +{ + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; + u64 to_free = 0; + + spin_lock(&src->lock); + src->reserved -= num_bytes; + src->size -= num_bytes; + spin_unlock(&src->lock); + + spin_lock(&delayed_refs_rsv->lock); + if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) { + u64 delta = delayed_refs_rsv->size - + delayed_refs_rsv->reserved; + if (num_bytes > delta) { + to_free = num_bytes - delta; + num_bytes = delta; + } + } else { + to_free = num_bytes; + num_bytes = 0; + } + + if (num_bytes) + delayed_refs_rsv->reserved += num_bytes; + if (delayed_refs_rsv->reserved >= delayed_refs_rsv->size) + delayed_refs_rsv->full = 1; + spin_unlock(&delayed_refs_rsv->lock); + + if (num_bytes) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, num_bytes, 1); + if (to_free) + space_info_add_old_bytes(fs_info, delayed_refs_rsv->space_info, + to_free); +} + +/** + * btrfs_throttle_delayed_refs - throttle based on our delayed refs usage. + * @fs_info - the fs_info for our fs. + * @flush - control how we can flush for this reservation. + * + * This will refill the delayed block_rsv up to 1 items size worth of space and + * will return -ENOSPC if we can't make the reservation. + */ +int btrfs_throttle_delayed_refs(struct btrfs_fs_info *fs_info, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; + u64 limit = btrfs_calc_trans_metadata_size(fs_info, 1); + u64 num_bytes = 0; + int ret = -ENOSPC; + + spin_lock(&block_rsv->lock); + if (block_rsv->reserved < block_rsv->size) { + num_bytes = block_rsv->size - block_rsv->reserved; + num_bytes = min(num_bytes, limit); + } + spin_unlock(&block_rsv->lock); + + if (!num_bytes) + return 0; + + ret = reserve_metadata_bytes(fs_info->extent_root, block_rsv, + num_bytes, flush); + if (ret) + return ret; + block_rsv_add_bytes(block_rsv, num_bytes, 0); + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, num_bytes, 1); + return 0; +} + + /* * This is for space we already have accounted in space_info->bytes_may_use, so * basically when we're returning space from block_rsv's. @@ -5699,6 +5796,31 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, return ret; } +static u64 __btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *block_rsv, + u64 num_bytes, u64 *qgroup_to_release) +{ + struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; + struct btrfs_block_rsv *target = delayed_rsv; + + if (target->full || target == block_rsv) + target = global_rsv; + + if (block_rsv->space_info != target->space_info) + target = NULL; + + return block_rsv_release_bytes(fs_info, block_rsv, target, num_bytes, + qgroup_to_release); +} + +void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *block_rsv, + u64 num_bytes) +{ + __btrfs_block_rsv_release(fs_info, block_rsv, num_bytes, NULL); +} + /** * btrfs_inode_rsv_release - release any excessive reservation. * @inode - the inode we need to release from. @@ -5713,7 +5835,6 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) { struct btrfs_fs_info *fs_info = inode->root->fs_info; - struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; struct btrfs_block_rsv *block_rsv = &inode->block_rsv; u64 released = 0; u64 qgroup_to_release = 0; @@ -5723,8 +5844,8 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) * are releasing 0 bytes, and then we'll just get the reservation over * the size free'd. */ - released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv, 0, - &qgroup_to_release); + released = __btrfs_block_rsv_release(fs_info, block_rsv, 0, + &qgroup_to_release); if (released > 0) trace_btrfs_space_reservation(fs_info, "delalloc", btrfs_ino(inode), released, 0); @@ -5735,16 +5856,26 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) qgroup_to_release); } -void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, - struct btrfs_block_rsv *block_rsv, - u64 num_bytes) +/** + * btrfs_delayed_refs_rsv_release - release a ref head's reservation. + * @fs_info - the fs_info for our fs. + * @nr - the number of items to drop. + * + * This drops the delayed ref head's count from the delayed refs rsv and free's + * any excess reservation we had. + */ +void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr) { + struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + u64 num_bytes = btrfs_calc_trans_metadata_size(fs_info, nr); + u64 released = 0; - if (global_rsv == block_rsv || - block_rsv->space_info != global_rsv->space_info) - global_rsv = NULL; - block_rsv_release_bytes(fs_info, block_rsv, global_rsv, num_bytes, NULL); + released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv, + num_bytes, NULL); + if (released) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, released, 0); } static void update_global_block_rsv(struct btrfs_fs_info *fs_info) @@ -5809,9 +5940,10 @@ static void init_global_block_rsv(struct btrfs_fs_info *fs_info) fs_info->trans_block_rsv.space_info = space_info; fs_info->empty_block_rsv.space_info = space_info; fs_info->delayed_block_rsv.space_info = space_info; + fs_info->delayed_refs_rsv.space_info = space_info; - fs_info->extent_root->block_rsv = &fs_info->global_block_rsv; - fs_info->csum_root->block_rsv = &fs_info->global_block_rsv; + fs_info->extent_root->block_rsv = &fs_info->delayed_refs_rsv; + fs_info->csum_root->block_rsv = &fs_info->delayed_refs_rsv; fs_info->dev_root->block_rsv = &fs_info->global_block_rsv; fs_info->tree_root->block_rsv = &fs_info->global_block_rsv; if (fs_info->quota_root) @@ -5831,8 +5963,34 @@ static void release_global_block_rsv(struct btrfs_fs_info *fs_info) WARN_ON(fs_info->chunk_block_rsv.reserved > 0); WARN_ON(fs_info->delayed_block_rsv.size > 0); WARN_ON(fs_info->delayed_block_rsv.reserved > 0); + WARN_ON(fs_info->delayed_refs_rsv.reserved > 0); + WARN_ON(fs_info->delayed_refs_rsv.size > 0); } +/* + * btrfs_update_delayed_refs_rsv - adjust the size of the delayed refs rsv + * @trans - the trans that may have generated delayed refs + * + * This is to be called anytime we may have adjusted trans->delayed_ref_updates, + * it'll calculate the additional size and add it to the delayed_refs_rsv. + */ +void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; + u64 num_bytes; + + if (!trans->delayed_ref_updates) + return; + + num_bytes = btrfs_calc_trans_metadata_size(fs_info, + trans->delayed_ref_updates); + spin_lock(&delayed_rsv->lock); + delayed_rsv->size += num_bytes; + delayed_rsv->full = 0; + spin_unlock(&delayed_rsv->lock); + trans->delayed_ref_updates = 0; +} /* * To be called after all the new block groups attached to the transaction @@ -6126,6 +6284,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, u64 old_val; u64 byte_in_group; int factor; + int ret = 0; /* block accounting for super block */ spin_lock(&info->delalloc_root_lock); @@ -6139,8 +6298,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, while (total) { cache = btrfs_lookup_block_group(info, bytenr); - if (!cache) - return -ENOENT; + if (!cache) { + ret = -ENOENT; + break; + } factor = btrfs_bg_type_to_factor(cache->flags); /* @@ -6199,6 +6360,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, list_add_tail(&cache->dirty_list, &trans->transaction->dirty_bgs); trans->transaction->num_dirty_bgs++; + trans->delayed_ref_updates++; btrfs_get_block_group(cache); } spin_unlock(&trans->transaction->dirty_bgs_lock); @@ -6216,7 +6378,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, total -= num_bytes; bytenr += num_bytes; } - return 0; + + /* Modified block groups are accounted for in the delayed_refs_rsv. */ + btrfs_update_delayed_refs_rsv(trans); + return ret; } static u64 first_logical_byte(struct btrfs_fs_info *fs_info, u64 search_start) @@ -8230,7 +8395,12 @@ use_block_rsv(struct btrfs_trans_handle *trans, goto again; } - if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) { + /* + * The global reserve still exists to save us from ourselves, so don't + * warn_on if we are short on our delayed refs reserve. + */ + if (block_rsv->type != BTRFS_BLOCK_RSV_DELREFS && + btrfs_test_opt(fs_info, ENOSPC_DEBUG)) { static DEFINE_RATELIMIT_STATE(_rs, DEFAULT_RATELIMIT_INTERVAL * 10, /*DEFAULT_RATELIMIT_BURST*/ 1); @@ -10146,6 +10316,7 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) add_block_group_free_space(trans, block_group); /* already aborted the transaction if it failed. */ next: + btrfs_delayed_refs_rsv_release(fs_info, 1); list_del_init(&block_group->bg_list); } trans->can_flush_pending_bgs = can_flush_pending_bgs; @@ -10223,6 +10394,8 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used, link_block_group(cache); list_add_tail(&cache->bg_list, &trans->new_bgs); + trans->delayed_ref_updates++; + btrfs_update_delayed_refs_rsv(trans); set_avail_alloc_bits(fs_info, type); return 0; @@ -10260,6 +10433,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, int factor; struct btrfs_caching_control *caching_ctl = NULL; bool remove_em; + bool remove_rsv = false; block_group = btrfs_lookup_block_group(fs_info, group_start); BUG_ON(!block_group); @@ -10324,6 +10498,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, if (!list_empty(&block_group->dirty_list)) { list_del_init(&block_group->dirty_list); + remove_rsv = true; btrfs_put_block_group(block_group); } spin_unlock(&trans->transaction->dirty_bgs_lock); @@ -10533,6 +10708,8 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, ret = btrfs_del_item(trans, root, path); out: + if (remove_rsv) + btrfs_delayed_refs_rsv_release(fs_info, 1); btrfs_free_path(path); return ret; } diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 212fa71317d6..cd00ec869c96 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5382,7 +5382,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, * Try to steal from the global reserve if there is space for * it. */ - if (!btrfs_check_space_for_delayed_refs(trans, fs_info) && + if (!btrfs_check_space_for_delayed_refs(fs_info) && !btrfs_block_rsv_migrate(global_rsv, rsv, min_size, 0)) return trans; diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 3b84f5015029..117e0c4a914a 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -455,7 +455,7 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, bool enforce_qgroups) { struct btrfs_fs_info *fs_info = root->fs_info; - + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *h; struct btrfs_transaction *cur_trans; u64 num_bytes = 0; @@ -484,13 +484,28 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, * the appropriate flushing if need be. */ if (num_items && root != fs_info->chunk_root) { + struct btrfs_block_rsv *rsv = &fs_info->trans_block_rsv; + u64 delayed_refs_bytes = 0; + qgroup_reserved = num_items * fs_info->nodesize; ret = btrfs_qgroup_reserve_meta_pertrans(root, qgroup_reserved, enforce_qgroups); if (ret) return ERR_PTR(ret); + /* + * We want to reserve all the bytes we may need all at once, so + * we only do 1 enospc flushing cycle per transaction start. We + * accomplish this by simply assuming we'll do 2 x num_items + * worth of delayed refs updates in this trans handle, and + * refill that amount for whatever is missing in the reserve. + */ num_bytes = btrfs_calc_trans_metadata_size(fs_info, num_items); + if (delayed_refs_rsv->full == 0) { + delayed_refs_bytes = num_bytes; + num_bytes <<= 1; + } + /* * Do the reservation for the relocation root creation */ @@ -499,8 +514,24 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, reloc_reserved = true; } - ret = btrfs_block_rsv_add(root, &fs_info->trans_block_rsv, - num_bytes, flush); + ret = btrfs_block_rsv_add(root, rsv, num_bytes, flush); + if (ret) + goto reserve_fail; + if (delayed_refs_bytes) { + btrfs_migrate_to_delayed_refs_rsv(fs_info, rsv, + delayed_refs_bytes); + num_bytes -= delayed_refs_bytes; + } + } else if (num_items == 0 && flush == BTRFS_RESERVE_FLUSH_ALL && + !delayed_refs_rsv->full) { + /* + * Some people call with btrfs_start_transaction(root, 0) + * because they can be throttled, but have some other mechanism + * for reserving space. We still want these guys to refill the + * delayed block_rsv so just add 1 items worth of reservation + * here. + */ + ret = btrfs_throttle_delayed_refs(fs_info, flush); if (ret) goto reserve_fail; } @@ -759,7 +790,7 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) { struct btrfs_fs_info *fs_info = trans->fs_info; - if (btrfs_check_space_for_delayed_refs(trans, fs_info)) + if (btrfs_check_space_for_delayed_refs(fs_info)) return 1; return !!btrfs_block_rsv_check(&fs_info->global_block_rsv, 5); @@ -768,22 +799,12 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) int btrfs_should_end_transaction(struct btrfs_trans_handle *trans) { struct btrfs_transaction *cur_trans = trans->transaction; - int updates; - int err; smp_mb(); if (cur_trans->state >= TRANS_STATE_BLOCKED || cur_trans->delayed_refs.flushing) return 1; - updates = trans->delayed_ref_updates; - trans->delayed_ref_updates = 0; - if (updates) { - err = btrfs_run_delayed_refs(trans, updates * 2); - if (err) /* Error code will also eval true */ - return err; - } - return should_end_transaction(trans); } @@ -813,11 +834,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, { struct btrfs_fs_info *info = trans->fs_info; struct btrfs_transaction *cur_trans = trans->transaction; - u64 transid = trans->transid; - unsigned long cur = trans->delayed_ref_updates; int lock = (trans->type != TRANS_JOIN_NOLOCK); int err = 0; - int must_run_delayed_refs = 0; if (refcount_read(&trans->use_count) > 1) { refcount_dec(&trans->use_count); @@ -828,27 +846,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); - - trans->delayed_ref_updates = 0; - if (!trans->sync) { - must_run_delayed_refs = - btrfs_should_throttle_delayed_refs(trans, info); - cur = max_t(unsigned long, cur, 32); - - /* - * don't make the caller wait if they are from a NOLOCK - * or ATTACH transaction, it will deadlock with commit - */ - if (must_run_delayed_refs == 1 && - (trans->type & (__TRANS_JOIN_NOLOCK | __TRANS_ATTACH))) - must_run_delayed_refs = 2; - } - - btrfs_trans_release_metadata(trans); - trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) btrfs_create_pending_block_groups(trans); @@ -893,10 +890,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, } kmem_cache_free(btrfs_trans_handle_cachep, trans); - if (must_run_delayed_refs) { - btrfs_async_run_delayed_refs(info, cur, transid, - must_run_delayed_refs == 1); - } return err; } diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index b401c4e36394..7d205e50b09c 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -1048,6 +1048,8 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_ITEMS, "FLUSH_DELAYED_ITEMS"}, \ { FLUSH_DELALLOC, "FLUSH_DELALLOC"}, \ { FLUSH_DELALLOC_WAIT, "FLUSH_DELALLOC_WAIT"}, \ + { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ + { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) From patchwork Fri Sep 28 11:17:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5945F15A7 for ; Fri, 28 Sep 2018 11:18:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50EFE2B0CF for ; Fri, 28 Sep 2018 11:18:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45CD32B0F4; Fri, 28 Sep 2018 11:18:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC8C02B0CF for ; Fri, 28 Sep 2018 11:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729377AbeI1Rlz (ORCPT ); Fri, 28 Sep 2018 13:41:55 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:41434 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729359AbeI1Rlz (ORCPT ); Fri, 28 Sep 2018 13:41:55 -0400 Received: by mail-qt1-f195.google.com with SMTP id m15-v6so6125384qtp.8 for ; Fri, 28 Sep 2018 04:18:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Ayf4FJ5WuWBTP9J+ecILAGKV0NvK3Tj5jK4kHqTh68A=; b=sH5tldjsYgRDJQhZKcyM5klkN5PxKDtIOEA8AVPa8xwbT5/D3kr8/tnScj8mgtZ0qw gOvQPEx2tUdSp4H6GnINE4cnAVGJPJ5O2GeU3lJeZ5At14ius4Da16N07MvO+MgX5B7V edBFLpOqzGz6dJu9N5LxQ6OoswkqQzv2A4IiOLjGW3c/reMbc3lwEdRKmSspgeFfnJDw vw0l19wOyR27RYMrQ7fiIX7N0FvhPEWbWTUfghyPJlrCAlQQtc/FBPWDEnkZeE35b7MS qsu2ZJ6NuZo5r9ujDPfiiaGsrzPtxx69O3bNx5bmLqAh7bYgDgG9Djy2NQCAFZ0B8d3w +KIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Ayf4FJ5WuWBTP9J+ecILAGKV0NvK3Tj5jK4kHqTh68A=; b=PbGvgPSUSLI2A6zfHZY4iDNVBLVWJuhLSlwOE7o19KQYfJD3gdvfveHgKU6hOd+N6k edX77M0g2Or+eC8fKg2JIPNRrdKQpix+zSRz5vXYqFicJnKd7yvUvwcU6s7LQvtqkLDT +dJXMx7J395sB4UAievGfIrakicMkEOiRf6tG0AyjOWHKY2eROTFkyE/mK3lZnLugCcp ZYIwPT8P8iiB+f7q80U7/iTO07DQpUmTyVoeqoM5s7iEt8Azvxh3CTCSOnRP73XqqLaj Z7ZGokCbMLsORJW9L5rqZnqEYwN1Y/jch9vy8CD4DdLtomLmfcr9J35prCLwDUuQqUn/ MGMQ== X-Gm-Message-State: ABuFfoiUZwYpBumxSzqm5oxxBkq8PMK6dSk7GiQbzi38GKdnZlo8vLf2 UZFoVF5UWuTV3DkLkmvf112v0g== X-Google-Smtp-Source: ACcGV60wSmwj2Dve0nAhGYuza+mjs6ePYMg8Ti5WMqzXTOKh+MvwcKnhArQDj/z+3wqznx6b10nmXA== X-Received: by 2002:a0c:962e:: with SMTP id 43-v6mr11784158qvx.129.1538133518160; Fri, 28 Sep 2018 04:18:38 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c2-v6sm3216185qkj.79.2018.09.28.04.18.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:37 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 07/42] btrfs: check if free bgs for commit Date: Fri, 28 Sep 2018 07:17:46 -0400 Message-Id: <20180928111821.24376-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP may_commit_transaction will skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1213f573eea2..da73b3e5bc39 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes) return 0; - /* See if there is enough pinned space to make this reservation */ - if (__percpu_counter_compare(&space_info->total_bytes_pinned, - bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) + trans = btrfs_join_transaction(fs_info->extent_root); + if (IS_ERR(trans)) + return -ENOSPC; + + /* + * See if there is enough pinned space to make this reservation, or if + * we have bg's that are going to be freed, allowing us to possibly do a + * chunk allocation the next loop through. + */ + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + __percpu_counter_compare(&space_info->total_bytes_pinned, bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) goto commit; /* @@ -4841,7 +4849,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * this reservation. */ if (space_info != delayed_rsv->space_info) - return -ENOSPC; + goto enospc; spin_lock(&delayed_rsv->lock); reclaim_bytes += delayed_rsv->reserved; @@ -4855,17 +4863,14 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, bytes -= reclaim_bytes; if (__percpu_counter_compare(&space_info->total_bytes_pinned, - bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) { - return -ENOSPC; - } - + bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) + goto enospc; commit: - trans = btrfs_join_transaction(fs_info->extent_root); - if (IS_ERR(trans)) - return -ENOSPC; - return btrfs_commit_transaction(trans); +enospc: + btrfs_end_transaction(trans); + return -ENOSPC; } /* From patchwork Fri Sep 28 11:17:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 800DF6CB for ; Fri, 28 Sep 2018 11:18:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75B192B061 for ; Fri, 28 Sep 2018 11:18:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A1932B0F2; Fri, 28 Sep 2018 11:18:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1821A2B061 for ; Fri, 28 Sep 2018 11:18:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729395AbeI1Rl6 (ORCPT ); Fri, 28 Sep 2018 13:41:58 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34987 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rl6 (ORCPT ); Fri, 28 Sep 2018 13:41:58 -0400 Received: by mail-qt1-f195.google.com with SMTP id z14-v6so6183533qtn.2 for ; Fri, 28 Sep 2018 04:18:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=rkuApPuIkmKt2plAeQOPYfOXk6MNyqgwWrbGueCragM=; b=K7XZJUnTzbtV4H88ycFBxSCsRcDYYILLLzEdYCMt03gkkIzhTVR3sJJUMmptXTtfsg NbN0vvVXQ7lmdVZQAwTDkD0YnGRDBj2rigrD3DDp1HAHoeTXx3hzlgQ3HKvnn/+0ShWJ WR8lFZ8nc79dKRBiD8+rZ6hJFfHH5xMwsuu9K9DG46vnr9KZJZFUJYX7Z1U1vEgd7lmT NxrR6P5vXEW4DbUR1QonPPdQHgCXh6mv/zOB936oPsZTgO8+3J+Ghyhype/wH5EAyFLG Jgm3OsVBjKkJ6AYoHSumSupi+bhIZ/eRTsdcv/w7IQVkMJpaGusYTu7yK3eqlS5P9XcE 3ryQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=rkuApPuIkmKt2plAeQOPYfOXk6MNyqgwWrbGueCragM=; b=PGKCiQjsBRjvqdYCnMhT6j6+3xVWEgR3pFhxBhdg6wOwKGwFA22NbBGNdMIPLqJ7/G Ju+dzNE8v2hTOplxcQFIskWh9EYBPO3ARc1rpzotULsD2L0NLAiOdrMEbDxuoGnvA1bd g0sID2y9AlvAnHRScNXDOHM7Z5twqVUo9xU626tMdfDF6OwUKPLMrJ1InZvNj6vzdSlk t0vpaI78NnVZTEsAIicyikMvc3HYIRiLb3ZlrXegClPkOfwdrYJgZdFpfego6yFB+7Kw gNnx18YzbrKKKPuMs5RvRBzA72aY8jITbH4pnl6HSriWbRllrNu3LzENCEhiDmqVKRhV bdoA== X-Gm-Message-State: ABuFfog+PpdRExo+cs4rq0irwbcaD8yhvDb7QbJLVui0XzEdU76NpkFY E45p4PAJk/oFXfF5nrXNqrR0L/LvDus= X-Google-Smtp-Source: ACcGV62Byyeam3x3ZNKuM4dQjkZoVcQGKRUtSOUJDzyxAWIspRpUjCXaw+/Cn2Dvksx9txMYvdMiXg== X-Received: by 2002:a0c:9e2a:: with SMTP id p42-v6mr11431093qve.54.1538133520350; Fri, 28 Sep 2018 04:18:40 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f10-v6sm2387876qtq.52.2018.09.28.04.18.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:39 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 08/42] btrfs: dump block_rsv whe dumping space info Date: Fri, 28 Sep 2018 07:17:47 -0400 Message-Id: <20180928111821.24376-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For enospc_debug having the block rsvs is super helpful to see if we've done something wrong. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval Reviewed-by: David Sterba --- fs/btrfs/extent-tree.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index da73b3e5bc39..c9913c59686b 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -7918,6 +7918,15 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, return ret; } +#define DUMP_BLOCK_RSV(fs_info, rsv_name) \ +do { \ + struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \ + spin_lock(&__rsv->lock); \ + btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \ + __rsv->size, __rsv->reserved); \ + spin_unlock(&__rsv->lock); \ +} while (0) + static void dump_space_info(struct btrfs_fs_info *fs_info, struct btrfs_space_info *info, u64 bytes, int dump_block_groups) @@ -7937,6 +7946,12 @@ static void dump_space_info(struct btrfs_fs_info *fs_info, info->bytes_readonly); spin_unlock(&info->lock); + DUMP_BLOCK_RSV(fs_info, global_block_rsv); + DUMP_BLOCK_RSV(fs_info, trans_block_rsv); + DUMP_BLOCK_RSV(fs_info, chunk_block_rsv); + DUMP_BLOCK_RSV(fs_info, delayed_block_rsv); + DUMP_BLOCK_RSV(fs_info, delayed_refs_rsv); + if (!dump_block_groups) return; From patchwork Fri Sep 28 11:17:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D532B15A7 for ; Fri, 28 Sep 2018 11:18:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CBD912B061 for ; Fri, 28 Sep 2018 11:18:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BFF362B0CF; Fri, 28 Sep 2018 11:18:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 671532B061 for ; Fri, 28 Sep 2018 11:18:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729406AbeI1RmA (ORCPT ); Fri, 28 Sep 2018 13:42:00 -0400 Received: from mail-qt1-f178.google.com ([209.85.160.178]:42022 "EHLO mail-qt1-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1Rl7 (ORCPT ); Fri, 28 Sep 2018 13:41:59 -0400 Received: by mail-qt1-f178.google.com with SMTP id z8-v6so6114830qto.9 for ; Fri, 28 Sep 2018 04:18:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=yYx+AcpbxeLRdPqDImtMM4yf4Rg6f5T1yjgQiHsfUlY=; b=xyh5YsQd1dY496OnahxCJ7gwUX3hHNiNBHNW9p5RSC9F/NzgUB1S3jN1kyBY+1Par3 jiKQ/L5LaLdF1jdbMlY13eeJIpcu4S8o3UQnpv5PzXVEzIGGxdBnqP74EzzOIz081gUP p48XLCB032UGouf07coDJkC0A1eJ4ckAEFABgBr/NbF+0ZCn9otUdl1ESi+0EZr4plEs sOlX5n/hwaag6d8qmOLpScNRbsaIYhAqCW20JZ0FpxB5iGuWEIwGcQtnG7PB/NdAe6xG O18txdxley0ywn/7HgA/eujEpq3L8Wm+gipN1gOCM5lhyDtoL/Ut8TobcpVNJ1SPcStO +SfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=yYx+AcpbxeLRdPqDImtMM4yf4Rg6f5T1yjgQiHsfUlY=; b=JhG5OOd4/P7uWHjoJER7bJVh+k3fqHxQRX46MmcV1RKlWoLFsr3SM2jSYC6TVLQg95 Cf+ym53vhsvVS0Sem0MsXpLKJPsPIY52Uki61wjqlTYAOomFlIN6IEe3QV/NJKGRLTUs PFwjMuTnZCogHe4zrc5E1AhvQMy0QSLSBMLSFHG/OPP72P1amwz2IW6RMHd3LC8gx8Eq RbUYUvGAMx7aZ9CyN/RQpT+WbjJzt08g8r4LKMVZvJ+TAxA7KIsZevM2pBJ0G8Bt4hUZ WfS6CbsBMfLwP0G3YSTxgwWSEFBfyGJfXLm9xmsq1J2bL1B0864XlM13jma/IJ40+owd qIFQ== X-Gm-Message-State: ABuFfojYwh/RXS6jkM2bDhZbOs53vS10EjosoQT/yPQQc4gjulNKXsb8 L80XP2YyD5pufjlYOxJs77QoNabjfnA= X-Google-Smtp-Source: ACcGV63URrSCe0p12sWYeHvFkdYX48zvn9fdIiis94hC+ELl7QHnekpLMn+eFED0naOYiXjiJ8ffHA== X-Received: by 2002:aed:32e4:: with SMTP id z91-v6mr9050397qtd.106.1538133522147; Fri, 28 Sep 2018 04:18:42 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f184-v6sm2610770qkc.23.2018.09.28.04.18.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:41 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 09/42] btrfs: release metadata before running delayed refs Date: Fri, 28 Sep 2018 07:17:48 -0400 Message-Id: <20180928111821.24376-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We want to release the unused reservation we have since it refills the delayed refs reserve, which will make everything go smoother when running the delayed refs if we're short on our reservation. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 117e0c4a914a..a0f19ca0bd6c 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1922,6 +1922,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } + btrfs_trans_release_metadata(trans); + trans->block_rsv = NULL; + /* make a pass through all the delayed refs we have so far * any runnings procs may add more while we are here */ @@ -1931,9 +1934,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } - btrfs_trans_release_metadata(trans); - trans->block_rsv = NULL; - cur_trans = trans->transaction; /* From patchwork Fri Sep 28 11:17:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 197FD6CB for ; Fri, 28 Sep 2018 11:18:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10DCD2B061 for ; Fri, 28 Sep 2018 11:18:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 051E02B0CF; Fri, 28 Sep 2018 11:18:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A39532B061 for ; Fri, 28 Sep 2018 11:18:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729418AbeI1RmC (ORCPT ); Fri, 28 Sep 2018 13:42:02 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:46158 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmC (ORCPT ); Fri, 28 Sep 2018 13:42:02 -0400 Received: by mail-qk1-f194.google.com with SMTP id q12-v6so3530730qkl.13 for ; Fri, 28 Sep 2018 04:18:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=+HzhX5veQVR7KYguCDJIe+MJkMCf434goypDockwhKY=; b=Yv2f3IeLb0lelUrvrsnzmAzNzjBYP7ID/Sf6aD9tQIVsJTggG0w7xY/N1uBwyxAd4I u6zZWzhbYNYY4ankDK09cfyc+CICrSUJVpAw7hBM24KgDCj6OJyJcNPkXO6on+4QjlyQ rEsJT/ieOJpedFzQrObbu2Ku1WfJqyOffoIDjdfhapx/UjWYN2dqjDWJ4UCre40AJHfv RgWMM55OcBVFGqUoZOHUlrLMTusMx2s4SYFuBghA+1M6WfmAOqO/rUNEgvvlME8EA2O5 itHNUy4dAAZKbzsEr6gv9c2RvhXVrtzNa+ZxjRTURPVw8gPIbSjGnTVgKWPcLGr1/VwD oVog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+HzhX5veQVR7KYguCDJIe+MJkMCf434goypDockwhKY=; b=rUE8gF2EyuS8iA3mVHibonM0VeMlvTw2QQUifDJxqm/DgRLjZirJv1tiNqw/CuWBF2 GnQbV6zealTYWizangiQFmbPy5b9B+IwxMjuz98S9vXJHv6EqtBNY6wx9u0WLW8MCfAI CVgk9oQgHem/593vEKdXInEcIx3tlp7BhePyuY/AdFUdFc2twaqeQlF4Kqfp78RGqPRh V99E9zPwTp48UMcuy6Yn4/T+8+NqFVCma2dSz5L/Yotj1SzUBi2d6abMcbJuR4AAETid 1xwsAsqVGjUbq5jmb/x2i8F4aGdXIsWi+i/emyHmYCPrv/rKVZg77UuhTYSG5fLlDw5P eBEw== X-Gm-Message-State: ABuFfojFe7nE2aQxytrNEsdcobARwUeA5CeuGd5+17GP5PXANLwBZjiZ cbyzAtvvBmqjg+J0ynmiI43fs+bB/+w= X-Google-Smtp-Source: ACcGV60xMtDNKU9hxUosg8ZkY7YbMZ/L8HhT46lel3/jQM90hDP/Kva+Soc3igIw3/hK7bnSaZHjVA== X-Received: by 2002:a37:aac5:: with SMTP id t188-v6mr11490373qke.250.1538133524032; Fri, 28 Sep 2018 04:18:44 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id t15-v6sm2578317qke.74.2018.09.28.04.18.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:43 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 10/42] btrfs: protect space cache inode alloc with nofs Date: Fri, 28 Sep 2018 07:17:49 -0400 Message-Id: <20180928111821.24376-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/free-space-cache.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index c3888c113d81..e077ad3b4549 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "ctree.h" #include "free-space-cache.h" #include "transaction.h" @@ -47,6 +48,7 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root, struct btrfs_free_space_header *header; struct extent_buffer *leaf; struct inode *inode = NULL; + unsigned nofs_flag; int ret; key.objectid = BTRFS_FREE_SPACE_OBJECTID; @@ -68,7 +70,13 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root, btrfs_disk_key_to_cpu(&location, &disk_key); btrfs_release_path(path); + /* + * We are often under a trans handle at this point, so we need to make + * sure NOFS is set to keep us from deadlocking. + */ + nofs_flag = memalloc_nofs_save(); inode = btrfs_iget(fs_info->sb, &location, root, NULL); + memalloc_nofs_restore(nofs_flag); if (IS_ERR(inode)) return inode; From patchwork Fri Sep 28 11:17:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5828B15A7 for ; Fri, 28 Sep 2018 11:18:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E4BD2B0C5 for ; Fri, 28 Sep 2018 11:18:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 42C302B0F2; Fri, 28 Sep 2018 11:18:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C78902B0C5 for ; Fri, 28 Sep 2018 11:18:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729429AbeI1RmE (ORCPT ); Fri, 28 Sep 2018 13:42:04 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:40136 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmE (ORCPT ); Fri, 28 Sep 2018 13:42:04 -0400 Received: by mail-qk1-f194.google.com with SMTP id c126-v6so3556743qkd.7 for ; Fri, 28 Sep 2018 04:18:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=09uUQnLegRaGDa0gAKaXhW+r7F/us0LEAwdMnUEs3rc=; b=UTVAeyO1rD3U0Vi+yP2tIQqOB55NjYupSWEkX0IfZuqsGLKr9woE9Q/uLbTb/eRojC gp/SOsBuV8MSecePRD+E2eFIWlhCgE+zpnRtnGtkjIn+6/lpO2P8mWnf8VlbXX2wM07q CXaSyeXL+qhT1Cr5AFR4oN0lDpxK0vcqpNqXw8Xqd32L2oDnT3pMGwV/2bePtPqlxWEv EjMccYD+wFCR5TDJXRI5l777VNtK/I4hbM1TEQv90jyMcyMaO+0lb4hsFRrlrbsb1QHi EnXuCuE93ucgHjBxgx+SUP2sH8QFyzMMaAWjJCPjj+Ry4TbLyoYBW2Us6eMszJzJFgSu Cy0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=09uUQnLegRaGDa0gAKaXhW+r7F/us0LEAwdMnUEs3rc=; b=s6gWg54OulHsD2LdTxLBhtQZa4NsScOUvvz+ZraSoKad53xvoW8WZwpKF1hu4ovwtM dCFiX5tcgIOWO2YRt2wp1bAr97yBwoOlacMdny3RqVimKG7XLGpVaG+q4VL2u8pwH055 7541TTBm81So5kFwiOTMoje//ENQnKLRR9yk6txoOCScWTurHOC0+3dpnwYyLm2pfmRf RMCr4NCv17xqpMMJZ5dOK66HQSVo+6pRiUGHH3wUyTF1deQPmnD331Wa/N43lslDsJ4+ Ak7g6B+EKuhdvyH42ap6dQAXqLnbpTyXWNtE+PwODdVGAO/DgqzInhbsHTh4iYRFOWmm ZBgQ== X-Gm-Message-State: ABuFfojgoucDyN41jQk+n3rjJ8TIlpB80+g/gNhpxN2EutGLba+K+uL2 rAfa9FFsCGT/K52vrp6e7yZgYgSgUH4= X-Google-Smtp-Source: ACcGV60CPA7tJufJTV2vBvfjF/TqC/yLeFrp19DsAlcY4nn73zOuy769JwM7jiNZSV47e7NGWW4L1w== X-Received: by 2002:a37:8106:: with SMTP id c6-v6mr11203883qkd.261.1538133526015; Fri, 28 Sep 2018 04:18:46 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e195-v6sm2816447qka.2.2018.09.28.04.18.44 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:44 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 11/42] btrfs: fix truncate throttling Date: Fri, 28 Sep 2018 07:17:50 -0400 Message-Id: <20180928111821.24376-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We have a bunch of magic to make sure we're throttling delayed refs when truncating a file. Now that we have a delayed refs rsv and a mechanism for refilling that reserve simply use that instead of all of this magic. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 79 ++++++++++++-------------------------------------------- 1 file changed, 17 insertions(+), 62 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index cd00ec869c96..5a91055a13b2 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -4493,31 +4493,6 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry) return err; } -static int truncate_space_check(struct btrfs_trans_handle *trans, - struct btrfs_root *root, - u64 bytes_deleted) -{ - struct btrfs_fs_info *fs_info = root->fs_info; - int ret; - - /* - * This is only used to apply pressure to the enospc system, we don't - * intend to use this reservation at all. - */ - bytes_deleted = btrfs_csum_bytes_to_leaves(fs_info, bytes_deleted); - bytes_deleted *= fs_info->nodesize; - ret = btrfs_block_rsv_add(root, &fs_info->trans_block_rsv, - bytes_deleted, BTRFS_RESERVE_NO_FLUSH); - if (!ret) { - trace_btrfs_space_reservation(fs_info, "transaction", - trans->transid, - bytes_deleted, 1); - trans->bytes_reserved += bytes_deleted; - } - return ret; - -} - /* * Return this if we need to call truncate_block for the last bit of the * truncate. @@ -4562,7 +4537,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u64 bytes_deleted = 0; bool be_nice = false; bool should_throttle = false; - bool should_end = false; BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY); @@ -4775,15 +4749,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, btrfs_abort_transaction(trans, ret); break; } - if (btrfs_should_throttle_delayed_refs(trans, fs_info)) - btrfs_async_run_delayed_refs(fs_info, - trans->delayed_ref_updates * 2, - trans->transid, 0); if (be_nice) { - if (truncate_space_check(trans, root, - extent_num_bytes)) { - should_end = true; - } if (btrfs_should_throttle_delayed_refs(trans, fs_info)) should_throttle = true; @@ -4795,7 +4761,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, if (path->slots[0] == 0 || path->slots[0] != pending_del_slot || - should_throttle || should_end) { + should_throttle) { if (pending_del_nr) { ret = btrfs_del_items(trans, root, path, pending_del_slot, @@ -4807,23 +4773,24 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, pending_del_nr = 0; } btrfs_release_path(path); - if (should_throttle) { - unsigned long updates = trans->delayed_ref_updates; - if (updates) { - trans->delayed_ref_updates = 0; - ret = btrfs_run_delayed_refs(trans, - updates * 2); - if (ret) - break; - } - } + /* - * if we failed to refill our space rsv, bail out - * and let the transaction restart + * We can generate a lot of delayed refs, so we need to + * throttle every once and a while and make sure we're + * adding enough space to keep up with the work we are + * generating. Since we hold a transaction here we + * can't flush, and we don't want to FLUSH_LIMIT because + * we could have generated too many delayed refs to + * actually allocate, so just bail if we're short and + * let the normal reservation dance happen higher up. */ - if (should_end) { - ret = -EAGAIN; - break; + if (should_throttle) { + ret = btrfs_throttle_delayed_refs(fs_info, + BTRFS_RESERVE_NO_FLUSH); + if (ret) { + ret = -EAGAIN; + break; + } } goto search_again; } else { @@ -4849,18 +4816,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, } btrfs_free_path(path); - - if (be_nice && bytes_deleted > SZ_32M && (ret >= 0 || ret == -EAGAIN)) { - unsigned long updates = trans->delayed_ref_updates; - int err; - - if (updates) { - trans->delayed_ref_updates = 0; - err = btrfs_run_delayed_refs(trans, updates * 2); - if (err) - ret = err; - } - } return ret; } From patchwork Fri Sep 28 11:17:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619689 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC7ED15A7 for ; Fri, 28 Sep 2018 11:18:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A48FF2B0C5 for ; Fri, 28 Sep 2018 11:18:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 98EE72B0F2; Fri, 28 Sep 2018 11:18:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 443642B0C5 for ; Fri, 28 Sep 2018 11:18:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729436AbeI1RmF (ORCPT ); Fri, 28 Sep 2018 13:42:05 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:45309 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmF (ORCPT ); Fri, 28 Sep 2018 13:42:05 -0400 Received: by mail-qt1-f196.google.com with SMTP id l2-v6so6092598qtr.12 for ; Fri, 28 Sep 2018 04:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=471cQ/dE/uVCNlSocUpXaqoy8Ktro672PmaGzD5v8vY=; b=kB5k4RIVewxdSRXEl8RLr6OZZ4YF8a/ksuTYinBDE1tn90vOiybY8dMPDgDcL9WGSi hJ6VNclRPggaiaTIeArSaFUvrBhr+nK3lH7SHFoytj6NM23azETBWJI2Sv4zqEDMROOS UgTCi7lLLckNZOmHFf0Xkr5fWm3QUB+zy7t6E2Qev3C6SPNBRRocd5APFlauCvpem1fo Rt0Ws21TX/tTfAPT3mXKH99aD9fYd0Ovbrch31FTtHureSamO+3/xVV2iHNBS8KiIj1z xBUTtj/O5ZpSyQD4BzCJlYblV3mLiLgUD/kiXK5sw/jWpFn0Ia2Ds/DDTdN3N+6wJXMS H+Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=471cQ/dE/uVCNlSocUpXaqoy8Ktro672PmaGzD5v8vY=; b=Qyz5YXnYBrjCtf83nDosVw1qfse/m9TAlPBapPc33Zfm4XnAiv8dPoEYRABIDwQytA EWA4geVt4tBQejoKpzb4nN2QiKlI4SzAt6bopy34nttElE9KcW81uppEcH+m0i5vqY7n 93feMEfoa/cYhZUKFgsi9OOoltyg5/dvoKr0RpIHLDGsYxgcD08QvfWXtay7SVsf3KaJ hcycbT4xAe29mrgx5IWkewFbtaZwpJlx3tbWu0uSlZNXFRM6x/EX7pops2pyQsz4O5wI iVxsATWR8TOCUSJHnc/Y1iPFCyE/9V6tpLOZRogPYLG04+PMPJsLlazagBup6r6PWk5V naag== X-Gm-Message-State: ABuFfojnf6AW+41Dx642G80fGKajTotcNOG8EhXWYIEoabUut0j+GmNN GaJ1YUr3ZOkWfa576AuuG10Wo5iIz+U= X-Google-Smtp-Source: ACcGV61cf6Z2RoC6JNOddQOikl3iAaGiCJnFN6ueghNTLjYCzwqCRSs1WJSXHPnycanWKlzuIsxSIA== X-Received: by 2002:aed:3384:: with SMTP id v4-v6mr11926048qtd.267.1538133527773; Fri, 28 Sep 2018 04:18:47 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id h58-v6sm3132728qtk.60.2018.09.28.04.18.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:46 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 12/42] btrfs: don't use global rsv for chunk allocation Date: Fri, 28 Sep 2018 07:17:51 -0400 Message-Id: <20180928111821.24376-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We've done this forever because of the voodoo around knowing how much space we have. However we have better ways of doing this now, and on normal file systems we'll easily have a global reserve of 512MiB, and since metadata chunks are usually 1GiB that means we'll allocate metadata chunks more readily. Instead use the actual used amount when determining if we need to allocate a chunk or not. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c9913c59686b..c0f6110419b2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4374,21 +4374,12 @@ static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global) static int should_alloc_chunk(struct btrfs_fs_info *fs_info, struct btrfs_space_info *sinfo, int force) { - struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; u64 bytes_used = btrfs_space_info_used(sinfo, false); u64 thresh; if (force == CHUNK_ALLOC_FORCE) return 1; - /* - * We need to take into account the global rsv because for all intents - * and purposes it's used space. Don't worry about locking the - * global_rsv, it doesn't change except when the transaction commits. - */ - if (sinfo->flags & BTRFS_BLOCK_GROUP_METADATA) - bytes_used += calc_global_rsv_need_space(global_rsv); - /* * in limited mode, we want to have some free space up to * about 1% of the FS size. From patchwork Fri Sep 28 11:17:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7A1F6CB for ; Fri, 28 Sep 2018 11:18:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F3452B0C5 for ; Fri, 28 Sep 2018 11:18:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 934E72B0F2; Fri, 28 Sep 2018 11:18:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28EDD2B0CF for ; Fri, 28 Sep 2018 11:18:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729440AbeI1RmH (ORCPT ); Fri, 28 Sep 2018 13:42:07 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:45313 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmH (ORCPT ); Fri, 28 Sep 2018 13:42:07 -0400 Received: by mail-qt1-f195.google.com with SMTP id l2-v6so6092683qtr.12 for ; Fri, 28 Sep 2018 04:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=aHABd3s47VJA+IGNtJA9Inn+i4GdiqS25/qy6diaJ4I=; b=BmHiG9g7knyfBbRejaYbMKh3Y0flDf0Sdb9fWMyFTfiBYmR9hlSVA1oUcl7SwyJ8xf l9q12YsynxEVw19MBcvu9RJVvSnPSRytE7gU+N9bzTZ1q1oTawxPRSnD77YTASrdJ7+D u+l6Zo56993yh5zsebH8GNvfKwsS2mgIimTvsk7EeG1zMrhFz1KdJbIDBc5HxtiH+43M n315NWcw1kSd/RzbcLfaVhUzgV2H3w/eeCptX8W0kKEF+G6yBkMgURssqno79HBGlUFD 2HxLeORPiFjJ93Q/9fI3HmtA8ZiCLjJGq4xKBQn26R2YWpzavOHzskFFtlAI5cjMI8kU jZjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=aHABd3s47VJA+IGNtJA9Inn+i4GdiqS25/qy6diaJ4I=; b=jVIBiBLV8iq5bJmaaKOw/Iu9lsal2DYYH/XfiYLSxCKlwnRD0VeGW0PZrCxrezat7D nuWFOseMcZgBwBxBFkxO2v2rSYYvjPhSpemorDZX9LWkLq2oGqeLjeLTBIXi7b52d+sP 4rSIbzPtRWH66OI1BQ3Sh4vp+NXPl+ucyS9W85+OxPr6FIQ4FI7NId+PPY4yBMqmjLIS XEtgudM6YiMx/td17iFyiFe8/uivsyA6gn4N/JqVR83PB3H3woFIQyLxnMfKlWLy32a1 z4kh6Elhjqtf13iz6NB92DLyalefvv2UzWMf3eV6jcBHrwDSlUCkBdxfGql1N8adrOM6 I0EQ== X-Gm-Message-State: ABuFfohIDqVGVhB+eZl2GUSwFqGivy7J70L1wopqipvdHzfiSYDKiwyj A1SmXaFanum6HZn1m+HN+XQFYnPo0Ww= X-Google-Smtp-Source: ACcGV63bhhCdHMmvvV6wg0P2euP3aL5WmgttnIXS0ZDoojnx4Uw7KavQPqw0HR8XLL51ny9j4HTSug== X-Received: by 2002:ac8:2866:: with SMTP id 35-v6mr12183272qtr.18.1538133529633; Fri, 28 Sep 2018 04:18:49 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id l24-v6sm2528065qtk.85.2018.09.28.04.18.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:48 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 13/42] btrfs: add ALLOC_CHUNK_FORCE to the flushing code Date: Fri, 28 Sep 2018 07:17:52 -0400 Message-Id: <20180928111821.24376-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With my change to no longer take into account the global reserve for metadata allocation chunks we have this side-effect for mixed block group fs'es where we are no longer allocating enough chunks for the data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE step to the flushing state machine. This will only get used if we've already made a full loop through the flushing machinery and tried committing the transaction. If we have then we can try and force a chunk allocation since we likely need it to make progress. This resolves the issues I was seeing with the mixed bg tests in xfstests with my previous patch. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 ++- fs/btrfs/extent-tree.c | 18 +++++++++++++++++- include/trace/events/btrfs.h | 1 + 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 1a2c3b629af2..29db902511c1 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2730,7 +2730,8 @@ enum btrfs_flush_state { FLUSH_DELALLOC = 5, FLUSH_DELALLOC_WAIT = 6, ALLOC_CHUNK = 7, - COMMIT_TRANS = 8, + ALLOC_CHUNK_FORCE = 8, + COMMIT_TRANS = 9, }; int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c0f6110419b2..cd2280962c8c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4914,6 +4914,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_end_transaction(trans); break; case ALLOC_CHUNK: + case ALLOC_CHUNK_FORCE: trans = btrfs_join_transaction(root); if (IS_ERR(trans)) { ret = PTR_ERR(trans); @@ -4921,7 +4922,9 @@ static void flush_space(struct btrfs_fs_info *fs_info, } ret = do_chunk_alloc(trans, btrfs_metadata_alloc_profile(fs_info), - CHUNK_ALLOC_NO_FORCE); + (state == ALLOC_CHUNK) ? + CHUNK_ALLOC_NO_FORCE : + CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); if (ret > 0 || ret == -ENOSPC) ret = 0; @@ -5057,6 +5060,19 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) commit_cycles--; } + /* + * We don't want to force a chunk allocation until we've tried + * pretty hard to reclaim space. Think of the case where we + * free'd up a bunch of space and so have a lot of pinned space + * to reclaim. We would rather use that than possibly create a + * underutilized metadata chunk. So if this is our first run + * through the flushing state machine skip ALLOC_CHUNK_FORCE and + * commit the transaction. If nothing has changed the next go + * around then we can force a chunk allocation. + */ + if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles) + flush_state++; + if (flush_state > COMMIT_TRANS) { commit_cycles++; if (commit_cycles > 2) { diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index 7d205e50b09c..fdb23181b5b7 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -1051,6 +1051,7 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ + { ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) TRACE_EVENT(btrfs_flush_space, From patchwork Fri Sep 28 11:17:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B9196CB for ; Fri, 28 Sep 2018 11:18:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41F122B061 for ; Fri, 28 Sep 2018 11:18:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 364492B0CF; Fri, 28 Sep 2018 11:18:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA94D2B061 for ; Fri, 28 Sep 2018 11:18:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729444AbeI1RmJ (ORCPT ); Fri, 28 Sep 2018 13:42:09 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:39830 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmJ (ORCPT ); Fri, 28 Sep 2018 13:42:09 -0400 Received: by mail-qk1-f193.google.com with SMTP id q5-v6so594886qki.6 for ; Fri, 28 Sep 2018 04:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ZS+Ro2q0p6YcbYs63sQ3GFXPklxc57Dq9Ikc/xk3dZs=; b=wtuOPbf1NvjgvZlWJJ2Oy3XrETx92rphs2HxZgfxllIPsXBXQ5gyDeX9Ioffvnd6g7 54U5tcmY/MJDXC4C5rDgtCgSBbSF7NFGFW3nhElTsQpJzz0c+rRd7iyPmsf/ccrFb7rP owzlRgbTMH21bTN2AyKrY0fbj4bCTkeSvfNDvAC/gGVQxoKAaX5Rqh2VxmSuzJxABklY Q0oP71KGgMKRojASwC2nZGUjcoL8rRNVOBJifR3dZYkWdPJA5TPvVw0QJaia7Zusak3l saQa7nOoby+wJraGYLHgVe54LS9z6xELSMdFQNiFSE4tzlUQcqzN7Souc4fH7+w5d2YO 38KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ZS+Ro2q0p6YcbYs63sQ3GFXPklxc57Dq9Ikc/xk3dZs=; b=hG3/U30AOOJfiUMEB9lsT76oM2IBPCnCWe5EuRBaLoShWpJotkyZ9jqsIKisqEeJxW UTanNbY7lYv4Ajgi0QO0VRgoyY3L1pVpk8WB7jpprP0WemVTI/gzB1pUKYMojWFHK9Dt rv5nnIA/AcZLuyZ3Y1BoQVrqPydMmhKn45wvAFeDcNziYYKm1l+BAXKTYdbU+C1Nt96B banscu9ngSBhsEKryefETK7NIIjirTDwxldCJ/xLjAd8kEog5klx0pzpD4+hZfgGuVK4 A1dxmeNeE2StV6YmbiBRjvkZPG7bi6IxngpR1TOJv+xB+I+PU56dR5GHWh/Bp8nY0ZSM JX4w== X-Gm-Message-State: ABuFfojsIWlJuxuMmrWN2y+RAZt36mBXjyKL5h/zhr1NI69BC28m1I0R 0fjqSh7qETzKsNzkfC9aPZ2jcFLTvjA= X-Google-Smtp-Source: ACcGV615TvH/T83Xqjkds7Vkw/li72+fuPhRPjh4yhpNWAfEmFOCc/CW8HdWvTybvbgf8nPF3hLiVA== X-Received: by 2002:ae9:ef11:: with SMTP id d17-v6mr11503904qkg.80.1538133531641; Fri, 28 Sep 2018 04:18:51 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o5-v6sm3213326qtc.63.2018.09.28.04.18.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:50 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 14/42] btrfs: reset max_extent_size properly Date: Fri, 28 Sep 2018 07:17:53 -0400 Message-Id: <20180928111821.24376-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we use up our block group before allocating a new one we'll easily get a max_extent_size that's set really really low, which will result in a lot of fragmentation. We need to make sure we're resetting the max_extent_size when we add a new chunk or add new space. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index cd2280962c8c..f84537a1d7eb 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4573,6 +4573,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags, goto out; } else { ret = 1; + space_info->max_extent_size = 0; } space_info->force_alloc = CHUNK_ALLOC_NO_FORCE; @@ -6671,6 +6672,7 @@ static int btrfs_free_reserved_bytes(struct btrfs_block_group_cache *cache, space_info->bytes_readonly += num_bytes; cache->reserved -= num_bytes; space_info->bytes_reserved -= num_bytes; + space_info->max_extent_size = 0; if (delalloc) cache->delalloc_bytes -= num_bytes; From patchwork Fri Sep 28 11:17:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619695 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7915115A7 for ; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 700C02B0C5 for ; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 646112B0CF; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D7332B061 for ; Fri, 28 Sep 2018 11:18:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729450AbeI1RmM (ORCPT ); Fri, 28 Sep 2018 13:42:12 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:35851 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmL (ORCPT ); Fri, 28 Sep 2018 13:42:11 -0400 Received: by mail-qk1-f194.google.com with SMTP id a85-v6so3577780qkg.3 for ; Fri, 28 Sep 2018 04:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=SV6VzMZmuQU6kRBrptnXRYkuIetlbfVghoSBpsgva9M=; b=F6Xi0Z4GI/Scnb46KjxSZzGz7LC20OMYxyOXJ/idyIpV2nscaMRiCJYKcAkA3pTZA2 aPQMMDxW8lZo09PaBFcwwnGK+RIzmRKTLP6BErtlL3oqNpz+2KY0iJpwL2ex1fTRKndm 6adRZxderLNOLWbgNCoORJ8IyMYtUv6L98gEBBaBrvBVi20ckZcThxils0YS8k1FcMOb c1rw3QuJbNIJQMU7R3zj1ApZhTmfOvm48SCFusYVRWQ5congmMIknCXUofLA90CeooFr SkHQGTY4xJXWshedqJxDXXhkwg6Sz2xiNVIVoxxBJmD/qdTEGPE31dQcts9FP0icTGnc Z/Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=SV6VzMZmuQU6kRBrptnXRYkuIetlbfVghoSBpsgva9M=; b=SC8Y/b1pXIzMM8qbJOgQ3BXggNSuVSfwk44U1wDEP856TT8JZiQfjyFxjjnmPKUqh+ bn5Cy96DY+5Mnohc4LOH4RPnAxJ30Q2xJDu+WRhjpeC8Nl70JFq5xbLgLtagXjKvyMkv BP+U/Ps9kPKVMcpX2qfgAJA3fF0WsIuZ9q4ULGFJnkR0/FPQr2FHSuzP63NGHyW7zsKk iHiqf/LZ5eZFgcw9BBUz5CGHptvQRsVHFgZgrMrU6rrdE0zf/AIqSCsxmDp0i2/OBoDg UQJIsbpmlwk1pbxL3fveTvUZ4Yivd7tpSr/Iw4PFTN5y4sNmoVdjdNJZe9oBHxBwrnGs tYjQ== X-Gm-Message-State: ABuFfog2HYudH3BnRUZ0Nr9yBjmw1mjcEnEFsOE1VLviHNna7NXee1NK zSk6LKyPk/iGqumrBxdBtmwSCoOB/2A= X-Google-Smtp-Source: ACcGV613RtOAzqvKRYNFH7H0Q2Z8EdpRwrXn/eLWv92RPUoFCeZ949/zbzLT/7QsFEfdQ+f5JzfkEQ== X-Received: by 2002:a37:a48d:: with SMTP id n135-v6mr4707404qke.160.1538133533373; Fri, 28 Sep 2018 04:18:53 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 23-v6sm2711992qkn.11.2018.09.28.04.18.52 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:52 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 15/42] btrfs: don't enospc all tickets on flush failure Date: Fri, 28 Sep 2018 07:17:54 -0400 Message-Id: <20180928111821.24376-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With the introduction of the per-inode block_rsv it became possible to have really really large reservation requests made because of data fragmentation. Since the ticket stuff assumed that we'd always have relatively small reservation requests it just killed all tickets if we were unable to satisfy the current request. However this is generally not the case anymore. So fix this logic to instead see if we had a ticket that we were able to give some reservation to, and if we were continue the flushing loop again. Likewise we make the tickets use the space_info_add_old_bytes() method of returning what reservation they did receive in hopes that it could satisfy reservations down the line. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f84537a1d7eb..7a53f6a29ebc 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4779,6 +4779,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } struct reserve_ticket { + u64 orig_bytes; u64 bytes; int error; struct list_head list; @@ -5000,7 +5001,7 @@ static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info, !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)); } -static void wake_all_tickets(struct list_head *head) +static bool wake_all_tickets(struct list_head *head) { struct reserve_ticket *ticket; @@ -5009,7 +5010,10 @@ static void wake_all_tickets(struct list_head *head) list_del_init(&ticket->list); ticket->error = -ENOSPC; wake_up(&ticket->wait); + if (ticket->bytes != ticket->orig_bytes) + return true; } + return false; } /* @@ -5077,8 +5081,12 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) if (flush_state > COMMIT_TRANS) { commit_cycles++; if (commit_cycles > 2) { - wake_all_tickets(&space_info->tickets); - space_info->flush = 0; + if (wake_all_tickets(&space_info->tickets)) { + flush_state = FLUSH_DELAYED_ITEMS_NR; + commit_cycles--; + } else { + space_info->flush = 0; + } } else { flush_state = FLUSH_DELAYED_ITEMS_NR; } @@ -5130,10 +5138,11 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, u64 orig_bytes) + struct reserve_ticket *ticket) { DEFINE_WAIT(wait); + u64 reclaim_bytes = 0; int ret = 0; spin_lock(&space_info->lock); @@ -5154,14 +5163,12 @@ static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, ret = ticket->error; if (!list_empty(&ticket->list)) list_del_init(&ticket->list); - if (ticket->bytes && ticket->bytes < orig_bytes) { - u64 num_bytes = orig_bytes - ticket->bytes; - space_info->bytes_may_use -= num_bytes; - trace_btrfs_space_reservation(fs_info, "space_info", - space_info->flags, num_bytes, 0); - } + if (ticket->bytes && ticket->bytes < ticket->orig_bytes) + reclaim_bytes = ticket->orig_bytes - ticket->bytes; spin_unlock(&space_info->lock); + if (reclaim_bytes) + space_info_add_old_bytes(fs_info, space_info, reclaim_bytes); return ret; } @@ -5187,6 +5194,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, { struct reserve_ticket ticket; u64 used; + u64 reclaim_bytes = 0; int ret = 0; ASSERT(orig_bytes); @@ -5222,6 +5230,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, * the list and we will do our own flushing further down. */ if (ret && flush != BTRFS_RESERVE_NO_FLUSH) { + ticket.orig_bytes = orig_bytes; ticket.bytes = orig_bytes; ticket.error = 0; init_waitqueue_head(&ticket.wait); @@ -5262,25 +5271,21 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, return ret; if (flush == BTRFS_RESERVE_FLUSH_ALL) - return wait_reserve_ticket(fs_info, space_info, &ticket, - orig_bytes); + return wait_reserve_ticket(fs_info, space_info, &ticket); ret = 0; priority_reclaim_metadata_space(fs_info, space_info, &ticket); spin_lock(&space_info->lock); if (ticket.bytes) { - if (ticket.bytes < orig_bytes) { - u64 num_bytes = orig_bytes - ticket.bytes; - space_info->bytes_may_use -= num_bytes; - trace_btrfs_space_reservation(fs_info, "space_info", - space_info->flags, - num_bytes, 0); - - } + if (ticket.bytes < orig_bytes) + reclaim_bytes = orig_bytes - ticket.bytes; list_del_init(&ticket.list); ret = -ENOSPC; } spin_unlock(&space_info->lock); + + if (reclaim_bytes) + space_info_add_old_bytes(fs_info, space_info, reclaim_bytes); ASSERT(list_empty(&ticket.list)); return ret; } From patchwork Fri Sep 28 11:17:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619697 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1AAB3CF1 for ; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D90ED2B061 for ; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD9D42B0CF; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E63F2B061 for ; Fri, 28 Sep 2018 11:18:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729454AbeI1RmN (ORCPT ); Fri, 28 Sep 2018 13:42:13 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:46823 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeI1RmM (ORCPT ); Fri, 28 Sep 2018 13:42:12 -0400 Received: by mail-qt1-f193.google.com with SMTP id h22-v6so6089287qtr.13 for ; Fri, 28 Sep 2018 04:18:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=KEzZC/iUqIPWQQpjLKXekOjt3SWza9+3dFW42ZSpeyc=; b=mpH4vl1e1qsnvjKyFGYH9j8MemRFrNotOWn+pPNkJ0rOFBCRldF9oSy1Q9FrudfUzZ WVdlALbx3O3cU6fzVPK80L3gewph8OE6gZqIn3gN8Z23PEUu/JPSb5l08b5WR5IPlOqf uubaBRyvT3BvBiObMNcLN/nwXSwa3vF9GPnfeEEHKEkA8P22VwIxEjhZkREOy6HPLgF/ VZnuAW/Rbal09bUBBKyXwEk6es4yVbgjBMdyerrPbniWHGgGmNqUtJFkNpfZrn2vRsuH x9r1DI2x9FF8thxFH5P8WnWBa3/5uz9c4rhhhpliOMGLfvFDve28KAFjGACnLrFy9/AW BFag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=KEzZC/iUqIPWQQpjLKXekOjt3SWza9+3dFW42ZSpeyc=; b=CsmrMGuZHynVwH1CzbXEpet4idShDyfLnapQGT1RE+8uj2b8R+FD/klQFfEoqo7JfR ZGVHCKGYUjCxfrOm8YQghbYs3qG3La1hMotTp7S/eUhCB6eB4Po8I2q3Yp1C34m15OuT JPtSpdvliSjlzgt86Sl0Z2X+O/eOmURLedsbICr9UYN1nyrdkEguzCEafAwtBwsWpJty vWN88VI/9+PAgppoNmUXvXSVCa4dSfUv0EFgiEdZSct+pGVEhNawVFEGEhQCImCy4N6b qb850wwB8depEKafwzxgq1P2qhpLUOHVGX5ZqphMX8AjDDllWitZLPo1dG5WtV12aRhT k0Ag== X-Gm-Message-State: ABuFfoh/4uDPDanT1IOdJYjWZFOXNhgAJeI3RQTOj8IetgR+V3LrFj4k xCVdTiBqoW6bPwQ/A89L910hu2iqIG8= X-Google-Smtp-Source: ACcGV61WDEQ8CD6Ygx1jwzatkVT1/F2+h3Hin7G8t7VZVFulCNUFbbcRU3C3NLldI9w5/EowYkaT5g== X-Received: by 2002:ac8:1770:: with SMTP id u45-v6mr369521qtk.285.1538133535166; Fri, 28 Sep 2018 04:18:55 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y143-v6sm2650665qky.95.2018.09.28.04.18.54 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:54 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 16/42] btrfs: loop in inode_rsv_refill Date: Fri, 28 Sep 2018 07:17:55 -0400 Message-Id: <20180928111821.24376-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With severe fragmentation we can end up with our inode rsv size being huge during writeout, which would cause us to need to make very large metadata reservations. However we may not actually need that much once writeout is complete. So instead try to make our reservation, and if we couldn't make it re-calculate our new reservation size and try again. If our reservation size doesn't change between tries then we know we are actually out of space and can error out. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 7a53f6a29ebc..461b8076928b 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5781,10 +5781,11 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, { struct btrfs_root *root = inode->root; struct btrfs_block_rsv *block_rsv = &inode->block_rsv; - u64 num_bytes = 0; + u64 num_bytes = 0, last = 0; u64 qgroup_num_bytes = 0; int ret = -ENOSPC; +again: spin_lock(&block_rsv->lock); if (block_rsv->reserved < block_rsv->size) num_bytes = block_rsv->size - block_rsv->reserved; @@ -5796,6 +5797,13 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, if (num_bytes == 0) return 0; + /* + * If our reservation size hasn't changed since the last time we tried + * to make an allocation we can just bail. + */ + if (last && last == num_bytes) + return -ENOSPC; + ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_num_bytes, true); if (ret) return ret; @@ -5809,8 +5817,22 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, spin_lock(&block_rsv->lock); block_rsv->qgroup_rsv_reserved += qgroup_num_bytes; spin_unlock(&block_rsv->lock); - } else + } else { btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes); + + /* + * If we are fragmented we can end up with a lot of outstanding + * extents which will make our size be much larger than our + * reserved amount. If we happen to try to do a reservation + * here that may result in us trying to do a pretty hefty + * reservation, which we may not need once delalloc flushing + * happens. If this is the case try and do the reserve again. + */ + if (flush == BTRFS_RESERVE_FLUSH_ALL) { + last = num_bytes; + goto again; + } + } return ret; } From patchwork Fri Sep 28 11:17:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 211BB6CB for ; Fri, 28 Sep 2018 11:19:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18A982B061 for ; Fri, 28 Sep 2018 11:19:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CF822B0CF; Fri, 28 Sep 2018 11:19:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ACD862B061 for ; Fri, 28 Sep 2018 11:18:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729459AbeI1RmP (ORCPT ); Fri, 28 Sep 2018 13:42:15 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34735 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmP (ORCPT ); Fri, 28 Sep 2018 13:42:15 -0400 Received: by mail-qt1-f195.google.com with SMTP id x23-v6so6181483qtr.1 for ; Fri, 28 Sep 2018 04:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=plAorLc8fDr39W0E6gJzb3bU7gor3NEIbpaym2AY9dc=; b=DYvsiw7WBddLcuzMn6MM151tKY2mru/yxknTU8OaWobGouaAXjjCK3n6A+aHFp2FMG Qfkq+vwF+05GYdRimHfl/uH3WXkRYinAML95RnZ5Ur4PHoDGAzVkCwHEjf6hjKRd/WHI 3qxrFvFjwJG7LYZ4FCsyjFgg5XQrmSaF5T/UKuB6DcyvGHSIxNRPJxq351gTm5PzpzgD 6brXg21iPyvoTy1LblKb95CHPHdKmeNl1ivIIFRqNmfRhObT1wQCaq5rF/8iMAHVK4wT 4OjQw8s4mTwjvJTP/wwa0DGmoNMb+Ddsi22eGvtEbnIOhc6+DD0UoghOGgHwFETF7Beu FJUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=plAorLc8fDr39W0E6gJzb3bU7gor3NEIbpaym2AY9dc=; b=WJpYOrLRiSoKCIH/yFBvlCu0ady3vhuDliea67C/ovpMsG35eVLN0RCNO/cO1aqmGl SnhUv7rZvofmf6LoaHAhPA9Pf6r09o0FaAjzbv8FFaRnqf0v1/onbcQ/eRO9TCaAEhWg X7IriQJDjtsl7729qlEUmn8RBrL2R2LeHqm/anDXhyI7m6DVAXgXiBIP5Oy9eUb1LJCE bD/pMzCJE9LWjr9+d9xbHQW5YO4ItRWk49YZvPFabfs3L+QscjICUhgc99FZ4MCEm4ld eILguPEtrOEl+XlxhZcglPBraQHdGHpDiwOVEaY22W4MH9TUSREv7wXekTRT7Qb5Qqf7 Y1Zg== X-Gm-Message-State: ABuFfohF/6kyEU8d60ft3ncnxFkoUMEOq9tPi4kNGeJokb+d4hcgiqDm qMM7XyqduzX3Z/G72ADvpT+y5A== X-Google-Smtp-Source: ACcGV62+kUT27Z6X+Nt3vQU2cl4vsUa3MTyB2JRq4F0qUKtKluq9tso8wH1Ytt+0JyO1ReDaezQqYg== X-Received: by 2002:aed:3b97:: with SMTP id r23-v6mr11857892qte.77.1538133536928; Fri, 28 Sep 2018 04:18:56 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m6-v6sm242906qta.50.2018.09.28.04.18.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:56 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 17/42] btrfs: run delayed iputs before committing Date: Fri, 28 Sep 2018 07:17:56 -0400 Message-Id: <20180928111821.24376-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Delayed iputs means we can have final iputs of deleted inodes in the queue, which could potentially generate a lot of pinned space that could be free'd. So before we decide to commit the transaction for ENOPSC reasons, run the delayed iputs so that any potential space is free'd up. If there is and we freed enough we can then commit the transaction and potentially be able to make our reservation. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 461b8076928b..eb64b28196b8 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4823,6 +4823,15 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes) return 0; + /* + * If we have pending delayed iputs then we could free up a bunch of + * pinned space, so make sure we run the iputs before we do our pinned + * bytes check below. + */ + mutex_lock(&fs_info->cleaner_delayed_iput_mutex); + btrfs_run_delayed_iputs(fs_info); + mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return -ENOSPC; From patchwork Fri Sep 28 11:17:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619701 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 67B8615A7 for ; Fri, 28 Sep 2018 11:19:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F63D2B061 for ; Fri, 28 Sep 2018 11:19:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 53D642B0CF; Fri, 28 Sep 2018 11:19:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2C852B061 for ; Fri, 28 Sep 2018 11:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729464AbeI1RmQ (ORCPT ); Fri, 28 Sep 2018 13:42:16 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:41990 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmQ (ORCPT ); Fri, 28 Sep 2018 13:42:16 -0400 Received: by mail-qt1-f195.google.com with SMTP id z8-v6so6115544qto.9 for ; Fri, 28 Sep 2018 04:18:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=BgDRhtJ88t7ugG2XXq1rTUKV8IL4k8Uo9zOi0L9ZWBs=; b=GVKNiXf7Dyb6kVr8XMTZ46QsYZdt3MUXWXMYt7JLIARzq0vR3EdNgTR9Sh9/a90dFw X3JkIQyCcfhI4S588z4UTlWT3ffz4Sr4Lkio7Dzf8PT+cb8K/+GwRxASK1wYoFOafPeC 8D4Dnfu0oHFnXyOUFYo+iKUxKGRHty3w+2NvEik6BOGd4ngXLP31uBKey7fEvkZh6Psn dM/K3nAKqLcU5Bl8Jx7Ay2xZgRcJt2rdnx4l7UdLoL3Wg0ttxfWw0S0Yvt1nk8h7JDyA /XxVjt9AlPknHH77ricXLpmml/EwTBQPL42T7qhmhnapHfFvwsVaC2TdEgL04IGEPcj3 LNXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=BgDRhtJ88t7ugG2XXq1rTUKV8IL4k8Uo9zOi0L9ZWBs=; b=cGSyq+FNRyWIyiU1Pk93sryNvnkSL7J44i+56vEZ2vFIMML3LVt3DEnSBgCdDEZbqd dekSKvjyc2uVDFuuhAcbn7WXcjU88EhHrMILtELU3KfVNQ5TyX1UdwtBuVnzuVZefjOw K+kW45YbtLG5NmYPm2t7zVQfNXtMR59PpNWLXZrvHDgMI2Bt7ielg4zuo32aQaEKK7HN 08Ro0a2Vc0qQp6YOfrD5CqAhYNsfcOWUcC+0uDRitWp1UEilvKqBPqBiC9TVq8DUsZZG foLPGWd8jHeexNSwQstI5LzKWSK4iaM/6fSPoKBhyQOvFdOnhThg9I73ZRakdNXawlsk UsVA== X-Gm-Message-State: ABuFfoi0EPA1iyKQOQOuwowjEehUNTVYeVtPKQhM1iB6d06TEHPe9TFI SCvzr7GQlCLuJ4ZDzUPjByYhqFCPnXs= X-Google-Smtp-Source: ACcGV62sJzU/nEYkf6mMibqzRJA3N0pXGpgu/fPep9SHkuU7m5j5+UjS2ce67zjo4stgkXZ5DRRsWw== X-Received: by 2002:aed:20f1:: with SMTP id 104-v6mr12402380qtb.81.1538133538685; Fri, 28 Sep 2018 04:18:58 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y143-v6sm2650738qky.95.2018.09.28.04.18.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:57 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 18/42] btrfs: move the dio_sem higher up the callchain Date: Fri, 28 Sep 2018 07:17:57 -0400 Message-Id: <20180928111821.24376-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We're getting a lockdep splat because we take the dio_sem under the log_mutex. What we really need is to protect fsync() from logging an extent map for an extent we never waited on higher up, so just guard the whole thing with dio_sem. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 095f0bb86bb7..c07110edb9de 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out; inode_lock(inode); + + /* + * We take the dio_sem here because the tree log stuff can race with + * lockless dio writes and get an extent map logged for an extent we + * never waited on. We need it this high up for lockdep reasons. + */ + down_write(&BTRFS_I(inode)->dio_sem); + atomic_inc(&root->log_batch); /* @@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) */ ret = btrfs_wait_ordered_range(inode, start, len); if (ret) { + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) * checked called fsync. */ ret = filemap_check_wb_err(inode->i_mapping, file->f_wb_err); + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans = btrfs_start_transaction(root, 0); if (IS_ERR(trans)) { ret = PTR_ERR(trans); + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) * file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); /* diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 1650dc44a5e3..66b7e059b765 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, INIT_LIST_HEAD(&extents); - down_write(&inode->dio_sem); write_lock(&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start; @@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, } WARN_ON(!list_empty(&extents)); write_unlock(&tree->lock); - up_write(&inode->dio_sem); btrfs_release_path(path); if (!ret) From patchwork Fri Sep 28 11:17:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E7EE15A7 for ; Fri, 28 Sep 2018 11:19:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65D0F2B0C5 for ; Fri, 28 Sep 2018 11:19:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5A6662B0F2; Fri, 28 Sep 2018 11:19:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D65512B0C5 for ; Fri, 28 Sep 2018 11:19:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729469AbeI1RmS (ORCPT ); Fri, 28 Sep 2018 13:42:18 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:43566 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmS (ORCPT ); Fri, 28 Sep 2018 13:42:18 -0400 Received: by mail-qt1-f196.google.com with SMTP id q41-v6so1329473qtq.10 for ; Fri, 28 Sep 2018 04:19:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=unYdeV0h0D7vZF7/WYt36r/3Sg+lYY+VUgHQLhqzUMw=; b=XsW1EyeUClvQ8Ps1JysRieIV87VDLs0ErMS+B7yT4CeDndSP7K1nVKLn3/Kx6NjMUY Amx5EIRgVfuNxFszFMLiI4SIMeyTQVzK2k7KZ7BSXEWO1Qc1APUbXt82Na6EH2mSK6dE 6aIpVHd64hSZJ5fx5vkWnrWG+82eT1LXhT8USmnrJeOjAClPrRqTOpYgqfzyFYT9nbkF PyysXX4M5rV7tKe7OAGGNIJcYpDWQyPNxO4b8kXq611oHBZza81qUidarcQn2rpBadEc UQ8+qN+AvYCAvw2nN1sX0shTTj3Njkt/g+MbTEFaZ0l55U1+OSzh5/hMdnOeqr5P636c v4jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=unYdeV0h0D7vZF7/WYt36r/3Sg+lYY+VUgHQLhqzUMw=; b=nYzaifeeuoCRjzXAuN6ap6NOjCRySzR8b6PftT95OQ8n7svjT3FxGAWivyRNxTja2q nip5ccs8sNucIBzzIOHt5XBs9YrE8RtHf0mLn/DNxEluQPLoKOcKs6yuewIXnw4LAUac 2P09hYcTUe4ne3FSTIJPebTI1uJBKmDbi+ruyoEquF1TFlBc+2sbfPKo+cCXsm0M6cdp N8wEko/N1ikxy19Vov7SkqviLIgMLR03ODS+UVxVN3r/hqC+ufVZpVONyuNsPnULH5c2 Ml/Yjd72jLPzSPaXbU4Hp8UidwD8LshZVgCCRHsmD6CVh5OPXFtyBBm5EhC4oTdlcZaf GK2A== X-Gm-Message-State: ABuFfoj49YhLNK7o7DEOLmYNP49RYtYOd/Gdho6L2uRhCS2i/EUkvGIp yRqLsc/VP5pQ91VnrB2nKt/8+w== X-Google-Smtp-Source: ACcGV61bawu+SWB9HFk6xAA6D9xUrcpKtG3BusKM3WIeBc6la9louPV0fmn0unFm3JsZ2dWX0OEJKg== X-Received: by 2002:ac8:3ac3:: with SMTP id x61-v6mr2585262qte.85.1538133540738; Fri, 28 Sep 2018 04:19:00 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id k126-v6sm2587158qkb.91.2018.09.28.04.18.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:18:59 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 19/42] btrfs: set max_extent_size properly Date: Fri, 28 Sep 2018 07:17:58 -0400 Message-Id: <20180928111821.24376-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We can't use entry->bytes if our entry is a bitmap entry, we need to use entry->max_extent_size in that case. Fix up all the logic to make this consistent. Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index e077ad3b4549..2e96ee7da3ec 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1770,6 +1770,18 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl, return -1; } +static void set_max_extent_size(struct btrfs_free_space *entry, + u64 *max_extent_size) +{ + if (entry->bitmap) { + if (entry->max_extent_size > *max_extent_size) + *max_extent_size = entry->max_extent_size; + } else { + if (entry->bytes > *max_extent_size) + *max_extent_size = entry->bytes; + } +} + /* Cache the size of the max extent in bytes */ static struct btrfs_free_space * find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, @@ -1791,8 +1803,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, for (node = &entry->offset_index; node; node = rb_next(node)) { entry = rb_entry(node, struct btrfs_free_space, offset_index); if (entry->bytes < *bytes) { - if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + set_max_extent_size(entry, max_extent_size); continue; } @@ -1810,8 +1821,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, } if (entry->bytes < *bytes + align_off) { - if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + set_max_extent_size(entry, max_extent_size); continue; } @@ -1823,8 +1833,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, *offset = tmp; *bytes = size; return entry; - } else if (size > *max_extent_size) { - *max_extent_size = size; + } else { + set_max_extent_size(entry, max_extent_size); } continue; } @@ -2684,8 +2694,7 @@ static u64 btrfs_alloc_from_bitmap(struct btrfs_block_group_cache *block_group, err = search_bitmap(ctl, entry, &search_start, &search_bytes, true); if (err) { - if (search_bytes > *max_extent_size) - *max_extent_size = search_bytes; + set_max_extent_size(entry, max_extent_size); return 0; } @@ -2722,8 +2731,8 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, entry = rb_entry(node, struct btrfs_free_space, offset_index); while (1) { - if (entry->bytes < bytes && entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + if (entry->bytes < bytes) + set_max_extent_size(entry, max_extent_size); if (entry->bytes < bytes || (!entry->bitmap && entry->offset < min_start)) { From patchwork Fri Sep 28 11:17:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619705 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B6956CB for ; Fri, 28 Sep 2018 11:19:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71C442B061 for ; Fri, 28 Sep 2018 11:19:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 664BA2B0CF; Fri, 28 Sep 2018 11:19:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 115112B061 for ; Fri, 28 Sep 2018 11:19:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729474AbeI1RmU (ORCPT ); Fri, 28 Sep 2018 13:42:20 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:36830 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmU (ORCPT ); Fri, 28 Sep 2018 13:42:20 -0400 Received: by mail-qt1-f193.google.com with SMTP id e26-v6so6167660qtq.3 for ; Fri, 28 Sep 2018 04:19:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MzDmzB2k3QZIyenv0r3E5KuNr4U54ItlfnVj2WDyMrY=; b=fRkvgD9Tm5fiueF2TUbneyJxwl+/0Cowu0WADeq2v6arQSrd0LYfIBCINPBn3bvQq4 8x8MmB9sgRVQVb55yq+TXPMnNqJkiwJqiWcT3BCDyVYKUOgjcC7B7pJx2mo9HHtmB0x/ IVabXuodqC3t/H61F5I8K/2jA9SQ25LG1Hp4XS8CMp8K4+ms4BtA+u7Hsjlea9JXNLc0 ig0ocS52GtGRq/4qYJl4XaQ348Ne0A5vOQdAp5GHz37huJwkXarIUMxkg7VhumVML8k6 EIgj4j3fNU6jwnKNGl5cbhzeHImdFfoKNyOds2vmRdHesgTNm67ZL+ID7HJSFotD6zri 5TGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MzDmzB2k3QZIyenv0r3E5KuNr4U54ItlfnVj2WDyMrY=; b=aHhupkjye97o5uxBcz+5dkph3EabsdGaY+Q95T9IqXhK++ffsv+qyF6XTAJerL4fHx We8pjWCiCuLEO7JOrfDtoKZcIiSc1L0Q422U7OPcbd+CjAEkv781EOYP1mT6k04Ky1wu senwZz20KI6O5uaDCZZ8MbL2/hNY0LMjF0PPreq0utpJwrfdnZ30EyVXwjgdwyd/HK9O LRMzZL1yNaafC64jFwmDLveIXDsOSqCmXxRQvNoCfVqvRMrl48rli15W9RAfN+gmBq25 X8W39mJ4pe2Ise7u+31k54qPvLZXRR9beZyvZNOg3syLYjrPtZm1sU4rSuH48LALbmMw ITtw== X-Gm-Message-State: ABuFfog8QJo/jBikQqz/yQQ4UnL0GLyEwq3zcAemwbkECAOnVoKVRpUC y6Y1FWT/k7WkR/X/Y9W0FWhKgw== X-Google-Smtp-Source: ACcGV63NO/uep9xRQOix8R+JJuYaabg2d2U9ouTkivv0q4Y/luZsHE1hFGEydFHU6mUApeing+Pfvw== X-Received: by 2002:aed:2c64:: with SMTP id f91-v6mr11819172qtd.310.1538133542543; Fri, 28 Sep 2018 04:19:02 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s73-v6sm3227842qkl.65.2018.09.28.04.19.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:01 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 20/42] btrfs: don't use ctl->free_space for max_extent_size Date: Fri, 28 Sep 2018 07:17:59 -0400 Message-Id: <20180928111821.24376-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik max_extent_size is supposed to be the largest contiguous range for the space info, and ctl->free_space is the total free space in the block group. We need to keep track of these separately and _only_ use the max_free_space if we don't have a max_extent_size, as that means our original request was too large to search any of the block groups for and therefore wouldn't have a max_extent_size set. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index eb64b28196b8..40503438ef6c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -7492,6 +7492,7 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, struct btrfs_block_group_cache *block_group = NULL; u64 search_start = 0; u64 max_extent_size = 0; + u64 max_free_space = 0; u64 empty_cluster = 0; struct btrfs_space_info *space_info; int loop = 0; @@ -7787,8 +7788,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, spin_lock(&ctl->tree_lock); if (ctl->free_space < num_bytes + empty_cluster + empty_size) { - if (ctl->free_space > max_extent_size) - max_extent_size = ctl->free_space; + max_free_space = max(max_free_space, + ctl->free_space); spin_unlock(&ctl->tree_lock); goto loop; } @@ -7955,6 +7956,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, } out: if (ret == -ENOSPC) { + if (!max_extent_size) + max_extent_size = max_free_space; spin_lock(&space_info->lock); space_info->max_extent_size = max_extent_size; spin_unlock(&space_info->lock); From patchwork Fri Sep 28 11:18:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619707 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFBA515A7 for ; Fri, 28 Sep 2018 11:19:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5C562B061 for ; Fri, 28 Sep 2018 11:19:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BA5432B0CF; Fri, 28 Sep 2018 11:19:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B6562B061 for ; Fri, 28 Sep 2018 11:19:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729478AbeI1RmW (ORCPT ); Fri, 28 Sep 2018 13:42:22 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:38147 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmW (ORCPT ); Fri, 28 Sep 2018 13:42:22 -0400 Received: by mail-qt1-f195.google.com with SMTP id z13-v6so6144075qts.5 for ; Fri, 28 Sep 2018 04:19:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z3nhKV1wBqQcfXnDoMHMl2GEjOrb6h1HQ06C9nkbnUE=; b=cdBc5Q6KuuD06XXPh1mFzp5TW+/7H4ANit7XsOnhww4nFYTdj+Ei5lZdLzH0WjxRIk zFVyx2lwsl8ES7NVUje/y51OeDxW5XsNNwpvkW5l4A2sW4xZkQuNH8sYxL+Lo1aeqoqd Nq6JAVv2WqD6L+ciesSDtCFrbvxrb+K5mU8eaoP8ALeFe7IKuXAJl8zrLL61vi8gu2cI V3dtJnO8kXeYH8eEb6JPH6CJm7tHhRUNDy64Se86iq7ibK85dhctjYUWi5/nHMNNCsYV EBOwPKd6zw2PHqSsyd6lGlfi2lSCL5ny0uR3xXSsqP95EvEkpLemsyF6kapmNs7I0Rw+ rffg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z3nhKV1wBqQcfXnDoMHMl2GEjOrb6h1HQ06C9nkbnUE=; b=RLPc6S59EGFuOP0t2LjTOM0VELcIJHQoGcLU3mweTdIgNrh5INNm7s4768sBFphAIm loNdW2GZcrNwSLtxHr/vNhYeu6sAs52eZC9PYAtdDtp52nGvcqEG92w7foRT7G26IY/E 0xIFxGRCeYZSk9F0ySnDXBIPmrbJTnasAWidpahdgjjDjoMK+w3zXM1PeAX7wIoTekV5 +my0ZtgtZJ/M5NzyJdDhAjPJ3TRXmE2NwzN8Uz7QMrmiVF4mmhDQJPQg+p7gkX81v15J bGYkL5LM/3xUimEGhca8bTjfO8ZVDkMB2hmxAs0/AaOhvFlZrbULUZAVw7Ocypyrooif D6tw== X-Gm-Message-State: ABuFfoj75hqE4yu9cRbPZkdxJj457NGxEHsMVmgEcGaFEQ6TDnq934sh mdamYhQL4qyf6BgWTNAGdKkv/Q== X-Google-Smtp-Source: ACcGV60AGalAUmVNdqHGKkBexmdl+jSkQFtfKLVDVFggfWZXhFIw2g3Wt8TNzWD7ug2UmJPOienNyg== X-Received: by 2002:ac8:6b13:: with SMTP id w19-v6mr12239761qts.317.1538133544259; Fri, 28 Sep 2018 04:19:04 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f10-v6sm2388374qtq.52.2018.09.28.04.19.03 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:03 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 21/42] btrfs: reset max_extent_size on clear in a bitmap Date: Fri, 28 Sep 2018 07:18:00 -0400 Message-Id: <20180928111821.24376-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We need to clear the max_extent_size when we clear bits from a bitmap since it could have been from the range that contains the max_extent_size. Reviewed-by: Liu Bo Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 2e96ee7da3ec..d2a863a2ee24 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1687,6 +1687,8 @@ static inline void __bitmap_clear_bits(struct btrfs_free_space_ctl *ctl, bitmap_clear(info->bitmap, start, count); info->bytes -= bytes; + if (info->max_extent_size > ctl->unit) + info->max_extent_size = 0; } static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl, From patchwork Fri Sep 28 11:18:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619709 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CC7F15A7 for ; Fri, 28 Sep 2018 11:19:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32C5D2B061 for ; Fri, 28 Sep 2018 11:19:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2798C2B0CF; Fri, 28 Sep 2018 11:19:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C07F32B061 for ; Fri, 28 Sep 2018 11:19:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729483AbeI1RmY (ORCPT ); Fri, 28 Sep 2018 13:42:24 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:39847 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmY (ORCPT ); Fri, 28 Sep 2018 13:42:24 -0400 Received: by mail-qk1-f194.google.com with SMTP id q5-v6so595207qki.6 for ; Fri, 28 Sep 2018 04:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=vfQB1tHlwUhp446i68IaYEtvkGVFvtWut8g/b3eBAlk=; b=oMQjhGq73M/j/vpwJYUcd1s9HO4OGoFYrNBUhr1cBwwMTow0G5dAaJSY4JUv2YoDQ5 TbFcxWda9/ncAeWNHbheXvIudezGPga5yzc7uRnnQsyUvBQgAN0yCplWIWP3fkVRoNNG Tusl+fERqHorz1x7/N0hFCqiAfDLjBU58zTeSrith2rZE/WLhWqSEuPHQy+y7Dq5usK2 xx03ahS9kaD6OYsH53lNd9v8ESjr4xGPn3M9vxgR0MM+d2xELxvzkxLSp2z3S7mzeRkq 97m/qbwCwGYOGu1WzjE2Ruc/t4BmmA8CiqhuprgAby8qSmHZLcjSWwTPaU4YSanqnUo9 I8zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=vfQB1tHlwUhp446i68IaYEtvkGVFvtWut8g/b3eBAlk=; b=e5RJoIG/Dc9iOvZdqWD8gEFd1c2MhyI/vlnf3HmPa7vJBMkmBwcS1CPx0fZs17XPzW M8ZN9FogK0Kwl/xH/kTxknzdPnoqJFvrgf9pkOr3WqcsNE488qRCxD3CLMdJk51NHrtb 0qreZrVakMP+KZIQB+LJ9/BTZ6k7neXiX6XNmDGvUked5pJ3QF53tdLINVFnh5rjNm8b YUS0FKXHiJZVsj1vbd8Hk6oesGVLt0bEnh0+AWcDiPudtaalQmZsPhz29pe0jEyu3eNo N9ic4l9PNu/HVy1x8XXuxHGMpTTd5rlgtQoJEA2MNPuAjoNC9TU6Zm+L+nSrQrBlD6UF Alkg== X-Gm-Message-State: ABuFfojkWVK0uRRBOz1BF6hDQsVYqzsRrX7pMJIGH+nQ2UptNqryWiy7 5XfW+8QiBhdAPCM2SBF1obWZAw== X-Google-Smtp-Source: ACcGV60bFU18KpOEOnxw1D9+wYnwqVX/c8DuBEEpN+XPpuOVwovCN19L8mLYsRDCt9CNDfPVMBMShQ== X-Received: by 2002:a37:28c7:: with SMTP id o68-v6mr11503452qko.350.1538133546109; Fri, 28 Sep 2018 04:19:06 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o7-v6sm2211760qtp.75.2018.09.28.04.19.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:05 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 22/42] btrfs: only run delayed refs if we're committing Date: Fri, 28 Sep 2018 07:18:01 -0400 Message-Id: <20180928111821.24376-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I noticed in a giant dbench run that we spent a lot of time on lock contention while running transaction commit. This is because dbench results in a lot of fsync()'s that do a btrfs_transaction_commit(), and they all run the delayed refs first thing, so they all contend with each other. This leads to seconds of 0 throughput. Change this to only run the delayed refs if we're the ones committing the transaction. This makes the latency go away and we get no more lock contention. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index a0f19ca0bd6c..39a2bddb0b29 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1925,15 +1925,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - /* make a pass through all the delayed refs we have so far - * any runnings procs may add more while we are here - */ - ret = btrfs_run_delayed_refs(trans, 0); - if (ret) { - btrfs_end_transaction(trans); - return ret; - } - cur_trans = trans->transaction; /* @@ -1946,12 +1937,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) if (!list_empty(&trans->new_bgs)) btrfs_create_pending_block_groups(trans); - ret = btrfs_run_delayed_refs(trans, 0); - if (ret) { - btrfs_end_transaction(trans); - return ret; - } - if (!test_bit(BTRFS_TRANS_DIRTY_BG_RUN, &cur_trans->flags)) { int run_it = 0; @@ -2022,6 +2007,15 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) spin_unlock(&fs_info->trans_lock); } + /* + * We are now the only one in the commit area, we can run delayed refs + * without hitting a bunch of lock contention from a lot of people + * trying to commit the transaction at once. + */ + ret = btrfs_run_delayed_refs(trans, 0); + if (ret) + goto cleanup_transaction; + extwriter_counter_dec(cur_trans, trans->type); ret = btrfs_start_delalloc_flush(fs_info); From patchwork Fri Sep 28 11:18:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619711 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E9446CB for ; Fri, 28 Sep 2018 11:19:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64D512B061 for ; Fri, 28 Sep 2018 11:19:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 596132B0CF; Fri, 28 Sep 2018 11:19:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 052AD2B061 for ; Fri, 28 Sep 2018 11:19:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729488AbeI1RmZ (ORCPT ); Fri, 28 Sep 2018 13:42:25 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:39344 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1RmZ (ORCPT ); Fri, 28 Sep 2018 13:42:25 -0400 Received: by mail-qt1-f196.google.com with SMTP id c25-v6so6145483qtp.6 for ; Fri, 28 Sep 2018 04:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=N+dz60mOwxy8LSi9HFgdT6MWdMfa3FlIMDFMCSuxx48=; b=dz16+1LRQ95OLS0zid0fsAbDlQMCGJF0df2In3q2MRQrzMvVNgfv/aIiklVBotuGd/ 9FcZ8C1RcTHGe7uSnrtICKiEysx0DnODrzJn3ZqMZAqp3uEDTgrnPPDFQ4o58lA4MEY3 r7hmvR+yH0nZYAvF6jbxiexJl1BgeMDIm7Y3hcC08cxEQ/JD21YNO5b02uqvYTQDY2j1 2CZ03c3DIacrSXy++rUagl3Hm9Y49Ir0NV7vFGDeA7jEY9i2t0qpqvVjD97apGBq1YyC 1jTNX1hKqQicg8wxZ6fOkgstnXQvbB8cfKNoR1E83Rk9uPfHypW6cld0YjsgEfPXObzQ zSAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=N+dz60mOwxy8LSi9HFgdT6MWdMfa3FlIMDFMCSuxx48=; b=Munzt0Wk4liraDO+HyvTk58mmRhxAUkNcdBFAARAsdCmg5ynYs76HaR7OxwIU24PUB nCcVsk7+n/wnRKpbaKSP1gpe5hsumnzdUchsnWlDFVGNDj5J57H++4OaznfV4xfrgq3K 19/GF9iAqeWIXOoHZB+HjdipiInQf/5cBwt4XPyRpSZUqYzV9hATmSJjMxBQdw8dIbN3 v8By6+zI2bLR5WFd4MZLj/58idO8Vs2jC8Gi/O4ztHs+L/On+DHdFGTuf71GPlnx7ISF mLUDowIueJVNG5/3+tXyXGHFSURTnqfu3BqFVtIjtGz7NwUKjV0xlH60Tv/JdGh2Z7uI anQA== X-Gm-Message-State: ABuFfoipre5UMOf7oEmNRa2BpDcbYEocQccBUGiax9uPDaSxTg94YzoA SmmU6DU8gvSbyrrEh4oEDtmDcQ== X-Google-Smtp-Source: ACcGV60cfb/HzJMDQzIup8J0AYvBYsux+r1rZvs5tI3ir9kV6ojZvSKSTygRS5QN7JdJZjA2Q5ItIQ== X-Received: by 2002:a0c:b9a9:: with SMTP id v41-v6mr11398673qvf.63.1538133547845; Fri, 28 Sep 2018 04:19:07 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id t28-v6sm3185174qki.41.2018.09.28.04.19.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:06 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 23/42] btrfs: make sure we create all new bgs Date: Fri, 28 Sep 2018 07:18:02 -0400 Message-Id: <20180928111821.24376-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allocating new chunks modifies both the extent and chunk tree, which can trigger new chunk allocations. So instead of doing list_for_each_safe, just do while (!list_empty()) so we make sure we don't exit with other pending bg's still on our list. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 40503438ef6c..01bfb02101c1 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -10357,7 +10357,7 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info) void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) { struct btrfs_fs_info *fs_info = trans->fs_info; - struct btrfs_block_group_cache *block_group, *tmp; + struct btrfs_block_group_cache *block_group; struct btrfs_root *extent_root = fs_info->extent_root; struct btrfs_block_group_item item; struct btrfs_key key; @@ -10365,7 +10365,10 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) bool can_flush_pending_bgs = trans->can_flush_pending_bgs; trans->can_flush_pending_bgs = false; - list_for_each_entry_safe(block_group, tmp, &trans->new_bgs, bg_list) { + while (!list_empty(&trans->new_bgs)) { + block_group = list_first_entry(&trans->new_bgs, + struct btrfs_block_group_cache, + bg_list); if (ret) goto next; From patchwork Fri Sep 28 11:18:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619713 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2D6CE6CB for ; Fri, 28 Sep 2018 11:19:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 246472B061 for ; Fri, 28 Sep 2018 11:19:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 18F4C2B0CF; Fri, 28 Sep 2018 11:19:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE0792B061 for ; Fri, 28 Sep 2018 11:19:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729493AbeI1Rm1 (ORCPT ); Fri, 28 Sep 2018 13:42:27 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:44513 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1Rm1 (ORCPT ); Fri, 28 Sep 2018 13:42:27 -0400 Received: by mail-qk1-f193.google.com with SMTP id g184-v6so3538159qkc.11 for ; Fri, 28 Sep 2018 04:19:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=JFxGVw1HrusCY3GFgaJkHeYCW8OyW70/VljEmT/1Dxg=; b=sPHp7jUkzCEtCaZ7TZa7nJAet+wGxKfK1z05Y5h1YbKKNkBMZ3/GK8hysREi9l+K0D HRx7R2PlFziw+uQconyApQK5bELe7bh2xiIMnZSs+zt1K4Q8K9bADbgcSp/DxKQilPwZ kJKmb9t8u3LMxvKkkuIsPQr1zR9PgACPki2HfLGrBE/6t0lwgDYAONqPY5jYwK26f/Vc 1QbFS24IM8gxOqDDXPMJP3TwBUypBPzA9EHRO5lVcmYblwczgFrC8yoqG6Zns7wUhGvj ND4ajw2GkanK3QxrSHYK/heYrR8D0zMRohZYzn/KzqgKSyGXhFzRls1nXL+P3+WtNFgm Am0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=JFxGVw1HrusCY3GFgaJkHeYCW8OyW70/VljEmT/1Dxg=; b=gDSbPTro20rCT+SPkx15OuknftfLPYpCOaqh/CwhQ+6cyN45T2jqfUFJlowoJ1Cdl6 dqlwEdfnGFIRDGiG6h5SDHH5S840z+s45Vr6Jq3nxBfWxYIoGcCJSpwXG5PpRqEHV1cD ptFh+79wM70KYzNKKIMkzwJsSftSSKXluqe1u1JlOgnHXe5JLmqnuDcCQQMD5nWXSBUa TcW3Da9T3LsGkgCN4KbzUce4CZrZMaqB7AUwtMLRxWRawBW7Mc1sDNC2LMCJVUSNrHdk lIFVvuaRlSX433PJqVqs5vi8YQgi4h7EHjMPvi32e2yboQsPuLN+5YOfRVG1IZdNzlJU nclw== X-Gm-Message-State: ABuFfoh/LIuH4svZ94ypCxpIEOdvkoaP4y0A23qBz0/OxGIEn1fL9lAM h8TkipIpEv1RghWKZrWmsUtgCDqc58w= X-Google-Smtp-Source: ACcGV61fpQPr+ZCr17Vf2ioB5cRs5iudMVRVSC7oDvWvJ0MfP8WK/0nR7X76X+I7yIsCnCOS5/vCnQ== X-Received: by 2002:a37:f53:: with SMTP id z80-v6mr10666503qkg.278.1538133549566; Fri, 28 Sep 2018 04:19:09 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v12-v6sm2700178qtg.82.2018.09.28.04.19.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:08 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 24/42] btrfs: assert on non-empty delayed iputs Date: Fri, 28 Sep 2018 07:18:03 -0400 Message-Id: <20180928111821.24376-25-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I ran into an issue where there was some reference being held on an inode that I couldn't track. This assert wasn't triggered, but it at least rules out we're doing something stupid. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/disk-io.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 377ad9c1cb17..39bd158466cd 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3979,6 +3979,7 @@ void close_ctree(struct btrfs_fs_info *fs_info) kthread_stop(fs_info->transaction_kthread); kthread_stop(fs_info->cleaner_kthread); + ASSERT(list_empty(&fs_info->delayed_iputs)); set_bit(BTRFS_FS_CLOSING_DONE, &fs_info->flags); btrfs_free_qgroup_config(fs_info); From patchwork Fri Sep 28 11:18:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619715 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA4EB15A7 for ; Fri, 28 Sep 2018 11:19:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D1E2B2B061 for ; Fri, 28 Sep 2018 11:19:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C68772B0CF; Fri, 28 Sep 2018 11:19:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 610842B061 for ; Fri, 28 Sep 2018 11:19:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729498AbeI1Rm3 (ORCPT ); Fri, 28 Sep 2018 13:42:29 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:37286 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1Rm3 (ORCPT ); Fri, 28 Sep 2018 13:42:29 -0400 Received: by mail-qk1-f196.google.com with SMTP id c13-v6so3570651qkm.4 for ; Fri, 28 Sep 2018 04:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=rvsXCa4Gbgrz3G3GJWzyW6ZCwXPWHXoo2lvIWdgxiik=; b=sKYZKAYWXip5EtXzV1gCTggr1INyI+c0vwJRJ2ApWUdJLY/M4rx3Ukpd05P7UKmalh m4Rgb+MP8jcjz25A1A23foU/0x8xO5qFbPRpbl8acmmOg28FVDz0Kfj/SrusMU6wQSc3 h6iSZ82fexUN1MfZ3rAVNyX06VrvxCx3J9iYUXQ7TIf6vS9uUsqZm2NySNOxsJMYCmeL 9lKlkRA9DRGrZUOf5ZWhtMVYRjR/5w7ateQndbq+Xgx9mYNFec+3rKaJ+Io2WcQABSe6 uJxMvqOxl9qCP93lTOQFfKJ2XbYe6KtjV19u85/eWbUVn2Nubv28+SLuLAbjmn5bDmsW AseQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=rvsXCa4Gbgrz3G3GJWzyW6ZCwXPWHXoo2lvIWdgxiik=; b=NMMvAw3ly8ocbwnMzGH3aD9V4zapZiQitxugBB156qL2+MoOrvsG+FHxtIEH7PhwDL vwEeNZeO1cX1fN1Vwj0Fcnyj7/bZ/y543Oke+Dif8a91nXF2iHljjNMDAYP4yqob6yVD u8GSmYwnzm0nQCJ9kESBXday0tGTyQFnFJLdtvweP22bKO7EFGcW9RGAizkA6h8qw42i LayJOIUHrDQ5ncDCHRaSnbo3KH22Xf0QMVhZn9l4dw6V1Rm59SbSIPENv4qr4y1qhROE 4l8mfhDh1i5ce/rFKCuCkHCiDDneKvTH75IHMuUB5ZAKGwlzaYmNa4CkQpZImfsZKN0j FW4A== X-Gm-Message-State: ABuFfoiTyBzP780W03wcSbe62JcU6vgV4gMFWP0WMx8sMEQUHp/hFLWR 79AxN9zBOpmho0VcVbKCL8K0Wg== X-Google-Smtp-Source: ACcGV61NEgXmkeJMGERdWlqtvQSEwITzbQRt+DDY5jOI5ZLDYFpAHW7vdt0dJDih6e1fZZcRcVQgUg== X-Received: by 2002:a37:8786:: with SMTP id j128-v6mr11414992qkd.32.1538133551385; Fri, 28 Sep 2018 04:19:11 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f23-v6sm2596552qtp.5.2018.09.28.04.19.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:10 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 25/42] btrfs: pass delayed_refs_root to btrfs_delayed_ref_lock Date: Fri, 28 Sep 2018 07:18:04 -0400 Message-Id: <20180928111821.24376-26-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't need the trans except to get the delayed_refs_root, so just pass the delayed_refs_root into btrfs_delayed_ref_lock and call it a day. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 5 +---- fs/btrfs/delayed-ref.h | 2 +- fs/btrfs/extent-tree.c | 2 +- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 96ce087747b2..87778645bf4a 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -197,12 +197,9 @@ find_ref_head(struct rb_root *root, u64 bytenr, return NULL; } -int btrfs_delayed_ref_lock(struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock(struct btrfs_delayed_ref_root *delayed_refs, struct btrfs_delayed_ref_head *head) { - struct btrfs_delayed_ref_root *delayed_refs; - - delayed_refs = &trans->transaction->delayed_refs; lockdep_assert_held(&delayed_refs->lock); if (mutex_trylock(&head->mutex)) return 0; diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index 7769177b489e..ee636d7a710a 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -255,7 +255,7 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head * btrfs_find_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_refs, u64 bytenr); -int btrfs_delayed_ref_lock(struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock(struct btrfs_delayed_ref_root *delayed_refs, struct btrfs_delayed_ref_head *head); static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) { diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 01bfb02101c1..34105bc5eef7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2600,7 +2600,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, /* grab the lock that says we are going to process * all the refs for this head */ - ret = btrfs_delayed_ref_lock(trans, locked_ref); + ret = btrfs_delayed_ref_lock(delayed_refs, locked_ref); spin_unlock(&delayed_refs->lock); /* * we may have dropped the spin lock to get the head From patchwork Fri Sep 28 11:18:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619717 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 132B115A7 for ; Fri, 28 Sep 2018 11:19:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BB832B061 for ; Fri, 28 Sep 2018 11:19:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 004A12B0CF; Fri, 28 Sep 2018 11:19:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FBF72B061 for ; Fri, 28 Sep 2018 11:19:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729503AbeI1Rmb (ORCPT ); Fri, 28 Sep 2018 13:42:31 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:39354 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1Rmb (ORCPT ); Fri, 28 Sep 2018 13:42:31 -0400 Received: by mail-qt1-f195.google.com with SMTP id c25-v6so6145707qtp.6 for ; Fri, 28 Sep 2018 04:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=X+PT81/Zp3zcYtUcLLi8M0fbP5w87s6IcVZs1kVoVUA=; b=NV4v21Y2b3kHq4kkKPvlC/HvRP7dcCk5/lM4olHWJqicPfAvk4YlW5W+vwEExSSn+U 9Klw3yT2SpKHOPvpq7eOsZUbrcVFTg78Q67bskkB25pijXy745SrlYGnQgyz1dVds6fc q2ICyVysqVjA5GGnwH8+ZFEaDbh3mbSG7HdWOzHoaQhIOqgHNwnbokDSy1v/njASGI6e EkDnHMZiRqHym76IdGbzBd+qfF9lMEzK2LgbmpbmyEZuZPULvyjkPAL+ketPWEy5wS9x rZM33NI9CqQhc3WSRY/pZQvofpUHefymIrpvTcgq/mj0JNnEieIgr58LGTMBqrOjJDLO Pv1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=X+PT81/Zp3zcYtUcLLi8M0fbP5w87s6IcVZs1kVoVUA=; b=SCgKSMzaKpTc50OhjdN5ugYp14GQ01+TqDo2Owstb5QtI6ipoWG6JmxL5NUaEKLSEm 838eOHQZEQGgejlqGQSuyUbHOggVBGHS/mlTdvnIVz7iEQ1k2AIvb3IGnAqhRvUy6Jq8 dFNicOChIfIH2exNuz15Vlf22kEqqJgrqvRyRqO2/Vnyf8cMIdvlFg5Un8bDlwdjk7Du qkrNeya3olqh4d6XwbsPt+xfqIpYySTkxvQ+ZYgpW+RUftFi8fWo/wWI9Mx7GK1K4189 Y+mtuRiqLKWdbrLqOoBHM5raINpLsBRRQiOtVEEoaEtkYrnZVwDpQSH4yRE7dDra8X97 bKUA== X-Gm-Message-State: ABuFfojIaKmHGd6Hw878xfsazBFFkr6AecIl2fjYN/riGLqTtceppIaX lwYZXPKbVmoiR1bI8AYP8o1UeKH8bZM= X-Google-Smtp-Source: ACcGV619t5wDGk18XGmTPD80PIF5VY5Br8iF9/rHCYFPTwct07QOPYzTrYr+c5w8+PyUsnIjpIhmCA== X-Received: by 2002:ac8:1793:: with SMTP id o19-v6mr11941511qtj.98.1538133553176; Fri, 28 Sep 2018 04:19:13 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e29-v6sm3092790qte.47.2018.09.28.04.19.12 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:12 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 26/42] btrfs: make btrfs_destroy_delayed_refs use btrfs_delayed_ref_lock Date: Fri, 28 Sep 2018 07:18:05 -0400 Message-Id: <20180928111821.24376-27-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We have this open coded in btrfs_destroy_delayed_refs, use the helper instead. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 39bd158466cd..121ab180a78a 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4214,16 +4214,9 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); - if (!mutex_trylock(&head->mutex)) { - refcount_inc(&head->refs); - spin_unlock(&delayed_refs->lock); - - mutex_lock(&head->mutex); - mutex_unlock(&head->mutex); - btrfs_put_delayed_ref_head(head); - spin_lock(&delayed_refs->lock); + if (btrfs_delayed_ref_lock(delayed_refs, head)) continue; - } + spin_lock(&head->lock); while ((n = rb_first(&head->ref_tree)) != NULL) { ref = rb_entry(n, struct btrfs_delayed_ref_node, From patchwork Fri Sep 28 11:18:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619719 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92D0D6CB for ; Fri, 28 Sep 2018 11:19:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89F152B061 for ; Fri, 28 Sep 2018 11:19:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E03F2B0CF; Fri, 28 Sep 2018 11:19:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3118B2B061 for ; Fri, 28 Sep 2018 11:19:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729507AbeI1Rmd (ORCPT ); Fri, 28 Sep 2018 13:42:33 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39356 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1Rmd (ORCPT ); Fri, 28 Sep 2018 13:42:33 -0400 Received: by mail-qt1-f193.google.com with SMTP id c25-v6so6145762qtp.6 for ; Fri, 28 Sep 2018 04:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=cvSqJmOxERWLztXjenJ/CRRumlDa3ZymappVew1ga8M=; b=ec/MB5LieKBlAkYByyUs5658fGHiSbkKfgPaAHg8WFpxWJaRiD0yL9lkBYvSQboL3D Ga0YFNkRoUfrSIUPpHJY0xHi3hROGeNIhmmLu7pLYwr0+kJPp3Mii4XM+CT5e9ChzVYH UNjlmy9WfOsfoTa1+0Qobx37D0Y8KjGj/Ueb2ORuvrGUGIRPbwislzRtUgJRa7tokLiL Qc0EaMTWC4gCLXMfQKpMd8pmbtNqYlqnNskOm/es96JLjHKHRHwqch4OFwTvUlSqSq1M 48Tx+M9TD9dyA2GqhVXYxy4O2DnhTv+S8n0FJ74lKdEcMRco7fx6jdZiFfQmN2zQSUfo pa/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=cvSqJmOxERWLztXjenJ/CRRumlDa3ZymappVew1ga8M=; b=g7shYies4juhESjhwwfpH6gePLGjARQyLhiL3fvYLOzhIr+I5OVSU63YT74Wu+hKPW wnltsvAIozijmNo1nfBYM/UaTfPaW7fd/j+4IbvNn44tGr1I7vhxP7nsgSgHXpCenJpz KKRN9AZzZUh5tR/yMZg2Bd2iUbeMQU4hnkA/aQ7CiPx3p2geR7hjWmtz0clL4GqVrH9+ dAPeU7Y7LC+H4dga6Z8cOxF6xyHXFLmJISQokb62BdIhTJoVp3Y3HeRffHwZv4TRUY2m 8xjX9kxNn4rSrcLgC67bzxZ0/EpJx/HFEk1O1mxQt0HodsEAs0bC3lQ7JLhNgxEuGdDG Ob4g== X-Gm-Message-State: ABuFfogRMz+3hraUS4nAIU5+4Die+9tLWApb5vlHnXj5kyfvm5w0UpRb +F0uPpCF3zW+w3zkmIK683uxQA== X-Google-Smtp-Source: ACcGV62JjHZrhVYccC1dlnHJZoORWOcC9mo0MLxwClYOW5qlikf1Y7odyzkYH/8uwcZ42lzvL9Dinw== X-Received: by 2002:a05:6214:1091:: with SMTP id o17mr4520661qvr.193.1538133555085; Fri, 28 Sep 2018 04:19:15 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 14-v6sm47319qke.3.2018.09.28.04.19.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:14 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 27/42] btrfs: make btrfs_destroy_delayed_refs use btrfs_delete_ref_head Date: Fri, 28 Sep 2018 07:18:06 -0400 Message-Id: <20180928111821.24376-28-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of open coding this stuff use the helper instead. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 121ab180a78a..fe1f229320ef 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4232,12 +4232,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, if (head->must_insert_reserved) pin_bytes = true; btrfs_free_delayed_extent_op(head->extent_op); - delayed_refs->num_heads--; - if (head->processing == 0) - delayed_refs->num_heads_ready--; - atomic_dec(&delayed_refs->num_entries); - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); + btrfs_delete_ref_head(delayed_refs, head); spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); mutex_unlock(&head->mutex); From patchwork Fri Sep 28 11:18:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619721 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BADBA15A7 for ; Fri, 28 Sep 2018 11:19:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B17C12B061 for ; Fri, 28 Sep 2018 11:19:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5DF12B0CF; Fri, 28 Sep 2018 11:19:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41FB42B061 for ; Fri, 28 Sep 2018 11:19:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729512AbeI1Rmf (ORCPT ); Fri, 28 Sep 2018 13:42:35 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:38103 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeI1Rmf (ORCPT ); Fri, 28 Sep 2018 13:42:35 -0400 Received: by mail-qk1-f193.google.com with SMTP id 204-v6so2899771qkd.5 for ; Fri, 28 Sep 2018 04:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=WfeTrzL+iK+qU6MThxl04x4RCBEh/Z5V2g9l9HBhkpE=; b=Gsofs28lRId9Cgh/lz6SrMfmEBz01DeEUwo+wc2dWJwMtJ0zP90Ocg8nk3VtL7CzNq ozgROBT93s/RGDSulsuvwyKoLTHEGX0GUdzDMc5ZYXVJHedsMOr4Y044DkMO8qVVPSO8 Zy0jEqjNUXBwI1YVzw7qzoVsiKn9JgRXwH2Sjf+7OQJx65ZaNJvUES2gV33etKGI8Feh AHWnnF55DJmp3dvhXAvAmLBi24ITufsIueoXqKQdFvr+bo1+B4BDpoNN6/W9bjxrVf9O FihU2WH8WoYzwV+7OD9RHQFCW0qj1xdQ0zsu+NAzz3Z+RJO8N98EF2F/aOZP3SZ0Beth 62TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=WfeTrzL+iK+qU6MThxl04x4RCBEh/Z5V2g9l9HBhkpE=; b=TJNbdipvKCryiQjUh80sBLS/SkTusnEEHy/383z4VY7xbfE/JED3s3e+sIrS1M0FG0 QsokIBUQ2I1azup1cay830kTETc8AF93akyqoNdEcuxXy9zdRCGWxkMNDijDu2RjCpfI shU05Xpbcf1PTyJqJXIuAFsn5bLKbMZsqlwc8K2YZNAsk/mTSloWWU4Soh/qoR9WIimH XxVazTVfK9AgbTKFi7j9wH2jqo3VhbxszeGNX4/sLrl/fUicu8r+oQ6B+QwoZVq0RWXA UDJdHpsG4bDDOi2B9EnxFfaQj+k3qA6g+cOikihwXJeDEo6rzZdT4uKaAgjm7ijOyqCz 1TgA== X-Gm-Message-State: ABuFfoigAn4N2kLzLTRcHlysQZrkTUKsWa9sr8rr2bb7oIXzWxId2jTu ZT+FFTI3hopppEtAP8hIPXYPWw== X-Google-Smtp-Source: ACcGV61hbjYDNy4m8G5wUzkc44lgtqpElA4fYHLEiRZ9BbrQxHRSguGINbgvNUnr3zd1bqbXC/pyAQ== X-Received: by 2002:a37:a107:: with SMTP id k7-v6mr11325367qke.276.1538133556988; Fri, 28 Sep 2018 04:19:16 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c15-v6sm2509317qkm.42.2018.09.28.04.19.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:16 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 28/42] btrfs: handle delayed ref head accounting cleanup in abort Date: Fri, 28 Sep 2018 07:18:07 -0400 Message-Id: <20180928111821.24376-29-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We weren't doing any of the accounting cleanup when we aborted transactions. Fix this by making cleanup_ref_head_accounting global and calling it from the abort code, this fixes the issue where our accounting was all wrong after the fs aborts. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 5 +++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/extent-tree.c | 13 ++++++------- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 29db902511c1..e40356ca0295 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -35,6 +35,7 @@ struct btrfs_trans_handle; struct btrfs_transaction; struct btrfs_pending_snapshot; +struct btrfs_delayed_ref_root; extern struct kmem_cache *btrfs_trans_handle_cachep; extern struct kmem_cache *btrfs_bit_radix_cachep; extern struct kmem_cache *btrfs_path_cachep; @@ -2623,6 +2624,10 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long count); int btrfs_async_run_delayed_refs(struct btrfs_fs_info *fs_info, unsigned long count, u64 transid, int wait); +void +btrfs_cleanup_ref_head_accounting(struct btrfs_fs_info *fs_info, + struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head); int btrfs_lookup_data_extent(struct btrfs_fs_info *fs_info, u64 start, u64 len); int btrfs_lookup_extent_info(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info, u64 bytenr, diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index fe1f229320ef..54fbdc944a3f 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4240,6 +4240,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, if (pin_bytes) btrfs_pin_extent(fs_info, head->bytenr, head->num_bytes, 1); + btrfs_cleanup_ref_head_accounting(fs_info, delayed_refs, head); btrfs_put_delayed_ref_head(head); cond_resched(); spin_lock(&delayed_refs->lock); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 34105bc5eef7..7245a198ad31 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2475,12 +2475,11 @@ static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans, return ret ? ret : 1; } -static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, - struct btrfs_delayed_ref_head *head) +void +btrfs_cleanup_ref_head_accounting(struct btrfs_fs_info *fs_info, + struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head) { - struct btrfs_fs_info *fs_info = trans->fs_info; - struct btrfs_delayed_ref_root *delayed_refs = - &trans->transaction->delayed_refs; int nr_items = 1; if (head->total_ref_mod < 0) { @@ -2558,7 +2557,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } - cleanup_ref_head_accounting(trans, head); + btrfs_cleanup_ref_head_accounting(fs_info, delayed_refs, head); trace_run_delayed_ref_head(fs_info, head, 0); btrfs_delayed_ref_unlock(head); @@ -7223,7 +7222,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (head->must_insert_reserved) ret = 1; - cleanup_ref_head_accounting(trans, head); + btrfs_cleanup_ref_head_accounting(trans->fs_info, delayed_refs, head); mutex_unlock(&head->mutex); btrfs_put_delayed_ref_head(head); return ret; From patchwork Fri Sep 28 11:18:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C439215A7 for ; Fri, 28 Sep 2018 11:19:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B66982B061 for ; Fri, 28 Sep 2018 11:19:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A96542B0CF; Fri, 28 Sep 2018 11:19:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54D5D2B061 for ; Fri, 28 Sep 2018 11:19:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729518AbeI1Rmh (ORCPT ); Fri, 28 Sep 2018 13:42:37 -0400 Received: from mail-qk1-f170.google.com ([209.85.222.170]:43995 "EHLO mail-qk1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729513AbeI1Rmh (ORCPT ); Fri, 28 Sep 2018 13:42:37 -0400 Received: by mail-qk1-f170.google.com with SMTP id v18-v6so3545631qka.10 for ; Fri, 28 Sep 2018 04:19:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=P0ipw3j2I3ux8QOjibv1ucfjdjXUNT2FexmyaP5BF3E=; b=CgOQ0H8uZs4lZnX19xSAV9Vl/qfNenrBVOn0R9WmTEdsUV5VGSN80/NFPHZPL0lewU v0YpGGPmTGz/s2rJpMRF9uT52oPF4In1MKLnqqlfJU9ZIHti4D38+u30tThRH/7zdIuq QvON8yTfpE58kXtUyk36OyqzECCQcHYeS+s4KkDrk+etCU0K0y7shSxzRefgiZdwtW6f PMmMhJEFyT9n36/DYqpb5v+hs3Tr7xqy/ZfGSC+NkWf8vEQ2a14A2nAky1TQGLCEiJhm w2wmWN+XSmLOLS+lDj4PiCb30sjh/P6rTp9NYr3xeiUkA6IZiFSS86LnK1jZwAI3f5l/ Id7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=P0ipw3j2I3ux8QOjibv1ucfjdjXUNT2FexmyaP5BF3E=; b=ORvI4IWAu8rLlRiKMcSdilD05dPm0ksKecrjGgQiSTygaHPQlh4S8BiS0p0tAo5oWo asQiQyV+DtK4dfobWyb0AsqnY/Qk/0bBw/hpVAxT31/nHD2LeqIu84zkRwL9sRSw/CE8 2xz/gQNVo1WKocktbyR8ivby7WWRpQ8ugnSbriGarf48n7ei4jhYVOsNTlA28zIuw/gx 3F+yfQtu/j2mNdi6ieB1W9xuTZBrD4blsHrpgJyE9oItgOkK6PsN41meQ8w07FYGA9H+ tENYzneggg++Nx4DnPsilho/G/1qV3KYw4LW9Ypz08P4BU8pNtAozRfOpUx3c6ya52OL OwWQ== X-Gm-Message-State: ABuFfoinohqKCEzndeUavFYwXsEnXDMNAwseu9ASWtrmqBy5w/cPREWz 014H1cV1gMAwrZwyVIkjlIfycg== X-Google-Smtp-Source: ACcGV61H+3oNjLJOoVMLRBuFLMMc3Ihd+z09seBjaKXqVnYuMX76PUJw1GLPbhzTtgwm0CWH2Hviuw== X-Received: by 2002:a37:a84:: with SMTP id 126-v6mr8650627qkk.267.1538133558802; Fri, 28 Sep 2018 04:19:18 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u36-v6sm2927608qtc.46.2018.09.28.04.19.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:17 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 29/42] btrfs: call btrfs_create_pending_block_groups unconditionally Date: Fri, 28 Sep 2018 07:18:08 -0400 Message-Id: <20180928111821.24376-30-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The first thing we do is loop through the list, this if (!list_empty()) btrfs_create_pending_block_groups(); thing is just wasted space. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 3 +-- fs/btrfs/transaction.c | 6 ++---- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 7245a198ad31..91e45cb14d45 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2978,8 +2978,7 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, } if (run_all) { - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); spin_lock(&delayed_refs->lock); node = rb_first(&delayed_refs->href_root); diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 39a2bddb0b29..46ca775a709e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -846,8 +846,7 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); btrfs_trans_release_chunk_metadata(trans); @@ -1934,8 +1933,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing = 1; smp_wmb(); - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); if (!test_bit(BTRFS_TRANS_DIRTY_BG_RUN, &cur_trans->flags)) { int run_it = 0; From patchwork Fri Sep 28 11:18:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619725 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F6CA6CB for ; Fri, 28 Sep 2018 11:19:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17AF52B061 for ; Fri, 28 Sep 2018 11:19:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0C3D32B0CF; Fri, 28 Sep 2018 11:19:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B01F02B061 for ; Fri, 28 Sep 2018 11:19:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729523AbeI1Rmi (ORCPT ); Fri, 28 Sep 2018 13:42:38 -0400 Received: from mail-qk1-f179.google.com ([209.85.222.179]:46783 "EHLO mail-qk1-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729513AbeI1Rmi (ORCPT ); Fri, 28 Sep 2018 13:42:38 -0400 Received: by mail-qk1-f179.google.com with SMTP id q12-v6so3531722qkl.13 for ; Fri, 28 Sep 2018 04:19:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=FaedjnvqEPjhcquf2LaidURx/buWzJK+i4n0WtJhwI8=; b=ZsCcd1DBGRmB+4KwglFPY+NQCZgUE7cxMH7Ld0M5QX6NNhCZQvpjLILZ/46rVoYu+R 2zOcZwYHeqPt3KBfvEOrWiP28aZGesSyrzw6hz8gygEgJVD/27p8jZo0/IQuvdzvRTwj OTjXD+kqo9WE7sOt7NM1OlAxsGf0U83SrSeBI9rf7wE6OyBL37HOUEZQ1phAPgFtQkLm JmVN3hLMxOD0pxXcuhknpX1IZQVHSieWqGriMA3cPN2z3aRVY79YUA80FqMZH7SerYGe 2ERDe+2OlS2gEaiHj53dzHpTc5bk8gtaznafzW0XwxsLYrdE3b2H8L3mJJb0Huy5NR+0 im/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=FaedjnvqEPjhcquf2LaidURx/buWzJK+i4n0WtJhwI8=; b=QNDBE5miF3Yug1X2I78io2dFMP0hZVzQsaX9fqajKc5YPHrnWXlyj1YQNw7sBdKSES dgrUKTgL04xUNHLOZO2MWxXMxprXxp2v+tZq9x8AyOT0uKmOrGNU6TUIiFEvj4Y0czUR uAay3L0aMDMIEwLPTrh3OmPhs25W6ajlUrd53Si5jBzOFqcwSL9cKfXp2utI8kITdLp0 PtUgSQeV0pdQm3cfcAxWLktL2j2SK2EgWjzxQkQqoqB07KxFyA8djQ5uxQDGF7ZfhEUP 4GWf9rjlz+VqE86iF+L31uNOS4PHvHnHliWIOp6n0bSwO/ee2Mmvwr/73EtWSZkZegGL j0sw== X-Gm-Message-State: ABuFfojGMgtWHXV642MxWoGBEbjmjEU1GbmOIcpuqy/4kYHB9MJayzwJ JhN4vjcu/UmU75oEP2b2PtKbMw== X-Google-Smtp-Source: ACcGV63q/xlzTEA9gO+cpKCoWp7aVuCzPQFdYBnXnljRcQcwAc5MGBxReIu5LFxsPK4OLpN0lkVpYA== X-Received: by 2002:a37:6388:: with SMTP id x130-v6mr4700039qkb.118.1538133560776; Fri, 28 Sep 2018 04:19:20 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a11-v6sm2748774qte.13.2018.09.28.04.19.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:19 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 30/42] btrfs: just delete pending bgs if we are aborted Date: Fri, 28 Sep 2018 07:18:09 -0400 Message-Id: <20180928111821.24376-31-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We still need to do all of the accounting cleanup for pending block groups if we abort. So set the ret to trans->aborted so if we aborted the cleanup happens and everybody is happy. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 91e45cb14d45..ac282eb535a8 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -10359,9 +10359,15 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) struct btrfs_root *extent_root = fs_info->extent_root; struct btrfs_block_group_item item; struct btrfs_key key; - int ret = 0; + int ret; bool can_flush_pending_bgs = trans->can_flush_pending_bgs; + /* + * If we aborted the transaction with pending bg's we need to just + * cleanup the list and carry on. + */ + ret = trans->aborted; + trans->can_flush_pending_bgs = false; while (!list_empty(&trans->new_bgs)) { block_group = list_first_entry(&trans->new_bgs, From patchwork Fri Sep 28 11:18:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DED415A7 for ; Fri, 28 Sep 2018 11:19:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 350242B061 for ; Fri, 28 Sep 2018 11:19:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 298B22B0CF; Fri, 28 Sep 2018 11:19:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE3FF2B061 for ; Fri, 28 Sep 2018 11:19:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729527AbeI1Rmk (ORCPT ); Fri, 28 Sep 2018 13:42:40 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:44809 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729513AbeI1Rmk (ORCPT ); Fri, 28 Sep 2018 13:42:40 -0400 Received: by mail-qt1-f195.google.com with SMTP id c56-v6so1241450qtd.11 for ; Fri, 28 Sep 2018 04:19:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=esyabqCBu+9iDj6gymgoZZfIo7W+xlWCseOBiKeoQzQ=; b=Oese1BH95QAWz361InMKFeP7xWkTBwxNBbYSicJAaQ8ykWpj8wdyXebyoQYNsRtqwd P7S/NIrBjLSYPh18EQShviSJ3y4kCRGqUTgTS/qq6vb41ZZXxmwoOjdj7b0cniGYzUVQ eJthzNje4j9Z+gRmKUc5fbgklzixkD6E3qM7r2cK78aCctON7+wGl/53jpSVLlOQGi2E FYx/pBZUz/W67TRSK7b3mQshz1xqXl732sGWH5WRl5P1siv2iEih5UeQCWJEEZ5WY/vC v7f8rdAaDLurlb6KPOp1FX7TzNku8Zqk6Rr6nx4uHecFq9Lj3MDMxHyrleLBSm76HPmA q6rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=esyabqCBu+9iDj6gymgoZZfIo7W+xlWCseOBiKeoQzQ=; b=c18p8F7PK3mIf0ysOqkadYKjMaBGZwjy99lBncJCaAJnYvO4X/YrRudTHGU7NRklI6 jVfJIyNHIXxKSEsi7CzIU2BVIizQpoY5KIOwYlGNjNsBnQ08r4ebvhhNJPbtjkBccmxX 1mIPzKMEbUZSg4MlY+GItYirEoIdW1JuX1JqBwVEnWD1VRtRIefRcAvgd1CSMpu+Js+l +dUM50bIE3jTkxVFzIhhND/2LD7Bwz/KnnUsvAvrRzPvns1AK4TPmwP7ZTZ5lAfWpRpp Qs/njrI6F6cpCFhQ0kDjy4BouFpM7YZoLHgkRrskf9/G/+t5YcYa8V1QBGXZ3u4SqwMU 5dTg== X-Gm-Message-State: ABuFfohSfaQcoBrt11FDJSn6IMBmyyVDcCchnkcjb2SObxg1xjia2xPV KuIQ9zxKKNz8kFgf2XuGFJocHg== X-Google-Smtp-Source: ACcGV62o+RfiMAWSwQ6LXQGAIF8UW7BLjXCJ0CjodJkoiHYZjdTkEcTr8sqtPam6QeVglyhn7KPcRg== X-Received: by 2002:a0c:83c4:: with SMTP id k62-v6mr11502062qva.88.1538133562704; Fri, 28 Sep 2018 04:19:22 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b9-v6sm2416140qtp.79.2018.09.28.04.19.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:21 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 31/42] btrfs: cleanup pending bgs on transaction abort Date: Fri, 28 Sep 2018 07:18:10 -0400 Message-Id: <20180928111821.24376-32-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We may abort the transaction during a commit and not have a chance to run the pending bgs stuff, which will leave block groups on our list and cause us accounting issues and leaked memory. Fix this by running the pending bgs when we cleanup a transaction. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 46ca775a709e..9168efaca37e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -2280,6 +2280,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_scrub_continue(fs_info); cleanup_transaction: btrfs_trans_release_metadata(trans); + /* This cleans up the pending block groups list properly. */ + if (!trans->aborted) + trans->aborted = ret; + btrfs_create_pending_block_groups(trans); btrfs_trans_release_chunk_metadata(trans); trans->block_rsv = NULL; btrfs_warn(fs_info, "Skipping commit of aborted transaction."); From patchwork Fri Sep 28 11:18:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619729 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C691B6CB for ; Fri, 28 Sep 2018 11:19:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD7912B061 for ; Fri, 28 Sep 2018 11:19:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1FD82B0CF; Fri, 28 Sep 2018 11:19:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 559412B061 for ; Fri, 28 Sep 2018 11:19:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729530AbeI1Rmn (ORCPT ); Fri, 28 Sep 2018 13:42:43 -0400 Received: from mail-qt1-f178.google.com ([209.85.160.178]:46492 "EHLO mail-qt1-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729513AbeI1Rmm (ORCPT ); Fri, 28 Sep 2018 13:42:42 -0400 Received: by mail-qt1-f178.google.com with SMTP id h22-v6so6090642qtr.13 for ; Fri, 28 Sep 2018 04:19:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=OTZOuDgDt/R5spujlNFzi4h/V31eI1viGWGcVEUBX/Y=; b=g3f0vPm8untRqZQJPQGfmRU69iIfSg4kuTVdkqXkFjSzxYwprjdBpQnzbaQjCkpJkm 0JY3g6kS82hHFkxMNu1BBwzXIOnWAwEyz7tyBoR3YvkF7Pg7rk1+Rk3H2KaSGuJg8dro /T3zGvZEfkSQ5RflpqKNa/HG9ymE4s5t+iRV6w3EpYqf1pPUrGSq1HxT+AjXv+eo8Fc1 u8ammbLSJded8uNmFgFAu5ztwAXXmPo4SSPaKWQLPqil1cBNCS3FX4z7rAKroyPu6fxM nK2TMj44uj2AxmT9M7IyS5xkkzd7wR+tmZpkKasb4JOdUcBZEOTIgxUycibA4RMbBjrA wEfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=OTZOuDgDt/R5spujlNFzi4h/V31eI1viGWGcVEUBX/Y=; b=I4VVY5oryz1d9AVAVhyrOg1loWCjRCxx3bT64s/ctDU1rVzB9KCjoHAElBXIc5Xu3A 77q/RtuBbwExpCb9jBUwa6IDjIjlB1Pmku6tviwCr02qHzmJADuISOZSYGNNmvirhN6j IHgf3os50ih8iy0X8UxJtGkGUDAusrB1Fq0LC8gdT1Doi6Fab5Gnrc+9lxgB/oJ+F/Mz 1BfCkzi3lZSuslaTMhJEiwE7NWnNgkP1QZv4snD6RNtCZNMw7ic6N3T8l1ka9EC9F01f 6jauOIJ5vI6MChaHXLEE5pptTq+b1tbEFwwj8fihnV6XrSng4GcvIyYVjEoJfHBWIGhg J0zw== X-Gm-Message-State: ABuFfogV8P3pefvhijLy7SoLUUS9V6mik8BdeS1SOIFVjxsRCJq4lUEG KgpMgAxEI3EyM2DqGjbr1LZIGA== X-Google-Smtp-Source: ACcGV62a9eD2fSQ/FPceO9RauWOLul1Pa/QA2GzXuoRHx5gq/+HvUnDS0tQshEDel+iP01Gjriw5HA== X-Received: by 2002:ac8:37f2:: with SMTP id e47-v6mr11579448qtc.32.1538133564532; Fri, 28 Sep 2018 04:19:24 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u184-v6sm2611219qkc.87.2018.09.28.04.19.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:23 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 32/42] btrfs: only free reserved extent if we didn't insert it Date: Fri, 28 Sep 2018 07:18:11 -0400 Message-Id: <20180928111821.24376-33-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When we insert the file extent once the ordered extent completes we free the reserved extent reservation as it'll have been migrated to the bytes_used counter. However if we error out after this step we'll still clear the reserved extent reservation, resulting in a negative accounting of the reserved bytes for the block group and space info. Fix this by only doing the free if we didn't successfully insert a file extent for this extent. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/inode.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 5a91055a13b2..2b257d14bd3d 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -2992,6 +2992,7 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) bool truncated = false; bool range_locked = false; bool clear_new_delalloc_bytes = false; + bool clear_reserved_extent = true; if (!test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags) && @@ -3095,10 +3096,12 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) logical_len, logical_len, compress_type, 0, 0, BTRFS_FILE_EXTENT_REG); - if (!ret) + if (!ret) { + clear_reserved_extent = false; btrfs_release_delalloc_bytes(fs_info, ordered_extent->start, ordered_extent->disk_len); + } } unpin_extent_cache(&BTRFS_I(inode)->extent_tree, ordered_extent->file_offset, ordered_extent->len, @@ -3159,8 +3162,13 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) * wrong we need to return the space for this ordered extent * back to the allocator. We only free the extent in the * truncated case if we didn't write out the extent at all. + * + * If we made it past insert_reserved_file_extent before we + * errored out then we don't need to do this as the accounting + * has already been done. */ if ((ret || !logical_len) && + clear_reserved_extent && !test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags)) btrfs_free_reserved_extent(fs_info, From patchwork Fri Sep 28 11:18:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619731 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 351926CB for ; Fri, 28 Sep 2018 11:19:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 297592B061 for ; Fri, 28 Sep 2018 11:19:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1BEC52B0F2; Fri, 28 Sep 2018 11:19:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC9212B061 for ; Fri, 28 Sep 2018 11:19:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729535AbeI1Rmo (ORCPT ); Fri, 28 Sep 2018 13:42:44 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:38190 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729513AbeI1Rmo (ORCPT ); Fri, 28 Sep 2018 13:42:44 -0400 Received: by mail-qt1-f193.google.com with SMTP id z13-v6so6144982qts.5 for ; Fri, 28 Sep 2018 04:19:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=SPKf73KNGAa6s3Pq3EYrX+Ny6AS9kkFExw/KdZENJxk=; b=0zJ8DkUQr3aW1g/iHvsqSVMmRXrO/eKievaTdO6lOKeToiPtXsLjbcBCDDFYImyO7D akNgYJANygRbmfx5AWorFRdWSMiNHG+S6JS5nxGQ8A8oau5EK5cJFnZ/Me8lz9lQ9yvQ FXhv6ZGjJ0Ev9V3fxWfttyYFCDjuaRvR9qt9hFv0s1QxlPb4cabggCqEMPgKmJ+lZ1c9 8eCMtw61oDUCDXzjdilyQ8u0gHj/SWKACIKacq2n+lQR1fkInw1T26c2vtcbWzvN8DJD yHF4szzSckkPMWN/Swubw/5uOvB3vdxO6/2qoIyZM8Nor/NbpZM6UoLFC9CCpApLRGwf dWDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=SPKf73KNGAa6s3Pq3EYrX+Ny6AS9kkFExw/KdZENJxk=; b=OS7Kpg21uAf4eKKxutt3tV8ajzNEzazISZ1XM4AOkgoatAkJ38SEondO0OD9FOCP0A SWtK1XjfMKcTQohBwcHCsu5JSG9LJmuuY8I+5X0d/S1Iy65U9hNmjn6RP1EKX341mmrQ kmYH9iL2CzFzvosTy3yr5SGi7q9DAv9teLt0XyXr1Evp4mrTZjf9j0IK76aGU3fZmMqS hfDycI/xhv7uIsMDEoogn8ISHFudX9HZE3teDQecVS3IxeYYAxiOi9jw+5jA0q1o/VpQ Wz/EN+3ovN1jnrxxmslQalxVJF8n7V8rQMMFnCLvmNeuLs3ho1goy0XN8wZimsv3Jco3 X0jw== X-Gm-Message-State: ABuFfohEJTr/6ctUCEIqLDVNzpEWKzhv57as+2Eu7eR9/0tvs24lKaIO 6uYIUS+iqObl1P61NDP3E0Ln4g== X-Google-Smtp-Source: ACcGV61M1LgCPpK32JIppLQx7pmKYOo4K66RQu9IxXeD0mN8HTg00AZQI32MJMVIqFkZL2ypqspaOg== X-Received: by 2002:a0c:91fd:: with SMTP id r58-v6mr1170861qvr.30.1538133566309; Fri, 28 Sep 2018 04:19:26 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s51-v6sm3213687qta.71.2018.09.28.04.19.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:25 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 33/42] btrfs: fix insert_reserved error handling Date: Fri, 28 Sep 2018 07:18:12 -0400 Message-Id: <20180928111821.24376-34-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We were not handling the reserved byte accounting properly for data references. Metadata was fine, if it errored out the error paths would free the bytes_reserved count and pin the extent, but it even missed one of the error cases. So instead move this handling up into run_one_delayed_ref so we are sure that both cases are properly cleaned up in case of a transaction abort. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index ac282eb535a8..922dd509591a 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2405,6 +2405,9 @@ static int run_one_delayed_ref(struct btrfs_trans_handle *trans, insert_reserved); else BUG(); + if (ret && insert_reserved) + btrfs_pin_extent(trans->fs_info, node->bytenr, + node->num_bytes, 1); return ret; } @@ -8253,21 +8256,14 @@ static int alloc_reserved_tree_block(struct btrfs_trans_handle *trans, } path = btrfs_alloc_path(); - if (!path) { - btrfs_free_and_pin_reserved_extent(fs_info, - extent_key.objectid, - fs_info->nodesize); + if (!path) return -ENOMEM; - } path->leave_spinning = 1; ret = btrfs_insert_empty_item(trans, fs_info->extent_root, path, &extent_key, size); if (ret) { btrfs_free_path(path); - btrfs_free_and_pin_reserved_extent(fs_info, - extent_key.objectid, - fs_info->nodesize); return ret; } From patchwork Fri Sep 28 11:18:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619733 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F285B6CB for ; Fri, 28 Sep 2018 11:19:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAD9C2B0F2 for ; Fri, 28 Sep 2018 11:19:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD02F2B0F4; Fri, 28 Sep 2018 11:19:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BFBA2B0C5 for ; Fri, 28 Sep 2018 11:19:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729539AbeI1Rmq (ORCPT ); Fri, 28 Sep 2018 13:42:46 -0400 Received: from mail-qk1-f170.google.com ([209.85.222.170]:45735 "EHLO mail-qk1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729513AbeI1Rmq (ORCPT ); Fri, 28 Sep 2018 13:42:46 -0400 Received: by mail-qk1-f170.google.com with SMTP id q20-v6so3532266qkc.12 for ; Fri, 28 Sep 2018 04:19:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=o3aniGDG3zSjx1n1+LpysocBJ3oenkOpLCURB29A0Uc=; b=UQIBHyfdBnZqPJN7DSbVKETuAVs5PBIMQ2I17Y+LLv4XxcJ0y2bNuvNMn/YZzDitzK EYUnttxSlLB+0PDk6d075ESYOzDwCfX2WDEruq8047iwRmOaYe0dj+Li/pjQHe2QgAJa iwEUg2LqgwNDbK1O8kxm1bS3cNE/lvlDYeZFHmZ10vB/yJbF0qUGfCg37892YAaIlDSl NfaIXWGMsesX+BpsETT5/HEcYaGSnzAuKbo0X9av3DwU16QtIpG7ZLWp6iklFrrxaN6p xzxMw6OtIBZf0aEC2Y42UsK3KyV1GOUGAd2d8SG1k6BMZQ0yA6bpUy0evQRc9+RzR500 sRfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=o3aniGDG3zSjx1n1+LpysocBJ3oenkOpLCURB29A0Uc=; b=tyCb0uNGIUsoOb5kvUMStDC0XMXiyqKJJl4FwAhW8Cj38VHDkhL73814U6dL3ocgCJ Qr/1HLrqm8NEbDsZs2wyDu6RPFJR+pj/S/PJRbu+/2rJOMxk+IUFy1bVTiaRIQsqb01Z 4m85agtdJKY4j0SvGIXlzB2aUDfDIIfzL7QuhQLO6Wn+UBmuHL+ZiTSQydhOaLctT4pu uufDZt/WOtCDKllfTX9rrr59ps2Np/6Sq9J1kuqVYkyX1upc639HzRTmQzDEZgJDi5hw 9sxR6TMaOZfNoIXDt3AwqJTEcx/0io25JJoNxqPVvcjdKTq2EOKMsFEytfISPoLj7eoC Ru+w== X-Gm-Message-State: ABuFfog+he36F833KHsslKP2jRmZefZ2jfL/BJTKBy77qEGzoHjtWgvv tXyuP+7jBpZQrMV9J36FeBDOjA== X-Google-Smtp-Source: ACcGV62GK5armn7lzfc7ddpxfG54iB7hyzfkp2sOm4L1KmtecpR2HCIGIinjBFpO5l0/xAwccNYUcA== X-Received: by 2002:a37:8406:: with SMTP id g6-v6mr11210070qkd.148.1538133568068; Fri, 28 Sep 2018 04:19:28 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f53-v6sm3221242qtk.40.2018.09.28.04.19.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:27 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 34/42] btrfs: wait on ordered extents on abort cleanup Date: Fri, 28 Sep 2018 07:18:13 -0400 Message-Id: <20180928111821.24376-35-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we flip read-only before we initiate writeback on all dirty pages for ordered extents we've created then we'll have ordered extents left over on umount, which results in all sorts of bad things happening. Fix this by making sure we wait on ordered extents if we have to do the aborted transaction cleanup stuff. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 54fbdc944a3f..51b2a5bf25e5 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4188,6 +4188,14 @@ static void btrfs_destroy_all_ordered_extents(struct btrfs_fs_info *fs_info) spin_lock(&fs_info->ordered_root_lock); } spin_unlock(&fs_info->ordered_root_lock); + + /* + * We need this here because if we've been flipped read-only we won't + * get sync() from the umount, so we need to make sure any ordered + * extents that haven't had their dirty pages IO start writeout yet + * actually get run and error out properly. + */ + btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, (u64)-1); } static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, From patchwork Fri Sep 28 11:18:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619735 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5EC415A7 for ; Fri, 28 Sep 2018 11:19:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CAA12B061 for ; Fri, 28 Sep 2018 11:19:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9116E2B0F4; Fri, 28 Sep 2018 11:19:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A0CB2B061 for ; Fri, 28 Sep 2018 11:19:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729542AbeI1Rms (ORCPT ); Fri, 28 Sep 2018 13:42:48 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:33158 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729368AbeI1Rms (ORCPT ); Fri, 28 Sep 2018 13:42:48 -0400 Received: by mail-qt1-f196.google.com with SMTP id i10-v6so6194616qtp.0 for ; Fri, 28 Sep 2018 04:19:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ja6BEdk2d172wcHDS/gitUITWM0Dn5OfC8ykxBklKgo=; b=zaZsb4n+orpW4Yu7Z22PhRRjFpeRt2pXSJlIdXo2VHDrxjIVFQH37h6TW+5GnXn87Y xP7dgM3hQWufP+z5OW2M2aVC2migqv8/4GFzf8cVEp/Fx69rh/Xlq9XCKEngfTjWL3PY EjGKiJVwg4Dc+aVRhXDp+CcRHW5A/Wxep38st1VkpLmaGh4TrfILByhtaB2fXasY48Yw fgWDGG+f3jzlxAvY1b9nvLV/mXUN+/pK6QY5NTSZyMbYdswllwbfYY7SMvRUEImz3hpR +pSiAkq73oID+gPBxDaONVFJ/cQsVC2+inQMVS1/JRSdjti650HVlRTuSigCvZtMAwRJ Aiew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ja6BEdk2d172wcHDS/gitUITWM0Dn5OfC8ykxBklKgo=; b=SMY+ZsFfs4r2VxgMzSWJcf99Svzv8a4/EvkHYvFC0Mfh0mswH9A8ly5xtX4w8+EvZv d6I3EAx4Mz152zGNjTT9qiS5hq/K2wEvREGQUjGC5d/R6z5WOTm5BKPXpfLnonwwAvQ4 z3tW4jWiDoGsxnRd6WkM6kfeSkcbHv+dwktlaDuRcm5fwqZ1JUgxw38FYoMXevv3V+kh zl268roQ2vXorUUuuGhhRnfuzQ+HFF8pu6P/RG+wIm+VOhcs3Ubb/eK5RjpCexJ0GpVv VFhzB2QEbhPo6eYROocr+RcmTgMYcRTx5u80+VJZ3AL6t4Qyr5VRlQKniHxwWGGnfxMl 8SAA== X-Gm-Message-State: ABuFfogxLjV28Dut5EczFQRp8CpIRqCp4EZIN7NN6MFoenh+S0tiLrak 0UZ2E0qqOMBoPH5gl9UzzYrOP5uORTQ= X-Google-Smtp-Source: ACcGV63Gr/06adrxpVKqPZIiHGG+Dcpo/IpRjjWPSmX/VLrPDF4sRf4P+SU3DHj8hSqFwHoRwEA8Xg== X-Received: by 2002:aed:2807:: with SMTP id r7-v6mr11887644qtd.68.1538133569946; Fri, 28 Sep 2018 04:19:29 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b9-v6sm2416296qtp.79.2018.09.28.04.19.28 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:28 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 35/42] MAINTAINERS: update my email address for btrfs Date: Fri, 28 Sep 2018 07:18:14 -0400 Message-Id: <20180928111821.24376-36-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP My work email is completely useless, switch it to my personal address so I get emails on a account I actually pay attention to. Signed-off-by: Josef Bacik --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index 32fbc6f732d4..7723dc958e99 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3095,7 +3095,7 @@ F: drivers/gpio/gpio-bt8xx.c BTRFS FILE SYSTEM M: Chris Mason -M: Josef Bacik +M: Josef Bacik M: David Sterba L: linux-btrfs@vger.kernel.org W: http://btrfs.wiki.kernel.org/ From patchwork Fri Sep 28 11:18:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619737 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A90715A7 for ; Fri, 28 Sep 2018 11:19:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 319002B061 for ; Fri, 28 Sep 2018 11:19:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25B352B0F4; Fri, 28 Sep 2018 11:19:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 161212B061 for ; Fri, 28 Sep 2018 11:19:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729543AbeI1Rmu (ORCPT ); Fri, 28 Sep 2018 13:42:50 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:45994 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1Rmt (ORCPT ); Fri, 28 Sep 2018 13:42:49 -0400 Received: by mail-qk1-f193.google.com with SMTP id q20-v6so3532355qkc.12 for ; Fri, 28 Sep 2018 04:19:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=v3HuVx6e30n18glhWe6CLEo+SQ0QfuV0FvwRHevY9qI=; b=qo8cy6r4VCTYoBYD99WKLdV866gA33LtqlMMBSBihuUBMRfDLDNbdhF5BiIazo6/rE y0/os18K0P2rTxhCNgi9Kp6ETQkfFsgELx4BLEn19RXDwl5nV/YwXt7XWoPlWHu7jXpx TnSHIHH2orSv326S4JEFK9348LQ0RTcfWzbsnEyx8DalMOXIkq8f3HMpffa4aOhb/eSo 7mRNyUO1LNJUd1mstd4FqHxmVIBxNcWCNnRZcn0L/a2f3id7Dr+Dqg6QR0FuOFuxZDud pa/3m/Z9Q0kOO2ChQWMMq053BKRUSObN4h8UTwW986EWaU1hqPotzFU40RmICzbCmX8u 8o+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=v3HuVx6e30n18glhWe6CLEo+SQ0QfuV0FvwRHevY9qI=; b=VjinpipXIdtkmZeSc5PtkmAqtP+cHxk2Qp40t+umdHzSezc7l8p+s1Z3C2Mg05I0r1 kBrApgjQTpTHqOC0zryYV3YV/eME/kpXUEzvDr9ijYzmcRX3vLUEJgkHiBdooCNIjWi/ x1wn2CwpXFBBIdK9biEtLHrDWyaHh6y6D/289iI9AWTm7xTPBDYJyXG1dZToYSp6VeT1 f5Zw5HwEy2M4+aYmSxO3Z35ZVujd2Roz799l2A++Z90iGVomhdzYuSg3YABq8+mva4kY Nb23So0bCkCH5SbaQjkzM9B9ernoX/juH6hObjTwUU2MzCjxpvXUOCyeEMFkQPC2iSQo 6Qnw== X-Gm-Message-State: ABuFfoh0qIPyCrY3cAgdSEYVkc7RTFZEgtP+0KmPRkBwSKrMu2BNAVSL Bn34YPKB4k/AkRDcp+0qb+/BAg== X-Google-Smtp-Source: ACcGV63tg/jAXLswVRx0JOAjUfFcjaYJYdNBmYA94WCbHwRxgORLYvRReo256lzLocXdRTGfHcBgng== X-Received: by 2002:a37:e104:: with SMTP id c4-v6mr11426712qkm.28.1538133571767; Fri, 28 Sep 2018 04:19:31 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id n83-v6sm2549515qki.60.2018.09.28.04.19.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:30 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 36/42] btrfs: wait on caching when putting the bg cache Date: Fri, 28 Sep 2018 07:18:15 -0400 Message-Id: <20180928111821.24376-37-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is because we aren't waiting for the caching threads to be done before freeing everything up, so to fix this make sure we wait on any outstanding caching that's being done before we free up the block group, so we're sure to be done with all IO by the time we get to btrfs_stop_all_workers(). This fixes the panic I was seeing consistently in testing. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 922dd509591a..262e0f7f2ea1 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -9890,6 +9890,7 @@ void btrfs_put_block_group_cache(struct btrfs_fs_info *info) block_group = btrfs_lookup_first_block_group(info, last); while (block_group) { + wait_block_group_cache_done(block_group); spin_lock(&block_group->lock); if (block_group->iref) break; From patchwork Fri Sep 28 11:18:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619739 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E47576CB for ; Fri, 28 Sep 2018 11:19:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA4E52B061 for ; Fri, 28 Sep 2018 11:19:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE6BD2B0FD; Fri, 28 Sep 2018 11:19:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C0AC02B061 for ; Fri, 28 Sep 2018 11:19:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729547AbeI1Rmv (ORCPT ); Fri, 28 Sep 2018 13:42:51 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:39397 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1Rmv (ORCPT ); Fri, 28 Sep 2018 13:42:51 -0400 Received: by mail-qt1-f194.google.com with SMTP id c25-v6so6146541qtp.6 for ; Fri, 28 Sep 2018 04:19:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ANeznX0lp5d9NlWDDlAsdKM+75WpkJ49Ux0qD1evMCM=; b=G+jIjuh1F0ReR+rc4SULRsAqiSLqm/f5loGvH9R+Y6G7i/xM/rAzVgrH0bRr5+Ca+K frX2RNhco+1YL4e6D5WkXfb3WWvckAXvIn3VpS7ttsAYPEGpFHUuGz8dgzRpN/zzm6ce UmE5V6GeUt9m2C5DBCrVVvS33Dk7W61AYV/BUaPVMG5hHwI3SubpyMinKnGAjgPSodGE gNVXbOvPV9V1XjTXiOJ0W8LlSamjEuE0xGH0OKk8+ppLuL7SLtPdpCI2L4iEOushS8Ix fdroiFe68ZTJXdinO9QPrVpU55Y86e4bmirSCoosfW22Z8k/5b/EpCObuVK3sDre5op4 8deQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ANeznX0lp5d9NlWDDlAsdKM+75WpkJ49Ux0qD1evMCM=; b=qAwJwE2tq9NGu/IcGJ/BRjMMrRuaAWim440IeZ+UR7aMBhsSxPIER5af39vOB3N3jh 33UkrPFEk5jaM80cdLcUIcE7t1WgqVvQOBGJyQU9FQjJjzFsMXQXpD+F192lpybqGzQf 5DtJdQoEXUSwhsghPYzFV1BM7QrX39BIQaEdgLV3XEygLNhogVnS9n/+cLPOtf44uRLs m6rn8mw6bLc3vtvP/mXG3JjN0ziZjapxmPBkNugTkr/Ny8BEml7xZPZrqXN9e2Dp3TAX hOgqK9vkNgXNL8WplMc6Yw0cKrdBy+Ew3Jb8/hkRRJHmE+nNS2fi2xAHbpfNj8be8GLv snjg== X-Gm-Message-State: ABuFfohLfbLGRUru+w+XSAJPm0d7qQCClngQQbdg7tNnkHoS6+5uYc/b V+/MtODm74Z1glFHur6pFeQ4DN8WQ9U= X-Google-Smtp-Source: ACcGV63e+BL5BVKUaRLcviAjn+IjSpdlbYM9vZj/h4Pvbe1yFUsDXZPtVxEPNSKzTNk5WZMzfdH8Zg== X-Received: by 2002:a05:6214:46:: with SMTP id c6mr3092417qvr.128.1538133573576; Fri, 28 Sep 2018 04:19:33 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c47-v6sm2958505qtd.70.2018.09.28.04.19.32 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:32 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 37/42] btrfs: wakeup cleaner thread when adding delayed iput Date: Fri, 28 Sep 2018 07:18:16 -0400 Message-Id: <20180928111821.24376-38-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The cleaner thread usually takes care of delayed iputs, with the exception of the btrfs_end_transaction_throttle path. The cleaner thread only gets woken up every 30 seconds, so instead wake it up to do it's work so that we can free up that space as quickly as possible. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 2b257d14bd3d..0a1671fb03bf 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -3323,6 +3323,7 @@ void btrfs_add_delayed_iput(struct inode *inode) ASSERT(list_empty(&binode->delayed_iput)); list_add_tail(&binode->delayed_iput, &fs_info->delayed_iputs); spin_unlock(&fs_info->delayed_iput_lock); + wake_up_process(fs_info->cleaner_kthread); } void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info) From patchwork Fri Sep 28 11:18:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32A006CB for ; Fri, 28 Sep 2018 11:19:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 280E72B061 for ; Fri, 28 Sep 2018 11:19:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1BE142B0F4; Fri, 28 Sep 2018 11:19:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B3E212B061 for ; Fri, 28 Sep 2018 11:19:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729551AbeI1Rmx (ORCPT ); Fri, 28 Sep 2018 13:42:53 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:38214 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1Rmx (ORCPT ); Fri, 28 Sep 2018 13:42:53 -0400 Received: by mail-qt1-f194.google.com with SMTP id z13-v6so6145420qts.5 for ; Fri, 28 Sep 2018 04:19:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=RHqXkFYQ/isH8yVGHYsgnjCqUZbpOKGf9EwG0OJglnw=; b=gbRchkiRnzH5GMzLrpiJ66VziIn1QNi794B6dLadUzsRNQWLQCG8/tRCDYBDz7WbTn D2IpEgcZc43TF+DmFmegjsabopl8HW/74CrqiEWUl4hpRG8JCvKLMiQoKmcKuo/SePTX iw1+D1PAf81ei39X8lI1qVvUY4IGaT148VizP4oY2QT19qfts8t/S08TLubXJyI/VXUH e0Sfyt4Zd6egwqrnup/pYkBBPzoyXh3tOS5//Rm6zNan+clIDc5bVYWV6gsyKJe+wS0i P1SG3LrMLtdgTAXA4Mb1eXKpuZ2eReyF8oWo31kd/7k6dz5x858vyzQGtMA6QUQCDcbx anBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=RHqXkFYQ/isH8yVGHYsgnjCqUZbpOKGf9EwG0OJglnw=; b=GphcrQ8M4eL93jc4GVjhBRiU3Ad9xqNxwPhGQkONGQfsueIoaHanqoeJmgJ79WZKJa YBm224xFjfKJFyixz3z85QUlLKh1kqcQUZbFGeGWwW5OG2s/5mFcJvP2Cj4aqdlFyRP8 y2BvDRsZxM831ZdQgERvy8aEzsNWvr/j802L+qP9FFK9HZ5/7ZTC8paan83wPRPRf+Hx jXpHJEp4byUalSoz5Li6yFfSsTb+MbX6qZbm2JEY24ZGZv6HxRLIBu4RUmjG3JdjZXc9 f1rzN+zTImvJnbiotF+8Kpc+3nFnliOA7mofd6wQi9LiY4+YwvmF+DAeAwDMHC2VV2Q7 MnDA== X-Gm-Message-State: ABuFfogiKYK8mxWZ0UFVaSmgQEwun/QSuj5PQnFzWeFlpMbQZnWu/46w uzzLyhqmnQU1y1FNDtn9TaqV9lpcbqU= X-Google-Smtp-Source: ACcGV61nNfesoSZsZWYhlcPxuO3DsmFwWZ7TAg6HaLyZOlE7zYt562b662N9LqJbv+RFHnBh1/iTvA== X-Received: by 2002:a0c:f40c:: with SMTP id h12-v6mr6274301qvl.78.1538133575303; Fri, 28 Sep 2018 04:19:35 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id k185-v6sm2551197qkd.27.2018.09.28.04.19.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:34 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 38/42] btrfs: be more explicit about allowed flush states Date: Fri, 28 Sep 2018 07:18:17 -0400 Message-Id: <20180928111821.24376-39-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For FLUSH_LIMIT flushers we really can only allocate chunks and flush delayed inode items, everything else is problematic. I added a bunch of new states and it lead to weirdness in the FLUSH_LIMIT case because I forgot about how it worked. So instead explicitly declare the states that are ok for flushing with FLUSH_LIMIT and use that for our state machine. Then as we add new things that are safe we can just add them to this list. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 262e0f7f2ea1..a7ba0d0e8de1 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5110,12 +5110,18 @@ void btrfs_init_async_reclaim_work(struct work_struct *work) INIT_WORK(work, btrfs_async_reclaim_metadata_space); } +static const enum btrfs_flush_state priority_flush_states[] = { + FLUSH_DELAYED_ITEMS_NR, + FLUSH_DELAYED_ITEMS, + ALLOC_CHUNK, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) { u64 to_reclaim; - int flush_state = FLUSH_DELAYED_ITEMS_NR; + int flush_state = 0; spin_lock(&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info, @@ -5127,7 +5133,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, + priority_flush_states[flush_state]); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) { @@ -5135,15 +5142,7 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, return; } spin_unlock(&space_info->lock); - - /* - * Priority flushers can't wait on delalloc without - * deadlocking. - */ - if (flush_state == FLUSH_DELALLOC || - flush_state == FLUSH_DELALLOC_WAIT) - flush_state = ALLOC_CHUNK; - } while (flush_state < COMMIT_TRANS); + } while (flush_state < ARRAY_SIZE(priority_flush_states)); } static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Fri Sep 28 11:18:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619745 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B0246CB for ; Fri, 28 Sep 2018 11:19:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 908952B061 for ; Fri, 28 Sep 2018 11:19:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 84D5C2B0F4; Fri, 28 Sep 2018 11:19:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED6292B061 for ; Fri, 28 Sep 2018 11:19:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729554AbeI1Rm4 (ORCPT ); Fri, 28 Sep 2018 13:42:56 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:33176 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1Rmz (ORCPT ); Fri, 28 Sep 2018 13:42:55 -0400 Received: by mail-qt1-f194.google.com with SMTP id i10-v6so6194935qtp.0 for ; Fri, 28 Sep 2018 04:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=LU0+VWWWubdNmxxLWHJ3F1vkXtQo+CBzk4lWtJGRwKk=; b=s3rvmWeSk8G4Pn4Tfrz4vt6RmUW4McbmawLNthU+LHGeE151wTp5Wz30OTpIosfWJT BpoVk79D653/aEjdt+SQjn+BJDnPcSHSGBP5n6aBW9V3n9YgFjGovRj7Bdk66uZN8ft+ nQ6whrvX7nOMKfyRGhWtnhO2taf16aQv3f5Q9eOYQoWuS6mo1OWHrUEwLx2X8oy4QRSp 9y9P10X1hyi9BCI5+3PFu0k3zuTXQdtkXG1TCJBLTKCYj0vSZQdxIHmZ08XDRCbByLXj zT3SPFJ0qO1GHXasXwh8ZNiMA2ZNn/suLVzg459r/VNOR1ELgjsyqJq1joE3zN9XE9zJ SStw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=LU0+VWWWubdNmxxLWHJ3F1vkXtQo+CBzk4lWtJGRwKk=; b=BSAVD9uyk7/TX2fzPg2S4P5yuZ9F5fbDYujdZSKttBLtzwXuAwkZuCp1MXOYvZj8ji FIdRy84EiviaAco/jH5eHDa4EVftlGAYtQAoSRxJIpLJ5T0FB6GSDrqwYgVczmfi43xk kDeyDEfVBSJ0+YTETMkvy3qhLBMBUlMZpC1etkaUk7Su86QnAPfIUaYGOjlrCOXgIWvL JCb1YrkEvzOVjfC/4iFACE7l1O80Si3BjcVLqI/8UG4izYlvSuYhXqCPs58uNcYLtIlO CXkShVsbX3RXWzcjwpCJ4pHPKLFhrsLCArK7B2VXAQJaCLeWDMweiEa93n/gVJQ9NH5F bdyQ== X-Gm-Message-State: ABuFfojXFOCeAUYTspdkC3LnMM16GTFtbYcvfXPTu07aipfGa39k5tJ0 DXEVSGFr0zLN852I90iy4tctsw== X-Google-Smtp-Source: ACcGV60djeCNCU+5USVF0a7VrZUV1N53qUIcNDeo5jrFGr9QsVL5QMxFpWQnoJTyC2V8D7FULGFlWg== X-Received: by 2002:a0c:be01:: with SMTP id k1-v6mr11188406qvg.169.1538133577094; Fri, 28 Sep 2018 04:19:37 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id f85-v6sm3252406qkh.35.2018.09.28.04.19.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:36 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 39/42] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue Date: Fri, 28 Sep 2018 07:18:18 -0400 Message-Id: <20180928111821.24376-40-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The throttle path doesn't take cleaner_delayed_iput_mutex, which means we could think we're done flushing iputs in the data space reservation path when we could have a throttler doing an iput. There's no real reason to serialize the delayed iput flushing, so instead of taking the cleaner_delayed_iput_mutex whenever we flush the delayed iputs just replace it with an atomic counter and a waitqueue. This removes the short (or long depending on how big the inode is) window where we think there are no more pending iputs when there really are some. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 4 +++- fs/btrfs/disk-io.c | 5 ++--- fs/btrfs/extent-tree.c | 9 +++++---- fs/btrfs/inode.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index e40356ca0295..1ef0b1649cad 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -894,7 +894,8 @@ struct btrfs_fs_info { spinlock_t delayed_iput_lock; struct list_head delayed_iputs; - struct mutex cleaner_delayed_iput_mutex; + atomic_t nr_delayed_iputs; + wait_queue_head_t delayed_iputs_wait; /* this protects tree_mod_seq_list */ spinlock_t tree_mod_seq_lock; @@ -3212,6 +3213,7 @@ int btrfs_orphan_cleanup(struct btrfs_root *root); int btrfs_cont_expand(struct inode *inode, loff_t oldsize, loff_t size); void btrfs_add_delayed_iput(struct inode *inode); void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info); +int btrfs_wait_on_delayed_iputs(struct btrfs_fs_info *fs_info); int btrfs_prealloc_file_range(struct inode *inode, int mode, u64 start, u64 num_bytes, u64 min_size, loff_t actual_len, u64 *alloc_hint); diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 51b2a5bf25e5..3dce9ff72e41 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1692,9 +1692,7 @@ static int cleaner_kthread(void *arg) goto sleep; } - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); btrfs_run_delayed_iputs(fs_info); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); again = btrfs_clean_one_deleted_snapshot(root); mutex_unlock(&fs_info->cleaner_mutex); @@ -2677,7 +2675,6 @@ int open_ctree(struct super_block *sb, mutex_init(&fs_info->delete_unused_bgs_mutex); mutex_init(&fs_info->reloc_mutex); mutex_init(&fs_info->delalloc_root_mutex); - mutex_init(&fs_info->cleaner_delayed_iput_mutex); seqlock_init(&fs_info->profiles_lock); INIT_LIST_HEAD(&fs_info->dirty_cowonly_roots); @@ -2699,6 +2696,7 @@ int open_ctree(struct super_block *sb, atomic_set(&fs_info->defrag_running, 0); atomic_set(&fs_info->qgroup_op_seq, 0); atomic_set(&fs_info->reada_works_cnt, 0); + atomic_set(&fs_info->nr_delayed_iputs, 0); atomic64_set(&fs_info->tree_mod_seq, 0); fs_info->sb = sb; fs_info->max_inline = BTRFS_DEFAULT_MAX_INLINE; @@ -2776,6 +2774,7 @@ int open_ctree(struct super_block *sb, init_waitqueue_head(&fs_info->transaction_wait); init_waitqueue_head(&fs_info->transaction_blocked_wait); init_waitqueue_head(&fs_info->async_submit_wait); + init_waitqueue_head(&fs_info->delayed_iputs_wait); INIT_LIST_HEAD(&fs_info->pinned_chunks); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index a7ba0d0e8de1..77bc53ad84e9 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4258,8 +4258,9 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) * operations. Wait for it to finish so that * more space is released. */ - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + ret = btrfs_wait_on_delayed_iputs(fs_info); + if (ret) + return ret; goto again; } else { btrfs_end_transaction(trans); @@ -4829,9 +4830,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * pinned space, so make sure we run the iputs before we do our pinned * bytes check below. */ - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); btrfs_run_delayed_iputs(fs_info); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + wait_event(fs_info->delayed_iputs_wait, + atomic_read(&fs_info->nr_delayed_iputs) == 0); trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 0a1671fb03bf..ab8242b10601 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -3319,6 +3319,7 @@ void btrfs_add_delayed_iput(struct inode *inode) if (atomic_add_unless(&inode->i_count, -1, 1)) return; + atomic_inc(&fs_info->nr_delayed_iputs); spin_lock(&fs_info->delayed_iput_lock); ASSERT(list_empty(&binode->delayed_iput)); list_add_tail(&binode->delayed_iput, &fs_info->delayed_iputs); @@ -3338,11 +3339,31 @@ void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info) list_del_init(&inode->delayed_iput); spin_unlock(&fs_info->delayed_iput_lock); iput(&inode->vfs_inode); + if (atomic_dec_and_test(&fs_info->nr_delayed_iputs)) + wake_up(&fs_info->delayed_iputs_wait); spin_lock(&fs_info->delayed_iput_lock); } spin_unlock(&fs_info->delayed_iput_lock); } +/** + * btrfs_wait_on_delayed_iputs - wait on the delayed iputs to be done running + * @fs_info - the fs_info for this fs + * @return - EINTR if we were killed, 0 if nothing's pending + * + * This will wait on any delayed iputs that are currently running with KILLABLE + * set. Once they are all done running we will return, unless we are killed in + * which case we return EINTR. + */ +int btrfs_wait_on_delayed_iputs(struct btrfs_fs_info *fs_info) +{ + int ret = wait_event_killable(fs_info->delayed_iputs_wait, + atomic_read(&fs_info->nr_delayed_iputs) == 0); + if (ret) + return -EINTR; + return 0; +} + /* * This creates an orphan entry for the given inode in case something goes wrong * in the middle of an unlink. From patchwork Fri Sep 28 11:18:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619747 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A08C56CB for ; Fri, 28 Sep 2018 11:19:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 985332B061 for ; Fri, 28 Sep 2018 11:19:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C7B62B0F4; Fri, 28 Sep 2018 11:19:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C0702B061 for ; Fri, 28 Sep 2018 11:19:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729557AbeI1Rm5 (ORCPT ); Fri, 28 Sep 2018 13:42:57 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:33362 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1Rm4 (ORCPT ); Fri, 28 Sep 2018 13:42:56 -0400 Received: by mail-qk1-f196.google.com with SMTP id 84-v6so3595160qkf.0 for ; Fri, 28 Sep 2018 04:19:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ClFA1IppPtkTYcFpzPGI9iJqwl/ciCdtCKaL1iQ0hJo=; b=wJ3twJIV0AP0IzKg432zczGQI0GzmMg85cwYcv5GMRggM7MTnaon7s0lSQF5uptocI qa1wRlpCgbz0Dy9T+i/O+G/HdFhv/nPrJy13DN6waTyWlq2wR8UD06meZTT+4GQ6oo3g u3juzb3ikQoNHlyVksYQPoXbsxapVM69JLDvQzRjWrma2TctR039x4QMPUcfm6nxcZu/ cteJk9hGZDn985ScPHAyfQNh8Ckjrx939yiwQbCTjt0enJEkEVdeb//rE3E6U7hESiil 9kkQ90zJSe3+ITqjndUYL3YcpPW6URC5iXMbaCrHVF/y/Oe1KGI9Wuc5o3kGMqS+r5Yk 6tgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ClFA1IppPtkTYcFpzPGI9iJqwl/ciCdtCKaL1iQ0hJo=; b=CJSSiyB+oZ+CloWHG8rgRc2cZ6JaFf9noBX39qsGyp9EVkcnedpfrbabxdum8HKa11 PGDRYZUcDwNSJzygwvursiu3f6/9a6JQnCTv98xUqY5PrMybd0gbPIxJxMYnrVkj1vZC TrrqhHb1sgs34/u0nMEERf1BVPIAiAX61BAldExNsMBya82S1ts/41Y0XOv9MmTWgJfL ZYdywB4YDmSs3pl5UrDvZe3FKRTiTnqi2fHFwiopg6zoBqLauj/Ren+RpKB47fV3e7BU ahrXxyNa0an2RyTPyXKzl8z2Vv3cYc60O09/298CmHwhe5Pb5Tlfmcws77tGZXzndUtW b0VA== X-Gm-Message-State: ABuFfohCe2qxDcfgFf91JFD8lwZXkITmboCtJHOQjo9Izg/n7HEmEk2w UfZY+HcZhr5JfElG/nIctvlGUVQXxww= X-Google-Smtp-Source: ACcGV639ynIomDL2c2foqsVHBS8UQH6Z8a7RJWySpOD76xQMO3r7B3v2urDbOGzHYSWGmbGmOS9slw== X-Received: by 2002:a37:5e82:: with SMTP id s124-v6mr11386086qkb.313.1538133578821; Fri, 28 Sep 2018 04:19:38 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g14-v6sm3179563qtc.66.2018.09.28.04.19.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:37 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 40/42] btrfs: drop min_size from evict_refill_and_join Date: Fri, 28 Sep 2018 07:18:19 -0400 Message-Id: <20180928111821.24376-41-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't need it, rsv->size is set once and never changes throughout its lifetime, so just use that for the reserve size. Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/inode.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ab8242b10601..dbcca915e681 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5339,8 +5339,7 @@ static void evict_inode_truncate_pages(struct inode *inode) } static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, - struct btrfs_block_rsv *rsv, - u64 min_size) + struct btrfs_block_rsv *rsv) { struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; @@ -5350,7 +5349,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, struct btrfs_trans_handle *trans; int ret; - ret = btrfs_block_rsv_refill(root, rsv, min_size, + ret = btrfs_block_rsv_refill(root, rsv, rsv->size, BTRFS_RESERVE_FLUSH_LIMIT); if (ret && ++failures > 2) { @@ -5368,7 +5367,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, * it. */ if (!btrfs_check_space_for_delayed_refs(fs_info) && - !btrfs_block_rsv_migrate(global_rsv, rsv, min_size, 0)) + !btrfs_block_rsv_migrate(global_rsv, rsv, rsv->size, 0)) return trans; /* If not, commit and try again. */ @@ -5384,7 +5383,6 @@ void btrfs_evict_inode(struct inode *inode) struct btrfs_trans_handle *trans; struct btrfs_root *root = BTRFS_I(inode)->root; struct btrfs_block_rsv *rsv; - u64 min_size; int ret; trace_btrfs_inode_evict(inode); @@ -5394,8 +5392,6 @@ void btrfs_evict_inode(struct inode *inode) return; } - min_size = btrfs_calc_trunc_metadata_size(fs_info, 1); - evict_inode_truncate_pages(inode); if (inode->i_nlink && @@ -5428,13 +5424,13 @@ void btrfs_evict_inode(struct inode *inode) rsv = btrfs_alloc_block_rsv(fs_info, BTRFS_BLOCK_RSV_TEMP); if (!rsv) goto no_delete; - rsv->size = min_size; + rsv->size = btrfs_calc_trunc_metadata_size(fs_info, 1); rsv->failfast = 1; btrfs_i_size_write(BTRFS_I(inode), 0); while (1) { - trans = evict_refill_and_join(root, rsv, min_size); + trans = evict_refill_and_join(root, rsv); if (IS_ERR(trans)) goto free_rsv; @@ -5459,7 +5455,7 @@ void btrfs_evict_inode(struct inode *inode) * If it turns out that we are dropping too many of these, we might want * to add a mechanism for retrying these after a commit. */ - trans = evict_refill_and_join(root, rsv, min_size); + trans = evict_refill_and_join(root, rsv); if (!IS_ERR(trans)) { trans->block_rsv = rsv; btrfs_orphan_del(trans, BTRFS_I(inode)); From patchwork Fri Sep 28 11:18:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619749 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A766115A7 for ; Fri, 28 Sep 2018 11:19:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9EB742B061 for ; Fri, 28 Sep 2018 11:19:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 930A72B0F4; Fri, 28 Sep 2018 11:19:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 362A32B061 for ; Fri, 28 Sep 2018 11:19:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729560AbeI1Rm7 (ORCPT ); Fri, 28 Sep 2018 13:42:59 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:37326 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1Rm7 (ORCPT ); Fri, 28 Sep 2018 13:42:59 -0400 Received: by mail-qk1-f195.google.com with SMTP id c13-v6so3571414qkm.4 for ; Fri, 28 Sep 2018 04:19:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=am1ak1Lg8slURG/v1F1bdWqQUehd7FnCdJ9TMX1sZm0=; b=dL0MrWJAoyRdcRMr/CFVu3GZAnY1JF54EeD8nh6VuiQYQta3wkIKspgynxc8aqYO5q 39lMmcOjMN0w8b64QaEnenT9TATol8PDC2eZdRWt1LZd8Ms9uOoaqrILqi+loSlpC7a2 L9aXY17bXE97uHZJ524vPYNc/RdP4OT2HQR4EWpJpcBq60uNFYuDfvvsH5Ec/kdjYzSy 3AnuXMvnV6jJpHKqKfzIfohQPWnjHam7POOV/Fs5k2OhxsUZU33vDHL0IH2ZgO2EMSZc NymFWDUGOhGSwCoFE9yf+lK2XGB6PHllv0mR9zRApMyuZHsJOKOfg8aYHFHoSmKJnYAe th7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=am1ak1Lg8slURG/v1F1bdWqQUehd7FnCdJ9TMX1sZm0=; b=iU0lJLsk7csIZmNZ36I3ebjMTahvgaZ4KBR6KReq31Vc4PkYQHoYFWeVrKVHlJSb14 urlqZF3ciuDSPdhHQp9cIc9+kguKhSi8pLFvXV5Mu4wUcW4PtWylHPt9Iug8wwtGzl78 m8KwM8WJ63/h82/5Ilahon14Miv3YCYh1D5puHKWkSPib4SYI3pKyhERY1zBgB8PBCGj oCSqnWk7IC5Cujmp1qaRx8Fbi0/SPhofVul1he9nUDf6ouVMZqwNPZzSK2yc/yMvG9sN 7RqJrP2uRaTd58v9ruQSnUSkFPj8zuEIC3aV7R4OXo/o6ubhGMffaWIwojH23f+JwSXF MNFA== X-Gm-Message-State: ABuFfogx9qrYPzW9heao1Q7oBJvaoFUsTSBHU7cofKjym/hgezNAq7T9 S1ZftC0qddk7d7s02/lzixmpiA== X-Google-Smtp-Source: ACcGV62tLufoddP1P+jaP3Syn3Ofdpo7wTmhNeeFFHp7RvNMPQQAleKXc13keb9Cxue5H2ikto+mFg== X-Received: by 2002:a37:5c81:: with SMTP id q123-v6mr11212405qkb.8.1538133580737; Fri, 28 Sep 2018 04:19:40 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c85-v6sm2439672qke.89.2018.09.28.04.19.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:39 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 41/42] btrfs: reserve extra space during evict() Date: Fri, 28 Sep 2018 07:18:20 -0400 Message-Id: <20180928111821.24376-42-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We could generate a lot of delayed refs in evict but never have any left over space from our block rsv to make up for that fact. So reserve some extra space and give it to the transaction so it can be used to refill the delayed refs rsv every loop through the truncate path. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index dbcca915e681..9f7da5e3c741 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5343,13 +5343,15 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, { struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + u64 delayed_refs_extra = btrfs_calc_trans_metadata_size(fs_info, 1); int failures = 0; for (;;) { struct btrfs_trans_handle *trans; int ret; - ret = btrfs_block_rsv_refill(root, rsv, rsv->size, + ret = btrfs_block_rsv_refill(root, rsv, + rsv->size + delayed_refs_extra, BTRFS_RESERVE_FLUSH_LIMIT); if (ret && ++failures > 2) { @@ -5358,9 +5360,28 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, return ERR_PTR(-ENOSPC); } + /* + * Evict can generate a large amount of delayed refs without + * having a way to add space back since we exhaust our temporary + * block rsv. We aren't allowed to do FLUSH_ALL in this case + * because we could deadlock with so many things in the flushing + * code, so we have to try and hold some extra space to + * compensate for our delayed ref generation. If we can't get + * that space then we need see if we can steal our minimum from + * the global reserve. We will be ratelimited by the amount of + * space we have for the delayed refs rsv, so we'll end up + * committing and trying again. + */ trans = btrfs_join_transaction(root); - if (IS_ERR(trans) || !ret) + if (IS_ERR(trans) || !ret) { + if (!IS_ERR(trans)) { + trans->block_rsv = &fs_info->trans_block_rsv; + trans->bytes_reserved = delayed_refs_extra; + btrfs_block_rsv_migrate(rsv, trans->block_rsv, + delayed_refs_extra, 1); + } return trans; + } /* * Try to steal from the global reserve if there is space for From patchwork Fri Sep 28 11:18:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10619751 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2362C15A7 for ; Fri, 28 Sep 2018 11:19:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1AE002B061 for ; Fri, 28 Sep 2018 11:19:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0F38D2B0F4; Fri, 28 Sep 2018 11:19:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4B5A2B061 for ; Fri, 28 Sep 2018 11:19:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729563AbeI1RnA (ORCPT ); Fri, 28 Sep 2018 13:43:00 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:43667 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbeI1RnA (ORCPT ); Fri, 28 Sep 2018 13:43:00 -0400 Received: by mail-qt1-f194.google.com with SMTP id q41-v6so1331531qtq.10 for ; Fri, 28 Sep 2018 04:19:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=sV09l7dO3avmvRHRl1FabUbd/mHTvojy+8waApybFR4=; b=JnIsHloaL5IVXTRWMo8NpVnqf60amNdc8hSjSwEEHS75ri0QIkp+rkyqexWBAe/RfJ kvSVLN/Zrl36FSoxpy1K3SqIzzuWwqyAeRvvb6hLFrPKbFrl7zob9iSNuzkSZkEX+z4X GSfuCbs579Q1z5qtq8dhKrHy2YFOBXKDphYH9238aTKboF5qdGwCnBvJnOJuGYPwTsWt 5JudYLeujPloD+12ysWqnC/NKqLsm5+YQKLt2dA5W6MyoBYk0prDI10AUYb5X5vL7neU vkITs0jJH3M/Ki/5zoYaz9DYLDPrT42P27MLWsnchOX6IGNBih6mzNfgD1pTNzYAzerF RC0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=sV09l7dO3avmvRHRl1FabUbd/mHTvojy+8waApybFR4=; b=pSOJFCVXVkCrS3+OmVQ7LP3t8VumLPPms8PEerz0b9zkcXUR6s17B7SgOL22/NQ9a6 kivUs4pOM0OV3aMeu82TgTjvIdciypwjhyFUMJevtv2jVdwFvq+nPPgmQkpTWw2fQ0fz tFuJat3JxaVjboV70GFDhXdhAv2r9AcDqtDaCuf+suBD6KLAAWkDruqrWLUvuxzPwE7s d2hFHM0aJX00oJgfXk3r+MFzbu+kDJid84Rtn3GWH+dNRHCe9nsMWikCB8C+IqigRgO5 eORqJy2xQBZ6YcIJuVm5K6jGmm/IVmWQg91NzrC9+sFYC5MRt+lDY2vkkb46cnyIQILY HoDw== X-Gm-Message-State: ABuFfojDQB6wdCps3jTosxwpv65eDjJhBqi9efGKtit74oY5i1iqMk3W JRa7MPAcyULVvf6UcIbXGtO6WnSVVas= X-Google-Smtp-Source: ACcGV61uS6FjiEU0OB+t1/dqEqENK7Jrfm2U+6hY+wq6XzolYGZ78qWgUo3R25EKZV4V0xGYMYuGPA== X-Received: by 2002:ad4:404a:: with SMTP id r10-v6mr11556258qvp.246.1538133582592; Fri, 28 Sep 2018 04:19:42 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v18-v6sm3180056qtj.94.2018.09.28.04.19.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Sep 2018 04:19:41 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 42/42] btrfs: don't run delayed_iputs in commit Date: Fri, 28 Sep 2018 07:18:21 -0400 Message-Id: <20180928111821.24376-43-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180928111821.24376-1-josef@toxicpanda.com> References: <20180928111821.24376-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This could result in a really bad case where we do something like evict evict_refill_and_join btrfs_commit_transaction btrfs_run_delayed_iputs evict evict_refill_and_join btrfs_commit_transaction ... forever We have plenty of other places where we run delayed iputs that are much safer, let those do the work. Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 9168efaca37e..c91dc36fccae 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -2265,15 +2265,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) kmem_cache_free(btrfs_trans_handle_cachep, trans); - /* - * If fs has been frozen, we can not handle delayed iputs, otherwise - * it'll result in deadlock about SB_FREEZE_FS. - */ - if (current != fs_info->transaction_kthread && - current != fs_info->cleaner_kthread && - !test_bit(BTRFS_FS_FROZEN, &fs_info->flags)) - btrfs_run_delayed_iputs(fs_info); - return ret; scrub_continue: