From patchwork Thu Oct 11 19:53:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B5CF1679F for ; Thu, 11 Oct 2018 19:54:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2EA72C09B for ; Thu, 11 Oct 2018 19:54:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 972812C0A2; Thu, 11 Oct 2018 19:54:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 357222C09B for ; Thu, 11 Oct 2018 19:54:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726775AbeJLDXW (ORCPT ); Thu, 11 Oct 2018 23:23:22 -0400 Received: from mail-qt1-f170.google.com ([209.85.160.170]:37818 "EHLO mail-qt1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726754AbeJLDXW (ORCPT ); Thu, 11 Oct 2018 23:23:22 -0400 Received: by mail-qt1-f170.google.com with SMTP id d14-v6so11349818qto.4 for ; Thu, 11 Oct 2018 12:54:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=J1nvikXF93vBk79skKC0f+gf8yb5S1jfySvhFcqnzF4=; b=A03nV5WIDGvK/biQc4opVP5LC4Cb7l/OY0JOVLO7M/6nx99DtP8uj4ZLcauLw3ACHp jGrxUafG+6AuO+nNXFcqOHqGyBvYC3ZHj/6jmvDm4MKciK1vhY5gIXxg17jArfzCUicK z1tP/deP//UOsVblzzU0UuKyaxQtA9Tfa1w1ViQyD3YgJe5aXYdv1T7+lq4zqBYMlDq1 DE4CcNNNQdNB1hbilW9dGGH/RpMNRfX58GSckOARewdd0xvu6KZKhpwgPsivo3YoyqCF XGsbygr/HUDifoATcAHcaOwEGg/4Y8yiEb45tN7mVFz2UzGa1vLzZ28WwJGsmlVdktji B3Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=J1nvikXF93vBk79skKC0f+gf8yb5S1jfySvhFcqnzF4=; b=RqrwQL+ES4jwQDyKSPxbhEKBDwVQRUEgLOvtSIoAtpf484gGZGJYY2ZKXtVI3CNgz2 HNz0C1Dde2v9e9vRGe9OxLx+awKPRl25wRNs42Ojz1k3RkQPjd/cdCDwPNgdxwkU7ynE UdGp3ZGphZLQ6T1ROy2AjqrsDCeMQGFd05fE5FoAxs9m/Hwq98K0+4d82hKUZ0IV26X3 V3UDPnw692a7aW5c+uE3/QiRdNchg/jZIfxaCKjftjx/DcuMitE2yxOXVe/UrMC/Rahv 3Ec2/h3HYrMVfgWBJnc1WPRmaETjE/l/uFzgofblaF/l4UgM4H2+sYpmq+KepFw816IC Uzbw== X-Gm-Message-State: ABuFfojyPj6qD2e3oLqTXqnnv7RKru54wBGB/eqQH46X9ygs3WmP8+wQ pUb9ku2qt88tc14jUyapzkqiXHzfybw= X-Google-Smtp-Source: ACcGV60msZpBkQXKhLWyp12rOJgN6NCSLK5eCUDbb4OgpwweJkkPIIFXYp0FxdMV+7eITPkzxisyJg== X-Received: by 2002:a0c:95e6:: with SMTP id t35mr3084144qvt.163.1539287675614; Thu, 11 Oct 2018 12:54:35 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id r20-v6sm15593994qkl.12.2018.10.11.12.54.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:34 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 01/42] btrfs: add btrfs_delete_ref_head helper Date: Thu, 11 Oct 2018 15:53:50 -0400 Message-Id: <20181011195431.3441-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We do this dance in cleanup_ref_head and check_ref_cleanup, unify it into a helper and cleanup the calling functions. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/delayed-ref.c | 14 ++++++++++++++ fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/extent-tree.c | 22 +++------------------- 3 files changed, 19 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 62ff545ba1f7..3a9e4ac21794 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -393,6 +393,20 @@ btrfs_select_ref_head(struct btrfs_trans_handle *trans) return head; } +void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head) +{ + lockdep_assert_held(&delayed_refs->lock); + lockdep_assert_held(&head->lock); + + rb_erase(&head->href_node, &delayed_refs->href_root); + RB_CLEAR_NODE(&head->href_node); + atomic_dec(&delayed_refs->num_entries); + delayed_refs->num_heads--; + if (head->processing == 0) + delayed_refs->num_heads_ready--; +} + /* * Helper to insert the ref_node to the tail or merge with tail. * diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index d9f2a4ebd5db..7769177b489e 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -261,7 +261,8 @@ static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) { mutex_unlock(&head->mutex); } - +void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head); struct btrfs_delayed_ref_head * btrfs_select_ref_head(struct btrfs_trans_handle *trans); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f77226d8020a..d24a0de4a2e7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2492,12 +2492,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, spin_unlock(&delayed_refs->lock); return 1; } - delayed_refs->num_heads--; - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); + btrfs_delete_ref_head(delayed_refs, head); spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); - atomic_dec(&delayed_refs->num_entries); trace_run_delayed_ref_head(fs_info, head, 0); @@ -6984,22 +6981,9 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (!mutex_trylock(&head->mutex)) goto out; - /* - * at this point we have a head with no other entries. Go - * ahead and process it. - */ - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); - atomic_dec(&delayed_refs->num_entries); - - /* - * we don't take a ref on the node because we're removing it from the - * tree, so we just steal the ref the tree was holding. - */ - delayed_refs->num_heads--; - if (head->processing == 0) - delayed_refs->num_heads_ready--; + btrfs_delete_ref_head(delayed_refs, head); head->processing = 0; + spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); From patchwork Thu Oct 11 19:53:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637291 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E115157A for ; Thu, 11 Oct 2018 19:54:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BB682C09B for ; Thu, 11 Oct 2018 19:54:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 800002C0A2; Thu, 11 Oct 2018 19:54:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B1302C09B for ; Thu, 11 Oct 2018 19:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726789AbeJLDXY (ORCPT ); Thu, 11 Oct 2018 23:23:24 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:41260 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726754AbeJLDXX (ORCPT ); Thu, 11 Oct 2018 23:23:23 -0400 Received: by mail-qk1-f194.google.com with SMTP id 23-v6so6256421qkh.8 for ; Thu, 11 Oct 2018 12:54:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/1YdvlrrEVlQm7E7WHAhXkpqC+TBu9F+Y13pNbn/ob8=; b=lZe+Djg158/nt16yQMD+e+o+2rIoTZQQieIhDC+EQixoQtoHHaixmp3/sZjAlG1jTz WycVJCfBl3o+SMhoZM2B8+pTe9IGElaaXdKQO4ST9z7pRtoM71G6XpcBcZ52tcgC6u/q ibdNE1M2azMsDlTjn3sVi7ZXtu+OYt0S6RmjIPJivaChEC+yoctVA6BbFvmrohUPYzYT 9m869Ytkk3W9na0i6oLa5qMpBbXe/n58igRgnCQ0mGWAb1knaRWLEiMB2WedO3iu5ZMI AJ6/3NHV1sGc5Ph9xBAxts6vGfo7SmTseZYjirQzMFDOR8FJhD0uDR1uScOSv9nQdPd/ +VhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/1YdvlrrEVlQm7E7WHAhXkpqC+TBu9F+Y13pNbn/ob8=; b=MrF1IaX0HIHxRX9yATCrD7Qwx1Sr/NlnfODTORrOoT7YpW41YfkaP3Xmp5DtLwvclI bYuMVHK6PWawU4bk2oH7Psz9+wgsULZdNOGlqtCy8ZewSKhTpNI0Ipg5s6YrUCg+ev09 EoXEMXR8Kts1Mc/QtVAJi/AyC6YF67Cz0AHhFkfPhggr97+RU428wHnOUslHvgiltKWa xZlPUxdClkjfeo9cYIqJqJf4T9/G7FOxqTgpE3h2GJ0Be51Yi2TQwHG9CmvxcNOXfIqA CaC2W8ef7UyijwmWqyXM0nTQ+wqEskK+san8F/ob4p2LIkPq2tYO23T/o2qtH1nvlgYU B2Dw== X-Gm-Message-State: ABuFfoiNX82N9yJBfVlfgOTQhZCQYVDKhDO5Zh9k6og3xogYJS9m35Jl OzdgVycHRWNSGLzObzHfshsHNA== X-Google-Smtp-Source: ACcGV60gRH0OVRGFbKAmD4+0dQoDCX9gjgt+Jidi6kf+eP+4l0YE7KvP6kP2Hz07JaUWFp+3AAwtVA== X-Received: by 2002:a37:8b46:: with SMTP id n67-v6mr2958885qkd.262.1539287677453; Thu, 11 Oct 2018 12:54:37 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y7-v6sm11909357qkf.38.2018.10.11.12.54.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:36 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 02/42] btrfs: add cleanup_ref_head_accounting helper Date: Thu, 11 Oct 2018 15:53:51 -0400 Message-Id: <20181011195431.3441-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We were missing some quota cleanups in check_ref_cleanup, so break the ref head accounting cleanup into a helper and call that from both check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that we don't screw up accounting in the future for other things that we add. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 67 +++++++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index d24a0de4a2e7..a44d55e36e11 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2461,6 +2461,41 @@ static int cleanup_extent_op(struct btrfs_trans_handle *trans, return ret ? ret : 1; } +static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + + if (head->total_ref_mod < 0) { + struct btrfs_space_info *space_info; + u64 flags; + + if (head->is_data) + flags = BTRFS_BLOCK_GROUP_DATA; + else if (head->is_system) + flags = BTRFS_BLOCK_GROUP_SYSTEM; + else + flags = BTRFS_BLOCK_GROUP_METADATA; + space_info = __find_space_info(fs_info, flags); + ASSERT(space_info); + percpu_counter_add_batch(&space_info->total_bytes_pinned, + -head->num_bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH); + + if (head->is_data) { + spin_lock(&delayed_refs->lock); + delayed_refs->pending_csums -= head->num_bytes; + spin_unlock(&delayed_refs->lock); + } + } + + /* Also free its reserved qgroup space */ + btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, + head->qgroup_reserved); +} + static int cleanup_ref_head(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head *head) { @@ -2496,31 +2531,6 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); - trace_run_delayed_ref_head(fs_info, head, 0); - - if (head->total_ref_mod < 0) { - struct btrfs_space_info *space_info; - u64 flags; - - if (head->is_data) - flags = BTRFS_BLOCK_GROUP_DATA; - else if (head->is_system) - flags = BTRFS_BLOCK_GROUP_SYSTEM; - else - flags = BTRFS_BLOCK_GROUP_METADATA; - space_info = __find_space_info(fs_info, flags); - ASSERT(space_info); - percpu_counter_add_batch(&space_info->total_bytes_pinned, - -head->num_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - - if (head->is_data) { - spin_lock(&delayed_refs->lock); - delayed_refs->pending_csums -= head->num_bytes; - spin_unlock(&delayed_refs->lock); - } - } - if (head->must_insert_reserved) { btrfs_pin_extent(fs_info, head->bytenr, head->num_bytes, 1); @@ -2530,9 +2540,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } - /* Also free its reserved qgroup space */ - btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, - head->qgroup_reserved); + cleanup_ref_head_accounting(trans, head); + + trace_run_delayed_ref_head(fs_info, head, 0); btrfs_delayed_ref_unlock(head); btrfs_put_delayed_ref_head(head); return 0; @@ -6991,6 +7001,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (head->must_insert_reserved) ret = 1; + cleanup_ref_head_accounting(trans, head); mutex_unlock(&head->mutex); btrfs_put_delayed_ref_head(head); return ret; From patchwork Thu Oct 11 19:53:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637293 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AC9B112B for ; Thu, 11 Oct 2018 19:54:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 280A32C09B for ; Thu, 11 Oct 2018 19:54:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C8142C0A2; Thu, 11 Oct 2018 19:54:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B23302C09B for ; Thu, 11 Oct 2018 19:54:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726798AbeJLDXZ (ORCPT ); Thu, 11 Oct 2018 23:23:25 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:44912 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726754AbeJLDXZ (ORCPT ); Thu, 11 Oct 2018 23:23:25 -0400 Received: by mail-qt1-f195.google.com with SMTP id c56-v6so11326267qtd.11 for ; Thu, 11 Oct 2018 12:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rHTVMCxRLsTEVUkODyGqkPIz399o/p3gyRkwe0nkz4U=; b=zA32ceDWv1Ij4Z3iHARbZw4p6ITa3KabVq+UWI3qqq4ONN+lr1FDt3tYppYBGZdEiO riW9HxX/81Of5AKKIFeCfLVK5Rpj8Ws0ttwad58OMnPGRM7hOwx2ho/ArS0CPReJtEH0 EFSHedOugCI/Hw8K80JqnIS4s1YA2ZzSgcrxq32MMj8lPT3g7YYMqZIVDJzvWo/pdyqB Sla1X+fnY7tEWLsHQXxJqQ3QuB9IgC51ce8xyahJsz4KRIHp+5QDLfJT7LH0CL069IH0 Lelsusi/TNqvmdDoVpKzZEWtLKV0vtZoReRNtWYgV+oMVGmFopMFi4Hr5TeidqwICogq 552A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rHTVMCxRLsTEVUkODyGqkPIz399o/p3gyRkwe0nkz4U=; b=YVM+zZ4WlUaNJuQ/Hbk2hTVvhEtw1nd1luGu15MoOmn4grOTRhhVO08HGtH2rlglty 4/Kvr4rlGvpAHN2qHowMm5vhg4fJkygLAOo359Oet/aurtHAWxodyPR/11sHXNekyJkw JhmyoNtD+6LaLEyM+ihu5wk1o7FLwUfoLR7Mo5GG24lJTBLew7XzhPmMy4l2XVkKnOBY 0Ru+3lBZzpFbIMObREnbTLj7Mok/L8JW4vNOH9pZp6wHuelV+CfKbV170+mSWXSU+ms1 xPmjIBrZkMzlJypJ9UZBkj2BSDr/hFouueTkga+sWiqnyLxoSV7L+TbBPAXKBcrndwlr Buhg== X-Gm-Message-State: ABuFfogLdVSN6EBuPlQNnGX+7Jg82TotBbKgjvKgMGiMcjpiwLWuCR3c z2+gE71yui4fWH3r7hSkI0BePw== X-Google-Smtp-Source: ACcGV61G1B1C9wnJvamlp0IeCN2ZGXxmomXEd4tQochh+kzwlAcED4DlkDGktNd8LrRpHxJHUfo14A== X-Received: by 2002:a0c:8c86:: with SMTP id p6-v6mr3036751qvb.246.1539287679386; Thu, 11 Oct 2018 12:54:39 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id j57-v6sm16958692qtj.96.2018.10.11.12.54.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:38 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 03/42] btrfs: cleanup extent_op handling Date: Thu, 11 Oct 2018 15:53:52 -0400 Message-Id: <20181011195431.3441-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik The cleanup_extent_op function actually would run the extent_op if it needed running, which made the name sort of a misnomer. Change it to run_and_cleanup_extent_op, and move the actual cleanup work to cleanup_extent_op so it can be used by check_ref_cleanup() in order to unify the extent op handling. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index a44d55e36e11..98f36dfeccb0 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2442,19 +2442,33 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_ref btrfs_delayed_ref_unlock(head); } -static int cleanup_extent_op(struct btrfs_trans_handle *trans, - struct btrfs_delayed_ref_head *head) +static struct btrfs_delayed_extent_op * +cleanup_extent_op(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) { struct btrfs_delayed_extent_op *extent_op = head->extent_op; - int ret; if (!extent_op) - return 0; - head->extent_op = NULL; + return NULL; + if (head->must_insert_reserved) { + head->extent_op = NULL; btrfs_free_delayed_extent_op(extent_op); - return 0; + return NULL; } + return extent_op; +} + +static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) +{ + struct btrfs_delayed_extent_op *extent_op = + cleanup_extent_op(trans, head); + int ret; + + if (!extent_op) + return 0; + head->extent_op = NULL; spin_unlock(&head->lock); ret = run_delayed_extent_op(trans, head, extent_op); btrfs_free_delayed_extent_op(extent_op); @@ -2506,7 +2520,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, delayed_refs = &trans->transaction->delayed_refs; - ret = cleanup_extent_op(trans, head); + ret = run_and_cleanup_extent_op(trans, head); if (ret < 0) { unselect_delayed_ref_head(delayed_refs, head); btrfs_debug(fs_info, "run_delayed_extent_op returned %d", ret); @@ -6977,12 +6991,8 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (!RB_EMPTY_ROOT(&head->ref_tree)) goto out; - if (head->extent_op) { - if (!head->must_insert_reserved) - goto out; - btrfs_free_delayed_extent_op(head->extent_op); - head->extent_op = NULL; - } + if (cleanup_extent_op(trans, head) != NULL) + goto out; /* * waiting for the lock here would deadlock. If someone else has it From patchwork Thu Oct 11 19:53:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637295 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15E81112B for ; Thu, 11 Oct 2018 19:54:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 023FE2C09B for ; Thu, 11 Oct 2018 19:54:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EA6242C0A2; Thu, 11 Oct 2018 19:54:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9EE592C09B for ; Thu, 11 Oct 2018 19:54:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726822AbeJLDX1 (ORCPT ); Thu, 11 Oct 2018 23:23:27 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39936 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726754AbeJLDX1 (ORCPT ); Thu, 11 Oct 2018 23:23:27 -0400 Received: by mail-qt1-f193.google.com with SMTP id b4-v6so11329156qtc.7 for ; Thu, 11 Oct 2018 12:54:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E8JF94eruCVpfwCQjNtPgc8hW7YmBCNIla9LHGUTiOs=; b=eTZla7w7cFPTBdvpqJGjVk+CfGa5cQveyXJb85iYTh4cAWBpy3XYuM1iQtAd8F8Jnp /NbZBeKjPwMn+23RtNAi868PAjVIbFiyj0cmCIIWWOjaOM6ZTKUuTPPWGVTqaH77qDms In+sAO89fQ4J/WeaUvC9QO9QO55HlDAWJYHtCN00bvebbKa9Rgy+ANyhNTkMjE0weP/5 J/vnCAtgNysJI0i2fZ27j4VeMpbGkgmK+RjtSLAF9NCHfwaqSOj/kShZzTEjPoDwOkMy Sgs+P0BunbpSRieq32RxFOuNDlY0QejwcmVXWfWATjjxyweeThl44ANBvEesqFd9ZWfM 1AsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E8JF94eruCVpfwCQjNtPgc8hW7YmBCNIla9LHGUTiOs=; b=NeVOLSFUEMbEK1lK3dx2DVSoWDqf9LjEnEWWXOW00QZhnCbs1ykiMVwxLIecmaKqh7 Y+DHdQaFPtMFK/dz5gvxn5zlyUPAojIUxyBi1t5ibwOVDoK2OO7IAm4gULx3ySUjVZqG bYkDKYMUPCKdx7z3HA0pDHUM4GuE5O5yv9D+qMvVRQfr2yo92u4xXo7P3KQ/IX5ox0yZ 7CC3QZ4ZLDrJWVdVV76addjZyJF1ZByAOjfOxPO1bW2m56rDv+bPA5Xr1MM3wsLU7eVN lrgJu5FGbKIRcv/QtKOY3frmscR1XJTiWGhOnof/oEQs2TcyQyEdPQ2WkIMMNKOibWAD IaAw== X-Gm-Message-State: ABuFfohvwlATPhp1D9sT7jYY6Em2ZbpURJjDjUofNqf6FVXiincBpCqP QiXzfkhH5fdZyBDH5EbA207I8Td0Hf0= X-Google-Smtp-Source: ACcGV62KhV7XgOag9+YpFGfQV9YutTPSarvJK+crLuarQSmfZKI+P9kJj9jijKeQnfaWp5VTTOCklA== X-Received: by 2002:ac8:6a0a:: with SMTP id t10-v6mr2948726qtr.376.1539287681211; Thu, 11 Oct 2018 12:54:41 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id t29-v6sm15635933qkt.44.2018.10.11.12.54.40 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:40 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 04/42] btrfs: only track ref_heads in delayed_ref_updates Date: Thu, 11 Oct 2018 15:53:53 -0400 Message-Id: <20181011195431.3441-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We use this number to figure out how many delayed refs to run, but __btrfs_run_delayed_refs really only checks every time we need a new delayed ref head, so we always run at least one ref head completely no matter what the number of items on it. Fix the accounting to only be adjusted when we add/remove a ref head. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 3a9e4ac21794..27f7dd4e3d52 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -234,8 +234,6 @@ static inline void drop_delayed_ref(struct btrfs_trans_handle *trans, ref->in_tree = 0; btrfs_put_delayed_ref(ref); atomic_dec(&delayed_refs->num_entries); - if (trans->delayed_ref_updates) - trans->delayed_ref_updates--; } static bool merge_ref(struct btrfs_trans_handle *trans, @@ -460,7 +458,6 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans, if (ref->action == BTRFS_ADD_DELAYED_REF) list_add_tail(&ref->add_list, &href->ref_add_list); atomic_inc(&root->num_entries); - trans->delayed_ref_updates++; spin_unlock(&href->lock); return ret; } From patchwork Thu Oct 11 19:53:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637297 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32A4C112B for ; Thu, 11 Oct 2018 19:54:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 203CF2C09B for ; Thu, 11 Oct 2018 19:54:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 14CE72C0A2; Thu, 11 Oct 2018 19:54:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5A052C09B for ; Thu, 11 Oct 2018 19:54:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726830AbeJLDX3 (ORCPT ); Thu, 11 Oct 2018 23:23:29 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:41907 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726754AbeJLDX3 (ORCPT ); Thu, 11 Oct 2018 23:23:29 -0400 Received: by mail-qt1-f195.google.com with SMTP id l41-v6so11340767qtl.8 for ; Thu, 11 Oct 2018 12:54:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=+yz3n6lATRx6h1PNpaQrCmKxcZVsI4n1UZsZzxSFbKc=; b=xxbRFGkUyKXht8tR2XYFQb/KDSgNVi61b7NNQvgItAylgvO8JbopaYhkmtgiWDvSK8 cJuDig3BLhK3KQQ5gFRynF/jQY7YEvN4gDkkGXMtyYdo5LfjmVj7uOya7L+F693Nfqsl KBlVMfcNkH5n0jBInbMTxxqO+08QocpbX6mHxZe/crvdGxNoPkcHLB0oHydu2w36tlkA BoFES1nX9TD/2f3Esw5iQ28CGh4skE6V9pugLOy1daLK9wjbDu5vklOND6VMYSFp7vfM 2r+4uxStM6OnNyE/i3NCzcq3PNiovzYE5gbNVGxWk4YYib8HS75BsatUA75XQYiwkTrT JkfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+yz3n6lATRx6h1PNpaQrCmKxcZVsI4n1UZsZzxSFbKc=; b=MRV2B8mPnP9Em5IjQlxVFEHlj1OgukMQaCSqQgY6wJdsqyP7Zj0vpSQj/zWSGydciz SdxhPFkf2dF6ZbMwNjI2ipnqpVMtxRLt7Kpd7JZLmvriAE4kKz/n/2oiRglHwTSnP+qx jklFVxzi7ztt0A31LGSrBaAd8p+H/V4lThtPWyBIkukMuNOtXOB0KDTBL7/9uNWnYqYe p2ddkDlSlNNNzVWw06qAjIBpJW+u3KB2L1Hqr++w20bLvG++xL36HznZa1/kipP2P71S lb0euTAOsDLUiydICiL8ub11uUxAH8lYcj5ZQ/cCxI32+6xvXA3btdZWMHuTZ48DhdR+ 4FBA== X-Gm-Message-State: ABuFfojA9h60ppfDsJCY01OeDoB59Kbms8mjeGPdZ/nmJLZmz7YtCdIB 9RqHtchDq9/0AW0gi4RebQn8Lw== X-Google-Smtp-Source: ACcGV63xRggPBMfZ9q9vvcu69WcVMQOzHnLno6tuKjDOx4aVEgjhbdD3Mn4nX78Y0cimJKn2y1IHkg== X-Received: by 2002:a0c:be01:: with SMTP id k1-v6mr3064601qvg.226.1539287683088; Thu, 11 Oct 2018 12:54:43 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id l18-v6sm16619240qki.83.2018.10.11.12.54.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:42 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 05/42] btrfs: only count ref heads run in __btrfs_run_delayed_refs Date: Thu, 11 Oct 2018 15:53:54 -0400 Message-Id: <20181011195431.3441-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We pick the number of ref's to run based on the number of ref heads, and only make the decision to stop once we've processed entire ref heads, so only count the ref heads we've run and bail once we've hit the number of ref heads we wanted to process. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 98f36dfeccb0..b32bd38390dd 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2592,6 +2592,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, spin_unlock(&delayed_refs->lock); break; } + count++; /* grab the lock that says we are going to process * all the refs for this head */ @@ -2605,7 +2606,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, */ if (ret == -EAGAIN) { locked_ref = NULL; - count++; continue; } } @@ -2633,7 +2633,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unselect_delayed_ref_head(delayed_refs, locked_ref); locked_ref = NULL; cond_resched(); - count++; continue; } @@ -2651,7 +2650,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, return ret; } locked_ref = NULL; - count++; continue; } @@ -2702,7 +2700,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, } btrfs_put_delayed_ref(ref); - count++; cond_resched(); } From patchwork Thu Oct 11 19:53:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 401B9157A for ; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 268CA2C09B for ; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1AEE92C0A2; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F8042C09B for ; Thu, 11 Oct 2018 19:54:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726846AbeJLDXd (ORCPT ); Thu, 11 Oct 2018 23:23:33 -0400 Received: from mail-qk1-f171.google.com ([209.85.222.171]:40362 "EHLO mail-qk1-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726754AbeJLDXd (ORCPT ); Thu, 11 Oct 2018 23:23:33 -0400 Received: by mail-qk1-f171.google.com with SMTP id a13-v6so6268504qkc.7 for ; Thu, 11 Oct 2018 12:54:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UMAxEiV+68Nd9dMH4VurZacgpPCSyaixMmaSB9avthc=; b=DvmQo0oEmq9J+EzWzASQGAxHzGli/nqLdcecSdLwjj10vswrzGwVIOThP1RWKOw49g sHif5eHUZ68DWd2uRFcOmwxm6MSvcUv3Bx+z7FT08av4NPpKowDUu7j4uXScz/kNOxGb JTE1bzfxAFbvDwKXCh3opOvjn96JLsCIOTtPbl99betmvc0MRKujNzqKVDTwIhaZzuup KnoVa8f34xozuW7oEMJc6e9d2e9u/52K6h+xTFB5tDikbP4D/Lrg/wgc4i7lSUNgd1vw QNoFkjMruGyeSj5kZgFqjQBxbJk7Z2BAzmc1fesxo4I8XTH4c14r/HE7GqMsP0iD6l7D 6vLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UMAxEiV+68Nd9dMH4VurZacgpPCSyaixMmaSB9avthc=; b=nYzqIpCmJERP09EwrUVZCdrQMC5AAUnzKVByF+koP2qO08fWvKBct0Iu7Gwl5qpBWQ fLMZFFJGz05LBj1EZ8yKCYETtaEBcisaBhN8v2+BrJ+R56ueZCannEjG88pUWniPFxns cUXhfYFnTd5flxemvxTZfPKdC64CoG36nPKZwdZtmPkSRKuGeTzmacosZqELQET7yC8G dXAXsVFJgbX7dMuuDgi7lIFck8v2RQC3B3bKFDSJO6VfHj+K0gOA6ieqGLHsfhp+2Tkz 0lXPRz0k3rtnq5r6p282nCUiv9Y4oa4HByFJHU8qi84bJ3g1tnuY3XMEOAk4tVfgJ8pV 6Ofw== X-Gm-Message-State: ABuFfohdDS1pZ623gEYtF38yro901d4MNbG2JEyVNAljVijpfVZ8/AmY iEsebGwNnRj5pwo00vBs7+ER5b6qHs8= X-Google-Smtp-Source: ACcGV62aGPvBc9pyErtpekLV4whet166c6JQxaduy8rxmpSYhGAulR8zLijFwOrlkRQGzGMn1aiAMg== X-Received: by 2002:a37:6845:: with SMTP id d66-v6mr3081188qkc.9.1539287685184; Thu, 11 Oct 2018 12:54:45 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id q21-v6sm8069079qta.77.2018.10.11.12.54.43 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:44 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 06/42] btrfs: introduce delayed_refs_rsv Date: Thu, 11 Oct 2018 15:53:55 -0400 Message-Id: <20181011195431.3441-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong. So instead give them their own block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism. For now the delayed_refs_rsv will hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via the delayed_refs so that will stay there. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 66f1d3895bca..1a2c3b629af2 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -452,8 +452,9 @@ struct btrfs_space_info { #define BTRFS_BLOCK_RSV_TRANS 3 #define BTRFS_BLOCK_RSV_CHUNK 4 #define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8 struct btrfs_block_rsv { u64 size; @@ -794,6 +795,8 @@ struct btrfs_fs_info { struct btrfs_block_rsv chunk_block_rsv; /* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; + /* block reservation for delayed refs */ + struct btrfs_block_rsv delayed_refs_rsv; struct btrfs_block_rsv empty_block_rsv; @@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info, int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, - struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info); void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info, const u64 start); void btrfs_wait_block_group_reservations(struct btrfs_block_group_cache *bg); @@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum { enum btrfs_flush_state { FLUSH_DELAYED_ITEMS_NR = 1, FLUSH_DELAYED_ITEMS = 2, - FLUSH_DELALLOC = 3, - FLUSH_DELALLOC_WAIT = 4, - ALLOC_CHUNK = 5, - COMMIT_TRANS = 6, + FLUSH_DELAYED_REFS_NR = 3, + FLUSH_DELAYED_REFS = 4, + FLUSH_DELALLOC = 5, + FLUSH_DELALLOC_WAIT = 6, + ALLOC_CHUNK = 7, + COMMIT_TRANS = 8, }; int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes); @@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info, void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *block_rsv, u64 num_bytes); +void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr); +void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs(struct btrfs_fs_info *fs_info, + enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *src, + u64 num_bytes); int btrfs_inc_block_group_ro(struct btrfs_block_group_cache *cache); void btrfs_dec_block_group_ro(struct btrfs_block_group_cache *cache); void btrfs_put_block_group_cache(struct btrfs_fs_info *info); diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 27f7dd4e3d52..96ce087747b2 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans, * existing and update must have the same bytenr */ static noinline void -update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head *existing, struct btrfs_delayed_ref_head *update, int *old_ref_mod_ret) { + struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + struct btrfs_fs_info *fs_info = trans->fs_info; int old_ref_mod; BUG_ON(existing->is_data != update->is_data); @@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs, * versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { - if (existing->total_ref_mod >= 0 && old_ref_mod < 0) + u64 csum_items = + btrfs_csum_bytes_to_leaves(fs_info, + existing->num_bytes); + + if (existing->total_ref_mod >= 0 && old_ref_mod < 0) { delayed_refs->pending_csums -= existing->num_bytes; - if (existing->total_ref_mod < 0 && old_ref_mod >= 0) + btrfs_delayed_refs_rsv_release(fs_info, csum_items); + } + if (existing->total_ref_mod < 0 && old_ref_mod >= 0) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } } spin_unlock(&existing->lock); } @@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); - update_existing_head_ref(delayed_refs, existing, head_ref, + update_existing_head_ref(trans, existing, head_ref, old_ref_mod); /* * we've updated the existing ref, free the newly @@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, } else { if (old_ref_mod) *old_ref_mod = 0; - if (head_ref->is_data && head_ref->ref_mod < 0) + if (head_ref->is_data && head_ref->ref_mod < 0) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += + btrfs_csum_bytes_to_leaves(trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++; atomic_inc(&delayed_refs->num_entries); @@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret = insert_delayed_ref(trans, delayed_refs, head_ref, &ref->node); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); trace_add_delayed_tree_ref(fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ? @@ -866,6 +882,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, ret = insert_delayed_ref(trans, delayed_refs, head_ref, &ref->node); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); trace_add_delayed_data_ref(trans->fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ? @@ -903,6 +920,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, NULL, NULL, NULL); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); return 0; } diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5124c15705ce..377ad9c1cb17 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2692,6 +2692,9 @@ int open_ctree(struct super_block *sb, btrfs_init_block_rsv(&fs_info->empty_block_rsv, BTRFS_BLOCK_RSV_EMPTY); btrfs_init_block_rsv(&fs_info->delayed_block_rsv, BTRFS_BLOCK_RSV_DELOPS); + btrfs_init_block_rsv(&fs_info->delayed_refs_rsv, + BTRFS_BLOCK_RSV_DELREFS); + atomic_set(&fs_info->async_delalloc_pages, 0); atomic_set(&fs_info->defrag_running, 0); atomic_set(&fs_info->qgroup_op_seq, 0); @@ -4419,6 +4422,7 @@ void btrfs_cleanup_dirty_bgs(struct btrfs_transaction *cur_trans, spin_unlock(&cur_trans->dirty_bgs_lock); btrfs_put_block_group(cache); + btrfs_delayed_refs_rsv_release(fs_info, 1); spin_lock(&cur_trans->dirty_bgs_lock); } spin_unlock(&cur_trans->dirty_bgs_lock); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index b32bd38390dd..1213f573eea2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2481,6 +2481,7 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info = trans->fs_info; struct btrfs_delayed_ref_root *delayed_refs = &trans->transaction->delayed_refs; + int nr_items = 1; if (head->total_ref_mod < 0) { struct btrfs_space_info *space_info; @@ -2502,12 +2503,15 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, spin_lock(&delayed_refs->lock); delayed_refs->pending_csums -= head->num_bytes; spin_unlock(&delayed_refs->lock); + nr_items += btrfs_csum_bytes_to_leaves(fs_info, + head->num_bytes); } } /* Also free its reserved qgroup space */ btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, head->qgroup_reserved); + btrfs_delayed_refs_rsv_release(fs_info, nr_items); } static int cleanup_ref_head(struct btrfs_trans_handle *trans, @@ -2802,40 +2806,22 @@ u64 btrfs_csum_bytes_to_leaves(struct btrfs_fs_info *fs_info, u64 csum_bytes) return num_csums; } -int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, - struct btrfs_fs_info *fs_info) +bool btrfs_check_space_for_delayed_refs( struct btrfs_fs_info *fs_info) { - struct btrfs_block_rsv *global_rsv; - u64 num_heads = trans->transaction->delayed_refs.num_heads_ready; - u64 csum_bytes = trans->transaction->delayed_refs.pending_csums; - unsigned int num_dirty_bgs = trans->transaction->num_dirty_bgs; - u64 num_bytes, num_dirty_bgs_bytes; - int ret = 0; - - num_bytes = btrfs_calc_trans_metadata_size(fs_info, 1); - num_heads = heads_to_leaves(fs_info, num_heads); - if (num_heads > 1) - num_bytes += (num_heads - 1) * fs_info->nodesize; - num_bytes <<= 1; - num_bytes += btrfs_csum_bytes_to_leaves(fs_info, csum_bytes) * - fs_info->nodesize; - num_dirty_bgs_bytes = btrfs_calc_trans_metadata_size(fs_info, - num_dirty_bgs); - global_rsv = &fs_info->global_block_rsv; - - /* - * If we can't allocate any more chunks lets make sure we have _lots_ of - * wiggle room since running delayed refs can create more delayed refs. - */ - if (global_rsv->space_info->full) { - num_dirty_bgs_bytes <<= 1; - num_bytes <<= 1; - } + struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; + u64 reserved; + bool ret = false; spin_lock(&global_rsv->lock); - if (global_rsv->reserved <= num_bytes + num_dirty_bgs_bytes) - ret = 1; + reserved = global_rsv->reserved; spin_unlock(&global_rsv->lock); + + spin_lock(&delayed_refs_rsv->lock); + reserved += delayed_refs_rsv->reserved; + if (delayed_refs_rsv->size >= reserved) + ret = true; + spin_unlock(&delayed_refs_rsv->lock); return ret; } @@ -2855,7 +2841,7 @@ int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans, if (val >= NSEC_PER_SEC / 2) return 2; - return btrfs_check_space_for_delayed_refs(trans, fs_info); + return btrfs_check_space_for_delayed_refs(fs_info) ? 1 : 0; } struct async_delayed_refs { @@ -3610,6 +3596,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) */ mutex_lock(&trans->transaction->cache_write_mutex); while (!list_empty(&dirty)) { + bool drop_reserve = true; + cache = list_first_entry(&dirty, struct btrfs_block_group_cache, dirty_list); @@ -3682,6 +3670,7 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) list_add_tail(&cache->dirty_list, &cur_trans->dirty_bgs); btrfs_get_block_group(cache); + drop_reserve = false; } spin_unlock(&cur_trans->dirty_bgs_lock); } else if (ret) { @@ -3692,6 +3681,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) /* if its not on the io list, we need to put the block group */ if (should_put) btrfs_put_block_group(cache); + if (drop_reserve) + btrfs_delayed_refs_rsv_release(fs_info, 1); if (ret) break; @@ -3840,6 +3831,7 @@ int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans, /* if its not on the io list, we need to put the block group */ if (should_put) btrfs_put_block_group(cache); + btrfs_delayed_refs_rsv_release(fs_info, 1); spin_lock(&cur_trans->dirty_bgs_lock); } spin_unlock(&cur_trans->dirty_bgs_lock); @@ -4816,8 +4808,10 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *trans; u64 bytes; + u64 reclaim_bytes = 0; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) @@ -4850,12 +4844,16 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, return -ENOSPC; spin_lock(&delayed_rsv->lock); - if (delayed_rsv->size > bytes) - bytes = 0; - else - bytes -= delayed_rsv->size; + reclaim_bytes += delayed_rsv->reserved; spin_unlock(&delayed_rsv->lock); + spin_lock(&delayed_refs_rsv->lock); + reclaim_bytes += delayed_refs_rsv->reserved; + spin_unlock(&delayed_refs_rsv->lock); + if (reclaim_bytes >= bytes) + goto commit; + bytes -= reclaim_bytes; + if (__percpu_counter_compare(&space_info->total_bytes_pinned, bytes, BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) { @@ -4905,6 +4903,20 @@ static void flush_space(struct btrfs_fs_info *fs_info, shrink_delalloc(fs_info, num_bytes * 2, num_bytes, state == FLUSH_DELALLOC_WAIT); break; + case FLUSH_DELAYED_REFS_NR: + case FLUSH_DELAYED_REFS: + trans = btrfs_join_transaction(root); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + break; + } + if (state == FLUSH_DELAYED_REFS_NR) + nr = calc_reclaim_items_nr(fs_info, num_bytes); + else + nr = 0; + btrfs_run_delayed_refs(trans, nr); + btrfs_end_transaction(trans); + break; case ALLOC_CHUNK: trans = btrfs_join_transaction(root); if (IS_ERR(trans)) { @@ -5377,6 +5389,91 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info, return 0; } +/** + * btrfs_migrate_to_delayed_refs_rsv - transfer bytes to our delayed refs rsv. + * @fs_info - the fs info for our fs. + * @src - the source block rsv to transfer from. + * @num_bytes - the number of bytes to transfer. + * + * This transfers up to the num_bytes amount from the src rsv to the + * delayed_refs_rsv. Any extra bytes are returned to the space info. + */ +void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *src, + u64 num_bytes) +{ + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; + u64 to_free = 0; + + spin_lock(&src->lock); + src->reserved -= num_bytes; + src->size -= num_bytes; + spin_unlock(&src->lock); + + spin_lock(&delayed_refs_rsv->lock); + if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) { + u64 delta = delayed_refs_rsv->size - + delayed_refs_rsv->reserved; + if (num_bytes > delta) { + to_free = num_bytes - delta; + num_bytes = delta; + } + } else { + to_free = num_bytes; + num_bytes = 0; + } + + if (num_bytes) + delayed_refs_rsv->reserved += num_bytes; + if (delayed_refs_rsv->reserved >= delayed_refs_rsv->size) + delayed_refs_rsv->full = 1; + spin_unlock(&delayed_refs_rsv->lock); + + if (num_bytes) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, num_bytes, 1); + if (to_free) + space_info_add_old_bytes(fs_info, delayed_refs_rsv->space_info, + to_free); +} + +/** + * btrfs_throttle_delayed_refs - throttle based on our delayed refs usage. + * @fs_info - the fs_info for our fs. + * @flush - control how we can flush for this reservation. + * + * This will refill the delayed block_rsv up to 1 items size worth of space and + * will return -ENOSPC if we can't make the reservation. + */ +int btrfs_throttle_delayed_refs(struct btrfs_fs_info *fs_info, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; + u64 limit = btrfs_calc_trans_metadata_size(fs_info, 1); + u64 num_bytes = 0; + int ret = -ENOSPC; + + spin_lock(&block_rsv->lock); + if (block_rsv->reserved < block_rsv->size) { + num_bytes = block_rsv->size - block_rsv->reserved; + num_bytes = min(num_bytes, limit); + } + spin_unlock(&block_rsv->lock); + + if (!num_bytes) + return 0; + + ret = reserve_metadata_bytes(fs_info->extent_root, block_rsv, + num_bytes, flush); + if (ret) + return ret; + block_rsv_add_bytes(block_rsv, num_bytes, 0); + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, num_bytes, 1); + return 0; +} + + /* * This is for space we already have accounted in space_info->bytes_may_use, so * basically when we're returning space from block_rsv's. @@ -5699,6 +5796,31 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, return ret; } +static u64 __btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *block_rsv, + u64 num_bytes, u64 *qgroup_to_release) +{ + struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; + struct btrfs_block_rsv *target = delayed_rsv; + + if (target->full || target == block_rsv) + target = global_rsv; + + if (block_rsv->space_info != target->space_info) + target = NULL; + + return block_rsv_release_bytes(fs_info, block_rsv, target, num_bytes, + qgroup_to_release); +} + +void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *block_rsv, + u64 num_bytes) +{ + __btrfs_block_rsv_release(fs_info, block_rsv, num_bytes, NULL); +} + /** * btrfs_inode_rsv_release - release any excessive reservation. * @inode - the inode we need to release from. @@ -5713,7 +5835,6 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) { struct btrfs_fs_info *fs_info = inode->root->fs_info; - struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; struct btrfs_block_rsv *block_rsv = &inode->block_rsv; u64 released = 0; u64 qgroup_to_release = 0; @@ -5723,8 +5844,8 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) * are releasing 0 bytes, and then we'll just get the reservation over * the size free'd. */ - released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv, 0, - &qgroup_to_release); + released = __btrfs_block_rsv_release(fs_info, block_rsv, 0, + &qgroup_to_release); if (released > 0) trace_btrfs_space_reservation(fs_info, "delalloc", btrfs_ino(inode), released, 0); @@ -5735,16 +5856,26 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) qgroup_to_release); } -void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, - struct btrfs_block_rsv *block_rsv, - u64 num_bytes) +/** + * btrfs_delayed_refs_rsv_release - release a ref head's reservation. + * @fs_info - the fs_info for our fs. + * @nr - the number of items to drop. + * + * This drops the delayed ref head's count from the delayed refs rsv and free's + * any excess reservation we had. + */ +void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr) { + struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + u64 num_bytes = btrfs_calc_trans_metadata_size(fs_info, nr); + u64 released = 0; - if (global_rsv == block_rsv || - block_rsv->space_info != global_rsv->space_info) - global_rsv = NULL; - block_rsv_release_bytes(fs_info, block_rsv, global_rsv, num_bytes, NULL); + released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv, + num_bytes, NULL); + if (released) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, released, 0); } static void update_global_block_rsv(struct btrfs_fs_info *fs_info) @@ -5809,9 +5940,10 @@ static void init_global_block_rsv(struct btrfs_fs_info *fs_info) fs_info->trans_block_rsv.space_info = space_info; fs_info->empty_block_rsv.space_info = space_info; fs_info->delayed_block_rsv.space_info = space_info; + fs_info->delayed_refs_rsv.space_info = space_info; - fs_info->extent_root->block_rsv = &fs_info->global_block_rsv; - fs_info->csum_root->block_rsv = &fs_info->global_block_rsv; + fs_info->extent_root->block_rsv = &fs_info->delayed_refs_rsv; + fs_info->csum_root->block_rsv = &fs_info->delayed_refs_rsv; fs_info->dev_root->block_rsv = &fs_info->global_block_rsv; fs_info->tree_root->block_rsv = &fs_info->global_block_rsv; if (fs_info->quota_root) @@ -5831,8 +5963,34 @@ static void release_global_block_rsv(struct btrfs_fs_info *fs_info) WARN_ON(fs_info->chunk_block_rsv.reserved > 0); WARN_ON(fs_info->delayed_block_rsv.size > 0); WARN_ON(fs_info->delayed_block_rsv.reserved > 0); + WARN_ON(fs_info->delayed_refs_rsv.reserved > 0); + WARN_ON(fs_info->delayed_refs_rsv.size > 0); } +/* + * btrfs_update_delayed_refs_rsv - adjust the size of the delayed refs rsv + * @trans - the trans that may have generated delayed refs + * + * This is to be called anytime we may have adjusted trans->delayed_ref_updates, + * it'll calculate the additional size and add it to the delayed_refs_rsv. + */ +void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; + u64 num_bytes; + + if (!trans->delayed_ref_updates) + return; + + num_bytes = btrfs_calc_trans_metadata_size(fs_info, + trans->delayed_ref_updates); + spin_lock(&delayed_rsv->lock); + delayed_rsv->size += num_bytes; + delayed_rsv->full = 0; + spin_unlock(&delayed_rsv->lock); + trans->delayed_ref_updates = 0; +} /* * To be called after all the new block groups attached to the transaction @@ -6126,6 +6284,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, u64 old_val; u64 byte_in_group; int factor; + int ret = 0; /* block accounting for super block */ spin_lock(&info->delalloc_root_lock); @@ -6139,8 +6298,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, while (total) { cache = btrfs_lookup_block_group(info, bytenr); - if (!cache) - return -ENOENT; + if (!cache) { + ret = -ENOENT; + break; + } factor = btrfs_bg_type_to_factor(cache->flags); /* @@ -6199,6 +6360,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, list_add_tail(&cache->dirty_list, &trans->transaction->dirty_bgs); trans->transaction->num_dirty_bgs++; + trans->delayed_ref_updates++; btrfs_get_block_group(cache); } spin_unlock(&trans->transaction->dirty_bgs_lock); @@ -6216,7 +6378,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, total -= num_bytes; bytenr += num_bytes; } - return 0; + + /* Modified block groups are accounted for in the delayed_refs_rsv. */ + btrfs_update_delayed_refs_rsv(trans); + return ret; } static u64 first_logical_byte(struct btrfs_fs_info *fs_info, u64 search_start) @@ -8230,7 +8395,12 @@ use_block_rsv(struct btrfs_trans_handle *trans, goto again; } - if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) { + /* + * The global reserve still exists to save us from ourselves, so don't + * warn_on if we are short on our delayed refs reserve. + */ + if (block_rsv->type != BTRFS_BLOCK_RSV_DELREFS && + btrfs_test_opt(fs_info, ENOSPC_DEBUG)) { static DEFINE_RATELIMIT_STATE(_rs, DEFAULT_RATELIMIT_INTERVAL * 10, /*DEFAULT_RATELIMIT_BURST*/ 1); @@ -10146,6 +10316,7 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) add_block_group_free_space(trans, block_group); /* already aborted the transaction if it failed. */ next: + btrfs_delayed_refs_rsv_release(fs_info, 1); list_del_init(&block_group->bg_list); } trans->can_flush_pending_bgs = can_flush_pending_bgs; @@ -10223,6 +10394,8 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used, link_block_group(cache); list_add_tail(&cache->bg_list, &trans->new_bgs); + trans->delayed_ref_updates++; + btrfs_update_delayed_refs_rsv(trans); set_avail_alloc_bits(fs_info, type); return 0; @@ -10260,6 +10433,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, int factor; struct btrfs_caching_control *caching_ctl = NULL; bool remove_em; + bool remove_rsv = false; block_group = btrfs_lookup_block_group(fs_info, group_start); BUG_ON(!block_group); @@ -10324,6 +10498,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, if (!list_empty(&block_group->dirty_list)) { list_del_init(&block_group->dirty_list); + remove_rsv = true; btrfs_put_block_group(block_group); } spin_unlock(&trans->transaction->dirty_bgs_lock); @@ -10533,6 +10708,8 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, ret = btrfs_del_item(trans, root, path); out: + if (remove_rsv) + btrfs_delayed_refs_rsv_release(fs_info, 1); btrfs_free_path(path); return ret; } diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 212fa71317d6..cd00ec869c96 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5382,7 +5382,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, * Try to steal from the global reserve if there is space for * it. */ - if (!btrfs_check_space_for_delayed_refs(trans, fs_info) && + if (!btrfs_check_space_for_delayed_refs(fs_info) && !btrfs_block_rsv_migrate(global_rsv, rsv, min_size, 0)) return trans; diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 3b84f5015029..117e0c4a914a 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -455,7 +455,7 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, bool enforce_qgroups) { struct btrfs_fs_info *fs_info = root->fs_info; - + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *h; struct btrfs_transaction *cur_trans; u64 num_bytes = 0; @@ -484,13 +484,28 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, * the appropriate flushing if need be. */ if (num_items && root != fs_info->chunk_root) { + struct btrfs_block_rsv *rsv = &fs_info->trans_block_rsv; + u64 delayed_refs_bytes = 0; + qgroup_reserved = num_items * fs_info->nodesize; ret = btrfs_qgroup_reserve_meta_pertrans(root, qgroup_reserved, enforce_qgroups); if (ret) return ERR_PTR(ret); + /* + * We want to reserve all the bytes we may need all at once, so + * we only do 1 enospc flushing cycle per transaction start. We + * accomplish this by simply assuming we'll do 2 x num_items + * worth of delayed refs updates in this trans handle, and + * refill that amount for whatever is missing in the reserve. + */ num_bytes = btrfs_calc_trans_metadata_size(fs_info, num_items); + if (delayed_refs_rsv->full == 0) { + delayed_refs_bytes = num_bytes; + num_bytes <<= 1; + } + /* * Do the reservation for the relocation root creation */ @@ -499,8 +514,24 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, reloc_reserved = true; } - ret = btrfs_block_rsv_add(root, &fs_info->trans_block_rsv, - num_bytes, flush); + ret = btrfs_block_rsv_add(root, rsv, num_bytes, flush); + if (ret) + goto reserve_fail; + if (delayed_refs_bytes) { + btrfs_migrate_to_delayed_refs_rsv(fs_info, rsv, + delayed_refs_bytes); + num_bytes -= delayed_refs_bytes; + } + } else if (num_items == 0 && flush == BTRFS_RESERVE_FLUSH_ALL && + !delayed_refs_rsv->full) { + /* + * Some people call with btrfs_start_transaction(root, 0) + * because they can be throttled, but have some other mechanism + * for reserving space. We still want these guys to refill the + * delayed block_rsv so just add 1 items worth of reservation + * here. + */ + ret = btrfs_throttle_delayed_refs(fs_info, flush); if (ret) goto reserve_fail; } @@ -759,7 +790,7 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) { struct btrfs_fs_info *fs_info = trans->fs_info; - if (btrfs_check_space_for_delayed_refs(trans, fs_info)) + if (btrfs_check_space_for_delayed_refs(fs_info)) return 1; return !!btrfs_block_rsv_check(&fs_info->global_block_rsv, 5); @@ -768,22 +799,12 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) int btrfs_should_end_transaction(struct btrfs_trans_handle *trans) { struct btrfs_transaction *cur_trans = trans->transaction; - int updates; - int err; smp_mb(); if (cur_trans->state >= TRANS_STATE_BLOCKED || cur_trans->delayed_refs.flushing) return 1; - updates = trans->delayed_ref_updates; - trans->delayed_ref_updates = 0; - if (updates) { - err = btrfs_run_delayed_refs(trans, updates * 2); - if (err) /* Error code will also eval true */ - return err; - } - return should_end_transaction(trans); } @@ -813,11 +834,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, { struct btrfs_fs_info *info = trans->fs_info; struct btrfs_transaction *cur_trans = trans->transaction; - u64 transid = trans->transid; - unsigned long cur = trans->delayed_ref_updates; int lock = (trans->type != TRANS_JOIN_NOLOCK); int err = 0; - int must_run_delayed_refs = 0; if (refcount_read(&trans->use_count) > 1) { refcount_dec(&trans->use_count); @@ -828,27 +846,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); - - trans->delayed_ref_updates = 0; - if (!trans->sync) { - must_run_delayed_refs = - btrfs_should_throttle_delayed_refs(trans, info); - cur = max_t(unsigned long, cur, 32); - - /* - * don't make the caller wait if they are from a NOLOCK - * or ATTACH transaction, it will deadlock with commit - */ - if (must_run_delayed_refs == 1 && - (trans->type & (__TRANS_JOIN_NOLOCK | __TRANS_ATTACH))) - must_run_delayed_refs = 2; - } - - btrfs_trans_release_metadata(trans); - trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) btrfs_create_pending_block_groups(trans); @@ -893,10 +890,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, } kmem_cache_free(btrfs_trans_handle_cachep, trans); - if (must_run_delayed_refs) { - btrfs_async_run_delayed_refs(info, cur, transid, - must_run_delayed_refs == 1); - } return err; } diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index b401c4e36394..7d205e50b09c 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -1048,6 +1048,8 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_ITEMS, "FLUSH_DELAYED_ITEMS"}, \ { FLUSH_DELALLOC, "FLUSH_DELALLOC"}, \ { FLUSH_DELALLOC_WAIT, "FLUSH_DELALLOC_WAIT"}, \ + { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ + { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) From patchwork Thu Oct 11 19:53:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637299 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33D13157A for ; Thu, 11 Oct 2018 19:54:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 201D12C09F for ; Thu, 11 Oct 2018 19:54:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 107E42C0A7; Thu, 11 Oct 2018 19:54:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A458A2C09F for ; Thu, 11 Oct 2018 19:54:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726867AbeJLDXe (ORCPT ); Thu, 11 Oct 2018 23:23:34 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:37727 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXd (ORCPT ); Thu, 11 Oct 2018 23:23:33 -0400 Received: by mail-qt1-f195.google.com with SMTP id d14-v6so11350539qto.4 for ; Thu, 11 Oct 2018 12:54:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Ayf4FJ5WuWBTP9J+ecILAGKV0NvK3Tj5jK4kHqTh68A=; b=z0M0JogW4LDmVUjaioPW0Eug1EDQdPaj4T1bR90j07KOEo4C10jyloe7BnvQrTvoGk RFTFqF2OBN7FK2h+5TXNEu0HhYJ5SsCP6ZHy4/qTbW4x8+SeON89xE6QmjW52OWDEUBV bIXv6VF21u9zUSbB+oEWDcHozDUBkkMv+l2XukhlK0Ka2lJcH00mzvFIz+bFi3+dtagT 8tmVUIk/+dzVyVblIH2nMGQuH1hHNKYjyv5X7+QpHrQgN/ukJpLM6zim7Thx6P4y83W8 OVplL8KpsNyBJzZrJRvEe0G9idWniyJkhC/aJ1m5tcCvIiwqnm50rFKwbY2pPUcsWOcS S3Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Ayf4FJ5WuWBTP9J+ecILAGKV0NvK3Tj5jK4kHqTh68A=; b=iuxFUGsN1rRxCnCqMPlxN3bOKekVDSSatnE3WW70Ekk2FXFVxsnXGTnUtZ53rM19fn KKz+nVU2Rod2tsyPXvXF7SZh3wo+9mChqSiM2TbNfa0f7J4JgeCPG2AoTDO/+qFZekzC hVFpwyuZoOcfTA4k4JUGvCgf3YfnYB+9IL1jeJ//pmPSe931FyxBJUtFKw6zGeG2bQyq vG1Eobb72vslSZ7GkayPxBTCUdCMIlbaaCtizDp45BJ1c9EJOQdQvTtD1JI+iMHWad09 DpCpwCcF4vKG7L0F0kHN/1FGWRfLUYjPanOsCt0vp/FT555vAYmwyCfVJ7IVaekv/7Re 0Tiw== X-Gm-Message-State: ABuFfoiXKpwnbGENxM+LTaMvnf/stiHZ0K0gbZP9dzt5mV7UBAqKUAne VVlSMKMWwbGEa+x6nuCBnu1boyxBPPs= X-Google-Smtp-Source: ACcGV63k0hVlLoBroDgPx0xT0BuJeRBOsoDI4BMCn7KPs5w8ENogV3qLlssI65x61ESqEfdYmjPs9Q== X-Received: by 2002:a0c:facc:: with SMTP id p12mr3120157qvo.103.1539287687125; Thu, 11 Oct 2018 12:54:47 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a13-v6sm15928303qkg.94.2018.10.11.12.54.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:46 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 07/42] btrfs: check if free bgs for commit Date: Thu, 11 Oct 2018 15:53:56 -0400 Message-Id: <20181011195431.3441-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP may_commit_transaction will skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1213f573eea2..da73b3e5bc39 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes) return 0; - /* See if there is enough pinned space to make this reservation */ - if (__percpu_counter_compare(&space_info->total_bytes_pinned, - bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) + trans = btrfs_join_transaction(fs_info->extent_root); + if (IS_ERR(trans)) + return -ENOSPC; + + /* + * See if there is enough pinned space to make this reservation, or if + * we have bg's that are going to be freed, allowing us to possibly do a + * chunk allocation the next loop through. + */ + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + __percpu_counter_compare(&space_info->total_bytes_pinned, bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) goto commit; /* @@ -4841,7 +4849,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * this reservation. */ if (space_info != delayed_rsv->space_info) - return -ENOSPC; + goto enospc; spin_lock(&delayed_rsv->lock); reclaim_bytes += delayed_rsv->reserved; @@ -4855,17 +4863,14 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, bytes -= reclaim_bytes; if (__percpu_counter_compare(&space_info->total_bytes_pinned, - bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) { - return -ENOSPC; - } - + bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) + goto enospc; commit: - trans = btrfs_join_transaction(fs_info->extent_root); - if (IS_ERR(trans)) - return -ENOSPC; - return btrfs_commit_transaction(trans); +enospc: + btrfs_end_transaction(trans); + return -ENOSPC; } /* From patchwork Thu Oct 11 19:53:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637303 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99FB9112B for ; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 878112C09B for ; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C1672C0A2; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26CC82C09F for ; Thu, 11 Oct 2018 19:54:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726893AbeJLDXf (ORCPT ); Thu, 11 Oct 2018 23:23:35 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:35201 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXf (ORCPT ); Thu, 11 Oct 2018 23:23:35 -0400 Received: by mail-qt1-f193.google.com with SMTP id v19-v6so11357204qtg.2 for ; Thu, 11 Oct 2018 12:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=lr4oPNsJzTCgyemE9htPT5VmmZwFNCKLK46w3mgICjE=; b=Ifri+2a6WTOu9XtHQCNpcURG4yU7x9WaMeqe8xf6GBJxTQzrSKQroWKHgjamhjM8LP Hoi8V0Yt1swnUTSFhiRNCbZV/dvqK8mYOVx3vqLE5NNJR9t8fn7ap99om4JQ+/EU8wOk gQkrqcxde4bmBPuwTO382aBquX8dMe3PBQ76wLKy6I0OIADhKuNv7uz5PRvtcBmcgZH9 IHbCpfu223BYaSu+Ep7B47vLZhglygkbINvCKwJ8XP/M2O2Qnq3/ScHz5aklv0zch1Fd CNUl/IK+9dnDpvvbzo5+Yeax53jPPyS9wKhJhp642W0Bkay9UMJtcERTODqYuMUmvs1O uwmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=lr4oPNsJzTCgyemE9htPT5VmmZwFNCKLK46w3mgICjE=; b=AGRq6msAwDocvcNyAT7f3Ai9l5DbYlPx7hkiOrQvJ/El80lG4YT+0xnuSjBLxdz3Xb 1jWsDcPIYjE4/Ih6h9FD8/VcO8vvTIVL8dj+90x1LJRWQfQ7xKO0Tb1xWb/ZfLLNDY8p N4sTIOiD5mF/q6hHjZ+PyN+aLuwf62t5faSBs9cd/spTVsXXYSNpmidVyB11OYkCadwb vpN1QP9LE0/3tv69CPCqGZvUwVp1xo01XCzsPhveey2hshZbHmdUf058wRq6SsaS9wnI XWeMZ3sutbuUD45AzmlRZ/daW/JuQ5+TOeyL7/DEJ2VsDv5vJe/Br2lKjtBTBfweWmVf NGAA== X-Gm-Message-State: ABuFfojOfGpvPp/V1/SJTlbtAe0W7/eT9K4L1PIgYnIjQ2ogILMH25jx hVNp8C2DvfuvE9WqOl0bjJ4mhnnAki4= X-Google-Smtp-Source: ACcGV636oLkonoK2QNe+nNEiZA25yyC5KFAzkSNuz1ckplz8fQ9rVp/xTwJmvc7uSubL20U+PPGmPg== X-Received: by 2002:a0c:997a:: with SMTP id i55-v6mr3132449qvd.71.1539287689025; Thu, 11 Oct 2018 12:54:49 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c44-v6sm17790093qtb.53.2018.10.11.12.54.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:48 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 08/42] btrfs: dump block_rsv whe dumping space info Date: Thu, 11 Oct 2018 15:53:57 -0400 Message-Id: <20181011195431.3441-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For enospc_debug having the block rsvs is super helpful to see if we've done something wrong. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval Reviewed-by: David Sterba --- fs/btrfs/extent-tree.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index da73b3e5bc39..c9913c59686b 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -7918,6 +7918,15 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, return ret; } +#define DUMP_BLOCK_RSV(fs_info, rsv_name) \ +do { \ + struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \ + spin_lock(&__rsv->lock); \ + btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \ + __rsv->size, __rsv->reserved); \ + spin_unlock(&__rsv->lock); \ +} while (0) + static void dump_space_info(struct btrfs_fs_info *fs_info, struct btrfs_space_info *info, u64 bytes, int dump_block_groups) @@ -7937,6 +7946,12 @@ static void dump_space_info(struct btrfs_fs_info *fs_info, info->bytes_readonly); spin_unlock(&info->lock); + DUMP_BLOCK_RSV(fs_info, global_block_rsv); + DUMP_BLOCK_RSV(fs_info, trans_block_rsv); + DUMP_BLOCK_RSV(fs_info, chunk_block_rsv); + DUMP_BLOCK_RSV(fs_info, delayed_block_rsv); + DUMP_BLOCK_RSV(fs_info, delayed_refs_rsv); + if (!dump_block_groups) return; From patchwork Thu Oct 11 19:53:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637305 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3999D157A for ; Thu, 11 Oct 2018 19:54:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 292FF2C09B for ; Thu, 11 Oct 2018 19:54:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1BF582C0A7; Thu, 11 Oct 2018 19:54:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE36C2C09B for ; Thu, 11 Oct 2018 19:54:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726903AbeJLDXh (ORCPT ); Thu, 11 Oct 2018 23:23:37 -0400 Received: from mail-qk1-f173.google.com ([209.85.222.173]:36338 "EHLO mail-qk1-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXh (ORCPT ); Thu, 11 Oct 2018 23:23:37 -0400 Received: by mail-qk1-f173.google.com with SMTP id a85-v6so6278228qkg.3 for ; Thu, 11 Oct 2018 12:54:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=yYx+AcpbxeLRdPqDImtMM4yf4Rg6f5T1yjgQiHsfUlY=; b=mHB99gDsbhUNvWIWPjLY0Pw9l6iUNxJU3AewQx7C1hrIXha7BzwTI1UmJ9WSZyCa6K Q/bAgCCzpzbawiqE4aMpZR6WeppSsNoiHDBBb32akCRhvbwz+5mjcKEyoSHHLgrHzAgp Vsnu/rCxgkyZPEK8Jfvp+yz2QQviTLF0hguGIlhy/MDZgPNDHNZkHzuto2YOu8hJPtz6 kqdocH1F3eqmYyJ9PKHEqFPPSXJmQKccMdJu2xKJ2t/Umb47OGYmwC2WCJZMu0vh/4bR Iq2VUuI28IaHWEWU0QoCQZr7SwWTMqZ899rUUTL2uUg8DYykOvsHwJBMiYndhLN3F4sr PBoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=yYx+AcpbxeLRdPqDImtMM4yf4Rg6f5T1yjgQiHsfUlY=; b=hRxMqUjv8tavWnLZuPFcnJohK8KaJliIzrcYBciGsiUJQcL7PNX+je6owydOMQZ/sO gwSCIG0cROL81O1fz690p7mqCl/r88Z4QVmkiNmDlQzBgCPUanBpoUpP/8nVvUG6QQq5 Mpa4paGYHgnYn7jOiMEj+bF47tSLU/3bPRQKNeg+JfxJF7DAHnAQNyvplTYQaEAOSvin nl1Xzf07MIEakPMPDvegvw00YwlprJ3w/1b81EilTemWvCtAVTYlrjzRWu9HFCHw5g3Q s4fu27eqYnLVbAeCLAjtT71KwIPo9dwEbK1HH4Ir9qt349o8M2Eb8YZpIPjWK3CYsj58 rXIw== X-Gm-Message-State: ABuFfohxW4ZKCqumCwMEUobGbIKi9duZMiNUHkOOQyYZ59kMovn2N5eb sK8e0iJ4lTKGj8mGAJSJ1DXu0Q== X-Google-Smtp-Source: ACcGV618jFIlWNGFek61r+IP5wdDyVEjkoU3zDPeUpBf5/BueOMHuFT1L9ffvM+A2e84bm9zaeIXKA== X-Received: by 2002:a37:f8c:: with SMTP id 12-v6mr2994137qkp.224.1539287690821; Thu, 11 Oct 2018 12:54:50 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g4-v6sm6762627qkb.64.2018.10.11.12.54.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:49 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 09/42] btrfs: release metadata before running delayed refs Date: Thu, 11 Oct 2018 15:53:58 -0400 Message-Id: <20181011195431.3441-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We want to release the unused reservation we have since it refills the delayed refs reserve, which will make everything go smoother when running the delayed refs if we're short on our reservation. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 117e0c4a914a..a0f19ca0bd6c 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1922,6 +1922,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } + btrfs_trans_release_metadata(trans); + trans->block_rsv = NULL; + /* make a pass through all the delayed refs we have so far * any runnings procs may add more while we are here */ @@ -1931,9 +1934,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } - btrfs_trans_release_metadata(trans); - trans->block_rsv = NULL; - cur_trans = trans->transaction; /* From patchwork Thu Oct 11 19:53:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71C52157A for ; Thu, 11 Oct 2018 19:54:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61AD92C09F for ; Thu, 11 Oct 2018 19:54:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 55C872C0A7; Thu, 11 Oct 2018 19:54:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 016542C09F for ; Thu, 11 Oct 2018 19:54:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726920AbeJLDXk (ORCPT ); Thu, 11 Oct 2018 23:23:40 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34353 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXj (ORCPT ); Thu, 11 Oct 2018 23:23:39 -0400 Received: by mail-qt1-f195.google.com with SMTP id o17-v6so11371760qtr.1 for ; Thu, 11 Oct 2018 12:54:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=gKSGvLkh9Ok2LxOaudtm+P6nYNrWuwCgPKPCVFl6C1Q=; b=aQqNZkgPZHWGfdixvbmz4TJGZqGOkep+EDTJ8Rv4nj97sHPGG3AFJXNPw/zi2QFyFw U8c0dRtXIcUFN23pTycO8eCCmhlKPVUquC+wty3CqRkwvgzoTnDdbFj2HJ8CymkEY/XQ poxwGqx9DJGLtD5zIZCnoUveG7FfR0Z/ww8D0TQ0+E1rA480XMMU+jSeRUgGMGYvJZo3 GJ560kzh58BybO5tB+cl6tjBXTu5wsgAKj6E8MBuEoBVZ721Ph7zotdsAp1qCoUfXVHd mvPVGsi9YqHiEk3dF+uRu0f5wy84wcm4sAUl3sOwTMn7s8RjYRny0cy/nQFr1Ralsfzm BYQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=gKSGvLkh9Ok2LxOaudtm+P6nYNrWuwCgPKPCVFl6C1Q=; b=aNlUiliGM6mG+mgA8ITadf5bXO3RsoLU75cxBoCCpk6uJNeZYyRSPpCUA8PbctAJtx 2rnHIO/50zALrt3OIj7X7lIK3wa3EkdOqBqDWFyTAJ91h6ktLtpX46Yv90Q3YVoC2HDy cTs5ovJ8873eRmJW7F259/eTI9JnpP2N6nIUGVH9eYMjZwJ1kp4ejlAULaPuzj+8bm/7 8EnY84BxqFsB1N5yuBXjpDFJfpoCq4HPwl+lXnqGr9c8c+LEkYkvr/5kgc53EQeDfeK0 OGUjfbkV4OZrlC7xO2DLShUAS26cKlgGr7Gmy9WLmQwLObseoRFLCSqDs2x/WZ+WgtDv Mu6A== X-Gm-Message-State: ABuFfojr8mUhnJoSMBwf9sJPTz8yFIjiQHN3NXPbWsNVgDZm2qQCj9VL 81Wk5RvKy/gnJyMSNDZ3iizGvQ== X-Google-Smtp-Source: ACcGV63DRo+LS9QI8qklLxJjLNf2gEgwwXsymY/Nd5tzoZ+OdZbn0YgnEhqTZrKiHgQa1T/VDJimlw== X-Received: by 2002:aed:3445:: with SMTP id w63-v6mr3044414qtd.339.1539287692744; Thu, 11 Oct 2018 12:54:52 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u38-v6sm15919995qtc.94.2018.10.11.12.54.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:51 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 10/42] btrfs: protect space cache inode alloc with nofs Date: Thu, 11 Oct 2018 15:53:59 -0400 Message-Id: <20181011195431.3441-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/free-space-cache.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index c3888c113d81..e077ad3b4549 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "ctree.h" #include "free-space-cache.h" #include "transaction.h" @@ -47,6 +48,7 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root, struct btrfs_free_space_header *header; struct extent_buffer *leaf; struct inode *inode = NULL; + unsigned nofs_flag; int ret; key.objectid = BTRFS_FREE_SPACE_OBJECTID; @@ -68,7 +70,13 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root, btrfs_disk_key_to_cpu(&location, &disk_key); btrfs_release_path(path); + /* + * We are often under a trans handle at this point, so we need to make + * sure NOFS is set to keep us from deadlocking. + */ + nofs_flag = memalloc_nofs_save(); inode = btrfs_iget(fs_info->sb, &location, root, NULL); + memalloc_nofs_restore(nofs_flag); if (IS_ERR(inode)) return inode; From patchwork Thu Oct 11 19:54:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E65C2112B for ; Thu, 11 Oct 2018 19:54:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D4E2D2C09B for ; Thu, 11 Oct 2018 19:54:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C92C22C0A7; Thu, 11 Oct 2018 19:54:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5C9462C09B for ; Thu, 11 Oct 2018 19:54:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726923AbeJLDXm (ORCPT ); Thu, 11 Oct 2018 23:23:42 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:46742 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXl (ORCPT ); Thu, 11 Oct 2018 23:23:41 -0400 Received: by mail-qt1-f193.google.com with SMTP id d8-v6so11324062qtk.13 for ; Thu, 11 Oct 2018 12:54:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=09uUQnLegRaGDa0gAKaXhW+r7F/us0LEAwdMnUEs3rc=; b=pLMvbuojEWMGfKA0SNegALIOfFJZ8r9E6ZeGGMBrV9R0tONDm2ZXCWrJqx+3abW5kT bfIZiNW3m4y0WQliDP8A4AAli2AnVIkI15MO6cTuxEhRgvfWrdzaPfgXJXMosg45NYfL X7ybOaj2j5xz3fUhfgFxZZr7jfuuOIavnY4ngtdo/GwxMpHeFba4uDUxXx5EDZ3aWCFK zfq66GKp7yprxZAQLnO1/6qa7hn2tUoq709DvNoYQN/oq8xv60YzJuvRJpustrUsKs6d z5qvI8qCtJfamLiRnLTYJaybk9OcVc11XNx2UQBhrQqs+sZ9b5gswOQa+GQrLc6r2Q3q LsVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=09uUQnLegRaGDa0gAKaXhW+r7F/us0LEAwdMnUEs3rc=; b=L57kg2cmhbo0VT+BqBa8yt9JB64M33OXoArYg3MsIcHESS39qfpqnvAppiDtoF/cLD IizzHPyopveyQvj6fCHr/5Ao/s0cRTjayuJ2xOONXckPAS5X5HcYvuEXZTFilyNNq9t9 f5Y7WqMWXOxkigkrct4yDFpsKljgqUKXPJtJUfrcxsyq2yoGsgQC5QAMsI2IemO9+6Zo rgdkQ4PS1lYr3pCXDtTZiFhS/9LZ43NOU2raLAEtflJ4Yp6n/XUudh8nivIT64NcoNe9 /lPQx/NUNSKmJpq8b6pk+b5MiCQJoDQxL7pBylSPceWD4bq0kvkR8o7g0T8Y5A+pkOdn eqzg== X-Gm-Message-State: ABuFfoiNXw0amlmMYBZtj99wUWtXyuLGfzPOaDnCs2j+WsRmWdyAVBL+ 3gmqa/Cuk3ithrAfNVRZDJZATA== X-Google-Smtp-Source: ACcGV63AfG2rxLX4/HotszEZ/Do4UpVVvYHd369FCZENGVmL4F+gK6aTBZAY0P9Yf14bQj2Tb+ONGQ== X-Received: by 2002:ac8:2278:: with SMTP id p53-v6mr3067191qtp.200.1539287694551; Thu, 11 Oct 2018 12:54:54 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id r13-v6sm16332204qte.19.2018.10.11.12.54.53 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:53 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 11/42] btrfs: fix truncate throttling Date: Thu, 11 Oct 2018 15:54:00 -0400 Message-Id: <20181011195431.3441-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We have a bunch of magic to make sure we're throttling delayed refs when truncating a file. Now that we have a delayed refs rsv and a mechanism for refilling that reserve simply use that instead of all of this magic. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 79 ++++++++++++-------------------------------------------- 1 file changed, 17 insertions(+), 62 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index cd00ec869c96..5a91055a13b2 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -4493,31 +4493,6 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry) return err; } -static int truncate_space_check(struct btrfs_trans_handle *trans, - struct btrfs_root *root, - u64 bytes_deleted) -{ - struct btrfs_fs_info *fs_info = root->fs_info; - int ret; - - /* - * This is only used to apply pressure to the enospc system, we don't - * intend to use this reservation at all. - */ - bytes_deleted = btrfs_csum_bytes_to_leaves(fs_info, bytes_deleted); - bytes_deleted *= fs_info->nodesize; - ret = btrfs_block_rsv_add(root, &fs_info->trans_block_rsv, - bytes_deleted, BTRFS_RESERVE_NO_FLUSH); - if (!ret) { - trace_btrfs_space_reservation(fs_info, "transaction", - trans->transid, - bytes_deleted, 1); - trans->bytes_reserved += bytes_deleted; - } - return ret; - -} - /* * Return this if we need to call truncate_block for the last bit of the * truncate. @@ -4562,7 +4537,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u64 bytes_deleted = 0; bool be_nice = false; bool should_throttle = false; - bool should_end = false; BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY); @@ -4775,15 +4749,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, btrfs_abort_transaction(trans, ret); break; } - if (btrfs_should_throttle_delayed_refs(trans, fs_info)) - btrfs_async_run_delayed_refs(fs_info, - trans->delayed_ref_updates * 2, - trans->transid, 0); if (be_nice) { - if (truncate_space_check(trans, root, - extent_num_bytes)) { - should_end = true; - } if (btrfs_should_throttle_delayed_refs(trans, fs_info)) should_throttle = true; @@ -4795,7 +4761,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, if (path->slots[0] == 0 || path->slots[0] != pending_del_slot || - should_throttle || should_end) { + should_throttle) { if (pending_del_nr) { ret = btrfs_del_items(trans, root, path, pending_del_slot, @@ -4807,23 +4773,24 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, pending_del_nr = 0; } btrfs_release_path(path); - if (should_throttle) { - unsigned long updates = trans->delayed_ref_updates; - if (updates) { - trans->delayed_ref_updates = 0; - ret = btrfs_run_delayed_refs(trans, - updates * 2); - if (ret) - break; - } - } + /* - * if we failed to refill our space rsv, bail out - * and let the transaction restart + * We can generate a lot of delayed refs, so we need to + * throttle every once and a while and make sure we're + * adding enough space to keep up with the work we are + * generating. Since we hold a transaction here we + * can't flush, and we don't want to FLUSH_LIMIT because + * we could have generated too many delayed refs to + * actually allocate, so just bail if we're short and + * let the normal reservation dance happen higher up. */ - if (should_end) { - ret = -EAGAIN; - break; + if (should_throttle) { + ret = btrfs_throttle_delayed_refs(fs_info, + BTRFS_RESERVE_NO_FLUSH); + if (ret) { + ret = -EAGAIN; + break; + } } goto search_again; } else { @@ -4849,18 +4816,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, } btrfs_free_path(path); - - if (be_nice && bytes_deleted > SZ_32M && (ret >= 0 || ret == -EAGAIN)) { - unsigned long updates = trans->delayed_ref_updates; - int err; - - if (updates) { - trans->delayed_ref_updates = 0; - err = btrfs_run_delayed_refs(trans, updates * 2); - if (err) - ret = err; - } - } return ret; } From patchwork Thu Oct 11 19:54:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 411255CAF for ; Thu, 11 Oct 2018 19:54:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EDD42C09B for ; Thu, 11 Oct 2018 19:54:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 231562C0A2; Thu, 11 Oct 2018 19:54:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3FB82C09F for ; Thu, 11 Oct 2018 19:54:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726933AbeJLDXn (ORCPT ); Thu, 11 Oct 2018 23:23:43 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39388 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXn (ORCPT ); Thu, 11 Oct 2018 23:23:43 -0400 Received: by mail-qt1-f193.google.com with SMTP id e22-v6so11363360qto.6 for ; Thu, 11 Oct 2018 12:54:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=471cQ/dE/uVCNlSocUpXaqoy8Ktro672PmaGzD5v8vY=; b=FOsqJNvn24sHi7ORMBYzLsqvT0UMaMgxkwncGisB5ylg42sVq3CPXOdD8h4Nv5J6aC WUGoacLstDVilw9VYvMQQ2fNX/JyiE6r9rOgj6pffVsuji5A4Y+eSXtTn51vhMp+kWCD fwVlVzri1ChZqUkA0gTvubsDm1WjuFMu2NSrh1sTyynN9wr1Cji8AfSus0rh3kJba439 CaeAL9glo5wAui179bakEJeHzhTGPy6HKjN0ZjYorVoC80W0la+BXoT1UNe+7bNshDy9 lMY/o2zVM480FNqstZ5niK6hTEPb0hzyayRZ/dRljcZbF7qrfswPtF7zOGXHKADhrgU1 a8cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=471cQ/dE/uVCNlSocUpXaqoy8Ktro672PmaGzD5v8vY=; b=EFkL5Tb2xTH3PdnHgMmcEtem8Y/bTZznqIZTWJ03Pl32Zvca9oUBXSAfhqHZ1ieL57 yh7EdTVn6DSnY9yr6jcatzDkNeimOgdOgVNGPl+h1gGSIw7IbayALdpEqj24D5tqYEK4 91P/jdqFaEaitFzsZwtvDeCZvv6L9ShhWFpPplM1vhEOeNxHEe6ilEzPooV+wykhhR1y 23b9CE5DwxRZI16RdBd28wkn8VqzJakKUy0g2B8FnXrKnInSK6iTU9e8YEoK9mO8IRXF 49fv9SNGQlmMiTauyrq14LVIndgqaaMQ5v+H2Fc954rCSw5J0JVVN9o8K0q6B0rLkc8j m5cQ== X-Gm-Message-State: ABuFfojS2gUChf0crA41rFV+33HHSNrCSX2GjTonD3Zb6ZfXX98+q/lz Di4p/kQ8JRhB8CeXQlrMA6Z6dd7O4MI= X-Google-Smtp-Source: ACcGV60Zm6Jl1x0WWY5OLNWRyq2004hannA5ykBEPV0fwG+FqxRZN+dAKpejc0t2Q0uc35T2xvG0sA== X-Received: by 2002:a0c:f744:: with SMTP id e4-v6mr2962445qvo.197.1539287696772; Thu, 11 Oct 2018 12:54:56 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y35-v6sm15817398qtc.49.2018.10.11.12.54.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:55 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 12/42] btrfs: don't use global rsv for chunk allocation Date: Thu, 11 Oct 2018 15:54:01 -0400 Message-Id: <20181011195431.3441-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We've done this forever because of the voodoo around knowing how much space we have. However we have better ways of doing this now, and on normal file systems we'll easily have a global reserve of 512MiB, and since metadata chunks are usually 1GiB that means we'll allocate metadata chunks more readily. Instead use the actual used amount when determining if we need to allocate a chunk or not. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c9913c59686b..c0f6110419b2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4374,21 +4374,12 @@ static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global) static int should_alloc_chunk(struct btrfs_fs_info *fs_info, struct btrfs_space_info *sinfo, int force) { - struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; u64 bytes_used = btrfs_space_info_used(sinfo, false); u64 thresh; if (force == CHUNK_ALLOC_FORCE) return 1; - /* - * We need to take into account the global rsv because for all intents - * and purposes it's used space. Don't worry about locking the - * global_rsv, it doesn't change except when the transaction commits. - */ - if (sinfo->flags & BTRFS_BLOCK_GROUP_METADATA) - bytes_used += calc_global_rsv_need_space(global_rsv); - /* * in limited mode, we want to have some free space up to * about 1% of the FS size. From patchwork Thu Oct 11 19:54:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637313 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD469112B for ; Thu, 11 Oct 2018 19:55:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C589B2C09B for ; Thu, 11 Oct 2018 19:55:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B666E2C0A2; Thu, 11 Oct 2018 19:55:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46C252C09B for ; Thu, 11 Oct 2018 19:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726942AbeJLDXp (ORCPT ); Thu, 11 Oct 2018 23:23:45 -0400 Received: from mail-qt1-f176.google.com ([209.85.160.176]:37879 "EHLO mail-qt1-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXp (ORCPT ); Thu, 11 Oct 2018 23:23:45 -0400 Received: by mail-qt1-f176.google.com with SMTP id d14-v6so11351255qto.4 for ; Thu, 11 Oct 2018 12:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=aHABd3s47VJA+IGNtJA9Inn+i4GdiqS25/qy6diaJ4I=; b=RSWcyGIVJczwNh091o/fuS7b7ZHecW0oGJ5eqfjXKuBiyYnerzyG+7SlvP0Q302Q4s +/KvZX7x3fAHzXVfG6iauM35UAx2pf+McT4eSgOqeygzBu3tRtqjrPIWOR5cTQxs1RTa ImLN8XtUeE+PQAj/ubwal+7XLo9YnjUQhW5cXqtT5Lq0lM+BDXi2sPH+w8rUYBPY1LBF p1iLYgrdtPoUUXcHrmYT8PaHesPzSYCmxyPj2wpTLGhGtq077aQZlh2GtewL63kYKlv/ Ggn4wUOAQarxagQHEnNw4KquaLY38y7vnxF1t1gfdkhotPj9uEgBgvSe7jiU6y1DcJTp EulQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=aHABd3s47VJA+IGNtJA9Inn+i4GdiqS25/qy6diaJ4I=; b=j6/RHCcO8i0wj0+XJ6pgnuPrm0i4SoQqgUkGaZecN4G7HZLsCQx8CeIPSx2LQ74K8Q FSx9ZtWMSo310Ly8P3OEi7nzmEN0QLMdwfX0ttU5M9Rj2RbI8DUJh/cO0IzAccW7hZsE dTZewyL7Uxz40d+oixM1GdCBVrqIWsvodHDDzVFny9lpsWdKpeAeE8KpaHPk6O3PiLOR uV1rIiolIN9Jkkf3oks3/YUJ8zswkuVtUSTqixMbwQVDM0vhZPka5zoHrtZHI/2D24xG F3UOP+ug69wobAWItFHZABJTERWXW4GFmFKh8VvK3nDYqyhfjqAXITqq38uVzysTAZiU j/yw== X-Gm-Message-State: ABuFfohvANTHbbS8FWtnoHaGpzEVSZCMr8U5anCkYKulaPhtHsBs6nbZ koNKNCwXTOws4pghazJuB0t8iQ== X-Google-Smtp-Source: ACcGV62++wzVzTQ7ZG2umvQiRoyRt/ePG6F1EqiP0poobHaqcePsMY6PAXjCZ8Q6ma/7yQjINN5VNQ== X-Received: by 2002:ac8:6717:: with SMTP id e23-v6mr2859353qtp.371.1539287698585; Thu, 11 Oct 2018 12:54:58 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o41-v6sm4707809qto.38.2018.10.11.12.54.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:57 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 13/42] btrfs: add ALLOC_CHUNK_FORCE to the flushing code Date: Thu, 11 Oct 2018 15:54:02 -0400 Message-Id: <20181011195431.3441-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With my change to no longer take into account the global reserve for metadata allocation chunks we have this side-effect for mixed block group fs'es where we are no longer allocating enough chunks for the data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE step to the flushing state machine. This will only get used if we've already made a full loop through the flushing machinery and tried committing the transaction. If we have then we can try and force a chunk allocation since we likely need it to make progress. This resolves the issues I was seeing with the mixed bg tests in xfstests with my previous patch. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 ++- fs/btrfs/extent-tree.c | 18 +++++++++++++++++- include/trace/events/btrfs.h | 1 + 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 1a2c3b629af2..29db902511c1 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2730,7 +2730,8 @@ enum btrfs_flush_state { FLUSH_DELALLOC = 5, FLUSH_DELALLOC_WAIT = 6, ALLOC_CHUNK = 7, - COMMIT_TRANS = 8, + ALLOC_CHUNK_FORCE = 8, + COMMIT_TRANS = 9, }; int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c0f6110419b2..cd2280962c8c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4914,6 +4914,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_end_transaction(trans); break; case ALLOC_CHUNK: + case ALLOC_CHUNK_FORCE: trans = btrfs_join_transaction(root); if (IS_ERR(trans)) { ret = PTR_ERR(trans); @@ -4921,7 +4922,9 @@ static void flush_space(struct btrfs_fs_info *fs_info, } ret = do_chunk_alloc(trans, btrfs_metadata_alloc_profile(fs_info), - CHUNK_ALLOC_NO_FORCE); + (state == ALLOC_CHUNK) ? + CHUNK_ALLOC_NO_FORCE : + CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); if (ret > 0 || ret == -ENOSPC) ret = 0; @@ -5057,6 +5060,19 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) commit_cycles--; } + /* + * We don't want to force a chunk allocation until we've tried + * pretty hard to reclaim space. Think of the case where we + * free'd up a bunch of space and so have a lot of pinned space + * to reclaim. We would rather use that than possibly create a + * underutilized metadata chunk. So if this is our first run + * through the flushing state machine skip ALLOC_CHUNK_FORCE and + * commit the transaction. If nothing has changed the next go + * around then we can force a chunk allocation. + */ + if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles) + flush_state++; + if (flush_state > COMMIT_TRANS) { commit_cycles++; if (commit_cycles > 2) { diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index 7d205e50b09c..fdb23181b5b7 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -1051,6 +1051,7 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ + { ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) TRACE_EVENT(btrfs_flush_space, From patchwork Thu Oct 11 19:54:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BEFEF157A for ; Thu, 11 Oct 2018 19:55:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD5752C09F for ; Thu, 11 Oct 2018 19:55:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A17A42C0A7; Thu, 11 Oct 2018 19:55:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 48F032C0A2 for ; Thu, 11 Oct 2018 19:55:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726946AbeJLDXr (ORCPT ); Thu, 11 Oct 2018 23:23:47 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:32925 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXq (ORCPT ); Thu, 11 Oct 2018 23:23:46 -0400 Received: by mail-qt1-f195.google.com with SMTP id q40-v6so11375492qte.0 for ; Thu, 11 Oct 2018 12:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ZS+Ro2q0p6YcbYs63sQ3GFXPklxc57Dq9Ikc/xk3dZs=; b=Qt5t8ogIpodUYLCziHQQlZV9/oqDYrsme07kKhebrZ8JermHmoM/uXQwzufxEod3iB Ym9JBgikuUrcOFC2KoNdNT0X4XCn0Hc2pI18kWJd3vtQpgEBhi5eU33yjHop5bX8xb4w mz4DLbQuOT7mOa9AG+iIlVk5ynWFvaBllU9V57tXeJcxmzAyNJwYiLLRWY6CbIDvmHKl UXkKUDEa9lrULfG8i8FX8BNMIibDHUgc+6P0DVDNOzJ0ylCOf2WFMRPpkfeqHrgmR0YF QfyNKwm7lE5E6ChlmVMTZrLjk73er2XEsTdrVl3nMwYbMH7kn2OqQm9MOPkw019EkBmM QrXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ZS+Ro2q0p6YcbYs63sQ3GFXPklxc57Dq9Ikc/xk3dZs=; b=qbcNaHHx3xFJbPXNpn4Nus//DBrtiRK/UpjUpRnRHTvbhdzK1v4WjOuWu8TG9DLfa/ 5SOMjkuiJyI+SH8QSfedyQcp3BmOUj+bHfMJdoQZ8pE0FJ/7J77pOn3zqHS61Xdead+P xo6dwQb6GITGRZSJb4x1Cpy7HuEqReaI2kLggEtJVG+8zqpD7lEPaaCFhF0JXwAlmVgR lxGd4dhkbBCoA0XiWHAlk+YalMIHvP7n3ejQvYrJ3WJssqDibHMZ+cQUsWlJSxtt5gNN UKFklf69lBLMswvyv9SLUhzplzO3xqxHOTRVfxR79QvJm2n5sqI44Za8PfovMTBwdIG1 ucqQ== X-Gm-Message-State: ABuFfojrwo+gkdOAGM0dVEa6viqFR7POlpcF0dqBk0jh+SKkhoVMKmiq zSCm+ZVpcyX45IDhpYOGhVq7Vg== X-Google-Smtp-Source: ACcGV62K7vrB3Dzt18/m5LTCJi65nt2NGYLNEI7QR/CVp/s/V74mMlgrsa/UNVjpMbc8xwQYZ8cHDg== X-Received: by 2002:ac8:276a:: with SMTP id h39-v6mr3008270qth.274.1539287700676; Thu, 11 Oct 2018 12:55:00 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a68-v6sm4854073qka.54.2018.10.11.12.54.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:54:59 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 14/42] btrfs: reset max_extent_size properly Date: Thu, 11 Oct 2018 15:54:03 -0400 Message-Id: <20181011195431.3441-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we use up our block group before allocating a new one we'll easily get a max_extent_size that's set really really low, which will result in a lot of fragmentation. We need to make sure we're resetting the max_extent_size when we add a new chunk or add new space. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/extent-tree.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index cd2280962c8c..f84537a1d7eb 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4573,6 +4573,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags, goto out; } else { ret = 1; + space_info->max_extent_size = 0; } space_info->force_alloc = CHUNK_ALLOC_NO_FORCE; @@ -6671,6 +6672,7 @@ static int btrfs_free_reserved_bytes(struct btrfs_block_group_cache *cache, space_info->bytes_readonly += num_bytes; cache->reserved -= num_bytes; space_info->bytes_reserved -= num_bytes; + space_info->max_extent_size = 0; if (delalloc) cache->delalloc_bytes -= num_bytes; From patchwork Thu Oct 11 19:54:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637317 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26F37157A for ; Thu, 11 Oct 2018 19:55:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 144732C09B for ; Thu, 11 Oct 2018 19:55:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08DDC2C0A2; Thu, 11 Oct 2018 19:55:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 881062C09B for ; Thu, 11 Oct 2018 19:55:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726958AbeJLDXt (ORCPT ); Thu, 11 Oct 2018 23:23:49 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:37764 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXt (ORCPT ); Thu, 11 Oct 2018 23:23:49 -0400 Received: by mail-qt1-f195.google.com with SMTP id d14-v6so11351498qto.4 for ; Thu, 11 Oct 2018 12:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=SV6VzMZmuQU6kRBrptnXRYkuIetlbfVghoSBpsgva9M=; b=XeKAnBStOd15/qlUk6mlCtvIcNpufvyBuuIsnReZu4EZzv6PDH3o4ksIOxYk4QqWXn PibGm/yShI2lX1v2lQaq+G1TzGIhdRyeIU6NEmwHCoR8IPpac/x9xtuAMF5TJklMUcxo bqj1Ve7BQbEvhM6M3R59dyjdbahMnvKKOhfThuG0iW8IVQXchatU2yThRg5p+F1DhLAM QmUYCB26q4uYIdgMyN4fQUhk5/X7hSzmgegJqDGNEy8ik36CIH4Ln0YO59NwgPVeIXyf VOcTSWPrtt7+hKB9NYBU2OJMzJck25IrIbpMLJeh5W+4UwWpYA40b1sXI5MzZaCCo364 0ERQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=SV6VzMZmuQU6kRBrptnXRYkuIetlbfVghoSBpsgva9M=; b=Z0eY9u1Rw/ccaHCbDhH8YfiSQZiLpOBulZhh77Ayq2AihZliH4/ZHD7OovtSavz5Ek /wvQqcyg3NADDmG1JV6vc5nJuW7g+4cmQpdAW7MNGVIcIffWO2NVCgSw5WFaVSLvMpTU j6Y98vV0AEZ1Vr6FAQ3Ypfr8WImRYwsLC6m/RCzJuGDWMrrlH8r9mvCYDnnTNwduj7Ba 1c0rUBEiZ59IJ8IWv/l9//m6jGU1tMGrz89f+Lc/qNGwgLRZT/n94MjRNVcLiWv6jBkV 9f8h8dgCvW0JKc+6NeJ6oQnvYmiv/+KaLf2pgeZcdOMi9UpnvREzWqSgccy/h6EkEY1j XY7w== X-Gm-Message-State: ABuFfojs5a2rXihZqgJRb/25GPGJ1lhjYSEdJA9fdTPjUdGcJWoPE5kC +raGK1hbeWK91Ax4H1uDF3lYjg== X-Google-Smtp-Source: ACcGV60hCxwbaVdZXTItLHBiD0NIH0cGEjGEZxNCcHjECR3Zsgb2QW9tYohTjRCSo+47jOXLPk/93w== X-Received: by 2002:ac8:4a93:: with SMTP id l19-v6mr2788080qtq.97.1539287702486; Thu, 11 Oct 2018 12:55:02 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u18-v6sm17402725qtb.63.2018.10.11.12.55.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:01 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 15/42] btrfs: don't enospc all tickets on flush failure Date: Thu, 11 Oct 2018 15:54:04 -0400 Message-Id: <20181011195431.3441-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With the introduction of the per-inode block_rsv it became possible to have really really large reservation requests made because of data fragmentation. Since the ticket stuff assumed that we'd always have relatively small reservation requests it just killed all tickets if we were unable to satisfy the current request. However this is generally not the case anymore. So fix this logic to instead see if we had a ticket that we were able to give some reservation to, and if we were continue the flushing loop again. Likewise we make the tickets use the space_info_add_old_bytes() method of returning what reservation they did receive in hopes that it could satisfy reservations down the line. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f84537a1d7eb..7a53f6a29ebc 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4779,6 +4779,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } struct reserve_ticket { + u64 orig_bytes; u64 bytes; int error; struct list_head list; @@ -5000,7 +5001,7 @@ static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info, !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)); } -static void wake_all_tickets(struct list_head *head) +static bool wake_all_tickets(struct list_head *head) { struct reserve_ticket *ticket; @@ -5009,7 +5010,10 @@ static void wake_all_tickets(struct list_head *head) list_del_init(&ticket->list); ticket->error = -ENOSPC; wake_up(&ticket->wait); + if (ticket->bytes != ticket->orig_bytes) + return true; } + return false; } /* @@ -5077,8 +5081,12 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) if (flush_state > COMMIT_TRANS) { commit_cycles++; if (commit_cycles > 2) { - wake_all_tickets(&space_info->tickets); - space_info->flush = 0; + if (wake_all_tickets(&space_info->tickets)) { + flush_state = FLUSH_DELAYED_ITEMS_NR; + commit_cycles--; + } else { + space_info->flush = 0; + } } else { flush_state = FLUSH_DELAYED_ITEMS_NR; } @@ -5130,10 +5138,11 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, u64 orig_bytes) + struct reserve_ticket *ticket) { DEFINE_WAIT(wait); + u64 reclaim_bytes = 0; int ret = 0; spin_lock(&space_info->lock); @@ -5154,14 +5163,12 @@ static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, ret = ticket->error; if (!list_empty(&ticket->list)) list_del_init(&ticket->list); - if (ticket->bytes && ticket->bytes < orig_bytes) { - u64 num_bytes = orig_bytes - ticket->bytes; - space_info->bytes_may_use -= num_bytes; - trace_btrfs_space_reservation(fs_info, "space_info", - space_info->flags, num_bytes, 0); - } + if (ticket->bytes && ticket->bytes < ticket->orig_bytes) + reclaim_bytes = ticket->orig_bytes - ticket->bytes; spin_unlock(&space_info->lock); + if (reclaim_bytes) + space_info_add_old_bytes(fs_info, space_info, reclaim_bytes); return ret; } @@ -5187,6 +5194,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, { struct reserve_ticket ticket; u64 used; + u64 reclaim_bytes = 0; int ret = 0; ASSERT(orig_bytes); @@ -5222,6 +5230,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, * the list and we will do our own flushing further down. */ if (ret && flush != BTRFS_RESERVE_NO_FLUSH) { + ticket.orig_bytes = orig_bytes; ticket.bytes = orig_bytes; ticket.error = 0; init_waitqueue_head(&ticket.wait); @@ -5262,25 +5271,21 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, return ret; if (flush == BTRFS_RESERVE_FLUSH_ALL) - return wait_reserve_ticket(fs_info, space_info, &ticket, - orig_bytes); + return wait_reserve_ticket(fs_info, space_info, &ticket); ret = 0; priority_reclaim_metadata_space(fs_info, space_info, &ticket); spin_lock(&space_info->lock); if (ticket.bytes) { - if (ticket.bytes < orig_bytes) { - u64 num_bytes = orig_bytes - ticket.bytes; - space_info->bytes_may_use -= num_bytes; - trace_btrfs_space_reservation(fs_info, "space_info", - space_info->flags, - num_bytes, 0); - - } + if (ticket.bytes < orig_bytes) + reclaim_bytes = orig_bytes - ticket.bytes; list_del_init(&ticket.list); ret = -ENOSPC; } spin_unlock(&space_info->lock); + + if (reclaim_bytes) + space_info_add_old_bytes(fs_info, space_info, reclaim_bytes); ASSERT(list_empty(&ticket.list)); return ret; } From patchwork Thu Oct 11 19:54:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CDD0112B for ; Thu, 11 Oct 2018 19:55:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89EF92C09B for ; Thu, 11 Oct 2018 19:55:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E6C52C0A7; Thu, 11 Oct 2018 19:55:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 004042C09B for ; Thu, 11 Oct 2018 19:55:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726970AbeJLDXv (ORCPT ); Thu, 11 Oct 2018 23:23:51 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:33573 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXu (ORCPT ); Thu, 11 Oct 2018 23:23:50 -0400 Received: by mail-qk1-f194.google.com with SMTP id 84-v6so6291283qkf.0 for ; Thu, 11 Oct 2018 12:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=8rOVak+zFvn8kwTuxufp2i+ZKKWKap8qoVBHOiPi4Bk=; b=s5BhkSD6WGrnQiDBtmyJMtJvSfCMAhn4mm9BGU1mrqTNpOtDIZ+PXUHk8ffz6yZP23 xQ+IVXWXMuVWGQtCIn8I1rlJcsRpKv8PYu44eOoLangLx7p04V0ivWVKmqIvkXpBHQQh 01xVvu26tH+nt/cDKMlbcVccU4GWiuOsoyim1j2qnDYKmKPeEMhw++6IkfVI0zPkV/CZ MVxDLODVXDzKwdGAfwGFFwSk8R+EQrwviXBqbIupmEyirZ+IMgpEhOZCZMP7tCx5aBv+ QkhOauAxbxhFqFraWPCb4Fb2qZVwCXL1wLv1Mgy6Pdn3LxJiVE9Xk5oX1avtHG1nB72N u9nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=8rOVak+zFvn8kwTuxufp2i+ZKKWKap8qoVBHOiPi4Bk=; b=NOH1u9SnqM9rOSbNaF+bABYYW6mlocr65cC5HLOzDcjcvgjXFCZMANqNTFHeZ3FZLc zpwl0cff2TrFnndo7XBoocL49JUneFWVyl6Ts0gRUQABIuYrQS/k668zPvx9TbPlTae5 EG/XY+ndW2xHeXbH17U12KVS5FeIrdT6FA1qvwgkOjxcBvQ6u7Lzn5nvr9RfN4Zp+lQf GO3gEuffLuN/WgNI6Bwyzcwmk3pHI+VHazSqYJrlgpVy6CZQG1BkqN+Zg/oi/hYAiGSt rZO3r7H4B9Kd7t67hGR8wykDYCASKsE9DActtJWG/AlJF2QgKgBUAVOqC0PAaYfJdW6I xv2g== X-Gm-Message-State: ABuFfoioIMZTxOabv34Q9m6IUwy9mTCMPETZMmUGuRclnPaNz/yGJlaZ OxdeWT34JcKHQjPozfiBfr9Nh/jX5Ao= X-Google-Smtp-Source: ACcGV60GXkUmiDG6H2a0fstKUlY+6ZRiPDEtCuchuoZhvX03q4SoYrmXYtt22B5ag+G2Mhsm64qa1g== X-Received: by 2002:a37:650b:: with SMTP id z11-v6mr2835444qkb.221.1539287704335; Thu, 11 Oct 2018 12:55:04 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m71-v6sm13169793qke.71.2018.10.11.12.55.03 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:03 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 16/42] btrfs: loop in inode_rsv_refill Date: Thu, 11 Oct 2018 15:54:05 -0400 Message-Id: <20181011195431.3441-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With severe fragmentation we can end up with our inode rsv size being huge during writeout, which would cause us to need to make very large metadata reservations. However we may not actually need that much once writeout is complete. So instead try to make our reservation, and if we couldn't make it re-calculate our new reservation size and try again. If our reservation size doesn't change between tries then we know we are actually out of space and can error out. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 56 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 7a53f6a29ebc..3aba96442472 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5766,6 +5766,21 @@ int btrfs_block_rsv_refill(struct btrfs_root *root, return ret; } +static inline void __get_refill_bytes(struct btrfs_block_rsv *block_rsv, + u64 *metadata_bytes, u64 *qgroup_bytes) +{ + *metadata_bytes = 0; + *qgroup_bytes = 0; + + spin_lock(&block_rsv->lock); + if (block_rsv->reserved < block_rsv->size) + *metadata_bytes = block_rsv->size - block_rsv->reserved; + if (block_rsv->qgroup_rsv_reserved < block_rsv->qgroup_rsv_size) + *qgroup_bytes = block_rsv->qgroup_rsv_size - + block_rsv->qgroup_rsv_reserved; + spin_unlock(&block_rsv->lock); +} + /** * btrfs_inode_rsv_refill - refill the inode block rsv. * @inode - the inode we are refilling. @@ -5781,25 +5796,37 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, { struct btrfs_root *root = inode->root; struct btrfs_block_rsv *block_rsv = &inode->block_rsv; - u64 num_bytes = 0; + u64 num_bytes = 0, last = 0; u64 qgroup_num_bytes = 0; int ret = -ENOSPC; - spin_lock(&block_rsv->lock); - if (block_rsv->reserved < block_rsv->size) - num_bytes = block_rsv->size - block_rsv->reserved; - if (block_rsv->qgroup_rsv_reserved < block_rsv->qgroup_rsv_size) - qgroup_num_bytes = block_rsv->qgroup_rsv_size - - block_rsv->qgroup_rsv_reserved; - spin_unlock(&block_rsv->lock); - + __get_refill_bytes(block_rsv, &num_bytes, &qgroup_num_bytes); if (num_bytes == 0) return 0; - ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_num_bytes, true); - if (ret) - return ret; - ret = reserve_metadata_bytes(root, block_rsv, num_bytes, flush); + do { + ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_num_bytes, true); + if (ret) + return ret; + ret = reserve_metadata_bytes(root, block_rsv, num_bytes, flush); + if (ret) { + btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes); + last = num_bytes; + /* + * If we are fragmented we can end up with a lot of + * outstanding extents which will make our size be much + * larger than our reserved amount. If we happen to + * try to do a reservation here that may result in us + * trying to do a pretty hefty reservation, which we may + * not need once delalloc flushing happens. If this is + * the case try and do the reserve again. + */ + if (flush == BTRFS_RESERVE_FLUSH_ALL) + __get_refill_bytes(block_rsv, &num_bytes, + &qgroup_num_bytes); + } + } while (ret && last != num_bytes); + if (!ret) { block_rsv_add_bytes(block_rsv, num_bytes, 0); trace_btrfs_space_reservation(root->fs_info, "delalloc", @@ -5809,8 +5836,7 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, spin_lock(&block_rsv->lock); block_rsv->qgroup_rsv_reserved += qgroup_num_bytes; spin_unlock(&block_rsv->lock); - } else - btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes); + } return ret; } From patchwork Thu Oct 11 19:54:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11F1F112B for ; Thu, 11 Oct 2018 19:55:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F330A2C09F for ; Thu, 11 Oct 2018 19:55:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E70E92C0A8; Thu, 11 Oct 2018 19:55:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 976482C09F for ; Thu, 11 Oct 2018 19:55:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726982AbeJLDXw (ORCPT ); Thu, 11 Oct 2018 23:23:52 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:39994 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXw (ORCPT ); Thu, 11 Oct 2018 23:23:52 -0400 Received: by mail-qt1-f196.google.com with SMTP id b4-v6so11330708qtc.7 for ; Thu, 11 Oct 2018 12:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=3KmYan+SJv1KntXTmAyylbTgpGl4Ql82kSHN+uqa4Eo=; b=SuiUPlfBWbGFt4P0yJzdOZU5PF3l7QQY7g5KVTlK48wEC87hV3TCaRgtJxT2qexOaL WfJyp64v+8AbiKt9e2gt5g53PC34mrGSI69Rw1CWTf03348iJ/noDp7xtONdJ/vmG56L 4pGtL/G/3Xt+KEWJGNdDt41jF36BkP2wrgxdAZhSjnjWcwOydIHCqFsqwlXLCvO8de3T jZdNM5b+D8K/+LVCC7ceG6sR6YTCYk9gVzGn5LeC0zaQlIXH/ELuZzJI7xl3LN0kfrcM 5WwyZ9IyMcg/lzAhvI/64Aj7Qna4dFEqosrJle7Srdrm3gpVqJXvceP6baxyx+sIop/z S+rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=3KmYan+SJv1KntXTmAyylbTgpGl4Ql82kSHN+uqa4Eo=; b=evDoZGIte39qCNhsqpxeddONzJ1OZYyRLxZJFRyz/QB53mxiwTwxvqTdvk9BILrmyE /exIjxuK5mXM1Sxq1U3NIKtZujxKyqMhr2GwdawPI7ABKtH3/JkAbjugxeUkzREIJEMN JLQjeBplKLUol4SWZAV3R7db/bogtUnxFkN+bS9urblKMkUqRnYFXTHUtUMYHXSTG3TA 1Z9RAS4YdzP0GBJwkYbFElBC8q8pMXQWH2Vw3vZQpJFNbUXaz/oNoeA60GS32dp8mrb5 RcUqII6gddAEsxfGH/hClgWjRIWQESoQA6Zn9UNqUZ7FtXUIGaMU2dGnliAtc7YRR6RI C2nA== X-Gm-Message-State: ABuFfoh4L5KC9sknIyBg+Ws3vDY2H9kG7bpIMcOqOygRUrTdVmK2Q/Uj 9Bh+qNzd8MLYGWcF1AEYDIM0j0t3+9Q= X-Google-Smtp-Source: ACcGV62ImT7Lg7cDkc+mZUrQtw6HI+fbI5/2eQ/m49UMgoemMi2xl4odauJsxFU1dnPNM0KdX8z84g== X-Received: by 2002:ac8:427:: with SMTP id v39-v6mr2954080qtg.59.1539287706266; Thu, 11 Oct 2018 12:55:06 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m54-v6sm11452976qtb.97.2018.10.11.12.55.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:05 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 17/42] btrfs: run delayed iputs before committing Date: Thu, 11 Oct 2018 15:54:06 -0400 Message-Id: <20181011195431.3441-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Delayed iputs means we can have final iputs of deleted inodes in the queue, which could potentially generate a lot of pinned space that could be free'd. So before we decide to commit the transaction for ENOPSC reasons, run the delayed iputs so that any potential space is free'd up. If there is and we freed enough we can then commit the transaction and potentially be able to make our reservation. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 3aba96442472..6e7bc3197737 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4823,6 +4823,15 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes) return 0; + /* + * If we have pending delayed iputs then we could free up a bunch of + * pinned space, so make sure we run the iputs before we do our pinned + * bytes check below. + */ + mutex_lock(&fs_info->cleaner_delayed_iput_mutex); + btrfs_run_delayed_iputs(fs_info); + mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return -ENOSPC; From patchwork Thu Oct 11 19:54:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637323 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DE68112B for ; Thu, 11 Oct 2018 19:55:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A3F72C09B for ; Thu, 11 Oct 2018 19:55:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1EB8E2C0A7; Thu, 11 Oct 2018 19:55:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74E6A2C09B for ; Thu, 11 Oct 2018 19:55:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726990AbeJLDXy (ORCPT ); Thu, 11 Oct 2018 23:23:54 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:46784 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDXy (ORCPT ); Thu, 11 Oct 2018 23:23:54 -0400 Received: by mail-qt1-f194.google.com with SMTP id d8-v6so11324872qtk.13 for ; Thu, 11 Oct 2018 12:55:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=2AifJsoYa8VVGHqyfqs2+9dIHUSfR9sXlFu5O/p8l7g=; b=NpO4SkJY2sWiUrIZ1cMlLbyuXAFs/M37Cym4C7JSnvPzxJ/7hsQHWoAjBQSJXx3kLM tDNkn/gmhzYl4W6u+y/fT1li6e83FUUMb2pgbXYNpCnz9Gfep7gKHCWSwf+2mYoEdMmF OplAgK+p1CPKrpWGB8MULJF3uoIMDAR3VSxMaBT3H55aB9O0ny4+MDv1yAYEDdZYmtGV he1p79/HvRoT4DN3EX+SMRB3q+OhPIQzkXoyMm9l0fw9pe1K7D/um3Y+Z2Vz7vDidyW+ BdBhFcJO4XSw/Go7YfJOAeKahFmarRATn/CdgcK9vMxkUqh7eS56TCLT3Usm9WC13a3e 90Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=2AifJsoYa8VVGHqyfqs2+9dIHUSfR9sXlFu5O/p8l7g=; b=jtLShts5BrqH2PqQXWAFSm8Ck7WcNzZOjT/o3h0AJnOlEiKu7Wa0d0ZKcZFOXEiHZL 2nKMKcAzEvz0Zjg0xfwwsq4CDIQ3mtqtWlkqJrWkxyTTqOsG6RfugikpN415E16EODyI HgsuMjSbup6urOkrJHw3iS3Gx/dWr6W5AVJW/pr4gHtT6KNaqzupf8gpHjJCR/NpVTR4 9FZZD+Qy7tyYHxLJ4dbocnT9z39lbw8rXRZ2ec1kowdIrDcgbdyFWMj6Rab+5D8x//zQ QAgrV1LelMVKD6Ked9w/QrEAEYtainbkjkS3lmvxUsyP+oAiXXsTMTukURcV7vQ8CzR+ nDbQ== X-Gm-Message-State: ABuFfojnTeWIcSogLdKYOhrGzCoDxlCgoZQymJnErn3FkrJcGemqGQQa muv33gq4+MzHcwLBHzZQ5uVGPd8zcnc= X-Google-Smtp-Source: ACcGV61qKnpkEDoq48KFj6SAYa2bpCHAxXg6Y7QZXNOLZdW8kWg7rMkr1Mgyh5AKjujURlKbJEYp8A== X-Received: by 2002:aed:2f82:: with SMTP id m2-v6mr2990853qtd.85.1539287708066; Thu, 11 Oct 2018 12:55:08 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id l43-v6sm17609830qtc.80.2018.10.11.12.55.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:07 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 18/42] btrfs: move the dio_sem higher up the callchain Date: Thu, 11 Oct 2018 15:54:07 -0400 Message-Id: <20181011195431.3441-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We're getting a lockdep splat because we take the dio_sem under the log_mutex. What we really need is to protect fsync() from logging an extent map for an extent we never waited on higher up, so just guard the whole thing with dio_sem. ====================================================== WARNING: possible circular locking dependency detected 4.18.0-rc4-xfstests-00025-g5de5edbaf1d4 #411 Not tainted ------------------------------------------------------ aio-dio-invalid/30928 is trying to acquire lock: 0000000092621cfd (&mm->mmap_sem){++++}, at: get_user_pages_unlocked+0x5a/0x1e0 but task is already holding lock: 00000000cefe6b35 (&ei->dio_sem){++++}, at: btrfs_direct_IO+0x3be/0x400 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #5 (&ei->dio_sem){++++}: lock_acquire+0xbd/0x220 down_write+0x51/0xb0 btrfs_log_changed_extents+0x80/0xa40 btrfs_log_inode+0xbaf/0x1000 btrfs_log_inode_parent+0x26f/0xa80 btrfs_log_dentry_safe+0x50/0x70 btrfs_sync_file+0x357/0x540 do_fsync+0x38/0x60 __ia32_sys_fdatasync+0x12/0x20 do_fast_syscall_32+0x9a/0x2f0 entry_SYSENTER_compat+0x84/0x96 -> #4 (&ei->log_mutex){+.+.}: lock_acquire+0xbd/0x220 __mutex_lock+0x86/0xa10 btrfs_record_unlink_dir+0x2a/0xa0 btrfs_unlink+0x5a/0xc0 vfs_unlink+0xb1/0x1a0 do_unlinkat+0x264/0x2b0 do_fast_syscall_32+0x9a/0x2f0 entry_SYSENTER_compat+0x84/0x96 -> #3 (sb_internal#2){.+.+}: lock_acquire+0xbd/0x220 __sb_start_write+0x14d/0x230 start_transaction+0x3e6/0x590 btrfs_evict_inode+0x475/0x640 evict+0xbf/0x1b0 btrfs_run_delayed_iputs+0x6c/0x90 cleaner_kthread+0x124/0x1a0 kthread+0x106/0x140 ret_from_fork+0x3a/0x50 -> #2 (&fs_info->cleaner_delayed_iput_mutex){+.+.}: lock_acquire+0xbd/0x220 __mutex_lock+0x86/0xa10 btrfs_alloc_data_chunk_ondemand+0x197/0x530 btrfs_check_data_free_space+0x4c/0x90 btrfs_delalloc_reserve_space+0x20/0x60 btrfs_page_mkwrite+0x87/0x520 do_page_mkwrite+0x31/0xa0 __handle_mm_fault+0x799/0xb00 handle_mm_fault+0x7c/0xe0 __do_page_fault+0x1d3/0x4a0 async_page_fault+0x1e/0x30 -> #1 (sb_pagefaults){.+.+}: lock_acquire+0xbd/0x220 __sb_start_write+0x14d/0x230 btrfs_page_mkwrite+0x6a/0x520 do_page_mkwrite+0x31/0xa0 __handle_mm_fault+0x799/0xb00 handle_mm_fault+0x7c/0xe0 __do_page_fault+0x1d3/0x4a0 async_page_fault+0x1e/0x30 -> #0 (&mm->mmap_sem){++++}: __lock_acquire+0x42e/0x7a0 lock_acquire+0xbd/0x220 down_read+0x48/0xb0 get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_fast+0xa4/0x150 iov_iter_get_pages+0xc3/0x340 do_direct_IO+0xf93/0x1d70 __blockdev_direct_IO+0x32d/0x1c20 btrfs_direct_IO+0x227/0x400 generic_file_direct_write+0xcf/0x180 btrfs_file_write_iter+0x308/0x58c aio_write+0xf8/0x1d0 io_submit_one+0x3a9/0x620 __ia32_compat_sys_io_submit+0xb2/0x270 do_int80_syscall_32+0x5b/0x1a0 entry_INT80_compat+0x88/0xa0 other info that might help us debug this: Chain exists of: &mm->mmap_sem --> &ei->log_mutex --> &ei->dio_sem Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&ei->dio_sem); lock(&ei->log_mutex); lock(&ei->dio_sem); lock(&mm->mmap_sem); *** DEADLOCK *** 1 lock held by aio-dio-invalid/30928: #0: 00000000cefe6b35 (&ei->dio_sem){++++}, at: btrfs_direct_IO+0x3be/0x400 stack backtrace: CPU: 0 PID: 30928 Comm: aio-dio-invalid Not tainted 4.18.0-rc4-xfstests-00025-g5de5edbaf1d4 #411 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014 Call Trace: dump_stack+0x7c/0xbb print_circular_bug.isra.37+0x297/0x2a4 check_prev_add.constprop.45+0x781/0x7a0 ? __lock_acquire+0x42e/0x7a0 validate_chain.isra.41+0x7f0/0xb00 __lock_acquire+0x42e/0x7a0 lock_acquire+0xbd/0x220 ? get_user_pages_unlocked+0x5a/0x1e0 down_read+0x48/0xb0 ? get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_fast+0xa4/0x150 iov_iter_get_pages+0xc3/0x340 do_direct_IO+0xf93/0x1d70 ? __alloc_workqueue_key+0x358/0x490 ? __blockdev_direct_IO+0x14b/0x1c20 __blockdev_direct_IO+0x32d/0x1c20 ? btrfs_run_delalloc_work+0x40/0x40 ? can_nocow_extent+0x490/0x490 ? kvm_clock_read+0x1f/0x30 ? can_nocow_extent+0x490/0x490 ? btrfs_run_delalloc_work+0x40/0x40 btrfs_direct_IO+0x227/0x400 ? btrfs_run_delalloc_work+0x40/0x40 generic_file_direct_write+0xcf/0x180 btrfs_file_write_iter+0x308/0x58c aio_write+0xf8/0x1d0 ? kvm_clock_read+0x1f/0x30 ? __might_fault+0x3e/0x90 io_submit_one+0x3a9/0x620 ? io_submit_one+0xe5/0x620 __ia32_compat_sys_io_submit+0xb2/0x270 do_int80_syscall_32+0x5b/0x1a0 entry_INT80_compat+0x88/0xa0 Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 095f0bb86bb7..c07110edb9de 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out; inode_lock(inode); + + /* + * We take the dio_sem here because the tree log stuff can race with + * lockless dio writes and get an extent map logged for an extent we + * never waited on. We need it this high up for lockdep reasons. + */ + down_write(&BTRFS_I(inode)->dio_sem); + atomic_inc(&root->log_batch); /* @@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) */ ret = btrfs_wait_ordered_range(inode, start, len); if (ret) { + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) * checked called fsync. */ ret = filemap_check_wb_err(inode->i_mapping, file->f_wb_err); + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans = btrfs_start_transaction(root, 0); if (IS_ERR(trans)) { ret = PTR_ERR(trans); + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) * file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); /* diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 1650dc44a5e3..66b7e059b765 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, INIT_LIST_HEAD(&extents); - down_write(&inode->dio_sem); write_lock(&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start; @@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, } WARN_ON(!list_empty(&extents)); write_unlock(&tree->lock); - up_write(&inode->dio_sem); btrfs_release_path(path); if (!ret) From patchwork Thu Oct 11 19:54:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8309F157A for ; Thu, 11 Oct 2018 19:55:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E9812C09B for ; Thu, 11 Oct 2018 19:55:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 62B5B2C0A2; Thu, 11 Oct 2018 19:55:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 05ADD2C09B for ; Thu, 11 Oct 2018 19:55:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727001AbeJLDX4 (ORCPT ); Thu, 11 Oct 2018 23:23:56 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:44996 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDX4 (ORCPT ); Thu, 11 Oct 2018 23:23:56 -0400 Received: by mail-qt1-f194.google.com with SMTP id c56-v6so11328292qtd.11 for ; Thu, 11 Oct 2018 12:55:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=unYdeV0h0D7vZF7/WYt36r/3Sg+lYY+VUgHQLhqzUMw=; b=eZ42HfhcqWQUvBF2Yinm8WUofbjgnWa8MGrmWZ4pFMpqPcoJJ//XvMBncqJCefAuj0 lsgtquwQbZmYEFR4g32MuEcUF/NBewQIp15lEl1T3vRujBaUQ1CzG3RWUCZTVnMg5R5v 0Q5GXimGYlqj4SjlTctv2x5QS9YhMMfgNmJQnKNIswSAOFg2gK+xgxSBLMc8qJdIUmoD u8WqDg9sezB/8GGJF6KN7fnT7FLwthl6+Z4r4X1HCIO7147y3mLTnqxcmUvUynlJEodW /PV9J/CzbmyzZOHGpu/8rrae4/X/sWSGBaRrJ29E+c8K/doqAf9gES7utbf1JDqFfhxB cStw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=unYdeV0h0D7vZF7/WYt36r/3Sg+lYY+VUgHQLhqzUMw=; b=hZA29Vufa9JQYgvhsAzi6TkVbyYDa3JqqTfZQ0oKVvliWTQE2ZqkSNsXh2Crga+FkR 9Zof3r9mIXlr0p/cVkSfBTZnQz6elarOUenmeJ+zqOVYeQz0kM151sevvgaJyqEGmv4b lI1MhHN9A2e4VpbkgNcqoTmTwj9bygWlvYHpBA2UacHVW2ZJKUQjerYr/S+a3SE9rbA/ jMLJgek5PF69XwefioDqze+z2K6+DJSv8sdRkJEMBnM1rf9DmbRuBKKrCopAALXWT+/a 1aThJT37C3JT4FIKXyUzzGaUhWFPJM+GP6zAhaxASVb2RH7dTklyJJYsmCLGOsCLOTCO qskg== X-Gm-Message-State: ABuFfojyQXM7/pVay0CgoWo82EPKmU7NuKrw514iA/ScTC19h3L96wwj GdTKvRgmOr1b4FHYJa4amW01rA== X-Google-Smtp-Source: ACcGV60lS2TtmgK2ZvxZ+JZQ2EhPXvCkV3iv5fyCmxYBFSc2Phr6dsso8SKMtPwUDFc/RnVZVGYpZQ== X-Received: by 2002:a0c:8545:: with SMTP id n63-v6mr3019575qva.205.1539287709990; Thu, 11 Oct 2018 12:55:09 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y26-v6sm13448316qty.52.2018.10.11.12.55.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:09 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 19/42] btrfs: set max_extent_size properly Date: Thu, 11 Oct 2018 15:54:08 -0400 Message-Id: <20181011195431.3441-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We can't use entry->bytes if our entry is a bitmap entry, we need to use entry->max_extent_size in that case. Fix up all the logic to make this consistent. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/free-space-cache.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index e077ad3b4549..2e96ee7da3ec 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1770,6 +1770,18 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl, return -1; } +static void set_max_extent_size(struct btrfs_free_space *entry, + u64 *max_extent_size) +{ + if (entry->bitmap) { + if (entry->max_extent_size > *max_extent_size) + *max_extent_size = entry->max_extent_size; + } else { + if (entry->bytes > *max_extent_size) + *max_extent_size = entry->bytes; + } +} + /* Cache the size of the max extent in bytes */ static struct btrfs_free_space * find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, @@ -1791,8 +1803,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, for (node = &entry->offset_index; node; node = rb_next(node)) { entry = rb_entry(node, struct btrfs_free_space, offset_index); if (entry->bytes < *bytes) { - if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + set_max_extent_size(entry, max_extent_size); continue; } @@ -1810,8 +1821,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, } if (entry->bytes < *bytes + align_off) { - if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + set_max_extent_size(entry, max_extent_size); continue; } @@ -1823,8 +1833,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, *offset = tmp; *bytes = size; return entry; - } else if (size > *max_extent_size) { - *max_extent_size = size; + } else { + set_max_extent_size(entry, max_extent_size); } continue; } @@ -2684,8 +2694,7 @@ static u64 btrfs_alloc_from_bitmap(struct btrfs_block_group_cache *block_group, err = search_bitmap(ctl, entry, &search_start, &search_bytes, true); if (err) { - if (search_bytes > *max_extent_size) - *max_extent_size = search_bytes; + set_max_extent_size(entry, max_extent_size); return 0; } @@ -2722,8 +2731,8 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, entry = rb_entry(node, struct btrfs_free_space, offset_index); while (1) { - if (entry->bytes < bytes && entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + if (entry->bytes < bytes) + set_max_extent_size(entry, max_extent_size); if (entry->bytes < bytes || (!entry->bitmap && entry->offset < min_start)) { From patchwork Thu Oct 11 19:54:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637327 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A451B157A for ; Thu, 11 Oct 2018 19:55:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 910C92C09B for ; Thu, 11 Oct 2018 19:55:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 855AC2C0A2; Thu, 11 Oct 2018 19:55:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34B802C09B for ; Thu, 11 Oct 2018 19:55:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727006AbeJLDX6 (ORCPT ); Thu, 11 Oct 2018 23:23:58 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:38035 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDX6 (ORCPT ); Thu, 11 Oct 2018 23:23:58 -0400 Received: by mail-qt1-f196.google.com with SMTP id l9-v6so11356198qtf.5 for ; Thu, 11 Oct 2018 12:55:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=u9vRDoMMODgf2y/Ifp4zyo+ltN+0lwuXKLQSeAL0348=; b=jJGlwfg2RhpGa7yu/J7ug1ujoStRxyRpcVVadnqBIsZkUY38V5XODQFz6fWXVd0mh9 vus2TNdUYxO4DktkGkiZez+oR5+px1mTPj9yeo+eYVY5BGWtpioGTxr6uv1EJUdGpjHS XycmPq3C/WsudF1Hj+e0hwOdyzgJ776ApPAGMvQfHu1Z3GrdiOWhn/j4K7TvbklXM9dV 3wALgGku+SGkha+Emb0N7Bss9uQB0UG2+0BuklrZl/Eq4f6Ubmm0G4EvIg8jryg92/GI MZ8sLhr9QN0uHRx3rsv0dfqLB12Eyz/ajg0e3dvexcR0GAhAJbt8jlxwHhsV3b5ogOuA 3YWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=u9vRDoMMODgf2y/Ifp4zyo+ltN+0lwuXKLQSeAL0348=; b=CoUiOQSXQ9FHGtLIyfE7G5Reqi/vNCfxZKIathgQrB175k3ZFVufXZceN3Z8akAUBL he1ngqbU4SeNy8XQL5AXe6q53rXhiiN0XhGluhQ6PrVo+usSYwNJGr9SIBNvgCRJTuTu 9ChkQxKsw9GAr9sTfN3ADYDKPqc4tggWCQBqRwVVCfhcOug98PwR3bMBbILv5rcoVJbi 9G0VF1EzYTb8QByWhGjoXAVLcwbuwMttP7iPLCpkNLGHyl3KzKjB4exZoES2IHsugCzt RzJ/E+nA+MNCIJNDvC+MDM4gEQfftCySklygwNaznZKbiOhlN1OIMspeliZqq6aLHBjc XYDw== X-Gm-Message-State: ABuFfojP/MJiaxOv85o7yTy1VC9jQQIIpAKc4rU0Z08dYgRkljDhzkTm 5UJjXdVFgIZ6hA29RrV+wFvFKugnayA= X-Google-Smtp-Source: ACcGV60uIe+3RVv6bNq5OM0uJLt4h/l+XuCLsyLKwct0ysB5HLyh+eDEEQaW5K03wAZxe3bnTe0FCQ== X-Received: by 2002:a0c:b29d:: with SMTP id r29-v6mr2950126qve.70.1539287711935; Thu, 11 Oct 2018 12:55:11 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c8-v6sm10982505qkb.81.2018.10.11.12.55.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:10 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 20/42] btrfs: don't use ctl->free_space for max_extent_size Date: Thu, 11 Oct 2018 15:54:09 -0400 Message-Id: <20181011195431.3441-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik max_extent_size is supposed to be the largest contiguous range for the space info, and ctl->free_space is the total free space in the block group. We need to keep track of these separately and _only_ use the max_free_space if we don't have a max_extent_size, as that means our original request was too large to search any of the block groups for and therefore wouldn't have a max_extent_size set. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 6e7bc3197737..4f48d047a1ec 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -7496,6 +7496,7 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, struct btrfs_block_group_cache *block_group = NULL; u64 search_start = 0; u64 max_extent_size = 0; + u64 max_free_space = 0; u64 empty_cluster = 0; struct btrfs_space_info *space_info; int loop = 0; @@ -7791,8 +7792,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, spin_lock(&ctl->tree_lock); if (ctl->free_space < num_bytes + empty_cluster + empty_size) { - if (ctl->free_space > max_extent_size) - max_extent_size = ctl->free_space; + max_free_space = max(max_free_space, + ctl->free_space); spin_unlock(&ctl->tree_lock); goto loop; } @@ -7959,6 +7960,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, } out: if (ret == -ENOSPC) { + if (!max_extent_size) + max_extent_size = max_free_space; spin_lock(&space_info->lock); space_info->max_extent_size = max_extent_size; spin_unlock(&space_info->lock); From patchwork Thu Oct 11 19:54:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637329 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A144B112B for ; Thu, 11 Oct 2018 19:55:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E5942C09B for ; Thu, 11 Oct 2018 19:55:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 82E192C0A2; Thu, 11 Oct 2018 19:55:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D87F2C09B for ; Thu, 11 Oct 2018 19:55:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727013AbeJLDYA (ORCPT ); Thu, 11 Oct 2018 23:24:00 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:45007 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYA (ORCPT ); Thu, 11 Oct 2018 23:24:00 -0400 Received: by mail-qt1-f196.google.com with SMTP id c56-v6so11328583qtd.11 for ; Thu, 11 Oct 2018 12:55:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z3nhKV1wBqQcfXnDoMHMl2GEjOrb6h1HQ06C9nkbnUE=; b=b3KCe1EWRzyLQiTGj7FwJp9/DIi+ZZnKj+VPNGL4dTvrM0LbRmtKyc9e3t0y0TI+6R mL5aScEgl32muQoVRaBzEjP/Gjb9GMO1QsIVC+ozW6zXSHExbiVjiJTXE20oKbjvTaaR uegv2Ey+g7pcnT/MbUv7T3gRP2uLVZtz0EM6+6WU0IdACv91NHcw43yCzW1SmyuAcRQa yyP65SRKaU6m3Pc2/SLORMsWO74B6ugxf9IRkOZfO63Kq4qdfdcw7SkZeXExM4GbGYTw qiOu8CZh/o6+PZmrYu4e5VZtVsTKneu0S4k6olgQZbAR2HPFZaudHRl6Tto8+3EcbbPJ tS/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z3nhKV1wBqQcfXnDoMHMl2GEjOrb6h1HQ06C9nkbnUE=; b=G3YE+MVAA6P6Jv4sTlK5XS2xW2u+TDRzFKgPN8bPWskFEY4UwsaufX3bpRZgNcbykA FBBefS0YFRVvUlIxdTMycRtlHZpdg4mamyfE4C8FGMvDZ8VBD3tdiT8jzHUbvAOS326D 4XfAjZg/pIK93QGqsypAFPBcYVNTFZLpEtoLguMD02USA5+8d8s1qkVufRxyFwg+0Kse 2RY2YKOj0fD1eoyfCT+/mpwZWvXHW30QAxL+2Xa+zmoWvdy5wGW4Rp72Y6Tan7iYVUIE H/Vq53UAxZoQ+Zzahn6tHyso4egAL7L8dLRhUFt+rZbee/Jts/DBadUiZV4pJ0Ex5CKf Tp5g== X-Gm-Message-State: ABuFfojBl1T9LSiRp/tOTq8DxkhSV47hDrsg0sSPMvnchfOdfeSCgjOj jXfCICnvi+hx9Mibc7ktXADbXw== X-Google-Smtp-Source: ACcGV61Y2Ax0IdCm3cEj9kXmdLWejTxQnenuNLNo3O1VZ/yMJedBP4sFh6rozmgtOJo5I3d4FVD+mw== X-Received: by 2002:a0c:ae04:: with SMTP id y4-v6mr2970287qvc.233.1539287713800; Thu, 11 Oct 2018 12:55:13 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m13-v6sm8907378qtm.20.2018.10.11.12.55.12 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:12 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Cc: Josef Bacik Subject: [PATCH 21/42] btrfs: reset max_extent_size on clear in a bitmap Date: Thu, 11 Oct 2018 15:54:10 -0400 Message-Id: <20181011195431.3441-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We need to clear the max_extent_size when we clear bits from a bitmap since it could have been from the range that contains the max_extent_size. Reviewed-by: Liu Bo Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 2e96ee7da3ec..d2a863a2ee24 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1687,6 +1687,8 @@ static inline void __bitmap_clear_bits(struct btrfs_free_space_ctl *ctl, bitmap_clear(info->bitmap, start, count); info->bytes -= bytes; + if (info->max_extent_size > ctl->unit) + info->max_extent_size = 0; } static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl, From patchwork Thu Oct 11 19:54:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637331 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56C99112B for ; Thu, 11 Oct 2018 19:55:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4614F2C09B for ; Thu, 11 Oct 2018 19:55:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A9412C0A2; Thu, 11 Oct 2018 19:55:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D79D72C09B for ; Thu, 11 Oct 2018 19:55:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727029AbeJLDYC (ORCPT ); Thu, 11 Oct 2018 23:24:02 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:42712 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYC (ORCPT ); Thu, 11 Oct 2018 23:24:02 -0400 Received: by mail-qk1-f194.google.com with SMTP id g20-v6so6257315qke.9 for ; Thu, 11 Oct 2018 12:55:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=vfQB1tHlwUhp446i68IaYEtvkGVFvtWut8g/b3eBAlk=; b=vLn//HymRhBz3ifqOuzuMIGbc82nV4RZ0mhv7FUNJ79nXMLHvyz4dnPZlfweaBnr5X otc8UZxKhk4fIa/PoBx7xNFxW1zFDqS66F+X7Y5oV9gplaO8JHHHn+2xfZN6NXYdbICW 0RuonW5nrnyLrnCwNB4yra/2QkU1+dbgAly3c+Ng3MGH19SQ+6OOQQYnRtj0xIKaa1OB qPQJEEEAJybQAyLsJlnvjcLJi3AGpaF181l8f6EvS6CB8vcMd0fgzjluSgV2jBQALJDI ER+lZc6s7yHXkyWbPyXC/Bp39B8O35vYK+W4/sKk6enepa1AkBK5GGrtzffh2nZ1/nuB /IvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=vfQB1tHlwUhp446i68IaYEtvkGVFvtWut8g/b3eBAlk=; b=eeTSbgi2j2ruWkMPMeqc7+NkqohI/skXBeBmdSbWreur26Uuw3B51OzcDjqYxhR7Q4 GWua69SmjujKRwfdbZxJLOlMqTuc3GCGpbjIEO+IOo5j7/+qzicbSkN7e6tCGFwoA99W /mUuQiEVQTonqDzSwCCI5R0rNcbu/65EuA+/xowmJVbf3acYpCiGkIZzChSS7dVEYSZ7 hAsf62Y4SYf7HGsUQzEbxz8owtyNtidXCmP4OMhemaeM7xxvyWwBbxCYtHU7hKzbyiTq gT2UZGF+fxNxGIg8myZKlyZrBSNa82ZiJ7jzvI5ZZMzBtdfcpUdtLqoTsj9LqyPhJhk0 xQuQ== X-Gm-Message-State: ABuFfohjQOS7+oIFPMCTiJmhg89GC8dnAg8UUmDDBkcBfXLQkH8uHcJu JikekWf3I3Iht3w7jHjzAazFg3mt9D0= X-Google-Smtp-Source: ACcGV60TVXcO1Yz1E/Ay4q8njlMOAVJ2dPzprdmsvUgonzMBsIIve6ZGBk9Zm+nXNZqfODLXYqhAVA== X-Received: by 2002:a37:f8f:: with SMTP id 15-v6mr3006273qkp.300.1539287715636; Thu, 11 Oct 2018 12:55:15 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id x191-v6sm13929123qkb.93.2018.10.11.12.55.14 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:14 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 22/42] btrfs: only run delayed refs if we're committing Date: Thu, 11 Oct 2018 15:54:11 -0400 Message-Id: <20181011195431.3441-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I noticed in a giant dbench run that we spent a lot of time on lock contention while running transaction commit. This is because dbench results in a lot of fsync()'s that do a btrfs_transaction_commit(), and they all run the delayed refs first thing, so they all contend with each other. This leads to seconds of 0 throughput. Change this to only run the delayed refs if we're the ones committing the transaction. This makes the latency go away and we get no more lock contention. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index a0f19ca0bd6c..39a2bddb0b29 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1925,15 +1925,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - /* make a pass through all the delayed refs we have so far - * any runnings procs may add more while we are here - */ - ret = btrfs_run_delayed_refs(trans, 0); - if (ret) { - btrfs_end_transaction(trans); - return ret; - } - cur_trans = trans->transaction; /* @@ -1946,12 +1937,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) if (!list_empty(&trans->new_bgs)) btrfs_create_pending_block_groups(trans); - ret = btrfs_run_delayed_refs(trans, 0); - if (ret) { - btrfs_end_transaction(trans); - return ret; - } - if (!test_bit(BTRFS_TRANS_DIRTY_BG_RUN, &cur_trans->flags)) { int run_it = 0; @@ -2022,6 +2007,15 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) spin_unlock(&fs_info->trans_lock); } + /* + * We are now the only one in the commit area, we can run delayed refs + * without hitting a bunch of lock contention from a lot of people + * trying to commit the transaction at once. + */ + ret = btrfs_run_delayed_refs(trans, 0); + if (ret) + goto cleanup_transaction; + extwriter_counter_dec(cur_trans, trans->type); ret = btrfs_start_delalloc_flush(fs_info); From patchwork Thu Oct 11 19:54:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637333 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 950D1157A for ; Thu, 11 Oct 2018 19:55:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8266A2C0A2 for ; Thu, 11 Oct 2018 19:55:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7695A2C0A8; Thu, 11 Oct 2018 19:55:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24AF32C0A2 for ; Thu, 11 Oct 2018 19:55:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727039AbeJLDYE (ORCPT ); Thu, 11 Oct 2018 23:24:04 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:34424 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYE (ORCPT ); Thu, 11 Oct 2018 23:24:04 -0400 Received: by mail-qt1-f194.google.com with SMTP id o17-v6so11373378qtr.1 for ; Thu, 11 Oct 2018 12:55:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=hXRxupQSb58mFrmcwAlimg01JbJ9bnLWSvle0nmy4+w=; b=voITN7rgg3K8b1R3/0Z09264yrhpbzEHh/yNkWu643tNCx7G8l7PuBbh05DZ9ASrZ7 uVlRoRG174TyzbGc+kRfFYRzfTkb7IZZVZbYTN3rCt4DoEyqY7VfamO8W0K+6X6ztyzF w512998YtuvrbRo5muEwo2RU4qaSvBGBw1OhzrdQRDZ5qJCDDcB7Q6kfnh4OSlwO8EI9 trx5IdCn8n+yjrIO6paHups9D8aTPyOrpMmVQcbDSXbSWFMcKAzE3zwj4wdxtt1RATuB QZciaGqgQzLV+SsljFK1DH4RIW2DaSP2o0tiGncYl2DflEGuJcnhuuVdZnje+YNb3EHR 1ugA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=hXRxupQSb58mFrmcwAlimg01JbJ9bnLWSvle0nmy4+w=; b=hqSz4TgRLACFd/79TrFQFB3uzPF2szaeO0HpB+UcW4bf4BoaPl0NUzjpqOsVYQgbsC 7fXn3+HrMg7ZyNRYhhNqiyRUyDadLJZ4FosH2viDAY4HibRzIgbkwtc1k148gqH/Z+h5 lDFiL5774k6RReS7f/FPgb0qRYFdm8rd2NywKqkJ2XIakFEauQpmoVpvjX9sC6oOrkwN wFda1lrgn7HIhSfBZIqXxiMx5WErpWsRI/MNE5CbmuNcAcmYmf1KTQzbuPqown4PkjUp Dv9xwukh9ZWjKpL5ZYXrJv4bqerMFKAiJLlkYkw28xhHQlRNnvXb7qJdB8kmiddae+vc NQbw== X-Gm-Message-State: ABuFfohRCotD8ZdkWT7lhImLZXlVs3thUkBSjki91bFHwZjNZzfe7cce fEIo/BGkj6cwqhO8AGzUehb3wA== X-Google-Smtp-Source: ACcGV60IDSUNSGjj88YJouuKkpHI00Y3a2AxMlpkQANy7w+zoOvxOU8XqQ9qf4fO8/q+9itEkJ1PFQ== X-Received: by 2002:a0c:f98b:: with SMTP id t11mr812683qvn.146.1539287717585; Thu, 11 Oct 2018 12:55:17 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 144-v6sm14006454qkk.63.2018.10.11.12.55.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:16 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 23/42] btrfs: make sure we create all new bgs Date: Thu, 11 Oct 2018 15:54:12 -0400 Message-Id: <20181011195431.3441-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allocating new chunks modifies both the extent and chunk tree, which can trigger new chunk allocations. So instead of doing list_for_each_safe, just do while (!list_empty()) so we make sure we don't exit with other pending bg's still on our list. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Reviewed-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 4f48d047a1ec..ec8ede2b1dec 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -10361,7 +10361,7 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info) void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) { struct btrfs_fs_info *fs_info = trans->fs_info; - struct btrfs_block_group_cache *block_group, *tmp; + struct btrfs_block_group_cache *block_group; struct btrfs_root *extent_root = fs_info->extent_root; struct btrfs_block_group_item item; struct btrfs_key key; @@ -10369,7 +10369,10 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) bool can_flush_pending_bgs = trans->can_flush_pending_bgs; trans->can_flush_pending_bgs = false; - list_for_each_entry_safe(block_group, tmp, &trans->new_bgs, bg_list) { + while (!list_empty(&trans->new_bgs)) { + block_group = list_first_entry(&trans->new_bgs, + struct btrfs_block_group_cache, + bg_list); if (ret) goto next; From patchwork Thu Oct 11 19:54:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637335 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22AC6157A for ; Thu, 11 Oct 2018 19:55:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 114272C09B for ; Thu, 11 Oct 2018 19:55:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05C452C0A2; Thu, 11 Oct 2018 19:55:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A247D2C09B for ; Thu, 11 Oct 2018 19:55:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727044AbeJLDYF (ORCPT ); Thu, 11 Oct 2018 23:24:05 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:39930 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYF (ORCPT ); Thu, 11 Oct 2018 23:24:05 -0400 Received: by mail-qk1-f195.google.com with SMTP id q5-v6so6273722qki.6 for ; Thu, 11 Oct 2018 12:55:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=5ErWA8iDFY+6lz4JimjzCOpUtgurbUZk5oK0tyfx1Sk=; b=vXUEaHu6EI4QOrztq8r6jxGq+TwWOTeI5byEwAKRUsqU6R+X3VvX2+B21eEhzqbzL2 ORwT6hHFL5iVETNhjVOdNvUA05cawKOW+IfZzVMu1CtdBuCSrsIG5W9lVdtaJ2bHdhrl QtmjTMEm7eOEuEitlHTkdtIEElAawnS/PoRkKWxQq5MD4g7TYxXqDqs5PsURMJ256CJE RsqVCZjkAM1OIvEPoz0L0+KxBIxuTpY6CbZx6JZJYUD+fTSg4jaUQNvNQ6Y/0nZ8zT7G u/8OX117Djh4JvsqvmGgzys7mvnf28cZ/0ZTzhbnahXUTEZcsu75hRMnA0V7Sjamp0bX bFIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=5ErWA8iDFY+6lz4JimjzCOpUtgurbUZk5oK0tyfx1Sk=; b=UclJKpTyHX2yyqgI+bZ7mUi2xTdOBY+e0C+Sv/jDCrLNAoPscMFFZMFN+Wdtxayqnr KX5OTrZlDDas+zkaGqcPH2TVSKH3f8Ht4SKr7gs7rV5pUD3MUe3AzYVYufr7oMtM1u8Y pcWdq11uFnrGnZ5rJPKXVodIg9ryIADNTPH8DqTyefMQjAtb1K7kGZpVgTrB7W4adyT6 Dr3YdaPuWGlrQFKTDI0zsTOpJCIBa8d3TxbLC1XcUdRDkqq4JZVajfCHMBJ/30zotFd9 k30q+6jr2vB4w1erEKfEsLHgVlLLmuu7mhClgwU7RYwQhg/+rulqiqDp9CbHMQyK7Omb dkLA== X-Gm-Message-State: ABuFfoj8IPSYzTieNLjw2isiSOKu/dmBFjcchOnsquvkfOAFM55f6LE9 gjrJpm/ujdGcimaL9TMpv3Xui1jZLlg= X-Google-Smtp-Source: ACcGV62fJ6FlKbyoJqiIJ0PeckMX20CGJ9vAC/RdcRZ6skbM4XjsJJNg8FDE5w1BWq2JIDo6iJIt0w== X-Received: by 2002:a37:aa44:: with SMTP id t65-v6mr3049606qke.121.1539287719398; Thu, 11 Oct 2018 12:55:19 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 18-v6sm18686875qtm.13.2018.10.11.12.55.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:18 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 24/42] btrfs: assert on non-empty delayed iputs Date: Thu, 11 Oct 2018 15:54:13 -0400 Message-Id: <20181011195431.3441-25-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I ran into an issue where there was some reference being held on an inode that I couldn't track. This assert wasn't triggered, but it at least rules out we're doing something stupid. Reviewed-by: Omar Sandoval Reviewed-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 377ad9c1cb17..39bd158466cd 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3979,6 +3979,7 @@ void close_ctree(struct btrfs_fs_info *fs_info) kthread_stop(fs_info->transaction_kthread); kthread_stop(fs_info->cleaner_kthread); + ASSERT(list_empty(&fs_info->delayed_iputs)); set_bit(BTRFS_FS_CLOSING_DONE, &fs_info->flags); btrfs_free_qgroup_config(fs_info); From patchwork Thu Oct 11 19:54:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637337 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EC62112B for ; Thu, 11 Oct 2018 19:55:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DBF22C09F for ; Thu, 11 Oct 2018 19:55:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 821182C0A7; Thu, 11 Oct 2018 19:55:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2062A2C09B for ; Thu, 11 Oct 2018 19:55:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727056AbeJLDYI (ORCPT ); Thu, 11 Oct 2018 23:24:08 -0400 Received: from mail-qk1-f181.google.com ([209.85.222.181]:39307 "EHLO mail-qk1-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYI (ORCPT ); Thu, 11 Oct 2018 23:24:08 -0400 Received: by mail-qk1-f181.google.com with SMTP id q5-v6so6273778qki.6 for ; Thu, 11 Oct 2018 12:55:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=lizMbT7vCIz2W88dy4Yz105drP7CcDdejIERE88viZs=; b=v5xhD/f+xmgqZ41UgjJVWwSwCaS+rZJ/nR0QPIhVL418AuQQeEEjyi8EHwVm194c+2 PobGPkFGWqlniiMUBr8f4ANNGkLe9D7S/qpMRTL16n7S4bn5xK+TXh4nYRcCiVpz6Cpd q+uMKbmHXbUqZoFZ6KqfutDyW36sBzTqfAbsJZgDWdr9P3s4oHJFYpBg1scND+cx41nr 7vCvG0VbrpFE3KAee2Eb94gew6+PKS5nvTb4Bl0dXHngdfvdTTFWoN/6UTa78+P93Y31 JPWfkrmdtqSrILKLqnOyprImmvOaVI4sk6WNPGrpBsZ/tnDGOJUFowf8Ks9hZzYOmN6s 4Xew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=lizMbT7vCIz2W88dy4Yz105drP7CcDdejIERE88viZs=; b=Ll+NiSe4GL2am82yXXWgq4bHRTjpuMDQVL1tsjHqJaJAn/cLFL8WJ1EKXv9sFntUFw 9QWUeNcyE8q6OUNr3vV4AL4w7Np7rf5L7/m6jV5WnlqfiDMaCtVubrz2Xnv+elffZcQa FRrHhCRYh7/iwk8ZT44OR2GFu1REY6DCxNMM4WYL7NKuBkr5rO0n/+Mcx0q8uxDzIgfR xqxS0Jbc0pVrk1wRnhFXtVNiqLW7sC9tldp+StxVYckpJrF2vmGHZda7XwciMIC9YXqa qUCvdFSHHEkpmAZW8TZvrAxNLcDqZGMaApHUIvCs5SMY1GQVnFQIrT3q7aZ/o31z9z0T RY8A== X-Gm-Message-State: ABuFfoh24dnAe1HAqPYsG36SvfWzFBOc2XLpdMz/hh1GWjhrlxbPXIOw dhyC/h5HcW9DjnBXLTHjdSSKiZq9cmE= X-Google-Smtp-Source: ACcGV63Z5DKT9KrOeyNMhA3NsUjR5jMhQjmvvs0vcPShqOGjEmBMiTn6XZsv4YOP25w7fiAEFHA3Cg== X-Received: by 2002:a37:5946:: with SMTP id n67-v6mr2991971qkb.202.1539287721256; Thu, 11 Oct 2018 12:55:21 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c36-v6sm20041113qta.56.2018.10.11.12.55.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:20 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 25/42] btrfs: pass delayed_refs_root to btrfs_delayed_ref_lock Date: Thu, 11 Oct 2018 15:54:14 -0400 Message-Id: <20181011195431.3441-26-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't need the trans except to get the delayed_refs_root, so just pass the delayed_refs_root into btrfs_delayed_ref_lock and call it a day. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 5 +---- fs/btrfs/delayed-ref.h | 2 +- fs/btrfs/extent-tree.c | 2 +- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 96ce087747b2..87778645bf4a 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -197,12 +197,9 @@ find_ref_head(struct rb_root *root, u64 bytenr, return NULL; } -int btrfs_delayed_ref_lock(struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock(struct btrfs_delayed_ref_root *delayed_refs, struct btrfs_delayed_ref_head *head) { - struct btrfs_delayed_ref_root *delayed_refs; - - delayed_refs = &trans->transaction->delayed_refs; lockdep_assert_held(&delayed_refs->lock); if (mutex_trylock(&head->mutex)) return 0; diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index 7769177b489e..ee636d7a710a 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -255,7 +255,7 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head * btrfs_find_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_refs, u64 bytenr); -int btrfs_delayed_ref_lock(struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock(struct btrfs_delayed_ref_root *delayed_refs, struct btrfs_delayed_ref_head *head); static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) { diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index ec8ede2b1dec..609f49c69c8d 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2600,7 +2600,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, /* grab the lock that says we are going to process * all the refs for this head */ - ret = btrfs_delayed_ref_lock(trans, locked_ref); + ret = btrfs_delayed_ref_lock(delayed_refs, locked_ref); spin_unlock(&delayed_refs->lock); /* * we may have dropped the spin lock to get the head From patchwork Thu Oct 11 19:54:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637339 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E90D157A for ; Thu, 11 Oct 2018 19:55:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D4F42C09B for ; Thu, 11 Oct 2018 19:55:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 118FC2C0A2; Thu, 11 Oct 2018 19:55:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B53242C09F for ; Thu, 11 Oct 2018 19:55:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727082AbeJLDYK (ORCPT ); Thu, 11 Oct 2018 23:24:10 -0400 Received: from mail-qt1-f173.google.com ([209.85.160.173]:43337 "EHLO mail-qt1-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYJ (ORCPT ); Thu, 11 Oct 2018 23:24:09 -0400 Received: by mail-qt1-f173.google.com with SMTP id q41-v6so11336205qtq.10 for ; Thu, 11 Oct 2018 12:55:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=X+PT81/Zp3zcYtUcLLi8M0fbP5w87s6IcVZs1kVoVUA=; b=JpFhFPSbLvkIgXUEZk5rcRryuq6R7zg44GGSlm0d3AafpFaqZ3HfNtb+GJ7HdNrp04 fhfznbSEXItaJK47iFYL3cD5MtAfmkrsN9jl91O8w1flkOOR6JYwihXBrSd0kEXwjW+S pg3wqUB03YtPbw6ntjL5Ozp0vBlqsw+0dI1rnDg17tSSGq8GlNV/un2YHDZ+XeLaV7TN jVLPCzk9cr4bDm3HCayhfN0wU+5W140bBOkaAlXC8Hd+SfYt6630vJCS7xtefrcuU/bF qO3YZIQDJyAlWgeZf8c1vbS8Kt08fGqz021UapLJ5ZlglmZlns+qFYdJbbRqx/l8IFbz Mogg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=X+PT81/Zp3zcYtUcLLi8M0fbP5w87s6IcVZs1kVoVUA=; b=IdcmLCBCGNv+yJits2XsiZcwjbt5N3af16eZuXzyat39Jzx3bItfooam39cfkzCzp+ eoD9RPMMOCFYihTtaBOJSqA89H5grLrgJkJFpAYbUGcYbx5xRe+Jf+UNmXr2wYuNTjT3 QiDSVhMaSF2Aemgk8qLyvccnu+s0fO4WOPucXl3GXCberl3tP71yJsY6YZVkPFmRPKn8 FSptDt+YUjObABog6nYVQ+HzXU0wUL0sJkDc2OlKug4lckT2CIwvcYKyJkZrwx9thTz/ 2aVzkDJ5PialYdRKhspYFZwZnFCacou4H7Lnaj4NmZ44RKVCSX+VRoC+Zl3LnSugY//q TbiQ== X-Gm-Message-State: ABuFfoiQgAMHISKYJfwIDkd0dKYE7PwBNQ85PG64yAcAI109TBJBZK9o tdni3zl/wu7fzDD7pYmbXcdBWg== X-Google-Smtp-Source: ACcGV62Ki5yRZA3PI3vpbkwmuZvVUCIXcXZ0rfDHsGXfTlU9WyXCmApMIwuLKrxRo8B5SFquXEznJw== X-Received: by 2002:ac8:ec3:: with SMTP id w3-v6mr3024168qti.138.1539287723036; Thu, 11 Oct 2018 12:55:23 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e2-v6sm10869159qkm.55.2018.10.11.12.55.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:22 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 26/42] btrfs: make btrfs_destroy_delayed_refs use btrfs_delayed_ref_lock Date: Thu, 11 Oct 2018 15:54:15 -0400 Message-Id: <20181011195431.3441-27-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We have this open coded in btrfs_destroy_delayed_refs, use the helper instead. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 39bd158466cd..121ab180a78a 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4214,16 +4214,9 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); - if (!mutex_trylock(&head->mutex)) { - refcount_inc(&head->refs); - spin_unlock(&delayed_refs->lock); - - mutex_lock(&head->mutex); - mutex_unlock(&head->mutex); - btrfs_put_delayed_ref_head(head); - spin_lock(&delayed_refs->lock); + if (btrfs_delayed_ref_lock(delayed_refs, head)) continue; - } + spin_lock(&head->lock); while ((n = rb_first(&head->ref_tree)) != NULL) { ref = rb_entry(n, struct btrfs_delayed_ref_node, From patchwork Thu Oct 11 19:54:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637341 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B7E7157A for ; Thu, 11 Oct 2018 19:55:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 893C32C09B for ; Thu, 11 Oct 2018 19:55:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7DC1D2C0A2; Thu, 11 Oct 2018 19:55:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31F662C09B for ; Thu, 11 Oct 2018 19:55:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727087AbeJLDYL (ORCPT ); Thu, 11 Oct 2018 23:24:11 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:45882 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYL (ORCPT ); Thu, 11 Oct 2018 23:24:11 -0400 Received: by mail-qt1-f194.google.com with SMTP id e10-v6so11332168qtq.12 for ; Thu, 11 Oct 2018 12:55:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=cvSqJmOxERWLztXjenJ/CRRumlDa3ZymappVew1ga8M=; b=bakCm/W7dzitw8SXDNZKH1WgGK+mvuyoB2OWMM4Dh3zT74C5AfJbfpH7LWHSe7rNhM RjvAQtqV/xv1WtTad4f7nyeoFksT6J2Iplw9cvE0jjPzuUPkFVhQugvMG7+u2n7G0Uwf hG4jFjWqyJRkucNw//U38BFDY0xDXxKl0ODr0Sj7QgXI+KZh8pmVXy1/G8XfzD7yxo1A c9GbJnryxxqzKmDXJR9N9U686mrjnsd/10g5I7G/rkKwQK83jleVq4FmpvIQ0jCmzhgQ 2S1jWbZlpcFv8CPgwUugEztAK2KDEHaXvUCQ6O7jR2O8YIzC8n1YnwdcAdbYSVmh8t1M JxqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=cvSqJmOxERWLztXjenJ/CRRumlDa3ZymappVew1ga8M=; b=BfBZn3tJlzhNhWNbsB1ZgUFIQx9Wa7P8t/0rXatP+/oaLpInwJ+dwoWVPnonY087Fb qCYaasLaM2qv4gx34tsaDHo2ad97ccjEBYIEn55m7waVfZUj0E2K/qfTI0MdK9xWM6f7 vBsEWzQ9AfQ3oJYsh+6AeM0+8LGVnf5+zJbVsEv52o/85+5OWE0M8ohRnZEV+ZOY2PW+ awHUKf71Ny7UPHsZiqXVrJfFYUO5AwyhIHSKltdymSgjERPUHyD9YZW4jYTZEXQLzM0t g6lithQStaSeyJ0jdi1ntKADj9vD3A8x3XScDtw/ICCJx2UudqOVvcuFKFM5wsXOUfcI rHPA== X-Gm-Message-State: ABuFfohikjnN4ucJuyfil/LLtKRKIxPz9Nw0QRmHFywYJr+gy+NP0JBo x9ZO3VpwXyUekemu0W4FfTytaA== X-Google-Smtp-Source: ACcGV60zGcToZZT86dT2a/zd2D/e7SbNSZNMKqHOPa8Ll82aXO+c7IsIeS4VyR+N2MYXNrUGk1MKoQ== X-Received: by 2002:a0c:9925:: with SMTP id h34-v6mr3089852qvd.81.1539287724802; Thu, 11 Oct 2018 12:55:24 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y28-v6sm9484497qtj.9.2018.10.11.12.55.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:23 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 27/42] btrfs: make btrfs_destroy_delayed_refs use btrfs_delete_ref_head Date: Thu, 11 Oct 2018 15:54:16 -0400 Message-Id: <20181011195431.3441-28-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of open coding this stuff use the helper instead. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 121ab180a78a..fe1f229320ef 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4232,12 +4232,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, if (head->must_insert_reserved) pin_bytes = true; btrfs_free_delayed_extent_op(head->extent_op); - delayed_refs->num_heads--; - if (head->processing == 0) - delayed_refs->num_heads_ready--; - atomic_dec(&delayed_refs->num_entries); - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); + btrfs_delete_ref_head(delayed_refs, head); spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); mutex_unlock(&head->mutex); From patchwork Thu Oct 11 19:54:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637343 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E56DD112B for ; Thu, 11 Oct 2018 19:55:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D371C2C09B for ; Thu, 11 Oct 2018 19:55:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C7C112C0A2; Thu, 11 Oct 2018 19:55:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 656912C09B for ; Thu, 11 Oct 2018 19:55:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727101AbeJLDYN (ORCPT ); Thu, 11 Oct 2018 23:24:13 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:38084 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYN (ORCPT ); Thu, 11 Oct 2018 23:24:13 -0400 Received: by mail-qt1-f195.google.com with SMTP id l9-v6so11357220qtf.5 for ; Thu, 11 Oct 2018 12:55:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=zIQ3GFsTodaoX0WP9NBZI59U9Zr/d1xI6Oori98CGvk=; b=sGu+DSqBzmhV9SJQQuQrTakrHBYnxnaKmfPEeD9It8LHGOMNaWYO0YItLehIEpuEel ktrBqAAnrFjO0hfLkQM2LwjFqV0iPfqCKCNxeNZKzE982M6/JZKxHXucaBsNaiEhBvvr xbZw5W8inPmnPu9Kql6KYsdg6IVpN9HiGsefeYBByonbEVvsdFm7wGEKkfrsj73u9xq0 mAeavjgpsPz0lDeZ3B98tVSRyUHWxElHNWbVFWmjM1eShLRF7f/TN68rde/LuYxp0pDI SOIDVbgW5iuPtF2jTsoUnIFY29wfyXfX6mT+uZdfjuqWHNtle3BtGjhHR3rGOkxhaPgB AOCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=zIQ3GFsTodaoX0WP9NBZI59U9Zr/d1xI6Oori98CGvk=; b=J6KurfDNWTe6ROAcKXu6VLu39iUUgB0kpeTWGZ3FKL3SOG1Q4QsiDXkXd0p4QH1XSu L7Lem1wEMBGzJBC4v4780uG0muquL+dapTfQiCQP+B4n9JOUqBV3wPEYpsbSlG+zTLh4 3RDoM5GIMpewn2elU+LaP4czu+zz4uqBhR4gZYWVoskA1GtUpNS7ehjoTWHq9fbljvW5 mSS1cnA3WQHrR5LqAxLCWoLH5oyRwAgsVqRqUS5dgOGymBXHJ2cY44AWlcjupMt3EHpO JMy0BswnHXz9JvsuyqKtyRAZ8yc4LoAPcICUuYnKuEMvCcP+XAPXBEMg8deo14BrX3qR oHnw== X-Gm-Message-State: ABuFfoge8d+NN3q0lFLA30fZR0BVZOx1YrVQEfPrUy3sKlh2Qpul10Ec 9581WgCvTq7ETh6ULC8lrzunFbf5zs4= X-Google-Smtp-Source: ACcGV60aD8J7hmyepG5uTFYrmjYXdjtFpWnFhmw1o1UDsJDfl869fWhz5pqfaS4iuNQl4jiiHcAKEQ== X-Received: by 2002:aed:2867:: with SMTP id r94-v6mr3039654qtd.112.1539287726521; Thu, 11 Oct 2018 12:55:26 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m71-v6sm13170311qke.71.2018.10.11.12.55.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:25 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 28/42] btrfs: handle delayed ref head accounting cleanup in abort Date: Thu, 11 Oct 2018 15:54:17 -0400 Message-Id: <20181011195431.3441-29-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We weren't doing any of the accounting cleanup when we aborted transactions. Fix this by making cleanup_ref_head_accounting global and calling it from the abort code, this fixes the issue where our accounting was all wrong after the fs aborts. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 5 +++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/extent-tree.c | 13 ++++++------- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 29db902511c1..e40356ca0295 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -35,6 +35,7 @@ struct btrfs_trans_handle; struct btrfs_transaction; struct btrfs_pending_snapshot; +struct btrfs_delayed_ref_root; extern struct kmem_cache *btrfs_trans_handle_cachep; extern struct kmem_cache *btrfs_bit_radix_cachep; extern struct kmem_cache *btrfs_path_cachep; @@ -2623,6 +2624,10 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long count); int btrfs_async_run_delayed_refs(struct btrfs_fs_info *fs_info, unsigned long count, u64 transid, int wait); +void +btrfs_cleanup_ref_head_accounting(struct btrfs_fs_info *fs_info, + struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head); int btrfs_lookup_data_extent(struct btrfs_fs_info *fs_info, u64 start, u64 len); int btrfs_lookup_extent_info(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info, u64 bytenr, diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index fe1f229320ef..54fbdc944a3f 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4240,6 +4240,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, if (pin_bytes) btrfs_pin_extent(fs_info, head->bytenr, head->num_bytes, 1); + btrfs_cleanup_ref_head_accounting(fs_info, delayed_refs, head); btrfs_put_delayed_ref_head(head); cond_resched(); spin_lock(&delayed_refs->lock); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 609f49c69c8d..3ca42f4cd462 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2475,12 +2475,11 @@ static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans, return ret ? ret : 1; } -static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, - struct btrfs_delayed_ref_head *head) +void +btrfs_cleanup_ref_head_accounting(struct btrfs_fs_info *fs_info, + struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head) { - struct btrfs_fs_info *fs_info = trans->fs_info; - struct btrfs_delayed_ref_root *delayed_refs = - &trans->transaction->delayed_refs; int nr_items = 1; if (head->total_ref_mod < 0) { @@ -2558,7 +2557,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } - cleanup_ref_head_accounting(trans, head); + btrfs_cleanup_ref_head_accounting(fs_info, delayed_refs, head); trace_run_delayed_ref_head(fs_info, head, 0); btrfs_delayed_ref_unlock(head); @@ -7227,7 +7226,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (head->must_insert_reserved) ret = 1; - cleanup_ref_head_accounting(trans, head); + btrfs_cleanup_ref_head_accounting(trans->fs_info, delayed_refs, head); mutex_unlock(&head->mutex); btrfs_put_delayed_ref_head(head); return ret; From patchwork Thu Oct 11 19:54:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637345 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69686157A for ; Thu, 11 Oct 2018 19:55:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5576F2C09B for ; Thu, 11 Oct 2018 19:55:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 49F922C0A2; Thu, 11 Oct 2018 19:55:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6C6E2C09B for ; Thu, 11 Oct 2018 19:55:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727123AbeJLDYP (ORCPT ); Thu, 11 Oct 2018 23:24:15 -0400 Received: from mail-qt1-f175.google.com ([209.85.160.175]:44596 "EHLO mail-qt1-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYP (ORCPT ); Thu, 11 Oct 2018 23:24:15 -0400 Received: by mail-qt1-f175.google.com with SMTP id c56-v6so11329632qtd.11 for ; Thu, 11 Oct 2018 12:55:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=0W52/LLOm7ykBXZDSlAQ977l0m2qfHMeXXMlb1zP6II=; b=MSlxF8gfkZAdET8AymXmSs949eL5Cqo0vITJiKS24/Zavq+47TLlTrb+0+0BFPzD8J H80kV/hqyaPw6vltB9jOM8rHMmCrMRdp+zBoyIxhrQ3wau4GLCz3QE/BAhPMoYgD4xja eyxITJEZkckVpFLJWDPxpx9PQ4GgnaG62sNd/kISqT010jMQ25XN9dLAc9kzBlyq/RIh BtBARZhD4BQxGtpKqEd5sN6Ri1iL+ZGn2JXi4PvDuGpX00LRr7qgoicM14HHKdHvN2rV 16UyCdkZdpJApiYPh7GOr490jJN6g2GdhYcw5oY7r7P1ya7kqDRAYYsWMEbpChxcLN3m L9oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=0W52/LLOm7ykBXZDSlAQ977l0m2qfHMeXXMlb1zP6II=; b=eDvW/g2zQNaRZgFZ0sO75z5AeiZfhBq1eRwwRw+z8l2hThvU/QfFbHL/Vad6XYFWlW Mc/f6Gk/kwr1KZlu/swJZqFcjWHvU1tmk9Gj2QY+vIo374f3ox2uRaSkLnAzhx5p8L4Y iOWZIewsItU9qdcZggsP7wuwgcqk9WSCefsWsb9DW4/48aUAANKq3l7GQJk9+TC8gF1i 5nhPZwswz8UJB78KkISQNyJN/ghJ9TNy7kMGb7g312CQzwcPCZ22vRJ5wzlZBXzrCNde /QEuVD88etj1eTrIjdg7y/TbpKYFfhEK3r0mNKxE+KoVhf+L9o4uqWYN3mShbv7esF2l bbUg== X-Gm-Message-State: ABuFfoiwEbTU8dapepJKKVlDOpq6JXoUgac7nTILk4NajZtt7j0kTgnT AKf7rVtQCTXVECzwBfilCwS3EA== X-Google-Smtp-Source: ACcGV62FBTyI3UpWALCwPqLt+/IpPkPnm0qVdvLWfoj9IOphk39psPQ3Wyvq0gAQO0gjL8RCAgvg5w== X-Received: by 2002:ac8:1644:: with SMTP id x4-v6mr3058860qtk.67.1539287728318; Thu, 11 Oct 2018 12:55:28 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o7-v6sm15101388qkc.67.2018.10.11.12.55.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:27 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 29/42] btrfs: call btrfs_create_pending_block_groups unconditionally Date: Thu, 11 Oct 2018 15:54:18 -0400 Message-Id: <20181011195431.3441-30-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The first thing we do is loop through the list, this if (!list_empty()) btrfs_create_pending_block_groups(); thing is just wasted space. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 3 +-- fs/btrfs/transaction.c | 6 ++---- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 3ca42f4cd462..6fa7d19bcbb1 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2978,8 +2978,7 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, } if (run_all) { - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); spin_lock(&delayed_refs->lock); node = rb_first(&delayed_refs->href_root); diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 39a2bddb0b29..46ca775a709e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -846,8 +846,7 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); btrfs_trans_release_chunk_metadata(trans); @@ -1934,8 +1933,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing = 1; smp_wmb(); - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); if (!test_bit(BTRFS_TRANS_DIRTY_BG_RUN, &cur_trans->flags)) { int run_it = 0; From patchwork Thu Oct 11 19:54:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5C377112B for ; Thu, 11 Oct 2018 19:55:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49F742C09B for ; Thu, 11 Oct 2018 19:55:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3E4962C0A2; Thu, 11 Oct 2018 19:55:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF0E22C09B for ; Thu, 11 Oct 2018 19:55:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727129AbeJLDYR (ORCPT ); Thu, 11 Oct 2018 23:24:17 -0400 Received: from mail-qk1-f180.google.com ([209.85.222.180]:45805 "EHLO mail-qk1-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYQ (ORCPT ); Thu, 11 Oct 2018 23:24:16 -0400 Received: by mail-qk1-f180.google.com with SMTP id m8-v6so6267878qka.12 for ; Thu, 11 Oct 2018 12:55:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=U3FEzmZTOlWuf95OZRDugO+ySzVMXr2hm7FmI8EAbWE=; b=ODOJisoEVAukyhSlaltYmP12vd7QIeNsbIKOtSPRhVrMKnTQ8g4frDrdd/zsu9H3Dr C8rY0TkU5Q11VWACjb1FnF7Ge/XTufszZUNAOPyD8FX8gK7LME2KlP7tGS9LmRQyXl4u fcYS9E7rWk2BKOrU5IDXoNSSAjckKkFvC+9OG+eBAGzjnZS66Zrjl2txh3AhLP60S4Fk ioSX1rquuT14Ruc2mKiZjOZNKXkX6CI+BU6RFzgvZ1uZ/Oz+YDS9VihkXSiF0A2/Qacm 7siQYC7ymYsIfu9L5+IPwpitZ4xg61mZVwv2ynQuD9Yi8+Q9HhTyvS+B1Cq1QN38xXD5 9eQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=U3FEzmZTOlWuf95OZRDugO+ySzVMXr2hm7FmI8EAbWE=; b=CUDZarfrQ0Llvx/k3PRFC+iT0lSk4ZCQ7oAUtessDV40qGN7og6O1vYpQF/tNgREvK 9bm8YJPoJEeZTq8+wXRkV00fzN1vDiLLndLJi90fSy5tNksZyBmkwOras5vzQ5chEutt 2CtKK+kMIGbOKqa0xsvQSDcKC/acxIFJeVfSRAx0s5TUOXKsGE3bHuk63yUBYkR91hXn nuBHOPPJwhMJk0Qq0211vT9SR8n8vsiDTW0Kw82qe/jfsy8J94efR40nJQcK6kNeMMPw 4hOy6eWABlnfa1pas4m7Aehqv5KR9wJdtw4uwtNgF8yXxpef9YWztGGpXXuTwhHaTp6v 9GUw== X-Gm-Message-State: ABuFfogZe0X7BsO2KSJbNtxSrLNNisPPkjgqubX6fkcupD1epjiJ/36u m44MKenB6BUGlutAZb56z3DHGqW4qaI= X-Google-Smtp-Source: ACcGV60eR2h08A3V4ohEg336J0XGBYu6QU0e0t7M7s/86WS7HraQz7F4Q0pVMkICsHdnPBxreuHIRQ== X-Received: by 2002:a37:9dd5:: with SMTP id g204-v6mr2965236qke.41.1539287730187; Thu, 11 Oct 2018 12:55:30 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id x34-v6sm14373573qth.86.2018.10.11.12.55.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:29 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 30/42] btrfs: just delete pending bgs if we are aborted Date: Thu, 11 Oct 2018 15:54:19 -0400 Message-Id: <20181011195431.3441-31-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We still need to do all of the accounting cleanup for pending block groups if we abort. So set the ret to trans->aborted so if we aborted the cleanup happens and everybody is happy. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 6fa7d19bcbb1..7e66a6351aad 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -10363,9 +10363,15 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) struct btrfs_root *extent_root = fs_info->extent_root; struct btrfs_block_group_item item; struct btrfs_key key; - int ret = 0; + int ret; bool can_flush_pending_bgs = trans->can_flush_pending_bgs; + /* + * If we aborted the transaction with pending bg's we need to just + * cleanup the list and carry on. + */ + ret = trans->aborted; + trans->can_flush_pending_bgs = false; while (!list_empty(&trans->new_bgs)) { block_group = list_first_entry(&trans->new_bgs, From patchwork Thu Oct 11 19:54:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637349 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD054112B for ; Thu, 11 Oct 2018 19:55:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B99062C09B for ; Thu, 11 Oct 2018 19:55:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AE4B62C0A2; Thu, 11 Oct 2018 19:55:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B1742C09B for ; Thu, 11 Oct 2018 19:55:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727148AbeJLDYS (ORCPT ); Thu, 11 Oct 2018 23:24:18 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:36382 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYS (ORCPT ); Thu, 11 Oct 2018 23:24:18 -0400 Received: by mail-qt1-f193.google.com with SMTP id u34-v6so11347004qth.3 for ; Thu, 11 Oct 2018 12:55:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=esyabqCBu+9iDj6gymgoZZfIo7W+xlWCseOBiKeoQzQ=; b=yDiT1o+5fzw+4rnmqdumVE/VMjvjTp4ow5PoUTlsmN0o7qZhlW/MTuPVoDguXQAzxs YCOiDKcDIwy9iSnHEe2PMVcYa91id5G/GEWV+1PF4NjpWW5LKBIAK1vqZ7ubJHndo442 ZuIkftt79lAAjgyGoHVmjPIP51s2w0L7xIlO3P+XapGM7ETAyEzy44jDyHGrW2KMEtdb dxZy4XbKs35rXIT0RQBOUZ05nUjUFPn5jqjFqigT1IWU/e1xD7/zEcBak2iA5H8PWhy5 vLzyTuKm+aNBc+a6DSQLtSHgII/QSaOa54PoRwxolbszpnE7/4/wbUFN7fEXkqIQQE95 sTtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=esyabqCBu+9iDj6gymgoZZfIo7W+xlWCseOBiKeoQzQ=; b=WYzW6AmZ92NvXAaJ14T3QJrln8OGxtDvyzfHhhylt29BJNPUKptjqJvPLh/LG4qB3+ dmxH/CLMruBiwBeyZQEPpnY1IadUKb3PboCmATy/uNV/yhOUwBcBmbiyHgy17TVzUXjS jAr7pwX+PVUpDQjxZjHB9pQzPyjqSPCCLRaeQp+Jbh/LhuXpFycEXh5dz4kbpcy1hwb5 hgLOqhV6ixYPH1laIWdE4BZgCaYGdxyFFRlaOzLOu7TPUroCE6SOzlQhxglU8RSZ4Nq4 yEGw7BuBpJi0I9Pl1IFWSohqVjuCBxtVfLxYmTKq/GMr4GeYPQ5EedxvEBnICa+B7izI 3uUg== X-Gm-Message-State: ABuFfohBbZ+HSg4OeZ1KjKVIare8B+Ounefb8avYEuxhJMRWcIKYPWLf QIIU1+rKQkGJSEmD++K0b5r2fw== X-Google-Smtp-Source: ACcGV61JYMFDZs3W+InRR9hwVp9tkCJEMdYGGqVNw1eEKWNoFhPhWxRrhXQcg+GMCMUoZSNF2/70hQ== X-Received: by 2002:aed:34e1:: with SMTP id x88-v6mr3074559qtd.156.1539287731904; Thu, 11 Oct 2018 12:55:31 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 144-v6sm14006883qkk.63.2018.10.11.12.55.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:31 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 31/42] btrfs: cleanup pending bgs on transaction abort Date: Thu, 11 Oct 2018 15:54:20 -0400 Message-Id: <20181011195431.3441-32-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We may abort the transaction during a commit and not have a chance to run the pending bgs stuff, which will leave block groups on our list and cause us accounting issues and leaked memory. Fix this by running the pending bgs when we cleanup a transaction. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 46ca775a709e..9168efaca37e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -2280,6 +2280,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_scrub_continue(fs_info); cleanup_transaction: btrfs_trans_release_metadata(trans); + /* This cleans up the pending block groups list properly. */ + if (!trans->aborted) + trans->aborted = ret; + btrfs_create_pending_block_groups(trans); btrfs_trans_release_chunk_metadata(trans); trans->block_rsv = NULL; btrfs_warn(fs_info, "Skipping commit of aborted transaction."); From patchwork Thu Oct 11 19:54:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637351 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2797157A for ; Thu, 11 Oct 2018 19:55:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF50B2C09B for ; Thu, 11 Oct 2018 19:55:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A3D032C0A2; Thu, 11 Oct 2018 19:55:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 498122C09B for ; Thu, 11 Oct 2018 19:55:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727160AbeJLDYU (ORCPT ); Thu, 11 Oct 2018 23:24:20 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:36392 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726831AbeJLDYU (ORCPT ); Thu, 11 Oct 2018 23:24:20 -0400 Received: by mail-qt1-f195.google.com with SMTP id u34-v6so11347182qth.3 for ; Thu, 11 Oct 2018 12:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=OTZOuDgDt/R5spujlNFzi4h/V31eI1viGWGcVEUBX/Y=; b=iZxiPsgPRItyhWi8ZxSgaHbF0t6l8SViKHUR3QLZOC0R86LrRBtpg1zlLUtpGdyBEL nlVNL6az2gBGpDR4EBGRbuWemojo92PrrLdL8dM408UZAbep3gOnGdIYqNMuMS08Emp+ xYc6Wmp8yM7bya/JRvCyfsQjy2nPMvM/9aUMbTgQJDa771XZXFGLPUCV5r4zWW2U45ET sDmBz8H5p0E2LLZnZueKpuSDL50XsM3yhNWYnhwKrkMSWcctv/03Lwje2lVc4zwvBwBy aJ7zaiqW5UktLWeLmWocxof6ZnpwQ4MKr6FUxec7QFyJq5kFhgdiNKI9m02UWQqSpMwa R7fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=OTZOuDgDt/R5spujlNFzi4h/V31eI1viGWGcVEUBX/Y=; b=K0c2cxqqU+kiE6tvF+05XwHcoBrumKqmk1UjYX5wOXarMHEqg4SfDwM6jBJiDZsd1F Af0rvi17d3xrUmbpYfd5Dno8KPULiOohJQ0Aov5oEnLOSlDt1AbZc3mxFkQMS6EDZ1ha jlNYB3Wbj5w4MHrC2SU/+eMpd03NXuaO0N9IncVB2SOxuwZffngoEGap3vsDC5Kmj88E EHdntT03fX3AWNszZ4a831kftBKVlahTZoVsSFngwRAeB333PRxzXrHBLH8MxViU/GHU jPfNU/YgVsF9sWI7k028/U3pofTklhFP2tPHyIxOHI4/FhvYq/cLYspJJXjBTIP4gl9U RTUw== X-Gm-Message-State: ABuFfojSGW9ZqVjoQpZ6g4pwl5PLZ/Hu+Le3F5N9MbpLZ7BcRUSqMUkC aM1MFZ835PCMvdvAJArhamHNjg== X-Google-Smtp-Source: ACcGV634Co96nEp3xOiNXgHQh3rvqDDZCQxGqN72+ZWE4AMrxd4+0eVbaasg8u5Ug0Ki3tzTbivV4Q== X-Received: by 2002:ac8:248c:: with SMTP id s12-v6mr2954057qts.116.1539287733810; Thu, 11 Oct 2018 12:55:33 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id l43-v6sm17610599qtc.80.2018.10.11.12.55.32 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:32 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 32/42] btrfs: only free reserved extent if we didn't insert it Date: Thu, 11 Oct 2018 15:54:21 -0400 Message-Id: <20181011195431.3441-33-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When we insert the file extent once the ordered extent completes we free the reserved extent reservation as it'll have been migrated to the bytes_used counter. However if we error out after this step we'll still clear the reserved extent reservation, resulting in a negative accounting of the reserved bytes for the block group and space info. Fix this by only doing the free if we didn't successfully insert a file extent for this extent. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval Reviewed-by: Filipe Manana --- fs/btrfs/inode.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 5a91055a13b2..2b257d14bd3d 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -2992,6 +2992,7 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) bool truncated = false; bool range_locked = false; bool clear_new_delalloc_bytes = false; + bool clear_reserved_extent = true; if (!test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags) && @@ -3095,10 +3096,12 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) logical_len, logical_len, compress_type, 0, 0, BTRFS_FILE_EXTENT_REG); - if (!ret) + if (!ret) { + clear_reserved_extent = false; btrfs_release_delalloc_bytes(fs_info, ordered_extent->start, ordered_extent->disk_len); + } } unpin_extent_cache(&BTRFS_I(inode)->extent_tree, ordered_extent->file_offset, ordered_extent->len, @@ -3159,8 +3162,13 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) * wrong we need to return the space for this ordered extent * back to the allocator. We only free the extent in the * truncated case if we didn't write out the extent at all. + * + * If we made it past insert_reserved_file_extent before we + * errored out then we don't need to do this as the accounting + * has already been done. */ if ((ret || !logical_len) && + clear_reserved_extent && !test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags)) btrfs_free_reserved_extent(fs_info, From patchwork Thu Oct 11 19:54:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637353 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB176157A for ; Thu, 11 Oct 2018 19:55:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99D702C09B for ; Thu, 11 Oct 2018 19:55:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8EA642C0A2; Thu, 11 Oct 2018 19:55:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D8472C09B for ; Thu, 11 Oct 2018 19:55:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727166AbeJLDYW (ORCPT ); Thu, 11 Oct 2018 23:24:22 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:40085 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726786AbeJLDYW (ORCPT ); Thu, 11 Oct 2018 23:24:22 -0400 Received: by mail-qt1-f193.google.com with SMTP id b4-v6so11332737qtc.7 for ; Thu, 11 Oct 2018 12:55:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=xTZQcRzq0IzMsRcNbawv7mMyVU41r6nyXAFW5TJVAig=; b=PezPKDiOFOkTFR+xmFUjKxEKK3jL1aZoJKN9ps+QCnV1HxZfiJluJ/40fiWCw6H6US rTAzTfZtuFvkMW/JbH5+bPVCfkrmvGbx0GhGCfZlcIxAJocJ1U3UEylIFtVn41MEzukL 80EZ5AeMhcLhtGmRybEB6rq6loXGxAVWJBnwoS4J9dOhb+ff9mELiFR3niWqWdhmX7VF pLBcQAdUtOkLbZUjiomNxgNEtt8KC8sqfKgTkEhyQP15rVnpaAjWygs1HnTd+fFbNEUl af0rwnAW5neMDZ3lb/BkND1D1v2nr4WdBVVCyLzKMumQscQ8dT4pJmWdGp1pUJkwfaEd agWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=xTZQcRzq0IzMsRcNbawv7mMyVU41r6nyXAFW5TJVAig=; b=l9P7QaOUwJItXjQr9YX8U64ThLKskzl+vzqQV8nxLV4EqWNvtPCGCMOo5qIY3lLYqa uxFLi5Q99KTAqA6nkrOJNwJRGMfTcxwPNJF8oplOrnjTLmcsN36sg6hAimrO0xYMq7Z3 0Kb83RNdCPnRM9fu4Mwvbe4rFoTB0qtnYuQnCdw5p2W9roQiutraGoL5JSXGb1MjSKHQ mKuKJpEEx4s4Plrb9LYH7+N3p9cHSI+hLq5FAeAhVrj09TlM1Ta94BMDo+mPzSu03woK pKKVEoC8jOX1OUF6hDk/iOdrJSJOpoe8XVwgT+1ZPWSofbW4JRp6fkU3pj7euRsqJ5Je By4g== X-Gm-Message-State: ABuFfohu0UBm/znhgzJ9g4n03oTfSPa5iwdJgKaOFqSto/8S0fFVFU5z 8B1o56rrNQdNhotUO4liNYuyrg== X-Google-Smtp-Source: ACcGV62FPEWgnCaiO5lWNXprm0bQAsoaIN3QxW0JDDsS8FWwakddKQ6V4vrqgpzZxXTHMlVaETqfhg== X-Received: by 2002:ac8:34bc:: with SMTP id w57-v6mr3002914qtb.239.1539287735563; Thu, 11 Oct 2018 12:55:35 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id k1-v6sm14538356qte.76.2018.10.11.12.55.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:34 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 33/42] btrfs: fix insert_reserved error handling Date: Thu, 11 Oct 2018 15:54:22 -0400 Message-Id: <20181011195431.3441-34-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We were not handling the reserved byte accounting properly for data references. Metadata was fine, if it errored out the error paths would free the bytes_reserved count and pin the extent, but it even missed one of the error cases. So instead move this handling up into run_one_delayed_ref so we are sure that both cases are properly cleaned up in case of a transaction abort. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/extent-tree.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 7e66a6351aad..6174d1b7875b 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2405,6 +2405,9 @@ static int run_one_delayed_ref(struct btrfs_trans_handle *trans, insert_reserved); else BUG(); + if (ret && insert_reserved) + btrfs_pin_extent(trans->fs_info, node->bytenr, + node->num_bytes, 1); return ret; } @@ -8257,21 +8260,14 @@ static int alloc_reserved_tree_block(struct btrfs_trans_handle *trans, } path = btrfs_alloc_path(); - if (!path) { - btrfs_free_and_pin_reserved_extent(fs_info, - extent_key.objectid, - fs_info->nodesize); + if (!path) return -ENOMEM; - } path->leave_spinning = 1; ret = btrfs_insert_empty_item(trans, fs_info->extent_root, path, &extent_key, size); if (ret) { btrfs_free_path(path); - btrfs_free_and_pin_reserved_extent(fs_info, - extent_key.objectid, - fs_info->nodesize); return ret; } From patchwork Thu Oct 11 19:54:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637355 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29B12112B for ; Thu, 11 Oct 2018 19:55:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 182DC2C09B for ; Thu, 11 Oct 2018 19:55:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CF9B2C0A2; Thu, 11 Oct 2018 19:55:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEAA82C09B for ; Thu, 11 Oct 2018 19:55:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727180AbeJLDYY (ORCPT ); Thu, 11 Oct 2018 23:24:24 -0400 Received: from mail-qt1-f171.google.com ([209.85.160.171]:41216 "EHLO mail-qt1-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726786AbeJLDYX (ORCPT ); Thu, 11 Oct 2018 23:24:23 -0400 Received: by mail-qt1-f171.google.com with SMTP id l41-v6so11344407qtl.8 for ; Thu, 11 Oct 2018 12:55:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=o3aniGDG3zSjx1n1+LpysocBJ3oenkOpLCURB29A0Uc=; b=bViF86CAKld+3GCaFGw6VsagrED0khahvoZpu8hkBNPk8rc365CJUf7/4Pwf6VNJrg 70dqZCAfvvYvY2ioSk33LC507qWuj1b4jWl/DOFzLFWYSSwYTUrojYHY2Sha2AHunY1L pNxpUjKADA+lcLl2uM4KFAqWppHym+Yp7qPV0SsIm4IvMdlIk4ndGwCaSMZf1682XQc5 nQo9UirqLK3fYeQj4i4eqkryz+V+8xy2mx1h1O3DcHglture3YgGc7EqJ2L3MvuI057s dRX0XzIqEVDWhUKQiELguyrxtrXXnMkx3f1jKbtlrUPy+b+Km6ftmEjWjp3V7n4WbQWT Rkug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=o3aniGDG3zSjx1n1+LpysocBJ3oenkOpLCURB29A0Uc=; b=akstax8A/buwRp12/t+1N8ZTMRAoX54CpnZ9bVJS47G1xXVA9bXJ2GtaENZ3W3WwrT y9ijvauLrR1XTiTsW3UzovHYjTfw0STy9VpO47GU8JdTIF74zB7YUrj8LcE4aOou9Yfm E2DwmKhAE5pnw+8P8X6aNrP9lH4sDm75s6w395ns+GlJohHXzi5U6I7NOK4FBOYENNCM PgeQzKh4s2Y+z8WF6rDMpV0TGtcN+n+50v5MU9s6piWkPKmMEgtHu218Lam5xYOtnEtq NJp8xsr9AMRa56uFpVYSl37i1nmITwgCmEdZMSPOrbKVUpsw5v5jdh5RPxoItrY+4iDK E26w== X-Gm-Message-State: ABuFfojuhi7U7oEiB7blbU4iTOZsLs2Ptuf2PaJSmI3ONrP1dBjPlBWS v4cSo4ICLMwsFngdHCT7yPA0Rw== X-Google-Smtp-Source: ACcGV62O16I73fvmLDVlSN1JT8ifVX0h1nfSA9JZYVeE0e35kNtfDbtFNMlETRJ2WHTCrhPXBW+RdQ== X-Received: by 2002:ac8:7697:: with SMTP id g23-v6mr3035221qtr.117.1539287737376; Thu, 11 Oct 2018 12:55:37 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y20-v6sm13867785qkb.46.2018.10.11.12.55.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:36 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 34/42] btrfs: wait on ordered extents on abort cleanup Date: Thu, 11 Oct 2018 15:54:23 -0400 Message-Id: <20181011195431.3441-35-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we flip read-only before we initiate writeback on all dirty pages for ordered extents we've created then we'll have ordered extents left over on umount, which results in all sorts of bad things happening. Fix this by making sure we wait on ordered extents if we have to do the aborted transaction cleanup stuff. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 54fbdc944a3f..51b2a5bf25e5 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4188,6 +4188,14 @@ static void btrfs_destroy_all_ordered_extents(struct btrfs_fs_info *fs_info) spin_lock(&fs_info->ordered_root_lock); } spin_unlock(&fs_info->ordered_root_lock); + + /* + * We need this here because if we've been flipped read-only we won't + * get sync() from the umount, so we need to make sure any ordered + * extents that haven't had their dirty pages IO start writeout yet + * actually get run and error out properly. + */ + btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, (u64)-1); } static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, From patchwork Thu Oct 11 19:54:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637357 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E602A157A for ; Thu, 11 Oct 2018 19:55:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D52DB2C09B for ; Thu, 11 Oct 2018 19:55:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9E1C2C0A2; Thu, 11 Oct 2018 19:55:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 807E72C09B for ; Thu, 11 Oct 2018 19:55:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727188AbeJLDY0 (ORCPT ); Thu, 11 Oct 2018 23:24:26 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:41058 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726786AbeJLDYZ (ORCPT ); Thu, 11 Oct 2018 23:24:25 -0400 Received: by mail-qt1-f194.google.com with SMTP id l41-v6so11344519qtl.8 for ; Thu, 11 Oct 2018 12:55:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ja6BEdk2d172wcHDS/gitUITWM0Dn5OfC8ykxBklKgo=; b=F/CST1XXZHkqVInWQkziWJQl41RMcWsBLPSdrKigqWw4jo86/k+uWBM7o7jAuzziNF BjifsDR6M0TW+wH3Q0iExVAdcTVWTmzLGd/BjMkZEy6e4GYHkUdLQSfEnCxHzhZZMAnR HgEJv/5ZfKWaGS/52W4rmdqz3Wk9sDFK7FMW9c8KRTY0BLk1cUEv5oM/BRFKUCPYfTn+ pjC5xAXHIw9d4JqCbug2fk1afkTESdp7QDuNf8ssQKVIt+E0aK/WXzrPijVFPxEFwULU O3OZlSRAI28Ederh/AhCrkdvJgNgWphslSHrbAwQN9HNjp/tghIxnuYkfkAYic/EA5JJ iE5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ja6BEdk2d172wcHDS/gitUITWM0Dn5OfC8ykxBklKgo=; b=n+w2MCLscpqIcV6ZyndHUHx562lVbqc6LvaCzQNrAEStKDi8a5dyK4wzgt+tbTplWj GBYYugVR9GGJjDDPtSrqT2ISe+Qgs5CFCuhdMcvBwfoJpjCYL7wBFXGZFtdB21lCePAp XVw/G4T0pOJNZsCXvaD4n91SkCysnWz5fMmz8gPUfn4Ljp+mk/dya356HbCwnkwCzKMD 4eKTBmF/ipmV/VfqgsQ5nBoSzW0VxQkV62jDsgyOiDsvs4ApTfU3GBijgFIOkypu2Ixq ZlWzj92xSac6aJpuFz/q3l3Qun2FSA8B0jzyrzt+eYSIroSDGpAztKvb/rSc4em5LggU wImA== X-Gm-Message-State: ABuFfojsSDnt7lJ3vbMNS/lNCTNkoiXm4hTE4txXffn6Ha0SerohQvh9 jX1R4MUXwQQwsW88Hz3SYQ3KEoVS7ic= X-Google-Smtp-Source: ACcGV61xGLhxTMF+f1wCozsREz+YclrMF2OuBR6rsRiD6VtgENnrHAPWLaw4Q9+NhlFs+w5hRXhgBg== X-Received: by 2002:ac8:b0d:: with SMTP id e13-v6mr2996643qti.375.1539287739191; Thu, 11 Oct 2018 12:55:39 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o7-v6sm15101716qkc.67.2018.10.11.12.55.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:38 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 35/42] MAINTAINERS: update my email address for btrfs Date: Thu, 11 Oct 2018 15:54:24 -0400 Message-Id: <20181011195431.3441-36-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP My work email is completely useless, switch it to my personal address so I get emails on a account I actually pay attention to. Signed-off-by: Josef Bacik --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index 32fbc6f732d4..7723dc958e99 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3095,7 +3095,7 @@ F: drivers/gpio/gpio-bt8xx.c BTRFS FILE SYSTEM M: Chris Mason -M: Josef Bacik +M: Josef Bacik M: David Sterba L: linux-btrfs@vger.kernel.org W: http://btrfs.wiki.kernel.org/ From patchwork Thu Oct 11 19:54:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637359 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F24FA157A for ; Thu, 11 Oct 2018 19:55:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E097B2C09B for ; Thu, 11 Oct 2018 19:55:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D530C2C0A2; Thu, 11 Oct 2018 19:55:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69D352C09B for ; Thu, 11 Oct 2018 19:55:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727194AbeJLDY1 (ORCPT ); Thu, 11 Oct 2018 23:24:27 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:43913 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726786AbeJLDY1 (ORCPT ); Thu, 11 Oct 2018 23:24:27 -0400 Received: by mail-qt1-f193.google.com with SMTP id q41-v6so11337394qtq.10 for ; Thu, 11 Oct 2018 12:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=91liZIK+wxIF7dRdH1otjGSMGHeQwahR4ZLfPo8AmG8=; b=v/TtigltNv8iu0A4OOER5xeVDELJDu+ylZUhS/Y0Eo2MtY+2ODahH2ZKfEj6ZcnxrF 35S2HvDSeyPIKhAqV2B76rV1G0AgCgc2JdeWLVSNnWi2z6sfNHaOxM8bjbeBlcQfZHtj XMqddfcf0YENay4onUheOzOCQotpxnKuNkdSnXBk3MgTjxXqOPvIFbp6A1KIa7qXp2DA RPoKnjEIobPNYAwDsg3OeM7yEEigBD7fObqPvR91oKbnDXXrYTDpz5Z/+LfySXzOiznJ 4DEUwq68qgUor/bGVjYkd+AN5FZdQmtMXi/XUo5fPvuQYxIdEbvS0yyq/QPHL7yzpyQd WbQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=91liZIK+wxIF7dRdH1otjGSMGHeQwahR4ZLfPo8AmG8=; b=AJGS9d3g7Wmnmwfy1JAFpkIbx/syRd92VNzIVWKKIsXku3vXJ9e/1OADAyQoL4j9It tuqncgMrS1XdD9sDO8WFknuMjLMo0zhRbWfYvBOcvNZ07/qyAcf/f8GG8Qd8iwM6/pEY L9Ry2KNsSgTPhcKixEBGS5IcFtjomU50/Y2S749FgrIuOUqvnDsPJAuSacCaTnIAMnMF voC/JzZ9K239fVxMZs2NRIECCGeSV28mCzzU4FROTqIZCUVD0YUWPIpQM9urThC/d52l h1rqzwYpQfA03jLdfKczl0jjKIaRoY+y3NUoE5PYFMHJv3QTitR/IvlTvbZ0456LaV5d y00Q== X-Gm-Message-State: ABuFfohV7jPrQNhsZ35YfWkK1Na33c6/ciDnnR/z6D7lJi7iBmaRdHEZ RpJAtCnuxESzgE2y41hqn3RxAQ== X-Google-Smtp-Source: ACcGV62d8xlbIahJXMdbeEBhKdCju+vZsZBjE2S7VmmaXocH0iZxSSCbSNC8UDdrqKO0k7TYWm9nPQ== X-Received: by 2002:aed:25d1:: with SMTP id y17-v6mr2954267qtc.217.1539287741071; Thu, 11 Oct 2018 12:55:41 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s17-v6sm16895967qtj.31.2018.10.11.12.55.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:40 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 36/42] btrfs: wait on caching when putting the bg cache Date: Thu, 11 Oct 2018 15:54:25 -0400 Message-Id: <20181011195431.3441-37-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is because we aren't waiting for the caching threads to be done before freeing everything up, so to fix this make sure we wait on any outstanding caching that's being done before we free up the block group, so we're sure to be done with all IO by the time we get to btrfs_stop_all_workers(). This fixes the panic I was seeing consistently in testing. ------------[ cut here ]------------ kernel BUG at fs/btrfs/volumes.c:6112! SMP PTI Modules linked in: CPU: 1 PID: 27165 Comm: kworker/u4:7 Not tainted 4.16.0-02155-g3553e54a578d-dirty #875 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014 Workqueue: btrfs-cache btrfs_cache_helper RIP: 0010:btrfs_map_bio+0x346/0x370 RSP: 0000:ffffc900061e79d0 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff880071542e00 RCX: 0000000000533000 RDX: ffff88006bb74380 RSI: 0000000000000008 RDI: ffff880078160000 RBP: 0000000000000001 R08: ffff8800781cd200 R09: 0000000000503000 R10: ffff88006cd21200 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: ffff8800781cd200 R15: ffff880071542e00 FS: 0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000817ffc4 CR3: 0000000078314000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: btree_submit_bio_hook+0x8a/0xd0 submit_one_bio+0x5d/0x80 read_extent_buffer_pages+0x18a/0x320 btree_read_extent_buffer_pages+0xbc/0x200 ? alloc_extent_buffer+0x359/0x3e0 read_tree_block+0x3d/0x60 read_block_for_search.isra.30+0x1a5/0x360 btrfs_search_slot+0x41b/0xa10 btrfs_next_old_leaf+0x212/0x470 caching_thread+0x323/0x490 normal_work_helper+0xc5/0x310 process_one_work+0x141/0x340 worker_thread+0x44/0x3c0 kthread+0xf8/0x130 ? process_one_work+0x340/0x340 ? kthread_bind+0x10/0x10 ret_from_fork+0x35/0x40 Code: ff ff 48 8b 4c 24 28 48 89 de 48 8b 7c 24 08 e8 d1 e5 04 00 89 c3 e9 08 ff ff ff 4d 89 c6 49 89 df e9 27 fe ff ff e8 5a 3a bb ff <0f> 0b 0f 0b e9 57 ff ff ff 48 8b 7c 24 08 4c 89 f9 4c 89 ea 48 RIP: btrfs_map_bio+0x346/0x370 RSP: ffffc900061e79d0 ---[ end trace 827eb13e50846033 ]--- Kernel panic - not syncing: Fatal exception Kernel Offset: disabled ---[ end Kernel panic - not syncing: Fatal exception Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 6174d1b7875b..4b74d8a97f7c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -9894,6 +9894,7 @@ void btrfs_put_block_group_cache(struct btrfs_fs_info *info) block_group = btrfs_lookup_first_block_group(info, last); while (block_group) { + wait_block_group_cache_done(block_group); spin_lock(&block_group->lock); if (block_group->iref) break; From patchwork Thu Oct 11 19:54:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637361 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE962112B for ; Thu, 11 Oct 2018 19:55:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD1552C09B for ; Thu, 11 Oct 2018 19:55:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C16D62C0A2; Thu, 11 Oct 2018 19:55:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 738D42C09B for ; Thu, 11 Oct 2018 19:55:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727215AbeJLDY3 (ORCPT ); Thu, 11 Oct 2018 23:24:29 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:37663 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726786AbeJLDY3 (ORCPT ); Thu, 11 Oct 2018 23:24:29 -0400 Received: by mail-qk1-f196.google.com with SMTP id x8-v6so6281747qka.4 for ; Thu, 11 Oct 2018 12:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=zZszbBJ2wYqnRaCO5pCUOw7tHC5U99aK2TNTpFeSZ9Q=; b=nBDCdAUw2Q0vveKaqzIG6aPL4eSDLhbWZWC8o2o9qvuYTE8wO/YzylvlKP0y+rse8M Npunmex3nMXrmgiUZ0qPNp6uUYUAyP9ZBHM48c//VPIjyLl2FfADll8KRaaRYTkwOKS9 6UG7uZoy0hwFzYWwjMVKCDmroB/purcO1lnxNKTE+TjQlmd4eQ/Q3pD59jwyIetQKgwI L26UJ6YcyDqeccRth1UBHEH81DV1iCmYhl6LTuAnwaFU71bcVpt+eY84h059Sq/1f2n8 G+lheiY6zR2oln9RVl76GANaO0bRC6M+WdX6VVvXWfZp0zsQIfnK116eleVZ/rwpt8Sb zxbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=zZszbBJ2wYqnRaCO5pCUOw7tHC5U99aK2TNTpFeSZ9Q=; b=Tv6AAqm65cpEUND57jte3vHOdFb2KQGYYm0zxn6tO8hnMVVRuHUw5cLgOfLppiccgO 3KY21c+o/aE7qmq3km0vQc5yzwjPemFDUzHFIbeLubjf9GOAbIg3rmHRrG7WhUbKcrot 1JFRSLPwV5jX9TBK+iWXLaVAv9075iKBwhFM59vRUzhimsFkWK9LA/Zg6eGDQjXnIayW OZHT1W+UzQ4ItQtrkhyc6vqUEWhOS7EsvAxBa3pTwJOD59EGUDdrAApyUL209J2N2Prr lM/MqJnB0jktJWEdCVJVsaXbhkjOWoqULHG0O/HJ6W0vPQUToWG3pUNc5Wa1dP/2Pz0Z YDyg== X-Gm-Message-State: ABuFfohY9f7WMkNncGx12+3InuEQXP0qdNev8+xG/shi5+RRGXwa8Ufs 3cmWA6HELJ9dOpGOFlpkblk46HddSNo= X-Google-Smtp-Source: ACcGV63lQ7KJVLOL5Fo3+8fXiJi218vxi2wuRm8D5yRB7JuBFg9x5/9wHNKkN1DUKJI96jqqOD7Edw== X-Received: by 2002:a37:c03:: with SMTP id 3-v6mr2825227qkm.51.1539287743005; Thu, 11 Oct 2018 12:55:43 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m64-v6sm13210156qkf.18.2018.10.11.12.55.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:42 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 37/42] btrfs: wakeup cleaner thread when adding delayed iput Date: Thu, 11 Oct 2018 15:54:26 -0400 Message-Id: <20181011195431.3441-38-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The cleaner thread usually takes care of delayed iputs, with the exception of the btrfs_end_transaction_throttle path. The cleaner thread only gets woken up every 30 seconds, so instead wake it up to do it's work so that we can free up that space as quickly as possible. Reviewed-by: Filipe Manana Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 2b257d14bd3d..0a1671fb03bf 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -3323,6 +3323,7 @@ void btrfs_add_delayed_iput(struct inode *inode) ASSERT(list_empty(&binode->delayed_iput)); list_add_tail(&binode->delayed_iput, &fs_info->delayed_iputs); spin_unlock(&fs_info->delayed_iput_lock); + wake_up_process(fs_info->cleaner_kthread); } void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info) From patchwork Thu Oct 11 19:54:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637363 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C722C157A for ; Thu, 11 Oct 2018 19:55:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B72602C09B for ; Thu, 11 Oct 2018 19:55:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ABA8A2C0A2; Thu, 11 Oct 2018 19:55:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 555D32C09B for ; Thu, 11 Oct 2018 19:55:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727222AbeJLDYb (ORCPT ); Thu, 11 Oct 2018 23:24:31 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:45949 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726786AbeJLDYb (ORCPT ); Thu, 11 Oct 2018 23:24:31 -0400 Received: by mail-qt1-f195.google.com with SMTP id e10-v6so11333437qtq.12 for ; Thu, 11 Oct 2018 12:55:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=cKf7nfGjyX5ciBrLw5SDFaPPqW28rfuFK6izaRjmL0w=; b=O3e1h9QeEZSMKxzl2IioVQNgr9MDFJXM7VNh7ITcvLQI/gGWHgOjANZox0Tma9xyKs ZDtyhC1BWsiDePBIPWnAVkkWQdOv5oG5WoKRjWMSWiQ3tPsvIwbXHJk897IPwTFyv1cQ ghxqT5uvs0r2S5zxBK4OrFboijrIu+TSj/9ouUCqsJoaG9oqk0M7p+Dfqp0FLcYnfGkb gxAh91XNm+uIMD252XTnZAaGrlfp/cLJj99f3K0bAS+LWEcl838P+hX3izAE8Q4kxV/V 0UazdBcDk/Y4xIFqxsmRmp02lNV0OQZor9lWyXL77sLxhoZEfJ1ED5Ga/I+KhQKnedY3 LjbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=cKf7nfGjyX5ciBrLw5SDFaPPqW28rfuFK6izaRjmL0w=; b=R65ReoQilgiQC9Ty8I+mPd+cszKyRoLJxU8pgY6nSS9Fll8YgiVbC0Tby/afC5id85 y25sgi+9qnTSiez2IL3jeovo9zvuXhZHyK5YfEke+EwN5w6EHMMDFZPR7AXGCU/Bd82A dDuKmcOQzkTWbI0/awJgTOx58syR+ve049UU4lF4rRN8gCv2B3yxsdF3XbKwgA52EtlE Kh9xj2CZuKl0PGF54d7DX79xZpA+DyGKlE+UZIwl/+gvBf8bUoqrUDmvas6XHZ7dgu6X yQ73EXWLr0+4mfKuIGapzrLFKA7CKPobCBA4ZtHToYabvQmL+wSoyJnjMQvgA4GtetW/ QIOw== X-Gm-Message-State: ABuFfoibS0CQ49YEGdOQyPXVD5c98t41bX6Acchr2XwD4ckzwRmrQGb7 40LZEoXJMGDm0vgkiK+tYlgiTw== X-Google-Smtp-Source: ACcGV633GscVhDMaHVSSgp1X4zd8YHjFiaIkipY/lQuo6a7ddT00olXh6GbiEdQkIRDDHaRG6caGSg== X-Received: by 2002:ac8:2595:: with SMTP id e21-v6mr3006957qte.233.1539287744917; Thu, 11 Oct 2018 12:55:44 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id z24-v6sm14286162qtg.42.2018.10.11.12.55.43 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:43 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 38/42] btrfs: be more explicit about allowed flush states Date: Thu, 11 Oct 2018 15:54:27 -0400 Message-Id: <20181011195431.3441-39-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For FLUSH_LIMIT flushers we really can only allocate chunks and flush delayed inode items, everything else is problematic. I added a bunch of new states and it lead to weirdness in the FLUSH_LIMIT case because I forgot about how it worked. So instead explicitly declare the states that are ok for flushing with FLUSH_LIMIT and use that for our state machine. Then as we add new things that are safe we can just add them to this list. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 4b74d8a97f7c..be18b40d2d48 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5110,12 +5110,18 @@ void btrfs_init_async_reclaim_work(struct work_struct *work) INIT_WORK(work, btrfs_async_reclaim_metadata_space); } +static const enum btrfs_flush_state priority_flush_states[] = { + FLUSH_DELAYED_ITEMS_NR, + FLUSH_DELAYED_ITEMS, + ALLOC_CHUNK, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) { u64 to_reclaim; - int flush_state = FLUSH_DELAYED_ITEMS_NR; + int flush_state = 0; spin_lock(&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info, @@ -5127,7 +5133,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, + priority_flush_states[flush_state]); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) { @@ -5135,15 +5142,7 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, return; } spin_unlock(&space_info->lock); - - /* - * Priority flushers can't wait on delalloc without - * deadlocking. - */ - if (flush_state == FLUSH_DELALLOC || - flush_state == FLUSH_DELALLOC_WAIT) - flush_state = ALLOC_CHUNK; - } while (flush_state < COMMIT_TRANS); + } while (flush_state < ARRAY_SIZE(priority_flush_states)); } static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Thu Oct 11 19:54:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637365 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F044112B for ; Thu, 11 Oct 2018 19:55:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BEF42C09B for ; Thu, 11 Oct 2018 19:55:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 407A02C0A2; Thu, 11 Oct 2018 19:55:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B34872C09B for ; Thu, 11 Oct 2018 19:55:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727233AbeJLDYe (ORCPT ); Thu, 11 Oct 2018 23:24:34 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34510 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726697AbeJLDYd (ORCPT ); Thu, 11 Oct 2018 23:24:33 -0400 Received: by mail-qt1-f195.google.com with SMTP id o17-v6so11375221qtr.1 for ; Thu, 11 Oct 2018 12:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=/30ZTKbU29s7YJsoPZYEMnXg6WTSYKSBz6kyYmrLk4g=; b=PumpNAogdtjLajrjdAKuxbCCNWIlvPhBowv4l2E9WX+nXZ43rXq7XJichZeCuR1tyN 5CTHS73XNDIdlJjorcEXys+Mm1CPiTuSFqbh4m7nt77bLBokXnxY5ghFEzl9GJBh+ZH+ 9bxW7hjZi39lAfv0f+uYjdUCHI7K92xm9uP3rQKKDPyrymndZnMyqnU+pXeAvlSoi1m3 TnhMFwlnnZe0wtJmJ/nR+RA8wYg9HHaWu5EbHJSvAUVjx8CmWlR5nN7QN2HkP2nkMRS2 pIATCYBSYHX38fIvDcp46rCvR3IIyANQp7btK9nIW76++8tRgQOQVTTiXoLX9xISfF6+ 09WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=/30ZTKbU29s7YJsoPZYEMnXg6WTSYKSBz6kyYmrLk4g=; b=q2onfYz9Bd2JQj86eiBx2C6aZ/RImj8At0HPWlHuV9xzzLLjSbe5XpI5gEt7eKGUph VHlpYEbSDjWSD3y2umWqRTzhq7tebZRbuCwxFzjuAgN1r9tO3JH64dLJz0yVeB1/yJwe ngvnNaKZkkwiaA9OTn5UFwYOa5z+bXaC/ZppLGrgpnOZ/cXwg8tnSDWfBMO6hI7XU4lC E1UPgMdT67PVLV9kS25SL0DOobll81/CDU+8fHUGwVH/BIVpxR6o2rLzJwAUzROXpht8 6K2yNhl9q1GflV8bkohVagb4eOBFsSyqkwX+T8d9vz+oZlNDZb9dw6g4K7ZqbT8RUqgm e6BQ== X-Gm-Message-State: ABuFfogeKNJyTsVCISKvFh7JpaLN+jjM7jEAsqz6/QRtnr8j5PPX5H7X S7SM0YYKPRFAxk2fwW4DWA+WZw== X-Google-Smtp-Source: ACcGV63j3eaZMm8xh9zphB7z/lS1mHhqcRzBlAWfohj0L91nMFe3PlUCdPWCfrqhERjn0cgAkiIP4w== X-Received: by 2002:a0c:b5d3:: with SMTP id o19-v6mr3096892qvf.218.1539287746742; Thu, 11 Oct 2018 12:55:46 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o96-v6sm4524691qte.22.2018.10.11.12.55.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:45 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 39/42] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue Date: Thu, 11 Oct 2018 15:54:28 -0400 Message-Id: <20181011195431.3441-40-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The throttle path doesn't take cleaner_delayed_iput_mutex, which means we could think we're done flushing iputs in the data space reservation path when we could have a throttler doing an iput. There's no real reason to serialize the delayed iput flushing, so instead of taking the cleaner_delayed_iput_mutex whenever we flush the delayed iputs just replace it with an atomic counter and a waitqueue. This removes the short (or long depending on how big the inode is) window where we think there are no more pending iputs when there really are some. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 4 +++- fs/btrfs/disk-io.c | 5 ++--- fs/btrfs/extent-tree.c | 9 +++++---- fs/btrfs/inode.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index e40356ca0295..1ef0b1649cad 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -894,7 +894,8 @@ struct btrfs_fs_info { spinlock_t delayed_iput_lock; struct list_head delayed_iputs; - struct mutex cleaner_delayed_iput_mutex; + atomic_t nr_delayed_iputs; + wait_queue_head_t delayed_iputs_wait; /* this protects tree_mod_seq_list */ spinlock_t tree_mod_seq_lock; @@ -3212,6 +3213,7 @@ int btrfs_orphan_cleanup(struct btrfs_root *root); int btrfs_cont_expand(struct inode *inode, loff_t oldsize, loff_t size); void btrfs_add_delayed_iput(struct inode *inode); void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info); +int btrfs_wait_on_delayed_iputs(struct btrfs_fs_info *fs_info); int btrfs_prealloc_file_range(struct inode *inode, int mode, u64 start, u64 num_bytes, u64 min_size, loff_t actual_len, u64 *alloc_hint); diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 51b2a5bf25e5..3dce9ff72e41 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1692,9 +1692,7 @@ static int cleaner_kthread(void *arg) goto sleep; } - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); btrfs_run_delayed_iputs(fs_info); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); again = btrfs_clean_one_deleted_snapshot(root); mutex_unlock(&fs_info->cleaner_mutex); @@ -2677,7 +2675,6 @@ int open_ctree(struct super_block *sb, mutex_init(&fs_info->delete_unused_bgs_mutex); mutex_init(&fs_info->reloc_mutex); mutex_init(&fs_info->delalloc_root_mutex); - mutex_init(&fs_info->cleaner_delayed_iput_mutex); seqlock_init(&fs_info->profiles_lock); INIT_LIST_HEAD(&fs_info->dirty_cowonly_roots); @@ -2699,6 +2696,7 @@ int open_ctree(struct super_block *sb, atomic_set(&fs_info->defrag_running, 0); atomic_set(&fs_info->qgroup_op_seq, 0); atomic_set(&fs_info->reada_works_cnt, 0); + atomic_set(&fs_info->nr_delayed_iputs, 0); atomic64_set(&fs_info->tree_mod_seq, 0); fs_info->sb = sb; fs_info->max_inline = BTRFS_DEFAULT_MAX_INLINE; @@ -2776,6 +2774,7 @@ int open_ctree(struct super_block *sb, init_waitqueue_head(&fs_info->transaction_wait); init_waitqueue_head(&fs_info->transaction_blocked_wait); init_waitqueue_head(&fs_info->async_submit_wait); + init_waitqueue_head(&fs_info->delayed_iputs_wait); INIT_LIST_HEAD(&fs_info->pinned_chunks); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index be18b40d2d48..882b55b79497 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4258,8 +4258,9 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) * operations. Wait for it to finish so that * more space is released. */ - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + ret = btrfs_wait_on_delayed_iputs(fs_info); + if (ret) + return ret; goto again; } else { btrfs_end_transaction(trans); @@ -4829,9 +4830,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * pinned space, so make sure we run the iputs before we do our pinned * bytes check below. */ - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); btrfs_run_delayed_iputs(fs_info); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + wait_event(fs_info->delayed_iputs_wait, + atomic_read(&fs_info->nr_delayed_iputs) == 0); trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 0a1671fb03bf..ab8242b10601 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -3319,6 +3319,7 @@ void btrfs_add_delayed_iput(struct inode *inode) if (atomic_add_unless(&inode->i_count, -1, 1)) return; + atomic_inc(&fs_info->nr_delayed_iputs); spin_lock(&fs_info->delayed_iput_lock); ASSERT(list_empty(&binode->delayed_iput)); list_add_tail(&binode->delayed_iput, &fs_info->delayed_iputs); @@ -3338,11 +3339,31 @@ void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info) list_del_init(&inode->delayed_iput); spin_unlock(&fs_info->delayed_iput_lock); iput(&inode->vfs_inode); + if (atomic_dec_and_test(&fs_info->nr_delayed_iputs)) + wake_up(&fs_info->delayed_iputs_wait); spin_lock(&fs_info->delayed_iput_lock); } spin_unlock(&fs_info->delayed_iput_lock); } +/** + * btrfs_wait_on_delayed_iputs - wait on the delayed iputs to be done running + * @fs_info - the fs_info for this fs + * @return - EINTR if we were killed, 0 if nothing's pending + * + * This will wait on any delayed iputs that are currently running with KILLABLE + * set. Once they are all done running we will return, unless we are killed in + * which case we return EINTR. + */ +int btrfs_wait_on_delayed_iputs(struct btrfs_fs_info *fs_info) +{ + int ret = wait_event_killable(fs_info->delayed_iputs_wait, + atomic_read(&fs_info->nr_delayed_iputs) == 0); + if (ret) + return -EINTR; + return 0; +} + /* * This creates an orphan entry for the given inode in case something goes wrong * in the middle of an unlink. From patchwork Thu Oct 11 19:54:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637367 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F8DA112B for ; Thu, 11 Oct 2018 19:55:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F23A92C09B for ; Thu, 11 Oct 2018 19:55:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E65462C0A2; Thu, 11 Oct 2018 19:55:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8460B2C09B for ; Thu, 11 Oct 2018 19:55:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727242AbeJLDYf (ORCPT ); Thu, 11 Oct 2018 23:24:35 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:35915 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726697AbeJLDYf (ORCPT ); Thu, 11 Oct 2018 23:24:35 -0400 Received: by mail-qk1-f193.google.com with SMTP id a85-v6so6280367qkg.3 for ; Thu, 11 Oct 2018 12:55:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=0pFXVY2NKlGxnx5addw7+EqJLIsrQdT/gx6gstwLZiU=; b=XVMH9OQZ8Gaj7bTG235SKlI0BYOZZY6TX53Kw2KO/78QusTAC453h7XnKCrr7ciPpd ts4iT3M4I92RzeOyDha5GhjL/kCgETuz6DJf/6M8JaFzrN9g6a1yVbBZcD7VWWNTpvJt NVXZuguktRuTHWmwi7GkqZKXoL/D51tWPHqYW8bQVlO/CsgwYZROp27Ify8v/po59KXH /M+G8cc7AkQSW0Ta4BK4FtXFB/sgumu8pTl4NzAI0xqHzbG29EZwcGUbUw1zgXp+b/HG Tajw3DZt7/mGD2FR8f1+99+q6BiHlu8p0xGTukyS4K6WqyCv/Brgjccb0s4SezueoOGW zAIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=0pFXVY2NKlGxnx5addw7+EqJLIsrQdT/gx6gstwLZiU=; b=ixUg7Yvzfo3ZDv2i7b4E8jri4ilXSRcmT+EnB5PjXRlT+tt7zLPG7zzdVJM9UW8nxu IDfuGv1dR668FsoqTg/Bz+ye8NBEsK0vVzHF3KmyzfDYuYcE7xdjBE8Hews2ZOaMwHva HI+XrUCEblgErdifQQSR0Q2eD3YWtO75M7IgV+qJjT+eKhzdm/UtK4v6QU5/6KjN05Y7 MYEQxJqJAmx8db9f6xhELtR1LiUW4rQwYC2Q/1p6dxFkayp6MAHfzUpfVrw+DlT/ycB8 ms+/Fm+YGg9xFB+jJI90yVsL4TFejgRRufr6tU3ooRfFKCMOtt7z08K2Vfj6Rpl+Z2pk a/0g== X-Gm-Message-State: ABuFfogj/6ZEtLbjcHlBJsAhKlGggNeKEFdBYDKD0uaDyZXeweHGMSRv bG6tuFSm8UliNQK6MQPdsGWJL2Mi5x8= X-Google-Smtp-Source: ACcGV622s9d3Dzpz2MbYdPg4DdLIbquMLvYzGOF04bLzDv7dIehfxiFCqUdDAyie5OSLgFt44U6V7w== X-Received: by 2002:a37:a795:: with SMTP id q143-v6mr3091371qke.92.1539287748649; Thu, 11 Oct 2018 12:55:48 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y186-v6sm10810553qkd.68.2018.10.11.12.55.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:47 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 40/42] btrfs: drop min_size from evict_refill_and_join Date: Thu, 11 Oct 2018 15:54:29 -0400 Message-Id: <20181011195431.3441-41-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't need it, rsv->size is set once and never changes throughout its lifetime, so just use that for the reserve size. Reviewed-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ab8242b10601..dbcca915e681 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5339,8 +5339,7 @@ static void evict_inode_truncate_pages(struct inode *inode) } static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, - struct btrfs_block_rsv *rsv, - u64 min_size) + struct btrfs_block_rsv *rsv) { struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; @@ -5350,7 +5349,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, struct btrfs_trans_handle *trans; int ret; - ret = btrfs_block_rsv_refill(root, rsv, min_size, + ret = btrfs_block_rsv_refill(root, rsv, rsv->size, BTRFS_RESERVE_FLUSH_LIMIT); if (ret && ++failures > 2) { @@ -5368,7 +5367,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, * it. */ if (!btrfs_check_space_for_delayed_refs(fs_info) && - !btrfs_block_rsv_migrate(global_rsv, rsv, min_size, 0)) + !btrfs_block_rsv_migrate(global_rsv, rsv, rsv->size, 0)) return trans; /* If not, commit and try again. */ @@ -5384,7 +5383,6 @@ void btrfs_evict_inode(struct inode *inode) struct btrfs_trans_handle *trans; struct btrfs_root *root = BTRFS_I(inode)->root; struct btrfs_block_rsv *rsv; - u64 min_size; int ret; trace_btrfs_inode_evict(inode); @@ -5394,8 +5392,6 @@ void btrfs_evict_inode(struct inode *inode) return; } - min_size = btrfs_calc_trunc_metadata_size(fs_info, 1); - evict_inode_truncate_pages(inode); if (inode->i_nlink && @@ -5428,13 +5424,13 @@ void btrfs_evict_inode(struct inode *inode) rsv = btrfs_alloc_block_rsv(fs_info, BTRFS_BLOCK_RSV_TEMP); if (!rsv) goto no_delete; - rsv->size = min_size; + rsv->size = btrfs_calc_trunc_metadata_size(fs_info, 1); rsv->failfast = 1; btrfs_i_size_write(BTRFS_I(inode), 0); while (1) { - trans = evict_refill_and_join(root, rsv, min_size); + trans = evict_refill_and_join(root, rsv); if (IS_ERR(trans)) goto free_rsv; @@ -5459,7 +5455,7 @@ void btrfs_evict_inode(struct inode *inode) * If it turns out that we are dropping too many of these, we might want * to add a mechanism for retrying these after a commit. */ - trans = evict_refill_and_join(root, rsv, min_size); + trans = evict_refill_and_join(root, rsv); if (!IS_ERR(trans)) { trans->block_rsv = rsv; btrfs_orphan_del(trans, BTRFS_I(inode)); From patchwork Thu Oct 11 19:54:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637369 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B5088112B for ; Thu, 11 Oct 2018 19:55:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A41792C09B for ; Thu, 11 Oct 2018 19:55:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 986642C0A2; Thu, 11 Oct 2018 19:55:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B6D52C09B for ; Thu, 11 Oct 2018 19:55:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727248AbeJLDYh (ORCPT ); Thu, 11 Oct 2018 23:24:37 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:41095 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726697AbeJLDYh (ORCPT ); Thu, 11 Oct 2018 23:24:37 -0400 Received: by mail-qt1-f196.google.com with SMTP id l41-v6so11345248qtl.8 for ; Thu, 11 Oct 2018 12:55:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=am1ak1Lg8slURG/v1F1bdWqQUehd7FnCdJ9TMX1sZm0=; b=JIc2h/TK/UQ2UZ8nJ39ub+qXfDm7HiabmkcL7kG08mCd8vs4VcWZMi7sfEyfQc8ppZ 3MZdK1ySpjiZ34cUFRLIU2w+0vgeVOjDVfA3p2Os7XDxQYVbVMxQDB6naInJL6cNc2Ga Jj7SdboMYkKvGScL+kfgw8dpF2JDzpyUTJW6oLojR80TV88JwwSTCmg/Lc/X2X7VxZIy wbQkle3tRs6jawyYYm7Mquynm22E1gt5iLMjP5sZmgkNE3FtiGNdg3dAH1lNxQSJx0KF DEIKCkHZmx68+DWv1RZFQxvZTjPL3RoWm+D2nO4tDP16WXXBPd6Gt2USCFCKOxO0aW+4 KcUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=am1ak1Lg8slURG/v1F1bdWqQUehd7FnCdJ9TMX1sZm0=; b=GCgamZK+u0xmr/ZBENNu2vjDbuBl5mx8xYLSTuRaSvqWepAERyDwK5KNgum99vRr+I KBPdgwi7OkJeOZfSTYAck5iWxukCvqa4oyKd0rZhof5pZpItkf4cjzFBYgDAfUjQRDuK U3Sx+kQFW3+39t0OgwiRiwuzbyK6FuZrVDTX6mop41zfeAzVUraJUzmcya6Tunv27hTY GPff5Wq63debwunKBo5DoqdsctnuQqEPhakuaSk6BVr/weVQcA2OTivy0zanTruT7AG7 v5/qYKRh9DUQP39fYYQzqTLEDn38Aj1IrxLyHHhwY9RihYs5Ik5RS/1JgHv7jPLj6Mml q7Ug== X-Gm-Message-State: ABuFfoj26TinDDd33ZYGjiWeUNPExBpt4MVsWY3t6sJ+lR0E6dcguyg/ W6S1R1ZIUqjnqHQyMPEukKGe1vLEziM= X-Google-Smtp-Source: ACcGV62A+JeSEnl62DvKN5+9Y+unzc6stDHh/GwB8J0I2L6eVF//ENDFm0e8YIBozuuPac2ErzQCuw== X-Received: by 2002:ac8:2595:: with SMTP id e21-v6mr3007216qte.233.1539287750466; Thu, 11 Oct 2018 12:55:50 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a13-v6sm15930322qkg.94.2018.10.11.12.55.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:49 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 41/42] btrfs: reserve extra space during evict() Date: Thu, 11 Oct 2018 15:54:30 -0400 Message-Id: <20181011195431.3441-42-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We could generate a lot of delayed refs in evict but never have any left over space from our block rsv to make up for that fact. So reserve some extra space and give it to the transaction so it can be used to refill the delayed refs rsv every loop through the truncate path. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index dbcca915e681..9f7da5e3c741 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5343,13 +5343,15 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, { struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + u64 delayed_refs_extra = btrfs_calc_trans_metadata_size(fs_info, 1); int failures = 0; for (;;) { struct btrfs_trans_handle *trans; int ret; - ret = btrfs_block_rsv_refill(root, rsv, rsv->size, + ret = btrfs_block_rsv_refill(root, rsv, + rsv->size + delayed_refs_extra, BTRFS_RESERVE_FLUSH_LIMIT); if (ret && ++failures > 2) { @@ -5358,9 +5360,28 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, return ERR_PTR(-ENOSPC); } + /* + * Evict can generate a large amount of delayed refs without + * having a way to add space back since we exhaust our temporary + * block rsv. We aren't allowed to do FLUSH_ALL in this case + * because we could deadlock with so many things in the flushing + * code, so we have to try and hold some extra space to + * compensate for our delayed ref generation. If we can't get + * that space then we need see if we can steal our minimum from + * the global reserve. We will be ratelimited by the amount of + * space we have for the delayed refs rsv, so we'll end up + * committing and trying again. + */ trans = btrfs_join_transaction(root); - if (IS_ERR(trans) || !ret) + if (IS_ERR(trans) || !ret) { + if (!IS_ERR(trans)) { + trans->block_rsv = &fs_info->trans_block_rsv; + trans->bytes_reserved = delayed_refs_extra; + btrfs_block_rsv_migrate(rsv, trans->block_rsv, + delayed_refs_extra, 1); + } return trans; + } /* * Try to steal from the global reserve if there is space for From patchwork Thu Oct 11 19:54:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10637371 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8D13157A for ; Thu, 11 Oct 2018 19:55:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B81562C09B for ; Thu, 11 Oct 2018 19:55:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9BDC2C0A2; Thu, 11 Oct 2018 19:55:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 60A7F2C09B for ; Thu, 11 Oct 2018 19:55:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727298AbeJLDYm (ORCPT ); Thu, 11 Oct 2018 23:24:42 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:46916 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727252AbeJLDYi (ORCPT ); Thu, 11 Oct 2018 23:24:38 -0400 Received: by mail-qt1-f195.google.com with SMTP id d8-v6so11327661qtk.13 for ; Thu, 11 Oct 2018 12:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=sV09l7dO3avmvRHRl1FabUbd/mHTvojy+8waApybFR4=; b=QJHkhUxTtQCT3P7+4G825BxRv6Zb3O/MQjoykCcyuMzzTi+g5a8tHxkMMMgD6mwKmo AU9hRRofkamjx9oF0knlek5kojdexn2Cxr28/nW1FprCl2bdtdPApZP63BAE4U9CmF6y tvHO+Fu+UC1zzxI67l+1MRor0PgRxAzZSt6+CKTYKwoDcsJG2aSvIEQssu2Btxxx23yD 2qJAvU5hOc8JBmkZu/Yv/JFjphZwVmNYeKJ6gLeK3AqRe7nhD2Kd2dmaLPNCUozn5Cq1 lEnUlLtDFOpFf+TAMjLzOVboaifZpg9pCYedqzdqzlrqWk3cnj5TVy8WXg+3AYfYSZ9g z3nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=sV09l7dO3avmvRHRl1FabUbd/mHTvojy+8waApybFR4=; b=bR4mFcg98sWFOkISII78DeoZeB8yLPW0hNIaGRqkvsRAc39vJ17y3CHMEOqmzrhryw 1+gxmOCU/+0ufLd2KODeaDMrxS2q/jHLHQO5Th7juJKSX5lsdhdD5FojrKyC5BwUbEeL 5kqQRt8CzcdQM3ol3epiS7gYGOIoQ6AgH26XScKyFTIFjpuQ4zgXgSAq+NUQ0Vj+ECYv Zr+a1lZzzUhh+7EkNCwUhW3l96gGkDyvfON6L9+v+AA78vtg0VgzwOMtfaOUmf7j82RF ILxA8G3E3jjqGTLBRJik13o+6pMI1pGYAiXYmuHmVONYvqvZnKDE65N1k1qDximRR3yw aTpg== X-Gm-Message-State: ABuFfoiRl7ZrbRlVASY2CCNdx5QwupuDl9Unh3xK6qh4fygmciHco/Yy kZtThyxOJr0Ur4xp/hvhe4J66S30jD8= X-Google-Smtp-Source: ACcGV60N5A174vlAs4DYMdFjDp6joNWacNStxT1y6RKC4qNax2Ctpyj18gmcFwvdxZZk3px1WyM7Og== X-Received: by 2002:a0c:c346:: with SMTP id j6mr3104905qvi.48.1539287752336; Thu, 11 Oct 2018 12:55:52 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id i27-v6sm17965684qti.57.2018.10.11.12.55.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Oct 2018 12:55:51 -0700 (PDT) From: Josef Bacik To: kernel-team@fb.com, linux-btrfs@vger.kernel.org Subject: [PATCH 42/42] btrfs: don't run delayed_iputs in commit Date: Thu, 11 Oct 2018 15:54:31 -0400 Message-Id: <20181011195431.3441-43-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181011195431.3441-1-josef@toxicpanda.com> References: <20181011195431.3441-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This could result in a really bad case where we do something like evict evict_refill_and_join btrfs_commit_transaction btrfs_run_delayed_iputs evict evict_refill_and_join btrfs_commit_transaction ... forever We have plenty of other places where we run delayed iputs that are much safer, let those do the work. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/transaction.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 9168efaca37e..c91dc36fccae 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -2265,15 +2265,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) kmem_cache_free(btrfs_trans_handle_cachep, trans); - /* - * If fs has been frozen, we can not handle delayed iputs, otherwise - * it'll result in deadlock about SB_FREEZE_FS. - */ - if (current != fs_info->transaction_kthread && - current != fs_info->cleaner_kthread && - !test_bit(BTRFS_FS_FROZEN, &fs_info->flags)) - btrfs_run_delayed_iputs(fs_info); - return ret; scrub_continue: