From patchwork Fri Oct 12 19:32:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639145 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D9E1157A for ; Fri, 12 Oct 2018 19:33:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F5DD2C4FA for ; Fri, 12 Oct 2018 19:33:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 641072C527; Fri, 12 Oct 2018 19:33:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E4392C4FA for ; Fri, 12 Oct 2018 19:33:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726762AbeJMDHC (ORCPT ); Fri, 12 Oct 2018 23:07:02 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:37399 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725935AbeJMDHC (ORCPT ); Fri, 12 Oct 2018 23:07:02 -0400 Received: by mail-qt1-f195.google.com with SMTP id d14-v6so15061116qto.4 for ; Fri, 12 Oct 2018 12:33:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=J1nvikXF93vBk79skKC0f+gf8yb5S1jfySvhFcqnzF4=; b=SHU5fAyz5XNLgGKxkw4xZlIt4Jh0IHCuD644icXLTKHduTq5piUEdHoyt7/ts0TlsH kUH/GHQFpxWJx3kM+PVbrL2vOfSqpub+drUSzRNzvzDnbUlJXRZoK5dGPQ6cPQFhc/3/ xsEjUWkCM9szfolpcw7Gwspds82bkxTQiO9PHVCWit1pLfsndGwkOFdIajWk269z5fW2 KBgP87dlhKJsHf26wAblcMAgx0W+NEMiAxK+CqvNEDx168KO9DMOk0PzufoiGPCMM94N VKyepeDJ1fj3LubY2/fb8T6CosEs5htnX2gRmb5TwHz52evztzEfeeechysAOFlEDxLz I1wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=J1nvikXF93vBk79skKC0f+gf8yb5S1jfySvhFcqnzF4=; b=bs4ZJzn0ZrYNya/p3bKobYQ7HJ1Spz5E88EnskMUIJufYbZU1MMLkgkocgmrRj0dEr DYflGfLV97W50bp/P8peqUeRGE/og6t6MPl9k3Ob0KfDkTpnDXT7b0pSEZNGMVWFIa8L KZmo+bzbn1KIJeL9g7mH5r761AtqOpG33jglussd/NogYrvnPhXhWROhqhWA7rBt6pil aGVQSzm4C1EVwHBlzbZCqeg0HQclmbLiQ8LdUZ6yfA6R3Ch+HX4CisIhUyhKRK7PIeUy i+hD0exVeZYpAwPZWU+fKQe/ZIZZjCggz4zTnFuLoDa/DF+TZYhiLSQEdd7xvrvIY01v LUfw== X-Gm-Message-State: ABuFfogdDGyJ2MZ3Vc8J0wnaG54DciX6AG34OGk2DjsvEfJ1cZ9wRUYt 8etdKqqKvtJ7iX9o/ERx1fZCsgL8NZs= X-Google-Smtp-Source: ACcGV606mE5fqsS6YuSLZ9GpITIJm5rrJkbuSMIcwUt9FfPoYgKXFkrxz2mjjroNP+tOMGqgJHqaTA== X-Received: by 2002:ac8:540d:: with SMTP id b13-v6mr6688964qtq.242.1539372781506; Fri, 12 Oct 2018 12:33:01 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id k185-v6sm1393771qkd.27.2018.10.12.12.33.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:00 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 01/42] btrfs: add btrfs_delete_ref_head helper Date: Fri, 12 Oct 2018 15:32:15 -0400 Message-Id: <20181012193256.13735-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We do this dance in cleanup_ref_head and check_ref_cleanup, unify it into a helper and cleanup the calling functions. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/delayed-ref.c | 14 ++++++++++++++ fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/extent-tree.c | 22 +++------------------- 3 files changed, 19 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 62ff545ba1f7..3a9e4ac21794 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -393,6 +393,20 @@ btrfs_select_ref_head(struct btrfs_trans_handle *trans) return head; } +void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head) +{ + lockdep_assert_held(&delayed_refs->lock); + lockdep_assert_held(&head->lock); + + rb_erase(&head->href_node, &delayed_refs->href_root); + RB_CLEAR_NODE(&head->href_node); + atomic_dec(&delayed_refs->num_entries); + delayed_refs->num_heads--; + if (head->processing == 0) + delayed_refs->num_heads_ready--; +} + /* * Helper to insert the ref_node to the tail or merge with tail. * diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index d9f2a4ebd5db..7769177b489e 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -261,7 +261,8 @@ static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) { mutex_unlock(&head->mutex); } - +void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head); struct btrfs_delayed_ref_head * btrfs_select_ref_head(struct btrfs_trans_handle *trans); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f77226d8020a..d24a0de4a2e7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2492,12 +2492,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, spin_unlock(&delayed_refs->lock); return 1; } - delayed_refs->num_heads--; - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); + btrfs_delete_ref_head(delayed_refs, head); spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); - atomic_dec(&delayed_refs->num_entries); trace_run_delayed_ref_head(fs_info, head, 0); @@ -6984,22 +6981,9 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (!mutex_trylock(&head->mutex)) goto out; - /* - * at this point we have a head with no other entries. Go - * ahead and process it. - */ - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); - atomic_dec(&delayed_refs->num_entries); - - /* - * we don't take a ref on the node because we're removing it from the - * tree, so we just steal the ref the tree was holding. - */ - delayed_refs->num_heads--; - if (head->processing == 0) - delayed_refs->num_heads_ready--; + btrfs_delete_ref_head(delayed_refs, head); head->processing = 0; + spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); From patchwork Fri Oct 12 19:32:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639147 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA922157A for ; Fri, 12 Oct 2018 19:33:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC7292C4FA for ; Fri, 12 Oct 2018 19:33:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A0AA42C527; Fri, 12 Oct 2018 19:33:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D3602C4FA for ; Fri, 12 Oct 2018 19:33:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726770AbeJMDHF (ORCPT ); Fri, 12 Oct 2018 23:07:05 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:34223 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725935AbeJMDHE (ORCPT ); Fri, 12 Oct 2018 23:07:04 -0400 Received: by mail-qk1-f196.google.com with SMTP id p6-v6so8369083qkg.1 for ; Fri, 12 Oct 2018 12:33:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/1YdvlrrEVlQm7E7WHAhXkpqC+TBu9F+Y13pNbn/ob8=; b=Z2l6+vhhczKihir2zLc0QNqK70PDGxPRriU4KhAWIdz2Ya6oHSY4KRWveoQwVB4+lY ++yip40rdI8jNunMS7S9purUs+KIu8HbNdwu37YToap+O77elMDS4i3K2crx/fYLpSri QWo0sDKT5JL7CDcO8B48fBGuKfV/npW0LUGRu0duyFMWqaAnWB6u1NnUJUBomWIOwKkH vRZN0RyKQSBUOj9VHX2+/4OjRkxDY9/qSXcAJ2ancnwuRpszCRaQclv62wzu+D/5pDAZ yQBQfhhHO+B0fj2eBYfyU/PMadOawgVsCiKEzdeHTWtUhfZFYv48W1E7CA47ny7oXnLG JtDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/1YdvlrrEVlQm7E7WHAhXkpqC+TBu9F+Y13pNbn/ob8=; b=tfFE9C7k7gh1kWLEkc4ENKGC2+Xo7BkAHyhw19c/1GykzMhB+Fxbq6HIlRqRCZvqLC 0vUzEY6Li3fcdGLItZEXsT/2pUPxesswydBSQImPXOImlGL0nX67PcG3c0Eq2aisF2Q0 GakoHfqq9rWQSlpeuDOURdTxfT5A/k27ZUdwAZfDxRnKs29dzPb++gOW7R2VNnDRyB1N xEP+Te/9d4OhbfoUzUeTBC4K3xWHmnoKaRqRWf9UFSA7hlpqOdIoEiwp9qKLadWMJcBk k908ONYr2n8mBQVFpBpV+v5t8BAwbsCq1TEvlk7Kb4yiffZ1gCsfluh7JtSvBRx2dHh/ jfFA== X-Gm-Message-State: ABuFfogpUuz1OwLFEqAYfZrd4mwiGRnYgXj82uhovobC/AyG4RyG/rns lN2FveP2pGHEKPq0x/w/KGJYTgDSrD0= X-Google-Smtp-Source: ACcGV60WHY5G5NsYES0gRKhfpFeyMjo3YpJKsYdP9ycNmASP31Q8JanCoMG51cvzmE9HDB38j5TFyA== X-Received: by 2002:ae9:ef11:: with SMTP id d17-v6mr7186828qkg.80.1539372783484; Fri, 12 Oct 2018 12:33:03 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y26-v6sm1253675qty.52.2018.10.12.12.33.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:02 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 02/42] btrfs: add cleanup_ref_head_accounting helper Date: Fri, 12 Oct 2018 15:32:16 -0400 Message-Id: <20181012193256.13735-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We were missing some quota cleanups in check_ref_cleanup, so break the ref head accounting cleanup into a helper and call that from both check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that we don't screw up accounting in the future for other things that we add. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 67 +++++++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index d24a0de4a2e7..a44d55e36e11 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2461,6 +2461,41 @@ static int cleanup_extent_op(struct btrfs_trans_handle *trans, return ret ? ret : 1; } +static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + + if (head->total_ref_mod < 0) { + struct btrfs_space_info *space_info; + u64 flags; + + if (head->is_data) + flags = BTRFS_BLOCK_GROUP_DATA; + else if (head->is_system) + flags = BTRFS_BLOCK_GROUP_SYSTEM; + else + flags = BTRFS_BLOCK_GROUP_METADATA; + space_info = __find_space_info(fs_info, flags); + ASSERT(space_info); + percpu_counter_add_batch(&space_info->total_bytes_pinned, + -head->num_bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH); + + if (head->is_data) { + spin_lock(&delayed_refs->lock); + delayed_refs->pending_csums -= head->num_bytes; + spin_unlock(&delayed_refs->lock); + } + } + + /* Also free its reserved qgroup space */ + btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, + head->qgroup_reserved); +} + static int cleanup_ref_head(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head *head) { @@ -2496,31 +2531,6 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); - trace_run_delayed_ref_head(fs_info, head, 0); - - if (head->total_ref_mod < 0) { - struct btrfs_space_info *space_info; - u64 flags; - - if (head->is_data) - flags = BTRFS_BLOCK_GROUP_DATA; - else if (head->is_system) - flags = BTRFS_BLOCK_GROUP_SYSTEM; - else - flags = BTRFS_BLOCK_GROUP_METADATA; - space_info = __find_space_info(fs_info, flags); - ASSERT(space_info); - percpu_counter_add_batch(&space_info->total_bytes_pinned, - -head->num_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - - if (head->is_data) { - spin_lock(&delayed_refs->lock); - delayed_refs->pending_csums -= head->num_bytes; - spin_unlock(&delayed_refs->lock); - } - } - if (head->must_insert_reserved) { btrfs_pin_extent(fs_info, head->bytenr, head->num_bytes, 1); @@ -2530,9 +2540,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } - /* Also free its reserved qgroup space */ - btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, - head->qgroup_reserved); + cleanup_ref_head_accounting(trans, head); + + trace_run_delayed_ref_head(fs_info, head, 0); btrfs_delayed_ref_unlock(head); btrfs_put_delayed_ref_head(head); return 0; @@ -6991,6 +7001,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (head->must_insert_reserved) ret = 1; + cleanup_ref_head_accounting(trans, head); mutex_unlock(&head->mutex); btrfs_put_delayed_ref_head(head); return ret; From patchwork Fri Oct 12 19:32:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639149 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A072E17E3 for ; Fri, 12 Oct 2018 19:33:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90CE02C39F for ; Fri, 12 Oct 2018 19:33:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 855232C519; Fri, 12 Oct 2018 19:33:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2639B2C39F for ; Fri, 12 Oct 2018 19:33:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726771AbeJMDHH (ORCPT ); Fri, 12 Oct 2018 23:07:07 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:45225 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725935AbeJMDHG (ORCPT ); Fri, 12 Oct 2018 23:07:06 -0400 Received: by mail-qt1-f194.google.com with SMTP id e10-v6so15040573qtq.12 for ; Fri, 12 Oct 2018 12:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rHTVMCxRLsTEVUkODyGqkPIz399o/p3gyRkwe0nkz4U=; b=ob7X3CVr+I302weDMTaQgmJLUgRtHBIfMlTtZgKoZGEBc/mtbCvO5wXH8aVC/mOhou 4SQE9f7Shi643gRhSGkRGSUpjcfloULuUifaiVBR4stkjjEo0yXVySr0Lpx/yGeJivxl x/EWNAYIreb+6MHTL1HlGjozGnF5gLYfWm0MpeKTdALYqpt2BxBuV3Wnqcjzkz0dHfCp CV6WCNuYyDzfzcC7KoGSR0PtS7aCZb6lDWtqV6gKyreLTV5Ejnlu7XLKLd4QVK8GdMFo mUd3JhKo3q6hJnU1fCrTV+Cp6w3IzzJJmIUPsNahr5fRo9K7oaU6CL/9P9IeKbu2Kgdg 7phA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rHTVMCxRLsTEVUkODyGqkPIz399o/p3gyRkwe0nkz4U=; b=lIw9ZgY+w2ZjtiptJw70U9RpwpiHqX4CBr7axxpXaq6fGOHH9qzXFCPIlJMwu0J8TN bIJtVPD1cT9LekOrJgHnP5tR215QeCgeZ47G288OXF7L9zn0st15cAcbqwsVcXMNJcKp /GW0L3A+3HbZiO+wFHil9rCunquygZiC5aRFAaP4OI4876j2jdS71PPtGgkcGIC+wjDo 809Vxw7Z5BJhMWZdwGiGMAmgHRkihrpfiz8gGnKR0ktI764rSSYwzCGTxsLvTPUS2zbk 9EYd4S5aF6Nt8DNu2xSjLEBVsBO536MgiufeIe7DgCZxRsg/6mFgdYZ4oVDSTGGCb4ai fxBg== X-Gm-Message-State: ABuFfojyqpzSUKSdszKcPhQHuLboUIKQjCnd1UVmXwm5YAVJN0wzMisk kjoAzB1kSW1IX/7EtFeLDPUcYexk3Z8= X-Google-Smtp-Source: ACcGV61Uv9kzHPLirg/qySYd5n/a/HGekK9t0T4TQnFhJ1Re9OiKc0aVIbs0UeIi8kKlXI9L2aB3Ag== X-Received: by 2002:a0c:d4e5:: with SMTP id y34mr7396065qvh.106.1539372785293; Fri, 12 Oct 2018 12:33:05 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c36-v6sm2002877qta.56.2018.10.12.12.33.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:04 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 03/42] btrfs: cleanup extent_op handling Date: Fri, 12 Oct 2018 15:32:17 -0400 Message-Id: <20181012193256.13735-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik The cleanup_extent_op function actually would run the extent_op if it needed running, which made the name sort of a misnomer. Change it to run_and_cleanup_extent_op, and move the actual cleanup work to cleanup_extent_op so it can be used by check_ref_cleanup() in order to unify the extent op handling. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index a44d55e36e11..98f36dfeccb0 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2442,19 +2442,33 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_ref btrfs_delayed_ref_unlock(head); } -static int cleanup_extent_op(struct btrfs_trans_handle *trans, - struct btrfs_delayed_ref_head *head) +static struct btrfs_delayed_extent_op * +cleanup_extent_op(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) { struct btrfs_delayed_extent_op *extent_op = head->extent_op; - int ret; if (!extent_op) - return 0; - head->extent_op = NULL; + return NULL; + if (head->must_insert_reserved) { + head->extent_op = NULL; btrfs_free_delayed_extent_op(extent_op); - return 0; + return NULL; } + return extent_op; +} + +static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans, + struct btrfs_delayed_ref_head *head) +{ + struct btrfs_delayed_extent_op *extent_op = + cleanup_extent_op(trans, head); + int ret; + + if (!extent_op) + return 0; + head->extent_op = NULL; spin_unlock(&head->lock); ret = run_delayed_extent_op(trans, head, extent_op); btrfs_free_delayed_extent_op(extent_op); @@ -2506,7 +2520,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, delayed_refs = &trans->transaction->delayed_refs; - ret = cleanup_extent_op(trans, head); + ret = run_and_cleanup_extent_op(trans, head); if (ret < 0) { unselect_delayed_ref_head(delayed_refs, head); btrfs_debug(fs_info, "run_delayed_extent_op returned %d", ret); @@ -6977,12 +6991,8 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (!RB_EMPTY_ROOT(&head->ref_tree)) goto out; - if (head->extent_op) { - if (!head->must_insert_reserved) - goto out; - btrfs_free_delayed_extent_op(head->extent_op); - head->extent_op = NULL; - } + if (cleanup_extent_op(trans, head) != NULL) + goto out; /* * waiting for the lock here would deadlock. If someone else has it From patchwork Fri Oct 12 19:32:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639151 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D965117E3 for ; Fri, 12 Oct 2018 19:33:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB3012C39F for ; Fri, 12 Oct 2018 19:33:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BF6EC2C519; Fri, 12 Oct 2018 19:33:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71AE12C39F for ; Fri, 12 Oct 2018 19:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726779AbeJMDHI (ORCPT ); Fri, 12 Oct 2018 23:07:08 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:39995 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725935AbeJMDHI (ORCPT ); Fri, 12 Oct 2018 23:07:08 -0400 Received: by mail-qt1-f196.google.com with SMTP id b4-v6so15035359qtc.7 for ; Fri, 12 Oct 2018 12:33:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E8JF94eruCVpfwCQjNtPgc8hW7YmBCNIla9LHGUTiOs=; b=0f7x9n3zgieKfRC3UgRX4ExEzt57PcehiQaAHUSajpjKvs/RkfaNxFbxSvxWYC864Y 8QCkDNrw3fBjXVp5uxt9zJ+EQBkO7oftXmMKiiJjj4MpMHLRYt6C3fZH1cbGY50b9nQc ViDniPQQR8D0JVct6IC2tgiRBbit5LJDB3XGpXZCiSEvjQyqraqBDDqn798vlt6nZKrJ 9+238CshwBue05X13GPSZVDqLt50RQJ8a1todhoCl7n3H8AnmTK4nTYkmGJpcVkeuTjD EEi6O9k+u7CFkm/2MsGqIVGfB6IHAWh2IaY1lTaEjLthzTCf/ULsnzy10s5/4/UgKeGl chWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E8JF94eruCVpfwCQjNtPgc8hW7YmBCNIla9LHGUTiOs=; b=PfhYWiXtMXIg/9nF/+1Cx2pVtQ29NadBpzGi+iubFfK92+LFVOs/w+lw1YeAltz1Xn D4TL2p8a/zC5vELNWcz7araXT28sRdPG7O1+pPFK9RWBbjH3p9Zu/VrueTLW9eX9GPgG +1mbBMEhG0eUM65mhehgcj02gLCIwBpkw+20D3uDjFeuNoKVzhklCcbF/2uuLCDU7Ebr iSkEb/DxmorZqANjE4YzsRAGcW26FkrXiPm/deu+4aCiFRMjllNB3ecDPMfVtR7rIlS6 NVK7yofIRSW1piRHUvJOf1vNXt7ripjZwKs5vcccd7EL9hYiw0Am9s7yYwd6OmUcLZJI tIZg== X-Gm-Message-State: ABuFfojazS3N/Xj1NTqytgmTbxVmUdPaMGisz+yVQg+PVpZTT35gRFSx kU/JRO935JqsdvwUkf2WcKHW3FNbiLY= X-Google-Smtp-Source: ACcGV62nfF9JF/Ayu2skCxpcAHgcr6y7HnPmdcKvw744tGu0eNZdM/7S4h/YHSi6vNhkBx92R+qZmQ== X-Received: by 2002:ac8:721a:: with SMTP id a26-v6mr7145958qtp.64.1539372787168; Fri, 12 Oct 2018 12:33:07 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id j125-v6sm1093251qke.61.2018.10.12.12.33.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:06 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 04/42] btrfs: only track ref_heads in delayed_ref_updates Date: Fri, 12 Oct 2018 15:32:18 -0400 Message-Id: <20181012193256.13735-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We use this number to figure out how many delayed refs to run, but __btrfs_run_delayed_refs really only checks every time we need a new delayed ref head, so we always run at least one ref head completely no matter what the number of items on it. Fix the accounting to only be adjusted when we add/remove a ref head. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 3a9e4ac21794..27f7dd4e3d52 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -234,8 +234,6 @@ static inline void drop_delayed_ref(struct btrfs_trans_handle *trans, ref->in_tree = 0; btrfs_put_delayed_ref(ref); atomic_dec(&delayed_refs->num_entries); - if (trans->delayed_ref_updates) - trans->delayed_ref_updates--; } static bool merge_ref(struct btrfs_trans_handle *trans, @@ -460,7 +458,6 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans, if (ref->action == BTRFS_ADD_DELAYED_REF) list_add_tail(&ref->add_list, &href->ref_add_list); atomic_inc(&root->num_entries); - trans->delayed_ref_updates++; spin_unlock(&href->lock); return ret; } From patchwork Fri Oct 12 19:32:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639153 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE342157A for ; Fri, 12 Oct 2018 19:33:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD1501FF41 for ; Fri, 12 Oct 2018 19:33:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE1B11FFE4; Fri, 12 Oct 2018 19:33:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 517312C39F for ; Fri, 12 Oct 2018 19:33:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726791AbeJMDHK (ORCPT ); Fri, 12 Oct 2018 23:07:10 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:35780 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725935AbeJMDHK (ORCPT ); Fri, 12 Oct 2018 23:07:10 -0400 Received: by mail-qt1-f194.google.com with SMTP id d21-v6so2674742qtq.2 for ; Fri, 12 Oct 2018 12:33:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=+yz3n6lATRx6h1PNpaQrCmKxcZVsI4n1UZsZzxSFbKc=; b=BWZukbTMFhXPPTpyFXkRBYarxYItqcNa36ERumq0ge1fDZoUlBxkm01OTCay9SW969 o7MUjGMgOW7J2EW3FkDDkUXB3IrK61JtiGG+NXms9qEPIv04Hx+OXhXiesJZT5lFtAai lAJi8ynOx5etN2jlpeNt/7rYANIdIPeeGBo0ykivI3SM0lmnjLTDf3qgudH8s8WppCAS LtPMK5GNr4nXjC1rGd19QHD/IMxsI78QtjSJ9VtiLz+1BV2diQLLOaDXYiavEfHBKDpy SyhSQg+bgWkri4sXTgt+gos3oKOMaDl5r2YChU5oy1DINp3rGewM7NcOOcUpARjnRM51 Plbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+yz3n6lATRx6h1PNpaQrCmKxcZVsI4n1UZsZzxSFbKc=; b=RFoT5ynVTwrDQW95UKlN4whyYt3z3oX/Fl5Ztld5mjCrhtVdTyXkayG13hLUJtJL8b QWkmHejZxaWbVetvwDOWaNiVsSyli2U9wrhlzuJmN/a1hjvXW0gey/mMPZgcH+YvsB6v J3b/56yXiYMuzTckfJuCvNy9nxUnMJjy6WWL06ybArzQFOZwHOetSwLYVE7SywMPA6wG PyG7S8dQ1gjYXqc5jh++txKGfujVkq9YaBPuDZrKT3BaAK+wL90YUVJURlTexoqX6n+L 9+CmpCvSnLqJZ6wpuGlBCIZ9T5/krCiBN3TSf/Lp5HCVL6pUDqyQBgbhZEJy2vaCAHbq E3Ag== X-Gm-Message-State: ABuFfohURAXhRYOB/vjUt0QYZtHj+gaQj5tLDNptAbNW8g6dfbmJkr60 +rToJh1AG7idWsHB2hwj9ee7YTaANRw= X-Google-Smtp-Source: ACcGV60Up3Gcj3ldfz+k1VoUHvYlgzZgyiOvC7Uu7AU5uZtK1dCUVB1fKhcc+TgM0Fq7P2NUg2RF5w== X-Received: by 2002:aed:3eac:: with SMTP id n41-v6mr6892468qtf.196.1539372788991; Fri, 12 Oct 2018 12:33:08 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v57-v6sm1347908qtk.51.2018.10.12.12.33.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:08 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 05/42] btrfs: only count ref heads run in __btrfs_run_delayed_refs Date: Fri, 12 Oct 2018 15:32:19 -0400 Message-Id: <20181012193256.13735-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We pick the number of ref's to run based on the number of ref heads, and only make the decision to stop once we've processed entire ref heads, so only count the ref heads we've run and bail once we've hit the number of ref heads we wanted to process. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 98f36dfeccb0..b32bd38390dd 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2592,6 +2592,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, spin_unlock(&delayed_refs->lock); break; } + count++; /* grab the lock that says we are going to process * all the refs for this head */ @@ -2605,7 +2606,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, */ if (ret == -EAGAIN) { locked_ref = NULL; - count++; continue; } } @@ -2633,7 +2633,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unselect_delayed_ref_head(delayed_refs, locked_ref); locked_ref = NULL; cond_resched(); - count++; continue; } @@ -2651,7 +2650,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, return ret; } locked_ref = NULL; - count++; continue; } @@ -2702,7 +2700,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, } btrfs_put_delayed_ref(ref); - count++; cond_resched(); } From patchwork Fri Oct 12 19:32:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639157 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4686217E3 for ; Fri, 12 Oct 2018 19:33:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 348711FF41 for ; Fri, 12 Oct 2018 19:33:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 286C3201BD; Fri, 12 Oct 2018 19:33:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 670AE1FF41 for ; Fri, 12 Oct 2018 19:33:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726799AbeJMDHN (ORCPT ); Fri, 12 Oct 2018 23:07:13 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:40007 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725935AbeJMDHN (ORCPT ); Fri, 12 Oct 2018 23:07:13 -0400 Received: by mail-qt1-f193.google.com with SMTP id b4-v6so15035574qtc.7 for ; Fri, 12 Oct 2018 12:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UMAxEiV+68Nd9dMH4VurZacgpPCSyaixMmaSB9avthc=; b=pClg3QKF/o+qbju6Xnt+dbj9d9QvLfVjFWdle/i9BuG0BmT5+OXrBPFnOnGQlzZRoo wv59nOfFrt33cmAbRT8YEgIalv90b7LxgrAKNEHVKLP52SlHQ5Riab97SBTHdAj6QfWi 5XSml+je7JaYpP6gdAY1SwVkYdp9HT5XXQTlYKv8lrX6U1IwwuNi2xCDMAzGsQgELVL4 o//2DYD2eNglrgleMStV8Q0hlnJF89cNmiYnq0seZk+6M+FdoXYBn80a3X2xQIx8cXcX NLGaOenM2IIIxsJClSgT/FEqnU5d/20yr2dlyxcWDRSgTnrEM7bJgURu/JSITbxSvw5Q bNxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UMAxEiV+68Nd9dMH4VurZacgpPCSyaixMmaSB9avthc=; b=Lns1f/rXJ9+krCFobdUngWBXHcI0w1sl83rMkbix3sPw7vy6DVcvJFHIHfk47rc+2o AvAbpRDCNEFP9atkqFSO+zWf6cwNhftYQgYPqzvAGUR6+Vpb1AAY4VXmHSOqRonvU4Em gvu7g8A11siVOlB+CH1qTVOxOaSnYNFqMze9wzx0O54gJPTPd3LGOoiVQ1mzz2eKzkUt rWyroqyYqOr68i9uHPPoa5uHZrQXp6cTvsbOPxMb1ZTyukB99QFtNaJXpAW1ggpahgfE Rk4sXkUXBGQ9oVvTyOKv1u9tz9EPkHGcEbREVGON2fHhmcyWEvmH7ZbZFf6zxKXCJWsQ ijeQ== X-Gm-Message-State: ABuFfogwQYMOmCRpptvYUqWVBeVMo/UqHWHFHNrGRImyqnJZZc4/rCRJ x7SLxaa/w4Dof6VqRyDnqjKn8lCc/AI= X-Google-Smtp-Source: ACcGV62ICNmUlFG+Zt732YWjXPKOzZPIkfN6zaX6wMAhqKjh55vWSSwsiGA7KZRFXFhmldcBdtdkzw== X-Received: by 2002:aed:38c8:: with SMTP id k66-v6mr7128343qte.263.1539372790874; Fri, 12 Oct 2018 12:33:10 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b5-v6sm1200429qkf.4.2018.10.12.12.33.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:09 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 06/42] btrfs: introduce delayed_refs_rsv Date: Fri, 12 Oct 2018 15:32:20 -0400 Message-Id: <20181012193256.13735-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong. So instead give them their own block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism. For now the delayed_refs_rsv will hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via the delayed_refs so that will stay there. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 66f1d3895bca..1a2c3b629af2 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -452,8 +452,9 @@ struct btrfs_space_info { #define BTRFS_BLOCK_RSV_TRANS 3 #define BTRFS_BLOCK_RSV_CHUNK 4 #define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8 struct btrfs_block_rsv { u64 size; @@ -794,6 +795,8 @@ struct btrfs_fs_info { struct btrfs_block_rsv chunk_block_rsv; /* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; + /* block reservation for delayed refs */ + struct btrfs_block_rsv delayed_refs_rsv; struct btrfs_block_rsv empty_block_rsv; @@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info, int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, - struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info); void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info, const u64 start); void btrfs_wait_block_group_reservations(struct btrfs_block_group_cache *bg); @@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum { enum btrfs_flush_state { FLUSH_DELAYED_ITEMS_NR = 1, FLUSH_DELAYED_ITEMS = 2, - FLUSH_DELALLOC = 3, - FLUSH_DELALLOC_WAIT = 4, - ALLOC_CHUNK = 5, - COMMIT_TRANS = 6, + FLUSH_DELAYED_REFS_NR = 3, + FLUSH_DELAYED_REFS = 4, + FLUSH_DELALLOC = 5, + FLUSH_DELALLOC_WAIT = 6, + ALLOC_CHUNK = 7, + COMMIT_TRANS = 8, }; int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes); @@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info, void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *block_rsv, u64 num_bytes); +void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr); +void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs(struct btrfs_fs_info *fs_info, + enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *src, + u64 num_bytes); int btrfs_inc_block_group_ro(struct btrfs_block_group_cache *cache); void btrfs_dec_block_group_ro(struct btrfs_block_group_cache *cache); void btrfs_put_block_group_cache(struct btrfs_fs_info *info); diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 27f7dd4e3d52..96ce087747b2 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans, * existing and update must have the same bytenr */ static noinline void -update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head *existing, struct btrfs_delayed_ref_head *update, int *old_ref_mod_ret) { + struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + struct btrfs_fs_info *fs_info = trans->fs_info; int old_ref_mod; BUG_ON(existing->is_data != update->is_data); @@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs, * versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { - if (existing->total_ref_mod >= 0 && old_ref_mod < 0) + u64 csum_items = + btrfs_csum_bytes_to_leaves(fs_info, + existing->num_bytes); + + if (existing->total_ref_mod >= 0 && old_ref_mod < 0) { delayed_refs->pending_csums -= existing->num_bytes; - if (existing->total_ref_mod < 0 && old_ref_mod >= 0) + btrfs_delayed_refs_rsv_release(fs_info, csum_items); + } + if (existing->total_ref_mod < 0 && old_ref_mod >= 0) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } } spin_unlock(&existing->lock); } @@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); - update_existing_head_ref(delayed_refs, existing, head_ref, + update_existing_head_ref(trans, existing, head_ref, old_ref_mod); /* * we've updated the existing ref, free the newly @@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, } else { if (old_ref_mod) *old_ref_mod = 0; - if (head_ref->is_data && head_ref->ref_mod < 0) + if (head_ref->is_data && head_ref->ref_mod < 0) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += + btrfs_csum_bytes_to_leaves(trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++; atomic_inc(&delayed_refs->num_entries); @@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret = insert_delayed_ref(trans, delayed_refs, head_ref, &ref->node); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); trace_add_delayed_tree_ref(fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ? @@ -866,6 +882,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, ret = insert_delayed_ref(trans, delayed_refs, head_ref, &ref->node); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); trace_add_delayed_data_ref(trans->fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ? @@ -903,6 +920,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, NULL, NULL, NULL); spin_unlock(&delayed_refs->lock); + btrfs_update_delayed_refs_rsv(trans); return 0; } diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5124c15705ce..377ad9c1cb17 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2692,6 +2692,9 @@ int open_ctree(struct super_block *sb, btrfs_init_block_rsv(&fs_info->empty_block_rsv, BTRFS_BLOCK_RSV_EMPTY); btrfs_init_block_rsv(&fs_info->delayed_block_rsv, BTRFS_BLOCK_RSV_DELOPS); + btrfs_init_block_rsv(&fs_info->delayed_refs_rsv, + BTRFS_BLOCK_RSV_DELREFS); + atomic_set(&fs_info->async_delalloc_pages, 0); atomic_set(&fs_info->defrag_running, 0); atomic_set(&fs_info->qgroup_op_seq, 0); @@ -4419,6 +4422,7 @@ void btrfs_cleanup_dirty_bgs(struct btrfs_transaction *cur_trans, spin_unlock(&cur_trans->dirty_bgs_lock); btrfs_put_block_group(cache); + btrfs_delayed_refs_rsv_release(fs_info, 1); spin_lock(&cur_trans->dirty_bgs_lock); } spin_unlock(&cur_trans->dirty_bgs_lock); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index b32bd38390dd..1213f573eea2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2481,6 +2481,7 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info = trans->fs_info; struct btrfs_delayed_ref_root *delayed_refs = &trans->transaction->delayed_refs; + int nr_items = 1; if (head->total_ref_mod < 0) { struct btrfs_space_info *space_info; @@ -2502,12 +2503,15 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, spin_lock(&delayed_refs->lock); delayed_refs->pending_csums -= head->num_bytes; spin_unlock(&delayed_refs->lock); + nr_items += btrfs_csum_bytes_to_leaves(fs_info, + head->num_bytes); } } /* Also free its reserved qgroup space */ btrfs_qgroup_free_delayed_ref(fs_info, head->qgroup_ref_root, head->qgroup_reserved); + btrfs_delayed_refs_rsv_release(fs_info, nr_items); } static int cleanup_ref_head(struct btrfs_trans_handle *trans, @@ -2802,40 +2806,22 @@ u64 btrfs_csum_bytes_to_leaves(struct btrfs_fs_info *fs_info, u64 csum_bytes) return num_csums; } -int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, - struct btrfs_fs_info *fs_info) +bool btrfs_check_space_for_delayed_refs( struct btrfs_fs_info *fs_info) { - struct btrfs_block_rsv *global_rsv; - u64 num_heads = trans->transaction->delayed_refs.num_heads_ready; - u64 csum_bytes = trans->transaction->delayed_refs.pending_csums; - unsigned int num_dirty_bgs = trans->transaction->num_dirty_bgs; - u64 num_bytes, num_dirty_bgs_bytes; - int ret = 0; - - num_bytes = btrfs_calc_trans_metadata_size(fs_info, 1); - num_heads = heads_to_leaves(fs_info, num_heads); - if (num_heads > 1) - num_bytes += (num_heads - 1) * fs_info->nodesize; - num_bytes <<= 1; - num_bytes += btrfs_csum_bytes_to_leaves(fs_info, csum_bytes) * - fs_info->nodesize; - num_dirty_bgs_bytes = btrfs_calc_trans_metadata_size(fs_info, - num_dirty_bgs); - global_rsv = &fs_info->global_block_rsv; - - /* - * If we can't allocate any more chunks lets make sure we have _lots_ of - * wiggle room since running delayed refs can create more delayed refs. - */ - if (global_rsv->space_info->full) { - num_dirty_bgs_bytes <<= 1; - num_bytes <<= 1; - } + struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; + u64 reserved; + bool ret = false; spin_lock(&global_rsv->lock); - if (global_rsv->reserved <= num_bytes + num_dirty_bgs_bytes) - ret = 1; + reserved = global_rsv->reserved; spin_unlock(&global_rsv->lock); + + spin_lock(&delayed_refs_rsv->lock); + reserved += delayed_refs_rsv->reserved; + if (delayed_refs_rsv->size >= reserved) + ret = true; + spin_unlock(&delayed_refs_rsv->lock); return ret; } @@ -2855,7 +2841,7 @@ int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans, if (val >= NSEC_PER_SEC / 2) return 2; - return btrfs_check_space_for_delayed_refs(trans, fs_info); + return btrfs_check_space_for_delayed_refs(fs_info) ? 1 : 0; } struct async_delayed_refs { @@ -3610,6 +3596,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) */ mutex_lock(&trans->transaction->cache_write_mutex); while (!list_empty(&dirty)) { + bool drop_reserve = true; + cache = list_first_entry(&dirty, struct btrfs_block_group_cache, dirty_list); @@ -3682,6 +3670,7 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) list_add_tail(&cache->dirty_list, &cur_trans->dirty_bgs); btrfs_get_block_group(cache); + drop_reserve = false; } spin_unlock(&cur_trans->dirty_bgs_lock); } else if (ret) { @@ -3692,6 +3681,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans) /* if its not on the io list, we need to put the block group */ if (should_put) btrfs_put_block_group(cache); + if (drop_reserve) + btrfs_delayed_refs_rsv_release(fs_info, 1); if (ret) break; @@ -3840,6 +3831,7 @@ int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans, /* if its not on the io list, we need to put the block group */ if (should_put) btrfs_put_block_group(cache); + btrfs_delayed_refs_rsv_release(fs_info, 1); spin_lock(&cur_trans->dirty_bgs_lock); } spin_unlock(&cur_trans->dirty_bgs_lock); @@ -4816,8 +4808,10 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *trans; u64 bytes; + u64 reclaim_bytes = 0; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) @@ -4850,12 +4844,16 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, return -ENOSPC; spin_lock(&delayed_rsv->lock); - if (delayed_rsv->size > bytes) - bytes = 0; - else - bytes -= delayed_rsv->size; + reclaim_bytes += delayed_rsv->reserved; spin_unlock(&delayed_rsv->lock); + spin_lock(&delayed_refs_rsv->lock); + reclaim_bytes += delayed_refs_rsv->reserved; + spin_unlock(&delayed_refs_rsv->lock); + if (reclaim_bytes >= bytes) + goto commit; + bytes -= reclaim_bytes; + if (__percpu_counter_compare(&space_info->total_bytes_pinned, bytes, BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) { @@ -4905,6 +4903,20 @@ static void flush_space(struct btrfs_fs_info *fs_info, shrink_delalloc(fs_info, num_bytes * 2, num_bytes, state == FLUSH_DELALLOC_WAIT); break; + case FLUSH_DELAYED_REFS_NR: + case FLUSH_DELAYED_REFS: + trans = btrfs_join_transaction(root); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + break; + } + if (state == FLUSH_DELAYED_REFS_NR) + nr = calc_reclaim_items_nr(fs_info, num_bytes); + else + nr = 0; + btrfs_run_delayed_refs(trans, nr); + btrfs_end_transaction(trans); + break; case ALLOC_CHUNK: trans = btrfs_join_transaction(root); if (IS_ERR(trans)) { @@ -5377,6 +5389,91 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info, return 0; } +/** + * btrfs_migrate_to_delayed_refs_rsv - transfer bytes to our delayed refs rsv. + * @fs_info - the fs info for our fs. + * @src - the source block rsv to transfer from. + * @num_bytes - the number of bytes to transfer. + * + * This transfers up to the num_bytes amount from the src rsv to the + * delayed_refs_rsv. Any extra bytes are returned to the space info. + */ +void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *src, + u64 num_bytes) +{ + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; + u64 to_free = 0; + + spin_lock(&src->lock); + src->reserved -= num_bytes; + src->size -= num_bytes; + spin_unlock(&src->lock); + + spin_lock(&delayed_refs_rsv->lock); + if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) { + u64 delta = delayed_refs_rsv->size - + delayed_refs_rsv->reserved; + if (num_bytes > delta) { + to_free = num_bytes - delta; + num_bytes = delta; + } + } else { + to_free = num_bytes; + num_bytes = 0; + } + + if (num_bytes) + delayed_refs_rsv->reserved += num_bytes; + if (delayed_refs_rsv->reserved >= delayed_refs_rsv->size) + delayed_refs_rsv->full = 1; + spin_unlock(&delayed_refs_rsv->lock); + + if (num_bytes) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, num_bytes, 1); + if (to_free) + space_info_add_old_bytes(fs_info, delayed_refs_rsv->space_info, + to_free); +} + +/** + * btrfs_throttle_delayed_refs - throttle based on our delayed refs usage. + * @fs_info - the fs_info for our fs. + * @flush - control how we can flush for this reservation. + * + * This will refill the delayed block_rsv up to 1 items size worth of space and + * will return -ENOSPC if we can't make the reservation. + */ +int btrfs_throttle_delayed_refs(struct btrfs_fs_info *fs_info, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; + u64 limit = btrfs_calc_trans_metadata_size(fs_info, 1); + u64 num_bytes = 0; + int ret = -ENOSPC; + + spin_lock(&block_rsv->lock); + if (block_rsv->reserved < block_rsv->size) { + num_bytes = block_rsv->size - block_rsv->reserved; + num_bytes = min(num_bytes, limit); + } + spin_unlock(&block_rsv->lock); + + if (!num_bytes) + return 0; + + ret = reserve_metadata_bytes(fs_info->extent_root, block_rsv, + num_bytes, flush); + if (ret) + return ret; + block_rsv_add_bytes(block_rsv, num_bytes, 0); + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, num_bytes, 1); + return 0; +} + + /* * This is for space we already have accounted in space_info->bytes_may_use, so * basically when we're returning space from block_rsv's. @@ -5699,6 +5796,31 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, return ret; } +static u64 __btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *block_rsv, + u64 num_bytes, u64 *qgroup_to_release) +{ + struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; + struct btrfs_block_rsv *target = delayed_rsv; + + if (target->full || target == block_rsv) + target = global_rsv; + + if (block_rsv->space_info != target->space_info) + target = NULL; + + return block_rsv_release_bytes(fs_info, block_rsv, target, num_bytes, + qgroup_to_release); +} + +void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, + struct btrfs_block_rsv *block_rsv, + u64 num_bytes) +{ + __btrfs_block_rsv_release(fs_info, block_rsv, num_bytes, NULL); +} + /** * btrfs_inode_rsv_release - release any excessive reservation. * @inode - the inode we need to release from. @@ -5713,7 +5835,6 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) { struct btrfs_fs_info *fs_info = inode->root->fs_info; - struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; struct btrfs_block_rsv *block_rsv = &inode->block_rsv; u64 released = 0; u64 qgroup_to_release = 0; @@ -5723,8 +5844,8 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) * are releasing 0 bytes, and then we'll just get the reservation over * the size free'd. */ - released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv, 0, - &qgroup_to_release); + released = __btrfs_block_rsv_release(fs_info, block_rsv, 0, + &qgroup_to_release); if (released > 0) trace_btrfs_space_reservation(fs_info, "delalloc", btrfs_ino(inode), released, 0); @@ -5735,16 +5856,26 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) qgroup_to_release); } -void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info, - struct btrfs_block_rsv *block_rsv, - u64 num_bytes) +/** + * btrfs_delayed_refs_rsv_release - release a ref head's reservation. + * @fs_info - the fs_info for our fs. + * @nr - the number of items to drop. + * + * This drops the delayed ref head's count from the delayed refs rsv and free's + * any excess reservation we had. + */ +void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr) { + struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + u64 num_bytes = btrfs_calc_trans_metadata_size(fs_info, nr); + u64 released = 0; - if (global_rsv == block_rsv || - block_rsv->space_info != global_rsv->space_info) - global_rsv = NULL; - block_rsv_release_bytes(fs_info, block_rsv, global_rsv, num_bytes, NULL); + released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv, + num_bytes, NULL); + if (released) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", + 0, released, 0); } static void update_global_block_rsv(struct btrfs_fs_info *fs_info) @@ -5809,9 +5940,10 @@ static void init_global_block_rsv(struct btrfs_fs_info *fs_info) fs_info->trans_block_rsv.space_info = space_info; fs_info->empty_block_rsv.space_info = space_info; fs_info->delayed_block_rsv.space_info = space_info; + fs_info->delayed_refs_rsv.space_info = space_info; - fs_info->extent_root->block_rsv = &fs_info->global_block_rsv; - fs_info->csum_root->block_rsv = &fs_info->global_block_rsv; + fs_info->extent_root->block_rsv = &fs_info->delayed_refs_rsv; + fs_info->csum_root->block_rsv = &fs_info->delayed_refs_rsv; fs_info->dev_root->block_rsv = &fs_info->global_block_rsv; fs_info->tree_root->block_rsv = &fs_info->global_block_rsv; if (fs_info->quota_root) @@ -5831,8 +5963,34 @@ static void release_global_block_rsv(struct btrfs_fs_info *fs_info) WARN_ON(fs_info->chunk_block_rsv.reserved > 0); WARN_ON(fs_info->delayed_block_rsv.size > 0); WARN_ON(fs_info->delayed_block_rsv.reserved > 0); + WARN_ON(fs_info->delayed_refs_rsv.reserved > 0); + WARN_ON(fs_info->delayed_refs_rsv.size > 0); } +/* + * btrfs_update_delayed_refs_rsv - adjust the size of the delayed refs rsv + * @trans - the trans that may have generated delayed refs + * + * This is to be called anytime we may have adjusted trans->delayed_ref_updates, + * it'll calculate the additional size and add it to the delayed_refs_rsv. + */ +void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; + u64 num_bytes; + + if (!trans->delayed_ref_updates) + return; + + num_bytes = btrfs_calc_trans_metadata_size(fs_info, + trans->delayed_ref_updates); + spin_lock(&delayed_rsv->lock); + delayed_rsv->size += num_bytes; + delayed_rsv->full = 0; + spin_unlock(&delayed_rsv->lock); + trans->delayed_ref_updates = 0; +} /* * To be called after all the new block groups attached to the transaction @@ -6126,6 +6284,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, u64 old_val; u64 byte_in_group; int factor; + int ret = 0; /* block accounting for super block */ spin_lock(&info->delalloc_root_lock); @@ -6139,8 +6298,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, while (total) { cache = btrfs_lookup_block_group(info, bytenr); - if (!cache) - return -ENOENT; + if (!cache) { + ret = -ENOENT; + break; + } factor = btrfs_bg_type_to_factor(cache->flags); /* @@ -6199,6 +6360,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, list_add_tail(&cache->dirty_list, &trans->transaction->dirty_bgs); trans->transaction->num_dirty_bgs++; + trans->delayed_ref_updates++; btrfs_get_block_group(cache); } spin_unlock(&trans->transaction->dirty_bgs_lock); @@ -6216,7 +6378,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, total -= num_bytes; bytenr += num_bytes; } - return 0; + + /* Modified block groups are accounted for in the delayed_refs_rsv. */ + btrfs_update_delayed_refs_rsv(trans); + return ret; } static u64 first_logical_byte(struct btrfs_fs_info *fs_info, u64 search_start) @@ -8230,7 +8395,12 @@ use_block_rsv(struct btrfs_trans_handle *trans, goto again; } - if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) { + /* + * The global reserve still exists to save us from ourselves, so don't + * warn_on if we are short on our delayed refs reserve. + */ + if (block_rsv->type != BTRFS_BLOCK_RSV_DELREFS && + btrfs_test_opt(fs_info, ENOSPC_DEBUG)) { static DEFINE_RATELIMIT_STATE(_rs, DEFAULT_RATELIMIT_INTERVAL * 10, /*DEFAULT_RATELIMIT_BURST*/ 1); @@ -10146,6 +10316,7 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) add_block_group_free_space(trans, block_group); /* already aborted the transaction if it failed. */ next: + btrfs_delayed_refs_rsv_release(fs_info, 1); list_del_init(&block_group->bg_list); } trans->can_flush_pending_bgs = can_flush_pending_bgs; @@ -10223,6 +10394,8 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used, link_block_group(cache); list_add_tail(&cache->bg_list, &trans->new_bgs); + trans->delayed_ref_updates++; + btrfs_update_delayed_refs_rsv(trans); set_avail_alloc_bits(fs_info, type); return 0; @@ -10260,6 +10433,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, int factor; struct btrfs_caching_control *caching_ctl = NULL; bool remove_em; + bool remove_rsv = false; block_group = btrfs_lookup_block_group(fs_info, group_start); BUG_ON(!block_group); @@ -10324,6 +10498,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, if (!list_empty(&block_group->dirty_list)) { list_del_init(&block_group->dirty_list); + remove_rsv = true; btrfs_put_block_group(block_group); } spin_unlock(&trans->transaction->dirty_bgs_lock); @@ -10533,6 +10708,8 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, ret = btrfs_del_item(trans, root, path); out: + if (remove_rsv) + btrfs_delayed_refs_rsv_release(fs_info, 1); btrfs_free_path(path); return ret; } diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 212fa71317d6..cd00ec869c96 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5382,7 +5382,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, * Try to steal from the global reserve if there is space for * it. */ - if (!btrfs_check_space_for_delayed_refs(trans, fs_info) && + if (!btrfs_check_space_for_delayed_refs(fs_info) && !btrfs_block_rsv_migrate(global_rsv, rsv, min_size, 0)) return trans; diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 3b84f5015029..117e0c4a914a 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -455,7 +455,7 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, bool enforce_qgroups) { struct btrfs_fs_info *fs_info = root->fs_info; - + struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *h; struct btrfs_transaction *cur_trans; u64 num_bytes = 0; @@ -484,13 +484,28 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, * the appropriate flushing if need be. */ if (num_items && root != fs_info->chunk_root) { + struct btrfs_block_rsv *rsv = &fs_info->trans_block_rsv; + u64 delayed_refs_bytes = 0; + qgroup_reserved = num_items * fs_info->nodesize; ret = btrfs_qgroup_reserve_meta_pertrans(root, qgroup_reserved, enforce_qgroups); if (ret) return ERR_PTR(ret); + /* + * We want to reserve all the bytes we may need all at once, so + * we only do 1 enospc flushing cycle per transaction start. We + * accomplish this by simply assuming we'll do 2 x num_items + * worth of delayed refs updates in this trans handle, and + * refill that amount for whatever is missing in the reserve. + */ num_bytes = btrfs_calc_trans_metadata_size(fs_info, num_items); + if (delayed_refs_rsv->full == 0) { + delayed_refs_bytes = num_bytes; + num_bytes <<= 1; + } + /* * Do the reservation for the relocation root creation */ @@ -499,8 +514,24 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, reloc_reserved = true; } - ret = btrfs_block_rsv_add(root, &fs_info->trans_block_rsv, - num_bytes, flush); + ret = btrfs_block_rsv_add(root, rsv, num_bytes, flush); + if (ret) + goto reserve_fail; + if (delayed_refs_bytes) { + btrfs_migrate_to_delayed_refs_rsv(fs_info, rsv, + delayed_refs_bytes); + num_bytes -= delayed_refs_bytes; + } + } else if (num_items == 0 && flush == BTRFS_RESERVE_FLUSH_ALL && + !delayed_refs_rsv->full) { + /* + * Some people call with btrfs_start_transaction(root, 0) + * because they can be throttled, but have some other mechanism + * for reserving space. We still want these guys to refill the + * delayed block_rsv so just add 1 items worth of reservation + * here. + */ + ret = btrfs_throttle_delayed_refs(fs_info, flush); if (ret) goto reserve_fail; } @@ -759,7 +790,7 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) { struct btrfs_fs_info *fs_info = trans->fs_info; - if (btrfs_check_space_for_delayed_refs(trans, fs_info)) + if (btrfs_check_space_for_delayed_refs(fs_info)) return 1; return !!btrfs_block_rsv_check(&fs_info->global_block_rsv, 5); @@ -768,22 +799,12 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) int btrfs_should_end_transaction(struct btrfs_trans_handle *trans) { struct btrfs_transaction *cur_trans = trans->transaction; - int updates; - int err; smp_mb(); if (cur_trans->state >= TRANS_STATE_BLOCKED || cur_trans->delayed_refs.flushing) return 1; - updates = trans->delayed_ref_updates; - trans->delayed_ref_updates = 0; - if (updates) { - err = btrfs_run_delayed_refs(trans, updates * 2); - if (err) /* Error code will also eval true */ - return err; - } - return should_end_transaction(trans); } @@ -813,11 +834,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, { struct btrfs_fs_info *info = trans->fs_info; struct btrfs_transaction *cur_trans = trans->transaction; - u64 transid = trans->transid; - unsigned long cur = trans->delayed_ref_updates; int lock = (trans->type != TRANS_JOIN_NOLOCK); int err = 0; - int must_run_delayed_refs = 0; if (refcount_read(&trans->use_count) > 1) { refcount_dec(&trans->use_count); @@ -828,27 +846,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); - - trans->delayed_ref_updates = 0; - if (!trans->sync) { - must_run_delayed_refs = - btrfs_should_throttle_delayed_refs(trans, info); - cur = max_t(unsigned long, cur, 32); - - /* - * don't make the caller wait if they are from a NOLOCK - * or ATTACH transaction, it will deadlock with commit - */ - if (must_run_delayed_refs == 1 && - (trans->type & (__TRANS_JOIN_NOLOCK | __TRANS_ATTACH))) - must_run_delayed_refs = 2; - } - - btrfs_trans_release_metadata(trans); - trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) btrfs_create_pending_block_groups(trans); @@ -893,10 +890,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, } kmem_cache_free(btrfs_trans_handle_cachep, trans); - if (must_run_delayed_refs) { - btrfs_async_run_delayed_refs(info, cur, transid, - must_run_delayed_refs == 1); - } return err; } diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index b401c4e36394..7d205e50b09c 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -1048,6 +1048,8 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_ITEMS, "FLUSH_DELAYED_ITEMS"}, \ { FLUSH_DELALLOC, "FLUSH_DELALLOC"}, \ { FLUSH_DELALLOC_WAIT, "FLUSH_DELALLOC_WAIT"}, \ + { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ + { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) From patchwork Fri Oct 12 19:32:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639155 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9199117E3 for ; Fri, 12 Oct 2018 19:33:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83D181FFE4 for ; Fri, 12 Oct 2018 19:33:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 77A112022B; Fri, 12 Oct 2018 19:33:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BD821FFE4 for ; Fri, 12 Oct 2018 19:33:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726814AbeJMDHO (ORCPT ); Fri, 12 Oct 2018 23:07:14 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:43414 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHO (ORCPT ); Fri, 12 Oct 2018 23:07:14 -0400 Received: by mail-qk1-f196.google.com with SMTP id 12-v6so8329507qkj.10 for ; Fri, 12 Oct 2018 12:33:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Q7Ot7S7niLzFArpknfZ6fBkmt7T9NBBADarihIFw9qo=; b=pYbfImKs0nrP2vdJMti+2NI1xWgIksINAgmFQcFaKhUSRvZe0nmkRlC4bWGOb1luu2 wdR50fhQ9RBZg7RJUf1VUrYES8mJ7AYclq7EUE/fesAoLAe+oTU8vPKzeS94e9MlaIxK FqaY6b0n1QDcw1pJQJwC76gr/6wCJp0RPm2cNke9sNnCUn3fys3uRbBSidePXAQlii2C vKglbi7FZ6o91COZI4i5CFIWhe8FjnEdC0B/PVb1k3S67cyqBLRHzw0i+3SQD2lPUs+d T9KisnxwuXbOmp+RduuT0hXUVqapWH8VOX/2pj9TaYEZkrSTeyIDNQUsVSqX7dWovcPG K2wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Q7Ot7S7niLzFArpknfZ6fBkmt7T9NBBADarihIFw9qo=; b=kyzNrGKa9NeuVkftgwQsybVW7fWEVNlZhYw/LEceUcvfpjdO5BHR9aOv9MM2drMz2g G0YcSTqtesPJBJUwmqwlEETU5A+BqqCFOz7FbW8WvQxdzRcGtJUX4B+W7l5DfGkKOU4a DH8S7ZKvVVJI5hu3HtcfTs0e+QT1ZrQDS+cJr06p6LrLvjnEo7Ap5pSpH9zT1/AFUOK5 bJyyNeO3dHzp9KZQMGgYzkspRi6PCzhwVlkXt1xVBuL9l9I26zUA3WRshtceaFjG70P0 GR5QMgOyo4VFTmVuBRYRroAI14pLvrJzaB902yeDsq2rLP/zICZl95yCwtkxkaFFxyoO jpiA== X-Gm-Message-State: ABuFfogg5BaTqR3fxbuA/IMEs5cNIg8gmpqXLj6IYs4Br8z/WAPDF3XH VZgTOJuNGgiJ8aurjFYY2P7ng+/49QQ= X-Google-Smtp-Source: ACcGV61EBaa6geU/jmH/zVicwere7gx0iGPzs8NiAO4a8ijPU8orPFTUoILCMAMuEEuw0gymZnIoLg== X-Received: by 2002:a37:7006:: with SMTP id l6-v6mr6643785qkc.243.1539372792888; Fri, 12 Oct 2018 12:33:12 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g34-v6sm1700735qta.79.2018.10.12.12.33.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:11 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 07/42] btrfs: check if free bgs for commit Date: Fri, 12 Oct 2018 15:32:21 -0400 Message-Id: <20181012193256.13735-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP may_commit_transaction will skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1213f573eea2..4aab49debfc3 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes) return 0; - /* See if there is enough pinned space to make this reservation */ - if (__percpu_counter_compare(&space_info->total_bytes_pinned, - bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) + trans = btrfs_join_transaction(fs_info->extent_root); + if (IS_ERR(trans)) + return PTR_ERR(trans); + + /* + * See if there is enough pinned space to make this reservation, or if + * we have bg's that are going to be freed, allowing us to possibly do a + * chunk allocation the next loop through. + */ + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + __percpu_counter_compare(&space_info->total_bytes_pinned, bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) goto commit; /* @@ -4841,7 +4849,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * this reservation. */ if (space_info != delayed_rsv->space_info) - return -ENOSPC; + goto enospc; spin_lock(&delayed_rsv->lock); reclaim_bytes += delayed_rsv->reserved; @@ -4855,17 +4863,14 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, bytes -= reclaim_bytes; if (__percpu_counter_compare(&space_info->total_bytes_pinned, - bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) { - return -ENOSPC; - } - + bytes, + BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0) + goto enospc; commit: - trans = btrfs_join_transaction(fs_info->extent_root); - if (IS_ERR(trans)) - return -ENOSPC; - return btrfs_commit_transaction(trans); +enospc: + btrfs_end_transaction(trans); + return -ENOSPC; } /* From patchwork Fri Oct 12 19:32:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639159 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F887157A for ; Fri, 12 Oct 2018 19:33:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 111101FF41 for ; Fri, 12 Oct 2018 19:33:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 052782022B; Fri, 12 Oct 2018 19:33:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A201E1FF41 for ; Fri, 12 Oct 2018 19:33:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726827AbeJMDHQ (ORCPT ); Fri, 12 Oct 2018 23:07:16 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:39002 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHQ (ORCPT ); Fri, 12 Oct 2018 23:07:16 -0400 Received: by mail-qk1-f196.google.com with SMTP id q5-v6so8348083qki.6 for ; Fri, 12 Oct 2018 12:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=SMLqLDnd0lkJbGe6WWsh7Fynol7PpuIRXW5wGtVdu70=; b=JiA4tXB5jnsd48iUHAZwfVJ6PUADW4w5Cuo7wzBIDS/iaJteAPuQCcN0ir4DGYRioH W1CcbJk49OqgAj8dccaNil4/VMrm7ZO1xfDpngrzjuFEMcaL1JjkAoSUfzqBHBywsYpm m4nVEYeZOzn2ksbh/0TYcPrHGy6uhC86nGsFfY5UBwLpH5Ub6QsJ4V1AIEFS6tfZbnl2 u6is3DuiC2ccivYeu92WIwrobZULY2h4V1relhguvsbQTroJXksNByr1Hc1gCpquMJeM YOjOuMdDb6gefrwwXNSX2fl/+gDPdjkEpepiayibufuT/tz5yR7wfptrF366OWfcavWn CBPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=SMLqLDnd0lkJbGe6WWsh7Fynol7PpuIRXW5wGtVdu70=; b=Y/lN8RstCkFcN7+GPEe0Xest/WP4mDOXx030VjOGfk0MTWT3HZAxDjq0yVyteG453m xrwFLx4pPP4t35nT6mjmL9Xi9aeBPXLlNURUJ16argXHHxqUFqYBaPrXgQxQjwgPASN9 MBu4P0mUzFKpDRPeHKCTkymXCdOQgj4teWvJCxtINneHRnFl7Qs+0v4914fD4gt+vjtQ nLIJgnONjtA3epPwZ5ixzPI6vKvPKFCbBetAFIouv3xTjvdj9zoQs+6P7v1cfFNoXKQZ Ag1kaDEACcgbkbMbcUZUgC/gBarW1Dw6VoTqg9sN59Ot1N2J+vQfxKSimEG00QXO2GjD GjVw== X-Gm-Message-State: ABuFfojG13tNmddBZ0FTw7tAdHstsjBMs+52/liaz5PZPilf+QoGGkL5 jMZZKDaHfrfV3eWQjd1H3WqAoM93r8g= X-Google-Smtp-Source: ACcGV619OoQMsl9ypAHr1JBEGTIuvbT/TIFiU4uFYT0zPqdwtNfoMWe3EEsinRLAUGXeuRuKEP84Ew== X-Received: by 2002:a37:2b8d:: with SMTP id r13-v6mr6762818qkr.259.1539372794883; Fri, 12 Oct 2018 12:33:14 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id w81-v6sm1645347qka.87.2018.10.12.12.33.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:13 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 08/42] btrfs: dump block_rsv whe dumping space info Date: Fri, 12 Oct 2018 15:32:22 -0400 Message-Id: <20181012193256.13735-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For enospc_debug having the block rsvs is super helpful to see if we've done something wrong. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval Reviewed-by: David Sterba --- fs/btrfs/extent-tree.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 4aab49debfc3..8b00c658deb3 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -7918,6 +7918,15 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, return ret; } +#define DUMP_BLOCK_RSV(fs_info, rsv_name) \ +do { \ + struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \ + spin_lock(&__rsv->lock); \ + btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \ + __rsv->size, __rsv->reserved); \ + spin_unlock(&__rsv->lock); \ +} while (0) + static void dump_space_info(struct btrfs_fs_info *fs_info, struct btrfs_space_info *info, u64 bytes, int dump_block_groups) @@ -7937,6 +7946,12 @@ static void dump_space_info(struct btrfs_fs_info *fs_info, info->bytes_readonly); spin_unlock(&info->lock); + DUMP_BLOCK_RSV(fs_info, global_block_rsv); + DUMP_BLOCK_RSV(fs_info, trans_block_rsv); + DUMP_BLOCK_RSV(fs_info, chunk_block_rsv); + DUMP_BLOCK_RSV(fs_info, delayed_block_rsv); + DUMP_BLOCK_RSV(fs_info, delayed_refs_rsv); + if (!dump_block_groups) return; From patchwork Fri Oct 12 19:32:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639161 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 869533CF1 for ; Fri, 12 Oct 2018 19:33:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79E541FF41 for ; Fri, 12 Oct 2018 19:33:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6DDE6201BD; Fri, 12 Oct 2018 19:33:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BB161FF41 for ; Fri, 12 Oct 2018 19:33:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726836AbeJMDHS (ORCPT ); Fri, 12 Oct 2018 23:07:18 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:45044 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHR (ORCPT ); Fri, 12 Oct 2018 23:07:17 -0400 Received: by mail-qt1-f196.google.com with SMTP id c56-v6so15043461qtd.11 for ; Fri, 12 Oct 2018 12:33:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=yYx+AcpbxeLRdPqDImtMM4yf4Rg6f5T1yjgQiHsfUlY=; b=skXY3pP1DrFOdaOO5OkC1wUb6KrG+WXjvnuvCO5zH0wiq/Pm2fTcI5P/OuJe8q999n kgqdbaj6P90mIcNIATAKMWRq4XrEKdwkvI9/9XWX7dTIbG6N/TVQ2Gbtsr3hQH28EWjJ lrBrc6luGk5TlR1lYLELvN+YXWQGStEFB++OR96TO3seFEHcbGT7iGfBdRMpco4t3cDb 6YSYPO+NoIHUkeJe1vYspE6o6OkdhY/3hV5c/IBPJl78e90RAPBFDnKLRyLKN8uG1wjq WUqeHcz1B6Gb8neY1JYzM3mK/0xMprhaizF35lAFsePqvcHCpcGN/x/qzDUF6oNeCkj+ vMRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=yYx+AcpbxeLRdPqDImtMM4yf4Rg6f5T1yjgQiHsfUlY=; b=qtZza7fbL9oCpeDMv/A31ekUBy9vv7u/J6NTfgW1N0JC+u+N02kJWBN+lfRByVFQXd ySPrqRizyQycN+7HyPEpNkRGCMWb5GPZuvxytgO6U3xP54KXrjIDTphVHsi0trTMh1XV 9dEK7uRAFMurDJ9X+HE2A4PeRYBoRAuEQM+IfGgqIYmHfByNOwupdJUo2EmODuLIQlho zS6dLuZfsQAXfTdoiFJoZlS/sCZVAc0qmM7Rg9Bxl/u6NoAJL8uf9aeXeJ6w68XDLcMD q3AR1UWGeSIX10FKm56WXBWBsnc73Owpyw7Lkda4WAGbmVjBlfbtf8r3MH9eNurq2jnS WTIA== X-Gm-Message-State: ABuFfohGxkjaB9+8bjpTXSJ5RnLPm6zm1XmQkZu2V6HA0lBdUrz2c/N/ cdagFeBYcVoXrlPP5L8z0YXbJ1aJ6Mk= X-Google-Smtp-Source: ACcGV61+GUrPwZKPZ4dJ2PIzFj5ghXqJ9+NZoPRw9qtncUl4fv1V4OnL6quyl+BQ7p1JUG2FP/Nfsw== X-Received: by 2002:a0c:d97a:: with SMTP id t55mr7109584qvj.208.1539372796675; Fri, 12 Oct 2018 12:33:16 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g2-v6sm1065682qkf.91.2018.10.12.12.33.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:15 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 09/42] btrfs: release metadata before running delayed refs Date: Fri, 12 Oct 2018 15:32:23 -0400 Message-Id: <20181012193256.13735-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We want to release the unused reservation we have since it refills the delayed refs reserve, which will make everything go smoother when running the delayed refs if we're short on our reservation. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 117e0c4a914a..a0f19ca0bd6c 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1922,6 +1922,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } + btrfs_trans_release_metadata(trans); + trans->block_rsv = NULL; + /* make a pass through all the delayed refs we have so far * any runnings procs may add more while we are here */ @@ -1931,9 +1934,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } - btrfs_trans_release_metadata(trans); - trans->block_rsv = NULL; - cur_trans = trans->transaction; /* From patchwork Fri Oct 12 19:32:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639163 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AEC7A17E3 for ; Fri, 12 Oct 2018 19:33:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB191FF41 for ; Fri, 12 Oct 2018 19:33:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 94025201BD; Fri, 12 Oct 2018 19:33:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BF391FF41 for ; Fri, 12 Oct 2018 19:33:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726846AbeJMDHU (ORCPT ); Fri, 12 Oct 2018 23:07:20 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:45048 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHT (ORCPT ); Fri, 12 Oct 2018 23:07:19 -0400 Received: by mail-qt1-f195.google.com with SMTP id c56-v6so15043555qtd.11 for ; Fri, 12 Oct 2018 12:33:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=gKSGvLkh9Ok2LxOaudtm+P6nYNrWuwCgPKPCVFl6C1Q=; b=jhMkBXMPf5+C0w4bcEUFX1qSF/Koy9dEJNUXrEwpYkkS/b7yHxx3OGllE8ohfg1wV8 J9IwVXafhWNiA3+PQvtwQGp3FxyWqN+S5X6YqXAhVqlDCUpppcJm0L5Kgw7Mb78QAqnb PKpsuI/F1jkoXBnDqYCtfkBcSPxT+YLOvPqxV1LRFCsDkf9vzZu+z+oJDVccJM+f6HwY vCFaB1fQXpEvem1QIeJK0aS0VDJ/QVyWfc3cBIgqfdEaHB77xgO1enFFqO3WHwFBBH5D c4F4qTAEjIENBUL7ug8cdV6grXZ31TefXwYFtx2h1MC5R+eWbEKcborcyfkAOcHLMu7C Uqrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=gKSGvLkh9Ok2LxOaudtm+P6nYNrWuwCgPKPCVFl6C1Q=; b=k3pz3E6JaK70O8Ag+fl/LaXwjJ7l8PVbWZ71EwoVGxueGGB2Hz9ohknsRagd44FOVL g1RIh7I909UwbLaWnIerak5id9Ye+t3bNpjQ65Cf5WXgmircKH3/Bz37iTRN8udw81v7 sPdvjij46B5O2aECJHLKPpZP9MQKiUsMEu8Pzc3LY9ZLLzJ1HP2nYy5h2IHar3jxaUef 18mrk0BTNODrnpERAp7h24HLb9kDviiZcrkR291m+YaP7EXPOBRmUH4UYZ3egaKbf3t4 OogyCX50Nt/Phd3dsBIamXKP8vAVYSxIVdwF3sTNj64ohEkh9tBVjHtq+PPF4lAv4hwn d6rA== X-Gm-Message-State: ABuFfohaD+Wq8lrAsV3Vru4iuEwY9dK/aoGO8ZTKPN8T43MwsRRjd8zD 5oOZsJzSwPFRLd78DYMzUuCiz9HnEio= X-Google-Smtp-Source: ACcGV63KeDpZYcxEm/V4nRr2/c8OpXmMx9UcW4j8S597gdVYmluw+pxKDuD+m0qxrqk7PIhxR4iGhg== X-Received: by 2002:aed:3b7b:: with SMTP id q56-v6mr6837893qte.51.1539372798712; Fri, 12 Oct 2018 12:33:18 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id w5-v6sm1437897qkw.69.2018.10.12.12.33.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:17 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 10/42] btrfs: protect space cache inode alloc with nofs Date: Fri, 12 Oct 2018 15:32:24 -0400 Message-Id: <20181012193256.13735-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/free-space-cache.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index c3888c113d81..e077ad3b4549 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "ctree.h" #include "free-space-cache.h" #include "transaction.h" @@ -47,6 +48,7 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root, struct btrfs_free_space_header *header; struct extent_buffer *leaf; struct inode *inode = NULL; + unsigned nofs_flag; int ret; key.objectid = BTRFS_FREE_SPACE_OBJECTID; @@ -68,7 +70,13 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root, btrfs_disk_key_to_cpu(&location, &disk_key); btrfs_release_path(path); + /* + * We are often under a trans handle at this point, so we need to make + * sure NOFS is set to keep us from deadlocking. + */ + nofs_flag = memalloc_nofs_save(); inode = btrfs_iget(fs_info->sb, &location, root, NULL); + memalloc_nofs_restore(nofs_flag); if (IS_ERR(inode)) return inode; From patchwork Fri Oct 12 19:32:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639165 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECDB6157A for ; Fri, 12 Oct 2018 19:33:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEEC01FF41 for ; Fri, 12 Oct 2018 19:33:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D335A201BD; Fri, 12 Oct 2018 19:33:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 655C51FF41 for ; Fri, 12 Oct 2018 19:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726861AbeJMDHX (ORCPT ); Fri, 12 Oct 2018 23:07:23 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:33475 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHX (ORCPT ); Fri, 12 Oct 2018 23:07:23 -0400 Received: by mail-qt1-f194.google.com with SMTP id q40-v6so15083780qte.0 for ; Fri, 12 Oct 2018 12:33:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=09uUQnLegRaGDa0gAKaXhW+r7F/us0LEAwdMnUEs3rc=; b=hjUsEk57aQgcQAlklygzfUJWcpB011TV45foxKQ3boafHCWPqPcBhaSNdE16BuC6RB BoTIzsSVVuqkgVb4UNLboOzR3TGr9okTPknoesaVDEwY1kOQKkoHTGaoiBf3Asa4bUY7 Gt0++E5pm3s/3PC6BtHK75R341eWDV5jrnAjifiteVjRs0q9dvCLy3lvJ2TAuiZAjhSR WBKPZUxVTZWKxRQpLYSFfrZAldmxc3S4e1iFv2Z1rE8oMVbTKygHCj2WgdAkhEwPGI73 ogsbADsbEMJ7OYqmXce0WGlo4MAIoU+CiA9oNAk06ZdImMI2Z28lwHJ3Ft9j1pecG9i0 iqeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=09uUQnLegRaGDa0gAKaXhW+r7F/us0LEAwdMnUEs3rc=; b=elBQvMNygTKmkEDEBScaaXpGLr0Gwed4e53GWdjIU7pa0nWC0H4dAt4aIJ4w3SUlUo 95XtlW2BIeWCvUmJ7HseVcYU4A34/VWzETsulK2sa0vqlNOU+1wpHD5PbBGdTuXran4v zS0TUFn/uvAOr0kwpfl+WyIbkLSr+Wzxtle4XgnYiRYx9jzngTGsDpPZpPjJe+WUXma4 BT/g5hyBr6cQXCi5CIYmUtg1vOwngL2m1Gz63u8/i+jLIXhmw6RFwLTaJyKnVNwHmjtD X0ghrk0tuDcBOgxJlBWeXNAVjbo8t+kiVmZ7b0MxVLBLtJI7cYREqcUySg4B9U31Lpya s+DA== X-Gm-Message-State: ABuFfoj08Omzwj76a7MxSu4CQf5gK9TJMV/B3sGdQ64ij7msvkcxx1VV ymtLXPyR1COH6gPoVpw2AjS5ZiVeHXQ= X-Google-Smtp-Source: ACcGV62jymBiZTblRGNfkbQug4eiM72uSf5Buo2UFMW40KdbK/WBf0ubG4w2aiRmQ4RI0NR5CA3uPA== X-Received: by 2002:a0c:d1d6:: with SMTP id k22-v6mr7581355qvh.156.1539372801376; Fri, 12 Oct 2018 12:33:21 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m9-v6sm1697236qkl.32.2018.10.12.12.33.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:20 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 11/42] btrfs: fix truncate throttling Date: Fri, 12 Oct 2018 15:32:25 -0400 Message-Id: <20181012193256.13735-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We have a bunch of magic to make sure we're throttling delayed refs when truncating a file. Now that we have a delayed refs rsv and a mechanism for refilling that reserve simply use that instead of all of this magic. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 79 ++++++++++++-------------------------------------------- 1 file changed, 17 insertions(+), 62 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index cd00ec869c96..5a91055a13b2 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -4493,31 +4493,6 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry) return err; } -static int truncate_space_check(struct btrfs_trans_handle *trans, - struct btrfs_root *root, - u64 bytes_deleted) -{ - struct btrfs_fs_info *fs_info = root->fs_info; - int ret; - - /* - * This is only used to apply pressure to the enospc system, we don't - * intend to use this reservation at all. - */ - bytes_deleted = btrfs_csum_bytes_to_leaves(fs_info, bytes_deleted); - bytes_deleted *= fs_info->nodesize; - ret = btrfs_block_rsv_add(root, &fs_info->trans_block_rsv, - bytes_deleted, BTRFS_RESERVE_NO_FLUSH); - if (!ret) { - trace_btrfs_space_reservation(fs_info, "transaction", - trans->transid, - bytes_deleted, 1); - trans->bytes_reserved += bytes_deleted; - } - return ret; - -} - /* * Return this if we need to call truncate_block for the last bit of the * truncate. @@ -4562,7 +4537,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u64 bytes_deleted = 0; bool be_nice = false; bool should_throttle = false; - bool should_end = false; BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY); @@ -4775,15 +4749,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, btrfs_abort_transaction(trans, ret); break; } - if (btrfs_should_throttle_delayed_refs(trans, fs_info)) - btrfs_async_run_delayed_refs(fs_info, - trans->delayed_ref_updates * 2, - trans->transid, 0); if (be_nice) { - if (truncate_space_check(trans, root, - extent_num_bytes)) { - should_end = true; - } if (btrfs_should_throttle_delayed_refs(trans, fs_info)) should_throttle = true; @@ -4795,7 +4761,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, if (path->slots[0] == 0 || path->slots[0] != pending_del_slot || - should_throttle || should_end) { + should_throttle) { if (pending_del_nr) { ret = btrfs_del_items(trans, root, path, pending_del_slot, @@ -4807,23 +4773,24 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, pending_del_nr = 0; } btrfs_release_path(path); - if (should_throttle) { - unsigned long updates = trans->delayed_ref_updates; - if (updates) { - trans->delayed_ref_updates = 0; - ret = btrfs_run_delayed_refs(trans, - updates * 2); - if (ret) - break; - } - } + /* - * if we failed to refill our space rsv, bail out - * and let the transaction restart + * We can generate a lot of delayed refs, so we need to + * throttle every once and a while and make sure we're + * adding enough space to keep up with the work we are + * generating. Since we hold a transaction here we + * can't flush, and we don't want to FLUSH_LIMIT because + * we could have generated too many delayed refs to + * actually allocate, so just bail if we're short and + * let the normal reservation dance happen higher up. */ - if (should_end) { - ret = -EAGAIN; - break; + if (should_throttle) { + ret = btrfs_throttle_delayed_refs(fs_info, + BTRFS_RESERVE_NO_FLUSH); + if (ret) { + ret = -EAGAIN; + break; + } } goto search_again; } else { @@ -4849,18 +4816,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, } btrfs_free_path(path); - - if (be_nice && bytes_deleted > SZ_32M && (ret >= 0 || ret == -EAGAIN)) { - unsigned long updates = trans->delayed_ref_updates; - int err; - - if (updates) { - trans->delayed_ref_updates = 0; - err = btrfs_run_delayed_refs(trans, updates * 2); - if (err) - ret = err; - } - } return ret; } From patchwork Fri Oct 12 19:32:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639167 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5398A17E3 for ; Fri, 12 Oct 2018 19:33:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4697B1FF41 for ; Fri, 12 Oct 2018 19:33:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3B208201BD; Fri, 12 Oct 2018 19:33:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB6BA1FF41 for ; Fri, 12 Oct 2018 19:33:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726868AbeJMDHZ (ORCPT ); Fri, 12 Oct 2018 23:07:25 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:33482 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHZ (ORCPT ); Fri, 12 Oct 2018 23:07:25 -0400 Received: by mail-qt1-f195.google.com with SMTP id q40-v6so15083925qte.0 for ; Fri, 12 Oct 2018 12:33:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=PXodpEXCKEY9gKL1VbyQfz/+OWJov9JEkUK4lJNrhOs=; b=WJbSrUR069pa+1pwBlv0QiVg2G4G02o12/TKTQ6uHM1K2SwSyoymUNr8CZgn0lIlz/ +DhuOqXTDKN72d0hr42clHg0fDXRB7m/tNkuCD7nj/L9nPEcEKsycnCiM//vdd6X6A6x kKGZ5jsIkMNxPzb00iyUe0N4oHVtxBC2zGFJELIx5KL83XWLov94MD8/cOcvJYlkP7DL KgV068OcHFCL2zTAmvR9xzc7xyNdNnwTcWj214qmDqmUOFLVXT7ixuLYjmGf5hS+9ZGn /lOVHhmjDURqO3dmKtIPa/0etTzhE/N/hZm3gTUL37b8UAQrIEVJUR2qQS2Dm3+WMAMi IppQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=PXodpEXCKEY9gKL1VbyQfz/+OWJov9JEkUK4lJNrhOs=; b=ZTB/c1sXp7kIO1C0LrCPIZIuzLKmiIKkv9HjIb4pU3zF0rHxo35f0L5uJyKYOLBtgq tRWvCEcneSesDsTzeDlLcVbR5x1MDYWAzqWW+i5024TyaCLGWUY7jT9WDaDd7OTH9IsK yWFfB1Yv0qzz8B0qph05lT84HRx2bQDlg/j7crpUO67CS3soO+fEaJ8TVsovARhQ09U4 IEsNmfL13sKKjFHuNSBhcJe2Bz6h9x9c/bnfWmXhk6zIHz1AvqhwaYd2k+TH51kOitJM OCpdt0yY5hnhg/nPw39oiaaISPUXGRf+VKknpEQY1962dGOsYMDTj3QPbAksvzzW59v8 /h1g== X-Gm-Message-State: ABuFfogqtSiJf84prtAcMDVgNFA2fOdo5I2ylCI5nSIAyKnm52TzmtSA tuGZgD22FHJIvHOQzmzkwRh4tpwQwZo= X-Google-Smtp-Source: ACcGV61U9dz/1XkxYb9I+163Akl2vqfKIxlY/c7e7nA+jveCRuu33AwgDhHUuXhjwS3UB0YbHp0kTw== X-Received: by 2002:aed:2747:: with SMTP id n65-v6mr7205513qtd.103.1539372804359; Fri, 12 Oct 2018 12:33:24 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id x68-v6sm966630qkd.50.2018.10.12.12.33.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:23 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 12/42] btrfs: don't use global rsv for chunk allocation Date: Fri, 12 Oct 2018 15:32:26 -0400 Message-Id: <20181012193256.13735-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We've done this forever because of the voodoo around knowing how much space we have. However we have better ways of doing this now, and on normal file systems we'll easily have a global reserve of 512MiB, and since metadata chunks are usually 1GiB that means we'll allocate metadata chunks more readily. Instead use the actual used amount when determining if we need to allocate a chunk or not. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 8b00c658deb3..828b82db439c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4374,21 +4374,12 @@ static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global) static int should_alloc_chunk(struct btrfs_fs_info *fs_info, struct btrfs_space_info *sinfo, int force) { - struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; u64 bytes_used = btrfs_space_info_used(sinfo, false); u64 thresh; if (force == CHUNK_ALLOC_FORCE) return 1; - /* - * We need to take into account the global rsv because for all intents - * and purposes it's used space. Don't worry about locking the - * global_rsv, it doesn't change except when the transaction commits. - */ - if (sinfo->flags & BTRFS_BLOCK_GROUP_METADATA) - bytes_used += calc_global_rsv_need_space(global_rsv); - /* * in limited mode, we want to have some free space up to * about 1% of the FS size. From patchwork Fri Oct 12 19:32:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639169 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57D4017E3 for ; Fri, 12 Oct 2018 19:33:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47CC51FF41 for ; Fri, 12 Oct 2018 19:33:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3BD97201BD; Fri, 12 Oct 2018 19:33:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C65241FF41 for ; Fri, 12 Oct 2018 19:33:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726871AbeJMDH2 (ORCPT ); Fri, 12 Oct 2018 23:07:28 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:35828 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDH2 (ORCPT ); Fri, 12 Oct 2018 23:07:28 -0400 Received: by mail-qt1-f196.google.com with SMTP id d21-v6so2675594qtq.2 for ; Fri, 12 Oct 2018 12:33:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=09Q7MBchTgKXgOMnrZdzeN5nZLK67ToDfhlEUzOVZJY=; b=Yp4MgjXC8pPd3N14awl7gbolXvQDqn+FPRYiSgxhuN7HyEw8AYcezPKlfUu3jhdjBZ hEWuK1rR9v1wMIOgkxiiCpDk5NWd+m+HBb7wmkqWghBcZ+aGU7RzlIsqGJx5TlSfsZ8q VLLBzbMHNLVwCLJZwgOPhA3x3QPTKafZ7ix+GjOsy/kzi3YdTnsY5yk04K/m44/Kp8ek /cGf3ABKS9a73gK38bQPsWsRMNtWtoMZsJM8LZHvu2CAx0fa0JeEVdrVTAokYIXpOIz1 0dlkHlHUqogaxx7FUoEYu/VNoWcOgYuHtPTD2dEFLJDjNLz62a/K7ifSS4LAj1exFoKT LFGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=09Q7MBchTgKXgOMnrZdzeN5nZLK67ToDfhlEUzOVZJY=; b=guLYbdaVG3/jxRiS3psiWtCtLzKe6vcCObQ4cV0RTCh2RwzdAbOhA3kWd3OVmBxgrD 398xJd5gldwCuJLxQGleXkmXKYqY7uMrpj5xllgq6/seYg7226gawSuCEntxAsbLGxvq l/Xn4JhuhqsJa5fU1IJxrRdcLQbiKZzme9Y8x5m59CHqRl6Or+BdRm0d2B0QH/JBoUG1 ZzpWgBTLbGPyfpvw9rJmgrCDJC1tZ3w155xk8l3INo0pExuw2ojLVi22kGFzGbhp5BNy 0K8rZe513UJ5s8/MnVRRD2dCbmYkEl5dbL+KzlhEB1fLWUWVt3yqNp38q86CbYFZmI78 nibw== X-Gm-Message-State: ABuFfojnqmr2kpwE5gya0PbkOqpJRqDq6fmkfrn3N4PSRlSDlLuPxgUP vdEo0+OeqruMse9EvVu/AxkqPVqvYC0= X-Google-Smtp-Source: ACcGV60EnPxQBmj7EOmYcH3+N90V3xOhUqlvst2ODmAQOg179yLec4O9WySLmv9XhBI/317anXetrQ== X-Received: by 2002:ac8:4590:: with SMTP id l16-v6mr6953750qtn.385.1539372807215; Fri, 12 Oct 2018 12:33:27 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id j57-v6sm1293746qtj.96.2018.10.12.12.33.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:26 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 13/42] btrfs: add ALLOC_CHUNK_FORCE to the flushing code Date: Fri, 12 Oct 2018 15:32:27 -0400 Message-Id: <20181012193256.13735-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With my change to no longer take into account the global reserve for metadata allocation chunks we have this side-effect for mixed block group fs'es where we are no longer allocating enough chunks for the data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE step to the flushing state machine. This will only get used if we've already made a full loop through the flushing machinery and tried committing the transaction. If we have then we can try and force a chunk allocation since we likely need it to make progress. This resolves the issues I was seeing with the mixed bg tests in xfstests with my previous patch. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 ++- fs/btrfs/extent-tree.c | 18 +++++++++++++++++- include/trace/events/btrfs.h | 1 + 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 1a2c3b629af2..29db902511c1 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2730,7 +2730,8 @@ enum btrfs_flush_state { FLUSH_DELALLOC = 5, FLUSH_DELALLOC_WAIT = 6, ALLOC_CHUNK = 7, - COMMIT_TRANS = 8, + ALLOC_CHUNK_FORCE = 8, + COMMIT_TRANS = 9, }; int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 828b82db439c..640ec640b21c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4914,6 +4914,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_end_transaction(trans); break; case ALLOC_CHUNK: + case ALLOC_CHUNK_FORCE: trans = btrfs_join_transaction(root); if (IS_ERR(trans)) { ret = PTR_ERR(trans); @@ -4921,7 +4922,9 @@ static void flush_space(struct btrfs_fs_info *fs_info, } ret = do_chunk_alloc(trans, btrfs_metadata_alloc_profile(fs_info), - CHUNK_ALLOC_NO_FORCE); + (state == ALLOC_CHUNK) ? + CHUNK_ALLOC_NO_FORCE : + CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); if (ret > 0 || ret == -ENOSPC) ret = 0; @@ -5057,6 +5060,19 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) commit_cycles--; } + /* + * We don't want to force a chunk allocation until we've tried + * pretty hard to reclaim space. Think of the case where we + * free'd up a bunch of space and so have a lot of pinned space + * to reclaim. We would rather use that than possibly create a + * underutilized metadata chunk. So if this is our first run + * through the flushing state machine skip ALLOC_CHUNK_FORCE and + * commit the transaction. If nothing has changed the next go + * around then we can force a chunk allocation. + */ + if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles) + flush_state++; + if (flush_state > COMMIT_TRANS) { commit_cycles++; if (commit_cycles > 2) { diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index 7d205e50b09c..fdb23181b5b7 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -1051,6 +1051,7 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ + { ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) TRACE_EVENT(btrfs_flush_space, From patchwork Fri Oct 12 19:32:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639171 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32E7D157A for ; Fri, 12 Oct 2018 19:33:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24ACF1FF41 for ; Fri, 12 Oct 2018 19:33:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 18EFE201BD; Fri, 12 Oct 2018 19:33:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8A9E1FF41 for ; Fri, 12 Oct 2018 19:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726887AbeJMDHb (ORCPT ); Fri, 12 Oct 2018 23:07:31 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:43077 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHb (ORCPT ); Fri, 12 Oct 2018 23:07:31 -0400 Received: by mail-qt1-f193.google.com with SMTP id q41-v6so15042979qtq.10 for ; Fri, 12 Oct 2018 12:33:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=M5rFuwWprh+KHg3eMVqEhuKS6R++ogrBjwetuDUUZ8A=; b=hjqGgyVxPZEGhBBgr6//k4jMJqu0HhwTTF/4cIsqVG+9oBK9WAbNYs8kmTp/HGdIKf FFzZuea+GTSrNLe0+8WArAFdaf8GSytE1zMHANNo9+7egRrIpvW9nVZiX8V7QGtehe2g zml9+r9o3oVVdEAiqd/2UDZeKg3HTr3fD2Amfagjy8zH2FBX3SQsZGGYYNtHWs6CUftU fiJEdup3xdYyPNhYHmfsBURmIhHtOBrN+TkfMO3EM7t9Zj8QIynY6qs0HkXjytFhxKoz Hg278X5gq2pggNHY/xFi7EmgN5xERP9Gak7qff5TyDu2B7DHtvJqhSr3qhsEcxowSVgG lMsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=M5rFuwWprh+KHg3eMVqEhuKS6R++ogrBjwetuDUUZ8A=; b=YA3QQ+Io3ri8xp9PUyPLDoLKKeSSVZnKgUmnLwtLbm10dysh4r/ePfj+zGCtbV2ad6 /Hp/f0JtrO50KTQYMhVtRPrqGO470kXTGvv4FnApe0AeWGSxl2oypn0BHSkV2SKzGyjp glg3foPKule0EgwTy1KKOzOelTgIikNBLVyAs8F7xS6Hn+T0PGAg2EzEj4tYzVSGVt45 ketzEZrYUjQjjmRYMs65fnJuv8WrPM7/CM0dxwxgg1M8o2dbz6l596XG0eKyooh7J0IS c5WLVnZ+HyCyoVTYPNdMbrlGuxFZEqDHdOcF1FzgxwZT7ZDGvRHCq7hKO+Xpw0W+FhZi 9/xQ== X-Gm-Message-State: ABuFfogq1RG4wKTTygm6aesSxiEjcPuGrqQOkqra8wihBs5ISZeSC0nc meBZ3uD8LjO6iulNvhDSILcc3xxpwaU= X-Google-Smtp-Source: ACcGV63qY60JYWwe0TPF71E+xjlz28IoRCFWz0Bk0q6sswtGdtdalfNCSBmjd70/23gBhwoe6WF+Ug== X-Received: by 2002:ac8:6bc8:: with SMTP id b8-v6mr7086591qtt.213.1539372810311; Fri, 12 Oct 2018 12:33:30 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id i26-v6sm1637167qta.34.2018.10.12.12.33.28 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:29 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 14/42] btrfs: reset max_extent_size properly Date: Fri, 12 Oct 2018 15:32:28 -0400 Message-Id: <20181012193256.13735-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we use up our block group before allocating a new one we'll easily get a max_extent_size that's set really really low, which will result in a lot of fragmentation. We need to make sure we're resetting the max_extent_size when we add a new chunk or add new space. Reviewed-by: Filipe Manana Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 640ec640b21c..2b1704331d21 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4573,6 +4573,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags, goto out; } else { ret = 1; + space_info->max_extent_size = 0; } space_info->force_alloc = CHUNK_ALLOC_NO_FORCE; @@ -6671,6 +6672,7 @@ static int btrfs_free_reserved_bytes(struct btrfs_block_group_cache *cache, space_info->bytes_readonly += num_bytes; cache->reserved -= num_bytes; space_info->bytes_reserved -= num_bytes; + space_info->max_extent_size = 0; if (delalloc) cache->delalloc_bytes -= num_bytes; From patchwork Fri Oct 12 19:32:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639173 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E00417E3 for ; Fri, 12 Oct 2018 19:33:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F2AA1FF41 for ; Fri, 12 Oct 2018 19:33:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 537EF201BD; Fri, 12 Oct 2018 19:33:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C98461FF41 for ; Fri, 12 Oct 2018 19:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726932AbeJMDHf (ORCPT ); Fri, 12 Oct 2018 23:07:35 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:34817 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHe (ORCPT ); Fri, 12 Oct 2018 23:07:34 -0400 Received: by mail-qt1-f194.google.com with SMTP id d21-v6so2675854qtq.2 for ; Fri, 12 Oct 2018 12:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=uGlgRfNL5ut1DOfgOnKRZEG30dKHwa5Lx3HW3k8vIVw=; b=FvPAm/EmzzVekkz9WT5HX/r6eXI76EH7uq0UPZdZ0TBBnZuLyF9OVPZzDxAyh65Mc9 U4QQco3fyymunKZYc7TC6//DJsjWQlASthw/pUbhytp/wJw7s6k+cl7MWkk3fgHZS7YX qUyijDHYYdggylKmV9Rn/H0V7JlF6hYOFTIx41jbISg+PuKE3XB0e9QzoQNmfeiKZtDX Mwz90wlQngWP451zJARWleoOgG/lKHA93zGeB+RvTCvzZoKNk1xiBn4q2Vqe5vHyzJiL kM2zv44bjY5pWkfKec5efpTofIHY4LIBMK/7RTKgsuSMDTV7ixP4g8/27848h1gw8zV+ TzbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=uGlgRfNL5ut1DOfgOnKRZEG30dKHwa5Lx3HW3k8vIVw=; b=nVHlcFV0FeXtbKqGzoq9naCt21CrOPW2MhQ/36TQ5iaIjQgGyYcq9XMxSxxsgOY7Dx uI1Wy6Iur9imBi56inELNUwOVnEfY0UN4KpRuuvdAhcMdZb7WsWpHJ37LhmXeKql4OJH S9RpFZ6RxlUlHq7wuxYH95jrwxfxJJK6NTCbj96+j2Ng4imGv77lc04ZAXx6JBPABWoh 2BXwCIyq88HEzcG5CgO18/bnAtPIkFH4uBN6Zt4u5JdKGyrMJLcS/tgaaRoPVRG7Yu7Z jyZ67NaP1wxrIUrI3EPVniv735yfUD41aG40j0M2W7SkMws+WKBOdhM8AblY//mgR1SG G7sw== X-Gm-Message-State: ABuFfohEvMkRfMKmlWWOWOzGACPGyn6n8tV7+3UDNeaMdUb4F5BqL4Pz CmU9SuiVeylSyczGxmKouZ4ihX4xjXw= X-Google-Smtp-Source: ACcGV61lFyjJ8uZ0JNfC2eTAi01YEV23Tg2sHeycQGQIMWtHI8PstQdvI0qAvsQzRHaHa2rnXdqWNA== X-Received: by 2002:ac8:101a:: with SMTP id z26-v6mr6958319qti.308.1539372812839; Fri, 12 Oct 2018 12:33:32 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id r82-v6sm1947475qkh.28.2018.10.12.12.33.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:31 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 15/42] btrfs: don't enospc all tickets on flush failure Date: Fri, 12 Oct 2018 15:32:29 -0400 Message-Id: <20181012193256.13735-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With the introduction of the per-inode block_rsv it became possible to have really really large reservation requests made because of data fragmentation. Since the ticket stuff assumed that we'd always have relatively small reservation requests it just killed all tickets if we were unable to satisfy the current request. However this is generally not the case anymore. So fix this logic to instead see if we had a ticket that we were able to give some reservation to, and if we were continue the flushing loop again. Likewise we make the tickets use the space_info_add_old_bytes() method of returning what reservation they did receive in hopes that it could satisfy reservations down the line. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 2b1704331d21..19449ee93693 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4779,6 +4779,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } struct reserve_ticket { + u64 orig_bytes; u64 bytes; int error; struct list_head list; @@ -5000,7 +5001,7 @@ static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info, !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)); } -static void wake_all_tickets(struct list_head *head) +static bool wake_all_tickets(struct list_head *head) { struct reserve_ticket *ticket; @@ -5009,7 +5010,10 @@ static void wake_all_tickets(struct list_head *head) list_del_init(&ticket->list); ticket->error = -ENOSPC; wake_up(&ticket->wait); + if (ticket->bytes != ticket->orig_bytes) + return true; } + return false; } /* @@ -5077,8 +5081,12 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) if (flush_state > COMMIT_TRANS) { commit_cycles++; if (commit_cycles > 2) { - wake_all_tickets(&space_info->tickets); - space_info->flush = 0; + if (wake_all_tickets(&space_info->tickets)) { + flush_state = FLUSH_DELAYED_ITEMS_NR; + commit_cycles--; + } else { + space_info->flush = 0; + } } else { flush_state = FLUSH_DELAYED_ITEMS_NR; } @@ -5130,10 +5138,11 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, u64 orig_bytes) + struct reserve_ticket *ticket) { DEFINE_WAIT(wait); + u64 reclaim_bytes = 0; int ret = 0; spin_lock(&space_info->lock); @@ -5154,14 +5163,12 @@ static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, ret = ticket->error; if (!list_empty(&ticket->list)) list_del_init(&ticket->list); - if (ticket->bytes && ticket->bytes < orig_bytes) { - u64 num_bytes = orig_bytes - ticket->bytes; - space_info->bytes_may_use -= num_bytes; - trace_btrfs_space_reservation(fs_info, "space_info", - space_info->flags, num_bytes, 0); - } + if (ticket->bytes && ticket->bytes < ticket->orig_bytes) + reclaim_bytes = ticket->orig_bytes - ticket->bytes; spin_unlock(&space_info->lock); + if (reclaim_bytes) + space_info_add_old_bytes(fs_info, space_info, reclaim_bytes); return ret; } @@ -5187,6 +5194,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, { struct reserve_ticket ticket; u64 used; + u64 reclaim_bytes = 0; int ret = 0; ASSERT(orig_bytes); @@ -5222,6 +5230,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, * the list and we will do our own flushing further down. */ if (ret && flush != BTRFS_RESERVE_NO_FLUSH) { + ticket.orig_bytes = orig_bytes; ticket.bytes = orig_bytes; ticket.error = 0; init_waitqueue_head(&ticket.wait); @@ -5262,25 +5271,21 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, return ret; if (flush == BTRFS_RESERVE_FLUSH_ALL) - return wait_reserve_ticket(fs_info, space_info, &ticket, - orig_bytes); + return wait_reserve_ticket(fs_info, space_info, &ticket); ret = 0; priority_reclaim_metadata_space(fs_info, space_info, &ticket); spin_lock(&space_info->lock); if (ticket.bytes) { - if (ticket.bytes < orig_bytes) { - u64 num_bytes = orig_bytes - ticket.bytes; - space_info->bytes_may_use -= num_bytes; - trace_btrfs_space_reservation(fs_info, "space_info", - space_info->flags, - num_bytes, 0); - - } + if (ticket.bytes < orig_bytes) + reclaim_bytes = orig_bytes - ticket.bytes; list_del_init(&ticket.list); ret = -ENOSPC; } spin_unlock(&space_info->lock); + + if (reclaim_bytes) + space_info_add_old_bytes(fs_info, space_info, reclaim_bytes); ASSERT(list_empty(&ticket.list)); return ret; } From patchwork Fri Oct 12 19:32:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639179 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4DBD217E3 for ; Fri, 12 Oct 2018 19:33:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F00A2C546 for ; Fri, 12 Oct 2018 19:33:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3392D2C555; Fri, 12 Oct 2018 19:33:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF2772C546 for ; Fri, 12 Oct 2018 19:33:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727002AbeJMDHq (ORCPT ); Fri, 12 Oct 2018 23:07:46 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34329 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbeJMDHh (ORCPT ); Fri, 12 Oct 2018 23:07:37 -0400 Received: by mail-qt1-f195.google.com with SMTP id o17-v6so15081949qtr.1 for ; Fri, 12 Oct 2018 12:33:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=vHabzEqM4andqo36SRSVv9D6V894NYsX3WbIWx+RRlM=; b=NKttTl5l4WQbbj+Wy9FAfdXXirZUyutam7wnpsBICTZdfDVCpMCQ5/4PWQeQ8gpW16 gbj28GiscKxeF9OJLb/+yT3W4Uh01F6jcHa5K64LEhtiUV35HEermkQAZ7qCdh/C4Orm TsySBzGQRln4DPJ2HIb4K1UXsTArvqYRfdiSi0IlDAO6lkLUvao8Q48uQysr/gWGgMIn riMOOP44Vz1EbUw+lgcXvpFn8TPFCK21jCQ3o/2BQiLLwDlwY1yEJNFkF9fkitf96BNP QhuP0HRB3O88I2AF3b/HGu+G9qDEOVoIdr0OnHd6ayBN4e0DPhfDFkT7Ku+JErlQfHif UnBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=vHabzEqM4andqo36SRSVv9D6V894NYsX3WbIWx+RRlM=; b=YdNJpoxp5MJ5xer66vPHTm6D+OrSlhnpwUklE7d/WqX6cj6Rvv1ZCsLmAlf1zEjJ3w m03EkYuJrSHnN+KjfE6gSyiuEDSqUUrvBzeYOuXbZl+lIBFVYvIDc5sOiEl9CIk90IAi jAgGT+hhL4IQALLoao9XNBLMSJJvSRFqTm7+PwGyQ/ETdFNuoxS4Ncl8qAHHxm+KQ78J GK+xpQASry88qqoKt7r/OJYb9Gm0e7K4OkfN6whNvONRpr0J+t3fnoTdzAGfZtNEMSwY q0bZl7UG8UepcEQbnAUV5HP0a5+ZGttkwJ6ouZBOdrB4Nr8arU+9zGWIieYyHPW703Jp UtRw== X-Gm-Message-State: ABuFfohuloQIO2xYJwul2VRAUAgQzFUHz/+gYGrc3ZkLY9zdBdFhcJCQ 7jBf3eiF5bhZy0K1ZKyFOhKxdLkS7aw= X-Google-Smtp-Source: ACcGV61YkOSjfdg94+24oMvmZoDR6to2HAaIU1vZl9DU0RC8Eyc8/pGWhIFznVYnJX/SPdqG8q/c8w== X-Received: by 2002:ac8:2278:: with SMTP id p53-v6mr7136442qtp.200.1539372815530; Fri, 12 Oct 2018 12:33:35 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id z46-v6sm1589896qth.87.2018.10.12.12.33.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:34 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 16/42] btrfs: loop in inode_rsv_refill Date: Fri, 12 Oct 2018 15:32:30 -0400 Message-Id: <20181012193256.13735-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With severe fragmentation we can end up with our inode rsv size being huge during writeout, which would cause us to need to make very large metadata reservations. However we may not actually need that much once writeout is complete. So instead try to make our reservation, and if we couldn't make it re-calculate our new reservation size and try again. If our reservation size doesn't change between tries then we know we are actually out of space and can error out. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 56 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 19449ee93693..3757ca42b8e0 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5766,6 +5766,21 @@ int btrfs_block_rsv_refill(struct btrfs_root *root, return ret; } +static inline void __get_refill_bytes(struct btrfs_block_rsv *block_rsv, + u64 *metadata_bytes, u64 *qgroup_bytes) +{ + *metadata_bytes = 0; + *qgroup_bytes = 0; + + spin_lock(&block_rsv->lock); + if (block_rsv->reserved < block_rsv->size) + *metadata_bytes = block_rsv->size - block_rsv->reserved; + if (block_rsv->qgroup_rsv_reserved < block_rsv->qgroup_rsv_size) + *qgroup_bytes = block_rsv->qgroup_rsv_size - + block_rsv->qgroup_rsv_reserved; + spin_unlock(&block_rsv->lock); +} + /** * btrfs_inode_rsv_refill - refill the inode block rsv. * @inode - the inode we are refilling. @@ -5781,25 +5796,37 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, { struct btrfs_root *root = inode->root; struct btrfs_block_rsv *block_rsv = &inode->block_rsv; - u64 num_bytes = 0; + u64 num_bytes = 0, last = 0; u64 qgroup_num_bytes = 0; int ret = -ENOSPC; - spin_lock(&block_rsv->lock); - if (block_rsv->reserved < block_rsv->size) - num_bytes = block_rsv->size - block_rsv->reserved; - if (block_rsv->qgroup_rsv_reserved < block_rsv->qgroup_rsv_size) - qgroup_num_bytes = block_rsv->qgroup_rsv_size - - block_rsv->qgroup_rsv_reserved; - spin_unlock(&block_rsv->lock); - + __get_refill_bytes(block_rsv, &num_bytes, &qgroup_num_bytes); if (num_bytes == 0) return 0; - ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_num_bytes, true); - if (ret) - return ret; - ret = reserve_metadata_bytes(root, block_rsv, num_bytes, flush); + do { + ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_num_bytes, true); + if (ret) + return ret; + ret = reserve_metadata_bytes(root, block_rsv, num_bytes, flush); + if (ret) { + btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes); + last = num_bytes; + /* + * If we are fragmented we can end up with a lot of + * outstanding extents which will make our size be much + * larger than our reserved amount. If we happen to + * try to do a reservation here that may result in us + * trying to do a pretty hefty reservation, which we may + * not need once delalloc flushing happens. If this is + * the case try and do the reserve again. + */ + if (flush == BTRFS_RESERVE_FLUSH_ALL) + __get_refill_bytes(block_rsv, &num_bytes, + &qgroup_num_bytes); + } + } while (ret && last != num_bytes); + if (!ret) { block_rsv_add_bytes(block_rsv, num_bytes, 0); trace_btrfs_space_reservation(root->fs_info, "delalloc", @@ -5809,8 +5836,7 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, spin_lock(&block_rsv->lock); block_rsv->qgroup_rsv_reserved += qgroup_num_bytes; spin_unlock(&block_rsv->lock); - } else - btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes); + } return ret; } From patchwork Fri Oct 12 19:32:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639175 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6530157A for ; Fri, 12 Oct 2018 19:33:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 953832C54A for ; Fri, 12 Oct 2018 19:33:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 836992C546; Fri, 12 Oct 2018 19:33:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2CBFE2C546 for ; Fri, 12 Oct 2018 19:33:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726982AbeJMDHl (ORCPT ); Fri, 12 Oct 2018 23:07:41 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:39777 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726933AbeJMDHj (ORCPT ); Fri, 12 Oct 2018 23:07:39 -0400 Received: by mail-qt1-f195.google.com with SMTP id e22-v6so15072445qto.6 for ; Fri, 12 Oct 2018 12:33:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=2w+WsPjL8r93Nw50uJe/lUsSq3OzjnbW2tYS17GINdo=; b=hhQw7Se96f1p4VOc1r9EdzOg4KBxP4LHpXyoYWaLT7lSJwmED/rJAynkWaE294Aegj lITxhCN4MNX8n+uZjBJUMDHgUwb1l9D3lib/awxO4ZJl/yWbo4NrbinjrCWX2GtD36zz p9BYyscRhUxVIXoJagyocQl+aD/OtzpUtIDGEifnoxX1cSy3OxS/V3xCWW3+DqV0r/HD RgUqtkPJZiFju+8ZqZBq8NOdGB2R9httDoFtgKu0235nO4xcBFKBdn7rWiYpKkpbwGkV OuQ8ibYQbXfdXAI0bJlPPAP3ppOMTe4g6Id28O57lKBl0IDTatMdL4Lpw3BBa7EeZHXF LR+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=2w+WsPjL8r93Nw50uJe/lUsSq3OzjnbW2tYS17GINdo=; b=rIHW/DXvHPYtD1T5qB2Q5HVSOiDs82w1AJ3vOXNJy49BmJWEeoDtDzCAkJLIt65vmz Urq3ZlCdNoD80LxUKapownTe6p8/dIPEhkr+/MI/Ij0R4jVK4nC2iPIsL0Ik+OkbP8ui XyVQZaYJvCNSPIbfTA0tWYC1zfTKt1s6Sq51+vMSmNnoz5R4BB3Bxm0s+ICvf80DTrsN ptzw6To8NMmPSyNJchq+xq7wm3sl+AaEdLScKfhusUMCISz0v4YZ0tba1XozNmuwyseA OSU+K/9lkj2gQSLOmTPSFgxhslzm5gh3N1E/SXbHUOTfmYbaslvIigscednJyQZEB0YD 7T9w== X-Gm-Message-State: ABuFfoi2VSLmah3ilBpstXvWj2N77pz4R0LwcHlaY7ucJ+jwE+ATlakA J5TF2G/x5MowVNnu1HAIMG5OPLeyv9U= X-Google-Smtp-Source: ACcGV62BOG5DUNHqLMvqHS1gYB3bdjO52svtVrMu8I+36bBLSw3F2zlkL6EEm15lhzwrgrsqiHscbA== X-Received: by 2002:ac8:46da:: with SMTP id h26-v6mr7124015qto.369.1539372818368; Fri, 12 Oct 2018 12:33:38 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id w17-v6sm1602437qth.47.2018.10.12.12.33.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:37 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 17/42] btrfs: run delayed iputs before committing Date: Fri, 12 Oct 2018 15:32:31 -0400 Message-Id: <20181012193256.13735-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Delayed iputs means we can have final iputs of deleted inodes in the queue, which could potentially generate a lot of pinned space that could be free'd. So before we decide to commit the transaction for ENOPSC reasons, run the delayed iputs so that any potential space is free'd up. If there is and we freed enough we can then commit the transaction and potentially be able to make our reservation. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 3757ca42b8e0..ef852d1911d1 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4823,6 +4823,15 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes) return 0; + /* + * If we have pending delayed iputs then we could free up a bunch of + * pinned space, so make sure we run the iputs before we do our pinned + * bytes check below. + */ + mutex_lock(&fs_info->cleaner_delayed_iput_mutex); + btrfs_run_delayed_iputs(fs_info); + mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); From patchwork Fri Oct 12 19:32:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639177 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BFC4157A for ; Fri, 12 Oct 2018 19:33:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D8262C546 for ; Fri, 12 Oct 2018 19:33:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 720502C555; Fri, 12 Oct 2018 19:33:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3F772C546 for ; Fri, 12 Oct 2018 19:33:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726989AbeJMDHo (ORCPT ); Fri, 12 Oct 2018 23:07:44 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:46667 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726933AbeJMDHn (ORCPT ); Fri, 12 Oct 2018 23:07:43 -0400 Received: by mail-qk1-f193.google.com with SMTP id q12-v6so8326695qkl.13 for ; Fri, 12 Oct 2018 12:33:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=2AifJsoYa8VVGHqyfqs2+9dIHUSfR9sXlFu5O/p8l7g=; b=hYA/5QOl789sf6jAucWJb8RT5EQ2Gq0Rdqd6o0SaSs18AROxmIFW+Pa1viTMuwWwQJ 7DQyDre/FqrbRMczMMGvt64cvCQmqsG8g3naXMUxz/M8QLB8JdNHp5N8oocoenriIFNd d+KQMBr/wLEE74IA0mRHH7uuB2AAGyk7TEYJWwTy52dSWvTXkCfqmIjTzIvnUl4ylEKV ITIhoNTjQAbdechZcorPt3LlUAYa2f0DfO/V1ASAA0BvwDhQqI1g0PVpXP6jEzBfvevZ eV/VxokKIZawpErZ6ZKBzcuWwttV+QWFeee7xS7RbKZM+hBYWRUBhRt/xrKjalquoxWA xyxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=2AifJsoYa8VVGHqyfqs2+9dIHUSfR9sXlFu5O/p8l7g=; b=aI1+a9kaL58I8QqcplD7GLJ/4NcOYtub/JayuCrh8XmDn2Jd5n3bOrXdPwHUmrD2UW +zw9tlScrWRX5R4CT9SmpgL+COIcrdaHu/L9Mdb2FPEy03j8PrFzzosZHQ/pmI8/V6XL 9wN8pe5kgnB24zGVKzOlY1UgrdCzkhmF982o+3xThWBkXmEzo5CGibYuJhPgkW3lkqxu oLvjy5D7bJhMzZ5Y5dMT9nK13HD+G2XxV7KPZCI/F1jrx9+6TXo1scYGm9mCBCjWcMvT BExslnDH9Z14Uisb3jDl8RbkeODwp9ZTv9e3NJMASLFZbtbjTKwMGC8DeZzEDlFyvKcW CCYQ== X-Gm-Message-State: ABuFfogN7FIVvcm6/DI53+dAPOaiWe9BBN1OVQqbtBYhT6FrrChV5MFF VyCN5XV7qXubjmqKFzLUZE1KGzEGMVA= X-Google-Smtp-Source: ACcGV61Y3enHMAuI1QYvJq62xKeTfO9UfHSqzU+MhYhNUSiFyUHD6TxrANFr/Toij/QEiLqwJ/GpzQ== X-Received: by 2002:a37:6cc3:: with SMTP id h186-v6mr7032971qkc.49.1539372821224; Fri, 12 Oct 2018 12:33:41 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e195-v6sm1482164qka.2.2018.10.12.12.33.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:40 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 18/42] btrfs: move the dio_sem higher up the callchain Date: Fri, 12 Oct 2018 15:32:32 -0400 Message-Id: <20181012193256.13735-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We're getting a lockdep splat because we take the dio_sem under the log_mutex. What we really need is to protect fsync() from logging an extent map for an extent we never waited on higher up, so just guard the whole thing with dio_sem. ====================================================== WARNING: possible circular locking dependency detected 4.18.0-rc4-xfstests-00025-g5de5edbaf1d4 #411 Not tainted ------------------------------------------------------ aio-dio-invalid/30928 is trying to acquire lock: 0000000092621cfd (&mm->mmap_sem){++++}, at: get_user_pages_unlocked+0x5a/0x1e0 but task is already holding lock: 00000000cefe6b35 (&ei->dio_sem){++++}, at: btrfs_direct_IO+0x3be/0x400 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #5 (&ei->dio_sem){++++}: lock_acquire+0xbd/0x220 down_write+0x51/0xb0 btrfs_log_changed_extents+0x80/0xa40 btrfs_log_inode+0xbaf/0x1000 btrfs_log_inode_parent+0x26f/0xa80 btrfs_log_dentry_safe+0x50/0x70 btrfs_sync_file+0x357/0x540 do_fsync+0x38/0x60 __ia32_sys_fdatasync+0x12/0x20 do_fast_syscall_32+0x9a/0x2f0 entry_SYSENTER_compat+0x84/0x96 -> #4 (&ei->log_mutex){+.+.}: lock_acquire+0xbd/0x220 __mutex_lock+0x86/0xa10 btrfs_record_unlink_dir+0x2a/0xa0 btrfs_unlink+0x5a/0xc0 vfs_unlink+0xb1/0x1a0 do_unlinkat+0x264/0x2b0 do_fast_syscall_32+0x9a/0x2f0 entry_SYSENTER_compat+0x84/0x96 -> #3 (sb_internal#2){.+.+}: lock_acquire+0xbd/0x220 __sb_start_write+0x14d/0x230 start_transaction+0x3e6/0x590 btrfs_evict_inode+0x475/0x640 evict+0xbf/0x1b0 btrfs_run_delayed_iputs+0x6c/0x90 cleaner_kthread+0x124/0x1a0 kthread+0x106/0x140 ret_from_fork+0x3a/0x50 -> #2 (&fs_info->cleaner_delayed_iput_mutex){+.+.}: lock_acquire+0xbd/0x220 __mutex_lock+0x86/0xa10 btrfs_alloc_data_chunk_ondemand+0x197/0x530 btrfs_check_data_free_space+0x4c/0x90 btrfs_delalloc_reserve_space+0x20/0x60 btrfs_page_mkwrite+0x87/0x520 do_page_mkwrite+0x31/0xa0 __handle_mm_fault+0x799/0xb00 handle_mm_fault+0x7c/0xe0 __do_page_fault+0x1d3/0x4a0 async_page_fault+0x1e/0x30 -> #1 (sb_pagefaults){.+.+}: lock_acquire+0xbd/0x220 __sb_start_write+0x14d/0x230 btrfs_page_mkwrite+0x6a/0x520 do_page_mkwrite+0x31/0xa0 __handle_mm_fault+0x799/0xb00 handle_mm_fault+0x7c/0xe0 __do_page_fault+0x1d3/0x4a0 async_page_fault+0x1e/0x30 -> #0 (&mm->mmap_sem){++++}: __lock_acquire+0x42e/0x7a0 lock_acquire+0xbd/0x220 down_read+0x48/0xb0 get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_fast+0xa4/0x150 iov_iter_get_pages+0xc3/0x340 do_direct_IO+0xf93/0x1d70 __blockdev_direct_IO+0x32d/0x1c20 btrfs_direct_IO+0x227/0x400 generic_file_direct_write+0xcf/0x180 btrfs_file_write_iter+0x308/0x58c aio_write+0xf8/0x1d0 io_submit_one+0x3a9/0x620 __ia32_compat_sys_io_submit+0xb2/0x270 do_int80_syscall_32+0x5b/0x1a0 entry_INT80_compat+0x88/0xa0 other info that might help us debug this: Chain exists of: &mm->mmap_sem --> &ei->log_mutex --> &ei->dio_sem Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&ei->dio_sem); lock(&ei->log_mutex); lock(&ei->dio_sem); lock(&mm->mmap_sem); *** DEADLOCK *** 1 lock held by aio-dio-invalid/30928: #0: 00000000cefe6b35 (&ei->dio_sem){++++}, at: btrfs_direct_IO+0x3be/0x400 stack backtrace: CPU: 0 PID: 30928 Comm: aio-dio-invalid Not tainted 4.18.0-rc4-xfstests-00025-g5de5edbaf1d4 #411 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014 Call Trace: dump_stack+0x7c/0xbb print_circular_bug.isra.37+0x297/0x2a4 check_prev_add.constprop.45+0x781/0x7a0 ? __lock_acquire+0x42e/0x7a0 validate_chain.isra.41+0x7f0/0xb00 __lock_acquire+0x42e/0x7a0 lock_acquire+0xbd/0x220 ? get_user_pages_unlocked+0x5a/0x1e0 down_read+0x48/0xb0 ? get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_fast+0xa4/0x150 iov_iter_get_pages+0xc3/0x340 do_direct_IO+0xf93/0x1d70 ? __alloc_workqueue_key+0x358/0x490 ? __blockdev_direct_IO+0x14b/0x1c20 __blockdev_direct_IO+0x32d/0x1c20 ? btrfs_run_delalloc_work+0x40/0x40 ? can_nocow_extent+0x490/0x490 ? kvm_clock_read+0x1f/0x30 ? can_nocow_extent+0x490/0x490 ? btrfs_run_delalloc_work+0x40/0x40 btrfs_direct_IO+0x227/0x400 ? btrfs_run_delalloc_work+0x40/0x40 generic_file_direct_write+0xcf/0x180 btrfs_file_write_iter+0x308/0x58c aio_write+0xf8/0x1d0 ? kvm_clock_read+0x1f/0x30 ? __might_fault+0x3e/0x90 io_submit_one+0x3a9/0x620 ? io_submit_one+0xe5/0x620 __ia32_compat_sys_io_submit+0xb2/0x270 do_int80_syscall_32+0x5b/0x1a0 entry_INT80_compat+0x88/0xa0 Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana --- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 095f0bb86bb7..c07110edb9de 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out; inode_lock(inode); + + /* + * We take the dio_sem here because the tree log stuff can race with + * lockless dio writes and get an extent map logged for an extent we + * never waited on. We need it this high up for lockdep reasons. + */ + down_write(&BTRFS_I(inode)->dio_sem); + atomic_inc(&root->log_batch); /* @@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) */ ret = btrfs_wait_ordered_range(inode, start, len); if (ret) { + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) * checked called fsync. */ ret = filemap_check_wb_err(inode->i_mapping, file->f_wb_err); + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans = btrfs_start_transaction(root, 0); if (IS_ERR(trans)) { ret = PTR_ERR(trans); + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); goto out; } @@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) * file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ + up_write(&BTRFS_I(inode)->dio_sem); inode_unlock(inode); /* diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 1650dc44a5e3..66b7e059b765 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, INIT_LIST_HEAD(&extents); - down_write(&inode->dio_sem); write_lock(&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start; @@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, } WARN_ON(!list_empty(&extents)); write_unlock(&tree->lock); - up_write(&inode->dio_sem); btrfs_release_path(path); if (!ret) From patchwork Fri Oct 12 19:32:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639181 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F12117E3 for ; Fri, 12 Oct 2018 19:33:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2074C2C546 for ; Fri, 12 Oct 2018 19:33:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 14C632C555; Fri, 12 Oct 2018 19:33:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A99FE2C546 for ; Fri, 12 Oct 2018 19:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727000AbeJMDHp (ORCPT ); Fri, 12 Oct 2018 23:07:45 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:43449 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726933AbeJMDHp (ORCPT ); Fri, 12 Oct 2018 23:07:45 -0400 Received: by mail-qk1-f195.google.com with SMTP id 12-v6so8330285qkj.10 for ; Fri, 12 Oct 2018 12:33:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YPU1+9kFJB8jUn5q1U/VWyfL9ar9cpfy/opBHehMoyg=; b=L+N44KSlgTRVrVQY1oPt+aLD38/F4sM/0E+fygpe1dD49axIO2HGoquHCGsBZgwwiq 74bl1EQ1sScwV3DMs7uB0bN8jPYuJybBUd0JJkK2OtdHs9jgIbRAU0ETD2gZTaz7Aq26 WuSSXXrdB39iwh0Xy8nBBDkZA9wwjC/s4dJsDENM9LUyC09NxmhUwa7G/pGF/EV9EICC DCMh1mOPCuA051IaVWnVLhOWdIUcFsL+PISP91J/mk+ccFHMGc3B0vZhzbWQ2o7H2McC lX2pT+Zke720omGJRdYqf4GRnRY/YFJpieocCt52wdrzcbWFh2y5DlOM5kQL5Br6BwW2 fPvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YPU1+9kFJB8jUn5q1U/VWyfL9ar9cpfy/opBHehMoyg=; b=BgSgkdA+NtRzQx9BuHVzZj9UP8PsFZQmcalkhrZUHdb/VDWMDjfyd0HqvkuQfwSDPC gkaFGwPor8NrIom3hzDVuwliBrrKI2yFv8u+S3YP0/USpHLwHCpxnFTJjWcRZevOr4nT rp7g2pWofmbaZwd6hSiUCipWEZXB6QOpqrIXIqp59RZZKO3Si7/g44mFQll3Z5D2vTZW RlJvV21pXHl/cNGW7jaIux0NEPolkc6sb4Mv3T/Kl/la4fiQ2X26rstFZ56lTgSncL1+ IyBOf5iuol5xJWzLwUJZtuyr58UPV/dJaAg06+Enu0RZvOtRvT9LL0GyDqPW79f30jTA ME1g== X-Gm-Message-State: ABuFfoij/yEelYpSsdI083PYELgnX7AUZ5QPzp52nO747gMwDVDp3z9K WnF6phYSSPvl1GK7yrwU9kYTb8okek4= X-Google-Smtp-Source: ACcGV62mEs9B1W6pTtcUop2PEkIfOWAwEu4aGi+NooXnNRXRRpBRdb/pNwpk96YaO3y75bunHQsj9w== X-Received: by 2002:a37:ba42:: with SMTP id k63-v6mr7048989qkf.103.1539372823809; Fri, 12 Oct 2018 12:33:43 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id d12-v6sm1528890qte.75.2018.10.12.12.33.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:42 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 19/42] btrfs: set max_extent_size properly Date: Fri, 12 Oct 2018 15:32:33 -0400 Message-Id: <20181012193256.13735-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We can't use entry->bytes if our entry is a bitmap entry, we need to use entry->max_extent_size in that case. Fix up all the logic to make this consistent. Signed-off-by: Josef Bacik Reviewed-by: David Sterba --- fs/btrfs/free-space-cache.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index e077ad3b4549..2dd773e96530 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1770,6 +1770,13 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl, return -1; } +static inline u64 get_max_extent_size(struct btrfs_free_space *entry) +{ + if (entry->bitmap) + return entry->max_extent_size; + return entry->bytes; +} + /* Cache the size of the max extent in bytes */ static struct btrfs_free_space * find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, @@ -1791,8 +1798,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, for (node = &entry->offset_index; node; node = rb_next(node)) { entry = rb_entry(node, struct btrfs_free_space, offset_index); if (entry->bytes < *bytes) { - if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + *max_extent_size = max(get_max_extent_size(entry), + *max_extent_size); continue; } @@ -1810,8 +1817,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, } if (entry->bytes < *bytes + align_off) { - if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + *max_extent_size = max(get_max_extent_size(entry), + *max_extent_size); continue; } @@ -1823,8 +1830,10 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, *offset = tmp; *bytes = size; return entry; - } else if (size > *max_extent_size) { - *max_extent_size = size; + } else { + *max_extent_size = + max(get_max_extent_size(entry), + *max_extent_size); } continue; } @@ -2684,8 +2693,8 @@ static u64 btrfs_alloc_from_bitmap(struct btrfs_block_group_cache *block_group, err = search_bitmap(ctl, entry, &search_start, &search_bytes, true); if (err) { - if (search_bytes > *max_extent_size) - *max_extent_size = search_bytes; + *max_extent_size = max(get_max_extent_size(entry), + *max_extent_size); return 0; } @@ -2722,8 +2731,9 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, entry = rb_entry(node, struct btrfs_free_space, offset_index); while (1) { - if (entry->bytes < bytes && entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; + if (entry->bytes < bytes) + *max_extent_size = max(get_max_extent_size(entry), + *max_extent_size); if (entry->bytes < bytes || (!entry->bitmap && entry->offset < min_start)) { From patchwork Fri Oct 12 19:32:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639183 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29E8A157A for ; Fri, 12 Oct 2018 19:33:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B97E2C546 for ; Fri, 12 Oct 2018 19:33:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1022F2C555; Fri, 12 Oct 2018 19:33:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B020B2C546 for ; Fri, 12 Oct 2018 19:33:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727006AbeJMDHr (ORCPT ); Fri, 12 Oct 2018 23:07:47 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:33546 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726750AbeJMDHr (ORCPT ); Fri, 12 Oct 2018 23:07:47 -0400 Received: by mail-qt1-f194.google.com with SMTP id q40-v6so15085066qte.0 for ; Fri, 12 Oct 2018 12:33:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LRl8jIfrXlaS55TP3QwSq+B/0ZwhXv7LeNXybWqMCSk=; b=WQrokF6rvoHdjAd6KhjZlIlOdp2etCYcMktA5FLJW/2pojw/7bgK/bgUuGyeQpKNX7 wNnh4aiU8H1efyo9Tschab7d8Ekq/E/+BjMPIMX1qZDytMHz+OjnMJunNIkVY9WL13Bo hAkn/TgMyLO6/zB+MG2zjOjo3ptwROrymJfG+1ZjdayeyYUXlBmcx//R6gUA/s+jRgWl kyDzFT1P6mWMduvxjY0vadhmbx/AhwD6K6n4XKoR9KIGoYK/7sgeAcVEVViBYxrJxbCr DqE7pfnAmflcljPRGqnjWoZAfiZ0SVPgCzUW15q2cxwbcrIPZ3nXPh5Im7NZ2qH1nesP fTVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LRl8jIfrXlaS55TP3QwSq+B/0ZwhXv7LeNXybWqMCSk=; b=LSnd8dcoKbwkT25LqMpZ9niDedfLwK5OFCYP/uIS+CYmwaIFynIDNJ9uUjWnoNnsT5 cvwWFCUvP9kI9r+KYSTcf25fccm1mDYwO+m1BFl07YhhV9MLqM8MMF1w9hStJljXhxeL Ej9bGlf1gUm8FQ70Lav3NRjnCPKYcNmoSL4XGo0c9kf0YfD07cWxetsyqY2Sp23Cksgr PLQh2+qHTyThRARLhjwt+VhWn8Wj0aMFEv9zYEjzFehmwlpx9zjJZxDMlcoserxKmNnB kYVfPf3m4x+dyzzA6ZNBNI8pPjhSIybeQVuYzVYXoC1210P5grk8mUGRDnBCvtnNTfG1 9UGg== X-Gm-Message-State: ABuFfohPZuK2WSe9dCvOJTYSFPjZWEoQVwyhCB2dLQve4voIM/Qo4aR3 n3LJ25CjeBSlXCorKrl5FyQqbTOi790= X-Google-Smtp-Source: ACcGV62oTttESlkWFLsLV6Pwy6AIEp3zNM3f4bKdH4ZA6dKPR/2aC9lWkWnP3mETcswie1Am++9Qcg== X-Received: by 2002:a0c:d6c3:: with SMTP id l3-v6mr7161928qvi.67.1539372826063; Fri, 12 Oct 2018 12:33:46 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v57-v6sm1349454qtk.51.2018.10.12.12.33.44 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:44 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 20/42] btrfs: don't use ctl->free_space for max_extent_size Date: Fri, 12 Oct 2018 15:32:34 -0400 Message-Id: <20181012193256.13735-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik max_extent_size is supposed to be the largest contiguous range for the space info, and ctl->free_space is the total free space in the block group. We need to keep track of these separately and _only_ use the max_free_space if we don't have a max_extent_size, as that means our original request was too large to search any of the block groups for and therefore wouldn't have a max_extent_size set. Reviewed-by: Filipe Manana Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index ef852d1911d1..6721698ab8aa 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -7496,6 +7496,7 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, struct btrfs_block_group_cache *block_group = NULL; u64 search_start = 0; u64 max_extent_size = 0; + u64 max_free_space = 0; u64 empty_cluster = 0; struct btrfs_space_info *space_info; int loop = 0; @@ -7791,8 +7792,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, spin_lock(&ctl->tree_lock); if (ctl->free_space < num_bytes + empty_cluster + empty_size) { - if (ctl->free_space > max_extent_size) - max_extent_size = ctl->free_space; + max_free_space = max(max_free_space, + ctl->free_space); spin_unlock(&ctl->tree_lock); goto loop; } @@ -7959,6 +7960,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, } out: if (ret == -ENOSPC) { + if (!max_extent_size) + max_extent_size = max_free_space; spin_lock(&space_info->lock); space_info->max_extent_size = max_extent_size; spin_unlock(&space_info->lock); From patchwork Fri Oct 12 19:32:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639185 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F356917E3 for ; Fri, 12 Oct 2018 19:33:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E38802C546 for ; Fri, 12 Oct 2018 19:33:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D7E182C555; Fri, 12 Oct 2018 19:33:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E58B2C546 for ; Fri, 12 Oct 2018 19:33:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727013AbeJMDHt (ORCPT ); Fri, 12 Oct 2018 23:07:49 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:32965 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726750AbeJMDHt (ORCPT ); Fri, 12 Oct 2018 23:07:49 -0400 Received: by mail-qk1-f195.google.com with SMTP id 84-v6so8372084qkf.0 for ; Fri, 12 Oct 2018 12:33:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=l5PjtYy8R8KZGhwEXTXGsXgmvOxQha2RMnXjcyOlSUU=; b=WXFF5zcwa0S4zbrj1/vMHw7MNq9HM4/q/GYYjj2QASJUyzDfv9QN20ieIz011tqLwX X/4aLk9dkILVKMe0kIudbMECme1yN/s0FDgDuMYFs7wosAnQRPM0MAEuxxdlJgEczd4i TGPIsLL6EU6D9JKbmkPSPCKyysvm/P7+SGZP6jv8UYTkgBTkWS2sGGXBcwoUTXktTNT3 FiHuX7Nh3JGz0ADIe8q/tyacANvETtFMMehFNCKZhbsEZwEESfZpbE8lWV4dyELaQ5gI mRlrlky9H/BgvgScQMlSPIwNzZ4Ig8qk2inRYRgSyRux0ud0eZJogJp99dZZl4d1hRtG C8HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=l5PjtYy8R8KZGhwEXTXGsXgmvOxQha2RMnXjcyOlSUU=; b=tv5rqA3/tygEcksvvf7RM1OzbQJLOtRF/KXKHEe3eb/LacLFQ946yWiuBGdkCZgXGM tY5q/3tzItbrFn3he5OsYOSnvxcqwmHJ0taLnt5VPHbgvdtisDv2LrdASW1I0DH/UP0m uB/xo/cPEQqqhDJAzvpzcCtinLrLnPJ9X07gNku9k9jrwAPa328wwLvuAZtYQ0KHHSGA nW8nwzc9SL7e6iCbtV7/lCXkZBPheTuSCCt+HZu4C4kuNiJirSnqROhGB4++c0Cgd8KJ hUKAlu9xKroRqHMO1F30G3CIMmqmzU5VhhR90GYmT1zad5wG+XkeVtyqaKWd1hSoCluG RPbw== X-Gm-Message-State: ABuFfogCdkrezdXxIfGd5Pp1X3hfFuf61z5g/B1qw5mhvZ6GCgUZOLDf 9J3b71zn0FWgvs4BFMYQi5kNR6vy9N0= X-Google-Smtp-Source: ACcGV63rcMzFFZKhPuH/crjgI1EgnyeFgIsOwPE10olzItQ8JDchuzqoXrPkMpa4mKy7Us++HdT1gg== X-Received: by 2002:a37:a9cd:: with SMTP id s196-v6mr7059826qke.342.1539372827954; Fri, 12 Oct 2018 12:33:47 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id j18-v6sm1986821qkh.20.2018.10.12.12.33.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:47 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH 21/42] btrfs: reset max_extent_size on clear in a bitmap Date: Fri, 12 Oct 2018 15:32:35 -0400 Message-Id: <20181012193256.13735-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik We need to clear the max_extent_size when we clear bits from a bitmap since it could have been from the range that contains the max_extent_size. Reviewed-by: Liu Bo Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 2dd773e96530..411acffbd45d 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -1687,6 +1687,8 @@ static inline void __bitmap_clear_bits(struct btrfs_free_space_ctl *ctl, bitmap_clear(info->bitmap, start, count); info->bytes -= bytes; + if (info->max_extent_size > ctl->unit) + info->max_extent_size = 0; } static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl, From patchwork Fri Oct 12 19:32:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639187 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9BC1017E3 for ; Fri, 12 Oct 2018 19:33:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C6782C54A for ; Fri, 12 Oct 2018 19:33:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8068A2C55C; Fri, 12 Oct 2018 19:33:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 180EC2C54A for ; Fri, 12 Oct 2018 19:33:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727028AbeJMDHw (ORCPT ); Fri, 12 Oct 2018 23:07:52 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:46516 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbeJMDHw (ORCPT ); Fri, 12 Oct 2018 23:07:52 -0400 Received: by mail-qt1-f193.google.com with SMTP id d8-v6so15041765qtk.13 for ; Fri, 12 Oct 2018 12:33:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=vfQB1tHlwUhp446i68IaYEtvkGVFvtWut8g/b3eBAlk=; b=sSc9ygjJPsMtOsf7c2ES0lIvSLUj1HHBm0ojY9Iy6SNHSazo9rROwR3Ouh3eZvM6KM BPpnvSgIL9TFNa319KAb5Osciyhl9bOW9WT2ZmP3FIKNfkQMJUqmFzI9oF8Q64DY0ifl /wd3rFilAVpIH16Nn8Fj/budwb3zft/SehmD/Z8Z5PHxHBCBSEDYbfs4ludiEkLxe1j/ 7DVvSe4ZboKAx2U7PDnrbco+cWYfWqdM0f75g6KQnqro2yucwWcqAaCLnIb1GfzSu0AJ fcQa32UGBI94XdWDyNa5CUJY5MSiDXl3ZC7JaCzHLhGAEkJz6K+6K9Nj8Us3YrWOytjw n9Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=vfQB1tHlwUhp446i68IaYEtvkGVFvtWut8g/b3eBAlk=; b=CJx0XrMgaseYCPdcR27dNkX+iDhA8gQom5246W9UlhwCTH10L692ujjj96VbRZOVYt su0P+vQIdf+YFMbZGJ7WrZqrQILOffAtJRklvqm5at2r+UJVIc8pXdNW9GVKvpccpRD+ /l+sNXbrM4PaS877yix0UwRzJ7FpkWRhuxowo8ZqaufUzmM8iRcFYrAlhP1NmpUl5vYs 7SSX8r2GL+pBlZO7aAUxP1gPXt47uFxVul1vFotPAVuNtqcgSq5lVHu3zACtk1XArnUB 4A/N+D/u9d38beMBKiQ0wTZ90fmCzI3lhrEk03ApC7JTtostRKhz0fwPv62aT9+8ko8z gLPg== X-Gm-Message-State: ABuFfohs2DrzxQmZCe/IzBU0FcFXdTbnYiRvJ1C+8CUPWNxGzE/mPkoL 4swld2iEPmQjA4KvEpLLauY44yA3WXE= X-Google-Smtp-Source: ACcGV62ukNDpYosL8wAQAoqyQEojcp4Ct2EBZ21EhME99X2WLEWA48x688fReADXzFOXOIHM/U+lqA== X-Received: by 2002:aed:3807:: with SMTP id j7-v6mr7181883qte.140.1539372830771; Fri, 12 Oct 2018 12:33:50 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y51-v6sm1486841qth.66.2018.10.12.12.33.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:49 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 22/42] btrfs: only run delayed refs if we're committing Date: Fri, 12 Oct 2018 15:32:36 -0400 Message-Id: <20181012193256.13735-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I noticed in a giant dbench run that we spent a lot of time on lock contention while running transaction commit. This is because dbench results in a lot of fsync()'s that do a btrfs_transaction_commit(), and they all run the delayed refs first thing, so they all contend with each other. This leads to seconds of 0 throughput. Change this to only run the delayed refs if we're the ones committing the transaction. This makes the latency go away and we get no more lock contention. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index a0f19ca0bd6c..39a2bddb0b29 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1925,15 +1925,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - /* make a pass through all the delayed refs we have so far - * any runnings procs may add more while we are here - */ - ret = btrfs_run_delayed_refs(trans, 0); - if (ret) { - btrfs_end_transaction(trans); - return ret; - } - cur_trans = trans->transaction; /* @@ -1946,12 +1937,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) if (!list_empty(&trans->new_bgs)) btrfs_create_pending_block_groups(trans); - ret = btrfs_run_delayed_refs(trans, 0); - if (ret) { - btrfs_end_transaction(trans); - return ret; - } - if (!test_bit(BTRFS_TRANS_DIRTY_BG_RUN, &cur_trans->flags)) { int run_it = 0; @@ -2022,6 +2007,15 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) spin_unlock(&fs_info->trans_lock); } + /* + * We are now the only one in the commit area, we can run delayed refs + * without hitting a bunch of lock contention from a lot of people + * trying to commit the transaction at once. + */ + ret = btrfs_run_delayed_refs(trans, 0); + if (ret) + goto cleanup_transaction; + extwriter_counter_dec(cur_trans, trans->type); ret = btrfs_start_delalloc_flush(fs_info); From patchwork Fri Oct 12 19:32:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639189 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 127A6157A for ; Fri, 12 Oct 2018 19:33:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04ABF2C55C for ; Fri, 12 Oct 2018 19:33:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ECAAB2C569; Fri, 12 Oct 2018 19:33:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DDB9F2C54A for ; Fri, 12 Oct 2018 19:33:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727034AbeJMDHz (ORCPT ); Fri, 12 Oct 2018 23:07:55 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:32975 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbeJMDHy (ORCPT ); Fri, 12 Oct 2018 23:07:54 -0400 Received: by mail-qk1-f196.google.com with SMTP id 84-v6so8372233qkf.0 for ; Fri, 12 Oct 2018 12:33:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=iwqcae1qLLQVCE/PGD/ciqt8xARB3jAN063PUm5bmoo=; b=jUygUYNCoAdq7AmY6YYV50mJYxEK/WS2R6yrI4RrID34sJraCcho+G+ofSKFRrIQEs RLNyz1hzS5B0KoCI9gG5JD5IAsVFqWVtae0ePr+4FbjGgjFfho831MEhqw40OxNGViNa 3/jEDcQtdK+rAvyyWxpCI6LH5RIG0PETDb+1LY0nFx9e7pAWidBp1HXC2TTKHWy6wY+M zGYCN0I4DukWncXLWHUtVQoVjVtmP8IfPAwjx2Bo7bX5viLD4FgfLOgEPcYc7sWAltkJ wZUVy4VKvGYB8HGVzOXYNFh3crtADL58LzqzM1XX3ceZOzRuqv8cQLTzgbzTraoISV3W 9B9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=iwqcae1qLLQVCE/PGD/ciqt8xARB3jAN063PUm5bmoo=; b=mhcRnpV/gTruwUldd+6pgrdTgOXnl/HG0QpuhAV/G33JZWF0DxfPrGDjVEoUiSeN/6 C/3KkwbfExUnyOW2r+HusRt1TR4EJhOAN9J6b627FAnACKh6kWXkqsbKZOb41W5BHULS G7ilfxSmX2NbQr8gouTbCsQ8xN1YbJOUTJ319Z+hEkkW2tUxHFy9ri+RTk/QHR+nXKpF +XgFqtZLUAL2/FTbeJTu/mJqlYvjbJf+xB0jury++aM0kbd+vexgVoljtWyc4iIKaZFr JqdlFctWS3GUrMrTZWQFsP5ez8T9OKsSuqChPYxEK62oa9/vKzWmHerbzQc+YBTwzCOV stQQ== X-Gm-Message-State: ABuFfojwCR4Fky86VscYt6z3INpnFYikKE+52Y6jLeTEZlWnMftCJ5i3 tc/SSWB9b6cveJUNhbMRjMW2C6XCXxA= X-Google-Smtp-Source: ACcGV60pKON7m2wnEkI6zhaGBwM9xhZRFr/y5bFkpj86wk8l01z6Abl3lTdvxmBBeGSltdr/+/t3dQ== X-Received: by 2002:a37:8305:: with SMTP id f5-v6mr7090298qkd.230.1539372833218; Fri, 12 Oct 2018 12:33:53 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id r78-v6sm1605656qki.16.2018.10.12.12.33.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:52 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 23/42] btrfs: make sure we create all new bgs Date: Fri, 12 Oct 2018 15:32:37 -0400 Message-Id: <20181012193256.13735-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allocating new chunks modifies both the extent and chunk tree, which can trigger new chunk allocations. So instead of doing list_for_each_safe, just do while (!list_empty()) so we make sure we don't exit with other pending bg's still on our list. Reviewed-by: Omar Sandoval Reviewed-by: Liu Bo Reviewed-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 6721698ab8aa..0d7fc07c34ab 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -10361,7 +10361,7 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info) void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) { struct btrfs_fs_info *fs_info = trans->fs_info; - struct btrfs_block_group_cache *block_group, *tmp; + struct btrfs_block_group_cache *block_group; struct btrfs_root *extent_root = fs_info->extent_root; struct btrfs_block_group_item item; struct btrfs_key key; @@ -10369,7 +10369,10 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) bool can_flush_pending_bgs = trans->can_flush_pending_bgs; trans->can_flush_pending_bgs = false; - list_for_each_entry_safe(block_group, tmp, &trans->new_bgs, bg_list) { + while (!list_empty(&trans->new_bgs)) { + block_group = list_first_entry(&trans->new_bgs, + struct btrfs_block_group_cache, + bg_list); if (ret) goto next; From patchwork Fri Oct 12 19:32:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639191 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6B0317E3 for ; Fri, 12 Oct 2018 19:33:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C91982C546 for ; Fri, 12 Oct 2018 19:33:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BD01E2C569; Fri, 12 Oct 2018 19:33:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 677F62C546 for ; Fri, 12 Oct 2018 19:33:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727042AbeJMDH4 (ORCPT ); Fri, 12 Oct 2018 23:07:56 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:45929 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbeJMDH4 (ORCPT ); Fri, 12 Oct 2018 23:07:56 -0400 Received: by mail-qk1-f196.google.com with SMTP id m8-v6so8341282qka.12 for ; Fri, 12 Oct 2018 12:33:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=5ErWA8iDFY+6lz4JimjzCOpUtgurbUZk5oK0tyfx1Sk=; b=ydruAyxWw+sUhnZcgM3UxK9ILhXYl9GXsn+wElfZY/6XMO248WlkjgELKSeFSmiSQN 6yj2YBIgiDYNf0jHxGMECdtPffvzDYzgQHxLyvSrJxWwbF1U+bBmImpF1eijV+vJnEhK HHS5XjYrw6akh/M1q188dOAHfxZGP1H5O5uootp4Co09VWTita7iGdkGMIScsGT3zoAB n9b6xIoljiik3dKZHI1KUYYwzUeX69VKkeBpoTgfO/O5NPpF6SSb//8eNcz5ZSh2e92K H8Olz8ADTwakb4bYev17999S9kVpOgL2FSfUN8gatB5/cDBjuV+OY6tqnS0i7sdrCSAe N6rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=5ErWA8iDFY+6lz4JimjzCOpUtgurbUZk5oK0tyfx1Sk=; b=QL3a4KE6N2aT91RmCvXkJ7AIYZAGjOYHZqemLNIh+pkTomSbRMzP+pJlr5lz68ZP9j OUDGqJsRA+PnKw4fTSIjuxZ7WWUMuYGm/m6ymZb3ATV5cmiZIc/7f7ZDedleNMC7rxCl kMWX+yPS1dO5PeryNkz97CjIBOUALnXfVpRTahitLhoZXpRStwK+a4fh4+F3I4oGdLhB 7sDTEgB8CRBfohz4kLmtpLEhO+bwZZuma/JeagU4ZYAAaOSspmFXjIA3U3eFy/hzxeTT qg8hrKrid3MbHJpwMbrde1sjLDEhI9bR2iic4uhyzlLb507wRjhGzJw0R8CvSiiwAUXD NLuQ== X-Gm-Message-State: ABuFfojgsQeg2tdEPZ7QWuqVKlCpTUGZqgHaQKDaBbAzfw5JMUBRvB3M JKWA5nbw7sBT3oFSBIkR5wjQSRYcLUQ= X-Google-Smtp-Source: ACcGV63AuLgDwZuzC5DuzX3Y84SdKZEKU4qP5MvkPLiFahqMuOpWeQ0d+vq7l3ORt+BPDyowQR+94A== X-Received: by 2002:a37:b345:: with SMTP id c66-v6mr7214202qkf.47.1539372835019; Fri, 12 Oct 2018 12:33:55 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e2-v6sm1093970qkb.41.2018.10.12.12.33.53 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:54 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 24/42] btrfs: assert on non-empty delayed iputs Date: Fri, 12 Oct 2018 15:32:38 -0400 Message-Id: <20181012193256.13735-25-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I ran into an issue where there was some reference being held on an inode that I couldn't track. This assert wasn't triggered, but it at least rules out we're doing something stupid. Reviewed-by: Omar Sandoval Reviewed-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 377ad9c1cb17..39bd158466cd 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3979,6 +3979,7 @@ void close_ctree(struct btrfs_fs_info *fs_info) kthread_stop(fs_info->transaction_kthread); kthread_stop(fs_info->cleaner_kthread); + ASSERT(list_empty(&fs_info->delayed_iputs)); set_bit(BTRFS_FS_CLOSING_DONE, &fs_info->flags); btrfs_free_qgroup_config(fs_info); From patchwork Fri Oct 12 19:32:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639199 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0292B157A for ; Fri, 12 Oct 2018 19:34:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E87172C575 for ; Fri, 12 Oct 2018 19:34:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD3532C57A; Fri, 12 Oct 2018 19:34:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83E862C575 for ; Fri, 12 Oct 2018 19:34:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727072AbeJMDH6 (ORCPT ); Fri, 12 Oct 2018 23:07:58 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:41974 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727044AbeJMDH6 (ORCPT ); Fri, 12 Oct 2018 23:07:58 -0400 Received: by mail-qt1-f194.google.com with SMTP id l41-v6so15053456qtl.8 for ; Fri, 12 Oct 2018 12:33:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=iYf43solQFPvuH12W6PP9YSCdqMp0PBiM9fgZKcOmF8=; b=0A/pmy+fF/MSShxF+E8A8K4Q1bexMSX/diWGRrLcLjtohaI5gaX7//w2Gm2qUvyWmD U9+8rZlO3sVu+WLTWjIKOu0n52nemtCBr9p0A6BmKfEpqinUi6P0tCxjKD6fRDgrXQ3I QLovmREkXB+LFtPNCDPyle3Z9xg4jOprXw6vQ7YSXHbNd/0jtOrJUXP7jbXlTD//FjpD w3IHAuGvs0/4VfaA8C7KFkKWBHebH11HXlTjrU8ko2fapCJ7NeEquJnvmNdhQO8vvrkw 4F/B5NEIg0kFmWsnqLziZpQBDbS9cAa8uDrW5g2l6qxErvfqNej0n/DjxQ5SHso2dugR 9UdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=iYf43solQFPvuH12W6PP9YSCdqMp0PBiM9fgZKcOmF8=; b=dxlYzCWCYzRcNJs7WmHahBwQqlQkbfgN/0c1mvzUXyzNvBq4e1n+b98rIGlGJEmiKg 186ero4UuhY8cWTXQFmWFDPViDJgM4HkQyWXdTJJOhSyDVn0uY/bqVM3BdXD4/uFd8mp WTKclYolqO1YXCKVq+zNIAuwMZ5UdNcTr/SdkCvAJ28tD1haBedJ65JjKCy8j2OhCY5G 6OU0ChsbwhTbXfJNy1LKkmuEydyF8ZtFDNqkDlf5Y8fCZ0IyN8jzq3kqel+Ph8BBlsc/ 33Rpva20ANkEOzVAgxGzeaYoEO47KZWeZoSmFVh6hPPTZ8UK8hCjBUO2b6KTWkbY0oTd NEDA== X-Gm-Message-State: ABuFfoh60RPC8bwzXnf6GIsb8EtuVfPw5/ZrLiIfUAWcySuQXfCctiCD SJF7Kr+mTgFlFDshYUEmIqUh7K3lhck= X-Google-Smtp-Source: ACcGV63QJQMcoF+2gfRY77N9eqxXrT5XbNQ2c4MlMecLldwE35JDTb1MnE6DnkGn/4ELJQs0WYARHA== X-Received: by 2002:ac8:3414:: with SMTP id u20-v6mr6734532qtb.237.1539372836775; Fri, 12 Oct 2018 12:33:56 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s71-v6sm1555623qkl.86.2018.10.12.12.33.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:55 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 25/42] btrfs: pass delayed_refs_root to btrfs_delayed_ref_lock Date: Fri, 12 Oct 2018 15:32:39 -0400 Message-Id: <20181012193256.13735-26-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't need the trans except to get the delayed_refs_root, so just pass the delayed_refs_root into btrfs_delayed_ref_lock and call it a day. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 5 +---- fs/btrfs/delayed-ref.h | 2 +- fs/btrfs/extent-tree.c | 2 +- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 96ce087747b2..87778645bf4a 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -197,12 +197,9 @@ find_ref_head(struct rb_root *root, u64 bytenr, return NULL; } -int btrfs_delayed_ref_lock(struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock(struct btrfs_delayed_ref_root *delayed_refs, struct btrfs_delayed_ref_head *head) { - struct btrfs_delayed_ref_root *delayed_refs; - - delayed_refs = &trans->transaction->delayed_refs; lockdep_assert_held(&delayed_refs->lock); if (mutex_trylock(&head->mutex)) return 0; diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index 7769177b489e..ee636d7a710a 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -255,7 +255,7 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans, struct btrfs_delayed_ref_head * btrfs_find_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_refs, u64 bytenr); -int btrfs_delayed_ref_lock(struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock(struct btrfs_delayed_ref_root *delayed_refs, struct btrfs_delayed_ref_head *head); static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) { diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 0d7fc07c34ab..529c2aefbbf4 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2600,7 +2600,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, /* grab the lock that says we are going to process * all the refs for this head */ - ret = btrfs_delayed_ref_lock(trans, locked_ref); + ret = btrfs_delayed_ref_lock(delayed_refs, locked_ref); spin_unlock(&delayed_refs->lock); /* * we may have dropped the spin lock to get the head From patchwork Fri Oct 12 19:32:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639193 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 802C717E3 for ; Fri, 12 Oct 2018 19:34:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72E772C55C for ; Fri, 12 Oct 2018 19:34:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 675C62C574; Fri, 12 Oct 2018 19:34:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 16D132C55C for ; Fri, 12 Oct 2018 19:34:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727098AbeJMDIB (ORCPT ); Fri, 12 Oct 2018 23:08:01 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:39068 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbeJMDIA (ORCPT ); Fri, 12 Oct 2018 23:08:00 -0400 Received: by mail-qk1-f194.google.com with SMTP id q5-v6so8349318qki.6 for ; Fri, 12 Oct 2018 12:33:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=X+PT81/Zp3zcYtUcLLi8M0fbP5w87s6IcVZs1kVoVUA=; b=UVY+Z6BSGBlz2keV5PZRsp4zE6xgRVvxNUpCgpc/Rs1R3hBacFcoeW9Yiz+F04ir6E mDQJq5J3gvlmeWxqIt4uq5zIn81+rEAIsS5XoNDcpf9I2N28j7d+GegKwM1UTFF7MzWT Fb0xT8qbqvskIWmF8tTzkE7pZmuPdUTOYmNc1xWa9lXzdOrFTpuwMXdsnsj1IeQeej9U BIFt+HYOLi33Jiaxdy5HfQXdDWKXoCG/mG81YRfxWMVQOPfk10BqP6E9DsTc+pPP2Pqo 4qvjXuF/3BURi3e1zKhjAOBYGHgCYbVTxuBcIYTRiWdqCdhSaIn+nWMYX14lNRuQD7GR +iFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=X+PT81/Zp3zcYtUcLLi8M0fbP5w87s6IcVZs1kVoVUA=; b=HxNY5MJqG7v3J5b5AFefJpNVWpEcNZMF117mizY+RBxNX1fASEhzBNOxTqdVkdNGJP lJjumzNybA5V0+LmJ9iRVP0vQYy64wC6E1gHTrI4jWNYXHCSYyHFVMdlsNVZaUvPaCRW 58UK04g90OaGKqC6YAgST609Oy911XF2hNGwcU0412GlumIHxrK5DaDsNBPFwv8y/EVr y/un4L8wCDxUC1IEz+4gxGn2TCCOzjaPYi4gdMHAlKSTMUOWl82gCMTjVIWMkJti+zqr jTMqKUjDQjHtuw+n+G49x5V1r4q/wMs3yJmrCTuUdrh+bxhUmLeh7btwtWKr6XH+cRs1 NvCg== X-Gm-Message-State: ABuFfoijhg5N98pkTUw0jtVlmQ/IkZNnvgvOgYuKrkHe9s6wD8GdANLF xt/YcIAnRyLnhk7zMd0VsLKqv73Z1rQ= X-Google-Smtp-Source: ACcGV63nrzeSYSF/n1xVDQS3N45THNLMSLzcXtG2AFb3BB+ZnnFacwVYpS67NUByxFBFfnZRMrIumA== X-Received: by 2002:a37:8d83:: with SMTP id p125-v6mr6985486qkd.97.1539372838539; Fri, 12 Oct 2018 12:33:58 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a68-v6sm1459249qka.54.2018.10.12.12.33.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:57 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 26/42] btrfs: make btrfs_destroy_delayed_refs use btrfs_delayed_ref_lock Date: Fri, 12 Oct 2018 15:32:40 -0400 Message-Id: <20181012193256.13735-27-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We have this open coded in btrfs_destroy_delayed_refs, use the helper instead. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 39bd158466cd..121ab180a78a 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4214,16 +4214,9 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); - if (!mutex_trylock(&head->mutex)) { - refcount_inc(&head->refs); - spin_unlock(&delayed_refs->lock); - - mutex_lock(&head->mutex); - mutex_unlock(&head->mutex); - btrfs_put_delayed_ref_head(head); - spin_lock(&delayed_refs->lock); + if (btrfs_delayed_ref_lock(delayed_refs, head)) continue; - } + spin_lock(&head->lock); while ((n = rb_first(&head->ref_tree)) != NULL) { ref = rb_entry(n, struct btrfs_delayed_ref_node, From patchwork Fri Oct 12 19:32:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639195 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B768157A for ; Fri, 12 Oct 2018 19:34:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5DC152C55C for ; Fri, 12 Oct 2018 19:34:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 51E572C574; Fri, 12 Oct 2018 19:34:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 029362C55C for ; Fri, 12 Oct 2018 19:34:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727099AbeJMDIC (ORCPT ); Fri, 12 Oct 2018 23:08:02 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:40137 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbeJMDIB (ORCPT ); Fri, 12 Oct 2018 23:08:01 -0400 Received: by mail-qt1-f195.google.com with SMTP id b4-v6so15038146qtc.7 for ; Fri, 12 Oct 2018 12:34:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=cvSqJmOxERWLztXjenJ/CRRumlDa3ZymappVew1ga8M=; b=H3FabDKZ7+9haDuK1v7H4BWbbHIWygwqdJADpInWsOKs1F17gRiIQW9SvI4xsOuOtR z3Pgpe6jBGR3wch19p9P6FgldZjpDggYyw9K/p1Mk9tXqX64zojh14dCl0BqDszG0AM6 TJCU16aVxfGcHwjdBRQ7biWLK2s++TOGUkwqeZ8rb/YDlCNJNLA1WYgSEmtrh3OBJgPN bFubkolI0OuOdIwrgYrRf+hesVNUeIJthVB1FVQLkYJwrxg3o9hTyAtUw8LJK1KSKsDP pECrJnwzNy+X0Flurq7jDlDxqdEQ+C2I30sbNLlPGG+0K+DZEABGjp/OC2t6aWMjlNw0 l2/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=cvSqJmOxERWLztXjenJ/CRRumlDa3ZymappVew1ga8M=; b=DusydO3F4LVXrEU39Zth48icKV81izL3IzsOvJBhJC+v7gSccK/YR4PtKFPYlNIPmS phYjD9SUO4tHRe2rEHMBlLx/m+j5QcJO3lEZ2F11Hnhv27shJA5J6ajE5fQ40ah8AOS8 BDnikcbfXUMpPAo2fwk5fSZ70dLnFaJ7vy086o5Vom4cGi+pV3mb82UbnJmTan7QM+gW IpsavwLOluvQLe2Y3jVNCYEPvRhGkGofxFhR27TosXaKXi79gLs7iy0Rki7Jg8ffzPny CJYoWXzlP9buTGm1ZsqBCewvT8NkDoFqy6HY4Diiy9ohn4z7C3r3sHG3s1NuC3msMc7m EXOg== X-Gm-Message-State: ABuFfojGRIfOMYwmXsqzvqB0ySY1h312pQkJ1YcXhn2RCahSH9wx2/9I pyRN9Rs6IuXh8K/RuJ8wnoUINLNaCWk= X-Google-Smtp-Source: ACcGV60fTOPwEGE6nMGiQxE5aKG8MqtczbMeNmMOsnSOFVlmjCSRm0AzchGhppvsCpJGQfWEZ9f3MQ== X-Received: by 2002:a0c:92f3:: with SMTP id c48mr7198912qvc.39.1539372840409; Fri, 12 Oct 2018 12:34:00 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g127-v6sm942810qke.59.2018.10.12.12.33.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:33:59 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 27/42] btrfs: make btrfs_destroy_delayed_refs use btrfs_delete_ref_head Date: Fri, 12 Oct 2018 15:32:41 -0400 Message-Id: <20181012193256.13735-28-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of open coding this stuff use the helper instead. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 121ab180a78a..fe1f229320ef 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4232,12 +4232,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, if (head->must_insert_reserved) pin_bytes = true; btrfs_free_delayed_extent_op(head->extent_op); - delayed_refs->num_heads--; - if (head->processing == 0) - delayed_refs->num_heads_ready--; - atomic_dec(&delayed_refs->num_entries); - rb_erase(&head->href_node, &delayed_refs->href_root); - RB_CLEAR_NODE(&head->href_node); + btrfs_delete_ref_head(delayed_refs, head); spin_unlock(&head->lock); spin_unlock(&delayed_refs->lock); mutex_unlock(&head->mutex); From patchwork Fri Oct 12 19:32:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639197 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 07F29157A for ; Fri, 12 Oct 2018 19:34:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EF02D2C574 for ; Fri, 12 Oct 2018 19:34:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E36612C569; Fri, 12 Oct 2018 19:34:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 811A22C546 for ; Fri, 12 Oct 2018 19:34:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727101AbeJMDIE (ORCPT ); Fri, 12 Oct 2018 23:08:04 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:37571 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726530AbeJMDIE (ORCPT ); Fri, 12 Oct 2018 23:08:04 -0400 Received: by mail-qt1-f196.google.com with SMTP id d14-v6so15064117qto.4 for ; Fri, 12 Oct 2018 12:34:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=LupJVcNaGZD7v7kwZmH4pcQfZSwxchu3Okq8BUpSizQ=; b=Z9sT6E3imDMgkTGOwBcSxwLvdQ0ARxuBB6Ep433QcBtsEvM8gK18PTXBLkPSj0UkmQ zsaZ7Cq92Ss2Ml0Y5oRxbAi6hqH2jPLmZy++ZXakEd98ekWELw2pqEjWUEtOkCNuKAwf 0M3nrwnUPB0h76OUUGvs33rfM5SAAbcBAIe4lcBLoOed53bFhOzpeDQ7AFemMmsYhMNO jpOmRcy7KmWbVGBgEe1onSgjzHIk09KvJrh0W/0Gr2UMS7J66GuGB1346AYfAlTutmyI rd6Uz157DVz66GWFxGUB2FK6vOtJySt45mlloWWuRwU2G8cAqV6w0YZK2YpdpOyzrkXW 2d+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=LupJVcNaGZD7v7kwZmH4pcQfZSwxchu3Okq8BUpSizQ=; b=mKNEDeHZ1GlKM7coBP1G3IZb0iXgM0b8UAJSCyRvGJ2TD3Qz73lNIj4rH3FqOY1M0I yBYPAW4q/u6CVAYkzDJKW6i9rqS1PSLaMY50KZdN0O+7WDXoZW01Y6IW7o3/u1Qq7f6B amw+oRKCIS6xGBScj/sV0wrdbeKYPMX7EkIvOpmduMF4G+a1dYlXAelC60QrD8GtGgbq OIwUSzGtIUJDUXQnAEd607omUgz9+sfYTSH6JBq1WYyziY8Bejh+J+1qY5Iz14nVNOz7 bkik3OzyI/JdRHtwTF/uNhDoWawj85Ci3JEXLY3XRxgMiqr1e3D62A1UMQzbcgwiR7XZ emDg== X-Gm-Message-State: ABuFfoinzUXnsyMje6Mwj1p7Sw6zeDc4hzL93binFfR0RwBoq5ybCp+S XlEo9WbGTJHjDsjmgvH2yACAF8wr9ZE= X-Google-Smtp-Source: ACcGV63JASmwY2WzdF5HTHJrk3vazIaaJlDGjhordg0ukTCW06gP4dQdYUWM1L7ZzCJxnj1TI0B84A== X-Received: by 2002:ad4:40cb:: with SMTP id x11mr2716638qvp.52.1539372842551; Fri, 12 Oct 2018 12:34:02 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id z46-v6sm1591181qth.87.2018.10.12.12.34.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:01 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 28/42] btrfs: handle delayed ref head accounting cleanup in abort Date: Fri, 12 Oct 2018 15:32:42 -0400 Message-Id: <20181012193256.13735-29-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We weren't doing any of the accounting cleanup when we aborted transactions. Fix this by making cleanup_ref_head_accounting global and calling it from the abort code, this fixes the issue where our accounting was all wrong after the fs aborts. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 5 +++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/extent-tree.c | 13 ++++++------- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 29db902511c1..e40356ca0295 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -35,6 +35,7 @@ struct btrfs_trans_handle; struct btrfs_transaction; struct btrfs_pending_snapshot; +struct btrfs_delayed_ref_root; extern struct kmem_cache *btrfs_trans_handle_cachep; extern struct kmem_cache *btrfs_bit_radix_cachep; extern struct kmem_cache *btrfs_path_cachep; @@ -2623,6 +2624,10 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long count); int btrfs_async_run_delayed_refs(struct btrfs_fs_info *fs_info, unsigned long count, u64 transid, int wait); +void +btrfs_cleanup_ref_head_accounting(struct btrfs_fs_info *fs_info, + struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head); int btrfs_lookup_data_extent(struct btrfs_fs_info *fs_info, u64 start, u64 len); int btrfs_lookup_extent_info(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info, u64 bytenr, diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index fe1f229320ef..54fbdc944a3f 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4240,6 +4240,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, if (pin_bytes) btrfs_pin_extent(fs_info, head->bytenr, head->num_bytes, 1); + btrfs_cleanup_ref_head_accounting(fs_info, delayed_refs, head); btrfs_put_delayed_ref_head(head); cond_resched(); spin_lock(&delayed_refs->lock); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 529c2aefbbf4..5b5a28b69fa9 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2475,12 +2475,11 @@ static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans, return ret ? ret : 1; } -static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans, - struct btrfs_delayed_ref_head *head) +void +btrfs_cleanup_ref_head_accounting(struct btrfs_fs_info *fs_info, + struct btrfs_delayed_ref_root *delayed_refs, + struct btrfs_delayed_ref_head *head) { - struct btrfs_fs_info *fs_info = trans->fs_info; - struct btrfs_delayed_ref_root *delayed_refs = - &trans->transaction->delayed_refs; int nr_items = 1; if (head->total_ref_mod < 0) { @@ -2558,7 +2557,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } - cleanup_ref_head_accounting(trans, head); + btrfs_cleanup_ref_head_accounting(fs_info, delayed_refs, head); trace_run_delayed_ref_head(fs_info, head, 0); btrfs_delayed_ref_unlock(head); @@ -7227,7 +7226,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans, if (head->must_insert_reserved) ret = 1; - cleanup_ref_head_accounting(trans, head); + btrfs_cleanup_ref_head_accounting(trans->fs_info, delayed_refs, head); mutex_unlock(&head->mutex); btrfs_put_delayed_ref_head(head); return ret; From patchwork Fri Oct 12 19:32:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639201 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19D9A17E3 for ; Fri, 12 Oct 2018 19:34:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0B00F2C574 for ; Fri, 12 Oct 2018 19:34:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F36AC2C576; Fri, 12 Oct 2018 19:34:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F1132C574 for ; Fri, 12 Oct 2018 19:34:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727122AbeJMDIG (ORCPT ); Fri, 12 Oct 2018 23:08:06 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:42093 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726530AbeJMDIG (ORCPT ); Fri, 12 Oct 2018 23:08:06 -0400 Received: by mail-qk1-f193.google.com with SMTP id g20-v6so8335755qke.9 for ; Fri, 12 Oct 2018 12:34:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=A9aQdPUEOSexy0JQIA2ztVH7gL7ZKLlpRHnZdfmk2Vo=; b=BH2HhYNnD8+XxYt96UgUrhL8Wctors+vRNGGi56OZyXBWZu3s8tb2/T0HKVXbqZa8c S5xsdYWJfhMcRBIoUS9mK1bl5ebdzS+yL9zTgkGe3fUAJjA3FkCRVbYzOc5ZH4Lh9H+6 IWQOYT0JHuo7i5hzFWgs2OM4EVQsFbifJfr6hwbF8cd5mlJqqMJ9jpteuQjyFj8N0mAS gQTf6rpDyoqIJbX660ksSl51fkC813c/yt1GYYpS8XvXh4FIVheURGWXy70+eD16RiEA roz15uERls5dkqWMp/5+lr4u2ipUgRiirQDH7eulVhJtVeGE45o9fVmnApwdWH3M7CPP gzRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=A9aQdPUEOSexy0JQIA2ztVH7gL7ZKLlpRHnZdfmk2Vo=; b=hHooHYJ/73KKDoZ90f9/+AQfyPmNUtFfgJSy58GG6+SHjQRQGGu+oKdVtQm/LdldWw K9gFKWtZWRAtz2sBGuYUzXB9frEjfVp8oV1Fu1SYh04FAZy/kB8LMb9GYX9Gn0ve/40H lmmpt8005NukglYv/dvtVBUFQv/Riga0aaUvmHOwcIV3eT+Kzpsegc0FtwlDKV4ZNr/Z YGKgS4+Pfl8+++2leNMB6R+jD1J7pwGthMPAwlxhoWjFEDNbedZpXYkTTJKTzItOZvmg GFk/R4RTfJ64lgwxbDvHmbNrEgAayJIF/UEA8AvpRQNL8aF8DJI8+dH+6/ROyJBvokDA Z5iw== X-Gm-Message-State: ABuFfohC6rtazBsbiX+l7rQ1OcTigOJVGwp172M148muKYiK+s2RPTJf JqJd+CMWL9pKGH+WA6EMuqXEkglDX84= X-Google-Smtp-Source: ACcGV61k+eHoVAQVkco+4vAyUnZRxrwYQcLTXrBQ0phQsUP/KkN5HzEA0+f1e1YjtMP4o3hYy20r1Q== X-Received: by 2002:ae9:ef11:: with SMTP id d17-v6mr7189519qkg.80.1539372844520; Fri, 12 Oct 2018 12:34:04 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 140-v6sm1461573qkj.13.2018.10.12.12.34.03 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:03 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 29/42] btrfs: call btrfs_create_pending_block_groups unconditionally Date: Fri, 12 Oct 2018 15:32:43 -0400 Message-Id: <20181012193256.13735-30-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The first thing we do is loop through the list, this if (!list_empty()) btrfs_create_pending_block_groups(); thing is just wasted space. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 3 +-- fs/btrfs/transaction.c | 6 ++---- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 5b5a28b69fa9..14495c030301 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2978,8 +2978,7 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, } if (run_all) { - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); spin_lock(&delayed_refs->lock); node = rb_first(&delayed_refs->href_root); diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 39a2bddb0b29..46ca775a709e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -846,8 +846,7 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, btrfs_trans_release_metadata(trans); trans->block_rsv = NULL; - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); btrfs_trans_release_chunk_metadata(trans); @@ -1934,8 +1933,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing = 1; smp_wmb(); - if (!list_empty(&trans->new_bgs)) - btrfs_create_pending_block_groups(trans); + btrfs_create_pending_block_groups(trans); if (!test_bit(BTRFS_TRANS_DIRTY_BG_RUN, &cur_trans->flags)) { int run_it = 0; From patchwork Fri Oct 12 19:32:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639203 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 894FC17E3 for ; Fri, 12 Oct 2018 19:34:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BCAB2C546 for ; Fri, 12 Oct 2018 19:34:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 701E32C574; Fri, 12 Oct 2018 19:34:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 66A8F2C546 for ; Fri, 12 Oct 2018 19:34:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727128AbeJMDII (ORCPT ); Fri, 12 Oct 2018 23:08:08 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:38790 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726530AbeJMDII (ORCPT ); Fri, 12 Oct 2018 23:08:08 -0400 Received: by mail-qt1-f193.google.com with SMTP id l9-v6so15069500qtf.5 for ; Fri, 12 Oct 2018 12:34:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=2v3HNp+SiUuKkJAWzeR46bMWpcvQbLX2jWxeWYdX+Ok=; b=NCgWsohdYsXKratC8htSWExXwEtFINv2XfjX2Qu1N+u1XPgQ5T84AzWbkyvxfz5LhV LSyRjrWZXBwl3iuJfFofjI8+/Gz6ivTxbTBrK/B69L6tkjAdADmD2whIv+sHutOArFLh oemeg525qzuczeB8o6fU/fQwLXvONTqTYUHStE3R/9dr3zwpreVsa6UIFeJPbvsEKL5X QHhTkRZzxaPzPLeyCZrFkq1tSZu577iFxMjPPD618IUQ9GFlzCJfHClLZjOSE3jS8Fkg baWg++BzAquN7LxFauImwAvEAYIXh5q796zMGtyRGEFkzCfY2iVgC9GmgeqttzvDd7j9 +Odw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=2v3HNp+SiUuKkJAWzeR46bMWpcvQbLX2jWxeWYdX+Ok=; b=Ddg290kNPZAlfrtasx9gTDRu26hHQt3InTjB4e6ie+KtfUfjI305UoUFEJC0mxb/Mo gn0OiIDtvrfluiZgtOwp3wdkteY3qkbACodzHLkaBUNGTirVatrGT/UumPKBvlu6xNfV QiL7iVN/j2p398ZNdYxuAprQh4PMPaoBIIZh2xtW/TvoqNMCCsmVU2HZAXHs7PH6tpKD aLXDCXwTmGOgP1PL+hcRSBvi+yGhW5GqT932NvtXVF013nXGBmNbAB4pDQ3wnka/EaEY 3pVtEyMnRBIk1aEVawUBEnoDbfusqGRdgcNWWFlHlDwpWEuQ/LqskzSU/X9IFnWuzZZj Vexg== X-Gm-Message-State: ABuFfoj+VkZ9iXfavgbdtafoyqs9TmC27zLjSl8t0LJ6l6elBL+DJ2iq hhmD0WW/0gN/VsBOVrUxSshZPDDqlSs= X-Google-Smtp-Source: ACcGV62If6XmjTofKNOvwcwwt2UF3ZpFXdNl7G1XCwoYCObJdIhc0VrzZLDYQqsm3tpYQm7qm/7UDA== X-Received: by 2002:aed:2647:: with SMTP id z65-v6mr7020018qtc.301.1539372846646; Fri, 12 Oct 2018 12:34:06 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id n67-v6sm1803906qkh.66.2018.10.12.12.34.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:05 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 30/42] btrfs: just delete pending bgs if we are aborted Date: Fri, 12 Oct 2018 15:32:44 -0400 Message-Id: <20181012193256.13735-31-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We still need to do all of the accounting cleanup for pending block groups if we abort. So set the ret to trans->aborted so if we aborted the cleanup happens and everybody is happy. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 14495c030301..933cba06c9fb 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -10363,9 +10363,15 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans) struct btrfs_root *extent_root = fs_info->extent_root; struct btrfs_block_group_item item; struct btrfs_key key; - int ret = 0; + int ret; bool can_flush_pending_bgs = trans->can_flush_pending_bgs; + /* + * If we aborted the transaction with pending bg's we need to just + * cleanup the list and carry on. + */ + ret = trans->aborted; + trans->can_flush_pending_bgs = false; while (!list_empty(&trans->new_bgs)) { block_group = list_first_entry(&trans->new_bgs, From patchwork Fri Oct 12 19:32:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639205 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9BC58157A for ; Fri, 12 Oct 2018 19:34:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D1EA2C546 for ; Fri, 12 Oct 2018 19:34:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 80C1F2C574; Fri, 12 Oct 2018 19:34:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 313BB2C546 for ; Fri, 12 Oct 2018 19:34:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727129AbeJMDIK (ORCPT ); Fri, 12 Oct 2018 23:08:10 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:37584 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726227AbeJMDIJ (ORCPT ); Fri, 12 Oct 2018 23:08:09 -0400 Received: by mail-qt1-f193.google.com with SMTP id d14-v6so15064398qto.4 for ; Fri, 12 Oct 2018 12:34:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=esyabqCBu+9iDj6gymgoZZfIo7W+xlWCseOBiKeoQzQ=; b=2HFFGqV0bAxAaOijWNshjE6G/88Qxic6X1sRVPFM+yr2ufTDIVKOfskbiPZF/jySdz ioCEJkxvsbJxF2+O3+6IuZmmgRmh1DGvTb++EBLly2oZwBu9ebl6Aq+ij+2G18UxwTpW gQDxJ2BTF1xv0EXfe2y0A4W/0p5fHzem58OMrAQypH9+lZGinkFEjKhWbCkYUPrpIfaz 2gMNvkLcEYQSd24IM+h08M8roeqzxFWNdTdHk4Or/aW/srvTvDnDDdCdYoqe90Fc+1Dw TzWesY1YZpDzFPCyPLhTZ0/xyt3j7g8jm9uDa5AOXZIhPoYsZvMBMwa3I3Vg/DexSSlp haTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=esyabqCBu+9iDj6gymgoZZfIo7W+xlWCseOBiKeoQzQ=; b=QERhAsMuwPK+AdAwyJj72omT/Nm4pA3YfB3zJfJrnZeoCC/vs4VVn+wF5x2ZAK0fcl VKWz1TMG7dENTR6O5iCz2vpXvpeMvf2RjMQMgon+D/N7voBOJEuOT3d+n6H1udjkE+P3 bjvwDINO8SqBjxMN84FxdciPyP2I+9/VlI0jFQ3TJcw8hNIKl28hsQF8tesxYfRn/zrx uRfnwgun0iNWm6e1jC0eQm6P8WgYGRqSEPt5BN28t8biTPJJcU++LEReC8O/yuHNZInd p874mIPHjNItgZ8MK8D0kJ8C4fS+fgPArI8YBPXtha/G0u8dUxFC4PXj7iqz3q2Bcw5s dKrg== X-Gm-Message-State: ABuFfogSl1C9LW9xnxvGtCZ1QzehGQN7Y9k09WuaPAC+yJI8tyS/jJNQ 1KV9kZTGQ0Br72D9MjzESP94IvDszHI= X-Google-Smtp-Source: ACcGV602pW3005r1d1Qjhbenv9cdAaQ+RxK/U/137oG2v4HcE4dc6zI8aO6FzXq3EQbvdoCQuF/D3w== X-Received: by 2002:a0c:fd8c:: with SMTP id p12-v6mr7104537qvr.198.1539372848521; Fri, 12 Oct 2018 12:34:08 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b134-v6sm1519145qka.45.2018.10.12.12.34.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:07 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 31/42] btrfs: cleanup pending bgs on transaction abort Date: Fri, 12 Oct 2018 15:32:45 -0400 Message-Id: <20181012193256.13735-32-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We may abort the transaction during a commit and not have a chance to run the pending bgs stuff, which will leave block groups on our list and cause us accounting issues and leaked memory. Fix this by running the pending bgs when we cleanup a transaction. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 46ca775a709e..9168efaca37e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -2280,6 +2280,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_scrub_continue(fs_info); cleanup_transaction: btrfs_trans_release_metadata(trans); + /* This cleans up the pending block groups list properly. */ + if (!trans->aborted) + trans->aborted = ret; + btrfs_create_pending_block_groups(trans); btrfs_trans_release_chunk_metadata(trans); trans->block_rsv = NULL; btrfs_warn(fs_info, "Skipping commit of aborted transaction."); From patchwork Fri Oct 12 19:32:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639207 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2533F17E3 for ; Fri, 12 Oct 2018 19:34:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17B342C55C for ; Fri, 12 Oct 2018 19:34:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0BDB22C575; Fri, 12 Oct 2018 19:34:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A358A2C55C for ; Fri, 12 Oct 2018 19:34:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727141AbeJMDIM (ORCPT ); Fri, 12 Oct 2018 23:08:12 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:43199 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726227AbeJMDIM (ORCPT ); Fri, 12 Oct 2018 23:08:12 -0400 Received: by mail-qt1-f194.google.com with SMTP id q41-v6so15045068qtq.10 for ; Fri, 12 Oct 2018 12:34:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=OTZOuDgDt/R5spujlNFzi4h/V31eI1viGWGcVEUBX/Y=; b=BznmhAh1nQe++duAhbVQugXyBiF/Df+N5Lwb+2mjrpkID+der8ZW9AwHTqikimwwi5 x43VFVHWS2aSqHwCh6fFEWcfjgARDZHMmQOag2TjGhvDMucQ2nVYWTxR6XXBfpv3jj6z F3+xnSdy/tC+k2A/uXZKBaXEjvUTrFthlMfggE0lhUmBNZIZgdVlZoa2PQDdrsPD8SvN x3FlmgnNUH0geCTViQQ3IDeiZxGVWpWrSTDVPy6OCkqPlCfvDmk2W9UF0DlDCpaBGuPc LM7GiJk9SXDdrJlTu/v0GFQsBwW9XNgOKovt+1ZqHFpQ2vKaXG8nSDFg3M4T691XGxWm glxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=OTZOuDgDt/R5spujlNFzi4h/V31eI1viGWGcVEUBX/Y=; b=YbGy/GjPmUwEfjHJvDPJBQhSkDGInbYOLXWo03fN+SwhBTrMj/PiHvyf1795AmGBDa AHsAae+loHisaaCKrqITE2K4hGIe9V2Vkm3+dVax9Dkm5S3J8xv1v1PG9WCsYlceZFvM sQPdgqxIT0DLQY1b0rTAWdD1zPpmrnTbVUMQX5cEqWayYsGXTcX1ndqF+h3gOS9mtYLc NKwa7HCTS01ddR4pQiX9uU2NmpGxBMjVKQemws1GmYAiaGG4CKTv9jQwdBejqtffwvLl KcsH0UZ2jNYFSm0rHESwubStXVP08jvj+TgZepaKrzrcv7lLPbmPk54IhGkf+m4lVao3 uHBg== X-Gm-Message-State: ABuFfoguDO7Qa+KQzmbXsC5ROHQTtpBXL3XkSiqIBv7qu2oz8waBqEF5 jZLaKhi7LikbwGFK771Fr1DwuUKvP8k= X-Google-Smtp-Source: ACcGV60nzGyhC1itymQUMd78E589/B2gMwK2XHn7lmlG3RwZeeWQYpILwizuEynfVCUaCTlVMZyjGg== X-Received: by 2002:ac8:7287:: with SMTP id v7-v6mr7072272qto.126.1539372850771; Fri, 12 Oct 2018 12:34:10 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c2-v6sm1139792qkj.79.2018.10.12.12.34.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:09 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 32/42] btrfs: only free reserved extent if we didn't insert it Date: Fri, 12 Oct 2018 15:32:46 -0400 Message-Id: <20181012193256.13735-33-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When we insert the file extent once the ordered extent completes we free the reserved extent reservation as it'll have been migrated to the bytes_used counter. However if we error out after this step we'll still clear the reserved extent reservation, resulting in a negative accounting of the reserved bytes for the block group and space info. Fix this by only doing the free if we didn't successfully insert a file extent for this extent. Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/inode.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 5a91055a13b2..2b257d14bd3d 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -2992,6 +2992,7 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) bool truncated = false; bool range_locked = false; bool clear_new_delalloc_bytes = false; + bool clear_reserved_extent = true; if (!test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags) && @@ -3095,10 +3096,12 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) logical_len, logical_len, compress_type, 0, 0, BTRFS_FILE_EXTENT_REG); - if (!ret) + if (!ret) { + clear_reserved_extent = false; btrfs_release_delalloc_bytes(fs_info, ordered_extent->start, ordered_extent->disk_len); + } } unpin_extent_cache(&BTRFS_I(inode)->extent_tree, ordered_extent->file_offset, ordered_extent->len, @@ -3159,8 +3162,13 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) * wrong we need to return the space for this ordered extent * back to the allocator. We only free the extent in the * truncated case if we didn't write out the extent at all. + * + * If we made it past insert_reserved_file_extent before we + * errored out then we don't need to do this as the accounting + * has already been done. */ if ((ret || !logical_len) && + clear_reserved_extent && !test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags)) btrfs_free_reserved_extent(fs_info, From patchwork Fri Oct 12 19:32:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A634157A for ; Fri, 12 Oct 2018 19:34:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B9452C546 for ; Fri, 12 Oct 2018 19:34:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FE502C574; Fri, 12 Oct 2018 19:34:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFD122C546 for ; Fri, 12 Oct 2018 19:34:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727148AbeJMDIP (ORCPT ); Fri, 12 Oct 2018 23:08:15 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:44723 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726227AbeJMDIO (ORCPT ); Fri, 12 Oct 2018 23:08:14 -0400 Received: by mail-qk1-f196.google.com with SMTP id y8-v6so8339783qka.11 for ; Fri, 12 Oct 2018 12:34:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=OmnZZhmWbwXgq96ydypXLUVdgaCmw0VaXUN9znKi4C4=; b=itXGjxim6NfMLCtd+mEWueQ/RG+2jt+Jf8HkEeMSCXfPU6C7pwqpZFtO7/cNXSxTTc nA/osNt34LcHp3+t3Z5fSlUun45a6KUq1BkcIunMtySFKVfz26c8+uEBO4DNt4OQsr95 EPnQb+6JhGpVos7HlMKi1+qoHINpi/UtG6uKk6cGJ9cR30B5tRnjbmtrJZ5zetOuFY/X LH5Wc7bjHEKFm6WBsviXX2DT+bWHT8JpxwdkyTjfoW3btP8G0KEKIbJwQAyZlKPrOKRL PMBdhS3zqLB1VYl9XHgC/S7YqOxbqGUSrDoYBS0ba8t4ARZsvTX/12jfSaxjOgVZ4ie7 d+lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=OmnZZhmWbwXgq96ydypXLUVdgaCmw0VaXUN9znKi4C4=; b=mYNgn1bISun5j7KkQuy9fpWzNhJkq8XtIpiTueuQWmSoLErskeUn+fnBOYsh3jK62+ kgHEl5pGVrlTLiVoYziByTgs7gvhCUCuArX2RggFCtrg3781Eu6XdYkP/pMonAGMUSJF 0axj0ffuuyrP3LN5A3oq15Zu9sH5yqaSlP4RaFUMjki4GFzk6Bx8vqNqUBq5Ey74WLGt VvsHVn929duOuLW64v1cuEEunnLhAv6O8/KSxd0IPKqE0Jb6w7JQkCwWe7vhYe5oLu7F 8qsf/5r26zsnVuAejmvbPozYI1zO4KeBXfpOjZ43+6E4b7Rhh7W712U6/aD6mGE0X1N8 hzLQ== X-Gm-Message-State: ABuFfohgTKx6S9CdzYj1JHDzFp3jXTfDyc4Tztv7tK/1oWIrAM/9yLIY 0GiC00MbaAKW92UsZuyYOXsf4WGJRSw= X-Google-Smtp-Source: ACcGV61u3sGiLZ8bkAshUuQ4ZmDDUwZxKLODqjqvokNzRyxsqZ28LZewQ0PCx44ZoD2GKODpeBzMlQ== X-Received: by 2002:a37:15e7:: with SMTP id 100-v6mr7069328qkv.151.1539372852829; Fri, 12 Oct 2018 12:34:12 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a17-v6sm1575380qta.0.2018.10.12.12.34.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:11 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 33/42] btrfs: fix insert_reserved error handling Date: Fri, 12 Oct 2018 15:32:47 -0400 Message-Id: <20181012193256.13735-34-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We were not handling the reserved byte accounting properly for data references. Metadata was fine, if it errored out the error paths would free the bytes_reserved count and pin the extent, but it even missed one of the error cases. So instead move this handling up into run_one_delayed_ref so we are sure that both cases are properly cleaned up in case of a transaction abort. Reviewed-by: David Sterba Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 933cba06c9fb..9b5366932298 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2405,6 +2405,9 @@ static int run_one_delayed_ref(struct btrfs_trans_handle *trans, insert_reserved); else BUG(); + if (ret && insert_reserved) + btrfs_pin_extent(trans->fs_info, node->bytenr, + node->num_bytes, 1); return ret; } @@ -8257,21 +8260,14 @@ static int alloc_reserved_tree_block(struct btrfs_trans_handle *trans, } path = btrfs_alloc_path(); - if (!path) { - btrfs_free_and_pin_reserved_extent(fs_info, - extent_key.objectid, - fs_info->nodesize); + if (!path) return -ENOMEM; - } path->leave_spinning = 1; ret = btrfs_insert_empty_item(trans, fs_info->extent_root, path, &extent_key, size); if (ret) { btrfs_free_path(path); - btrfs_free_and_pin_reserved_extent(fs_info, - extent_key.objectid, - fs_info->nodesize); return ret; } From patchwork Fri Oct 12 19:32:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69104157A for ; Fri, 12 Oct 2018 19:34:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B5262C546 for ; Fri, 12 Oct 2018 19:34:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FE082C569; Fri, 12 Oct 2018 19:34:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E348E2C546 for ; Fri, 12 Oct 2018 19:34:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727160AbeJMDIR (ORCPT ); Fri, 12 Oct 2018 23:08:17 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:40369 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726227AbeJMDIQ (ORCPT ); Fri, 12 Oct 2018 23:08:16 -0400 Received: by mail-qk1-f196.google.com with SMTP id a13-v6so8345884qkc.7 for ; Fri, 12 Oct 2018 12:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=o3aniGDG3zSjx1n1+LpysocBJ3oenkOpLCURB29A0Uc=; b=oPkG/N9GSND2Hn2d4ZXschpHbq9qm0Q/I76A1Zi5E1rGHy6ENLGgqD5gBFZrdKWlQM NK1FgM2/N9gfTmAn1S81IvSBGPjvgVy6REnwo78ca9buuB3iNvsxEeAhY5b4Pb3wDADP Gbpx5bJgKFqoy2+DCntfoAlpsOHLgdwLimMJ6IhsEjtnSAIh9te1B5zq13m4MVTWTvDm fGwL1lerBWW11aRDFxxUAnoZLRAiXj9nmtxRrlFgNNZ/NFeUTg4unWBjFPgFIbPcUflZ Q7yKeDueIcj8eZPT6hRL9T2FrHfuor9S1qmJvLXFXeuhtHBWC99dc524iT6f6BiFHVV9 BaWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=o3aniGDG3zSjx1n1+LpysocBJ3oenkOpLCURB29A0Uc=; b=OqfLtFnbR7obkJg2YFDtP7QAbr83A4ueCI5fQlSXYm/hR1UKLQOvlTaV/Jk1AJMoLl gbGHdV826/pJULN07FRwMTrGjwIzztK5WxZ27kwzX7EeARr7Wn7bVsf1s8xQ/kwZER12 FYKqnNu3/G2ng0awk+9Ccgk3TAWxo0ADd+l8ZdYQ85njnU0hRMVqDd8/bDWJiDnzlpIu zWY0cVZp0l/x1IU0XUd7GQul/sg84JRCESt+uOiI9wpQY8a9rPqEq87KZViR+I9Olu95 46AXOCF8+bttljotYzUSUYHe8vBysOL/PcXfod1jNbLOXpm3OmW2Y32HyXITn8sHg+Qb 3maQ== X-Gm-Message-State: ABuFfoiOvQQPfeU3iZEZOuWKuljddwAw0FfZ/3KqDj6c2HFOeS1wzqfv voM0T/0Y2Vf/S8lO5Qjv/0DzoV3T+Ps= X-Google-Smtp-Source: ACcGV63CVQoPGc5OIrfbu1vYe56DWVD9X64kArsv0y0oRABiEXxZ/LpzDSgK98MKkfiuSQ0vofI+CA== X-Received: by 2002:ae9:e30e:: with SMTP id v14-v6mr7098104qkf.356.1539372855768; Fri, 12 Oct 2018 12:34:15 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m64-v6sm1209232qkf.18.2018.10.12.12.34.14 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:14 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 34/42] btrfs: wait on ordered extents on abort cleanup Date: Fri, 12 Oct 2018 15:32:48 -0400 Message-Id: <20181012193256.13735-35-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we flip read-only before we initiate writeback on all dirty pages for ordered extents we've created then we'll have ordered extents left over on umount, which results in all sorts of bad things happening. Fix this by making sure we wait on ordered extents if we have to do the aborted transaction cleanup stuff. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 54fbdc944a3f..51b2a5bf25e5 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4188,6 +4188,14 @@ static void btrfs_destroy_all_ordered_extents(struct btrfs_fs_info *fs_info) spin_lock(&fs_info->ordered_root_lock); } spin_unlock(&fs_info->ordered_root_lock); + + /* + * We need this here because if we've been flipped read-only we won't + * get sync() from the umount, so we need to make sure any ordered + * extents that haven't had their dirty pages IO start writeout yet + * actually get run and error out properly. + */ + btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, (u64)-1); } static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, From patchwork Fri Oct 12 19:32:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639213 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 891FF157A for ; Fri, 12 Oct 2018 19:34:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B4902C546 for ; Fri, 12 Oct 2018 19:34:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6F9C02C569; Fri, 12 Oct 2018 19:34:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 224652C546 for ; Fri, 12 Oct 2018 19:34:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727162AbeJMDIU (ORCPT ); Fri, 12 Oct 2018 23:08:20 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:41008 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726227AbeJMDIU (ORCPT ); Fri, 12 Oct 2018 23:08:20 -0400 Received: by mail-qt1-f194.google.com with SMTP id l41-v6so15054644qtl.8 for ; Fri, 12 Oct 2018 12:34:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ja6BEdk2d172wcHDS/gitUITWM0Dn5OfC8ykxBklKgo=; b=jDGBbrxyO5K2LDatw6+hvmpr4HwUbD60+98ZOeDVMZMUCXdBclOeP/e2zIHm5BplEm ylNdpu+4bVlgXWjm2z1NvexOE53x2qQg6xyd/aEVqQSoaEbpO+hQEmEODH7buRXQZ2Qu GUit+qcEugAM6axUbemshegABkEi/Fij2QHqUorz7cL4ZfAxy2gxb8qEqpU9fZZbZ5No POuc6w7o8SDB2f9JNYJOeDtY+NvIjquTzzAr9DNXcfR3AaS4Z4bXWYInIMTHgxOXlP1g fnlQDu7xlHZ8kV/gUzxEaGSY5l09FNR8AXgGlcXt3KZYzetaH9NS2XvURKsrBDTSIPZ9 arqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ja6BEdk2d172wcHDS/gitUITWM0Dn5OfC8ykxBklKgo=; b=sr24EES/3X3EILMQafpto3reakPPi8QVXXILxV8ypppbZnAF0vDEBSHSA7LyymSF9X 9xY5cpse5WFD7VgfBK4xK3z36R312BdpnU9Ug3ku/TWQBiqEUy80YloBvVZKLQquY4rZ 7B8/ZyUoI3rdJ0ilf/lTpBL+tKwiHLASO7vZO35vr0Mq9aPkFlE9vJuJhr6bPrCAv4Qx XWVcZAnZCP7WxQO5akWDwcuwmb+vjt1Gr09h/zukc+XHTOOWI05qkKCXHnaaiNthTV/7 zzsQyOudvfuc9IGsyxp/pdoXE4bPWEYObu9OWzsT/qGvG2aeX2Xj+CRsu5THw+HI58Ay Ot9Q== X-Gm-Message-State: ABuFfojJ0HAIJMG8ApGJlwxsZ4i7AMmlaLrBieQyvERxft9rA7zfOMMS /PVcNBYgorycFdvkhMsFIq3C4i9jtYA= X-Google-Smtp-Source: ACcGV61B1GGRUeprZJLoUFbTmbmJ+akAWhKHU/qXQbRhB2BQut2JFqAs/GvOltKkwM4c1sYvzWjeLQ== X-Received: by 2002:ac8:373a:: with SMTP id o55-v6mr7165115qtb.224.1539372858722; Fri, 12 Oct 2018 12:34:18 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v57-v6sm1350785qtk.51.2018.10.12.12.34.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:17 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 35/42] MAINTAINERS: update my email address for btrfs Date: Fri, 12 Oct 2018 15:32:49 -0400 Message-Id: <20181012193256.13735-36-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP My work email is completely useless, switch it to my personal address so I get emails on a account I actually pay attention to. Signed-off-by: Josef Bacik --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index 32fbc6f732d4..7723dc958e99 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3095,7 +3095,7 @@ F: drivers/gpio/gpio-bt8xx.c BTRFS FILE SYSTEM M: Chris Mason -M: Josef Bacik +M: Josef Bacik M: David Sterba L: linux-btrfs@vger.kernel.org W: http://btrfs.wiki.kernel.org/ From patchwork Fri Oct 12 19:32:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639215 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 629AF157A for ; Fri, 12 Oct 2018 19:34:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5403D2C546 for ; Fri, 12 Oct 2018 19:34:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 481182C569; Fri, 12 Oct 2018 19:34:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE3042C546 for ; Fri, 12 Oct 2018 19:34:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726967AbeJMDIY (ORCPT ); Fri, 12 Oct 2018 23:08:24 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:33654 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDIX (ORCPT ); Fri, 12 Oct 2018 23:08:23 -0400 Received: by mail-qt1-f195.google.com with SMTP id q40-v6so15087091qte.0 for ; Fri, 12 Oct 2018 12:34:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=CmIQJ2t4XtqsjVffqXoBO7xUxEc9XCxX0h3fJq21XS8=; b=jH9FB/nllsXsHUyfQHofb/3U/Dtctfx5CW99Q/6QH/tcS8jmT0xgWwdrS+Z5gIk6AL j2ti7iuUv2XPLavGOlWFYQCxGpv80PQg/Jy/sAZXq6Odk/p8RBH9K9OdKhISHrQis8CB Pd4f9Nv1Y2UHlhpeMN5ooPLZ5D5IwKk6kLjoHDlqvjgXg/ApjVXjJPhkx7yU5qtB/QZc tpayMVGy/qdIhylD+3cqTsuM3rOPTvyMDEDzjRpjk1Ns70TL8ncn9L69UgwkaOdn20rf lTzJ711kHC51RrlOv/RbHq6rKDqvGsVzjL9Q1hZDRMvJcCeOAITTrGSFb7HXTiMv9Sp5 cJLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=CmIQJ2t4XtqsjVffqXoBO7xUxEc9XCxX0h3fJq21XS8=; b=OQ15B+C+Lj92Ve0smFHwTD6F10gPYPRmcJk26ehNTY+WZEelc6+NoESjNIv+nPSBn9 05hnbvMP7PnvL1IOO2jLt2j7g2R65XFc5MeG+EoQ29DFQM6GOrpFuVeO9+8LFoTYjMEh EoWze9FJJSADwQNyEIqgPkwFn4tEpLa0MAKNd4RhW4dCxRyFh0gcWnD7vsuiIJj2pdrq EmQ3BLPNl9p51yC16dHMeRnFPr9LInrSq4ebNGEGNQrKAjBisSEuOiTQu5HtgFfybZwp jqABc5K7E+l6oYaIr2BLpVa8O5WhadfQdgTycKZiF9JX/5oqnI2wJVWI773SNtTmYpAd C02A== X-Gm-Message-State: ABuFfohCOvWw6c3cDnqbuE2x1x/QhCLpa+UR4RKrvEuDCp9RQ6b6Ev0w y9xev14XXAcfQYvLwQUzSyoKF+x0nZk= X-Google-Smtp-Source: ACcGV63ft6QL35tMqzHOQdH5Z/94Jrfmz3R6Y82O+6UE7I0O45xMAwvSv4ug1lZWSGQkUTagEJdhBA== X-Received: by 2002:a0c:b62c:: with SMTP id f44-v6mr7531092qve.184.1539372861952; Fri, 12 Oct 2018 12:34:21 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id y26-v6sm1256695qty.52.2018.10.12.12.34.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:20 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 36/42] btrfs: wait on caching when putting the bg cache Date: Fri, 12 Oct 2018 15:32:50 -0400 Message-Id: <20181012193256.13735-37-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is because we aren't waiting for the caching threads to be done before freeing everything up, so to fix this make sure we wait on any outstanding caching that's being done before we free up the block group, so we're sure to be done with all IO by the time we get to btrfs_stop_all_workers(). This fixes the panic I was seeing consistently in testing. ------------[ cut here ]------------ kernel BUG at fs/btrfs/volumes.c:6112! SMP PTI Modules linked in: CPU: 1 PID: 27165 Comm: kworker/u4:7 Not tainted 4.16.0-02155-g3553e54a578d-dirty #875 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014 Workqueue: btrfs-cache btrfs_cache_helper RIP: 0010:btrfs_map_bio+0x346/0x370 RSP: 0000:ffffc900061e79d0 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff880071542e00 RCX: 0000000000533000 RDX: ffff88006bb74380 RSI: 0000000000000008 RDI: ffff880078160000 RBP: 0000000000000001 R08: ffff8800781cd200 R09: 0000000000503000 R10: ffff88006cd21200 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: ffff8800781cd200 R15: ffff880071542e00 FS: 0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000817ffc4 CR3: 0000000078314000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: btree_submit_bio_hook+0x8a/0xd0 submit_one_bio+0x5d/0x80 read_extent_buffer_pages+0x18a/0x320 btree_read_extent_buffer_pages+0xbc/0x200 ? alloc_extent_buffer+0x359/0x3e0 read_tree_block+0x3d/0x60 read_block_for_search.isra.30+0x1a5/0x360 btrfs_search_slot+0x41b/0xa10 btrfs_next_old_leaf+0x212/0x470 caching_thread+0x323/0x490 normal_work_helper+0xc5/0x310 process_one_work+0x141/0x340 worker_thread+0x44/0x3c0 kthread+0xf8/0x130 ? process_one_work+0x340/0x340 ? kthread_bind+0x10/0x10 ret_from_fork+0x35/0x40 Code: ff ff 48 8b 4c 24 28 48 89 de 48 8b 7c 24 08 e8 d1 e5 04 00 89 c3 e9 08 ff ff ff 4d 89 c6 49 89 df e9 27 fe ff ff e8 5a 3a bb ff <0f> 0b 0f 0b e9 57 ff ff ff 48 8b 7c 24 08 4c 89 f9 4c 89 ea 48 RIP: btrfs_map_bio+0x346/0x370 RSP: ffffc900061e79d0 ---[ end trace 827eb13e50846033 ]--- Kernel panic - not syncing: Fatal exception Kernel Offset: disabled ---[ end Kernel panic - not syncing: Fatal exception Signed-off-by: Josef Bacik Reviewed-by: Omar Sandoval --- fs/btrfs/extent-tree.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 9b5366932298..4bacb5042d08 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -9894,6 +9894,7 @@ void btrfs_put_block_group_cache(struct btrfs_fs_info *info) block_group = btrfs_lookup_first_block_group(info, last); while (block_group) { + wait_block_group_cache_done(block_group); spin_lock(&block_group->lock); if (block_group->iref) break; From patchwork Fri Oct 12 19:32:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639217 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E17517E3 for ; Fri, 12 Oct 2018 19:34:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 60B4B2C546 for ; Fri, 12 Oct 2018 19:34:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 553DD2C569; Fri, 12 Oct 2018 19:34:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 06C342C546 for ; Fri, 12 Oct 2018 19:34:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727164AbeJMDI0 (ORCPT ); Fri, 12 Oct 2018 23:08:26 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:37628 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDI0 (ORCPT ); Fri, 12 Oct 2018 23:08:26 -0400 Received: by mail-qt1-f193.google.com with SMTP id d14-v6so15065223qto.4 for ; Fri, 12 Oct 2018 12:34:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=zZszbBJ2wYqnRaCO5pCUOw7tHC5U99aK2TNTpFeSZ9Q=; b=pgziAFro1R6lxCBjS1Sg+32a1iqvPlpMfohfHpMcD/QeZyBHeRlkbchDuWiG/Lu1qf 5ln/VsTBAWxkupqwHpAIVJ77iSBz6rCIu+2swMUIMvujDzLWHebGZddfT77Rk5z3h1RC KdtH2mbKAkkH6mR/GWGAwHngU0XAJLN3XmCfBVE/RecotHZJhSVlxsVAbOsxHp+nfatK Od2Vm7ZuXVuRVTgxGyLGh0AwfpQgz0Ki/fHSKxfkIhIUV+1EKNWEEc4Lfaldj0o8dRqg ELNkrpyPocfxHPKLSEbPA7yRuYhvozWj8s3eeobZH41FYB7ebUOSKgMglaV4MpAFJ9vC 0OXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=zZszbBJ2wYqnRaCO5pCUOw7tHC5U99aK2TNTpFeSZ9Q=; b=NQS+EAdDAmbHU4IQwTltSimnEEuFjQe8Layj6nGBxypmmK++Mr5mlsonWdSNJCaFu+ sz3Ow2Ed74hy8ospm15Up2DGPyeIRx8X8ZPLpQPi1HJBh9nJr0CSRMrylUS9COBXupaA BRi9bf/FQaQLYZb6PLi1xteDrVmQ1mjbF2rWpP+ptee7RRURDPMzIpaI403uwFvqo+qV v6W9HOcI3YBO/cVx/AIVWefr2FyLEx2XnFAzQMDqR2WnVDUKwJNA+FZdLxTXibycWHbt gL5uX4Yn/GLGgN2w0KQijeJgP4DcXoleW1XWGS2YFH0QjNaRka7Eg7zLsk/Ivnhv4TjT Ocaw== X-Gm-Message-State: ABuFfojRwkW4E3m7aLwR25fzvPa9oT7iymwQ7tsvr8BIKNObgofd285b LbuU5cw5cDU5ugJgB8+ztohQUHXIFrg= X-Google-Smtp-Source: ACcGV62ZL4qM+jCuipY5MweN25SRh6oQtUR6BuMjB09/A0DiBfX0BwMPYNDmjM74ZFOoprgzS8d6nA== X-Received: by 2002:ac8:7482:: with SMTP id v2-v6mr7147733qtq.251.1539372864452; Fri, 12 Oct 2018 12:34:24 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e24-v6sm1476954qtc.78.2018.10.12.12.34.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:23 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 37/42] btrfs: wakeup cleaner thread when adding delayed iput Date: Fri, 12 Oct 2018 15:32:51 -0400 Message-Id: <20181012193256.13735-38-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The cleaner thread usually takes care of delayed iputs, with the exception of the btrfs_end_transaction_throttle path. The cleaner thread only gets woken up every 30 seconds, so instead wake it up to do it's work so that we can free up that space as quickly as possible. Reviewed-by: Filipe Manana Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 2b257d14bd3d..0a1671fb03bf 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -3323,6 +3323,7 @@ void btrfs_add_delayed_iput(struct inode *inode) ASSERT(list_empty(&binode->delayed_iput)); list_add_tail(&binode->delayed_iput, &fs_info->delayed_iputs); spin_unlock(&fs_info->delayed_iput_lock); + wake_up_process(fs_info->cleaner_kthread); } void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info) From patchwork Fri Oct 12 19:32:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639219 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF60217E3 for ; Fri, 12 Oct 2018 19:34:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A02F32C546 for ; Fri, 12 Oct 2018 19:34:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 947D62C569; Fri, 12 Oct 2018 19:34:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3AC8A2C546 for ; Fri, 12 Oct 2018 19:34:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727170AbeJMDI3 (ORCPT ); Fri, 12 Oct 2018 23:08:29 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:36878 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDI3 (ORCPT ); Fri, 12 Oct 2018 23:08:29 -0400 Received: by mail-qk1-f193.google.com with SMTP id x8-v6so8356848qka.4 for ; Fri, 12 Oct 2018 12:34:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=DGKAE6OrSG2VtkUO5QHGUrYV9YOGfUXps164Bg2g4BM=; b=wujtVS+Uwn4ajp7Jr5YHSOhIlNDmw9g3cUmzLaIdNI3wSRo0mEWf6a9xwf13/Ar+/v ehHW/jTzUk3EivGPoXGlAiS8G2u95z5FdVaTAt20hTQQbrulBJNVTJxIJxkkArkQmb0J jz65UL/KlTAONjDa5YjrUPFEzcYbX+zivaekAmqS0JF2juobyadW1rjZ1Wz4Df1Qsz5D lwBC5jqDZHTJ3Xarv5T+9Pk+5yp4wzM90nVl4KFM8Wmu0fN/xwx/WxNBM4S7DDgii/qp S44MWrii52PV3o0WV//70XBNcvr+Y8kp5cbK8MiaL4A45/pRjtFCalQ0Mf3iN/0c4R+w WZLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=DGKAE6OrSG2VtkUO5QHGUrYV9YOGfUXps164Bg2g4BM=; b=PFi67itHg0DIMzVnrfMbFDzVCjzeVzTpoVrUBdqYPAGtXrloMbU+pGAOMxfnoNKfuN ogDRz/PvEiWbB50vMmHNSlDHoA0D3ByyFcWcECFraBYQAjD+rhS+lDtFMrD571y1eafN Odx3VLhpMWzBhVimkmh66MzfLDClmOqMhwiRAxYf86teg5ErqYAFCPE/4bJa+VsRl/Pq LshFrY+octG/kmlzuUTXUMtqbj0M3URtEz99wxYKrNNKJg5FiJRUFi+fvCr47+Xa78li O9LzSSwMRf7l6ockRG60w3Y8Fr7EVH1ofzcNaWqB1Ui18cOju6czIN3MpI1K/Q3rcE53 RieA== X-Gm-Message-State: ABuFfohILjT1KadBF1tE/uDPIV/xIQlU8gDaRiWdrZ22d+HJDtluRW+2 CmDGxcG5iUe21Sk1au8D9VLRcQddfFE= X-Google-Smtp-Source: ACcGV62lbdepFizHwVOYuqF2NK0zwNeHCkR/B1+iz1Ktg/481viUmsyoXtHLeCp073F6x54IsAowAw== X-Received: by 2002:a37:d413:: with SMTP id l19-v6mr6797439qki.21.1539372867467; Fri, 12 Oct 2018 12:34:27 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id i27-v6sm1805191qti.57.2018.10.12.12.34.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:26 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 38/42] btrfs: be more explicit about allowed flush states Date: Fri, 12 Oct 2018 15:32:52 -0400 Message-Id: <20181012193256.13735-39-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For FLUSH_LIMIT flushers we really can only allocate chunks and flush delayed inode items, everything else is problematic. I added a bunch of new states and it lead to weirdness in the FLUSH_LIMIT case because I forgot about how it worked. So instead explicitly declare the states that are ok for flushing with FLUSH_LIMIT and use that for our state machine. Then as we add new things that are safe we can just add them to this list. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 4bacb5042d08..2056d5d37dfe 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5110,12 +5110,18 @@ void btrfs_init_async_reclaim_work(struct work_struct *work) INIT_WORK(work, btrfs_async_reclaim_metadata_space); } +static const enum btrfs_flush_state priority_flush_states[] = { + FLUSH_DELAYED_ITEMS_NR, + FLUSH_DELAYED_ITEMS, + ALLOC_CHUNK, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) { u64 to_reclaim; - int flush_state = FLUSH_DELAYED_ITEMS_NR; + int flush_state = 0; spin_lock(&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info, @@ -5127,7 +5133,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, + priority_flush_states[flush_state]); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) { @@ -5135,15 +5142,7 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, return; } spin_unlock(&space_info->lock); - - /* - * Priority flushers can't wait on delalloc without - * deadlocking. - */ - if (flush_state == FLUSH_DELALLOC || - flush_state == FLUSH_DELALLOC_WAIT) - flush_state = ALLOC_CHUNK; - } while (flush_state < COMMIT_TRANS); + } while (flush_state < ARRAY_SIZE(priority_flush_states)); } static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Fri Oct 12 19:32:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639221 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9DEE157A for ; Fri, 12 Oct 2018 19:34:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABAAB2C546 for ; Fri, 12 Oct 2018 19:34:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9FE232C569; Fri, 12 Oct 2018 19:34:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1DF042C546 for ; Fri, 12 Oct 2018 19:34:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727173AbeJMDIc (ORCPT ); Fri, 12 Oct 2018 23:08:32 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:43257 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDIc (ORCPT ); Fri, 12 Oct 2018 23:08:32 -0400 Received: by mail-qt1-f193.google.com with SMTP id q41-v6so15046050qtq.10 for ; Fri, 12 Oct 2018 12:34:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=376B5Ewk9SX6zKU3F++7M86mkOiT1w+Wgn5fDYhS55g=; b=SexBi4y6aJiZZOmDM9kDlDU7WqHdFJNdOgWRBuIcdFMHYQcr5qvn7h+5rqYghVy2rj q+0ysKs1bsG0WfHISMBIs1Yh3Q/SIoR4AetDzEsg5Us3+U5dJLZy994tmsQgNl5fN5XS KKClGrtqBI4rDBDUrl267sMwXJ4iG6Qkc/kGHjNjT73Mz+9STusUUdHIM25pxhurTx4q UtAQK2AG67JPiSmvLhVlbvTrX9UKp+grb637aYr7KlccKcOHelotRTtku1nii1kELeV5 Xne5xOF3N4h1XaV3nfkPTspwRFouYDCVSAreQmrjyBRVW21VRbv8WLsnQcF2KqAhm8D/ nj7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=376B5Ewk9SX6zKU3F++7M86mkOiT1w+Wgn5fDYhS55g=; b=CyEVVvszTGK130o8m7droqW04jqxKgcPkPjleV1vGFwktKSknlJgFALQ74qeVp0duu QQYbJPpOTnZIF5lae/kP0RH9JdeT5nMefZ2TUA0LYc+TSH9n4DmsH6vUk1A62yFHJ0QS 8NLWysDvWbcX8YgqnCW0h5J2J5TLK+nHhFFcEoZtUL/203LdjEnrYz3bX/POdGHw+N6t /SoTsjiGGT4CRl0FOo8MUDjiJ0+nxIXU2QEGdZHJngMqzUdHJxHBzAzhXaIRPLc8GFnm Bo2TrRWdOQXRDtdPDyyYTmB+b01nFKWOseixwyAIoBZxWwx2WLtmGeOzjnBRJsfAUPPd VIAQ== X-Gm-Message-State: ABuFfogGkH2RrLrYr01xDO9X2y5hYMcx5ItC1GCLYtewU29ksaibSUQM TfGDG8hjjssO5OvDr1TerNOZA0c4XXs= X-Google-Smtp-Source: ACcGV63PpgvLaN00tundypUNXk6GDKTbUv1yYjUgRoE4Ii7IGff12dW3aVo01Bngo0TH80/ZNgwRDA== X-Received: by 2002:ac8:145:: with SMTP id f5-v6mr6958476qtg.221.1539372870015; Fri, 12 Oct 2018 12:34:30 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a17-v6sm1576227qta.0.2018.10.12.12.34.28 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:29 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 39/42] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue Date: Fri, 12 Oct 2018 15:32:53 -0400 Message-Id: <20181012193256.13735-40-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The throttle path doesn't take cleaner_delayed_iput_mutex, which means we could think we're done flushing iputs in the data space reservation path when we could have a throttler doing an iput. There's no real reason to serialize the delayed iput flushing, so instead of taking the cleaner_delayed_iput_mutex whenever we flush the delayed iputs just replace it with an atomic counter and a waitqueue. This removes the short (or long depending on how big the inode is) window where we think there are no more pending iputs when there really are some. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 4 +++- fs/btrfs/disk-io.c | 5 ++--- fs/btrfs/extent-tree.c | 9 +++++---- fs/btrfs/inode.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index e40356ca0295..1ef0b1649cad 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -894,7 +894,8 @@ struct btrfs_fs_info { spinlock_t delayed_iput_lock; struct list_head delayed_iputs; - struct mutex cleaner_delayed_iput_mutex; + atomic_t nr_delayed_iputs; + wait_queue_head_t delayed_iputs_wait; /* this protects tree_mod_seq_list */ spinlock_t tree_mod_seq_lock; @@ -3212,6 +3213,7 @@ int btrfs_orphan_cleanup(struct btrfs_root *root); int btrfs_cont_expand(struct inode *inode, loff_t oldsize, loff_t size); void btrfs_add_delayed_iput(struct inode *inode); void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info); +int btrfs_wait_on_delayed_iputs(struct btrfs_fs_info *fs_info); int btrfs_prealloc_file_range(struct inode *inode, int mode, u64 start, u64 num_bytes, u64 min_size, loff_t actual_len, u64 *alloc_hint); diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 51b2a5bf25e5..3dce9ff72e41 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1692,9 +1692,7 @@ static int cleaner_kthread(void *arg) goto sleep; } - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); btrfs_run_delayed_iputs(fs_info); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); again = btrfs_clean_one_deleted_snapshot(root); mutex_unlock(&fs_info->cleaner_mutex); @@ -2677,7 +2675,6 @@ int open_ctree(struct super_block *sb, mutex_init(&fs_info->delete_unused_bgs_mutex); mutex_init(&fs_info->reloc_mutex); mutex_init(&fs_info->delalloc_root_mutex); - mutex_init(&fs_info->cleaner_delayed_iput_mutex); seqlock_init(&fs_info->profiles_lock); INIT_LIST_HEAD(&fs_info->dirty_cowonly_roots); @@ -2699,6 +2696,7 @@ int open_ctree(struct super_block *sb, atomic_set(&fs_info->defrag_running, 0); atomic_set(&fs_info->qgroup_op_seq, 0); atomic_set(&fs_info->reada_works_cnt, 0); + atomic_set(&fs_info->nr_delayed_iputs, 0); atomic64_set(&fs_info->tree_mod_seq, 0); fs_info->sb = sb; fs_info->max_inline = BTRFS_DEFAULT_MAX_INLINE; @@ -2776,6 +2774,7 @@ int open_ctree(struct super_block *sb, init_waitqueue_head(&fs_info->transaction_wait); init_waitqueue_head(&fs_info->transaction_blocked_wait); init_waitqueue_head(&fs_info->async_submit_wait); + init_waitqueue_head(&fs_info->delayed_iputs_wait); INIT_LIST_HEAD(&fs_info->pinned_chunks); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 2056d5d37dfe..d0a7b0589826 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4258,8 +4258,9 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) * operations. Wait for it to finish so that * more space is released. */ - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + ret = btrfs_wait_on_delayed_iputs(fs_info); + if (ret) + return ret; goto again; } else { btrfs_end_transaction(trans); @@ -4829,9 +4830,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * pinned space, so make sure we run the iputs before we do our pinned * bytes check below. */ - mutex_lock(&fs_info->cleaner_delayed_iput_mutex); btrfs_run_delayed_iputs(fs_info); - mutex_unlock(&fs_info->cleaner_delayed_iput_mutex); + wait_event(fs_info->delayed_iputs_wait, + atomic_read(&fs_info->nr_delayed_iputs) == 0); trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 0a1671fb03bf..ab8242b10601 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -3319,6 +3319,7 @@ void btrfs_add_delayed_iput(struct inode *inode) if (atomic_add_unless(&inode->i_count, -1, 1)) return; + atomic_inc(&fs_info->nr_delayed_iputs); spin_lock(&fs_info->delayed_iput_lock); ASSERT(list_empty(&binode->delayed_iput)); list_add_tail(&binode->delayed_iput, &fs_info->delayed_iputs); @@ -3338,11 +3339,31 @@ void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info) list_del_init(&inode->delayed_iput); spin_unlock(&fs_info->delayed_iput_lock); iput(&inode->vfs_inode); + if (atomic_dec_and_test(&fs_info->nr_delayed_iputs)) + wake_up(&fs_info->delayed_iputs_wait); spin_lock(&fs_info->delayed_iput_lock); } spin_unlock(&fs_info->delayed_iput_lock); } +/** + * btrfs_wait_on_delayed_iputs - wait on the delayed iputs to be done running + * @fs_info - the fs_info for this fs + * @return - EINTR if we were killed, 0 if nothing's pending + * + * This will wait on any delayed iputs that are currently running with KILLABLE + * set. Once they are all done running we will return, unless we are killed in + * which case we return EINTR. + */ +int btrfs_wait_on_delayed_iputs(struct btrfs_fs_info *fs_info) +{ + int ret = wait_event_killable(fs_info->delayed_iputs_wait, + atomic_read(&fs_info->nr_delayed_iputs) == 0); + if (ret) + return -EINTR; + return 0; +} + /* * This creates an orphan entry for the given inode in case something goes wrong * in the middle of an unlink. From patchwork Fri Oct 12 19:32:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639223 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D201617E3 for ; Fri, 12 Oct 2018 19:34:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C39732C55C for ; Fri, 12 Oct 2018 19:34:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B83CF2C574; Fri, 12 Oct 2018 19:34:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 560EB2C55C for ; Fri, 12 Oct 2018 19:34:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727182AbeJMDIe (ORCPT ); Fri, 12 Oct 2018 23:08:34 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:39917 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDIe (ORCPT ); Fri, 12 Oct 2018 23:08:34 -0400 Received: by mail-qt1-f194.google.com with SMTP id e22-v6so15075127qto.6 for ; Fri, 12 Oct 2018 12:34:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=0pFXVY2NKlGxnx5addw7+EqJLIsrQdT/gx6gstwLZiU=; b=xWDwe50uOk2SNqQssrArVUqWrahx7FZjCeFOktq26Fb1Yh7UH6mPGAU3e5U+4igK0z wucjgxJ3z6ELN7MoiYvz1B3/qnA5D9xgOlfkIyMGxDy+qk87FJRRQZdJO+5Wequikslr n5hFVuKCMapWMELEXWLaQkjlb2ZT/imWVXDRLKkPWpzWoOEbs9G8t4o187STPpBNeKQ5 m/td5aCuIvvInbFtO37TzLroMbtvNYBeLdHqRDtluJcRG7tKNjHiDzCf0zgrmrKEH3PK rDcpvy03Sm+8poMUrY83TR6vO/l7XuAGK4bnXNp0dU/UaGWOhyrdJT5SqIIYq+eBpgzk NWwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=0pFXVY2NKlGxnx5addw7+EqJLIsrQdT/gx6gstwLZiU=; b=F6CeatpT4h4CTpxvdoNELu2nuKejBLriV9xZU5aV+8f2jNQ2nVeQDX8SvvHb9sHCVk NdJ2UFSD+azcjATQ5t4QOE7RhKoTAD8Z/e1BxL1qlIV9RkfULKMBQO3rtRIp2FzlllDz Cxu2sk0Awv2xgajmDlLvn4RYW+u988YcUwf7TrWmjoqkZoGo9XpzmLlYxLdv2Thrip+K Mb9TdZi8/GitoI1OfmBKSO28FNeXFbP9P49rMpiZEHP4/VUlnVSM/eT9Caurlj/XgVNb lojs1DyDQS/yeNZ2fdiMwEKx+G6LT0bTWF1pgC/s6oDCMZhKM3KfR+6Lc2kZb5G/WAUa tYcQ== X-Gm-Message-State: ABuFfointYKbdlHTboEVD9p5YHtEPhIBLeBYAjSqtsynZvJ79QSVMrSO qUgY3anXCtfDEa16SIeYy6O1zw1I8HM= X-Google-Smtp-Source: ACcGV60R1NRDAYomDYzO5BHthY7CzZKyFMxduwV+rPIjd3f1dXxX0hMRniXGEShqkQmedA+wa+21Rg== X-Received: by 2002:a0c:af49:: with SMTP id j9mr7381392qvc.72.1539372872647; Fri, 12 Oct 2018 12:34:32 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c8-v6sm1023590qkb.81.2018.10.12.12.34.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:31 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 40/42] btrfs: drop min_size from evict_refill_and_join Date: Fri, 12 Oct 2018 15:32:54 -0400 Message-Id: <20181012193256.13735-41-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't need it, rsv->size is set once and never changes throughout its lifetime, so just use that for the reserve size. Reviewed-by: David Sterba Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index ab8242b10601..dbcca915e681 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5339,8 +5339,7 @@ static void evict_inode_truncate_pages(struct inode *inode) } static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, - struct btrfs_block_rsv *rsv, - u64 min_size) + struct btrfs_block_rsv *rsv) { struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; @@ -5350,7 +5349,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, struct btrfs_trans_handle *trans; int ret; - ret = btrfs_block_rsv_refill(root, rsv, min_size, + ret = btrfs_block_rsv_refill(root, rsv, rsv->size, BTRFS_RESERVE_FLUSH_LIMIT); if (ret && ++failures > 2) { @@ -5368,7 +5367,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, * it. */ if (!btrfs_check_space_for_delayed_refs(fs_info) && - !btrfs_block_rsv_migrate(global_rsv, rsv, min_size, 0)) + !btrfs_block_rsv_migrate(global_rsv, rsv, rsv->size, 0)) return trans; /* If not, commit and try again. */ @@ -5384,7 +5383,6 @@ void btrfs_evict_inode(struct inode *inode) struct btrfs_trans_handle *trans; struct btrfs_root *root = BTRFS_I(inode)->root; struct btrfs_block_rsv *rsv; - u64 min_size; int ret; trace_btrfs_inode_evict(inode); @@ -5394,8 +5392,6 @@ void btrfs_evict_inode(struct inode *inode) return; } - min_size = btrfs_calc_trunc_metadata_size(fs_info, 1); - evict_inode_truncate_pages(inode); if (inode->i_nlink && @@ -5428,13 +5424,13 @@ void btrfs_evict_inode(struct inode *inode) rsv = btrfs_alloc_block_rsv(fs_info, BTRFS_BLOCK_RSV_TEMP); if (!rsv) goto no_delete; - rsv->size = min_size; + rsv->size = btrfs_calc_trunc_metadata_size(fs_info, 1); rsv->failfast = 1; btrfs_i_size_write(BTRFS_I(inode), 0); while (1) { - trans = evict_refill_and_join(root, rsv, min_size); + trans = evict_refill_and_join(root, rsv); if (IS_ERR(trans)) goto free_rsv; @@ -5459,7 +5455,7 @@ void btrfs_evict_inode(struct inode *inode) * If it turns out that we are dropping too many of these, we might want * to add a mechanism for retrying these after a commit. */ - trans = evict_refill_and_join(root, rsv, min_size); + trans = evict_refill_and_join(root, rsv); if (!IS_ERR(trans)) { trans->block_rsv = rsv; btrfs_orphan_del(trans, BTRFS_I(inode)); From patchwork Fri Oct 12 19:32:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639225 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24F4B17E3 for ; Fri, 12 Oct 2018 19:34:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17AFC2C55C for ; Fri, 12 Oct 2018 19:34:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0C40E2C574; Fri, 12 Oct 2018 19:34:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A181E2C55C for ; Fri, 12 Oct 2018 19:34:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727193AbeJMDIg (ORCPT ); Fri, 12 Oct 2018 23:08:36 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:40393 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDIg (ORCPT ); Fri, 12 Oct 2018 23:08:36 -0400 Received: by mail-qk1-f196.google.com with SMTP id a13-v6so8346440qkc.7 for ; Fri, 12 Oct 2018 12:34:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=am1ak1Lg8slURG/v1F1bdWqQUehd7FnCdJ9TMX1sZm0=; b=gx2Y6kcsHJ+pVpuknpzs13ZOWqkQbNRz+KAaoZl8n3q+UzN/rHyRhKtq0IO7OjBne6 V7q3syiIcAa9tgVMjstbIg0FCNQtBKviOLFYQEzEQGNvIvs16izxvgJnomNe4ojOrjlN CUALc8UpoY1IErNEeXzNWZAkhq3n3Wt91utwkY9ymsfjx6/SdDOvsYtXnBDg2m3rCMTw qT7cZhUBpIQd6uYPeiG/4cjqQm/co3CD4ObOjLYimwC8h6Xf6DSZWgD75sQWQKDuWTll P3sRT+kVflYhroI0NBZAvBHfn4YZpw9/OL7acSWKHxpsHS72vSeg3lgzQ9yQx1NUO66M B+lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=am1ak1Lg8slURG/v1F1bdWqQUehd7FnCdJ9TMX1sZm0=; b=q/3uC0Z7E3NDDPPkTKJ6AqQRsc4EKdrDSmYk8daHRHTnt/R93JEacCxs5zaBylYk7w x9GZd8XIYXSKU+3tnZiTdNSuB/9pLMVK0bvJ5f7VOFBwmw54HTeFJ7V5qJbBmEXOm3jM mm8beMJH2imvfIKToKZ5bWkLrRsfVyC3ZltDHurkPqtd2qgRxBfoCA+0O5FjrCAV106U dnyMypi4jG+wO4qoZoplcoGiYHus+LY6jdDjeWeXGKh/Q/J/I3POf0ppwWFvbgbAagQN Rxwd8HiZLf/o2exNmrO1ZplnMyzRXF+ddyWPYNLAagUpnWq7h7nawIO/YFGnMeTPdNyf KFmw== X-Gm-Message-State: ABuFfoiCI/jNS43ai1xblbeuiGSdjnxiVy5/iYENELXq03SMTvoK1HPG xX1454gMKmB2u/1hBdYEp89DT1BHKtM= X-Google-Smtp-Source: ACcGV62E9oZnMwI/UFpvFRmBjIBrxiA7O5m8cWTA0JkKrnsEV1r2UhaePEPKAr9nka+DbBmDhj8Cuw== X-Received: by 2002:a37:ad11:: with SMTP id f17-v6mr6877028qkm.329.1539372874778; Fri, 12 Oct 2018 12:34:34 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id i27-v6sm1805641qti.57.2018.10.12.12.34.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:33 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 41/42] btrfs: reserve extra space during evict() Date: Fri, 12 Oct 2018 15:32:55 -0400 Message-Id: <20181012193256.13735-42-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We could generate a lot of delayed refs in evict but never have any left over space from our block rsv to make up for that fact. So reserve some extra space and give it to the transaction so it can be used to refill the delayed refs rsv every loop through the truncate path. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index dbcca915e681..9f7da5e3c741 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5343,13 +5343,15 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, { struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; + u64 delayed_refs_extra = btrfs_calc_trans_metadata_size(fs_info, 1); int failures = 0; for (;;) { struct btrfs_trans_handle *trans; int ret; - ret = btrfs_block_rsv_refill(root, rsv, rsv->size, + ret = btrfs_block_rsv_refill(root, rsv, + rsv->size + delayed_refs_extra, BTRFS_RESERVE_FLUSH_LIMIT); if (ret && ++failures > 2) { @@ -5358,9 +5360,28 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, return ERR_PTR(-ENOSPC); } + /* + * Evict can generate a large amount of delayed refs without + * having a way to add space back since we exhaust our temporary + * block rsv. We aren't allowed to do FLUSH_ALL in this case + * because we could deadlock with so many things in the flushing + * code, so we have to try and hold some extra space to + * compensate for our delayed ref generation. If we can't get + * that space then we need see if we can steal our minimum from + * the global reserve. We will be ratelimited by the amount of + * space we have for the delayed refs rsv, so we'll end up + * committing and trying again. + */ trans = btrfs_join_transaction(root); - if (IS_ERR(trans) || !ret) + if (IS_ERR(trans) || !ret) { + if (!IS_ERR(trans)) { + trans->block_rsv = &fs_info->trans_block_rsv; + trans->bytes_reserved = delayed_refs_extra; + btrfs_block_rsv_migrate(rsv, trans->block_rsv, + delayed_refs_extra, 1); + } return trans; + } /* * Try to steal from the global reserve if there is space for From patchwork Fri Oct 12 19:32:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10639227 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64756157A for ; Fri, 12 Oct 2018 19:34:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 57AB62C55C for ; Fri, 12 Oct 2018 19:34:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C3A72C574; Fri, 12 Oct 2018 19:34:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2D362C55C for ; Fri, 12 Oct 2018 19:34:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726713AbeJMDIi (ORCPT ); Fri, 12 Oct 2018 23:08:38 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:41052 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726158AbeJMDIi (ORCPT ); Fri, 12 Oct 2018 23:08:38 -0400 Received: by mail-qt1-f194.google.com with SMTP id l41-v6so15055644qtl.8 for ; Fri, 12 Oct 2018 12:34:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=sV09l7dO3avmvRHRl1FabUbd/mHTvojy+8waApybFR4=; b=O9wDp53xXiGiI3+RRjMyNCCkLOpSTIYQJZRk7Bh93aO53bdpyA7ePzQfpU+4nhgDG7 0vwMWoOMTjj2k6HBSQmES1cBEH5cJ55lBsTC4NVlc5eqQPMs6toz7lbQHQW5tgKEFzdh Ly+SnJYxaJDXxk+NAgGYyfL/trmWWTDl8xqheQ9iacg9BjFznG1KiPobCSLXbs0ZmecO mld5UHJfh91yrVUEiOSaTsneb+Ktd8aVeMO5jykoS2XnC+8lS7eJhcHuUrp9ErWrThb5 TJHcud8/rl0nvbGIJxiI4BpfCN/jKWyi17ip1qsQoav+JY+MfXuPHByNK85jCw4z05Ry 4Buw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=sV09l7dO3avmvRHRl1FabUbd/mHTvojy+8waApybFR4=; b=Agc3faOshTAis80ClufY6Xet4CxoiwSW9VAJ8rWa0ZNJoqyyAEP68gnAPRHpLRRwEx C1n00VhXigX3mkpcJPwEH6SxZ+yFqhGKROTZw2LATTXdmlSad9ahUOD+VJWvJh1d+xcv hYc7VHTT8CX7dS0C1yIzLzqLat+q3JqHKSj5MizjqKC4imyH6NRSdJ3L0HjDRo0KvX5L SZaONAaUdiPNtzndEqVAMUjq+5a9/IWepFszv60LAZAR7j8ZR8Aa05D4jKYusGdrbQN8 B4MFn3Yixm8BsUUJ+R6kaJ4mivXEs6LSLsvfjwyCBym/LN19LzQlCiS5oJxlgtv1EDHO /1hg== X-Gm-Message-State: ABuFfogG7M+IHj4dbG9E7EKjqVTHaMA8JFyGOH2UQc7+IOYZ/8xUR8mB 1To/ADVYb1i7L4a6CJqOgZ7ByWLdXKs= X-Google-Smtp-Source: ACcGV61T5EgiRiaJGKCAH17qJ+KPi4G70kaNp1UxKATWuQsk3oVKnY+VwKGiJjfszq4XSgUg8icSPA== X-Received: by 2002:aed:34e1:: with SMTP id x88-v6mr7167409qtd.156.1539372876570; Fri, 12 Oct 2018 12:34:36 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m9-v6sm1699980qkl.32.2018.10.12.12.34.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Oct 2018 12:34:35 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 42/42] btrfs: don't run delayed_iputs in commit Date: Fri, 12 Oct 2018 15:32:56 -0400 Message-Id: <20181012193256.13735-43-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20181012193256.13735-1-josef@toxicpanda.com> References: <20181012193256.13735-1-josef@toxicpanda.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This could result in a really bad case where we do something like evict evict_refill_and_join btrfs_commit_transaction btrfs_run_delayed_iputs evict evict_refill_and_join btrfs_commit_transaction ... forever We have plenty of other places where we run delayed iputs that are much safer, let those do the work. Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana Reviewed-by: David Sterba --- fs/btrfs/transaction.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 9168efaca37e..c91dc36fccae 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -2265,15 +2265,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) kmem_cache_free(btrfs_trans_handle_cachep, trans); - /* - * If fs has been frozen, we can not handle delayed iputs, otherwise - * it'll result in deadlock about SB_FREEZE_FS. - */ - if (current != fs_info->transaction_kthread && - current != fs_info->cleaner_kthread && - !test_bit(BTRFS_FS_FROZEN, &fs_info->flags)) - btrfs_run_delayed_iputs(fs_info); - return ret; scrub_continue: