From patchwork Fri Aug 18 20:09:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 9909913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BB876600CC for ; Fri, 18 Aug 2017 20:09:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE1C728AC9 for ; Fri, 18 Aug 2017 20:09:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A245128AFC; Fri, 18 Aug 2017 20:09:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2078228AC9 for ; Fri, 18 Aug 2017 20:09:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751843AbdHRUJW (ORCPT ); Fri, 18 Aug 2017 16:09:22 -0400 Received: from mail-qk0-f195.google.com ([209.85.220.195]:36251 "EHLO mail-qk0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751687AbdHRUJV (ORCPT ); Fri, 18 Aug 2017 16:09:21 -0400 Received: by mail-qk0-f195.google.com with SMTP id u139so1907817qka.3 for ; Fri, 18 Aug 2017 13:09:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id; bh=7D2zbk/MgRkBDveY02lctL5C8LobWQXHxMKx+HP0178=; b=ZFtZYqfMcbLJSukZoRHXn2jRdou8tcZHKCGxtn2zlnfqmIw5SIk7eKd4G7+3e1/tHB IKkX6EoQ4T3iVVtZLU6zLZRE+EN8Cbgeoy3jT8uzt+P2vscNsXE868IWxOD3S5j22HVU SE/ULON6bvXtgXLgOXDUhPHdMnUl0Uw0kxpUEzSrATjrIkxw0l6M0Swh5RqJrc4MbSta ifBDzlm6LlRjEpLuLGGifG178R0RRMH300b1hbBrfKLnSSGE0yk8hcBc0nR/ailUe7c+ wRxIf3ikd1RrN6JGZXFj//5TkSR+I9VUonEf+Y6buRopNL8MlKhr+jmBlrSr1NsP61zx kpvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=7D2zbk/MgRkBDveY02lctL5C8LobWQXHxMKx+HP0178=; b=XHg9eY1NdULbHE3SnJhwFIlfcGC7wP48FMXSjAKgKAez6DLycKGNdzP+YkSf9VxelG Lb1HTmjH18zdWellwRwdFrfqEiEf4cExa71VGw4GYG6to6pn4jyhSmImr6DlmMsgE7lv i7ytvaS2kVQBgNzlLrYwml6fUreAi+HiYmqkWX/2v41YWCwAvHPDY4A/bYPHSv6du+4z ySnC/IsbDWwWVzyoBdkKR7EbNJdds5I1zmAeCqdsVeQ/2hsCBWXU8pfGTdU9TMpTbEbq pwvhGI+wOFceTzlZJXjsAobZ7VJ2t1aNtuYzX2L3s9s1Ke0WbG8jRkIcxnQ+7V05ySmX inUw== X-Gm-Message-State: AHYfb5gsE3zYEx8lDDbpN82N3s+oMxPgs5MtXmK5TNXUSxwyCvIN12y4 8PofDsNmmVCmpV6vWxNPIQ== X-Received: by 10.55.25.133 with SMTP id 5mr12758919qkz.232.1503086960620; Fri, 18 Aug 2017 13:09:20 -0700 (PDT) Received: from localhost ([2606:a000:4381:1201:225:22ff:feb3:e51a]) by smtp.gmail.com with ESMTPSA id v40sm4441848qta.29.2017.08.18.13.09.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Aug 2017 13:09:20 -0700 (PDT) From: josef@toxicpanda.com X-Google-Original-From: jbacik@fb.com To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Josef Bacik Subject: [PATCH] btrfs: change how we decide to commit transactions during flushing Date: Fri, 18 Aug 2017 16:09:19 -0400 Message-Id: <1503086959-23467-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 2.7.4 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Josef Bacik Nikolay reported that generic/273 was failing currently with ENOSPC. Turns out this is because we get to the point where the outstanding reservations are greater than the pinned space on the fs. This is a mistake, previously we used the current reservation amount in may_commit_transaction, not the entire outstanding reservation amount. Fix this to find the minimum byte size needed to make progress in flushing, and pass that into may_commit_transaction. From there we can make a smarter decision on whether to commit the transaction or not. This fixes the failure in generic/273. Reported-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 40 +++++++++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index fac42b8..4509e47 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4872,8 +4872,12 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, return -ENOSPC; spin_lock(&delayed_rsv->lock); + if (delayed_rsv->size > bytes) + bytes = 0; + else + bytes -= delayed_rsv->size; if (percpu_counter_compare(&space_info->total_bytes_pinned, - bytes - delayed_rsv->size) < 0) { + bytes) < 0) { spin_unlock(&delayed_rsv->lock); return -ENOSPC; } @@ -4901,7 +4905,7 @@ struct reserve_ticket { */ static void flush_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, u64 num_bytes, - int state) + u64 min_bytes, int state) { struct btrfs_root *root = fs_info->extent_root; struct btrfs_trans_handle *trans; @@ -4944,7 +4948,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case COMMIT_TRANS: ret = may_commit_transaction(fs_info, space_info, - num_bytes, 0); + min_bytes, 0); break; default: ret = -ENOSPC; @@ -4959,17 +4963,23 @@ static void flush_space(struct btrfs_fs_info *fs_info, static inline u64 btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - bool system_chunk) + u64 *min_reclaim, bool system_chunk) { struct reserve_ticket *ticket; u64 used; u64 expected; u64 to_reclaim = 0; - list_for_each_entry(ticket, &space_info->tickets, list) + list_for_each_entry(ticket, &space_info->tickets, list) { + if (min_reclaim && *min_reclaim == 0) + *min_reclaim = ticket->bytes; to_reclaim += ticket->bytes; - list_for_each_entry(ticket, &space_info->priority_tickets, list) + } + list_for_each_entry(ticket, &space_info->priority_tickets, list) { + if (min_reclaim && *min_reclaim == 0) + *min_reclaim = ticket->bytes; to_reclaim += ticket->bytes; + } if (to_reclaim) return to_reclaim; @@ -4992,6 +5002,8 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info, to_reclaim = 0; to_reclaim = min(to_reclaim, space_info->bytes_may_use + space_info->bytes_reserved); + if (min_reclaim) + *min_reclaim = to_reclaim; return to_reclaim; } @@ -5005,7 +5017,7 @@ static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info, if ((space_info->bytes_used + space_info->bytes_reserved) >= thresh) return 0; - if (!btrfs_calc_reclaim_metadata_size(fs_info, space_info, + if (!btrfs_calc_reclaim_metadata_size(fs_info, space_info, NULL, system_chunk)) return 0; @@ -5038,13 +5050,14 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) int flush_state; int commit_cycles = 0; u64 last_tickets_id; + u64 min_reclaim = 0; fs_info = container_of(work, struct btrfs_fs_info, async_reclaim_work); space_info = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); spin_lock(&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info, - false); + &min_reclaim, false); if (!to_reclaim) { space_info->flush = 0; spin_unlock(&space_info->lock); @@ -5055,15 +5068,18 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) flush_state = FLUSH_DELAYED_ITEMS_NR; do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, min_reclaim, + flush_state); spin_lock(&space_info->lock); if (list_empty(&space_info->tickets)) { space_info->flush = 0; spin_unlock(&space_info->lock); return; } + min_reclaim = 0; to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info, + &min_reclaim, false); if (last_tickets_id == space_info->tickets_id) { flush_state++; @@ -5097,11 +5113,12 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct reserve_ticket *ticket) { u64 to_reclaim; + u64 min_reclaim = 0; int flush_state = FLUSH_DELAYED_ITEMS_NR; spin_lock(&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info, - false); + &min_reclaim, false); if (!to_reclaim) { spin_unlock(&space_info->lock); return; @@ -5109,7 +5126,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, min_reclaim, + flush_state); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) {