From patchwork Fri Jan 31 22:35:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360645 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31F0D13A4 for ; Fri, 31 Jan 2020 22:36:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 067DD20707 for ; Fri, 31 Jan 2020 22:36:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="LqyrVQGs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726494AbgAaWgU (ORCPT ); Fri, 31 Jan 2020 17:36:20 -0500 Received: from mail-qk1-f195.google.com ([209.85.222.195]:45392 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726385AbgAaWgU (ORCPT ); Fri, 31 Jan 2020 17:36:20 -0500 Received: by mail-qk1-f195.google.com with SMTP id x1so8186081qkl.12 for ; Fri, 31 Jan 2020 14:36:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Vg21yZU611ABgBg8hmJ4dMx7Bqb2oKc4eL/3Bbeki1I=; b=LqyrVQGsGxPAMjP+EwEp2aRfHKOUu1bPGngSVS4FthqgJwmJyiZHP8POqsz4ENQe7J 12EIeL8rwmSQr9alMBeUAgQqhk5EW9AXfWJqMleYGwWiHM1hWEQCJwlkAaNLnXwQ+pnS hBWxYSF9MGdQzDbDr/3NyH0kuvvJRSS550PhYmLhanKifksOwZcjK2Q/K2Yf/R5QhF+B q/SCHw4mB1bopAk31H/X2OxkByWuuoDtm7L2y7cMOllZII2ScR9lwsuuqFa0U39EvwWM 2/jytzvZO7okJWph92H34yPaa76VJSxe4sMxvVBS547zvAQ03TLtsusW+vs7IPpevXk0 qGbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Vg21yZU611ABgBg8hmJ4dMx7Bqb2oKc4eL/3Bbeki1I=; b=mG21hYnsBQfDNwWRza6qTej+URo1drPTwTgF2yKq3yD+eE9XpwX1URZvsdhzhOC9cY sdj6sKNLxsNalqS5NoWUNo6mJJksRur7xOHMqVcOiupTF+C0F3Ol7erayijhJSS2cf2A z0Qg0k68JKLYpZTd8BgofXdDXqwKx/zG4kf6lsIuh2xG+pf6AdQxlrK7AEzMYyLidPTM 4IvspG7G/Xp4weFgBq8qYAK/UP43GVpiIS6UEk9o4e7xJh3s17mADPUfOJAoENzDpoZj abn12dGOZpdouA66EIBZTjoLeiv/XBPUc8k48bMYpU9mFRe4smCpuBDHk4TH+FrVqXZu fMIg== X-Gm-Message-State: APjAAAVPmK+KY/l70cFz5ga0fvIu5xQOkMwUfe0tqC4HTdrh5p0vAcM8 wUNleyhKFRFkWrQTswq8mEYpJl8uYBj8zg== X-Google-Smtp-Source: APXvYqxnwFUmA4BKT1G/sEppybvdIgD3iYzDoJyu264gIuSN5A8ja5rK3fix0O2DrpuKx/RpXwNgdg== X-Received: by 2002:a05:620a:1269:: with SMTP id b9mr13289213qkl.312.1580510178585; Fri, 31 Jan 2020 14:36:18 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o17sm5586031qtq.93.2020.01.31.14.36.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:17 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 01/23] btrfs: change nr to u64 in btrfs_start_delalloc_roots Date: Fri, 31 Jan 2020 17:35:51 -0500 Message-Id: <20200131223613.490779-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have btrfs_wait_ordered_roots() which takes a u64 for nr, but btrfs_start_delalloc_roots() that takes an int for nr, which makes using them in conjunction, especially for something like (u64)-1, annoying and inconsistent. Fix btrfs_start_delalloc_roots() to take a u64 for nr and adjust start_delalloc_inodes() and it's callers appropriately. This means we've adjusted start_delalloc_inodes() to take a pointer of nr since we want to preserve the ability for start-delalloc_inodes() to return an error, so simply make it do the nr adjusting as necessary. Part of adjusting the callers to this means changing btrfs_writeback_inodes_sb_nr() to take a u64 for items. This may be confusing because it seems unrelated, but the caller of btrfs_writeback_inodes_sb_nr() already passes in a u64, it's just the function variable that needs to be changed. Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov --- fs/btrfs/ctree.h | 2 +- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/inode.c | 27 +++++++++++---------------- fs/btrfs/ioctl.c | 2 +- fs/btrfs/space-info.c | 2 +- 5 files changed, 15 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 8a2c1665baad..6afa0885a9bb 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2884,7 +2884,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u32 min_type); int btrfs_start_delalloc_snapshot(struct btrfs_root *root); -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr); +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr); int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end, unsigned int extra_bits, struct extent_state **cached_state); diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index f639dde2a679..6ff08eb3c35d 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -593,7 +593,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info, * flush all outstanding I/O and inode extent mappings before the * copy operation is declared as being finished */ - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) { mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return ret; diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 9320f13778ce..5c6ce78bff1d 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -9619,7 +9619,8 @@ static struct btrfs_delalloc_work *btrfs_alloc_delalloc_work(struct inode *inode * some fairly slow code that needs optimization. This walks the list * of all the inodes with pending delalloc and forces them to disk. */ -static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) +static int start_delalloc_inodes(struct btrfs_root *root, u64 *nr, + bool snapshot) { struct btrfs_inode *binode; struct inode *inode; @@ -9659,9 +9660,11 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) list_add_tail(&work->list, &works); btrfs_queue_work(root->fs_info->flush_workers, &work->work); - ret++; - if (nr != -1 && ret >= nr) - goto out; + if (*nr != U64_MAX) { + (*nr)--; + if (*nr == 0) + goto out; + } cond_resched(); spin_lock(&root->delalloc_lock); } @@ -9686,18 +9689,15 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) int btrfs_start_delalloc_snapshot(struct btrfs_root *root) { struct btrfs_fs_info *fs_info = root->fs_info; - int ret; + u64 nr = U64_MAX; if (test_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state)) return -EROFS; - ret = start_delalloc_inodes(root, -1, true); - if (ret > 0) - ret = 0; - return ret; + return start_delalloc_inodes(root, &nr, true); } -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr) { struct btrfs_root *root; struct list_head splice; @@ -9720,15 +9720,10 @@ int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) &fs_info->delalloc_roots); spin_unlock(&fs_info->delalloc_root_lock); - ret = start_delalloc_inodes(root, nr, false); + ret = start_delalloc_inodes(root, &nr, false); btrfs_put_root(root); if (ret < 0) goto out; - - if (nr != -1) { - nr -= ret; - WARN_ON(nr < 0); - } spin_lock(&fs_info->delalloc_root_lock); } spin_unlock(&fs_info->delalloc_root_lock); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index ecb6b188df15..442c89502f06 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -5510,7 +5510,7 @@ long btrfs_ioctl(struct file *file, unsigned int case BTRFS_IOC_SYNC: { int ret; - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) return ret; ret = btrfs_sync_fs(inode->i_sb, 1); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 01297c5b2666..edda1ee0455e 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -310,7 +310,7 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, } static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, int nr_items) + unsigned long nr_pages, u64 nr_items) { struct super_block *sb = fs_info->sb; From patchwork Fri Jan 31 22:35:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360647 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D89213A4 for ; Fri, 31 Jan 2020 22:36:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7BAAA20707 for ; Fri, 31 Jan 2020 22:36:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="UaVu8eZ/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726514AbgAaWgW (ORCPT ); Fri, 31 Jan 2020 17:36:22 -0500 Received: from mail-qt1-f193.google.com ([209.85.160.193]:32856 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726213AbgAaWgW (ORCPT ); Fri, 31 Jan 2020 17:36:22 -0500 Received: by mail-qt1-f193.google.com with SMTP id d5so6742910qto.0 for ; Fri, 31 Jan 2020 14:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cRM9PPePTz8q2Yy0WH5FdM9n9537Y5XGRyzvVXRZq9s=; b=UaVu8eZ/D4HgVObvZSpVavHrODBLpLKMspkKUYJuRJCJ0KbW3Pte4qH88e6P8oNdT2 LWnohSzRrTwP28LSdZ08kXSKYqgTHzm36ddjub4VoJ5Ur7GKV1icIUxkZ/fapwhDg8Ox ydzEuRFrKP4QewWVcGfKD1zfbFpuz6ABx8ioq8aQ2hSsf78Y1P4GjIQoUjab9FXDUTd0 qQhJsmLCVLSG69KfFHpEI0+YXN9yv+7/j7UOeiE3gz7qDnPrSJcKXXxMFsSjuCV39Sj7 fEORHzHc/JashtZehxBnlSxwF5tqv/jfhW1UVm8EswqoacEQEnauViPZpgtTop/HK6TL D6dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cRM9PPePTz8q2Yy0WH5FdM9n9537Y5XGRyzvVXRZq9s=; b=e7jOo6agEZSgW/PcHhiRkO7YPEFQehuVdKwWk35jBO6Qobv/betOiAPED2you9YcgB 4j2QDxs/spdb4PEBSwwIp2dQX60cflcqQuMsRtiCGW3GL89OJaBjl6ankjLTwORXp5CK /2BatEDLSRVMq+X8Hjuq3dafg9/UN2yBBvLkxMx/VbeUu1iVWGNAiKfPdI/z1EagL9+/ 0nP45/4OuxLQ/ffe/r+6BwFHhcNixRlC3DKB/kTR8SQR0ljXbkk4OMKwKp9SAWQlOzVZ OoIX45NQsX8JjHM1d8Wo+hgn3k4lOQf8OkFUTP5fy8kEmJ1j8L0dpzlmwWq2kBxSnrxQ 2/Hg== X-Gm-Message-State: APjAAAU2qB1oeK6jqpa+s1Bukg8mSM3dXgaqdMyfdCfQnfKhWsyWM2TU LMjglViIzIXUiTiAL9TPP3qbIIYg0o0Mtw== X-Google-Smtp-Source: APXvYqygoTO3R+4WAQjPOCi710FgSshd8BnaW3j10DaWyAZ45aRKvT6yvEecVkqtX9gx/7ClqARw9A== X-Received: by 2002:ac8:969:: with SMTP id z38mr12610591qth.203.1580510180254; Fri, 31 Jan 2020 14:36:20 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id i16sm5259045qkh.120.2020.01.31.14.36.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:19 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 02/23] btrfs: remove orig from shrink_delalloc Date: Fri, 31 Jan 2020 17:35:52 -0500 Message-Id: <20200131223613.490779-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We don't use this anywhere inside of shrink_delalloc, remove it. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn --- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index edda1ee0455e..671c3a379224 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -350,7 +350,7 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, * shrink metadata reservation for delalloc */ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - u64 orig, bool wait_ordered) + bool wait_ordered) { struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; @@ -569,7 +569,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, num_bytes, + shrink_delalloc(fs_info, num_bytes * 2, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Fri Jan 31 22:35:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360649 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A03C14B4 for ; Fri, 31 Jan 2020 22:36:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EBA2120CC7 for ; Fri, 31 Jan 2020 22:36:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="tW6Et8xF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726530AbgAaWgY (ORCPT ); Fri, 31 Jan 2020 17:36:24 -0500 Received: from mail-qt1-f194.google.com ([209.85.160.194]:44906 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726385AbgAaWgX (ORCPT ); Fri, 31 Jan 2020 17:36:23 -0500 Received: by mail-qt1-f194.google.com with SMTP id w8so6679440qts.11 for ; Fri, 31 Jan 2020 14:36:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OmcZmrW0/o/7qkPrbKGRRluJBaRYm1tr9cIsIRSlx1o=; b=tW6Et8xFEqsxst/b0QVYfuARZG0G369f0nlBlU3rU2NDDqHY08uqM6CIUzCzmNcPB4 ygH2aYVI+wbLR+GAdln3bM8yXtewmmpepBx01i/nAvOtXTFR49QoCoZuWf992/fB0ECY Z9Si7whLhZk8O65mqtpHBcXUwlnvq1/9my4Lnk8SJKCbEJPiOvtDaBPERcB/e/rhJkXb WrSPFYPpEB48nBI352SRzL1ZVpqXZIpQzjWF2s7pJBYIhYNie8/H24ddxVSZAkqRsqmX txwkDiuw+e5Z3cVU8eTeKcR7AcCqMFIkGMfLTFNQo6rkjTv6pA+3oLC0Q2Z+GhvmiR1S hVTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OmcZmrW0/o/7qkPrbKGRRluJBaRYm1tr9cIsIRSlx1o=; b=J4JTWNcR7FK/MtYrrv+Iksc54tuug3gL+yzjO6ohjek+vusr/mTRBn3VbA9O98/uwv oLe2X3pvAi1yU+LJKMLSmE0dZ4QYqw06UZG230VJnBwLj3RypjS/Vpzf72GWYEIv4wWZ VIA8iYrls8VuZ77WY6AAWfvsvS5IRTJwkmoHDs2+Ol5vLe9ow73tcTYzRMcKgoXeTUyS vp5p2/a4SMg4uiCKyi5GpFBs1lt7STkkn+Dw8wIfOVAVBRsLmCPgt5jcu7tQrOC7YH9h jel63mzb6rLbBvN5CM3H2YjCUJGbYryOLb4Jbby/Rpfca+PKMGSzZFf9K3tHxjSwAegQ /L8Q== X-Gm-Message-State: APjAAAVo5lpKQ3gQYfDe6iAFfVOSNC7wjTSsQRYqwN4H6BpzTX+8SOZh JnUsgeQetdIPv7yXH0+PBV2WPODNoNuTGA== X-Google-Smtp-Source: APXvYqyeXd8Y547vW1FA3KyCea1U7ThhreLwfEpV+M460jem2DmSJAQ4tDBYKb454yMXVEWPTfGRsg== X-Received: by 2002:ac8:3853:: with SMTP id r19mr13151040qtb.69.1580510181865; Fri, 31 Jan 2020 14:36:21 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s42sm5718033qtk.87.2020.01.31.14.36.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:21 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 03/23] btrfs: handle U64_MAX for shrink_delalloc Date: Fri, 31 Jan 2020 17:35:53 -0500 Message-Id: <20200131223613.490779-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data allocations are going to want to pass in U64_MAX for flushing space, adjust shrink_delalloc to handle this properly. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn Reviewed-by: Johannes Thumshirn --- fs/btrfs/space-info.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 671c3a379224..924cee245e4a 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -363,8 +363,19 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, int loops; /* Calc the number of the pages we need flush for space reservation */ - items = calc_reclaim_items_nr(fs_info, to_reclaim); - to_reclaim = items * EXTENT_SIZE_PER_ITEM; + if (to_reclaim == U64_MAX) { + items = U64_MAX; + } else { + /* + * to_reclaim is set to however much metadata we need to + * reclaim, but reclaiming that much data doesn't really track + * exactly, so increase the amount to reclaim by 2x in order to + * make sure we're flushing enough delalloc to hopefully reclaim + * some metadata reservations. + */ + items = calc_reclaim_items_nr(fs_info, to_reclaim) * 2; + to_reclaim = items * EXTENT_SIZE_PER_ITEM; + } trans = (struct btrfs_trans_handle *)current->journal_info; space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); @@ -569,7 +580,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, + shrink_delalloc(fs_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Fri Jan 31 22:35:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D9F4414B4 for ; Fri, 31 Jan 2020 22:36:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B7AFB20707 for ; Fri, 31 Jan 2020 22:36:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="DZcWOy/5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726598AbgAaWg0 (ORCPT ); Fri, 31 Jan 2020 17:36:26 -0500 Received: from mail-qk1-f193.google.com ([209.85.222.193]:36019 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726385AbgAaWgZ (ORCPT ); Fri, 31 Jan 2020 17:36:25 -0500 Received: by mail-qk1-f193.google.com with SMTP id w25so8247835qki.3 for ; Fri, 31 Jan 2020 14:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oPYxQWUatMHclwU0kDL2t7eqZfxhtfFwQr2wE2TCs3Q=; b=DZcWOy/5mIaQ7MAun3PqoOnliM6uppmfy+IM+afr+Vr/cdX1+QQeJvWAMpvcUNZQ35 sbRgM/kL1bATvl/v4LqIW+/sch92wBf+fjueNSlFewdU1i3CAQxXCK35vPt0kDHb2Gy3 ALLqJfiKuvJ3BGBQBOL7O3YVQAsyWtSUDoqIjnSxpWvI1yHPbJ3lpQVKVwYuzngQenlP oPziWvQ+HXrU4WVxe+SxZoUXedrfJRIcRF2JAyglhx3MHh539tSsApID+ZCvqQkSBnM5 g5iUMaZizw79tR2bWm4x9XAnNO3CLRONJdXcX6q2DcMVTEDGQIhsV7hdX93aJCmHaiJa /6LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oPYxQWUatMHclwU0kDL2t7eqZfxhtfFwQr2wE2TCs3Q=; b=RYawt2BaK70K00PTCQe2F9AyNNcG4UPQ5+IFzmKKv+Soq2XooD1gH6LI9nJ3Nnn0Ex d7v0EuNOP449tGrqVxkA7bf/dg6OrtgrWIJJVyEbll6O67RAiVuO++QTPnu9uRAqg/hf 2RntUiW37sJ1b7DFYyblSibJ+FI1rh939DSHYr6eAzTATW2kmadWIdznMM4VVQCsQb7Y 6xT+ZqNdIQXUkkQNElbVfxh8fbaDoN4aQwzPYs3eIRTjlKARWmEfREFV5yyCiochcD1+ fP664FArq6ZqzN+kqRc4PQYH2DJrXrUEpLFDd9bSvkMw3jwrMy6saIoFxEptfHNoGkow xRSQ== X-Gm-Message-State: APjAAAVU7BwQEeKEKD416kamLbKzeqFns3NcGMBMahmgty8FeUwsVWsA O8/QE8dC98WcdiemW0eUJ6BUQk4Ljqr8HQ== X-Google-Smtp-Source: APXvYqxsOmGl6ipuyb1SXFYM5CM2Y8dbZ6UgOrwcVRjhlwvUz2RUheBGz+DAj6Qct/9Iaqo1yNN21A== X-Received: by 2002:a37:7685:: with SMTP id r127mr13352484qkc.166.1580510183459; Fri, 31 Jan 2020 14:36:23 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id z34sm5712947qtd.42.2020.01.31.14.36.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:22 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 04/23] btrfs: make shrink_delalloc take space_info as an arg Date: Fri, 31 Jan 2020 17:35:54 -0500 Message-Id: <20200131223613.490779-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently shrink_delalloc just looks up the metadata space info, but this won't work if we're trying to reclaim space for data chunks. We get the right space_info we want passed into flush_space, so simply pass that along to shrink_delalloc. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn --- fs/btrfs/space-info.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 924cee245e4a..17e2b5a53cb5 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -349,10 +349,10 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, /* * shrink metadata reservation for delalloc */ -static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - bool wait_ordered) +static void shrink_delalloc(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + u64 to_reclaim, bool wait_ordered) { - struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; @@ -378,7 +378,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } trans = (struct btrfs_trans_handle *)current->journal_info; - space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); @@ -580,7 +579,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes, + shrink_delalloc(fs_info, space_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Fri Jan 31 22:35:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360653 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24DCE13A4 for ; Fri, 31 Jan 2020 22:36:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 02FEE20707 for ; Fri, 31 Jan 2020 22:36:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="z/sjmPf7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726637AbgAaWg2 (ORCPT ); Fri, 31 Jan 2020 17:36:28 -0500 Received: from mail-qk1-f195.google.com ([209.85.222.195]:41612 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726385AbgAaWg1 (ORCPT ); Fri, 31 Jan 2020 17:36:27 -0500 Received: by mail-qk1-f195.google.com with SMTP id u19so553233qku.8 for ; Fri, 31 Jan 2020 14:36:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kXAG1/yIQ4AmblYfBjycP6z9OteijNC9KxrI0HvaUWI=; b=z/sjmPf774cxEIXqqmqGhXu3dVo94L/rWkIPR6Lqyv3FHpcjbQpXxy6izENM721kxd yyfEelEMIpiQkxvF3GekNiHa/hmPjhI9hCHMysNLYBszN267BBKK7NW347Qd3IocBn9D ngCem6lAWjA2cMQ4WjM9N0W/htuTWcKRib6moyyk1D6QtQYhUfpc/XkBM2+CL7mMkHXo Z2uBCX2ziibCxjAxvESALmuX1aSYZ8SVvBOAhUjdWms0kxzrmND3StjvcEh7bQppSlz/ PyvMl6vbFwfIMedk3BtS1Za8Yf+ap1YoHGn5hP0dN6Bsw2Lw8BhwhFhMm3kXGmy1yMeJ m4hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kXAG1/yIQ4AmblYfBjycP6z9OteijNC9KxrI0HvaUWI=; b=ro0bIMUY9GedbvgYtcA2b5ZBlLPTKuYWs4dfe4lW0Hw8uiVcYki/OlCnlK8sA6dpR9 qKKlbMkVdSfKLcQhZ6GDcNqajH6E/8PuXIi0JkW/KOT1C+8s67Uo4u2QuFUYwHaDi9cE M4G2Nahw8Q3jhWhgE45XVnxoPFVYSsL80nw4qqEe37OhOe/vJpX2r8svyFVA0xQ67BFP BaFYwX63YsvNdAQ65TWSkuSGv6MKU8dJUWZ78QZnBsAyIPETXF+CJWk6XCvNqYJxfGe/ odYpYIHkOcbVvWBoCO1/YEdRwecJrPTuRl++zh9hEI5cP7WktevjH+wTf8LkArVM0bXs LLqw== X-Gm-Message-State: APjAAAUdk/+Ncojh2falNkVvhXlzIhridX2bWe8DIHIRPLDOYnlPqHf6 /WYAsKmP6amil0AcuRLyKrN0iyI4YCn2kw== X-Google-Smtp-Source: APXvYqxuI8dis0waLgWtsdACt4YUVJjagJ+2vcZSCkXUUMoSzvlxOu7qPKGA4TzxYzqE7w1Cy9SdVA== X-Received: by 2002:a05:620a:62c:: with SMTP id 12mr13037985qkv.154.1580510185137; Fri, 31 Jan 2020 14:36:25 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id k17sm5098153qkj.71.2020.01.31.14.36.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:24 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 05/23] btrfs: make ALLOC_CHUNK use the space info flags Date: Fri, 31 Jan 2020 17:35:55 -0500 Message-Id: <20200131223613.490779-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have traditionally used flush_space() to flush metadata space, so we've been unconditionally using btrfs_metadata_alloc_profile() for our profile to allocate a chunk. However if we're going to use this for data we need to use btrfs_get_alloc_profile() on the space_info we pass in. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn --- fs/btrfs/space-info.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 17e2b5a53cb5..5a92851af2b3 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -604,7 +604,8 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; } ret = btrfs_chunk_alloc(trans, - btrfs_metadata_alloc_profile(fs_info), + btrfs_get_alloc_profile(fs_info, + space_info->flags), (state == ALLOC_CHUNK) ? CHUNK_ALLOC_NO_FORCE : CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); From patchwork Fri Jan 31 22:35:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360655 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47C7D13A4 for ; Fri, 31 Jan 2020 22:36:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2522C20705 for ; Fri, 31 Jan 2020 22:36:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="vQlpdNpt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726664AbgAaWg3 (ORCPT ); Fri, 31 Jan 2020 17:36:29 -0500 Received: from mail-qk1-f194.google.com ([209.85.222.194]:34167 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726636AbgAaWg2 (ORCPT ); Fri, 31 Jan 2020 17:36:28 -0500 Received: by mail-qk1-f194.google.com with SMTP id d10so8259570qke.1 for ; Fri, 31 Jan 2020 14:36:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p6I+xp02X/JD3iHfzrWtjMJkBBWxQ9LOIvKViwUqQCE=; b=vQlpdNptVqS1217P0utlHY+30aTu9pBia7qdsZAlfEAm5mz4vsqlz4oUPeoiAwAnnP 4Ex/CtBTycoDaK+IQ1z0HnETirxvoMXbFztseoE/uly5+rkw+ccfruwSgJSmRxFpK2m+ ZpBlFNGGGfZc/MhzDk9UabKE+DzkrBaqFy1bqmU4PlVJpPtP52heObhxebaqqnQlw1L8 RA55in1uJn9jwn9ncDoZmJYGFIlortmEjbVkxmeeTDrxs+x+cGo7KuQ4ZhVBzMq7pkZX 0cB+D0Aw8F+Vm0OzED6l1bIImbiW92Rso6hqq33RgoSBFfw9rKMe425o5xptUZL58FFh +30A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p6I+xp02X/JD3iHfzrWtjMJkBBWxQ9LOIvKViwUqQCE=; b=pxnQTaGMz3Vm/sGl14dY2WIr5XAu6NhkB0pFz/wqscorFAnjrr34tQaE3PZ8DNz4vB YdCRsrdrRq5wtWUlQ+O4drCoip7DFDfjEYUumZyo6k7CPiIGjTFXI9pY1Iln/jM0aNFf q7UbcCQMUBSt2ZbtWypc6DE3jcVx+pj4RhueXyz1BI2XvwbH5nUftPJU70SEVpNZPaso mHzvldivF54k78gsgEG5dZw5D2MCfwjp0dHDYGi8N/eqajDzIesO3NTwXuItKHORHPus kqtWw1kx2CPvzUQp4zZMafPhaKPboHOrVbjxuJfXeWohHC0qnx6/kFUa/gbeQCfwz3AU OCUw== X-Gm-Message-State: APjAAAVhPG78Zw2klzRq/pozxusD2Ndokz5Ot/pjEHeb1giSukTFXEAC CSPsHzBdvSH/j8cqsqwAruJPsk5xoRxq/w== X-Google-Smtp-Source: APXvYqzDCTDSpz9iuSGem+D2VS9YgUYHX3zhegLdgOlGp0aSW5pji10hO2Ufd6rUnALDFmhVgHQSXw== X-Received: by 2002:a37:e203:: with SMTP id g3mr13104164qki.479.1580510187137; Fri, 31 Jan 2020 14:36:27 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id d20sm5727874qto.2.2020.01.31.14.36.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:26 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 06/23] btrfs: call btrfs_try_granting_tickets when freeing reserved bytes Date: Fri, 31 Jan 2020 17:35:56 -0500 Message-Id: <20200131223613.490779-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We were missing a call to btrfs_try_granting_tickets in btrfs_free_reserved_bytes, so add it to handle the case where we're able to satisfy an allocation because we've freed a pending reservation. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 77ec0597bd17..616d0dd69394 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -2932,6 +2932,8 @@ void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, if (delalloc) cache->delalloc_bytes -= num_bytes; spin_unlock(&cache->lock); + + btrfs_try_granting_tickets(cache->fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Fri Jan 31 22:35:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360657 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B95CD13A4 for ; Fri, 31 Jan 2020 22:36:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 980F620CC7 for ; Fri, 31 Jan 2020 22:36:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="YPmYPC6e" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726731AbgAaWgb (ORCPT ); Fri, 31 Jan 2020 17:36:31 -0500 Received: from mail-qv1-f65.google.com ([209.85.219.65]:39101 "EHLO mail-qv1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726708AbgAaWga (ORCPT ); Fri, 31 Jan 2020 17:36:30 -0500 Received: by mail-qv1-f65.google.com with SMTP id y8so4051127qvk.6 for ; Fri, 31 Jan 2020 14:36:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P5ZEakxJOYOrx3rWdH/AtGjR1lP2wcMDdRkL7MIN6FI=; b=YPmYPC6e70U9U2uaYGqD9Y5HRDIXEfX5qi7bnSfAG/WqifbLy/F8ykwQBnqflLg8UK PuSgG9UaepdvTea8SNdNuGQB8K1HPfXxzo3L0ZF4xER/IbP3uHF57l486LvDKqzmSSOW Q868qLb6FOL7suLsVaXWFBWGm0rZ3BzBOjB6vq0PWm3FgEkgQcgecibsy6mj3quVAf/k aeeZBDE62/MTCV4dvCrCwTRXDX/glvIrxvSrPSY4Abs4uDBSIfig32n2n8W0Jk2ld6Dt pQmi3Nrovyc7GLWPdDxEz0g2l/n+Bkemp/1aDMZ5Yy++gYtBn2KvjtFZTsWbkAv7s8lJ uO7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P5ZEakxJOYOrx3rWdH/AtGjR1lP2wcMDdRkL7MIN6FI=; b=HQz6YCxZ0rTASAQQnIwDAe+pApmpOPWSp8MYteI4o/M9vlp4PdurXDa0FyGHyRonWv lLIvwMCI/p+qy8sbeZrE3/zUQnWkcYqdgBzhzFRcJ+6gmLeJK/KC1JE2GOFGt2ZMNau8 lrOxy6NWkWpGkai6hz7Q61lN9SRkbbaX/2xDuKaDKpcVsc0x8Vu7xNTQQt9+d/g/q5YD pi0EsFNFqQ+/1LS7VpvvdFvdh/GnaM5PjuL6zy4U2i5RRxpO5f1I++w+HIIzbKaB7h7z ZGMD1pCB2mUdM/RKDU2swR5U1/2Ef1uyZYjkVbtygHw42+YCBbTD4wIvn5olRIpwqg73 mHMg== X-Gm-Message-State: APjAAAVLTg3R1ANneVxvRH/Fjzp5vCxxYP0LYcyFm2Flz/2gHsgV4lHV c4PcYw+xlo/KpmpD5ArhvH7RT7upTZY5Iw== X-Google-Smtp-Source: APXvYqyLoa6EzEoPQLAyRnbVKswlYMQzHmo+L6jmFYhHpRRTfqxWopa3Q7u5uHWRta0DrcOXYkSXgQ== X-Received: by 2002:a0c:e150:: with SMTP id c16mr12686850qvl.51.1580510188995; Fri, 31 Jan 2020 14:36:28 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g53sm5808344qtk.76.2020.01.31.14.36.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:28 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 07/23] btrfs: call btrfs_try_granting_tickets when unpinning anything Date: Fri, 31 Jan 2020 17:35:57 -0500 Message-Id: <20200131223613.490779-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When unpinning we were only calling btrfs_try_granting_tickets() if global_rsv->space_info == space_info, which is problematic because we use ticketing for SYSTEM chunks, and want to use it for DATA as well. Fix this by moving this call outside of that if statement. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent-tree.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 0783341ef2e7..dfa810854850 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2885,11 +2885,10 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info, len -= to_add; } spin_unlock(&global_rsv->lock); - /* Add to any tickets we may have */ - if (len) - btrfs_try_granting_tickets(fs_info, - space_info); } + /* Add to any tickets we may have */ + if (!readonly && return_free_space && len) + btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Fri Jan 31 22:35:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360659 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4E5D159A for ; Fri, 31 Jan 2020 22:36:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C3714214D8 for ; Fri, 31 Jan 2020 22:36:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="X7Z+GDQP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726793AbgAaWge (ORCPT ); Fri, 31 Jan 2020 17:36:34 -0500 Received: from mail-qk1-f193.google.com ([209.85.222.193]:37476 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726677AbgAaWgc (ORCPT ); Fri, 31 Jan 2020 17:36:32 -0500 Received: by mail-qk1-f193.google.com with SMTP id 21so8242285qky.4 for ; Fri, 31 Jan 2020 14:36:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/CCJGAmj8zhaN6UFv+rbjnkqAKaxFdKlrvRJCkn7m3c=; b=X7Z+GDQPSbNCRumXH2mo30ChtBFTqUm4sl51iNwh/hMR+n5fPEEGEuzXUbs4frrDT2 OcWvPnEX0j/bUNK/qqtJBxuEU3BQqG5uvAcVTvm08/TzBHogyxFe/jWdojYPNtVGSDF0 mhquPbrFqzsxwGP0nJdgCCf+OE5Qb2r3uD8/mguLyVWCNCI5o49PWnvZEwtRQAn0zIKV RmFeg9AYEZLWfyE4+zMiNVDAe/GoCT98efJDEQaTv9AcPAAWpKMzbzaTtQn2vjcuOD9L RNE6orAlfQvcjPaMbfvI85RvI7hMyPBrc/iUY1g52J9mSadSKi2moeyyGMEF+Y1TMdjQ pvxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/CCJGAmj8zhaN6UFv+rbjnkqAKaxFdKlrvRJCkn7m3c=; b=N2UzadeSbLJHW/1KPAcGN6YAKrzQbZi9Q8zd/IcDuUCMhLA6u4U9Pr17uCAhM4rC0c h5fi7eNvk3rfdFxxmMfA/WVoQe4+btvn/4WVGv3VnBuKk3EXQ2gr+dn5cGqmS4zzawgu c1zR2xiRs7MyM01wRnjeDN+4zO3kD8guX4oKQuIaUSTWURuvNzjg+Op9aWibsLeo+sAy l95+Bb8Tsj+hJYfYfRz3BHntJxF+EZOyJ3cm3NzSwjNM3fNEvz8d4kCU63ji+6exoToS sOELr/EgHJaaGo9KgINV+N3h5D8qHgsjQigCvH9GAVjZ28T7f0w7Ur1ahgJpEm8azG4d 64XA== X-Gm-Message-State: APjAAAUPXdNHQPK8+SAZTmcybYsoREK1mr7i2ouUHze6NBc3uwFIb17V CD6Rufd1k1Kke+ByrZZ5QUy55pw9neirag== X-Google-Smtp-Source: APXvYqyW7DOz5Q1CqB4Wy6A5G0PEuKRiqr3Q06at1gsMW2+IqTXMhEtwdzb9J24ohuA4YwANcdeVNA== X-Received: by 2002:a37:274a:: with SMTP id n71mr12604517qkn.302.1580510190678; Fri, 31 Jan 2020 14:36:30 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id h34sm6001285qtc.62.2020.01.31.14.36.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:29 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 08/23] btrfs: call btrfs_try_granting_tickets when reserving space Date: Fri, 31 Jan 2020 17:35:58 -0500 Message-Id: <20200131223613.490779-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org If we have compression on we could free up more space than we reserved, and thus be able to make a space reservation. Add the call for this scenario. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 616d0dd69394..aca4510f1f2d 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -2899,6 +2899,13 @@ int btrfs_add_reserved_bytes(struct btrfs_block_group *cache, space_info, -ram_bytes); if (delalloc) cache->delalloc_bytes += num_bytes; + + /* + * Compression can use less space than we reserved, so wake + * tickets if that happens. + */ + if (num_bytes < ram_bytes) + btrfs_try_granting_tickets(cache->fs_info, space_info); } spin_unlock(&cache->lock); spin_unlock(&space_info->lock); From patchwork Fri Jan 31 22:35:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360661 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 914AF13A4 for ; Fri, 31 Jan 2020 22:36:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7062A20CC7 for ; Fri, 31 Jan 2020 22:36:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="BxdXSBQZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726803AbgAaWge (ORCPT ); Fri, 31 Jan 2020 17:36:34 -0500 Received: from mail-qt1-f194.google.com ([209.85.160.194]:33940 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726773AbgAaWgd (ORCPT ); Fri, 31 Jan 2020 17:36:33 -0500 Received: by mail-qt1-f194.google.com with SMTP id h12so6745957qtu.1 for ; Fri, 31 Jan 2020 14:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m89TUixA1F2eueluk4MjDZtHVKs272tFOVWQ6OnEAc8=; b=BxdXSBQZxwrwyT/ML5kGDaklv/AN9RZ68dN4QOm0CJ22+BzyhtJ9xqeE0TnfYOXgf3 aF3OKB6NEsITyewpeP9muqLR+b4nMmnEQmbMVoEB/SileHzpBfniEVyAWB5dDXh20knN CNReuGEsxovytfmgqyY4Cb7lM2sfi8UhctVCq/aKqlsIwdsSNYlBDrowu2tViXhm1iVI Y2OItosnhhSPW3liX+vYsn9+bMnBG1FHzybxAyTNOIiYWDdCzjESUPHiQ2JtE/aRzy5Y QQ5i/aPc6BKaP6y4py0Kwq0rwaHSxsK9b6dJRjAtn/+CSn8wMjG+KeJLL7403YmSCk/k MMaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m89TUixA1F2eueluk4MjDZtHVKs272tFOVWQ6OnEAc8=; b=Kngpyxt8GIL+wpr7L9GLuW89DQ2g4LTWpM0Gg6VpK02AJgSY3/ZJzJ+SbCr6VxzMqh K5PtXpFxc4g6f1pzIxfBpqPBXq/B+eRAwQYuZwrLwAVlPJBy4wiDzipfCEdKnyZ4a1KH ahumq57BEiiIPUnuuNLvRaaMtXUh7lUsEVe5/GvCR1jmqUDJHN3uuXATn/p2q4S3udKF sS30IX0V/XSxAc1nEseuBV0BIQpVVRGUxkw/7mVYXq45lBQgtCegmEYZrK6/LyVNofdF pqL/9pAYt9+copZe3nYZvlBm48HDbDtL96jHu57jqRLiQbHu7Zjc5BUU1oHencofrVvQ ltMg== X-Gm-Message-State: APjAAAXWBDujB7Lg+9xMZL6X5kRhqDW4rV6+eDWTBEmVcH3eUWCnyOPB PSBUutaNDbJTvwrAahA0KCBGDXrKO8HUIA== X-Google-Smtp-Source: APXvYqwWlu+0CnKGw5v5kycFBIxfmVeZo4wJ2Bl/jW40wZSBvLjUyIoqlIZYFbI2ju7IMs+6HA7CJQ== X-Received: by 2002:ac8:1206:: with SMTP id x6mr13312630qti.55.1580510192362; Fri, 31 Jan 2020 14:36:32 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id j1sm5176808qkl.86.2020.01.31.14.36.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:31 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 09/23] btrfs: use the btrfs_space_info_free_bytes_may_use helper for delalloc Date: Fri, 31 Jan 2020 17:35:59 -0500 Message-Id: <20200131223613.490779-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We are going to use the ticket infrastructure for data, so use the btrfs_space_info_free_bytes_may_use() helper in btrfs_free_reserved_data_space_noquota() so we get the try_granting_tickets call when we free our reservation. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik Reviewed-by: Johannes Thumshirn --- fs/btrfs/delalloc-space.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index 4cdac4d834f5..08cfef49f88b 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -179,9 +179,7 @@ void btrfs_free_reserved_data_space_noquota(struct inode *inode, u64 start, start = round_down(start, fs_info->sectorsize); data_sinfo = fs_info->data_sinfo; - spin_lock(&data_sinfo->lock); - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, -len); - spin_unlock(&data_sinfo->lock); + btrfs_space_info_free_bytes_may_use(fs_info, data_sinfo, len); } /* From patchwork Fri Jan 31 22:36:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360663 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1713E13A4 for ; Fri, 31 Jan 2020 22:36:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E9CA220CC7 for ; Fri, 31 Jan 2020 22:36:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="z2mA7ZW2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726813AbgAaWgg (ORCPT ); Fri, 31 Jan 2020 17:36:36 -0500 Received: from mail-qt1-f195.google.com ([209.85.160.195]:43461 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726773AbgAaWgf (ORCPT ); Fri, 31 Jan 2020 17:36:35 -0500 Received: by mail-qt1-f195.google.com with SMTP id d18so6682217qtj.10 for ; Fri, 31 Jan 2020 14:36:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2b5P75+IzoL0lqF+p1WcmspeZm2Y3yh4egu0l6pb7eo=; b=z2mA7ZW2iGURSgtx7Td4mGqFrE8uQ8+M8YgLs4N0sPX2YlUkPhi8wojkFySQW3eV77 FrG9LEIi2wXaCCKHCOMr5EZ3LlleNcf7rvnJcMmRfkCzdZ0pxzHg4sJvNsbn/Djv8nss Fh++5U3/3MmSzfbguKMsQvCxLYeAiCUUde0Ak0PVELC7O9zUhOk5APuEyaFWrYnRmEzS xOs6ucAwcUj5KlEX5cEGrPy75ddJRfDF9xkN29qpMUn9KDHQnUD8UBs+WiXaGq3WTQc8 3nfiIEcVwas2Zbrf3VnZal3wZHN/5p4J8Q7n+QdoB6OXPRQ23ASndpEnfWhxXYZ7KIsf iPRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2b5P75+IzoL0lqF+p1WcmspeZm2Y3yh4egu0l6pb7eo=; b=WDLwLEm4iViVpQ3Zcf1CPnFg//z9h0KA18S9EHJ6xz4D1+Y1MfbHlcJVE3LzRHP7Bn hP0mTi4XQFwCtmbELPhYESn7n2To4bhwcGC0XK8mr8086zF/22/+59hjGcmsDCX4ekx4 rdUzWf/cLpAV2VawjjccTWY3+hXjLEntlZO/iYI86CeO3mUMb5AM3jXDM4tARlqriilT h64W3UVcy28ErwfosoQmb+Fnpipp0pt0kwIYDg4izF7Cn2zzZc5/yy1jj0WvQi46ZCYL cNZgVAYWhPYW/JmSfty12Nm10QTVMXh3seNgMAAY77TFgftSt0N/IO3irqE1RpiX/wnz nR2w== X-Gm-Message-State: APjAAAXvkteu7NmOO8gvv+AlOX97e27G6l00jHfazVAVn+kQd1ntozH7 mlfkDfMALeQtAtwcogBXnTLZdw+JgMelHQ== X-Google-Smtp-Source: APXvYqxrvgNo8mdeARaoFXud2Hnpk57D0NBYmB79H45AqJZGLmwn7853EW1eyWk6OxLJpkbF+cKRVw== X-Received: by 2002:ac8:3418:: with SMTP id u24mr12938167qtb.87.1580510194003; Fri, 31 Jan 2020 14:36:34 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 16sm5307547qkm.93.2020.01.31.14.36.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:33 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 10/23] btrfs: use btrfs_start_delalloc_roots in shrink_delalloc Date: Fri, 31 Jan 2020 17:36:00 -0500 Message-Id: <20200131223613.490779-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The original iteration of flushing had us flushing delalloc and then checking to see if we could make our reservation, thus we were very careful about how many pages we would flush at once. But now that everything is async and we satisfy tickets as the space becomes available we don't have to keep track of any of this, simply try and flush the number of dirty inodes we may have in order to reclaim space to make our reservation. This cleans up our delalloc flushing significantly. The async_pages stuff is dropped because btrfs_start_delalloc_roots() handles the case that we generate async extents for us, so we no longer require this extra logic. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 55 +------------------------------------------ 1 file changed, 1 insertion(+), 54 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 5a92851af2b3..a69e3d9057ff 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -309,28 +309,6 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, up_read(&info->groups_sem); } -static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, u64 nr_items) -{ - struct super_block *sb = fs_info->sb; - - if (down_read_trylock(&sb->s_umount)) { - writeback_inodes_sb_nr(sb, nr_pages, WB_REASON_FS_FREE_SPACE); - up_read(&sb->s_umount); - } else { - /* - * We needn't worry the filesystem going from r/w to r/o though - * we don't acquire ->s_umount mutex, because the filesystem - * should guarantee the delalloc inodes list be empty after - * the filesystem is readonly(all dirty pages are written to - * the disk). - */ - btrfs_start_delalloc_roots(fs_info, nr_items); - if (!current->journal_info) - btrfs_wait_ordered_roots(fs_info, nr_items, 0, (u64)-1); - } -} - static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, u64 to_reclaim) { @@ -356,10 +334,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; - u64 async_pages; u64 items; long time_left; - unsigned long nr_pages; int loops; /* Calc the number of the pages we need flush for space reservation */ @@ -400,37 +376,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, loops = 0; while ((delalloc_bytes || dio_bytes) && loops < 3) { - nr_pages = min(delalloc_bytes, to_reclaim) >> PAGE_SHIFT; - - /* - * Triggers inode writeback for up to nr_pages. This will invoke - * ->writepages callback and trigger delalloc filling - * (btrfs_run_delalloc_range()). - */ - btrfs_writeback_inodes_sb_nr(fs_info, nr_pages, items); - - /* - * We need to wait for the compressed pages to start before - * we continue. - */ - async_pages = atomic_read(&fs_info->async_delalloc_pages); - if (!async_pages) - goto skip_async; - - /* - * Calculate how many compressed pages we want to be written - * before we continue. I.e if there are more async pages than we - * require wait_event will wait until nr_pages are written. - */ - if (async_pages <= nr_pages) - async_pages = 0; - else - async_pages -= nr_pages; + btrfs_start_delalloc_roots(fs_info, items); - wait_event(fs_info->async_submit_wait, - atomic_read(&fs_info->async_delalloc_pages) <= - (int)async_pages); -skip_async: spin_lock(&space_info->lock); if (list_empty(&space_info->tickets) && list_empty(&space_info->priority_tickets)) { From patchwork Fri Jan 31 22:36:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360665 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDA8014B4 for ; Fri, 31 Jan 2020 22:36:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9C12420CC7 for ; Fri, 31 Jan 2020 22:36:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="xwXFLlWx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726836AbgAaWgj (ORCPT ); Fri, 31 Jan 2020 17:36:39 -0500 Received: from mail-qk1-f196.google.com ([209.85.222.196]:45422 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726773AbgAaWgi (ORCPT ); Fri, 31 Jan 2020 17:36:38 -0500 Received: by mail-qk1-f196.google.com with SMTP id x1so8186731qkl.12 for ; Fri, 31 Jan 2020 14:36:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tvdnCxP2qzDN2fH1vgdT3kTHfzo9HmvwH2IHfqMpIPU=; b=xwXFLlWxc5PGbzaSCD01jS5qADpmGDnZbae92feEPmSKgMKZ6P8igWf9hLwkG5z/wX Zci9y9pnfLtT5UwBP528F94rd5+RpSckORzm1n7hYObn/duWyPcDCeUcekpXCOq0D9O4 CUCkHKM+xXYXBUA3GL6p+v10OPoyyeWfQCDX7fBFKav+kgb/kClpaHjNImoL6g6HbxTf g/jPkW5irToCDEaeKjdYVqq3CpqKR2zgcNGgzW1XkgRyckwU83ENFZ7xp3viw5Gt2LYA TPz61bka1YtY5+J00DirYhRAoYsL89kAChEb7uzDAk2z5lNYpard1YGFf6pcNPSHYoqH qDlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tvdnCxP2qzDN2fH1vgdT3kTHfzo9HmvwH2IHfqMpIPU=; b=aopLATMs88Fa0HG1Dka8VPFmvzt/kxaI5139uysOvZaDGdgIMHH9BJM8uKu95qIInY PNqgRg2Nw8IUDyVjLMv96YCB2LyJkx1HBUY9PDo7vPNVxxSbo644UyQt/QPNpaKgckyh xe/DtbCKKAjFO0hqCixwpqxmJns2WD8/UKs5HuTJqVl5uZt4rIaSCYvkmocS0NVnkNNZ MRNPncDz4DwnLOuvSy2nOkFNiKy+tfjH4igBz7Q5DZb45/i23bgxL+ljQZHeAXfR2mja eUWHrOyTWeYqdXOEiI1KR9x3xB5oZAI9jjwA0SxK1IsrM3jw1H1/RnaAUe7CvDT2A/BC MiLA== X-Gm-Message-State: APjAAAUqp17ypFkOkVRFFtFi9KvOSHdleDB59oB07zQPbUvR67P9ik5k peMfpue2prl94QjUNdn4FWAiUmEvUKk2ww== X-Google-Smtp-Source: APXvYqzFHuEjf07WmtuUR+2ss3k1Uyi6iQ+VvywgQo+nPQA0uZ3uwA+JSgY+kfudTp0AxY7Ev4DtJA== X-Received: by 2002:a37:a183:: with SMTP id k125mr13005297qke.381.1580510195638; Fri, 31 Jan 2020 14:36:35 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id n25sm5074188qkh.88.2020.01.31.14.36.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:34 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 11/23] btrfs: check tickets after waiting on ordered extents Date: Fri, 31 Jan 2020 17:36:01 -0500 Message-Id: <20200131223613.490779-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Right now if the space is free'd up after the ordered extents complete (which is likely since the reservations are held until they complete), we would do extra delalloc flushing before we'd notice that we didn't have any more tickets. Fix this by moving the tickets check after our wait_ordered_extents check. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index a69e3d9057ff..955f59f4b1d0 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -378,14 +378,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, while ((delalloc_bytes || dio_bytes) && loops < 3) { btrfs_start_delalloc_roots(fs_info, items); - spin_lock(&space_info->lock); - if (list_empty(&space_info->tickets) && - list_empty(&space_info->priority_tickets)) { - spin_unlock(&space_info->lock); - break; - } - spin_unlock(&space_info->lock); - loops++; if (wait_ordered && !trans) { btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1); @@ -394,6 +386,15 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, if (time_left) break; } + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets) && + list_empty(&space_info->priority_tickets)) { + spin_unlock(&space_info->lock); + break; + } + spin_unlock(&space_info->lock); + delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes); From patchwork Fri Jan 31 22:36:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360667 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9241814B4 for ; Fri, 31 Jan 2020 22:36:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 715B920CC7 for ; Fri, 31 Jan 2020 22:36:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="acVDBCgT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726859AbgAaWgj (ORCPT ); Fri, 31 Jan 2020 17:36:39 -0500 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39199 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726814AbgAaWgi (ORCPT ); Fri, 31 Jan 2020 17:36:38 -0500 Received: by mail-qt1-f193.google.com with SMTP id c5so6699734qtj.6 for ; Fri, 31 Jan 2020 14:36:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ccL/mgGwlka9YTOV7kQZS4bJ9ygzaqfarB8fdvzP3+E=; b=acVDBCgTDWzGnWjEPBn0k+esz6SA5w14d4JsjjTfCGW3flZUVMI3OGE+9AybS516F3 FXmzN9E46DA88RW1ELUrrnd1DaW/cqX2MvqUnvlsLu1vTLRLT8ArmAlh0EhTVwRyeT7c 8xyYguq3Ml686UyAPcjWdXqsdixw+lN+cr3tsbGqacz2Whv5AfYz6+BctqFHfUc+ZM9k j1hIDR38VykrSjyAETHLbzLsgdS7qbNhB2HJ3lsZ4jkhhEY8m1Jo3aa6IY34V6lg6fQA Oh2gQ9JbNMVqjPw8tX2AyYE7kuB08h4dX5ZhhSGENxJMtJy6YbUdoFEw+9nt+dR7dQO+ BvlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ccL/mgGwlka9YTOV7kQZS4bJ9ygzaqfarB8fdvzP3+E=; b=tsxxLZXEVB5dYpD4Oji4YIGkfunD8GqiEmpPGQr+gnBqIvfllDAU7sjHyKLjQFeIF/ Fdl2D/T3VxKU1cnxH5Sj6VI9ZtizNN/c9B/+etvPQO3IUMXjPotG/laVXDfVBdJgCwXU RhPPtcBSN8hyLcASiNTUIfVGed8INRfxvjhyAA44pWQp7BrqOPtItbJkdBlwIgRTyVRV 5SC17t3PWUuZnBJjeGCTJvSxZ7uX2lj5D6uV2M0Xnwpdi+ouO/B3WKnTHDjc1jraboWy eIrp8wgSUvRNTPbyIslC9tlmbxO6JN+0s6/V75BUiMV8FfT3VBndad7prqLVjEwBxPmQ V/CA== X-Gm-Message-State: APjAAAUawbMYZ7rQHQ+/ILzDdWanjhR5jDSzP7wBuqz0ezZsiBQRaKe9 j97YoR5z1yvs1jkVVzp2se379b2X/RJdBQ== X-Google-Smtp-Source: APXvYqzWIfDv4Wff2g/0CKA9ADma+fn17Nw+G825/wd0rwF7tAlayEHC2QJ77L1li5cUsWnqPzFJuQ== X-Received: by 2002:aed:2bc2:: with SMTP id e60mr13211550qtd.115.1580510197251; Fri, 31 Jan 2020 14:36:37 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id s20sm5224907qkj.100.2020.01.31.14.36.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:36 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 12/23] btrfs: add the data transaction commit logic into may_commit_transaction Date: Fri, 31 Jan 2020 17:36:02 -0500 Message-Id: <20200131223613.490779-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data space flushing currently unconditionally commits the transaction twice in a row, and the last time it checks if there's enough pinned extents to satisfy it's reservation before deciding to commit the transaction for the 3rd and final time. Encode this logic into may_commit_transaction(). In the next patch we will pass in U64_MAX for bytes_needed the first two times, and the final time we will pass in the actual bytes we need so the normal logic will apply. This patch exists soley to make the logical changes I will make to the flushing state machine separate to make it easier to bisect any performance related regressions. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 955f59f4b1d0..9f8ef2a09ad9 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -412,20 +412,32 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info) + struct btrfs_space_info *space_info, + u64 bytes_needed) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *trans; - u64 bytes_needed; u64 reclaim_bytes = 0; u64 cur_free_bytes = 0; + bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; + /* + * If we are data and have passed in U64_MAX we just want to + * unconditionally commit the transaction to match the previous data + * flushing behavior. + */ + if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && + bytes_needed == U64_MAX) { + do_commit = true; + goto check_pinned; + } + spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -439,7 +451,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : 0; + bytes_needed = (ticket) ? ticket->bytes : bytes_needed; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -450,6 +462,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; +check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -459,7 +472,8 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (do_commit || + test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -570,7 +584,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info); + ret = may_commit_transaction(fs_info, space_info, num_bytes); break; default: ret = -ENOSPC; From patchwork Fri Jan 31 22:36:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360669 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB88013A4 for ; Fri, 31 Jan 2020 22:36:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BA40120CC7 for ; Fri, 31 Jan 2020 22:36:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="Zn57qzmO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726881AbgAaWgl (ORCPT ); Fri, 31 Jan 2020 17:36:41 -0500 Received: from mail-qk1-f194.google.com ([209.85.222.194]:41632 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726869AbgAaWgk (ORCPT ); Fri, 31 Jan 2020 17:36:40 -0500 Received: by mail-qk1-f194.google.com with SMTP id u19so553712qku.8 for ; Fri, 31 Jan 2020 14:36:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=EHvMaa3HgJQBaukxZ/rw4Q8tIRtNIO6L/belh7SMG74=; b=Zn57qzmON1nszVjm2/meCW3nlQs16PincdPQRm/xSxkGCjxJA+U2L3Dw13u6XvxUBH G6vO12QlNCQSEXl9jCGy23cXp1EGyS983h2zk62CMWWu03V6twI3JhzOR2k0YAVwICbv k3yVGz4P0gI4Qj5etIqxFxeiSmIQe2wiYFF7raKJNMJHWeB7LbrFWACxzyr3avLhhTJU QyPL36MK/yGdZeM+9r5JVYCM3uk97YnKHb5v6U586bnvxo+ECbK91tvI2g4OrZBYoSkH L8i0pHUlyY4eQxXkUSzLcyBaxG8BkENPDzHhJqYpuiYQF2+TQqAm3BextY92RsswkdED xqWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EHvMaa3HgJQBaukxZ/rw4Q8tIRtNIO6L/belh7SMG74=; b=d93zF4iNO2NxqkLV3DFT2IHc7zo7H4llKAjMTNeABs9A3ExLxw6HqFl9aLP5y+5A7r 6dgatgTZWeuBubDBZQKtyOzxzfxsfkfn8uQGfufmHrYe8jsr0hKpBkXXiGP7e451YmhZ 0uJcycCSm7FVqgooX5vBiM8Gg82Z54mMsf/IQvV6HnalidWFDd2Qx2ka51exWHkJ1/9b +AKt077qcSiV7SjHlp0U33vQE6+aWzC6SExabiGLmZLGFbNlZiLMlE9stzYFl/YvsuIl G/S0/ZmWpHfD3Ya4yxiItpKDv07qHCz1QyA2HpFrlQEhNwjHA1ZBkuWgu0T26FrGSpgG //pA== X-Gm-Message-State: APjAAAVgKu1JgXC6GDx2/iJkRUFG8mlIqt0MrqZiIzaCmqH65gAxxyzd 6iUOG9KnKC4W/QFlIfDCaApbZMqCkqvGgQ== X-Google-Smtp-Source: APXvYqyy9FZCT8aSim75r3kPCH++vRcU2sWi0cMPkLWfyk/eGmy5OEqtMsuXBAFQT1J1UKiSSyKLyA== X-Received: by 2002:a37:a881:: with SMTP id r123mr13427460qke.275.1580510198916; Fri, 31 Jan 2020 14:36:38 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b30sm5189205qka.48.2020.01.31.14.36.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:38 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 13/23] btrfs: add flushing states for handling data reservations Date: Fri, 31 Jan 2020 17:36:03 -0500 Message-Id: <20200131223613.490779-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently the way we do data reservations is by seeing if we have enough space in our space_info. If we do not and we're a normal inode we'll 1) Attempt to force a chunk allocation until we can't anymore. 2) If that fails we'll flush delalloc, then commit the transaction, then run the delayed iputs. If we are a free space inode we're only allowed to force a chunk allocation. In order to use the normal flushing mechanism we need to encode this into a flush state array for normal inodes. Since both will start with allocating chunks until the space info is full there is no need to add this as a flush state, this will be handled specially. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/space-info.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 6afa0885a9bb..865b24a1759e 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2526,6 +2526,8 @@ enum btrfs_reserve_flush_enum { */ BTRFS_RESERVE_FLUSH_LIMIT, BTRFS_RESERVE_FLUSH_EVICT, + BTRFS_RESERVE_FLUSH_DATA, + BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE, BTRFS_RESERVE_FLUSH_ALL, }; diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 9f8ef2a09ad9..ad203717269c 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -816,6 +816,12 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + COMMIT_TRANS, + RUN_DELAYED_IPUTS, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, From patchwork Fri Jan 31 22:36:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360671 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 718A413A4 for ; Fri, 31 Jan 2020 22:36:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4627720CC7 for ; Fri, 31 Jan 2020 22:36:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="UeUg+bEr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726905AbgAaWgo (ORCPT ); Fri, 31 Jan 2020 17:36:44 -0500 Received: from mail-qt1-f179.google.com ([209.85.160.179]:34546 "EHLO mail-qt1-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726869AbgAaWgn (ORCPT ); Fri, 31 Jan 2020 17:36:43 -0500 Received: by mail-qt1-f179.google.com with SMTP id h12so6746168qtu.1 for ; Fri, 31 Jan 2020 14:36:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=w9cOHckmeyg7HG5B862CYyORuWxhrzt1vxCZ6FY+sko=; b=UeUg+bErrhu92miRK1qAv8s+G5QvVO4UYnW68YHC5bosJiSk2bmIuM6PVKMQMR+VJV ebRmiDwq5gE2C7A18KIHLgCdMlYbtsbS6/o4a4G5z1oJ+AygFk4xp5omEnbeQfzDS1mO 2q5aw4FHZysgyFd6BssRfaV2JLVRt6W6/5hITIaIW2dHcj5+fPR4OcSPiv0HmVpFce8R 1EyXnzfxqQ+x9L4bejP/bU5aTmACOOBXyqzXiOdrZ6SNs4h7V2yIqM4tucJ/jIQwpfbF oB6IQ1FU4f24UvQYYtbm1vrc4ci0ATXIZJgg+jVvr8nADZqAgi9Ju0p0vnG7pshbVox+ xvpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w9cOHckmeyg7HG5B862CYyORuWxhrzt1vxCZ6FY+sko=; b=eK0Z46jbxQKDcC54pupeLIRGMlm7vL6INlRs+UyGiNk/G5A6CS0oynt7ikJGblme5g 9WHEx9Jw3pg8pC1pFlpu+EX48tu3pbWWht3w0E/UaQvDkqTEY+ILWt8dY4UGP/WQMzIW x+QgdLpTFu+fAKRZXQwx21gn5u4oUOHMe5sAUBrr69AWDgyNim5/b8D0PdzPofqMgcTr 7ebakNM/yEXlOhUqq7iNc2kaMQk+g1Tj3OQO07v8f7ZP+X+LvwV6KqjTDGRffMsmlKpn IlhDc3WeAgP3/GiUE8czUtcN6oBDm2zbWAVIqIL9PmHcjKW9v9B2aMg1SumNdyvRjQK9 pkEw== X-Gm-Message-State: APjAAAVLBoACZlJqW295l9DdDkaLbbJ9dJMfq4YBtOtdZMeDZvXXwJOc dn/yHdBtb5lTIbPeZQ5hdw1x3/q/usDExQ== X-Google-Smtp-Source: APXvYqwbto2jEskxMKJI+FJ11SW2Iff6HNBtAwVWuskGjHBxRKZdumIcTtezgPwjm/U4svbzL5PUow== X-Received: by 2002:ac8:3847:: with SMTP id r7mr13298105qtb.381.1580510200662; Fri, 31 Jan 2020 14:36:40 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o17sm5586530qtq.93.2020.01.31.14.36.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:39 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 14/23] btrfs: add btrfs_reserve_data_bytes and use it Date: Fri, 31 Jan 2020 17:36:04 -0500 Message-Id: <20200131223613.490779-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Create a new function btrfs_reserve_data_bytes() in order to handle data reservations. This uses the new flush types and flush states to handle making data reservations. This patch specifically does not change any functionality, and is purposefully not cleaned up in order to make bisection easier for the future patches. The new helper is identical to the old helper in how it handles data reservations. We first try to force a chunk allocation, and then we run through the flush states all at once and in the same order that they were done with the old helper. Subsequent patches will clean this up and change the behavior of the flushing, and it is important to keep those changes separate so we can easily bisect down to the patch that caused the regression, rather than the patch that made us start using the new infrastructure. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/delalloc-space.c | 119 ++------------------------------------ fs/btrfs/space-info.c | 92 +++++++++++++++++++++++++++++ fs/btrfs/space-info.h | 2 + 3 files changed, 98 insertions(+), 115 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index 08cfef49f88b..c13d8609cc99 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -13,126 +13,15 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) { struct btrfs_root *root = inode->root; struct btrfs_fs_info *fs_info = root->fs_info; - struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = 0; - int need_commit = 2; - int have_pinned_space; + enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_DATA; /* Make sure bytes are sectorsize aligned */ bytes = ALIGN(bytes, fs_info->sectorsize); - if (btrfs_is_free_space_inode(inode)) { - need_commit = 0; - ASSERT(current->journal_info); - } - -again: - /* Make sure we have enough space to handle the data first */ - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - - if (used + bytes > data_sinfo->total_bytes) { - struct btrfs_trans_handle *trans; - - /* - * If we don't have enough free bytes in this space then we need - * to alloc a new chunk. - */ - if (!data_sinfo->full) { - u64 alloc_target; - - data_sinfo->force_alloc = CHUNK_ALLOC_FORCE; - spin_unlock(&data_sinfo->lock); - - alloc_target = btrfs_data_alloc_profile(fs_info); - /* - * It is ugly that we don't call nolock join - * transaction for the free space inode case here. - * But it is safe because we only do the data space - * reservation for the free space cache in the - * transaction context, the common join transaction - * just increase the counter of the current transaction - * handler, doesn't try to acquire the trans_lock of - * the fs. - */ - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - - ret = btrfs_chunk_alloc(trans, alloc_target, - CHUNK_ALLOC_NO_FORCE); - btrfs_end_transaction(trans); - if (ret < 0) { - if (ret != -ENOSPC) - return ret; - else { - have_pinned_space = 1; - goto commit_trans; - } - } - - goto again; - } - - /* - * If we don't have enough pinned space to deal with this - * allocation, and no removed chunk in current transaction, - * don't bother committing the transaction. - */ - have_pinned_space = __percpu_counter_compare( - &data_sinfo->total_bytes_pinned, - used + bytes - data_sinfo->total_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - spin_unlock(&data_sinfo->lock); - - /* Commit the current transaction and try again */ -commit_trans: - if (need_commit) { - need_commit--; - - if (need_commit > 0) { - btrfs_start_delalloc_roots(fs_info, -1); - btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, - (u64)-1); - } - - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - if (have_pinned_space >= 0 || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, - &trans->transaction->flags) || - need_commit > 0) { - ret = btrfs_commit_transaction(trans); - if (ret) - return ret; - /* - * The cleaner kthread might still be doing iput - * operations. Wait for it to finish so that - * more space is released. We don't need to - * explicitly run the delayed iputs here because - * the commit_transaction would have woken up - * the cleaner. - */ - ret = btrfs_wait_on_delayed_iputs(fs_info); - if (ret) - return ret; - goto again; - } else { - btrfs_end_transaction(trans); - } - } - - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", - data_sinfo->flags, bytes, 1); - return -ENOSPC; - } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - spin_unlock(&data_sinfo->lock); + if (btrfs_is_free_space_inode(inode)) + flush = BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE; - return 0; + return btrfs_reserve_data_bytes(fs_info, bytes, flush); } int btrfs_check_data_free_space(struct inode *inode, diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index ad203717269c..6b71f6d3a348 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1082,3 +1082,95 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, } return ret; } + +/** + * btrfs_reserve_data_bytes - try to reserve data bytes for an allocation + * @root - the root we are allocating for + * @bytes - the number of bytes we need + * @flush - how we are allowed to flush + * + * This will reserve bytes from the data space info. If there is not enough + * space then we will attempt to flush space as specified ty flush. + */ +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; + const enum btrfs_flush_state *states = NULL; + u64 used; + int states_nr = 0; + int commit_cycles = 2; + int ret = -ENOSPC; + + ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); + + if (flush == BTRFS_RESERVE_FLUSH_DATA) { + states = data_flush_states; + states_nr = ARRAY_SIZE(data_flush_states); + } + + spin_lock(&data_sinfo->lock); +again: + used = btrfs_space_info_used(data_sinfo, true); + + if (used + bytes > data_sinfo->total_bytes) { + u64 prev_total_bytes = data_sinfo->total_bytes; + int flush_state = 0; + + spin_unlock(&data_sinfo->lock); + + /* + * Everybody can force chunk allocation, so try this first to + * see if we can just bail here and make our reservation. + */ + flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); + spin_lock(&data_sinfo->lock); + if (prev_total_bytes < data_sinfo->total_bytes) + goto again; + spin_unlock(&data_sinfo->lock); + + /* + * Cycle through the rest of the flushing options for our flush + * type, then try again. + */ + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + /* + * Previously we unconditionally committed the + * transaction twice before finally checking against + * pinned space before committing the final time. We + * also skipped flushing delalloc the final pass + * through. + */ + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = bytes; + } + + flush_space(fs_info, data_sinfo, flush_bytes, + states[flush_state]); + flush_state++; + } + + if (!commit_cycles) + goto out; + + commit_cycles--; + spin_lock(&data_sinfo->lock); + goto again; + } + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; + spin_unlock(&data_sinfo->lock); +out: + if (ret) + trace_btrfs_space_reservation(fs_info, + "space_info:enospc", + data_sinfo->flags, bytes, 1); + return ret; +} diff --git a/fs/btrfs/space-info.h b/fs/btrfs/space-info.h index 24514cd2c6c1..179f757c4a6b 100644 --- a/fs/btrfs/space-info.h +++ b/fs/btrfs/space-info.h @@ -141,5 +141,7 @@ static inline void btrfs_space_info_free_bytes_may_use( btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush); #endif /* BTRFS_SPACE_INFO_H */ From patchwork Fri Jan 31 22:36:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360673 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD671159A for ; Fri, 31 Jan 2020 22:36:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B23BF20CC7 for ; Fri, 31 Jan 2020 22:36:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="oLZBlIsA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726907AbgAaWgp (ORCPT ); Fri, 31 Jan 2020 17:36:45 -0500 Received: from mail-qk1-f194.google.com ([209.85.222.194]:37494 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726885AbgAaWgn (ORCPT ); Fri, 31 Jan 2020 17:36:43 -0500 Received: by mail-qk1-f194.google.com with SMTP id 21so8242715qky.4 for ; Fri, 31 Jan 2020 14:36:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=p3zRGXxK8EZ0Bwaao4g3Hwd+cSIC6ePOzvPRwrer+MY=; b=oLZBlIsAXSah9bs4xPBNt73INlS4D+BlAxMHf1ivtNcEBKXIzY95KUwf5ND/vlHsc6 K+p87Zz2wVsYqiEecJWOZgWsAQXC2fQbNZt4zDYILlmVWTj1KeZChWcEDvQhP2qPzxxv et5dpDsWtZQ6cbWg9fs3djDRQhnAnCiyP2YQII79o6dNfy/YiCxEXv7NTXon7SeS6HK1 XzRNZelaoiQeCkQxv9saPZpfbMql4kSdx5O3W7YoBAh3x4qif0c2txx1iKg7eB7F2EgF w7keWx2WWeIyITRzloFCE+82NwpmOtOMSeonJuH3J2AeK0k7pPx1Py9SZJQtwVnZ6AGq U+qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p3zRGXxK8EZ0Bwaao4g3Hwd+cSIC6ePOzvPRwrer+MY=; b=kX2PVcXwhYqqD0Eo1UgGWVqWJRovTJH7WW6/vtb5TmMzgJg+dIT2hvyQbVykOV537K LvodPhwtrisZd0eAttdM4vDgAjhDhr7Mbd2Zx47wSduk9C2S592KgsjCPiUu38wTK30A 5Byrsy4evllJxG05A0ADCRACwRl1I2J1MXnjtvbW2Qeh9k0eHwdmXb6JqsAFVBwzcIWY kYxbf9pwkKxiwShZOUNTY87m0agQiSXuU67Pf+qaMrCgfJEwtat1Fq6w8v19mursm0Ns WlUT53b/ksBW3Nai3j02azK6psExk4BlQLjxazx2vdQ4O+Kr+bWKNxiZKn3TXj7hyJu9 f9TQ== X-Gm-Message-State: APjAAAW5oj1cxY5P7UibmS6zfKdBYrTmIbFopluaiwmrLO2vPpR8D3Fi O50NCJfGZLO4NGdqOeRLF8GgP0SAUM94Sw== X-Google-Smtp-Source: APXvYqzqE+W3dVLLXGbs0OgG8yf9n07GOsZE8pQmlXQYPToola+wyzAh74ii/R/0lbp2s5hBNoJuig== X-Received: by 2002:a37:2e82:: with SMTP id u124mr12584944qkh.294.1580510202361; Fri, 31 Jan 2020 14:36:42 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 205sm362457qkd.61.2020.01.31.14.36.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:41 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 15/23] btrfs: use ticketing for data space reservations Date: Fri, 31 Jan 2020 17:36:05 -0500 Message-Id: <20200131223613.490779-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have all the infrastructure in place, use the ticketing infrastructure to make data allocations. This still maintains the exact same flushing behavior, but now we're using tickets to get our reservations satisfied. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 128 ++++++++++++++++++++++-------------------- 1 file changed, 68 insertions(+), 60 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 6b71f6d3a348..43c5775bcbc6 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -829,7 +829,7 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, int states_nr) { u64 to_reclaim; - int flush_state; + int flush_state = 0; spin_lock(&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info); @@ -839,7 +839,6 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); - flush_state = 0; do { flush_space(fs_info, space_info, to_reclaim, states[flush_state]); flush_state++; @@ -852,6 +851,54 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, } while (flush_state < states_nr); } +static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + struct reserve_ticket *ticket, + const enum btrfs_flush_state *states, + int states_nr) +{ + int flush_state = 0; + int commit_cycles = 2; + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + } +again: + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = ticket->bytes; + } + + flush_space(fs_info, space_info, flush_bytes, + states[flush_state]); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + flush_state++; + } + if (commit_cycles) { + commit_cycles--; + flush_state = 0; + goto again; + } +} + static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) @@ -917,6 +964,15 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; + case BTRFS_RESERVE_FLUSH_DATA: + priority_reclaim_data_space(fs_info, space_info, ticket, + data_flush_states, + ARRAY_SIZE(data_flush_states)); + break; + case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: + priority_reclaim_data_space(fs_info, space_info, ticket, + NULL, 0); + break; default: ASSERT(0); break; @@ -1096,78 +1152,30 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - const enum btrfs_flush_state *states = NULL; u64 used; - int states_nr = 0; - int commit_cycles = 2; int ret = -ENOSPC; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - if (flush == BTRFS_RESERVE_FLUSH_DATA) { - states = data_flush_states; - states_nr = ARRAY_SIZE(data_flush_states); - } - spin_lock(&data_sinfo->lock); -again: used = btrfs_space_info_used(data_sinfo, true); if (used + bytes > data_sinfo->total_bytes) { - u64 prev_total_bytes = data_sinfo->total_bytes; - int flush_state = 0; + struct reserve_ticket ticket; + init_waitqueue_head(&ticket.wait); + ticket.bytes = bytes; + ticket.error = 0; + list_add_tail(&ticket.list, &data_sinfo->priority_tickets); spin_unlock(&data_sinfo->lock); - /* - * Everybody can force chunk allocation, so try this first to - * see if we can just bail here and make our reservation. - */ - flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); - spin_lock(&data_sinfo->lock); - if (prev_total_bytes < data_sinfo->total_bytes) - goto again; + ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, + flush); + } else { + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; spin_unlock(&data_sinfo->lock); - - /* - * Cycle through the rest of the flushing options for our flush - * type, then try again. - */ - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - /* - * Previously we unconditionally committed the - * transaction twice before finally checking against - * pinned space before committing the final time. We - * also skipped flushing delalloc the final pass - * through. - */ - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = bytes; - } - - flush_space(fs_info, data_sinfo, flush_bytes, - states[flush_state]); - flush_state++; - } - - if (!commit_cycles) - goto out; - - commit_cycles--; - spin_lock(&data_sinfo->lock); - goto again; } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); -out: if (ret) trace_btrfs_space_reservation(fs_info, "space_info:enospc", From patchwork Fri Jan 31 22:36:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360675 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C4C3414B4 for ; Fri, 31 Jan 2020 22:36:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A3FDD20CC7 for ; Fri, 31 Jan 2020 22:36:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="1XwkJKmI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726916AbgAaWgr (ORCPT ); Fri, 31 Jan 2020 17:36:47 -0500 Received: from mail-qk1-f194.google.com ([209.85.222.194]:43751 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726885AbgAaWgq (ORCPT ); Fri, 31 Jan 2020 17:36:46 -0500 Received: by mail-qk1-f194.google.com with SMTP id j20so8202663qka.10 for ; Fri, 31 Jan 2020 14:36:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8eSIp50bber6rSeu3XOdTcZqeJcik5LQlOAfamDNLKw=; b=1XwkJKmIHI6aR+F3DLLdsJW114jhs2Wex2dGwN4ZqDGtdbiDpZdUaFH45Eyv2Zp9aA 3nDg/qgKg3ibwO5xBi3uLieG+0l9t6+BfXLtGjBiUof8nyXpe46mFWT4EeKZf8CTDb2Y 2DIOmar9KU5d0EQ+kERcgkJlJhZjPARGu1l2n9RNpmI6BcU8A0WlGNG+uYGnN8u4+Jrf 6OhNlUOfsUh3h+u8TufK//nl3YyPNqmX9fg1bODfEdAqzoEss1/Mfvbi0r8TJX/CfgSC bGYPoY4GsLLBprIXPWRbj+MvkWrYMDrpEhG2fnthJp4xp5MqkifUKWtEUYPFygK0/GF7 iL6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8eSIp50bber6rSeu3XOdTcZqeJcik5LQlOAfamDNLKw=; b=cubs8vwQD0ouqeFgGgKo+4abp/9nMaEOrLFkRxnW3nhk1hZ9gJsDOVecsPC40Cwyww gan7uCE8kaN+5P743KXHNcDw2utov1YW0DUmc7ze1HP2/tG5y/AlxP+MiYFMU7AVX5R9 SyT0G1Gcqw7CsbwqsTxpM3nD5bhzHa8MbNqzPhNhAaC9TJPCQoE2xhSkgBZOHyQiOq6p 3WK4O1PkxcPaJt/7oP48hSUsY0k91FTQQ49CD8vt3k1PLh6DH7++lcsTAg32tdB3bf85 C3SSYSvq5NCHSWtufY1h4bs6kQKJuA1I+SH6KGmShEX3lHvn8JLZxvg3n1mQNmw5Rlse OGxg== X-Gm-Message-State: APjAAAVod9xRnXRLmRaUpeEBZStxiVMRMcFGljHtlL26MN/TAX/kPKh9 ZcwDSpjA0ZW4q+O8JZpFpIlfXrXjp3Vqyg== X-Google-Smtp-Source: APXvYqz2c7xKCknZI/N6vHB34y1377q8QXxSb/5TJww3NLA7h4XjvPi5dzyHNEs9zICDEgAPLfSYRg== X-Received: by 2002:a37:9ace:: with SMTP id c197mr13300009qke.482.1580510204277; Fri, 31 Jan 2020 14:36:44 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id k50sm5808948qtc.90.2020.01.31.14.36.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:43 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 16/23] btrfs: serialize data reservations if we are flushing Date: Fri, 31 Jan 2020 17:36:06 -0500 Message-Id: <20200131223613.490779-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Nikolay reported a problem where generic/371 would fail sometimes with a slow drive. The gist of the test is that we fallocate a file in parallel with a pwrite of a different file. These two files combined are smaller than the file system, but sometimes the pwrite would ENOSPC. A fair bit of investigation uncovered the fact that the fallocate workload was racing in and grabbing the free space that the pwrite workload was trying to free up so it could make its own reservation. After a few loops of this eventually the pwrite workload would error out with an ENOSPC. We've had the same problem with metadata as well, and we serialized all metadata allocations to satisfy this problem. This wasn't usually a problem with data because data reservations are more straightforward, but obviously could still happen. Fix this by not allowing reservations to occur if there are any pending tickets waiting to be satisfied on the space info. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 43c5775bcbc6..97379524bac8 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1154,13 +1154,17 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; u64 used; int ret = -ENOSPC; + bool pending_tickets; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); spin_lock(&data_sinfo->lock); used = btrfs_space_info_used(data_sinfo, true); + pending_tickets = !list_empty(&data_sinfo->tickets) || + !list_empty(&data_sinfo->priority_tickets); - if (used + bytes > data_sinfo->total_bytes) { + if (pending_tickets || + used + bytes > data_sinfo->total_bytes) { struct reserve_ticket ticket; init_waitqueue_head(&ticket.wait); From patchwork Fri Jan 31 22:36:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360677 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2049F13A4 for ; Fri, 31 Jan 2020 22:36:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F27F520CC7 for ; Fri, 31 Jan 2020 22:36:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="FuFPgpFG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726930AbgAaWgs (ORCPT ); Fri, 31 Jan 2020 17:36:48 -0500 Received: from mail-qt1-f196.google.com ([209.85.160.196]:33955 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726913AbgAaWgr (ORCPT ); Fri, 31 Jan 2020 17:36:47 -0500 Received: by mail-qt1-f196.google.com with SMTP id h12so6746300qtu.1 for ; Fri, 31 Jan 2020 14:36:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uIXnSRIo56eU+s2OzaeIRmu/GBcbXO8fptF5VEIM4I0=; b=FuFPgpFGLQX4ynFuyJ+fK7/Aohq3iHmZH0+5ASFRw4CqlIba3jXyYb/K+tKW19Aa60 kZPiaVXkP/W1oZMhI4UAyaslcaCCdmnT5DkAjRclryDqNEMLqdHFkoxWhGRmKWBoA/aq 49xvycL/A8RBsEJ9ouoTsITWDx+jIgOUrZe7T0n75YtPGuN7XCwMm9QcKNLxjd6Fk+07 N9RCXYtB+U8+WHg8BHA6vTi+jwVxOdguSOkRPlNafIioNvfzPlRwillHB04EVEuFW4Or ZMFIQ4dNir8xAu58y+RKYcApmPgi1h8b9GmawH1IBQ7w/t/4qMCtQ4asnucsLxQrMabv Wrrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uIXnSRIo56eU+s2OzaeIRmu/GBcbXO8fptF5VEIM4I0=; b=RkSvLvVxMuTrn3jpp5do7n69Cc/392su4rEEDuG1qzyTBVuA253xuuZyX8oiginHyL rh1k7BqRuZXmlSp24tZfXkp+ru1wVF8njvMRa+E7ZAsaDllk4dW3VPYwjnT5rPbCURFj II5rQ43S/aXX13VF3+Mh+rjlDbnm97VPiqKAUNx+cK7JfmUDATf0ijgkuSVAouPqfhd1 jOMTIR2CHGNOq4iisS2iCl7UJr487euoGPpDKX/8IaLNX8cdldDjcWlnUL7iO7nh36Ka rXy05B/W5JtMjRrBnaTemkDb7eLib0/mq+6zyzcWKuwvh2X6HqKFCqt52WasrWhEzvcS eDRA== X-Gm-Message-State: APjAAAV+7eHBnw6N4kaDUho8uU9Xri3RjzNDE+j2zQ4WCmzO5qfCcI8k qfhRuyp/EAQGSREJpOOGbhJ3Omj2MHTbug== X-Google-Smtp-Source: APXvYqy66JWY79v6syRY0RwHl2iun1rSQsGTYz1v6sbnCxs4wA09QNwDQaZ0tLkrjGBEbDzHXk3Qog== X-Received: by 2002:ac8:6f73:: with SMTP id u19mr12986458qtv.102.1580510205952; Fri, 31 Jan 2020 14:36:45 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id 69sm5198054qkk.106.2020.01.31.14.36.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:45 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 17/23] btrfs: use the same helper for data and metadata reservations Date: Fri, 31 Jan 2020 17:36:07 -0500 Message-Id: <20200131223613.490779-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that data reservations follow the same pattern as metadata reservations we can simply rename __reserve_metadata_bytes to __reserve_bytes and use that helper for data reservations. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 48 +++++++++++++------------------------------ 1 file changed, 14 insertions(+), 34 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 97379524bac8..13a3692a0122 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1016,10 +1016,9 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, * regain reservations will be made and this will fail if there is not enough * space already. */ -static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 orig_bytes, - enum btrfs_reserve_flush_enum flush) +static int __reserve_bytes(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, u64 orig_bytes, + enum btrfs_reserve_flush_enum flush) { struct reserve_ticket ticket; u64 used; @@ -1119,8 +1118,8 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; int ret; - ret = __reserve_metadata_bytes(fs_info, block_rsv->space_info, - orig_bytes, flush); + ret = __reserve_bytes(fs_info, block_rsv->space_info, orig_bytes, + flush); if (ret == -ENOSPC && unlikely(root->orphan_cleanup_state == ORPHAN_CLEANUP_STARTED)) { if (block_rsv != global_rsv && @@ -1152,37 +1151,18 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = -ENOSPC; - bool pending_tickets; + int ret; + ASSERT(flush == BTRFS_RESERVE_FLUSH_DATA || + flush == BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - pending_tickets = !list_empty(&data_sinfo->tickets) || - !list_empty(&data_sinfo->priority_tickets); - - if (pending_tickets || - used + bytes > data_sinfo->total_bytes) { - struct reserve_ticket ticket; - - init_waitqueue_head(&ticket.wait); - ticket.bytes = bytes; - ticket.error = 0; - list_add_tail(&ticket.list, &data_sinfo->priority_tickets); - spin_unlock(&data_sinfo->lock); - - ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, - flush); - } else { - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); - } - if (ret) - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", + ret = __reserve_bytes(fs_info, data_sinfo, bytes, flush); + if (ret == -ENOSPC) { + trace_btrfs_space_reservation(fs_info, "space_info:enospc", data_sinfo->flags, bytes, 1); + if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) + btrfs_dump_space_info(fs_info, data_sinfo, bytes, 0); + } return ret; } From patchwork Fri Jan 31 22:36:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360679 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E244A14B4 for ; Fri, 31 Jan 2020 22:36:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C196520CC7 for ; Fri, 31 Jan 2020 22:36:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="ZC3/uUmb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726943AbgAaWgu (ORCPT ); Fri, 31 Jan 2020 17:36:50 -0500 Received: from mail-qv1-f65.google.com ([209.85.219.65]:40499 "EHLO mail-qv1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726913AbgAaWgt (ORCPT ); Fri, 31 Jan 2020 17:36:49 -0500 Received: by mail-qv1-f65.google.com with SMTP id dp13so4039965qvb.7 for ; Fri, 31 Jan 2020 14:36:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=33hR81qqU939Th9PD2Iibmlu9uXeO2M0EG13L9JDzGI=; b=ZC3/uUmbX2Tsjbc+X1IkqnxsGYH8PWnii0CpbhTpF5KeyLT9mpsI9MocsHF8o2Mr/H RTSH1kbQhg6Mw2BWcskzlglKERNE06HWPPkbvEXOuVircZqeKhyfIyc+YVtNHmazT4n2 iI6pmojMfLsbMBfSF4I4zi+dRTVyt9JLvquGhJDCGi5nnhgjHckfZJSkbxX+I6MRmJ06 /by/n95lIaroKn3pwdATJJ6qNhNg65wuJIxW/tgfr8U/IL1MMUnX0eP/Cz4EptF5LwW6 EYRsEK7jF+DJxHhU45h+wHiwKPUReH+VopY1OMi3WsIw+zIcKxysux1VNarmt03SVwxg 7xLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=33hR81qqU939Th9PD2Iibmlu9uXeO2M0EG13L9JDzGI=; b=niGkmKIBb1BQfc7WLt/7ji7KpBOeA/IpZ+11uFT8m6OYvrksOQ6LGfk9o9y52Y63Gh UMo9GvZxjkq26bVsY4GZAMnlxoG5WUjcF04cqyRqPvkJhDSvSTHASV8zQZGS+/phzrng CSomUDkEfRtkXZrzXfDkB0FGl1LUNc7WeZSCBZw8Sm0HFvaxgAxHfQrQ09v96zMop87g NgO0HaS2k5qPhBG8Uv4xdu/ayHOuSG+El3qnt6cTDF4QS1vKjUA/eupTJVqrlIUAwQa8 7B9UraT/t/1WxDOKqy9QmsbWHgZSIcWrSrgJRBN4IxWZuTkfEouKvMZjAMjJrh2gUR15 6UoQ== X-Gm-Message-State: APjAAAVAriPxch2zKQQfK7rrP8K9QTmg8tEpOiWV3NAXJtJJ6qdNPjgT IdhzFVQJhKhOH1L1R6OOyrFHUVa+LLsPdA== X-Google-Smtp-Source: APXvYqwQDJ6A+1G8I1jDDUUNuw031skL0hyursEOeisXJK3KiOtysFB9DwPNrjcNHBgBUUAijgoA/Q== X-Received: by 2002:a0c:f513:: with SMTP id j19mr12575699qvm.206.1580510207581; Fri, 31 Jan 2020 14:36:47 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v10sm5568289qtq.58.2020.01.31.14.36.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:46 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 18/23] btrfs: drop the commit_cycles stuff for data reservations Date: Fri, 31 Jan 2020 17:36:08 -0500 Message-Id: <20200131223613.490779-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This was an old wart left over from how we previously did data reservations. Before we could have people race in and take a reservation while we were flushing space, so we needed to make sure we looped a few times before giving up. Now that we're using the ticketing infrastructure we don't have to worry about this and can drop the logic altogether. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 13a3692a0122..3060754a3341 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -858,7 +858,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, int states_nr) { int flush_state = 0; - int commit_cycles = 2; while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); @@ -869,21 +868,9 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } -again: - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = ticket->bytes; - } - flush_space(fs_info, space_info, flush_bytes, - states[flush_state]); + while (flush_state < states_nr) { + flush_space(fs_info, space_info, U64_MAX, states[flush_state]); spin_lock(&space_info->lock); if (ticket->bytes == 0) { spin_unlock(&space_info->lock); @@ -892,11 +879,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); flush_state++; } - if (commit_cycles) { - commit_cycles--; - flush_state = 0; - goto again; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Fri Jan 31 22:36:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360681 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9513014B4 for ; Fri, 31 Jan 2020 22:36:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 73C8820CC7 for ; Fri, 31 Jan 2020 22:36:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="HimvHlFW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726959AbgAaWgv (ORCPT ); Fri, 31 Jan 2020 17:36:51 -0500 Received: from mail-qk1-f195.google.com ([209.85.222.195]:42064 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726947AbgAaWgu (ORCPT ); Fri, 31 Jan 2020 17:36:50 -0500 Received: by mail-qk1-f195.google.com with SMTP id q15so8203544qke.9 for ; Fri, 31 Jan 2020 14:36:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PcYOcjsXO51RZRYs5Squ6ZI+EtuEXP9Zp6JzVyb12nY=; b=HimvHlFWZY4jdWz1y0G4WvBPQbX8lobyn8opMXcoumRwj3Hw1pz3o2Om1tFKhwIVHu F0L7dE0+4d1P0eVc9tZ/rH8ChxotFz+5/rwvH2GjSa6UOU8js0msLPUd9mFCM/IAp7PD CIRHv0rs/uW8HUZQPWtBQSL0cQWndkpXdCQdq7mQvLyXNNhskGnFSWTRJPGDyDRlZBKc acSrYll8bziCWZ5f1y3JtdpmHETfwsC8MIOjAlOM1HVSzOe4DXronpJ7HaSnrF9qIIc3 3KziYfT2z1lrorcFqqbVXM2hg18aGNYzgSJ0eMA+d2ABQ/mBTa3FkLciHrv9WCRvSV3z VeCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PcYOcjsXO51RZRYs5Squ6ZI+EtuEXP9Zp6JzVyb12nY=; b=gcIRGa+Kh/hAHe7TjMA5Ymt4V05P7ncKLtWXng81k7gBCsNAKeYZjuIzMdaJjfePuP 2JxhZFFdpz42U8Q6zBxBMCnVxfUIhFbDTIyjMtZVUuJKrekA9i8gesAVYMmpB2OB3+6M DfWTqzR2CRwTapuOa7cvd1mMbD2dNqixGX7o8u4ATZ60uZFb/WCalcNkoDZBr9WunKht 5/k16mlOE2CC7/8yuCnWRVeB59WSgB3WJea5yxXdSAUe+DfyVZ0wqEuWK5yyrmBSbl0J joxUSDQmb30PIBxGcX82SjhkT9E8LqBHBWOg2oTorsUZ+w1vYUylKxJ85r8qphCZdX+X VRaw== X-Gm-Message-State: APjAAAUqK3Htkvq1uwxgPDTPtHrn2iem30w5PZ5AAP+VXWnvOfdd4D7i uzohtXCEzUqRjOgTrob8NLfybeWbgrmibw== X-Google-Smtp-Source: APXvYqwEAozPpC6egCcVQyPja6igyGF965Lz8/9emTsSWipHu9VAnA+fK+skKmH+49Gd1DZGHUYvlw== X-Received: by 2002:a05:620a:2218:: with SMTP id m24mr13292375qkh.442.1580510209247; Fri, 31 Jan 2020 14:36:49 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b17sm294624qtr.36.2020.01.31.14.36.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:48 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 19/23] btrfs: don't pass bytes_needed to may_commit_transaction Date: Fri, 31 Jan 2020 17:36:09 -0500 Message-Id: <20200131223613.490779-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This was put into place in order to mirror the way data flushing handled committing the transaction. Now that we do not loop on committing the transaction simply force a transaction commit if we are data. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 3060754a3341..cef14a4d4167 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -412,14 +412,14 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 bytes_needed) + struct btrfs_space_info *space_info) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_trans_handle *trans; u64 reclaim_bytes = 0; + u64 bytes_needed; u64 cur_free_bytes = 0; bool do_commit = false; @@ -428,12 +428,10 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, return -EAGAIN; /* - * If we are data and have passed in U64_MAX we just want to - * unconditionally commit the transaction to match the previous data - * flushing behavior. + * If we are data just force the commit, we aren't likely to do this + * over and over again. */ - if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && - bytes_needed == U64_MAX) { + if (space_info->flags & BTRFS_BLOCK_GROUP_DATA) { do_commit = true; goto check_pinned; } @@ -451,7 +449,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : bytes_needed; + bytes_needed = (ticket) ? ticket->bytes : 0; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -584,7 +582,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info, num_bytes); + ret = may_commit_transaction(fs_info, space_info); break; default: ret = -ENOSPC; From patchwork Fri Jan 31 22:36:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360683 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E85BB13A4 for ; Fri, 31 Jan 2020 22:36:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C78E320CC7 for ; Fri, 31 Jan 2020 22:36:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="vgo82ca5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726976AbgAaWgx (ORCPT ); Fri, 31 Jan 2020 17:36:53 -0500 Received: from mail-qt1-f193.google.com ([209.85.160.193]:41230 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726947AbgAaWgw (ORCPT ); Fri, 31 Jan 2020 17:36:52 -0500 Received: by mail-qt1-f193.google.com with SMTP id l19so6685292qtq.8 for ; Fri, 31 Jan 2020 14:36:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=n5tXTLLcQsz2ZfJ8D+G2Vlr1YbUYyIGuXV27Rmuhx6U=; b=vgo82ca5VUt1Gjote4uRYkpagO0Txc5DNPmEUyDq3XaIYTY233b4o2+I7hpMIiQ917 XS5PsBaDGWAtVzyQgDW14llzDBqWbOQzzu0t5sRAV2qiw9BSskw2Aej8HpcZVrUIesbi gj54QOx9thNx4DWtZgzECdIYkIsXckKKfOsrHlD2GF3BblZQHCtcQ85SGMgugLb6Qoe2 wnmaQXO+QT/xpIWFOEOL18Oc1SwGaBo5SvVbRaP9h9/BC7oTsraEu4k/N8RZiD9F+7I/ mKy318mp/PrRBus2cVRf2YU4knuaVLmjRxp0uAWtNqrAR2mlpvA+rStyEvhrVuIVs723 e/rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n5tXTLLcQsz2ZfJ8D+G2Vlr1YbUYyIGuXV27Rmuhx6U=; b=NjXYp1x+tsqtiVMkSTI+3dYDhDCOFepJBiG5DL18jIK62eDbMayk4rX72nQH6NVIWq +KWWrENIqrZcGurMumrX/bzfkGTJJ6El4V1HJVLSIUjq40HL90xg67pDES/01GPMM4ZJ q7jOMZkJv8kL+zCUJm/mM8j1h/AD+iYZn6rZnUPa9oIX+LNI9ooFlIBngzP+0BDnDdid pR5qqYAK1TzFDtM737S3HwRnOi04DQAy096VNES2ZUfywbo4wZDdvvamqzWCvrih90h3 5Mjtk+XgPY3syUQHGKVUGeqmJghY6jFkkU0zY6uMjDtHSIhqgyUDASezB6GhJpmF9gmq PwwA== X-Gm-Message-State: APjAAAVllmOIEFDGca+bJPKhwMaUBn6wgXNhmv/XMx9JCufSlNSb+MoG ldFZqZqKp1z95OgZ5dlD0s+4MtmUcJpwlA== X-Google-Smtp-Source: APXvYqwcTJtNpKFlz7U/2L3QHYScO/zwRKXJr6dCrzJJwHeZyRQnL9k4piOrembORH9gvaTcolgIhQ== X-Received: by 2002:ac8:5548:: with SMTP id o8mr13386531qtr.338.1580510211054; Fri, 31 Jan 2020 14:36:51 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id o55sm5899716qtf.46.2020.01.31.14.36.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:50 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 20/23] btrfs: don't force commit if we are data Date: Fri, 31 Jan 2020 17:36:10 -0500 Message-Id: <20200131223613.490779-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We used to unconditionally commit the transaction at least 2 times and then on the 3rd try check against pinned space to make sure committing the transaction was worth the effort. This is overkill, we know nobody is going to steal our reservation, and if we can't make our reservation with the pinned amount simply bail out. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index cef14a4d4167..0c2d8e66cf5e 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -421,21 +421,11 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, u64 reclaim_bytes = 0; u64 bytes_needed; u64 cur_free_bytes = 0; - bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; - /* - * If we are data just force the commit, we aren't likely to do this - * over and over again. - */ - if (space_info->flags & BTRFS_BLOCK_GROUP_DATA) { - do_commit = true; - goto check_pinned; - } - spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -460,7 +450,6 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; -check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -470,8 +459,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (do_commit || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) From patchwork Fri Jan 31 22:36:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360685 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A70513A4 for ; Fri, 31 Jan 2020 22:36:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5AC7020CC7 for ; Fri, 31 Jan 2020 22:36:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="MFniYDWz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726987AbgAaWgy (ORCPT ); Fri, 31 Jan 2020 17:36:54 -0500 Received: from mail-qt1-f196.google.com ([209.85.160.196]:33961 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726947AbgAaWgy (ORCPT ); Fri, 31 Jan 2020 17:36:54 -0500 Received: by mail-qt1-f196.google.com with SMTP id h12so6746471qtu.1 for ; Fri, 31 Jan 2020 14:36:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2Ume/qlDre8Ea67Er3EDH0IdgTNboJRMluDWsWiEk1o=; b=MFniYDWzSsfc2a4tUbg0GXez6yLgeZmRrLn27/uUR5ie29Y+XnanyhWiijlg5snlV7 9ErS/96kBIYTEg0uBffflr2tpWXTVtn3LtszYK+gW6bCBsDfXqvu0ImEYDqHSWy3rzpG tKLqq/KscW4+nlIMvrZXi+mX7uwOmgM3QqkP369gPxx4MrV2H1Ydd2WDnaL4y3fd3m1D eDdaGZoyIGXU5rGw23SY0onHW571OGT5trAbKdfHL9kICI12itERhqAzifCLVxH2yT+z 17vV4Tc1fJz7NyHCa9DNw4/E4RLSGpyzRJGDJN60aJTlH8s/FjoSMSYrqaGBvaApODez csng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2Ume/qlDre8Ea67Er3EDH0IdgTNboJRMluDWsWiEk1o=; b=Jsgp01+nnhIkZ01ob7PgTIx+I9DMTwPjzfXyZvUBEwDYLskzERbwngJkdd3/ypmfz9 d45DkFBaRiUHIImtt40XF688+hHy8FxHYWq+Ep6fCdoxQepI8r7lgwCLKMZyUaPL58dl DF9lkRaoWOn359W4qj7Ri8Ue9lK58S3LoHLRmLgCfVHP8jpVBwoXC1NJ8wKU1QhOy3BR fNbhViWQwpTTVEaNf/G/r1ryqc/EU31HTe8H2YO9T+kKa2yrCStX/Ko6eU1prXelUHIf ay0geD/XSlaoZoI/UY8snsiPqrf9Sep1mmMA5XeOXxKt68HKvQrK0Cj/OICof+tV4nOr W44Q== X-Gm-Message-State: APjAAAVFQm510n8xnbBE5qPy/ZTTFsuZjWBIs77p7RvWZAGXK0JqpcGF KFynnLZHGXReXOuWOSu1KDAggU2ctiq3NA== X-Google-Smtp-Source: APXvYqyXx95Hm9buow1zyldVk78s8gFPtlg+OqzREgW0ssHcLHUOa7q8g5P7f7tV097AlecXlZ/o4Q== X-Received: by 2002:aed:3fce:: with SMTP id w14mr13721762qth.0.1580510212723; Fri, 31 Jan 2020 14:36:52 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id m95sm5664140qte.41.2020.01.31.14.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:52 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 21/23] btrfs: run delayed iputs before committing the transaction for data Date: Fri, 31 Jan 2020 17:36:11 -0500 Message-Id: <20200131223613.490779-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Before we were waiting on iputs after we committed the transaction, but this doesn't really make much sense. We want to reclaim any space we may have in order to be more likely to commit the transaction, due to pinned space being added by running the delayed iputs. Fix this by making delayed iputs run before committing the transaction. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 0c2d8e66cf5e..c86fad4174f1 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -804,8 +804,8 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, - COMMIT_TRANS, RUN_DELAYED_IPUTS, + COMMIT_TRANS, }; static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, From patchwork Fri Jan 31 22:36:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360687 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CDA514B4 for ; Fri, 31 Jan 2020 22:36:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3B48720CC7 for ; Fri, 31 Jan 2020 22:36:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="CJi0Kl4U" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726992AbgAaWg4 (ORCPT ); Fri, 31 Jan 2020 17:36:56 -0500 Received: from mail-qt1-f193.google.com ([209.85.160.193]:38731 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726947AbgAaWgz (ORCPT ); Fri, 31 Jan 2020 17:36:55 -0500 Received: by mail-qt1-f193.google.com with SMTP id c24so6718344qtp.5 for ; Fri, 31 Jan 2020 14:36:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+UrJOCU5WbPWQtIPp2MB0dRS8vSO02ENQco0IBK98lU=; b=CJi0Kl4UOxAfPvJFyR04IlxuFnAuvkQ2d6bVhqSTJOrWgvZudzbL1uAAeyeaB70jwH Phuc4ePRLdzgW+oem3KY8uEw5X2y3YYxG810wY6t02hjdOY70oogNjboaVkrvTkWx2vO StTyztydbsA+pVfeYpWNVtAxmQcNa56IuzJoPgCR578vM5MMW0pV34XspeF85bOjmuu6 QTo7CiAoMjqNfr89IgshzRCNkfgQMYV8BDTR3sBjmlost/XWj9fNko7H9rKL41hYe5Ii qBwKCzn/Fr+2kgE75b403N4q3pQd7maHtCnNTnlia/qkGl7FDQMmHDrhZ4M4KH7BXY7V s/mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+UrJOCU5WbPWQtIPp2MB0dRS8vSO02ENQco0IBK98lU=; b=dBuVfkICZHkfYmeGriPCn7EHeuWLPfKF22QB/XijRqtE3SOAaPSYPPct7QJr/QTKNA o3H1KrQI1tHIShaUbGnnJWNLAGCgNbBecUvNG06KTmnDPSwPT/v24/A6YnydFnqNsPHt BPm4L0WVVJ1iiNYbqnIKl61KGiWX8eBdq1J+9Y66m5+kyj/L+VCVy4nUGmH2kYm1Gnyw /f5SHFk9RfTcm1q63SmfmA3HXAsrIWEp3rq2UfKIK6IV5GickG8CzB76dCsqO6R2paaP CpC494648H67IMleZMbynRUbAfrKW4/LqZAUo0sAZOMnJSYO2grkU/2leswfE7I87LiS 9Fvw== X-Gm-Message-State: APjAAAVjb9UW+2dFbV3Ywx1mjaFe0gbkqAWDSqOituC1vPdZF896wR3Y xrTjVC9IjKpoA4pf/mMcA3fw57hWFS+OFQ== X-Google-Smtp-Source: APXvYqytSFImqOi/JMF2R29CPbkytU5hgHZhzNcYHwnzKUEc83FvhhiRqYZxeK4DmPi02617qhLkXQ== X-Received: by 2002:ac8:36ab:: with SMTP id a40mr12838337qtc.60.1580510214384; Fri, 31 Jan 2020 14:36:54 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id g62sm5226311qkd.25.2020.01.31.14.36.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:53 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 22/23] btrfs: flush delayed refs when trying to reserve data space Date: Fri, 31 Jan 2020 17:36:12 -0500 Message-Id: <20200131223613.490779-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We can end up with free'd extents in the delayed refs, and thus may_commit_transaction() may not think we have enough pinned space to commit the transaction and we'll ENOSPC early. Handle this by running the delayed refs in order to make sure pinned is uptodate before we try to commit the transaction. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index c86fad4174f1..5b0dc1046daa 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -805,6 +805,7 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, COMMIT_TRANS, }; From patchwork Fri Jan 31 22:36:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11360689 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CB8814B4 for ; Fri, 31 Jan 2020 22:37:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1121120705 for ; Fri, 31 Jan 2020 22:37:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="LiQbawZ5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727024AbgAaWg7 (ORCPT ); Fri, 31 Jan 2020 17:36:59 -0500 Received: from mail-qk1-f196.google.com ([209.85.222.196]:40252 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726319AbgAaWg7 (ORCPT ); Fri, 31 Jan 2020 17:36:59 -0500 Received: by mail-qk1-f196.google.com with SMTP id t204so8213826qke.7 for ; Fri, 31 Jan 2020 14:36:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=F1kgxxdEuvekc6AuLbOv+zeBfkBqv+pniLCWvc0BX5Q=; b=LiQbawZ5rSoXlwSb99XhMJ5ovjr1aYHCR0yJcB9oUarXiTOhI3onhfIMPwDKOBJR+l 9jBaH3Zvrrsx3N4WzUWPUMd6qg2HJSGnt1LSgQvLRMy4VGjASzcjWR5x2mA0JmlBZn3E AcCCLlQHeTVOvNjZqJJnjZvppBcfR/JSTdfuL8ZJaeyGXRT0RD64d1CUTXbkjhlzPbXj 1onZc/6JbpmA8pxw1cWEgEQ5Tok5KRAnRnnvKYL2/ADLrlUkN5bBBeEGjkGDvLmNaZX+ d4RRUnkOTbuXDv3EXXdg6Enqi+4JEF1hTErGwRSDVFwAzkAAl7i1tv8VCUHc/4ZqBf5s 88Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=F1kgxxdEuvekc6AuLbOv+zeBfkBqv+pniLCWvc0BX5Q=; b=VdSa8hM4KsFOdKGsjo6uJ9EsgFz5Abc0F9cq2t0IZyIhK5h70X6tFISWpUmBEj3X/T AX7C6El6mv/Rv/uAYwnV0o1viV+wujHK+4Y6RWNkiee6ji5MvFPb/T+ZZZuQh5L7IuwI zGcGn0Tj7v9M6U8jrng6msluv+adBwI3jkdbJZFkhZ/Qa33HrMTrfScdGlNEIUhPnXZL wpvrsfxmZwnTO6avmxQpFZRCIJaKxXUWsGZ7vkUg+lwoosVDmjXlECaxjBA0OtgR0Uad rMbjm4i1whgKADm/68WVPNOrgTFpAzNYFvIT3MfucMdPE5WAU8N5hsPjZUvctq8/7BER bd0g== X-Gm-Message-State: APjAAAVniqsKpwu7zhMREFOaXujRFqB34hPJKMuSbDblomMNcx2Rh9zb 3AvddNzYpsP+ipJXTIMotHBwAPdnwOROGw== X-Google-Smtp-Source: APXvYqwUOvGWn+Zz6CL6P6fowPiaxXA6C8UdKIG+9SKSi03XiMgUV64KYcLIo6tlG/ule+GAH+nJpA== X-Received: by 2002:a05:620a:1522:: with SMTP id n2mr13232754qkk.108.1580510216121; Fri, 31 Jan 2020 14:36:56 -0800 (PST) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id e13sm5714559qtq.26.2020.01.31.14.36.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2020 14:36:55 -0800 (PST) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 23/23] btrfs: do async reclaim for data reservations Date: Fri, 31 Jan 2020 17:36:13 -0500 Message-Id: <20200131223613.490779-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200131223613.490779-1-josef@toxicpanda.com> References: <20200131223613.490779-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have the data ticketing stuff in place, move normal data reservations to use an async reclaim helper to satisfy tickets. Before we could have multiple tasks race in and both allocate chunks, resulting in more data chunks than we would necessarily need. Serializing these allocations and making a single thread responsible for flushing will only allocate chunks as needed, as well as cut down on transaction commits and other flush related activities. Priority reservations will still work as they have before, simply trying to allocate a chunk until they can make their reservation. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov but look below for one --- fs/btrfs/ctree.h | 3 +- fs/btrfs/disk-io.c | 2 +- fs/btrfs/space-info.c | 123 ++++++++++++++++++++++++++++++------------ 3 files changed, 91 insertions(+), 37 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 865b24a1759e..709823a23c62 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -493,7 +493,7 @@ enum btrfs_orphan_cleanup_state { ORPHAN_CLEANUP_DONE = 2, }; -void btrfs_init_async_reclaim_work(struct work_struct *work); +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info); /* fs_info */ struct reloc_control; @@ -917,6 +917,7 @@ struct btrfs_fs_info { /* Used to reclaim the metadata space in the background. */ struct work_struct async_reclaim_work; + struct work_struct async_data_reclaim_work; spinlock_t unused_bgs_lock; struct list_head unused_bgs; diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 56d0a24aec74..825242f5c3f7 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2753,7 +2753,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info) fs_info->check_integrity_print_mask = 0; #endif btrfs_init_balance(fs_info); - btrfs_init_async_reclaim_work(&fs_info->async_reclaim_work); + btrfs_init_async_reclaim_work(fs_info); spin_lock_init(&fs_info->block_group_cache_lock); fs_info->block_group_cache_tree = RB_ROOT; diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 5b0dc1046daa..4249a6711200 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -780,9 +780,83 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } -void btrfs_init_async_reclaim_work(struct work_struct *work) +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, + COMMIT_TRANS, +}; + +static void btrfs_async_reclaim_data_space(struct work_struct *work) { - INIT_WORK(work, btrfs_async_reclaim_metadata_space); + struct btrfs_fs_info *fs_info; + struct btrfs_space_info *space_info; + u64 last_tickets_id; + int flush_state = 0; + + fs_info = container_of(work, struct btrfs_fs_info, + async_data_reclaim_work); + space_info = fs_info->data_sinfo; + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + } + + while (flush_state < ARRAY_SIZE(data_flush_states)) { + flush_space(fs_info, space_info, U64_MAX, + data_flush_states[flush_state]); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + + if (last_tickets_id == space_info->tickets_id) { + flush_state++; + } else { + last_tickets_id = space_info->tickets_id; + flush_state = 0; + } + + if (flush_state >= ARRAY_SIZE(data_flush_states)) { + if (space_info->full) { + if (maybe_fail_all_tickets(fs_info, + space_info)) + flush_state = 0; + else + space_info->flush = 0; + } else { + flush_state = 0; + } + } + spin_unlock(&space_info->lock); + } +} + +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info) +{ + INIT_WORK(&fs_info->async_reclaim_work, + btrfs_async_reclaim_metadata_space); + INIT_WORK(&fs_info->async_data_reclaim_work, + btrfs_async_reclaim_data_space); } static const enum btrfs_flush_state priority_flush_states[] = { @@ -802,13 +876,6 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; -static const enum btrfs_flush_state data_flush_states[] = { - FLUSH_DELALLOC_WAIT, - RUN_DELAYED_IPUTS, - FLUSH_DELAYED_REFS, - COMMIT_TRANS, -}; - static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, @@ -840,12 +907,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, - const enum btrfs_flush_state *states, - int states_nr) + struct reserve_ticket *ticket) { - int flush_state = 0; - while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); spin_lock(&space_info->lock); @@ -855,17 +918,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } - - while (flush_state < states_nr) { - flush_space(fs_info, space_info, U64_MAX, states[flush_state]); - spin_lock(&space_info->lock); - if (ticket->bytes == 0) { - spin_unlock(&space_info->lock); - return; - } - spin_unlock(&space_info->lock); - flush_state++; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, @@ -920,6 +972,7 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, int ret; switch (flush) { + case BTRFS_RESERVE_FLUSH_DATA: case BTRFS_RESERVE_FLUSH_ALL: wait_reserve_ticket(fs_info, space_info, ticket); break; @@ -933,14 +986,8 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; - case BTRFS_RESERVE_FLUSH_DATA: - priority_reclaim_data_space(fs_info, space_info, ticket, - data_flush_states, - ARRAY_SIZE(data_flush_states)); - break; case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: - priority_reclaim_data_space(fs_info, space_info, ticket, - NULL, 0); + priority_reclaim_data_space(fs_info, space_info, ticket); break; default: ASSERT(0); @@ -989,6 +1036,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, u64 orig_bytes, enum btrfs_reserve_flush_enum flush) { + struct work_struct *async_work; struct reserve_ticket ticket; u64 used; int ret = 0; @@ -997,6 +1045,11 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, ASSERT(orig_bytes); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_ALL); + if (flush == BTRFS_RESERVE_FLUSH_DATA) + async_work = &fs_info->async_data_reclaim_work; + else + async_work = &fs_info->async_reclaim_work; + spin_lock(&space_info->lock); ret = -ENOSPC; used = btrfs_space_info_used(space_info, true); @@ -1026,7 +1079,8 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, ticket.bytes = orig_bytes; ticket.error = 0; init_waitqueue_head(&ticket.wait); - if (flush == BTRFS_RESERVE_FLUSH_ALL) { + if (flush == BTRFS_RESERVE_FLUSH_ALL || + flush == BTRFS_RESERVE_FLUSH_DATA) { list_add_tail(&ticket.list, &space_info->tickets); if (!space_info->flush) { space_info->flush = 1; @@ -1034,8 +1088,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, space_info->flags, orig_bytes, flush, "enospc"); - queue_work(system_unbound_wq, - &fs_info->async_reclaim_work); + queue_work(system_unbound_wq, async_work); } } else { list_add_tail(&ticket.list,