From patchwork Tue Jun 30 13:58:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634079 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D698A912 for ; Tue, 30 Jun 2020 13:59:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA6DA207F9 for ; Tue, 30 Jun 2020 13:59:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="kmVfuw7K" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388171AbgF3N7a (ORCPT ); Tue, 30 Jun 2020 09:59:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727053AbgF3N73 (ORCPT ); Tue, 30 Jun 2020 09:59:29 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9ADAC061755 for ; Tue, 30 Jun 2020 06:59:27 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id 145so16127161qke.9 for ; Tue, 30 Jun 2020 06:59:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q3V6irZLnsZrtYJPX/dX6i+oAV4ziCPw7VoqYDNQ8cw=; b=kmVfuw7KRp2QDzQw2BE6qtwS9shky7fVGMkSiy9x36cmyUsJlBFBpv8NMGU6+F00et 5x4xIL4VAl8fKyDBINbLhlvTZZ7UJK4x8Mtzjieu8iTZbv2zztrVYvUKcutxu/cbu4lO a4bofwrXwVvAPwpVrSedRKzbpMyK+Bjhj7dPX/rWRFEPURCIhD4xBu6fqDjkpenMf8Lu NYhUkMr8xY4iP6Mcik1APXNb3chFNlOceZC2aX2LQc25q29g4NPeAQu4ZskjBvD1oW3P qBvh9JHrgrtAzdZENdlHVmObm/x9NB4CBszwiSENNlT2tVf34P4Y2BLdCQXBmGuMhQej hu6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q3V6irZLnsZrtYJPX/dX6i+oAV4ziCPw7VoqYDNQ8cw=; b=OdJsDEyShWjiQWlZoA81lxA0VCUpU0R6szBdWWt0bwqkgEk6RvUd4GnIryLKBKZVhL xVqGXhIqbO4EeoEkB4snLmh7RIJPV3yLiK3tXzLnf0NMO1EkSWABDnAoL85qyb6MeD3p 9aMJk/Si+N7W0nGeO7LO3lTnggEeUWXHdq/oGiAdTO5aaNUqygbOBPNK3I8W4KrCT3ZK qrdbAgR5uQnqvXi7DjGrw2SgBGYzOq6QN0VpzPgvz+kB5H5LFGWyISb/ddbcxu+fBueQ uMyrMwEF/ahVRmcIbRS1oirLyvG70uSVIW/yNhe8bUet59LGh1NDoTF3hJP9gdrMIf70 HjPg== X-Gm-Message-State: AOAM5310ch6MfeemNRUgv8Wp133TlHOatJOYkF9Ry1f9M/uaFzPpHt0M DKh3ull7TmAW1Lijp1wSHS9ArmilrL7TkA== X-Google-Smtp-Source: ABdhPJwdSuxsVcdNMl8zX/CaLH6DVT8ZSGeMu3+qWeUlFMLdTSB9QhDKK8KvylThrNTkGmrLKK0fIQ== X-Received: by 2002:a37:2781:: with SMTP id n123mr20206037qkn.106.1593525566630; Tue, 30 Jun 2020 06:59:26 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id g4sm2797749qtp.89.2020.06.30.06.59.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:25 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 01/23] btrfs: change nr to u64 in btrfs_start_delalloc_roots Date: Tue, 30 Jun 2020 09:58:59 -0400 Message-Id: <20200630135921.745612-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have btrfs_wait_ordered_roots() which takes a u64 for nr, but btrfs_start_delalloc_roots() that takes an int for nr, which makes using them in conjunction, especially for something like (u64)-1, annoying and inconsistent. Fix btrfs_start_delalloc_roots() to take a u64 for nr and adjust start_delalloc_inodes() and it's callers appropriately. This means we've adjusted start_delalloc_inodes() to take a pointer of nr since we want to preserve the ability for start-delalloc_inodes() to return an error, so simply make it do the nr adjusting as necessary. Part of adjusting the callers to this means changing btrfs_writeback_inodes_sb_nr() to take a u64 for items. This may be confusing because it seems unrelated, but the caller of btrfs_writeback_inodes_sb_nr() already passes in a u64, it's just the function variable that needs to be changed. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/inode.c | 27 +++++++++++---------------- fs/btrfs/ioctl.c | 2 +- fs/btrfs/space-info.c | 2 +- 5 files changed, 15 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index e40eb210670d..93cd1ff878cb 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2926,7 +2926,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u32 min_type); int btrfs_start_delalloc_snapshot(struct btrfs_root *root); -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr); +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr); int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end, unsigned int extra_bits, struct extent_state **cached_state); diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index db93909b25e0..769ac3098880 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -630,7 +630,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info, * flush all outstanding I/O and inode extent mappings before the * copy operation is declared as being finished */ - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) { mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return ret; diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index d301550b9c70..b1ca1e3fcc54 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -9394,7 +9394,8 @@ static struct btrfs_delalloc_work *btrfs_alloc_delalloc_work(struct inode *inode * some fairly slow code that needs optimization. This walks the list * of all the inodes with pending delalloc and forces them to disk. */ -static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) +static int start_delalloc_inodes(struct btrfs_root *root, u64 *nr, + bool snapshot) { struct btrfs_inode *binode; struct inode *inode; @@ -9434,9 +9435,11 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) list_add_tail(&work->list, &works); btrfs_queue_work(root->fs_info->flush_workers, &work->work); - ret++; - if (nr != -1 && ret >= nr) - goto out; + if (*nr != U64_MAX) { + (*nr)--; + if (*nr == 0) + goto out; + } cond_resched(); spin_lock(&root->delalloc_lock); } @@ -9461,18 +9464,15 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) int btrfs_start_delalloc_snapshot(struct btrfs_root *root) { struct btrfs_fs_info *fs_info = root->fs_info; - int ret; + u64 nr = U64_MAX; if (test_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state)) return -EROFS; - ret = start_delalloc_inodes(root, -1, true); - if (ret > 0) - ret = 0; - return ret; + return start_delalloc_inodes(root, &nr, true); } -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr) { struct btrfs_root *root; struct list_head splice; @@ -9495,15 +9495,10 @@ int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) &fs_info->delalloc_roots); spin_unlock(&fs_info->delalloc_root_lock); - ret = start_delalloc_inodes(root, nr, false); + ret = start_delalloc_inodes(root, &nr, false); btrfs_put_root(root); if (ret < 0) goto out; - - if (nr != -1) { - nr -= ret; - WARN_ON(nr < 0); - } spin_lock(&fs_info->delalloc_root_lock); } spin_unlock(&fs_info->delalloc_root_lock); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index b3e4c632d80c..0ffc4b2d2627 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -4829,7 +4829,7 @@ long btrfs_ioctl(struct file *file, unsigned int case BTRFS_IOC_SYNC: { int ret; - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) return ret; ret = btrfs_sync_fs(inode->i_sb, 1); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 41ee88633769..571b87cf1c41 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -477,7 +477,7 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, } static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, int nr_items) + unsigned long nr_pages, u64 nr_items) { struct super_block *sb = fs_info->sb; From patchwork Tue Jun 30 13:59:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634081 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4ED0A912 for ; Tue, 30 Jun 2020 13:59:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3803B207E8 for ; Tue, 30 Jun 2020 13:59:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="x6AoSOid" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388187AbgF3N7b (ORCPT ); Tue, 30 Jun 2020 09:59:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388177AbgF3N7a (ORCPT ); Tue, 30 Jun 2020 09:59:30 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5E14C061755 for ; Tue, 30 Jun 2020 06:59:29 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id h18so9291448qvl.3 for ; Tue, 30 Jun 2020 06:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NpLoVxtYuKq3xuBN6YWOovH3J+ch/wI4zzBLssFwCZc=; b=x6AoSOidWYQ/0pSAI/cqWcxBwOmYMz+mvy7YpzTTAI+hXtvJbKRUAvonwkd395kf96 Lbjk3N3WdwxFL6wSZZk8+VKLsu6pPXnEdj/LdOcqel+8NCGwKiOGtavLXrGRKR7fQ8wR MG7aNRVY9QxSzHwvedCOQqlxTKJpfuboqyncqzjKjEjDEoyivvMd2/6iohMWOoWAqPiF 5wK+Sck9yb8UE32X6r4RAp//kaFqiStevZbeVVHTWeYVhMhJfD01c3vZtrb5KeMz0sX4 lpf24YPTczSmlW18+2o6+HlNH1h4rwrPAGezL6d6HXor3PHpnyoDytg+NZ83DcxgAbFp 5Z4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NpLoVxtYuKq3xuBN6YWOovH3J+ch/wI4zzBLssFwCZc=; b=jRWooVgb/CtDPrKMW+ZRbBe35/nv6EYdAZVZ3FJMwuXk7IPP/IT49f2K9NTSkkFIUq YPYhgcaXFngZy3C1PwcYApJBpNLxnXAbAghuP1xm+cLEpF5aSrCm+pNBfnV+FCE9cDF6 31NoeYeb5buIZMokEgzR82erItmZpZFqa7uAwrCkHmt4v7ghaIQrNc1UvmC2g0h7WUct c/mebgO9ye0Z44uc+S6T5kdHs0O94+iYG0cftBkt+a+kbtNf/ZEX6IDSjCXxgEFpyYTE mApPtVktjuwnVwDoKQ8a9Q/6XQOTIb6JVFkOZcU8wR0B3VXoUO9wXPYdzcEbFKrMh9ZE CIDg== X-Gm-Message-State: AOAM5303ifZQaXM20Y+jx24VikkXlK7TjEZ0+RuGJ5oIq1O5pTFOWAmA LQN4Q9238iRp3NLuZlgrUXoSipdqMI2MVg== X-Google-Smtp-Source: ABdhPJyZoW3Ltlj9nDVGrQ731dTdVOnzlkrhdrMX6NOXmWXwFVn6pvVChDxTscc+ey30o/odLU3AxQ== X-Received: by 2002:a0c:fe01:: with SMTP id x1mr19561062qvr.246.1593525568711; Tue, 30 Jun 2020 06:59:28 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id p125sm2847058qke.78.2020.06.30.06.59.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:28 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 02/23] btrfs: remove orig from shrink_delalloc Date: Tue, 30 Jun 2020 09:59:00 -0400 Message-Id: <20200630135921.745612-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We don't use this anywhere inside of shrink_delalloc, remove it. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 571b87cf1c41..266b9887fda4 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -517,7 +517,7 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, * shrink metadata reservation for delalloc */ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - u64 orig, bool wait_ordered) + bool wait_ordered) { struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; @@ -742,7 +742,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, num_bytes, + shrink_delalloc(fs_info, num_bytes * 2, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Tue Jun 30 13:59:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634083 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8095F912 for ; Tue, 30 Jun 2020 13:59:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69604207E8 for ; Tue, 30 Jun 2020 13:59:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="Qc1pQgOW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388196AbgF3N7d (ORCPT ); Tue, 30 Jun 2020 09:59:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388159AbgF3N7c (ORCPT ); Tue, 30 Jun 2020 09:59:32 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1E80C061755 for ; Tue, 30 Jun 2020 06:59:31 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id i3so15563745qtq.13 for ; Tue, 30 Jun 2020 06:59:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o8OvljpuILSWk6dPIuW+DVZm/J9jdzzmI+Wl8q8HkOY=; b=Qc1pQgOWhtkPeFHuaOLmoHDDxxx1hCLLOC0QenFJhmcxSiULhbnVyz6pua8zpD7HHh 6a8HbkJ8isjBjbPUUKCAy9AxqXvNirGZUHjf/AQiVUjwvV+T8Rb2JdYCDDtcBGYzQMKg ua5Z1+Xg0F3ruWdXiMkV8/BPNRKYrN6RV43E96rU/Mo5/wbsYa5wkZm20XvuHzsNqo7U 47maVFT+P4YiXwJPV4Ht8MViEyXxD7eFfc71WmFb3d8KeIszTH9LQS1BuY+0u40n51Sd ipdfqe9jk5c6RX+ozJ0RmnuVEXoyu/+E1GXZBuYs0R2lwR/WQWKgZbMVd700XTrqYwau UkPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o8OvljpuILSWk6dPIuW+DVZm/J9jdzzmI+Wl8q8HkOY=; b=Lu/mcbM7kMXFISMpxrHkKjXWRqNrtcTF8RSe8WLdC3qINPvf6ZMeBSrSsg7UJ8Nda2 Lz5b7l7++A8u3DEfS7IM6bebdyJ31r8dTSie3Vq0qoHe1OIcxkFh1S/XnA1HafBvcAKj f9j5r2j7fwmI+djkujRGxUhP3mjZfeLWwXW1Cy4B9lMgyHWs6V9JI9zcVqQ3mdIGKcb2 XVs/SziJGpKi50+X55Z0kcmwz9MyVoyy/s9Hd+SsOaPYNsBGnRAbSCBO5Ymo8plMsz6V vlxii3F4omleE63Pr09nEYlygvxAAhMHF/sfRIDTVdJOYOy6vpgrGktQPuSsOHlGY5oG nznA== X-Gm-Message-State: AOAM533Qr0AuGRqN7E4+DPvywpP9Sifnk0CTPd3fdxVMOwvfBvP3ZYg7 0cWgOgiDWKq1RbI3yXomiSM/fxfOjuQJVQ== X-Google-Smtp-Source: ABdhPJwgQrKcB8wJiece15D7cYIrNFeaz3fD32eTXYrrQlVHS1liJ66uEVpHhSq7s8c+jjNz15DKtQ== X-Received: by 2002:ac8:260b:: with SMTP id u11mr21184884qtu.380.1593525570676; Tue, 30 Jun 2020 06:59:30 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id i22sm2837074qki.4.2020.06.30.06.59.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:30 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 03/23] btrfs: handle U64_MAX for shrink_delalloc Date: Tue, 30 Jun 2020 09:59:01 -0400 Message-Id: <20200630135921.745612-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data allocations are going to want to pass in U64_MAX for flushing space, adjust shrink_delalloc to handle this properly. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 266b9887fda4..c2a2326f9a79 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -530,8 +530,19 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, int loops; /* Calc the number of the pages we need flush for space reservation */ - items = calc_reclaim_items_nr(fs_info, to_reclaim); - to_reclaim = items * EXTENT_SIZE_PER_ITEM; + if (to_reclaim == U64_MAX) { + items = U64_MAX; + } else { + /* + * to_reclaim is set to however much metadata we need to + * reclaim, but reclaiming that much data doesn't really track + * exactly, so increase the amount to reclaim by 2x in order to + * make sure we're flushing enough delalloc to hopefully reclaim + * some metadata reservations. + */ + items = calc_reclaim_items_nr(fs_info, to_reclaim) * 2; + to_reclaim = items * EXTENT_SIZE_PER_ITEM; + } trans = (struct btrfs_trans_handle *)current->journal_info; space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); @@ -742,7 +753,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, + shrink_delalloc(fs_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Tue Jun 30 13:59:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634085 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3A89912 for ; Tue, 30 Jun 2020 13:59:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D5C07207E8 for ; Tue, 30 Jun 2020 13:59:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="BLrsBDt9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388224AbgF3N7f (ORCPT ); Tue, 30 Jun 2020 09:59:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388159AbgF3N7e (ORCPT ); Tue, 30 Jun 2020 09:59:34 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63A3EC061755 for ; Tue, 30 Jun 2020 06:59:34 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id g13so15595313qtv.8 for ; Tue, 30 Jun 2020 06:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G701bsI8IliaVfeT9eLjM7iHLcC7Fvjl92brNTxJI4s=; b=BLrsBDt9z6R8jN1SBp+dORIjca8JDkxrkhExpWaqgyUj8PydNbHhA33lTubFNwE7XI y6SLwNeEGxz2kR0+HKyEwrGSql/+4l+hJYqXuFHaqTsf5x1QdhSqwWCOMB8K0vynRDjw tA9GhVb6LAqZWoKEiQUgYG1OU3zTdpPj+59sbRLKHGOfLYOuP1yG3XeJIFTBsCIWPLxc huRYvOL0gLn467iXoa1JD5eaSzaSH1suW46kCfsVcq/qN8DIGafjiCwX4CCbf+1z74ga X9PmyHhun6T+iQHudWI1YySpqPJ5zKH9HcBq8HZjRojUR1Ya3LH01TrtdLsJH0LWzlkc 4fHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G701bsI8IliaVfeT9eLjM7iHLcC7Fvjl92brNTxJI4s=; b=iZYtypE/15TmXXxe/pXjgpCnxfeC8Dc5hU5CHVCqUOhVDlGSF0fLJndPfDNWsw+G7R R8pIiQcU5JHqdfIKbkSMQsi/qsUx4hQaRq8WUEkurbyPU5WRL+cdvLbRMLUhyC+XtDui eggx9KUKBc0cA5SfTBrz8yXqoErWuc0pZMzNJB8Qk7UXUnrA19vOrcMzL8niHxXMcp9l QJkGXDCq4D9sujgq0uH35c6+P6EzFlfykpkgbiyZ0JPJbmRDzmcXD/KEcR2FYVUtkHUN QMaVK8vv97F1lgs+KfY+xJxyhfOvbjNoCS45SbCkHShlHh8wL7Z6PtQh7AlKyVL1lS1w /rmA== X-Gm-Message-State: AOAM531307+LlY4BoZsMxyzJAgQpbqpv1Ox2x3i4rcYJ5/i/YfUGvYSX Q/7GI5+DVfILq6oiFkICBUxxdtYOQCwTqw== X-Google-Smtp-Source: ABdhPJzBbsrxGaDh34KVRo96fRU480LgocJb85PWxm2WDJtiqsWuDSR2z1TJhSr214lJylDNnEgBYQ== X-Received: by 2002:aed:3e2e:: with SMTP id l43mr21001410qtf.287.1593525573241; Tue, 30 Jun 2020 06:59:33 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id c27sm2853402qkl.125.2020.06.30.06.59.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:32 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 04/23] btrfs: make shrink_delalloc take space_info as an arg Date: Tue, 30 Jun 2020 09:59:02 -0400 Message-Id: <20200630135921.745612-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently shrink_delalloc just looks up the metadata space info, but this won't work if we're trying to reclaim space for data chunks. We get the right space_info we want passed into flush_space, so simply pass that along to shrink_delalloc. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index c2a2326f9a79..dda0e9ec68e3 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -516,10 +516,10 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, /* * shrink metadata reservation for delalloc */ -static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - bool wait_ordered) +static void shrink_delalloc(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + u64 to_reclaim, bool wait_ordered) { - struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; @@ -545,7 +545,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } trans = (struct btrfs_trans_handle *)current->journal_info; - space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); @@ -753,7 +752,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes, + shrink_delalloc(fs_info, space_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Tue Jun 30 13:59:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634093 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 02092912 for ; Tue, 30 Jun 2020 13:59:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D95FA20774 for ; Tue, 30 Jun 2020 13:59:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="NbLysFfJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388234AbgF3N7h (ORCPT ); Tue, 30 Jun 2020 09:59:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728637AbgF3N7g (ORCPT ); Tue, 30 Jun 2020 09:59:36 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6555FC061755 for ; Tue, 30 Jun 2020 06:59:36 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id e13so18621490qkg.5 for ; Tue, 30 Jun 2020 06:59:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wvZIkows8auYehqDoOck2uZN7Jfa9bG4yskXCxHR+6g=; b=NbLysFfJxHCSs455ruFLSiVY6PdzFmGgpzVRZSyspWLxVfIrLwQorkdcy9P97tl8/M d2gOrI0+lSjMTL30+G5vtu3E0TJ8BUz6h/l9YQGzErTf0PnpCrDZI8yYdCVd0cxM1wuH CTsQ5eIvOAJWUmjGQITjLKsalsXCMbdr6/ltszaOzxatGzQssHShMr56Wq7TDH7psM0M eFVSw+GXYAjAe8hLmySLfSYvrXbMsvi2ki/V3uNgP41ttCkkCukq1EjZBwT4RsVFmUdM BtR85a8IgTXVBUqxX32nhRTNsLkQshMa9ZCHh3OqHqV+xEumA87Qb4aa6cgVZ4Ik8NhN TwsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wvZIkows8auYehqDoOck2uZN7Jfa9bG4yskXCxHR+6g=; b=eZI0kcSevPknek4or+E9ZZ/jIzBfngoDEKWCRadsru3IJkfOZOighCH9CMVzYDhnDC sU7AjsxFNUY57kANffQy/IYmZXh7FhQUzBcZZVHWdMS/OOKmrW7d3OezzXlu8c3siwnK WePF2jDOjKl8Pi6fP7Q9boupXCfqV1NQmEIK6evAcyzIqOeZIERUhBQPoHRPQlFH11Co 92we6JmhY1zcEEALaWqbCiRUQO4l7EElfisCdE4UJvoYjo17KTYkuVbs9lCYUwqrQR7r dqtX3vRRmzIXQKL4/F2d9C87TmbroSO7eTd5kGXPINg1DJctlCvZEkIsKB/M3iQwPoKX LbrQ== X-Gm-Message-State: AOAM531LN0YZ5yod/VUgjQQ5CJdCEQ2EQuyyNWEWx24n3OJEvgoNkxBz qE85kkA6HaRBCkkp/Li45r2OnNd+Ybj9MA== X-Google-Smtp-Source: ABdhPJwVXaEfdW0/0vzwYm5UwKEsTB3hN96tYZ8IPOKC2a6/XavYupnfbSqKBwv886xaPOzSFyeJng== X-Received: by 2002:a05:620a:2f4:: with SMTP id a20mr20230371qko.227.1593525575211; Tue, 30 Jun 2020 06:59:35 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id j16sm2920233qtp.92.2020.06.30.06.59.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:34 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 05/23] btrfs: make ALLOC_CHUNK use the space info flags Date: Tue, 30 Jun 2020 09:59:03 -0400 Message-Id: <20200630135921.745612-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have traditionally used flush_space() to flush metadata space, so we've been unconditionally using btrfs_metadata_alloc_profile() for our profile to allocate a chunk. However if we're going to use this for data we need to use btrfs_get_alloc_profile() on the space_info we pass in. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index dda0e9ec68e3..3f0e6fa31937 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -777,7 +777,8 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; } ret = btrfs_chunk_alloc(trans, - btrfs_metadata_alloc_profile(fs_info), + btrfs_get_alloc_profile(fs_info, + space_info->flags), (state == ALLOC_CHUNK) ? CHUNK_ALLOC_NO_FORCE : CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); From patchwork Tue Jun 30 13:59:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C62A13B6 for ; Tue, 30 Jun 2020 13:59:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 04114207F9 for ; Tue, 30 Jun 2020 13:59:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="shv+V0ia" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388340AbgF3N7l (ORCPT ); Tue, 30 Jun 2020 09:59:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388263AbgF3N7i (ORCPT ); Tue, 30 Jun 2020 09:59:38 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 787A1C061755 for ; Tue, 30 Jun 2020 06:59:38 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id 80so18620533qko.7 for ; Tue, 30 Jun 2020 06:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pTMh1XdIBMBbevjWIsHmClI1dbdzDP2/JU+FBcW3VOM=; b=shv+V0iaGC2PXnKdKrvjgXPQt0ZWEtp0bSFG+f8nz4uq27JwIL5Ahcye+zPSdYcNet RQHqu/X29xiZRwH2DblBqdzexO6OdVUmzbUmaNidvHgeUKB2RnX+10mbz5hzv4N76ncU TLm32ZjeEjgFa3RVQl4ZrqrRVt5Sh3qEjJvGYpROleBmkqR35czbwdbsP5QLUbL59TpP z7snoAuZK0xXiq4OlIr+JRgrz7AZ73DMDjZGxXK5oQl0CJ8AyU+LPvXAJ+EC2hEs44N0 B+HVdFAHz0DwaVbsbDG0V+sV7hxWQ62SQXzOTzh1hw08pEAivo7Lkh7C5nUi3H7wu2H4 i3qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pTMh1XdIBMBbevjWIsHmClI1dbdzDP2/JU+FBcW3VOM=; b=jcKx2h0JxzVC8qQ4Q8xhNR66g/544A9hkPTvyeVLZxC+4cKCrzVAnFb9+/SdA0khil 5Wb+pA/B+peJirHgSl7TLT2uIMbWoUsCfb53R6PlGmOR4ULy8FFADXKaJ+J/47jphJBR 3pJ6uciKN8nkXnM/D2ge7yTKsp+YTkyG69k5aAFzo60HK5dJGzCyavQknlFDk2aj1uYb v05CnjVwIrPyi0wcPnHYlVVjQLhqod7cy2IXa2DleRD45xjM0J7DXEQTNGKKAHa3MzwI A1/R7wfzXaCuCubKeA9Oxff4L8IT6SjxkNO6N2oFxZzfYEPbylKWZjTBNhdAKkiHpdtM gkLw== X-Gm-Message-State: AOAM533Z6gLf5sK3APmMyNz65qbqtylZEpMAZT7PGJThd3bYgFlE5eAI gGKSHn3cFxIPdvZ+Rc3WVBV0gPL4LV58Lg== X-Google-Smtp-Source: ABdhPJwN9PYrFX8S3cntJW+vZnBqvp0DXs5HxwXUQ7+XRYzQ8A72ZkxUAMYlKtM3eQi65e+gFmObNw== X-Received: by 2002:a37:a444:: with SMTP id n65mr19418517qke.289.1593525577338; Tue, 30 Jun 2020 06:59:37 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id u22sm3274156qtb.23.2020.06.30.06.59.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:36 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 06/23] btrfs: call btrfs_try_granting_tickets when freeing reserved bytes Date: Tue, 30 Jun 2020 09:59:04 -0400 Message-Id: <20200630135921.745612-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We were missing a call to btrfs_try_granting_tickets in btrfs_free_reserved_bytes, so add it to handle the case where we're able to satisfy an allocation because we've freed a pending reservation. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 09b796a081dd..7b04cf4eeff3 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3090,6 +3090,8 @@ void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, if (delalloc) cache->delalloc_bytes -= num_bytes; spin_unlock(&cache->lock); + + btrfs_try_granting_tickets(cache->fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Tue Jun 30 13:59:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634089 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 89B0B912 for ; Tue, 30 Jun 2020 13:59:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71DB2207E8 for ; Tue, 30 Jun 2020 13:59:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="rxPaGioK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388349AbgF3N7o (ORCPT ); Tue, 30 Jun 2020 09:59:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388310AbgF3N7k (ORCPT ); Tue, 30 Jun 2020 09:59:40 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82232C061755 for ; Tue, 30 Jun 2020 06:59:40 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id j10so15576305qtq.11 for ; Tue, 30 Jun 2020 06:59:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1eyMNWoi2hC8qlRuAY9xprTphnhdSNxjekwZC/C82Cw=; b=rxPaGioK9eQLYv2ggBL4lTq+uuymCO/2+1iIg0sVvmoXyGaIltIyKloWkEGirfusBm 2cZyHmWZNyPJdGwgb5UtN8jWsyb77ql+ysMs1cPp9szfvmAN1ehJZpSJSLzd7qHVrum3 rejzitX2SNRTuMjiZyPzU/wKlsMfv+zmL5wpA8G32n1xe5kIi+ttYOsasZKvGmj+HiEr 5i8g3mHGWLXbCBzYG7mefAMMCjH1k/wXsqgQ49bT94b2FLg6/vjtAvddX+/R4AmsbuSa Efw0uvp/pOzqC03Y/7J3HAJ2A14PCm0dKZS92dkR5LK/QAX/GYVS8p2ae1Ppd6IcY9EA UdCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1eyMNWoi2hC8qlRuAY9xprTphnhdSNxjekwZC/C82Cw=; b=Oy7NL7gCuJjiE2nU9H0e4pSye28g1f7JI/6ie5BeHRgfdBx5ynoMjMfICxdkU0hNMa PwhFaPzdtu2BVIrG5K9t1fdicVYKehk4rqnNCGW+Edv/ry6LZ+5cpWfolBNUMDlnluli BUerLRL5XJDF3q4h8C/CMyAMfmhB+rSfqiUvZbD4wS4H6F/w8cRkHIwbbhooRjjiGQZQ oFgS649ufiG0NUF49XIZbjVInJawfWxXLr4GkCqDhEip7Qafooyv5fa6Jhiwhk0Gdm+h 0TFuC8ZuSViqsTh+AY53jswlrF5uuUF6Krk6i6p994YzRGzwCiT79ByNM2PtVFZJgO9+ JAdQ== X-Gm-Message-State: AOAM531as1i56jy4jnM/R32bmimNwdzbQVUgCqX2BnAkoolqf66/kS1M +6erucVGjUSVgCtvcCssC8iKtfSIlICKdQ== X-Google-Smtp-Source: ABdhPJzEd8TGZfUSdHGOMJzcl9NOywxOlWBK6fbKR2w2b68m0STG7+9cj4BC4FjDFyZ3nQeU1jpaWg== X-Received: by 2002:ac8:2bfb:: with SMTP id n56mr21474120qtn.281.1593525579406; Tue, 30 Jun 2020 06:59:39 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id c7sm3360066qta.95.2020.06.30.06.59.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:38 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 07/23] btrfs: call btrfs_try_granting_tickets when unpinning anything Date: Tue, 30 Jun 2020 09:59:05 -0400 Message-Id: <20200630135921.745612-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When unpinning we were only calling btrfs_try_granting_tickets() if global_rsv->space_info == space_info, which is problematic because we use ticketing for SYSTEM chunks, and want to use it for DATA as well. Fix this by moving this call outside of that if statement. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c0bc35f932bf..4b8c59318f6e 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2844,11 +2844,10 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info, len -= to_add; } spin_unlock(&global_rsv->lock); - /* Add to any tickets we may have */ - if (len) - btrfs_try_granting_tickets(fs_info, - space_info); } + /* Add to any tickets we may have */ + if (!readonly && return_free_space && len) + btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Tue Jun 30 13:59:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634091 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C280D13B6 for ; Tue, 30 Jun 2020 13:59:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB114207E8 for ; Tue, 30 Jun 2020 13:59:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="WtYv4Ogn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388342AbgF3N7p (ORCPT ); Tue, 30 Jun 2020 09:59:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388263AbgF3N7n (ORCPT ); Tue, 30 Jun 2020 09:59:43 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1C79C03E979 for ; Tue, 30 Jun 2020 06:59:42 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id o38so15603965qtf.6 for ; Tue, 30 Jun 2020 06:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lSf+3a72PAiNlMwNS/uTdBTwmHFW8V8FvMYLaHzUIks=; b=WtYv4Ogn4jd+Z2/PNE+xaT2B3WGNckkYHtyF7qat9AVCRaF9vqpDGPdZf2UR2CMh6y A+CdXANY5WkxL1V5rClke4oTw6C3HzTIKquMGhOfyr3xujUe0wI7D82HDOJ5VVyD0aFH vhDcIEtbvo9IuDi0kxdeoDySX1i7CdZZXPzOgD6LfRA7qNw6wx1X7Z1wF/B138xTExT6 /Ps4+b/tt9lgAoQeSRrtHakXInSaoNtyHGOcFvIPj9246ldH8eZUzxBFLKXwdhgEKm4G hRwgevzIikXc10tLtZYp1zruhdx6fKIWzcK4GHbAkjiRp/tih6Cpexsdi9sicZHAOcB3 OrPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lSf+3a72PAiNlMwNS/uTdBTwmHFW8V8FvMYLaHzUIks=; b=b/BrQdFV+YSONKvHo2++wMRBQRoKW2Qj5SdOIAutzPSzgr7CJh8b2myo/lQYWzPAes AcrKOlRC/MLPK5qHfDUdZaEk7WTSvXQIzTp3xx+TEwkW73xNtoovp50Vr6rE12Q5o4Rs ghssUMkxOskzKE2eb23sTmAvuIYXAeMRFMtnHtqw/ZfPzOGS4avmdd1bKZJnBsmEVRjz KhaZGK5kGkyZFfLnHLdjrlRcmknskPyzwcHS1F4bCF0XUx4CjaOsqUJdFojdsvZyF/NO dh724xk2fseTiNRI2d1Q2o5VLrnIe5RJRtQY8EBoDbqYjLv8hF6bH/gL5nsg7OYHwKbP yFng== X-Gm-Message-State: AOAM533ZzVSnpEQet1ItsJU8ysH9NEjdLKDuRfOjyx0zwSIqnDHpNNhF 0CqsBKUf6QSjnZYPePaHXlz7pZmPo/80Rg== X-Google-Smtp-Source: ABdhPJySIfTJX3Wym0yI1pCrObonSdfW4g0rBob8+EowYx3DmIjS33cguU5ctdOCqzAljuJhZ8nPDQ== X-Received: by 2002:ac8:7b57:: with SMTP id m23mr389835qtu.379.1593525581520; Tue, 30 Jun 2020 06:59:41 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id v10sm3243494qtk.14.2020.06.30.06.59.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:40 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 08/23] btrfs: call btrfs_try_granting_tickets when reserving space Date: Tue, 30 Jun 2020 09:59:06 -0400 Message-Id: <20200630135921.745612-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org If we have compression on we could free up more space than we reserved, and thus be able to make a space reservation. Add the call for this scenario. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 7b04cf4eeff3..df0ae5ab49bb 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3057,6 +3057,13 @@ int btrfs_add_reserved_bytes(struct btrfs_block_group *cache, space_info, -ram_bytes); if (delalloc) cache->delalloc_bytes += num_bytes; + + /* + * Compression can use less space than we reserved, so wake + * tickets if that happens. + */ + if (num_bytes < ram_bytes) + btrfs_try_granting_tickets(cache->fs_info, space_info); } spin_unlock(&cache->lock); spin_unlock(&space_info->lock); From patchwork Tue Jun 30 13:59:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634095 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7655114B7 for ; Tue, 30 Jun 2020 13:59:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 57860207E8 for ; Tue, 30 Jun 2020 13:59:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="DFY9bK6o" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388370AbgF3N7q (ORCPT ); Tue, 30 Jun 2020 09:59:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388356AbgF3N7p (ORCPT ); Tue, 30 Jun 2020 09:59:45 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5936CC061755 for ; Tue, 30 Jun 2020 06:59:45 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id j80so18631620qke.0 for ; Tue, 30 Jun 2020 06:59:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SuOlF8IpTn7Y/ZwFI/h0PcotCrvJ7Q9POZ2Ys6bf7Po=; b=DFY9bK6ove4w5+qG3b2vq2K+5nyHFnfz4OMaVU6lhLLxDaS7R52UcaYn9KHwHVaTHu kQxs3EABgzStiHVFHHBcw7+TR1BdR38icPXhigjhtHgEjMy0Ckrruzpq5314T/MJ++i1 R46HpFFTXkWeRB9/q1zZJCgK0LzvpgvIO1yCtqUCiMG3Sac6UPXb4k1mmgqGC1bhIFzh YyLoPDyn/News1aSL9CwQ6Y64Q524JY47x1Slj0EvAoYdCLDCP9UG6ei2AAheaN7ZHql 1KEA90yPTMOturB/smkOHrXqBFfIY5rKRiThTaGc/7hUd/pQD0qdL05JqM4gHZA1+wIx KL+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SuOlF8IpTn7Y/ZwFI/h0PcotCrvJ7Q9POZ2Ys6bf7Po=; b=YCqubG2zrRO4QsCzpTi5G3WZpSunrXfddisELLUgKl9ajOh5K1qRCJl5mT4hL6m2Po sc9jIEG9cIPMf9DudKHCuINExC81D3dg3EAsJUd6KQWujngzYUF4acTW5HTTuI8GUK9g BmoXcKGnaJQocQSuLRRdXZwuVo9zR81Z7r8qmZZ51EnCkbeAM93pY7+HTMI7R2kcQ7Js 2/N1AcgRMvJQ0xVsqEGbpek/Mds16W2fdcWo9WDdrG+Y5h+kwuVSpDbGiIcNzKA686z0 3PlelZ+qTWv7lkJ61swm+PaMLCB8UD4hUwqxxPel9IgxlnR9vDqFSmQe5tbxyHb5MKeR W7mA== X-Gm-Message-State: AOAM530PKQDrKUQvSW51dAblkO+ri0GtdEcbI7hqg5eCPD/bolsa2vuI wndHIfvRv2+LyFbyKzMFh1cZv1t3FI5Xmg== X-Google-Smtp-Source: ABdhPJzFD81jkMmew0muRzK9R3EQWTcLSX4IgkcdHPypIjWiIOOEhzOnWhmRq/c/D2HDDI9ytC7atg== X-Received: by 2002:a37:474d:: with SMTP id u74mr19157678qka.195.1593525584115; Tue, 30 Jun 2020 06:59:44 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t138sm3016955qka.15.2020.06.30.06.59.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:43 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 09/23] btrfs: use the btrfs_space_info_free_bytes_may_use helper for delalloc Date: Tue, 30 Jun 2020 09:59:07 -0400 Message-Id: <20200630135921.745612-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We are going to use the ticket infrastructure for data, so use the btrfs_space_info_free_bytes_may_use() helper in btrfs_free_reserved_data_space_noquota() so we get the try_granting_tickets call when we free our reservation. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index d05648f882ca..fcd75531272b 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -278,9 +278,7 @@ void btrfs_free_reserved_data_space_noquota(struct inode *inode, ASSERT(IS_ALIGNED(len, fs_info->sectorsize)); data_sinfo = fs_info->data_sinfo; - spin_lock(&data_sinfo->lock); - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, -len); - spin_unlock(&data_sinfo->lock); + btrfs_space_info_free_bytes_may_use(fs_info, data_sinfo, len); } /* From patchwork Tue Jun 30 13:59:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634099 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9BE3912 for ; Tue, 30 Jun 2020 13:59:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A090820774 for ; Tue, 30 Jun 2020 13:59:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="FidGbfmu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388377AbgF3N7t (ORCPT ); Tue, 30 Jun 2020 09:59:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387687AbgF3N7r (ORCPT ); Tue, 30 Jun 2020 09:59:47 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C9D1C061755 for ; Tue, 30 Jun 2020 06:59:47 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id 80so18621067qko.7 for ; Tue, 30 Jun 2020 06:59:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VnXtZZSzIs9CL8rkYhMWWfG8VP/VpNFi+WpvwqR/TWE=; b=FidGbfmu+Y84gsQumxE8Ace+PqnJU4ZkWF7zxvkaNZ41hd//eYLt9pz1W3oGVBEm6X +nmmgB6IJzrocNA7Mg4yGZlf/nCu+oXgVc6/QNz9NI3u8NGLjMTyGtATdl9cudsd3gwi o0bwDS/F+MifskQFeh/asPIYqZv2tjnPNCtEspEfnV5ZYUoZ3XgD2o4ow9YjGiD+SP5e P6LL3ZJp9Ynhc+STIByk1scD1I8nesIF6Z8kRl20JaRZS6Fi0l/xR9VFr73DgEoMJkra RgMJ6EQAjSCutbOJoeFV7sPmISq1o0kHD1ayLA41rDPQp8qzq7rk42em6vv7SPJ2b2v6 PTtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VnXtZZSzIs9CL8rkYhMWWfG8VP/VpNFi+WpvwqR/TWE=; b=ThYzEiHjfDtmytmoVLGT7qDwKMisMn8dn34XvATkUvSLjFnB/2YPP3UVvwRWj5GAq+ 1uaft+GbySr1sX0xj2CwTvJF6y5dPvcTrQk2Ctmx2YUU3yHCyH9K5EZ7x/+FRnpO8WR2 FOq6KqoQ5XqhiUXXIBm/y0PIZTZxbFogXx6W9bXMBVUBmm0M/FvJZm760i26Gdl3b/if 6Nx94U/0PRiuGwGaHYCcSgPscQbK5Tsj3pAC8JB7zxN1OKe3UvG01fD/lqXr8RXbz9Zd AcIR533sNbxmUtA8wwdniqZxYD1CLOA8/3stLEcJXOIlOV+bT8AxkxeBZiWWSMpKJ+59 RC+g== X-Gm-Message-State: AOAM531YsgMO8LABAmDXoUR2uFnoVLLz28990s6MrLTWXdafYhoH2q2/ Gp739FcMR+DEYiNqgPm620nkCVWrmtwd3g== X-Google-Smtp-Source: ABdhPJzwqYjpWMPLsDpXwo10J4bo+GnKtL69zef3pf/BnNPgyvnXDLs/p1iJ6WWtSgg+0M5PH9TvmQ== X-Received: by 2002:a37:c246:: with SMTP id j6mr18992459qkm.444.1593525586304; Tue, 30 Jun 2020 06:59:46 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id w28sm2819396qkw.92.2020.06.30.06.59.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:45 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 10/23] btrfs: use btrfs_start_delalloc_roots in shrink_delalloc Date: Tue, 30 Jun 2020 09:59:08 -0400 Message-Id: <20200630135921.745612-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The original iteration of flushing had us flushing delalloc and then checking to see if we could make our reservation, thus we were very careful about how many pages we would flush at once. But now that everything is async and we satisfy tickets as the space becomes available we don't have to keep track of any of this, simply try and flush the number of dirty inodes we may have in order to reclaim space to make our reservation. This cleans up our delalloc flushing significantly. The async_pages stuff is dropped because btrfs_start_delalloc_roots() handles the case that we generate async extents for us, so we no longer require this extra logic. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 55 +------------------------------------------ 1 file changed, 1 insertion(+), 54 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 3f0e6fa31937..1d3af37b0ad0 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -476,28 +476,6 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, up_read(&info->groups_sem); } -static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, u64 nr_items) -{ - struct super_block *sb = fs_info->sb; - - if (down_read_trylock(&sb->s_umount)) { - writeback_inodes_sb_nr(sb, nr_pages, WB_REASON_FS_FREE_SPACE); - up_read(&sb->s_umount); - } else { - /* - * We needn't worry the filesystem going from r/w to r/o though - * we don't acquire ->s_umount mutex, because the filesystem - * should guarantee the delalloc inodes list be empty after - * the filesystem is readonly(all dirty pages are written to - * the disk). - */ - btrfs_start_delalloc_roots(fs_info, nr_items); - if (!current->journal_info) - btrfs_wait_ordered_roots(fs_info, nr_items, 0, (u64)-1); - } -} - static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, u64 to_reclaim) { @@ -523,10 +501,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; - u64 async_pages; u64 items; long time_left; - unsigned long nr_pages; int loops; /* Calc the number of the pages we need flush for space reservation */ @@ -567,37 +543,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, loops = 0; while ((delalloc_bytes || dio_bytes) && loops < 3) { - nr_pages = min(delalloc_bytes, to_reclaim) >> PAGE_SHIFT; - - /* - * Triggers inode writeback for up to nr_pages. This will invoke - * ->writepages callback and trigger delalloc filling - * (btrfs_run_delalloc_range()). - */ - btrfs_writeback_inodes_sb_nr(fs_info, nr_pages, items); - - /* - * We need to wait for the compressed pages to start before - * we continue. - */ - async_pages = atomic_read(&fs_info->async_delalloc_pages); - if (!async_pages) - goto skip_async; - - /* - * Calculate how many compressed pages we want to be written - * before we continue. I.e if there are more async pages than we - * require wait_event will wait until nr_pages are written. - */ - if (async_pages <= nr_pages) - async_pages = 0; - else - async_pages -= nr_pages; + btrfs_start_delalloc_roots(fs_info, items); - wait_event(fs_info->async_submit_wait, - atomic_read(&fs_info->async_delalloc_pages) <= - (int)async_pages); -skip_async: spin_lock(&space_info->lock); if (list_empty(&space_info->tickets) && list_empty(&space_info->priority_tickets)) { From patchwork Tue Jun 30 13:59:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634101 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69ECA912 for ; Tue, 30 Jun 2020 13:59:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5120E20774 for ; Tue, 30 Jun 2020 13:59:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="AdDmwQAt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388392AbgF3N7v (ORCPT ); Tue, 30 Jun 2020 09:59:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387687AbgF3N7u (ORCPT ); Tue, 30 Jun 2020 09:59:50 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DF68C061755 for ; Tue, 30 Jun 2020 06:59:50 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id q198so18637529qka.2 for ; Tue, 30 Jun 2020 06:59:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DuyJL+ch1I9ONsvNvZudfQkXqBUgJDCWj2ulphkQ96w=; b=AdDmwQAtl9MgdIerLu3JWtJwX0N9/DEuxFnfmjhBaMRlaH4rrc/qRQOAxqquG+/StW k/wOdIVRo8ncNeVwN37sZuOXAVrRchvtMCxgAhrZTbIpiRljpLT8KUjtrWq1QnrBntSd ma+iuYljVcbNM6Cd7B+MbjcTl8tg6CAnV9qjuoLrXMU6/InNKgzZvSAhMDXqX2zCp6QH drKFb1zpZvSdWIaAxuY3zSpYV/Wx73nxByTR+1Kv6dJF7A6uQB0PnrZEG2+0Zyc1oA7U mBL83qRqzfvorg+gOnm/tIGVWx4Ru006dFMYCgwHTqo3srS1IZwB4cX1FifdG6rRBR1z J6BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DuyJL+ch1I9ONsvNvZudfQkXqBUgJDCWj2ulphkQ96w=; b=Qi6ragOsJ0zVVji0KomjsFCs1O6gJSRWoKNE48feGftc0g+GWACEZu/C2YMsoTrCE2 Yh+PdfXXlVjBi0x+Mu7T0u+b3pYcmzwkDKc8If9A8pXCpaPrVivi1+J+vFN2ODUBeUtP QkXp5AEllFAQUhDY0V036SSL17emulZhAOvUI11USUO3GVJMKxL8Nf1j5YLXNoP8bIc3 0DT2fLmhJ+0ibjW9716+l1FZOHtNzWjZQeFJYO5Yx+STsucfWa9xK5SnWVE0+Hbl5m/D 8JquYCLOhx/FIBHUxVG6yPH+0N57yj0YhsWirDlyVi8F2lmog4Uf/3y54g7pB4o5Bt1U kUuQ== X-Gm-Message-State: AOAM533vQzAqC8dVKofHQxiZA5yftlubAwA8QFMEBCslZqytxesloiSY qwSGpY/ggJjfcXtsAmbkqGrMJbbFZoHxtg== X-Google-Smtp-Source: ABdhPJxRxgx78dNklXMuRYLBXfPSNwt8/KrB0oCdzurNXUXiKJe2SVhn9tLvy8TvnZet48w+pXsDdQ== X-Received: by 2002:a37:b341:: with SMTP id c62mr19462342qkf.128.1593525588972; Tue, 30 Jun 2020 06:59:48 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t35sm3118712qth.79.2020.06.30.06.59.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:47 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 11/23] btrfs: check tickets after waiting on ordered extents Date: Tue, 30 Jun 2020 09:59:09 -0400 Message-Id: <20200630135921.745612-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Right now if the space is free'd up after the ordered extents complete (which is likely since the reservations are held until they complete), we would do extra delalloc flushing before we'd notice that we didn't have any more tickets. Fix this by moving the tickets check after our wait_ordered_extents check. Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 1d3af37b0ad0..0b6566a3da61 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -545,14 +545,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, while ((delalloc_bytes || dio_bytes) && loops < 3) { btrfs_start_delalloc_roots(fs_info, items); - spin_lock(&space_info->lock); - if (list_empty(&space_info->tickets) && - list_empty(&space_info->priority_tickets)) { - spin_unlock(&space_info->lock); - break; - } - spin_unlock(&space_info->lock); - loops++; if (wait_ordered && !trans) { btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1); @@ -561,6 +553,15 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, if (time_left) break; } + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets) && + list_empty(&space_info->priority_tickets)) { + spin_unlock(&space_info->lock); + break; + } + spin_unlock(&space_info->lock); + delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes); From patchwork Tue Jun 30 13:59:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634103 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D5E56912 for ; Tue, 30 Jun 2020 13:59:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BBAC520780 for ; Tue, 30 Jun 2020 13:59:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="mrZ/oxuB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388404AbgF3N7y (ORCPT ); Tue, 30 Jun 2020 09:59:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387687AbgF3N7w (ORCPT ); Tue, 30 Jun 2020 09:59:52 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5560C061755 for ; Tue, 30 Jun 2020 06:59:52 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id di5so4362989qvb.11 for ; Tue, 30 Jun 2020 06:59:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wOl3r3XCtXoWmA1dQ3UTMbZ8Zu77pCaRE4U/fVaboV8=; b=mrZ/oxuBp7wq8wbDoDfGRnZqdPL5vr9apnjPXOHqLvqI3F5WoVKrrcAXgkR/JK7jTA d4P0XoNG4w3kHgdK5wRZT7tbM6xAIf1NwWljqOz8uSg5NOH8MFNSlxpHTwbrwakDgrvv 8mnubPI/0xgcj5DBmu05H/C2a+woKK1o+1YZbt/u3dmgiRmKqX+4HUfmWYlVVEzsGpbl 4W1vvYmZ5YXI0cfqvhaQlBRF4FX0Ml14+90G+nSc1wHLWxrFSgW7mEX8/qoavVJqpIuc U+yurnhxczrIXoLuBN54zsipw9kLji9VhtuhLhKspUg8H2otpWvc0LGwXO1Tk0PBQqMi yAhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wOl3r3XCtXoWmA1dQ3UTMbZ8Zu77pCaRE4U/fVaboV8=; b=b6ub7VpJrouqKYJWnu/0Vu/dS6bC95/UbM9gQLU1jy5tAHA8aIFuLv7p+9SfLzg193 hYPdDfUJ8u+GusIXQHitCfA7jjRciGf4eGsjepQ8WHBTHH2+MxXOeCEL/sPRm5okyasL CuSBmja+ddyBRUynU7FLIeAa45jVuldeQPB5TCPDJB9r2efv/m0/vTsl/ZpjHOY8UYhD bF3RFdG5+QK2/g9pT78O2i+ROpkoiZLGWNmiU2XOhvHoMBIG68ipitZCabfenqpZ2gxA QcdO7oOLscNe8W9huP5Oalwhs48VQD+PzDUrO5Qxi7aPSg5xt1PkVd2aisq/4Em8jnFQ oLVQ== X-Gm-Message-State: AOAM530ECNaFRJlLZAwJ7NjMCH5Bb//al/EKED7Tvp1Xa9aWNp2LlAc2 HwE58p33esudgU/8ebaisOhITaPrrm0YlQ== X-Google-Smtp-Source: ABdhPJzogDRCtJVvXsa4TR0q5rCqcURZ+Z+7pm7RHwibcP7obFkrl0BYCokRCe0wme0WdemEIUX+Cg== X-Received: by 2002:a0c:f0c8:: with SMTP id d8mr20499884qvl.217.1593525591552; Tue, 30 Jun 2020 06:59:51 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id n28sm3316812qtf.8.2020.06.30.06.59.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:50 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 12/23] btrfs: add flushing states for handling data reservations Date: Tue, 30 Jun 2020 09:59:10 -0400 Message-Id: <20200630135921.745612-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently the way we do data reservations is by seeing if we have enough space in our space_info. If we do not and we're a normal inode we'll 1) Attempt to force a chunk allocation until we can't anymore. 2) If that fails we'll flush delalloc, then commit the transaction, then run the delayed iputs. If we are a free space inode we're only allowed to force a chunk allocation. In order to use the normal flushing mechanism we need to encode this into a flush state array for normal inodes. Since both will start with allocating chunks until the space info is full there is no need to add this as a flush state, this will be handled specially. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/space-info.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 93cd1ff878cb..efa72a204b91 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2569,6 +2569,8 @@ enum btrfs_reserve_flush_enum { */ BTRFS_RESERVE_FLUSH_LIMIT, BTRFS_RESERVE_FLUSH_EVICT, + BTRFS_RESERVE_FLUSH_DATA, + BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE, BTRFS_RESERVE_FLUSH_ALL, BTRFS_RESERVE_FLUSH_ALL_STEAL, }; diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 0b6566a3da61..e041b1d58e28 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1018,6 +1018,12 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + COMMIT_TRANS, + RUN_DELAYED_IPUTS, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, From patchwork Tue Jun 30 13:59:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 76AD8912 for ; Tue, 30 Jun 2020 13:59:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5C53520774 for ; Tue, 30 Jun 2020 13:59:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="F7COuy9+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388449AbgF3N74 (ORCPT ); Tue, 30 Jun 2020 09:59:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387687AbgF3N7y (ORCPT ); Tue, 30 Jun 2020 09:59:54 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0348C061755 for ; Tue, 30 Jun 2020 06:59:54 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id r22so18575767qke.13 for ; Tue, 30 Jun 2020 06:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U1ZK2JnWt4MssSxX5c4s0HVBW4N4hDnot5PGEAT5Zkw=; b=F7COuy9++EDhtt9FL4oZuKVrANNiBfHhODtKeGtH0wNqlbZPOxmj9/Wdx/3urDAcxi cZPE9k0bCJnS0t0W5XRPoO9DV6sgj+29w0CP5p5cpvjf4AUMrFkNGw66+CZUBM5J+Bls 73JucgbxKW67v5rOgUY3+TKnpnR8Ja0LFv5xiLPrRzrHBsK5zsmLoXBLyhuxcvsO8BLW xgHJOWDJYJN103W2PlyyD8xjLOnogGoUENGa6z3yPtfRwJPYCFoHathrWNjgg5wfT2as kDykIutzk6kZax645Z0AvrgxV7IwyVxCBJWF+S6GEkqLF4DkFuIOJPfGgoFulDwp3qUG MvRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U1ZK2JnWt4MssSxX5c4s0HVBW4N4hDnot5PGEAT5Zkw=; b=D8P8m4UMMwFMGUzYNi97VScfOamkDBSjHL1GQYllsqE1+xH8AOecUVALxMWcEUeVx8 bDcS6WrP/qR8JfCT7QWRcAArAYYjt/0HyMRHI4+l34wjcNrX1hJEDqJ7sP0EswhBUCNs v+jfkffmWr4GK3NFfBtzhKM37pqhqStGHGxApzfpAZmeH7gJGqOmLeDxgO+FGhVQwVUl zggsqpAmVF/GhT2Bs/bNlBoPH4YIc/F7DPWlWNdDSfjuf9iZxJQWcPnYQ20N3gATRB9W L5Rz03CbJbKzm/+mE1LDrXJtFrXMbWmiXtazQSo+9DRO3OuzPfdgFziOx+1XjuSng3Zb 5R0Q== X-Gm-Message-State: AOAM532yzK7SUT8y69bh7qSTI8SEuxTYu7u3xBCa82PJyBffcaZabBnY 7/w6U5HqFVyFXbNw+W4/hvYaoN+cEQy5yA== X-Google-Smtp-Source: ABdhPJwVOc0HINCp3DC+l4gRmoaDdoElB190AAT6LNNgEuWh+pST/AJpaF2rv4WixK84vyBdqkgOjw== X-Received: by 2002:ae9:ef83:: with SMTP id d125mr19947017qkg.287.1593525593466; Tue, 30 Jun 2020 06:59:53 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t35sm3118863qth.79.2020.06.30.06.59.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:52 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 13/23] btrfs: add the data transaction commit logic into may_commit_transaction Date: Tue, 30 Jun 2020 09:59:11 -0400 Message-Id: <20200630135921.745612-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data space flushing currently unconditionally commits the transaction twice in a row, and the last time it checks if there's enough pinned extents to satisfy it's reservation before deciding to commit the transaction for the 3rd and final time. Encode this logic into may_commit_transaction(). In the next patch we will pass in U64_MAX for bytes_needed the first two times, and the final time we will pass in the actual bytes we need so the normal logic will apply. This patch exists soley to make the logical changes I will make to the flushing state machine separate to make it easier to bisect any performance related regressions. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index e041b1d58e28..fb63ddc31540 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -579,21 +579,33 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info) + struct btrfs_space_info *space_info, + u64 bytes_needed) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_block_rsv *trans_rsv = &fs_info->trans_block_rsv; struct btrfs_trans_handle *trans; - u64 bytes_needed; u64 reclaim_bytes = 0; u64 cur_free_bytes = 0; + bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; + /* + * If we are data and have passed in U64_MAX we just want to + * unconditionally commit the transaction to match the previous data + * flushing behavior. + */ + if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && + bytes_needed == U64_MAX) { + do_commit = true; + goto check_pinned; + } + spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -607,7 +619,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : 0; + bytes_needed = (ticket) ? ticket->bytes : bytes_needed; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -618,6 +630,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; +check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -627,7 +640,8 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (do_commit || + test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -743,7 +757,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info); + ret = may_commit_transaction(fs_info, space_info, num_bytes); break; default: ret = -ENOSPC; From patchwork Tue Jun 30 13:59:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634107 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E7CB813B6 for ; Tue, 30 Jun 2020 13:59:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CA6A520774 for ; Tue, 30 Jun 2020 13:59:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="fpXvkKYI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388457AbgF3N76 (ORCPT ); Tue, 30 Jun 2020 09:59:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387687AbgF3N75 (ORCPT ); Tue, 30 Jun 2020 09:59:57 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5311C061755 for ; Tue, 30 Jun 2020 06:59:56 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id v19so15572700qtq.10 for ; Tue, 30 Jun 2020 06:59:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JDUQUmQLxpGqOAZY3JDcU4AQdki79qhLR0HALOpieRo=; b=fpXvkKYImdSTvVZP04M5/LzurDCN3psNtEW3luSMKTHiNZ23IGVbvVxGxEGCyK8dCK S4x5KdEPPEvnxUVQGx1K9RTizFBrWDFxyAUzFO3i6wq0olAZ0WbLcsjN7CPS6feJ9KEX 4afTECP+BE02Sn2pDgYaoC9Hba4JEda03Aziw9vUBka9plrLy9bnRcqIgjN3kpkmhfVV cdh1iKb5BSCHF/0ufq5g9vn8M+Lyv46+bZusfGrWr/SmN3Zn5VwwmYMUREbvxqk4XNl7 Zvt1nI32CUxZRMhnbcqz46PTS0WY6Lc/MLC0Ec22CiXqNP5PBKboGgOVwyjdZyLciVdM qQkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JDUQUmQLxpGqOAZY3JDcU4AQdki79qhLR0HALOpieRo=; b=jGwv6ROIdP4I6uCNmCEpukw2jlKdcwoTUA8vXnJnPluW9RaXd7DMZ6Pyg5Ufvuiuf/ MKcdn5fm8xSDbLDhPXxXft24UTUgvIFyK6BwQTCE2ZqWMjQQ1OcSMq2RSck6a1cOCJwP SCfBKHN/E/W3HPYOP6Ph9Vh3jRaZX1GF+0/WupIbepH4yRXOt1Ax98nao28C4K+/7BxL b52AJhNjojS/aDmMyuQYwG/xHDdWrNzs0B6DHXCfOpRrcmIvcxa73Wpuvd4mwcYgqvOQ lJJH7YQhkIPHTwcWqdAy9pWT41rze32YU+dBa5rlyyb8pgyZqKeFesYalpAcnNBoqOuy 26LA== X-Gm-Message-State: AOAM530xPe6fuFBuN/ysGNCDWZVE1ONOaZ1YZlg5SAjXXW8wOpxXEHKB Obzh61OOqIqt5QsV1rPsZC1xsHfSg0zJ3g== X-Google-Smtp-Source: ABdhPJzDXoxn5xgXRErwatYBNJGrZBH1WNlmStqWWRypdiHbnIDEXY7v1SuucUXjvtKmmgn+PqJvfA== X-Received: by 2002:ac8:66d1:: with SMTP id m17mr21025929qtp.88.1593525595560; Tue, 30 Jun 2020 06:59:55 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id i35sm3042946qtd.96.2020.06.30.06.59.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:54 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 14/23] btrfs: add btrfs_reserve_data_bytes and use it Date: Tue, 30 Jun 2020 09:59:12 -0400 Message-Id: <20200630135921.745612-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Create a new function btrfs_reserve_data_bytes() in order to handle data reservations. This uses the new flush types and flush states to handle making data reservations. This patch specifically does not change any functionality, and is purposefully not cleaned up in order to make bisection easier for the future patches. The new helper is identical to the old helper in how it handles data reservations. We first try to force a chunk allocation, and then we run through the flush states all at once and in the same order that they were done with the old helper. Subsequent patches will clean this up and change the behavior of the flushing, and it is important to keep those changes separate so we can easily bisect down to the patch that caused the regression, rather than the patch that made us start using the new infrastructure. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 119 ++------------------------------------ fs/btrfs/space-info.c | 92 +++++++++++++++++++++++++++++ fs/btrfs/space-info.h | 2 + 3 files changed, 98 insertions(+), 115 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index fcd75531272b..52e84552d662 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -115,126 +115,15 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) { struct btrfs_root *root = inode->root; struct btrfs_fs_info *fs_info = root->fs_info; - struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = 0; - int need_commit = 2; - int have_pinned_space; + enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_DATA; /* Make sure bytes are sectorsize aligned */ bytes = ALIGN(bytes, fs_info->sectorsize); - if (btrfs_is_free_space_inode(inode)) { - need_commit = 0; - ASSERT(current->journal_info); - } - -again: - /* Make sure we have enough space to handle the data first */ - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - - if (used + bytes > data_sinfo->total_bytes) { - struct btrfs_trans_handle *trans; - - /* - * If we don't have enough free bytes in this space then we need - * to alloc a new chunk. - */ - if (!data_sinfo->full) { - u64 alloc_target; - - data_sinfo->force_alloc = CHUNK_ALLOC_FORCE; - spin_unlock(&data_sinfo->lock); - - alloc_target = btrfs_data_alloc_profile(fs_info); - /* - * It is ugly that we don't call nolock join - * transaction for the free space inode case here. - * But it is safe because we only do the data space - * reservation for the free space cache in the - * transaction context, the common join transaction - * just increase the counter of the current transaction - * handler, doesn't try to acquire the trans_lock of - * the fs. - */ - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - - ret = btrfs_chunk_alloc(trans, alloc_target, - CHUNK_ALLOC_NO_FORCE); - btrfs_end_transaction(trans); - if (ret < 0) { - if (ret != -ENOSPC) - return ret; - else { - have_pinned_space = 1; - goto commit_trans; - } - } - - goto again; - } - - /* - * If we don't have enough pinned space to deal with this - * allocation, and no removed chunk in current transaction, - * don't bother committing the transaction. - */ - have_pinned_space = __percpu_counter_compare( - &data_sinfo->total_bytes_pinned, - used + bytes - data_sinfo->total_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - spin_unlock(&data_sinfo->lock); - - /* Commit the current transaction and try again */ -commit_trans: - if (need_commit) { - need_commit--; - - if (need_commit > 0) { - btrfs_start_delalloc_roots(fs_info, -1); - btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, - (u64)-1); - } - - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - if (have_pinned_space >= 0 || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, - &trans->transaction->flags) || - need_commit > 0) { - ret = btrfs_commit_transaction(trans); - if (ret) - return ret; - /* - * The cleaner kthread might still be doing iput - * operations. Wait for it to finish so that - * more space is released. We don't need to - * explicitly run the delayed iputs here because - * the commit_transaction would have woken up - * the cleaner. - */ - ret = btrfs_wait_on_delayed_iputs(fs_info); - if (ret) - return ret; - goto again; - } else { - btrfs_end_transaction(trans); - } - } - - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", - data_sinfo->flags, bytes, 1); - return -ENOSPC; - } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - spin_unlock(&data_sinfo->lock); + if (btrfs_is_free_space_inode(inode)) + flush = BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE; - return 0; + return btrfs_reserve_data_bytes(fs_info, bytes, flush); } int btrfs_check_data_free_space(struct inode *inode, diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index fb63ddc31540..799ee6090693 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1327,3 +1327,95 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, } return ret; } + +/** + * btrfs_reserve_data_bytes - try to reserve data bytes for an allocation + * @root - the root we are allocating for + * @bytes - the number of bytes we need + * @flush - how we are allowed to flush + * + * This will reserve bytes from the data space info. If there is not enough + * space then we will attempt to flush space as specified ty flush. + */ +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; + const enum btrfs_flush_state *states = NULL; + u64 used; + int states_nr = 0; + int commit_cycles = 2; + int ret = -ENOSPC; + + ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); + + if (flush == BTRFS_RESERVE_FLUSH_DATA) { + states = data_flush_states; + states_nr = ARRAY_SIZE(data_flush_states); + } + + spin_lock(&data_sinfo->lock); +again: + used = btrfs_space_info_used(data_sinfo, true); + + if (used + bytes > data_sinfo->total_bytes) { + u64 prev_total_bytes = data_sinfo->total_bytes; + int flush_state = 0; + + spin_unlock(&data_sinfo->lock); + + /* + * Everybody can force chunk allocation, so try this first to + * see if we can just bail here and make our reservation. + */ + flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); + spin_lock(&data_sinfo->lock); + if (prev_total_bytes < data_sinfo->total_bytes) + goto again; + spin_unlock(&data_sinfo->lock); + + /* + * Cycle through the rest of the flushing options for our flush + * type, then try again. + */ + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + /* + * Previously we unconditionally committed the + * transaction twice before finally checking against + * pinned space before committing the final time. We + * also skipped flushing delalloc the final pass + * through. + */ + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = bytes; + } + + flush_space(fs_info, data_sinfo, flush_bytes, + states[flush_state]); + flush_state++; + } + + if (!commit_cycles) + goto out; + + commit_cycles--; + spin_lock(&data_sinfo->lock); + goto again; + } + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; + spin_unlock(&data_sinfo->lock); +out: + if (ret) + trace_btrfs_space_reservation(fs_info, + "space_info:enospc", + data_sinfo->flags, bytes, 1); + return ret; +} diff --git a/fs/btrfs/space-info.h b/fs/btrfs/space-info.h index c3c64019950a..5646393b928c 100644 --- a/fs/btrfs/space-info.h +++ b/fs/btrfs/space-info.h @@ -149,5 +149,7 @@ static inline void btrfs_space_info_free_bytes_may_use( btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush); #endif /* BTRFS_SPACE_INFO_H */ From patchwork Tue Jun 30 13:59:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A23BD13B6 for ; Tue, 30 Jun 2020 14:00:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 87B4720780 for ; Tue, 30 Jun 2020 14:00:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="FZl22Sa3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388472AbgF3OAA (ORCPT ); Tue, 30 Jun 2020 10:00:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387687AbgF3N76 (ORCPT ); Tue, 30 Jun 2020 09:59:58 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6EABC061755 for ; Tue, 30 Jun 2020 06:59:58 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id x62so15611233qtd.3 for ; Tue, 30 Jun 2020 06:59:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZXI03dVwmDfyBQ4Mz+3HeqE9PLEvcemuppqFHIP6ego=; b=FZl22Sa3M98w9B/S0kqUC2Vh7AumdGDlpEa1v4n/9W3GI01duvHLTGoJGWa4QgqNpW /LqNf6+sY4beTNEYBU9z88pRm2/Ru+sR74MmzvfkHaAktfTe7QmhxvuHm00KElI0pNc/ Qms7WlaorduAUtY8ww7bq29OOcBLl7q283qMdm4EKiUoqsABnWexSXAvXL4WOelGWwp8 HT8q/DDmGElcVnD8xmJpoAeiAVO13z2kHMbVClXgAKjNkzGk+Nmn3WVPdK0Ka7qpiE3+ 5N49orHeuWfiWiMVfe+O5b+SVh5vrre7td8UwPhBcvSHXZ3Pd4BOeKnpyeh8Vz3U+OsW 3tTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZXI03dVwmDfyBQ4Mz+3HeqE9PLEvcemuppqFHIP6ego=; b=o2VsNcupLUKN4QfK5vPU5AJuI5TWJjrt1ybckK4K7/EdSMKmFGp1SQlq6E6wJi/H0J B9b3fNZexyr4HTX3AcG89twhOjQnPDKNwR7g8A4UtxL/O3QyDsDv4sTNf4vfpDArbgqV 8Kzl6aWGwPlslDpbj2+T22jghEClsk7bflKJJ0A4/5figs6Yyx2+t2YayGZi2t0kfWJa Ia1B7Iq6dMeQc4G1LO/ofCsBb6X4p9iNi5pJF64rKzhievUC4H9Saruss2hdpIVcxmGL yC4ouw5rzMvrgOdjrWrjaxUWvkwiOuJCLGjKZHeNWbDgpP40m0pnbCQmhVUk3m4LouKu bUhg== X-Gm-Message-State: AOAM533w0+XuCE4kVJTAIs2M/mvqvrpUcF4V+xMT7xvnokvJ3XK3OxA3 IZZqEx4oyur5LsqaenunyywU+Bc6dbaY8Q== X-Google-Smtp-Source: ABdhPJwsc6WmMSTPNaE0iWi3N1VR2+JjNlIAH+NWMvx1k247ofueKVzXCvjF8gHI5kHnKhRG7fb7lg== X-Received: by 2002:aed:2cc5:: with SMTP id g63mr20792813qtd.59.1593525597485; Tue, 30 Jun 2020 06:59:57 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id 21sm3198562qkj.56.2020.06.30.06.59.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:56 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 15/23] btrfs: use ticketing for data space reservations Date: Tue, 30 Jun 2020 09:59:13 -0400 Message-Id: <20200630135921.745612-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have all the infrastructure in place, use the ticketing infrastructure to make data allocations. This still maintains the exact same flushing behavior, but now we're using tickets to get our reservations satisfied. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 125 ++++++++++++++++++++++-------------------- 1 file changed, 67 insertions(+), 58 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 799ee6090693..ee4747917b81 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1068,6 +1068,54 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, } while (flush_state < states_nr); } +static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + struct reserve_ticket *ticket, + const enum btrfs_flush_state *states, + int states_nr) +{ + int flush_state = 0; + int commit_cycles = 2; + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + } +again: + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = ticket->bytes; + } + + flush_space(fs_info, space_info, flush_bytes, + states[flush_state]); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + flush_state++; + } + if (commit_cycles) { + commit_cycles--; + flush_state = 0; + goto again; + } +} + static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) @@ -1134,6 +1182,15 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; + case BTRFS_RESERVE_FLUSH_DATA: + priority_reclaim_data_space(fs_info, space_info, ticket, + data_flush_states, + ARRAY_SIZE(data_flush_states)); + break; + case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: + priority_reclaim_data_space(fs_info, space_info, ticket, + NULL, 0); + break; default: ASSERT(0); break; @@ -1341,78 +1398,30 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - const enum btrfs_flush_state *states = NULL; u64 used; - int states_nr = 0; - int commit_cycles = 2; int ret = -ENOSPC; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - if (flush == BTRFS_RESERVE_FLUSH_DATA) { - states = data_flush_states; - states_nr = ARRAY_SIZE(data_flush_states); - } - spin_lock(&data_sinfo->lock); -again: used = btrfs_space_info_used(data_sinfo, true); if (used + bytes > data_sinfo->total_bytes) { - u64 prev_total_bytes = data_sinfo->total_bytes; - int flush_state = 0; + struct reserve_ticket ticket; + init_waitqueue_head(&ticket.wait); + ticket.bytes = bytes; + ticket.error = 0; + list_add_tail(&ticket.list, &data_sinfo->priority_tickets); spin_unlock(&data_sinfo->lock); - /* - * Everybody can force chunk allocation, so try this first to - * see if we can just bail here and make our reservation. - */ - flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); - spin_lock(&data_sinfo->lock); - if (prev_total_bytes < data_sinfo->total_bytes) - goto again; + ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, + flush); + } else { + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; spin_unlock(&data_sinfo->lock); - - /* - * Cycle through the rest of the flushing options for our flush - * type, then try again. - */ - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - /* - * Previously we unconditionally committed the - * transaction twice before finally checking against - * pinned space before committing the final time. We - * also skipped flushing delalloc the final pass - * through. - */ - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = bytes; - } - - flush_space(fs_info, data_sinfo, flush_bytes, - states[flush_state]); - flush_state++; - } - - if (!commit_cycles) - goto out; - - commit_cycles--; - spin_lock(&data_sinfo->lock); - goto again; } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); -out: if (ret) trace_btrfs_space_reservation(fs_info, "space_info:enospc", From patchwork Tue Jun 30 13:59:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634111 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC476912 for ; Tue, 30 Jun 2020 14:00:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1930207E8 for ; Tue, 30 Jun 2020 14:00:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="cCoLTgj9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387687AbgF3OAF (ORCPT ); Tue, 30 Jun 2020 10:00:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388552AbgF3OAA (ORCPT ); Tue, 30 Jun 2020 10:00:00 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE294C061755 for ; Tue, 30 Jun 2020 07:00:00 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id 145so16129043qke.9 for ; Tue, 30 Jun 2020 07:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xx2N/cJIt3WdYApbuDV2o6kZRueGF1hgYbsNJDc02iU=; b=cCoLTgj9sRl6Aw1qeeyUY19y3Cca3xFKqLN2Jc0G14mLWS3DEmyrZCcxzA9nhaonYg nrkQG3lu1jv0RLZNDJuNSNfBWZOcW7akYGHhAK/jTcGfR2JZYo8KKtjLNbIjRy4vo/ii qS1EYQlreWIudgSdj9bCk+DNRqxvbTWs4FpqLW30iP6uEHg/1r+oVtGUVUDjhwvA6cpn Pko/6+A7oFiP0DWIgoCBTGSjOyWXw4X49799an1/IZP8D7DzS1wzjiPVjJU6aMWvbwvv YeiUcCej6k2GhKk23204DPWC4eAhqWxrCyREsnl+sp9s/oqfaL++xBvUHx924YZUT0FW LTaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xx2N/cJIt3WdYApbuDV2o6kZRueGF1hgYbsNJDc02iU=; b=XWU7uJVVWoAviAO4H0iTpj73hMgHyafTm+OyrcQJWBwX855vGojZeGbUAln7Q2qY5y DLZBclthEvn+ZFVbq6cCVu/TcRiSl5CYq69qecNUQA1ckj1htfjrG7hAAVAOmDGktf7U X4wW4+30aIK4oaLjK7CTVkMVKTokhHJ0sg5FkcHpmXEujTjzhaiOODyJqZRKYglmVsY+ RF8CG6V/fr9pS56c7TzWvJg00cNAmkG1yhw1Wb+52IXPGdirZ0d5GbBn28Dg7jcPX4/Q cRhE8f8hZUSmXMiSP7xniuDNHu3/JSE4Ws6uIQjehXScZpnO9Sl8g1Vb3aDjRNK1BIaA AQzA== X-Gm-Message-State: AOAM530OsXF6wyJ1/AjZHMT6rH3EdnEC1mJy7Fram1+NX4JlmzMyhrz0 ba6vdVWMgsFUhoQm4Ce4iI5YsYzXbircpA== X-Google-Smtp-Source: ABdhPJxxm2eYP8aLOnYolhs9VXTyaD1TmJiRQpEOkiSetcQITN4vhf08R9IRk39v7oQpudkwmMOjzQ== X-Received: by 2002:a05:620a:782:: with SMTP id 2mr10005566qka.50.1593525599493; Tue, 30 Jun 2020 06:59:59 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id j198sm3065070qke.71.2020.06.30.06.59.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 06:59:58 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 16/23] btrfs: serialize data reservations if we are flushing Date: Tue, 30 Jun 2020 09:59:14 -0400 Message-Id: <20200630135921.745612-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Nikolay reported a problem where generic/371 would fail sometimes with a slow drive. The gist of the test is that we fallocate a file in parallel with a pwrite of a different file. These two files combined are smaller than the file system, but sometimes the pwrite would ENOSPC. A fair bit of investigation uncovered the fact that the fallocate workload was racing in and grabbing the free space that the pwrite workload was trying to free up so it could make its own reservation. After a few loops of this eventually the pwrite workload would error out with an ENOSPC. We've had the same problem with metadata as well, and we serialized all metadata allocations to satisfy this problem. This wasn't usually a problem with data because data reservations are more straightforward, but obviously could still happen. Fix this by not allowing reservations to occur if there are any pending tickets waiting to be satisfied on the space info. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index ee4747917b81..d9a6aefb3a9f 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1400,13 +1400,17 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; u64 used; int ret = -ENOSPC; + bool pending_tickets; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); spin_lock(&data_sinfo->lock); used = btrfs_space_info_used(data_sinfo, true); + pending_tickets = !list_empty(&data_sinfo->tickets) || + !list_empty(&data_sinfo->priority_tickets); - if (used + bytes > data_sinfo->total_bytes) { + if (pending_tickets || + used + bytes > data_sinfo->total_bytes) { struct reserve_ticket ticket; init_waitqueue_head(&ticket.wait); From patchwork Tue Jun 30 13:59:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B85E6912 for ; Tue, 30 Jun 2020 14:00:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F771207E8 for ; Tue, 30 Jun 2020 14:00:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="oIKQVJjU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388743AbgF3OAF (ORCPT ); Tue, 30 Jun 2020 10:00:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388702AbgF3OAD (ORCPT ); Tue, 30 Jun 2020 10:00:03 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAEBDC061755 for ; Tue, 30 Jun 2020 07:00:02 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id u17so15628103qtq.1 for ; Tue, 30 Jun 2020 07:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hp9/XdwrBAg/OBpZpU0Fky59EByG1ziNL9v3oIP8KJo=; b=oIKQVJjUgcAhRwyUTKv9ph52Zletl7E1TQJOW0vEtcWqzzMFTyOK/+saMAY9yRy6fh Fe4yE3Wfcd8fWnZScvnqm430V00pvhVV2pHo6cMi550LGdU2RHzFKoyqBnWrmCcEjvTa a6f02hxtpo/ln/HGmsHEdUGsoMuNrxE1KPQZybpPbrIWeeunoI/95BDN39LMu6P0qiP7 D+NXo8qgnZERVcS8lmu4smb2hPYPQNkbnV/BwdNqvdbhq2foAv51jjuv1Ym38B2y9xXN PfGT4qqLPLuHeDzcxkZlBiimCQU0/CtOL66NMQIKIGN78kIxIWiRh8ILoLK9nX+FPifs cfxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hp9/XdwrBAg/OBpZpU0Fky59EByG1ziNL9v3oIP8KJo=; b=oTV9M+84tlm0A5qVlv1Y+XaHuCM+FeEY6Q2WeciQUj31zK3pcFBZ9UGOtp0k3L61gx TOqa5yOBK7IRV1+qv2i5LHwdT8Z3DxgpRI6g/mYBi9pPNNXoKBZDcUa30lfSiV4vVxfE bDfnFLtwL0sNDotWb6aED4iIW4Xsau36cuWz8OSKeRew2C6bBWhuIA0PhkR2QjaMFhxE WFw2LRqtfhhAQCvkzNTIYY/1rDLq446oaKcX1/Z75Unfo93gAs+p6EMyzj/5ymjSjIWI Ke36k4WIvFmJPuffxdirfsDF1rhBXXOzjRW4J4IK9Fef4YLdcfb/EW6AyLG2M2J0nSuz he1Q== X-Gm-Message-State: AOAM531pkpD9tpvrXZyYaeomRPPbAR7YM3Bw/JLHgGbuOiCPwR70QGJN BODo/kUmtdoIVOav7Ud/JCNo2g0+1z3AKw== X-Google-Smtp-Source: ABdhPJxT5zy0T4f9b56BEUXmqUkeRNwu/SsZOJjW49XataAu6hc+utGyUGZQhQoL/Xa2+tTiVUKotw== X-Received: by 2002:ac8:110a:: with SMTP id c10mr21519430qtj.163.1593525601572; Tue, 30 Jun 2020 07:00:01 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id z4sm3085538qkb.66.2020.06.30.07.00.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:00 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 17/23] btrfs: use the same helper for data and metadata reservations Date: Tue, 30 Jun 2020 09:59:15 -0400 Message-Id: <20200630135921.745612-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that data reservations follow the same pattern as metadata reservations we can simply rename __reserve_metadata_bytes to __reserve_bytes and use that helper for data reservations. Things to keep in mind, btrfs_can_overcommit() returns 0 for data, because we can never overcommit. We also will never pass in FLUSH_ALL for data, so we'll simply be added to the priority list and go straight into handle_reserve_ticket. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 48 +++++++++++++------------------------------ 1 file changed, 14 insertions(+), 34 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index d9a6aefb3a9f..b5a007fab743 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1250,10 +1250,9 @@ static inline bool is_normal_flushing(enum btrfs_reserve_flush_enum flush) * regain reservations will be made and this will fail if there is not enough * space already. */ -static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 orig_bytes, - enum btrfs_reserve_flush_enum flush) +static int __reserve_bytes(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, u64 orig_bytes, + enum btrfs_reserve_flush_enum flush) { struct reserve_ticket ticket; u64 used; @@ -1365,8 +1364,8 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; int ret; - ret = __reserve_metadata_bytes(fs_info, block_rsv->space_info, - orig_bytes, flush); + ret = __reserve_bytes(fs_info, block_rsv->space_info, orig_bytes, + flush); if (ret == -ENOSPC && unlikely(root->orphan_cleanup_state == ORPHAN_CLEANUP_STARTED)) { if (block_rsv != global_rsv && @@ -1398,37 +1397,18 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = -ENOSPC; - bool pending_tickets; + int ret; + ASSERT(flush == BTRFS_RESERVE_FLUSH_DATA || + flush == BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - pending_tickets = !list_empty(&data_sinfo->tickets) || - !list_empty(&data_sinfo->priority_tickets); - - if (pending_tickets || - used + bytes > data_sinfo->total_bytes) { - struct reserve_ticket ticket; - - init_waitqueue_head(&ticket.wait); - ticket.bytes = bytes; - ticket.error = 0; - list_add_tail(&ticket.list, &data_sinfo->priority_tickets); - spin_unlock(&data_sinfo->lock); - - ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, - flush); - } else { - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); - } - if (ret) - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", + ret = __reserve_bytes(fs_info, data_sinfo, bytes, flush); + if (ret == -ENOSPC) { + trace_btrfs_space_reservation(fs_info, "space_info:enospc", data_sinfo->flags, bytes, 1); + if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) + btrfs_dump_space_info(fs_info, data_sinfo, bytes, 0); + } return ret; } From patchwork Tue Jun 30 13:59:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83B7D912 for ; Tue, 30 Jun 2020 14:00:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A56C20774 for ; Tue, 30 Jun 2020 14:00:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="FFpl2WEB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388745AbgF3OAH (ORCPT ); Tue, 30 Jun 2020 10:00:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730122AbgF3OAF (ORCPT ); Tue, 30 Jun 2020 10:00:05 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5453C061755 for ; Tue, 30 Jun 2020 07:00:04 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id c30so14748682qka.10 for ; Tue, 30 Jun 2020 07:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CNK3PHTWRwKiFmRo4WUgIpd2m4+LI0mQWgvoNTUBZAU=; b=FFpl2WEBxbKXZfkczkZhJAnsksngF62PNLRtT8o3rI99pxl0TgXH5DfLXuLj/wYS0a JzRH3YiyTB5vduJrorZU0gK8y91ETvP3ZFhMJ1+pkMaGRUdIErx9Gi45zspAbirAMQ8V l3qYHVkewNWXNlR6QzP4uRVKaqj0IJhHAc4nEvvnHH/Gg3Sp+WkLTfTYYQLTMNtWDXck rF53wjAKdlEfgLWHvRjcC/+7Rmg9x2feLBHoFiwOkcdfW4r4AIjbgZdWQsqFSnZMgjli 5u+KEIk6a7ex36o/uhhjG56qcpYUDy3qfoJtVszcNK0WKiA7YBq2U3bLZQWw4IUjkbFO hWAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CNK3PHTWRwKiFmRo4WUgIpd2m4+LI0mQWgvoNTUBZAU=; b=rGffEGg0R/CUidRdNIUgA66thKfCFFCj18Z0kRf5ZjkX30RFyGfC+kZAIM8nDVm2u/ cLfmlBcQfLJMEk7ehAAsOlKvBBIPJmXyudEdhQoWqYYjCitLt44f1UMOaNmHXpNx2S8T SBhCEkbiBnbSKgpWNEh+H/Q21YuDOimJSXa8Ta9zG4QF915TuZJ221xv87FNMclQo+6c 7AspaZgH2I+tyMlwz8MyO4UnXpACSNOnBEFrVC24GKEtMJybLB+kaB5GmAcpWEW/255k HxmX7YJ5fj28XQ1ZPLL2WiX6HUb28So3TFd72qKgfGYpotd7pME4N0KXu6f5xMIVQGqm 3TYQ== X-Gm-Message-State: AOAM532kPMK5ZmXA8NKgVpfyP9e0aT1VkG+4QeRN1D779Uq6wKmgX/Lm I7xTIIUISYsiyZ82KbcBuQcvpP0CgY34eQ== X-Google-Smtp-Source: ABdhPJxxVHvd90pIp9JhFdDVrSyJIMWlYBa4cVeyVBhl0Khwpnt31FPGqJdZRsz3SsXVaZcvj+1z5g== X-Received: by 2002:ae9:ef04:: with SMTP id d4mr19812518qkg.41.1593525603622; Tue, 30 Jun 2020 07:00:03 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id q29sm619037qtc.10.2020.06.30.07.00.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:03 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 18/23] btrfs: drop the commit_cycles stuff for data reservations Date: Tue, 30 Jun 2020 09:59:16 -0400 Message-Id: <20200630135921.745612-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This was an old wart left over from how we previously did data reservations. Before we could have people race in and take a reservation while we were flushing space, so we needed to make sure we looped a few times before giving up. Now that we're using the ticketing infrastructure we don't have to worry about this and can drop the logic altogether. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index b5a007fab743..c88a31210b9a 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1075,7 +1075,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, int states_nr) { int flush_state = 0; - int commit_cycles = 2; while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); @@ -1086,21 +1085,9 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } -again: - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = ticket->bytes; - } - flush_space(fs_info, space_info, flush_bytes, - states[flush_state]); + while (flush_state < states_nr) { + flush_space(fs_info, space_info, U64_MAX, states[flush_state]); spin_lock(&space_info->lock); if (ticket->bytes == 0) { spin_unlock(&space_info->lock); @@ -1109,11 +1096,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); flush_state++; } - if (commit_cycles) { - commit_cycles--; - flush_state = 0; - goto again; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Tue Jun 30 13:59:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 86621138C for ; Tue, 30 Jun 2020 14:00:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5FAA920774 for ; Tue, 30 Jun 2020 14:00:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="xTFD3PGd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388762AbgF3OAL (ORCPT ); Tue, 30 Jun 2020 10:00:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730122AbgF3OAI (ORCPT ); Tue, 30 Jun 2020 10:00:08 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8092C061755 for ; Tue, 30 Jun 2020 07:00:07 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id z2so15613680qts.5 for ; Tue, 30 Jun 2020 07:00:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SUVfzbZ8KXMiSzGE2ItvCM3pO93EvG2UsX28ywarkAg=; b=xTFD3PGdwva5ucXyvT5NLhs6s3c5VgIoW1KdccfMQwf0wdZR1JlCsbCP/tNAm589aC cEaYSESwKy9VrypwVZJPCh9QSv529hKLPaNyaCp8uHh+EZy7LqwJTwvfvh4Ko0ymGXOP vU2KpPwsP42tdOOTIHhxYqlJ2Vk5evU4KZlZU0fHauWDCSRcT5KZVHZriDGK1lVqWMob 629+uoqWCIKYjyYDg+njUEpe7l2IdfMOAa+OTI10aP2zmfYq0rStWruhFEgSmiKGENEc ff30YdhzEBr/X8IMPqZXYQcp9e6BV6RuPaXvqrSJ4MF/K1+AKCpe9DIpQ47fPC6UhvaS zaQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SUVfzbZ8KXMiSzGE2ItvCM3pO93EvG2UsX28ywarkAg=; b=NPdkp1fvtSCBMr5qP/A7a/IxHrb+iitNyykoN7Pn0HjC5kU49XPAJhBCwvL/DTu+QZ 8WtE5GdEmBrnedREky4tmAOyOamFib80AB28q4zbWZzSL4rXl1Fk+A6EQkeUt/tIdiLY yW1IIhqvQZfPEqQtmqTPpVuh5K7Ec4AFgz2ClbUWRx+XegJ2l3Xn7psSZhJyup/aA1qW CtF/6Qfgw1kOj8BZxKwSb4JQmHvj186v5ohZ7aJ+FLmZgUGrk2Ov55U/1lNRskGcw2CX RimfndbK7YmrJ4fEH60u4IBxpEdcLLg1BjH8B8woXM+TTOjNW4asDamtTZV6H0COyzwg eogA== X-Gm-Message-State: AOAM530CeGrSBi8/oDTHk6zRU3v4VpY+7rR2igr4oEOg+Z38C3RbTJ8m bn4YUHM2LpR1o7rVo3HDlQ4rd+VDKc71kQ== X-Google-Smtp-Source: ABdhPJx6yhOIO0mLBEJ7Oh+aZ9I/7lJ1eQJFjwFLEU+ds4bdD1sUHr6yCUesRq1pFYFZ5mFjPR8kjg== X-Received: by 2002:ac8:458b:: with SMTP id l11mr21316753qtn.111.1593525605818; Tue, 30 Jun 2020 07:00:05 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id o12sm2877055qtl.48.2020.06.30.07.00.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:05 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 19/23] btrfs: don't force commit if we are data Date: Tue, 30 Jun 2020 09:59:17 -0400 Message-Id: <20200630135921.745612-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We used to unconditionally commit the transaction at least 2 times and then on the 3rd try check against pinned space to make sure committing the transaction was worth the effort. This is overkill, we know nobody is going to steal our reservation, and if we can't make our reservation with the pinned amount simply bail out. This also cleans up the passing of bytes_needed to may_commit_transaction, as that was the thing we added into place in order to accomplish this behavior. We no longer need it so remove that mess. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index c88a31210b9a..490dab7779e8 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -579,8 +579,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 bytes_needed) + struct btrfs_space_info *space_info) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; @@ -588,24 +587,13 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *trans_rsv = &fs_info->trans_block_rsv; struct btrfs_trans_handle *trans; u64 reclaim_bytes = 0; + u64 bytes_needed; u64 cur_free_bytes = 0; - bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; - /* - * If we are data and have passed in U64_MAX we just want to - * unconditionally commit the transaction to match the previous data - * flushing behavior. - */ - if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && - bytes_needed == U64_MAX) { - do_commit = true; - goto check_pinned; - } - spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -619,7 +607,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : bytes_needed; + bytes_needed = (ticket) ? ticket->bytes : 0; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -630,7 +618,6 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; -check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -640,8 +627,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (do_commit || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -757,7 +743,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info, num_bytes); + ret = may_commit_transaction(fs_info, space_info); break; default: ret = -ENOSPC; From patchwork Tue Jun 30 13:59:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5582713B6 for ; Tue, 30 Jun 2020 14:00:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3DA24207E8 for ; Tue, 30 Jun 2020 14:00:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="zgoG1QNi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388766AbgF3OAN (ORCPT ); Tue, 30 Jun 2020 10:00:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388748AbgF3OAK (ORCPT ); Tue, 30 Jun 2020 10:00:10 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D046C061755 for ; Tue, 30 Jun 2020 07:00:10 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id q198so18638851qka.2 for ; Tue, 30 Jun 2020 07:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ur2leh3Ja0qBtgy0KaPVx6zXttZV5oVwu1hQROwoy9w=; b=zgoG1QNihlCYgZLeP54+qnc6bUpIrhXNjTpiRyGkArvJUO5231J+6bnK3SMnCI9BqY ecMKomGryi/ABobIK4ilhJp99qeiUTuN4+h5Ttq6Z6cv1UWL4hB8oIIE80ZcbZSDGjUe 5kAraRIKKbQMKAJ88g19Tx1vggoacbWtSSYWONIqbj81IamZ3128R27W3EJYLNiluX4N WO2K+lRPvbyxWNRKuFw1+/9p5AwZ3jvsR4EmQVNmLRaqi1d57vB+gHD+p8efzerYjjsf u49Lq3bUFGTA9Er0H6ZVYU0ISwCJjDUDHYCjIt1DHSUCPbAx8IrooXabvSrtJ8Ddpxik vgXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ur2leh3Ja0qBtgy0KaPVx6zXttZV5oVwu1hQROwoy9w=; b=Lt0QQnT+Y6C9AhtJYkmsdU56p8vN5cR+K93wPaiwxvMxachDD2cqvFGx38erLWSrtt CdOOPDaynRygcqp92eXmCPYDsE/UcGmfOu3A7iP6reJfiwHIgUv9ABqcM15oQ9EJb2Wq wop4OJ3n3XkP8PbAomR9nCrCNAvGW3eEPskDr2iC06k6kWnL8TKXo+WO4/p7t0894Sxz TKt6JbyDn3Z60jDc2+k3DygrX+KBdmOR9bqlGImpuKQa3XulAISvdiPIpucfoUw5mRBr Lp3wKI0AfLtiSpSQLQJ4jJz7okKi3Ov4+bbNrGmjlxr8iOBJY1S6tGxAZKz2EWhzrTMm pSGA== X-Gm-Message-State: AOAM531pJZuOR3DgmgDatLZXxy2+KbTxIjgVocPYon6/jDxWilJYNwVM y/j6MTF00iATyGBzfleNU2vt6xno65mZfQ== X-Google-Smtp-Source: ABdhPJzcXL+gXWqEH0S6idx7HdpRqYo3kw2z42zdsoz27Hi6Npupj7QwDVbp/JFnIJmTcZBJGNbmbA== X-Received: by 2002:a37:a00a:: with SMTP id j10mr19315635qke.87.1593525608424; Tue, 30 Jun 2020 07:00:08 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id y143sm3269520qka.22.2020.06.30.07.00.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:07 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 20/23] btrfs: run delayed iputs before committing the transaction for data Date: Tue, 30 Jun 2020 09:59:18 -0400 Message-Id: <20200630135921.745612-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Before we were waiting on iputs after we committed the transaction, but this doesn't really make much sense. We want to reclaim any space we may have in order to be more likely to commit the transaction, due to pinned space being added by running the delayed iputs. Fix this by making delayed iputs run before committing the transaction. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 490dab7779e8..ba09085b6122 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1020,8 +1020,8 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, - COMMIT_TRANS, RUN_DELAYED_IPUTS, + COMMIT_TRANS, }; static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, From patchwork Tue Jun 30 13:59:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634121 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D369F912 for ; Tue, 30 Jun 2020 14:00:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BBB4D206E9 for ; Tue, 30 Jun 2020 14:00:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="dDzcsr4S" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388769AbgF3OAQ (ORCPT ); Tue, 30 Jun 2020 10:00:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388626AbgF3OAM (ORCPT ); Tue, 30 Jun 2020 10:00:12 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D01DC061755 for ; Tue, 30 Jun 2020 07:00:12 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id l6so18626229qkc.6 for ; Tue, 30 Jun 2020 07:00:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lim8CI6lg+mJedJh/Lpz8jxCpGnkjsJYsJSjoJQNZIU=; b=dDzcsr4SPXKKDBVRGaPOl90jsZYvamL/mxZDzCVQc8XC9irQKONvYAItyL2fb+kFK7 Gy08I3EgFlmevIAGt3yQKIQwWKoqbD393wPvOkiePh78ETfcFsBQ+qD1j09CoUuKNIWG b6G9Rbh6/IvL7hTd1tKaK6OpKsEc6d75qx0UwIGLaV3yi04hrxlObG8moMcGf8P7lXPj VyBEk4dk/DJsi07y99jU2rj2x9WPZHO4JC9G9f8a59Jl4UbAssEeSbl9+CZ5gfCnZcoe 5poysWzUxLSaKlCNCEfFL6cN6SkST1YZZiOfrc8smynDjrzV7GPpeXSQwEr2HeP3wNAw 7ENw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lim8CI6lg+mJedJh/Lpz8jxCpGnkjsJYsJSjoJQNZIU=; b=tBPlyIwAGTShTggF4lfhkVlQHYtx0beCt5rLEwve6cSV1vkkeYdnybOQplkaVQvWY5 ZQm3TYJj7JEmiUBY461ZLtG7NJpcp4JOJe+Cq5z6PNFVlzmYIfdwGUWeDHcWbpHSfp89 26TcweLZHVkUW0lsLzxrGNaElSv+wmy+QSlkkWj5a4oyJN7p4DnJzhhr0J15XHSOjYl0 +wZkuldqKQnYA3yAUpNmkxzOCsxaTcLz1S1ddem6XT1Cg4/IEMhnRyNgjuoMtrWTLftn Lrahc7gU+nKe6qYJxLKrfSwGw45U00kZ/4DXVLFRojgin7W+CuKLbDGovBq+Qkd/nXfY DCiw== X-Gm-Message-State: AOAM5321Fq8vfmFuL0ZF4ItwTlMIdxGclVh4lnn/eIBf+utU5at385iR Gv2q55hvUcr/qeH7VNIB5gQB5Go7ECJ2Fg== X-Google-Smtp-Source: ABdhPJywjt1+jgt+lUu2oXtQNAFU9iR3oa7jEWKJPJv9ZDwrQGGl6OnJaxmtkqbFcWKumjG2ghsuKA== X-Received: by 2002:a37:8905:: with SMTP id l5mr19135937qkd.302.1593525610425; Tue, 30 Jun 2020 07:00:10 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id k45sm3663340qtc.62.2020.06.30.07.00.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:09 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 21/23] btrfs: flush delayed refs when trying to reserve data space Date: Tue, 30 Jun 2020 09:59:19 -0400 Message-Id: <20200630135921.745612-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We can end up with free'd extents in the delayed refs, and thus may_commit_transaction() may not think we have enough pinned space to commit the transaction and we'll ENOSPC early. Handle this by running the delayed refs in order to make sure pinned is uptodate before we try to commit the transaction. Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index ba09085b6122..8b1a5b644d2f 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1021,6 +1021,7 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, COMMIT_TRANS, }; From patchwork Tue Jun 30 13:59:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5DDC138C for ; Tue, 30 Jun 2020 14:00:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A76B6206E9 for ; Tue, 30 Jun 2020 14:00:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="F9QYl8Cw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388468AbgF3OAR (ORCPT ); Tue, 30 Jun 2020 10:00:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388748AbgF3OAO (ORCPT ); Tue, 30 Jun 2020 10:00:14 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E256C061755 for ; Tue, 30 Jun 2020 07:00:14 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id m9so9307020qvx.5 for ; Tue, 30 Jun 2020 07:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DfVkz+2UGd3pKpWLk7eMITHTv6YrhNmgPwwAkCDHqPA=; b=F9QYl8Cw0MKqv7arFdC3XCgBr4f4iUw31eQAa9jXFk/BVvOXMuNeYcO5T3diY6x8Qx 6xPRywW/D5bRx5TMXfPiP9I39KmiFWusMXONHZRx88DlTaySOVKCFI0NG3wmzEOvwl7i vKn5p4nH76MgRC6Bk0e39tldhkVtw/M9czhk8gg4Vdcbea/S/Osa9Vao55+vQBssGvqb O+B37xrJs0KMlpYwF3HbhwAFn5ngww0Vrn9TrRSuwRLH1EIc8EYWv4V/EaEktupvcH2x URwD2g10cuuVjPZc0Re7GX+FZvNNv455cyFECgZqJEGTbrFJXJo0M4B+YLSZJmlxQR03 6Blg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DfVkz+2UGd3pKpWLk7eMITHTv6YrhNmgPwwAkCDHqPA=; b=aC2zEUADhPpqqIPrVfxTmUNvJ4sb7czJQww3oOFu0MpcOY/a2ECqJTMhlvGVBTpX4O okn87YjwMnOaVksOl+xf5N0a5OQM3qKMYPvgP7sUijyuR0hiyFi+EDuOkF31Wm3ZRkOe gCOv2PX7lj7u4YLFv2i4xzYDG/v3d1dcEVSWnXcYVlZjsIn+5sOFN6jdlZ50fqQr2gKN MmaoKbpxRpse082YO27IoDpUGgAxYWNE6+teEy9Q62IvJ65sgTy3K78wvVU7RYOkdmrQ 0c933gtsdtafZN9v0cvXoT/QjtM5rrBwfbndDKMRc9XqbkAluDmnX/1G0QmiN9mg50Mt 3zAw== X-Gm-Message-State: AOAM530Q78PXPvbm86HwN2lCvjIxvh7Tt5JhUQjlQPgFXwVZHHJ15Bzj rVfrzj8Ey7SyAZCyNW21MSj0bQSnG4R2jQ== X-Google-Smtp-Source: ABdhPJwi0AqOhBAO+beSGmQTFIUNDh3qEwL1VRyZ6okrmgS16fNyFMvHK5rzsZYTwVG6/ehbygInoA== X-Received: by 2002:a0c:b791:: with SMTP id l17mr20613821qve.44.1593525612635; Tue, 30 Jun 2020 07:00:12 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id w18sm2880054qtn.3.2020.06.30.07.00.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:11 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 22/23] btrfs: do async reclaim for data reservations Date: Tue, 30 Jun 2020 09:59:20 -0400 Message-Id: <20200630135921.745612-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have the data ticketing stuff in place, move normal data reservations to use an async reclaim helper to satisfy tickets. Before we could have multiple tasks race in and both allocate chunks, resulting in more data chunks than we would necessarily need. Serializing these allocations and making a single thread responsible for flushing will only allocate chunks as needed, as well as cut down on transaction commits and other flush related activities. Priority reservations will still work as they have before, simply trying to allocate a chunk until they can make their reservation. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 +- fs/btrfs/disk-io.c | 3 +- fs/btrfs/space-info.c | 123 ++++++++++++++++++++++++++++++------------ fs/btrfs/super.c | 1 + 4 files changed, 93 insertions(+), 37 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index efa72a204b91..998bda3132fd 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -494,7 +494,7 @@ enum btrfs_orphan_cleanup_state { ORPHAN_CLEANUP_DONE = 2, }; -void btrfs_init_async_reclaim_work(struct work_struct *work); +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info); /* fs_info */ struct reloc_control; @@ -916,6 +916,7 @@ struct btrfs_fs_info { /* Used to reclaim the metadata space in the background. */ struct work_struct async_reclaim_work; + struct work_struct async_data_reclaim_work; spinlock_t unused_bgs_lock; struct list_head unused_bgs; diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 7c07578866f3..0907e02eea54 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2713,7 +2713,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info) fs_info->check_integrity_print_mask = 0; #endif btrfs_init_balance(fs_info); - btrfs_init_async_reclaim_work(&fs_info->async_reclaim_work); + btrfs_init_async_reclaim_work(fs_info); spin_lock_init(&fs_info->block_group_cache_lock); fs_info->block_group_cache_tree = RB_ROOT; @@ -4033,6 +4033,7 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) btrfs_cleanup_defrag_inodes(fs_info); cancel_work_sync(&fs_info->async_reclaim_work); + cancel_work_sync(&fs_info->async_data_reclaim_work); /* Cancel or finish ongoing discard work */ btrfs_discard_cleanup(fs_info); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 8b1a5b644d2f..0a2fdfaa9fe6 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -996,9 +996,83 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } -void btrfs_init_async_reclaim_work(struct work_struct *work) +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, + COMMIT_TRANS, +}; + +static void btrfs_async_reclaim_data_space(struct work_struct *work) { - INIT_WORK(work, btrfs_async_reclaim_metadata_space); + struct btrfs_fs_info *fs_info; + struct btrfs_space_info *space_info; + u64 last_tickets_id; + int flush_state = 0; + + fs_info = container_of(work, struct btrfs_fs_info, + async_data_reclaim_work); + space_info = fs_info->data_sinfo; + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + } + + while (flush_state < ARRAY_SIZE(data_flush_states)) { + flush_space(fs_info, space_info, U64_MAX, + data_flush_states[flush_state]); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + + if (last_tickets_id == space_info->tickets_id) { + flush_state++; + } else { + last_tickets_id = space_info->tickets_id; + flush_state = 0; + } + + if (flush_state >= ARRAY_SIZE(data_flush_states)) { + if (space_info->full) { + if (maybe_fail_all_tickets(fs_info, + space_info)) + flush_state = 0; + else + space_info->flush = 0; + } else { + flush_state = 0; + } + } + spin_unlock(&space_info->lock); + } +} + +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info) +{ + INIT_WORK(&fs_info->async_reclaim_work, + btrfs_async_reclaim_metadata_space); + INIT_WORK(&fs_info->async_data_reclaim_work, + btrfs_async_reclaim_data_space); } static const enum btrfs_flush_state priority_flush_states[] = { @@ -1018,13 +1092,6 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; -static const enum btrfs_flush_state data_flush_states[] = { - FLUSH_DELALLOC_WAIT, - RUN_DELAYED_IPUTS, - FLUSH_DELAYED_REFS, - COMMIT_TRANS, -}; - static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, @@ -1057,12 +1124,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, - const enum btrfs_flush_state *states, - int states_nr) + struct reserve_ticket *ticket) { - int flush_state = 0; - while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); spin_lock(&space_info->lock); @@ -1072,17 +1135,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } - - while (flush_state < states_nr) { - flush_space(fs_info, space_info, U64_MAX, states[flush_state]); - spin_lock(&space_info->lock); - if (ticket->bytes == 0) { - spin_unlock(&space_info->lock); - return; - } - spin_unlock(&space_info->lock); - flush_state++; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, @@ -1137,6 +1189,7 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, int ret; switch (flush) { + case BTRFS_RESERVE_FLUSH_DATA: case BTRFS_RESERVE_FLUSH_ALL: case BTRFS_RESERVE_FLUSH_ALL_STEAL: wait_reserve_ticket(fs_info, space_info, ticket); @@ -1151,14 +1204,8 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; - case BTRFS_RESERVE_FLUSH_DATA: - priority_reclaim_data_space(fs_info, space_info, ticket, - data_flush_states, - ARRAY_SIZE(data_flush_states)); - break; case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: - priority_reclaim_data_space(fs_info, space_info, ticket, - NULL, 0); + priority_reclaim_data_space(fs_info, space_info, ticket); break; default: ASSERT(0); @@ -1223,6 +1270,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, u64 orig_bytes, enum btrfs_reserve_flush_enum flush) { + struct work_struct *async_work; struct reserve_ticket ticket; u64 used; int ret = 0; @@ -1231,6 +1279,11 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, ASSERT(orig_bytes); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_ALL); + if (flush == BTRFS_RESERVE_FLUSH_DATA) + async_work = &fs_info->async_data_reclaim_work; + else + async_work = &fs_info->async_reclaim_work; + spin_lock(&space_info->lock); ret = -ENOSPC; used = btrfs_space_info_used(space_info, true); @@ -1272,7 +1325,8 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, init_waitqueue_head(&ticket.wait); ticket.steal = (flush == BTRFS_RESERVE_FLUSH_ALL_STEAL); if (flush == BTRFS_RESERVE_FLUSH_ALL || - flush == BTRFS_RESERVE_FLUSH_ALL_STEAL) { + flush == BTRFS_RESERVE_FLUSH_ALL_STEAL || + flush == BTRFS_RESERVE_FLUSH_DATA) { list_add_tail(&ticket.list, &space_info->tickets); if (!space_info->flush) { space_info->flush = 1; @@ -1280,8 +1334,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, space_info->flags, orig_bytes, flush, "enospc"); - queue_work(system_unbound_wq, - &fs_info->async_reclaim_work); + queue_work(system_unbound_wq, async_work); } } else { list_add_tail(&ticket.list, diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 3c9ebd4f2b61..66094f076385 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1880,6 +1880,7 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) * the filesystem is busy. */ cancel_work_sync(&fs_info->async_reclaim_work); + cancel_work_sync(&fs_info->async_data_reclaim_work); btrfs_discard_cleanup(fs_info); From patchwork Tue Jun 30 13:59:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11634123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 00F66912 for ; Tue, 30 Jun 2020 14:00:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DC61A206BE for ; Tue, 30 Jun 2020 14:00:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="uwSQlrpd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388626AbgF3OAR (ORCPT ); Tue, 30 Jun 2020 10:00:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388770AbgF3OAQ (ORCPT ); Tue, 30 Jun 2020 10:00:16 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0D9FC03E979 for ; Tue, 30 Jun 2020 07:00:15 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id q198so18639283qka.2 for ; Tue, 30 Jun 2020 07:00:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xHhyavkxTL8alcSYgeq4qkBfKvKJ2Z9T73hkmCD1E2U=; b=uwSQlrpdFZEwrENeLB+KSPz0ERVzjFsMM6F7o2b1wZlw+wr2ph9x2NASYd4wlnKCcw TWGOwIxsanEgWuRI5VsLYXSaW39Eh7mdwYdiS1kSQD9rYosS8Svs7uUhTRcGlseTUeFf GtJG36X/qmJICxU9lxv4UP5mFoqXjowuFXzNXX6HzxPmI3dnWpeXIu6CeIMx5vBH3xLH fOpVxKctcLKkUCOZVbCH/E7TSRgJ064BEWXIQlX0RPqFrHT2K25J19Hyr88MPIXmqtJe KzeM4klSSVzUZQ+WsuFNcn0mAkxh5nVm1r2937m6lQAeHSG5L10R6hPu9lF02/i2zObA L8VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xHhyavkxTL8alcSYgeq4qkBfKvKJ2Z9T73hkmCD1E2U=; b=j2RC3svGrhvCh7r5YdVlgUnmsQssu/KfQnN1z3t4xZ/H7yOUC7ttKVBQJDd5P83vat REa7v4MarbCJqJPfuci0mvyQSIMrt22bSED8FtYV5ocZX1CdtpN5T1FjmZC0C6p9b1ft zORY0KRt20JmnjyMl7N0CnwCPhT0By9hDz3+k8smRYxViSbCf/tHubPopBEzSMeRlRrs YH8aBwv7seF3tVB2q4MOoXRMled8IPNZC5HJyDNEEqqk6hFrTBhEfh0EAYXVYjhgHXM5 AUMlmf3jiyxZC+bU/aNF5KJI8RjdxwgGNhW0pnam0deyA6dFOmAxhKopY466FYX0Zl6x hPUA== X-Gm-Message-State: AOAM531CTMNh0qZOKMqQveScBfAKQzAjMbFAXQeJIXhD0yt5RRYVfskj 4mv5D8Jzzp7i6c0BPp1k1HvrtJbPXSo54Q== X-Google-Smtp-Source: ABdhPJwtzZ+ZAcOhtieMLVoS0MQ5ar3BmhYdUEhJEBxOqnDx0ESic0+ewHFefSU5XM7LLrK0x/CMlA== X-Received: by 2002:a37:3d4:: with SMTP id 203mr18910194qkd.420.1593525614612; Tue, 30 Jun 2020 07:00:14 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id h185sm3105057qkf.85.2020.06.30.07.00.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 07:00:14 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 23/23] btrfs: add a comment explaining the data flush steps Date: Tue, 30 Jun 2020 09:59:21 -0400 Message-Id: <20200630135921.745612-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200630135921.745612-1-josef@toxicpanda.com> References: <20200630135921.745612-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The data flushing steps are not obvious to people other than myself and Chris. Write a giant comment explaining the reasoning behind each flush step for data as well as why it is in that particular order. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 47 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 0a2fdfaa9fe6..818f0a6caacd 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -996,6 +996,53 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } +/* + * FLUSH_DELALLOC_WAIT: + * Space is free'd from flushing delalloc in one of two ways. + * + * 1) compression is on and we allocate less space than we reserved. + * 2) We are overwriting existing space. + * + * For #1 that extra space is reclaimed as soon as the delalloc pages are + * cow'ed, by way of btrfs_add_reserved_bytes() which adds the actual extent + * length to ->bytes_reserved, and subtracts the reserved space from + * ->bytes_may_use. + * + * For #2 this is trickier. Once the ordered extent runs we will drop the + * extent in the range we are overwriting, which creates a delayed ref for + * that freed extent. This however is not reclaimed until the transaction + * commits, thus the next stages. + * + * RUN_DELAYED_IPUTS + * If we are freeing inodes, we want to make sure all delayed iputs have + * completed, because they could have been on an inode with i_nlink == 0, and + * thus have been trunated and free'd up space. But again this space is not + * immediately re-usable, it comes in the form of a delayed ref, which must be + * run and then the transaction must be committed. + * + * FLUSH_DELAYED_REFS + * The above two cases generate delayed refs that will affect + * ->total_bytes_pinned. However this counter can be inconsistent with + * reality if there are outstanding delayed refs. This is because we adjust + * the counter based soley on the current set of delayed refs and disregard + * any on-disk state which might include more refs. So for example, if we + * have an extent with 2 references, but we only drop 1, we'll see that there + * is a negative delayed ref count for the extent and assume that the space + * will be free'd, and thus increase ->total_bytes_pinned. + * + * Running the delayed refs gives us the actual real view of what will be + * freed at the transaction commit time. This stage will not actually free + * space for us, it just makes sure that may_commit_transaction() has all of + * the information it needs to make the right decision. + * + * COMMIT_TRANS + * This is where we reclaim all of the pinned space generated by the previous + * two stages. We will not commit the transaction if we don't think we're + * likely to satisfy our request, which means if our current free space + + * total_bytes_pinned < reservation we will not commit. This is why the + * previous states are actually important, to make sure we know for sure + * whether committing the transaction will allow us to make progress. + */ static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS,