From patchwork Tue Jul 7 15:42:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11648989 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 259E76C1 for ; Tue, 7 Jul 2020 15:42:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 085C7206F6 for ; Tue, 7 Jul 2020 15:42:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="xH9Fhkux" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728746AbgGGPmy (ORCPT ); Tue, 7 Jul 2020 11:42:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPmx (ORCPT ); Tue, 7 Jul 2020 11:42:53 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BFEAC061755 for ; Tue, 7 Jul 2020 08:42:53 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id b185so27939164qkg.1 for ; Tue, 07 Jul 2020 08:42:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q3V6irZLnsZrtYJPX/dX6i+oAV4ziCPw7VoqYDNQ8cw=; b=xH9FhkuxdD5bw+bx3J5l8d0+ifM99OuP+wxOiQ/S0q5uR8T9pxgbpQi2VfCDhxlWgK qGW7Xraf+iiXuzz3JkYUUYR2hn+pVdrp6BWY9bfyjeS+3bwwwB4v7009sTqbjfc1cthV UIoLeJJZFUajWaDlhnytMY2utbGTfDJvz2Va6fiQqhMgzb6cU9lXe4wQ1jw1/Iod7jJS yKGo9ruADQoUi2w4Oc2xMYp8dZaIvcpdfV9zQnB7hePCWRgIFZdeu/r6szVIInfDtZRr Jgdc1x35XuQ/SazCoyGBuNVcRyaKBUxzP0cHTjfrlU7a+Pj10IALKhZ777paJlvS/FN2 h+xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q3V6irZLnsZrtYJPX/dX6i+oAV4ziCPw7VoqYDNQ8cw=; b=M+ifh1Yr6QJTgBONdlITn1V/mBsp2FxVjHrz2w8HeKk1vefltQ4pCtx+LFIKoB/kot FZuxoB+ncgeqrnpou/YMu4k6pStZnFIORz6BM+dBeqwc/vj+bPYKu6aks2blKvea6h89 9Yq51rsrDLlCEHVMv1VgDACDafYDIjhYaUKIZ4UAYr7RwNVAllImNgLBL+7kkduLZ+Gx Q6rja3P71rLb+x9qIAoicAzxqcp3M/43LaciPFtUR3JlxLmYK/fGWdu8Z/HblmlHh2TB oyluUL6mQl9jKrow/vzC58JDrwHIJcKTdJ7Sd7f+DD25QlbFgXeHMPnaAy+Qp/Myihf+ dFgA== X-Gm-Message-State: AOAM531oqO0Qwexxdci47vmcD1F3scWZceaL40jV+xtirc1LVoW+ZoRQ /+2msvzCA0e/6wb2pseGgwKbuevLll0V2A== X-Google-Smtp-Source: ABdhPJw9zjsuiuEvxfFy6AqZAbqQ3vzEDgGIb47HJ+/kaBe1sZ52Xwqvi2KbVrJXUBe/ZBTOG6YwQQ== X-Received: by 2002:a05:620a:2eb:: with SMTP id a11mr49904420qko.383.1594136572365; Tue, 07 Jul 2020 08:42:52 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id i35sm25856201qtd.96.2020.07.07.08.42.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:42:51 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 01/23] btrfs: change nr to u64 in btrfs_start_delalloc_roots Date: Tue, 7 Jul 2020 11:42:24 -0400 Message-Id: <20200707154246.52844-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have btrfs_wait_ordered_roots() which takes a u64 for nr, but btrfs_start_delalloc_roots() that takes an int for nr, which makes using them in conjunction, especially for something like (u64)-1, annoying and inconsistent. Fix btrfs_start_delalloc_roots() to take a u64 for nr and adjust start_delalloc_inodes() and it's callers appropriately. This means we've adjusted start_delalloc_inodes() to take a pointer of nr since we want to preserve the ability for start-delalloc_inodes() to return an error, so simply make it do the nr adjusting as necessary. Part of adjusting the callers to this means changing btrfs_writeback_inodes_sb_nr() to take a u64 for items. This may be confusing because it seems unrelated, but the caller of btrfs_writeback_inodes_sb_nr() already passes in a u64, it's just the function variable that needs to be changed. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/inode.c | 27 +++++++++++---------------- fs/btrfs/ioctl.c | 2 +- fs/btrfs/space-info.c | 2 +- 5 files changed, 15 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index e40eb210670d..93cd1ff878cb 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2926,7 +2926,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u32 min_type); int btrfs_start_delalloc_snapshot(struct btrfs_root *root); -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr); +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr); int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end, unsigned int extra_bits, struct extent_state **cached_state); diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index db93909b25e0..769ac3098880 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -630,7 +630,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info, * flush all outstanding I/O and inode extent mappings before the * copy operation is declared as being finished */ - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) { mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return ret; diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index d301550b9c70..b1ca1e3fcc54 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -9394,7 +9394,8 @@ static struct btrfs_delalloc_work *btrfs_alloc_delalloc_work(struct inode *inode * some fairly slow code that needs optimization. This walks the list * of all the inodes with pending delalloc and forces them to disk. */ -static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) +static int start_delalloc_inodes(struct btrfs_root *root, u64 *nr, + bool snapshot) { struct btrfs_inode *binode; struct inode *inode; @@ -9434,9 +9435,11 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) list_add_tail(&work->list, &works); btrfs_queue_work(root->fs_info->flush_workers, &work->work); - ret++; - if (nr != -1 && ret >= nr) - goto out; + if (*nr != U64_MAX) { + (*nr)--; + if (*nr == 0) + goto out; + } cond_resched(); spin_lock(&root->delalloc_lock); } @@ -9461,18 +9464,15 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) int btrfs_start_delalloc_snapshot(struct btrfs_root *root) { struct btrfs_fs_info *fs_info = root->fs_info; - int ret; + u64 nr = U64_MAX; if (test_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state)) return -EROFS; - ret = start_delalloc_inodes(root, -1, true); - if (ret > 0) - ret = 0; - return ret; + return start_delalloc_inodes(root, &nr, true); } -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr) { struct btrfs_root *root; struct list_head splice; @@ -9495,15 +9495,10 @@ int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) &fs_info->delalloc_roots); spin_unlock(&fs_info->delalloc_root_lock); - ret = start_delalloc_inodes(root, nr, false); + ret = start_delalloc_inodes(root, &nr, false); btrfs_put_root(root); if (ret < 0) goto out; - - if (nr != -1) { - nr -= ret; - WARN_ON(nr < 0); - } spin_lock(&fs_info->delalloc_root_lock); } spin_unlock(&fs_info->delalloc_root_lock); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index b3e4c632d80c..0ffc4b2d2627 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -4829,7 +4829,7 @@ long btrfs_ioctl(struct file *file, unsigned int case BTRFS_IOC_SYNC: { int ret; - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) return ret; ret = btrfs_sync_fs(inode->i_sb, 1); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 41ee88633769..571b87cf1c41 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -477,7 +477,7 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, } static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, int nr_items) + unsigned long nr_pages, u64 nr_items) { struct super_block *sb = fs_info->sb; From patchwork Tue Jul 7 15:42:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11648991 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85000912 for ; Tue, 7 Jul 2020 15:42:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E3D020738 for ; Tue, 7 Jul 2020 15:42:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="oW7pS4M1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728769AbgGGPmz (ORCPT ); Tue, 7 Jul 2020 11:42:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPmz (ORCPT ); Tue, 7 Jul 2020 11:42:55 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BE02C061755 for ; Tue, 7 Jul 2020 08:42:55 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id q198so38534965qka.2 for ; Tue, 07 Jul 2020 08:42:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NpLoVxtYuKq3xuBN6YWOovH3J+ch/wI4zzBLssFwCZc=; b=oW7pS4M1Omavo2E7RbFBI91o0X8F3hHGdwpq2dDtgN+sXhNGujqp/VeziDIdd5a01u yBI4JvHk9xo4Yspx5Y6xnFqNjD5riVMLjW8JrYOPL3Lk5Fc0UoXR58FGmLx6VWTlz6Eo ki8l6ci723JMlHUu9xwy/aMalNdCj7WrvOYa5YGAUgJZJTpbIABr7SnNy5vVwHdP1b/+ P0A5J6ozQYbygHaivDBYLGeK9UB3QTrOJDBx+SExQa4zpzrnirdCqpneiEuvhGblmr4k qoXIcBdf5bWVnFpJ+Ct+3596oy48rpsDM06+Ktg/LoOfSmfHsOBZJLK/AJDdY0RCpk+6 abPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NpLoVxtYuKq3xuBN6YWOovH3J+ch/wI4zzBLssFwCZc=; b=Jc3uXsIMcdoQqr2zOuluBNxONJNO/ZtfvmY+vaEHQ92Jl+U+KT3+bDqPkgCzLUdPCy Hw4PKdr3pzkYexrgHPmMQ6nJImZ8MGgEnxOPpu37IGTuYwuBlY85AUKExNgBkG4xaiEW cH4Fe0Dpatg+fjHRZeYXGP1TcdXov/yqX1WzjsPZ9IZELSq11eI3X2qcTe7qPTgWRZDw OIGLI1E56Gjr0FnQR59JaMMEyocoknuIbWq+zHz3Bgrxxqfieu7wX+G7bGFyFNI/FBtq q4FEIDtoybZW8nMMqF3qPqAyJRfZqwS2JQ2U2ZddobQpz3zpzMbUda8yOHs9EgrdfLe7 Wj0A== X-Gm-Message-State: AOAM531DxzB4p4Bes/UIC1xg4+hdu+2vLaWAtGe63aDzAaDFCzz7I4px L3Aw9jCCyC/T0NwyJEAfs9s+cod71ostKA== X-Google-Smtp-Source: ABdhPJxRLulU+0W5ieoSVsv0frkar7rXNCQGdP72UF6yXUEEu81sFqijXn5DRGtxX5NYX5sN2uM5Og== X-Received: by 2002:a37:51c6:: with SMTP id f189mr53133172qkb.339.1594136574186; Tue, 07 Jul 2020 08:42:54 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id q189sm24382921qke.21.2020.07.07.08.42.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:42:53 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 02/23] btrfs: remove orig from shrink_delalloc Date: Tue, 7 Jul 2020 11:42:25 -0400 Message-Id: <20200707154246.52844-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We don't use this anywhere inside of shrink_delalloc, remove it. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 571b87cf1c41..266b9887fda4 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -517,7 +517,7 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, * shrink metadata reservation for delalloc */ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - u64 orig, bool wait_ordered) + bool wait_ordered) { struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; @@ -742,7 +742,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, num_bytes, + shrink_delalloc(fs_info, num_bytes * 2, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Tue Jul 7 15:42:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11648993 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E9E2912 for ; Tue, 7 Jul 2020 15:42:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 66D9320738 for ; Tue, 7 Jul 2020 15:42:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="2BquOguN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728782AbgGGPm5 (ORCPT ); Tue, 7 Jul 2020 11:42:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPm5 (ORCPT ); Tue, 7 Jul 2020 11:42:57 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5AF1C061755 for ; Tue, 7 Jul 2020 08:42:56 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id z63so38490435qkb.8 for ; Tue, 07 Jul 2020 08:42:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o8OvljpuILSWk6dPIuW+DVZm/J9jdzzmI+Wl8q8HkOY=; b=2BquOguNc/naZ89O+MuIPCWRDVHCi3IaABhoCdecqlCgCmdyw5tzXtB+H8JfEcJRop Dk9MF6P6fbiCAdCKryWj6VjjjvHk8sVVsHtgHY1urdOk1qBDIHGKD5C+hqn5o1My+PPD V6RjOVICpnovv8mu8+0eiIQd1guC26LL+Pm6+BPgPCsT9pqW5S+bkMv18rA+H9CbI3Pf u1vTm0FzmiUWoSJcwe6HqIpABjEas96qPMZcQAFEC4yq0GilqGSjzlBjZPD8rc0WraBw AD5MQylNMiJ6C8SPXZv9TFFhyIpP3Y7s+muWWg2h+xCsZzRGgW+PTiWQAwkxfxQSYkyf daOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o8OvljpuILSWk6dPIuW+DVZm/J9jdzzmI+Wl8q8HkOY=; b=kHucHq/aXD9SFbc8FOVCG27Gum1f01krrNqzKH7OSKugJ4TQb1J6/Cs+itDoOuXWlx xkkM47emQTeZJaS4YiLvxSsSGbMR1PHHNzqw8qh04sb/CNfXSf1v+sHavTYsAIcU4FMY NprAvlYjXNzlIAtwiXZ50gFEkOAsxC9qBVujO8furUhSV47k7V6r7pV4HDs9ereIInOf 4544yEgPF7u9SdPPBO2NrqfFXKgkuuOtIKvdVFB9Gn8NvDlFsgMU3SFf6COCpcaHzaxB JYC0UDfOLQHalZYXbvSQNmd5lnWJU4685WJsaY0W1hcfqcTpnsLm67h+Tf7bLkt63WTf 68Jw== X-Gm-Message-State: AOAM530wGYP2GIVaI2FOQD8dq2W86Xf/anmqZFMaiW0XvYMz31nONWSz 915RgWgtpia2JUI/7nf8ZI3LNEJQJUwb2A== X-Google-Smtp-Source: ABdhPJza1IVEMCL4ifrMS+7TQAEM4iN5ESvF0f2QkKRtWfp9XpWR9/3hJYx6zC8aYAzJpucaENjrpQ== X-Received: by 2002:a05:620a:1282:: with SMTP id w2mr50482946qki.196.1594136575848; Tue, 07 Jul 2020 08:42:55 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id u22sm27986992qtb.23.2020.07.07.08.42.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:42:55 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 03/23] btrfs: handle U64_MAX for shrink_delalloc Date: Tue, 7 Jul 2020 11:42:26 -0400 Message-Id: <20200707154246.52844-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data allocations are going to want to pass in U64_MAX for flushing space, adjust shrink_delalloc to handle this properly. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 266b9887fda4..c2a2326f9a79 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -530,8 +530,19 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, int loops; /* Calc the number of the pages we need flush for space reservation */ - items = calc_reclaim_items_nr(fs_info, to_reclaim); - to_reclaim = items * EXTENT_SIZE_PER_ITEM; + if (to_reclaim == U64_MAX) { + items = U64_MAX; + } else { + /* + * to_reclaim is set to however much metadata we need to + * reclaim, but reclaiming that much data doesn't really track + * exactly, so increase the amount to reclaim by 2x in order to + * make sure we're flushing enough delalloc to hopefully reclaim + * some metadata reservations. + */ + items = calc_reclaim_items_nr(fs_info, to_reclaim) * 2; + to_reclaim = items * EXTENT_SIZE_PER_ITEM; + } trans = (struct btrfs_trans_handle *)current->journal_info; space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); @@ -742,7 +753,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, + shrink_delalloc(fs_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Tue Jul 7 15:42:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11648995 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1FBAE6C1 for ; Tue, 7 Jul 2020 15:43:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07B84206F6 for ; Tue, 7 Jul 2020 15:43:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="iv5kMzQs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728802AbgGGPm7 (ORCPT ); Tue, 7 Jul 2020 11:42:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPm6 (ORCPT ); Tue, 7 Jul 2020 11:42:58 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFA47C061755 for ; Tue, 7 Jul 2020 08:42:58 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id x62so32069595qtd.3 for ; Tue, 07 Jul 2020 08:42:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G701bsI8IliaVfeT9eLjM7iHLcC7Fvjl92brNTxJI4s=; b=iv5kMzQs86E40i/JvIrGQWiMFUP7iUfhbMC8jB4lfCanihaW2FA4zR84VZH/T46QIO 22Z5X8rNCpICJgnbM+oi9bLMEWl32nmcQr+ZpVN9gyg+otx2egfg1l5vS9dsrDjThtz6 oxzzW2ev7XoeDkz+KAZjJy5V7VC9dBoduquBEodzejmWDls8u5k/A6d54j50gzuPVnYA VpuujOTlH4ro9rGLx/vpx9e5fkEp82ZPw+9NSc4aMHYQIj/QfCku81sxZn1uPIvhNaaI fyXcxOJR6qJUCYA/vHFzfQIhLd3jvdqPoCofFdzM1eRqQ+mbl33d37cDmOMGniB4PjII LKzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G701bsI8IliaVfeT9eLjM7iHLcC7Fvjl92brNTxJI4s=; b=fcLcCYzoP0WzAOIww3ZmKnmYVJkbISkDd4wtopvFYc/xyLFxhjVwHWJZ9L/WmnJet6 iHTA488JO5RPfg7Nc0nvan9ZcQLtPyWiwPf5q41+QhZ7BajQqPDBaA+LrWmvYSNAhqWQ gek4otx5x95DHuj1ylAPBuX5y4DZvI0WRgZgu6QXcnTiLK9Mh/NRfLEM6tfbxE0cDW+W 5H8RhHqX0YXIikUgmzzRkPS13Jwfj8CwXvcMTlr1VJhiWWTIL0i1xzHUz2+CyTf+bBcL kAHM3jtbXYwjW1bvZc7itYaNfgXhHktDLKFNUNU3jnTS85GpdZTNCmHfcHduxepBTqCa to8Q== X-Gm-Message-State: AOAM53373DuATqDxGcrHtJz/URuFHJs8fISgikEOS1m7Klriuw22GeGR 4ajLianOdE/Y5sW14E8YJ31/i120YrOZVg== X-Google-Smtp-Source: ABdhPJwJHe96ah9j0Hk8NTELcYnJvWWhntJzdI2I8oEUcdOybKWMrx3rWmJj2xei2CFkaBkcH+COfg== X-Received: by 2002:ac8:53c1:: with SMTP id c1mr55815861qtq.193.1594136577630; Tue, 07 Jul 2020 08:42:57 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id x34sm2071940qth.62.2020.07.07.08.42.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:42:57 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 04/23] btrfs: make shrink_delalloc take space_info as an arg Date: Tue, 7 Jul 2020 11:42:27 -0400 Message-Id: <20200707154246.52844-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently shrink_delalloc just looks up the metadata space info, but this won't work if we're trying to reclaim space for data chunks. We get the right space_info we want passed into flush_space, so simply pass that along to shrink_delalloc. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index c2a2326f9a79..dda0e9ec68e3 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -516,10 +516,10 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, /* * shrink metadata reservation for delalloc */ -static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - bool wait_ordered) +static void shrink_delalloc(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + u64 to_reclaim, bool wait_ordered) { - struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; @@ -545,7 +545,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } trans = (struct btrfs_trans_handle *)current->journal_info; - space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); @@ -753,7 +752,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes, + shrink_delalloc(fs_info, space_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Tue Jul 7 15:42:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11648997 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CCC2C6C1 for ; Tue, 7 Jul 2020 15:43:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B3D8D206F6 for ; Tue, 7 Jul 2020 15:43:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="HwEgY5rP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728809AbgGGPnB (ORCPT ); Tue, 7 Jul 2020 11:43:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnA (ORCPT ); Tue, 7 Jul 2020 11:43:00 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56F9CC061755 for ; Tue, 7 Jul 2020 08:43:00 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id di5so14078924qvb.11 for ; Tue, 07 Jul 2020 08:43:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wvZIkows8auYehqDoOck2uZN7Jfa9bG4yskXCxHR+6g=; b=HwEgY5rPoUvKCiOz9tpTZLddEjtoA1d/9GSeGDsUlbKX39urjo9EEdQ9a0/NQE90Np P4+urA92pJfCotzIV9IO/Q/xMaqy6aECzo0Hagtvu21ybd5+eWK3+D6YWRE42hQIg63E z6LweUhFs/8jDfD1u8FPq1SZovFE+OEGelG8Mn1qPbXv4r5DwRlbSA7t6EjXqG0iI6FB bw5G4rUEKB05SQoLrsTh0CLTqCwYOCs4bWp+W6yz5LGXwbtvtL0QszhRy9FTBGac6srY nTEDhUgAZXD8h6oqttyWZKX+HvpJKx+oAY/t2PtYlQeq8DB5ot1UGVlwcWyRgFepQzxa H2Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wvZIkows8auYehqDoOck2uZN7Jfa9bG4yskXCxHR+6g=; b=PI1mdH2yu9agr0TUWjgTO5TLHZSm8OQJzpHunUSEYTF/v4CJOPmA/PoBEtSexcYqtp eRrPIs5u3vKZ6YUUFZ3mEHS4vp+A3Xnz5/4uQBsQhxu4mtjtPkYl7IqWt0algyKWf/B3 6jUXjgPSj29bmWbSkJvixaK4M5PpB2xuz3sIlhEaaN8Xq90QDEMYvEInXdEvDi3It9DN JOr+UUrhRqZ6KprvUom5wKqBIwUeYMXGTR8TIN59acbS8Nn1Qf8K+cYN8OFtzoOnjXME /G89RYlnDJ+/wJFzY6uUks7SAr9qHw1M1TRIus1aqSPcDIu9vAenAg8quz0ztueLks31 OEjA== X-Gm-Message-State: AOAM531oSdItocr0MXrMl2WhyLI1SdtrMXHFSD5WmWB+rkrQMHjrdSFl IO9Hy7+ZXBdVx2odAxGnh8nNsWCoKYSBtw== X-Google-Smtp-Source: ABdhPJzbxhdemk+gJMxiMEMl+CjHQleDohmKEBrM4bWsOGk3eLoqvCN7f7ZGAzY9ep2+zK/VYQiVhw== X-Received: by 2002:a0c:a811:: with SMTP id w17mr17666528qva.164.1594136579248; Tue, 07 Jul 2020 08:42:59 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t9sm24901086qke.68.2020.07.07.08.42.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:42:58 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 05/23] btrfs: make ALLOC_CHUNK use the space info flags Date: Tue, 7 Jul 2020 11:42:28 -0400 Message-Id: <20200707154246.52844-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have traditionally used flush_space() to flush metadata space, so we've been unconditionally using btrfs_metadata_alloc_profile() for our profile to allocate a chunk. However if we're going to use this for data we need to use btrfs_get_alloc_profile() on the space_info we pass in. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index dda0e9ec68e3..3f0e6fa31937 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -777,7 +777,8 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; } ret = btrfs_chunk_alloc(trans, - btrfs_metadata_alloc_profile(fs_info), + btrfs_get_alloc_profile(fs_info, + space_info->flags), (state == ALLOC_CHUNK) ? CHUNK_ALLOC_NO_FORCE : CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); From patchwork Tue Jul 7 15:42:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11648999 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 836ED912 for ; Tue, 7 Jul 2020 15:43:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B114206F6 for ; Tue, 7 Jul 2020 15:43:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="e7GCYwKy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728819AbgGGPnC (ORCPT ); Tue, 7 Jul 2020 11:43:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnC (ORCPT ); Tue, 7 Jul 2020 11:43:02 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DD1CC061755 for ; Tue, 7 Jul 2020 08:43:02 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id b185so27939626qkg.1 for ; Tue, 07 Jul 2020 08:43:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pTMh1XdIBMBbevjWIsHmClI1dbdzDP2/JU+FBcW3VOM=; b=e7GCYwKyx97WB+O03DlWeP0sf2PA7c0KOzzptttLOA6Eqwd6FmsN+oKbpW+rSLttD/ /faiifI8KgAjs5S5NrbdcaWxFhTSMnYrb0k6idPfX36JLkQ4zi7vIhfMowbP+yAwLY27 eke217om93w2wyEICwA1uO9ScSVQ3jzc49VuFigUt3QEQ+rzOfObuxDF9NVkl2odhZAy Fx3xCg4Y1kwwJXMmxzNPA+uNfTrtDN6ms5l2JtDv3hFETll966DMA7gIWyRxloWxBs/X hUoL2WKPFsxDkmfGfKJbrKzbDK+cFIGRTOvGq8ui/zSiaKPCHp+LxQUaAPkIHZyfHNBu Dkng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pTMh1XdIBMBbevjWIsHmClI1dbdzDP2/JU+FBcW3VOM=; b=fmDEi44JY/p5l+/HrGmuOuXqTAqFTQWJxVnxpl7/D3mCYK6JJGx5SgTSMZiMDeWwpG VjVOOvz/cvFIZvir5Kz2Dye5O2NXP8YLdJ44UX7nx6EJwM+RSvtpIdO+qhyLuZHSvjLk teSiek/qIP9nj1Q+WWq6eMJxqh93WeScjdEKdMwx/KDCT+iPwsMoZa8dzuW7kPgvdPUq IQfTkKWTAh/lk4zyhNCaY2geBnoJVBR3bmomIzLOXOqekRHPc4FZBisQHWCgvaIWxzid t4KcNfDQSlowZn7opuOIAgliHrkNaSBnaUvkMIg5YFVTJwr002CY0QRJHwUBFtZfsS6e /oJA== X-Gm-Message-State: AOAM530i2To3ur58IM2hzNKTGhAYIcP2t288zZNLWYCG3FGzDA3XNz/x jptlncmcIXDxcK83bsuKOKGRY+wleuDdCg== X-Google-Smtp-Source: ABdhPJzbsbjWByINZzTpSmwSoETjpi1oSfmF3IdmhDnKbUJR+rTQp2GdvBXkU3l+8BvhjOxoCOKM+g== X-Received: by 2002:a05:620a:958:: with SMTP id w24mr53768226qkw.20.1594136581011; Tue, 07 Jul 2020 08:43:01 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t57sm26732549qtc.91.2020.07.07.08.43.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:00 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 06/23] btrfs: call btrfs_try_granting_tickets when freeing reserved bytes Date: Tue, 7 Jul 2020 11:42:29 -0400 Message-Id: <20200707154246.52844-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We were missing a call to btrfs_try_granting_tickets in btrfs_free_reserved_bytes, so add it to handle the case where we're able to satisfy an allocation because we've freed a pending reservation. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 09b796a081dd..7b04cf4eeff3 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3090,6 +3090,8 @@ void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, if (delalloc) cache->delalloc_bytes -= num_bytes; spin_unlock(&cache->lock); + + btrfs_try_granting_tickets(cache->fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Tue Jul 7 15:42:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649001 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57A14912 for ; Tue, 7 Jul 2020 15:43:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3FCF720738 for ; Tue, 7 Jul 2020 15:43:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="ymtrum+y" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728820AbgGGPnE (ORCPT ); Tue, 7 Jul 2020 11:43:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnE (ORCPT ); Tue, 7 Jul 2020 11:43:04 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB372C061755 for ; Tue, 7 Jul 2020 08:43:03 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id 6so6784212qtt.0 for ; Tue, 07 Jul 2020 08:43:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1eyMNWoi2hC8qlRuAY9xprTphnhdSNxjekwZC/C82Cw=; b=ymtrum+ykktESV3a2hBp2Kt9vWIcLzXc0dAqpvUJJQGyFddqnEQCd3kAXu/DlqEfAd dkPgIIVYKXWNCPX01NtgkBaqqHzu0iLXxfB1T9SgW5U60HLWApyuXXzD/MNDFe59oSpT KeNmd1DHoCbmJu+GVTZlmTDWJIu5T+2118C01jthega2bTyrQKI81f+qZzc7dqv+egmL eBoHQU9/h2hwj6/Mz+k8KiFIEJqbsHf63+Xs/aMETousTMcCwvrTavPPrdonOSChh5la LLDrNYTgKDkaBKG2Pj/k1BzRyT9IXlWsxl9GY+6mX/+VMVS5yhcn+Z/gYO+ts/qIculh AZOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1eyMNWoi2hC8qlRuAY9xprTphnhdSNxjekwZC/C82Cw=; b=TCxuk+oO4/b2NUFRKsp8MljXoAHEEiqpxIosMlRgo4WHMAe6g4khoLNS5B5S0ohMC0 6GzH+i1qywMhG+vyczh+zffHynJytM6F7aAmjwAFnNVdvMHMZEAQz3AHoU7QR/lni9V2 68cXrKx3bIfwoze7YZArGOj893u3UvgibxkypwgF6GArDJ8QoUf2Bh0P3kaLzkgJlYDl pgDr76enS4M1J76ImzO57qtLlml3oAkk9GimwbTOtqlWuM21GkD2ccrCV3mRrd0QQ8gh C1gc6HaC1XGHqtd68aGc2QPh3b9+wZ4BvgLkxObo8IeS5gx6ylfU4T1FJsSJU0Q2dxly sLGA== X-Gm-Message-State: AOAM53055rZOgU+ZW7APnH9QOkTDQpUFrVmDNM+XPy75iNbFDFRczvdk RPnAqDFzJCbD23e37YMUKcJYsRABlle9nw== X-Google-Smtp-Source: ABdhPJx4ZsjfvWJ6N0NZtWAvSzAjEAF/vINHoh82r+kqhXXv7HxUOzJ1GDg23jyfSq17XD2SLElTWA== X-Received: by 2002:ac8:1baf:: with SMTP id z44mr56901328qtj.129.1594136582743; Tue, 07 Jul 2020 08:43:02 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id o145sm24010818qke.2.2020.07.07.08.43.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:02 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 07/23] btrfs: call btrfs_try_granting_tickets when unpinning anything Date: Tue, 7 Jul 2020 11:42:30 -0400 Message-Id: <20200707154246.52844-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When unpinning we were only calling btrfs_try_granting_tickets() if global_rsv->space_info == space_info, which is problematic because we use ticketing for SYSTEM chunks, and want to use it for DATA as well. Fix this by moving this call outside of that if statement. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c0bc35f932bf..4b8c59318f6e 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2844,11 +2844,10 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info, len -= to_add; } spin_unlock(&global_rsv->lock); - /* Add to any tickets we may have */ - if (len) - btrfs_try_granting_tickets(fs_info, - space_info); } + /* Add to any tickets we may have */ + if (!readonly && return_free_space && len) + btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Tue Jul 7 15:42:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649003 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1071A6C1 for ; Tue, 7 Jul 2020 15:43:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED98D20738 for ; Tue, 7 Jul 2020 15:43:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="qQSHzAu0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728828AbgGGPnH (ORCPT ); Tue, 7 Jul 2020 11:43:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnG (ORCPT ); Tue, 7 Jul 2020 11:43:06 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D0BEC061755 for ; Tue, 7 Jul 2020 08:43:06 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id 6so6784347qtt.0 for ; Tue, 07 Jul 2020 08:43:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lSf+3a72PAiNlMwNS/uTdBTwmHFW8V8FvMYLaHzUIks=; b=qQSHzAu0Z9u1rIpgsaLLocTLmxh3eUhk4tNIXeyYS2g+XbqG++UmI9SH4PZKq2ERsh nmNKMxr8t5RYdn2dZmz+qsMJ5LbrIFl5P5GAz/oGqUM2KGJuYz8LSD+IDr73gYTBZLZt eHWIyx1XYqgFvGLJ6Fhn6y6FP9ZlZcXyXOPxHyHu4C2B8Oq5cxWpGItcX+offgXXM8aN 8xe+duFV2JDhwfGntfOgqypsQQvM9cM1gp3FjtH2E4l8nOua972TyMmIcx7/PQu2kHSi Og00Rhb5apPkTGQzU5hPKrgLKlkPr977tNXrPMCEEpV+jO7zdtRGwjtoq5ZzdfEvR9+S 6Yyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lSf+3a72PAiNlMwNS/uTdBTwmHFW8V8FvMYLaHzUIks=; b=PjzaJLO1p/DrgdXzcqxX1gwEE6rjKhEvqgDQKnTx2hYuEyCdRvYoaRLkJkwiURoHHX Z4DJClTVW9D3fS8nGrVBz983UNLLpacc6ny83vLxGES7nG1eEPgM4oAYEoCTRxmj4Jqd QAdk9xlrYJaR2UFlwLBw8iRZvVja9dt1dIUl7L24GkHMtmMCBbD6AxS88aSddd/h+S98 5+QwVlju4N3Q3KZSDdGTS3LAOPxqR46vc/AB89fr9CZoDJ40JOhll24MFqPTftK/OrU7 oBV6bWUajgyXUXs8uaukexRn1p8bpHjViBbL4eZeE6QAtqU+v5KIOU9YTrNA5xw0SW0Q Tc3w== X-Gm-Message-State: AOAM5328RUycAzDejmzelJpaH11+PwQzu+oTZeGJyYZPft2dXg8LJHmV zI2cZwZ2875s4sN5ahr+ZV/rS+NkkUSHWQ== X-Google-Smtp-Source: ABdhPJwoiWLdAhq3L1li4PgjFmIfNY+Vtn/dPkyTElVXvJo+7oq4Yx09LRvfwdkkvoDMUe6S9HF6gg== X-Received: by 2002:ac8:554f:: with SMTP id o15mr56311653qtr.278.1594136585513; Tue, 07 Jul 2020 08:43:05 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id f54sm29637141qte.76.2020.07.07.08.43.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:04 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 08/23] btrfs: call btrfs_try_granting_tickets when reserving space Date: Tue, 7 Jul 2020 11:42:31 -0400 Message-Id: <20200707154246.52844-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org If we have compression on we could free up more space than we reserved, and thus be able to make a space reservation. Add the call for this scenario. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 7b04cf4eeff3..df0ae5ab49bb 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3057,6 +3057,13 @@ int btrfs_add_reserved_bytes(struct btrfs_block_group *cache, space_info, -ram_bytes); if (delalloc) cache->delalloc_bytes += num_bytes; + + /* + * Compression can use less space than we reserved, so wake + * tickets if that happens. + */ + if (num_bytes < ram_bytes) + btrfs_try_granting_tickets(cache->fs_info, space_info); } spin_unlock(&cache->lock); spin_unlock(&space_info->lock); From patchwork Tue Jul 7 15:42:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649005 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBBC3912 for ; Tue, 7 Jul 2020 15:43:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D41F220738 for ; Tue, 7 Jul 2020 15:43:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="rkr2Zf/Q" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728832AbgGGPnK (ORCPT ); Tue, 7 Jul 2020 11:43:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnJ (ORCPT ); Tue, 7 Jul 2020 11:43:09 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92B51C061755 for ; Tue, 7 Jul 2020 08:43:08 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id w34so12024937qte.1 for ; Tue, 07 Jul 2020 08:43:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SuOlF8IpTn7Y/ZwFI/h0PcotCrvJ7Q9POZ2Ys6bf7Po=; b=rkr2Zf/QlddTmfBfb7LGOebR2lh6X585cp+9mr2K06HghZR65Q7OGYSV3vHmyVL+A7 q0EAUxLyHivwItorUE9N8ntbAp3qP4Qt0Q35ydNxnqYq+4CtRW/lDDh5G7S9lJjl5tNj Qv2wd5hOK0aOJNXHKUsjZP3DMpLJf0WxsQhAa9Sy3sTh29xd4SeAhxXDBSZCaQpOnk4U 6Pmhtjty/+0WxNR7orbibOxlJGMnch7lTLTYr1YId+bf9Nte025hhrmy8sVPi3/hNsoQ fJNEYgHXh/1GMEKt6LJK63SXn86NYcxCadPWaemQq9JMaj/1gMJGTzwhW3PN2OfTlkln ji+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SuOlF8IpTn7Y/ZwFI/h0PcotCrvJ7Q9POZ2Ys6bf7Po=; b=LJk7r3LRCpbgoPSHNcqxsyaMwwAoA3EUndvWJ768JGxA892fzdkicLuq2PH/yXxY06 I5EvHxPnECGFsU9Dr/Kxp8JGKhcYEtpaLmjHt/0o1ql/CaZKF6pneR1SbIqTznz9qMDV bqfscE8vz1/zEgxuio2qCnw2RgditKQbJs4Bmd4MlYPiz+GxeLIqvjWGfNWsePb0CHtO Xg4nugPUv1UfQoDe5IVnpWFmKu/BI5M92/MRnkO0sWGYWB0HXTR/t75BDkOMja83KR1P doabfdYQGmOx8gNCMdGlDMJWfpytKbHWjUBxd3si+Zkj6dGu0Ny2ILAorQC5F9oMZeC2 rzbA== X-Gm-Message-State: AOAM5335MlmeJmLQiMdkUYCo71kACJHiestgySiCxEM0PPv4JamoDwCI 8AkttetdZHpC7nhTWpge/DaRWQRRk+2DWQ== X-Google-Smtp-Source: ABdhPJy0nxE1nX9Ra5/NOLjOuRqW7puLS74kfVW5bWGc5L5R/gmj2mnMXGHYi06nLybGzCRjxhmHpA== X-Received: by 2002:ac8:13c9:: with SMTP id i9mr56583234qtj.284.1594136587526; Tue, 07 Jul 2020 08:43:07 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id w44sm28069759qtj.86.2020.07.07.08.43.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:07 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 09/23] btrfs: use the btrfs_space_info_free_bytes_may_use helper for delalloc Date: Tue, 7 Jul 2020 11:42:32 -0400 Message-Id: <20200707154246.52844-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We are going to use the ticket infrastructure for data, so use the btrfs_space_info_free_bytes_may_use() helper in btrfs_free_reserved_data_space_noquota() so we get the try_granting_tickets call when we free our reservation. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index d05648f882ca..fcd75531272b 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -278,9 +278,7 @@ void btrfs_free_reserved_data_space_noquota(struct inode *inode, ASSERT(IS_ALIGNED(len, fs_info->sectorsize)); data_sinfo = fs_info->data_sinfo; - spin_lock(&data_sinfo->lock); - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, -len); - spin_unlock(&data_sinfo->lock); + btrfs_space_info_free_bytes_may_use(fs_info, data_sinfo, len); } /* From patchwork Tue Jul 7 15:42:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649007 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E82356C1 for ; Tue, 7 Jul 2020 15:43:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D089A206F6 for ; Tue, 7 Jul 2020 15:43:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="FWFpRi9a" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728489AbgGGPnL (ORCPT ); Tue, 7 Jul 2020 11:43:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnK (ORCPT ); Tue, 7 Jul 2020 11:43:10 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A6F3C061755 for ; Tue, 7 Jul 2020 08:43:10 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id l6so38544539qkc.6 for ; Tue, 07 Jul 2020 08:43:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VnXtZZSzIs9CL8rkYhMWWfG8VP/VpNFi+WpvwqR/TWE=; b=FWFpRi9auHgbOpdg4/8gu3ammiG5fR7Inj3VOXtdZTnjPf7DVY2/p7+fMRXDvSGcOT TvsuZjuc3w5bik31oDSJkXxGvIWdkdNZDOUl8gt5CH6vwtgg1/GxuxxpTJVLiJwYqPi+ d6DhlmsUjLYnuL2KM2cktxK34L5MVmiLxmXAyliAmfWLXeF9e3HP78klDv41mItyIM9L dqPnobMneQ0lpwkYMQtB4jdGrwTrWulzdLVDbYIXPQivft4vSiesXINkDjIfif3kM+H2 i+c4NCJ3rK+o7f8RKT7mq+6rQq9Lv3nzDWe9yo0QPQYMR57wN3UwRdyurCcYGxfFnrJ2 1xjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VnXtZZSzIs9CL8rkYhMWWfG8VP/VpNFi+WpvwqR/TWE=; b=WwKR+eJN8dphBhz1JpmlCQ90mepDFoQSV4jtDJwOMyufSKOKmEyEfvEi9ZsVGmuAVC JYKBgvTnp5bPUlC3YBxAB8gm/n00TKmx3mW2Z9Vx3YK+ZUazXQClCjuxdE0+rDWtMSKK hJcXCpqzAEazN3otCNyLfhS/E20NkPUqBGL0mrUd/3zGPxgmhaC3E9vHfIWWfnnxEf8/ Y8P3ReRvSUJv4+tauZiqPKrlGwf3CcxRBjGeFc/FovCVTEI9AMuCzb2kyzyPaztJzdnO GbN8Qp1cl8+ev4S6VBbUKSx1CjQ/wRGmg6BKoddbYMOTE0PEqjVAP3QpFZx8K75hmPUF 7EDg== X-Gm-Message-State: AOAM530NGpMkHmPt9qXLMNC6+yGELT1iqpp8Cnl9gZRSDXWT/UB6o4JZ 4bz2uyVyIW35E3NOWu+AdAIXwSAd0lH/FA== X-Google-Smtp-Source: ABdhPJyn31aDJwMXuyum5lK/WF2jjfSxziDQXmslpa/iZU0kDi9fgZ1pdSYx0LmmOs+qq/AdAXwEtA== X-Received: by 2002:a37:689:: with SMTP id 131mr42578968qkg.468.1594136589201; Tue, 07 Jul 2020 08:43:09 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id r6sm16092555qtt.81.2020.07.07.08.43.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:08 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 10/23] btrfs: use btrfs_start_delalloc_roots in shrink_delalloc Date: Tue, 7 Jul 2020 11:42:33 -0400 Message-Id: <20200707154246.52844-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The original iteration of flushing had us flushing delalloc and then checking to see if we could make our reservation, thus we were very careful about how many pages we would flush at once. But now that everything is async and we satisfy tickets as the space becomes available we don't have to keep track of any of this, simply try and flush the number of dirty inodes we may have in order to reclaim space to make our reservation. This cleans up our delalloc flushing significantly. The async_pages stuff is dropped because btrfs_start_delalloc_roots() handles the case that we generate async extents for us, so we no longer require this extra logic. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 55 +------------------------------------------ 1 file changed, 1 insertion(+), 54 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 3f0e6fa31937..1d3af37b0ad0 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -476,28 +476,6 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, up_read(&info->groups_sem); } -static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, u64 nr_items) -{ - struct super_block *sb = fs_info->sb; - - if (down_read_trylock(&sb->s_umount)) { - writeback_inodes_sb_nr(sb, nr_pages, WB_REASON_FS_FREE_SPACE); - up_read(&sb->s_umount); - } else { - /* - * We needn't worry the filesystem going from r/w to r/o though - * we don't acquire ->s_umount mutex, because the filesystem - * should guarantee the delalloc inodes list be empty after - * the filesystem is readonly(all dirty pages are written to - * the disk). - */ - btrfs_start_delalloc_roots(fs_info, nr_items); - if (!current->journal_info) - btrfs_wait_ordered_roots(fs_info, nr_items, 0, (u64)-1); - } -} - static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, u64 to_reclaim) { @@ -523,10 +501,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; - u64 async_pages; u64 items; long time_left; - unsigned long nr_pages; int loops; /* Calc the number of the pages we need flush for space reservation */ @@ -567,37 +543,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, loops = 0; while ((delalloc_bytes || dio_bytes) && loops < 3) { - nr_pages = min(delalloc_bytes, to_reclaim) >> PAGE_SHIFT; - - /* - * Triggers inode writeback for up to nr_pages. This will invoke - * ->writepages callback and trigger delalloc filling - * (btrfs_run_delalloc_range()). - */ - btrfs_writeback_inodes_sb_nr(fs_info, nr_pages, items); - - /* - * We need to wait for the compressed pages to start before - * we continue. - */ - async_pages = atomic_read(&fs_info->async_delalloc_pages); - if (!async_pages) - goto skip_async; - - /* - * Calculate how many compressed pages we want to be written - * before we continue. I.e if there are more async pages than we - * require wait_event will wait until nr_pages are written. - */ - if (async_pages <= nr_pages) - async_pages = 0; - else - async_pages -= nr_pages; + btrfs_start_delalloc_roots(fs_info, items); - wait_event(fs_info->async_submit_wait, - atomic_read(&fs_info->async_delalloc_pages) <= - (int)async_pages); -skip_async: spin_lock(&space_info->lock); if (list_empty(&space_info->tickets) && list_empty(&space_info->priority_tickets)) { From patchwork Tue Jul 7 15:42:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649009 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 786196C1 for ; Tue, 7 Jul 2020 15:43:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60B9F206F6 for ; Tue, 7 Jul 2020 15:43:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="n+36MXH0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728840AbgGGPnM (ORCPT ); Tue, 7 Jul 2020 11:43:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnM (ORCPT ); Tue, 7 Jul 2020 11:43:12 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0843CC061755 for ; Tue, 7 Jul 2020 08:43:12 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id b25so10646604qto.2 for ; Tue, 07 Jul 2020 08:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DuyJL+ch1I9ONsvNvZudfQkXqBUgJDCWj2ulphkQ96w=; b=n+36MXH0PgSk3PfNET7i3so/vIcOM833DeKzDyfvv7rR0KX1L+r/FxFebroisPbBYp rcnskeqebH2/hxQGCmLzhPG/dXAasTk6CTUvoIzoN9JKN8Q6lxJzzUEWEByiFiowK/hV VABjuIRJMLzq1A8XAl+VJMgl+8Zu2f7ij6XreQHnH4bf9dq55vTd7LjS7HBzHdYYnz74 l5HBSGN3Tr0AYZRL5eXpoNVt2eBkTJpR+gFoO5e8I9j6At0/Dm+nYAddLSIvudziNzkE CGn7gsFEc+g8sE/RSGCtLAFkZrFlyvKyaYEViN4PxSsZaA8/DvY5dyHmXoIHQefL4IU3 +duQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DuyJL+ch1I9ONsvNvZudfQkXqBUgJDCWj2ulphkQ96w=; b=S58GyoW7T9KC6YlGA3Sheh5VsJMiuJix0BwbSjejulrXViFABR96514SHQzRVeKdGa jIJsqsdkcyjEXQ0+BuSu8nmXUYUYPGzJv2rBnW+eme/wFZTdQjFdbpoMFWSe+Nfm4qWd Ue7rlkPGtasEbXVY8ag2BF0v8EupjjU1z2Vs4cxjHvKamadIG0FUTlBoo7kMX6qN5C+U OetgH55Em+nnG3iMCgOVJXwq5UfLFSZ8COSldt/DqA1ZfCiD6IqIY+8Z1I2XACwieZp6 rXh4xQjW31payFAEYjKQkc/GXqS6BoKYfHcb0bqUQUhzi7iv6G2/w0+uEhFVW/NlRpu1 GEvg== X-Gm-Message-State: AOAM533eZ8p0CC8nTdlxxheoeHz0YYgajfnT3X2iINeTicDgduM7eXDm 4j0Rg88ZgjdAE+MEFBo+GrFiu9S7TJ1XOg== X-Google-Smtp-Source: ABdhPJw1sENNZzlnYzjAzG3BLFteYVM3C7ZD7w6mvk5Sa3DwW73M9YN+zAYv2n5LJZcM932SBk+CxA== X-Received: by 2002:ac8:24d:: with SMTP id o13mr26639944qtg.154.1594136590936; Tue, 07 Jul 2020 08:43:10 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id a20sm11110316qtw.54.2020.07.07.08.43.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:10 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 11/23] btrfs: check tickets after waiting on ordered extents Date: Tue, 7 Jul 2020 11:42:34 -0400 Message-Id: <20200707154246.52844-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Right now if the space is free'd up after the ordered extents complete (which is likely since the reservations are held until they complete), we would do extra delalloc flushing before we'd notice that we didn't have any more tickets. Fix this by moving the tickets check after our wait_ordered_extents check. Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 1d3af37b0ad0..0b6566a3da61 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -545,14 +545,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, while ((delalloc_bytes || dio_bytes) && loops < 3) { btrfs_start_delalloc_roots(fs_info, items); - spin_lock(&space_info->lock); - if (list_empty(&space_info->tickets) && - list_empty(&space_info->priority_tickets)) { - spin_unlock(&space_info->lock); - break; - } - spin_unlock(&space_info->lock); - loops++; if (wait_ordered && !trans) { btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1); @@ -561,6 +553,15 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, if (time_left) break; } + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets) && + list_empty(&space_info->priority_tickets)) { + spin_unlock(&space_info->lock); + break; + } + spin_unlock(&space_info->lock); + delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes); From patchwork Tue Jul 7 15:42:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649011 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1AFB26C1 for ; Tue, 7 Jul 2020 15:43:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0330C20738 for ; Tue, 7 Jul 2020 15:43:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="veM8OZDD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728849AbgGGPnO (ORCPT ); Tue, 7 Jul 2020 11:43:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728844AbgGGPnN (ORCPT ); Tue, 7 Jul 2020 11:43:13 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5876C061755 for ; Tue, 7 Jul 2020 08:43:13 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id d27so32074073qtg.4 for ; Tue, 07 Jul 2020 08:43:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wOl3r3XCtXoWmA1dQ3UTMbZ8Zu77pCaRE4U/fVaboV8=; b=veM8OZDDxbHJ83QdQrZtBUvdDKDYAzMl3lANMcx23NxFPUt5MvHzD05WamxqbhJghv FuMkrStDjEkVqT72ihu67wXmQol5sQ1DNdnr4csK4fxU5uEq1LHFkeX2VsCppKJhoyKe mv7JPnbhQe1qUG0AoiZO9vYgzi3VQCmcy98sSGnvMD42ZLOzEhpqSuvNtMf5P1Z2686w oNLaL5hj1YThxGxvBRbHVcgGCfQmFhZSpfGtsr7VdoCV+NX7UTo4CkFANxmiVA+xpbAw F3+VQUILzQpsQhr2/+QMA9EbjABpwZL7KwV4mRo0jsUzQ9vFNUb+VupNj8pBh+e4sT3H YZeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wOl3r3XCtXoWmA1dQ3UTMbZ8Zu77pCaRE4U/fVaboV8=; b=IphahFqtHibaYeSJIN3IbmUUyCPSwTa7khI5QovOZYuic3ciwlAFEFcLJCr8ufrxWC 7N+3h5d0J95jgnsw8GsdcHDjMnSdeWM1tWyfPqtZKk6bkI/s8x+dthAxPk42c73VboHo MoceRDxVMXqjVlcjAlY/mzGlIgFDjTMkUuu8V+2cmIwd/ASOiy9ncLsidOGXuKvkcyc4 F5Y5BhG9w4ZvgrHMudfP/1o2B3MOtPRZ39+MV9oKvXw8E0cNNNGZuijxZWiyaSV3hv6v 9sffMlu55uceG+URpgOm3tFNQToyM/auLvjM4ZYg8ejhVte61v8bDLFY/jaL8XQ6f8M9 rlBQ== X-Gm-Message-State: AOAM530tOCKQpls1/qq3jQgbrFKIloOW8dz2Wm1IXZ99tK3Y5oC8r0G2 0ogQlKzwpr2HO3Mqg9n5/b1iCuf3tEiK5Q== X-Google-Smtp-Source: ABdhPJzZvIWbDqegIjOTjwrb4+FFirzHnFcYXm69ksbyd7KLJQABs7tG5sLncFEWF6Jbg5kC+2KK5A== X-Received: by 2002:ac8:1090:: with SMTP id a16mr57530037qtj.181.1594136592628; Tue, 07 Jul 2020 08:43:12 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id a185sm24356091qkg.3.2020.07.07.08.43.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:12 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 12/23] btrfs: add flushing states for handling data reservations Date: Tue, 7 Jul 2020 11:42:35 -0400 Message-Id: <20200707154246.52844-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently the way we do data reservations is by seeing if we have enough space in our space_info. If we do not and we're a normal inode we'll 1) Attempt to force a chunk allocation until we can't anymore. 2) If that fails we'll flush delalloc, then commit the transaction, then run the delayed iputs. If we are a free space inode we're only allowed to force a chunk allocation. In order to use the normal flushing mechanism we need to encode this into a flush state array for normal inodes. Since both will start with allocating chunks until the space info is full there is no need to add this as a flush state, this will be handled specially. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/space-info.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 93cd1ff878cb..efa72a204b91 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2569,6 +2569,8 @@ enum btrfs_reserve_flush_enum { */ BTRFS_RESERVE_FLUSH_LIMIT, BTRFS_RESERVE_FLUSH_EVICT, + BTRFS_RESERVE_FLUSH_DATA, + BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE, BTRFS_RESERVE_FLUSH_ALL, BTRFS_RESERVE_FLUSH_ALL_STEAL, }; diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 0b6566a3da61..e041b1d58e28 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1018,6 +1018,12 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + COMMIT_TRANS, + RUN_DELAYED_IPUTS, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, From patchwork Tue Jul 7 15:42:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649013 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C3A86C1 for ; Tue, 7 Jul 2020 15:43:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0D672206F6 for ; Tue, 7 Jul 2020 15:43:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="AdNJl3Cj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728859AbgGGPnQ (ORCPT ); Tue, 7 Jul 2020 11:43:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnP (ORCPT ); Tue, 7 Jul 2020 11:43:15 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A16FEC061755 for ; Tue, 7 Jul 2020 08:43:15 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id 6so6784814qtt.0 for ; Tue, 07 Jul 2020 08:43:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fi9/qSfeBO4Bgao0/IUMPXS8eqOJwidDnbibJXsQZhg=; b=AdNJl3Cjg60U5ZY22170jeR9JWgzHOcoKNDTedMtOaYxiQCQIlyQfFFm6lD+CIaZWF LZwWznj93xOv4C0e3gqslolWo4b9fv0Jc8Mp+7phGWLOfxpPelK2W28EnsNou9X8lZFO GKaDNFHzr1vtFD8N70GF2+SG5awBhK7tQPqh39jnDx8bA72SjHqM+B+DJLoH895G+5sO q1sYFzu2po/PdOJipy7bQ0qgDfVZ8qMouGQtFyUuVpjK/ZGMm0a4vDJ6E7GHsGlUCyUS KgWntQNikTtGLfhu/CGDHtPndd5TLwKSiU37ISGpRcwY84O6o4bcCkLUtkCmXJDo/Js3 b7ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fi9/qSfeBO4Bgao0/IUMPXS8eqOJwidDnbibJXsQZhg=; b=WgjSFLLGs8+J/rUr5pFYlbDvlReAJhJrbqgORr0yoWoNAemS6guUAaCTg3di4lxBeH AI7q78Ft7d0SoTeUrfpcXiB/FubBE1dw40e96nAsK1raIQFkUAs9cgVX2/NkOl6C2yCz rghazY2aXihhtS2YYdyFJCWd47dK49wIPiT7Ks8hUBHebtc7BOBBa4tICbAz0d4ZGQ1O yNu291fhgqO2gaBlRf1a8ppcCF03PaKK/qe7dekCa829EWdbCN5jjlP98kVYm3Mrkrqv sPLZH33WmXApvTOUa/9AAVF2QqN76mL0uWGtGNm5g5OKrV+CEVk8iGcCPOco33wtzwMR dgig== X-Gm-Message-State: AOAM533tKYj1ezwNCi0SUdm9HJZ+ZjkPji81fPbheIcGTL4HqKnlJrbk J6Ll/qohjh3pvkSoCav2pml8khs8LNs5Ew== X-Google-Smtp-Source: ABdhPJy409905/UqIEePZIzKz+WbpG+zvd8CPUnZGzjsMQN0w2GLiOJw2vmx6f2pApPqZX/4P4vUbw== X-Received: by 2002:ac8:7650:: with SMTP id i16mr11003480qtr.215.1594136594535; Tue, 07 Jul 2020 08:43:14 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id d14sm25403835qti.41.2020.07.07.08.43.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:13 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 13/23] btrfs: add the data transaction commit logic into may_commit_transaction Date: Tue, 7 Jul 2020 11:42:36 -0400 Message-Id: <20200707154246.52844-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data space flushing currently unconditionally commits the transaction twice in a row, and the last time it checks if there's enough pinned extents to satisfy it's reservation before deciding to commit the transaction for the 3rd and final time. Encode this logic into may_commit_transaction(). In the next patch we will pass in U64_MAX for bytes_needed the first two times, and the final time we will pass in the actual bytes we need so the normal logic will apply. This patch exists soley to make the logical changes I will make to the flushing state machine separate to make it easier to bisect any performance related regressions. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index e041b1d58e28..e29c3e318d1b 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -579,21 +579,33 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info) + struct btrfs_space_info *space_info, + u64 bytes_needed) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_block_rsv *trans_rsv = &fs_info->trans_block_rsv; struct btrfs_trans_handle *trans; - u64 bytes_needed; u64 reclaim_bytes = 0; u64 cur_free_bytes = 0; + bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; + /* + * If we are data and have passed in U64_MAX we just want to + * unconditionally commit the transaction to match the previous data + * flushing behavior. + */ + if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && + bytes_needed == U64_MAX) { + do_commit = true; + goto check_pinned; + } + spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -607,7 +619,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : 0; + bytes_needed = (ticket) ? ticket->bytes : bytes_needed; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -618,6 +630,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; +check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -627,7 +640,8 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (do_commit || + test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -635,7 +649,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, /* * See if there is some space in the delayed insertion reservation for - * this reservation. + * this reservation. If the space_info's don't match (like for DATA or + * SYSTEM) then just enospc, reclaiming this space won't recover any + * space to satisfy those reservations. */ if (space_info != delayed_rsv->space_info) goto enospc; @@ -743,7 +759,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info); + ret = may_commit_transaction(fs_info, space_info, num_bytes); break; default: ret = -ENOSPC; From patchwork Tue Jul 7 15:42:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649015 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D223912 for ; Tue, 7 Jul 2020 15:43:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 412C520738 for ; Tue, 7 Jul 2020 15:43:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="NVd3Ti6V" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728870AbgGGPnS (ORCPT ); Tue, 7 Jul 2020 11:43:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726911AbgGGPnR (ORCPT ); Tue, 7 Jul 2020 11:43:17 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DA26C061755 for ; Tue, 7 Jul 2020 08:43:17 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id r22so38505219qke.13 for ; Tue, 07 Jul 2020 08:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w0lNIaPsPy9NuRGlgPnoLHnx+aMFktjaGX1SWCs7fFY=; b=NVd3Ti6VG8prRJwW2pKAeievZ0B7vwQmOg0dvcqL7QrqKKDBvS0Utx39iLz1QKgu/D B3inJE9JVnI/ZtyQoN9R8WZFjcJrV+FCfjW/CUIqUnaXWejeIJPfSrwsgcsysXRG0dvi YQ016CAutnd2QHR/njB64gp4kOQWgzBiejlEgtLa56vSl84tmHaZR4k/d5/q7O27/4im jtdGXW4HsTzgqOY75NGRmogO053YKYcd7Q+gSrKksFqYFNEzXeU9Rdv4+VY8Af03SXJB dv2Xf+wdAtL/WnLCZhPP0evuAA7La4Y5PQVCYRs3zjjGMJjbDSjd5AInrdPIRHExGLxy CmDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w0lNIaPsPy9NuRGlgPnoLHnx+aMFktjaGX1SWCs7fFY=; b=ru5bf4H2Mm184m3FIUXeW9KNtnXtKTYZ4Ky4k99kNrprJJRQCRr0RY+AiF/Pkl7yj/ 1+o+J9N5ra7yzekUDZNSZn0P0OIJ0fEL3DxF2SSIsDtGi9Rsa+U628wTskitMqHoDPfG T3yqC6sPVoj9Gfpd0htDaRgkN8TMwkemqZ4GXdF3kORSASvrMyMcXvkRrSFjxBhf50jI um5RhPxAqn87vNucr1uju9dBgjrLcQvwhwXVRikKvW6k8KzFetPSZKc46nOy7bohce9p Okt8R+JHYx6Ea2uIiarX2UknlnFyW3dx2oNMNopllo0xzC9/rh/njEyEr3pDICwUjtir bfrw== X-Gm-Message-State: AOAM532eeahpgj9kI87magJjA5Lv1cjJojMH8qhbJsignRz//+442bo+ 70kRKQGws+WhRA1pIR0+J771ldc8hoNcvw== X-Google-Smtp-Source: ABdhPJz6FLcW8ftHpJDHvX/0GcedbSfF8Ciko4XZba8cIr7TVBblh3rPP+iGWoF+pLwRge1d4WsXnw== X-Received: by 2002:a37:d02:: with SMTP id 2mr36919235qkn.382.1594136596415; Tue, 07 Jul 2020 08:43:16 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id s24sm27197881qtb.63.2020.07.07.08.43.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:15 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 14/23] btrfs: add btrfs_reserve_data_bytes and use it Date: Tue, 7 Jul 2020 11:42:37 -0400 Message-Id: <20200707154246.52844-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Create a new function btrfs_reserve_data_bytes() in order to handle data reservations. This uses the new flush types and flush states to handle making data reservations. This patch specifically does not change any functionality, and is purposefully not cleaned up in order to make bisection easier for the future patches. The new helper is identical to the old helper in how it handles data reservations. We first try to force a chunk allocation, and then we run through the flush states all at once and in the same order that they were done with the old helper. Subsequent patches will clean this up and change the behavior of the flushing, and it is important to keep those changes separate so we can easily bisect down to the patch that caused the regression, rather than the patch that made us start using the new infrastructure. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 119 ++------------------------------------ fs/btrfs/space-info.c | 92 +++++++++++++++++++++++++++++ fs/btrfs/space-info.h | 2 + 3 files changed, 98 insertions(+), 115 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index fcd75531272b..52e84552d662 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -115,126 +115,15 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) { struct btrfs_root *root = inode->root; struct btrfs_fs_info *fs_info = root->fs_info; - struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = 0; - int need_commit = 2; - int have_pinned_space; + enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_DATA; /* Make sure bytes are sectorsize aligned */ bytes = ALIGN(bytes, fs_info->sectorsize); - if (btrfs_is_free_space_inode(inode)) { - need_commit = 0; - ASSERT(current->journal_info); - } - -again: - /* Make sure we have enough space to handle the data first */ - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - - if (used + bytes > data_sinfo->total_bytes) { - struct btrfs_trans_handle *trans; - - /* - * If we don't have enough free bytes in this space then we need - * to alloc a new chunk. - */ - if (!data_sinfo->full) { - u64 alloc_target; - - data_sinfo->force_alloc = CHUNK_ALLOC_FORCE; - spin_unlock(&data_sinfo->lock); - - alloc_target = btrfs_data_alloc_profile(fs_info); - /* - * It is ugly that we don't call nolock join - * transaction for the free space inode case here. - * But it is safe because we only do the data space - * reservation for the free space cache in the - * transaction context, the common join transaction - * just increase the counter of the current transaction - * handler, doesn't try to acquire the trans_lock of - * the fs. - */ - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - - ret = btrfs_chunk_alloc(trans, alloc_target, - CHUNK_ALLOC_NO_FORCE); - btrfs_end_transaction(trans); - if (ret < 0) { - if (ret != -ENOSPC) - return ret; - else { - have_pinned_space = 1; - goto commit_trans; - } - } - - goto again; - } - - /* - * If we don't have enough pinned space to deal with this - * allocation, and no removed chunk in current transaction, - * don't bother committing the transaction. - */ - have_pinned_space = __percpu_counter_compare( - &data_sinfo->total_bytes_pinned, - used + bytes - data_sinfo->total_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - spin_unlock(&data_sinfo->lock); - - /* Commit the current transaction and try again */ -commit_trans: - if (need_commit) { - need_commit--; - - if (need_commit > 0) { - btrfs_start_delalloc_roots(fs_info, -1); - btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, - (u64)-1); - } - - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - if (have_pinned_space >= 0 || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, - &trans->transaction->flags) || - need_commit > 0) { - ret = btrfs_commit_transaction(trans); - if (ret) - return ret; - /* - * The cleaner kthread might still be doing iput - * operations. Wait for it to finish so that - * more space is released. We don't need to - * explicitly run the delayed iputs here because - * the commit_transaction would have woken up - * the cleaner. - */ - ret = btrfs_wait_on_delayed_iputs(fs_info); - if (ret) - return ret; - goto again; - } else { - btrfs_end_transaction(trans); - } - } - - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", - data_sinfo->flags, bytes, 1); - return -ENOSPC; - } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - spin_unlock(&data_sinfo->lock); + if (btrfs_is_free_space_inode(inode)) + flush = BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE; - return 0; + return btrfs_reserve_data_bytes(fs_info, bytes, flush); } int btrfs_check_data_free_space(struct inode *inode, diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index e29c3e318d1b..f1a9733507a6 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1329,3 +1329,95 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, } return ret; } + +/** + * btrfs_reserve_data_bytes - try to reserve data bytes for an allocation + * @root - the root we are allocating for + * @bytes - the number of bytes we need + * @flush - how we are allowed to flush + * + * This will reserve bytes from the data space info. If there is not enough + * space then we will attempt to flush space as specified ty flush. + */ +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; + const enum btrfs_flush_state *states = NULL; + u64 used; + int states_nr = 0; + int commit_cycles = 2; + int ret = -ENOSPC; + + ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); + + if (flush == BTRFS_RESERVE_FLUSH_DATA) { + states = data_flush_states; + states_nr = ARRAY_SIZE(data_flush_states); + } + + spin_lock(&data_sinfo->lock); +again: + used = btrfs_space_info_used(data_sinfo, true); + + if (used + bytes > data_sinfo->total_bytes) { + u64 prev_total_bytes = data_sinfo->total_bytes; + int flush_state = 0; + + spin_unlock(&data_sinfo->lock); + + /* + * Everybody can force chunk allocation, so try this first to + * see if we can just bail here and make our reservation. + */ + flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); + spin_lock(&data_sinfo->lock); + if (prev_total_bytes < data_sinfo->total_bytes) + goto again; + spin_unlock(&data_sinfo->lock); + + /* + * Cycle through the rest of the flushing options for our flush + * type, then try again. + */ + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + /* + * Previously we unconditionally committed the + * transaction twice before finally checking against + * pinned space before committing the final time. We + * also skipped flushing delalloc the final pass + * through. + */ + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = bytes; + } + + flush_space(fs_info, data_sinfo, flush_bytes, + states[flush_state]); + flush_state++; + } + + if (!commit_cycles) + goto out; + + commit_cycles--; + spin_lock(&data_sinfo->lock); + goto again; + } + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; + spin_unlock(&data_sinfo->lock); +out: + if (ret) + trace_btrfs_space_reservation(fs_info, + "space_info:enospc", + data_sinfo->flags, bytes, 1); + return ret; +} diff --git a/fs/btrfs/space-info.h b/fs/btrfs/space-info.h index c3c64019950a..5646393b928c 100644 --- a/fs/btrfs/space-info.h +++ b/fs/btrfs/space-info.h @@ -149,5 +149,7 @@ static inline void btrfs_space_info_free_bytes_may_use( btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush); #endif /* BTRFS_SPACE_INFO_H */ From patchwork Tue Jul 7 15:42:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17FF2912 for ; Tue, 7 Jul 2020 15:43:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F382120738 for ; Tue, 7 Jul 2020 15:43:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="lR3vPbOq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728880AbgGGPnU (ORCPT ); Tue, 7 Jul 2020 11:43:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728871AbgGGPnT (ORCPT ); Tue, 7 Jul 2020 11:43:19 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72C8BC061755 for ; Tue, 7 Jul 2020 08:43:19 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id t7so19004539qvl.8 for ; Tue, 07 Jul 2020 08:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yuVvSmbvxuhymntrmL400oC1JrktwIyNh8bfQgD1sDI=; b=lR3vPbOq2afZPifxjWJXc/z6iVPPHsJgxqnuSMA37HtC39/jUb84FKPP41NgDUqsnn uE3pZATYJycHeig/YDHcsXwvO1toqrapSzBmfPhHqaaaT11DHdrEdd3/YCGRghtrapaS S2KotLIx6VBSTwRZLYxrrf5c3UC/Yo/F+D5CTOIBItcAqhq4eENDaTc6hBr89v6Dz5MH Q6OlAo3q4u71ez/cD76EJMV9jme1584sXssLVhwx7ooKZA6wMQHLkYr5c6vhOpjeGZuv cG9zzp2CCqWpQEeZRoI9xPKDQD444sfYxsu+o1niDbAMMcAIBsc9tR2BpXKYcSfCHAPS G8mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yuVvSmbvxuhymntrmL400oC1JrktwIyNh8bfQgD1sDI=; b=sw4WOQKj/K9pCsExLWp+uRxlDRnJ9N4uI1mYu8WMEZ5IN2tyUfll32ay1BborLBkh/ cy/zd1w82bnBYeYiO2DHNlzocZIichHBbuzCYgVMHz1Ffsa+tMMDXkQnSahnO7jcJtFF ftpd6GaAr0VGgfWxoilh5/i2O6mepp2Uhk4zi5jlSSR1kNu8rkyIg5GVLDWRj15QAhC+ LF5QzW8FbqigJd6MMiUV4GeQaPebSk3zRYjuTVzTD1rjWefTzOYLOgqM+72UOxOShuN5 dTNZ6Ew8ralapM1CAQ+KHA+4Qq7XKXUBFvbrSmgIyPJmEONoD8La8nROQIT2k9r8KOxg kzag== X-Gm-Message-State: AOAM531YbAG9PvWW2P+WytQXWEZQRI3237Ia7pfkwohPEZoydgLElRfK 4mc4mEHXyRgvx7TH+YY3SME1DWG3bgv7Qw== X-Google-Smtp-Source: ABdhPJy+z/35XlfuAMEAKbFF/qaCvxGkt5ZJO3XMZhv9izVxIegTXN5lpuWe2TTUuWfjOUnRSGx+9Q== X-Received: by 2002:ad4:588f:: with SMTP id dz15mr23863179qvb.140.1594136598262; Tue, 07 Jul 2020 08:43:18 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id o187sm23903889qke.76.2020.07.07.08.43.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:17 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 15/23] btrfs: use ticketing for data space reservations Date: Tue, 7 Jul 2020 11:42:38 -0400 Message-Id: <20200707154246.52844-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have all the infrastructure in place, use the ticketing infrastructure to make data allocations. This still maintains the exact same flushing behavior, but now we're using tickets to get our reservations satisfied. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 126 +++++++++++++++++++++++------------------- 1 file changed, 68 insertions(+), 58 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index f1a9733507a6..7fdd571a6031 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1070,6 +1070,54 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, } while (flush_state < states_nr); } +static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + struct reserve_ticket *ticket, + const enum btrfs_flush_state *states, + int states_nr) +{ + int flush_state = 0; + int commit_cycles = 2; + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + } +again: + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = ticket->bytes; + } + + flush_space(fs_info, space_info, flush_bytes, + states[flush_state]); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + flush_state++; + } + if (commit_cycles) { + commit_cycles--; + flush_state = 0; + goto again; + } +} + static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) @@ -1136,6 +1184,15 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; + case BTRFS_RESERVE_FLUSH_DATA: + priority_reclaim_data_space(fs_info, space_info, ticket, + data_flush_states, + ARRAY_SIZE(data_flush_states)); + break; + case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: + priority_reclaim_data_space(fs_info, space_info, ticket, + NULL, 0); + break; default: ASSERT(0); break; @@ -1343,78 +1400,31 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - const enum btrfs_flush_state *states = NULL; u64 used; - int states_nr = 0; - int commit_cycles = 2; int ret = -ENOSPC; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - if (flush == BTRFS_RESERVE_FLUSH_DATA) { - states = data_flush_states; - states_nr = ARRAY_SIZE(data_flush_states); - } - spin_lock(&data_sinfo->lock); -again: used = btrfs_space_info_used(data_sinfo, true); if (used + bytes > data_sinfo->total_bytes) { - u64 prev_total_bytes = data_sinfo->total_bytes; - int flush_state = 0; + struct reserve_ticket ticket; + init_waitqueue_head(&ticket.wait); + ticket.bytes = bytes; + ticket.error = 0; + list_add_tail(&ticket.list, &data_sinfo->priority_tickets); + data_sinfo->reclaim_size += bytes; spin_unlock(&data_sinfo->lock); - /* - * Everybody can force chunk allocation, so try this first to - * see if we can just bail here and make our reservation. - */ - flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); - spin_lock(&data_sinfo->lock); - if (prev_total_bytes < data_sinfo->total_bytes) - goto again; + ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, + flush); + } else { + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; spin_unlock(&data_sinfo->lock); - - /* - * Cycle through the rest of the flushing options for our flush - * type, then try again. - */ - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - /* - * Previously we unconditionally committed the - * transaction twice before finally checking against - * pinned space before committing the final time. We - * also skipped flushing delalloc the final pass - * through. - */ - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = bytes; - } - - flush_space(fs_info, data_sinfo, flush_bytes, - states[flush_state]); - flush_state++; - } - - if (!commit_cycles) - goto out; - - commit_cycles--; - spin_lock(&data_sinfo->lock); - goto again; } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); -out: if (ret) trace_btrfs_space_reservation(fs_info, "space_info:enospc", From patchwork Tue Jul 7 15:42:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15E17912 for ; Tue, 7 Jul 2020 15:43:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED6C120738 for ; Tue, 7 Jul 2020 15:43:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="s4gl3/8G" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728912AbgGGPnX (ORCPT ); Tue, 7 Jul 2020 11:43:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPnW (ORCPT ); Tue, 7 Jul 2020 11:43:22 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AE85C061755 for ; Tue, 7 Jul 2020 08:43:22 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id j80so38546403qke.0 for ; Tue, 07 Jul 2020 08:43:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G9f1vvaCUtFJNzvH+33kN/DhJJU7is1uTY6EG9KbLBM=; b=s4gl3/8GbiJMhh22bGpZW7Dd7kUbw5pcsFiRy5eZ23AREjdolTg6RoJhGKeB7NFB5K i45q0l2KWVA4f90zTCFOUTFHrNCbD9/VxyzzZUFQzX+PWr7+3ITnDKUjaYd1+3u7ZeBS ZvI08Q+ChMa9svgj2LhOsNQ1Wc0zTGZ/Cvh1zRNj57r+sZ7SaET/tkhlaKDimrJ/j10z GIg2uHb4NiQfCocRLtT7LJgdpBXddPP1Q9Y2hkaca0E4A5Ad7l9gfhcG8TQwadnh1hYj E7jqTWIlP1mX/XWg536gU8v9qLIn6NBdiu+qSpcIhLdvq5CrjtdxkAuZ6Sza/ttwxJVg RUeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G9f1vvaCUtFJNzvH+33kN/DhJJU7is1uTY6EG9KbLBM=; b=T7SHzR76MOXm2Oxucy2zVuGMwTk9MkQz8fDt+FdrbC5eXKR96gK/LA+5HFEWG+DffI 0ydHBPsPx/mdHMpFgzbtAyqmq31efsKOo25qnYDDuPS2kfwHxyJjlasbVM6sy/VvAE3L KKfwiSPVOaZkigcBSiGKjyT5Th7zJ7yJSB94BU9D7yNCIO2FluE2Nq26YztJGZCeOLaG HCDqi5quRzzKACeXSG7LQrT6/uLFIoOo1eQNvqtKWmiFEMbQ7eLQltHtojrthK/UTK74 wdrzXw+WSOJCwpdLL787cL31fx6qHuSGgDppLwJTdhEpDIOarTjYFL1WWCiZyZHFDQHw pCTg== X-Gm-Message-State: AOAM532Y1qvVdb2AvlZ5ZuDeuCFY3X8sqhnLEnI20b/1ghHUBDk9bm8+ CfqONvJw1WrddutxbEP/eAe42m0U/beZ5Q== X-Google-Smtp-Source: ABdhPJyHUu48ZTBV46bHvxewGwuA2CKQ03GLd5b+yUkustcbQLjgETz2UbylzKKgG2MUUJhNxcfgTA== X-Received: by 2002:a05:620a:7ea:: with SMTP id k10mr51836498qkk.418.1594136601069; Tue, 07 Jul 2020 08:43:21 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id g17sm18309815qto.73.2020.07.07.08.43.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:20 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 16/23] btrfs: serialize data reservations if we are flushing Date: Tue, 7 Jul 2020 11:42:39 -0400 Message-Id: <20200707154246.52844-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Nikolay reported a problem where generic/371 would fail sometimes with a slow drive. The gist of the test is that we fallocate a file in parallel with a pwrite of a different file. These two files combined are smaller than the file system, but sometimes the pwrite would ENOSPC. A fair bit of investigation uncovered the fact that the fallocate workload was racing in and grabbing the free space that the pwrite workload was trying to free up so it could make its own reservation. After a few loops of this eventually the pwrite workload would error out with an ENOSPC. We've had the same problem with metadata as well, and we serialized all metadata allocations to satisfy this problem. This wasn't usually a problem with data because data reservations are more straightforward, but obviously could still happen. Fix this by not allowing reservations to occur if there are any pending tickets waiting to be satisfied on the space info. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 7fdd571a6031..abb57ab6bfc9 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1402,13 +1402,17 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; u64 used; int ret = -ENOSPC; + bool pending_tickets; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); spin_lock(&data_sinfo->lock); used = btrfs_space_info_used(data_sinfo, true); + pending_tickets = !list_empty(&data_sinfo->tickets) || + !list_empty(&data_sinfo->priority_tickets); - if (used + bytes > data_sinfo->total_bytes) { + if (pending_tickets || + used + bytes > data_sinfo->total_bytes) { struct reserve_ticket ticket; init_waitqueue_head(&ticket.wait); From patchwork Tue Jul 7 15:42:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB824912 for ; Tue, 7 Jul 2020 15:43:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B08DC20738 for ; Tue, 7 Jul 2020 15:43:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="UGxLZhWB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728920AbgGGPnY (ORCPT ); Tue, 7 Jul 2020 11:43:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPnY (ORCPT ); Tue, 7 Jul 2020 11:43:24 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D8FB5C061755 for ; Tue, 7 Jul 2020 08:43:23 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id b4so38491776qkn.11 for ; Tue, 07 Jul 2020 08:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JkWrdQ70XFawvSa4oKhSlL1L1YJE6jcK1mafGy9eKn0=; b=UGxLZhWByk0yQP4rzXJNKLTVjRRCbQNOPIHhrk7+hm41N5irzWiNCfYvfgSobGLXxc jnLYqjMpUyCEbJLwo+Ko3Vpaz3Oc9IU+RKWEbt1hIwu2VKoirNUjvQ26hrCsWNvgfarl T9NbsVjkUt8NkxoYUU2jpATWcKcaZCzlG5HbXB0abFRJ7GEr1cKKL/ucO11m7f2S0P+F PmSdnEvbrWyWAZAcXcFIHogjg+NU6vUPl3aJdvuQORpyV8Jz4GnAk/4KP69Pm+6GUbfe hzXLPShwfMYrL5DDmGYe5/LGZWFDLpdmQ9EAaj5gd0iYjCsiZ0tmPfiZgrebmAi58xRc I2sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JkWrdQ70XFawvSa4oKhSlL1L1YJE6jcK1mafGy9eKn0=; b=jQEjJz0KVvMHruEGY2v9lc+pkQSe55uvuNMWe8+NdERqh2Rlm1o5m6R+xM6U6MzimD OdxPfo0WCpsHXHAv6qT4hZ7D3Crti7r8fHILVkxOkgsK1HZGgszLiQipubqoP/whDnKn KCHS+QxH9lztZYVS1edc8bGylVALLykCb/T+ShoxA5fZ2dgQVXIbwuhJNZrvFiAEjQqK nZmnMjwG4f0q8TeCW3na2nm+Vmb0ALiUBS/gaXz/f47MzXhaoSxKW28nOqh3NqGB8b2U l1aU5ChqC9+N/OlCWX0vG6VKgiqu6ETAcx+JLhqYlvTPuc4IbDGqsomC2sOnX98FKtKJ 6E3g== X-Gm-Message-State: AOAM531p0XcmAX6PnLWxkP2TakVOaSl/AkURATfxiT3upHYng0v8C4e+ f9b4moaCw7U+HNVNPawzAoDXdsKb/gfowg== X-Google-Smtp-Source: ABdhPJyEl3IJLDlVNRzFej0Ax7hVoyVt1ux5Fcv7rfkPf1HP/KRwGD9W4sOQsW62Gk/kF4i+47/wXA== X-Received: by 2002:a37:7a42:: with SMTP id v63mr54081535qkc.258.1594136602753; Tue, 07 Jul 2020 08:43:22 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id a6sm24045111qkd.69.2020.07.07.08.43.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:22 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 17/23] btrfs: use the same helper for data and metadata reservations Date: Tue, 7 Jul 2020 11:42:40 -0400 Message-Id: <20200707154246.52844-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that data reservations follow the same pattern as metadata reservations we can simply rename __reserve_metadata_bytes to __reserve_bytes and use that helper for data reservations. Things to keep in mind, btrfs_can_overcommit() returns 0 for data, because we can never overcommit. We also will never pass in FLUSH_ALL for data, so we'll simply be added to the priority list and go straight into handle_reserve_ticket. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 49 +++++++++++++------------------------------ 1 file changed, 14 insertions(+), 35 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index abb57ab6bfc9..1e070cd485bc 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1252,10 +1252,9 @@ static inline bool is_normal_flushing(enum btrfs_reserve_flush_enum flush) * regain reservations will be made and this will fail if there is not enough * space already. */ -static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 orig_bytes, - enum btrfs_reserve_flush_enum flush) +static int __reserve_bytes(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, u64 orig_bytes, + enum btrfs_reserve_flush_enum flush) { struct reserve_ticket ticket; u64 used; @@ -1367,8 +1366,8 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; int ret; - ret = __reserve_metadata_bytes(fs_info, block_rsv->space_info, - orig_bytes, flush); + ret = __reserve_bytes(fs_info, block_rsv->space_info, orig_bytes, + flush); if (ret == -ENOSPC && unlikely(root->orphan_cleanup_state == ORPHAN_CLEANUP_STARTED)) { if (block_rsv != global_rsv && @@ -1400,38 +1399,18 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = -ENOSPC; - bool pending_tickets; + int ret; + ASSERT(flush == BTRFS_RESERVE_FLUSH_DATA || + flush == BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - pending_tickets = !list_empty(&data_sinfo->tickets) || - !list_empty(&data_sinfo->priority_tickets); - - if (pending_tickets || - used + bytes > data_sinfo->total_bytes) { - struct reserve_ticket ticket; - - init_waitqueue_head(&ticket.wait); - ticket.bytes = bytes; - ticket.error = 0; - list_add_tail(&ticket.list, &data_sinfo->priority_tickets); - data_sinfo->reclaim_size += bytes; - spin_unlock(&data_sinfo->lock); - - ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, - flush); - } else { - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); - } - if (ret) - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", + ret = __reserve_bytes(fs_info, data_sinfo, bytes, flush); + if (ret == -ENOSPC) { + trace_btrfs_space_reservation(fs_info, "space_info:enospc", data_sinfo->flags, bytes, 1); + if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) + btrfs_dump_space_info(fs_info, data_sinfo, bytes, 0); + } return ret; } From patchwork Tue Jul 7 15:42:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38DBC6C1 for ; Tue, 7 Jul 2020 15:43:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 20A2820738 for ; Tue, 7 Jul 2020 15:43:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="UGJONAHv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728928AbgGGPn0 (ORCPT ); Tue, 7 Jul 2020 11:43:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPnZ (ORCPT ); Tue, 7 Jul 2020 11:43:25 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89DC8C061755 for ; Tue, 7 Jul 2020 08:43:25 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id 80so38546261qko.7 for ; Tue, 07 Jul 2020 08:43:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2XLMhupLEmPrkQFbBbB2TlTxqawyDNYaqNQAW9M6vys=; b=UGJONAHve+yR7qhKhBo7y3IrcELz8R362igmAVCqccaXexRx9i+cR4eTfVjGDIoMKd 0PBiAI+Y7Kc65wr8NfLWeuAF4YuQiiceAH0UXiJVVV2Sj00xNVVPctnELmwQdS25cjxK ygjaOkiFhnj41mc5g/bvCc2ai0OLmDU4RJGqO6l2GdDl+2JnlklhZpVwtvaXLhy84mHM z63K8XFNDDgbqF3CCpmEoFKpAgL2COmosVQYUqBQG1fWM+LUyyoHH5I+ZKAodtDM7vmG FXyO4wps7mrVn16s59wiZZ/H5iNADwUVXPKs53sGV2gJeiirnVRZZYT9j8mp62TlXBDz K4Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2XLMhupLEmPrkQFbBbB2TlTxqawyDNYaqNQAW9M6vys=; b=OuGI1puQTdEWwOSo33ituHXaYK6MxvSZeQNdHi6oZirKCifJpTEzxoHC2lihl2Tp+X C7xVdL837i04owLR+yntgHO39BM4JFoznDsBBHBRG3prfC79nGG3gLOXwqt/5NK+owd8 NCVAJ5W0mmVyekBfspKpe5gwJpUQz+byXi+/bNvtM1IoG2xGP5f5Uhi2TcKbJziY8DAI 5NovuNCfW+Tpt7Iux2goWvIoiab41QI1bsIp+MyEqqMHFIoDi6q0uAu31kVV0tcl465I jvQVi/us10QBGRYP8TnDM7zVQBt4dVlSyPECsb7M/3b7yFYnudFxCU2zuZKzgNSKastV PCvQ== X-Gm-Message-State: AOAM533OAmq7A38EBsm4XAzxlpV6kGDEAYnJPE3NzOxpo4vCQN1zusMs CsaxbQbEFRVYj/6IxeF4LQjkyzSgVfMT1g== X-Google-Smtp-Source: ABdhPJwt/cvBnRoBHWR1oAEMXzaGHfsD2JM7JXIbu+nCQISkY86YlbjWCLb0Llo6qUIfN+AjSaFHxA== X-Received: by 2002:ae9:e519:: with SMTP id w25mr52923258qkf.327.1594136604436; Tue, 07 Jul 2020 08:43:24 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id j24sm22617309qkl.79.2020.07.07.08.43.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:23 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 18/23] btrfs: drop the commit_cycles stuff for data reservations Date: Tue, 7 Jul 2020 11:42:41 -0400 Message-Id: <20200707154246.52844-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This was an old wart left over from how we previously did data reservations. Before we could have people race in and take a reservation while we were flushing space, so we needed to make sure we looped a few times before giving up. Now that we're using the ticketing infrastructure we don't have to worry about this and can drop the logic altogether. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 1e070cd485bc..59d0ad1736e8 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1077,7 +1077,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, int states_nr) { int flush_state = 0; - int commit_cycles = 2; while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); @@ -1088,21 +1087,9 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } -again: - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = ticket->bytes; - } - flush_space(fs_info, space_info, flush_bytes, - states[flush_state]); + while (flush_state < states_nr) { + flush_space(fs_info, space_info, U64_MAX, states[flush_state]); spin_lock(&space_info->lock); if (ticket->bytes == 0) { spin_unlock(&space_info->lock); @@ -1111,11 +1098,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); flush_state++; } - if (commit_cycles) { - commit_cycles--; - flush_state = 0; - goto again; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Tue Jul 7 15:42:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D827912 for ; Tue, 7 Jul 2020 15:43:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 056AA2075B for ; Tue, 7 Jul 2020 15:43:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="CsAeHVMI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728931AbgGGPn2 (ORCPT ); Tue, 7 Jul 2020 11:43:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPn1 (ORCPT ); Tue, 7 Jul 2020 11:43:27 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E417C061755 for ; Tue, 7 Jul 2020 08:43:27 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id h18so19017604qvl.3 for ; Tue, 07 Jul 2020 08:43:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zlHWUUd+eF/vwrtsTrHXFQe7nd+UYBasDoIwJfuWDlw=; b=CsAeHVMImhm2IVUfHkxxfh5pJLKoJq8cRpc0fiySnfXdnYow9KBCLRqTtsZHuldopT 8ZkdDG5urptTV4a6LXrvy0KLaFxBz1Crm91SwJUl3kNFN/4aHCWVU5TTnAoD316kkXeJ nGkl89IVjjK7LAtL3Yb/gYmHx47N8Q5lb0ODMd57qHX5+nW7wy+I4Iz9VFDC+EQ/Ab7q Rblw2FT+Y8BQ7YbLOIKgbzFpw2vH2OTcMkppFlEHB7yM2/4TF76RK8awB/bizCTgKr/p R3Vub/nyb9S3mL0IRIgxuEMgHlBs89tFXdQCnWkwExzKjC/w3/K/iSfDRivgpSE6RfNW 1+Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zlHWUUd+eF/vwrtsTrHXFQe7nd+UYBasDoIwJfuWDlw=; b=ij2ohLC/QRTDmSXurbiwbPvMRkpzNywKHq//UssJEf0LAyFndd0gdnnjIwmlvacSa3 IVK267YUdF+Sk6O+5+4g9bxhuXdRbLOwEcqBvCLcQcMuiQeGB0742U/XI/LPoThsirkV bAamP5yhnu2LN+0GVIPx4zEQkriwiv+y/pH8cG+lKMgadOb5Fg6J8qzg+0GEiVKCYxwR OpLQz14YmfGUDcWeyblIPePklvaNyowzauWL97bKfUHZG1gFD8a1xTmJFP3IZUTs8iMH tCQ42RuiJfDEufTxsqmXRxeTbsgQFpv/JMSNBEGOUBxviPgaTJYaWK1CmCa7JT3BbpLs Ahcw== X-Gm-Message-State: AOAM532g/XYbshIO/wqidy24zx9q7V6GHqdAUIXz1yGh3UFCVS4JqVSb tVuU7wEbS0w7fTGcq85qWhG1Q+lvJ7ww9w== X-Google-Smtp-Source: ABdhPJy1extF9AAPx9+kLFDZmHsUo6vIFax3fouLGwUNhGls70xneGbAF71hEaNjNYM9wPy61e2CRw== X-Received: by 2002:a0c:f214:: with SMTP id h20mr51611381qvk.131.1594136606434; Tue, 07 Jul 2020 08:43:26 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id h16sm22303850qkl.96.2020.07.07.08.43.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:25 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 19/23] btrfs: don't force commit if we are data Date: Tue, 7 Jul 2020 11:42:42 -0400 Message-Id: <20200707154246.52844-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We used to unconditionally commit the transaction at least 2 times and then on the 3rd try check against pinned space to make sure committing the transaction was worth the effort. This is overkill, we know nobody is going to steal our reservation, and if we can't make our reservation with the pinned amount simply bail out. This also cleans up the passing of bytes_needed to may_commit_transaction, as that was the thing we added into place in order to accomplish this behavior. We no longer need it so remove that mess. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 59d0ad1736e8..8df0110bd97a 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -579,8 +579,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 bytes_needed) + struct btrfs_space_info *space_info) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; @@ -588,24 +587,13 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *trans_rsv = &fs_info->trans_block_rsv; struct btrfs_trans_handle *trans; u64 reclaim_bytes = 0; + u64 bytes_needed; u64 cur_free_bytes = 0; - bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; - /* - * If we are data and have passed in U64_MAX we just want to - * unconditionally commit the transaction to match the previous data - * flushing behavior. - */ - if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && - bytes_needed == U64_MAX) { - do_commit = true; - goto check_pinned; - } - spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -619,7 +607,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : bytes_needed; + bytes_needed = (ticket) ? ticket->bytes : 0; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -630,7 +618,6 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; -check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -640,8 +627,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (do_commit || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -759,7 +745,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info, num_bytes); + ret = may_commit_transaction(fs_info, space_info); break; default: ret = -ENOSPC; From patchwork Tue Jul 7 15:42:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21EBB912 for ; Tue, 7 Jul 2020 15:43:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09D8820738 for ; Tue, 7 Jul 2020 15:43:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="z0uzSsmW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728940AbgGGPna (ORCPT ); Tue, 7 Jul 2020 11:43:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPn3 (ORCPT ); Tue, 7 Jul 2020 11:43:29 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53719C061755 for ; Tue, 7 Jul 2020 08:43:29 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id a14so19013590qvq.6 for ; Tue, 07 Jul 2020 08:43:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VikI5MVVaTSxTXtFs/088Kv1GEt/DSUSVPLJNrvdXhc=; b=z0uzSsmWenHIe5xVwZEvfDEWnUTAuDY9tEbF2HiS4Q8wQWLGLGZyDfPB6bh0pF2VQE 3kEHKwa+tODPaxwcQn/3wTI+5Loa8nPh8OY3SZiASRdN+ZecgDDlpFuBOwF1/SXCAuUy AZz98TmdvnpEYPnUcI07wW0dQoPgMC/CZy7WPpePAP4aF++45xU0NbwuCvDHdNq8Tnrf oRvaf96V4fVKGejEFJuxEpgVoSEqgu/E3fEIiBMgVVx7QPVswI5dQyJ2QkFieGWtuMDu CFqaxcao7eXWfRU2vBtIi3FC9cDM8d9pf6V+ClcFvwhBoKblIoDknLXXltNppYAH+BfR lJNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VikI5MVVaTSxTXtFs/088Kv1GEt/DSUSVPLJNrvdXhc=; b=llkClpRha/lvNEj9eGnI4uu0/TcBHD7gVVlXWYZ5XpJzNkBCJBf/3mswj8vyqd09rm TYiB61w0psR7EBoYXrcZqadMV0ETP8dDeQrwx2ARqg/kDTCZGmV/Tcs5OJg4OyCo0lTl qcnhXWoIH4uW7hzJ0gVPB2t3KKSC03A21bV1QBjadIhkdyIHJ5dWzdg8TG1vaKEAYk0P QtNf3U0YvoItkTkZzzFBhHCusLucC3Da23iakbul2MLpPNDjxXKcKfJaus/dHWr455u9 N3YtHQlWtrPtybzCznUB+HIq+cE5duUclKfzFPvR+lrgUII0ln+6k6WHZGScJk6VVCIh 3Wyg== X-Gm-Message-State: AOAM5333CWJ/KIuorZKZ67Y4NLNBobagBTN/dx/JrxP4xTTXCAiARhpg fUZBvZDSzf7JMkzcld5683LO2CEkFY1AEw== X-Google-Smtp-Source: ABdhPJzk5JjV4BYHaTg8dFiuwC0ymyS2ZmruQqarVgDE2g4+emhBiGxGVogEfiYhrvto9b4OePsh/Q== X-Received: by 2002:ad4:4672:: with SMTP id z18mr53962616qvv.104.1594136608166; Tue, 07 Jul 2020 08:43:28 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id p186sm24374454qkf.33.2020.07.07.08.43.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:27 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 20/23] btrfs: run delayed iputs before committing the transaction for data Date: Tue, 7 Jul 2020 11:42:43 -0400 Message-Id: <20200707154246.52844-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Before we were waiting on iputs after we committed the transaction, but this doesn't really make much sense. We want to reclaim any space we may have in order to be more likely to commit the transaction, due to pinned space being added by running the delayed iputs. Fix this by making delayed iputs run before committing the transaction. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 8df0110bd97a..5a555f20ca20 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1022,8 +1022,8 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, - COMMIT_TRANS, RUN_DELAYED_IPUTS, + COMMIT_TRANS, }; static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, From patchwork Tue Jul 7 15:42:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C98FE6C1 for ; Tue, 7 Jul 2020 15:43:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B225C2075B for ; Tue, 7 Jul 2020 15:43:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="KkJ4n0ke" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728966AbgGGPnc (ORCPT ); Tue, 7 Jul 2020 11:43:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35732 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPnb (ORCPT ); Tue, 7 Jul 2020 11:43:31 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0137EC061755 for ; Tue, 7 Jul 2020 08:43:31 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id ed14so8508537qvb.2 for ; Tue, 07 Jul 2020 08:43:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w+sX17+UelazVGYPbbR1IqUU0+Cr26BH5FHiEiR34no=; b=KkJ4n0kecBTpBowW9VJHfGuaQahDuG2w28fGcmV8zdShkKqgfchmyeFB3jA+pzuqUH /t7we8SBJRjwNrFqe+xR8kRpgvgW0aPYQkPWOv0G9NzgIEn9Uha4KTG4HlS57FGBr3Hj zwugpXu2FM9Db9nuqzzl/kFDIkatElhFkpABecku7+RSRp9Uz/TrLy/EZ0GPVtcpZB5D YlzBOJBbxqeLaJc/srSo4/O+tiqeSps/H3nzOz3pf4ukK3odjIsssgRiSgfvJ8EJPyMH cbJqGlNImAawMLjbqdmuglYu8eLxJU3yAITQyN8F+DgEuTdRiPCkvREjhWySKX5TffHK 6Dkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w+sX17+UelazVGYPbbR1IqUU0+Cr26BH5FHiEiR34no=; b=DIv/IGDFRYX6tIN4qBoNOPLwnu4KinOPPU5wzfUFMfbiRn3HXCKw62/hfOUbwqJjAm Exj3GpaL4BLioqaivunzQwgCI914U9a3JU/2hHGpAJRTF9p55Wjs19/ZftQypKExXXFH wBpkBaAk/a4K11Cr2yAL0jGMjCgyh7iT8VgwY3lzj1LM5IuU5cdatxwl+W4vWOp/x8VA UuG7fQ6N2x2pY2SAEXdJrkmF9qw9qrTxaaxMyXAiffiFaRSW7a/8u3D1FuudrXfyliKk zoadWucvOdhH9ynkTIeIJX8DVg0LO9AGEmLNCrS5Rb/vokUrRV01EGvaYT5xvZW1g6We h3GQ== X-Gm-Message-State: AOAM53365lDHFa4+2DtbpW/mm6p4UlXYynlgiZmJtB42mjwxkr3WF1kj ZrL7nhwdLMU0GDtdqWkanoVjouEtR1HbjA== X-Google-Smtp-Source: ABdhPJzhX8tufUPJ2dxozteeK7KPI/jEvrfhbCVBUdguHznnNy8nHi8i7NkcIi99WMZHjEgPGz2UcA== X-Received: by 2002:ad4:5511:: with SMTP id az17mr52837798qvb.159.1594136609901; Tue, 07 Jul 2020 08:43:29 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id c27sm23651585qka.23.2020.07.07.08.43.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:29 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 21/23] btrfs: flush delayed refs when trying to reserve data space Date: Tue, 7 Jul 2020 11:42:44 -0400 Message-Id: <20200707154246.52844-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We can end up with free'd extents in the delayed refs, and thus may_commit_transaction() may not think we have enough pinned space to commit the transaction and we'll ENOSPC early. Handle this by running the delayed refs in order to make sure pinned is uptodate before we try to commit the transaction. Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 5a555f20ca20..db0eebe6758b 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1023,6 +1023,7 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, COMMIT_TRANS, }; From patchwork Tue Jul 7 15:42:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649031 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE9F56C1 for ; Tue, 7 Jul 2020 15:43:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 928112075B for ; Tue, 7 Jul 2020 15:43:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="tw0ajwnx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728971AbgGGPne (ORCPT ); Tue, 7 Jul 2020 11:43:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPnd (ORCPT ); Tue, 7 Jul 2020 11:43:33 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E06D6C061755 for ; Tue, 7 Jul 2020 08:43:32 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id w34so12025897qte.1 for ; Tue, 07 Jul 2020 08:43:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HwKnkctBatP8mq5wuRxIhxJyvILH7ub1QqZQoCTgCGo=; b=tw0ajwnx4uZkA6D7ysZyc5g3p6DA45yXLjzgQYwXJM5wMi/I0BcsDEzgKiz1M2/w+t 6a7b5gyWLz05+lN18LYRqaiNSv7D313ymRTIum6oXYzuCjuWrFbCMoMyzVqIIgib2pn2 eCmrpGTmlN0Kwj99s0q1vUrotwqUyxAP3NDWZRDIqgrcGcJ008gvsJ7b52gfrRjrR6Kq YjGVuNZkmgIp99al32slbyLQXVLZOL/+MMcX6/OynewIqmxFt23fvSzNP29is1P0/gaa wkZkYtYC+/PUAwNyu7iBzqfNKLTTYi3SKcWdrZZhGamZSIGkQ+l9tuuu+2O8byzHRDwC eBlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HwKnkctBatP8mq5wuRxIhxJyvILH7ub1QqZQoCTgCGo=; b=h2zlzmnGMsfEIi34Uw9EpcXvtoRqpC0efOIGhmeA7PB4nSN26aadm+TSEFWDmYB7Is jN9kBebe74c3wlL67dRNpMxkKUiBGD3sswuClAGHnJg4EUTKX5HU0+Bpc/Tra1B7CMqR Ivmg0NKQe27zUgPy2Pp0bCQmFTMNo1nZjmyTsqVsJFY+wMlLOMivlMckgsYjhQQd+lKz BCEnKAC89daBCKpg5kf7KddxU6MEBrGgfHrhdcRX0sYSH+uS74+AzVCYPD77OGYbFDb/ XvSOotTRcLl6wUBqQG/wrAR7KC/6pVMs+xeQqApE9q2uLHBlKOtx4X+Xg1Z8GlJB1pLd nPKA== X-Gm-Message-State: AOAM5308iRi+OkK6qVVMvE0AubPMa1c7xD3Q6O6KB9jIt2gcVKjTG3BT DKtLrdNz4/y+yPFUyO3nBP3Y1v6f3MJcFg== X-Google-Smtp-Source: ABdhPJzeeG4uChCZgULpUSyxQPraWxdai9VyDgjKm+mps7qGNkk7V6IBLYGqhhrTHRIHpCckhdPOGw== X-Received: by 2002:aed:2a75:: with SMTP id k50mr53586461qtf.27.1594136611727; Tue, 07 Jul 2020 08:43:31 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t5sm24215973qkh.46.2020.07.07.08.43.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:31 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 22/23] btrfs: do async reclaim for data reservations Date: Tue, 7 Jul 2020 11:42:45 -0400 Message-Id: <20200707154246.52844-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have the data ticketing stuff in place, move normal data reservations to use an async reclaim helper to satisfy tickets. Before we could have multiple tasks race in and both allocate chunks, resulting in more data chunks than we would necessarily need. Serializing these allocations and making a single thread responsible for flushing will only allocate chunks as needed, as well as cut down on transaction commits and other flush related activities. Priority reservations will still work as they have before, simply trying to allocate a chunk until they can make their reservation. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 +- fs/btrfs/disk-io.c | 3 +- fs/btrfs/space-info.c | 123 ++++++++++++++++++++++++++++++------------ fs/btrfs/super.c | 1 + 4 files changed, 93 insertions(+), 37 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index efa72a204b91..998bda3132fd 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -494,7 +494,7 @@ enum btrfs_orphan_cleanup_state { ORPHAN_CLEANUP_DONE = 2, }; -void btrfs_init_async_reclaim_work(struct work_struct *work); +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info); /* fs_info */ struct reloc_control; @@ -916,6 +916,7 @@ struct btrfs_fs_info { /* Used to reclaim the metadata space in the background. */ struct work_struct async_reclaim_work; + struct work_struct async_data_reclaim_work; spinlock_t unused_bgs_lock; struct list_head unused_bgs; diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 7c07578866f3..0907e02eea54 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2713,7 +2713,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info) fs_info->check_integrity_print_mask = 0; #endif btrfs_init_balance(fs_info); - btrfs_init_async_reclaim_work(&fs_info->async_reclaim_work); + btrfs_init_async_reclaim_work(fs_info); spin_lock_init(&fs_info->block_group_cache_lock); fs_info->block_group_cache_tree = RB_ROOT; @@ -4033,6 +4033,7 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) btrfs_cleanup_defrag_inodes(fs_info); cancel_work_sync(&fs_info->async_reclaim_work); + cancel_work_sync(&fs_info->async_data_reclaim_work); /* Cancel or finish ongoing discard work */ btrfs_discard_cleanup(fs_info); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index db0eebe6758b..bc99864c30d9 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -998,9 +998,83 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } -void btrfs_init_async_reclaim_work(struct work_struct *work) +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, + COMMIT_TRANS, +}; + +static void btrfs_async_reclaim_data_space(struct work_struct *work) { - INIT_WORK(work, btrfs_async_reclaim_metadata_space); + struct btrfs_fs_info *fs_info; + struct btrfs_space_info *space_info; + u64 last_tickets_id; + int flush_state = 0; + + fs_info = container_of(work, struct btrfs_fs_info, + async_data_reclaim_work); + space_info = fs_info->data_sinfo; + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + } + + while (flush_state < ARRAY_SIZE(data_flush_states)) { + flush_space(fs_info, space_info, U64_MAX, + data_flush_states[flush_state]); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + + if (last_tickets_id == space_info->tickets_id) { + flush_state++; + } else { + last_tickets_id = space_info->tickets_id; + flush_state = 0; + } + + if (flush_state >= ARRAY_SIZE(data_flush_states)) { + if (space_info->full) { + if (maybe_fail_all_tickets(fs_info, + space_info)) + flush_state = 0; + else + space_info->flush = 0; + } else { + flush_state = 0; + } + } + spin_unlock(&space_info->lock); + } +} + +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info) +{ + INIT_WORK(&fs_info->async_reclaim_work, + btrfs_async_reclaim_metadata_space); + INIT_WORK(&fs_info->async_data_reclaim_work, + btrfs_async_reclaim_data_space); } static const enum btrfs_flush_state priority_flush_states[] = { @@ -1020,13 +1094,6 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; -static const enum btrfs_flush_state data_flush_states[] = { - FLUSH_DELALLOC_WAIT, - RUN_DELAYED_IPUTS, - FLUSH_DELAYED_REFS, - COMMIT_TRANS, -}; - static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, @@ -1059,12 +1126,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, - const enum btrfs_flush_state *states, - int states_nr) + struct reserve_ticket *ticket) { - int flush_state = 0; - while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); spin_lock(&space_info->lock); @@ -1074,17 +1137,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } - - while (flush_state < states_nr) { - flush_space(fs_info, space_info, U64_MAX, states[flush_state]); - spin_lock(&space_info->lock); - if (ticket->bytes == 0) { - spin_unlock(&space_info->lock); - return; - } - spin_unlock(&space_info->lock); - flush_state++; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, @@ -1139,6 +1191,7 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, int ret; switch (flush) { + case BTRFS_RESERVE_FLUSH_DATA: case BTRFS_RESERVE_FLUSH_ALL: case BTRFS_RESERVE_FLUSH_ALL_STEAL: wait_reserve_ticket(fs_info, space_info, ticket); @@ -1153,14 +1206,8 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; - case BTRFS_RESERVE_FLUSH_DATA: - priority_reclaim_data_space(fs_info, space_info, ticket, - data_flush_states, - ARRAY_SIZE(data_flush_states)); - break; case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: - priority_reclaim_data_space(fs_info, space_info, ticket, - NULL, 0); + priority_reclaim_data_space(fs_info, space_info, ticket); break; default: ASSERT(0); @@ -1225,6 +1272,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, u64 orig_bytes, enum btrfs_reserve_flush_enum flush) { + struct work_struct *async_work; struct reserve_ticket ticket; u64 used; int ret = 0; @@ -1233,6 +1281,11 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, ASSERT(orig_bytes); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_ALL); + if (flush == BTRFS_RESERVE_FLUSH_DATA) + async_work = &fs_info->async_data_reclaim_work; + else + async_work = &fs_info->async_reclaim_work; + spin_lock(&space_info->lock); ret = -ENOSPC; used = btrfs_space_info_used(space_info, true); @@ -1274,7 +1327,8 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, init_waitqueue_head(&ticket.wait); ticket.steal = (flush == BTRFS_RESERVE_FLUSH_ALL_STEAL); if (flush == BTRFS_RESERVE_FLUSH_ALL || - flush == BTRFS_RESERVE_FLUSH_ALL_STEAL) { + flush == BTRFS_RESERVE_FLUSH_ALL_STEAL || + flush == BTRFS_RESERVE_FLUSH_DATA) { list_add_tail(&ticket.list, &space_info->tickets); if (!space_info->flush) { space_info->flush = 1; @@ -1282,8 +1336,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, space_info->flags, orig_bytes, flush, "enospc"); - queue_work(system_unbound_wq, - &fs_info->async_reclaim_work); + queue_work(system_unbound_wq, async_work); } } else { list_add_tail(&ticket.list, diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 3c9ebd4f2b61..66094f076385 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1880,6 +1880,7 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) * the filesystem is busy. */ cancel_work_sync(&fs_info->async_reclaim_work); + cancel_work_sync(&fs_info->async_data_reclaim_work); btrfs_discard_cleanup(fs_info); From patchwork Tue Jul 7 15:42:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11649033 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF669912 for ; Tue, 7 Jul 2020 15:43:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9841E2075B for ; Tue, 7 Jul 2020 15:43:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="Rm1BDpDm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728989AbgGGPng (ORCPT ); Tue, 7 Jul 2020 11:43:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727079AbgGGPne (ORCPT ); Tue, 7 Jul 2020 11:43:34 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAE07C061755 for ; Tue, 7 Jul 2020 08:43:34 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id c139so38488270qkg.12 for ; Tue, 07 Jul 2020 08:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=K7w32C9kwC8lzUHZISfRCie38Nyemq4QdTj1Gte1J8M=; b=Rm1BDpDmvVnUvbwEKDKw4qwxQqwpHK7Pe+zhgUpR7zVvdZ+OSR2e9sjpsTFzx02pas hsvNX2yYircw5tVQXxJCSsSkWmkDKtc+hDZacPJyeGN3mp20RGdOCYW78OXxumEAn/RJ 45c1V0hP/8LtS5bOHCSZ26g5MFkSS2cXSQFQ3d18OA6vB5Ql53PHTuU1AROBOXmKxwqK 5pqQSvnVNHYr19w2G+KSQ88wQ+W0+BMfNvZrVXUJamBJgnZih8Q0d1GrQiQeJu46JYy4 OE9rG4m+78lqrvhYVH5genn/BIvjZ5oLcpBkq/QB0y5j1ru3Kt9A4XEqneQjZJHR9QrY LhPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K7w32C9kwC8lzUHZISfRCie38Nyemq4QdTj1Gte1J8M=; b=MvaHfYriz2Jn7bRv/xgtrMjZlAfR/O9whr7v3y15NOCkesJHAZDI90oNSKSl0NwKcb WvJ0OB9dvgSAz1TGgASIHodZ6ZfZSpKCimMrjgbJs6jvc1hv7bk0L+SIG71WPk3GjcbM HrROOlPJyn3IB3zztZE0se+sOvl7DD1IM1bXXl5QvNsQw5keaqYNrM0PY2hTz5MdWuF3 mblez7C0l0b1jSkQr1s/3odhJ+gxKDJXn0BAc2HTWfjd+Xmdv8YltLAl99vc/sKhl3XA Z+fjssmmKTzFx63hurygl77dIKK+agXLZJonICJzmsv/kIp+8QfKdA8kYs3d7U45Z0uo n/cw== X-Gm-Message-State: AOAM5305mn6KIhQhQdsZwxSc6JsxV+vi/wlEov8xM636QSgnxPx6wCs0 doeTh0Tl53BNuMD6VgLTeRObQnpXeIoPbw== X-Google-Smtp-Source: ABdhPJzJbt4+1bE2ixy54tcUtjSUk/nOQAjFYKreDNCqwB/as+OiZJqf+3BTStvZjMKyMX/zljFwuw== X-Received: by 2002:a37:a4d0:: with SMTP id n199mr35672090qke.476.1594136613486; Tue, 07 Jul 2020 08:43:33 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id 73sm24809116qkk.38.2020.07.07.08.43.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 08:43:32 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 23/23] btrfs: add a comment explaining the data flush steps Date: Tue, 7 Jul 2020 11:42:46 -0400 Message-Id: <20200707154246.52844-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200707154246.52844-1-josef@toxicpanda.com> References: <20200707154246.52844-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The data flushing steps are not obvious to people other than myself and Chris. Write a giant comment explaining the reasoning behind each flush step for data as well as why it is in that particular order. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 47 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index bc99864c30d9..7fa7f580b4cc 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -998,6 +998,53 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } +/* + * FLUSH_DELALLOC_WAIT: + * Space is free'd from flushing delalloc in one of two ways. + * + * 1) compression is on and we allocate less space than we reserved. + * 2) We are overwriting existing space. + * + * For #1 that extra space is reclaimed as soon as the delalloc pages are + * cow'ed, by way of btrfs_add_reserved_bytes() which adds the actual extent + * length to ->bytes_reserved, and subtracts the reserved space from + * ->bytes_may_use. + * + * For #2 this is trickier. Once the ordered extent runs we will drop the + * extent in the range we are overwriting, which creates a delayed ref for + * that freed extent. This however is not reclaimed until the transaction + * commits, thus the next stages. + * + * RUN_DELAYED_IPUTS + * If we are freeing inodes, we want to make sure all delayed iputs have + * completed, because they could have been on an inode with i_nlink == 0, and + * thus have been trunated and free'd up space. But again this space is not + * immediately re-usable, it comes in the form of a delayed ref, which must be + * run and then the transaction must be committed. + * + * FLUSH_DELAYED_REFS + * The above two cases generate delayed refs that will affect + * ->total_bytes_pinned. However this counter can be inconsistent with + * reality if there are outstanding delayed refs. This is because we adjust + * the counter based soley on the current set of delayed refs and disregard + * any on-disk state which might include more refs. So for example, if we + * have an extent with 2 references, but we only drop 1, we'll see that there + * is a negative delayed ref count for the extent and assume that the space + * will be free'd, and thus increase ->total_bytes_pinned. + * + * Running the delayed refs gives us the actual real view of what will be + * freed at the transaction commit time. This stage will not actually free + * space for us, it just makes sure that may_commit_transaction() has all of + * the information it needs to make the right decision. + * + * COMMIT_TRANS + * This is where we reclaim all of the pinned space generated by the previous + * two stages. We will not commit the transaction if we don't think we're + * likely to satisfy our request, which means if our current free space + + * total_bytes_pinned < reservation we will not commit. This is why the + * previous states are actually important, to make sure we know for sure + * whether committing the transaction will allow us to make progress. + */ static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS,