From patchwork Wed Jul 8 13:59:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651701 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 438D2618 for ; Wed, 8 Jul 2020 14:00:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 25E672064B for ; Wed, 8 Jul 2020 14:00:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="jcrXwnUO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729469AbgGHOAV (ORCPT ); Wed, 8 Jul 2020 10:00:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45732 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAU (ORCPT ); Wed, 8 Jul 2020 10:00:20 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44369C061A0B for ; Wed, 8 Jul 2020 07:00:20 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id u8so20473297qvj.12 for ; Wed, 08 Jul 2020 07:00:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HnTx9Ul/fcQBFwOo+UODTyb97JsAdctKQxu5MvW+SPk=; b=jcrXwnUOYWDiGGt3qycVahW8Ua61YDREO3TPBubloOQh6Bvy3VB3nNGU3+/D/VVzMV tIIun5/fWDxk3CMmbDXl6BceHA02JIUyOQvOYZqaw2WhT0ZvKYFAcWJ1bejl7NIOSX9J R8p415cJHV0aKPGPeaPDHwcwSsJ1miBVeEMuwC0IJeelKdD/XyTwwh0qki1yXRAW7QnV 0u2ACARlKkJTMJY35j5b/f4oSdppVNw+u9Fapkn8o4Y1cz7Sj2Aon2FbLdoXOsf4d3pu AA0zmSEJCnXv1KZAxQOJZaWi/KK+Yv4pVN7a7QnnUALZQYoBqN5TrBvLV6/kRURAbilp 48+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HnTx9Ul/fcQBFwOo+UODTyb97JsAdctKQxu5MvW+SPk=; b=ROCqluf16nQ3ZX3SPQ5+2C9ENoMEmnUEhjS+vB9/HJ/U+NjRCitBSJebWyRAdwVGuX 9e2gY+s3VsuEN1jhXdoahggT5M3XIyezzUD1bAJnypB4dbVeRPSciMSUmBs4Kzi8RRSn k9VUwQjMCSztXPjVE5Yxacr4SwWX3ofULKgsJE4t0UBk1I6Zkc6pRioRe1JYdRQIdV33 dag0Y1FvFStHcX09ZLDtE4S2jYo/xIEWAgG6cVZXCDf2G9iAHI0K0VcGeMJUPeSHwRZm JI2WYElzzPvezANGyifSIMWQN5JvGt2nXGI89PoACT6R4Qu9S5REXeWb/yShtOGbYIrm 4VVg== X-Gm-Message-State: AOAM5327sAKutFno4DFR60afPULTX0QMQhv+nNwuHvoN57sst1o4MB85 rvmfaZtG3QVtbDgZivQuwKsLOkaFBba5HA== X-Google-Smtp-Source: ABdhPJzdnzcnfTTHwdxyShQzT7y6PVZJWlktr54who/qhR3WeGU1tHD2hTFPeNSltsx+U34GQaiELA== X-Received: by 2002:a0c:b61d:: with SMTP id f29mr48763870qve.249.1594216818974; Wed, 08 Jul 2020 07:00:18 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id e37sm30958585qtk.94.2020.07.08.07.00.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:18 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 01/23] btrfs: change nr to u64 in btrfs_start_delalloc_roots Date: Wed, 8 Jul 2020 09:59:51 -0400 Message-Id: <20200708140013.56994-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have btrfs_wait_ordered_roots() which takes a u64 for nr, but btrfs_start_delalloc_roots() that takes an int for nr, which makes using them in conjunction, especially for something like (u64)-1, annoying and inconsistent. Fix btrfs_start_delalloc_roots() to take a u64 for nr and adjust start_delalloc_inodes() and it's callers appropriately. This means we've adjusted start_delalloc_inodes() to take a pointer of nr since we want to preserve the ability for start-delalloc_inodes() to return an error, so simply make it do the nr adjusting as necessary. Part of adjusting the callers to this means changing btrfs_writeback_inodes_sb_nr() to take a u64 for items. This may be confusing because it seems unrelated, but the caller of btrfs_writeback_inodes_sb_nr() already passes in a u64, it's just the function variable that needs to be changed. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/inode.c | 27 +++++++++++---------------- fs/btrfs/ioctl.c | 2 +- fs/btrfs/space-info.c | 2 +- 5 files changed, 15 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index c96c27f66999..6f1ba19c6705 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2929,7 +2929,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, u32 min_type); int btrfs_start_delalloc_snapshot(struct btrfs_root *root); -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr); +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr); int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end, unsigned int extra_bits, struct extent_state **cached_state); diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index db93909b25e0..769ac3098880 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -630,7 +630,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info, * flush all outstanding I/O and inode extent mappings before the * copy operation is declared as being finished */ - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) { mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return ret; diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index d894d9e41aad..57a9a6f91572 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -9384,7 +9384,8 @@ static struct btrfs_delalloc_work *btrfs_alloc_delalloc_work(struct inode *inode * some fairly slow code that needs optimization. This walks the list * of all the inodes with pending delalloc and forces them to disk. */ -static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) +static int start_delalloc_inodes(struct btrfs_root *root, u64 *nr, + bool snapshot) { struct btrfs_inode *binode; struct inode *inode; @@ -9424,9 +9425,11 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) list_add_tail(&work->list, &works); btrfs_queue_work(root->fs_info->flush_workers, &work->work); - ret++; - if (nr != -1 && ret >= nr) - goto out; + if (*nr != U64_MAX) { + (*nr)--; + if (*nr == 0) + goto out; + } cond_resched(); spin_lock(&root->delalloc_lock); } @@ -9451,18 +9454,15 @@ static int start_delalloc_inodes(struct btrfs_root *root, int nr, bool snapshot) int btrfs_start_delalloc_snapshot(struct btrfs_root *root) { struct btrfs_fs_info *fs_info = root->fs_info; - int ret; + u64 nr = U64_MAX; if (test_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state)) return -EROFS; - ret = start_delalloc_inodes(root, -1, true); - if (ret > 0) - ret = 0; - return ret; + return start_delalloc_inodes(root, &nr, true); } -int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) +int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, u64 nr) { struct btrfs_root *root; struct list_head splice; @@ -9485,15 +9485,10 @@ int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, int nr) &fs_info->delalloc_roots); spin_unlock(&fs_info->delalloc_root_lock); - ret = start_delalloc_inodes(root, nr, false); + ret = start_delalloc_inodes(root, &nr, false); btrfs_put_root(root); if (ret < 0) goto out; - - if (nr != -1) { - nr -= ret; - WARN_ON(nr < 0); - } spin_lock(&fs_info->delalloc_root_lock); } spin_unlock(&fs_info->delalloc_root_lock); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index ab34179d7cbc..d4df1a7f53a8 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -4848,7 +4848,7 @@ long btrfs_ioctl(struct file *file, unsigned int case BTRFS_IOC_SYNC: { int ret; - ret = btrfs_start_delalloc_roots(fs_info, -1); + ret = btrfs_start_delalloc_roots(fs_info, U64_MAX); if (ret) return ret; ret = btrfs_sync_fs(inode->i_sb, 1); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index c7bd3fdd7792..ef6e264746fc 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -477,7 +477,7 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, } static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, int nr_items) + unsigned long nr_pages, u64 nr_items) { struct super_block *sb = fs_info->sb; From patchwork Wed Jul 8 13:59:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651703 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9ACB6618 for ; Wed, 8 Jul 2020 14:00:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83118206E2 for ; Wed, 8 Jul 2020 14:00:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="fOwHSlZZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729495AbgGHOAW (ORCPT ); Wed, 8 Jul 2020 10:00:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAW (ORCPT ); Wed, 8 Jul 2020 10:00:22 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03F55C061A0B for ; Wed, 8 Jul 2020 07:00:22 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id w27so11483583qtb.7 for ; Wed, 08 Jul 2020 07:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=psc9iFpr11NrkpfCqKSXQMbsM0IUWL8y4wKOAVQjwHQ=; b=fOwHSlZZact8L6FH/i+B3pi4NrtswBIdym1+HhbIUg1/HsPJBwVHt6SQGjpaDUOkPu COSpBuxu9GQX1shLLO0vRk5/5g4AQDzljHNGzMl8xnjzJYgXcBOL+Fb2NZtR9T/Wtbd2 uAsOUIh2F4mze/xe7HHOMXQmjsxubdqFsnemw2zSAY0pOVgz6e0fGStc6KlQPeY2Dv87 IAo9pQcvXN2VC/lXiZ6A8rEnXMCmQSfxLVJ1Gweh6ij4AHwkv1E/Z+DRpkKPpQ9uoOsy hs8LMLWzBizR84BmDoRlHDnnEm/4nRHga+c0iQ1nR62ohhYi8YVDafsM8t44FdFb3Qjq r7JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=psc9iFpr11NrkpfCqKSXQMbsM0IUWL8y4wKOAVQjwHQ=; b=hstiLu/8Eho2EiP1+hXTYd7O1pTMYCLEZe0hlFU1brv7Q/wUtSqd5oYn5dyo6H0u1D ZhaRCjJRr/p+8ar0CtWsI1lzTG6yiB0IErEnhOWd/bmReS6an9nVItfPlb4rwtMycv0C K3c8D5pxWjD7aMhpQDu6/EGiH9o/zH2Vppr0ng/AvEd2J3ANxQpM0ThjOPbbdfCJclYV KsjyHQCoMG6ic6MKlAiID8oYXiGluM8TyEqDunOxmM6tASe/YKniBmfhFmsVSckShyKT oMRYk43FGM6TwU5Nd4n46EUOU+hW2WNKDalNuWqvVTIx9SQNNcsdm1jicxgQ1LnlHl9z LgVQ== X-Gm-Message-State: AOAM530IfQg2xb/Y+lecmL/xIoN/C9vZRkANm38U/J87/qNYhTCwBxqR 70ZRV5seHxergz36joA2WzS7evyxvVCcQQ== X-Google-Smtp-Source: ABdhPJz+xU1Wk3jcqTHA1PhEtaFXP/8TUNFIcu4px2TTvkyPCpE23u27Iza1qx8EJKf1CTXo/dOYXA== X-Received: by 2002:ac8:4718:: with SMTP id f24mr56359872qtp.95.1594216820846; Wed, 08 Jul 2020 07:00:20 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id g1sm31094449qko.70.2020.07.08.07.00.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:20 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 02/23] btrfs: remove orig from shrink_delalloc Date: Wed, 8 Jul 2020 09:59:52 -0400 Message-Id: <20200708140013.56994-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We don't use this anywhere inside of shrink_delalloc, remove it. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index ef6e264746fc..10cae5b55235 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -517,7 +517,7 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, * shrink metadata reservation for delalloc */ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - u64 orig, bool wait_ordered) + bool wait_ordered) { struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; @@ -742,7 +742,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, num_bytes, + shrink_delalloc(fs_info, num_bytes * 2, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Wed Jul 8 13:59:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651705 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8408A618 for ; Wed, 8 Jul 2020 14:00:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BB862064B for ; Wed, 8 Jul 2020 14:00:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="SmN99C2n" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729578AbgGHOAY (ORCPT ); Wed, 8 Jul 2020 10:00:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAY (ORCPT ); Wed, 8 Jul 2020 10:00:24 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA777C08C5CE for ; Wed, 8 Jul 2020 07:00:23 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id d27so34521115qtg.4 for ; Wed, 08 Jul 2020 07:00:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tlhyn6Io+XWJV27UN298PnUC9Kg6hHIiKF1O0jj30xw=; b=SmN99C2nabPjvJiRco+1Us9kEeHohZM4xOh/K6jCMCrcxMc7VUOV6NGiFMi1bgTFC2 dsqRR1CrKI3I0E4i0S16NVBLR/kjxK19I2JTjQ76tmLDYcaamnGnTZ0j1PjcFfol0EO9 MFjS4WqMk1VkRkCj7DfGkO99T18tRIp3RDfgB664LM3FTDGVlNVpKeh1lgLRtL9LKJcU 5xhzSVj3bK0cmQ5pymafNJIXC/SOZr3gLKrNp0+6EZE0247VVp2iLt+Lk1vb6Pvl7+/T 7fswVOaDDUX4cWuVY9821xu49qQMzVvvLzKUxK8w78uWS+/4tHlPXBaxaxY2Z7XAx90d 7yKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tlhyn6Io+XWJV27UN298PnUC9Kg6hHIiKF1O0jj30xw=; b=byteDEo0Uu9fhR3BzumxLaPfQPp2YLazGrlD/Ktkn/YE2Qjo3E5OY+ompky3AZBunh rmQlUlxMsheM4RxN62YQFsQl+Y+O0W9a498arz/qyg/iRdsXSshuY96AhEXW5Yj3PmgD vy0AiJyfOrA+/kavuiqTYPfjUrak1NxK6UrFYJ0BcfGarxoEXcHZ3s0OeDTGRV+jsw0U a+gNMjK/NB3XkCuzqwDcPqQnHg6CF5n2V0fQNqJoW7U4R6R9GhnUnemvGYEcObyu23W5 Mv0x52fDqGT1eBlfrjN2oT/rCA454Yy1giUY5Ol4wgfSSGZORFEijE8U/NgDYmZzXnBx 04hQ== X-Gm-Message-State: AOAM533wKqiJRHpEMGvZRCYkU3RB4Be1BHEvRarfQwfEXcKcpSHjmYqx gRZuqQGv/OJSfDqy9ylSPwxwoj6+T5QgaQ== X-Google-Smtp-Source: ABdhPJy8UWGzlDr8OjZgHlCDuwS6ad0f39SjQlBy9YwKEf93ZuEOF1B82m4ge6YZ+1ECofgWuzSK1Q== X-Received: by 2002:ac8:4b4f:: with SMTP id e15mr62611366qts.305.1594216822674; Wed, 08 Jul 2020 07:00:22 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id f4sm29254230qtv.59.2020.07.08.07.00.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:22 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 03/23] btrfs: handle U64_MAX for shrink_delalloc Date: Wed, 8 Jul 2020 09:59:53 -0400 Message-Id: <20200708140013.56994-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data allocations are going to want to pass in U64_MAX for flushing space, adjust shrink_delalloc to handle this properly. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 10cae5b55235..78ad8f01785c 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -530,8 +530,19 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, int loops; /* Calc the number of the pages we need flush for space reservation */ - items = calc_reclaim_items_nr(fs_info, to_reclaim); - to_reclaim = items * EXTENT_SIZE_PER_ITEM; + if (to_reclaim == U64_MAX) { + items = U64_MAX; + } else { + /* + * to_reclaim is set to however much metadata we need to + * reclaim, but reclaiming that much data doesn't really track + * exactly, so increase the amount to reclaim by 2x in order to + * make sure we're flushing enough delalloc to hopefully reclaim + * some metadata reservations. + */ + items = calc_reclaim_items_nr(fs_info, to_reclaim) * 2; + to_reclaim = items * EXTENT_SIZE_PER_ITEM; + } trans = (struct btrfs_trans_handle *)current->journal_info; space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); @@ -742,7 +753,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes * 2, + shrink_delalloc(fs_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Wed Jul 8 13:59:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651707 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E730618 for ; Wed, 8 Jul 2020 14:00:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 45B4F206E2 for ; Wed, 8 Jul 2020 14:00:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="ChDc4FIF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729538AbgGHOA0 (ORCPT ); Wed, 8 Jul 2020 10:00:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAZ (ORCPT ); Wed, 8 Jul 2020 10:00:25 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9437C061A0B for ; Wed, 8 Jul 2020 07:00:25 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id b185so30958435qkg.1 for ; Wed, 08 Jul 2020 07:00:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rUsScBbpNuEa/GRgx27yvwOsrj7ecK3MJGcmAjuWUPo=; b=ChDc4FIFV79Irl73yq7dFllQS1sAUdS6Igp+oeIFzKaAB10rShC5T3odjbs2R8SEcS zJEvXuAR/EwrUfBS5tAqDQYaw6fTGWotQniwpiLvgWU/5lZbwEq8eAG/q0pAri+/0Ref siZ8zWqGiZGsV+ivhx7xCBpuqXL7A8XwcAN4TcBz/DoFWACKNglr2MFmWoy+cY4flD+I 0Vbvp+9BnLUkNQrRbOoJdJQRD4iTS5A0tkxNlNR0QXsvQ08Y+qBpUGi8Uso6juVbyKY3 ++dcHdzE78TZ3+h+e17AdZgnOtnjk2ECiiShY+Gu0sEoIKRX89ZhLRJ0LKupbPgn/M+P 7f9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rUsScBbpNuEa/GRgx27yvwOsrj7ecK3MJGcmAjuWUPo=; b=DEJEOllHXXMXgBYkD1HHy+I5DvJXXW/i52+IRrW21v0/lHcne3cqFCEOHF4Sy1A1Nu 8gkN5tB3HxGtNW5DQ9BiJRkBoTjgFOqCyosjva/Gqglrwp6FNG5iTgpMS/ZJRKEPo1QJ SIg9oa0q1mBCWG4hp1lvR3J3KXGHWsjwRlibFrXrOqzj2v51RV92RtdsdEKckmhF4No7 vtBCk5Fxfgbfvfb72dalAcVoFt9Cds2SujN7FOAj+eyexgFtUt5YSBKo5jokHMprtYzo DCWUNlgWDJRhG82Ydo+bE2AqAEa/JPFUlpO+IPkGjglV/lcB7FtoKGzXvJljO9y4tmFD xxrw== X-Gm-Message-State: AOAM532lo7WD+TJ8xuilyXh1Z9t3SNRJRLSc75mFNLVChyRsX2cw16UI bdq+TNGvhxf/QkXVxJ9hrL2JdPQ9dxbd6w== X-Google-Smtp-Source: ABdhPJwCUuR/kwgw3Y9a4ESjXstRaI/MA7w1YeV4Rq10YtQyIZbRdWGPNuivIxMoG3iwxNGxjEzU3g== X-Received: by 2002:a37:5bc6:: with SMTP id p189mr56119952qkb.112.1594216824601; Wed, 08 Jul 2020 07:00:24 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id s52sm29866172qtj.52.2020.07.08.07.00.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:24 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 04/23] btrfs: make shrink_delalloc take space_info as an arg Date: Wed, 8 Jul 2020 09:59:54 -0400 Message-Id: <20200708140013.56994-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently shrink_delalloc just looks up the metadata space info, but this won't work if we're trying to reclaim space for data chunks. We get the right space_info we want passed into flush_space, so simply pass that along to shrink_delalloc. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 78ad8f01785c..1bf40328b0ee 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -516,10 +516,10 @@ static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, /* * shrink metadata reservation for delalloc */ -static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, - bool wait_ordered) +static void shrink_delalloc(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + u64 to_reclaim, bool wait_ordered) { - struct btrfs_space_info *space_info; struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; @@ -545,7 +545,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, } trans = (struct btrfs_trans_handle *)current->journal_info; - space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); @@ -753,7 +752,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; case FLUSH_DELALLOC: case FLUSH_DELALLOC_WAIT: - shrink_delalloc(fs_info, num_bytes, + shrink_delalloc(fs_info, space_info, num_bytes, state == FLUSH_DELALLOC_WAIT); break; case FLUSH_DELAYED_REFS_NR: From patchwork Wed Jul 8 13:59:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651709 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE2F913B6 for ; Wed, 8 Jul 2020 14:00:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D6758206E2 for ; Wed, 8 Jul 2020 14:00:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="IA537qQ8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729618AbgGHOA2 (ORCPT ); Wed, 8 Jul 2020 10:00:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOA2 (ORCPT ); Wed, 8 Jul 2020 10:00:28 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38314C061A0B for ; Wed, 8 Jul 2020 07:00:28 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id 145so39064693qke.9 for ; Wed, 08 Jul 2020 07:00:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nNbN8KMVo5bDXJEMK3qzoB8OXC6pwWVBEBIb751xZOI=; b=IA537qQ8wthXj8g2awpJT0dhFUL5ItvTPhENcIqnusr9jr3ROVoctOOFdzNQw3T1ve UA+clttSTBhM04566doF9om4eGskRFNA/V1SMXsMp5xB/ca1lmezcdg20n+Ub1rM4Kt7 oJX3MG95y55cu/COYdFtYYBa27jd0gjZ1yzE73xoXD9yXZzQfZZ/dXDybsgkwoB3tQZI sbjyj6u+yufG3YvE9PEdBHDQzfyWnZRum4bLtiX40mVXSZG5iMWBJKC9MeHul3WaT6lg 9SQMpOe48Puo5VmoFhas7l5Fy0LKiM/xyxcd0ZnfY1236TYNY5PHGj1sZpSqErCCOFug rEug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nNbN8KMVo5bDXJEMK3qzoB8OXC6pwWVBEBIb751xZOI=; b=jrcypScp4YB/udgoIcvXiBDkONGMyvwRHGonIoTfEa5XNfOeB1muIJFEQrVcOZx36I vwhN/ZYVbwxgTShSyZHb6RqE8ZkWhU46kDg6ep28NjOZy2oKr3Pmo5y0+zo2tRtL2p/h uFOadI/GFQrOynewvezoG8HZdAA9pMw/6NGTz64NwZuoPTe203Ci0U+6YSEJPdqvaIGI Kih5lsYP4ByouDgC2e1S4pUiZ2Gh/GA1mQ4N/l95igl9bQWuanZ2hIWZa0s9wjONdArn luofHFoKZmYN/cdXMsCPmshgLtHqGJQwv9TbYzLJqO3gPXHcgyV6I+ZXQ3MyxwKiXWFd qAkQ== X-Gm-Message-State: AOAM533OB6aFCyWLiynMiUb2XYtg5pC2teT7g1SQUd2uXR4j2pvQpdQO SRBzxJyyCdw4A6pEPx8znGm+wvwcUDSprA== X-Google-Smtp-Source: ABdhPJzgk1ahwuIb5F4sQVaA2Rc5antjYJpoqpRRIFnHt7XYlgB10aRjlJwhizWcv9bVN1s/DH2l2w== X-Received: by 2002:a05:620a:15ab:: with SMTP id f11mr46332871qkk.199.1594216827066; Wed, 08 Jul 2020 07:00:27 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id m7sm29263827qti.6.2020.07.08.07.00.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:26 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 05/23] btrfs: make ALLOC_CHUNK use the space info flags Date: Wed, 8 Jul 2020 09:59:55 -0400 Message-Id: <20200708140013.56994-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We have traditionally used flush_space() to flush metadata space, so we've been unconditionally using btrfs_metadata_alloc_profile() for our profile to allocate a chunk. However if we're going to use this for data we need to use btrfs_get_alloc_profile() on the space_info we pass in. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 1bf40328b0ee..8450864c5b77 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -777,7 +777,8 @@ static void flush_space(struct btrfs_fs_info *fs_info, break; } ret = btrfs_chunk_alloc(trans, - btrfs_metadata_alloc_profile(fs_info), + btrfs_get_alloc_profile(fs_info, + space_info->flags), (state == ALLOC_CHUNK) ? CHUNK_ALLOC_NO_FORCE : CHUNK_ALLOC_FORCE); btrfs_end_transaction(trans); From patchwork Wed Jul 8 13:59:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651711 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B8C413B6 for ; Wed, 8 Jul 2020 14:00:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8449E206E2 for ; Wed, 8 Jul 2020 14:00:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="VO7gtwJU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729634AbgGHOAb (ORCPT ); Wed, 8 Jul 2020 10:00:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAb (ORCPT ); Wed, 8 Jul 2020 10:00:31 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E1E5C061A0B for ; Wed, 8 Jul 2020 07:00:31 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id e12so34496704qtr.9 for ; Wed, 08 Jul 2020 07:00:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wxLlTO38pSaivYDhn6/e+gzVkShJaxbwsw4D9UDfLoE=; b=VO7gtwJUZXFj1wLci8XkjO2c+0i00Yr+G2dHmu8gSY6TzN5G1zYXwmE32t5EtS6jJj c3wtVUAS8LhFkP9rDFA+EwIvYCaKbJfwaswLl7C/+PbKvFBP4jTNaJ/Twi8662p9FjM9 fTp8qRdSvdysixUmzz4HMHShVLAPK22Op3AKmJPIXAQJBezSq1SPaGYwlfIt57zgNpCL wqnDhCzup7aZwQ61VfSOcftEaDvPs704shnoDypjnkVmbY0qeJ9FwxitKGds5qvJI3XV UdcblTrjAf03Hm+fwEAfEEhAb/tHu6jR+9SvBYxHQU6QUUDWlRBFKRyhb7JVHv4ocvhl dYeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wxLlTO38pSaivYDhn6/e+gzVkShJaxbwsw4D9UDfLoE=; b=R9r1FiewyUQXRE6tlQkaZLugBZoTvkXdmfNQ0iE8qtYti/uawDxROj18/zFNLhaeVK kKFRPH7XMJ9O2Fnjyef0jHfi/HDRXc6YxrhoCQB48N2kFq1BIdw8ZFuN0QV+UZ5ovMH7 sT7urqrSclPmJ9GDEjuv8yYAQSj6W5giP+S1cWv9w0ygxb/HGqmMAxZQrq+8oU0uDIcF 0iw0Uo8CUs8mabDTEtcOn+bNOj7cm6PHFqZJG1xgoQpMvZbGg/3c/K4BpE4WYyssAc3g s4cqty3CzFPBypF5d/Nj2nqZP+XNoV0DWNuYTRYQaJ1DQ5WL5/CyDmrU/L9iNg8SjXBJ sgTg== X-Gm-Message-State: AOAM53134Y/R/ExMiTN+FUHgDQT8yjDZ+l+KlGmIocUPcjoFssLsh/D9 4+YVbc+gaAIsyrfVbpZLbdFYpnZOl6l8sw== X-Google-Smtp-Source: ABdhPJyodWiYuuXpbCBzGv1+V4KDbCd5g2ES+xtAGEYgUyThAVwxL2iD8aV52xUqG3VBNPoxRoGXQQ== X-Received: by 2002:ac8:27c9:: with SMTP id x9mr40593007qtx.172.1594216830109; Wed, 08 Jul 2020 07:00:30 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id z18sm31762633qta.51.2020.07.08.07.00.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:29 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 06/23] btrfs: call btrfs_try_granting_tickets when freeing reserved bytes Date: Wed, 8 Jul 2020 09:59:56 -0400 Message-Id: <20200708140013.56994-7-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We were missing a call to btrfs_try_granting_tickets in btrfs_free_reserved_bytes, so add it to handle the case where we're able to satisfy an allocation because we've freed a pending reservation. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 3aa78952a2b7..daf88891a40a 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3091,6 +3091,8 @@ void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, if (delalloc) cache->delalloc_bytes -= num_bytes; spin_unlock(&cache->lock); + + btrfs_try_granting_tickets(cache->fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Wed Jul 8 13:59:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651713 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF910618 for ; Wed, 8 Jul 2020 14:00:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C799B206E2 for ; Wed, 8 Jul 2020 14:00:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="ENrQ6f5b" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729636AbgGHOAe (ORCPT ); Wed, 8 Jul 2020 10:00:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAd (ORCPT ); Wed, 8 Jul 2020 10:00:33 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FB38C061A0B for ; Wed, 8 Jul 2020 07:00:33 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id b185so30959003qkg.1 for ; Wed, 08 Jul 2020 07:00:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1eyMNWoi2hC8qlRuAY9xprTphnhdSNxjekwZC/C82Cw=; b=ENrQ6f5bqdC5bdr9UFbCNIUlZpnYO8mX1mkv0j8m8xLhMyZiuUKAaX8faMWYy4yYEv nYYxUrjbDJ0egAe+ya0nG24QBcbeCkrpbJ1yZJfFE9DMNslBGvgdOeEs40j3ET9oCVDy /VUknqw/BB22D988Wxzxy3ciiwnoSCaz0ZJlUAip3zVGzBeu7VJmtU8kQG8P7IV6MnVA kM2Gka0zfww3XJRd0DmbpVqOIDvjnMvbGAiiGEYI63O2XRoYNi59btW6L4Y7ft8SPM1p Gp0rC2qc/jnC00bbNOqsAFz+U4VC0uJVjd9QtUy8nGj21zZTbhAHKtQs/V69G6er6y4f g5LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1eyMNWoi2hC8qlRuAY9xprTphnhdSNxjekwZC/C82Cw=; b=Rw8bUeuLwpAVEL5S+ORKKkV4n90iB5yQtDVUVlukdmz94SmPOYAuL25roUDcpdBobT hXj6vKRkdBk7g4Dumy/FM47Wkrr8a50Z8tkRPMutHImVEEJZg1NQ9iGdEFqw6tgWspo0 BApdqSisUa3G1pFy+EaWbq9FQwJcbYN4kRtfNU55K+IRXPyTqDU/1Tcv9dvaOCoKNCcI A9otQKDOe30Z8U+JyK6/s7bq9L7K6WrcTmB4C0qxrNp/f9kBo6Td+duHhjrp6+LePuXC 6n8aVyXk0ymvwf0toAaQv/KIuCu+G1/7Kn/7KTpykw/sR401EwA7Rct5l1CXWnCa93ut MZwQ== X-Gm-Message-State: AOAM531Qj/J1BwhHR+2Cu8X4Kt2nZvBmCjiAbBFu5eupzVkjp5Z+WfZI VLqV0LiePLmmvAiljjQD1RNHTnXKP3cWeA== X-Google-Smtp-Source: ABdhPJz7LXd7niqIlG0WIB+4SieK+OBHw52+MivnKQvbkrvXBpG2Xoheh0pF/nBpKth9rj7t/WSjng== X-Received: by 2002:a05:620a:4ca:: with SMTP id 10mr57955011qks.2.1594216832298; Wed, 08 Jul 2020 07:00:32 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id z36sm22784847qtd.22.2020.07.08.07.00.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:31 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 07/23] btrfs: call btrfs_try_granting_tickets when unpinning anything Date: Wed, 8 Jul 2020 09:59:57 -0400 Message-Id: <20200708140013.56994-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org When unpinning we were only calling btrfs_try_granting_tickets() if global_rsv->space_info == space_info, which is problematic because we use ticketing for SYSTEM chunks, and want to use it for DATA as well. Fix this by moving this call outside of that if statement. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index c0bc35f932bf..4b8c59318f6e 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2844,11 +2844,10 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info, len -= to_add; } spin_unlock(&global_rsv->lock); - /* Add to any tickets we may have */ - if (len) - btrfs_try_granting_tickets(fs_info, - space_info); } + /* Add to any tickets we may have */ + if (!readonly && return_free_space && len) + btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } From patchwork Wed Jul 8 13:59:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651715 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AEF1D13B6 for ; Wed, 8 Jul 2020 14:00:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 976F5206E2 for ; Wed, 8 Jul 2020 14:00:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="ONHtMbPX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729648AbgGHOAg (ORCPT ); Wed, 8 Jul 2020 10:00:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAg (ORCPT ); Wed, 8 Jul 2020 10:00:36 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5807DC061A0B for ; Wed, 8 Jul 2020 07:00:36 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id u12so34490509qth.12 for ; Wed, 08 Jul 2020 07:00:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YK9+ywm77Te+pOZHuYGGFVXJb/D8KOylDb/tS2FRu/g=; b=ONHtMbPXgkJLdG+/lcpjGSQPz27nHoR8UM9UtiQJf8kiWTDLs6GJdDXfYegk8+ZqN7 5wSj0XITBkax4DO2XB/c+ho7XjbrNNH8hS1chmx3sLYumjDc3A6Ve0mFLrGFJm3kC95+ PL7dZeUu6sauh6Iy/csRgePJs/aoha9KozGqsmOMZKIY3lwYiEiIR2ORAJxQYefbrJQC aMkZFQTASGRThm2JoXbfULR526/cDKKmgvcZiOHN3PpsUnhKBGJ04RatwJ4NieYRU8+l mFYGrW6Ws+y4zRALh5uf9ps+Ou3GZawHfm1RCYmtk14KEiHmdBKTWMQBV/Y1TsfOBQhD g9fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YK9+ywm77Te+pOZHuYGGFVXJb/D8KOylDb/tS2FRu/g=; b=Dm+pVrLrsahoOfN/3ytdTCtfejCIbwR6UGHHJfkBvekzVXnQKNWz/JlCfpax7aryml fDMO9Rw1l5i2+oMo9Zf0O/ua21R1NtOyqypmufVmRA6W+/BzVNRoUWBRHOdEie0Wml16 +XDQBa7Iq6rlJYXejMLcgh+DCJH2TG/6nvVIt4p8youfdkP4GWqxYEv7hgxeFvRVwOJI QRXDwutu5obicCGMgrpHjQx5FFzDzIH4VAknXI8xJYjKx+hm3GctQyaoceZtrTq4J+z5 CsiTVToMX81QRvQ4MovC/OgAUOak/VEak61c2vp6BG8ktdJ3VhM6ZwCfSz/LwADZX46s PFIg== X-Gm-Message-State: AOAM531LJhuuZRjuX/tS3FpEIn99PiFT/+wW04AwJHELtllhffUtvzI1 aoZ5m5tntzZZqfpXG4pk9D2JmzaajYUOAA== X-Google-Smtp-Source: ABdhPJxvYNnksohRteOA4NFZVK3KHNL1KpOytIxdYxd4Nj+sOsVnYvKKxKzuiGlE/JiNeRPJBO0hCA== X-Received: by 2002:ac8:1305:: with SMTP id e5mr60425904qtj.78.1594216835258; Wed, 08 Jul 2020 07:00:35 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id b53sm31248752qtc.65.2020.07.08.07.00.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:34 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 08/23] btrfs: call btrfs_try_granting_tickets when reserving space Date: Wed, 8 Jul 2020 09:59:58 -0400 Message-Id: <20200708140013.56994-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org If we have compression on we could free up more space than we reserved, and thus be able to make a space reservation. Add the call for this scenario. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index daf88891a40a..c7097cb4fc7d 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3058,6 +3058,13 @@ int btrfs_add_reserved_bytes(struct btrfs_block_group *cache, space_info, -ram_bytes); if (delalloc) cache->delalloc_bytes += num_bytes; + + /* + * Compression can use less space than we reserved, so wake + * tickets if that happens. + */ + if (num_bytes < ram_bytes) + btrfs_try_granting_tickets(cache->fs_info, space_info); } spin_unlock(&cache->lock); spin_unlock(&space_info->lock); From patchwork Wed Jul 8 13:59:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651717 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E004813B6 for ; Wed, 8 Jul 2020 14:00:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C847C2064B for ; Wed, 8 Jul 2020 14:00:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="d+7Q+fKi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729653AbgGHOAj (ORCPT ); Wed, 8 Jul 2020 10:00:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728932AbgGHOAi (ORCPT ); Wed, 8 Jul 2020 10:00:38 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61A8CC061A0B for ; Wed, 8 Jul 2020 07:00:38 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id t11so18413666qvk.1 for ; Wed, 08 Jul 2020 07:00:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/Ci5z3yMI3h5IFqCrR0RnubjoKZNhhjifg4W5m/gO7o=; b=d+7Q+fKi9A3tdYihFobpUsUw4s2Z0brjIPWkzyxmhSDdITzAMUyyz+LGgKOGYKrWeu S70M+2lHcewu0eRUJpPEzzoriokfDVCCHmeYHluFSyuWwlc/QtyIXZSzngJ0Znz0Du9h Vs4YYTsxDOlsAea6KpOC8tjeyrpQ23xyheA9hQWOR22oCxaq3U3U4L1YiJgGlGI4ozKq hfZmaZD5q+iJ2x3rpUe+U5R6P5wlrVCa/H7DHCiQDVrHA+5jmkm9ePtVRyw6S5J/O0HG bUOebyX5dsOcyNstkgFU69qn7FQmpunug+hmD0DZMP2C3uUpzC/zAzSic/KzvXaTHPFX 8OTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/Ci5z3yMI3h5IFqCrR0RnubjoKZNhhjifg4W5m/gO7o=; b=E1ab/4D1Up+vJpY8BOENKgbU1IzgOqhhvA8BUvvO8iK3lsh6Zf5URJZk/FjIb9e253 H5683cXzffulhZv5Ib2LE1j4wYBQytgOQ6d2i1U9PCYQValkcJFE9aXoWA5aNIaD1Pz7 NlbP/rnGW74AJaXvdUloFm7UWZ+WPWfRniq2rwL1BD+rDOl7IoXz3LzN87YMDN839Q9k LBA2P9/nIXeEuCJvFQ/Vk+5WS0SH8llAq4P2rL/+Tq0pUu8nzekc7YjQ1JLuNv588GNE OKDm5VmTTCV8pxNMuHtmLIIujUy/HL3EFC7fLwB+Y/qM8bxTz9FOeoZ+E0Qhw2W91Y37 jZRQ== X-Gm-Message-State: AOAM532WTA5nRBrXu21ZwB1ag79zsrzM9qQSk8xVFWk/B7Q/ESR1bTD6 g3KSQo6GxlV/Lg7G11mKBgrbGR2ziJ+VzA== X-Google-Smtp-Source: ABdhPJzp0HYy+GldPW3OdNbxFdWEd19YF9lLA8Rsxq3ABUfMeWk+2mQde+SRXIn+wiPF0NVmJ5rQOg== X-Received: by 2002:a0c:e903:: with SMTP id a3mr56969749qvo.144.1594216837080; Wed, 08 Jul 2020 07:00:37 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id i21sm30029717qke.9.2020.07.08.07.00.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:36 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov , Johannes Thumshirn Subject: [PATCH 09/23] btrfs: use the btrfs_space_info_free_bytes_may_use helper for delalloc Date: Wed, 8 Jul 2020 09:59:59 -0400 Message-Id: <20200708140013.56994-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We are going to use the ticket infrastructure for data, so use the btrfs_space_info_free_bytes_may_use() helper in btrfs_free_reserved_data_space_noquota() so we get the try_granting_tickets call when we free our reservation. Reviewed-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index 0e354e9e57d0..0a41bdcc14d1 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -277,9 +277,7 @@ void btrfs_free_reserved_data_space_noquota(struct btrfs_fs_info *fs_info, ASSERT(IS_ALIGNED(len, fs_info->sectorsize)); data_sinfo = fs_info->data_sinfo; - spin_lock(&data_sinfo->lock); - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, -len); - spin_unlock(&data_sinfo->lock); + btrfs_space_info_free_bytes_may_use(fs_info, data_sinfo, len); } /* From patchwork Wed Jul 8 14:00:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651719 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BFBE13B6 for ; Wed, 8 Jul 2020 14:00:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3F5AD2078B for ; Wed, 8 Jul 2020 14:00:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="kytZjOyC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729695AbgGHOAm (ORCPT ); Wed, 8 Jul 2020 10:00:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729656AbgGHOAl (ORCPT ); Wed, 8 Jul 2020 10:00:41 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25545C061A0B for ; Wed, 8 Jul 2020 07:00:41 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id di5so15545708qvb.11 for ; Wed, 08 Jul 2020 07:00:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZRKc63P5E/fKmHUwzOTIMk+4mmOzl1OUpVjs2y9i3Qw=; b=kytZjOyCYHF5oWqdLvrxUPfP5T6amtkdSVmyGKARmw/850CjMai8wKSfBeygv04AXd AKFeylCK33VV/R11tOk4okC+Cz6LytKIa7A5Q7PBB1rIuqQ5+Uow3FQoli9Vwbuv67UX 8OwuE8kPMovss2Yoy1okVOZRm3Yw721SX4GlB5y0UDg2OOkVRSm0dWg+W8SOY9oFWvwJ gHOqKMPQKwDLwkGIFAp7ADZn9yMzdQYWvp/EkcLrXMRcYKi9Vol6FnfFrevRg8n7QFIY MhzVq2mdnVhn8X6seEh1S6+yJiuulAY6kjTPxZa59X50NNvVoYu5B7ZKtUv/iKBj9MFY CKKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZRKc63P5E/fKmHUwzOTIMk+4mmOzl1OUpVjs2y9i3Qw=; b=WegpB66p9J53R1hwdQcIEdYW/TBeXzUFhOErj3BgboUyPkJ17ekcUGdGLgkXJE4/n5 EIuHsvLu2XPLuOCausXJ/CmhUyc1e9yT9d64XHnp0forjC7BDFK2yWWNYSIe0kfymvl6 VsihOIr/EPQKhW4SVV4cE/16aGOdrayn+w177NpmLaV94br1DWLkT0LXmIcE4YYmJ7T7 9rLeHm0k6vKUw1dQ+CwDB6uCsUEQSY74ZELN18agamLJP4O879df5iYKsyTMcAY1EONI 4IUZb3vD7BOfas4zcqLJ4CPwBfPGqRrrHgkMrKpH621gDdcmUAH5q9yB/tk0QkddrAGe duGQ== X-Gm-Message-State: AOAM531JVWgzdwtNSIBlX2Q5mXcxO39DTn/r9IIbfI8MwBUVJpzexxis GiLQDtNKLrJtZRqD1HnsvFW1EEklJpaVSQ== X-Google-Smtp-Source: ABdhPJwI/cHso0BV9C5m5iWgFahU6h0+4yFknME29mMDZQzdoxYV/dqMR3GfrF8+jZ30kU2XSSHiXA== X-Received: by 2002:a0c:f0c8:: with SMTP id d8mr58393570qvl.217.1594216839938; Wed, 08 Jul 2020 07:00:39 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id e25sm28069950qtc.93.2020.07.08.07.00.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:39 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 10/23] btrfs: use btrfs_start_delalloc_roots in shrink_delalloc Date: Wed, 8 Jul 2020 10:00:00 -0400 Message-Id: <20200708140013.56994-11-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The original iteration of flushing had us flushing delalloc and then checking to see if we could make our reservation, thus we were very careful about how many pages we would flush at once. But now that everything is async and we satisfy tickets as the space becomes available we don't have to keep track of any of this, simply try and flush the number of dirty inodes we may have in order to reclaim space to make our reservation. This cleans up our delalloc flushing significantly. The async_pages stuff is dropped because btrfs_start_delalloc_roots() handles the case that we generate async extents for us, so we no longer require this extra logic. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 55 +------------------------------------------ 1 file changed, 1 insertion(+), 54 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 8450864c5b77..de8a49de5fdb 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -476,28 +476,6 @@ void btrfs_dump_space_info(struct btrfs_fs_info *fs_info, up_read(&info->groups_sem); } -static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info, - unsigned long nr_pages, u64 nr_items) -{ - struct super_block *sb = fs_info->sb; - - if (down_read_trylock(&sb->s_umount)) { - writeback_inodes_sb_nr(sb, nr_pages, WB_REASON_FS_FREE_SPACE); - up_read(&sb->s_umount); - } else { - /* - * We needn't worry the filesystem going from r/w to r/o though - * we don't acquire ->s_umount mutex, because the filesystem - * should guarantee the delalloc inodes list be empty after - * the filesystem is readonly(all dirty pages are written to - * the disk). - */ - btrfs_start_delalloc_roots(fs_info, nr_items); - if (!current->journal_info) - btrfs_wait_ordered_roots(fs_info, nr_items, 0, (u64)-1); - } -} - static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info, u64 to_reclaim) { @@ -523,10 +501,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, struct btrfs_trans_handle *trans; u64 delalloc_bytes; u64 dio_bytes; - u64 async_pages; u64 items; long time_left; - unsigned long nr_pages; int loops; /* Calc the number of the pages we need flush for space reservation */ @@ -567,37 +543,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, loops = 0; while ((delalloc_bytes || dio_bytes) && loops < 3) { - nr_pages = min(delalloc_bytes, to_reclaim) >> PAGE_SHIFT; - - /* - * Triggers inode writeback for up to nr_pages. This will invoke - * ->writepages callback and trigger delalloc filling - * (btrfs_run_delalloc_range()). - */ - btrfs_writeback_inodes_sb_nr(fs_info, nr_pages, items); - - /* - * We need to wait for the compressed pages to start before - * we continue. - */ - async_pages = atomic_read(&fs_info->async_delalloc_pages); - if (!async_pages) - goto skip_async; - - /* - * Calculate how many compressed pages we want to be written - * before we continue. I.e if there are more async pages than we - * require wait_event will wait until nr_pages are written. - */ - if (async_pages <= nr_pages) - async_pages = 0; - else - async_pages -= nr_pages; + btrfs_start_delalloc_roots(fs_info, items); - wait_event(fs_info->async_submit_wait, - atomic_read(&fs_info->async_delalloc_pages) <= - (int)async_pages); -skip_async: spin_lock(&space_info->lock); if (list_empty(&space_info->tickets) && list_empty(&space_info->priority_tickets)) { From patchwork Wed Jul 8 14:00:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651721 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7E3C618 for ; Wed, 8 Jul 2020 14:00:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEFB62078B for ; Wed, 8 Jul 2020 14:00:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="0hFEMEsa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729699AbgGHOAo (ORCPT ); Wed, 8 Jul 2020 10:00:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729411AbgGHOAn (ORCPT ); Wed, 8 Jul 2020 10:00:43 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03B98C061A0B for ; Wed, 8 Jul 2020 07:00:43 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id k18so4200557qtm.10 for ; Wed, 08 Jul 2020 07:00:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=F24Fb8VJoZFaPNYAY7Qby2wBX99HVjadR1H3fE1tv5s=; b=0hFEMEsatxRUiSgQxuXZNKphY7PGIBw5vWllylpZXh8Cbxi7gCrknxQONBPs3It2DX IBQSS02i1Uo9/gLld5ANKL8urq+TVD3dba/WZPyfVUq4fmNCmbSe5wHazrTp+cSlhQCG N89VF9Y3Lx9dgN0JzozOqA/i7oqMYacq3oUh1EGkMR9TluSUogl8KyqNErn0j9Np0Cej EqKWo60HXeGh+hr3iBUmD6Cq1P0Gb4LYNsn4KNXpL0iXbRT6Egurh6FWfAn1WheYsBY1 jryQcFEHow4c2cYeC2NXko+Ip/O3H1LUBDrcy7guUzLXM50gdj4UwHH8j5LfAIpkiJGC bUzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=F24Fb8VJoZFaPNYAY7Qby2wBX99HVjadR1H3fE1tv5s=; b=TgiluIeUMVMfZ0VsrTagmRjCchhwEMfkUVPm4uPm8fSTCRZWbphxApvPCU4LN5yIIt VNLY3nuy007j/N8nsw6yOvb3R/iYtOSqT16PqMaX6LlKpBNPssQ/3CmbAgCbGjn9SaL1 Zmrm/HgBK5DuPQu9kYMKW7R6R1ZO9usob13RkL6GcxrGcfuGTn6aL5112LJ0g89PKoBP S0zUk88kElFLhWXCD6cpcPLQjNtiSONQQpSr8xpC1uOGSlB884aAH1PI3DdrLmt1tHMX 6qq/vo5YiX//eMxvZL05kw8aTQJGffx+hSaYM0xLs3ICQEJdatdceMcnnlK9yOgCrUe3 VwvA== X-Gm-Message-State: AOAM530vxnpXZDvPRx6OyDVYGs2O7NVX827xPQIRG+iJP/nDwOcSJhrg 9wjPO7tE+C/k766WbtRHU9JVwBSzzpsJUw== X-Google-Smtp-Source: ABdhPJxuP3iCSPDPBxJryAoFNtExcTGS8JylL0NsANVO8SP2toS/1lEjrpdTFojnhFcYr6y+ziWC/w== X-Received: by 2002:ac8:3528:: with SMTP id y37mr59841025qtb.308.1594216841853; Wed, 08 Jul 2020 07:00:41 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id m4sm32661903qtf.43.2020.07.08.07.00.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:41 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 11/23] btrfs: check tickets after waiting on ordered extents Date: Wed, 8 Jul 2020 10:00:01 -0400 Message-Id: <20200708140013.56994-12-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Right now if the space is free'd up after the ordered extents complete (which is likely since the reservations are held until they complete), we would do extra delalloc flushing before we'd notice that we didn't have any more tickets. Fix this by moving the tickets check after our wait_ordered_extents check. Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index de8a49de5fdb..3b5064a2a972 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -545,14 +545,6 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, while ((delalloc_bytes || dio_bytes) && loops < 3) { btrfs_start_delalloc_roots(fs_info, items); - spin_lock(&space_info->lock); - if (list_empty(&space_info->tickets) && - list_empty(&space_info->priority_tickets)) { - spin_unlock(&space_info->lock); - break; - } - spin_unlock(&space_info->lock); - loops++; if (wait_ordered && !trans) { btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1); @@ -561,6 +553,15 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, if (time_left) break; } + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets) && + list_empty(&space_info->priority_tickets)) { + spin_unlock(&space_info->lock); + break; + } + spin_unlock(&space_info->lock); + delalloc_bytes = percpu_counter_sum_positive( &fs_info->delalloc_bytes); dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes); From patchwork Wed Jul 8 14:00:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651725 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9485713B6 for ; Wed, 8 Jul 2020 14:00:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7C03120786 for ; Wed, 8 Jul 2020 14:00:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="nmVNR2sW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729714AbgGHOAp (ORCPT ); Wed, 8 Jul 2020 10:00:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729411AbgGHOAp (ORCPT ); Wed, 8 Jul 2020 10:00:45 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4B10C061A0B for ; Wed, 8 Jul 2020 07:00:44 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id 145so39065941qke.9 for ; Wed, 08 Jul 2020 07:00:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eAww/7Sh8UavgDUq52mopahUOAupYHHSLg89+z5kfPk=; b=nmVNR2sWMbejPqbnlZMrJcj2cMhUDO1uGg0LWnDVs9I4jxmopMRHBKe3IwIo6DZUr5 IAX/n9HEyOMgfBv7OV80LkSjkJKXvwcN51XPwTd+p3FWcZZGFcEEJQG3SZod4HynX40H +G8Zs4BkBJswE38NuI3gW74hKDEZzTJhFtjIO9C9QB2FbvmAKJWsDr46HzGRRJj2Kpjg 3qK6rJsiay1zaasOMmhTZV99kITgWwNUCPazL39vd5Q6fGJcn7AgBejKS2h6ov7AN71Q YoXClT8r6GMMh+2ctcP9WvQAhbE0BXQUlbCBdp+AU3PiSALM7mfMq35y2QmwKKgBfsCN ekWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eAww/7Sh8UavgDUq52mopahUOAupYHHSLg89+z5kfPk=; b=eFHbu4QqgG/jF+rMRAfXwKQBfb7tEPLf4jM73gVT/Cs40dz2mErobOMfVe5A03nOrr Cj9SsO76WdekDsoL4WJsBJ93Z7Zv1cvAajtaE0Boe/iUyqAi6wy4sKTllllsztFo6b31 DB3leyya778YAKgn6ADCtAkl9UNll8dfehVL39JDiBdUhM1KDo7DA7SAXIlIOP7SYAK2 wQ40q6uXSptLV/ykjUaQQ06TpqWC+W3rQDD0yTewX6O+uWKMsLX5NU+Qvp28DK595iOA GkkjHyTAYclsJkuQYYNPlJkJSPiXSdX8Kt4loiXvAxFfCIHL55RKzj3bGK7Xs7UuKMxv BlsQ== X-Gm-Message-State: AOAM53279d6FyEO9rHy8NH+4ZGRHyrQgI4fY7moh0IJG6wKla+VrBYEe dk37J5A2gfceLKkBxclyzGXrkO51eMbMkA== X-Google-Smtp-Source: ABdhPJxB2MPy25irrrQ4z3c0OrLAIJ23PCFDzKGQdY9Mzdkohj6/pEYmmgvTirhGxHLlHLYsx+UBcA== X-Received: by 2002:a37:6609:: with SMTP id a9mr57157609qkc.337.1594216843791; Wed, 08 Jul 2020 07:00:43 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id p36sm18874208qta.0.2020.07.08.07.00.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:43 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 12/23] btrfs: add flushing states for handling data reservations Date: Wed, 8 Jul 2020 10:00:02 -0400 Message-Id: <20200708140013.56994-13-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently the way we do data reservations is by seeing if we have enough space in our space_info. If we do not and we're a normal inode we'll 1) Attempt to force a chunk allocation until we can't anymore. 2) If that fails we'll flush delalloc, then commit the transaction, then run the delayed iputs. If we are a free space inode we're only allowed to force a chunk allocation. In order to use the normal flushing mechanism we need to encode this into a flush state array for normal inodes. Since both will start with allocating chunks until the space info is full there is no need to add this as a flush state, this will be handled specially. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/space-info.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 6f1ba19c6705..4eabad429440 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -2572,6 +2572,8 @@ enum btrfs_reserve_flush_enum { */ BTRFS_RESERVE_FLUSH_LIMIT, BTRFS_RESERVE_FLUSH_EVICT, + BTRFS_RESERVE_FLUSH_DATA, + BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE, BTRFS_RESERVE_FLUSH_ALL, BTRFS_RESERVE_FLUSH_ALL_STEAL, }; diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 3b5064a2a972..94da7b43e152 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1018,6 +1018,12 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + COMMIT_TRANS, + RUN_DELAYED_IPUTS, +}; + static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, From patchwork Wed Jul 8 14:00:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651727 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E4F913B6 for ; Wed, 8 Jul 2020 14:00:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 877092078B for ; Wed, 8 Jul 2020 14:00:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="t+/+GN2a" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729730AbgGHOAr (ORCPT ); Wed, 8 Jul 2020 10:00:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729411AbgGHOAr (ORCPT ); Wed, 8 Jul 2020 10:00:47 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0ECD8C061A0B for ; Wed, 8 Jul 2020 07:00:47 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id w27so11485127qtb.7 for ; Wed, 08 Jul 2020 07:00:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tQxqsUrlp4x7HrENuHXKxmhsqNJWIRD3LhIamFg7tBc=; b=t+/+GN2aVgbNoDx/yrevGd9OmIlTQKvVM/FddEpG5edBRvd/rD6u3q1nKETebtLU6T 3HdOCr2EZES1yxSBKREvabrx2VwSpogRZJTisXBI307u3ssASihN1WJq4PImr+jvcCYX JcwvrYj0UY7QSGFz397FiImuqgP59MujoJblH407QYVf9bwfSOKpPIUEZirqmqCYAG/H +6818QAy4J05ogYbms/dIIDZHCdzNjUQ/2/cyR8txXpvUdPQ9IZ8+B29XYPtcDfA7orC 0NP9Zu0GdWLB4LpCrE/ifodzcDUgqUw6o4RslN+j+nHwtaNygKCdlZytVEwouEUiM4ZZ l+Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tQxqsUrlp4x7HrENuHXKxmhsqNJWIRD3LhIamFg7tBc=; b=Wcqu6kjuckI8ojTMIP8xH8koyFJ0q+0PoR8A3D0Guuis9EgINA+40kyzmcqC0aepSZ Tgb33+HgKTlCOXxxKaUeVzyMZZa9eiLzI6JxiJaATyKXF2GqLtadVs9I9JrbxXqlsBGJ BNoggnlLbt+2+8dPXfLNFZu5q395rTUQ/glbyojsTjSDWtvyuBchBQq5nRjgWpZvPxXA eE3V4DaziJrDBJ8HA2lLYsxvZcZN/Q0KhqZzGI2E3/A2VOMAPG/+4MaAvsXE/WmN6ysf 82WmlCjLzdGsvQdz/UM5rKbMyJWyW8PB3FrOoMRRSDUQRBcdhP7zP4rx0tEyMYamOYMB KtVg== X-Gm-Message-State: AOAM532tHLx8BxXdIVS2ibCiFuE77TPY5F+XTDAtqxfSxoUusQUU765Z QY9lQX4g7nZemwdBlcnQMRGdD5+FvPRPAg== X-Google-Smtp-Source: ABdhPJz020YzqnE2uMzAz6JM7GMJyX3n9n/9YEr0ZDzrI9ykzXnqLldSZNag4OS2K3Kzv8TeaE0RWA== X-Received: by 2002:ac8:44a8:: with SMTP id a8mr45555016qto.29.1594216845759; Wed, 08 Jul 2020 07:00:45 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id q28sm31272728qtk.13.2020.07.08.07.00.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:45 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 13/23] btrfs: add the data transaction commit logic into may_commit_transaction Date: Wed, 8 Jul 2020 10:00:03 -0400 Message-Id: <20200708140013.56994-14-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Data space flushing currently unconditionally commits the transaction twice in a row, and the last time it checks if there's enough pinned extents to satisfy it's reservation before deciding to commit the transaction for the 3rd and final time. Encode this logic into may_commit_transaction(). In the next patch we will pass in U64_MAX for bytes_needed the first two times, and the final time we will pass in the actual bytes we need so the normal logic will apply. This patch exists soley to make the logical changes I will make to the flushing state machine separate to make it easier to bisect any performance related regressions. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 94da7b43e152..be0f117d4ccf 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -579,21 +579,33 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info) + struct btrfs_space_info *space_info, + u64 bytes_needed) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; struct btrfs_block_rsv *trans_rsv = &fs_info->trans_block_rsv; struct btrfs_trans_handle *trans; - u64 bytes_needed; u64 reclaim_bytes = 0; u64 cur_free_bytes = 0; + bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; + /* + * If we are data and have passed in U64_MAX we just want to + * unconditionally commit the transaction to match the previous data + * flushing behavior. + */ + if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && + bytes_needed == U64_MAX) { + do_commit = true; + goto check_pinned; + } + spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -607,7 +619,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : 0; + bytes_needed = (ticket) ? ticket->bytes : bytes_needed; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -618,6 +630,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; +check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -627,7 +640,8 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (do_commit || + test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -635,7 +649,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, /* * See if there is some space in the delayed insertion reservation for - * this reservation. + * this reservation. If the space_info's don't match (like for DATA or + * SYSTEM) then just enospc, reclaiming this space won't recover any + * space to satisfy those reservations. */ if (space_info != delayed_rsv->space_info) goto enospc; @@ -743,7 +759,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info); + ret = may_commit_transaction(fs_info, space_info, num_bytes); break; default: ret = -ENOSPC; From patchwork Wed Jul 8 14:00:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651729 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24B24618 for ; Wed, 8 Jul 2020 14:00:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07C7320786 for ; Wed, 8 Jul 2020 14:00:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="LudqmNq+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729745AbgGHOAu (ORCPT ); Wed, 8 Jul 2020 10:00:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOAt (ORCPT ); Wed, 8 Jul 2020 10:00:49 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46801C061A0B for ; Wed, 8 Jul 2020 07:00:49 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id e11so41615064qkm.3 for ; Wed, 08 Jul 2020 07:00:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gLDtuOZfkhQLfDpyxeUamy76vLbqLUpJJ6K47Azp3Ws=; b=LudqmNq+mRcMy9fnxKhBiPkyMOXrIhbEnEczDhTzAU6iaK6la6svf9jYC64CQ2oLd9 7WFphznUgjdx/4pyhuMpMDNx/sr4vbqdlrd1Ffmg/+1nWLYYE5gKXHWjby/g7sRaNUs8 iI3EO+XU5/GEDwpB0I3BvSB54lJvraOEE0n98bAC5OXldo1lE7Mq+lQ3um4/XTTc0h2d dWuFnv+QJoiHvCZ5VEXd8r6m+OPMhwqLqC8HNUUWoltwqpTj5Ij3j1FN9msf9h8Myq4w vX1TQc46RrqtHUe5a9GNor9eLsBczcPH8fhNFVyKvkwXX4jyimYM4LXN75XrvQxGZQyB ux+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gLDtuOZfkhQLfDpyxeUamy76vLbqLUpJJ6K47Azp3Ws=; b=Lqy9ptKbRlAfhvh+OTFSK7dzloWtRAcbcBZmjh4aHC7t9tVtZYDiHc2pKGR1aeh6gB 23bSBhVzgMUOe3bWL3IHqap5/RGeFPMkK+duApkEXE0GO8YD84hO5cxgaNQWcEDffIi1 TVzdTW8oGwkXSO89XQ958UGMAHRBrjng/IaNnIWyAXiJMOio+DpPLOz6GorrygM6bVTW ZjJM86bOzRdhGcNmHPBTZAZSKGVRHo4EFEkRwVABAdDz9C9FA7T/QCtwpOEuj1UsMh1S w5Gx3ngqah6ahkOgRFQfnR+8GMaw0s0rwtRb4tWHOZRg/UBRarne5KdBJWmAqq9N/CiA 2wow== X-Gm-Message-State: AOAM531Ai4m2ph5l9hnj8JrqvNEEibiz0GvIhR1LxulLS/81YSmAw6A3 w3O+xZj0d35212VE0oAV9hZFuEehX+VEcA== X-Google-Smtp-Source: ABdhPJxDT/r2mrQt553qJePIWOVUBHuNuTZuu2QuDEHQwYb+xVHIKU1dDUicvz/1IL/vSwqkwsQzPw== X-Received: by 2002:a37:97c5:: with SMTP id z188mr53626505qkd.48.1594216847910; Wed, 08 Jul 2020 07:00:47 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id c27sm26676701qka.23.2020.07.08.07.00.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:47 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 14/23] btrfs: add btrfs_reserve_data_bytes and use it Date: Wed, 8 Jul 2020 10:00:04 -0400 Message-Id: <20200708140013.56994-15-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Create a new function btrfs_reserve_data_bytes() in order to handle data reservations. This uses the new flush types and flush states to handle making data reservations. This patch specifically does not change any functionality, and is purposefully not cleaned up in order to make bisection easier for the future patches. The new helper is identical to the old helper in how it handles data reservations. We first try to force a chunk allocation, and then we run through the flush states all at once and in the same order that they were done with the old helper. Subsequent patches will clean this up and change the behavior of the flushing, and it is important to keep those changes separate so we can easily bisect down to the patch that caused the regression, rather than the patch that made us start using the new infrastructure. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 119 ++------------------------------------ fs/btrfs/space-info.c | 92 +++++++++++++++++++++++++++++ fs/btrfs/space-info.h | 2 + 3 files changed, 98 insertions(+), 115 deletions(-) diff --git a/fs/btrfs/delalloc-space.c b/fs/btrfs/delalloc-space.c index 0a41bdcc14d1..bacee09b7bfd 100644 --- a/fs/btrfs/delalloc-space.c +++ b/fs/btrfs/delalloc-space.c @@ -115,126 +115,15 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes) { struct btrfs_root *root = inode->root; struct btrfs_fs_info *fs_info = root->fs_info; - struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = 0; - int need_commit = 2; - int have_pinned_space; + enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_DATA; /* Make sure bytes are sectorsize aligned */ bytes = ALIGN(bytes, fs_info->sectorsize); - if (btrfs_is_free_space_inode(inode)) { - need_commit = 0; - ASSERT(current->journal_info); - } - -again: - /* Make sure we have enough space to handle the data first */ - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - - if (used + bytes > data_sinfo->total_bytes) { - struct btrfs_trans_handle *trans; - - /* - * If we don't have enough free bytes in this space then we need - * to alloc a new chunk. - */ - if (!data_sinfo->full) { - u64 alloc_target; - - data_sinfo->force_alloc = CHUNK_ALLOC_FORCE; - spin_unlock(&data_sinfo->lock); - - alloc_target = btrfs_data_alloc_profile(fs_info); - /* - * It is ugly that we don't call nolock join - * transaction for the free space inode case here. - * But it is safe because we only do the data space - * reservation for the free space cache in the - * transaction context, the common join transaction - * just increase the counter of the current transaction - * handler, doesn't try to acquire the trans_lock of - * the fs. - */ - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - - ret = btrfs_chunk_alloc(trans, alloc_target, - CHUNK_ALLOC_NO_FORCE); - btrfs_end_transaction(trans); - if (ret < 0) { - if (ret != -ENOSPC) - return ret; - else { - have_pinned_space = 1; - goto commit_trans; - } - } - - goto again; - } - - /* - * If we don't have enough pinned space to deal with this - * allocation, and no removed chunk in current transaction, - * don't bother committing the transaction. - */ - have_pinned_space = __percpu_counter_compare( - &data_sinfo->total_bytes_pinned, - used + bytes - data_sinfo->total_bytes, - BTRFS_TOTAL_BYTES_PINNED_BATCH); - spin_unlock(&data_sinfo->lock); - - /* Commit the current transaction and try again */ -commit_trans: - if (need_commit) { - need_commit--; - - if (need_commit > 0) { - btrfs_start_delalloc_roots(fs_info, -1); - btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, - (u64)-1); - } - - trans = btrfs_join_transaction(root); - if (IS_ERR(trans)) - return PTR_ERR(trans); - if (have_pinned_space >= 0 || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, - &trans->transaction->flags) || - need_commit > 0) { - ret = btrfs_commit_transaction(trans); - if (ret) - return ret; - /* - * The cleaner kthread might still be doing iput - * operations. Wait for it to finish so that - * more space is released. We don't need to - * explicitly run the delayed iputs here because - * the commit_transaction would have woken up - * the cleaner. - */ - ret = btrfs_wait_on_delayed_iputs(fs_info); - if (ret) - return ret; - goto again; - } else { - btrfs_end_transaction(trans); - } - } - - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", - data_sinfo->flags, bytes, 1); - return -ENOSPC; - } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - spin_unlock(&data_sinfo->lock); + if (btrfs_is_free_space_inode(inode)) + flush = BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE; - return 0; + return btrfs_reserve_data_bytes(fs_info, bytes, flush); } int btrfs_check_data_free_space(struct btrfs_inode *inode, diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index be0f117d4ccf..3a61cdfe4b21 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1329,3 +1329,95 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, } return ret; } + +/** + * btrfs_reserve_data_bytes - try to reserve data bytes for an allocation + * @root - the root we are allocating for + * @bytes - the number of bytes we need + * @flush - how we are allowed to flush + * + * This will reserve bytes from the data space info. If there is not enough + * space then we will attempt to flush space as specified ty flush. + */ +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush) +{ + struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; + const enum btrfs_flush_state *states = NULL; + u64 used; + int states_nr = 0; + int commit_cycles = 2; + int ret = -ENOSPC; + + ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); + + if (flush == BTRFS_RESERVE_FLUSH_DATA) { + states = data_flush_states; + states_nr = ARRAY_SIZE(data_flush_states); + } + + spin_lock(&data_sinfo->lock); +again: + used = btrfs_space_info_used(data_sinfo, true); + + if (used + bytes > data_sinfo->total_bytes) { + u64 prev_total_bytes = data_sinfo->total_bytes; + int flush_state = 0; + + spin_unlock(&data_sinfo->lock); + + /* + * Everybody can force chunk allocation, so try this first to + * see if we can just bail here and make our reservation. + */ + flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); + spin_lock(&data_sinfo->lock); + if (prev_total_bytes < data_sinfo->total_bytes) + goto again; + spin_unlock(&data_sinfo->lock); + + /* + * Cycle through the rest of the flushing options for our flush + * type, then try again. + */ + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + /* + * Previously we unconditionally committed the + * transaction twice before finally checking against + * pinned space before committing the final time. We + * also skipped flushing delalloc the final pass + * through. + */ + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = bytes; + } + + flush_space(fs_info, data_sinfo, flush_bytes, + states[flush_state]); + flush_state++; + } + + if (!commit_cycles) + goto out; + + commit_cycles--; + spin_lock(&data_sinfo->lock); + goto again; + } + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; + spin_unlock(&data_sinfo->lock); +out: + if (ret) + trace_btrfs_space_reservation(fs_info, + "space_info:enospc", + data_sinfo->flags, bytes, 1); + return ret; +} diff --git a/fs/btrfs/space-info.h b/fs/btrfs/space-info.h index c3c64019950a..5646393b928c 100644 --- a/fs/btrfs/space-info.h +++ b/fs/btrfs/space-info.h @@ -149,5 +149,7 @@ static inline void btrfs_space_info_free_bytes_may_use( btrfs_try_granting_tickets(fs_info, space_info); spin_unlock(&space_info->lock); } +int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, + enum btrfs_reserve_flush_enum flush); #endif /* BTRFS_SPACE_INFO_H */ From patchwork Wed Jul 8 14:00:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651731 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10D0B13B6 for ; Wed, 8 Jul 2020 14:00:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EDB7D20786 for ; Wed, 8 Jul 2020 14:00:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="mrDT8L4F" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729611AbgGHOAw (ORCPT ); Wed, 8 Jul 2020 10:00:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOAv (ORCPT ); Wed, 8 Jul 2020 10:00:51 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28C75C08C5CE for ; Wed, 8 Jul 2020 07:00:51 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id i3so34499959qtq.13 for ; Wed, 08 Jul 2020 07:00:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=68CpSjdjlQOq7XkTk1wbYwbFTcHMjsYHB8jLLgt1if8=; b=mrDT8L4F6uwobyJI2AqkkDj+cnhkl9O7XcfPLz2j5sjuLssgvnuEqy+gfaJaQIYSJn qYEymOt3KX/M/PEJBQQDpMBds2vd3yGQe+OJP4neZkbOoNhvkzpjOatCpmtxwUIi8YGi Um7j5Lam0xymoGodmI2IxDgXIEhtVpxvHieRWmDliC1ovF1zYQAtaJV9mOKI5lHwAisF NArWvkJ6UqyYis1Vp9k97ht3uFSqXjYOIw56K15WEvVI50w7FYt57FhurCeME6R2M/f1 dd1ttG1FpaeMSdVrBFshHHf3NX5IrhgZH8nR37GKgQJZEsBCItlxJUyckS71WsSHhU0R +VPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=68CpSjdjlQOq7XkTk1wbYwbFTcHMjsYHB8jLLgt1if8=; b=tWlvXIsBp3jH7wpXXJlRCiNNsP89CxePzMqCfXe5K6vrH0mAM1sQyyc2FG+hq/jaLK ou4/zEqdp6V+/5q4O9SsB9YRyxkJFnESgPVGsFjKmMQd7f7T3/POs5XabupAYRiIH178 /EwY4HAfeEzn7dl+rjwaY015r+i+8jGjQ3p0sspNbWPKlmaUpN0RaeMwvMMBevE6ouxF YI+GQH/fmmBxb+C0y8PTPPj5cIMo3b4NvP1fY09M66nR760Xu1ZjlkvRGbpcobmCtvu2 wy0oNjbARCSNQyRe7eDvYlJVf3mrQQdHNWCLilnQlmtWmDDj/s0GuN7OTg4RaKJCl9z2 pNZQ== X-Gm-Message-State: AOAM532tFx4pPFSzkiHPW+Iwi+O/PgB2YDiE8MgtYqjqYi/6YGS1Af25 6w17n1+amYz7xXsMcYbks/euNU3u4mzsNA== X-Google-Smtp-Source: ABdhPJz3R/iYu8zqJzQJUWcJoCmgZF3fIpy+P9Gk8QyUTG8r77ZAW+oZ0itXeeb2YiGvBFkcRktyMw== X-Received: by 2002:ac8:408c:: with SMTP id p12mr58688382qtl.332.1594216849873; Wed, 08 Jul 2020 07:00:49 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id x4sm23687041qkl.130.2020.07.08.07.00.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:49 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 15/23] btrfs: use ticketing for data space reservations Date: Wed, 8 Jul 2020 10:00:05 -0400 Message-Id: <20200708140013.56994-16-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have all the infrastructure in place, use the ticketing infrastructure to make data allocations. This still maintains the exact same flushing behavior, but now we're using tickets to get our reservations satisfied. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 126 +++++++++++++++++++++++------------------- 1 file changed, 68 insertions(+), 58 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 3a61cdfe4b21..a107370d144c 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1070,6 +1070,54 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, } while (flush_state < states_nr); } +static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, + struct reserve_ticket *ticket, + const enum btrfs_flush_state *states, + int states_nr) +{ + int flush_state = 0; + int commit_cycles = 2; + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + } +again: + while (flush_state < states_nr) { + u64 flush_bytes = U64_MAX; + + if (!commit_cycles) { + if (states[flush_state] == FLUSH_DELALLOC_WAIT) { + flush_state++; + continue; + } + if (states[flush_state] == COMMIT_TRANS) + flush_bytes = ticket->bytes; + } + + flush_space(fs_info, space_info, flush_bytes, + states[flush_state]); + spin_lock(&space_info->lock); + if (ticket->bytes == 0) { + spin_unlock(&space_info->lock); + return; + } + spin_unlock(&space_info->lock); + flush_state++; + } + if (commit_cycles) { + commit_cycles--; + flush_state = 0; + goto again; + } +} + static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket) @@ -1136,6 +1184,15 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; + case BTRFS_RESERVE_FLUSH_DATA: + priority_reclaim_data_space(fs_info, space_info, ticket, + data_flush_states, + ARRAY_SIZE(data_flush_states)); + break; + case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: + priority_reclaim_data_space(fs_info, space_info, ticket, + NULL, 0); + break; default: ASSERT(0); break; @@ -1343,78 +1400,31 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - const enum btrfs_flush_state *states = NULL; u64 used; - int states_nr = 0; - int commit_cycles = 2; int ret = -ENOSPC; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - if (flush == BTRFS_RESERVE_FLUSH_DATA) { - states = data_flush_states; - states_nr = ARRAY_SIZE(data_flush_states); - } - spin_lock(&data_sinfo->lock); -again: used = btrfs_space_info_used(data_sinfo, true); if (used + bytes > data_sinfo->total_bytes) { - u64 prev_total_bytes = data_sinfo->total_bytes; - int flush_state = 0; + struct reserve_ticket ticket; + init_waitqueue_head(&ticket.wait); + ticket.bytes = bytes; + ticket.error = 0; + list_add_tail(&ticket.list, &data_sinfo->priority_tickets); + data_sinfo->reclaim_size += bytes; spin_unlock(&data_sinfo->lock); - /* - * Everybody can force chunk allocation, so try this first to - * see if we can just bail here and make our reservation. - */ - flush_space(fs_info, data_sinfo, bytes, ALLOC_CHUNK_FORCE); - spin_lock(&data_sinfo->lock); - if (prev_total_bytes < data_sinfo->total_bytes) - goto again; + ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, + flush); + } else { + btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); + ret = 0; spin_unlock(&data_sinfo->lock); - - /* - * Cycle through the rest of the flushing options for our flush - * type, then try again. - */ - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - /* - * Previously we unconditionally committed the - * transaction twice before finally checking against - * pinned space before committing the final time. We - * also skipped flushing delalloc the final pass - * through. - */ - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = bytes; - } - - flush_space(fs_info, data_sinfo, flush_bytes, - states[flush_state]); - flush_state++; - } - - if (!commit_cycles) - goto out; - - commit_cycles--; - spin_lock(&data_sinfo->lock); - goto again; } - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); -out: if (ret) trace_btrfs_space_reservation(fs_info, "space_info:enospc", From patchwork Wed Jul 8 14:00:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651733 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B572513B6 for ; Wed, 8 Jul 2020 14:00:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E4E82078B for ; Wed, 8 Jul 2020 14:00:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="O9PhikjI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729758AbgGHOAx (ORCPT ); Wed, 8 Jul 2020 10:00:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOAw (ORCPT ); Wed, 8 Jul 2020 10:00:52 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F620C061A0B for ; Wed, 8 Jul 2020 07:00:52 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id m8so16268770qvk.7 for ; Wed, 08 Jul 2020 07:00:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=46HbDWQcKmcttSQB7AW4PupurGGvNNNEWKfXAY7s/Ks=; b=O9PhikjIA9JGPu5uRkpf9BDS+laBYRKRiP6mA0KwiQbGy3ylVwUBWoO0HN965dJXKW B/u6my+6n7nJt34CoAAZky57OUI4YxxHEmCw5xKckAaLs2icSsVCKlM+e1NEsI/zsehi ZeAnV59q5ff1Hb9pTJQsYGrux9PsbnujaUct7lWJliXOMj8z6O0fRR7ZvhYHaCKzjMSg fH/6w4Mob0CT4GAqbEzyZJbKANPQVmfBkJI6+Wu8cnpSbEd0PGCnL4xymqT1kRwaSQ9Y FFhOQ9r0mz74A4+fQJOJ59mbzjF9Ds83ETFH6W+HSA1yhfXVfyuatluM6aav+hvFpmBo fnDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=46HbDWQcKmcttSQB7AW4PupurGGvNNNEWKfXAY7s/Ks=; b=BbvXo5WWPbPGSPY+uJyG6Mc1wS7DbQd8126fLJq6r+a/mDtSoX1lCELhIIG1/DSN96 8CEnBch65B6FypGBlpm/YqoQMnn9xG1IYADPDW87prq8BiBndrQjNgFaEyihl6WvfORL fl8V/Ph1yyANku/o4bB/AcQWogn+8Iqg+Ixo39DTwD2DT6fw4XKZhg7wWx9LPRBhd2mj DkkMpeWmJWDtaevamrj51YQ9f5ZSsljkyLnOzl7eMewjG6S4TQUEtMq1LCQKuvd92r+p YbU8nBirzNJZ1HOImgJMJqj7Iu0P6s1aF322C/4VFkUt4bZRcvT27sEeTtsbGxE6q+lp 7oIw== X-Gm-Message-State: AOAM531o+7nO3bfrAm77908WvcT+hkIcPeR+XuIJ/GaVAWPMliu9qCUj +vh3ayLnBc7oVoCoKZgA8/OsaNGZe5aHSQ== X-Google-Smtp-Source: ABdhPJy0kEXBgaOEydd7n55oLy32XRWPzBfh+Zzu8+90kxb2qMYing3ZfCyXN/BsLXmn9YnaNnWmdA== X-Received: by 2002:a05:6214:949:: with SMTP id dn9mr54530724qvb.116.1594216851537; Wed, 08 Jul 2020 07:00:51 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id z187sm25898526qkb.102.2020.07.08.07.00.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:50 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 16/23] btrfs: serialize data reservations if we are flushing Date: Wed, 8 Jul 2020 10:00:06 -0400 Message-Id: <20200708140013.56994-17-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Nikolay reported a problem where generic/371 would fail sometimes with a slow drive. The gist of the test is that we fallocate a file in parallel with a pwrite of a different file. These two files combined are smaller than the file system, but sometimes the pwrite would ENOSPC. A fair bit of investigation uncovered the fact that the fallocate workload was racing in and grabbing the free space that the pwrite workload was trying to free up so it could make its own reservation. After a few loops of this eventually the pwrite workload would error out with an ENOSPC. We've had the same problem with metadata as well, and we serialized all metadata allocations to satisfy this problem. This wasn't usually a problem with data because data reservations are more straightforward, but obviously could still happen. Fix this by not allowing reservations to occur if there are any pending tickets waiting to be satisfied on the space info. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index a107370d144c..afdc774cebe9 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1402,13 +1402,17 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; u64 used; int ret = -ENOSPC; + bool pending_tickets; ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); spin_lock(&data_sinfo->lock); used = btrfs_space_info_used(data_sinfo, true); + pending_tickets = !list_empty(&data_sinfo->tickets) || + !list_empty(&data_sinfo->priority_tickets); - if (used + bytes > data_sinfo->total_bytes) { + if (pending_tickets || + used + bytes > data_sinfo->total_bytes) { struct reserve_ticket ticket; init_waitqueue_head(&ticket.wait); From patchwork Wed Jul 8 14:00:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651735 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E09D6618 for ; Wed, 8 Jul 2020 14:00:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C93D02078B for ; Wed, 8 Jul 2020 14:00:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="bVyVtMZs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729479AbgGHOAz (ORCPT ); Wed, 8 Jul 2020 10:00:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOAy (ORCPT ); Wed, 8 Jul 2020 10:00:54 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7CE4C061A0B for ; Wed, 8 Jul 2020 07:00:54 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id i3so34500159qtq.13 for ; Wed, 08 Jul 2020 07:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sdzRsWOXAGvB73dIlXqZ3dGxDme3VsWQ5fDO5h3OlXQ=; b=bVyVtMZsTU6AuejjaQNhXDcnvf1kWfl9zRgK36ILS/nfwORGM9YcWoXU1Y6C7LlXKQ QbRCBtvRyiAgh6c0xYSIvqLHD+Ac74qQNgmd0iqMnYBXoWjEuivU8g5xl6ImzfCYf8Q2 QIJmRV+NzwtycF8KbrTo6V9/MaP6c70eMrOoxvQuDGVGZCqkMY7DF+/HlQZ7VhkA97/H erRhZYVHp75OtfFuCkECWrpYjn5wb/6dSoHgMX80mqtdVOVlCqB2yruRN5KGC2OOT92K YeI2lDsbaU4jsSPrYh6d2W4uN8QqLQEtUwOtEZC0jzemdUTtK7kLi602meSaWjdoHHD5 2KFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sdzRsWOXAGvB73dIlXqZ3dGxDme3VsWQ5fDO5h3OlXQ=; b=Cyc4w9LWyaKQvT8GbzHoWenDYBGOlMHhnAjTvBaHHDXRMXGosnHeYhnVgctQmKuqaH Azn777y+qvQsb/SWkt9E/eMrfTfp60uag/jrsyZ5mybevhzypTt+1bH+eApyG2P1NDdD +NuXgXK1gE77oyp0EsO4HDhegC72kJvKTAlU27dEysTW4nBdmMqs9VS5oI+iVm00klvk RylZwetJAMB7+6Jdj2fHqsoDya3N7hKQhZKtss+bnEuAIVxjYgQ3+KUbKIA7iTSuYDy5 OHSdMEnWHYhqvUp/JOeI9VCp8mNXQiH0EydJrOvD5OVGUBAP6i34GFYbDPFwnH8+rBfL 2oJg== X-Gm-Message-State: AOAM533WkV5LA+DOtEz2VVXOjRCtXpnTAreSgii2st6yyTvQFJrV7m5/ 4xSBMhXfws21+uLzzWK8WNCfhOvTDl9W+g== X-Google-Smtp-Source: ABdhPJxnHQ+tjev3fVQnTM5pgN7IhZKFgFh0Y96KdfyHas9tdLkoA6AKTQXR24nHz+ySHdTkaBRpxQ== X-Received: by 2002:ac8:2928:: with SMTP id y37mr61026724qty.245.1594216853633; Wed, 08 Jul 2020 07:00:53 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id o20sm31783371qtk.56.2020.07.08.07.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:52 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 17/23] btrfs: use the same helper for data and metadata reservations Date: Wed, 8 Jul 2020 10:00:07 -0400 Message-Id: <20200708140013.56994-18-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that data reservations follow the same pattern as metadata reservations we can simply rename __reserve_metadata_bytes to __reserve_bytes and use that helper for data reservations. Things to keep in mind, btrfs_can_overcommit() returns 0 for data, because we can never overcommit. We also will never pass in FLUSH_ALL for data, so we'll simply be added to the priority list and go straight into handle_reserve_ticket. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 49 +++++++++++++------------------------------ 1 file changed, 14 insertions(+), 35 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index afdc774cebe9..605be9f4fcac 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1252,10 +1252,9 @@ static inline bool is_normal_flushing(enum btrfs_reserve_flush_enum flush) * regain reservations will be made and this will fail if there is not enough * space already. */ -static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 orig_bytes, - enum btrfs_reserve_flush_enum flush) +static int __reserve_bytes(struct btrfs_fs_info *fs_info, + struct btrfs_space_info *space_info, u64 orig_bytes, + enum btrfs_reserve_flush_enum flush) { struct reserve_ticket ticket; u64 used; @@ -1367,8 +1366,8 @@ int btrfs_reserve_metadata_bytes(struct btrfs_root *root, struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; int ret; - ret = __reserve_metadata_bytes(fs_info, block_rsv->space_info, - orig_bytes, flush); + ret = __reserve_bytes(fs_info, block_rsv->space_info, orig_bytes, + flush); if (ret == -ENOSPC && unlikely(root->orphan_cleanup_state == ORPHAN_CLEANUP_STARTED)) { if (block_rsv != global_rsv && @@ -1400,38 +1399,18 @@ int btrfs_reserve_data_bytes(struct btrfs_fs_info *fs_info, u64 bytes, enum btrfs_reserve_flush_enum flush) { struct btrfs_space_info *data_sinfo = fs_info->data_sinfo; - u64 used; - int ret = -ENOSPC; - bool pending_tickets; + int ret; + ASSERT(flush == BTRFS_RESERVE_FLUSH_DATA || + flush == BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_DATA); - spin_lock(&data_sinfo->lock); - used = btrfs_space_info_used(data_sinfo, true); - pending_tickets = !list_empty(&data_sinfo->tickets) || - !list_empty(&data_sinfo->priority_tickets); - - if (pending_tickets || - used + bytes > data_sinfo->total_bytes) { - struct reserve_ticket ticket; - - init_waitqueue_head(&ticket.wait); - ticket.bytes = bytes; - ticket.error = 0; - list_add_tail(&ticket.list, &data_sinfo->priority_tickets); - data_sinfo->reclaim_size += bytes; - spin_unlock(&data_sinfo->lock); - - ret = handle_reserve_ticket(fs_info, data_sinfo, &ticket, - flush); - } else { - btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes); - ret = 0; - spin_unlock(&data_sinfo->lock); - } - if (ret) - trace_btrfs_space_reservation(fs_info, - "space_info:enospc", + ret = __reserve_bytes(fs_info, data_sinfo, bytes, flush); + if (ret == -ENOSPC) { + trace_btrfs_space_reservation(fs_info, "space_info:enospc", data_sinfo->flags, bytes, 1); + if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) + btrfs_dump_space_info(fs_info, data_sinfo, bytes, 0); + } return ret; } From patchwork Wed Jul 8 14:00:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651737 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A042618 for ; Wed, 8 Jul 2020 14:00:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FC8B20786 for ; Wed, 8 Jul 2020 14:00:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="Mr5qqmiH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729774AbgGHOA5 (ORCPT ); Wed, 8 Jul 2020 10:00:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729762AbgGHOA4 (ORCPT ); Wed, 8 Jul 2020 10:00:56 -0400 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1FC1C08C5CE for ; Wed, 8 Jul 2020 07:00:56 -0700 (PDT) Received: by mail-qt1-x841.google.com with SMTP id d27so34523108qtg.4 for ; Wed, 08 Jul 2020 07:00:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HYvFTR2wkUcDKdml4Bzn2R6cx4pTZvSFMrHCayhIgeY=; b=Mr5qqmiH9QrWMRU2lz1gEjLPOFc2wMgaTFgrfyyM6RpgDS+UI28fltDw7gGhFzQcs2 fbr1cTNXG9f7wC3nAFoVDQSczxYxMs+4epS8kGY0Z0B4ufDzQxpHga/uAg3EXd9ZGZFL LpT4vOF56CcryDzhyG+bIpS8db8jFj5I374d59ysmwXNLbYuEZWYBq1B/VKX78kKiRZL YYqIW4gnObMnI0joSSAh86JT9u0xA28B6yQbH+R8rbOtuuCv05C8bczHQT50H51yLInY jh3wAYUeyJJPM6kiuYAa79jCk4ibz8Sg/TLR0AHe4eVNkoYER0pGBR2m4LOnlZX5Xf1b KDEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HYvFTR2wkUcDKdml4Bzn2R6cx4pTZvSFMrHCayhIgeY=; b=Oc35aBg0mVWoA8NGIFBz01J4+E3DI29VQb74Mcnaw+QoiocOzNkSVCiD40mBZeN+A3 yy5RUqBleW7NKeXPiGFyQSlkRRWNMAjR4SoVG3LfLMPoE+oJEqQXLt9L/+jCAb1uTOkv 275mcj47e5TiEDV3Z9EFp22CTGu+BnVaMnuzsei/qZdxaN5EHi9nS6IRAeG5X9Q4+Nsm gmKc/esHu5DgbKyIaMGKbpU/kt+u24YW/3JCIn74+lR/eOXKaeoIHhivqgvK0ZoDs7dM AU9P28HOfc7QdjQ/KtajojswvJij5hMHDIJ9WeY9lAtHwKQsnFaiNKf7ofK1eIDTi7Tv 2cKg== X-Gm-Message-State: AOAM5309WtzgxLXJxWXSJnVnCrlwSmPNjU+cxasqY8o1nEWkgb84Z+2X /C4QpojNnl088MWIG7EXfc49kEtSXm94Jw== X-Google-Smtp-Source: ABdhPJxsN9plOPJbViKqNftGfIcSOcOg6QzKSRWqFGtJDQ2BPNhlqLJelrsmcOtroOH9eUjGMoGW1g== X-Received: by 2002:ac8:3fac:: with SMTP id d41mr57082119qtk.98.1594216855533; Wed, 08 Jul 2020 07:00:55 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id x3sm29112611qkd.62.2020.07.08.07.00.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:54 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 18/23] btrfs: drop the commit_cycles stuff for data reservations Date: Wed, 8 Jul 2020 10:00:08 -0400 Message-Id: <20200708140013.56994-19-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This was an old wart left over from how we previously did data reservations. Before we could have people race in and take a reservation while we were flushing space, so we needed to make sure we looped a few times before giving up. Now that we're using the ticketing infrastructure we don't have to worry about this and can drop the logic altogether. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 605be9f4fcac..5efb3eb9253d 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1077,7 +1077,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, int states_nr) { int flush_state = 0; - int commit_cycles = 2; while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); @@ -1088,21 +1087,9 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } -again: - while (flush_state < states_nr) { - u64 flush_bytes = U64_MAX; - - if (!commit_cycles) { - if (states[flush_state] == FLUSH_DELALLOC_WAIT) { - flush_state++; - continue; - } - if (states[flush_state] == COMMIT_TRANS) - flush_bytes = ticket->bytes; - } - flush_space(fs_info, space_info, flush_bytes, - states[flush_state]); + while (flush_state < states_nr) { + flush_space(fs_info, space_info, U64_MAX, states[flush_state]); spin_lock(&space_info->lock); if (ticket->bytes == 0) { spin_unlock(&space_info->lock); @@ -1111,11 +1098,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, spin_unlock(&space_info->lock); flush_state++; } - if (commit_cycles) { - commit_cycles--; - flush_state = 0; - goto again; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, From patchwork Wed Jul 8 14:00:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651739 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7400113B6 for ; Wed, 8 Jul 2020 14:01:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5C6C32078B for ; Wed, 8 Jul 2020 14:01:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="17zYMwTn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729780AbgGHOA7 (ORCPT ); Wed, 8 Jul 2020 10:00:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOA6 (ORCPT ); Wed, 8 Jul 2020 10:00:58 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD800C061A0B for ; Wed, 8 Jul 2020 07:00:58 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id e11so41615775qkm.3 for ; Wed, 08 Jul 2020 07:00:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JvyDgN6grpnYsv6EV70BSXlJ3Z3zZRzxM5sfFDQKHac=; b=17zYMwTnSdK+orznCF1jojxVmUebaK7FyWO6Vl9k5zVK96o5OArnGNS8ifYvesNSSq /jsO9NUQPd3qgmURC1FN0zPGt0/WS1IY92BjSacEhjmRVeGVu3yphJ5hQwMln1HO+b3S 6sXCKVehWQDs67ueHjtbMMGBhe9muFFwoBjj0S1rB6Mgv2lNKnFyoGgSSxx8CbjBhb2p PfaFBi3J7Tz0TlL0Vi4TBgh6l8h315TLLrAc5IH3duyCk+URAndhfWLTF3ZfKzKKSK74 vkidBjGhqytYqFbbVtxm9ae3aQswMFxYwPUioJLUXtuLn0mkPSM0w67/1HqjVATB4KO1 2yFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JvyDgN6grpnYsv6EV70BSXlJ3Z3zZRzxM5sfFDQKHac=; b=ZeDccOW2CZqj61IP1tbcDIxTRQH+Jz0457XLjy7/Hng23su335B85XRMzPQXdPiJRI 67UwG0d8F9Nnj0tyiGX3Ki0+4wkQrI9demwBiLTirTawAJIGxokJcA24U4KDghs9+H/m UdLvxxpWEtgb2APADFiIltd1zTdY3H2jlt6p92YBoTOa+1nZMk1zB//mHUS4bERJlM6E k79emupTzYOb8JjJ/tUcoOkl6hzhD8FfUAqtIubs946sU9i2nX4El2ewwO5gtYJTpGu2 rRn9o3UeF7EehRJVT7HWDISEMLmTPQOwxzzpYjIuy4j6g1D5daDUKkGKYGQVVDbjDnj3 kXhw== X-Gm-Message-State: AOAM530Jqn6w6jlHvIUGgaPauB8YO7nceRnUPO84y3HBR3QmxOWpe0vi D9CWF/RG+s7k1Y/ZUnKhZ6rocOrSTDP9Xg== X-Google-Smtp-Source: ABdhPJz36cwYVGlff7zZEp/3YrXAizIAa2nCaWZn4wHmzIRbX3hVDNo5/pVvpO2X5h28Na6y3Us+SA== X-Received: by 2002:a05:620a:992:: with SMTP id x18mr55593945qkx.273.1594216857537; Wed, 08 Jul 2020 07:00:57 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id m26sm30020897qtc.83.2020.07.08.07.00.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:56 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 19/23] btrfs: don't force commit if we are data Date: Wed, 8 Jul 2020 10:00:09 -0400 Message-Id: <20200708140013.56994-20-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We used to unconditionally commit the transaction at least 2 times and then on the 3rd try check against pinned space to make sure committing the transaction was worth the effort. This is overkill, we know nobody is going to steal our reservation, and if we can't make our reservation with the pinned amount simply bail out. This also cleans up the passing of bytes_needed to may_commit_transaction, as that was the thing we added into place in order to accomplish this behavior. We no longer need it so remove that mess. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 5efb3eb9253d..5b46835766e3 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -579,8 +579,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, * will return -ENOSPC. */ static int may_commit_transaction(struct btrfs_fs_info *fs_info, - struct btrfs_space_info *space_info, - u64 bytes_needed) + struct btrfs_space_info *space_info) { struct reserve_ticket *ticket = NULL; struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; @@ -588,24 +587,13 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *trans_rsv = &fs_info->trans_block_rsv; struct btrfs_trans_handle *trans; u64 reclaim_bytes = 0; + u64 bytes_needed; u64 cur_free_bytes = 0; - bool do_commit = false; trans = (struct btrfs_trans_handle *)current->journal_info; if (trans) return -EAGAIN; - /* - * If we are data and have passed in U64_MAX we just want to - * unconditionally commit the transaction to match the previous data - * flushing behavior. - */ - if ((space_info->flags & BTRFS_BLOCK_GROUP_DATA) && - bytes_needed == U64_MAX) { - do_commit = true; - goto check_pinned; - } - spin_lock(&space_info->lock); cur_free_bytes = btrfs_space_info_used(space_info, true); if (cur_free_bytes < space_info->total_bytes) @@ -619,7 +607,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, else if (!list_empty(&space_info->tickets)) ticket = list_first_entry(&space_info->tickets, struct reserve_ticket, list); - bytes_needed = (ticket) ? ticket->bytes : bytes_needed; + bytes_needed = (ticket) ? ticket->bytes : 0; if (bytes_needed > cur_free_bytes) bytes_needed -= cur_free_bytes; @@ -630,7 +618,6 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, if (!bytes_needed) return 0; -check_pinned: trans = btrfs_join_transaction(fs_info->extent_root); if (IS_ERR(trans)) return PTR_ERR(trans); @@ -640,8 +627,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, * we have block groups that are going to be freed, allowing us to * possibly do a chunk allocation the next loop through. */ - if (do_commit || - test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || + if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) || __percpu_counter_compare(&space_info->total_bytes_pinned, bytes_needed, BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0) @@ -759,7 +745,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, btrfs_wait_on_delayed_iputs(fs_info); break; case COMMIT_TRANS: - ret = may_commit_transaction(fs_info, space_info, num_bytes); + ret = may_commit_transaction(fs_info, space_info); break; default: ret = -ENOSPC; From patchwork Wed Jul 8 14:00:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1192618 for ; Wed, 8 Jul 2020 14:01:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA0A92078B for ; Wed, 8 Jul 2020 14:01:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="bHUnXXQk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729792AbgGHOBB (ORCPT ); Wed, 8 Jul 2020 10:01:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729783AbgGHOBB (ORCPT ); Wed, 8 Jul 2020 10:01:01 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3790C061A0B for ; Wed, 8 Jul 2020 07:01:00 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id c30so37682588qka.10 for ; Wed, 08 Jul 2020 07:01:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=frCUE3S3/bN7rWND0qsjSP1NCZLi+CeRQ3+FfDokoQY=; b=bHUnXXQk8F+WO5qpAdwlMmQ8wigThK6hz8E5KC81OThUnRwza33Ft1akSWK9iSKsgF vriLQdpUhXcdLeu5MQZlDOXt3RajWMaFqx4AeWqiusw9zUv5HpV9F69ZRoht059hxeOO f+ag5l0T+onLmaqPYl2rEXi+JI1VUT4CUeog7JR5B9AacApi2//FrW/1xfl1fLuhogCW 0PofJsjnhP25nO2SeNJxPvd6UrFkq+k5O/jfiQBDKlx+MlnzOwrCmOUuzvIwqFgd40Gk R+kBH8KRhSvtHHsLDhCD2sKxxk7P8tCIP73Z9uMoOxB0CmBPf5/846dhIB+L8AtsG7bS E/Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=frCUE3S3/bN7rWND0qsjSP1NCZLi+CeRQ3+FfDokoQY=; b=cfe1OUvfS6WrQ7kWI/jHCr+iEI+Hzh8OSPDhNELajTnG+i/73/osArx7FMkCX/Mmte dA3MeyRBc+RbxLqBCjEkF30cPqonl7X9FEcv0BdLkv74vmG6KyJxfV7dy8VgfP6w4Q+Y 1FilSuAYfyADTrfkYFox9X8Sj9drnwyemmAX/qM0lHMqjCDS+OFEO/s0J2UlGWHJjPP3 RZCybz6esVoZM5bwZsW6HQu8LF/A0EMq51+/l56q4Nazo6s9HMJCyxMLuvUtrDU8opme 4PZyKzWWLkKyZpnY8WTGIRa4FLc91xQ26d9jJNpl7wP/T4eu8ZG13DT77WbMyXaRpusQ HvTg== X-Gm-Message-State: AOAM531M89gPUJCBG7s0lh0Qe4DzR0+ro/WOUT0zgwTGPbeDVoor95RF /2R/IPH6Ft08Uxtp1tVms7fqplFFQaTY/Q== X-Google-Smtp-Source: ABdhPJwGedtf90J1nd2PFOwS1KEw8BvwEZwrWtgfusUbXT415prvl6PYkFkYfmfLw0fObPnjzbKNuA== X-Received: by 2002:a05:620a:11b3:: with SMTP id c19mr56943887qkk.203.1594216859497; Wed, 08 Jul 2020 07:00:59 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id n64sm27340458qke.77.2020.07.08.07.00.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:00:58 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 20/23] btrfs: run delayed iputs before committing the transaction for data Date: Wed, 8 Jul 2020 10:00:10 -0400 Message-Id: <20200708140013.56994-21-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Before we were waiting on iputs after we committed the transaction, but this doesn't really make much sense. We want to reclaim any space we may have in order to be more likely to commit the transaction, due to pinned space being added by running the delayed iputs. Fix this by making delayed iputs run before committing the transaction. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 5b46835766e3..092f3f62a5ef 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1022,8 +1022,8 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, - COMMIT_TRANS, RUN_DELAYED_IPUTS, + COMMIT_TRANS, }; static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, From patchwork Wed Jul 8 14:00:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651743 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B41713B6 for ; Wed, 8 Jul 2020 14:01:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3441E2078B for ; Wed, 8 Jul 2020 14:01:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="qlTcLPWP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729805AbgGHOBD (ORCPT ); Wed, 8 Jul 2020 10:01:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOBC (ORCPT ); Wed, 8 Jul 2020 10:01:02 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79990C061A0B for ; Wed, 8 Jul 2020 07:01:02 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id k18so4201658qtm.10 for ; Wed, 08 Jul 2020 07:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8xmiqq+ZSGW48hf2wB/mM0xQesV//E9KoAcumehKGgc=; b=qlTcLPWP2aaLK6gQMfqIPIgkLHBcOTliv0YRnitEmaJCaLppOT18QV4xseZJj3PHk8 /MbzWH4oCmtP9V7pv5M3SFXqyIf/zAD54gxqeMK5XC3hzOFjR7Z3hREvkiQJSxU/tKG+ gyNTXjbHZ7GjVQVWu+BmmEoKB1Mh3thfxixxs5AAdgzRT+QEAnDtkFPxal22pbMYEkUh mXcuzVB1Dqofor0SDURDhqNlnRghzj9diMKDtCokyoB8qnnqTIt782uGoxJZOw4mY5Id FupPWO6zbgWAkEHB5124s8D3zn6dPCTTJ+0R5AIliAmV9fGPd/315j5rRBqaFpx3arJf Q0sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8xmiqq+ZSGW48hf2wB/mM0xQesV//E9KoAcumehKGgc=; b=oeB+lEsADIc7cs4UIaQx8sAAx7iarjso+B8U2GuUkJuB1dDq9DbIBjAviWCFwaDGX4 wUPq0BjaHtEaJfY6ZoxBWTr9EYu5yWpOcIXbMyWhYsQnF9LQvfXiGcXiQR+hUn70fj9M ncHTJoYTLDwXnqzx4HqGUNliFwOUbaHqNF05rUb600jopmulWeUQD0RogTtGUN5VfbDU YWRgFnWQDHoxEkPQYRwWC5G7ddgjTsXjX5L0BEWhlWy9i3Jwpkwzf09NRdvlZ0FkOuuK s6MUo5B5nGfoDKfsM0QPwsALTqQIwVCHm8cQPNbiRTldwtZ6c48qdi970zpfa0H9v4oX OvdA== X-Gm-Message-State: AOAM531zVLRBrhwBJ9O3xb0dGq/g17l5Holnk/WF7mhdDFzSqPQbi7Db MUuhlCZ+hOuZ/cJkYSsReGVKYtPz7g88Lw== X-Google-Smtp-Source: ABdhPJxYQ0sQmBqbLScS49yg2Q4Fgwqs0zcqm6aZHuYt69Vy9Cdw+jsNA6cRzof1YcrNm9IJj5V4Bw== X-Received: by 2002:ac8:4c85:: with SMTP id j5mr63025678qtv.219.1594216861420; Wed, 08 Jul 2020 07:01:01 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id w4sm31166898qtc.5.2020.07.08.07.01.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:01:00 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 21/23] btrfs: flush delayed refs when trying to reserve data space Date: Wed, 8 Jul 2020 10:00:11 -0400 Message-Id: <20200708140013.56994-22-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We can end up with free'd extents in the delayed refs, and thus may_commit_transaction() may not think we have enough pinned space to commit the transaction and we'll ENOSPC early. Handle this by running the delayed refs in order to make sure pinned is uptodate before we try to commit the transaction. Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 092f3f62a5ef..fce61f800150 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -1023,6 +1023,7 @@ static const enum btrfs_flush_state evict_flush_states[] = { static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, COMMIT_TRANS, }; From patchwork Wed Jul 8 14:00:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651745 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4C3F13B6 for ; Wed, 8 Jul 2020 14:01:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B444E20786 for ; Wed, 8 Jul 2020 14:01:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="EYeQWwxv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729810AbgGHOBH (ORCPT ); Wed, 8 Jul 2020 10:01:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729732AbgGHOBE (ORCPT ); Wed, 8 Jul 2020 10:01:04 -0400 Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A70B8C061A0B for ; Wed, 8 Jul 2020 07:01:04 -0700 (PDT) Received: by mail-qk1-x743.google.com with SMTP id e13so41560056qkg.5 for ; Wed, 08 Jul 2020 07:01:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Oe8IenWGfx01XjKFxdN5owSVLszP7xfQSpoHZWdTA2w=; b=EYeQWwxvF57D9MYHSLWo/bJo8cOLBlWvi+CN6u9aev+cRIJvXiUgYHJuX+ubLeWhYh Wwqa4mnFlIUGYwbSPpvi7NU23MsbiYobyjxUV+ZQkcV8+B2YnIJfWAxsd/K5qDoR9r0H fHgcadPziVsRk6hlBcAKHBWQFGuwvn5EOOhKQd5j9PonugbtooKX9pehOcX4D3RlAY/h U1xlxEfs7IopMckVVqnzJMFO7Fwqjv0Zz1/49LEXSBOPCcXXa5bG0BGwR1c5ptRecvKt ziqS1nDekfVaKqJm10LlpDkFzXCBpsEhicMYiHir/gA/4FTLaGH6F68UT8bJSVGpAnGJ nZ6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Oe8IenWGfx01XjKFxdN5owSVLszP7xfQSpoHZWdTA2w=; b=Z++3dFEQNlAjAaVAi/qi+cgHNC+GQJ5Fn7hNpsYQEmTyU0oVs+pb8mB25NLBk0BebZ d6UJwuMPbe/9LjAWvAtw3AJ9lU1+uSH3pvtx6Cd+YVEmQKQjb+DhENLn082zuBZCMSZK c0+u99OGiAG3lp2wLaQ92Q74U+o5UEFlKnCQyqpQ98M9HUNzhks7TKX39py/okEXZHZL fVsGDtez3IW501BY0LCw3KwaLnQ6I7S4/mfiE8cHXNXZvuih4zREj04vod+4o0l0ymQ7 GgtEADSZVVGbbDQYZfxbS4yphfnHjXYQnGQDaajKxKzojXrLFXHmCbl7H2wXqbVqRDle ofJw== X-Gm-Message-State: AOAM533MnagrNfgRyOo/OdpteD9poFWB1bzSohsG9CfTt0VrDyPEZwDT BSrO9fztZdDZJA5IxRUNl4A14LuwSY8I7A== X-Google-Smtp-Source: ABdhPJy7be3uov+oRNZKRYo331c0nBa5/cTcAAXWz6DszKUE5e7vrvj2t19E2TWNUX08YuyQwC6cjw== X-Received: by 2002:a37:7741:: with SMTP id s62mr41233306qkc.263.1594216863334; Wed, 08 Jul 2020 07:01:03 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id k197sm28251563qke.133.2020.07.08.07.01.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:01:02 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Nikolay Borisov Subject: [PATCH 22/23] btrfs: do async reclaim for data reservations Date: Wed, 8 Jul 2020 10:00:12 -0400 Message-Id: <20200708140013.56994-23-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we have the data ticketing stuff in place, move normal data reservations to use an async reclaim helper to satisfy tickets. Before we could have multiple tasks race in and both allocate chunks, resulting in more data chunks than we would necessarily need. Serializing these allocations and making a single thread responsible for flushing will only allocate chunks as needed, as well as cut down on transaction commits and other flush related activities. Priority reservations will still work as they have before, simply trying to allocate a chunk until they can make their reservation. Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 +- fs/btrfs/disk-io.c | 3 +- fs/btrfs/space-info.c | 123 ++++++++++++++++++++++++++++++------------ fs/btrfs/super.c | 1 + 4 files changed, 93 insertions(+), 37 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 4eabad429440..aa6532976404 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -494,7 +494,7 @@ enum btrfs_orphan_cleanup_state { ORPHAN_CLEANUP_DONE = 2, }; -void btrfs_init_async_reclaim_work(struct work_struct *work); +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info); /* fs_info */ struct reloc_control; @@ -917,6 +917,7 @@ struct btrfs_fs_info { /* Used to reclaim the metadata space in the background. */ struct work_struct async_reclaim_work; + struct work_struct async_data_reclaim_work; spinlock_t unused_bgs_lock; struct list_head unused_bgs; diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index aa97a39e4b15..08140caceb33 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2779,7 +2779,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info) fs_info->check_integrity_print_mask = 0; #endif btrfs_init_balance(fs_info); - btrfs_init_async_reclaim_work(&fs_info->async_reclaim_work); + btrfs_init_async_reclaim_work(fs_info); spin_lock_init(&fs_info->block_group_cache_lock); fs_info->block_group_cache_tree = RB_ROOT; @@ -4099,6 +4099,7 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) btrfs_cleanup_defrag_inodes(fs_info); cancel_work_sync(&fs_info->async_reclaim_work); + cancel_work_sync(&fs_info->async_data_reclaim_work); /* Cancel or finish ongoing discard work */ btrfs_discard_cleanup(fs_info); diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index fce61f800150..680218a7b0e5 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -998,9 +998,83 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } -void btrfs_init_async_reclaim_work(struct work_struct *work) +static const enum btrfs_flush_state data_flush_states[] = { + FLUSH_DELALLOC_WAIT, + RUN_DELAYED_IPUTS, + FLUSH_DELAYED_REFS, + COMMIT_TRANS, +}; + +static void btrfs_async_reclaim_data_space(struct work_struct *work) { - INIT_WORK(work, btrfs_async_reclaim_metadata_space); + struct btrfs_fs_info *fs_info; + struct btrfs_space_info *space_info; + u64 last_tickets_id; + int flush_state = 0; + + fs_info = container_of(work, struct btrfs_fs_info, + async_data_reclaim_work); + space_info = fs_info->data_sinfo; + + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + + while (!space_info->full) { + flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + last_tickets_id = space_info->tickets_id; + spin_unlock(&space_info->lock); + } + + while (flush_state < ARRAY_SIZE(data_flush_states)) { + flush_space(fs_info, space_info, U64_MAX, + data_flush_states[flush_state]); + spin_lock(&space_info->lock); + if (list_empty(&space_info->tickets)) { + space_info->flush = 0; + spin_unlock(&space_info->lock); + return; + } + + if (last_tickets_id == space_info->tickets_id) { + flush_state++; + } else { + last_tickets_id = space_info->tickets_id; + flush_state = 0; + } + + if (flush_state >= ARRAY_SIZE(data_flush_states)) { + if (space_info->full) { + if (maybe_fail_all_tickets(fs_info, + space_info)) + flush_state = 0; + else + space_info->flush = 0; + } else { + flush_state = 0; + } + } + spin_unlock(&space_info->lock); + } +} + +void btrfs_init_async_reclaim_work(struct btrfs_fs_info *fs_info) +{ + INIT_WORK(&fs_info->async_reclaim_work, + btrfs_async_reclaim_metadata_space); + INIT_WORK(&fs_info->async_data_reclaim_work, + btrfs_async_reclaim_data_space); } static const enum btrfs_flush_state priority_flush_states[] = { @@ -1020,13 +1094,6 @@ static const enum btrfs_flush_state evict_flush_states[] = { COMMIT_TRANS, }; -static const enum btrfs_flush_state data_flush_states[] = { - FLUSH_DELALLOC_WAIT, - RUN_DELAYED_IPUTS, - FLUSH_DELAYED_REFS, - COMMIT_TRANS, -}; - static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, @@ -1059,12 +1126,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, - struct reserve_ticket *ticket, - const enum btrfs_flush_state *states, - int states_nr) + struct reserve_ticket *ticket) { - int flush_state = 0; - while (!space_info->full) { flush_space(fs_info, space_info, U64_MAX, ALLOC_CHUNK_FORCE); spin_lock(&space_info->lock); @@ -1074,17 +1137,6 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, } spin_unlock(&space_info->lock); } - - while (flush_state < states_nr) { - flush_space(fs_info, space_info, U64_MAX, states[flush_state]); - spin_lock(&space_info->lock); - if (ticket->bytes == 0) { - spin_unlock(&space_info->lock); - return; - } - spin_unlock(&space_info->lock); - flush_state++; - } } static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, @@ -1139,6 +1191,7 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, int ret; switch (flush) { + case BTRFS_RESERVE_FLUSH_DATA: case BTRFS_RESERVE_FLUSH_ALL: case BTRFS_RESERVE_FLUSH_ALL_STEAL: wait_reserve_ticket(fs_info, space_info, ticket); @@ -1153,14 +1206,8 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, evict_flush_states, ARRAY_SIZE(evict_flush_states)); break; - case BTRFS_RESERVE_FLUSH_DATA: - priority_reclaim_data_space(fs_info, space_info, ticket, - data_flush_states, - ARRAY_SIZE(data_flush_states)); - break; case BTRFS_RESERVE_FLUSH_FREE_SPACE_INODE: - priority_reclaim_data_space(fs_info, space_info, ticket, - NULL, 0); + priority_reclaim_data_space(fs_info, space_info, ticket); break; default: ASSERT(0); @@ -1225,6 +1272,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, u64 orig_bytes, enum btrfs_reserve_flush_enum flush) { + struct work_struct *async_work; struct reserve_ticket ticket; u64 used; int ret = 0; @@ -1233,6 +1281,11 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, ASSERT(orig_bytes); ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_ALL); + if (flush == BTRFS_RESERVE_FLUSH_DATA) + async_work = &fs_info->async_data_reclaim_work; + else + async_work = &fs_info->async_reclaim_work; + spin_lock(&space_info->lock); ret = -ENOSPC; used = btrfs_space_info_used(space_info, true); @@ -1274,7 +1327,8 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, init_waitqueue_head(&ticket.wait); ticket.steal = (flush == BTRFS_RESERVE_FLUSH_ALL_STEAL); if (flush == BTRFS_RESERVE_FLUSH_ALL || - flush == BTRFS_RESERVE_FLUSH_ALL_STEAL) { + flush == BTRFS_RESERVE_FLUSH_ALL_STEAL || + flush == BTRFS_RESERVE_FLUSH_DATA) { list_add_tail(&ticket.list, &space_info->tickets); if (!space_info->flush) { space_info->flush = 1; @@ -1282,8 +1336,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, space_info->flags, orig_bytes, flush, "enospc"); - queue_work(system_unbound_wq, - &fs_info->async_reclaim_work); + queue_work(system_unbound_wq, async_work); } } else { list_add_tail(&ticket.list, diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index f90ee35465ab..b44e859c3d5c 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1868,6 +1868,7 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) * the filesystem is busy. */ cancel_work_sync(&fs_info->async_reclaim_work); + cancel_work_sync(&fs_info->async_data_reclaim_work); btrfs_discard_cleanup(fs_info); From patchwork Wed Jul 8 14:00:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11651747 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1432618 for ; Wed, 8 Jul 2020 14:01:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99DF02078B for ; Wed, 8 Jul 2020 14:01:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="gX4fb3wX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729815AbgGHOBH (ORCPT ); Wed, 8 Jul 2020 10:01:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729806AbgGHOBH (ORCPT ); Wed, 8 Jul 2020 10:01:07 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1285C061A0B for ; Wed, 8 Jul 2020 07:01:06 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id di5so15546468qvb.11 for ; Wed, 08 Jul 2020 07:01:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=453v9VzQmoK5i/nqibBwzH7YHdJp2eB/54xxH6Xi1B8=; b=gX4fb3wXzNkwkyhXjnt/yWfDUws9AfhEVY3PvZybJ5xLBziW14TZURkzqmRae9oAdj RtUc15uDvdBE+831QfrGhAhd2/F0aR24HoHqyJPOaJnDabJ2mBxGOJywSfkgtAMqRWIo sp8eWl0iNmBuIv/hPEGfxg8GVJv1Y5wFXdiPr49Nw2BPg/KgSSIRCpcw+apBNpbYO1es LSUhM6VykKlnve9KcnljAS5ReT5KBDLwn/FFIeQL8vKI2/BPFFfM3MWr0Bw16rM0T0Dr wxmRvQoezt7n+2M94KrSD+ztsKQm4j4yWkGicHPe92OCRA5pMnrBppBDsmJD5C7z3BtY Fpvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=453v9VzQmoK5i/nqibBwzH7YHdJp2eB/54xxH6Xi1B8=; b=rjljbqG/bJSrZn0SNFR5W4uv5ac8y0F5CF7gr3upBpKs8VVQcwjGlG2RBcxj3YDr9y sgYgnowIW7tsZ5jtG9CvmyHEtk9ghJsMh6L2pSHzI+XfoUoorfNL+qoiTy7s8dBXWlrQ qt7mvpBRWtNi9Kd//eRH90vM97VD9LmZZnWyI41w4M2VtnOE9bgKoJtevn623Idg5Zpr MmzonEVJGAXaeeVWgAVOtk2NobcLWL2yLfrVIklqHW6KtoaXPUkvz/hVbIIL2YG9ZXDb OFcBZinr8bcU3ZyElhf11ipGCWICXPFu4eD3lXKwLx5R24DQ0UGvB3UE0pPaqSf46gN0 PWFQ== X-Gm-Message-State: AOAM533+NUhfs+uu14PJNhKiCAN/OP8v3SiJR4rSakKLoXOJT3QBHqeo fsGsLyLpjeX/ScmCAQ5i0xWT6v05oXI7aw== X-Google-Smtp-Source: ABdhPJxHmIDxhirbr5MGH60Q6/Dui+B4IfTefnxShB8zpmktIp15JcxEM8t7GfB4PzL7oYX2mORzlA== X-Received: by 2002:ad4:504a:: with SMTP id m10mr45148897qvq.172.1594216865494; Wed, 08 Jul 2020 07:01:05 -0700 (PDT) Received: from localhost (cpe-174-109-172-136.nc.res.rr.com. [174.109.172.136]) by smtp.gmail.com with ESMTPSA id t9sm28151467qke.68.2020.07.08.07.01.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 07:01:04 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 23/23] btrfs: add a comment explaining the data flush steps Date: Wed, 8 Jul 2020 10:00:13 -0400 Message-Id: <20200708140013.56994-24-josef@toxicpanda.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200708140013.56994-1-josef@toxicpanda.com> References: <20200708140013.56994-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The data flushing steps are not obvious to people other than myself and Chris. Write a giant comment explaining the reasoning behind each flush step for data as well as why it is in that particular order. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 47 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 680218a7b0e5..ba2bb47cbb18 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-info.c @@ -998,6 +998,53 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) } while (flush_state <= COMMIT_TRANS); } +/* + * FLUSH_DELALLOC_WAIT: + * Space is free'd from flushing delalloc in one of two ways. + * + * 1) compression is on and we allocate less space than we reserved. + * 2) We are overwriting existing space. + * + * For #1 that extra space is reclaimed as soon as the delalloc pages are + * cow'ed, by way of btrfs_add_reserved_bytes() which adds the actual extent + * length to ->bytes_reserved, and subtracts the reserved space from + * ->bytes_may_use. + * + * For #2 this is trickier. Once the ordered extent runs we will drop the + * extent in the range we are overwriting, which creates a delayed ref for + * that freed extent. This however is not reclaimed until the transaction + * commits, thus the next stages. + * + * RUN_DELAYED_IPUTS + * If we are freeing inodes, we want to make sure all delayed iputs have + * completed, because they could have been on an inode with i_nlink == 0, and + * thus have been trunated and free'd up space. But again this space is not + * immediately re-usable, it comes in the form of a delayed ref, which must be + * run and then the transaction must be committed. + * + * FLUSH_DELAYED_REFS + * The above two cases generate delayed refs that will affect + * ->total_bytes_pinned. However this counter can be inconsistent with + * reality if there are outstanding delayed refs. This is because we adjust + * the counter based soley on the current set of delayed refs and disregard + * any on-disk state which might include more refs. So for example, if we + * have an extent with 2 references, but we only drop 1, we'll see that there + * is a negative delayed ref count for the extent and assume that the space + * will be free'd, and thus increase ->total_bytes_pinned. + * + * Running the delayed refs gives us the actual real view of what will be + * freed at the transaction commit time. This stage will not actually free + * space for us, it just makes sure that may_commit_transaction() has all of + * the information it needs to make the right decision. + * + * COMMIT_TRANS + * This is where we reclaim all of the pinned space generated by the previous + * two stages. We will not commit the transaction if we don't think we're + * likely to satisfy our request, which means if our current free space + + * total_bytes_pinned < reservation we will not commit. This is why the + * previous states are actually important, to make sure we know for sure + * whether committing the transaction will allow us to make progress. + */ static const enum btrfs_flush_state data_flush_states[] = { FLUSH_DELALLOC_WAIT, RUN_DELAYED_IPUTS,