From patchwork Mon Oct 7 20:17:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Zhou X-Patchwork-Id: 11178473 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16A1F139A for ; Mon, 7 Oct 2019 20:18:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E9CC421479 for ; Mon, 7 Oct 2019 20:18:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1570479497; bh=3qOnVvLxs1I/g+rPlLz35cb86WfaeefidX0KU6jYE24=; h=From:To:Cc:Subject:Date:In-Reply-To:References:In-Reply-To: References:List-ID:From; b=HesE/H/sARsSN9T4l/lG+J0LePqiGZayANUfBu2sEe/C1KKFWygJp0RxI2UEZb1jK I8/FYmVcNqakx15nIwOvaqD+eSXiDOngQnwMKGf5ZEwv9yav/qg/kVbsPBfjLSuL89 WwLQLaRDgFExdMDgBfc8DBqGzP18juusr9r7K7yo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729402AbfJGUSP (ORCPT ); Mon, 7 Oct 2019 16:18:15 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:37901 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729380AbfJGUSO (ORCPT ); Mon, 7 Oct 2019 16:18:14 -0400 Received: by mail-qk1-f196.google.com with SMTP id u186so13942834qkc.5 for ; Mon, 07 Oct 2019 13:18:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=ydAJYdps+yVy//BCuVYseS/Oav8Svt31Ns5WcndpjHM=; b=Rq+dhffEeyNx36PHKGx5avXT3hmzaihHEbUCxBl/4Fjv0L/3mFlDO+DD8uCxtQSKPi 9VZ7qiV5Gmk7/1uG7kf58bLMtJBj9Kw0Ml1eky3KUYUXcrvHhd5gzVnMh0pyserJ0NWa BDii6AUu2tRcB7UeP+kL/c83Lc4CuUu89sLcq6GLq8f1fmQMgLea1n9ns2dzNgPAiqEr H7U63U0lHZJ72LSA5j+64rN8pu1Nl6yRQsBOBLQ1nVAk77xezIQKMd1YZ+a93G53DO+c HCtkxV7QNiN6p/LE2Bj/PbWm+PPWR3i7mPxHHakNiv9wqQlb6PPg1FlTFibut9EmgmJ6 G2/A== X-Gm-Message-State: APjAAAUUhPBvCd5mhwUbqiU7tJT4T0PwcQyWXkHc6zMjGMn69grYZZX2 AR5TETqKtPsoBkcrZ1VSn3Q= X-Google-Smtp-Source: APXvYqyCnse2OVlmmP5jbjkElNNnd3QPowNCkYaeMm+CkzmcYas6ECxgcbSl/FbyFUZHxiJAyU6YoA== X-Received: by 2002:a37:a2c3:: with SMTP id l186mr24169465qke.461.1570479493477; Mon, 07 Oct 2019 13:18:13 -0700 (PDT) Received: from dennisz-mbp.thefacebook.com ([163.114.130.128]) by smtp.gmail.com with ESMTPSA id k2sm6904005qtm.42.2019.10.07.13.18.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 07 Oct 2019 13:18:12 -0700 (PDT) From: Dennis Zhou To: David Sterba , Chris Mason , Josef Bacik , Omar Sandoval Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, Dennis Zhou Subject: [PATCH 18/19] btrfs: increase the metadata allowance for the free_space_cache Date: Mon, 7 Oct 2019 16:17:49 -0400 Message-Id: <4b171367793ccdcd722e787e5ae0f4f547ed5c43.1570479299.git.dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: References: In-Reply-To: References: Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently, there is no way for the free space cache to recover from being serviced by purely bitmaps because the extent threshold is set to 0 in recalculate_thresholds() when we surpass the metadata allowance. This adds a recovery mechanism by keeping large extents out of the bitmaps and increases the metadata upper bound to 64KB. The recovery mechanism bypasses this upper bound, thus making it a soft upper bound. But, with the bypass being 1MB or greater, it shouldn't add unbounded overhead. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 480119016c0d..a0941d281a63 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -24,7 +24,8 @@ #include "discard.h" #define BITS_PER_BITMAP (PAGE_SIZE * 8UL) -#define MAX_CACHE_BYTES_PER_GIG SZ_32K +#define MAX_CACHE_BYTES_PER_GIG SZ_64K +#define FORCE_EXTENT_THRESHOLD SZ_1M struct btrfs_trim_range { u64 start; @@ -1686,26 +1687,17 @@ static void recalculate_thresholds(struct btrfs_free_space_ctl *ctl) ASSERT(ctl->total_bitmaps <= max_bitmaps); /* - * The goal is to keep the total amount of memory used per 1gb of space - * at or below 32k, so we need to adjust how much memory we allow to be - * used by extent based free space tracking + * We are trying to keep the total amount of memory used per 1gb of + * space to be MAX_CACHE_BYTES_PER_GIG. However, with a reclamation + * mechanism of pulling extents >= FORCE_EXTENT_THRESHOLD out of + * bitmaps, we may end up using more memory than this. */ if (size < SZ_1G) max_bytes = MAX_CACHE_BYTES_PER_GIG; else max_bytes = MAX_CACHE_BYTES_PER_GIG * div_u64(size, SZ_1G); - /* - * we want to account for 1 more bitmap than what we have so we can make - * sure we don't go over our overall goal of MAX_CACHE_BYTES_PER_GIG as - * we add more bitmaps. - */ - bitmap_bytes = (ctl->total_bitmaps + 1) * ctl->unit; - - if (bitmap_bytes >= max_bytes) { - ctl->extents_thresh = 0; - return; - } + bitmap_bytes = ctl->total_bitmaps * ctl->unit; /* * we want the extent entry threshold to always be at most 1/2 the max @@ -2086,6 +2078,10 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl, forced = true; #endif + /* this is a way to reclaim large regions from the bitmaps */ + if (!forced && info->bytes >= FORCE_EXTENT_THRESHOLD) + return false; + /* * If we are below the extents threshold then we can add this as an * extent, and don't have to deal with the bitmap