From patchwork Mon Oct 30 21:46:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Lyle X-Patchwork-Id: 10033307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D3CC86039A for ; Mon, 30 Oct 2017 21:47:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5CF828945 for ; Mon, 30 Oct 2017 21:47:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BAD0428949; Mon, 30 Oct 2017 21:47:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3051A28945 for ; Mon, 30 Oct 2017 21:47:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932945AbdJ3VrB (ORCPT ); Mon, 30 Oct 2017 17:47:01 -0400 Received: from mail-pf0-f196.google.com ([209.85.192.196]:50224 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932353AbdJ3Vq7 (ORCPT ); Mon, 30 Oct 2017 17:46:59 -0400 Received: by mail-pf0-f196.google.com with SMTP id b6so12071963pfh.7 for ; Mon, 30 Oct 2017 14:46:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lyle-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MqQdPNBeBHyBuI6zpcwhW8SoUNLaRTlTGUNP032w0dM=; b=LQFixGe7XaDHfhtB2Gnq9Jr/t+cK3k7It8I7dYEPAkknQYWypvO57DlXH1KEPKvhPi 0a4Op9JuCUnywKlDir2kq7VbQ75m79rkrSw377oVoeGaelC2GIXb8EJ00x08Yfm6Qi74 6CZoqsW+EIA0EsRpHzbpZE4rZY8zT+OYMKSSNYtSiTrveaAk9c9o+A4a31ry7mRi/oPz vy1LNrixRU1fMQ6G+bacAXBp9OQLrFP0AtmLCXhsTDHebX7u5LRQM5yO+ywF6Cu3xHWt +m7h2GYQrMGJ2PLEfU+/ySXPqakRG4DX4zCBE8wXdSVr9sRz30b6DpWl6NfuBp4FOwHV 3IAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MqQdPNBeBHyBuI6zpcwhW8SoUNLaRTlTGUNP032w0dM=; b=f30LDujKkZso1V7Jk+2e+u54WO/frsPvH3+5BTQVZYSONEtCARvC6kPfFELFx6xI9I A8Qyx595c6vwhtOyJ1uiTZ9H/pRCu/YRJyC1s0Lpn3BpVIkefxhUvfScc9xwVMQXBZZs gMRJU7iDuOYGINGAUQEIcfURN89z8u7r2YrSLx6e3QFH3c9Jy+QODmpqxmkpbv+5Uclz LLHNRD4IOTCRDuGr3zxJYqqhEYfnxKuV65mh/s940EqWqMZhoVPmymjTLal0TKfI21PZ nwrFQAYpVbtzINsuur67Zgpnt+ovdXHg8Ggrhdl+t3R5PfaGk/grpjCPHUF47DSesAfo G+KQ== X-Gm-Message-State: AMCzsaWqlLrJY5/ac2sW1rk029H7hmazW1UYw8becMEpsNkFrwsNeRkh y7oPJvHIeom/nVKlFRpt/415Yw== X-Google-Smtp-Source: ABhQp+TjtqlcMFkFyuGqLlJ0n1Z0rxpCaLpENbYDpE3py2VNQmPQJrX3hPR5J6zu+iyxsRw85TisXQ== X-Received: by 10.84.133.69 with SMTP id 63mr8380996plf.203.1509400018339; Mon, 30 Oct 2017 14:46:58 -0700 (PDT) Received: from localhost.localdomain (68-189-67-104.dhcp.prtv.ca.charter.com. [68.189.67.104]) by smtp.gmail.com with ESMTPSA id t78sm23733329pgb.93.2017.10.30.14.46.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Oct 2017 14:46:57 -0700 (PDT) From: Michael Lyle To: linux-bcache@vger.kernel.org, linux-block@vger.kernel.org Cc: axboe@fb.com, Tang Junhui , Michael Lyle Subject: [PATCH 3/5] bcache: update bucket_in_use in real time Date: Mon, 30 Oct 2017 14:46:33 -0700 Message-Id: <20171030214635.29888-4-mlyle@lyle.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171030214635.29888-1-mlyle@lyle.org> References: <20171030214635.29888-1-mlyle@lyle.org> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tang Junhui bucket_in_use is updated in gc thread which triggered by invalidating or writing sectors_to_gc dirty data, It's a long interval. Therefore, when we use it to compare with the threshold, it is often not timely, which leads to inaccurate judgment and often results in bucket depletion. We have send a patch before, by the means of updating bucket_in_use periodically In gc thread, which Coly thought that would lead high latency, In this patch, we add avail_nbuckets to record the count of available buckets, and we calculate bucket_in_use when alloc or free bucket in real time. [edited by ML: eliminated some whitespace errors] Signed-off-by: Tang Junhui Signed-off-by: Michael Lyle Reviewed-by: Michael Lyle Reviewed-by: Coly Li --- drivers/md/bcache/alloc.c | 10 ++++++++++ drivers/md/bcache/bcache.h | 1 + drivers/md/bcache/btree.c | 17 ++++++++++------- drivers/md/bcache/btree.h | 2 +- 4 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/md/bcache/alloc.c b/drivers/md/bcache/alloc.c index 4c40870e99f5..8c5a626343d4 100644 --- a/drivers/md/bcache/alloc.c +++ b/drivers/md/bcache/alloc.c @@ -442,6 +442,11 @@ long bch_bucket_alloc(struct cache *ca, unsigned reserve, bool wait) b->prio = INITIAL_PRIO; } + if (ca->set->avail_nbuckets > 0) { + ca->set->avail_nbuckets--; + bch_update_bucket_in_use(ca->set, &ca->set->gc_stats); + } + return r; } @@ -449,6 +454,11 @@ void __bch_bucket_free(struct cache *ca, struct bucket *b) { SET_GC_MARK(b, 0); SET_GC_SECTORS_USED(b, 0); + + if (ca->set->avail_nbuckets < ca->set->nbuckets) { + ca->set->avail_nbuckets++; + bch_update_bucket_in_use(ca->set, &ca->set->gc_stats); + } } void bch_bucket_free(struct cache_set *c, struct bkey *k) diff --git a/drivers/md/bcache/bcache.h b/drivers/md/bcache/bcache.h index 363ea6256b39..e274082330dc 100644 --- a/drivers/md/bcache/bcache.h +++ b/drivers/md/bcache/bcache.h @@ -581,6 +581,7 @@ struct cache_set { uint8_t need_gc; struct gc_stat gc_stats; size_t nbuckets; + size_t avail_nbuckets; struct task_struct *gc_thread; /* Where in the btree gc currently is */ diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c index 866dcf78ff8e..d8865e6ead37 100644 --- a/drivers/md/bcache/btree.c +++ b/drivers/md/bcache/btree.c @@ -1240,6 +1240,11 @@ void bch_initial_mark_key(struct cache_set *c, int level, struct bkey *k) __bch_btree_mark_key(c, level, k); } +void bch_update_bucket_in_use(struct cache_set *c, struct gc_stat *stats) +{ + stats->in_use = (c->nbuckets - c->avail_nbuckets) * 100 / c->nbuckets; +} + static bool btree_gc_mark_node(struct btree *b, struct gc_stat *gc) { uint8_t stale = 0; @@ -1651,9 +1656,8 @@ static void btree_gc_start(struct cache_set *c) mutex_unlock(&c->bucket_lock); } -static size_t bch_btree_gc_finish(struct cache_set *c) +static void bch_btree_gc_finish(struct cache_set *c) { - size_t available = 0; struct bucket *b; struct cache *ca; unsigned i; @@ -1690,6 +1694,7 @@ static size_t bch_btree_gc_finish(struct cache_set *c) } rcu_read_unlock(); + c->avail_nbuckets = 0; for_each_cache(ca, c, i) { uint64_t *i; @@ -1711,18 +1716,16 @@ static size_t bch_btree_gc_finish(struct cache_set *c) BUG_ON(!GC_MARK(b) && GC_SECTORS_USED(b)); if (!GC_MARK(b) || GC_MARK(b) == GC_MARK_RECLAIMABLE) - available++; + c->avail_nbuckets++; } } mutex_unlock(&c->bucket_lock); - return available; } static void bch_btree_gc(struct cache_set *c) { int ret; - unsigned long available; struct gc_stat stats; struct closure writes; struct btree_op op; @@ -1745,14 +1748,14 @@ static void bch_btree_gc(struct cache_set *c) pr_warn("gc failed!"); } while (ret); - available = bch_btree_gc_finish(c); + bch_btree_gc_finish(c); wake_up_allocators(c); bch_time_stats_update(&c->btree_gc_time, start_time); stats.key_bytes *= sizeof(uint64_t); stats.data <<= 9; - stats.in_use = (c->nbuckets - available) * 100 / c->nbuckets; + bch_update_bucket_in_use(c, &stats); memcpy(&c->gc_stats, &stats, sizeof(struct gc_stat)); trace_bcache_gc_end(c); diff --git a/drivers/md/bcache/btree.h b/drivers/md/bcache/btree.h index 73da1f5626cb..4073aca09a49 100644 --- a/drivers/md/bcache/btree.h +++ b/drivers/md/bcache/btree.h @@ -305,5 +305,5 @@ void bch_keybuf_del(struct keybuf *, struct keybuf_key *); struct keybuf_key *bch_keybuf_next(struct keybuf *); struct keybuf_key *bch_keybuf_next_rescan(struct cache_set *, struct keybuf *, struct bkey *, keybuf_pred_fn *); - +void bch_update_bucket_in_use(struct cache_set *c, struct gc_stat *stats); #endif