From patchwork Fri Apr 7 12:24:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Bates X-Patchwork-Id: 9669395 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A723F602A0 for ; Fri, 7 Apr 2017 12:24:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 960A02856A for ; Fri, 7 Apr 2017 12:24:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8AF2A285E9; Fri, 7 Apr 2017 12:24:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2484C2856A for ; Fri, 7 Apr 2017 12:24:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933563AbdDGMYL (ORCPT ); Fri, 7 Apr 2017 08:24:11 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:36017 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933605AbdDGMYK (ORCPT ); Fri, 7 Apr 2017 08:24:10 -0400 Received: by mail-wr0-f194.google.com with SMTP id o21so14482602wrb.3 for ; Fri, 07 Apr 2017 05:24:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/VXguuDDgKe6LDKZWL4XhBUb8lPxfojUgGwHWEa2PkI=; b=jgYs8cX3VwuUw5FkPOgB1A+Tw0e/DxBwCiJDFhzxRIGs1kAytLJiqRu+Jw+/8seSuX /ao0LRevxcMUxIGp6Uxppx1TfsgjdzDG1/GZq1o9mqvV0P1uYBeLArk449qXhhmMLMAI VZ9Xtk3qwPs+kXLF6xBWntZpZRCiNwI8+ASs0YJ6J3syU6zegHZZHuN8C6SCP1ZwseBE KQZcVtjEMIrOyvYKEnb5Oxmq4Lltl7Mfdq4J6wKzUDeLbIUaBuQM7HK8S4inKl6BWV1U TfbzrjgEf7n48TZ3Y2osrq7Aal4jXFTcTvyf9plHvtBOiMXSB7qJKA4cXmvJ2s+TLhJf bLtA== X-Gm-Message-State: AFeK/H0KUYq2c+Z9Hydt4HVhC47+Me0nfzU/8Giyd+Q+U0tC2PmCjAv2I7T1Nd48L7b1VA== X-Received: by 10.28.126.77 with SMTP id z74mr12209053wmc.15.1491567849459; Fri, 07 Apr 2017 05:24:09 -0700 (PDT) Received: from localhost.localdomain (51-171-151-230-dynamic.agg2.lky.bge-rtd.eircom.net. [51.171.151.230]) by smtp.gmail.com with ESMTPSA id v29sm5821502wrv.66.2017.04.07.05.24.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 07 Apr 2017 05:24:08 -0700 (PDT) From: sbates@raithlin.com To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Damien.LeMoal@wdc.com, osandov@osandov.com, sbates@raithlin.com, sagi@grimberg.me Subject: [PATCH v3 2/2] blk-mq: Add a polling specific stats function Date: Fri, 7 Apr 2017 06:24:03 -0600 Message-Id: <1491567843-26190-3-git-send-email-sbates@raithlin.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1491567843-26190-1-git-send-email-sbates@raithlin.com> References: <1491567843-26190-1-git-send-email-sbates@raithlin.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Stephen Bates Rather than bucketing IO statisics based on direction only we also bucket based on the IO size. This leads to improved polling performance. Update the bucket callback function and use it in the polling latency estimation. Signed-off-by: Stephen Bates --- block/blk-mq.c | 45 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 35 insertions(+), 10 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 061fc2c..5fd376b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -42,6 +42,25 @@ static LIST_HEAD(all_q_list); static void blk_mq_poll_stats_start(struct request_queue *q); static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb); +/* Must be consisitent with function below */ +#define BLK_MQ_POLL_STATS_BKTS 16 +static int blk_mq_poll_stats_bkt(const struct request *rq) +{ + int ddir, bytes, bucket; + + ddir = blk_stat_rq_ddir(rq); + bytes = blk_rq_bytes(rq); + + bucket = ddir + 2*(ilog2(bytes) - 9); + + if (bucket < 0) + return -1; + else if (bucket >= BLK_MQ_POLL_STATS_BKTS) + return ddir + BLK_MQ_POLL_STATS_BKTS - 2; + + return bucket; +} + /* * Check if any of the ctx's have pending work in this hardware queue */ @@ -2245,7 +2264,8 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set, q->mq_ops = set->ops; q->poll_cb = blk_stat_alloc_callback(blk_mq_poll_stats_fn, - blk_stat_rq_ddir, 2, q); + blk_mq_poll_stats_bkt, + BLK_MQ_POLL_STATS_BKTS, q); if (!q->poll_cb) goto err_exit; @@ -2663,11 +2683,12 @@ static void blk_mq_poll_stats_start(struct request_queue *q) static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb) { struct request_queue *q = cb->data; + int bucket; - if (cb->stat[READ].nr_samples) - q->poll_stat[READ] = cb->stat[READ]; - if (cb->stat[WRITE].nr_samples) - q->poll_stat[WRITE] = cb->stat[WRITE]; + for (bucket = 0; bucket < BLK_MQ_POLL_STATS_BKTS; bucket++) { + if (cb->stat[bucket].nr_samples) + q->poll_stat[bucket] = cb->stat[bucket]; + } } static unsigned long blk_mq_poll_nsecs(struct request_queue *q, @@ -2675,6 +2696,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q, struct request *rq) { unsigned long ret = 0; + int bucket; /* * If stats collection isn't on, don't sleep but turn it on for @@ -2689,12 +2711,15 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q, * For instance, if the completion latencies are tight, we can * get closer than just half the mean. This is especially * important on devices where the completion latencies are longer - * than ~10 usec. + * than ~10 usec. We do use the stats for the relevant IO size + * if available which does lead to better estimates. */ - if (req_op(rq) == REQ_OP_READ && q->poll_stat[READ].nr_samples) - ret = (q->poll_stat[READ].mean + 1) / 2; - else if (req_op(rq) == REQ_OP_WRITE && q->poll_stat[WRITE].nr_samples) - ret = (q->poll_stat[WRITE].mean + 1) / 2; + bucket = blk_mq_poll_stats_bkt(rq); + if (bucket < 0) + return ret; + + if (q->poll_stat[bucket].nr_samples) + ret = (q->poll_stat[bucket].mean + 1) / 2; return ret; }