From patchwork Wed Apr 5 17:39:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Bates X-Patchwork-Id: 9665343 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1F9DA60353 for ; Wed, 5 Apr 2017 17:40:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1535628285 for ; Wed, 5 Apr 2017 17:40:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0A1FF284BC; Wed, 5 Apr 2017 17:40:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 95C6228285 for ; Wed, 5 Apr 2017 17:40:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755633AbdDERjc (ORCPT ); Wed, 5 Apr 2017 13:39:32 -0400 Received: from mail-wr0-f195.google.com ([209.85.128.195]:35339 "EHLO mail-wr0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754199AbdDERjY (ORCPT ); Wed, 5 Apr 2017 13:39:24 -0400 Received: by mail-wr0-f195.google.com with SMTP id t20so4707142wra.2 for ; Wed, 05 Apr 2017 10:39:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bcw+kF5dHjwj7TaWjWLOZnG4BRGfU48MCTYo1o4lS30=; b=tazdxmivohX81bjWJSV3asqYI5iBbTZR8HQgr6FfS/ypy/wKeiqFOySXqdy2gEj7gn e2IPR8O6jYN+7jGaBEiuyaeRBYeqdIl+EuGycBW0I0hbRVYxvYSt64RmXhPhuJEXF2D2 w+cwd7BctZ2ZPWMsB02rUmV7N+d41miBNMmq1ufOV2kk0+A8J47nkShXS21jj9X/vL7l +ZrpiJ3Xexizd3hHbniKyT01SgXU/i9w96zCPnz4JkM9VUDR2wgSklL53f+3xZxAsBL9 TULZJ45p6nKn0H6bzGPr/egP/z7+xyuj+7m4Jmfs4OHGXVb305odltyUxXZ87y9N6TbK yqRw== X-Gm-Message-State: AFeK/H0ChjYwr7H21KSdI0BThwi/DMRhs2OQo5+wcUYr6HLfUcRrgPSl EtIhzEqMiFl1bA== X-Received: by 10.28.156.69 with SMTP id f66mr19768263wme.56.1491413963006; Wed, 05 Apr 2017 10:39:23 -0700 (PDT) Received: from localhost.localdomain (51-171-151-230-dynamic.agg2.lky.bge-rtd.eircom.net. [51.171.151.230]) by smtp.gmail.com with ESMTPSA id 134sm23074375wmj.6.2017.04.05.10.39.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 05 Apr 2017 10:39:22 -0700 (PDT) From: sbates@raithlin.com To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Damien.LeMoal@wdc.com, osandov@osandov.com, sbates@raithlin.com, sagi@grimberg.me Subject: [PATCH v2 2/2] blk-mq: Add a polling specific stats function Date: Wed, 5 Apr 2017 11:39:17 -0600 Message-Id: <1491413957-30885-3-git-send-email-sbates@raithlin.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1491413957-30885-1-git-send-email-sbates@raithlin.com> References: <1491413957-30885-1-git-send-email-sbates@raithlin.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Stephen Bates Rather than bucketing IO statisics based on direction only we also bucket based on the IO size. This leads to improved polling performance. Update the bucket callback function and use it in the polling latency estimation. Signed-off-by: Stephen Bates --- block/blk-mq.c | 53 +++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 10 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 061fc2c..8fb1fb0 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -42,6 +42,33 @@ static LIST_HEAD(all_q_list); static void blk_mq_poll_stats_start(struct request_queue *q); static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb); +static int blk_mq_poll_stats_bkt(const struct request *rq) +{ + int dir, bytes; + + dir = blk_stat_rq_ddir(rq); + bytes = blk_rq_bytes(rq); + + if (bytes <= 512) + return dir; + else if (bytes <= 4096) + return dir + 2; + else if (bytes <= 8192) + return dir + 4; + else if (bytes <= 16384) + return dir + 6; + else if (bytes <= 32768) + return dir + 8; + else if (bytes <= 65536) + return dir + 10; + else + return dir + 12; + + return -1; +} +/* Must be consisitent with function above */ +#define BLK_MQ_POLL_STATS_BKTS 14 + /* * Check if any of the ctx's have pending work in this hardware queue */ @@ -2245,7 +2272,8 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set, q->mq_ops = set->ops; q->poll_cb = blk_stat_alloc_callback(blk_mq_poll_stats_fn, - blk_stat_rq_ddir, 2, q); + blk_mq_poll_stats_bkt, + BLK_MQ_POLL_STATS_BKTS, q); if (!q->poll_cb) goto err_exit; @@ -2663,11 +2691,12 @@ static void blk_mq_poll_stats_start(struct request_queue *q) static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb) { struct request_queue *q = cb->data; + int bucket; - if (cb->stat[READ].nr_samples) - q->poll_stat[READ] = cb->stat[READ]; - if (cb->stat[WRITE].nr_samples) - q->poll_stat[WRITE] = cb->stat[WRITE]; + for (bucket = 0; bucket < BLK_MQ_POLL_STATS_BKTS; bucket++) { + if (cb->stat[bucket].nr_samples) + q->poll_stat[bucket] = cb->stat[bucket]; + } } static unsigned long blk_mq_poll_nsecs(struct request_queue *q, @@ -2675,6 +2704,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q, struct request *rq) { unsigned long ret = 0; + int bucket; /* * If stats collection isn't on, don't sleep but turn it on for @@ -2689,12 +2719,15 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q, * For instance, if the completion latencies are tight, we can * get closer than just half the mean. This is especially * important on devices where the completion latencies are longer - * than ~10 usec. + * than ~10 usec. We do use the stats for the relevant IO size + * if available which does lead to better estimates. */ - if (req_op(rq) == REQ_OP_READ && q->poll_stat[READ].nr_samples) - ret = (q->poll_stat[READ].mean + 1) / 2; - else if (req_op(rq) == REQ_OP_WRITE && q->poll_stat[WRITE].nr_samples) - ret = (q->poll_stat[WRITE].mean + 1) / 2; + bucket = blk_mq_poll_stats_bkt(rq); + if (bucket < 0) + return ret; + + if (q->poll_stat[bucket].nr_samples) + ret = (q->poll_stat[bucket].mean + 1) / 2; return ret; }