From patchwork Mon Apr 10 06:51:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 9672021 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2F61360244 for ; Mon, 10 Apr 2017 06:52:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2131D26490 for ; Mon, 10 Apr 2017 06:52:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 160AE26E3A; Mon, 10 Apr 2017 06:52:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E7B326490 for ; Mon, 10 Apr 2017 06:52:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752058AbdDJGwW (ORCPT ); Mon, 10 Apr 2017 02:52:22 -0400 Received: from mail-pf0-f176.google.com ([209.85.192.176]:32935 "EHLO mail-pf0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751864AbdDJGwW (ORCPT ); Mon, 10 Apr 2017 02:52:22 -0400 Received: by mail-pf0-f176.google.com with SMTP id s16so28860609pfs.0 for ; Sun, 09 Apr 2017 23:52:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=GFSPcR0Xti9ZNDTg6t5lGMFqME0y8SdzDentplNDHNI=; b=yX+To4p/Q2/jxdqpIkiFmFNGrGtxlwLCaMekGtm6LYmLg96mdfZDiREWSWJyTigM94 jt1O0Fm65xX60G/nDsv39Yia8b0fdpAAifAzoc8LvHD2VLRXy30mzvXq3TJ86SR/hEVA 9cZPKnJT+5BKAB61MV3cV5QSA+IAClnMlQgccO9H7JAPfkqSL+Y3zp23E4Tr166PJol/ Sy5/CxhTcpj9cphtSgcwum5GpE9FpLx3znhMuiFHut7XulP8TJhNwnSBmASRsD0DerF9 b1u1o2dCb+TR/3ALnPCWmdKoDx7evzFlAbSnuUBzW0QyAKwnTyr5zl5k5BlvHN0QbDnC 01gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=GFSPcR0Xti9ZNDTg6t5lGMFqME0y8SdzDentplNDHNI=; b=n2U/2e3B7pePrQw6oY0R6rgEm+Pwg1gTZUvqdVT/AwPoBZZs+PYAgIAX0dunxAzBgn A/WiXb1P0551AJzx2rnsmUM2e/Qd1Dyg96ne3ZVPkIG+sGd0Rjh4v7bi4eEzf6RlJDSr EfxSBK1zTfIKdYoxyb2z1ZdCkI2Mu7ij/BetgqfflETMF+mylcwo27/bk934jC23FRfK GerZxS0gth0tfn8pB65CSsHayPAUiIMnvBKWXDrxMbmuQk/jyqg/uLdnqwCzETLDMD7r err0mVwukDeqjHl0+tch/MjdZ0k3qwfY4Wo5W//8aSq9jDLtrA30ERG9/rlI1DAE4q2A rfoA== X-Gm-Message-State: AFeK/H0t7rlNAi9x5I/5BLdP5QMB0yucuUS8POVX7aqbPWb9lBXXmXs2e0NP8AIWZnHpqtF9 X-Received: by 10.99.240.83 with SMTP id s19mr54469045pgj.45.1491807141184; Sun, 09 Apr 2017 23:52:21 -0700 (PDT) Received: from vader.thefacebook.com ([2620:10d:c090:180::da80]) by smtp.gmail.com with ESMTPSA id b185sm10679391pfa.61.2017.04.09.23.52.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Apr 2017 23:52:20 -0700 (PDT) From: Omar Sandoval To: Jens Axboe , linux-block@vger.kernel.org Cc: kernel-team@fb.com Subject: [PATCH v3 4/5] blk-mq-sched: make completed_request() callback more useful Date: Sun, 9 Apr 2017 23:51:12 -0700 Message-Id: X-Mailer: git-send-email 2.12.2 In-Reply-To: References: In-Reply-To: References: Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval Currently, this callback is called right after put_request() and has no distinguishable purpose. Instead, let's call it before put_request() as soon as I/O has completed on the request, before we account it in blk-stat. With this, Kyber can enable stats when it sees a latency outlier and make sure the outlier gets accounted. Signed-off-by: Omar Sandoval --- block/blk-mq-sched.h | 11 +++-------- block/blk-mq.c | 5 ++++- include/linux/elevator.h | 2 +- 3 files changed, 8 insertions(+), 10 deletions(-) diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index f4bc186c3440..120c6abc37cc 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -82,17 +82,12 @@ blk_mq_sched_allow_merge(struct request_queue *q, struct request *rq, return true; } -static inline void -blk_mq_sched_completed_request(struct blk_mq_hw_ctx *hctx, struct request *rq) +static inline void blk_mq_sched_completed_request(struct request *rq) { - struct elevator_queue *e = hctx->queue->elevator; + struct elevator_queue *e = rq->q->elevator; if (e && e->type->ops.mq.completed_request) - e->type->ops.mq.completed_request(hctx, rq); - - BUG_ON(rq->internal_tag == -1); - - blk_mq_put_tag(hctx, hctx->sched_tags, rq->mq_ctx, rq->internal_tag); + e->type->ops.mq.completed_request(rq); } static inline void blk_mq_sched_started_request(struct request *rq) diff --git a/block/blk-mq.c b/block/blk-mq.c index 7138cd98146e..e2ef7b460924 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -350,7 +350,7 @@ void __blk_mq_finish_request(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, if (rq->tag != -1) blk_mq_put_tag(hctx, hctx->tags, ctx, rq->tag); if (sched_tag != -1) - blk_mq_sched_completed_request(hctx, rq); + blk_mq_put_tag(hctx, hctx->sched_tags, ctx, sched_tag); blk_mq_sched_restart(hctx); blk_queue_exit(q); } @@ -444,6 +444,9 @@ static void __blk_mq_complete_request(struct request *rq) { struct request_queue *q = rq->q; + if (rq->internal_tag != -1) + blk_mq_sched_completed_request(rq); + blk_mq_stat_add(rq); if (!q->softirq_done_fn) diff --git a/include/linux/elevator.h b/include/linux/elevator.h index b7ec315ee7e7..3a216318ae73 100644 --- a/include/linux/elevator.h +++ b/include/linux/elevator.h @@ -106,7 +106,7 @@ struct elevator_mq_ops { void (*insert_requests)(struct blk_mq_hw_ctx *, struct list_head *, bool); struct request *(*dispatch_request)(struct blk_mq_hw_ctx *); bool (*has_work)(struct blk_mq_hw_ctx *); - void (*completed_request)(struct blk_mq_hw_ctx *, struct request *); + void (*completed_request)(struct request *); void (*started_request)(struct request *); void (*requeue_request)(struct request *); struct request *(*former_request)(struct request_queue *, struct request *);