From patchwork Fri Apr 14 08:00:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 9680755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1FA3F60386 for ; Fri, 14 Apr 2017 08:01:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 114252835B for ; Fri, 14 Apr 2017 08:01:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 06077285B6; Fri, 14 Apr 2017 08:01:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C3DD2835B for ; Fri, 14 Apr 2017 08:01:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751989AbdDNIBR (ORCPT ); Fri, 14 Apr 2017 04:01:17 -0400 Received: from mail-pf0-f175.google.com ([209.85.192.175]:34270 "EHLO mail-pf0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752278AbdDNIBO (ORCPT ); Fri, 14 Apr 2017 04:01:14 -0400 Received: by mail-pf0-f175.google.com with SMTP id c198so38505858pfc.1 for ; Fri, 14 Apr 2017 01:01:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=GFSPcR0Xti9ZNDTg6t5lGMFqME0y8SdzDentplNDHNI=; b=jbm1oZs+mJ+GCsoC2o7rXCE/vOtufLuoqk/rXtjsYIFFdRpmDmS9u1VAgBdJSvWuS5 7v4WzpsM8vEOpQAGq5SpmizxKWJwRhtadVVfh2RMDyNIH7Tz0bcTPuXg8Tx3vbjOLPZu 5XqUoXPr9oSVlslW4XvfW9MBiCB37lY6hmqnEg4pQua0EOVg7GdO3G5g9wa0CxVWo+sr V9lLgQB06Mrt/esCWt5dhgZ7cvP2+gcdsukS9/QgorzqiBM9kO/p4jlFX3ekOv8ykWe1 jPruiObxyLpJXPmksm60Jn6+yvxhYQTAvFkXhIcu78UTOT9/bmetmD1zkDrrL2BDBUGZ /bJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=GFSPcR0Xti9ZNDTg6t5lGMFqME0y8SdzDentplNDHNI=; b=E1oydDqnAZHCQmPy9D9+4NXXsK+RFCnhrFSpJQCBLqAcESA/cqlMnI/dM3zWrW/UWx lKdildkzzfZOZwQT6eIDudk+66FcG3Nu3GGiSI1ZZFXtt6t0y3mxnozyFa05pObh16zG jMv6kxT+17cEbD45YAd3ETPw5hv8uvp9VjQ0+2ZY0EirboHGNkxILSW0gx/ctWUYCxq5 zzWq1wEUzbky6n4njzJ+2tw7PxDEU5LDgqNFAwwJThof67tSSEbnz6O3guza4M0EoIM+ AD3OwVtilC6yjE6fFSrv9PH3+Woq6i6q3G2rDYkYaP8+POssGrvrM7qyhf3c16+rk/sP et/g== X-Gm-Message-State: AN3rC/7TfH2eUHDh0WyWAqyIRU/wu2CxAk66iKS9hledrL2fxCmyLhEB eaaR7tOQSGdTadwZfFq5BQ== X-Received: by 10.98.221.141 with SMTP id w135mr5647242pff.109.1492156869048; Fri, 14 Apr 2017 01:01:09 -0700 (PDT) Received: from vader.thefacebook.com ([2620:10d:c090:180::b306]) by smtp.gmail.com with ESMTPSA id s21sm1913250pgg.65.2017.04.14.01.01.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Apr 2017 01:01:08 -0700 (PDT) From: Omar Sandoval To: Jens Axboe , linux-block@vger.kernel.org Cc: kernel-team@fb.com Subject: [PATCH v4 4/5] blk-mq-sched: make completed_request() callback more useful Date: Fri, 14 Apr 2017 01:00:01 -0700 Message-Id: X-Mailer: git-send-email 2.12.2 In-Reply-To: References: In-Reply-To: References: Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval Currently, this callback is called right after put_request() and has no distinguishable purpose. Instead, let's call it before put_request() as soon as I/O has completed on the request, before we account it in blk-stat. With this, Kyber can enable stats when it sees a latency outlier and make sure the outlier gets accounted. Signed-off-by: Omar Sandoval --- block/blk-mq-sched.h | 11 +++-------- block/blk-mq.c | 5 ++++- include/linux/elevator.h | 2 +- 3 files changed, 8 insertions(+), 10 deletions(-) diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index f4bc186c3440..120c6abc37cc 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -82,17 +82,12 @@ blk_mq_sched_allow_merge(struct request_queue *q, struct request *rq, return true; } -static inline void -blk_mq_sched_completed_request(struct blk_mq_hw_ctx *hctx, struct request *rq) +static inline void blk_mq_sched_completed_request(struct request *rq) { - struct elevator_queue *e = hctx->queue->elevator; + struct elevator_queue *e = rq->q->elevator; if (e && e->type->ops.mq.completed_request) - e->type->ops.mq.completed_request(hctx, rq); - - BUG_ON(rq->internal_tag == -1); - - blk_mq_put_tag(hctx, hctx->sched_tags, rq->mq_ctx, rq->internal_tag); + e->type->ops.mq.completed_request(rq); } static inline void blk_mq_sched_started_request(struct request *rq) diff --git a/block/blk-mq.c b/block/blk-mq.c index 7138cd98146e..e2ef7b460924 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -350,7 +350,7 @@ void __blk_mq_finish_request(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, if (rq->tag != -1) blk_mq_put_tag(hctx, hctx->tags, ctx, rq->tag); if (sched_tag != -1) - blk_mq_sched_completed_request(hctx, rq); + blk_mq_put_tag(hctx, hctx->sched_tags, ctx, sched_tag); blk_mq_sched_restart(hctx); blk_queue_exit(q); } @@ -444,6 +444,9 @@ static void __blk_mq_complete_request(struct request *rq) { struct request_queue *q = rq->q; + if (rq->internal_tag != -1) + blk_mq_sched_completed_request(rq); + blk_mq_stat_add(rq); if (!q->softirq_done_fn) diff --git a/include/linux/elevator.h b/include/linux/elevator.h index b7ec315ee7e7..3a216318ae73 100644 --- a/include/linux/elevator.h +++ b/include/linux/elevator.h @@ -106,7 +106,7 @@ struct elevator_mq_ops { void (*insert_requests)(struct blk_mq_hw_ctx *, struct list_head *, bool); struct request *(*dispatch_request)(struct blk_mq_hw_ctx *); bool (*has_work)(struct blk_mq_hw_ctx *); - void (*completed_request)(struct blk_mq_hw_ctx *, struct request *); + void (*completed_request)(struct request *); void (*started_request)(struct request *); void (*requeue_request)(struct request *); struct request *(*former_request)(struct request_queue *, struct request *);