From patchwork Tue Feb 21 21:29:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 9585757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4D31E600C1 for ; Tue, 21 Feb 2017 21:30:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 339A6285DD for ; Tue, 21 Feb 2017 21:30:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2895C28622; Tue, 21 Feb 2017 21:30:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF6C9285DD for ; Tue, 21 Feb 2017 21:30:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751455AbdBUVar (ORCPT ); Tue, 21 Feb 2017 16:30:47 -0500 Received: from mail-pg0-f44.google.com ([74.125.83.44]:36728 "EHLO mail-pg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751397AbdBUVaq (ORCPT ); Tue, 21 Feb 2017 16:30:46 -0500 Received: by mail-pg0-f44.google.com with SMTP id s67so30139675pgb.3 for ; Tue, 21 Feb 2017 13:30:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=mx5xQvJFAEitCMrtZ5Z4UHRDpWM0f2BDqdiFvvITtdk=; b=LJqlrDzmoiTZV2gxIsWreCVkclff6m+xlzqlLgh6kYe9dpRaLXFTRESbjZvmVkqyg2 DTHO0BGOPXECHHwavoqE7ZugBh9HcpFhT0/nTYNXohlczUfcYdMEc0kgjWFqKfEFiuPM wLIF3k0f6TtQvxiMLgLqijCwY0ikqHYUw8m4lYJgOLQjIIsfERYWLebak45q/fWk1t1K ksg5imUJoXSEhtH/zRvXCL8SOQd8SS4mJRX+RThgIffUcUoOacsbCG+CjUD3PYAcDjbb cLTFOM7xZCK9Ced09KMiWq3dKJykbde4wNaiIy9OKixX2VNuxay4F+o/yEq0797OVtwu Dsng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=mx5xQvJFAEitCMrtZ5Z4UHRDpWM0f2BDqdiFvvITtdk=; b=sLJlCXx9wgSPLVIryubTlVok+GH5pKxErm0tdwylfFGAnJmBoXYqKiL2jWIgDedoe9 AB+NIf8iPviP7fA7EcB3DIuxoi2RakIEDc9/DMHg/i+YVhkO+oGFPQ/0FyW8XzC+DTzG vbCh8o6JK4Bl1yxJdWWmlCx1ADTk9Kdnz+T5ZVwL6GaRxUd2YLkKi/Q8R947cvGIFsAw H7j87lFzXIFMEqBOZ83blIHsTRJMFv8HmjJ0QOgIub6fGbgD4/HDLvmqe2HX5lGejNYJ sS9Ut4CRPlqdHFDIV/TehcoMuntukNFDr6Pbe8k8L8xNMwEzGPBni+p3s4mtTQSnXtf3 bxLw== X-Gm-Message-State: AMke39nVros8NShOGfAkdfRohEXP4iGEV4Drpu3ewgUIVL6KlhM0jtcuuvztM+XU9iWx1rlN X-Received: by 10.99.174.71 with SMTP id e7mr38098872pgp.3.1487712646046; Tue, 21 Feb 2017 13:30:46 -0800 (PST) Received: from vader.thefacebook.com ([2620:10d:c090:200::2:7fc8]) by smtp.gmail.com with ESMTPSA id a76sm42617030pfe.131.2017.02.21.13.30.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Feb 2017 13:30:45 -0800 (PST) From: Omar Sandoval To: Jens Axboe , linux-block@vger.kernel.org Cc: kernel-team@fb.com Subject: [PATCH v2 2/2] blk-mq-sched: separate mark hctx and queue restart operations Date: Tue, 21 Feb 2017 13:29:57 -0800 Message-Id: <9e507577a8db5f856e0263a28ef908d1689a64c8.1487712413.git.osandov@fb.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <19bf336f1d329df8d12fbdc5ab8842f81880c71d.1487712413.git.osandov@fb.com> References: <19bf336f1d329df8d12fbdc5ab8842f81880c71d.1487712413.git.osandov@fb.com> In-Reply-To: <19bf336f1d329df8d12fbdc5ab8842f81880c71d.1487712413.git.osandov@fb.com> References: <19bf336f1d329df8d12fbdc5ab8842f81880c71d.1487712413.git.osandov@fb.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval In blk_mq_sched_dispatch_requests(), we call blk_mq_sched_mark_restart() after we dispatch requests left over on our hardware queue dispatch list. This is so we'll go back and dispatch requests from the scheduler. In this case, it's only necessary to restart the hardware queue that we are running; there's no reason to run other hardware queues just because we are using shared tags. So, split out blk_mq_sched_mark_restart() into two operations, one for just the hardware queue and one for the whole request queue. The core code only needs the hctx variant, but I/O schedulers will want to use both. This also requires adjusting blk_mq_sched_restart_queues() to always check the queue restart flag, not just when using shared tags. Signed-off-by: Omar Sandoval Signed-off-by: Jens Axboe --- block/blk-mq-sched.c | 20 ++++++++------------ block/blk-mq-sched.h | 26 ++++++++++++++++++-------- 2 files changed, 26 insertions(+), 20 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 9e8d6795a8c1..16df0a5e7046 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -205,7 +205,7 @@ void blk_mq_sched_dispatch_requests(struct blk_mq_hw_ctx *hctx) * needing a restart in that case. */ if (!list_empty(&rq_list)) { - blk_mq_sched_mark_restart(hctx); + blk_mq_sched_mark_restart_hctx(hctx); did_work = blk_mq_dispatch_rq_list(hctx, &rq_list); } else if (!has_sched_dispatch) { blk_mq_flush_busy_ctxs(hctx, &rq_list); @@ -331,20 +331,16 @@ static void blk_mq_sched_restart_hctx(struct blk_mq_hw_ctx *hctx) void blk_mq_sched_restart_queues(struct blk_mq_hw_ctx *hctx) { + struct request_queue *q = hctx->queue; unsigned int i; - if (!(hctx->flags & BLK_MQ_F_TAG_SHARED)) + if (test_bit(QUEUE_FLAG_RESTART, &q->queue_flags)) { + if (test_and_clear_bit(QUEUE_FLAG_RESTART, &q->queue_flags)) { + queue_for_each_hw_ctx(q, hctx, i) + blk_mq_sched_restart_hctx(hctx); + } + } else { blk_mq_sched_restart_hctx(hctx); - else { - struct request_queue *q = hctx->queue; - - if (!test_bit(QUEUE_FLAG_RESTART, &q->queue_flags)) - return; - - clear_bit(QUEUE_FLAG_RESTART, &q->queue_flags); - - queue_for_each_hw_ctx(q, hctx, i) - blk_mq_sched_restart_hctx(hctx); } } diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index 7b5f3b95c78e..a75b16b123f7 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -122,17 +122,27 @@ static inline bool blk_mq_sched_has_work(struct blk_mq_hw_ctx *hctx) return false; } -static inline void blk_mq_sched_mark_restart(struct blk_mq_hw_ctx *hctx) +/* + * Mark a hardware queue as needing a restart. + */ +static inline void blk_mq_sched_mark_restart_hctx(struct blk_mq_hw_ctx *hctx) { - if (!test_bit(BLK_MQ_S_SCHED_RESTART, &hctx->state)) { + if (!test_bit(BLK_MQ_S_SCHED_RESTART, &hctx->state)) set_bit(BLK_MQ_S_SCHED_RESTART, &hctx->state); - if (hctx->flags & BLK_MQ_F_TAG_SHARED) { - struct request_queue *q = hctx->queue; +} + +/* + * Mark a hardware queue and the request queue it belongs to as needing a + * restart. + */ +static inline void blk_mq_sched_mark_restart_queue(struct blk_mq_hw_ctx *hctx) +{ + struct request_queue *q = hctx->queue; - if (!test_bit(QUEUE_FLAG_RESTART, &q->queue_flags)) - set_bit(QUEUE_FLAG_RESTART, &q->queue_flags); - } - } + if (!test_bit(BLK_MQ_S_SCHED_RESTART, &hctx->state)) + set_bit(BLK_MQ_S_SCHED_RESTART, &hctx->state); + if (!test_bit(QUEUE_FLAG_RESTART, &q->queue_flags)) + set_bit(QUEUE_FLAG_RESTART, &q->queue_flags); } static inline bool blk_mq_sched_needs_restart(struct blk_mq_hw_ctx *hctx)