From patchwork Mon Dec 5 18:27:02 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9461311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0218B6071F for ; Mon, 5 Dec 2016 18:30:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2D4127C2D for ; Mon, 5 Dec 2016 18:30:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D78C327E78; Mon, 5 Dec 2016 18:30:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8036A27C2D for ; Mon, 5 Dec 2016 18:30:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752557AbcLESaQ (ORCPT ); Mon, 5 Dec 2016 13:30:16 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:59147 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752669AbcLES1U (ORCPT ); Mon, 5 Dec 2016 13:27:20 -0500 Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id uB5IJn6X026204; Mon, 5 Dec 2016 10:27:18 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=r9eF0WBSGc7vgOEXx6cZ2FywxhdgkiiJ/axM5JIALG0=; b=N+YVjoNTfCsnCuo56vuTzLICtnUkX8GOziz63TLOn/iOmdEldXV+1rTeIGWE0bkeTAZA NX9UPGkM1HFMaIfp+rowo2ttYFz0NUu0nMucvHKHFORnArJePStkbyDSZoFd6TkgYouZ HYjworyYqAz54J2mPEH/+g0lThe1c9uF0N8= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2758nsapy5-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 05 Dec 2016 10:27:18 -0800 Received: from localhost.localdomain (192.168.54.13) by mail.thefacebook.com (192.168.16.24) with Microsoft SMTP Server (TLS) id 14.3.294.0; Mon, 5 Dec 2016 10:27:17 -0800 From: Jens Axboe To: , , CC: , Jens Axboe Subject: [PATCH 3/7] block: use appropriate queue running functions Date: Mon, 5 Dec 2016 11:27:02 -0700 Message-ID: <1480962426-15767-4-git-send-email-axboe@fb.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1480962426-15767-1-git-send-email-axboe@fb.com> References: <1480962426-15767-1-git-send-email-axboe@fb.com> MIME-Version: 1.0 X-Originating-IP: [192.168.54.13] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-12-05_14:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Use MQ variants for MQ, legacy ones for legacy. Signed-off-by: Jens Axboe --- block/blk-core.c | 5 ++++- block/blk-exec.c | 10 ++++++++-- block/blk-flush.c | 14 ++++++++++---- block/elevator.c | 5 ++++- 4 files changed, 26 insertions(+), 8 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 813c448453bf..f0aa810a5fe2 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -340,7 +340,10 @@ void __blk_run_queue(struct request_queue *q) if (unlikely(blk_queue_stopped(q))) return; - __blk_run_queue_uncond(q); + if (WARN_ON_ONCE(q->mq_ops)) + blk_mq_run_hw_queues(q, true); + else + __blk_run_queue_uncond(q); } EXPORT_SYMBOL(__blk_run_queue); diff --git a/block/blk-exec.c b/block/blk-exec.c index 3356dff5508c..6c3f12b32f86 100644 --- a/block/blk-exec.c +++ b/block/blk-exec.c @@ -80,8 +80,14 @@ void blk_execute_rq_nowait(struct request_queue *q, struct gendisk *bd_disk, } __elv_add_request(q, rq, where); - __blk_run_queue(q); - spin_unlock_irq(q->queue_lock); + + if (q->mq_ops) { + spin_unlock_irq(q->queue_lock); + blk_mq_run_hw_queues(q, false); + } else { + __blk_run_queue(q); + spin_unlock_irq(q->queue_lock); + } } EXPORT_SYMBOL_GPL(blk_execute_rq_nowait); diff --git a/block/blk-flush.c b/block/blk-flush.c index 040c36b83ef7..8f2354d97e17 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -265,8 +265,10 @@ static void flush_end_io(struct request *flush_rq, int error) * kblockd. */ if (queued || fq->flush_queue_delayed) { - WARN_ON(q->mq_ops); - blk_run_queue_async(q); + if (q->mq_ops) + blk_mq_run_hw_queues(q, true); + else + blk_run_queue_async(q); } fq->flush_queue_delayed = 0; if (!blk_use_sched_path(q)) @@ -346,8 +348,12 @@ static void flush_data_end_io(struct request *rq, int error) * After populating an empty queue, kick it to avoid stall. Read * the comment in flush_end_io(). */ - if (blk_flush_complete_seq(rq, fq, REQ_FSEQ_DATA, error)) - blk_run_queue_async(q); + if (blk_flush_complete_seq(rq, fq, REQ_FSEQ_DATA, error)) { + if (q->mq_ops) + blk_mq_run_hw_queues(q, true); + else + blk_run_queue_async(q); + } } static void mq_flush_data_end_io(struct request *rq, int error) diff --git a/block/elevator.c b/block/elevator.c index a18a5db274e4..11d2cfee2bc1 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -627,7 +627,10 @@ void __elv_add_request(struct request_queue *q, struct request *rq, int where) * with anything. There's no point in delaying queue * processing. */ - __blk_run_queue(q); + if (q->mq_ops) + blk_mq_run_hw_queues(q, true); + else + __blk_run_queue(q); break; case ELEVATOR_INSERT_SORT_MERGE: