From patchwork Mon Oct 29 08:49:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658851 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1B5313B5 for ; Mon, 29 Oct 2018 08:49:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A58F029724 for ; Mon, 29 Oct 2018 08:49:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9A21229730; Mon, 29 Oct 2018 08:49:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B3B129724 for ; Mon, 29 Oct 2018 08:49:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729418AbeJ2RhI (ORCPT ); Mon, 29 Oct 2018 13:37:08 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:36674 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbeJ2RhI (ORCPT ); Mon, 29 Oct 2018 13:37:08 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8nK1J030773; Mon, 29 Oct 2018 08:49:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=J7hzrmRyP/DeIBJ0y33e7T6MdMhAdvKFF9gj5cwGn8A=; b=V7jHkB176GRoQSoIMWjxGfYzmZvzsn4G8Ri3YpGLEycPT4TZhqV4Ot0lXjYuexfrVCVQ jCbccjVPMbmGJdQtGKYufzaMbYUJHXNfxYF6qM4oBVmTk+3JWDCm6UqFlFgv3DbeUbFb NRlD/mclHDigD0kkM6HxtpA/zsPzvIA8D9oLqcph0jrFHxecI8xiH4Yi+asK4o9UoIGN LYtNEq3Aiv+nf8aLbuXcXWPbMe3X08ThzMsvzH4Vvm+9m69kllUxUI67byjS7zskj5xV nGaDC8CZ8tioiOoAKh0WXkDox1JGClYZu85+EO4Qq40fzYR7oFU6DzwJMIUQ2FPo9iNU hQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2nduckrxpq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:23 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8nMxt003013 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:22 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w9T8nMTB004540; Mon, 29 Oct 2018 08:49:22 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:49:21 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 1/4] blk-mq: refactor the code of issue request directly Date: Mon, 29 Oct 2018 16:49:08 +0800 Message-Id: <1540802951-5408-2-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290085 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly into one interface which is able to handle the return value from .queue_rq callback. To make the code clearer, introduce new helpers blk_mq_make_decision and enum mq_decision. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 104 +++++++++++++++++++++++++++++++++------------------------ 1 file changed, 60 insertions(+), 44 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index dcf10e3..0f6328b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1711,7 +1711,6 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, switch (ret) { case BLK_STS_OK: blk_mq_update_dispatch_busy(hctx, false); - *cookie = new_cookie; break; case BLK_STS_RESOURCE: case BLK_STS_DEV_RESOURCE: @@ -1720,86 +1719,103 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, break; default: blk_mq_update_dispatch_busy(hctx, false); - *cookie = BLK_QC_T_NONE; + new_cookie = BLK_QC_T_NONE; break; } + if (cookie) + *cookie = new_cookie; return ret; } -static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +enum mq_decision { + MQ_INSERT_QUEUE, + MQ_END_REQUEST, + MQ_DO_NOTHING, +}; + +static inline enum mq_decision + blk_mq_make_dicision(blk_status_t ret, bool bypass_insert) +{ + enum mq_decision dec; + + switch(ret) { + case BLK_STS_OK: + dec = MQ_DO_NOTHING; + break; + case BLK_STS_DEV_RESOURCE: + case BLK_STS_RESOURCE: + dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; + break; + default: + dec = bypass_insert ? MQ_DO_NOTHING : MQ_END_REQUEST; + break; + } + + return dec; +} + +static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, bool bypass_insert) { struct request_queue *q = rq->q; bool run_queue = true; + enum mq_decision dec; + blk_status_t ret = BLK_STS_RESOURCE; + int srcu_idx; + hctx_lock(hctx, &srcu_idx); /* - * RCU or SRCU read lock is needed before checking quiesced flag. + * hctx_lock is needed before checking quiesced flag. * - * When queue is stopped or quiesced, ignore 'bypass_insert' from - * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller, - * and avoid driver to try to dispatch again. + * When queue is stopped or quiesced, ignore 'bypass_insert', + * and return BLK_STS_OK to caller, and avoid driver to try to + * dispatch again. */ if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { run_queue = false; bypass_insert = false; - goto insert; + goto out_unlock; } if (q->elevator && !bypass_insert) - goto insert; + goto out_unlock; if (!blk_mq_get_dispatch_budget(hctx)) - goto insert; + goto out_unlock; if (!blk_mq_get_driver_tag(rq)) { blk_mq_put_dispatch_budget(hctx); - goto insert; + goto out_unlock; } - return __blk_mq_issue_directly(hctx, rq, cookie); -insert: - if (bypass_insert) - return BLK_STS_RESOURCE; - - blk_mq_sched_insert_request(rq, false, run_queue, false); - return BLK_STS_OK; -} - -static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, - struct request *rq, blk_qc_t *cookie) -{ - blk_status_t ret; - int srcu_idx; - - might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); - - hctx_lock(hctx, &srcu_idx); + ret = __blk_mq_issue_directly(hctx, rq, cookie); +out_unlock: + hctx_unlock(hctx, srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false); - if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE) - blk_mq_sched_insert_request(rq, false, true, false); - else if (ret != BLK_STS_OK) + dec = blk_mq_make_dicision(ret, bypass_insert); + switch(dec) { + case MQ_INSERT_QUEUE: + blk_mq_sched_insert_request(rq, false, run_queue, false); + break; + case MQ_END_REQUEST: blk_mq_end_request(rq, ret); + break; + default: + return ret; + } - hctx_unlock(hctx, srcu_idx); + return BLK_STS_OK; } blk_status_t blk_mq_request_issue_directly(struct request *rq) { - blk_status_t ret; - int srcu_idx; - blk_qc_t unused_cookie; struct blk_mq_ctx *ctx = rq->mq_ctx; struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(rq->q, ctx->cpu); - hctx_lock(hctx, &srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, &unused_cookie, true); - hctx_unlock(hctx, srcu_idx); - - return ret; + return blk_mq_try_issue_directly(hctx, rq, NULL, true); } void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, @@ -1921,13 +1937,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) data.hctx = blk_mq_map_queue(q, same_queue_rq->mq_ctx->cpu); blk_mq_try_issue_directly(data.hctx, same_queue_rq, - &cookie); + &cookie, false); } } else if ((q->nr_hw_queues > 1 && is_sync) || (!q->elevator && !data.hctx->dispatch_busy)) { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); - blk_mq_try_issue_directly(data.hctx, rq, &cookie); + blk_mq_try_issue_directly(data.hctx, rq, &cookie, false); } else { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); From patchwork Mon Oct 29 08:49:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658843 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AA291751 for ; Mon, 29 Oct 2018 08:49:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D6DA29724 for ; Mon, 29 Oct 2018 08:49:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 612F729733; Mon, 29 Oct 2018 08:49:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04CF429727 for ; Mon, 29 Oct 2018 08:49:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729424AbeJ2RhI (ORCPT ); Mon, 29 Oct 2018 13:37:08 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:36682 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729418AbeJ2RhI (ORCPT ); Mon, 29 Oct 2018 13:37:08 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8nPFL030827; Mon, 29 Oct 2018 08:49:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=ljr1AVEDeyqNV2z//1XrWvkspXgCo1lAbTrhPYSDkVY=; b=OBBfqoEseTQUF4I7mA3VeuDw1tqArlQDZ5yYYrg2X/Ecz2/chkRFq8LpZgywdpdjshB9 lJ3dddYX/nVuxG1yZWXaZC9KzQwpkiD6jWnZvC62DoUfIq/lZc2cUoJqXS3TdLCoAng0 SjN7wT8HM637QoSMaK05fvXWL5pbthwyUU+r30eTK7m1tlEFZ8CpZzA1529KnRJVLghl YVms6h/7gaXkz2w4rLSynU6yI7fkwdVPhLrMpj+QAOP6EOXlMZ26T1fuTJh97BZBbnzo vgafHZztRm7dbkD6FZVrSAUdOzHQYo0XZ51DgNp54hQOV3JaVJE1ZnrOngSdJI2tt0nE 2w== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2nduckrxpv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:24 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8nOdT003603 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:24 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w9T8nO1V027654; Mon, 29 Oct 2018 08:49:24 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:49:23 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 2/4] blk-mq: insert request without involving any io scheduler Date: Mon, 29 Oct 2018 16:49:09 +0800 Message-Id: <1540802951-5408-3-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290085 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When try to issue request directly, if the queue is stopped or quiesced, 'bypass_insert' will be ignored and return BLK_STS_OK to caller to avoid it dispatch request again. Then the request will be inserted with blk_mq_sched_insert_request. This is not correct for dm-rq case where we should avoid to pass through the underlying paths' io scheduler. To fix it, use blk_mq_request_bypass_insert to insert the request into hctx->dispatch directly. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 0f6328b..e0aa068 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1729,13 +1729,14 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, } enum mq_decision { + MQ_INSERT_DISPATCH, MQ_INSERT_QUEUE, MQ_END_REQUEST, MQ_DO_NOTHING, }; static inline enum mq_decision - blk_mq_make_dicision(blk_status_t ret, bool bypass_insert) + blk_mq_make_dicision(blk_status_t ret, bool bypass_insert, bool force) { enum mq_decision dec; @@ -1745,7 +1746,10 @@ static inline enum mq_decision break; case BLK_STS_DEV_RESOURCE: case BLK_STS_RESOURCE: - dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; + if (force) + dec = bypass_insert ? MQ_INSERT_DISPATCH : MQ_INSERT_QUEUE; + else + dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; break; default: dec = bypass_insert ? MQ_DO_NOTHING : MQ_END_REQUEST; @@ -1761,7 +1765,7 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, bool bypass_insert) { struct request_queue *q = rq->q; - bool run_queue = true; + bool run_queue = true, force = false; enum mq_decision dec; blk_status_t ret = BLK_STS_RESOURCE; int srcu_idx; @@ -1776,7 +1780,7 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, */ if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { run_queue = false; - bypass_insert = false; + force = true; goto out_unlock; } @@ -1795,8 +1799,11 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, out_unlock: hctx_unlock(hctx, srcu_idx); - dec = blk_mq_make_dicision(ret, bypass_insert); + dec = blk_mq_make_dicision(ret, bypass_insert, force); switch(dec) { + case MQ_INSERT_DISPATCH: + blk_mq_request_bypass_insert(rq, run_queue); + break; case MQ_INSERT_QUEUE: blk_mq_sched_insert_request(rq, false, run_queue, false); break; From patchwork Mon Oct 29 08:49:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658847 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 271E21751 for ; Mon, 29 Oct 2018 08:49:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B5D329724 for ; Mon, 29 Oct 2018 08:49:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0FB1029730; Mon, 29 Oct 2018 08:49:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABD8C29724 for ; Mon, 29 Oct 2018 08:49:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729493AbeJ2RhK (ORCPT ); Mon, 29 Oct 2018 13:37:10 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:57892 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729421AbeJ2RhK (ORCPT ); Mon, 29 Oct 2018 13:37:10 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8nJG2035226; Mon, 29 Oct 2018 08:49:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=FeJiBIM4gj+Uv4+k6aC45dq/teyiiFKjX9alY/b/HNs=; b=LoK3Rmqgbh4PyiXegUyDTex8LSuahmkkIBs8YDeV5mFSYYNhppMj61KB1qReFQhEVTsk yxXoPxGIseTX8WYRCodRu1Lf5paDgvY9IUh1nbKCj8XLgOMOGx5VNaZz9r7wQBhlIACS cFaNMlfxtAEzYt1tTgVv/+L0ftKaDl/Q67YqLvq8XDHYMnapJDOyBPhe/9fwJOMQrFOy qg8SGU6IfKozZvgMXxIwGp1VZrh4MVZO0QsLWvjBlHbmLIJyAm5CVhIJaMNBxztOaIye wHEXhlkccH2RrDGlhjokHNWWPYZQF/mhdjqo+vrMef+sZxKXcIN8KyXGNXLNRy1JK5N3 EA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2120.oracle.com with ESMTP id 2ncfypmakv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:26 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8nPK4003740 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:25 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w9T8nPIE027660; Mon, 29 Oct 2018 08:49:25 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:49:25 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 3/4] blk-mq: issue directly with bypass_insert 'false' in blk_mq_sched_insert_requests Date: Mon, 29 Oct 2018 16:49:10 +0800 Message-Id: <1540802951-5408-4-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290085 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is not necessary to issue request directly with bypass_insert 'true' in blk_mq_sched_insert_requests and insert the non-issued requests itself. Just set bypass_insert to 'false' and let blk_mq_try_issue_directly handle them. Signed-off-by: Jianchao Wang --- block/blk-mq-sched.c | 8 +++----- block/blk-mq.c | 10 +--------- 2 files changed, 4 insertions(+), 14 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 29bfe80..23cd97e 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -411,12 +411,10 @@ void blk_mq_sched_insert_requests(struct request_queue *q, * busy in case of 'none' scheduler, and this way may save * us one extra enqueue & dequeue to sw queue. */ - if (!hctx->dispatch_busy && !e && !run_queue_async) { + if (!hctx->dispatch_busy && !e && !run_queue_async) blk_mq_try_issue_list_directly(hctx, list); - if (list_empty(list)) - return; - } - blk_mq_insert_requests(hctx, ctx, list); + else + blk_mq_insert_requests(hctx, ctx, list); } blk_mq_run_hw_queue(hctx, run_queue_async); diff --git a/block/blk-mq.c b/block/blk-mq.c index e0aa068..9c6c858 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1834,15 +1834,7 @@ void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, queuelist); list_del_init(&rq->queuelist); - ret = blk_mq_request_issue_directly(rq); - if (ret != BLK_STS_OK) { - if (ret == BLK_STS_RESOURCE || - ret == BLK_STS_DEV_RESOURCE) { - list_add(&rq->queuelist, list); - break; - } - blk_mq_end_request(rq, ret); - } + ret = blk_mq_try_issue_directly(hctx, rq, NULL, false); } } From patchwork Mon Oct 29 08:49:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658845 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A103213B5 for ; Mon, 29 Oct 2018 08:49:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 95E1529724 for ; Mon, 29 Oct 2018 08:49:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A02929732; Mon, 29 Oct 2018 08:49:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A91D29724 for ; Mon, 29 Oct 2018 08:49:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729508AbeJ2RhM (ORCPT ); Mon, 29 Oct 2018 13:37:12 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:56008 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729421AbeJ2RhM (ORCPT ); Mon, 29 Oct 2018 13:37:12 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8nKaS180966; Mon, 29 Oct 2018 08:49:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=ioT/yFIjJIy+Qg/6RzlFcZD3/CAktU68agYXTE+or0o=; b=4w3vT04h2KK5lBnFElHos7vHtN+vwvrovHTgy6kRUO2WGkjVXOoKHhSkvuKEGMmEuYUQ jZBA3Km6z3GVwO1+IWK9B8upwqmv/qsdoYpLFdmyegvDWEodlHcFOU+77NtzmwZF+pSW ld1RYcPWUO0eCuy9EFxGJ7APSSizJSLZ8yDJ+OvvoTV+7q/uf6KxJfeYDYsEJPB0nRsi 1TM/9tp5C1/gNnPxiqytDC6nPDXxHQ/w5AO9qk3OfR6/nSnZVas50czQDYO0sixv8SMY K5U1tXReDM75xqmVAvWWVquwDW+pvAwu6P+LJcrsU+o1gSMfUGKqk70nXq7llZFoAS5E JQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2ncgnqm8ws-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:29 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8nRBR003828 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:49:28 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w9T8nReV004562; Mon, 29 Oct 2018 08:49:27 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:49:27 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 4/4] blk-mq: ensure hctx to be ran on mapped cpu when issue directly Date: Mon, 29 Oct 2018 16:49:11 +0800 Message-Id: <1540802951-5408-5-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802951-5408-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=947 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290085 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When issue request directly and the task is migrated out of the original cpu where it allocates request, hctx could be ran on the cpu where it is not mapped. To fix this, insert the request if BLK_MQ_F_BLOCKING is set, check whether the current is mapped to the hctx and invoke __blk_mq_issue_directly under preemption disabled. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 9c6c858..ced3346 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1770,6 +1770,17 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, blk_status_t ret = BLK_STS_RESOURCE; int srcu_idx; + if (hctx->flags & BLK_MQ_F_BLOCKING) { + force = true; + goto out; + } + + if (!cpumask_test_cpu(get_cpu(), hctx->cpumask)) { + put_cpu(); + force = true; + goto out; + } + hctx_lock(hctx, &srcu_idx); /* * hctx_lock is needed before checking quiesced flag. @@ -1798,7 +1809,8 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, ret = __blk_mq_issue_directly(hctx, rq, cookie); out_unlock: hctx_unlock(hctx, srcu_idx); - + put_cpu(); +out: dec = blk_mq_make_dicision(ret, bypass_insert, force); switch(dec) { case MQ_INSERT_DISPATCH: