From patchwork Mon Mar 20 20:39:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 9635285 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 721186020B for ; Mon, 20 Mar 2017 20:39:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64BB5269E2 for ; Mon, 20 Mar 2017 20:39:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59A2226E97; Mon, 20 Mar 2017 20:39:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 02D6E271CB for ; Mon, 20 Mar 2017 20:39:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754054AbdCTUjl (ORCPT ); Mon, 20 Mar 2017 16:39:41 -0400 Received: from bombadil.infradead.org ([65.50.211.133]:33978 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754024AbdCTUjk (ORCPT ); Mon, 20 Mar 2017 16:39:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=DFtA55KJo0qAvj65rYcvvZWYoznWRtNZvCZ/j0vp/7g=; b=t+AvlRo6gVbL30UpOyScN2sks PHj8Xg9p0AJiapiJXtIiJhZdedLW2rrc489TlkXs/CKyJ9tsrlNwOeRTjbZ0RCNKBeHAyBiGQZwI6 YHFBY20Ff1HEXaxJvgr0KJDQMn0LzFCpZwCEt9D51ljxzIXXlehijuCU4sIsnsI9WXVCp9ryCHDyg lzo3sMwf1bYQOTQvrKOM94dbjASMRj2wISKWW6C7/2UtZ9LL/xilW9Dql+WqCzZyLbM7TCC7ztyam xeppllGLRAUnawOWE9tY5xCXYj9OB9REUXrXz2tgaJWCAohmFmh+ptE30QOPriY95V8E1PslU7Tor RyJp2JldA==; Received: from sjc00ib2.hgst.com ([199.255.44.5] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.87 #1 (Red Hat Linux)) id 1cq45a-0007pg-Gs; Mon, 20 Mar 2017 20:39:38 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: Bart.VanAssche@sandisk.com, linux-block@vger.kernel.org Subject: [PATCH 3/4] blk-mq: improve blk_mq_try_issue_directly Date: Mon, 20 Mar 2017 16:39:29 -0400 Message-Id: <20170320203930.12533-4-hch@lst.de> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170320203930.12533-1-hch@lst.de> References: <20170320203930.12533-1-hch@lst.de> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rename blk_mq_try_issue_directly to __blk_mq_try_issue_directly and add a new wrapper that takes care of RCU / SRCU locking to avoid having boileplate code in the caller which would get duplicated with new callers. Signed-off-by: Christoph Hellwig Reviewed-by: Bart Van Assche --- block/blk-mq.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 53e49a3f6f0a..48748cb799ed 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1434,7 +1434,7 @@ static blk_qc_t request_to_qc_t(struct blk_mq_hw_ctx *hctx, struct request *rq) return blk_tag_to_qc_t(rq->internal_tag, hctx->queue_num, true); } -static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) +static void __blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) { struct request_queue *q = rq->q; struct blk_mq_queue_data bd = { @@ -1478,13 +1478,27 @@ static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) blk_mq_sched_insert_request(rq, false, true, true, false); } +static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, + struct request *rq, blk_qc_t *cookie) +{ + if (!(hctx->flags & BLK_MQ_F_BLOCKING)) { + rcu_read_lock(); + __blk_mq_try_issue_directly(rq, cookie); + rcu_read_unlock(); + } else { + unsigned int srcu_idx = srcu_read_lock(&hctx->queue_rq_srcu); + __blk_mq_try_issue_directly(rq, cookie); + srcu_read_unlock(&hctx->queue_rq_srcu, srcu_idx); + } +} + static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) { const int is_sync = op_is_sync(bio->bi_opf); const int is_flush_fua = op_is_flush(bio->bi_opf); struct blk_mq_alloc_data data = { .flags = 0 }; struct request *rq; - unsigned int request_count = 0, srcu_idx; + unsigned int request_count = 0; struct blk_plug *plug; struct request *same_queue_rq = NULL; blk_qc_t cookie; @@ -1582,18 +1596,8 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) } else /* is_sync */ old_rq = rq; blk_mq_put_ctx(data.ctx); - if (!old_rq) - goto done; - - if (!(data.hctx->flags & BLK_MQ_F_BLOCKING)) { - rcu_read_lock(); - blk_mq_try_issue_directly(old_rq, &cookie); - rcu_read_unlock(); - } else { - srcu_idx = srcu_read_lock(&data.hctx->queue_rq_srcu); - blk_mq_try_issue_directly(old_rq, &cookie); - srcu_read_unlock(&data.hctx->queue_rq_srcu, srcu_idx); - } + if (old_rq) + blk_mq_try_issue_directly(data.hctx, old_rq, &cookie); goto done; }