From patchwork Mon Oct 30 22:42:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10033375 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 778216039A for ; Mon, 30 Oct 2017 22:42:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68FC82898D for ; Mon, 30 Oct 2017 22:42:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5DBB72898F; Mon, 30 Oct 2017 22:42:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68B792898D for ; Mon, 30 Oct 2017 22:42:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752581AbdJ3WmQ (ORCPT ); Mon, 30 Oct 2017 18:42:16 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:5427 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752608AbdJ3WmK (ORCPT ); Mon, 30 Oct 2017 18:42:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1509403331; x=1540939331; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=687sZbqrIMjY8Z5SmYIRodzv0VcvZFTVBoMoxGU1QeM=; b=BykvZaYgdCkFnjbnGrBPM1o1YBCjBHHG2DslnXbSctwiSyZB3kOjP0zE e8qNEfsX4OzTFmh4VKQy/PKpd4OFWazD0OJeBDlP4/DzxJZpHwygrZkEY znVcLFaMRpdueopKD4Se6G/4mEmFd+R6w+q6HEE7DpQIpcsh9lUcCn0Hh r8urjoz371au0LypOKAsxh1TQuycNi5smBIZNig7zCiw6lHfxeqpHFDBb 0K/zjTAMVKvs63yAYhVwZiIGZUzpBZV2juxVAHjgnCXFrRA3FuotfVi9P 2W6EDp7jwS9aRFnX/wGcBHguAoIFq/gMe2EwOvcrDY3YQTQIsPZhjhObk Q==; X-IronPort-AV: E=Sophos;i="5.44,321,1505750400"; d="scan'208";a="60181980" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 31 Oct 2017 06:42:09 +0800 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP; 30 Oct 2017 15:40:17 -0700 Received: from unknown (HELO MILHUBIP04.sdcorp.global.sandisk.com) ([10.177.9.97]) by uls-op-cesaip02.wdc.com with ESMTP; 30 Oct 2017 15:42:08 -0700 Received: from milsmgip11.sandisk.com (10.177.9.6) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.319.2; Mon, 30 Oct 2017 15:42:07 -0700 X-AuditID: 0ab10959-41c5898000002c97-92-59f7aabf07af Received: from thinkpad-bart.int.fusionio.com ( [10.177.8.100]) by (Symantec Messaging Gateway) with SMTP id A0.BD.11415.FBAA7F95; Mon, 30 Oct 2017 15:42:07 -0700 (PDT) From: Bart Van Assche To: Jens Axboe CC: , , "Christoph Hellwig" , "Martin K . Petersen" , Oleksandr Natalenko , Ming Lei , Martin Steigerwald , "Bart Van Assche" , Johannes Thumshirn Subject: [PATCH v11 2/7] block: Introduce blk_get_request_flags() Date: Mon, 30 Oct 2017 15:42:00 -0700 Message-ID: <20171030224205.25212-3-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171030224205.25212-1-bart.vanassche@wdc.com> References: <20171030224205.25212-1-bart.vanassche@wdc.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrHJMWRmVeSWpSXmKPExsXCtZEjRXf/qu+RBku3qlqsvtvPZnHpzxdG i5WrjzJZPO6awWqx95a2Rff1HWwWy4//Y7JYMquZyeLQZCCx8MUmZgcuj8tnSz0mrufz2H2z gc3j5SUOj49Pb7F4vN93lc1j8+lqj8+b5DzaD3QzBXBGcdmkpOZklqUW6dslcGVM/riOvWCT acWtveuZGxiXa3cxcnJICJhINNz9xtbFyMUhJLCaUaL75HImkASbgJ7EqXn7wGwRAQWJnt8r 2UBsZoE3TBIPv0mC2MICjhKLel+zgtgsAqoSy493MYPYvALWEh/2PGGFWCAv8X7BfUYQm1PA RmLRpu/sILYQUM2uidOZJjByL2BkWMUolpuZU5ybnllgaKhXnJiXklmcrZecn7uJERxinJE7 GJ9OND/EyMTBKdXAmLwyO3Fl3Gpz7qc/nPXFvmrGrPBrnKD8aZKq8AvWm3x2M6piNpdO9Ij+ 2PpFY37MqUW8Dy3q6g33FAU/2rmg46hOtnaw6/47Qn95VJVTZdtU2Cs3f3dx3/RoBnfJNVnj 9wo3dT/OjE5UTPQQuO33K7dTO332k+yoXwcnCNQ80niuXtKhun+mEktxRqKhFnNRcSIAXEA6 WuEBAAA= MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A side effect of this patch is that the GFP mask that is passed to several allocation functions in the legacy block layer is changed from GFP_KERNEL into __GFP_DIRECT_RECLAIM. Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke Tested-by: Martin Steigerwald Cc: Christoph Hellwig Cc: Ming Lei Cc: Johannes Thumshirn --- block/blk-core.c | 50 +++++++++++++++++++++++++++++++++++--------------- include/linux/blkdev.h | 3 +++ 2 files changed, 38 insertions(+), 15 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index ec4eafb5af9f..396104c05b38 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1159,7 +1159,7 @@ int blk_update_nr_requests(struct request_queue *q, unsigned int nr) * @rl: request list to allocate from * @op: operation and flags * @bio: bio to allocate request for (can be %NULL) - * @gfp_mask: allocation mask + * @flags: BLQ_MQ_REQ_* flags * * Get a free request from @q. This function may fail under memory * pressure or if @q is dead. @@ -1169,7 +1169,7 @@ int blk_update_nr_requests(struct request_queue *q, unsigned int nr) * Returns request pointer on success, with @q->queue_lock *not held*. */ static struct request *__get_request(struct request_list *rl, unsigned int op, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, unsigned int flags) { struct request_queue *q = rl->q; struct request *rq; @@ -1178,6 +1178,8 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, struct io_cq *icq = NULL; const bool is_sync = op_is_sync(op); int may_queue; + gfp_t gfp_mask = flags & BLK_MQ_REQ_NOWAIT ? GFP_ATOMIC : + __GFP_DIRECT_RECLAIM; req_flags_t rq_flags = RQF_ALLOCED; lockdep_assert_held(q->queue_lock); @@ -1338,7 +1340,7 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, * @q: request_queue to allocate request from * @op: operation and flags * @bio: bio to allocate request for (can be %NULL) - * @gfp_mask: allocation mask + * @flags: BLK_MQ_REQ_* flags. * * Get a free request from @q. If %__GFP_DIRECT_RECLAIM is set in @gfp_mask, * this function keeps retrying under memory pressure and fails iff @q is dead. @@ -1348,7 +1350,7 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, * Returns request pointer on success, with @q->queue_lock *not held*. */ static struct request *get_request(struct request_queue *q, unsigned int op, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, unsigned int flags) { const bool is_sync = op_is_sync(op); DEFINE_WAIT(wait); @@ -1360,7 +1362,7 @@ static struct request *get_request(struct request_queue *q, unsigned int op, rl = blk_get_rl(q, bio); /* transferred to @rq on success */ retry: - rq = __get_request(rl, op, bio, gfp_mask); + rq = __get_request(rl, op, bio, flags); if (!IS_ERR(rq)) return rq; @@ -1369,7 +1371,7 @@ static struct request *get_request(struct request_queue *q, unsigned int op, return ERR_PTR(-EAGAIN); } - if (!gfpflags_allow_blocking(gfp_mask) || unlikely(blk_queue_dying(q))) { + if ((flags & BLK_MQ_REQ_NOWAIT) || unlikely(blk_queue_dying(q))) { blk_put_rl(rl); return rq; } @@ -1396,10 +1398,13 @@ static struct request *get_request(struct request_queue *q, unsigned int op, goto retry; } +/* flags: BLK_MQ_REQ_PREEMPT and/or BLK_MQ_REQ_NOWAIT. */ static struct request *blk_old_get_request(struct request_queue *q, - unsigned int op, gfp_t gfp_mask) + unsigned int op, unsigned int flags) { struct request *rq; + gfp_t gfp_mask = flags & BLK_MQ_REQ_NOWAIT ? GFP_ATOMIC : + __GFP_DIRECT_RECLAIM; int ret = 0; WARN_ON_ONCE(q->mq_ops); @@ -1412,7 +1417,7 @@ static struct request *blk_old_get_request(struct request_queue *q, if (ret) return ERR_PTR(ret); spin_lock_irq(q->queue_lock); - rq = get_request(q, op, NULL, gfp_mask); + rq = get_request(q, op, NULL, flags); if (IS_ERR(rq)) { spin_unlock_irq(q->queue_lock); blk_queue_exit(q); @@ -1426,25 +1431,40 @@ static struct request *blk_old_get_request(struct request_queue *q, return rq; } -struct request *blk_get_request(struct request_queue *q, unsigned int op, - gfp_t gfp_mask) +/** + * blk_get_request_flags - allocate a request + * @q: request queue to allocate a request for + * @op: operation (REQ_OP_*) and REQ_* flags, e.g. REQ_SYNC. + * @flags: BLK_MQ_REQ_* flags, e.g. BLK_MQ_REQ_NOWAIT. + */ +struct request *blk_get_request_flags(struct request_queue *q, unsigned int op, + unsigned int flags) { struct request *req; + WARN_ON_ONCE(op & REQ_NOWAIT); + WARN_ON_ONCE(flags & ~BLK_MQ_REQ_NOWAIT); + if (q->mq_ops) { - req = blk_mq_alloc_request(q, op, - (gfp_mask & __GFP_DIRECT_RECLAIM) ? - 0 : BLK_MQ_REQ_NOWAIT); + req = blk_mq_alloc_request(q, op, flags); if (!IS_ERR(req) && q->mq_ops->initialize_rq_fn) q->mq_ops->initialize_rq_fn(req); } else { - req = blk_old_get_request(q, op, gfp_mask); + req = blk_old_get_request(q, op, flags); if (!IS_ERR(req) && q->initialize_rq_fn) q->initialize_rq_fn(req); } return req; } +EXPORT_SYMBOL(blk_get_request_flags); + +struct request *blk_get_request(struct request_queue *q, unsigned int op, + gfp_t gfp_mask) +{ + return blk_get_request_flags(q, op, gfp_mask & __GFP_DIRECT_RECLAIM ? + 0 : BLK_MQ_REQ_NOWAIT); +} EXPORT_SYMBOL(blk_get_request); /** @@ -1870,7 +1890,7 @@ static blk_qc_t blk_queue_bio(struct request_queue *q, struct bio *bio) * Returns with the queue unlocked. */ blk_queue_enter_live(q); - req = get_request(q, bio->bi_opf, bio, GFP_NOIO); + req = get_request(q, bio->bi_opf, bio, 0); if (IS_ERR(req)) { blk_queue_exit(q); __wbt_done(q->rq_wb, wb_acct); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 72637028f3c9..05203175eb9c 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -924,6 +924,9 @@ extern void blk_rq_init(struct request_queue *q, struct request *rq); extern void blk_init_request_from_bio(struct request *req, struct bio *bio); extern void blk_put_request(struct request *); extern void __blk_put_request(struct request_queue *, struct request *); +extern struct request *blk_get_request_flags(struct request_queue *, + unsigned int op, + unsigned int flags); extern struct request *blk_get_request(struct request_queue *, unsigned int op, gfp_t gfp_mask); extern void blk_requeue_request(struct request_queue *, struct request *);