From patchwork Thu Nov 9 18:49:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10051691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 05FC760632 for ; Thu, 9 Nov 2017 18:50:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED1242AFD4 for ; Thu, 9 Nov 2017 18:50:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E11A92B06D; Thu, 9 Nov 2017 18:50:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4648F29AD9 for ; Thu, 9 Nov 2017 18:50:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753532AbdKISuU (ORCPT ); Thu, 9 Nov 2017 13:50:20 -0500 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:7504 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753105AbdKISuD (ORCPT ); Thu, 9 Nov 2017 13:50:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1510253404; x=1541789404; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=zGdy/QYU6MZIeQ5Udli/DeOptSV0SYOgA8Eijs3MOtE=; b=KDZhAIpqn9gAYkZdgynaCUSEpSdSJNKUIMJoFf4hhop91XrmVtP8JSvy o82gXWl3bCBCWXsb/KkQ/EA/7+e42uxdI4XwGrW7P1zTQbf/RYTcY7Pe6 VjUWLwA2uYxe5t7yGXoIpdlQjgh/+qmYWtIcZXLDp40ePVv19C0lTKC01 5f9XdlQ93CTfNaonnKApaBe2ffNrhz4uvfcPyclpvM5x30ozDMl8AQ9D6 dKiCR8fVjvJguP9t++O5DcpOJ5eUusvF0sWvuXtLytGnK+MJaOCWr0sYk pARYujYsj6CBw8SZ379nQqLGn3CbFX5NRjBssKbsR1JTf24SKvQ+j+AwE Q==; X-IronPort-AV: E=Sophos;i="5.44,370,1505750400"; d="scan'208";a="61801477" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 10 Nov 2017 02:50:03 +0800 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP; 09 Nov 2017 10:47:26 -0800 Received: from unknown (HELO MILHUBIP04.sdcorp.global.sandisk.com) ([10.177.9.97]) by uls-op-cesaip02.wdc.com with ESMTP; 09 Nov 2017 10:50:02 -0800 Received: from milsmgip11.sandisk.com (10.177.9.6) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.319.2; Thu, 9 Nov 2017 10:50:01 -0800 X-AuditID: 0ab10959-41c5898000002c97-f2-5a04a359a13e Received: from thinkpad-bart.int.fusionio.com ( [10.177.9.6]) by (Symantec Messaging Gateway) with SMTP id 82.AE.11415.953A40A5; Thu, 9 Nov 2017 10:50:01 -0800 (PST) From: Bart Van Assche To: Jens Axboe CC: , , "Christoph Hellwig" , "Martin K . Petersen" , Oleksandr Natalenko , Ming Lei , Martin Steigerwald , "Bart Van Assche" , Johannes Thumshirn Subject: [PATCH v12 2/7] block: Introduce blk_get_request_flags() Date: Thu, 9 Nov 2017 10:49:54 -0800 Message-ID: <20171109184959.4617-3-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20171109184959.4617-1-bart.vanassche@wdc.com> References: <20171109184959.4617-1-bart.vanassche@wdc.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLJMWRmVeSWpSXmKPExsXCtZGTTTdyMUuUwfTfqhar7/azWVz684XR YuXqo0wWj7tmsFrsvaVt0X19B5vF8uP/mCyWzGpmsjg0GUgsfLGJ2YHL4/LZUo+J6/k8dt9s YPN4eYnD4+PTWywe7/ddZfPYfLra4/MmOY/2A91MAZxRXDYpqTmZZalF+nYJXBnNL+eyF7w1 rWj6+ZC5gfGudhcjJ4eEgInE0x2fWbsYuTiEBFYyShxe9pwFJMEmoCdxat4+JhBbREBBouf3 SjYQm1ngDZPEw2+SILawgKPEwj19jCA2i4CqxJvJL8B6eQWsJLY1bGeFWCAvsfbtPbAaTgFr ib975gLN4QBaZiWx+z7PBEbuBYwMqxjFcjNzinPTMwsMDfWKE/NSMouz9ZLzczcxQgIscgfj 04nmhxiZODilGhgdxC5J/GVkkOdhmPxJeXG9vqer8apukaKwY0qqu+50ys6WaO9KchNQ7Vx+ NMBw+sTEruhCI+f2m4n+k8u02tfq9HhaasrHFC64MT+Kb+mGH2ytLKqTa6btifya6fHQpCPw 8YJgPqdZc+492+Rwjyc97MXxdalzri3K/68o8fHgeyaFZaLJSUosxRmJhlrMRcWJANTsMXng AQAA MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A side effect of this patch is that the GFP mask that is passed to several allocation functions in the legacy block layer is changed from GFP_KERNEL into __GFP_DIRECT_RECLAIM. Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke Tested-by: Martin Steigerwald Tested-by: Oleksandr Natalenko Cc: Christoph Hellwig Cc: Ming Lei Cc: Johannes Thumshirn --- block/blk-core.c | 50 +++++++++++++++++++++++++++++++++++--------------- include/linux/blkdev.h | 3 +++ 2 files changed, 38 insertions(+), 15 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index a4362849059a..0f7093dfc010 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1160,7 +1160,7 @@ int blk_update_nr_requests(struct request_queue *q, unsigned int nr) * @rl: request list to allocate from * @op: operation and flags * @bio: bio to allocate request for (can be %NULL) - * @gfp_mask: allocation mask + * @flags: BLQ_MQ_REQ_* flags * * Get a free request from @q. This function may fail under memory * pressure or if @q is dead. @@ -1170,7 +1170,7 @@ int blk_update_nr_requests(struct request_queue *q, unsigned int nr) * Returns request pointer on success, with @q->queue_lock *not held*. */ static struct request *__get_request(struct request_list *rl, unsigned int op, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, unsigned int flags) { struct request_queue *q = rl->q; struct request *rq; @@ -1179,6 +1179,8 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, struct io_cq *icq = NULL; const bool is_sync = op_is_sync(op); int may_queue; + gfp_t gfp_mask = flags & BLK_MQ_REQ_NOWAIT ? GFP_ATOMIC : + __GFP_DIRECT_RECLAIM; req_flags_t rq_flags = RQF_ALLOCED; lockdep_assert_held(q->queue_lock); @@ -1339,7 +1341,7 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, * @q: request_queue to allocate request from * @op: operation and flags * @bio: bio to allocate request for (can be %NULL) - * @gfp_mask: allocation mask + * @flags: BLK_MQ_REQ_* flags. * * Get a free request from @q. If %__GFP_DIRECT_RECLAIM is set in @gfp_mask, * this function keeps retrying under memory pressure and fails iff @q is dead. @@ -1349,7 +1351,7 @@ static struct request *__get_request(struct request_list *rl, unsigned int op, * Returns request pointer on success, with @q->queue_lock *not held*. */ static struct request *get_request(struct request_queue *q, unsigned int op, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, unsigned int flags) { const bool is_sync = op_is_sync(op); DEFINE_WAIT(wait); @@ -1361,7 +1363,7 @@ static struct request *get_request(struct request_queue *q, unsigned int op, rl = blk_get_rl(q, bio); /* transferred to @rq on success */ retry: - rq = __get_request(rl, op, bio, gfp_mask); + rq = __get_request(rl, op, bio, flags); if (!IS_ERR(rq)) return rq; @@ -1370,7 +1372,7 @@ static struct request *get_request(struct request_queue *q, unsigned int op, return ERR_PTR(-EAGAIN); } - if (!gfpflags_allow_blocking(gfp_mask) || unlikely(blk_queue_dying(q))) { + if ((flags & BLK_MQ_REQ_NOWAIT) || unlikely(blk_queue_dying(q))) { blk_put_rl(rl); return rq; } @@ -1397,10 +1399,13 @@ static struct request *get_request(struct request_queue *q, unsigned int op, goto retry; } +/* flags: BLK_MQ_REQ_PREEMPT and/or BLK_MQ_REQ_NOWAIT. */ static struct request *blk_old_get_request(struct request_queue *q, - unsigned int op, gfp_t gfp_mask) + unsigned int op, unsigned int flags) { struct request *rq; + gfp_t gfp_mask = flags & BLK_MQ_REQ_NOWAIT ? GFP_ATOMIC : + __GFP_DIRECT_RECLAIM; int ret = 0; WARN_ON_ONCE(q->mq_ops); @@ -1413,7 +1418,7 @@ static struct request *blk_old_get_request(struct request_queue *q, if (ret) return ERR_PTR(ret); spin_lock_irq(q->queue_lock); - rq = get_request(q, op, NULL, gfp_mask); + rq = get_request(q, op, NULL, flags); if (IS_ERR(rq)) { spin_unlock_irq(q->queue_lock); blk_queue_exit(q); @@ -1427,25 +1432,40 @@ static struct request *blk_old_get_request(struct request_queue *q, return rq; } -struct request *blk_get_request(struct request_queue *q, unsigned int op, - gfp_t gfp_mask) +/** + * blk_get_request_flags - allocate a request + * @q: request queue to allocate a request for + * @op: operation (REQ_OP_*) and REQ_* flags, e.g. REQ_SYNC. + * @flags: BLK_MQ_REQ_* flags, e.g. BLK_MQ_REQ_NOWAIT. + */ +struct request *blk_get_request_flags(struct request_queue *q, unsigned int op, + unsigned int flags) { struct request *req; + WARN_ON_ONCE(op & REQ_NOWAIT); + WARN_ON_ONCE(flags & ~BLK_MQ_REQ_NOWAIT); + if (q->mq_ops) { - req = blk_mq_alloc_request(q, op, - (gfp_mask & __GFP_DIRECT_RECLAIM) ? - 0 : BLK_MQ_REQ_NOWAIT); + req = blk_mq_alloc_request(q, op, flags); if (!IS_ERR(req) && q->mq_ops->initialize_rq_fn) q->mq_ops->initialize_rq_fn(req); } else { - req = blk_old_get_request(q, op, gfp_mask); + req = blk_old_get_request(q, op, flags); if (!IS_ERR(req) && q->initialize_rq_fn) q->initialize_rq_fn(req); } return req; } +EXPORT_SYMBOL(blk_get_request_flags); + +struct request *blk_get_request(struct request_queue *q, unsigned int op, + gfp_t gfp_mask) +{ + return blk_get_request_flags(q, op, gfp_mask & __GFP_DIRECT_RECLAIM ? + 0 : BLK_MQ_REQ_NOWAIT); +} EXPORT_SYMBOL(blk_get_request); /** @@ -1871,7 +1891,7 @@ static blk_qc_t blk_queue_bio(struct request_queue *q, struct bio *bio) * Returns with the queue unlocked. */ blk_queue_enter_live(q); - req = get_request(q, bio->bi_opf, bio, GFP_NOIO); + req = get_request(q, bio->bi_opf, bio, 0); if (IS_ERR(req)) { blk_queue_exit(q); __wbt_done(q->rq_wb, wb_acct); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 1437ef4d8037..1af5ddd0f631 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -927,6 +927,9 @@ extern void blk_rq_init(struct request_queue *q, struct request *rq); extern void blk_init_request_from_bio(struct request *req, struct bio *bio); extern void blk_put_request(struct request *); extern void __blk_put_request(struct request_queue *, struct request *); +extern struct request *blk_get_request_flags(struct request_queue *, + unsigned int op, + unsigned int flags); extern struct request *blk_get_request(struct request_queue *, unsigned int op, gfp_t gfp_mask); extern void blk_requeue_request(struct request_queue *, struct request *);