From patchwork Sat Nov 17 21:43:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 106A913BF for ; Sat, 17 Nov 2018 21:44:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 013722A4E6 for ; Sat, 17 Nov 2018 21:44:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E98912A4FF; Sat, 17 Nov 2018 21:44:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7A74D2A4E6 for ; Sat, 17 Nov 2018 21:44:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726029AbeKRICG (ORCPT ); Sun, 18 Nov 2018 03:02:06 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:34263 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725743AbeKRICG (ORCPT ); Sun, 18 Nov 2018 03:02:06 -0500 Received: by mail-pg1-f193.google.com with SMTP id 17so11857824pgg.1 for ; Sat, 17 Nov 2018 13:44:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PncRR/XNvCzHFApU1TwopG2wJbE77TRgVK2hhkw731Y=; b=aSBJLmT0aMpIufp82xThHgCg3ZyqhDefx8lfo0CMzaDL7a/HfZl1Ip6yImObblr605 GLxn1lSZr2TVrm1yznT1LRxF5kQkowtFMgidjV5c6jP4hE0DOoIRPgAknFNckT39mN/q F8fwK4gCOK1X7uDpcqmntsgCIOFj589lG/aB5JzERLc3GsrT21j+rP2nWfvMMZbhzvUc RoVwzx+U3AfBp2YIO59TbaewQNtQkuw09MQD7nLMnerxV0WAk1DPGecrhlO/mzGPe4TH hzWg3cxwMIZ8HkkSVL+vEiJtzD53eWbhvvMBEONsBQ4P0q8Nm1TKml7EAe1YsdYfL4EU gB5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PncRR/XNvCzHFApU1TwopG2wJbE77TRgVK2hhkw731Y=; b=NmyARzlAnmoLfsTLFt4HayvUAgfLDt7JnXIUZdqqUjVx8yKOJcikPSmD16fBE+3eqi DHwXuacZucU6+5I/Kuc70B/6kIP1qKj+pB6tiHO/3OWevGjrWMSY8LDFuqUjhUgBlklW zB+FNaOoGFaZi+Wdo0sUFrMRaHQlgxz4BccEDxUm5xzG0Auk7FXlORPbuxtwRq3SacfW faEs97niZrj7YnoM7h/QUeIa1a8xDco3Ogr5WtjoEaW4kjPcQgu+qI4Q39TjcYROSpr4 8XzqbZ7jQd+dReh5lgT9xMX2zqxb4HR+pYNXykI6o82WqyRDrkG76BRBkfu2WTQd7acq 3jnw== X-Gm-Message-State: AGRZ1gIpDVVWWCbWkrExORShHEX9pwNcZQb7VmMQQ3v/aCeGEVH+jr5Q kb1F0GzK+RF8zTFJ7PG/WCWxhgwE8rg= X-Google-Smtp-Source: AJdET5eHlLE3mnf1/TVZSIj/gHtJAPEkX89yr2ox5Rds9Ayax1KpaREZk/NgIP3XkkBf3fh9bG9syA== X-Received: by 2002:a62:d504:: with SMTP id d4mr4604539pfg.38.1542491042036; Sat, 17 Nov 2018 13:44:02 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:00 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 1/7] block: avoid ordered task state change for polled IO Date: Sat, 17 Nov 2018 14:43:48 -0700 Message-Id: <20181117214354.822-2-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For the core poll helper, the task state setting don't need to imply any atomics, as it's the current task itself that is being modified and we're not going to sleep. For IRQ driven, the wakeup path have the necessary barriers to not need us using the heavy handed version of the task state setting. Signed-off-by: Jens Axboe Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 4 ++-- fs/block_dev.c | 7 +++++-- fs/iomap.c | 3 ++- mm/page_io.c | 3 ++- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 32b246ed44c0..7fc4abb4cc36 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3331,12 +3331,12 @@ static bool __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) ret = q->mq_ops->poll(hctx, rq->tag); if (ret > 0) { hctx->poll_success++; - set_current_state(TASK_RUNNING); + __set_current_state(TASK_RUNNING); return true; } if (signal_pending_state(state, current)) - set_current_state(TASK_RUNNING); + __set_current_state(TASK_RUNNING); if (current->state == TASK_RUNNING) return true; diff --git a/fs/block_dev.c b/fs/block_dev.c index 4d79bc80fb41..64ba27b8b754 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -237,9 +237,11 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter, qc = submit_bio(&bio); for (;;) { - set_current_state(TASK_UNINTERRUPTIBLE); + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(bio.bi_private)) break; + if (!(iocb->ki_flags & IOCB_HIPRI) || !blk_poll(bdev_get_queue(bdev), qc)) io_schedule(); @@ -415,7 +417,8 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, int nr_pages) return -EIOCBQUEUED; for (;;) { - set_current_state(TASK_UNINTERRUPTIBLE); + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(dio->waiter)) break; diff --git a/fs/iomap.c b/fs/iomap.c index b0462b363bad..c5df035ace6f 100644 --- a/fs/iomap.c +++ b/fs/iomap.c @@ -1888,7 +1888,8 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, return -EIOCBQUEUED; for (;;) { - set_current_state(TASK_UNINTERRUPTIBLE); + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(dio->submit.waiter)) break; diff --git a/mm/page_io.c b/mm/page_io.c index 57572ff46016..a7271fa481f6 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -405,7 +405,8 @@ int swap_readpage(struct page *page, bool synchronous) bio_get(bio); qc = submit_bio(bio); while (synchronous) { - set_current_state(TASK_UNINTERRUPTIBLE); + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(bio->bi_private)) break; From patchwork Sat Nov 17 21:43:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687625 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5432413BF for ; Sat, 17 Nov 2018 21:44:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 450F22A4E6 for ; Sat, 17 Nov 2018 21:44:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 398962A4FF; Sat, 17 Nov 2018 21:44:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABDA72A4E6 for ; Sat, 17 Nov 2018 21:44:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726034AbeKRICH (ORCPT ); Sun, 18 Nov 2018 03:02:07 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:44414 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725743AbeKRICH (ORCPT ); Sun, 18 Nov 2018 03:02:07 -0500 Received: by mail-pl1-f196.google.com with SMTP id s5-v6so12758639plq.11 for ; Sat, 17 Nov 2018 13:44:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=m/skZUFQZ3fxw1cd1nVnD2mYBZ9RX6Jjcd9knLDBuZs=; b=DKgcnShtDt480++uiQ24V0eL/a+xa6pemKjcWqULvSS6Y45MFTI5tPLkV0Vd4opgsu L+0d6F9JCRLk0fCh3vS+KMTly7zb8WtwFe0jhipFyjGI42AMKTafqtlk602EutxNkA1n zU8N+N/a+mDix+/lvHc06Us0nMJrCpjK1ISmxJmu3iE3Fghp2APLDdobJfkY9p4Y6OQ3 svWgrLwKz25mFUiVdJTAVtD7+Shes56gqRnWP7mhSn0BKMSvwKUVT/U0Tgcjb7sxqQV5 HGjXrQumj9/5AMdnwvJjB+GHauJwPV1slPyXLt/HAwr/w+ueJCQzPXRAqYijnAgGevag O1Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=m/skZUFQZ3fxw1cd1nVnD2mYBZ9RX6Jjcd9knLDBuZs=; b=WwdrE/mODHlMOqvqp//OpafRzidsU7OyeWq8tbrwovGvrXVU8D+u5ZeSBn0CjoYknu vzOGmnbJxpjqGbIIa75m7fPrARFbJP8/RKQcGRM0y6e0FWCEVa3mwih9nUuW1ApEisU/ qgRtYmJFhsWsIgRFEQ57D1ahxpg8T1QG6qeNJ01j6xS+Ut/vWjqi5K6R0niqNZhyQxbi tll4SWiyWN7v5EEcn2tM8OeUUc7zRXF+Izxkdb1Ui/OaIY0RXpqJl22rP9vf+3Cnzh5z 09ZZBxhnmJNTmJ7g3rt9w+So9x8kslgds85YP09Y68WDP6KCxuSzH78W6pWcPaEMTf1k H35g== X-Gm-Message-State: AGRZ1gIRPiyvf8JQtqovN6blvyl+aprMeq8zQBr2x5q5GwnjFpVguXa/ 3uNYDfRxAE/SGhkvn/q4ugQluw1mm+c= X-Google-Smtp-Source: AJdET5eCx7AhSy9bbTbdkDeGKAuEzJuv/xP6P2W5QuBew1cKVeIXK12Ey+roVyjzrNnB6aZe6W2Nbw== X-Received: by 2002:a17:902:6a4:: with SMTP id 33-v6mr16756275plh.268.1542491043885; Sat, 17 Nov 2018 13:44:03 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:03 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 2/7] block: have ->poll_fn() return number of entries polled Date: Sat, 17 Nov 2018 14:43:49 -0700 Message-Id: <20181117214354.822-3-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We currently only really support sync poll, ie poll with 1 IO in flight. This prepares us for supporting async poll. Note that the returned value isn't necessarily 100% accurate. If poll races with IRQ completion, we assume that the fact that the task is now runnable means we found at least one entry. In reality it could be more than 1, or not even 1. This is fine, the caller will just need to take this into account. Signed-off-by: Jens Axboe Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 18 +++++++++--------- drivers/nvme/host/multipath.c | 4 ++-- include/linux/blkdev.h | 2 +- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 7fc4abb4cc36..52b1c97cd7c6 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -38,7 +38,7 @@ #include "blk-mq-sched.h" #include "blk-rq-qos.h" -static bool blk_mq_poll(struct request_queue *q, blk_qc_t cookie); +static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie); static void blk_mq_poll_stats_start(struct request_queue *q); static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb); @@ -3305,7 +3305,7 @@ static bool blk_mq_poll_hybrid_sleep(struct request_queue *q, return true; } -static bool __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) +static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) { struct request_queue *q = hctx->queue; long state; @@ -3318,7 +3318,7 @@ static bool __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) * straight to the busy poll loop. */ if (blk_mq_poll_hybrid_sleep(q, hctx, rq)) - return true; + return 1; hctx->poll_considered++; @@ -3332,30 +3332,30 @@ static bool __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) if (ret > 0) { hctx->poll_success++; __set_current_state(TASK_RUNNING); - return true; + return ret; } if (signal_pending_state(state, current)) __set_current_state(TASK_RUNNING); if (current->state == TASK_RUNNING) - return true; + return 1; if (ret < 0) break; cpu_relax(); } __set_current_state(TASK_RUNNING); - return false; + return 0; } -static bool blk_mq_poll(struct request_queue *q, blk_qc_t cookie) +static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie) { struct blk_mq_hw_ctx *hctx; struct request *rq; if (!test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) - return false; + return 0; hctx = q->queue_hw_ctx[blk_qc_t_to_queue_num(cookie)]; if (!blk_qc_t_is_internal(cookie)) @@ -3369,7 +3369,7 @@ static bool blk_mq_poll(struct request_queue *q, blk_qc_t cookie) * so we should be safe with just the NULL check. */ if (!rq) - return false; + return 0; } return __blk_mq_poll(hctx, rq); diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index b82b0d3ca39a..65539c8df11d 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -220,11 +220,11 @@ static blk_qc_t nvme_ns_head_make_request(struct request_queue *q, return ret; } -static bool nvme_ns_head_poll(struct request_queue *q, blk_qc_t qc) +static int nvme_ns_head_poll(struct request_queue *q, blk_qc_t qc) { struct nvme_ns_head *head = q->queuedata; struct nvme_ns *ns; - bool found = false; + int found = 0; int srcu_idx; srcu_idx = srcu_read_lock(&head->srcu); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 1ad6eafc43f2..e97c0a3b2262 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -283,7 +283,7 @@ static inline unsigned short req_get_ioprio(struct request *req) struct blk_queue_ctx; typedef blk_qc_t (make_request_fn) (struct request_queue *q, struct bio *bio); -typedef bool (poll_q_fn) (struct request_queue *q, blk_qc_t); +typedef int (poll_q_fn) (struct request_queue *q, blk_qc_t); struct bio_vec; typedef int (dma_drain_needed_fn)(struct request *); From patchwork Sat Nov 17 21:43:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4546213BF for ; Sat, 17 Nov 2018 21:44:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 362F02A4E6 for ; Sat, 17 Nov 2018 21:44:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2A6912A4FF; Sat, 17 Nov 2018 21:44:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA4C82A4E6 for ; Sat, 17 Nov 2018 21:44:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726142AbeKRICK (ORCPT ); Sun, 18 Nov 2018 03:02:10 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:34268 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725743AbeKRICJ (ORCPT ); Sun, 18 Nov 2018 03:02:09 -0500 Received: by mail-pg1-f196.google.com with SMTP id 17so11857862pgg.1 for ; Sat, 17 Nov 2018 13:44:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=H55bb8wp7wkS/nESuGn9Ie3eERQNTXp7uDD13Q7fiM4=; b=IEBDpUs3ZNrlI+QMxbKKjqF5obyBGhC6FICFH0YxTzbCDY2blqDM8QzbN9ClXwknAf xTB3H+wHHfLBi3yvs0Dla8zNDTOwTQF2HebNcN8RYnnGZiIqM03rTv8cvl4EIyHkuBZC Ad48j9lTCfIrZzOVXrLueB2yhbAL4ES/zgGeoleywE/oboBgoLCyx3hYLuQACTp89aZp zAP8vOthvXk/oKiMShcxLMY4RxcxFjFfRFRiz7yfM/m4IqZCz3EF07Kg22nd1ZEEU66U 3aYvUINUJ0RUkXstFfhCFFF5TTDqBCUi4e2v6tupiWMvPIUCA9TSHqYW1KRs069ZrwS7 jn3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=H55bb8wp7wkS/nESuGn9Ie3eERQNTXp7uDD13Q7fiM4=; b=ViHyWbZdvtL6D+wFZQJRurn00n76Qp2QwXdwiyjkszBYK7FTaVQfbLkPTcWVGJX2u1 hkeGnHHBdWAnJ42IwFaHyGdhCCEH7vZ0F4HZKcCJ/bnRoRGDPSBWBvlaMAoEwn7Qrv3e jKrZykbVPZs0gvhXqr+/crIPbA0Ad3aYbSkpxm+7EyjrhqKypSdE03fbVBWogRs1oOkf ctsaQNAX1Ms2AG3j7DccrV6KZKFJh6X+6GmI+SG5pAfcNW1W63KVoqZsqQNE4QNJjmWJ zTDtvZp7hNr8Wi9ySWR4HsAm0yH5l1XlGHB4wPXhEtIlFTOJJJVvOTahkUmciQEls3wW lUYw== X-Gm-Message-State: AA+aEWa16+amlVDiUJK+LxcBc2iEPae9YKCciJw9RPruiFFaM82ewW0/ P8oOlOJwVwTvtt/7vAZqz+TeM1schOc= X-Google-Smtp-Source: AJdET5cRg8RS/2biv1j/tjlkYMhCvUHjAFZbZfddtB3ivj8B+ABFPwQmIMFN43yZl1LlhP0mNldEXw== X-Received: by 2002:a62:9657:: with SMTP id c84mr3040385pfe.77.1542491045738; Sat, 17 Nov 2018 13:44:05 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:04 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 3/7] nvme-fc: remove unused poll implementation Date: Sat, 17 Nov 2018 14:43:50 -0700 Message-Id: <20181117214354.822-4-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This relies on the fc taget ops setting ->poll_queue, which nobody does. Otherwise it just checks if something has completed, which isn't very useful. Signed-off-by: Jens Axboe Reviewed-by: Christoph Hellwig Reviewed-by:  James Smart --- drivers/nvme/host/fc.c | 33 --------------------------------- 1 file changed, 33 deletions(-) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 98c3c77f48f6..de797c641265 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -2302,38 +2302,6 @@ nvme_fc_queue_rq(struct blk_mq_hw_ctx *hctx, return nvme_fc_start_fcp_op(ctrl, queue, op, data_len, io_dir); } -static struct blk_mq_tags * -nvme_fc_tagset(struct nvme_fc_queue *queue) -{ - if (queue->qnum == 0) - return queue->ctrl->admin_tag_set.tags[queue->qnum]; - - return queue->ctrl->tag_set.tags[queue->qnum - 1]; -} - -static int -nvme_fc_poll(struct blk_mq_hw_ctx *hctx, unsigned int tag) - -{ - struct nvme_fc_queue *queue = hctx->driver_data; - struct nvme_fc_ctrl *ctrl = queue->ctrl; - struct request *req; - struct nvme_fc_fcp_op *op; - - req = blk_mq_tag_to_rq(nvme_fc_tagset(queue), tag); - if (!req) - return 0; - - op = blk_mq_rq_to_pdu(req); - - if ((atomic_read(&op->state) == FCPOP_STATE_ACTIVE) && - (ctrl->lport->ops->poll_queue)) - ctrl->lport->ops->poll_queue(&ctrl->lport->localport, - queue->lldd_handle); - - return ((atomic_read(&op->state) != FCPOP_STATE_ACTIVE)); -} - static void nvme_fc_submit_async_event(struct nvme_ctrl *arg) { @@ -2404,7 +2372,6 @@ static const struct blk_mq_ops nvme_fc_mq_ops = { .init_request = nvme_fc_init_request, .exit_request = nvme_fc_exit_request, .init_hctx = nvme_fc_init_hctx, - .poll = nvme_fc_poll, .timeout = nvme_fc_timeout, }; From patchwork Sat Nov 17 21:43:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 751F414BD for ; Sat, 17 Nov 2018 21:44:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65CAE2A4E6 for ; Sat, 17 Nov 2018 21:44:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59EE22A4FF; Sat, 17 Nov 2018 21:44:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A5FE72A4E6 for ; Sat, 17 Nov 2018 21:44:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726177AbeKRICM (ORCPT ); Sun, 18 Nov 2018 03:02:12 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:36002 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725743AbeKRICL (ORCPT ); Sun, 18 Nov 2018 03:02:11 -0500 Received: by mail-pl1-f196.google.com with SMTP id y6-v6so5203450plt.3 for ; Sat, 17 Nov 2018 13:44:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QofLUS/7R4mLCtcen4lbUXt1JioAqr/1O2ft1UwoKL8=; b=gWTKHee69yAKFKHdwCgskn9kLzd7ij4QBwsY1+iYWFoI+PKMoE7MwdBCsGiZJGqHiy QBnRrGAg81KEnt0CQ856o7un3XDcEOKgtCTQlE8b+O1w+bXbmzEc6KbI4C+D/ELg+wj5 MLpaJs6tSjmDV6XJt3ezyJRYwizqOBLOlFFz3uqbATpSIz8HLSa3Ez+fEbgRRF+E7Fq0 2ZhP4WqU6ut3YbQtbz07c6sQJcyKpRBVzJS0arRqZEVaZT5PV6oTM2AY+zXCrNVt5Goo DG9Xl0Z6Cow3rytu6GYV21UU+mLdW35c2yNi/h+2XW3196gZPappO3cr10tz8xF2yyKg 9DXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QofLUS/7R4mLCtcen4lbUXt1JioAqr/1O2ft1UwoKL8=; b=eEyI1yRVPFUVvtLMfSF9qqjmglGbAuHuUXwKhCbpK9CtVXt9Rjmb5bCgBJTnuejoeQ Spn3PuJsgxfiZkQ4IV0WjHKTyaH8VCKg23ri3J7DXcP7vY72UnZtSYiHuDHMLnn9TIcn Q1RCy6G0l2byaOcMDmOFxWH5gRQ0wSJPRAQ8JW0WpdiDN/wYVlyKA0D15qVPBXqpuk5b ggd3sWxU5MIYKeTplqg3BMMjs8yuym4hQ2Ufx9KqlNgdKjftpi4LMMxYrDhZt0s8EPym Coboqgji8IUTKflL2R2TMdmsm7bQG/TkRDfpmpDsmJSZt6++uVOUz/OnpTZM4/uZEZBL HE9Q== X-Gm-Message-State: AGRZ1gKdiEDWZWxkCkkIQm1bYZwYu3wLCdQT8EZL8jCIPEgbi2iOorB4 G7xKr+XCe9kH+M5Xhb2HrW7gXbxbHUo= X-Google-Smtp-Source: AJdET5cdxc/QtCn4WQnpn7Z4OFch24fWP06mQ12YqLsCv1a79m4Le+F/+Tr2Q+f/vmdKbhKMi6ZFBA== X-Received: by 2002:a17:902:5a03:: with SMTP id q3-v6mr16632093pli.186.1542491047558; Sat, 17 Nov 2018 13:44:07 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:06 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 4/7] blk-mq: when polling for IO, look for any completion Date: Sat, 17 Nov 2018 14:43:51 -0700 Message-Id: <20181117214354.822-5-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we want to support async IO polling, then we have to allow finding completions that aren't just for the one we are looking for. Always pass in -1 to the mq_ops->poll() helper, and have that return how many events were found in this poll loop. Signed-off-by: Jens Axboe --- block/blk-mq.c | 69 +++++++++++++++++++++++----------------- drivers/nvme/host/pci.c | 14 ++++---- drivers/nvme/host/rdma.c | 36 ++++++++++----------- 3 files changed, 62 insertions(+), 57 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 52b1c97cd7c6..3ca00d712158 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3266,9 +3266,7 @@ static bool blk_mq_poll_hybrid_sleep(struct request_queue *q, * 0: use half of prev avg * >0: use this specific value */ - if (q->poll_nsec == -1) - return false; - else if (q->poll_nsec > 0) + if (q->poll_nsec > 0) nsecs = q->poll_nsec; else nsecs = blk_mq_poll_nsecs(q, hctx, rq); @@ -3305,21 +3303,36 @@ static bool blk_mq_poll_hybrid_sleep(struct request_queue *q, return true; } -static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) +static bool blk_mq_poll_hybrid(struct request_queue *q, + struct blk_mq_hw_ctx *hctx, blk_qc_t cookie) +{ + struct request *rq; + + if (q->poll_nsec == -1) + return false; + + if (!blk_qc_t_is_internal(cookie)) + rq = blk_mq_tag_to_rq(hctx->tags, blk_qc_t_to_tag(cookie)); + else { + rq = blk_mq_tag_to_rq(hctx->sched_tags, blk_qc_t_to_tag(cookie)); + /* + * With scheduling, if the request has completed, we'll + * get a NULL return here, as we clear the sched tag when + * that happens. The request still remains valid, like always, + * so we should be safe with just the NULL check. + */ + if (!rq) + return false; + } + + return blk_mq_poll_hybrid_sleep(q, hctx, rq); +} + +static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx) { struct request_queue *q = hctx->queue; long state; - /* - * If we sleep, have the caller restart the poll loop to reset - * the state. Like for the other success return cases, the - * caller is responsible for checking if the IO completed. If - * the IO isn't complete, we'll get called again and will go - * straight to the busy poll loop. - */ - if (blk_mq_poll_hybrid_sleep(q, hctx, rq)) - return 1; - hctx->poll_considered++; state = current->state; @@ -3328,7 +3341,7 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) hctx->poll_invoked++; - ret = q->mq_ops->poll(hctx, rq->tag); + ret = q->mq_ops->poll(hctx, -1U); if (ret > 0) { hctx->poll_success++; __set_current_state(TASK_RUNNING); @@ -3352,27 +3365,23 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, struct request *rq) static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie) { struct blk_mq_hw_ctx *hctx; - struct request *rq; if (!test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) return 0; hctx = q->queue_hw_ctx[blk_qc_t_to_queue_num(cookie)]; - if (!blk_qc_t_is_internal(cookie)) - rq = blk_mq_tag_to_rq(hctx->tags, blk_qc_t_to_tag(cookie)); - else { - rq = blk_mq_tag_to_rq(hctx->sched_tags, blk_qc_t_to_tag(cookie)); - /* - * With scheduling, if the request has completed, we'll - * get a NULL return here, as we clear the sched tag when - * that happens. The request still remains valid, like always, - * so we should be safe with just the NULL check. - */ - if (!rq) - return 0; - } - return __blk_mq_poll(hctx, rq); + /* + * If we sleep, have the caller restart the poll loop to reset + * the state. Like for the other success return cases, the + * caller is responsible for checking if the IO completed. If + * the IO isn't complete, we'll get called again and will go + * straight to the busy poll loop. + */ + if (blk_mq_poll_hybrid(q, hctx, cookie)) + return 1; + + return __blk_mq_poll(hctx); } unsigned int blk_mq_rq_cpu(struct request *rq) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 89874e23e422..1742c8ab8196 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1012,15 +1012,15 @@ static inline void nvme_update_cq_head(struct nvme_queue *nvmeq) } } -static inline bool nvme_process_cq(struct nvme_queue *nvmeq, u16 *start, - u16 *end, int tag) +static inline int nvme_process_cq(struct nvme_queue *nvmeq, u16 *start, + u16 *end, unsigned int tag) { - bool found = false; + int found = 0; *start = nvmeq->cq_head; - while (!found && nvme_cqe_pending(nvmeq)) { - if (nvmeq->cqes[nvmeq->cq_head].command_id == tag) - found = true; + while (nvme_cqe_pending(nvmeq)) { + if (tag == -1U || nvmeq->cqes[nvmeq->cq_head].command_id == tag) + found++; nvme_update_cq_head(nvmeq); } *end = nvmeq->cq_head; @@ -1062,7 +1062,7 @@ static irqreturn_t nvme_irq_check(int irq, void *data) static int __nvme_poll(struct nvme_queue *nvmeq, unsigned int tag) { u16 start, end; - bool found; + int found; if (!nvme_cqe_pending(nvmeq)) return 0; diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index d181cafedc58..53e44efc6d32 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -1409,12 +1409,11 @@ static void nvme_rdma_submit_async_event(struct nvme_ctrl *arg) WARN_ON_ONCE(ret); } -static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue, - struct nvme_completion *cqe, struct ib_wc *wc, int tag) +static void nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue, + struct nvme_completion *cqe, struct ib_wc *wc) { struct request *rq; struct nvme_rdma_request *req; - int ret = 0; rq = blk_mq_tag_to_rq(nvme_rdma_tagset(queue), cqe->command_id); if (!rq) { @@ -1422,7 +1421,7 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue, "tag 0x%x on QP %#x not found\n", cqe->command_id, queue->qp->qp_num); nvme_rdma_error_recovery(queue->ctrl); - return ret; + return; } req = blk_mq_rq_to_pdu(rq); @@ -1437,6 +1436,8 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue, nvme_rdma_error_recovery(queue->ctrl); } } else if (req->mr) { + int ret; + ret = nvme_rdma_inv_rkey(queue, req); if (unlikely(ret < 0)) { dev_err(queue->ctrl->ctrl.device, @@ -1445,19 +1446,14 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue, nvme_rdma_error_recovery(queue->ctrl); } /* the local invalidation completion will end the request */ - return 0; + return; } - if (refcount_dec_and_test(&req->ref)) { - if (rq->tag == tag) - ret = 1; + if (refcount_dec_and_test(&req->ref)) nvme_end_request(rq, req->status, req->result); - } - - return ret; } -static int __nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc, int tag) +static void __nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc) { struct nvme_rdma_qe *qe = container_of(wc->wr_cqe, struct nvme_rdma_qe, cqe); @@ -1465,11 +1461,10 @@ static int __nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc, int tag) struct ib_device *ibdev = queue->device->dev; struct nvme_completion *cqe = qe->data; const size_t len = sizeof(struct nvme_completion); - int ret = 0; if (unlikely(wc->status != IB_WC_SUCCESS)) { nvme_rdma_wr_error(cq, wc, "RECV"); - return 0; + return; } ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE); @@ -1484,16 +1479,15 @@ static int __nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc, int tag) nvme_complete_async_event(&queue->ctrl->ctrl, cqe->status, &cqe->result); else - ret = nvme_rdma_process_nvme_rsp(queue, cqe, wc, tag); + nvme_rdma_process_nvme_rsp(queue, cqe, wc); ib_dma_sync_single_for_device(ibdev, qe->dma, len, DMA_FROM_DEVICE); nvme_rdma_post_recv(queue, qe); - return ret; } static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc) { - __nvme_rdma_recv_done(cq, wc, -1); + __nvme_rdma_recv_done(cq, wc); } static int nvme_rdma_conn_established(struct nvme_rdma_queue *queue) @@ -1758,10 +1752,12 @@ static int nvme_rdma_poll(struct blk_mq_hw_ctx *hctx, unsigned int tag) struct ib_cqe *cqe = wc.wr_cqe; if (cqe) { - if (cqe->done == nvme_rdma_recv_done) - found |= __nvme_rdma_recv_done(cq, &wc, tag); - else + if (cqe->done == nvme_rdma_recv_done) { + __nvme_rdma_recv_done(cq, &wc); + found++; + } else { cqe->done(cq, &wc); + } } } From patchwork Sat Nov 17 21:43:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5171714BD for ; Sat, 17 Nov 2018 21:44:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 416AF2A4E6 for ; Sat, 17 Nov 2018 21:44:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 359F22A4FF; Sat, 17 Nov 2018 21:44:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C564A2A4E6 for ; Sat, 17 Nov 2018 21:44:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726366AbeKRICO (ORCPT ); Sun, 18 Nov 2018 03:02:14 -0500 Received: from mail-pg1-f194.google.com ([209.85.215.194]:34270 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725743AbeKRICN (ORCPT ); Sun, 18 Nov 2018 03:02:13 -0500 Received: by mail-pg1-f194.google.com with SMTP id 17so11857902pgg.1 for ; Sat, 17 Nov 2018 13:44:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9NsRKs6YTwkhIZOzQWW1FUqIkBWvxAV2VRe60tl9dWk=; b=uS8p11PIgTsBZ2OVoHfntotuImPHHZev20fFS6DyS4ZdN4ObrkpTji4rj8yuLKz4dk lW+oXBxTJa2K6xB6ACw3rfyWRxrSXoyz9eL0ZLCrlrgkZuf5slEBV+BEkeTz8EoF3kuC 8oYPvzoYFrMIYNsczI3x50LtdsSgwD/eqAQOe7rj6g76Gs+0cLHY8vkyb0iWJrXb8mNq 6kjjo22E+c9/qqBDqMbRSL1lFQh5puK96/2fT0hXCLfSwt37fSBS1cGSPodKowuMwwaU 11Lh3LcYOw/cuMCf2FpUoNHmMgxBgwrK8RHvswzS2ZHIAG8BDqPipz3ixMg5f6OaOMk1 2czg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9NsRKs6YTwkhIZOzQWW1FUqIkBWvxAV2VRe60tl9dWk=; b=Z+zQ90vIJvXiA4Ci5gy/yyQEGCLYR4YYFw7kBvEJ1wpwtZlwn4VDTdPDXKrg/c3W+j YjHP/ZKq3Sd6VEDkprtjcqdd6xbmvD1pGt8Up6k1niXzIBaBWrEp+HGMCUOaoTJIN3u/ maIAuteReD+zBFqvrcgh8PlDEe60l+I3UhJrm/vdlwBcUXVU3tHXAmp/6kiqcLJxXNZz CF9YUeOWV3aHM4vR43DOvP40oItVVz4QkcH+fO+Wd0EBjryL18gXK3jfLWmUwtDXJ3Mj iiUtq3iRzl5uwUwh0MKJ2S+I7K4GcayqlXQbfw7K+OXN/yCP+NEcPpcMUxwKRjJ9wRu+ HUPA== X-Gm-Message-State: AGRZ1gLfd/ZomxiI2n3HMybq2EjJbGki+ish4nluhaHDjo2tpl+NkJTT BSt6g6aGidNNdXdjai52ZUoNHuWISQE= X-Google-Smtp-Source: AJdET5frkBkXwysmnMi2dsF1C4K2bvFa3P6ONawyFXRbnF2DbCPdnJKfLENYbS1MbxntqmsvF8sZTQ== X-Received: by 2002:a63:c141:: with SMTP id p1mr14802826pgi.424.1542491049457; Sat, 17 Nov 2018 13:44:09 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:08 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 5/7] blk-mq: remove 'tag' parameter from mq_ops->poll() Date: Sat, 17 Nov 2018 14:43:52 -0700 Message-Id: <20181117214354.822-6-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We always pass in -1 now and none of the callers use the tag value, remove the parameter. Signed-off-by: Jens Axboe Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 2 +- drivers/nvme/host/pci.c | 8 ++++---- drivers/nvme/host/rdma.c | 2 +- include/linux/blk-mq.h | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 3ca00d712158..10a14baa04ff 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3341,7 +3341,7 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx) hctx->poll_invoked++; - ret = q->mq_ops->poll(hctx, -1U); + ret = q->mq_ops->poll(hctx); if (ret > 0) { hctx->poll_success++; __set_current_state(TASK_RUNNING); diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 1742c8ab8196..1280cc327096 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1075,14 +1075,14 @@ static int __nvme_poll(struct nvme_queue *nvmeq, unsigned int tag) return found; } -static int nvme_poll(struct blk_mq_hw_ctx *hctx, unsigned int tag) +static int nvme_poll(struct blk_mq_hw_ctx *hctx) { struct nvme_queue *nvmeq = hctx->driver_data; - return __nvme_poll(nvmeq, tag); + return __nvme_poll(nvmeq, -1); } -static int nvme_poll_noirq(struct blk_mq_hw_ctx *hctx, unsigned int tag) +static int nvme_poll_noirq(struct blk_mq_hw_ctx *hctx) { struct nvme_queue *nvmeq = hctx->driver_data; u16 start, end; @@ -1092,7 +1092,7 @@ static int nvme_poll_noirq(struct blk_mq_hw_ctx *hctx, unsigned int tag) return 0; spin_lock(&nvmeq->cq_lock); - found = nvme_process_cq(nvmeq, &start, &end, tag); + found = nvme_process_cq(nvmeq, &start, &end, -1); spin_unlock(&nvmeq->cq_lock); nvme_complete_cqes(nvmeq, start, end); diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 53e44efc6d32..4b2ba563acdb 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -1741,7 +1741,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx, return BLK_STS_IOERR; } -static int nvme_rdma_poll(struct blk_mq_hw_ctx *hctx, unsigned int tag) +static int nvme_rdma_poll(struct blk_mq_hw_ctx *hctx) { struct nvme_rdma_queue *queue = hctx->driver_data; struct ib_cq *cq = queue->ib_cq; diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 929e8abc5535..ca0520ca6437 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -132,7 +132,7 @@ typedef void (exit_request_fn)(struct blk_mq_tag_set *set, struct request *, typedef bool (busy_iter_fn)(struct blk_mq_hw_ctx *, struct request *, void *, bool); typedef bool (busy_tag_iter_fn)(struct request *, void *, bool); -typedef int (poll_fn)(struct blk_mq_hw_ctx *, unsigned int); +typedef int (poll_fn)(struct blk_mq_hw_ctx *); typedef int (map_queues_fn)(struct blk_mq_tag_set *set); typedef bool (busy_fn)(struct request_queue *); typedef void (complete_fn)(struct request *); From patchwork Sat Nov 17 21:43:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60A6D13BF for ; Sat, 17 Nov 2018 21:44:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50D072A4E6 for ; Sat, 17 Nov 2018 21:44:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 44A9F2A4FF; Sat, 17 Nov 2018 21:44:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D2732A4E6 for ; Sat, 17 Nov 2018 21:44:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726442AbeKRICP (ORCPT ); Sun, 18 Nov 2018 03:02:15 -0500 Received: from mail-pg1-f194.google.com ([209.85.215.194]:36128 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725743AbeKRICP (ORCPT ); Sun, 18 Nov 2018 03:02:15 -0500 Received: by mail-pg1-f194.google.com with SMTP id n2so4943390pgm.3 for ; Sat, 17 Nov 2018 13:44:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2NKbqnhR7MBhe3pEkuvFMU/yC7rDgMlxN/r5M6eW954=; b=dxjTf4TOJkGE3CaNtoTDTb5T1lMGBT8IpY7dT/X4CjJMejqHrzi3NaG2XA4QJAPP2n xfFBDJEjhzV22RIzT7JaVa0zsY7j4I6usTd6b7a2+Ktc7WDaBSPA+Y2mdNmh0kjCuZV6 BjTMwMLlk3PabqkpK0cI1oIfbMjIXjMqG2Q9Xmv3ulv4zl2B/4zjGI/+4YFhps18WXdY K2g4pk5VPVdhpkHf4SWlK718D1t8K3Gv/xb3pMi22gR577yhXqUIeuElUYSzITbdaHvJ 81paEf6UmoH+5IR9gymNPIueCK5ALejVJ+ruOiQq1bSXUeScdywnupmiU42oyrKF2JkP lmPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2NKbqnhR7MBhe3pEkuvFMU/yC7rDgMlxN/r5M6eW954=; b=fZ+1//q18gykjCAh2X5LM6K1pxMz8XNlpWNpcaGyVf8q9kC9MzcKByvmTLynKnS/jM Zi8bGoEGdd9JJKQsxPSHtaeL9oiIrEAIUP+ThiFz4rxkaQWYPLv4XdyPoDTmfE3ru5p1 hEBHsSZ/a9BiYAzY2ztFhrhc5yJfHO5WRcAR9YZsdmedm0H5H+fLPZOcUIYnxTKA7XaK 4PUWlvw9bBALLPzyWAdUqN9jARHKrbN+tTs/ggtGLtrcxAQzIpe35F63Z+x5o4RaaauW xmkM/TgWhRRz1AxIghwL3RCrzeggAwz0dTjcQ8PDv7petUbF/N4y6xARpBGBMKoRFOQV wgIg== X-Gm-Message-State: AGRZ1gJII5IQhwbfhrApxxrzzbe2ymhlk3JKrMZCf+L2y84aI72Y0HpQ YDtnauQFCER3skZnx36PHawNesf7kAc= X-Google-Smtp-Source: AJdET5eSZbSqX/R/H7IgpJSmSFpRUdP4Jd/Ma6dx7PyGpcvpM8U3KEJg7sSyUDQcrnrH0IBVEXXj/g== X-Received: by 2002:a63:e101:: with SMTP id z1mr14858719pgh.310.1542491051392; Sat, 17 Nov 2018 13:44:11 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:10 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 6/7] block: make blk_poll() take a parameter on whether to spin or not Date: Sat, 17 Nov 2018 14:43:53 -0700 Message-Id: <20181117214354.822-7-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP blk_poll() has always kept spinning until it found an IO. This is fine for SYNC polling, since we need to find one request we have pending, but in preparation for ASYNC polling it can be beneficial to just check if we have any entries available or not. Existing callers are converted to pass in 'spin == true', to retain the old behavior. Signed-off-by: Jens Axboe --- block/blk-core.c | 4 ++-- block/blk-mq.c | 10 +++++----- drivers/nvme/host/multipath.c | 4 ++-- drivers/nvme/target/io-cmd-bdev.c | 2 +- fs/block_dev.c | 4 ++-- fs/direct-io.c | 2 +- fs/iomap.c | 2 +- include/linux/blkdev.h | 4 ++-- mm/page_io.c | 2 +- 9 files changed, 17 insertions(+), 17 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 0b684a520a11..ccf40f853afd 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1284,14 +1284,14 @@ blk_qc_t submit_bio(struct bio *bio) } EXPORT_SYMBOL(submit_bio); -bool blk_poll(struct request_queue *q, blk_qc_t cookie) +bool blk_poll(struct request_queue *q, blk_qc_t cookie, bool spin) { if (!q->poll_fn || !blk_qc_t_valid(cookie)) return false; if (current->plug) blk_flush_plug_list(current->plug, false); - return q->poll_fn(q, cookie); + return q->poll_fn(q, cookie, spin); } EXPORT_SYMBOL_GPL(blk_poll); diff --git a/block/blk-mq.c b/block/blk-mq.c index 10a14baa04ff..0a2847d9248b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -38,7 +38,7 @@ #include "blk-mq-sched.h" #include "blk-rq-qos.h" -static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie); +static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie, bool spin); static void blk_mq_poll_stats_start(struct request_queue *q); static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb); @@ -3328,7 +3328,7 @@ static bool blk_mq_poll_hybrid(struct request_queue *q, return blk_mq_poll_hybrid_sleep(q, hctx, rq); } -static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx) +static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, bool spin) { struct request_queue *q = hctx->queue; long state; @@ -3353,7 +3353,7 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx) if (current->state == TASK_RUNNING) return 1; - if (ret < 0) + if (ret < 0 || !spin) break; cpu_relax(); } @@ -3362,7 +3362,7 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx) return 0; } -static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie) +static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie, bool spin) { struct blk_mq_hw_ctx *hctx; @@ -3381,7 +3381,7 @@ static int blk_mq_poll(struct request_queue *q, blk_qc_t cookie) if (blk_mq_poll_hybrid(q, hctx, cookie)) return 1; - return __blk_mq_poll(hctx); + return __blk_mq_poll(hctx, spin); } unsigned int blk_mq_rq_cpu(struct request *rq) diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 65539c8df11d..c83bb3302684 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -220,7 +220,7 @@ static blk_qc_t nvme_ns_head_make_request(struct request_queue *q, return ret; } -static int nvme_ns_head_poll(struct request_queue *q, blk_qc_t qc) +static int nvme_ns_head_poll(struct request_queue *q, blk_qc_t qc, bool spin) { struct nvme_ns_head *head = q->queuedata; struct nvme_ns *ns; @@ -230,7 +230,7 @@ static int nvme_ns_head_poll(struct request_queue *q, blk_qc_t qc) srcu_idx = srcu_read_lock(&head->srcu); ns = srcu_dereference(head->current_path[numa_node_id()], &head->srcu); if (likely(ns && nvme_path_is_optimized(ns))) - found = ns->queue->poll_fn(q, qc); + found = ns->queue->poll_fn(q, qc, spin); srcu_read_unlock(&head->srcu, srcu_idx); return found; } diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c index c1ec3475a140..f6971b45bc54 100644 --- a/drivers/nvme/target/io-cmd-bdev.c +++ b/drivers/nvme/target/io-cmd-bdev.c @@ -116,7 +116,7 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req) cookie = submit_bio(bio); - blk_poll(bdev_get_queue(req->ns->bdev), cookie); + blk_poll(bdev_get_queue(req->ns->bdev), cookie, true); } static void nvmet_bdev_execute_flush(struct nvmet_req *req) diff --git a/fs/block_dev.c b/fs/block_dev.c index 64ba27b8b754..d233a59ea364 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -243,7 +243,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter, break; if (!(iocb->ki_flags & IOCB_HIPRI) || - !blk_poll(bdev_get_queue(bdev), qc)) + !blk_poll(bdev_get_queue(bdev), qc, true)) io_schedule(); } __set_current_state(TASK_RUNNING); @@ -423,7 +423,7 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, int nr_pages) break; if (!(iocb->ki_flags & IOCB_HIPRI) || - !blk_poll(bdev_get_queue(bdev), qc)) + !blk_poll(bdev_get_queue(bdev), qc, true)) io_schedule(); } __set_current_state(TASK_RUNNING); diff --git a/fs/direct-io.c b/fs/direct-io.c index ea07d5a34317..a5a4e5a1423e 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -518,7 +518,7 @@ static struct bio *dio_await_one(struct dio *dio) dio->waiter = current; spin_unlock_irqrestore(&dio->bio_lock, flags); if (!(dio->iocb->ki_flags & IOCB_HIPRI) || - !blk_poll(dio->bio_disk->queue, dio->bio_cookie)) + !blk_poll(dio->bio_disk->queue, dio->bio_cookie, true)) io_schedule(); /* wake up sets us TASK_RUNNING */ spin_lock_irqsave(&dio->bio_lock, flags); diff --git a/fs/iomap.c b/fs/iomap.c index c5df035ace6f..74c1f37f0fd6 100644 --- a/fs/iomap.c +++ b/fs/iomap.c @@ -1896,7 +1896,7 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, if (!(iocb->ki_flags & IOCB_HIPRI) || !dio->submit.last_queue || !blk_poll(dio->submit.last_queue, - dio->submit.cookie)) + dio->submit.cookie, true)) io_schedule(); } __set_current_state(TASK_RUNNING); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e97c0a3b2262..c85ea63c6406 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -283,7 +283,7 @@ static inline unsigned short req_get_ioprio(struct request *req) struct blk_queue_ctx; typedef blk_qc_t (make_request_fn) (struct request_queue *q, struct bio *bio); -typedef int (poll_q_fn) (struct request_queue *q, blk_qc_t); +typedef int (poll_q_fn) (struct request_queue *q, blk_qc_t, bool spin); struct bio_vec; typedef int (dma_drain_needed_fn)(struct request *); @@ -867,7 +867,7 @@ extern void blk_execute_rq_nowait(struct request_queue *, struct gendisk *, int blk_status_to_errno(blk_status_t status); blk_status_t errno_to_blk_status(int errno); -bool blk_poll(struct request_queue *q, blk_qc_t cookie); +bool blk_poll(struct request_queue *q, blk_qc_t cookie, bool spin); static inline struct request_queue *bdev_get_queue(struct block_device *bdev) { diff --git a/mm/page_io.c b/mm/page_io.c index a7271fa481f6..5bdfd21c1bd9 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -410,7 +410,7 @@ int swap_readpage(struct page *page, bool synchronous) if (!READ_ONCE(bio->bi_private)) break; - if (!blk_poll(disk->queue, qc)) + if (!blk_poll(disk->queue, qc, true)) break; } __set_current_state(TASK_RUNNING); From patchwork Sat Nov 17 21:43:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10687635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9432613BF for ; Sat, 17 Nov 2018 21:44:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 849812A4E6 for ; Sat, 17 Nov 2018 21:44:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 78D9F2A4FF; Sat, 17 Nov 2018 21:44:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 21DC22A4E6 for ; Sat, 17 Nov 2018 21:44:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726510AbeKRICR (ORCPT ); Sun, 18 Nov 2018 03:02:17 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:37305 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726496AbeKRICR (ORCPT ); Sun, 18 Nov 2018 03:02:17 -0500 Received: by mail-pg1-f195.google.com with SMTP id 80so12122114pge.4 for ; Sat, 17 Nov 2018 13:44:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vAyTfU2587OXyFZkad4b/5LE5tx6YC2PCTyuDpDarDo=; b=IHqA9CKL+rkio6YR5t2/L/dA622exqxyAHzTW4fYgyRbubqPJnXqdtA01+FHQVeInG fFtdruNPvZLZvPodkwq9aIdUMAIMAIrBpHM1ws5rhw19zTnoKLgv5twY2jZ9yH9qqNBx sv5RNRBIoejP+yfYNE3iSBqgA0REULMMXJGIHWRiorZcr7P/nuVOM/c1Iiw1ejjuA/z5 6dkdRTzaNtjFCg+9BUl0JE5qqCDWWjyQzZRakvfZM28ETxB5uFd+k5zUeEq76L0svo6z 0WWFWJwMlcGg4MmwbmA6gfb/OkgYLEb3r/O09RvI1xvNy13e1lWf0CbpUZELGqUXjaYw +P0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vAyTfU2587OXyFZkad4b/5LE5tx6YC2PCTyuDpDarDo=; b=MIbZR5IVVpJWB0xm/F/xqzTZ36Yp31qKzYxwKxJkqxJbkpdAi6Sn+wcPAIjoYU+HQh coRteg9QgQjzVkQaGTmR+YWon9/oEHz4apVp6Gm+1RBpWDw698u+MDnsFSevKtC+x6Ez NChmugeUZL6yTtxU2tjGvBdzUdn51UGcctmNmPFf5FOGr4AHeFMZuehSDp3Ep063xZYz 2JVKHygt4MfcgkIo0l2QFm1vSVYPiUyev4hF5fxDTFwNukvp0GgEzrBKKeTnzKrw5fxq QQ8VdqS4WgbvQeEGEHAN5y7mcbOYSmZoBm8PM49/qm3VQqrH/JGvu7x8kcCrBwwuHFm9 TWcg== X-Gm-Message-State: AGRZ1gIy6XypMJ9V9J3zYgg8syhJnSp8y6dBRxLiwG2uOwvrhtT7qKIm zxkpWPECtyXqgJ3rEafWgA1l8qv20J0= X-Google-Smtp-Source: AJdET5eayCXDC9bJDvWKGVL4LFv1fj4oML/wfBkyeyIP1uGCdfP6ka+q13skl4/LeK6KGooJaON6ew== X-Received: by 2002:a62:d2c1:: with SMTP id c184mr7705356pfg.248.1542491053181; Sat, 17 Nov 2018 13:44:13 -0800 (PST) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id o84sm29006501pfi.172.2018.11.17.13.44.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 13:44:12 -0800 (PST) From: Jens Axboe To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Jens Axboe Subject: [PATCH 7/7] blk-mq: ensure mq_ops ->poll() is entered at least once Date: Sat, 17 Nov 2018 14:43:54 -0700 Message-Id: <20181117214354.822-8-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117214354.822-1-axboe@kernel.dk> References: <20181117214354.822-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Right now we immediately bail if need_resched() is true, but we need to do at least one loop in case we have entries waiting. So just invert the need_resched() check, putting it at the bottom of the loop. Signed-off-by: Jens Axboe Reviewed-by: Christoph Hellwig --- block/blk-mq.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 0a2847d9248b..4769c975b8c8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3336,7 +3336,7 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, bool spin) hctx->poll_considered++; state = current->state; - while (!need_resched()) { + do { int ret; hctx->poll_invoked++; @@ -3356,7 +3356,7 @@ static int __blk_mq_poll(struct blk_mq_hw_ctx *hctx, bool spin) if (ret < 0 || !spin) break; cpu_relax(); - } + } while (!need_resched()); __set_current_state(TASK_RUNNING); return 0;