From patchwork Tue Dec 11 23:36:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10725299 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 277DF1869 for ; Tue, 11 Dec 2018 23:37:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 196412B4A4 for ; Tue, 11 Dec 2018 23:37:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0DF1E2B72F; Tue, 11 Dec 2018 23:37:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B11A42B56E for ; Tue, 11 Dec 2018 23:37:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726246AbeLKXhE (ORCPT ); Tue, 11 Dec 2018 18:37:04 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:37392 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726219AbeLKXhB (ORCPT ); Tue, 11 Dec 2018 18:37:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=yIB3JTVbUriwq3pCLTQwSP546N6Ac2IalBzLS97RbvI=; b=IIFkeF7J52G+4D3oTJzUQo6Yd qrtvTrqdgPyNhMUjKx5hQj+Q/zcDWuTFVWiTOou1Jp73Ef3jKwJu3l8melTfjJ84B3uFz/vEgX64q dqk/K3hZjVQ1RbYBUxZZgALQUC4p8nAKH2VrzUM/3ALAccDQrXtkhOpYN6FHuVFiBkKwpGfjgUaaA /ldok8w1JHMU3JHDrldgvmXxb6fhQE/SLqszKTzpTDM9SoZ1d4ZH+M8M+ie1slo/nOeUyi8C9/8l3 /2pKeptPPTozmk+o3kKNbpTkCCgMjld5oqbNu4irK29VfNqqIxiVW6G7NUaLJGqDizndOvit6PaWQ MXWzQjCPg==; Received: from [2600:1700:65a0:78e0:514:7862:1503:8e4d] (helo=sagi-Latitude-E7470.lbits) by bombadil.infradead.org with esmtpsa (Exim 4.90_1 #2 (Red Hat Linux)) id 1gWraD-0003uQ-AD; Tue, 11 Dec 2018 23:36:57 +0000 From: Sagi Grimberg To: linux-nvme@lists.infradead.org Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, Christoph Hellwig , Keith Busch Subject: [PATCH RFC 1/4] nvme-fabrics: allow user to pass in nr_poll_queues Date: Tue, 11 Dec 2018 15:36:48 -0800 Message-Id: <20181211233652.9705-2-sagi@grimberg.me> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181211233652.9705-1-sagi@grimberg.me> References: <20181211233652.9705-1-sagi@grimberg.me> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This argument will specify how many polling I/O queues to connect when creating the controller. These I/O queues will host I/O that is set with REQ_HIPRI. Signed-off-by: Sagi Grimberg --- drivers/nvme/host/fabrics.c | 16 +++++++++++++++- drivers/nvme/host/fabrics.h | 3 +++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c index 066c3a02e08b..f43bd1eada1c 100644 --- a/drivers/nvme/host/fabrics.c +++ b/drivers/nvme/host/fabrics.c @@ -617,6 +617,7 @@ static const match_table_t opt_tokens = { { NVMF_OPT_HDR_DIGEST, "hdr_digest" }, { NVMF_OPT_DATA_DIGEST, "data_digest" }, { NVMF_OPT_NR_WRITE_QUEUES, "nr_write_queues=%d" }, + { NVMF_OPT_NR_POLL_QUEUES, "nr_poll_queues=%d" }, { NVMF_OPT_ERR, NULL } }; @@ -850,6 +851,18 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts, } opts->nr_write_queues = token; break; + case NVMF_OPT_NR_POLL_QUEUES: + if (match_int(args, &token)) { + ret = -EINVAL; + goto out; + } + if (token <= 0) { + pr_err("Invalid nr_poll_queues %d\n", token); + ret = -EINVAL; + goto out; + } + opts->nr_poll_queues = token; + break; default: pr_warn("unknown parameter or missing value '%s' in ctrl creation request\n", p); @@ -967,7 +980,8 @@ EXPORT_SYMBOL_GPL(nvmf_free_options); #define NVMF_ALLOWED_OPTS (NVMF_OPT_QUEUE_SIZE | NVMF_OPT_NR_IO_QUEUES | \ NVMF_OPT_KATO | NVMF_OPT_HOSTNQN | \ NVMF_OPT_HOST_ID | NVMF_OPT_DUP_CONNECT |\ - NVMF_OPT_DISABLE_SQFLOW | NVMF_OPT_NR_WRITE_QUEUES) + NVMF_OPT_DISABLE_SQFLOW | NVMF_OPT_NR_WRITE_QUEUES | \ + NVMF_OPT_NR_POLL_QUEUES) static struct nvme_ctrl * nvmf_create_ctrl(struct device *dev, const char *buf, size_t count) diff --git a/drivers/nvme/host/fabrics.h b/drivers/nvme/host/fabrics.h index 81b8fd1c0c5d..36f5daf05f08 100644 --- a/drivers/nvme/host/fabrics.h +++ b/drivers/nvme/host/fabrics.h @@ -62,6 +62,7 @@ enum { NVMF_OPT_HDR_DIGEST = 1 << 15, NVMF_OPT_DATA_DIGEST = 1 << 16, NVMF_OPT_NR_WRITE_QUEUES = 1 << 17, + NVMF_OPT_NR_POLL_QUEUES = 1 << 18, }; /** @@ -93,6 +94,7 @@ enum { * @hdr_digest: generate/verify header digest (TCP) * @data_digest: generate/verify data digest (TCP) * @nr_write_queues: number of queues for write I/O + * @nr_poll_queues: number of queues for polling I/O */ struct nvmf_ctrl_options { unsigned mask; @@ -113,6 +115,7 @@ struct nvmf_ctrl_options { bool hdr_digest; bool data_digest; unsigned int nr_write_queues; + unsigned int nr_poll_queues; }; /* From patchwork Tue Dec 11 23:36:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10725293 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACB3E1869 for ; Tue, 11 Dec 2018 23:37:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D3842B4A4 for ; Tue, 11 Dec 2018 23:37:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 91A2A2B61E; Tue, 11 Dec 2018 23:37:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 188D72B56E for ; Tue, 11 Dec 2018 23:37:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726243AbeLKXhC (ORCPT ); Tue, 11 Dec 2018 18:37:02 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:37432 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726237AbeLKXhB (ORCPT ); Tue, 11 Dec 2018 18:37:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=UFF+Hh408CPpQAU18uYhYTpU0TmXqVymdZh/b0o4UH0=; b=Fdtd1pYOQ1+JdK78pwQLHz3RM DFIzozjBCI3n7NDh79wl16HB78Oi9hzAdEgEWGzlcn+AtHNfpyM+JHKHpYExM4oZnRGuffz4LdQv6 Btpzjv1lx0ZEcilp7rFxV1zggNw8wwDF95H423vrnNTTd8oBOjS7E8Eij4v0V+7hpmwalR/6rJ0o5 KozDqN8y9TxoGuyiCPZ5xJSmzuxDVfcRVRWbGt2q/gShYwL6R9Mx4c/tmA47/Q1g+OcOdZPUNgc1d j5WAB4JYKkoEGxhYuZknDcH/TbGhDG2emzR5onEv6hoMVaXuxFxoSgQ8LPNdNyAlkC83aT2ubih1C TMDdNkXfg==; Received: from [2600:1700:65a0:78e0:514:7862:1503:8e4d] (helo=sagi-Latitude-E7470.lbits) by bombadil.infradead.org with esmtpsa (Exim 4.90_1 #2 (Red Hat Linux)) id 1gWraD-0003uQ-HW; Tue, 11 Dec 2018 23:36:57 +0000 From: Sagi Grimberg To: linux-nvme@lists.infradead.org Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, Christoph Hellwig , Keith Busch Subject: [PATCH RFC 2/4] rdma: introduce ib_change_cq_ctx Date: Tue, 11 Dec 2018 15:36:49 -0800 Message-Id: <20181211233652.9705-3-sagi@grimberg.me> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181211233652.9705-1-sagi@grimberg.me> References: <20181211233652.9705-1-sagi@grimberg.me> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow cq consumers to modify the cq polling context online. The consumer might want to allocate the cq with softirq/workqueue polling context for async (setup time) I/O, and when completed, switch the polling context to direct polling and get all the interrupts out of the way. One example is nvme-rdma driver that hooks into the block layer infrastructure for a polling queue map for latency sensitive I/O. Every nvmf queue starts with a connect message that is the slow path at setup time, and there is no need for polling (it is actually hurtful). Instead, allocate the polling queue cq with IB_POLL_SOFTIRQ and switch it to IB_POLL_DIRECT where it makes sense. Signed-off-by: Sagi Grimberg --- drivers/infiniband/core/cq.c | 102 ++++++++++++++++++++++++----------- include/rdma/ib_verbs.h | 1 + 2 files changed, 71 insertions(+), 32 deletions(-) diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c index b1e5365ddafa..c820eb954edc 100644 --- a/drivers/infiniband/core/cq.c +++ b/drivers/infiniband/core/cq.c @@ -80,7 +80,7 @@ EXPORT_SYMBOL(ib_process_cq_direct); static void ib_cq_completion_direct(struct ib_cq *cq, void *private) { - WARN_ONCE(1, "got unsolicited completion for CQ 0x%p\n", cq); + pr_debug("got unsolicited completion for CQ 0x%p\n", cq); } static int ib_poll_handler(struct irq_poll *iop, int budget) @@ -120,6 +120,33 @@ static void ib_cq_completion_workqueue(struct ib_cq *cq, void *private) queue_work(cq->comp_wq, &cq->work); } +static int __ib_cq_set_ctx(struct ib_cq *cq) +{ + switch (cq->poll_ctx) { + case IB_POLL_DIRECT: + cq->comp_handler = ib_cq_completion_direct; + break; + case IB_POLL_SOFTIRQ: + cq->comp_handler = ib_cq_completion_softirq; + + irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler); + ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); + break; + case IB_POLL_WORKQUEUE: + case IB_POLL_UNBOUND_WORKQUEUE: + cq->comp_handler = ib_cq_completion_workqueue; + INIT_WORK(&cq->work, ib_cq_poll_work); + ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); + cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ? + ib_comp_wq : ib_comp_unbound_wq; + break; + default: + return -EINVAL; + } + + return 0; +} + /** * __ib_alloc_cq - allocate a completion queue * @dev: device to allocate the CQ for @@ -164,28 +191,9 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, rdma_restrack_set_task(&cq->res, caller); rdma_restrack_add(&cq->res); - switch (cq->poll_ctx) { - case IB_POLL_DIRECT: - cq->comp_handler = ib_cq_completion_direct; - break; - case IB_POLL_SOFTIRQ: - cq->comp_handler = ib_cq_completion_softirq; - - irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler); - ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); - break; - case IB_POLL_WORKQUEUE: - case IB_POLL_UNBOUND_WORKQUEUE: - cq->comp_handler = ib_cq_completion_workqueue; - INIT_WORK(&cq->work, ib_cq_poll_work); - ib_req_notify_cq(cq, IB_CQ_NEXT_COMP); - cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ? - ib_comp_wq : ib_comp_unbound_wq; - break; - default: - ret = -EINVAL; + ret = __ib_cq_set_ctx(cq); + if (ret) goto out_free_wc; - } return cq; @@ -198,17 +206,8 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, } EXPORT_SYMBOL(__ib_alloc_cq); -/** - * ib_free_cq - free a completion queue - * @cq: completion queue to free. - */ -void ib_free_cq(struct ib_cq *cq) +static void __ib_cq_clear_ctx(struct ib_cq *cq) { - int ret; - - if (WARN_ON_ONCE(atomic_read(&cq->usecnt))) - return; - switch (cq->poll_ctx) { case IB_POLL_DIRECT: break; @@ -222,6 +221,20 @@ void ib_free_cq(struct ib_cq *cq) default: WARN_ON_ONCE(1); } +} + +/** + * ib_free_cq - free a completion queue + * @cq: completion queue to free. + */ +void ib_free_cq(struct ib_cq *cq) +{ + int ret; + + if (WARN_ON_ONCE(atomic_read(&cq->usecnt))) + return; + + __ib_cq_clear_ctx(cq); kfree(cq->wc); rdma_restrack_del(&cq->res); @@ -229,3 +242,28 @@ void ib_free_cq(struct ib_cq *cq) WARN_ON_ONCE(ret); } EXPORT_SYMBOL(ib_free_cq); + +/** + * ib_change_cq_ctx - change completion queue polling context dynamically + * @cq: the completion queue + * @poll_ctx: new context to poll the CQ from + * + * The caller must make sure that there is no inflight I/O when calling + * this (otherwise its just asking for trouble). If the cq polling context + * change fails, the old polling context is restored. + */ +int ib_change_cq_ctx(struct ib_cq *cq, enum ib_poll_context poll_ctx) +{ + enum ib_poll_context old_ctx = cq->poll_ctx; + int ret; + + __ib_cq_clear_ctx(cq); + cq->poll_ctx = poll_ctx; + ret = __ib_cq_set_ctx(cq); + if (ret) { + cq->poll_ctx = old_ctx; + __ib_cq_set_ctx(cq); + } + return ret; +} +EXPORT_SYMBOL(ib_change_cq_ctx); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 9c0c2132a2d6..c9d03d3a3cd4 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -3464,6 +3464,7 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, void ib_free_cq(struct ib_cq *cq); int ib_process_cq_direct(struct ib_cq *cq, int budget); +int ib_change_cq_ctx(struct ib_cq *cq, enum ib_poll_context poll_ctx); /** * ib_create_cq - Creates a CQ on the specified device. From patchwork Tue Dec 11 23:36:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10725279 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B88EA1869 for ; Tue, 11 Dec 2018 23:37:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA30C2B4A4 for ; Tue, 11 Dec 2018 23:37:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9C9F72B61E; Tue, 11 Dec 2018 23:37:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 424BB2B56E for ; Tue, 11 Dec 2018 23:37:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726218AbeLKXg7 (ORCPT ); Tue, 11 Dec 2018 18:36:59 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:37262 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726201AbeLKXg7 (ORCPT ); Tue, 11 Dec 2018 18:36:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=I4L4IN089RqfWMWAwDyCS6XI87ETgggO9pASeHabJFM=; b=dAekHtNCMJdsV2PmZ2Cdwc4Ky 5x4U+tda6gsF87y9nXMB9ymoA/gNJLZlq4uY1whMHjGU5levnI/E+0jEWzyH5M0h9p71xP6TlLbGk xuc7OW4YiRVeenvxWxF3B1BTG5p58idQOun6ny/JftYbRUzLdSghr/eyjE2K1HZJlb03bDASrDIfP dRqImLDoYTpuNEM9POGXagtTr0f+k4Xvik3HB43tRKZcFYnoESMc8KS8bvIi9AQw+YIvUcTqvzpib w+IX0/rEVCIfiwKmgEV6T74rVT6s+fuCS9fB0w9t0zwgVi6oE4kBj1olXsUBxsWkkDz7P/eIJhRGF v5MPYm0aQ==; Received: from [2600:1700:65a0:78e0:514:7862:1503:8e4d] (helo=sagi-Latitude-E7470.lbits) by bombadil.infradead.org with esmtpsa (Exim 4.90_1 #2 (Red Hat Linux)) id 1gWraD-0003uQ-Tj; Tue, 11 Dec 2018 23:36:57 +0000 From: Sagi Grimberg To: linux-nvme@lists.infradead.org Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, Christoph Hellwig , Keith Busch Subject: [PATCH RFC 3/4] nvme-rdma: implement polling queue map Date: Tue, 11 Dec 2018 15:36:50 -0800 Message-Id: <20181211233652.9705-4-sagi@grimberg.me> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181211233652.9705-1-sagi@grimberg.me> References: <20181211233652.9705-1-sagi@grimberg.me> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Every nvmf queue starts with a connect message that is the slow path at setup time, and there is no need for polling (it is actually hurtful). Instead, allocate the polling queue cq with IB_POLL_SOFTIRQ and switch it to IB_POLL_DIRECT where it makes sense. Signed-off-by: Sagi Grimberg --- drivers/nvme/host/rdma.c | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 6a7c546b4e74..590d006d0187 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -607,6 +607,11 @@ static int nvme_rdma_start_queue(struct nvme_rdma_ctrl *ctrl, int idx) else dev_info(ctrl->ctrl.device, "failed to connect queue: %d ret=%d\n", idx, ret); + + if (idx > ctrl->ctrl.opts->nr_io_queues + + ctrl->ctrl.opts->nr_write_queues) + ib_change_cq_ctx(ctrl->queues[idx].ib_cq, IB_POLL_DIRECT); + return ret; } @@ -646,6 +651,7 @@ static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl) ibdev->num_comp_vectors); nr_io_queues += min(opts->nr_write_queues, num_online_cpus()); + nr_io_queues += min(opts->nr_poll_queues, num_online_cpus()); ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues); if (ret) @@ -716,7 +722,7 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl, set->driver_data = ctrl; set->nr_hw_queues = nctrl->queue_count - 1; set->timeout = NVME_IO_TIMEOUT; - set->nr_maps = 2 /* default + read */; + set->nr_maps = HCTX_MAX_TYPES; } ret = blk_mq_alloc_tag_set(set); @@ -1742,6 +1748,14 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx, return BLK_STS_IOERR; } +static int nvme_rdma_poll(struct blk_mq_hw_ctx *hctx) +{ + struct nvme_rdma_queue *queue = hctx->driver_data; + struct ib_cq *cq = queue->ib_cq; + + return ib_process_cq_direct(cq, 16); +} + static void nvme_rdma_complete_rq(struct request *rq) { struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq); @@ -1772,6 +1786,21 @@ static int nvme_rdma_map_queues(struct blk_mq_tag_set *set) ctrl->device->dev, 0); blk_mq_rdma_map_queues(&set->map[HCTX_TYPE_READ], ctrl->device->dev, 0); + + if (ctrl->ctrl.opts->nr_poll_queues) { + set->map[HCTX_TYPE_POLL].nr_queues = + ctrl->ctrl.opts->nr_poll_queues; + set->map[HCTX_TYPE_POLL].queue_offset = + ctrl->ctrl.opts->nr_io_queues; + if (ctrl->ctrl.opts->nr_write_queues) + set->map[HCTX_TYPE_POLL].queue_offset += + ctrl->ctrl.opts->nr_write_queues; + } else { + set->map[HCTX_TYPE_POLL].nr_queues = + ctrl->ctrl.opts->nr_io_queues; + set->map[HCTX_TYPE_POLL].queue_offset = 0; + } + blk_mq_map_queues(&set->map[HCTX_TYPE_POLL]); return 0; } @@ -1783,6 +1812,7 @@ static const struct blk_mq_ops nvme_rdma_mq_ops = { .init_hctx = nvme_rdma_init_hctx, .timeout = nvme_rdma_timeout, .map_queues = nvme_rdma_map_queues, + .poll = nvme_rdma_poll, }; static const struct blk_mq_ops nvme_rdma_admin_mq_ops = { @@ -1927,7 +1957,8 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev, INIT_WORK(&ctrl->err_work, nvme_rdma_error_recovery_work); INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work); - ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues + 1; + ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues + + opts->nr_poll_queues + 1; ctrl->ctrl.sqsize = opts->queue_size - 1; ctrl->ctrl.kato = opts->kato; From patchwork Tue Dec 11 23:36:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10725289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD1471869 for ; Tue, 11 Dec 2018 23:37:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D85F2B4A4 for ; Tue, 11 Dec 2018 23:37:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 918782B61A; Tue, 11 Dec 2018 23:37:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 466C22B56E for ; Tue, 11 Dec 2018 23:37:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726230AbeLKXhB (ORCPT ); Tue, 11 Dec 2018 18:37:01 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:37382 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726232AbeLKXhB (ORCPT ); Tue, 11 Dec 2018 18:37:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=QDX1HDPRZDcH1UGedTfNRuSknUh2xNTsqrVUS3lY3bs=; b=NYXgIPF8Gipwgq5PYBDLQeOGc 3lLkXe72BRMbzhmKjUh3tuDeaHbLze7SDceG7Bm2dESCvAmQ0u8LRBoTaHenDfP7g3K5sS/EQLuYg J7CdTG09yltoOMJ+grh7qyVb30Fm0ruokVT3j/XoAbQb2aE4TjDBwFFhS4sh1ssZwoBGp/N5XahvA F42iLMup+GE0j2cdq3wIYng8A1BhmQnURaR4AcGxPqmm1P9VJrQf0UytTA+V4+UyOjeWbOcImOiJy 4h85qDJdeCG1rRrdiac+Jl3GrcD6ODr9DhjAIBgjB/MyoO5URm7ZToR2Ka4OVAhtlNpaTqttK4pVU oT8p2hWfQ==; Received: from [2600:1700:65a0:78e0:514:7862:1503:8e4d] (helo=sagi-Latitude-E7470.lbits) by bombadil.infradead.org with esmtpsa (Exim 4.90_1 #2 (Red Hat Linux)) id 1gWraE-0003uQ-67; Tue, 11 Dec 2018 23:36:58 +0000 From: Sagi Grimberg To: linux-nvme@lists.infradead.org Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, Christoph Hellwig , Keith Busch Subject: [PATCH RFC 4/4] nvme-multipath: disable polling for underlying namespace request queue Date: Tue, 11 Dec 2018 15:36:51 -0800 Message-Id: <20181211233652.9705-5-sagi@grimberg.me> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181211233652.9705-1-sagi@grimberg.me> References: <20181211233652.9705-1-sagi@grimberg.me> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since the multipath device does not support polling (yet) we cannot pass requests to the polling queue map as those will not generate interrupt so we cannot reap the completion. Signed-off-by: Sagi Grimberg --- drivers/nvme/host/core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f90576862736..511d399a6002 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -1547,6 +1547,8 @@ static void __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id) if (ns->head->disk) { nvme_update_disk_info(ns->head->disk, ns, id); blk_queue_stack_limits(ns->head->disk->queue, ns->queue); + /* XXX: multipath device does not support polling for now... */ + blk_queue_flag_clear(QUEUE_FLAG_POLL, ns->queue); } #endif } From patchwork Tue Dec 11 23:36:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10725281 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8FB0217FE for ; Tue, 11 Dec 2018 23:37:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8064B2B4A4 for ; Tue, 11 Dec 2018 23:37:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7453D2B61E; Tue, 11 Dec 2018 23:37:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F029C2B4A4 for ; Tue, 11 Dec 2018 23:37:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726227AbeLKXhA (ORCPT ); Tue, 11 Dec 2018 18:37:00 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:37304 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726201AbeLKXhA (ORCPT ); Tue, 11 Dec 2018 18:37:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=tmCUpodaC14HxdBvhUMvpC4Z0CrR1akJvM7Cmg0Tlls=; b=rgjFxYra0uaonpX4Sxq6fv9AG 2zk83m4OFwdBCLNa2+QB9RS891ufTmRQgdE3YGacgmgk6TfeWnfJsKte/JlKLZoJsnuDF2LKMsi5g Smgf50ccRgfisKNJDEY+KyZAAQdFrp6P14i88jBk7hiPrcJ7rbAS5cUoMJQUvC8+KK0P0SqvYpVvS 0bpzweECc5esYc92OGd2ZbNH2xAwhEUaHH3rWPsB8wDoLq8+1zWLV/Qlaf7noPpAxNMgBeO1ow0nQ ott0jrNgdzXyOb+2qLg3V2+9RIcrv3z0g/6zu/EsImDVMv3z4RJ226m8oWwrZQGiZu+ZIe21KiS6x GJQI/FKmA==; Received: from [2600:1700:65a0:78e0:514:7862:1503:8e4d] (helo=sagi-Latitude-E7470.lbits) by bombadil.infradead.org with esmtpsa (Exim 4.90_1 #2 (Red Hat Linux)) id 1gWraE-0003uQ-ET; Tue, 11 Dec 2018 23:36:58 +0000 From: Sagi Grimberg To: linux-nvme@lists.infradead.org Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, Christoph Hellwig , Keith Busch Subject: [PATCH RFC nvme-cli 5/4] fabrics: pass in number of polling queues Date: Tue, 11 Dec 2018 15:36:52 -0800 Message-Id: <20181211233652.9705-6-sagi@grimberg.me> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181211233652.9705-1-sagi@grimberg.me> References: <20181211233652.9705-1-sagi@grimberg.me> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP nr_poll_queues specifies the number of additional queues that will be connected for hosting polling latency critical I/O. Signed-off-by: Sagi Grimberg --- Documentation/nvme-connect.txt | 5 +++++ fabrics.c | 11 +++++++++++ 2 files changed, 16 insertions(+) diff --git a/Documentation/nvme-connect.txt b/Documentation/nvme-connect.txt index d4a0e6678475..5412472dbd35 100644 --- a/Documentation/nvme-connect.txt +++ b/Documentation/nvme-connect.txt @@ -17,6 +17,7 @@ SYNOPSIS [--hostnqn= | -q ] [--nr-io-queues=<#> | -i <#>] [--nr-write-queues=<#> | -W <#>] + [--nr-poll-queues=<#> | -P <#>] [--queue-size=<#> | -Q <#>] [--keep-alive-tmo=<#> | -k <#>] [--reconnect-delay=<#> | -c <#>] @@ -80,6 +81,10 @@ OPTIONS --nr-write-queues=<#>:: Adds additional queues that will be used for write I/O. +-P <#>:: +--nr-poll-queues=<#>:: + Adds additional queues that will be used for polling latency sensitive I/O. + -Q <#>:: --queue-size=<#>:: Overrides the default number of elements in the I/O queues created diff --git a/fabrics.c b/fabrics.c index bc6a0b7e4e21..214fce4c1b51 100644 --- a/fabrics.c +++ b/fabrics.c @@ -54,6 +54,7 @@ static struct config { char *hostid; int nr_io_queues; int nr_write_queues; + int nr_poll_queues; int queue_size; int keep_alive_tmo; int reconnect_delay; @@ -624,6 +625,8 @@ static int build_options(char *argstr, int max_len) cfg.nr_io_queues) || add_int_argument(&argstr, &max_len, "nr_write_queues", cfg.nr_write_queues) || + add_int_argument(&argstr, &max_len, "nr_poll_queues", + cfg.nr_poll_queues) || add_int_argument(&argstr, &max_len, "queue_size", cfg.queue_size) || add_int_argument(&argstr, &max_len, "keep_alive_tmo", cfg.keep_alive_tmo) || @@ -704,6 +707,13 @@ retry: p += len; } + if (cfg.nr_poll_queues) { + len = sprintf(p, ",nr_poll_queues=%d", cfg.nr_poll_queues); + if (len < 0) + return -EINVAL; + p += len; + } + if (cfg.host_traddr) { len = sprintf(p, ",host_traddr=%s", cfg.host_traddr); if (len < 0) @@ -1020,6 +1030,7 @@ int connect(const char *desc, int argc, char **argv) {"hostid", 'I', "LIST", CFG_STRING, &cfg.hostid, required_argument, "user-defined hostid (if default not used)"}, {"nr-io-queues", 'i', "LIST", CFG_INT, &cfg.nr_io_queues, required_argument, "number of io queues to use (default is core count)" }, {"nr-write-queues", 'W', "LIST", CFG_INT, &cfg.nr_write_queues, required_argument, "number of write queues to use (default 0)" }, + {"nr-poll-queues", 'W', "LIST", CFG_INT, &cfg.nr_poll_queues, required_argument, "number of poll queues to use (default 0)" }, {"queue-size", 'Q', "LIST", CFG_INT, &cfg.queue_size, required_argument, "number of io queue elements to use (default 128)" }, {"keep-alive-tmo", 'k', "LIST", CFG_INT, &cfg.keep_alive_tmo, required_argument, "keep alive timeout period in seconds" }, {"reconnect-delay", 'c', "LIST", CFG_INT, &cfg.reconnect_delay, required_argument, "reconnect timeout period in seconds" },