From patchwork Tue Feb 12 13:04:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10807875 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA16D746 for ; Tue, 12 Feb 2019 13:05:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C741D2A2A4 for ; Tue, 12 Feb 2019 13:05:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B5F6D2B663; Tue, 12 Feb 2019 13:05:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54D592A2A4 for ; Tue, 12 Feb 2019 13:05:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729573AbfBLNE6 (ORCPT ); Tue, 12 Feb 2019 08:04:58 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53190 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727750AbfBLNE6 (ORCPT ); Tue, 12 Feb 2019 08:04:58 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C539436809; Tue, 12 Feb 2019 13:04:57 +0000 (UTC) Received: from localhost (ovpn-8-25.pek2.redhat.com [10.72.8.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC6D36019F; Tue, 12 Feb 2019 13:04:53 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Bjorn Helgaas , Thomas Gleixner Cc: Jens Axboe , linux-block@vger.kernel.org, Sagi Grimberg , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Keith Busch , Ming Lei Subject: [PATCH V2 1/4] genirq/affinity: store irq set vectors in 'struct irq_affinity' Date: Tue, 12 Feb 2019 21:04:36 +0800 Message-Id: <20190212130439.14501-2-ming.lei@redhat.com> In-Reply-To: <20190212130439.14501-1-ming.lei@redhat.com> References: <20190212130439.14501-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 12 Feb 2019 13:04:57 +0000 (UTC) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently the array of irq set vectors is provided by driver. irq_create_affinity_masks() can be simplied a bit by treating the non-irq-set case as single irq set. So move this array into 'struct irq_affinity', and pre-define the max set number as 4, which should be enough for normal cases. Signed-off-by: Ming Lei --- drivers/nvme/host/pci.c | 5 ++--- include/linux/interrupt.h | 8 +++++--- kernel/irq/affinity.c | 10 ++++++---- 3 files changed, 13 insertions(+), 10 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 022ea1ee63f8..0086bdf80ea1 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2081,12 +2081,11 @@ static void nvme_calc_io_queues(struct nvme_dev *dev, unsigned int irq_queues) static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues) { struct pci_dev *pdev = to_pci_dev(dev->dev); - int irq_sets[2]; struct irq_affinity affd = { .pre_vectors = 1, - .nr_sets = ARRAY_SIZE(irq_sets), - .sets = irq_sets, + .nr_sets = 2, }; + int *irq_sets = affd.set_vectors; int result = 0; unsigned int irq_queues, this_p_queues; diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index 7c9434652f36..a20150627a32 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -259,6 +259,8 @@ struct irq_affinity_notify { void (*release)(struct kref *ref); }; +#define IRQ_MAX_SETS 4 + /** * struct irq_affinity - Description for automatic irq affinity assignements * @pre_vectors: Don't apply affinity to @pre_vectors at beginning of @@ -266,13 +268,13 @@ struct irq_affinity_notify { * @post_vectors: Don't apply affinity to @post_vectors at end of * the MSI(-X) vector space * @nr_sets: Length of passed in *sets array - * @sets: Number of affinitized sets + * @set_vectors: Number of affinitized sets */ struct irq_affinity { int pre_vectors; int post_vectors; int nr_sets; - int *sets; + int set_vectors[IRQ_MAX_SETS]; }; /** @@ -332,7 +334,7 @@ extern int irq_set_affinity_notifier(unsigned int irq, struct irq_affinity_notify *notify); struct irq_affinity_desc * -irq_create_affinity_masks(int nvec, const struct irq_affinity *affd); +irq_create_affinity_masks(int nvec, struct irq_affinity *affd); int irq_calc_affinity_vectors(int minvec, int maxvec, const struct irq_affinity *affd); diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c index 118b66d64a53..a97b7c33d2db 100644 --- a/kernel/irq/affinity.c +++ b/kernel/irq/affinity.c @@ -239,7 +239,7 @@ static int irq_build_affinity_masks(const struct irq_affinity *affd, * Returns the irq_affinity_desc pointer or NULL if allocation failed. */ struct irq_affinity_desc * -irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) +irq_create_affinity_masks(int nvecs, struct irq_affinity *affd) { int affvecs = nvecs - affd->pre_vectors - affd->post_vectors; int curvec, usedvecs; @@ -265,11 +265,13 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) * have multiple sets, build each sets affinity mask separately. */ nr_sets = affd->nr_sets; - if (!nr_sets) + if (!nr_sets) { nr_sets = 1; + affd->set_vectors[0] = affvecs; + } for (i = 0, usedvecs = 0; i < nr_sets; i++) { - int this_vecs = affd->sets ? affd->sets[i] : affvecs; + int this_vecs = affd->set_vectors[i]; int ret; ret = irq_build_affinity_masks(affd, curvec, this_vecs, @@ -316,7 +318,7 @@ int irq_calc_affinity_vectors(int minvec, int maxvec, const struct irq_affinity int i; for (i = 0, set_vecs = 0; i < affd->nr_sets; i++) - set_vecs += affd->sets[i]; + set_vecs += affd->set_vectors[i]; } else { get_online_cpus(); set_vecs = cpumask_weight(cpu_possible_mask); From patchwork Tue Feb 12 13:04:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10807879 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3EF621575 for ; Tue, 12 Feb 2019 13:05:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C84B2A2A4 for ; Tue, 12 Feb 2019 13:05:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2015D2B686; Tue, 12 Feb 2019 13:05:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BEC2B2B663 for ; Tue, 12 Feb 2019 13:05:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729586AbfBLNFG (ORCPT ); Tue, 12 Feb 2019 08:05:06 -0500 Received: from mx1.redhat.com ([209.132.183.28]:60684 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727750AbfBLNFG (ORCPT ); Tue, 12 Feb 2019 08:05:06 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 98A32C0524E1; Tue, 12 Feb 2019 13:05:05 +0000 (UTC) Received: from localhost (ovpn-8-25.pek2.redhat.com [10.72.8.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id CC1105D9D1; Tue, 12 Feb 2019 13:05:00 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Bjorn Helgaas , Thomas Gleixner Cc: Jens Axboe , linux-block@vger.kernel.org, Sagi Grimberg , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Keith Busch , Ming Lei Subject: [PATCH V2 2/4] genirq/affinity: add new callback for caculating set vectors Date: Tue, 12 Feb 2019 21:04:37 +0800 Message-Id: <20190212130439.14501-3-ming.lei@redhat.com> In-Reply-To: <20190212130439.14501-1-ming.lei@redhat.com> References: <20190212130439.14501-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 12 Feb 2019 13:05:05 +0000 (UTC) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently pre-caculated set vectors are provided by driver for allocating & spread vectors. This way only works when drivers passes same 'max_vecs' and 'min_vecs' to pci_alloc_irq_vectors_affinity(), also requires driver to retry the allocating & spread. As Bjorn and Keith mentioned, the current usage & interface for irq sets is a bit awkward because the retrying should have been avoided by providing one resonable 'min_vecs'. However, if 'min_vecs' isn't same with 'max_vecs', number of the allocated vectors is unknown before calling pci_alloc_irq_vectors_affinity(), then each set's vectors can't be pre-caculated. Add a new callback of .calc_sets into 'struct irq_affinity' so that driver can caculate set vectors after IRQ vector is allocated and before spread IRQ vectors. Add 'priv' so that driver may retrieve its private data via the 'struct irq_affinity'. Suggested-by: Thomas Gleixner Signed-off-by: Ming Lei --- include/linux/interrupt.h | 4 ++++ kernel/irq/affinity.c | 13 +++++++++---- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index a20150627a32..7a27f6ba1f2f 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -269,12 +269,16 @@ struct irq_affinity_notify { * the MSI(-X) vector space * @nr_sets: Length of passed in *sets array * @set_vectors: Number of affinitized sets + * @calc_sets: Callback for caculating set vectors + * @priv: Private data of @calc_sets */ struct irq_affinity { int pre_vectors; int post_vectors; int nr_sets; int set_vectors[IRQ_MAX_SETS]; + void (*calc_sets)(struct irq_affinity *, int nvecs); + void *priv; }; /** diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c index a97b7c33d2db..34abba63df4d 100644 --- a/kernel/irq/affinity.c +++ b/kernel/irq/affinity.c @@ -264,11 +264,14 @@ irq_create_affinity_masks(int nvecs, struct irq_affinity *affd) * Spread on present CPUs starting from affd->pre_vectors. If we * have multiple sets, build each sets affinity mask separately. */ - nr_sets = affd->nr_sets; - if (!nr_sets) { + if (affd->calc_sets) { + affd->calc_sets(affd, nvecs); + nr_sets = affd->nr_sets; + } else if (!affd->nr_sets) { nr_sets = 1; affd->set_vectors[0] = affvecs; - } + } else + nr_sets = affd->nr_sets; for (i = 0, usedvecs = 0; i < nr_sets; i++) { int this_vecs = affd->set_vectors[i]; @@ -314,7 +317,9 @@ int irq_calc_affinity_vectors(int minvec, int maxvec, const struct irq_affinity if (resv > minvec) return 0; - if (affd->nr_sets) { + if (affd->calc_sets) { + set_vecs = vecs; + } else if (affd->nr_sets) { int i; for (i = 0, set_vecs = 0; i < affd->nr_sets; i++) From patchwork Tue Feb 12 13:04:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10807881 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 979DF1575 for ; Tue, 12 Feb 2019 13:05:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 870612B5C6 for ; Tue, 12 Feb 2019 13:05:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B5AE2B686; Tue, 12 Feb 2019 13:05:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 173392B5C6 for ; Tue, 12 Feb 2019 13:05:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729600AbfBLNFQ (ORCPT ); Tue, 12 Feb 2019 08:05:16 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39572 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729450AbfBLNFQ (ORCPT ); Tue, 12 Feb 2019 08:05:16 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9633A59448; Tue, 12 Feb 2019 13:05:15 +0000 (UTC) Received: from localhost (ovpn-8-25.pek2.redhat.com [10.72.8.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id E1C6C1992A; Tue, 12 Feb 2019 13:05:08 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Bjorn Helgaas , Thomas Gleixner Cc: Jens Axboe , linux-block@vger.kernel.org, Sagi Grimberg , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Keith Busch , Ming Lei Subject: [PATCH V2 3/4] nvme-pci: avoid irq allocation retrying via .calc_sets Date: Tue, 12 Feb 2019 21:04:38 +0800 Message-Id: <20190212130439.14501-4-ming.lei@redhat.com> In-Reply-To: <20190212130439.14501-1-ming.lei@redhat.com> References: <20190212130439.14501-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 12 Feb 2019 13:05:15 +0000 (UTC) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently pre-caculate each set vectors, and this way requires same 'max_vecs' and 'min_vecs' passed to pci_alloc_irq_vectors_affinity(), then nvme_setup_irqs() has to retry in case of allocation failure. This usage & interface is a bit awkward because the retry should have been avoided by providing one reasonable 'min_vecs'. Implement the callback of .calc_sets, so that pci_alloc_irq_vectors_affinity() can calculate each set's vector after IRQ vectors is allocated and before spread IRQ, then NVMe's retry in case of irq allocation failure can be removed. Signed-off-by: Ming Lei Reviewed-by: Keith Busch --- drivers/nvme/host/pci.c | 62 +++++++++++++------------------------------------ 1 file changed, 16 insertions(+), 46 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 0086bdf80ea1..ca381894542a 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2078,14 +2078,25 @@ static void nvme_calc_io_queues(struct nvme_dev *dev, unsigned int irq_queues) } } +static void nvme_calc_irq_sets(struct irq_affinity *affd, int nvecs) +{ + struct nvme_dev *dev = affd->priv; + + nvme_calc_io_queues(dev, nvecs); + + affd->set_vectors[HCTX_TYPE_DEFAULT] = dev->io_queues[HCTX_TYPE_DEFAULT]; + affd->set_vectors[HCTX_TYPE_READ] = dev->io_queues[HCTX_TYPE_READ]; + affd->nr_sets = HCTX_TYPE_POLL; +} + static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues) { struct pci_dev *pdev = to_pci_dev(dev->dev); struct irq_affinity affd = { .pre_vectors = 1, - .nr_sets = 2, + .calc_sets = nvme_calc_irq_sets, + .priv = dev, }; - int *irq_sets = affd.set_vectors; int result = 0; unsigned int irq_queues, this_p_queues; @@ -2102,50 +2113,8 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues) } dev->io_queues[HCTX_TYPE_POLL] = this_p_queues; - /* - * For irq sets, we have to ask for minvec == maxvec. This passes - * any reduction back to us, so we can adjust our queue counts and - * IRQ vector needs. - */ - do { - nvme_calc_io_queues(dev, irq_queues); - irq_sets[0] = dev->io_queues[HCTX_TYPE_DEFAULT]; - irq_sets[1] = dev->io_queues[HCTX_TYPE_READ]; - if (!irq_sets[1]) - affd.nr_sets = 1; - - /* - * If we got a failure and we're down to asking for just - * 1 + 1 queues, just ask for a single vector. We'll share - * that between the single IO queue and the admin queue. - * Otherwise, we assign one independent vector to admin queue. - */ - if (irq_queues > 1) - irq_queues = irq_sets[0] + irq_sets[1] + 1; - - result = pci_alloc_irq_vectors_affinity(pdev, irq_queues, - irq_queues, - PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd); - - /* - * Need to reduce our vec counts. If we get ENOSPC, the - * platform should support mulitple vecs, we just need - * to decrease our ask. If we get EINVAL, the platform - * likely does not. Back down to ask for just one vector. - */ - if (result == -ENOSPC) { - irq_queues--; - if (!irq_queues) - return result; - continue; - } else if (result == -EINVAL) { - irq_queues = 1; - continue; - } else if (result <= 0) - return -EIO; - break; - } while (1); - + result = pci_alloc_irq_vectors_affinity(pdev, 1, irq_queues, + PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd); return result; } @@ -3021,6 +2990,7 @@ static struct pci_driver nvme_driver = { static int __init nvme_init(void) { + BUILD_BUG_ON(HCTX_TYPE_POLL > IRQ_MAX_SETS); return pci_register_driver(&nvme_driver); } From patchwork Tue Feb 12 13:04:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10807885 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB8B11575 for ; Tue, 12 Feb 2019 13:05:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA04A2B67F for ; Tue, 12 Feb 2019 13:05:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDE732B69F; Tue, 12 Feb 2019 13:05:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 73D682B686 for ; Tue, 12 Feb 2019 13:05:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729615AbfBLNFW (ORCPT ); Tue, 12 Feb 2019 08:05:22 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53544 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727750AbfBLNFW (ORCPT ); Tue, 12 Feb 2019 08:05:22 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BCF7BA7875; Tue, 12 Feb 2019 13:05:21 +0000 (UTC) Received: from localhost (ovpn-8-25.pek2.redhat.com [10.72.8.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 55991101E845; Tue, 12 Feb 2019 13:05:17 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Bjorn Helgaas , Thomas Gleixner Cc: Jens Axboe , linux-block@vger.kernel.org, Sagi Grimberg , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Keith Busch , Ming Lei Subject: [PATCH V2 4/4] genirq/affinity: Document .calc_sets as required in case of multiple sets Date: Tue, 12 Feb 2019 21:04:39 +0800 Message-Id: <20190212130439.14501-5-ming.lei@redhat.com> In-Reply-To: <20190212130439.14501-1-ming.lei@redhat.com> References: <20190212130439.14501-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 12 Feb 2019 13:05:21 +0000 (UTC) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Now NVMe has implemented the .calc_sets callback for caculating each set's vectors. For other cases of multiple irq sets, it isn't a good way to pre-caculate each set's vectors before allocating IRQ vectors because NVMe's same issue exists too. Document .calc_sets as required explicitly for multiple sets. Signed-off-by: Ming Lei Acked-by: Bjorn Helgaas --- drivers/pci/msi.c | 4 ++-- include/linux/interrupt.h | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c index 4c0b47867258..9f91fa713141 100644 --- a/drivers/pci/msi.c +++ b/drivers/pci/msi.c @@ -1039,7 +1039,7 @@ static int __pci_enable_msi_range(struct pci_dev *dev, int minvec, int maxvec, * If the caller is passing in sets, we can't support a range of * vectors. The caller needs to handle that. */ - if (affd && affd->nr_sets && minvec != maxvec) + if (affd && affd->nr_sets > 1 && !affd->calc_sets) return -EINVAL; if (WARN_ON_ONCE(dev->msi_enabled)) @@ -1097,7 +1097,7 @@ static int __pci_enable_msix_range(struct pci_dev *dev, * If the caller is passing in sets, we can't support a range of * supported vectors. The caller needs to handle that. */ - if (affd && affd->nr_sets && minvec != maxvec) + if (affd && affd->nr_sets > 1 && !affd->calc_sets) return -EINVAL; if (WARN_ON_ONCE(dev->msix_enabled)) diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index 7a27f6ba1f2f..a053f7fb0ff1 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -269,7 +269,8 @@ struct irq_affinity_notify { * the MSI(-X) vector space * @nr_sets: Length of passed in *sets array * @set_vectors: Number of affinitized sets - * @calc_sets: Callback for caculating set vectors + * @calc_sets: Callback for caculating set vectors, required for + * multiple irq sets. * @priv: Private data of @calc_sets */ struct irq_affinity {