From patchwork Sat Nov 27 01:24:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EAFEBC433EF for ; Sat, 27 Nov 2021 01:24:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233607.405467 (Exim 4.92) (envelope-from ) id 1mqmSA-0004Aq-Fh; Sat, 27 Nov 2021 01:24:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233607.405467; Sat, 27 Nov 2021 01:24:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSA-0004Aj-Bk; Sat, 27 Nov 2021 01:24:34 +0000 Received: by outflank-mailman (input) for mailman id 233607; Sat, 27 Nov 2021 01:24:33 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmS9-0003uw-BK for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:33 +0000 Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c62e1276-4f20-11ec-9787-a32c541c8605; Sat, 27 Nov 2021 02:24:32 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c62e1276-4f20-11ec-9787-a32c541c8605 Message-ID: <20211127000918.534790941@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FLvm8heMaeaty/kI8f2vBLPLCFbQls4zKlJo+z99yXU=; b=lLwdFr2xpc7l1dE+4T8c1MEKxJXFHIuawpdbeLObZ4rSaOX110baGDslF/rOU9wDbvXSVb 9FwK80ttcZC8FgPdvRGW7y+G22X8ncPFzpiSbdXExua+JKuSc4E2bG1vBFN78+SwFYmocu i4eGI4fj8p39sd3rd6Ikj8XDkzYWruMiT/gYryLTYgCEbJIM9Ls+cqlNHwu16K+5dQ/V0s 4Jni/zQOIv0JeWnIL4L/IqKFWqWgoygRU5fSIYZYHv4gyS/8gekFSB4AkkbcfaAi6tmBvg cxy9ZrieNVf0BFY+uwSNchlatRpkrPSBy8I+FDfJxHsLOviNLiig21t6lXu8gQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FLvm8heMaeaty/kI8f2vBLPLCFbQls4zKlJo+z99yXU=; b=33qL6FsZitG3y1vbXi3dduciYhZhSt51lSZUkO/B+lH+mEZreTJcZXMUe4QnrCYqlPVu8j IM/59Op77c6+5+AQ== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 01/10] genirq/msi: Add range argument to alloc/free MSI domain ops References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:31 +0100 (CET) In preparation for supporting range allocations for MSI-X, add a range argument to the MSI domain alloc/free function pointers and fixup all affected places. The range is supplied via a pointer to a struct msi_range which contains the first and last MSI index and the number of vectors to allocate/free. To support the sparse MSI-X allocations via pci_enable_msix_range() and pci_enable_msix_exact() the number of vectors can be smaller than the range defined by the first and last MSI index. This can be cleaned up later once the code is converted by converting these sparse allocations to an initial allocation on enable and expansion of the vector space at the required indices. Signed-off-by: Thomas Gleixner --- arch/powerpc/platforms/pseries/msi.c | 6 +++--- arch/x86/pci/xen.c | 10 +++++----- include/linux/msi.h | 30 +++++++++++++++++++++++------- kernel/irq/msi.c | 12 ++++++------ 4 files changed, 37 insertions(+), 21 deletions(-) --- a/arch/powerpc/platforms/pseries/msi.c +++ b/arch/powerpc/platforms/pseries/msi.c @@ -450,13 +450,13 @@ static void pseries_msi_ops_msi_free(str * RTAS can not disable one MSI at a time. It's all or nothing. Do it * at the end after all IRQs have been freed. */ -static void pseries_msi_domain_free_irqs(struct irq_domain *domain, - struct device *dev) +static void pseries_msi_domain_free_irqs(struct irq_domain *domain, struct device *dev, + struct msi_range *range) { if (WARN_ON_ONCE(!dev_is_pci(dev))) return; - __msi_domain_free_irqs(domain, dev); + __msi_domain_free_irqs(domain, dev, range); rtas_disable_msi(to_pci_dev(dev)); } --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -407,8 +407,8 @@ static void xen_pv_teardown_msi_irqs(str xen_teardown_msi_irqs(dev); } -static int xen_msi_domain_alloc_irqs(struct irq_domain *domain, - struct device *dev, int nvec) +static int xen_msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, + struct msi_range *range) { int type; @@ -420,11 +420,11 @@ static int xen_msi_domain_alloc_irqs(str else type = PCI_CAP_ID_MSI; - return xen_msi_ops.setup_msi_irqs(to_pci_dev(dev), nvec, type); + return xen_msi_ops.setup_msi_irqs(to_pci_dev(dev), range->ndesc, type); } -static void xen_msi_domain_free_irqs(struct irq_domain *domain, - struct device *dev) +static void xen_msi_domain_free_irqs(struct irq_domain *domain, struct device *dev, + struct msi_range *range) { if (WARN_ON_ONCE(!dev_is_pci(dev))) return; --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -191,6 +191,23 @@ struct msi_device_data { enum msi_desc_filter __iter_filter; }; +/** + * msi_range - Descriptor for a MSI index range + * @first: First index + * @last: Last index (inclusive) + * @ndesc: Number of descriptors for allocations + * + * @first = 0 and @last = UINT_MAX is the full range for an operation. + * + * Note: @ndesc can be less than the range defined by @first and @last to + * support sparse allocations from PCI/MSI-X. + */ +struct msi_range { + unsigned int first; + unsigned int last; + unsigned int ndesc; +}; + int msi_setup_device_data(struct device *dev); /* MSI device properties */ @@ -415,10 +432,10 @@ struct msi_domain_ops { msi_alloc_info_t *arg); void (*set_desc)(msi_alloc_info_t *arg, struct msi_desc *desc); - int (*domain_alloc_irqs)(struct irq_domain *domain, - struct device *dev, int nvec); - void (*domain_free_irqs)(struct irq_domain *domain, - struct device *dev); + int (*domain_alloc_irqs)(struct irq_domain *domain, struct device *dev, + struct msi_range *range); + void (*domain_free_irqs)(struct irq_domain *domain, struct device *dev, + struct msi_range *range); }; /** @@ -484,13 +501,12 @@ int msi_domain_set_affinity(struct irq_d struct irq_domain *msi_create_irq_domain(struct fwnode_handle *fwnode, struct msi_domain_info *info, struct irq_domain *parent); -int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, - int nvec); +int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, struct msi_range *range); int msi_domain_alloc_irqs_descs_locked(struct irq_domain *domain, struct device *dev, int nvec); int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, int nvec); -void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev); +void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev, struct msi_range *range); void msi_domain_free_irqs_descs_locked(struct irq_domain *domain, struct device *dev); void msi_domain_free_irqs(struct irq_domain *domain, struct device *dev); struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain); --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -869,8 +869,7 @@ static int msi_init_virq(struct irq_doma return 0; } -int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, - int nvec) +int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, struct msi_range *range) { struct msi_domain_info *info = domain->host_data; struct msi_domain_ops *ops = info->ops; @@ -880,7 +879,7 @@ int __msi_domain_alloc_irqs(struct irq_d int allocated = 0; int i, ret, virq; - ret = msi_domain_prepare_irqs(domain, dev, nvec, &arg); + ret = msi_domain_prepare_irqs(domain, dev, range->ndesc, &arg); if (ret) return ret; @@ -960,6 +959,7 @@ int msi_domain_alloc_irqs_descs_locked(s int nvec) { struct msi_domain_info *info = domain->host_data; + struct msi_range range = { .ndesc = nvec }; struct msi_domain_ops *ops = info->ops; int ret; @@ -969,7 +969,7 @@ int msi_domain_alloc_irqs_descs_locked(s if (ret) return ret; - ret = ops->domain_alloc_irqs(domain, dev, nvec); + ret = ops->domain_alloc_irqs(domain, dev, &range); if (ret) msi_domain_free_irqs_descs_locked(domain, dev); return ret; @@ -994,7 +994,7 @@ int msi_domain_alloc_irqs(struct irq_dom return ret; } -void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev) +void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev, struct msi_range *range) { struct msi_domain_info *info = domain->host_data; struct irq_data *irqd; @@ -1041,7 +1041,7 @@ void msi_domain_free_irqs_descs_locked(s lockdep_assert_held(&dev->msi.data->mutex); - ops->domain_free_irqs(domain, dev); + ops->domain_free_irqs(domain, dev, NULL); msi_domain_free_msi_descs(info, dev); } From patchwork Sat Nov 27 01:24:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A234C433F5 for ; Sat, 27 Nov 2021 01:24:57 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233608.405478 (Exim 4.92) (envelope-from ) id 1mqmSC-0004S5-NN; Sat, 27 Nov 2021 01:24:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233608.405478; Sat, 27 Nov 2021 01:24:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSC-0004Rq-Jc; Sat, 27 Nov 2021 01:24:36 +0000 Received: by outflank-mailman (input) for mailman id 233608; Sat, 27 Nov 2021 01:24:35 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSA-0003uw-Tl for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:35 +0000 Received: from galois.linutronix.de (galois.linutronix.de [2a0a:51c0:0:12e:550::1]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c71c6c94-4f20-11ec-9787-a32c541c8605; Sat, 27 Nov 2021 02:24:34 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c71c6c94-4f20-11ec-9787-a32c541c8605 Message-ID: <20211127000918.594818541@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EW+Gm4C9srsYC5SImaDJkppoIX48hbwPcUobStKFJEM=; b=ba0qGeiB41JUVA/nbXIZqI/ZzR+lBVMsJEDx5NRNT/VAMjLuu4gdSY/IW2XqyTiCDLdfWK AQojkSzf0utkC0bMvnh29aBEv80K2O2wGaYj6Aba7BmI3wxtr19BQARnAuvbb1iEcFmo6I hlDD4BF/OfoIMuJ2s540besiWNAKWIpBGrHUFqoqUKXdsLvTElmNbz3nO/aZYfFyRcip8+ EyAKNeFCcDrle450/Hx6NPInjM4wsqaFaUZ62JFjd2yxuWSRw0kkEbA6kTiDYHUWYhGPEY M30MNH/ErWNFGfXjykXzp8rnRca9Xg4qCwqGov4eS+2PP6X9zLCe0KqcivXq9A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EW+Gm4C9srsYC5SImaDJkppoIX48hbwPcUobStKFJEM=; b=HOol/oYi7k4NJTRqZmozHsVnqCuHGZv01jWHIiq8+jdIzZ0IwgPtzldd7npF1ZckQ/Ku5Z mXV0fPjO/GkB7DCA== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 02/10] genirq/msi: Add range argument to msi_domain_alloc/free_descs_locked() References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:33 +0100 (CET) In preparation for supporting range allocations for MSI-X, add a range argument to the msi_domain_alloc/free_descs_locked() functions and fixup all affected places. Hand in ranges which are covering the current use case. They will be refined in later steps. Signed-off-by: Thomas Gleixner --- drivers/pci/msi/irqdomain.c | 6 ++++-- include/linux/msi.h | 5 ++--- kernel/irq/msi.c | 21 ++++++++++++--------- 3 files changed, 18 insertions(+), 14 deletions(-) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -10,22 +10,24 @@ int pci_msi_setup_msi_irqs(struct pci_dev *dev, int nvec, int type) { + struct msi_range range = { .first = 0, .last = UINT_MAX, .ndesc = nvec}; struct irq_domain *domain; domain = dev_get_msi_domain(&dev->dev); if (domain && irq_domain_is_hierarchy(domain)) - return msi_domain_alloc_irqs_descs_locked(domain, &dev->dev, nvec); + return msi_domain_alloc_irqs_descs_locked(domain, &dev->dev, &range); return pci_msi_legacy_setup_msi_irqs(dev, nvec, type); } void pci_msi_teardown_msi_irqs(struct pci_dev *dev) { + struct msi_range range = { .first = 0, .last = UINT_MAX, }; struct irq_domain *domain; domain = dev_get_msi_domain(&dev->dev); if (domain && irq_domain_is_hierarchy(domain)) - msi_domain_free_irqs_descs_locked(domain, &dev->dev); + msi_domain_free_irqs_descs_locked(domain, &dev->dev, &range); else pci_msi_legacy_teardown_msi_irqs(dev); } --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -502,12 +502,11 @@ struct irq_domain *msi_create_irq_domain struct msi_domain_info *info, struct irq_domain *parent); int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, struct msi_range *range); -int msi_domain_alloc_irqs_descs_locked(struct irq_domain *domain, struct device *dev, - int nvec); +int msi_domain_alloc_irqs_descs_locked(struct irq_domain *domain, struct device *dev, struct msi_range *range); int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, int nvec); void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev, struct msi_range *range); -void msi_domain_free_irqs_descs_locked(struct irq_domain *domain, struct device *dev); +void msi_domain_free_irqs_descs_locked(struct irq_domain *domain, struct device *dev, struct msi_range *range); void msi_domain_free_irqs(struct irq_domain *domain, struct device *dev); struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain); --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -956,22 +956,21 @@ static int msi_domain_add_simple_msi_des * Return: %0 on success or an error code. */ int msi_domain_alloc_irqs_descs_locked(struct irq_domain *domain, struct device *dev, - int nvec) + struct msi_range *range) { struct msi_domain_info *info = domain->host_data; - struct msi_range range = { .ndesc = nvec }; struct msi_domain_ops *ops = info->ops; int ret; lockdep_assert_held(&dev->msi.data->mutex); - ret = msi_domain_add_simple_msi_descs(info, dev, nvec); + ret = msi_domain_add_simple_msi_descs(info, dev, range->ndesc); if (ret) return ret; - ret = ops->domain_alloc_irqs(domain, dev, &range); + ret = ops->domain_alloc_irqs(domain, dev, range); if (ret) - msi_domain_free_irqs_descs_locked(domain, dev); + msi_domain_free_irqs_descs_locked(domain, dev, range); return ret; } @@ -986,10 +985,11 @@ int msi_domain_alloc_irqs_descs_locked(s */ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, int nvec) { + struct msi_range range = { .first = 0, .last = UINT_MAX, .ndesc = nvec, }; int ret; msi_lock_descs(dev); - ret = msi_domain_alloc_irqs_descs_locked(domain, dev, nvec); + ret = msi_domain_alloc_irqs_descs_locked(domain, dev, &range); msi_unlock_descs(dev); return ret; } @@ -1034,14 +1034,15 @@ static void msi_domain_free_msi_descs(st * pair. Use this for MSI irqdomains which implement their own vector * allocation. */ -void msi_domain_free_irqs_descs_locked(struct irq_domain *domain, struct device *dev) +void msi_domain_free_irqs_descs_locked(struct irq_domain *domain, struct device *dev, + struct msi_range *range) { struct msi_domain_info *info = domain->host_data; struct msi_domain_ops *ops = info->ops; lockdep_assert_held(&dev->msi.data->mutex); - ops->domain_free_irqs(domain, dev, NULL); + ops->domain_free_irqs(domain, dev, range); msi_domain_free_msi_descs(info, dev); } @@ -1053,8 +1054,10 @@ void msi_domain_free_irqs_descs_locked(s */ void msi_domain_free_irqs(struct irq_domain *domain, struct device *dev) { + struct msi_range range = { .first = 0, .last = UINT_MAX, }; + msi_lock_descs(dev); - msi_domain_free_irqs_descs_locked(domain, dev); + msi_domain_free_irqs_descs_locked(domain, dev, &range); msi_unlock_descs(dev); } From patchwork Sat Nov 27 01:24:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E6411C4332F for ; Sat, 27 Nov 2021 01:24:56 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233609.405489 (Exim 4.92) (envelope-from ) id 1mqmSE-0004jJ-0J; Sat, 27 Nov 2021 01:24:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233609.405489; Sat, 27 Nov 2021 01:24:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSD-0004j3-S0; Sat, 27 Nov 2021 01:24:37 +0000 Received: by outflank-mailman (input) for mailman id 233609; Sat, 27 Nov 2021 01:24:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSC-0003uw-Ff for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:36 +0000 Received: from galois.linutronix.de (galois.linutronix.de [2a0a:51c0:0:12e:550::1]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c8076b8e-4f20-11ec-9787-a32c541c8605; Sat, 27 Nov 2021 02:24:35 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c8076b8e-4f20-11ec-9787-a32c541c8605 Message-ID: <20211127000918.664542907@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=rrdNHJUyUgKyNz6Pbn0ws4hWiMPwwbNMnZ1F31AYBD0=; b=MarT9tzc3uD/IHylu0qgFMXJqCIRF9E81IK9oU/QSiHl//TLD0MlLLyVDHc3E7fdnFbWu4 jNajrFTveOc0UcHgHrFVYFiOkVWUcp3VDcqqQXPjZNVJR5XW50P58xX/SZg2KrTAsXpTtY Tx+yiPXnYoApeFdOBCHvl1LeKN2JqGWS+8oSh99qUDZtlKU0CvhJVcdO1Ig3H9Vvi7C6vm Kbuso6YgToHZ6EpxYMF78Igydg+8TrnH0g8uLD1WNk02H7AaNPPJZHQyTMgOO2HrkZQhcQ qo2f721FKRrB+prTdNZtaax2ROXufT5XV4IAi0xugE7NPJhFzBmGup8OcxopZw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=rrdNHJUyUgKyNz6Pbn0ws4hWiMPwwbNMnZ1F31AYBD0=; b=z83mpLTnkklHBxL+vHIMKuuahXOWE70VTpa6amHX4Jp0eBu+uIRunBrhjFfksl44tytuvK ibg2EKm4G97OPyCw== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 03/10] genirq/msi: Make MSI descriptor alloc/free ready for range allocations References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:34 +0100 (CET) Convert the MSI descriptor related functions to ranges and fixup the call sites. Signed-off-by: Thomas Gleixner --- drivers/base/platform-msi.c | 3 ++- include/linux/msi.h | 7 ++++--- kernel/irq/msi.c | 38 +++++++++++++++++++------------------- 3 files changed, 25 insertions(+), 23 deletions(-) --- a/drivers/base/platform-msi.c +++ b/drivers/base/platform-msi.c @@ -320,11 +320,12 @@ struct irq_domain * void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs) { + struct msi_range range = { .first = virq, .last = virq + nr_irqs - 1, }; struct platform_msi_priv_data *data = domain->host_data; msi_lock_descs(data->dev); irq_domain_free_irqs_common(domain, virq, nr_irqs); - msi_free_msi_descs_range(data->dev, MSI_DESC_ALL, virq, nr_irqs); + msi_free_msi_descs_range(data->dev, MSI_DESC_ALL, &range); msi_unlock_descs(data->dev); } --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -321,8 +321,7 @@ static inline void pci_write_msi_msg(uns #endif /* CONFIG_PCI_MSI */ int msi_add_msi_desc(struct device *dev, struct msi_desc *init_desc); -void msi_free_msi_descs_range(struct device *dev, enum msi_desc_filter filter, - unsigned int base_index, unsigned int ndesc); +void msi_free_msi_descs_range(struct device *dev, enum msi_desc_filter filter, struct msi_range *range); /** * msi_free_msi_descs - Free MSI descriptors of a device @@ -330,7 +329,9 @@ void msi_free_msi_descs_range(struct dev */ static inline void msi_free_msi_descs(struct device *dev) { - msi_free_msi_descs_range(dev, MSI_DESC_ALL, 0, UINT_MAX); + struct msi_range range = { .first = 0, .last = UINT_MAX, }; + + msi_free_msi_descs_range(dev, MSI_DESC_ALL, &range); } void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg); --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -101,19 +101,19 @@ int msi_add_msi_desc(struct device *dev, * * Return: 0 on success or an appropriate failure code. */ -static int msi_add_simple_msi_descs(struct device *dev, unsigned int index, unsigned int ndesc) +static int msi_add_simple_msi_descs(struct device *dev, struct msi_range *range) { struct msi_desc *desc; - unsigned long i; + unsigned long idx; int ret; lockdep_assert_held(&dev->msi.data->mutex); - for (i = 0; i < ndesc; i++) { + for (idx = range->first; idx <= range->last; idx++) { desc = msi_alloc_desc(dev, 1, NULL); if (!desc) goto fail_mem; - ret = msi_insert_desc(dev->msi.data, desc, index + i); + ret = msi_insert_desc(dev->msi.data, desc, idx); if (ret) goto fail; } @@ -122,7 +122,7 @@ static int msi_add_simple_msi_descs(stru fail_mem: ret = -ENOMEM; fail: - msi_free_msi_descs_range(dev, MSI_DESC_NOTASSOCIATED, index, ndesc); + msi_free_msi_descs_range(dev, MSI_DESC_NOTASSOCIATED, range); return ret; } @@ -148,14 +148,14 @@ static bool msi_desc_match(struct msi_de * @ndesc: Number of descriptors to free */ void msi_free_msi_descs_range(struct device *dev, enum msi_desc_filter filter, - unsigned int base_index, unsigned int ndesc) + struct msi_range *range) { struct msi_desc *desc; unsigned long idx; lockdep_assert_held(&dev->msi.data->mutex); - xa_for_each_range(&dev->msi.data->store, idx, desc, base_index, base_index + ndesc - 1) { + xa_for_each_range(&dev->msi.data->store, idx, desc, range->first, range->last) { if (msi_desc_match(desc, filter)) { xa_erase(&dev->msi.data->store, idx); msi_free_desc(desc); @@ -746,17 +746,18 @@ int msi_domain_prepare_irqs(struct irq_d int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev, int virq_base, int nvec, msi_alloc_info_t *arg) { + struct msi_range range = { .first = virq_base, .last = virq_base + nvec - 1 }; struct msi_domain_info *info = domain->host_data; struct msi_domain_ops *ops = info->ops; struct msi_desc *desc; int ret, virq; msi_lock_descs(dev); - ret = msi_add_simple_msi_descs(dev, virq_base, nvec); + ret = msi_add_simple_msi_descs(dev, &range); if (ret) goto unlock; - for (virq = virq_base; virq < virq_base + nvec; virq++) { + for (virq = range.first; virq <= range.last; virq++) { desc = xa_load(&dev->msi.data->store, virq); desc->irq = virq; @@ -773,7 +774,7 @@ int msi_domain_populate_irqs(struct irq_ fail: for (--virq; virq >= virq_base; virq--) irq_domain_free_irqs_common(domain, virq, 1); - msi_free_msi_descs_range(dev, MSI_DESC_ALL, virq_base, nvec); + msi_free_msi_descs_range(dev, MSI_DESC_ALL, &range); unlock: msi_unlock_descs(dev); return ret; @@ -932,14 +933,13 @@ int __msi_domain_alloc_irqs(struct irq_d return 0; } -static int msi_domain_add_simple_msi_descs(struct msi_domain_info *info, - struct device *dev, - unsigned int num_descs) +static int msi_domain_add_simple_msi_descs(struct msi_domain_info *info, struct device *dev, + struct msi_range *range) { if (!(info->flags & MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS)) return 0; - return msi_add_simple_msi_descs(dev, 0, num_descs); + return msi_add_simple_msi_descs(dev, range); } /** @@ -964,7 +964,7 @@ int msi_domain_alloc_irqs_descs_locked(s lockdep_assert_held(&dev->msi.data->mutex); - ret = msi_domain_add_simple_msi_descs(info, dev, range->ndesc); + ret = msi_domain_add_simple_msi_descs(info, dev, range); if (ret) return ret; @@ -1017,11 +1017,11 @@ void __msi_domain_free_irqs(struct irq_d } } -static void msi_domain_free_msi_descs(struct msi_domain_info *info, - struct device *dev) +static void msi_domain_free_msi_descs(struct msi_domain_info *info, struct device *dev, + struct msi_range *range) { if (info->flags & MSI_FLAG_FREE_MSI_DESCS) - msi_free_msi_descs(dev); + msi_free_msi_descs_range(dev, MSI_DESC_ALL, range); } /** @@ -1043,7 +1043,7 @@ void msi_domain_free_irqs_descs_locked(s lockdep_assert_held(&dev->msi.data->mutex); ops->domain_free_irqs(domain, dev, range); - msi_domain_free_msi_descs(info, dev); + msi_domain_free_msi_descs(info, dev, range); } /** From patchwork Sat Nov 27 01:24:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 546C5C433EF for ; Sat, 27 Nov 2021 01:24:57 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233610.405500 (Exim 4.92) (envelope-from ) id 1mqmSF-000520-FE; Sat, 27 Nov 2021 01:24:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233610.405500; Sat, 27 Nov 2021 01:24:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSF-00050t-A6; Sat, 27 Nov 2021 01:24:39 +0000 Received: by outflank-mailman (input) for mailman id 233610; Sat, 27 Nov 2021 01:24:38 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSE-0004gG-2S for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:38 +0000 Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c8ef161c-4f20-11ec-b941-1df2895da90e; Sat, 27 Nov 2021 02:24:37 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c8ef161c-4f20-11ec-b941-1df2895da90e Message-ID: <20211127000918.723637256@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=wAZLS4DVWG/31wPrjnVkWgrCpcsyPlNJgofX29BhKpw=; b=ACnXSfNOqewTY1/+Ig8VGJiP7Eq/Hu3Mv+SGL6FsmDIQqkFvprL4HbnwzMigRxHJsUCQvX HYkIGqFfPnBUVK2Gp7ida8axPvoyCJrV5eUDwLgcQ+4I3ZmFIlXQq+B2eKHdf/fcE1U1kz nsJU3n6d/XTfa5q6bmvRi8fkPmovjaBqt6w9idFgxeAybfsJaqycBHPeVheTi6QyrV5Psx KOVZkP2bvoI2OhezNabehmrvAH9DGlp9mLePq4eOKWdL5Z4w8GKb6RlfsGS1+ycA+No2jo M8NZR22qHu1LYfXAVJz7WDPCBDty/gbpCOtMDvTOca33CRTlhogVdVU/Bm1JEw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=wAZLS4DVWG/31wPrjnVkWgrCpcsyPlNJgofX29BhKpw=; b=vw0tiCuP5ldLG61m2bCrJEknO6vxY4QAx1hiFFMZvgHkZRCNmVTavfIu2AmwcqvFT1i30V Q2Dw304vuVxzBZDw== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 04/10] genirq/msi: Prepare MSI domain alloc/free for range irq allocation References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:36 +0100 (CET) Make the iterators in the allocation and free functions range based. Signed-off-by: Thomas Gleixner --- kernel/irq/msi.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -877,6 +877,7 @@ int __msi_domain_alloc_irqs(struct irq_d msi_alloc_info_t arg = { }; unsigned int vflags = 0; struct msi_desc *desc; + unsigned long idx; int allocated = 0; int i, ret, virq; @@ -906,7 +907,10 @@ int __msi_domain_alloc_irqs(struct irq_d vflags |= VIRQ_NOMASK_QUIRK; } - msi_for_each_desc(desc, dev, MSI_DESC_NOTASSOCIATED) { + xa_for_each_range(&dev->msi.data->store, idx, desc, range->first, range->last) { + if (!msi_desc_match(desc, MSI_DESC_NOTASSOCIATED)) + continue; + ops->set_desc(&arg, desc); virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used, @@ -999,10 +1003,14 @@ void __msi_domain_free_irqs(struct irq_d struct msi_domain_info *info = domain->host_data; struct irq_data *irqd; struct msi_desc *desc; + unsigned long idx; int i; /* Only handle MSI entries which have an interrupt associated */ - msi_for_each_desc(desc, dev, MSI_DESC_ASSOCIATED) { + xa_for_each_range(&dev->msi.data->store, idx, desc, range->first, range->last) { + if (!msi_desc_match(desc, MSI_DESC_ASSOCIATED)) + continue; + /* Make sure all interrupts are deactivated */ for (i = 0; i < desc->nvec_used; i++) { irqd = irq_domain_get_irq_data(domain, desc->irq + i); From patchwork Sat Nov 27 01:24:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A9098C433FE for ; Sat, 27 Nov 2021 01:24:57 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233611.405511 (Exim 4.92) (envelope-from ) id 1mqmSG-0005J2-Pk; Sat, 27 Nov 2021 01:24:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233611.405511; Sat, 27 Nov 2021 01:24:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSG-0005IP-Kj; Sat, 27 Nov 2021 01:24:40 +0000 Received: by outflank-mailman (input) for mailman id 233611; Sat, 27 Nov 2021 01:24:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSF-0004gG-9g for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:39 +0000 Received: from galois.linutronix.de (galois.linutronix.de [2a0a:51c0:0:12e:550::1]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c9dc1b68-4f20-11ec-b941-1df2895da90e; Sat, 27 Nov 2021 02:24:38 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c9dc1b68-4f20-11ec-b941-1df2895da90e Message-ID: <20211127000918.779751933@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=CFmm+AWRwXcJQSLL7VY9/FlDxYE98PyZd7GrvqHxiD8=; b=e24VzNipUk4hxg1PHC6IAXr9weR7Gc/SoXDfA3RHuL3eUYXW0KQ0vmOx1Rup57wdQ4/c0M j0SsM53zaTsEp+xcsTJcjjvt9idxHze65/F9ku69wsdFH4O1IR3xnNt1kBpYgjtuq1EJx3 G4+8DqRMVr0EurGDQTIs906q+2dXCPyTndaD2DMJHPeXSLph/ufoqB5pa8guu1m5SJm17L kBtRWLOCwVpFplTZ//BIVum2PpChXwXEF/pgcDF3hqzjYmEihoy8xv/uTBJ/AHXGcWoyWZ FCUJcprzFqND4a981S/J+eyIOTEK4LUqOJJk4OnVzXGII9RmrCOyMXoWJHxGBg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=CFmm+AWRwXcJQSLL7VY9/FlDxYE98PyZd7GrvqHxiD8=; b=gCM9TtH7kdRRuRtrS2dC9nAFXYnyiZU34H6DDhO0H4O61lGny8X8mQzTO3e0hAM5LNj6ef CHes+qC3TZO1npAA== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 05/10] genirq/msi: Add domain info flag MSI_FLAG_CAN_EXPAND References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:37 +0100 (CET) Not all MSI domains support runtime expansions of PCI/MSI-X vectors. Add a domain flag so implementations can opt in. Signed-off-by: Thomas Gleixner --- include/linux/msi.h | 2 ++ 1 file changed, 2 insertions(+) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -494,6 +494,8 @@ enum { MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS = (1 << 9), /* Free MSI descriptors */ MSI_FLAG_FREE_MSI_DESCS = (1 << 10), + /* MSI vectors can be expanded after initial setup */ + MSI_FLAG_CAN_EXPAND = (1 << 11), }; int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask, From patchwork Sat Nov 27 01:24:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF686C433F5 for ; Sat, 27 Nov 2021 01:24:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233612.405522 (Exim 4.92) (envelope-from ) id 1mqmSJ-0005fp-5X; Sat, 27 Nov 2021 01:24:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233612.405522; Sat, 27 Nov 2021 01:24:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSI-0005fe-W4; Sat, 27 Nov 2021 01:24:42 +0000 Received: by outflank-mailman (input) for mailman id 233612; Sat, 27 Nov 2021 01:24:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSH-0004gG-2y for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:41 +0000 Received: from galois.linutronix.de (galois.linutronix.de [2a0a:51c0:0:12e:550::1]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id caccf37b-4f20-11ec-b941-1df2895da90e; Sat, 27 Nov 2021 02:24:40 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: caccf37b-4f20-11ec-b941-1df2895da90e Message-ID: <20211127000918.836550402@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QyFkdnL3oY21jCwm7LMhiRAQpgzR7HBWMPckiZs+cZQ=; b=GBjn+b8QrsutedncAZPAURSSsA4cAXX/k7rT1hR+oC058LdbCYvWngK4orYIZgTSd/uxwl Aa+UcI+mivU7IjeRpqO9KD1764v9FRykg9zl00tSY4UcIHx67r0Udm9yLYeRLWYyvQlAY6 3tnrDvlJ/tZEiowTqvphP2SenTauFkT8SnCUF1cB4tGasAweDA0p5ZZ58RCgRgZPFI8Umg 3P+G9THTmbwcT+pPPNufVw7efW/jGCbDtTlJsig9IzdDZBTC8T1dqdMjTApAGPtcoC3fG2 NLsuG3m+9Q1T5nNk6Bf1DIVl7xUvdlctpWZxI9N7809o4lW5h5sTyHaJN9pQmA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QyFkdnL3oY21jCwm7LMhiRAQpgzR7HBWMPckiZs+cZQ=; b=qbNWScsKoX9jhoddvCDdBjSKrocguh1IQ5ezIZ/W0XajOCHwP+R6ptVU1LrhlXA7w6l1L3 qq8DHdqoSrxdHCDw== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 06/10] PCI/MSI: Use range in allocation path References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:39 +0100 (CET) Make the allocation path range based to prepare for runtime expansion of MSI-X vectors. Signed-off-by: Thomas Gleixner --- drivers/pci/msi/irqdomain.c | 7 +++---- drivers/pci/msi/msi.c | 34 +++++++++++++++++++++------------- drivers/pci/msi/msi.h | 2 +- 3 files changed, 25 insertions(+), 18 deletions(-) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -8,16 +8,15 @@ #include "msi.h" -int pci_msi_setup_msi_irqs(struct pci_dev *dev, int nvec, int type) +int pci_msi_setup_msi_irqs(struct pci_dev *dev, struct msi_range *range, int type) { - struct msi_range range = { .first = 0, .last = UINT_MAX, .ndesc = nvec}; struct irq_domain *domain; domain = dev_get_msi_domain(&dev->dev); if (domain && irq_domain_is_hierarchy(domain)) - return msi_domain_alloc_irqs_descs_locked(domain, &dev->dev, &range); + return msi_domain_alloc_irqs_descs_locked(domain, &dev->dev, range); - return pci_msi_legacy_setup_msi_irqs(dev, nvec, type); + return pci_msi_legacy_setup_msi_irqs(dev, range->ndesc, type); } void pci_msi_teardown_msi_irqs(struct pci_dev *dev) --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -370,14 +370,16 @@ static int msi_setup_msi_desc(struct pci return ret; } -static int msi_verify_entries(struct pci_dev *dev) +static int msi_verify_entries(struct pci_dev *dev, struct msi_range *range) { struct msi_desc *entry; if (!dev->no_64bit_msi) return 0; - msi_for_each_desc(entry, &dev->dev, MSI_DESC_ALL) { + msi_for_each_desc_from(entry, &dev->dev, MSI_DESC_ALL, range->first) { + if (entry->msi_index > range->last) + return 0; if (entry->msg.address_hi) { pci_err(dev, "arch assigned 64-bit MSI address %#x%08x but device only supports 32 bits\n", entry->msg.address_hi, entry->msg.address_lo); @@ -402,6 +404,7 @@ static int msi_verify_entries(struct pci static int msi_capability_init(struct pci_dev *dev, int nvec, struct irq_affinity *affd) { + struct msi_range range = { .first = 0, .last = 0, .ndesc = nvec, }; struct irq_affinity_desc *masks = NULL; struct msi_desc *entry; int ret; @@ -421,11 +424,11 @@ static int msi_capability_init(struct pc pci_msi_mask(entry, msi_multi_mask(entry)); /* Configure MSI capability structure */ - ret = pci_msi_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSI); + ret = pci_msi_setup_msi_irqs(dev, &range, PCI_CAP_ID_MSI); if (ret) goto err; - ret = msi_verify_entries(dev); + ret = msi_verify_entries(dev, &range); if (ret) goto err; @@ -469,7 +472,8 @@ static void __iomem *msix_map_region(str } static int msix_setup_msi_descs(struct pci_dev *dev, void __iomem *base, - struct msix_entry *entries, int nvec, + struct msi_range *range, + struct msix_entry *entries, struct irq_affinity_desc *masks) { int ret, i, vec_count = pci_msix_vec_count(dev); @@ -485,8 +489,8 @@ static int msix_setup_msi_descs(struct p desc.pci.msi_attrib.default_irq = dev->irq; desc.pci.mask_base = base; - for (i = 0, curmsk = masks; i < nvec; i++, curmsk++) { - desc.msi_index = entries ? entries[i].entry : i; + for (i = 0, curmsk = masks; i < range->ndesc; i++, curmsk++) { + desc.msi_index = entries ? entries[i].entry : range->first + i; desc.affinity = masks ? curmsk : NULL; desc.pci.msi_attrib.is_virtual = desc.msi_index >= vec_count; desc.pci.msi_attrib.can_mask = !pci_msi_ignore_mask && @@ -500,6 +504,9 @@ static int msix_setup_msi_descs(struct p ret = msi_add_msi_desc(&dev->dev, &desc); if (ret) break; + + if (desc.msi_index > range->last) + range->last = desc.msi_index; } return ret; @@ -530,28 +537,28 @@ static void msix_mask_all(void __iomem * } static int msix_setup_interrupts(struct pci_dev *dev, void __iomem *base, - struct msix_entry *entries, int nvec, + struct msi_range *range, struct msix_entry *entries, struct irq_affinity *affd) { struct irq_affinity_desc *masks = NULL; int ret; if (affd) - masks = irq_create_affinity_masks(nvec, affd); + masks = irq_create_affinity_masks(range->ndesc, affd); msi_lock_descs(&dev->dev); - ret = msix_setup_msi_descs(dev, base, entries, nvec, masks); + ret = msix_setup_msi_descs(dev, base, range, entries, masks); if (ret) goto out_free; dev->dev.msi.data->properties = MSI_PROP_PCI_MSIX | MSI_PROP_64BIT; - ret = pci_msi_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSIX); + ret = pci_msi_setup_msi_irqs(dev, range, PCI_CAP_ID_MSIX); if (ret) goto out_free; /* Check if all MSI entries honor device restrictions */ - ret = msi_verify_entries(dev); + ret = msi_verify_entries(dev, range); if (ret) goto out_free; @@ -580,6 +587,7 @@ static int msix_setup_interrupts(struct static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries, int nvec, struct irq_affinity *affd) { + struct msi_range range = { .first = 0, .last = 0, .ndesc = nvec, }; void __iomem *base; int ret, tsize; u16 control; @@ -606,7 +614,7 @@ static int msix_capability_init(struct p /* Ensure that all table entries are masked. */ msix_mask_all(base, tsize); - ret = msix_setup_interrupts(dev, base, entries, nvec, affd); + ret = msix_setup_interrupts(dev, base, &range, entries, affd); if (ret) goto out_disable; --- a/drivers/pci/msi/msi.h +++ b/drivers/pci/msi/msi.h @@ -5,7 +5,7 @@ #define msix_table_size(flags) ((flags & PCI_MSIX_FLAGS_QSIZE) + 1) -extern int pci_msi_setup_msi_irqs(struct pci_dev *dev, int nvec, int type); +extern int pci_msi_setup_msi_irqs(struct pci_dev *dev, struct msi_range *range, int type); extern void pci_msi_teardown_msi_irqs(struct pci_dev *dev); #ifdef CONFIG_PCI_MSI_ARCH_FALLBACKS From patchwork Sat Nov 27 01:24:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81979C433FE for ; Sat, 27 Nov 2021 01:25:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233613.405527 (Exim 4.92) (envelope-from ) id 1mqmSJ-0005jT-KM; Sat, 27 Nov 2021 01:24:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233613.405527; Sat, 27 Nov 2021 01:24:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSJ-0005iG-Bb; Sat, 27 Nov 2021 01:24:43 +0000 Received: by outflank-mailman (input) for mailman id 233613; Sat, 27 Nov 2021 01:24:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSI-0004gG-HU for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:42 +0000 Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id cbb420a3-4f20-11ec-b941-1df2895da90e; Sat, 27 Nov 2021 02:24:41 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cbb420a3-4f20-11ec-b941-1df2895da90e Message-ID: <20211127000918.892733246@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=1qj+ON8bFgIPks8keDU/08NOHxNWcizsMPwkJuRGMrI=; b=TpryTAQxxx6RWppxDWnQp1sseu+ucPeEfW+EupZx2rCOEvok5NQEvK3qjscAvEXPnKHmSQ sI/urKNHo3gJgriNBWDb0QhvqSsY81joc5q1w9Se24pZmghuMFkTDjBC6QojPQnDdhfCXq Jnd9Cm1icNq2eo2JCoo4o5VuhYLt2S83AR83F1zOnMJdSgsUEQtcJ28VJMsqNX4AyeQMUs NMXJEnyUqSkE6kh7Jqf3FPdhM9yK0G+ORqpoJrzhgEA7oNSRg2YUTW5kCj5vZAEmp2zAI/ 6MSH6J30iS6UDk/IGKjB4QnhlPXmiW/MpzU/gzDQ3NNKK/qzN9ApaYBom2X9+g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=1qj+ON8bFgIPks8keDU/08NOHxNWcizsMPwkJuRGMrI=; b=a8sBRzgNgK5ykgWV3lttGcyQfCDrllQes3dDaXtlLCK/MalZhwpdvKqazKfXM2oQvBMoIk 0ArxoCNQu0M2rQDA== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 07/10] PCI/MSI: Make free related functions range based References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:41 +0100 (CET) In preparation of runtime expandable PCI/MSI-X vectors convert the related free functions to take ranges instead of assuming a zero based vector space. Signed-off-by: Thomas Gleixner --- drivers/pci/msi/irqdomain.c | 5 ++--- drivers/pci/msi/msi.c | 24 ++++++++++++++++-------- drivers/pci/msi/msi.h | 2 +- 3 files changed, 19 insertions(+), 12 deletions(-) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -19,14 +19,13 @@ int pci_msi_setup_msi_irqs(struct pci_de return pci_msi_legacy_setup_msi_irqs(dev, range->ndesc, type); } -void pci_msi_teardown_msi_irqs(struct pci_dev *dev) +void pci_msi_teardown_msi_irqs(struct pci_dev *dev, struct msi_range *range) { - struct msi_range range = { .first = 0, .last = UINT_MAX, }; struct irq_domain *domain; domain = dev_get_msi_domain(&dev->dev); if (domain && irq_domain_is_hierarchy(domain)) - msi_domain_free_irqs_descs_locked(domain, &dev->dev, &range); + msi_domain_free_irqs_descs_locked(domain, &dev->dev, range); else pci_msi_legacy_teardown_msi_irqs(dev); } --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -222,9 +222,12 @@ void pci_write_msi_msg(unsigned int irq, } EXPORT_SYMBOL_GPL(pci_write_msi_msg); -static void free_msi_irqs(struct pci_dev *dev) +static void free_msi_irqs(struct pci_dev *dev, struct msi_range *range, bool shutdown) { - pci_msi_teardown_msi_irqs(dev); + pci_msi_teardown_msi_irqs(dev, range); + + if (!shutdown) + return; if (dev->msix_base) { iounmap(dev->msix_base); @@ -443,7 +446,7 @@ static int msi_capability_init(struct pc err: pci_msi_unmask(entry, msi_multi_mask(entry)); - free_msi_irqs(dev); + free_msi_irqs(dev, &range, true); unlock: msi_unlock_descs(&dev->dev); kfree(masks); @@ -538,7 +541,7 @@ static void msix_mask_all(void __iomem * static int msix_setup_interrupts(struct pci_dev *dev, void __iomem *base, struct msi_range *range, struct msix_entry *entries, - struct irq_affinity *affd) + struct irq_affinity *affd, bool expand) { struct irq_affinity_desc *masks = NULL; int ret; @@ -566,7 +569,8 @@ static int msix_setup_interrupts(struct goto out_unlock; out_free: - free_msi_irqs(dev); + free_msi_irqs(dev, range, !expand); + out_unlock: msi_unlock_descs(&dev->dev); kfree(masks); @@ -614,7 +618,7 @@ static int msix_capability_init(struct p /* Ensure that all table entries are masked. */ msix_mask_all(base, tsize); - ret = msix_setup_interrupts(dev, base, &range, entries, affd); + ret = msix_setup_interrupts(dev, base, &range, entries, affd, false); if (ret) goto out_disable; @@ -728,12 +732,14 @@ static void pci_msi_shutdown(struct pci_ void pci_disable_msi(struct pci_dev *dev) { + struct msi_range range = { .first = 0, .last = 0, }; + if (!pci_msi_enable || !dev || !dev->msi_enabled) return; msi_lock_descs(&dev->dev); pci_msi_shutdown(dev); - free_msi_irqs(dev); + free_msi_irqs(dev, &range, true); msi_unlock_descs(&dev->dev); } EXPORT_SYMBOL(pci_disable_msi); @@ -817,12 +823,14 @@ static void pci_msix_shutdown(struct pci void pci_disable_msix(struct pci_dev *dev) { + struct msi_range range = { .first = 0, .last = UINT_MAX, }; + if (!pci_msi_enable || !dev || !dev->msix_enabled) return; msi_lock_descs(&dev->dev); pci_msix_shutdown(dev); - free_msi_irqs(dev); + free_msi_irqs(dev, &range, true); msi_unlock_descs(&dev->dev); } EXPORT_SYMBOL(pci_disable_msix); --- a/drivers/pci/msi/msi.h +++ b/drivers/pci/msi/msi.h @@ -6,7 +6,7 @@ #define msix_table_size(flags) ((flags & PCI_MSIX_FLAGS_QSIZE) + 1) extern int pci_msi_setup_msi_irqs(struct pci_dev *dev, struct msi_range *range, int type); -extern void pci_msi_teardown_msi_irqs(struct pci_dev *dev); +extern void pci_msi_teardown_msi_irqs(struct pci_dev *dev, struct msi_range *range); #ifdef CONFIG_PCI_MSI_ARCH_FALLBACKS extern int pci_msi_legacy_setup_msi_irqs(struct pci_dev *dev, int nvec, int type); From patchwork Sat Nov 27 01:24:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12641897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D8B94C433F5 for ; Sat, 27 Nov 2021 01:25:03 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233614.405544 (Exim 4.92) (envelope-from ) id 1mqmSM-0006NB-41; Sat, 27 Nov 2021 01:24:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233614.405544; Sat, 27 Nov 2021 01:24:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSL-0006MW-Ts; Sat, 27 Nov 2021 01:24:45 +0000 Received: by outflank-mailman (input) for mailman id 233614; Sat, 27 Nov 2021 01:24:44 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSK-0003uw-1a for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:44 +0000 Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id cc9d0993-4f20-11ec-9787-a32c541c8605; Sat, 27 Nov 2021 02:24:43 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cc9d0993-4f20-11ec-9787-a32c541c8605 Message-ID: <20211127000918.948090130@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976283; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=0A61IAyhwI6GY5iuZYdA+ROlAbbN0w/XOIQw6hDb9NM=; b=u9TKqnusy4chqm3EGles3FCrSf9gO2SOc5ZSS2v+xgZD7/jCXibBKaEp2usPlK43Ep4OyF nluuEvkjH9Zwh3EDRF4zD+tPCC7NyAfn/QQs0J3glLsMhER63aL9Hqy+jdJwnGVjLn/TbV A1QjvxuddB8p4pNSNzu2Ed0dDQ4CPhfiN1AsChvb5X7XicA6ZLMdEx2E9qFBFwpV2PZyN/ bjiLIBGIiTlLYA+LxdGOY9ii+Fv3KJ7hBe/zdjqFsVI8hi0ub1SYXHy+N8kfvQUCB9htBH LMNGk56OV16MgLVSmUat+bpOijccz06MfnH9BVG4zo9uiV1i1K7vl0fBXVgwKQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976283; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=0A61IAyhwI6GY5iuZYdA+ROlAbbN0w/XOIQw6hDb9NM=; b=xEM4t7hASH4Fv9+Q06S2RwK+mQnaOqtbNKdZ9iN52VWX1kdgv/gw59z1rFiY2d5SqPSweM DKCVIioTt0nAxJCw== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 08/10] PCI/MSI: Provide pci_msi_domain_supports_expand() References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:42 +0100 (CET) Not all irq domain implementations can support runtime MSI-X vector expansion as they assume zero based allocations or have other restrictions. The legacy PCI allocation functions are not suited for runtime vector expansion either. Add a function which allows to query whether runtime MSI-X vector expansion is supported or not. Signed-off-by: Thomas Gleixner --- drivers/pci/msi/irqdomain.c | 29 +++++++++++++++++++++++------ include/linux/msi.h | 2 ++ 2 files changed, 25 insertions(+), 6 deletions(-) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -8,12 +8,18 @@ #include "msi.h" +static struct irq_domain *pci_get_msi_domain(struct pci_dev *dev) +{ + struct irq_domain *domain = dev_get_msi_domain(&dev->dev); + + return domain && irq_domain_is_hierarchy(domain) ? domain : NULL; +} + int pci_msi_setup_msi_irqs(struct pci_dev *dev, struct msi_range *range, int type) { - struct irq_domain *domain; + struct irq_domain *domain = pci_get_msi_domain(dev); - domain = dev_get_msi_domain(&dev->dev); - if (domain && irq_domain_is_hierarchy(domain)) + if (domain) return msi_domain_alloc_irqs_descs_locked(domain, &dev->dev, range); return pci_msi_legacy_setup_msi_irqs(dev, range->ndesc, type); @@ -21,15 +27,26 @@ int pci_msi_setup_msi_irqs(struct pci_de void pci_msi_teardown_msi_irqs(struct pci_dev *dev, struct msi_range *range) { - struct irq_domain *domain; + struct irq_domain *domain = pci_get_msi_domain(dev); - domain = dev_get_msi_domain(&dev->dev); - if (domain && irq_domain_is_hierarchy(domain)) + if (domain) msi_domain_free_irqs_descs_locked(domain, &dev->dev, range); else pci_msi_legacy_teardown_msi_irqs(dev); } +bool pci_msi_domain_supports_expand(struct pci_dev *dev) +{ + struct irq_domain *domain = pci_get_msi_domain(dev); + struct msi_domain_info *info; + + if (!domain) + return false; + + info = domain->host_data; + return info->flags & MSI_FLAG_CAN_EXPAND; +} + /** * pci_msi_domain_write_msg - Helper to write MSI message to PCI config space * @irq_data: Pointer to interrupt data of the MSI interrupt --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -552,11 +552,13 @@ struct irq_domain *pci_msi_create_irq_do u32 pci_msi_domain_get_msi_rid(struct irq_domain *domain, struct pci_dev *pdev); struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev); bool pci_dev_has_special_msi_domain(struct pci_dev *pdev); +bool pci_msi_domain_supports_expand(struct pci_dev *dev); #else static inline struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev) { return NULL; } +static inline bool pci_msi_domain_supports_expand(struct pci_dev *dev) { return false; } #endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */ #endif /* LINUX_MSI_H */ From patchwork Sat Nov 27 01:24:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12642011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0972C43217 for ; Sat, 27 Nov 2021 01:25:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233615.405553 (Exim 4.92) (envelope-from ) id 1mqmSN-0006i2-Sb; Sat, 27 Nov 2021 01:24:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233615.405553; Sat, 27 Nov 2021 01:24:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSN-0006ga-GM; Sat, 27 Nov 2021 01:24:47 +0000 Received: by outflank-mailman (input) for mailman id 233615; Sat, 27 Nov 2021 01:24:45 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSL-0003uw-JR for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:45 +0000 Received: from galois.linutronix.de (galois.linutronix.de [193.142.43.55]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id cd8ccc73-4f20-11ec-9787-a32c541c8605; Sat, 27 Nov 2021 02:24:44 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cd8ccc73-4f20-11ec-9787-a32c541c8605 Message-ID: <20211127000919.004572849@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=iXWx8ngjVm94paCMjL44qtncyzw3jmQTRYweVvwdUTE=; b=twc9E/k5sGMn2fquLHv5+7zfix6uLYi9RH9SGLZSSA3ABcOvbp0/gPOpQlesgzHkbj1UVu fxR6AoIcwrtz/nn3bycIRy8P9kAGaK0ECHfhbEqIdbz1o3vq7H1JdPjQaWggom9fcYLCdX BOmuzr15HeqHdEIDMKSLq/WavFlqpW/xYMSc3r2wN7DXzv+PNwO5cRgSi7yjS1wpb/86Ru gJmjSdRE0skAiQjFeQ3BBC/okRhBX5BOlbe4JLQChHLlOjJiIRqAKAnL9t8GF853DwnxyU l0iBSahv+YZe+ToUNkpkfYscK/wzeDb2wgYipZlnYU4lySHkxy2LdxVpk1XRIg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=iXWx8ngjVm94paCMjL44qtncyzw3jmQTRYweVvwdUTE=; b=Cwze/7Fqot4zxUe/5Tt8x3Rdu1Qo6xyHVQISU76a2p6ptM3z6Ii3pkLaG0Yznt5n2sNBh3 EqDN2aJVSQRqTdBA== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 09/10] PCI/MSI: Provide pci_msix_expand_vectors[_at]() References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:44 +0100 (CET) Provide a new interface which allows to expand the MSI-X vector space if the underlying irq domain implementation supports it. Signed-off-by: Thomas Gleixner --- drivers/pci/msi/msi.c | 41 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 13 +++++++++++++ 2 files changed, 54 insertions(+) --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -1025,6 +1025,47 @@ int pci_alloc_irq_vectors_affinity(struc EXPORT_SYMBOL(pci_alloc_irq_vectors_affinity); /** + * pci_msix_expand_vectors_at - Expand MSI-X interrupts for a device + * + * @dev: PCI device to operate on + * @at: Allocate at MSI-X index. If @at == PCI_MSI_EXPAND_AUTO + * the function expands automatically after the last + * active index. + * @nvec: Number of vectors to allocate + * + * Expand the MSI-X vectors of a device after an initial enablement and + * allocation. + * + * Return: 0 if the allocation was successful, an error code otherwise. + */ +int pci_msix_expand_vectors_at(struct pci_dev *dev, unsigned int at, unsigned int nvec) +{ + struct msi_device_data *md = dev->dev.msi.data; + struct msi_range range = { .ndesc = nvec, }; + unsigned int max_vecs; + int ret; + + if (!pci_msi_enable || !dev || !dev->msix_enabled || !md) + return -ENOTSUPP; + + if (!pci_msi_domain_supports_expand(dev)) + return -ENOTSUPP; + + max_vecs = pci_msix_vec_count(dev); + if (!nvec || nvec > max_vecs) + return -EINVAL; + + range.first = at == PCI_MSIX_EXPAND_AUTO ? md->num_descs : at; + + if (range.first >= max_vecs || nvec > max_vecs - range.first) + return -ENOSPC; + + ret = msix_setup_interrupts(dev, dev->msix_base, &range, NULL, NULL, true); + return ret <= 0 ? ret : -ENOSPC;; +} +EXPORT_SYMBOL_GPL(pci_msix_expand_vectors_at); + +/** * pci_free_irq_vectors - free previously allocated IRQs for a device * @dev: PCI device to operate on * --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1534,6 +1534,7 @@ static inline int pci_enable_msix_exact( int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs, unsigned int max_vecs, unsigned int flags, struct irq_affinity *affd); +int pci_msix_expand_vectors_at(struct pci_dev *dev, unsigned int at, unsigned int nvec); void pci_free_irq_vectors(struct pci_dev *dev); int pci_irq_vector(struct pci_dev *dev, unsigned int nr); @@ -1565,6 +1566,11 @@ pci_alloc_irq_vectors_affinity(struct pc return -ENOSPC; } +static inline int pci_msix_expand_vectors_at(struct pci_dev *dev, unsigned int at, unsigned int nvec) +{ + return -ENOTSUPP; +} + static inline void pci_free_irq_vectors(struct pci_dev *dev) { } @@ -1582,6 +1588,13 @@ static inline const struct cpumask *pci_ } #endif +#define PCI_MSIX_EXPAND_AUTO (UINT_MAX) + +static inline int pci_msix_expand_vectors(struct pci_dev *dev, unsigned int nvec) +{ + return pci_msix_expand_vectors_at(dev, PCI_MSIX_EXPAND_AUTO, nvec); +} + /** * pci_irqd_intx_xlate() - Translate PCI INTx value to an IRQ domain hwirq * @d: the INTx IRQ domain From patchwork Sat Nov 27 01:24:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 12642013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7AE4C433EF for ; Sat, 27 Nov 2021 01:25:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.233616.405562 (Exim 4.92) (envelope-from ) id 1mqmSP-000766-Hd; Sat, 27 Nov 2021 01:24:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 233616.405562; Sat, 27 Nov 2021 01:24:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSP-00073z-83; Sat, 27 Nov 2021 01:24:49 +0000 Received: by outflank-mailman (input) for mailman id 233616; Sat, 27 Nov 2021 01:24:47 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mqmSN-0004gG-82 for xen-devel@lists.xenproject.org; Sat, 27 Nov 2021 01:24:47 +0000 Received: from galois.linutronix.de (galois.linutronix.de [2a0a:51c0:0:12e:550::1]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ce9696c0-4f20-11ec-b941-1df2895da90e; Sat, 27 Nov 2021 02:24:46 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ce9696c0-4f20-11ec-b941-1df2895da90e Message-ID: <20211127000919.061417652@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1637976286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=hCvJ3RXlnaO1z2W7zc0iNOJLM+aNENiBxAWv9ZJQoBs=; b=XC2Gl2AXYUosg2kIFrKUz3oPHKElC1X9JFf4x3rxNOfcEV8Sq7ff8QkEBIiBuBsTfaHNCM 65bhXgRy5LZ9CnQBXg7GbCNI4vggc91+DCTEERsiANj0TGuDaiFlcTbB0M47vNHEjihjSW 8DA00mBgmlvScWnWuX/6xFMQMcL0n5mNIlPxb5zZocO3pbdWx0Ofy1wQIIEqhY38Jgc0AN q500czOHWsLH6192qrfut6JygO3G0UlybO6Qtx41PQKuS/ypTmzA4msGtxAX4YLtPpouxb /3SyS/cHVyI76sDvXyavx8EUa0STgNuiEnp8ZEVMmX/DZiNkTIl1EM8TAZq2cQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1637976286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=hCvJ3RXlnaO1z2W7zc0iNOJLM+aNENiBxAWv9ZJQoBs=; b=ailFWK1MoN7Bi6dOYGgm53UBim2lp4UpqheAs+DlCTXZcPa4k+6E5dnX9FNo8gNNMKECaq WAcA1zeyb1QOLQAQ== From: Thomas Gleixner To: LKML Cc: Bjorn Helgaas , Marc Zygnier , Alex Williamson , Kevin Tian , Jason Gunthorpe , Megha Dey , Ashok Raj , Michael Ellerman , Andrew Cooper , Juergen Gross , linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [patch 10/10] x86/apic/msi: Support MSI-X vector expansion References: <20211126233124.618283684@linutronix.de> MIME-Version: 1.0 Date: Sat, 27 Nov 2021 02:24:45 +0100 (CET) The X86 PCI/MSI irq domaim implementation supports vector expansion out of the box. Make it available. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/apic/msi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -178,7 +178,7 @@ static struct msi_domain_ops pci_msi_dom static struct msi_domain_info pci_msi_domain_info = { .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_PCI_MSIX, + MSI_FLAG_PCI_MSIX | MSI_FLAG_CAN_EXPAND, .ops = &pci_msi_domain_ops, .chip = &pci_msi_controller, .handler = handle_edge_irq, @@ -226,7 +226,7 @@ static struct irq_chip pci_msi_ir_contro static struct msi_domain_info pci_msi_ir_domain_info = { .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX | MSI_FLAG_CAN_EXPAND, .ops = &pci_msi_domain_ops, .chip = &pci_msi_ir_controller, .handler = handle_edge_irq,