From patchwork Wed Feb 3 20:56:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2519AC433E0 for ; Wed, 3 Feb 2021 20:58:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB8C164F51 for ; Wed, 3 Feb 2021 20:58:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232795AbhBCU6P (ORCPT ); Wed, 3 Feb 2021 15:58:15 -0500 Received: from mga01.intel.com ([192.55.52.88]:12604 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232784AbhBCU6K (ORCPT ); Wed, 3 Feb 2021 15:58:10 -0500 IronPort-SDR: K21N+YgWdOY/YDXmcWfNW+1mtjdAo7tYF5zXgp/gVvjbMLG3YmtJI5kkS2jEVOQz2FUrnf/WXM CGctj5SPTNQQ== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="200084222" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="200084222" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:28 -0800 IronPort-SDR: K26s3NvHcap36eZnRUHuzYw+PL6N+gSX1FVIK6Zy3LajbF9ZmnRbBLhVGu6pzRDCSlak4Uww72 FqbIe5f0XY0A== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510543" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:28 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 01/12] x86/irq: Add DEV_MSI allocation type Date: Wed, 3 Feb 2021 12:56:34 -0800 Message-Id: <1612385805-3412-2-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner For the upcoming device MSI support a new allocation type is required. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- arch/x86/include/asm/hw_irq.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/include/asm/hw_irq.h b/arch/x86/include/asm/hw_irq.h index d465ece..0531b9c 100644 --- a/arch/x86/include/asm/hw_irq.h +++ b/arch/x86/include/asm/hw_irq.h @@ -41,6 +41,7 @@ enum irq_alloc_type { X86_IRQ_ALLOC_TYPE_DMAR, X86_IRQ_ALLOC_TYPE_AMDVI, X86_IRQ_ALLOC_TYPE_UV, + X86_IRQ_ALLOC_TYPE_DEV_MSI, }; struct ioapic_alloc_info { From patchwork Wed Feb 3 20:56:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB009C433E0 for ; Wed, 3 Feb 2021 20:59:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71F0C64F51 for ; Wed, 3 Feb 2021 20:59:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232856AbhBCU6q (ORCPT ); Wed, 3 Feb 2021 15:58:46 -0500 Received: from mga01.intel.com ([192.55.52.88]:12600 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231215AbhBCU63 (ORCPT ); Wed, 3 Feb 2021 15:58:29 -0500 IronPort-SDR: ySNOpoul3ISTM9r15ibGPiOlK2RQ8Kbr0pFRuX3CYep34Ep39HarwPmwx853yC1nEzQJldwnxl pQXyejo0jbsw== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="200084225" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="200084225" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:28 -0800 IronPort-SDR: VhbifczPcXhDwHkb/ZHA/S9Oq+urvMYrbvav4fHs8Grqtb1TY7uqLIePZwzijvKOn62Hn9xw9k TuLfQiddyJ2Q== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510548" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:28 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 02/12] x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI Date: Wed, 3 Feb 2021 12:56:35 -0800 Message-Id: <1612385805-3412-3-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner Rename it to x86_msi_prepare() and handle the allocation type setup depending on the device type. Add a new arch_msi_prepare define which will be utilized by the upcoming device MSI support. Define it to NULL if not provided by an architecture in the generic MSI header. One arch specific function for MSI support is truly enough. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- arch/x86/include/asm/msi.h | 4 +++- arch/x86/kernel/apic/msi.c | 27 ++++++++++++++++++++------- drivers/pci/controller/pci-hyperv.c | 2 +- include/linux/msi.h | 4 ++++ 4 files changed, 28 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/msi.h b/arch/x86/include/asm/msi.h index b85147d..9bd214e 100644 --- a/arch/x86/include/asm/msi.h +++ b/arch/x86/include/asm/msi.h @@ -6,9 +6,11 @@ typedef struct irq_alloc_info msi_alloc_info_t; -int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, +int x86_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg); +#define arch_msi_prepare x86_msi_prepare + /* Structs and defines for the X86 specific MSI message format */ typedef struct x86_msi_data { diff --git a/arch/x86/kernel/apic/msi.c b/arch/x86/kernel/apic/msi.c index 44ebe25..84b16c7 100644 --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -153,26 +153,39 @@ static struct irq_chip pci_msi_controller = { .flags = IRQCHIP_SKIP_SET_WAKE, }; -int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, - msi_alloc_info_t *arg) +static void pci_msi_prepare(struct device *dev, msi_alloc_info_t *arg) { - struct pci_dev *pdev = to_pci_dev(dev); - struct msi_desc *desc = first_pci_msi_entry(pdev); + struct msi_desc *desc = first_msi_entry(dev); - init_irq_alloc_info(arg, NULL); if (desc->msi_attrib.is_msix) { arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSIX; } else { arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSI; arg->flags |= X86_IRQ_ALLOC_CONTIGUOUS_VECTORS; } +} + +static void dev_msi_prepare(struct device *dev, msi_alloc_info_t *arg) +{ + arg->type = X86_IRQ_ALLOC_TYPE_DEV_MSI; +} + +int x86_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, + msi_alloc_info_t *arg) +{ + init_irq_alloc_info(arg, NULL); + + if (dev_is_pci(dev)) + pci_msi_prepare(dev, arg); + else + dev_msi_prepare(dev, arg); return 0; } -EXPORT_SYMBOL_GPL(pci_msi_prepare); +EXPORT_SYMBOL_GPL(x86_msi_prepare); static struct msi_domain_ops pci_msi_domain_ops = { - .msi_prepare = pci_msi_prepare, + .msi_prepare = x86_msi_prepare, }; static struct msi_domain_info pci_msi_domain_info = { diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 6db8d96..bfb47c2 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -1546,7 +1546,7 @@ static struct irq_chip hv_msi_irq_chip = { }; static struct msi_domain_ops hv_msi_ops = { - .msi_prepare = pci_msi_prepare, + .msi_prepare = arch_msi_prepare, .msi_free = hv_msi_free, }; diff --git a/include/linux/msi.h b/include/linux/msi.h index 360a0a7..89acc76 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -467,4 +467,8 @@ static inline struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev) } #endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */ +#ifndef arch_msi_prepare +# define arch_msi_prepare NULL +#endif + #endif /* LINUX_MSI_H */ From patchwork Wed Feb 3 20:56:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FB07C433E6 for ; Wed, 3 Feb 2021 20:59:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4FC9E64F6C for ; Wed, 3 Feb 2021 20:59:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232460AbhBCU6e (ORCPT ); Wed, 3 Feb 2021 15:58:34 -0500 Received: from mga01.intel.com ([192.55.52.88]:12604 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232713AbhBCU63 (ORCPT ); Wed, 3 Feb 2021 15:58:29 -0500 IronPort-SDR: NDRdotet4lCdBU3Myk/xvuoSsP1r1PY/RBTkdC2+tIkEI2KNYH9eIMirO3sZ+Z09lXOKYhsVeL oUAcIhDxMkgw== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="200084227" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="200084227" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:29 -0800 IronPort-SDR: vEsPwWDwRp/bB9vfE/RdJnwAKAqrjj0TGiGqfUE3tQfYHLLcWnbKyzD3O+7yTc1u8ecIHu2bMA A95QSLTBqfIg== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510552" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:28 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 03/12] platform-msi: Provide default irq_chip:: Ack Date: Wed, 3 Feb 2021 12:56:36 -0800 Message-Id: <1612385805-3412-4-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner For the upcoming device MSI support it's required to have a default irq_chip::ack implementation (irq_chip_ack_parent) so the drivers do not need to care. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- drivers/base/platform-msi.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c index 2c1e2e0..9d9ccfc 100644 --- a/drivers/base/platform-msi.c +++ b/drivers/base/platform-msi.c @@ -101,6 +101,8 @@ static void platform_msi_update_chip_ops(struct msi_domain_info *info) chip->irq_mask = irq_chip_mask_parent; if (!chip->irq_unmask) chip->irq_unmask = irq_chip_unmask_parent; + if (!chip->irq_ack) + chip->irq_ack = irq_chip_ack_parent; if (!chip->irq_eoi) chip->irq_eoi = irq_chip_eoi_parent; if (!chip->irq_set_affinity) From patchwork Wed Feb 3 20:56:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44BBFC433E0 for ; Wed, 3 Feb 2021 21:02:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E54764F65 for ; Wed, 3 Feb 2021 21:02:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232573AbhBCVBy (ORCPT ); Wed, 3 Feb 2021 16:01:54 -0500 Received: from mga01.intel.com ([192.55.52.88]:12604 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232852AbhBCU6q (ORCPT ); Wed, 3 Feb 2021 15:58:46 -0500 IronPort-SDR: v1D1CfHrryRk8k9lyEPtzuGeiFgs8eZGaeOlk2+3AflrDm80gAzZAr1FN8MQ/5XkI6rIFssBb1 Wz2rYPwXdFOw== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="200084229" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="200084229" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:29 -0800 IronPort-SDR: 4XvBfjP/iQqbBWLZ2szBUV+ObDD33PykBEc1uG1b8bXLvYgFtaa6445QuPrkFnLgTfY8JojNhn y11wMRRTP6ng== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510555" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:29 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 04/12] genirq/proc: Take buslock on affinity write Date: Wed, 3 Feb 2021 12:56:37 -0800 Message-Id: <1612385805-3412-5-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner Until now interrupt chips which support setting affinity are not locking the associated bus lock for two reasons: - All chips which support affinity setting do not use buslock because they just can operated directly on the hardware. - All chips which use buslock do not support affinity setting because their interrupt chips are not capable. These chips are usually connected over a bus like I2C, SPI etc. and have an interrupt output which is conneted to CPU interrupt of some sort. So there is no way to set the affinity on the chip itself. Upcoming hardware which is PCIE based sports a non standard MSI(X) variant which stores the MSI message in RAM which is associated to e.g. a device queue. The device manages this RAM and writes have to be issued via command queues or similar mechanisms which is obviously not possible from interrupt disabled, raw spinlock held context. The buslock mechanism of irq chips can be utilized to support that. The affinity write to the chip writes to shadow state, marks it pending and the irq chip's irq_bus_sync_unlock() callback handles the command queue and wait for completion similar to the other chip operations on I2C or SPI busses. Change the locking in irq_set_affinity() to bus_lock/unlock to help with that. There are a few other callers than the proc interface, but none of them is affected by this change as none of them affects an irq chip with bus lock support. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- kernel/irq/manage.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index dec3f73..85ede4e 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -443,16 +443,16 @@ int irq_update_affinity_desc(unsigned int irq, int __irq_set_affinity(unsigned int irq, const struct cpumask *mask, bool force) { - struct irq_desc *desc = irq_to_desc(irq); + struct irq_desc *desc; unsigned long flags; int ret; + desc = irq_get_desc_buslock(irq, &flags, IRQ_GET_DESC_CHECK_GLOBAL); if (!desc) return -EINVAL; - raw_spin_lock_irqsave(&desc->lock, flags); ret = irq_set_affinity_locked(irq_desc_get_irq_data(desc), mask, force); - raw_spin_unlock_irqrestore(&desc->lock, flags); + irq_put_desc_busunlock(desc, flags); return ret; } From patchwork Wed Feb 3 20:56:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 592EFC4332B for ; Wed, 3 Feb 2021 21:03:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 256AC64F6C for ; Wed, 3 Feb 2021 21:03:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232984AbhBCVDC (ORCPT ); Wed, 3 Feb 2021 16:03:02 -0500 Received: from mga07.intel.com ([134.134.136.100]:45277 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232311AbhBCU6O (ORCPT ); Wed, 3 Feb 2021 15:58:14 -0500 IronPort-SDR: HoVvMG4FJdxupcxn9AwxD06LcAq4Pd7TJp2JDaphAS8j+iN85rVOYuLXPBTIPdrcNjV+HQQbac LgbK2qBcTZNA== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191295" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191295" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:29 -0800 IronPort-SDR: nXgXuUUuY2ynH8GS8K/ua4Ou6tXdiK2QJjlN9r8Ghg07cXafWhsE5TtVqUVyq1a6xCrsEOuBhM VJXZqcvzNqCg== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510561" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:29 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 05/12] genirq/msi: Provide and use msi_domain_set_default_info_flags() Date: Wed, 3 Feb 2021 12:56:38 -0800 Message-Id: <1612385805-3412-6-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner MSI interrupts have some common flags which should be set not only for PCI/MSI interrupts. Move the PCI/MSI flag setting into a common function so it can be reused. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- drivers/pci/msi.c | 7 +------ include/linux/msi.h | 1 + kernel/irq/msi.c | 24 ++++++++++++++++++++++++ 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c index 3162f88..20d2512 100644 --- a/drivers/pci/msi.c +++ b/drivers/pci/msi.c @@ -1492,12 +1492,7 @@ struct irq_domain *pci_msi_create_irq_domain(struct fwnode_handle *fwnode, if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS) pci_msi_domain_update_chip_ops(info); - info->flags |= MSI_FLAG_ACTIVATE_EARLY; - if (IS_ENABLED(CONFIG_GENERIC_IRQ_RESERVATION_MODE)) - info->flags |= MSI_FLAG_MUST_REACTIVATE; - - /* PCI-MSI is oneshot-safe */ - info->chip->flags |= IRQCHIP_ONESHOT_SAFE; + msi_domain_set_default_info_flags(info); domain = msi_create_irq_domain(fwnode, info, parent); if (!domain) diff --git a/include/linux/msi.h b/include/linux/msi.h index 89acc76..d7a7f7d 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -448,6 +448,7 @@ int platform_msi_domain_alloc(struct irq_domain *domain, unsigned int virq, void platform_msi_domain_free(struct irq_domain *domain, unsigned int virq, unsigned int nvec); void *platform_msi_get_host_data(struct irq_domain *domain); +void msi_domain_set_default_info_flags(struct msi_domain_info *info); #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */ #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index dc0e2d7..3697909 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -70,6 +70,30 @@ void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg) EXPORT_SYMBOL_GPL(get_cached_msi_msg); #ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN +void msi_domain_set_default_info_flags(struct msi_domain_info *info) +{ + /* Required so that a device latches a valid MSI message on startup */ + info->flags |= MSI_FLAG_ACTIVATE_EARLY; + + /* + * Interrupt reservation mode allows to stear the MSI message of an + * inactive device to a special (usually spurious interrupt) target. + * This allows to prevent interrupt vector exhaustion e.g. on x86. + * But (PCI)MSI interrupts are activated early - see above - so the + * interrupt request/startup sequence would not try to allocate a + * usable vector which means that the device interrupts would end + * up on the special vector and issue spurious interrupt messages. + * Setting the reactivation flag ensures that when the interrupt + * is requested the activation is invoked again so that a real + * vector can be allocated. + */ + if (IS_ENABLED(CONFIG_GENERIC_IRQ_RESERVATION_MODE)) + info->flags |= MSI_FLAG_MUST_REACTIVATE; + + /* MSI is oneshot-safe at least in theory */ + info->chip->flags |= IRQCHIP_ONESHOT_SAFE; +} + static inline void irq_chip_write_msi_msg(struct irq_data *data, struct msi_msg *msg) { From patchwork Wed Feb 3 20:56:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC47AC43381 for ; Wed, 3 Feb 2021 21:03:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D6BE64F43 for ; Wed, 3 Feb 2021 21:03:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232859AbhBCVDB (ORCPT ); Wed, 3 Feb 2021 16:03:01 -0500 Received: from mga07.intel.com ([134.134.136.100]:45270 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232792AbhBCU6O (ORCPT ); Wed, 3 Feb 2021 15:58:14 -0500 IronPort-SDR: Yhal+xDrb36SOPmvHZEBK6Go741snErnQIfAC/OuTXeMIL0hQovZWtUphnB4roAziTipKY1NgZ i+aFuBiNjyug== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191297" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191297" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:30 -0800 IronPort-SDR: ES1Xf9ZIwk5IsL/BSza/sEsBAYTEtSL1hblroMmVJ9xn7G6oGgeA2QgbOVHLpecKZo7Kt9/4Xv ea6lX64WRRKA== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510567" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:29 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 06/12] platform-msi: Add device MSI infrastructure Date: Wed, 3 Feb 2021 12:56:39 -0800 Message-Id: <1612385805-3412-7-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner Add device specific MSI domain infrastructure for devices which have their own resource management and interrupt chip. These devices are not related to PCI and contrary to platform MSI they do not share a common resource and interrupt chip. They provide their own domain specific resource management and interrupt chip. This utilizes the new alloc/free override in a non evil way which avoids having yet another set of specialized alloc/free functions. Just using msi_domain_alloc/free_irqs() is sufficient While initially it was suggested and tried to piggyback device MSI on platform MSI, the better variant is to reimplement platform MSI on top of device MSI. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- drivers/base/platform-msi.c | 131 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/irqdomain.h | 1 + include/linux/msi.h | 24 ++++++++ kernel/irq/Kconfig | 4 ++ 4 files changed, 160 insertions(+) diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c index 9d9ccfc..6127b3b 100644 --- a/drivers/base/platform-msi.c +++ b/drivers/base/platform-msi.c @@ -419,3 +419,134 @@ int platform_msi_domain_alloc(struct irq_domain *domain, unsigned int virq, return err; } + +#ifdef CONFIG_DEVICE_MSI +/* + * Device specific MSI domain infrastructure for devices which have their + * own resource management and interrupt chip. These devices are not + * related to PCI and contrary to platform MSI they do not share a common + * resource and interrupt chip. They provide their own domain specific + * resource management and interrupt chip. + */ + +static void device_msi_free_msi_entries(struct device *dev) +{ + struct list_head *msi_list = dev_to_msi_list(dev); + struct msi_desc *entry, *tmp; + + list_for_each_entry_safe(entry, tmp, msi_list, list) { + list_del(&entry->list); + free_msi_entry(entry); + } +} + +/** + * device_msi_free_irqs - Free MSI interrupts assigned to a device + * @dev: Pointer to the device + * + * Frees the interrupt and the MSI descriptors. + */ +static void device_msi_free_irqs(struct irq_domain *domain, struct device *dev) +{ + __msi_domain_free_irqs(domain, dev); + device_msi_free_msi_entries(dev); +} + +/** + * device_msi_alloc_irqs - Allocate MSI interrupts for a device + * @dev: Pointer to the device + * @nvec: Number of vectors + * + * Allocates the required number of MSI descriptors and the corresponding + * interrupt descriptors. + */ +static int device_msi_alloc_irqs(struct irq_domain *domain, struct device *dev, int nvec) +{ + int i, ret = -ENOMEM; + + for (i = 0; i < nvec; i++) { + struct msi_desc *entry = alloc_msi_entry(dev, 1, NULL); + + if (!entry) + goto fail; + list_add_tail(&entry->list, dev_to_msi_list(dev)); + } + + ret = __msi_domain_alloc_irqs(domain, dev, nvec); + if (!ret) + return 0; +fail: + device_msi_free_msi_entries(dev); + return ret; +} + +static void device_msi_update_dom_ops(struct msi_domain_info *info) +{ + if (!info->ops->domain_alloc_irqs) + info->ops->domain_alloc_irqs = device_msi_alloc_irqs; + if (!info->ops->domain_free_irqs) + info->ops->domain_free_irqs = device_msi_free_irqs; + if (!info->ops->msi_prepare) + info->ops->msi_prepare = arch_msi_prepare; +} + +/** + * device_msi_create_msi_irq_domain - Create an irq domain for devices + * @fwnode: Firmware node of the interrupt controller + * @info: MSI domain info to configure the new domain + * @parent: Parent domain + */ +struct irq_domain *device_msi_create_irq_domain(struct fwnode_handle *fn, + struct msi_domain_info *info, + struct irq_domain *parent) +{ + struct irq_domain *domain; + + if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS) + platform_msi_update_chip_ops(info); + + if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS) + device_msi_update_dom_ops(info); + + msi_domain_set_default_info_flags(info); + + domain = msi_create_irq_domain(fn, info, parent); + if (domain) + irq_domain_update_bus_token(domain, DOMAIN_BUS_DEVICE_MSI); + return domain; +} + +#ifdef CONFIG_PCI +#include + +/** + * pci_subdevice_msi_create_irq_domain - Create an irq domain for subdevices + * @pdev: Pointer to PCI device for which the subdevice domain is created + * @info: MSI domain info to configure the new domain + */ +struct irq_domain *pci_subdevice_msi_create_irq_domain(struct pci_dev *pdev, + struct msi_domain_info *info) +{ + struct irq_domain *domain, *pdev_msi; + struct fwnode_handle *fn; + + /* + * Retrieve the MSI domain of the underlying PCI device's MSI + * domain. The PCI device domain's parent domain is also the parent + * domain of the new subdevice domain. + */ + pdev_msi = dev_get_msi_domain(&pdev->dev); + if (!pdev_msi) + return NULL; + + fn = irq_domain_alloc_named_fwnode(dev_name(&pdev->dev)); + if (!fn) + return NULL; + domain = device_msi_create_irq_domain(fn, info, pdev_msi->parent); + if (!domain) + irq_domain_free_fwnode(fn); + return domain; +} +EXPORT_SYMBOL_GPL(pci_subdevice_msi_create_irq_domain); +#endif /* CONFIG_PCI */ +#endif /* CONFIG_DEVICE_MSI */ diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h index 42d1968..06c88ba 100644 --- a/include/linux/irqdomain.h +++ b/include/linux/irqdomain.h @@ -85,6 +85,7 @@ enum irq_domain_bus_token { DOMAIN_BUS_TI_SCI_INTA_MSI, DOMAIN_BUS_WAKEUP, DOMAIN_BUS_VMD_MSI, + DOMAIN_BUS_DEVICE_MSI, }; /** diff --git a/include/linux/msi.h b/include/linux/msi.h index d7a7f7d..fbf2258 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -95,6 +95,18 @@ struct ti_sci_inta_msi_desc { }; /** + * device_msi_desc - Device MSI specific MSI descriptor data + * @priv: Pointer to device specific private data + * @priv_iomem: Pointer to device specific private io memory + * @hwirq: The hardware irq number in the device domain + */ +struct device_msi_desc { + void *priv; + void __iomem *priv_iomem; + u16 hwirq; +}; + +/** * struct msi_desc - Descriptor structure for MSI based interrupts * @list: List head for management * @irq: The base interrupt number @@ -166,6 +178,7 @@ struct msi_desc { struct platform_msi_desc platform; struct fsl_mc_msi_desc fsl_mc; struct ti_sci_inta_msi_desc inta; + struct device_msi_desc device_msi; }; }; @@ -451,6 +464,17 @@ void *platform_msi_get_host_data(struct irq_domain *domain); void msi_domain_set_default_info_flags(struct msi_domain_info *info); #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */ +#ifdef CONFIG_DEVICE_MSI +struct irq_domain *device_msi_create_irq_domain(struct fwnode_handle *fn, + struct msi_domain_info *info, + struct irq_domain *parent); + +# ifdef CONFIG_PCI +struct irq_domain *pci_subdevice_msi_create_irq_domain(struct pci_dev *pdev, + struct msi_domain_info *info); +# endif +#endif /* CONFIG_DEVICE_MSI */ + #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg); struct irq_domain *pci_msi_create_irq_domain(struct fwnode_handle *fwnode, diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig index d79ef24..7223327 100644 --- a/kernel/irq/Kconfig +++ b/kernel/irq/Kconfig @@ -89,6 +89,10 @@ config GENERIC_MSI_IRQ_DOMAIN select IRQ_DOMAIN_HIERARCHY select GENERIC_MSI_IRQ +config DEVICE_MSI + bool + select GENERIC_MSI_IRQ_DOMAIN + config IRQ_MSI_IOMMU bool From patchwork Wed Feb 3 20:56:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70861C433DB for ; Wed, 3 Feb 2021 21:02:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4C59764F72 for ; Wed, 3 Feb 2021 21:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232268AbhBCVCY (ORCPT ); Wed, 3 Feb 2021 16:02:24 -0500 Received: from mga07.intel.com ([134.134.136.100]:45277 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232827AbhBCU6b (ORCPT ); Wed, 3 Feb 2021 15:58:31 -0500 IronPort-SDR: d9tGYLYcFJeWmWZ1FWwK4UNFtoKcTPVYbp0VLiCvVQHyBY7OBPCG3zoMJVfXWaDMtVzB/Ri+ai sjxbhKI5QGgA== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191304" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191304" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:30 -0800 IronPort-SDR: lDo9vV/MuzOHHPofxqRXtGbAdWNRgvbAUKvbBGjO753BaEOXV7JitXKSTe0CQfbNmOqU4s3e7x fwWzf2myIIFg== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510573" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:30 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 07/12] irqdomain/msi: Provide msi_alloc/free_store() callbacks Date: Wed, 3 Feb 2021 12:56:40 -0800 Message-Id: <1612385805-3412-8-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner For devices which don't have a standard storage for MSI messages like the upcoming IMS (Interrupt Message Store) it's required to allocate storage space before allocating interrupts and after freeing them. This could be achieved with the existing callbacks, but that would be awkward because they operate on msi_alloc_info_t which is not uniform across architectures. Also these callbacks are invoked per interrupt but the allocation might have bulk requirements depending on the device. As such devices can operate on different architectures it is simpler to have separate callbacks which operate on struct device. The resulting storage information has to be stored in struct msi_desc so the underlying irq chip implementation can retrieve it for the relevant operations. Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- include/linux/msi.h | 8 ++++++++ kernel/irq/msi.c | 11 +++++++++++ 2 files changed, 19 insertions(+) diff --git a/include/linux/msi.h b/include/linux/msi.h index fbf2258..a6b419d 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -317,6 +317,10 @@ struct msi_domain_info; * function. * @domain_free_irqs: Optional function to override the default free * function. + * @msi_alloc_store: Optional callback to allocate storage in a device + * specific non-standard MSI store + * @msi_alloc_free: Optional callback to free storage in a device + * specific non-standard MSI store * * @get_hwirq, @msi_init and @msi_free are callbacks used by * msi_create_irq_domain() and related interfaces @@ -366,6 +370,10 @@ struct msi_domain_ops { struct device *dev, int nvec); void (*domain_free_irqs)(struct irq_domain *domain, struct device *dev); + int (*msi_alloc_store)(struct irq_domain *domain, + struct device *dev, int nvec); + void (*msi_free_store)(struct irq_domain *domain, + struct device *dev); }; /** diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 3697909..d70d92e 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -434,6 +434,12 @@ int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, if (ret) return ret; + if (ops->msi_alloc_store) { + ret = ops->msi_alloc_store(domain, dev, nvec); + if (ret) + return ret; + } + for_each_msi_entry(desc, dev) { ops->set_desc(&arg, desc); @@ -533,6 +539,8 @@ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev) { + struct msi_domain_info *info = domain->host_data; + struct msi_domain_ops *ops = info->ops; struct msi_desc *desc; for_each_msi_entry(desc, dev) { @@ -546,6 +554,9 @@ void __msi_domain_free_irqs(struct irq_domain *domain, struct device *dev) desc->irq = 0; } } + + if (ops->msi_free_store) + ops->msi_free_store(domain, dev); } /** From patchwork Wed Feb 3 20:56:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81006C433E6 for ; Wed, 3 Feb 2021 21:03:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4346C64F6C for ; Wed, 3 Feb 2021 21:03:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232335AbhBCVCV (ORCPT ); Wed, 3 Feb 2021 16:02:21 -0500 Received: from mga07.intel.com ([134.134.136.100]:45270 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232831AbhBCU6c (ORCPT ); Wed, 3 Feb 2021 15:58:32 -0500 IronPort-SDR: bMzPtbJcVsXP0h/g+lR/99OeqtXolQWog0IUvX0Bdi8a8yoCCVOCXx6sguKOgdOlsWrn/BJ44M eQHwVsT1fTrQ== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191306" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191306" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:30 -0800 IronPort-SDR: z24OHgspgXrmUufwOeutvV6tlHGf9al7sBOIJbNf8aGLHBQtLiLTIBb2vJCFLQOMgpXsLQm86z uHnTQ4xRdMlA== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510575" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:30 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 08/12] genirq: Set auxiliary data for an interrupt Date: Wed, 3 Feb 2021 12:56:41 -0800 Message-Id: <1612385805-3412-9-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Introduce a new function pointer in the irq_chip structure(irq_set_auxdata) which is responsible for updating data which is stored in a shared register or data storage. For example, the idxd driver uses the auxiliary data API to enable/set and disable PASID field that is in the IMS entry (introduced in a later patch) and that data are not typically present in MSI entry. Reviewed-by: Tony Luck Signed-off-by: Megha Dey --- include/linux/interrupt.h | 2 ++ include/linux/irq.h | 4 ++++ kernel/irq/manage.c | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index bb8ff90..d3f419b 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -496,6 +496,8 @@ extern int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which, extern int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which, bool state); +int irq_set_auxdata(unsigned int irq, unsigned int which, u64 val); + #ifdef CONFIG_IRQ_FORCED_THREADING # ifdef CONFIG_PREEMPT_RT # define force_irqthreads (true) diff --git a/include/linux/irq.h b/include/linux/irq.h index 4aeb1c4..568cdf5 100644 --- a/include/linux/irq.h +++ b/include/linux/irq.h @@ -491,6 +491,8 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d) * irq_request_resources * @irq_compose_msi_msg: optional to compose message content for MSI * @irq_write_msi_msg: optional to write message content for MSI + * @irq_set_auxdata: Optional function to update auxiliary data e.g. in + * shared registers * @irq_get_irqchip_state: return the internal state of an interrupt * @irq_set_irqchip_state: set the internal state of a interrupt * @irq_set_vcpu_affinity: optional to target a vCPU in a virtual machine @@ -538,6 +540,8 @@ struct irq_chip { void (*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg); void (*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg); + int (*irq_set_auxdata)(struct irq_data *data, unsigned int which, u64 auxval); + int (*irq_get_irqchip_state)(struct irq_data *data, enum irqchip_irq_state which, bool *state); int (*irq_set_irqchip_state)(struct irq_data *data, enum irqchip_irq_state which, bool state); diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 85ede4e..68ff559 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -2860,3 +2860,35 @@ bool irq_check_status_bit(unsigned int irq, unsigned int bitmask) return res; } EXPORT_SYMBOL_GPL(irq_check_status_bit); + +/** + * irq_set_auxdata - Set auxiliary data + * @irq: Interrupt to update + * @which: Selector which data to update + * @auxval: Auxiliary data value + * + * Function to update auxiliary data for an interrupt, e.g. to update data + * which is stored in a shared register or data storage (e.g. IMS). + */ +int irq_set_auxdata(unsigned int irq, unsigned int which, u64 val) +{ + struct irq_desc *desc; + struct irq_data *data; + unsigned long flags; + int res = -ENODEV; + + desc = irq_get_desc_buslock(irq, &flags, 0); + if (!desc) + return -EINVAL; + + for (data = &desc->irq_data; data; data = irqd_get_parent_data(data)) { + if (data->chip->irq_set_auxdata) { + res = data->chip->irq_set_auxdata(data, which, val); + break; + } + } + + irq_put_desc_busunlock(desc, flags); + return res; +} +EXPORT_SYMBOL_GPL(irq_set_auxdata); From patchwork Wed Feb 3 20:56:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E5DDC433E0 for ; Wed, 3 Feb 2021 20:59:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4918064F7E for ; Wed, 3 Feb 2021 20:59:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232540AbhBCU7K (ORCPT ); Wed, 3 Feb 2021 15:59:10 -0500 Received: from mga07.intel.com ([134.134.136.100]:45328 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232932AbhBCU64 (ORCPT ); Wed, 3 Feb 2021 15:58:56 -0500 IronPort-SDR: OmYE1v2p6Q4ispbK4IuHMYofFrEAg6o6dh5exzZY+GecGk+fali4j3YNiHaiJDvfoLPgVC5FQ6 FNiEY2a4pOfg== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191309" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191309" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:31 -0800 IronPort-SDR: QZlpHv60LEgw1Tva4TVObVrye2zuMOFvuGuGwnX7Sj3M3Y5+jChKDshA8MTt8ME2eyNSHooii+ 3nLE0Ox0sqAA== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510580" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:30 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 09/12] iommu/vt-d: Add DEV-MSI support Date: Wed, 3 Feb 2021 12:56:42 -0800 Message-Id: <1612385805-3412-10-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add required support in the interrupt remapping driver for devices which generate dev-msi interrupts and use the intel remapping domain as the parent domain. Set the source-id of all dev-msi interrupt requests to the parent PCI device associated with it. Reviewed-by: Tony Luck Signed-off-by: Megha Dey --- drivers/iommu/intel/irq_remapping.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index 685200a..18f1b53 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -1278,6 +1278,9 @@ static void intel_irq_remapping_prepare_irte(struct intel_ir_data *data, case X86_IRQ_ALLOC_TYPE_PCI_MSIX: set_msi_sid(irte, msi_desc_to_pci_dev(info->desc)); break; + case X86_IRQ_ALLOC_TYPE_DEV_MSI: + set_msi_sid(irte, to_pci_dev(info->desc->dev->parent)); + break; default: BUG_ON(1); break; @@ -1321,7 +1324,8 @@ static int intel_irq_remapping_alloc(struct irq_domain *domain, if (!info || !iommu) return -EINVAL; if (nr_irqs > 1 && info->type != X86_IRQ_ALLOC_TYPE_PCI_MSI && - info->type != X86_IRQ_ALLOC_TYPE_PCI_MSIX) + info->type != X86_IRQ_ALLOC_TYPE_PCI_MSIX && + info->type != X86_IRQ_ALLOC_TYPE_DEV_MSI) return -EINVAL; /* From patchwork Wed Feb 3 20:56:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EE85C433DB for ; Wed, 3 Feb 2021 21:01:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DAEB164F92 for ; Wed, 3 Feb 2021 21:01:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232597AbhBCVAl (ORCPT ); Wed, 3 Feb 2021 16:00:41 -0500 Received: from mga07.intel.com ([134.134.136.100]:45277 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232604AbhBCU7G (ORCPT ); Wed, 3 Feb 2021 15:59:06 -0500 IronPort-SDR: 6NTX7Nzcyc2QiXy2esZJv4jLcBgvFIsx3TEw5L5b0EoFtZDh8j1ZWg5pdVttSN41T6Ihd4xKs/ 9ppsRWPIvcHw== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191311" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191311" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:32 -0800 IronPort-SDR: G3Slvai/j9HhtC7F3TgFBs5ouMaKBSpqdfeBf6+JEg0o30xilyMw1L8grccIH4csUS1oJnnYCn 60kXa5ZMaZFA== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510587" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:31 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 10/12] iommu: Add capability IOMMU_CAP_VIOMMU_HINT Date: Wed, 3 Feb 2021 12:56:43 -0800 Message-Id: <1612385805-3412-11-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Lu Baolu Some IOMMU specification defines some kind of hint mechanism, through which BIOS can imply that OS runs in a virtualized environment. For example, the caching mode defined in VT-d spec and NpCache capability defined in the AMD IOMMU specification. This hint could also be used outside of the IOMMU subsystem, where it could be used with other known means (CPUID, smbios) to sense whether Linux is running in a virtualized environment. Add a capability bit so that it could be used there. Signed-off-by: Lu Baolu Signed-off-by: Megha Dey --- drivers/iommu/amd/iommu.c | 2 ++ drivers/iommu/intel/iommu.c | 20 ++++++++++++++++++++ drivers/iommu/virtio-iommu.c | 9 +++++++++ include/linux/iommu.h | 2 ++ 4 files changed, 33 insertions(+) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index f0adbc4..a851f37 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -2646,6 +2646,8 @@ static bool amd_iommu_capable(enum iommu_cap cap) return (irq_remapping_enabled == 1); case IOMMU_CAP_NOEXEC: return false; + case IOMMU_CAP_VIOMMU_HINT: + return amd_iommu_np_cache; default: break; } diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 06b00b5..905d6aa 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -5094,12 +5094,32 @@ static inline bool nested_mode_support(void) return ret; } +static inline bool caching_mode_supported(void) +{ + struct dmar_drhd_unit *drhd; + struct intel_iommu *iommu; + bool ret = false; + + rcu_read_lock(); + for_each_active_iommu(iommu, drhd) { + if (cap_caching_mode(iommu->cap)) { + ret = true; + break; + } + } + rcu_read_unlock(); + + return ret; +} + static bool intel_iommu_capable(enum iommu_cap cap) { if (cap == IOMMU_CAP_CACHE_COHERENCY) return domain_update_iommu_snooping(NULL) == 1; if (cap == IOMMU_CAP_INTR_REMAP) return irq_remapping_enabled == 1; + if (cap == IOMMU_CAP_VIOMMU_HINT) + return caching_mode_supported(); return false; } diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c index 2bfdd57..e4941ca 100644 --- a/drivers/iommu/virtio-iommu.c +++ b/drivers/iommu/virtio-iommu.c @@ -931,7 +931,16 @@ static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args) return iommu_fwspec_add_ids(dev, args->args, 1); } +static bool viommu_capable(enum iommu_cap cap) +{ + if (cap == IOMMU_CAP_VIOMMU_HINT) + return true; + + return false; +} + static struct iommu_ops viommu_ops = { + .capable = viommu_capable, .domain_alloc = viommu_domain_alloc, .domain_free = viommu_domain_free, .attach_dev = viommu_attach_dev, diff --git a/include/linux/iommu.h b/include/linux/iommu.h index b3f0e20..5e62bcc 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -94,6 +94,8 @@ enum iommu_cap { transactions */ IOMMU_CAP_INTR_REMAP, /* IOMMU supports interrupt isolation */ IOMMU_CAP_NOEXEC, /* IOMMU_NOEXEC flag */ + IOMMU_CAP_VIOMMU_HINT, /* IOMMU can detect a hit for running in + VM */ }; /* From patchwork Wed Feb 3 20:56:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA490C433E9 for ; Wed, 3 Feb 2021 21:03:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 67E6F64F5F for ; Wed, 3 Feb 2021 21:03:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232681AbhBCVDA (ORCPT ); Wed, 3 Feb 2021 16:03:00 -0500 Received: from mga03.intel.com ([134.134.136.65]:47710 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232321AbhBCU6O (ORCPT ); Wed, 3 Feb 2021 15:58:14 -0500 IronPort-SDR: Mx4TZwhzE6zhEg/Myi6s2qaaTrU5h1/Gd54PVLrRBqeBCJJ2YhHjeyp3BRlZsiZIYDxp3ESzWG 9G/9BB56nptQ== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="181190095" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="181190095" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:32 -0800 IronPort-SDR: jF6G3NJzGgokGR6lD1hAvEKPFO/Fs2jX0KhiYC7l727N4wANLkAtlkN2UpSdny1JbD6Y9igX/z 65VuTxmCQpCw== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510590" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:31 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com, Leon Romanovsky Subject: [PATCH 11/12] platform-msi: Add platform check for subdevice irq domain Date: Wed, 3 Feb 2021 12:56:44 -0800 Message-Id: <1612385805-3412-12-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Lu Baolu The pci_subdevice_msi_create_irq_domain() should fail if the underlying platform is not able to support IMS (Interrupt Message Storage). Otherwise, the isolation of interrupt is not guaranteed. For x86, IMS is only supported on bare metal for now. We could enable it in the virtualization environments in the future if interrupt HYPERCALL domain is supported or the hardware has the capability of interrupt isolation for subdevices. Cc: David Woodhouse Cc: Leon Romanovsky Cc: Kevin Tian Suggested-by: Thomas Gleixner Link: https://lore.kernel.org/linux-pci/87pn4nk7nn.fsf@nanos.tec.linutronix.de/ Link: https://lore.kernel.org/linux-pci/877dqrnzr3.fsf@nanos.tec.linutronix.de/ Link: https://lore.kernel.org/linux-pci/877dqqmc2h.fsf@nanos.tec.linutronix.de/ Signed-off-by: Lu Baolu Signed-off-by: Megha Dey --- arch/x86/pci/common.c | 74 +++++++++++++++++++++++++++++++++++++++++++++ drivers/base/platform-msi.c | 8 +++++ include/linux/msi.h | 1 + 3 files changed, 83 insertions(+) diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c index 3507f45..263ccf6 100644 --- a/arch/x86/pci/common.c +++ b/arch/x86/pci/common.c @@ -12,6 +12,8 @@ #include #include #include +#include +#include #include #include @@ -724,3 +726,75 @@ struct pci_dev *pci_real_dma_dev(struct pci_dev *dev) return dev; } #endif + +#ifdef CONFIG_DEVICE_MSI +/* + * We want to figure out which context we are running in. But the hardware + * does not introduce a reliable way (instruction, CPUID leaf, MSR, whatever) + * which can be manipulated by the VMM to let the OS figure out where it runs. + * So we go with the below probably on_bare_metal() function as a replacement + * for definitely on_bare_metal() to go forward only for the very simple reason + * that this is the only option we have. + */ +static const char * const vmm_vendor_name[] = { + "QEMU", "Bochs", "KVM", "Xen", "VMware", "VMW", "VMware Inc.", + "innotek GmbH", "Oracle Corporation", "Parallels", "BHYVE" +}; + +static void read_type0_virtual_machine(const struct dmi_header *dm, void *p) +{ + u8 *data = (u8 *)dm + 0x13; + + /* BIOS Information (Type 0) */ + if (dm->type != 0 || dm->length < 0x14) + return; + + /* Bit 4 of BIOS Characteristics Extension Byte 2*/ + if (*data & BIT(4)) + *((bool *)p) = true; +} + +static bool smbios_virtual_machine(void) +{ + bool bit_present = false; + + dmi_walk(read_type0_virtual_machine, &bit_present); + + return bit_present; +} + +static bool on_bare_metal(struct device *dev) +{ + int i; + + if (boot_cpu_has(X86_FEATURE_HYPERVISOR)) + return false; + + if (smbios_virtual_machine()) + return false; + + if (iommu_capable(dev->bus, IOMMU_CAP_VIOMMU_HINT)) + return false; + + for (i = 0; i < ARRAY_SIZE(vmm_vendor_name); i++) + if (dmi_match(DMI_SYS_VENDOR, vmm_vendor_name[i])) + return false; + + pr_info("System running on bare metal, report to bugzilla.kernel.org if not the case."); + + return true; +} + +bool arch_support_pci_device_ims(struct pci_dev *pdev) +{ + /* + * When we are running in a VMM context, the device IMS could only be + * enabled when the underlying hardware supports interrupt isolation + * of the subdevice, or any mechanism (trap, hypercall) is added so + * that changes in the interrupt message store could be managed by the + * VMM. For now, we only support the device IMS when we are running on + * the bare metal. + */ + return on_bare_metal(&pdev->dev); +} +#endif diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c index 6127b3b..d5ae26f 100644 --- a/drivers/base/platform-msi.c +++ b/drivers/base/platform-msi.c @@ -519,6 +519,11 @@ struct irq_domain *device_msi_create_irq_domain(struct fwnode_handle *fn, #ifdef CONFIG_PCI #include +bool __weak arch_support_pci_device_ims(struct pci_dev *pdev) +{ + return false; +} + /** * pci_subdevice_msi_create_irq_domain - Create an irq domain for subdevices * @pdev: Pointer to PCI device for which the subdevice domain is created @@ -530,6 +535,9 @@ struct irq_domain *pci_subdevice_msi_create_irq_domain(struct pci_dev *pdev, struct irq_domain *domain, *pdev_msi; struct fwnode_handle *fn; + if (!arch_support_pci_device_ims(pdev)) + return NULL; + /* * Retrieve the MSI domain of the underlying PCI device's MSI * domain. The PCI device domain's parent domain is also the parent diff --git a/include/linux/msi.h b/include/linux/msi.h index a6b419d..fa02542 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -478,6 +478,7 @@ struct irq_domain *device_msi_create_irq_domain(struct fwnode_handle *fn, struct irq_domain *parent); # ifdef CONFIG_PCI +bool arch_support_pci_device_ims(struct pci_dev *pdev); struct irq_domain *pci_subdevice_msi_create_irq_domain(struct pci_dev *pdev, struct msi_domain_info *info); # endif From patchwork Wed Feb 3 20:56:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dey, Megha" X-Patchwork-Id: 12065327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3A7CC433E0 for ; Wed, 3 Feb 2021 21:01:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA4EC64F7E for ; Wed, 3 Feb 2021 21:01:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232326AbhBCVAi (ORCPT ); Wed, 3 Feb 2021 16:00:38 -0500 Received: from mga07.intel.com ([134.134.136.100]:45270 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232630AbhBCU7H (ORCPT ); Wed, 3 Feb 2021 15:59:07 -0500 IronPort-SDR: +1unLB2ms/8EQhTQbmczJmTb8o9tsn5HcUw/2bTl8z/xrOgMkjWcjHC22/XlXfajuUYb0n9z4c 2P2Bxl6T49Jw== X-IronPort-AV: E=McAfee;i="6000,8403,9884"; a="245191313" X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="245191313" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2021 12:57:32 -0800 IronPort-SDR: kNFrrJsVhMO3bLHNeasxEUDsFOnsxNGIbaNz4E9HyH4ot3x9SbSPJ7uy7Qs0AR6Aittcb//z7K mSmWxepxRLYg== X-IronPort-AV: E=Sophos;i="5.79,399,1602572400"; d="scan'208";a="372510593" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Feb 2021 12:57:32 -0800 From: Megha Dey To: tglx@linutronix.de Cc: linux-kernel@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, dwmw@amazon.co.uk, x86@kernel.org, tony.luck@intel.com, dan.j.williams@intel.com, megha.dey@intel.com, jgg@mellanox.com, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, bhelgaas@google.com, maz@kernel.org, linux-pci@vger.kernel.org, baolu.lu@linux.intel.com, ravi.v.shankar@intel.com Subject: [PATCH 12/12] irqchip: Add IMS (Interrupt Message Store) driver Date: Wed, 3 Feb 2021 12:56:45 -0800 Message-Id: <1612385805-3412-13-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1612385805-3412-1-git-send-email-megha.dey@intel.com> References: <1612385805-3412-1-git-send-email-megha.dey@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Thomas Gleixner Generic IMS(Interrupt Message Store) irq chips and irq domain implementations for IMS based devices which store the interrupt messages in an array in device memory. Allocation and freeing of interrupts happens via the generic msi_domain_alloc/free_irqs() interface. No special purpose IMS magic required as long as the interrupt domain is stored in the underlying device struct. The irq_set_auxdata() is used to program the pasid into the IMS entry. [Megha : Fixed compile time errors Added necessary dependencies to IMS_MSI_ARRAY config Fixed polarity of IMS_VECTOR_CTRL Added reads after writes to flush writes to device Added set_desc ops to IMS msi domain ops Tested the IMS infrastructure with the IDXD driver] Reviewed-by: Tony Luck Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- drivers/irqchip/Kconfig | 14 +++ drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-ims-msi.c | 211 ++++++++++++++++++++++++++++++++++++ include/linux/irqchip/irq-ims-msi.h | 68 ++++++++++++ 4 files changed, 294 insertions(+) create mode 100644 drivers/irqchip/irq-ims-msi.c create mode 100644 include/linux/irqchip/irq-ims-msi.h diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index b147f22..b50c821 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -590,4 +590,18 @@ config MST_IRQ help Support MStar Interrupt Controller. +config IMS_MSI + depends on PCI + select DEVICE_MSI + bool + +config IMS_MSI_ARRAY + bool "IMS Interrupt Message Store MSI controller for device memory storage arrays" + depends on PCI + select IMS_MSI + select GENERIC_MSI_IRQ_DOMAIN + help + Support for IMS Interrupt Message Store MSI controller + with IMS slot storage in a slot array in device memory + endmenu diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 0ac93bf..658a6bd 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -113,3 +113,4 @@ obj-$(CONFIG_LOONGSON_PCH_PIC) += irq-loongson-pch-pic.o obj-$(CONFIG_LOONGSON_PCH_MSI) += irq-loongson-pch-msi.o obj-$(CONFIG_MST_IRQ) += irq-mst-intc.o obj-$(CONFIG_SL28CPLD_INTC) += irq-sl28cpld.o +obj-$(CONFIG_IMS_MSI) += irq-ims-msi.o diff --git a/drivers/irqchip/irq-ims-msi.c b/drivers/irqchip/irq-ims-msi.c new file mode 100644 index 0000000..fa23207 --- /dev/null +++ b/drivers/irqchip/irq-ims-msi.c @@ -0,0 +1,211 @@ +// SPDX-License-Identifier: GPL-2.0 +// (C) Copyright 2021 Thomas Gleixner +/* + * Shared interrupt chips and irq domains for IMS devices + */ +#include +#include +#include +#include +#include + +#include + +#ifdef CONFIG_IMS_MSI_ARRAY + +struct ims_array_data { + struct ims_array_info info; + unsigned long map[0]; +}; + +static inline void iowrite32_and_flush(u32 value, void __iomem *addr) +{ + iowrite32(value, addr); + ioread32(addr); +} + +static void ims_array_mask_irq(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + u32 __iomem *ctrl = &slot->ctrl; + + iowrite32_and_flush(ioread32(ctrl) | IMS_CTRL_VECTOR_MASKBIT, ctrl); +} + +static void ims_array_unmask_irq(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + u32 __iomem *ctrl = &slot->ctrl; + + iowrite32_and_flush(ioread32(ctrl) & ~IMS_CTRL_VECTOR_MASKBIT, ctrl); +} + +static void ims_array_write_msi_msg(struct irq_data *data, struct msi_msg *msg) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + + iowrite32(msg->address_lo, &slot->address_lo); + iowrite32(msg->address_hi, &slot->address_hi); + iowrite32_and_flush(msg->data, &slot->data); +} + +static int ims_array_set_auxdata(struct irq_data *data, unsigned int which, + u64 auxval) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + u32 val, __iomem *ctrl = &slot->ctrl; + + if (which != IMS_AUXDATA_CONTROL_WORD) + return -EINVAL; + if (auxval & ~(u64)IMS_CONTROL_WORD_AUXMASK) + return -EINVAL; + + val = ioread32(ctrl) & IMS_CONTROL_WORD_IRQMASK; + iowrite32_and_flush(val | (u32)auxval, ctrl); + return 0; +} + +static const struct irq_chip ims_array_msi_controller = { + .name = "IMS", + .irq_mask = ims_array_mask_irq, + .irq_unmask = ims_array_unmask_irq, + .irq_write_msi_msg = ims_array_write_msi_msg, + .irq_set_auxdata = ims_array_set_auxdata, + .irq_retrigger = irq_chip_retrigger_hierarchy, + .flags = IRQCHIP_SKIP_SET_WAKE, +}; + +static void ims_array_reset_slot(struct ims_slot __iomem *slot) +{ + iowrite32(0, &slot->address_lo); + iowrite32(0, &slot->address_hi); + iowrite32(0, &slot->data); + iowrite32_and_flush(IMS_CTRL_VECTOR_MASKBIT, &slot->ctrl); +} + +static void ims_array_free_msi_store(struct irq_domain *domain, + struct device *dev) +{ + struct msi_domain_info *info = domain->host_data; + struct ims_array_data *ims = info->data; + struct msi_desc *entry; + + for_each_msi_entry(entry, dev) { + if (entry->device_msi.priv_iomem) { + clear_bit(entry->device_msi.hwirq, ims->map); + ims_array_reset_slot(entry->device_msi.priv_iomem); + entry->device_msi.priv_iomem = NULL; + entry->device_msi.hwirq = 0; + } + } +} + +static int ims_array_alloc_msi_store(struct irq_domain *domain, + struct device *dev, int nvec) +{ + struct msi_domain_info *info = domain->host_data; + struct ims_array_data *ims = info->data; + struct msi_desc *entry; + + for_each_msi_entry(entry, dev) { + unsigned int idx; + + idx = find_first_zero_bit(ims->map, ims->info.max_slots); + if (idx >= ims->info.max_slots) + goto fail; + set_bit(idx, ims->map); + entry->device_msi.priv_iomem = &ims->info.slots[idx]; + ims_array_reset_slot(entry->device_msi.priv_iomem); + entry->device_msi.hwirq = idx; + } + return 0; + +fail: + ims_array_free_msi_store(domain, dev); + return -ENOSPC; +} + +struct ims_array_domain_template { + struct msi_domain_ops ops; + struct msi_domain_info info; +}; + +static void ims_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc) +{ + arg->desc = desc; + arg->hwirq = desc->device_msi.hwirq; +} + +static const struct ims_array_domain_template ims_array_domain_template = { + .ops = { + .msi_alloc_store = ims_array_alloc_msi_store, + .msi_free_store = ims_array_free_msi_store, + .set_desc = ims_set_desc, + }, + .info = { + .flags = MSI_FLAG_USE_DEF_DOM_OPS | + MSI_FLAG_USE_DEF_CHIP_OPS, + .handler = handle_edge_irq, + .handler_name = "edge", + }, +}; + +struct irq_domain * +pci_ims_array_create_msi_irq_domain(struct pci_dev *pdev, + struct ims_array_info *ims_info) +{ + struct ims_array_domain_template *info; + struct ims_array_data *data; + struct irq_domain *domain; + struct irq_chip *chip; + unsigned int size; + + /* Allocate new domain storage */ + info = kmemdup(&ims_array_domain_template, + sizeof(ims_array_domain_template), GFP_KERNEL); + if (!info) + return NULL; + /* Link the ops */ + info->info.ops = &info->ops; + + /* Allocate ims_info along with the bitmap */ + size = sizeof(*data); + size += BITS_TO_LONGS(ims_info->max_slots) * sizeof(unsigned long); + data = kzalloc(size, GFP_KERNEL); + if (!data) + goto err_info; + + data->info = *ims_info; + info->info.data = data; + + /* + * Allocate an interrupt chip because the core needs to be able to + * update it with default callbacks. + */ + chip = kmemdup(&ims_array_msi_controller, + sizeof(ims_array_msi_controller), GFP_KERNEL); + if (!chip) + goto err_data; + info->info.chip = chip; + + domain = pci_subdevice_msi_create_irq_domain(pdev, &info->info); + if (!domain) + goto err_chip; + + return domain; + +err_chip: + kfree(chip); +err_data: + kfree(data); +err_info: + kfree(info); + return NULL; +} +EXPORT_SYMBOL_GPL(pci_ims_array_create_msi_irq_domain); + +#endif /* CONFIG_IMS_MSI_ARRAY */ diff --git a/include/linux/irqchip/irq-ims-msi.h b/include/linux/irqchip/irq-ims-msi.h new file mode 100644 index 0000000..9ba767f --- /dev/null +++ b/include/linux/irqchip/irq-ims-msi.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* (C) Copyright 2021 Thomas Gleixner */ + +#ifndef _LINUX_IRQCHIP_IRQ_IMS_MSI_H +#define _LINUX_IRQCHIP_IRQ_IMS_MSI_H + +#include +#include + +/** + * ims_hw_slot - The hardware layout of an IMS based MSI message + * @address_lo: Lower 32bit address + * @address_hi: Upper 32bit address + * @data: Message data + * @ctrl: Control word + * + * This structure is used by both the device memory array and the queue + * memory variants of IMS. + */ +struct ims_slot { + u32 address_lo; + u32 address_hi; + u32 data; + u32 ctrl; +} __packed; + +/* + * The IMS control word utilizes bit 0-2 for interrupt control. The remaining + * bits can contain auxiliary data. + */ +#define IMS_CONTROL_WORD_IRQMASK GENMASK(2, 0) +#define IMS_CONTROL_WORD_AUXMASK GENMASK(31, 3) + +/* Auxiliary control word data related defines */ +enum { + IMS_AUXDATA_CONTROL_WORD, +}; + +/* Bit to mask the interrupt in ims_hw_slot::ctrl */ +#define IMS_CTRL_VECTOR_MASKBIT BIT(0) +#define IMS_CTRL_PASID_ENABLE BIT(3) +#define IMS_CTRL_PASID_SHIFT 12 + +/* Set pasid and enable bit for the IMS entry */ +static inline u32 ims_ctrl_pasid_aux(unsigned int pasid, bool enable) +{ + u32 auxval = pasid << IMS_CTRL_PASID_SHIFT; + + return enable ? auxval | IMS_CTRL_PASID_ENABLE : auxval; +} + +/** + * struct ims_array_info - Information to create an IMS array domain + * @slots: Pointer to the start of the array + * @max_slots: Maximum number of slots in the array + */ +struct ims_array_info { + struct ims_slot __iomem *slots; + unsigned int max_slots; +}; + +struct pci_dev; +struct irq_domain; + +struct irq_domain *pci_ims_array_create_msi_irq_domain(struct pci_dev *pdev, + struct ims_array_info *ims_info); + +#endif