From patchwork Thu Aug 19 12:02:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE767C4338F for ; Thu, 19 Aug 2021 12:04:51 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A30296112E for ; Thu, 19 Aug 2021 12:04:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A30296112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168603.307828 (Exim 4.92) (envelope-from ) id 1mGgmm-0002DG-NP; Thu, 19 Aug 2021 12:04:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168603.307828; Thu, 19 Aug 2021 12:04:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgmm-0002D9-Js; Thu, 19 Aug 2021 12:04:40 +0000 Received: by outflank-mailman (input) for mailman id 168603; Thu, 19 Aug 2021 12:04:39 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgml-0002Cs-7e for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:04:39 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 05a63d4a-9cc0-4d63-b142-d1e131694794; Thu, 19 Aug 2021 12:04:36 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8779C31B; Thu, 19 Aug 2021 05:04:36 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A3A503F70D; Thu, 19 Aug 2021 05:04:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 05a63d4a-9cc0-4d63-b142-d1e131694794 From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Paul Durrant , Daniel De Graaf , "Daniel P. Smith" Subject: [PATCH v1 01/14] xen/pci: Refactor MSI code that implements MSI functionality within XEN Date: Thu, 19 Aug 2021 13:02:41 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: MSI code that implements MSI functionality to support MSI within XEN is not usable on ARM. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code for ARM. No functional change intended. Signed-off-by: Rahul Singh Reviewed-by: Jan Beulich Reviewed-by: Daniel P. Smith --- xen/arch/x86/Kconfig | 1 + xen/drivers/passthrough/Makefile | 1 + xen/drivers/passthrough/msi.c | 96 ++++++++++++++++++++++++++++++++ xen/drivers/passthrough/pci.c | 54 +++++------------- xen/drivers/pci/Kconfig | 4 ++ xen/include/xen/msi.h | 56 +++++++++++++++++++ xen/xsm/flask/hooks.c | 8 +-- 7 files changed, 175 insertions(+), 45 deletions(-) create mode 100644 xen/drivers/passthrough/msi.c create mode 100644 xen/include/xen/msi.h diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 9b164db641..7b46ee98c5 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -19,6 +19,7 @@ config X86 select HAS_NS16550 select HAS_PASSTHROUGH select HAS_PCI + select HAS_PCI_MSI select HAS_PDX select HAS_SCHED_GRANULARITY select HAS_UBSAN diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/Makefile index 445690e3e5..a5efa22714 100644 --- a/xen/drivers/passthrough/Makefile +++ b/xen/drivers/passthrough/Makefile @@ -7,3 +7,4 @@ obj-y += iommu.o obj-$(CONFIG_HAS_PCI) += pci.o obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o obj-$(CONFIG_HAS_PCI) += ats.o +obj-$(CONFIG_HAS_PCI_MSI) += msi.o diff --git a/xen/drivers/passthrough/msi.c b/xen/drivers/passthrough/msi.c new file mode 100644 index 0000000000..15ad0f8160 --- /dev/null +++ b/xen/drivers/passthrough/msi.c @@ -0,0 +1,96 @@ +/* + * Copyright (C) 2008, Netronome Systems, Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; If not, see . + */ + +#include +#include +#include +#include + +int pdev_msix_assign(struct domain *d, struct pci_dev *pdev) +{ + int rc; + + if ( pdev->msix ) + { + rc = pci_reset_msix_state(pdev); + if ( rc ) + return rc; + msixtbl_init(d); + } + + return 0; +} + +int pdev_msi_init(struct pci_dev *pdev) +{ + unsigned int pos; + + INIT_LIST_HEAD(&pdev->msi_list); + + pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn), + PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSI); + if ( pos ) + { + uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos)); + + pdev->msi_maxvec = multi_msi_capable(ctrl); + } + + pos = pci_find_cap_offset(pdev->seg, pdev->bus, PCI_SLOT(pdev->devfn), + PCI_FUNC(pdev->devfn), PCI_CAP_ID_MSIX); + if ( pos ) + { + struct arch_msix *msix = xzalloc(struct arch_msix); + uint16_t ctrl; + + if ( !msix ) + return -ENOMEM; + + spin_lock_init(&msix->table_lock); + + ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos)); + msix->nr_entries = msix_table_size(ctrl); + + pdev->msix = msix; + } + + return 0; +} + +void pdev_msi_deinit(struct pci_dev *pdev) +{ + XFREE(pdev->msix); +} + +void pdev_dump_msi(const struct pci_dev *pdev) +{ + const struct msi_desc *msi; + + printk("- MSIs < "); + list_for_each_entry ( msi, &pdev->msi_list, list ) + printk("%d ", msi->irq); + printk(">"); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index fc4fa2e5c3..67f5686ab6 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -32,8 +32,8 @@ #include #include #include +#include #include -#include #include "ats.h" struct pci_seg { @@ -314,6 +314,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) { struct pci_dev *pdev; unsigned int pos; + int rc; list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->bus == bus && pdev->devfn == devfn ) @@ -327,35 +328,12 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) *((u8*) &pdev->bus) = bus; *((u8*) &pdev->devfn) = devfn; pdev->domain = NULL; - INIT_LIST_HEAD(&pdev->msi_list); - - pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn), - PCI_CAP_ID_MSI); - if ( pos ) - { - uint16_t ctrl = pci_conf_read16(pdev->sbdf, msi_control_reg(pos)); - - pdev->msi_maxvec = multi_msi_capable(ctrl); - } - pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn), - PCI_CAP_ID_MSIX); - if ( pos ) + rc = pdev_msi_init(pdev); + if ( rc ) { - struct arch_msix *msix = xzalloc(struct arch_msix); - uint16_t ctrl; - - if ( !msix ) - { - xfree(pdev); - return NULL; - } - spin_lock_init(&msix->table_lock); - - ctrl = pci_conf_read16(pdev->sbdf, msix_control_reg(pos)); - msix->nr_entries = msix_table_size(ctrl); - - pdev->msix = msix; + xfree(pdev); + return NULL; } list_add(&pdev->alldevs_list, &pseg->alldevs_list); @@ -449,7 +427,7 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev) } list_del(&pdev->alldevs_list); - xfree(pdev->msix); + pdev_msi_deinit(pdev); xfree(pdev); } @@ -1271,18 +1249,16 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev *pdev) static int _dump_pci_devices(struct pci_seg *pseg, void *arg) { struct pci_dev *pdev; - struct msi_desc *msi; printk("==== segment %04x ====\n", pseg->nr); list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) { - printk("%pp - %pd - node %-3d - MSIs < ", + printk("%pp - %pd - node %-3d ", &pdev->sbdf, pdev->domain, (pdev->node != NUMA_NO_NODE) ? pdev->node : -1); - list_for_each_entry ( msi, &pdev->msi_list, list ) - printk("%d ", msi->irq); - printk(">\n"); + pdev_dump_msi(pdev); + printk("\n"); } return 0; @@ -1422,13 +1398,9 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) ASSERT(pdev && (pdev->domain == hardware_domain || pdev->domain == dom_io)); - if ( pdev->msix ) - { - rc = pci_reset_msix_state(pdev); - if ( rc ) - goto done; - msixtbl_init(d); - } + rc = pdev_msix_assign(d, pdev); + if ( rc ) + goto done; pdev->fault.count = 0; diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig index 7da03fa13b..c6a7bc8007 100644 --- a/xen/drivers/pci/Kconfig +++ b/xen/drivers/pci/Kconfig @@ -1,3 +1,7 @@ config HAS_PCI bool + +config HAS_PCI_MSI + bool + depends on HAS_PCI diff --git a/xen/include/xen/msi.h b/xen/include/xen/msi.h new file mode 100644 index 0000000000..b2d5bc6f9d --- /dev/null +++ b/xen/include/xen/msi.h @@ -0,0 +1,56 @@ +/* + * Copyright (C) 2008, Netronome Systems, Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; If not, see . + */ + +#ifndef __XEN_MSI_H_ +#define __XEN_MSI_H_ + +#ifdef CONFIG_HAS_PCI_MSI + +#include + +int pdev_msix_assign(struct domain *d, struct pci_dev *pdev); +int pdev_msi_init(struct pci_dev *pdev); +void pdev_msi_deinit(struct pci_dev *pdev); +void pdev_dump_msi(const struct pci_dev *pdev); + +#else /* !CONFIG_HAS_PCI_MSI */ +static inline int pdev_msix_assign(struct domain *d, struct pci_dev *pdev) +{ + return 0; +} + +static inline int pdev_msi_init(struct pci_dev *pdev) +{ + return 0; +} + +static inline void pdev_msi_deinit(struct pci_dev *pdev) {} +static inline void pci_cleanup_msi(struct pci_dev *pdev) {} +static inline void pdev_dump_msi(const struct pci_dev *pdev) {} + +#endif /* CONFIG_HAS_PCI_MSI */ + +#endif /* __XEN_MSI_H */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index f1a1217c98..fdcfeb984c 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -21,7 +21,7 @@ #include #include #include -#ifdef CONFIG_HAS_PCI +#ifdef CONFIG_HAS_PCI_MSI #include #endif #include @@ -114,7 +114,7 @@ static int get_irq_sid(int irq, u32 *sid, struct avc_audit_data *ad) } return security_irq_sid(irq, sid); } -#ifdef CONFIG_HAS_PCI +#ifdef CONFIG_HAS_PCI_MSI { struct irq_desc *desc = irq_to_desc(irq); if ( desc->msi_desc && desc->msi_desc->dev ) { @@ -874,7 +874,7 @@ static int flask_map_domain_pirq (struct domain *d) static int flask_map_domain_msi (struct domain *d, int irq, const void *data, u32 *sid, struct avc_audit_data *ad) { -#ifdef CONFIG_HAS_PCI +#ifdef CONFIG_HAS_PCI_MSI const struct msi_info *msi = data; u32 machine_bdf = (msi->seg << 16) | (msi->bus << 8) | msi->devfn; @@ -940,7 +940,7 @@ static int flask_unmap_domain_pirq (struct domain *d) static int flask_unmap_domain_msi (struct domain *d, int irq, const void *data, u32 *sid, struct avc_audit_data *ad) { -#ifdef CONFIG_HAS_PCI +#ifdef CONFIG_HAS_PCI_MSI const struct pci_dev *pdev = data; u32 machine_bdf = (pdev->seg << 16) | (pdev->bus << 8) | pdev->devfn; From patchwork Thu Aug 19 12:02:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2A47C4338F for ; Thu, 19 Aug 2021 12:05:23 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 969346112E for ; Thu, 19 Aug 2021 12:05:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 969346112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168611.307838 (Exim 4.92) (envelope-from ) id 1mGgnK-0002nn-3N; Thu, 19 Aug 2021 12:05:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168611.307838; Thu, 19 Aug 2021 12:05:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgnK-0002ng-0J; Thu, 19 Aug 2021 12:05:14 +0000 Received: by outflank-mailman (input) for mailman id 168611; Thu, 19 Aug 2021 12:05:13 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgnJ-0002nY-1B for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:05:13 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 5b5ee02b-6e6e-42eb-966f-4c693724cdff; Thu, 19 Aug 2021 12:05:11 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2453531B; Thu, 19 Aug 2021 05:05:11 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3D98E3F70D; Thu, 19 Aug 2021 05:05:10 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5b5ee02b-6e6e-42eb-966f-4c693724cdff From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 02/14] xen/pci: solve compilation error on ARM with HAS_PCI enabled Date: Thu, 19 Aug 2021 13:02:42 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Compilation error is observed when HAS_PCI is enabled for ARM architecture. Add definition for arch_iommu_use_permitted() and arch_pci_clean_pirqs().Implement dummy functions for pci_conf_read*() to fix compilation error. pci.c: In function ‘deassign_device’: pci.c:849:49: error: implicit declaration of function ‘pci_to_dev’; did you mean ‘dt_to_dev’? [-Werror=implicit-function-declaration] pci_to_dev(pdev)); pci.c:18: undefined reference to `pci_conf_read16’ pci.c:880: undefined reference to `arch_pci_clean_pirqs’ pci.c:1392: undefined reference to `arch_iommu_use_permitted' Signed-off-by: Rahul Singh --- xen/arch/arm/Makefile | 1 + xen/arch/arm/pci/Makefile | 2 + xen/arch/arm/pci/pci-access.c | 61 +++++++++++++++++++++++++++++ xen/arch/arm/pci/pci.c | 32 +++++++++++++++ xen/drivers/passthrough/arm/iommu.c | 5 +++ xen/include/asm-arm/pci.h | 33 ++++++++++++++-- 6 files changed, 131 insertions(+), 3 deletions(-) create mode 100644 xen/arch/arm/pci/Makefile create mode 100644 xen/arch/arm/pci/pci-access.c create mode 100644 xen/arch/arm/pci/pci.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 3d3b97b5b4..0e14a5e5c8 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -6,6 +6,7 @@ ifneq ($(CONFIG_NO_PLAT),y) obj-y += platforms/ endif obj-$(CONFIG_TEE) += tee/ +obj-$(CONFIG_HAS_PCI) += pci/ obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o obj-y += bootfdt.init.o diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile new file mode 100644 index 0000000000..a9ee0b9b44 --- /dev/null +++ b/xen/arch/arm/pci/Makefile @@ -0,0 +1,2 @@ +obj-y += pci.o +obj-y += pci-access.o diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c new file mode 100644 index 0000000000..b938047c03 --- /dev/null +++ b/xen/arch/arm/pci/pci-access.c @@ -0,0 +1,61 @@ +/* + * Copyright (C) 2021 Arm Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg, + unsigned int len) +{ + return ~0U; +} + +static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg, + unsigned int len, uint32_t val) +{ +} + +/* + * Wrappers for all PCI configuration access functions. + */ + +#define PCI_OP_WRITE(size, type) \ + void pci_conf_write##size (pci_sbdf_t sbdf,unsigned int reg, type val) \ +{ \ + pci_config_write(sbdf, reg, size / 8, val); \ +} + +#define PCI_OP_READ(size, type) \ + type pci_conf_read##size (pci_sbdf_t sbdf, unsigned int reg) \ +{ \ + return pci_config_read(sbdf, reg, size / 8); \ +} + +PCI_OP_READ(8, u8) +PCI_OP_READ(16, u16) +PCI_OP_READ(32, u32) +PCI_OP_WRITE(8, u8) +PCI_OP_WRITE(16, u16) +PCI_OP_WRITE(32, u32) + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c new file mode 100644 index 0000000000..dc55d23778 --- /dev/null +++ b/xen/arch/arm/pci/pci.c @@ -0,0 +1,32 @@ +/* + * Copyright (C) 2021 Arm Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +int arch_pci_clean_pirqs(struct domain *d) +{ + return 0; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c index db3b07a571..fdec1c5547 100644 --- a/xen/drivers/passthrough/arm/iommu.c +++ b/xen/drivers/passthrough/arm/iommu.c @@ -135,3 +135,8 @@ void arch_iommu_domain_destroy(struct domain *d) void __hwdom_init arch_iommu_hwdom_init(struct domain *d) { } + +bool arch_iommu_use_permitted(const struct domain *d) +{ + return true; +} diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index de13359f65..61e43da088 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -1,7 +1,34 @@ -#ifndef __X86_PCI_H__ -#define __X86_PCI_H__ +/* + * Copyright (C) 2021 Arm Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#ifndef __ARM_PCI_H__ +#define __ARM_PCI_H__ + +#ifdef CONFIG_HAS_PCI + +#define pci_to_dev(pcidev) (&(pcidev)->arch.dev) + +/* Arch pci dev struct */ struct arch_pci_dev { + struct device dev; }; -#endif /* __X86_PCI_H__ */ +#else /*!CONFIG_HAS_PCI*/ + +struct arch_pci_dev { }; + +#endif /*!CONFIG_HAS_PCI*/ +#endif /* __ARM_PCI_H__ */ From patchwork Thu Aug 19 12:02:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AF61C4338F for ; Thu, 19 Aug 2021 12:05:45 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0DBEC61152 for ; Thu, 19 Aug 2021 12:05:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0DBEC61152 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168616.307849 (Exim 4.92) (envelope-from ) id 1mGgnh-0003Nu-DX; Thu, 19 Aug 2021 12:05:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168616.307849; Thu, 19 Aug 2021 12:05:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgnh-0003Nn-AF; Thu, 19 Aug 2021 12:05:37 +0000 Received: by outflank-mailman (input) for mailman id 168616; Thu, 19 Aug 2021 12:05:36 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgng-0003MP-6D for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:05:36 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id f3b74f15-2acf-4137-b983-af5b37feea4e; Thu, 19 Aug 2021 12:05:35 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C077431B; Thu, 19 Aug 2021 05:05:34 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F41733F70D; Thu, 19 Aug 2021 05:05:33 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f3b74f15-2acf-4137-b983-af5b37feea4e From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Jan Beulich , Paul Durrant Subject: [PATCH v1 03/14] xen/pci: solve compilation error on ARM with ACPI && HAS_PCI Date: Thu, 19 Aug 2021 13:02:43 +0100 Message-Id: <97d39d3ee398d6018bdcaf745f00d039df6a92ef.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Compilation error is observed when ACPI and HAS_PCI is enabled for ARM architecture. Move the code under CONFIG_X86 flag to gate the code for ARM. No functional change intended. Signed-off-by: Rahul Singh --- xen/drivers/passthrough/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 67f5686ab6..c23c8cb06b 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1150,7 +1150,7 @@ void __hwdom_init setup_hwdom_pci_devices( pcidevs_unlock(); } -#ifdef CONFIG_ACPI +#if defined(CONFIG_ACPI) && defined(CONFIG_X86) #include #include From patchwork Thu Aug 19 12:02:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 771ADC4338F for ; Thu, 19 Aug 2021 12:06:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3F83E6112E for ; Thu, 19 Aug 2021 12:06:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3F83E6112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168621.307861 (Exim 4.92) (envelope-from ) id 1mGgo1-0003v3-M4; Thu, 19 Aug 2021 12:05:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168621.307861; Thu, 19 Aug 2021 12:05:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgo1-0003us-Ip; Thu, 19 Aug 2021 12:05:57 +0000 Received: by outflank-mailman (input) for mailman id 168621; Thu, 19 Aug 2021 12:05:56 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgo0-0003uH-55 for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:05:56 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id ce6fb9f2-00e5-11ec-a5f6-12813bfff9fa; Thu, 19 Aug 2021 12:05:55 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 10B9531B; Thu, 19 Aug 2021 05:05:55 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 280E03F70D; Thu, 19 Aug 2021 05:05:54 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ce6fb9f2-00e5-11ec-a5f6-12813bfff9fa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 04/14] xen/arm: Add support for PCI init to initialize the PCI driver. Date: Thu, 19 Aug 2021 13:02:44 +0100 Message-Id: <999887f9b4b2ea06ae99e1e003f9e43aa272a19c.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: pci_init(..) will be called during xen startup to initialize and probe the PCI host-bridge driver. Signed-off-by: Rahul Singh --- xen/arch/arm/pci/pci.c | 54 ++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/device.h | 1 + 2 files changed, 55 insertions(+) diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c index dc55d23778..d1c9cf997d 100644 --- a/xen/arch/arm/pci/pci.c +++ b/xen/arch/arm/pci/pci.c @@ -14,13 +14,67 @@ * along with this program. If not, see . */ +#include +#include +#include +#include #include +#include int arch_pci_clean_pirqs(struct domain *d) { return 0; } +static int __init dt_pci_init(void) +{ + struct dt_device_node *np; + int rc; + + dt_for_each_device_node(dt_host, np) + { + rc = device_init(np, DEVICE_PCI, NULL); + if( !rc ) + continue; + /* + * Ignore the following error codes: + * - EBADF: Indicate the current is not an pci + * - ENODEV: The pci device is not present or cannot be used by + * Xen. + */ + else if ( rc != -EBADF && rc != -ENODEV ) + { + printk(XENLOG_ERR "No driver found in XEN or driver init error.\n"); + return rc; + } + } + + return 0; +} + +#ifdef CONFIG_ACPI +static void __init acpi_pci_init(void) +{ + printk(XENLOG_ERR "ACPI pci init not supported \n"); + return; +} +#else +static inline void __init acpi_pci_init(void) { } +#endif + +static int __init pci_init(void) +{ + if ( acpi_disabled ) + dt_pci_init(); + else + acpi_pci_init(); + + pci_segments_init(); + + return 0; +} +__initcall(pci_init); + /* * Local variables: * mode: C diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h index ee7cff2d44..5ecd5e7bd1 100644 --- a/xen/include/asm-arm/device.h +++ b/xen/include/asm-arm/device.h @@ -34,6 +34,7 @@ enum device_class DEVICE_SERIAL, DEVICE_IOMMU, DEVICE_GIC, + DEVICE_PCI, /* Use for error */ DEVICE_UNKNOWN, }; From patchwork Thu Aug 19 12:02:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50FDBC4338F for ; Thu, 19 Aug 2021 12:06:36 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 172E56112E for ; Thu, 19 Aug 2021 12:06:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 172E56112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168629.307882 (Exim 4.92) (envelope-from ) id 1mGgoU-0004uq-9d; Thu, 19 Aug 2021 12:06:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168629.307882; Thu, 19 Aug 2021 12:06:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgoU-0004uj-6j; Thu, 19 Aug 2021 12:06:26 +0000 Received: by outflank-mailman (input) for mailman id 168629; Thu, 19 Aug 2021 12:06:25 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgoT-0004oY-0Y for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:06:25 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id de9b866c-00e5-11ec-a5f6-12813bfff9fa; Thu, 19 Aug 2021 12:06:22 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1170231B; Thu, 19 Aug 2021 05:06:22 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27DDE3F70D; Thu, 19 Aug 2021 05:06:21 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: de9b866c-00e5-11ec-a5f6-12813bfff9fa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 05/14] xen/arm: PCI host bridge discovery within XEN on ARM Date: Thu, 19 Aug 2021 13:02:45 +0100 Message-Id: <412b6574170d96d6fc4196fab4bb2b9286a770c8.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 XEN during boot will read the PCI device tree node “reg” property and will map the PCI config space to the XEN memory. As of now "pci-host-ecam-generic" compatible board is supported. "linux,pci-domain" device tree property assigns a fixed PCI domain number to a host bridge, otherwise an unstable (across boots) unique number will be assigned by Linux.This property has to be in sync with XEN to access the PCI devices. XEN will read the “linux,pci-domain” property from the device tree node and configure the host bridge segment number accordingly. If this property is not available XEN will allocate the unique segment number to the host bridge. dt_get_pci_domain_nr(..) and dt_pci_bus_find_domain_nr(..) are directly imported from the Linux source tree. Signed-off-by: Rahul Singh --- xen/arch/arm/pci/Makefile | 2 + xen/arch/arm/pci/pci-host-common.c | 261 ++++++++++++++++++++++++++++ xen/arch/arm/pci/pci-host-generic.c | 55 ++++++ xen/include/asm-arm/pci.h | 28 +++ 4 files changed, 346 insertions(+) create mode 100644 xen/arch/arm/pci/pci-host-common.c create mode 100644 xen/arch/arm/pci/pci-host-generic.c diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile index a9ee0b9b44..f3d97f859e 100644 --- a/xen/arch/arm/pci/Makefile +++ b/xen/arch/arm/pci/Makefile @@ -1,2 +1,4 @@ obj-y += pci.o obj-y += pci-access.o +obj-y += pci-host-generic.o +obj-y += pci-host-common.o diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c new file mode 100644 index 0000000000..9dd9b02271 --- /dev/null +++ b/xen/arch/arm/pci/pci-host-common.c @@ -0,0 +1,261 @@ +/* + * Copyright (C) 2021 Arm Ltd. + * + * Based on Linux drivers/pci/ecam.c + * Copyright 2016 Broadcom. + * + * Based on Linux drivers/pci/controller/pci-host-common.c + * Based on Linux drivers/pci/controller/pci-host-generic.c + * Copyright (C) 2014 ARM Limited Will Deacon + * + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include +#include +#include +#include +#include + +/* + * List for all the pci host bridges. + */ + +static LIST_HEAD(pci_host_bridges); + +static atomic_t domain_nr = ATOMIC_INIT(-1); + +bool dt_pci_parse_bus_range(struct dt_device_node *dev, + struct pci_config_window *cfg) +{ + const __be32 *cells; + uint32_t len; + + cells = dt_get_property(dev, "bus-range", &len); + /* bus-range should at least be 2 cells */ + if ( !cells || (len < (sizeof(*cells) * 2)) ) + return false; + + cfg->busn_start = dt_next_cell(1, &cells); + cfg->busn_end = dt_next_cell(1, &cells); + + return true; +} + +static inline void __iomem *pci_remap_cfgspace(paddr_t start, size_t len) +{ + return ioremap_nocache(start, len); +} + +static void pci_ecam_free(struct pci_config_window *cfg) +{ + if ( cfg->win ) + iounmap(cfg->win); + + xfree(cfg); +} + +static struct pci_config_window *gen_pci_init(struct dt_device_node *dev, + int ecam_reg_idx) +{ + int err; + struct pci_config_window *cfg; + paddr_t addr, size; + + cfg = xzalloc(struct pci_config_window); + if ( !cfg ) + return NULL; + + err = dt_pci_parse_bus_range(dev, cfg); + if ( !err ) { + cfg->busn_start = 0; + cfg->busn_end = 0xff; + printk(XENLOG_ERR "%s:No bus range found for pci controller\n", + dt_node_full_name(dev)); + } else { + if ( cfg->busn_end > cfg->busn_start + 0xff ) + cfg->busn_end = cfg->busn_start + 0xff; + } + + /* Parse our PCI ecam register address*/ + err = dt_device_get_address(dev, ecam_reg_idx, &addr, &size); + if ( err ) + goto err_exit; + + cfg->phys_addr = addr; + cfg->size = size; + + /* + * On 64-bit systems, we do a single ioremap for the whole config space + * since we have enough virtual address range available. On 32-bit, we + * ioremap the config space for each bus individually. + * + * As of now only 64-bit is supported 32-bit is not supported. + */ + cfg->win = pci_remap_cfgspace(cfg->phys_addr, cfg->size); + if ( !cfg->win ) + goto err_exit_remap; + + printk("ECAM at [mem %lx-%lx] for [bus %x-%x] \n",cfg->phys_addr, + cfg->phys_addr + cfg->size - 1, cfg->busn_start, cfg->busn_end); + + return cfg; + +err_exit_remap: + printk(XENLOG_ERR "ECAM ioremap failed\n"); +err_exit: + pci_ecam_free(cfg); + return NULL; +} + +struct pci_host_bridge *pci_alloc_host_bridge(void) +{ + struct pci_host_bridge *bridge = xzalloc(struct pci_host_bridge); + + if ( !bridge ) + return NULL; + + INIT_LIST_HEAD(&bridge->node); + bridge->bus_start = ~0; + bridge->bus_end = ~0; + return bridge; +} + +void pci_add_host_bridge(struct pci_host_bridge *bridge) +{ + list_add_tail(&bridge->node, &pci_host_bridges); +} + +static int pci_get_new_domain_nr(void) +{ + return atomic_inc_return(&domain_nr); +} + +/* + * This function will try to obtain the host bridge domain number by + * finding a property called "linux,pci-domain" of the given device node. + * + * @node: device tree node with the domain information + * + * Returns the associated domain number from DT in the range [0-0xffff], or + * a negative value if the required property is not found. + */ +static int dt_get_pci_domain_nr(struct dt_device_node *node) +{ + u32 domain; + int error; + + error = dt_property_read_u32(node, "linux,pci-domain", &domain); + if ( !error ) + return -EINVAL; + + return (u16)domain; +} + +static int pci_bus_find_domain_nr(struct dt_device_node *dev) +{ + static int use_dt_domains = -1; + int domain; + + domain = dt_get_pci_domain_nr(dev); + + /* + * Check DT domain and use_dt_domains values. + * + * If DT domain property is valid (domain >= 0) and + * use_dt_domains != 0, the DT assignment is valid since this means + * we have not previously allocated a domain number by using + * pci_get_new_domain_nr(); we should also update use_dt_domains to + * 1, to indicate that we have just assigned a domain number from + * DT. + * + * If DT domain property value is not valid (ie domain < 0), and we + * have not previously assigned a domain number from DT + * (use_dt_domains != 1) we should assign a domain number by + * using the: + * + * pci_get_new_domain_nr() + * + * API and update the use_dt_domains value to keep track of method we + * are using to assign domain numbers (use_dt_domains = 0). + * + * All other combinations imply we have a platform that is trying + * to mix domain numbers obtained from DT and pci_get_new_domain_nr(), + * which is a recipe for domain mishandling and it is prevented by + * invalidating the domain value (domain = -1) and printing a + * corresponding error. + */ + if ( domain >= 0 && use_dt_domains ) + { + use_dt_domains = 1; + } + else if ( domain < 0 && use_dt_domains != 1 ) + { + use_dt_domains = 0; + domain = pci_get_new_domain_nr(); + } + else + { + printk(XENLOG_ERR "Inconsistent \"linux,pci-domain\" property in DT\n"); + BUG(); + } + + return domain; +} + +int pci_host_common_probe(struct dt_device_node *dev, + int ecam_reg_idx) +{ + struct pci_host_bridge *bridge; + struct pci_config_window *cfg; + int err; + + bridge = pci_alloc_host_bridge(); + if ( !bridge ) + return -ENOMEM; + + /* Parse and map our Configuration Space windows */ + cfg = gen_pci_init(dev, ecam_reg_idx); + if ( !cfg ) + { + err = -ENOMEM; + goto err_exit; + } + + bridge->dt_node = dev; + bridge->sysdata = cfg; + bridge->bus_start = cfg->busn_start; + bridge->bus_end = cfg->busn_end; + + bridge->segment = pci_bus_find_domain_nr(dev); + + pci_add_host_bridge(bridge); + + return 0; + +err_exit: + xfree(bridge); + return err; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c new file mode 100644 index 0000000000..13d0f7f999 --- /dev/null +++ b/xen/arch/arm/pci/pci-host-generic.c @@ -0,0 +1,55 @@ +/* + * Copyright (C) 2021 Arm Ltd. + * + * Based on Linux drivers/pci/controller/pci-host-common.c + * Based on Linux drivers/pci/controller/pci-host-generic.c + * Copyright (C) 2014 ARM Limited Will Deacon + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include +#include + +static const struct dt_device_match gen_pci_dt_match[] = { + { .compatible = "pci-host-ecam-generic" }, + { }, +}; + +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data) +{ + const struct dt_device_match *of_id; + + of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node); + + printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n", + dt_node_full_name(dev), of_id->compatible); + + return pci_host_common_probe(dev, 0); +} + +DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI) +.dt_match = gen_pci_dt_match, +.init = gen_pci_dt_init, +DT_DEVICE_END + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 61e43da088..58a51e724e 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -26,6 +26,34 @@ struct arch_pci_dev { struct device dev; }; +/* + * struct to hold the mappings of a config space window. This + * is expected to be used as sysdata for PCI controllers that + * use ECAM. + */ +struct pci_config_window { + paddr_t phys_addr; + paddr_t size; + uint8_t busn_start; + uint8_t busn_end; + void __iomem *win; +}; + +/* + * struct to hold pci host bridge information + * for a PCI controller. + */ +struct pci_host_bridge { + struct dt_device_node *dt_node; /* Pointer to the associated DT node */ + struct list_head node; /* Node in list of host bridges */ + uint16_t segment; /* Segment number */ + u8 bus_start; /* Bus start of this bridge. */ + u8 bus_end; /* Bus end of this bridge. */ + void *sysdata; /* Pointer to the config space window*/ +}; + +int pci_host_common_probe(struct dt_device_node *dev, + int ecam_reg_idx); #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { }; From patchwork Thu Aug 19 12:02:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57BF8C4338F for ; Thu, 19 Aug 2021 12:07:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1D84961153 for ; Thu, 19 Aug 2021 12:07:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1D84961153 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168642.307893 (Exim 4.92) (envelope-from ) id 1mGgox-0005kD-MO; Thu, 19 Aug 2021 12:06:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168642.307893; Thu, 19 Aug 2021 12:06:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgox-0005k6-JW; Thu, 19 Aug 2021 12:06:55 +0000 Received: by outflank-mailman (input) for mailman id 168642; Thu, 19 Aug 2021 12:06:54 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgow-0005h3-Pj for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:06:54 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id ef22e3cc-00e5-11ec-a5f6-12813bfff9fa; Thu, 19 Aug 2021 12:06:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DA88231B; Thu, 19 Aug 2021 05:06:49 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 009433F70D; Thu, 19 Aug 2021 05:06:48 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ef22e3cc-00e5-11ec-a5f6-12813bfff9fa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 06/14] xen/arm: Add support for PCI ecam operations Date: Thu, 19 Aug 2021 13:02:46 +0100 Message-Id: <1dc8286db35ced8281587135cfa582ea44b0185f.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Add support for PCI ecam operations to access the PCI configuration space. Signed-off-by: Rahul Singh --- xen/arch/arm/pci/Makefile | 1 + xen/arch/arm/pci/ecam.c | 63 +++++++++++++++++++++++++++++ xen/arch/arm/pci/pci-access.c | 53 ++++++++++++++++++++++++ xen/arch/arm/pci/pci-host-common.c | 13 +++++- xen/arch/arm/pci/pci-host-generic.c | 8 +++- xen/include/asm-arm/pci.h | 32 +++++++++++++++ 6 files changed, 167 insertions(+), 3 deletions(-) create mode 100644 xen/arch/arm/pci/ecam.c diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile index f3d97f859e..6f32fbbe67 100644 --- a/xen/arch/arm/pci/Makefile +++ b/xen/arch/arm/pci/Makefile @@ -2,3 +2,4 @@ obj-y += pci.o obj-y += pci-access.o obj-y += pci-host-generic.o obj-y += pci-host-common.o +obj-y += ecam.o diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c new file mode 100644 index 0000000000..91c691b41f --- /dev/null +++ b/xen/arch/arm/pci/ecam.c @@ -0,0 +1,63 @@ +/* + * Copyright (C) 2021 Arm Ltd. + * + * Based on Linux drivers/pci/ecam.c + * Copyright 2016 Broadcom + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include + +/* + * Function to implement the pci_ops ->map_bus method. + */ +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, + uint32_t sbdf, uint32_t where) +{ + const struct pci_config_window *cfg = bridge->sysdata; + unsigned int devfn_shift = cfg->ops->bus_shift - 8; + void __iomem *base; + + pci_sbdf_t sbdf_t = (pci_sbdf_t) sbdf ; + unsigned int busn = sbdf_t.bus; + + if ( busn < cfg->busn_start || busn > cfg->busn_end ) + return NULL; + + busn -= cfg->busn_start; + base = cfg->win + (busn << cfg->ops->bus_shift); + + return base + (PCI_DEVFN(sbdf_t.dev, sbdf_t.fn) << devfn_shift) + where; +} + +/* ECAM ops */ +const struct pci_ecam_ops pci_generic_ecam_ops = { + .bus_shift = 20, + .pci_ops = { + .map_bus = pci_ecam_map_bus, + .read = pci_generic_config_read, + .write = pci_generic_config_write, + } +}; + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c index b938047c03..f39f6a3a38 100644 --- a/xen/arch/arm/pci/pci-access.c +++ b/xen/arch/arm/pci/pci-access.c @@ -15,6 +15,59 @@ */ #include +#include + +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t reg, uint32_t len, uint32_t *value) +{ + void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg); + if (!addr) { + *value = ~0; + return -ENODEV; + } + + switch (len) + { + case 1: + *value = readb(addr); + break; + case 2: + *value = readw(addr); + break; + case 4: + *value = readl(addr); + break; + default: + BUG(); + } + + return 0; +} + +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t reg, uint32_t len, uint32_t value) +{ + void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg); + if (!addr) + return -ENODEV; + + switch (len) + { + case 1: + writeb(value, addr); + break; + case 2: + writew(value, addr); + break; + case 4: + writel(value, addr); + break; + default: + BUG(); + } + + return 0; +} static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len) diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index 9dd9b02271..c582527e92 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -68,6 +68,7 @@ static void pci_ecam_free(struct pci_config_window *cfg) } static struct pci_config_window *gen_pci_init(struct dt_device_node *dev, + const struct pci_ecam_ops *ops, int ecam_reg_idx) { int err; @@ -96,6 +97,7 @@ static struct pci_config_window *gen_pci_init(struct dt_device_node *dev, cfg->phys_addr = addr; cfg->size = size; + cfg->ops = ops; /* * On 64-bit systems, we do a single ioremap for the whole config space @@ -111,6 +113,13 @@ static struct pci_config_window *gen_pci_init(struct dt_device_node *dev, printk("ECAM at [mem %lx-%lx] for [bus %x-%x] \n",cfg->phys_addr, cfg->phys_addr + cfg->size - 1, cfg->busn_start, cfg->busn_end); + if ( ops->init ) + { + err = ops->init(cfg); + if (err) + goto err_exit; + } + return cfg; err_exit_remap: @@ -216,6 +225,7 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev) } int pci_host_common_probe(struct dt_device_node *dev, + const struct pci_ecam_ops *ops, int ecam_reg_idx) { struct pci_host_bridge *bridge; @@ -227,7 +237,7 @@ int pci_host_common_probe(struct dt_device_node *dev, return -ENOMEM; /* Parse and map our Configuration Space windows */ - cfg = gen_pci_init(dev, ecam_reg_idx); + cfg = gen_pci_init(dev, ops, ecam_reg_idx); if ( !cfg ) { err = -ENOMEM; @@ -236,6 +246,7 @@ int pci_host_common_probe(struct dt_device_node *dev, bridge->dt_node = dev; bridge->sysdata = cfg; + bridge->ops = &ops->pci_ops; bridge->bus_start = cfg->busn_start; bridge->bus_end = cfg->busn_end; diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c index 13d0f7f999..2d652e8910 100644 --- a/xen/arch/arm/pci/pci-host-generic.c +++ b/xen/arch/arm/pci/pci-host-generic.c @@ -23,20 +23,24 @@ #include static const struct dt_device_match gen_pci_dt_match[] = { - { .compatible = "pci-host-ecam-generic" }, + { .compatible = "pci-host-ecam-generic", + .data = &pci_generic_ecam_ops }, + { }, }; static int gen_pci_dt_init(struct dt_device_node *dev, const void *data) { const struct dt_device_match *of_id; + const struct pci_ecam_ops *ops; of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node); + ops = (struct pci_ecam_ops *) of_id->data; printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n", dt_node_full_name(dev), of_id->compatible); - return pci_host_common_probe(dev, 0); + return pci_host_common_probe(dev, ops, 0); } DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI) diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 58a51e724e..22866244d2 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -37,6 +37,7 @@ struct pci_config_window { uint8_t busn_start; uint8_t busn_end; void __iomem *win; + const struct pci_ecam_ops *ops; }; /* @@ -50,10 +51,41 @@ struct pci_host_bridge { u8 bus_start; /* Bus start of this bridge. */ u8 bus_end; /* Bus end of this bridge. */ void *sysdata; /* Pointer to the config space window*/ + const struct pci_ops *ops; }; +struct pci_ops { + void __iomem *(*map_bus)(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t offset); + int (*read)(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t reg, uint32_t len, uint32_t *value); + int (*write)(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t reg, uint32_t len, uint32_t value); +}; + +/* + * struct to hold pci ops and bus shift of the config window + * for a PCI controller. + */ +struct pci_ecam_ops { + unsigned int bus_shift; + struct pci_ops pci_ops; + int (*init)(struct pci_config_window *); +}; + +/* Default ECAM ops */ +extern const struct pci_ecam_ops pci_generic_ecam_ops; + int pci_host_common_probe(struct dt_device_node *dev, + const struct pci_ecam_ops *ops, int ecam_reg_idx); +int pci_generic_config_read(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t reg, uint32_t len, uint32_t *value); +int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf, + uint32_t reg, uint32_t len, uint32_t value); +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, + uint32_t sbdf, uint32_t where); + #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { }; From patchwork Thu Aug 19 12:02:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72766C4338F for ; Thu, 19 Aug 2021 12:07:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 42C7761152 for ; Thu, 19 Aug 2021 12:07:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 42C7761152 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168643.307904 (Exim 4.92) (envelope-from ) id 1mGgpA-0006C5-W6; Thu, 19 Aug 2021 12:07:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168643.307904; Thu, 19 Aug 2021 12:07:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgpA-0006Bu-SI; Thu, 19 Aug 2021 12:07:08 +0000 Received: by outflank-mailman (input) for mailman id 168643; Thu, 19 Aug 2021 12:07:07 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgp9-0006AQ-IC for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:07:07 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id f0161eb3-35f0-4f08-9ae0-feff75077d1e; Thu, 19 Aug 2021 12:07:06 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BD04C31B; Thu, 19 Aug 2021 05:07:05 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B8F533F70D; Thu, 19 Aug 2021 05:07:04 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f0161eb3-35f0-4f08-9ae0-feff75077d1e From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko , Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 07/14] xen/arm: Add support for Xilinx ZynqMP PCI host controller Date: Thu, 19 Aug 2021 13:02:47 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Oleksandr Andrushchenko Add support for Xilinx ZynqMP PCI host controller to map the PCI config space to the XEN memory. Signed-off-by: Oleksandr Andrushchenko --- xen/arch/arm/pci/Makefile | 1 + xen/arch/arm/pci/pci-host-zynqmp.c | 59 ++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile index 6f32fbbe67..1d045ade01 100644 --- a/xen/arch/arm/pci/Makefile +++ b/xen/arch/arm/pci/Makefile @@ -3,3 +3,4 @@ obj-y += pci-access.o obj-y += pci-host-generic.o obj-y += pci-host-common.o obj-y += ecam.o +obj-y += pci-host-zynqmp.o diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c new file mode 100644 index 0000000000..fe103e3855 --- /dev/null +++ b/xen/arch/arm/pci/pci-host-zynqmp.c @@ -0,0 +1,59 @@ +/* + * Copyright (C) 2020-2021 EPAM Systems + * + * Based on Linux drivers/pci/controller/pci-host-common.c + * Based on Linux drivers/pci/controller/pci-host-generic.c + * Based on xen/arch/arm/pci/pci-host-generic.c + * Copyright (C) 2014 ARM Limited Will Deacon + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include +#include + +static const struct dt_device_match gen_pci_dt_match[] = { + { .compatible = "xlnx,nwl-pcie-2.11", + .data = &pci_generic_ecam_ops }, + { }, +}; + +static int gen_pci_dt_init(struct dt_device_node *dev, const void *data) +{ + const struct dt_device_match *of_id; + const struct pci_ecam_ops *ops; + + of_id = dt_match_node(gen_pci_dt_match, dev->dev.of_node); + ops = (struct pci_ecam_ops *) of_id->data; + + printk(XENLOG_INFO "Found PCI host bridge %s compatible:%s \n", + dt_node_full_name(dev), of_id->compatible); + + return pci_host_common_probe(dev, ops, 2); +} + +DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI) +.dt_match = gen_pci_dt_match, +.init = gen_pci_dt_init, +DT_DEVICE_END + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Thu Aug 19 12:02:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FD49C4338F for ; Thu, 19 Aug 2021 12:07:41 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5529561152 for ; Thu, 19 Aug 2021 12:07:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5529561152 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168650.307916 (Exim 4.92) (envelope-from ) id 1mGgpU-0006uh-8n; Thu, 19 Aug 2021 12:07:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168650.307916; Thu, 19 Aug 2021 12:07:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgpU-0006ua-50; Thu, 19 Aug 2021 12:07:28 +0000 Received: by outflank-mailman (input) for mailman id 168650; Thu, 19 Aug 2021 12:07:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgpS-0006rb-Vn for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:07:27 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 630f03c5-527c-4461-8eff-72b6b89ecffd; Thu, 19 Aug 2021 12:07:25 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9548B31B; Thu, 19 Aug 2021 05:07:25 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AEF093F70D; Thu, 19 Aug 2021 05:07:24 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 630f03c5-527c-4461-8eff-72b6b89ecffd From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 08/14] xen:arm: Implement pci access functions Date: Thu, 19 Aug 2021 13:02:48 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Implement generic pci access functions to read/write the configuration space. Signed-off-by: Rahul Singh --- xen/arch/arm/pci/pci-access.c | 31 +++++++++++++++++++++++++++++- xen/arch/arm/pci/pci-host-common.c | 19 ++++++++++++++++++ xen/include/asm-arm/pci.h | 2 ++ 3 files changed, 51 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c index f39f6a3a38..b94de3c3ac 100644 --- a/xen/arch/arm/pci/pci-access.c +++ b/xen/arch/arm/pci/pci-access.c @@ -72,12 +72,41 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf, static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len) { - return ~0U; + uint32_t val = GENMASK(0, len * 8); + + struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus); + + if ( unlikely(!bridge) ) + { + printk(XENLOG_ERR "Unable to find bridge for "PRI_pci"\n", + sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn); + return val; + } + + if ( unlikely(!bridge->ops->read) ) + return val; + + bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, &val); + + return val; } static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, uint32_t val) { + struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus); + + if ( unlikely(!bridge) ) + { + printk(XENLOG_ERR "Unable to find bridge for "PRI_pci"\n", + sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn); + return; + } + + if ( unlikely(!bridge->ops->write) ) + return; + + bridge->ops->write(bridge, (uint32_t) sbdf.sbdf, reg, len, val); } /* diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index c582527e92..62715b4676 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -261,6 +261,25 @@ err_exit: return err; } +/* + * This function will lookup an hostbridge based on the segment and bus + * number. + */ +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus) +{ + struct pci_host_bridge *bridge; + + list_for_each_entry( bridge, &pci_host_bridges, node ) + { + if ( bridge->segment != segment ) + continue; + if ( (bus < bridge->bus_start) || (bus > bridge->bus_end) ) + continue; + return bridge; + } + + return NULL; +} /* * Local variables: * mode: C diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 22866244d2..756f8637ab 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -20,6 +20,7 @@ #ifdef CONFIG_HAS_PCI #define pci_to_dev(pcidev) (&(pcidev)->arch.dev) +#define PRI_pci "%04x:%02x:%02x.%u" /* Arch pci dev struct */ struct arch_pci_dev { @@ -86,6 +87,7 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, uint32_t sbdf, void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, uint32_t sbdf, uint32_t where); +struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus); #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { }; From patchwork Thu Aug 19 12:02:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8D76C4338F for ; Thu, 19 Aug 2021 12:07:59 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F2976112E for ; Thu, 19 Aug 2021 12:07:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5F2976112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168661.307927 (Exim 4.92) (envelope-from ) id 1mGgps-0007cw-Gv; Thu, 19 Aug 2021 12:07:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168661.307927; Thu, 19 Aug 2021 12:07:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgps-0007cp-Db; Thu, 19 Aug 2021 12:07:52 +0000 Received: by outflank-mailman (input) for mailman id 168661; Thu, 19 Aug 2021 12:07:51 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgpr-0007cb-OH for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:07:51 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id c59c1134-6d59-4ab3-94a4-2c3029f94130; Thu, 19 Aug 2021 12:07:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3AE0831B; Thu, 19 Aug 2021 05:07:50 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 53BD83F70D; Thu, 19 Aug 2021 05:07:49 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c59c1134-6d59-4ab3-94a4-2c3029f94130 From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 09/14] xen/arm: Add cmdline boot option "pci=on" Date: Thu, 19 Aug 2021 13:02:49 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Add cmdline boot option "pci=on" to enable/disable the PCI init during boot. Signed-off-by: Rahul Singh --- xen/arch/arm/pci/pci.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c index d1c9cf997d..dc63bbc2a2 100644 --- a/xen/arch/arm/pci/pci.c +++ b/xen/arch/arm/pci/pci.c @@ -62,8 +62,38 @@ static void __init acpi_pci_init(void) static inline void __init acpi_pci_init(void) { } #endif +static bool __initdata param_pci_enable; + +static int __init parse_pci_param(const char *arg) +{ + if ( !arg ) + { + param_pci_enable = false; + return 0; + } + + switch ( parse_bool(arg, NULL) ) + { + case 0: + param_pci_enable = false; + return 0; + case 1: + param_pci_enable = true; + return 0; + } + + return -EINVAL; +} +custom_param("pci", parse_pci_param); + static int __init pci_init(void) { + /* + * Enable PCI when has been enabled explicitly (pci=on) + */ + if ( !param_pci_enable) + return 0; + if ( acpi_disabled ) dt_pci_init(); else From patchwork Thu Aug 19 12:02:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3937C4338F for ; Thu, 19 Aug 2021 12:08:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6489561152 for ; Thu, 19 Aug 2021 12:08:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6489561152 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168667.307938 (Exim 4.92) (envelope-from ) id 1mGgqD-0008DQ-Qc; Thu, 19 Aug 2021 12:08:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168667.307938; Thu, 19 Aug 2021 12:08:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqD-0008DJ-NI; Thu, 19 Aug 2021 12:08:13 +0000 Received: by outflank-mailman (input) for mailman id 168667; Thu, 19 Aug 2021 12:08:13 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqC-00080Z-Vu for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:08:13 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 6f1d4a33-0e66-43fc-8496-4977d88b3586; Thu, 19 Aug 2021 12:08:10 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C101031B; Thu, 19 Aug 2021 05:08:09 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D9FA93F70D; Thu, 19 Aug 2021 05:08:08 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6f1d4a33-0e66-43fc-8496-4977d88b3586 From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 10/14] xen/arm: Discovering PCI devices and add the PCI devices in XEN. Date: Thu, 19 Aug 2021 13:02:50 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Hardware domain is in charge of doing the PCI enumeration and will discover the PCI devices and then will communicate to XEN via hyper call PHYSDEVOP_pci_device_add to add the PCI devices in XEN. Signed-off-by: Rahul Singh --- xen/arch/arm/physdev.c | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c index e91355fe22..ccce8f0eba 100644 --- a/xen/arch/arm/physdev.c +++ b/xen/arch/arm/physdev.c @@ -9,12 +9,45 @@ #include #include #include - +#include +#include int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) { - gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd); - return -ENOSYS; + int ret = 0; + + switch ( cmd ) + { +#ifdef CONFIG_HAS_PCI + case PHYSDEVOP_pci_device_add: { + struct physdev_pci_device_add add; + struct pci_dev_info pdev_info; + nodeid_t node = NUMA_NO_NODE; + + ret = -EFAULT; + if ( copy_from_guest(&add, arg, 1) != 0 ) + break; + + pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN); + if ( add.flags & XEN_PCI_DEV_VIRTFN ) + { + pdev_info.is_virtfn = 1; + pdev_info.physfn.bus = add.physfn.bus; + pdev_info.physfn.devfn = add.physfn.devfn; + } + else + pdev_info.is_virtfn = 0; + + ret = pci_add_device(add.seg, add.bus, add.devfn, &pdev_info, node); + break; + } +#endif + default: + gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd); + ret = -ENOSYS; + } + + return ret; } /* From patchwork Thu Aug 19 12:02:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A71CC4338F for ; Thu, 19 Aug 2021 12:08:39 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 04FDA61156 for ; Thu, 19 Aug 2021 12:08:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 04FDA61156 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168670.307949 (Exim 4.92) (envelope-from ) id 1mGgqU-0000If-3S; Thu, 19 Aug 2021 12:08:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168670.307949; Thu, 19 Aug 2021 12:08:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqU-0000Hc-0Q; Thu, 19 Aug 2021 12:08:30 +0000 Received: by outflank-mailman (input) for mailman id 168670; Thu, 19 Aug 2021 12:08:28 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqS-0000Gt-Ob for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:08:28 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 292733d4-00e6-11ec-a5f6-12813bfff9fa; Thu, 19 Aug 2021 12:08:27 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4F8C631B; Thu, 19 Aug 2021 05:08:27 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 18D1B3F70D; Thu, 19 Aug 2021 05:08:25 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 292733d4-00e6-11ec-a5f6-12813bfff9fa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Jan Beulich , Paul Durrant , =?utf-8?q?Rog?= =?utf-8?q?er_Pau_Monn=C3=A9?= Subject: [PATCH v1 11/14] xen/arm: Enable the existing x86 virtual PCI support for ARM. Date: Thu, 19 Aug 2021 13:02:51 +0100 Message-Id: <370f4f87c148eaee5ac5ec69346828e6473f0f2d.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 The existing VPCI support available for X86 is adapted for Arm. When the device is added to XEN via the hyper call “PHYSDEVOP_pci_device_add”, VPCI handler for the config space access is added to the Xen to emulate the PCI devices config space. A MMIO trap handler for the PCI ECAM space is registered in XEN so that when guest is trying to access the PCI config space,XEN will trap the access and emulate read/write using the VPCI and not the real PCI hardware. Signed-off-by: Rahul Singh --- xen/arch/arm/Makefile | 1 + xen/arch/arm/domain.c | 4 ++ xen/arch/arm/vpci.c | 96 +++++++++++++++++++++++++++++++++++ xen/arch/arm/vpci.h | 37 ++++++++++++++ xen/drivers/passthrough/pci.c | 7 +++ xen/drivers/vpci/Makefile | 3 +- xen/drivers/vpci/header.c | 2 + xen/include/asm-arm/domain.h | 5 +- xen/include/asm-arm/pci.h | 8 +++ xen/include/public/arch-arm.h | 4 ++ 10 files changed, 165 insertions(+), 2 deletions(-) create mode 100644 xen/arch/arm/vpci.c create mode 100644 xen/arch/arm/vpci.h diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 0e14a5e5c8..7cdce684a4 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -7,6 +7,7 @@ obj-y += platforms/ endif obj-$(CONFIG_TEE) += tee/ obj-$(CONFIG_HAS_PCI) += pci/ +obj-$(CONFIG_HAS_VPCI) += vpci.o obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o obj-y += bootfdt.init.o diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index 19c756ac3d..d99c653626 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -40,6 +40,7 @@ #include #include "vuart.h" +#include "vpci.h" DEFINE_PER_CPU(struct vcpu *, curr_vcpu); @@ -767,6 +768,9 @@ int arch_domain_create(struct domain *d, if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) ) goto fail; + if ( (rc = domain_vpci_init(d)) != 0 ) + goto fail; + return 0; fail: diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c new file mode 100644 index 0000000000..da8b1ca13c --- /dev/null +++ b/xen/arch/arm/vpci.c @@ -0,0 +1,96 @@ +/* + * xen/arch/arm/vpci.c + * Copyright (c) 2021 Arm Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ +#include +#include + +/* Do some sanity checks. */ +static bool vpci_mmio_access_allowed(unsigned int reg, unsigned int len) +{ + /* Check access size. */ + if ( len != 1 && len != 2 && len != 4 && len != 8 ) + return false; + + /* Check that access is size aligned. */ + if ( (reg & (len - 1)) ) + return false; + + return true; +} + +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, + register_t *r, void *p) +{ + unsigned int reg; + pci_sbdf_t sbdf; + uint32_t data = 0; + unsigned int size = 1U << info->dabt.size; + + sbdf.sbdf = (((info->gpa) & 0x0ffff000) >> 12); + reg = (((info->gpa) & 0x00000ffc) | (info->gpa & 3)); + + if ( !vpci_mmio_access_allowed(reg, size) ) + return 1; + + data = vpci_read(sbdf, reg, size); + + memcpy(r, &data, size); + + return 1; +} + +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, + register_t r, void *p) +{ + unsigned int reg; + pci_sbdf_t sbdf; + uint32_t data = r; + unsigned int size = 1U << info->dabt.size; + + sbdf.sbdf = (((info->gpa) & 0x0ffff000) >> 12); + reg = (((info->gpa) & 0x00000ffc) | (info->gpa & 3)); + + if ( !vpci_mmio_access_allowed(reg, size) ) + return 1; + + vpci_write(sbdf, reg, size, data); + + return 1; +} + +static const struct mmio_handler_ops vpci_mmio_handler = { + .read = vpci_mmio_read, + .write = vpci_mmio_write, +}; + +int domain_vpci_init(struct domain *d) +{ + if ( !has_vpci(d) ) + return 0; + + register_mmio_handler(d, &vpci_mmio_handler, + GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL); + + return 0; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ + diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h new file mode 100644 index 0000000000..8a093bb705 --- /dev/null +++ b/xen/arch/arm/vpci.h @@ -0,0 +1,37 @@ +/* + * xen/arch/arm/vpci.h + * Copyright (c) 2021 Arm Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#ifndef __ARCH_ARM_VPCI_H__ +#define __ARCH_ARM_VPCI_H__ + +#ifdef CONFIG_HAS_VPCI +int domain_vpci_init(struct domain *d); +#else +static inline int domain_vpci_init(struct domain *d) +{ + return 0; +} +#endif + +#endif /* __ARCH_ARM_VPCI_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index c23c8cb06b..56e261e9bd 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -767,6 +767,13 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, else iommu_enable_device(pdev); +#ifdef CONFIG_ARM + ret = vpci_add_handlers(pdev); + if ( ret ) { + printk(XENLOG_ERR "setup of vPCI for failed: %d\n",ret); + goto out; + } +#endif pci_enable_acs(pdev); out: diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile index 55d1bdfda0..1a1413b93e 100644 --- a/xen/drivers/vpci/Makefile +++ b/xen/drivers/vpci/Makefile @@ -1 +1,2 @@ -obj-y += vpci.o header.o msi.o msix.o +obj-y += vpci.o header.o +obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index ba9a036202..f8cd55e7c0 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd, * FIXME: punching holes after the p2m has been set up might be racy for * DomU usage, needs to be revisited. */ +#ifdef CONFIG_HAS_PCI_MSI if ( map && !rom_only && vpci_make_msix_hole(pdev) ) return; +#endif for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h index c9277b5c6d..d742b94bd6 100644 --- a/xen/include/asm-arm/domain.h +++ b/xen/include/asm-arm/domain.h @@ -262,7 +262,10 @@ static inline void arch_vcpu_block(struct vcpu *v) {} #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag) -#define has_vpci(d) ({ (void)(d); false; }) +/* For X86 VPCI is enabled and tested for PVH DOM0 only but + * for ARM we enable support VPCI for guest domain also. + */ +#define has_vpci(d) ({ (void)(d); IS_ENABLED(CONFIG_HAS_VPCI); }) #endif /* __ASM_DOMAIN_H__ */ diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 756f8637ab..c58152de80 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -27,6 +27,14 @@ struct arch_pci_dev { struct device dev; }; +/* Arch-specific MSI data for vPCI. */ +struct vpci_arch_msi { +}; + +/* Arch-specific MSI-X entry data for vPCI. */ +struct vpci_arch_msix_entry { +}; + /* * struct to hold the mappings of a config space window. This * is expected to be used as sysdata for PCI controllers that diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h index 64a2ca30da..0a9749e768 100644 --- a/xen/include/public/arch-arm.h +++ b/xen/include/public/arch-arm.h @@ -422,6 +422,10 @@ typedef uint64_t xen_callback_t; #define GUEST_PL011_BASE xen_mk_ullong(0x22000000) #define GUEST_PL011_SIZE xen_mk_ullong(0x00001000) +/* VPCI ECAM mappings */ +#define GUEST_VPCI_ECAM_BASE xen_mk_ullong(0x10000000) +#define GUEST_VPCI_ECAM_SIZE xen_mk_ullong(0x10000000) + /* * 16MB == 4096 pages reserved for guest to use as a region to map its * grant table in. From patchwork Thu Aug 19 12:02:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A1F5C4338F for ; Thu, 19 Aug 2021 12:08:46 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 36BFB61152 for ; Thu, 19 Aug 2021 12:08:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 36BFB61152 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168673.307960 (Exim 4.92) (envelope-from ) id 1mGgqd-0000mG-Io; Thu, 19 Aug 2021 12:08:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168673.307960; Thu, 19 Aug 2021 12:08:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqd-0000m3-FM; Thu, 19 Aug 2021 12:08:39 +0000 Received: by outflank-mailman (input) for mailman id 168673; Thu, 19 Aug 2021 12:08:38 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqc-0000kV-5Z for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:08:38 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 57438a67-03ce-48a9-a6f0-9c76bd89baf1; Thu, 19 Aug 2021 12:08:36 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A355B31B; Thu, 19 Aug 2021 05:08:36 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0124A3F70D; Thu, 19 Aug 2021 05:08:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 57438a67-03ce-48a9-a6f0-9c76bd89baf1 From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Volodymyr Babchuk Subject: [PATCH v1 12/14] arm/libxl: Emulated PCI device tree node in libxl Date: Thu, 19 Aug 2021 13:02:52 +0100 Message-Id: <31e48fe4b219ebaf5f1d9f3f7ca9b433caadc5b2.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: libxl will create an emulated PCI device tree node in the device tree to enable the guest OS to discover the virtual PCI during guest boot. Emulated PCI device tree node will only be created when there is any device assigned to guest. A new area has been reserved in the arm guest physical map at which the VPCI bus is declared in the device tree (reg and ranges parameters of the node). Signed-off-by: Rahul Singh --- tools/libs/light/libxl_arm.c | 109 ++++++++++++++++++++++++++ tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_parse.c | 2 + xen/include/public/arch-arm.h | 11 +++ xen/include/public/device_tree_defs.h | 1 + 5 files changed, 124 insertions(+) diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index e3140a6e00..a091e97e76 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -269,6 +269,58 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt, return fdt_property(fdt, "reg", regs, sizeof(regs)); } +static int fdt_property_values(libxl__gc *gc, void *fdt, + const char *name, unsigned num_cells, ...) +{ + uint32_t prop[num_cells]; + be32 *cells = &prop[0]; + int i; + va_list ap; + uint32_t arg; + + va_start(ap, num_cells); + for (i = 0 ; i < num_cells; i++) { + arg = va_arg(ap, uint32_t); + set_cell(&cells, 1, arg); + } + va_end(ap); + + return fdt_property(fdt, name, prop, sizeof(prop)); +} + +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt, + unsigned addr_cells, + unsigned size_cells, + unsigned num_regs, ...) +{ + uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)]; + be32 *cells = ®s[0]; + int i; + va_list ap; + uint64_t arg; + + va_start(ap, num_regs); + for (i = 0 ; i < num_regs; i++) { + /* Set the memory bit field */ + arg = va_arg(ap, uint64_t); + set_cell(&cells, 1, arg); + + /* Set the vpci bus address */ + arg = addr_cells ? va_arg(ap, uint64_t) : 0; + set_cell(&cells, addr_cells , arg); + + /* Set the cpu bus address where vpci address is mapped */ + set_cell(&cells, addr_cells, arg); + + /* Set the vpci size requested */ + arg = size_cells ? va_arg(ap, uint64_t) : 0; + set_cell(&cells, size_cells,arg); + } + va_end(ap); + + return fdt_property(fdt, "ranges", regs, sizeof(regs)); +} + static int make_root_properties(libxl__gc *gc, const libxl_version_info *vers, void *fdt) @@ -668,6 +720,57 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt, return 0; } +static int make_vpci_node(libxl__gc *gc, void *fdt, + const struct arch_info *ainfo, + struct xc_dom_image *dom) +{ + int res; + const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE; + const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE; + const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base); + + res = fdt_begin_node(fdt, name); + if (res) return res; + + res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic"); + if (res) return res; + + res = fdt_property_string(fdt, "device_type", "pci"); + if (res) return res; + + res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, + GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size); + if (res) return res; + + res = fdt_property_values(gc, fdt, "bus-range", 2, 0,17); + if (res) return res; + + res = fdt_property_cell(fdt, "#address-cells", 3); + if (res) return res; + + res = fdt_property_cell(fdt, "#size-cells", 2); + if (res) return res; + + res = fdt_property_string(fdt, "status", "okay"); + if (res) return res; + + res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, + GUEST_ROOT_SIZE_CELLS, 2, + GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE, + GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR, + GUEST_VPCI_PREFETCH_MEM_SIZE); + if (res) return res; + + res = fdt_property_values(gc, fdt, "msi-map", 4, 0, GUEST_PHANDLE_ITS, + 0, 0x10000); + if (res) return res; + + res = fdt_end_node(fdt); + if (res) return res; + + return 0; +} + static const struct arch_info *get_arch_info(libxl__gc *gc, const struct xc_dom_image *dom) { @@ -971,6 +1074,9 @@ next_resize: if (info->tee == LIBXL_TEE_TYPE_OPTEE) FDT( make_optee_node(gc, fdt) ); + if (libxl_defbool_val(info->arch_arm.vpci)) + FDT( make_vpci_node(gc, fdt, ainfo, dom) ); + if (pfdt) FDT( copy_partial_fdt(gc, fdt, pfdt) ); @@ -1189,6 +1295,9 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, /* ACPI is disabled by default */ libxl_defbool_setdefault(&b_info->acpi, false); + /* VPCI is disabled by default */ + libxl_defbool_setdefault(&b_info->arch_arm.vpci, false); + if (b_info->type != LIBXL_DOMAIN_TYPE_PV) return; diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index 3f9fff653a..78b1ddf0b8 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -644,6 +644,7 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("arch_arm", Struct(None, [("gic_version", libxl_gic_version), ("vuart", libxl_vuart_type), + ("vpci", libxl_defbool), ])), ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool), ])), diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index 17dddb4cd5..ffafbeffb4 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -1497,6 +1497,8 @@ void parse_config_data(const char *config_source, } if (d_config->num_pcidevs && c_info->type == LIBXL_DOMAIN_TYPE_PV) libxl_defbool_set(&b_info->u.pv.e820_host, true); + if (d_config->num_pcidevs) + libxl_defbool_set(&b_info->arch_arm.vpci, true); } if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) { diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h index 0a9749e768..01d13e669e 100644 --- a/xen/include/public/arch-arm.h +++ b/xen/include/public/arch-arm.h @@ -426,6 +426,17 @@ typedef uint64_t xen_callback_t; #define GUEST_VPCI_ECAM_BASE xen_mk_ullong(0x10000000) #define GUEST_VPCI_ECAM_SIZE xen_mk_ullong(0x10000000) +/* PCI-PCIe memory space types */ +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM xen_mk_ullong(0x42000000) +#define GUEST_VPCI_ADDR_TYPE_MEM xen_mk_ullong(0x02000000) + +/* Guest PCI-PCIe memory space where config space and BAR will be available.*/ +#define GUEST_VPCI_PREFETCH_MEM_ADDR xen_mk_ullong(0x900000000) +#define GUEST_VPCI_MEM_ADDR xen_mk_ullong(0x20000000) + +#define GUEST_VPCI_PREFETCH_MEM_SIZE xen_mk_ullong(0x2000000000) +#define GUEST_VPCI_MEM_SIZE xen_mk_ullong(0x04000000) + /* * 16MB == 4096 pages reserved for guest to use as a region to map its * grant table in. diff --git a/xen/include/public/device_tree_defs.h b/xen/include/public/device_tree_defs.h index 209d43de3f..d24d86b1fc 100644 --- a/xen/include/public/device_tree_defs.h +++ b/xen/include/public/device_tree_defs.h @@ -7,6 +7,7 @@ * onwards. Reserve a high value for the GIC phandle. */ #define GUEST_PHANDLE_GIC (65000) +#define GUEST_PHANDLE_ITS (64999) #define GUEST_ROOT_ADDRESS_CELLS 2 #define GUEST_ROOT_SIZE_CELLS 2 From patchwork Thu Aug 19 12:02:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446979 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2921C4320A for ; Thu, 19 Aug 2021 12:09:09 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7EEAC6112E for ; Thu, 19 Aug 2021 12:09:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7EEAC6112E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168683.307971 (Exim 4.92) (envelope-from ) id 1mGgqz-0001ch-RS; Thu, 19 Aug 2021 12:09:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168683.307971; Thu, 19 Aug 2021 12:09:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqz-0001cY-OR; Thu, 19 Aug 2021 12:09:01 +0000 Received: by outflank-mailman (input) for mailman id 168683; Thu, 19 Aug 2021 12:09:00 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqy-0001W6-GO for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:09:00 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 39acc188-00e6-11ec-a5f6-12813bfff9fa; Thu, 19 Aug 2021 12:08:55 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 128DC31B; Thu, 19 Aug 2021 05:08:55 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2C87D3F70D; Thu, 19 Aug 2021 05:08:54 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 39acc188-00e6-11ec-a5f6-12813bfff9fa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 13/14] xen/arm: Fixed error when PCI device is assigned to guest Date: Thu, 19 Aug 2021 13:02:53 +0100 Message-Id: <917720808121c3098690b51d55f2d60118ec6408.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: XEN_DOMCTL_ioport_permission, PHYSDEVOP_unmap_pirq, PHYSDEVOP_unmap_pirq are unimplemented for ARM. When libxl assigning a PCI device to the guest error is observed related to above functions. Implement dummy functions to fix the error. Signed-off-by: Rahul Singh --- xen/arch/arm/domctl.c | 2 ++ xen/arch/arm/physdev.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c index b7d27f37df..38813be893 100644 --- a/xen/arch/arm/domctl.c +++ b/xen/arch/arm/domctl.c @@ -173,6 +173,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, return rc; } + case XEN_DOMCTL_ioport_permission: + return 0; default: { int rc; diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c index ccce8f0eba..4a0affeada 100644 --- a/xen/arch/arm/physdev.c +++ b/xen/arch/arm/physdev.c @@ -42,6 +42,9 @@ int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) break; } #endif + case PHYSDEVOP_unmap_pirq: + case PHYSDEVOP_map_pirq: + break; default: gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd); ret = -ENOSYS; From patchwork Thu Aug 19 12:02:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12446981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A39CDC4338F for ; Thu, 19 Aug 2021 12:09:09 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 708FB61152 for ; Thu, 19 Aug 2021 12:09:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 708FB61152 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.168685.307982 (Exim 4.92) (envelope-from ) id 1mGgr1-0001tj-4y; Thu, 19 Aug 2021 12:09:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 168685.307982; Thu, 19 Aug 2021 12:09:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgr1-0001tW-0z; Thu, 19 Aug 2021 12:09:03 +0000 Received: by outflank-mailman (input) for mailman id 168685; Thu, 19 Aug 2021 12:09:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mGgqz-0001cS-MD for xen-devel@lists.xenproject.org; Thu, 19 Aug 2021 12:09:01 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id d601c046-67f9-4157-93be-4ba20d430c33; Thu, 19 Aug 2021 12:09:00 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6E29F31B; Thu, 19 Aug 2021 05:09:00 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 892853F70D; Thu, 19 Aug 2021 05:08:59 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d601c046-67f9-4157-93be-4ba20d430c33 From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v1 14/14] xen/arm: Add linux,pci-domain property for hwdom if not available. Date: Thu, 19 Aug 2021 13:02:54 +0100 Message-Id: <0b979ccab6a4c9b2070748709f737f41fa108a42.1629366665.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: If the property is not present in the device tree node for host bridge, XEN while creating the dtb for hwdom will create this property and assigns the already allocated segment to the host bridge so that XEN and linux will have the same segment for the host bridges. Signed-off-by: Rahul Singh --- xen/arch/arm/domain_build.c | 18 ++++++++++++++++++ xen/arch/arm/pci/pci-host-common.c | 21 +++++++++++++++++++++ xen/include/asm-arm/pci.h | 3 +++ 3 files changed, 42 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 6c86d52781..e0cf2ff19d 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -581,6 +581,24 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo, return res; } +#ifdef CONFIG_HAS_PCI + if ( dt_device_type_is_equal(node, "pci") ) + { + if ( !dt_find_property(node, "linux,pci-domain", NULL) ) + { + uint16_t segment; + + res = pci_get_host_bridge_segment(node, &segment); + if ( res < 0 ) + return res; + + res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment); + if ( res ) + return res; + } + } +#endif + /* * Override the property "status" to disable the device when it's * marked for passthrough. diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index 62715b4676..5e34252deb 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -280,6 +280,27 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus) return NULL; } + +/* + * This function will lookup an hostbridge based on config space address. + */ +int pci_get_host_bridge_segment(const struct dt_device_node *node, + uint16_t *segment) +{ + struct pci_host_bridge *bridge; + + list_for_each_entry( bridge, &pci_host_bridges, node ) + { + if ( bridge->dt_node != node ) + continue; + + *segment = bridge->segment; + return 0; + } + + return -EINVAL; +} + /* * Local variables: * mode: C diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index c58152de80..2d4610a23a 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -96,6 +96,9 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, uint32_t sbdf, uint32_t where); struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus); +int pci_get_host_bridge_segment(const struct dt_device_node *node, + uint16_t *segment); + #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { };