From patchwork Tue Jul 13 17:06:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandru Elisei X-Patchwork-Id: 12374659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99506C07E96 for ; Tue, 13 Jul 2021 17:05:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83E8561363 for ; Tue, 13 Jul 2021 17:05:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231837AbhGMRIc (ORCPT ); Tue, 13 Jul 2021 13:08:32 -0400 Received: from foss.arm.com ([217.140.110.172]:47582 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232724AbhGMRIb (ORCPT ); Tue, 13 Jul 2021 13:08:31 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5FD801FB; Tue, 13 Jul 2021 10:05:41 -0700 (PDT) Received: from monolith.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E2BB23F7D8; Tue, 13 Jul 2021 10:05:39 -0700 (PDT) From: Alexandru Elisei To: will@kernel.org, julien.thierry.kdev@gmail.com, kvm@vger.kernel.org Cc: andre.przywara@arm.com, sami.mujawar@arm.com, lorenzo.pieralisi@arm.com, maz@kernel.org, pierre.gondois@arm.com Subject: [PATCH v3 kvmtool 1/4] Move fdt_irq_fn typedef to fdt.h Date: Tue, 13 Jul 2021 18:06:28 +0100 Message-Id: <20210713170631.155595-2-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210713170631.155595-1-alexandru.elisei@arm.com> References: <20210713170631.155595-1-alexandru.elisei@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The device tree code passes the function generate_irq_prop() to MMIO devices to create the "interrupts" property. The typedef fdt_irq_fn is the type used to pass the function to the device. It makes more sense for the typedef to be in fdt.h with the rest of the device tree functions, so move it there. Reviewed-by: Andre Przywara Signed-off-by: Alexandru Elisei --- hw/rtc.c | 1 + include/kvm/fdt.h | 2 ++ include/kvm/kvm.h | 1 - 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/rtc.c b/hw/rtc.c index aec31c52a85a..9b8785a869dd 100644 --- a/hw/rtc.c +++ b/hw/rtc.c @@ -1,5 +1,6 @@ #include "kvm/rtc.h" +#include "kvm/fdt.h" #include "kvm/ioport.h" #include "kvm/kvm.h" diff --git a/include/kvm/fdt.h b/include/kvm/fdt.h index 4e6157256482..060c37b947cc 100644 --- a/include/kvm/fdt.h +++ b/include/kvm/fdt.h @@ -25,6 +25,8 @@ enum irq_type { IRQ_TYPE_LEVEL_MASK = (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH), }; +typedef void (*fdt_irq_fn)(void *fdt, u8 irq, enum irq_type irq_type); + extern char *fdt_stdout_path; /* Helper for the various bits of code that generate FDT nodes */ diff --git a/include/kvm/kvm.h b/include/kvm/kvm.h index 6c28afa3f0bb..56e9c8e347a0 100644 --- a/include/kvm/kvm.h +++ b/include/kvm/kvm.h @@ -44,7 +44,6 @@ struct kvm_cpu; typedef void (*mmio_handler_fn)(struct kvm_cpu *vcpu, u64 addr, u8 *data, u32 len, u8 is_write, void *ptr); -typedef void (*fdt_irq_fn)(void *fdt, u8 irq, enum irq_type irq_type); enum { KVM_VMSTATE_RUNNING, From patchwork Tue Jul 13 17:06:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandru Elisei X-Patchwork-Id: 12374661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB89CC11F66 for ; Tue, 13 Jul 2021 17:05:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B3ADD6128E for ; Tue, 13 Jul 2021 17:05:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232890AbhGMRIe (ORCPT ); Tue, 13 Jul 2021 13:08:34 -0400 Received: from foss.arm.com ([217.140.110.172]:47596 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232813AbhGMRId (ORCPT ); Tue, 13 Jul 2021 13:08:33 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7183B1FB; Tue, 13 Jul 2021 10:05:43 -0700 (PDT) Received: from monolith.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A1C973F7D8; Tue, 13 Jul 2021 10:05:41 -0700 (PDT) From: Alexandru Elisei To: will@kernel.org, julien.thierry.kdev@gmail.com, kvm@vger.kernel.org Cc: andre.przywara@arm.com, sami.mujawar@arm.com, lorenzo.pieralisi@arm.com, maz@kernel.org, pierre.gondois@arm.com Subject: [PATCH v3 kvmtool 2/4] arm/fdt.c: Don't generate the node if generator function is NULL Date: Tue, 13 Jul 2021 18:06:29 +0100 Message-Id: <20210713170631.155595-3-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210713170631.155595-1-alexandru.elisei@arm.com> References: <20210713170631.155595-1-alexandru.elisei@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Print a more helpful debugging message when a MMIO device hasn't set a function to generate an FDT node instead of causing a segmentation fault by dereferencing a NULL pointer. Reviewed-by: Andre Przywara Signed-off-by: Alexandru Elisei --- arm/fdt.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arm/fdt.c b/arm/fdt.c index 02091e9e0bee..7032985e99a3 100644 --- a/arm/fdt.c +++ b/arm/fdt.c @@ -171,7 +171,12 @@ static int setup_fdt(struct kvm *kvm) dev_hdr = device__first_dev(DEVICE_BUS_MMIO); while (dev_hdr) { generate_mmio_fdt_nodes = dev_hdr->data; - generate_mmio_fdt_nodes(fdt, dev_hdr, generate_irq_prop); + if (generate_mmio_fdt_nodes) { + generate_mmio_fdt_nodes(fdt, dev_hdr, generate_irq_prop); + } else { + pr_debug("Missing FDT node generator for MMIO device %d", + dev_hdr->dev_num); + } dev_hdr = device__next_dev(dev_hdr); } From patchwork Tue Jul 13 17:06:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandru Elisei X-Patchwork-Id: 12374663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71DF3C07E95 for ; Tue, 13 Jul 2021 17:05:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B45F6128E for ; Tue, 13 Jul 2021 17:05:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233090AbhGMRIf (ORCPT ); Tue, 13 Jul 2021 13:08:35 -0400 Received: from foss.arm.com ([217.140.110.172]:47606 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232813AbhGMRIf (ORCPT ); Tue, 13 Jul 2021 13:08:35 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 349FD1FB; Tue, 13 Jul 2021 10:05:45 -0700 (PDT) Received: from monolith.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B459F3F7D8; Tue, 13 Jul 2021 10:05:43 -0700 (PDT) From: Alexandru Elisei To: will@kernel.org, julien.thierry.kdev@gmail.com, kvm@vger.kernel.org Cc: andre.przywara@arm.com, sami.mujawar@arm.com, lorenzo.pieralisi@arm.com, maz@kernel.org, pierre.gondois@arm.com Subject: [PATCH v3 kvmtool 3/4] arm/arm64: Add PCI Express 1.1 support Date: Tue, 13 Jul 2021 18:06:30 +0100 Message-Id: <20210713170631.155595-4-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210713170631.155595-1-alexandru.elisei@arm.com> References: <20210713170631.155595-1-alexandru.elisei@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org PCI Express comes with an extended addressing scheme, which directly translated into a bigger device configuration space (256->4096 bytes) and bigger PCI configuration space (16->256 MB), as well as mandatory capabilities (power management [1] and PCI Express capability [2]). However, our virtio PCI implementation implements version 0.9 of the protocol and it still uses transitional PCI device ID's, so we have opted to omit the mandatory PCI Express capabilities. For VFIO, the power management and PCI Express capability are left for a subsequent patch. [1] PCI Express Base Specification Revision 1.1, section 7.6 [2] PCI Express Base Specification Revision 1.1, section 7.8 Signed-off-by: Alexandru Elisei Reviewed-by: Andre Przywara --- arm/include/arm-common/kvm-arch.h | 4 ++- arm/pci.c | 2 +- include/kvm/pci.h | 51 ++++++++++++++++++++++++++++--- pci.c | 5 +-- vfio/pci.c | 26 +++++++++------- 5 files changed, 68 insertions(+), 20 deletions(-) diff --git a/arm/include/arm-common/kvm-arch.h b/arm/include/arm-common/kvm-arch.h index 436b67b843fc..c645ac001bca 100644 --- a/arm/include/arm-common/kvm-arch.h +++ b/arm/include/arm-common/kvm-arch.h @@ -49,7 +49,7 @@ #define KVM_PCI_CFG_AREA ARM_AXI_AREA -#define ARM_PCI_CFG_SIZE (1ULL << 24) +#define ARM_PCI_CFG_SIZE (1ULL << 28) #define KVM_PCI_MMIO_AREA (KVM_PCI_CFG_AREA + ARM_PCI_CFG_SIZE) #define ARM_PCI_MMIO_SIZE (ARM_MEMORY_AREA - \ (ARM_AXI_AREA + ARM_PCI_CFG_SIZE)) @@ -77,6 +77,8 @@ #define VIRTIO_RING_ENDIAN (VIRTIO_ENDIAN_LE | VIRTIO_ENDIAN_BE) +#define ARCH_HAS_PCI_EXP 1 + static inline bool arm_addr_in_ioport_region(u64 phys_addr) { u64 limit = KVM_IOPORT_AREA + ARM_IOPORT_SIZE; diff --git a/arm/pci.c b/arm/pci.c index ed325fa4a811..2251f627d8b5 100644 --- a/arm/pci.c +++ b/arm/pci.c @@ -62,7 +62,7 @@ void pci__generate_fdt_nodes(void *fdt) _FDT(fdt_property_cell(fdt, "#address-cells", 0x3)); _FDT(fdt_property_cell(fdt, "#size-cells", 0x2)); _FDT(fdt_property_cell(fdt, "#interrupt-cells", 0x1)); - _FDT(fdt_property_string(fdt, "compatible", "pci-host-cam-generic")); + _FDT(fdt_property_string(fdt, "compatible", "pci-host-ecam-generic")); _FDT(fdt_property(fdt, "dma-coherent", NULL, 0)); _FDT(fdt_property(fdt, "bus-range", bus_range, sizeof(bus_range))); diff --git a/include/kvm/pci.h b/include/kvm/pci.h index bf81323d83b7..42d9e1c5645f 100644 --- a/include/kvm/pci.h +++ b/include/kvm/pci.h @@ -10,6 +10,7 @@ #include "kvm/devices.h" #include "kvm/msi.h" #include "kvm/fdt.h" +#include "kvm/kvm-arch.h" #define pci_dev_err(pci_hdr, fmt, ...) \ pr_err("[%04x:%04x] " fmt, pci_hdr->vendor_id, pci_hdr->device_id, ##__VA_ARGS__) @@ -32,10 +33,49 @@ #define PCI_CONFIG_BUS_FORWARD 0xcfa #define PCI_IO_SIZE 0x100 #define PCI_IOPORT_START 0x6200 -#define PCI_CFG_SIZE (1ULL << 24) struct kvm; +/* + * On some distributions, pci_regs.h doesn't define PCI_CFG_SPACE_SIZE and + * PCI_CFG_SPACE_EXP_SIZE, so we define our own. + */ +#define PCI_CFG_SIZE_LEGACY (1ULL << 24) +#define PCI_DEV_CFG_SIZE_LEGACY 256 +#define PCI_CFG_SIZE_EXTENDED (1ULL << 28) +#define PCI_DEV_CFG_SIZE_EXTENDED 4096 + +#ifdef ARCH_HAS_PCI_EXP +#define PCI_CFG_SIZE PCI_CFG_SIZE_EXTENDED +#define PCI_DEV_CFG_SIZE PCI_DEV_CFG_SIZE_EXTENDED + +union pci_config_address { + struct { +#if __BYTE_ORDER == __LITTLE_ENDIAN + unsigned reg_offset : 2; /* 1 .. 0 */ + unsigned register_number : 10; /* 11 .. 2 */ + unsigned function_number : 3; /* 14 .. 12 */ + unsigned device_number : 5; /* 19 .. 15 */ + unsigned bus_number : 8; /* 27 .. 20 */ + unsigned reserved : 3; /* 30 .. 28 */ + unsigned enable_bit : 1; /* 31 */ +#else + unsigned enable_bit : 1; /* 31 */ + unsigned reserved : 3; /* 30 .. 28 */ + unsigned bus_number : 8; /* 27 .. 20 */ + unsigned device_number : 5; /* 19 .. 15 */ + unsigned function_number : 3; /* 14 .. 12 */ + unsigned register_number : 10; /* 11 .. 2 */ + unsigned reg_offset : 2; /* 1 .. 0 */ +#endif + }; + u32 w; +}; + +#else +#define PCI_CFG_SIZE PCI_CFG_SIZE_LEGACY +#define PCI_DEV_CFG_SIZE PCI_DEV_CFG_SIZE_LEGACY + union pci_config_address { struct { #if __BYTE_ORDER == __LITTLE_ENDIAN @@ -58,6 +98,9 @@ union pci_config_address { }; u32 w; }; +#endif /* ARCH_HAS_PCI_EXP */ + +#define PCI_DEV_CFG_MASK (PCI_DEV_CFG_SIZE - 1) struct msix_table { struct msi_msg msg; @@ -110,14 +153,12 @@ typedef int (*bar_deactivate_fn_t)(struct kvm *kvm, int bar_num, void *data); #define PCI_BAR_OFFSET(b) (offsetof(struct pci_device_header, bar[b])) -#define PCI_DEV_CFG_SIZE 256 -#define PCI_DEV_CFG_MASK (PCI_DEV_CFG_SIZE - 1) struct pci_config_operations { void (*write)(struct kvm *kvm, struct pci_device_header *pci_hdr, - u8 offset, void *data, int sz); + u16 offset, void *data, int sz); void (*read)(struct kvm *kvm, struct pci_device_header *pci_hdr, - u8 offset, void *data, int sz); + u16 offset, void *data, int sz); }; struct pci_device_header { diff --git a/pci.c b/pci.c index d6da79e0a56a..e593033164c1 100644 --- a/pci.c +++ b/pci.c @@ -353,7 +353,8 @@ static void pci_config_bar_wr(struct kvm *kvm, void pci__config_wr(struct kvm *kvm, union pci_config_address addr, void *data, int size) { void *base; - u8 bar, offset; + u8 bar; + u16 offset; struct pci_device_header *pci_hdr; u8 dev_num = addr.device_number; u32 value = 0; @@ -392,7 +393,7 @@ void pci__config_wr(struct kvm *kvm, union pci_config_address addr, void *data, void pci__config_rd(struct kvm *kvm, union pci_config_address addr, void *data, int size) { - u8 offset; + u16 offset; struct pci_device_header *pci_hdr; u8 dev_num = addr.device_number; diff --git a/vfio/pci.c b/vfio/pci.c index 49ecd12a38cd..7784212f9ad9 100644 --- a/vfio/pci.c +++ b/vfio/pci.c @@ -313,7 +313,7 @@ out_unlock: } static void vfio_pci_msix_cap_write(struct kvm *kvm, - struct vfio_device *vdev, u8 off, + struct vfio_device *vdev, u16 off, void *data, int sz) { struct vfio_pci_device *pdev = &vdev->pci; @@ -345,7 +345,7 @@ static void vfio_pci_msix_cap_write(struct kvm *kvm, } static int vfio_pci_msi_vector_write(struct kvm *kvm, struct vfio_device *vdev, - u8 off, u8 *data, u32 sz) + u16 off, u8 *data, u32 sz) { size_t i; u32 mask = 0; @@ -393,7 +393,7 @@ static int vfio_pci_msi_vector_write(struct kvm *kvm, struct vfio_device *vdev, } static void vfio_pci_msi_cap_write(struct kvm *kvm, struct vfio_device *vdev, - u8 off, u8 *data, u32 sz) + u16 off, u8 *data, u32 sz) { u8 ctrl; struct msi_msg msg; @@ -553,7 +553,7 @@ out: } static void vfio_pci_cfg_read(struct kvm *kvm, struct pci_device_header *pci_hdr, - u8 offset, void *data, int sz) + u16 offset, void *data, int sz) { struct vfio_region_info *info; struct vfio_pci_device *pdev; @@ -571,7 +571,7 @@ static void vfio_pci_cfg_read(struct kvm *kvm, struct pci_device_header *pci_hdr } static void vfio_pci_cfg_write(struct kvm *kvm, struct pci_device_header *pci_hdr, - u8 offset, void *data, int sz) + u16 offset, void *data, int sz) { struct vfio_region_info *info; struct vfio_pci_device *pdev; @@ -658,15 +658,15 @@ static int vfio_pci_parse_caps(struct vfio_device *vdev) { int ret; size_t size; - u8 pos, next; + u16 pos, next; struct pci_cap_hdr *cap; - u8 virt_hdr[PCI_DEV_CFG_SIZE]; + u8 virt_hdr[PCI_DEV_CFG_SIZE_LEGACY]; struct vfio_pci_device *pdev = &vdev->pci; if (!(pdev->hdr.status & PCI_STATUS_CAP_LIST)) return 0; - memset(virt_hdr, 0, PCI_DEV_CFG_SIZE); + memset(virt_hdr, 0, PCI_DEV_CFG_SIZE_LEGACY); pos = pdev->hdr.capabilities & ~3; @@ -699,7 +699,7 @@ static int vfio_pci_parse_caps(struct vfio_device *vdev) /* Wipe remaining capabilities */ pos = PCI_STD_HEADER_SIZEOF; - size = PCI_DEV_CFG_SIZE - PCI_STD_HEADER_SIZEOF; + size = PCI_DEV_CFG_SIZE_LEGACY - PCI_STD_HEADER_SIZEOF; memcpy((void *)&pdev->hdr + pos, virt_hdr + pos, size); return 0; @@ -707,7 +707,7 @@ static int vfio_pci_parse_caps(struct vfio_device *vdev) static int vfio_pci_parse_cfg_space(struct vfio_device *vdev) { - ssize_t sz = PCI_DEV_CFG_SIZE; + ssize_t sz = PCI_DEV_CFG_SIZE_LEGACY; struct vfio_region_info *info; struct vfio_pci_device *pdev = &vdev->pci; @@ -812,7 +812,11 @@ static int vfio_pci_fixup_cfg_space(struct vfio_device *vdev) /* Install our fake Configuration Space */ info = &vdev->regions[VFIO_PCI_CONFIG_REGION_INDEX].info; - hdr_sz = PCI_DEV_CFG_SIZE; + /* + * We don't touch the extended configuration space, let's be cautious + * and not overwrite it all with zeros, or bad things might happen. + */ + hdr_sz = PCI_DEV_CFG_SIZE_LEGACY; if (pwrite(vdev->fd, &pdev->hdr, hdr_sz, info->offset) != hdr_sz) { vfio_dev_err(vdev, "failed to write %zd bytes to Config Space", hdr_sz); From patchwork Tue Jul 13 17:06:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandru Elisei X-Patchwork-Id: 12374665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25A2DC07E96 for ; Tue, 13 Jul 2021 17:05:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E44D60725 for ; Tue, 13 Jul 2021 17:05:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233166AbhGMRIh (ORCPT ); Tue, 13 Jul 2021 13:08:37 -0400 Received: from foss.arm.com ([217.140.110.172]:47616 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232724AbhGMRIh (ORCPT ); Tue, 13 Jul 2021 13:08:37 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E9F63D6E; Tue, 13 Jul 2021 10:05:46 -0700 (PDT) Received: from monolith.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7650C3F7D8; Tue, 13 Jul 2021 10:05:45 -0700 (PDT) From: Alexandru Elisei To: will@kernel.org, julien.thierry.kdev@gmail.com, kvm@vger.kernel.org Cc: andre.przywara@arm.com, sami.mujawar@arm.com, lorenzo.pieralisi@arm.com, maz@kernel.org, pierre.gondois@arm.com Subject: [PATCH v3 kvmtool 4/4] arm/arm64: vfio: Add PCI Express Capability Structure Date: Tue, 13 Jul 2021 18:06:31 +0100 Message-Id: <20210713170631.155595-5-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210713170631.155595-1-alexandru.elisei@arm.com> References: <20210713170631.155595-1-alexandru.elisei@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It turns out that some Linux drivers (like Realtek R8169) fall back to a device-specific configuration method if the device is not PCI Express capable: [ 1.433825] r8169 0000:00:00.0 enp0s0: No native access to PCI extended config space, falling back to CSI Add the PCI Express Capability Structure and populate it for assigned devices, as this is how the Linux PCI driver determines if a device is PCI Express capable. Because we don't emulate a PCI Express link, a root complex or any slot related properties, the PCI Express capability is kept as small as possible by ignoring those fields. Reviewed-by: Andre Przywara Signed-off-by: Alexandru Elisei --- include/kvm/pci.h | 24 ++++++++++++++++++++++++ vfio/pci.c | 18 ++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/include/kvm/pci.h b/include/kvm/pci.h index 42d9e1c5645f..0f2d5bbabdc3 100644 --- a/include/kvm/pci.h +++ b/include/kvm/pci.h @@ -46,6 +46,8 @@ struct kvm; #define PCI_DEV_CFG_SIZE_EXTENDED 4096 #ifdef ARCH_HAS_PCI_EXP +#define arch_has_pci_exp() (true) + #define PCI_CFG_SIZE PCI_CFG_SIZE_EXTENDED #define PCI_DEV_CFG_SIZE PCI_DEV_CFG_SIZE_EXTENDED @@ -73,6 +75,8 @@ union pci_config_address { }; #else +#define arch_has_pci_exp() (false) + #define PCI_CFG_SIZE PCI_CFG_SIZE_LEGACY #define PCI_DEV_CFG_SIZE PCI_DEV_CFG_SIZE_LEGACY @@ -143,6 +147,24 @@ struct pci_cap_hdr { u8 next; }; +struct pci_exp_cap { + u8 cap; + u8 next; + u16 cap_reg; + u32 dev_cap; + u16 dev_ctrl; + u16 dev_status; + u32 link_cap; + u16 link_ctrl; + u16 link_status; + u32 slot_cap; + u16 slot_ctrl; + u16 slot_status; + u16 root_ctrl; + u16 root_cap; + u32 root_status; +}; + struct pci_device_header; typedef int (*bar_activate_fn_t)(struct kvm *kvm, @@ -188,6 +210,8 @@ struct pci_device_header { u8 min_gnt; u8 max_lat; struct msix_cap msix; + /* Used only by architectures which support PCIE */ + struct pci_exp_cap pci_exp; } __attribute__((packed)); /* Pad to PCI config space size */ u8 __pad[PCI_DEV_CFG_SIZE]; diff --git a/vfio/pci.c b/vfio/pci.c index 7784212f9ad9..ea33fd6e5c5a 100644 --- a/vfio/pci.c +++ b/vfio/pci.c @@ -12,6 +12,11 @@ #include +/* Some distros don't have the define. */ +#ifndef PCI_CAP_EXP_RC_ENDPOINT_SIZEOF_V1 +#define PCI_CAP_EXP_RC_ENDPOINT_SIZEOF_V1 12 +#endif + /* Wrapper around UAPI vfio_irq_set */ union vfio_irq_eventfd { struct vfio_irq_set irq; @@ -623,6 +628,12 @@ static ssize_t vfio_pci_cap_size(struct pci_cap_hdr *cap_hdr) return PCI_CAP_MSIX_SIZEOF; case PCI_CAP_ID_MSI: return vfio_pci_msi_cap_size((void *)cap_hdr); + case PCI_CAP_ID_EXP: + /* + * We don't emulate any of the link, slot and root complex + * properties, so ignore them. + */ + return PCI_CAP_EXP_RC_ENDPOINT_SIZEOF_V1; default: pr_err("unknown PCI capability 0x%x", cap_hdr->type); return 0; @@ -694,6 +705,13 @@ static int vfio_pci_parse_caps(struct vfio_device *vdev) pdev->msi.pos = pos; pdev->irq_modes |= VFIO_PCI_IRQ_MODE_MSI; break; + case PCI_CAP_ID_EXP: + if (!arch_has_pci_exp()) + continue; + ret = vfio_pci_add_cap(vdev, virt_hdr, cap, pos); + if (ret) + return ret; + break; } }