From patchwork Wed Jun 14 22:56:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Graf X-Patchwork-Id: 13280526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C211EEB64D8 for ; Wed, 14 Jun 2023 22:57:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q9ZQ8-0008AH-FJ; Wed, 14 Jun 2023 18:56:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q9ZQ6-00088o-Ht; Wed, 14 Jun 2023 18:56:54 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q9ZQ4-0000gw-7Q; Wed, 14 Jun 2023 18:56:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1686783412; x=1718319412; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e84Oi7TM/cZXisj3YC6qPFuW1L/TG43ZAu/OlIMHjng=; b=Ob37v+jKhHVPKmEx2JDFwNvmmozy8MOxn8DjCBD1tw/NmGZ2LCfjO8OP RTYPBJS85ysnSSvv4/bPsJHGcabIPOMgO7yBDkvBFyshQf759I9m3eRpX MxSw9DtklfYrSFFJmsTUjbZgRaZSg2E5BQz1Cod4RumS8Dq11VZrs1uWh E=; X-IronPort-AV: E=Sophos;i="6.00,243,1681171200"; d="scan'208";a="345700640" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-iad-1e-m6i4x-3e1fab07.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-9102.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2023 22:56:41 +0000 Received: from EX19MTAUWC001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-iad-1e-m6i4x-3e1fab07.us-east-1.amazon.com (Postfix) with ESMTPS id 532C6806EE; Wed, 14 Jun 2023 22:56:37 +0000 (UTC) Received: from EX19D020UWC004.ant.amazon.com (10.13.138.149) by EX19MTAUWC001.ant.amazon.com (10.250.64.174) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.26; Wed, 14 Jun 2023 22:56:34 +0000 Received: from dev-dsk-graf-1a-5ce218e4.eu-west-1.amazon.com (10.253.83.51) by EX19D020UWC004.ant.amazon.com (10.13.138.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.26; Wed, 14 Jun 2023 22:56:32 +0000 From: Alexander Graf To: CC: , , Cameron Esfahani , Roman Bolshakov , Stefan Hajnoczi , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Marcel Apfelbaum , Paolo Bonzini , "Peter Maydell" Subject: [PATCH 07/12] gpex: Allow more than 4 legacy IRQs Date: Wed, 14 Jun 2023 22:56:24 +0000 Message-ID: <20230614225626.97734-3-graf@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230614225626.97734-1-graf@amazon.com> References: <20230614224038.86148-1-graf> <20230614225626.97734-1-graf@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.253.83.51] X-ClientProxiedBy: EX19D040UWB004.ant.amazon.com (10.13.138.91) To EX19D020UWC004.ant.amazon.com (10.13.138.149) Precedence: Bulk Received-SPF: pass client-ip=207.171.184.29; envelope-from=prvs=52269c73c=graf@amazon.de; helo=smtp-fw-9102.amazon.com X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Some boards such as vmapple don't do real legacy PCI IRQ swizzling. Instead, they just keep allocating more board IRQ lines for each new legacy IRQ. Let's support that mode by giving instantiators a new "nr_irqs" property they can use to support more than 4 legacy IRQ lines. In this mode, GPEX will export more IRQ lines, one for each device. Signed-off-by: Alexander Graf --- hw/arm/sbsa-ref.c | 2 +- hw/arm/virt.c | 2 +- hw/i386/microvm.c | 2 +- hw/loongarch/virt.c | 2 +- hw/mips/loongson3_virt.c | 2 +- hw/openrisc/virt.c | 12 ++++++------ hw/pci-host/gpex.c | 36 +++++++++++++++++++++++++++++++----- hw/riscv/virt.c | 12 ++++++------ hw/xtensa/virt.c | 2 +- include/hw/pci-host/gpex.h | 7 +++---- 10 files changed, 52 insertions(+), 27 deletions(-) diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c index de21200ff9..2715ea7b2f 100644 --- a/hw/arm/sbsa-ref.c +++ b/hw/arm/sbsa-ref.c @@ -647,7 +647,7 @@ static void create_pcie(SBSAMachineState *sms) /* Map IO port space */ sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, base_pio); - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, qdev_get_gpio_in(sms->gic, irq + i)); gpex_set_irq_num(GPEX_HOST(dev), i, irq + i); diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 9b9f7d9c68..cabb5d14f2 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1466,7 +1466,7 @@ static void create_pcie(VirtMachineState *vms) /* Map IO port space */ sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, base_pio); - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, qdev_get_gpio_in(vms->gic, irq + i)); gpex_set_irq_num(GPEX_HOST(dev), i, irq + i); diff --git a/hw/i386/microvm.c b/hw/i386/microvm.c index 7227a2156c..9ca007b870 100644 --- a/hw/i386/microvm.c +++ b/hw/i386/microvm.c @@ -139,7 +139,7 @@ static void create_gpex(MicrovmMachineState *mms) mms->gpex.mmio64.base, mmio64_alias); } - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, x86ms->gsi[mms->gpex.irq + i]); } diff --git a/hw/loongarch/virt.c b/hw/loongarch/virt.c index ceddec1b23..6a0c2f3103 100644 --- a/hw/loongarch/virt.c +++ b/hw/loongarch/virt.c @@ -512,7 +512,7 @@ static void loongarch_devices_init(DeviceState *pch_pic, LoongArchMachineState * memory_region_add_subregion(get_system_memory(), VIRT_PCI_IO_BASE, pio_alias); - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { sysbus_connect_irq(d, i, qdev_get_gpio_in(pch_pic, 16 + i)); gpex_set_irq_num(GPEX_HOST(gpex_dev), i, 16 + i); diff --git a/hw/mips/loongson3_virt.c b/hw/mips/loongson3_virt.c index 216812f660..acf9fead85 100644 --- a/hw/mips/loongson3_virt.c +++ b/hw/mips/loongson3_virt.c @@ -438,7 +438,7 @@ static inline void loongson3_virt_devices_init(MachineState *machine, virt_memmap[VIRT_PCIE_PIO].base, s->pio_alias); sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, virt_memmap[VIRT_PCIE_PIO].base); - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { irq = qdev_get_gpio_in(pic, PCIE_IRQ_BASE + i); sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, irq); gpex_set_irq_num(GPEX_HOST(dev), i, PCIE_IRQ_BASE + i); diff --git a/hw/openrisc/virt.c b/hw/openrisc/virt.c index f8a68a6a6b..16a5676c4b 100644 --- a/hw/openrisc/virt.c +++ b/hw/openrisc/virt.c @@ -318,7 +318,7 @@ static void create_pcie_irq_map(void *fdt, char *nodename, int irq_base, { int pin, dev; uint32_t irq_map_stride = 0; - uint32_t full_irq_map[GPEX_NUM_IRQS * GPEX_NUM_IRQS * 6] = {}; + uint32_t full_irq_map[PCI_NUM_PINS * PCI_NUM_PINS * 6] = {}; uint32_t *irq_map = full_irq_map; /* @@ -330,11 +330,11 @@ static void create_pcie_irq_map(void *fdt, char *nodename, int irq_base, * possible slot) seeing the interrupt-map-mask will allow the table * to wrap to any number of devices. */ - for (dev = 0; dev < GPEX_NUM_IRQS; dev++) { + for (dev = 0; dev < PCI_NUM_PINS; dev++) { int devfn = dev << 3; - for (pin = 0; pin < GPEX_NUM_IRQS; pin++) { - int irq_nr = irq_base + ((pin + PCI_SLOT(devfn)) % GPEX_NUM_IRQS); + for (pin = 0; pin < PCI_NUM_PINS; pin++) { + int irq_nr = irq_base + ((pin + PCI_SLOT(devfn)) % PCI_NUM_PINS); int i = 0; /* Fill PCI address cells */ @@ -357,7 +357,7 @@ static void create_pcie_irq_map(void *fdt, char *nodename, int irq_base, } qemu_fdt_setprop(fdt, nodename, "interrupt-map", full_irq_map, - GPEX_NUM_IRQS * GPEX_NUM_IRQS * + PCI_NUM_PINS * PCI_NUM_PINS * irq_map_stride * sizeof(uint32_t)); qemu_fdt_setprop_cells(fdt, nodename, "interrupt-map-mask", @@ -409,7 +409,7 @@ static void openrisc_virt_pcie_init(OR1KVirtState *state, memory_region_add_subregion(get_system_memory(), pio_base, alias); /* Connect IRQ lines. */ - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { pcie_irq = get_per_cpu_irq(cpus, num_cpus, irq_base + i); sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, pcie_irq); diff --git a/hw/pci-host/gpex.c b/hw/pci-host/gpex.c index a6752fac5e..7d5d996f34 100644 --- a/hw/pci-host/gpex.c +++ b/hw/pci-host/gpex.c @@ -32,6 +32,7 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "hw/irq.h" +#include "hw/pci/pci_bus.h" #include "hw/pci-host/gpex.h" #include "hw/qdev-properties.h" #include "migration/vmstate.h" @@ -50,7 +51,7 @@ static void gpex_set_irq(void *opaque, int irq_num, int level) int gpex_set_irq_num(GPEXHost *s, int index, int gsi) { - if (index >= GPEX_NUM_IRQS) { + if (index >= s->nr_irqs) { return -EINVAL; } @@ -74,14 +75,29 @@ static PCIINTxRoute gpex_route_intx_pin_to_irq(void *opaque, int pin) return route; } +static int gpex_swizzle_map_irq_fn(PCIDevice *pci_dev, int pin) +{ + PCIBus *bus = pci_device_root_bus(pci_dev); + + return (PCI_SLOT(pci_dev->devfn) + pin) % bus->nirq; +} + static void gpex_host_realize(DeviceState *dev, Error **errp) { PCIHostState *pci = PCI_HOST_BRIDGE(dev); GPEXHost *s = GPEX_HOST(dev); SysBusDevice *sbd = SYS_BUS_DEVICE(dev); PCIExpressHost *pex = PCIE_HOST_BRIDGE(dev); + pci_map_irq_fn map_irq_fn = pci_swizzle_map_irq_fn; int i; + s->irq = g_malloc0(s->nr_irqs * sizeof(*s->irq)); + s->irq_num = g_malloc0(s->nr_irqs * sizeof(*s->irq_num)); + + if (s->nr_irqs != PCI_NUM_PINS) { + map_irq_fn = gpex_swizzle_map_irq_fn; + } + pcie_host_mmcfg_init(pex, PCIE_MMCFG_SIZE_MAX); sysbus_init_mmio(sbd, &pex->mmio); @@ -128,19 +144,27 @@ static void gpex_host_realize(DeviceState *dev, Error **errp) sysbus_init_mmio(sbd, &s->io_ioport); } - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < s->nr_irqs; i++) { sysbus_init_irq(sbd, &s->irq[i]); s->irq_num[i] = -1; } - pci->bus = pci_register_root_bus(dev, "pcie.0", gpex_set_irq, - pci_swizzle_map_irq_fn, s, &s->io_mmio, - &s->io_ioport, 0, 4, TYPE_PCIE_BUS); + pci->bus = pci_register_root_bus(dev, "pcie.0", gpex_set_irq, map_irq_fn, + s, &s->io_mmio, &s->io_ioport, 0, + s->nr_irqs, TYPE_PCIE_BUS); pci_bus_set_route_irq_fn(pci->bus, gpex_route_intx_pin_to_irq); qdev_realize(DEVICE(&s->gpex_root), BUS(pci->bus), &error_fatal); } +static void gpex_host_unrealize(DeviceState *dev) +{ + GPEXHost *s = GPEX_HOST(dev); + + g_free(s->irq); + g_free(s->irq_num); +} + static const char *gpex_host_root_bus_path(PCIHostState *host_bridge, PCIBus *rootbus) { @@ -154,6 +178,7 @@ static Property gpex_host_properties[] = { */ DEFINE_PROP_BOOL("allow-unmapped-accesses", GPEXHost, allow_unmapped_accesses, true), + DEFINE_PROP_UINT32("nr-irqs", GPEXHost, nr_irqs, PCI_NUM_PINS), DEFINE_PROP_END_OF_LIST(), }; @@ -164,6 +189,7 @@ static void gpex_host_class_init(ObjectClass *klass, void *data) hc->root_bus_path = gpex_host_root_bus_path; dc->realize = gpex_host_realize; + dc->unrealize = gpex_host_unrealize; set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); dc->fw_name = "pci"; device_class_set_props(dc, gpex_host_properties); diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 245c7b97b2..4e772657c9 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -174,7 +174,7 @@ static void create_pcie_irq_map(RISCVVirtState *s, void *fdt, char *nodename, { int pin, dev; uint32_t irq_map_stride = 0; - uint32_t full_irq_map[GPEX_NUM_IRQS * GPEX_NUM_IRQS * + uint32_t full_irq_map[PCI_NUM_PINS * PCI_NUM_PINS * FDT_MAX_INT_MAP_WIDTH] = {}; uint32_t *irq_map = full_irq_map; @@ -186,11 +186,11 @@ static void create_pcie_irq_map(RISCVVirtState *s, void *fdt, char *nodename, * possible slot) seeing the interrupt-map-mask will allow the table * to wrap to any number of devices. */ - for (dev = 0; dev < GPEX_NUM_IRQS; dev++) { + for (dev = 0; dev < PCI_NUM_PINS; dev++) { int devfn = dev * 0x8; - for (pin = 0; pin < GPEX_NUM_IRQS; pin++) { - int irq_nr = PCIE_IRQ + ((pin + PCI_SLOT(devfn)) % GPEX_NUM_IRQS); + for (pin = 0; pin < PCI_NUM_PINS; pin++) { + int irq_nr = PCIE_IRQ + ((pin + PCI_SLOT(devfn)) % PCI_NUM_PINS); int i = 0; /* Fill PCI address cells */ @@ -216,7 +216,7 @@ static void create_pcie_irq_map(RISCVVirtState *s, void *fdt, char *nodename, } qemu_fdt_setprop(fdt, nodename, "interrupt-map", full_irq_map, - GPEX_NUM_IRQS * GPEX_NUM_IRQS * + PCI_NUM_PINS * PCI_NUM_PINS * irq_map_stride * sizeof(uint32_t)); qemu_fdt_setprop_cells(fdt, nodename, "interrupt-map-mask", @@ -1105,7 +1105,7 @@ static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem, sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, pio_base); - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { irq = qdev_get_gpio_in(irqchip, PCIE_IRQ + i); sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, irq); diff --git a/hw/xtensa/virt.c b/hw/xtensa/virt.c index b87f842e74..1d39807aef 100644 --- a/hw/xtensa/virt.c +++ b/hw/xtensa/virt.c @@ -93,7 +93,7 @@ static void create_pcie(MachineState *ms, CPUXtensaState *env, int irq_base, /* Connect IRQ lines. */ extints = xtensa_get_extints(env); - for (i = 0; i < GPEX_NUM_IRQS; i++) { + for (i = 0; i < PCI_NUM_PINS; i++) { void *q = extints[irq_base + i]; sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, q); diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h index b0240bd768..098dc4d1cc 100644 --- a/include/hw/pci-host/gpex.h +++ b/include/hw/pci-host/gpex.h @@ -32,8 +32,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(GPEXHost, GPEX_HOST) #define TYPE_GPEX_ROOT_DEVICE "gpex-root" OBJECT_DECLARE_SIMPLE_TYPE(GPEXRootState, GPEX_ROOT_DEVICE) -#define GPEX_NUM_IRQS 4 - struct GPEXRootState { /*< private >*/ PCIDevice parent_obj; @@ -51,8 +49,9 @@ struct GPEXHost { MemoryRegion io_mmio; MemoryRegion io_ioport_window; MemoryRegion io_mmio_window; - qemu_irq irq[GPEX_NUM_IRQS]; - int irq_num[GPEX_NUM_IRQS]; + uint32_t nr_irqs; + qemu_irq *irq; + int *irq_num; bool allow_unmapped_accesses; };