From patchwork Tue Jun 20 17:24:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E6FDEB64DB for ; Tue, 20 Jun 2023 17:26:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552057.861907 (Exim 4.92) (envelope-from ) id 1qBf6z-0004qF-3E; Tue, 20 Jun 2023 17:25:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552057.861907; Tue, 20 Jun 2023 17:25:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf6z-0004q8-0G; Tue, 20 Jun 2023 17:25:49 +0000 Received: by outflank-mailman (input) for mailman id 552057; Tue, 20 Jun 2023 17:25:48 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf6y-0004q1-Hx for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:25:48 +0000 Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [2607:f8b0:4864:20::736]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 7d88d7d6-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:25:46 +0200 (CEST) Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-7624e8ceef7so372280285a.2 for ; Tue, 20 Jun 2023 10:25:46 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.25.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:25:44 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7d88d7d6-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281945; x=1689873945; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SzWA4Xdu5chmj9W8/SSvQqa9W/rPnPlBar7MYCHNXR4=; b=nkl7c5jtmMqYWfcMCRh68XOL/ki9yoAqUWOOUdZYbHgQNtTW4Muko67OQmS60sOvg+ ggRq408cyY7W+oRbCFRd76IoQNOKl3V2vHKQHkvFccUNpu87dQ4GpH4aMYwPfdI5J+x7 fcVLBDtSLZCDXn3bXtIhHZ8ck7SFK/Mb/3LxBDBj76WswWRaSgSnrP3N3tDB9v9vant9 qq7YVX5+7iRJOD80LEHirfoFLBmQhRuBppzU5bTVbMqOeOcCNyiVGTUMQeYNW6keD053 wCMSFBO6mI5oqXjKQb80eh8jC0vmXZevKVW2FVEz8ctN2t8vOjb7GxNcXPRc2f+uwre3 xplw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281945; x=1689873945; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SzWA4Xdu5chmj9W8/SSvQqa9W/rPnPlBar7MYCHNXR4=; b=DfC5BfLNPRUpaPmqBB3HMpwNy1L87jI5IGJ0lJVeqMTLWdH2GZH2vhw5Jf0TrWczXH tq0iX+MbiUTqCG+b/KpYy/yfdbgii0zuDkRlF/QGhZAxZ0BViHycigu4arWBKUFghU9n Or0P+Ew6nX+gT/fnmI6plCZ/pA5eqf31ynIOebJHbN9ZnZYRfD/f1EGvcZs0JCQewxOd IwjpyXg1gcQiz6hsGI1EDtE6snNEhNz0JKi1ZsJbhDgsCitNITkZdmljhxQAqZsc2Y74 dWKrncpUAw8Rba8tWZzqjbeWjimkW2a6ImkEN3354FRN4p2pyPx4Z23PbVLUBV4Hq148 r/jg== X-Gm-Message-State: AC+VfDx/tPikEUJRxROL5InAOo5Lbyi/cMUHJ0wQ88kRnjJCXnSji53f 2gauWSKbHbqI67LG1pJ3PN4= X-Google-Smtp-Source: ACHHUZ6JBmTkFy7A8F5kZ4X/2RW/8CZZjXWkzqR1U6M/mNyG75yHCAfIog0YjAv0K3OGN+9EpieONQ== X-Received: by 2002:a05:620a:3d0d:b0:75f:403:77a8 with SMTP id tq13-20020a05620a3d0d00b0075f040377a8mr4458855qkn.31.1687281945053; Tue, 20 Jun 2023 10:25:45 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Paolo Bonzini , Richard Henderson , Eduardo Habkost , "Michael S. Tsirkin" , Marcel Apfelbaum , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 01/23] pc/xen: Xen Q35 support: provide IRQ handling for PCI devices Date: Tue, 20 Jun 2023 13:24:34 -0400 Message-Id: <1c547c5581ce6192b70c68f39de108cdb2c73f7e.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 The primary difference in PCI device IRQ management between Xen HVM and QEMU is that Xen PCI IRQs are "device-centric" while QEMU PCI IRQs are "chipset-centric". Namely, Xen uses PCI device BDF and INTx as coordinates to assert IRQ while QEMU finds out to which chipset PIRQ the IRQ is routed through the hierarchy of PCI buses and manages IRQ assertion on chipset side (as PIRQ inputs). Two callback functions are used for this purpose: .map_irq and .set_irq (named after corresponding structure fields). Corresponding Xen-specific callback functions are piix3_set_irq() and pci_slot_get_pirq(). In Xen case these functions do not operate on pirq pin numbers. Instead, they use a specific value to pass BDF/INTx information between .map_irq and .set_irq -- PCI device devfn and INTx pin number are combined into pseudo-PIRQ in pci_slot_get_pirq, which piix3_set_irq later decodes back into devfn and INTx number for passing to *set_pci_intx_level() call. For Xen on Q35 this scheme is still applicable, with the exception that function names are non-descriptive now and need to be renamed to show their common i440/Q35 nature. Proposed new names are: xen_pci_slot_get_pirq --> xen_cmn_pci_slot_get_pirq xen_piix3_set_irq --> xen_cmn_set_irq Another IRQ-related difference between i440 and Q35 is the number of PIRQ inputs and PIRQ routers (PCI IRQ links in terms of ACPI) available. i440 has 4 PCI interrupt links, while Q35 has 8 (PIRQA...PIRQH). Currently Xen have support for only 4 PCI links, so we describe only 4 of 8 PCI links in ACPI tables. Also, hvmloader disables PIRQ routing for PIRQE..PIRQH by writing 80h into corresponding PIRQ[n]_ROUT registers. All this PCI interrupt routing stuff is largely an ancient legacy from PIC era. It's hardly worth to extend number of PCI links supported as we normally deal with APIC mode and/or MSI interrupts. The only useful thing to do with PIRQE..PIRQH routing currently is to check if guest actually attempts to use it for some reason (despite ACPI PCI routing information provided). In this case, a warning is logged. Things have changed a bit in modern Qemu, and more changes to the IRQ mapping had to be done inside the lpc_ich9 to write the irqs and setup the mappings. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/i386/pc_piix.c | 3 +- hw/i386/xen/xen-hvm.c | 7 +++-- hw/isa/lpc_ich9.c | 53 ++++++++++++++++++++++++++++++++--- hw/isa/piix3.c | 2 +- include/hw/southbridge/ich9.h | 1 + include/hw/xen/xen.h | 4 +-- stubs/xen-hw-stub.c | 4 +-- 7 files changed, 61 insertions(+), 13 deletions(-) diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index d5b0dcd1fe..8c1b20f3bc 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -62,6 +62,7 @@ #endif #include "hw/xen/xen-x86.h" #include "hw/xen/xen.h" +#include "sysemu/xen.h" #include "migration/global_state.h" #include "migration/misc.h" #include "sysemu/numa.h" @@ -233,7 +234,7 @@ static void pc_init1(MachineState *machine, x86ms->above_4g_mem_size, pci_memory, ram_memory); pci_bus_map_irqs(pci_bus, - xen_enabled() ? xen_pci_slot_get_pirq + xen_enabled() ? xen_cmn_pci_slot_get_pirq : pc_pci_slot_get_pirq); pcms->bus = pci_bus; diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c index 56641a550e..540ac46639 100644 --- a/hw/i386/xen/xen-hvm.c +++ b/hw/i386/xen/xen-hvm.c @@ -15,6 +15,7 @@ #include "hw/pci/pci.h" #include "hw/pci/pci_host.h" #include "hw/i386/pc.h" +#include "hw/southbridge/ich9.h" #include "hw/irq.h" #include "hw/hw.h" #include "hw/i386/apic-msidef.h" @@ -136,14 +137,14 @@ typedef struct XenIOState { Notifier wakeup; } XenIOState; -/* Xen specific function for piix pci */ +/* Xen-specific functions for pci dev IRQ handling */ -int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num) +int xen_cmn_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num) { return irq_num + (PCI_SLOT(pci_dev->devfn) << 2); } -void xen_piix3_set_irq(void *opaque, int irq_num, int level) +void xen_cmn_set_irq(void *opaque, int irq_num, int level) { xen_set_pci_intx_level(xen_domid, 0, 0, irq_num >> 2, irq_num & 3, level); diff --git a/hw/isa/lpc_ich9.c b/hw/isa/lpc_ich9.c index 9c47a2f6c7..733a99d443 100644 --- a/hw/isa/lpc_ich9.c +++ b/hw/isa/lpc_ich9.c @@ -51,6 +51,9 @@ #include "hw/core/cpu.h" #include "hw/nvram/fw_cfg.h" #include "qemu/cutils.h" +#include "hw/xen/xen.h" +#include "sysemu/xen.h" +#include "hw/southbridge/piix.h" #include "hw/acpi/acpi_aml_interface.h" #include "trace.h" @@ -535,11 +538,49 @@ static int ich9_lpc_post_load(void *opaque, int version_id) return 0; } +static void ich9_lpc_config_write_xen(PCIDevice *d, + uint32_t addr, uint32_t val, int len) +{ + static bool pirqe_f_warned = false; + if (ranges_overlap(addr, len, ICH9_LPC_PIRQA_ROUT, 4)) { + /* handle PIRQA..PIRQD routing */ + /* Scan for updates to PCI link routes (0x60-0x63). */ + int i; + for (i = 0; i < len; i++) { + uint8_t v = (val >> (8 * i)) & 0xff; + if (v & 0x80) { + v = 0; + } + v &= 0xf; + if (((addr + i) >= PIIX_PIRQCA) && ((addr + i) <= PIIX_PIRQCD)) { + xen_set_pci_link_route(addr + i - PIIX_PIRQCA, v); + } + } + } else if (ranges_overlap(addr, len, ICH9_LPC_PIRQE_ROUT, 4)) { + while (len--) { + if (range_covers_byte(ICH9_LPC_PIRQE_ROUT, 4, addr) && + (val & 0x80) == 0) { + /* print warning only once */ + if (!pirqe_f_warned) { + pirqe_f_warned = true; + fprintf(stderr, "WARNING: guest domain attempted to use PIRQ%c " + "routing which is not supported for Xen/Q35 currently\n", + (char)(addr - ICH9_LPC_PIRQE_ROUT + 'E')); + break; + } + } + addr++, val >>= 8; + } + } +} + static void ich9_lpc_config_write(PCIDevice *d, uint32_t addr, uint32_t val, int len) { ICH9LPCState *lpc = ICH9_LPC_DEVICE(d); uint32_t rcba_old = pci_get_long(d->config + ICH9_LPC_RCBA); + if (xen_enabled()) + ich9_lpc_config_write_xen(d, addr, val, len); pci_default_write_config(d, addr, val, len); if (ranges_overlap(addr, len, ICH9_LPC_PMBASE, 4) || @@ -731,10 +772,14 @@ static void ich9_lpc_realize(PCIDevice *d, Error **errp) return; } - pci_bus_irqs(pci_bus, ich9_lpc_set_irq, d, ICH9_LPC_NB_PIRQS); - pci_bus_map_irqs(pci_bus, ich9_lpc_map_irq); - pci_bus_set_route_irq_fn(pci_bus, ich9_route_intx_pin_to_irq); - + if (xen_enabled()) { + pci_bus_irqs(pci_bus, xen_cmn_set_irq, d, ICH9_XEN_NUM_IRQ_SOURCES); + pci_bus_map_irqs(pci_bus, xen_cmn_pci_slot_get_pirq); + } else { + pci_bus_irqs(pci_bus, ich9_lpc_set_irq, d, ICH9_LPC_NB_PIRQS); + pci_bus_map_irqs(pci_bus, ich9_lpc_map_irq); + pci_bus_set_route_irq_fn(pci_bus, ich9_route_intx_pin_to_irq); + } ich9_lpc_pm_init(lpc); } diff --git a/hw/isa/piix3.c b/hw/isa/piix3.c index f9103ea45a..3d0545eb0e 100644 --- a/hw/isa/piix3.c +++ b/hw/isa/piix3.c @@ -420,7 +420,7 @@ static void piix3_xen_realize(PCIDevice *dev, Error **errp) * connected to the IOAPIC directly. * These additional routes can be discovered through ACPI. */ - pci_bus_irqs(pci_bus, xen_piix3_set_irq, piix3, XEN_PIIX_NUM_PIRQS); + pci_bus_irqs(pci_bus, xen_cmn_set_irq, piix3, XEN_PIIX_NUM_PIRQS); } static void piix3_xen_class_init(ObjectClass *klass, void *data) diff --git a/include/hw/southbridge/ich9.h b/include/hw/southbridge/ich9.h index fd01649d04..07b84d5227 100644 --- a/include/hw/southbridge/ich9.h +++ b/include/hw/southbridge/ich9.h @@ -130,6 +130,7 @@ struct ICH9LPCState { #define ICH9_A2_LPC_REVISION 0x2 #define ICH9_LPC_NB_PIRQS 8 /* PCI A-H */ +#define ICH9_XEN_NUM_IRQ_SOURCES 128 #define ICH9_LPC_PMBASE 0x40 #define ICH9_LPC_PMBASE_BASE_ADDRESS_MASK ICH9_MASK(32, 15, 7) diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h index 2bd8ec742d..a2c3d98eaa 100644 --- a/include/hw/xen/xen.h +++ b/include/hw/xen/xen.h @@ -37,9 +37,9 @@ extern uint32_t xen_domid; extern enum xen_mode xen_mode; extern bool xen_domid_restrict; -int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num); +int xen_cmn_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num); int xen_set_pci_link_route(uint8_t link, uint8_t irq); -void xen_piix3_set_irq(void *opaque, int irq_num, int level); +void xen_cmn_set_irq(void *opaque, int irq_num, int level); void xen_hvm_inject_msi(uint64_t addr, uint32_t data); int xen_is_pirq_msi(uint32_t msi_data); diff --git a/stubs/xen-hw-stub.c b/stubs/xen-hw-stub.c index 34a22f2ad7..f06fbf48c8 100644 --- a/stubs/xen-hw-stub.c +++ b/stubs/xen-hw-stub.c @@ -10,12 +10,12 @@ #include "hw/xen/xen.h" #include "hw/xen/xen-x86.h" -int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num) +int xen_cmn_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num) { return -1; } -void xen_piix3_set_irq(void *opaque, int irq_num, int level) +void xen_cmn_set_irq(void *opaque, int irq_num, int level) { } From patchwork Tue Jun 20 17:24:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F899EB64D7 for ; Tue, 20 Jun 2023 17:26:30 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552064.861927 (Exim 4.92) (envelope-from ) id 1qBf7M-0005iw-Ve; Tue, 20 Jun 2023 17:26:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552064.861927; Tue, 20 Jun 2023 17:26:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7M-0005ip-Q9; Tue, 20 Jun 2023 17:26:12 +0000 Received: by outflank-mailman (input) for mailman id 552064; Tue, 20 Jun 2023 17:26:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7K-0004q1-OA for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:10 +0000 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [2607:f8b0:4864:20::1133]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8ab5614f-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:08 +0200 (CEST) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-56fff21c2ebso53777097b3.3 for ; Tue, 20 Jun 2023 10:26:08 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:06 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8ab5614f-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281967; x=1689873967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pVhDjOa7FHuFiuCBrjeemQ/YptYJoZezJ1j93NWRipk=; b=bPEXGnB2OtY8PYvTjaUk45NIyGfaJM2vd0nUPIwq0dz+1vjogQHKnLJSGWeMF3FZiR BJQr02kmNxHweWckj3n4AC4pYWAIm3gXebmre0XVSa+MyCSfP7KmzjksZfpzuEP6yi2c HKrdqxIW3oi5jIN+o1+rFb7mcBEbqWQ+WLjCyNu2Le6YTLfO5bsApD3TyYLGfcWzXEpw MgywIHHn/Ex9ORIPODfah/nut/515nEu59JmlDBKfJz3jeZl5XAIcDawiSkUvxQm8mCn zahZTbbwNHQb4LnWfVlixwaX4ifBQy6qHwzgwKkO2Saqoz8hCtPT/HCRi58uaA1nlKxf A7Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281967; x=1689873967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pVhDjOa7FHuFiuCBrjeemQ/YptYJoZezJ1j93NWRipk=; b=c9fkDGSLh+JJjLeuqNTuZmw0lC0GFrJz53sMRNbxDJ0wGc+bYeowpwK21kZywn7wp7 3rvbLZhoNyr4kg9fn7cnNnLe9WWmH/lrw3SCiN22VWOkXG9mVw9/SgfF/8RJw57/3bij TtsBl3cX25eGKiMmj0WAs5o7M6H3MdaBs4WV+taEqHRHXatmmi+lNIF7jt25YnSc1Qqk uX9OrddZL/BDAe0Zl2ejrJe5HuVIf/G3y73rGAl05euzCpjXUJwlngMbtCvXK7EdSWTu 2KXV/B4knay20tpoWdSL9t8jXhi7Gu/G1b9Ujs7fvMEHhrJhPZhY49dBWzws/vYKKtse xo/w== X-Gm-Message-State: AC+VfDwTRJtds/sRBpHNP1rgAySGY0nt7JoYE9pYjrESFFZ3iKxrVQye GJxG1Ho9IrEbm6fDyELv0cA= X-Google-Smtp-Source: ACHHUZ6SdCuyO2ywOemJ8HmOK80tXYlK5MSf18GBMCkKWxXCffCWY1KnaRNAYZ9J1meknqeU888PzQ== X-Received: by 2002:a0d:f384:0:b0:56d:1528:c56f with SMTP id c126-20020a0df384000000b0056d1528c56fmr12677100ywf.16.1687281967212; Tue, 20 Jun 2023 10:26:07 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 06/23] xen/pt: XenHostPCIDevice: provide functions for PCI Capabilities and PCIe Extended Capabilities enumeration Date: Tue, 20 Jun 2023 13:24:40 -0400 Message-Id: <8a5ed3bf4f1abca1e7e3a2c6011b7f81620aedca.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 This patch introduces 2 new functions, - xen_host_pci_find_next_ext_cap (actually a reworked xen_host_pci_find_ext_cap_offset function which is unused) - xen_host_pci_find_next_cap These functions allow to search for PCI/PCIe capabilities in a uniform way. Both functions allow to search either a specific capability or any encountered next (by specifying CAP_ID_ANY as a capability ID) -- this may be useful when we merely need to traverse the capability list one-by-one. In both functions the 'pos' argument allows to continue searching from last position (0 means to start from beginning). In order not to probe PCIe Extended Capabilities existence every time, xen_host_pci_find_next_ext_cap makes use of the new 'has_pcie_ext_caps' field in XenHostPCIDevice structure which is filled only once (in xen_host_pci_device_get). Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen-host-pci-device.c | 91 ++++++++++++++++++++++++++++++++---- hw/xen/xen-host-pci-device.h | 5 +- 2 files changed, 85 insertions(+), 11 deletions(-) diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c index 8c6e9a1716..a7021a5d56 100644 --- a/hw/xen/xen-host-pci-device.c +++ b/hw/xen/xen-host-pci-device.c @@ -32,6 +32,7 @@ #define IORESOURCE_PREFETCH 0x00001000 /* No side effects */ #define IORESOURCE_MEM_64 0x00100000 +#define XEN_HOST_PCI_CAP_MAX 48 static void xen_host_pci_sysfs_path(const XenHostPCIDevice *d, const char *name, char *buf, ssize_t size) @@ -198,6 +199,19 @@ static bool xen_host_pci_dev_is_virtfn(XenHostPCIDevice *d) return !stat(path, &buf); } +static bool xen_host_pci_dev_has_pcie_ext_caps(XenHostPCIDevice *d) +{ + uint32_t header; + + if (xen_host_pci_get_long(d, PCI_CONFIG_SPACE_SIZE, &header)) + return false; + + if (header == 0 || header == ~0U) + return false; + + return true; +} + static void xen_host_pci_config_open(XenHostPCIDevice *d, Error **errp) { char path[PATH_MAX]; @@ -296,37 +310,93 @@ int xen_host_pci_set_block(XenHostPCIDevice *d, int pos, uint8_t *buf, int len) return xen_host_pci_config_write(d, pos, buf, len); } -int xen_host_pci_find_ext_cap_offset(XenHostPCIDevice *d, uint32_t cap) +int xen_host_pci_find_next_ext_cap(XenHostPCIDevice *d, int pos, uint32_t cap) { uint32_t header = 0; int max_cap = XEN_HOST_PCI_MAX_EXT_CAP; - int pos = PCI_CONFIG_SPACE_SIZE; + + if (!d->has_pcie_ext_caps) + return 0; + + if (!pos) { + pos = PCI_CONFIG_SPACE_SIZE; + } else { + if (xen_host_pci_get_long(d, pos, &header)) + return 0; + + pos = PCI_EXT_CAP_NEXT(header); + } do { + if (!pos || pos < PCI_CONFIG_SPACE_SIZE) { + break; + } + if (xen_host_pci_get_long(d, pos, &header)) { break; } /* * If we have no capabilities, this is indicated by cap ID, * cap version and next pointer all being 0. + * Also check for all F's returned (which means PCIe ext conf space + * is unreadable for some reason) */ - if (header == 0) { + if (header == 0 || header == ~0U) { break; } - if (PCI_EXT_CAP_ID(header) == cap) { + if (cap == CAP_ID_ANY) { + return pos; + } else if (PCI_EXT_CAP_ID(header) == cap) { return pos; } pos = PCI_EXT_CAP_NEXT(header); - if (pos < PCI_CONFIG_SPACE_SIZE) { + } while (--max_cap); + + return 0; +} + +int xen_host_pci_find_next_cap(XenHostPCIDevice *d, int pos, uint32_t cap) +{ + uint8_t id; + unsigned max_cap = XEN_HOST_PCI_CAP_MAX; + uint8_t status = 0; + uint8_t curpos; + + if (xen_host_pci_get_byte(d, PCI_STATUS, &status)) + return 0; + + if ((status & PCI_STATUS_CAP_LIST) == 0) + return 0; + + if (pos < PCI_CAPABILITY_LIST) { + curpos = PCI_CAPABILITY_LIST; + } else { + curpos = (uint8_t) pos; + } + + while (max_cap--) { + if (xen_host_pci_get_byte(d, curpos, &curpos)) + break; + if (!curpos) + break; + + if (cap == CAP_ID_ANY) + return curpos; + + if (xen_host_pci_get_byte(d, curpos + PCI_CAP_LIST_ID, &id)) break; - } - max_cap--; - } while (max_cap > 0); + if (id == 0xff) + break; + else if (id == cap) + return curpos; + + curpos += PCI_CAP_LIST_NEXT; + } - return -1; + return 0; } void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, @@ -376,7 +446,8 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, } d->class_code = v; - d->is_virtfn = xen_host_pci_dev_is_virtfn(d); + d->is_virtfn = xen_host_pci_dev_is_virtfn(d); + d->has_pcie_ext_caps = xen_host_pci_dev_has_pcie_ext_caps(d); return; diff --git a/hw/xen/xen-host-pci-device.h b/hw/xen/xen-host-pci-device.h index 4d8d34ecb0..37c5614a24 100644 --- a/hw/xen/xen-host-pci-device.h +++ b/hw/xen/xen-host-pci-device.h @@ -32,6 +32,7 @@ typedef struct XenHostPCIDevice { XenHostPCIIORegion rom; bool is_virtfn; + bool has_pcie_ext_caps; int config_fd; } XenHostPCIDevice; @@ -53,6 +54,8 @@ int xen_host_pci_set_long(XenHostPCIDevice *d, int pos, uint32_t data); int xen_host_pci_set_block(XenHostPCIDevice *d, int pos, uint8_t *buf, int len); -int xen_host_pci_find_ext_cap_offset(XenHostPCIDevice *s, uint32_t cap); +#define CAP_ID_ANY (~0U) +int xen_host_pci_find_next_cap(XenHostPCIDevice *s, int pos, uint32_t cap); +int xen_host_pci_find_next_ext_cap(XenHostPCIDevice *d, int pos, uint32_t cap); #endif /* XEN_HOST_PCI_DEVICE_H */ From patchwork Tue Jun 20 17:24:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7882EB64D7 for ; Tue, 20 Jun 2023 17:26:22 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552066.861937 (Exim 4.92) (envelope-from ) id 1qBf7O-0005zv-5M; Tue, 20 Jun 2023 17:26:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552066.861937; Tue, 20 Jun 2023 17:26:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7O-0005zo-21; Tue, 20 Jun 2023 17:26:14 +0000 Received: by outflank-mailman (input) for mailman id 552066; Tue, 20 Jun 2023 17:26:12 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7M-0004q1-OZ for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:12 +0000 Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [2607:f8b0:4864:20::72a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8bddd98e-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:10 +0200 (CEST) Received: by mail-qk1-x72a.google.com with SMTP id af79cd13be357-763a8918cbeso155210185a.3 for ; Tue, 20 Jun 2023 10:26:10 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:08 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8bddd98e-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281969; x=1689873969; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nxaTb/29l3CebuyhwApqyR6VcKjgYqFWVhZRRk7NUds=; b=UFwApTqEw15ynGiEMn4zGI1aRFSQUGsmKdhJK7bY8DvDybKXYs/QI0p/LA5BZIs0eo r4K3I7UnCrrl4ZIipvzffx4/tipAi6+P7+pjmEQi23LTYqE8nBmFG/9rdPuiN9zFzt9M UnDbRDx4B509g2OsHTMYvs6QnAcLHZ6xyNsN1ZmQDHcNDbOvIGQdiI2rRNmFwXzbmaEJ 4ma5FkfJ73AyHagCUq4UUuGWnk4Skt3LBSLD2io6CYEmHhdm9lXfxTwNLXs3mxlSuArE FJXjdyn8A7I1jaWErgjnbUBY9TRfRhVuuT4ZpQkxrVzKFWtmmgF8E4VkQHjeVaNSpUNK eTjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281969; x=1689873969; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nxaTb/29l3CebuyhwApqyR6VcKjgYqFWVhZRRk7NUds=; b=bZxfuHC/9MOMf+BdXVDIXWYd05uabOZFRdDtGb6/aMj1z2JljG9mRagne2gUqxIAS7 Wfzi0I2P3qKLh5GPUdbD5XFbntgTTT/dpiVZduFXo9T0yneDA8e8AInMDo6QxCuKLKiP KvQi5yBxpauI/aRYgBltWsIwm6BogNobpj47ikE8MM7OvZKiCb/yLg4cvxF3oKE6yUaS kLqEOF4qmfIGSqcaLeeOkxV195StqmQRL8Ufw8DITk6Q0KFZ2dFNJ4U0Aj+P9hrikEHv CzineQOGsq6FDcG74R//0w1LetPYPHu7zpfKtPBIxSuAU25PQccp91AadxDVgGu+dbn4 hfnQ== X-Gm-Message-State: AC+VfDzrrcHv50i1gw1YF5qMWlJPkBgThfsUZskRDKr+c2in8X5kGqnZ yERjAXg7u8gWM3M4Exntte8= X-Google-Smtp-Source: ACHHUZ7qEQizJx1HhPJarTNwVXdhkGcdKmod3Rc2GPN945a78/sJmry1z4uw6lP6dXJOmmKm7jBB9w== X-Received: by 2002:a05:620a:25c8:b0:763:b475:90ef with SMTP id y8-20020a05620a25c800b00763b47590efmr1998867qko.69.1687281969191; Tue, 20 Jun 2023 10:26:09 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 07/23] xen/pt: avoid reading PCIe device type and cap version multiple times Date: Tue, 20 Jun 2023 13:24:41 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 xen_pt_config_init.c reads Device/Port Type and Capability version fields in many places. Two functions are used for this purpose: get_capability_version and get_device_type. These functions perform PCI conf space reading every time they're called. Another bad thing is that these functions know nothing about where PCI Expess Capability is located, so its offset must be provided explicitly in function arguments. Their typical usage is like this: uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); uint8_t dev_type = get_device_type(s, real_offset - reg->offset); To avoid this, the PCI Express Capability register now being read only once and stored in XenHostPCIDevice structure (pcie_flags field). The capabiliy offset parameter is no longer needed, simplifying functions usage. Also, get_device_type and get_capability_version were renamed to more descriptive get_pcie_device_type and get_pcie_capability_version. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen-host-pci-device.c | 15 +++++++++++++++ hw/xen/xen-host-pci-device.h | 1 + hw/xen/xen_pt_config_init.c | 34 ++++++++++++++-------------------- 3 files changed, 30 insertions(+), 20 deletions(-) diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c index a7021a5d56..63481a859e 100644 --- a/hw/xen/xen-host-pci-device.c +++ b/hw/xen/xen-host-pci-device.c @@ -405,6 +405,7 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, { ERRP_GUARD(); unsigned int v; + int pcie_cap_pos; d->config_fd = -1; d->domain = domain; @@ -449,6 +450,20 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, d->is_virtfn = xen_host_pci_dev_is_virtfn(d); d->has_pcie_ext_caps = xen_host_pci_dev_has_pcie_ext_caps(d); + /* read and store PCIe Capabilities field for later use */ + pcie_cap_pos = xen_host_pci_find_next_cap(d, 0, PCI_CAP_ID_EXP); + + if (pcie_cap_pos) { + if (xen_host_pci_get_word(d, pcie_cap_pos + PCI_EXP_FLAGS, + &d->pcie_flags)) { + error_setg(errp, "Unable to read from PCI Express capability " + "structure at 0x%x", pcie_cap_pos); + goto error; + } + } else { + d->pcie_flags = 0xFFFF; + } + return; error: diff --git a/hw/xen/xen-host-pci-device.h b/hw/xen/xen-host-pci-device.h index 37c5614a24..2884c4b4b9 100644 --- a/hw/xen/xen-host-pci-device.h +++ b/hw/xen/xen-host-pci-device.h @@ -27,6 +27,7 @@ typedef struct XenHostPCIDevice { uint16_t device_id; uint32_t class_code; int irq; + uint16_t pcie_flags; XenHostPCIIORegion io_regions[PCI_NUM_REGIONS - 1]; XenHostPCIIORegion rom; diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 2b8680b112..47c8482f32 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -832,24 +832,18 @@ static XenPTRegInfo xen_pt_emu_reg_vendor[] = { * PCI Express Capability */ -static inline uint8_t get_capability_version(XenPCIPassthroughState *s, - uint32_t offset) +static inline uint8_t get_pcie_capability_version(XenPCIPassthroughState *s) { - uint8_t flag; - if (xen_host_pci_get_byte(&s->real_device, offset + PCI_EXP_FLAGS, &flag)) { - return 0; - } - return flag & PCI_EXP_FLAGS_VERS; + assert(s->real_device.pcie_flags != 0xFFFF); + + return (uint8_t) (s->real_device.pcie_flags & PCI_EXP_FLAGS_VERS); } -static inline uint8_t get_device_type(XenPCIPassthroughState *s, - uint32_t offset) +static inline uint8_t get_pcie_device_type(XenPCIPassthroughState *s) { - uint8_t flag; - if (xen_host_pci_get_byte(&s->real_device, offset + PCI_EXP_FLAGS, &flag)) { - return 0; - } - return (flag & PCI_EXP_FLAGS_TYPE) >> 4; + assert(s->real_device.pcie_flags != 0xFFFF); + + return (uint8_t) ((s->real_device.pcie_flags & PCI_EXP_FLAGS_TYPE) >> 4); } /* initialize Link Control register */ @@ -857,8 +851,8 @@ static int xen_pt_linkctrl_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data) { - uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); - uint8_t dev_type = get_device_type(s, real_offset - reg->offset); + uint8_t cap_ver = get_pcie_capability_version(s); + uint8_t dev_type = get_pcie_device_type(s); /* no need to initialize in case of Root Complex Integrated Endpoint * with cap_ver 1.x @@ -875,7 +869,7 @@ static int xen_pt_devctrl2_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data) { - uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); + uint8_t cap_ver = get_pcie_capability_version(s); /* no need to initialize in case of cap_ver 1.x */ if (cap_ver == 1) { @@ -890,7 +884,7 @@ static int xen_pt_linkctrl2_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data) { - uint8_t cap_ver = get_capability_version(s, real_offset - reg->offset); + uint8_t cap_ver = get_pcie_capability_version(s); uint32_t reg_field = 0; /* no need to initialize in case of cap_ver 1.x */ @@ -1590,8 +1584,8 @@ static int xen_pt_pcie_size_init(XenPCIPassthroughState *s, uint32_t base_offset, uint8_t *size) { PCIDevice *d = PCI_DEVICE(s); - uint8_t version = get_capability_version(s, base_offset); - uint8_t type = get_device_type(s, base_offset); + uint8_t version = get_pcie_capability_version(s); + uint8_t type = get_pcie_device_type(s); uint8_t pcie_size = 0; From patchwork Tue Jun 20 17:24:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5056EEB64DB for ; Tue, 20 Jun 2023 17:26:28 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552067.861947 (Exim 4.92) (envelope-from ) id 1qBf7P-0006GD-Eu; Tue, 20 Jun 2023 17:26:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552067.861947; Tue, 20 Jun 2023 17:26:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7P-0006G4-Aj; Tue, 20 Jun 2023 17:26:15 +0000 Received: by outflank-mailman (input) for mailman id 552067; Tue, 20 Jun 2023 17:26:14 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7O-0004q1-DZ for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:14 +0000 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [2607:f8b0:4864:20::112c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8d435b48-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:12 +0200 (CEST) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-56fff21c2ebso53778057b3.3 for ; Tue, 20 Jun 2023 10:26:12 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:10 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8d435b48-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281971; x=1689873971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=re4sewoUiAJAV4/kgcnlDrxKrE2Fa0zI8YeYhG23N0Y=; b=DfdES6JEvCQWTQ1WFDll4cGS97nAixB6PRsd09O5iFOLxZco6PKhbfyAEFkZAWyakT gCJzPrYhpd7a1DMuixAnPGnTmdPOTVwMM5rMfcTE2WmPXGz7o5UvRBvEZ0GteR42PRC2 lVCvO5GXFsYJItGC1Q2scC7ydoOZ9VTsDqbOD9BN/OuN69tnXeoy0DIkIAtRXjuaqzjN 013Eyyc3Oa4qordF985hTPAqy6uANH1i3Y6hPcC3nXertqQVA1BtmbjxNxMz81vSwaRD mIGdmNWqNoB5viq56p08Y7WwZY97IlnzgnqgCVOHaYsed6TmrP6sl/ttsrXBy3RlJ9hZ 116w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281971; x=1689873971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=re4sewoUiAJAV4/kgcnlDrxKrE2Fa0zI8YeYhG23N0Y=; b=cCJuS6cyNVXVAQEyPO3EFys1Gn59wOiduatSx6PaRrPOWtAqkb+JznZURK4ouj10Bn DgyJFOi8q1HTt1DmKncEJ55A/FS1W4atrpiDfek0IMm+aoHZj7bvtrySzhAuS8UP2dG2 miAFOpKmfxcYpsuZkHgDIhNfZFSNtgOim9pv/8Q3SkGOOykAFnMTcmrqnp3MchNIqnuA YSlYi+LRrvv5NSllUD/N7DY5ilPDH+LWlDAP8A5rhYleKi9YuhC4/WmPfzgFnFojJBq1 BFkfONE5/cm/VEzjtpwSBwFknIp2VM2Oyah4JZiTp3/4FBduZVJIvLpdLWqYTqgfUO9r K2xw== X-Gm-Message-State: AC+VfDwD48ND9gfINhSqwJEiKX+JP00obi2xy3pq7acty8rBuNyFeF3p tEfvkkDonf8lBKG8/C068e4= X-Google-Smtp-Source: ACHHUZ7xi/MCe/ilteffhPiFjo95RMXHLzj+R6XW+7HT26RsjzlIhW2AGxlvpmO2zHaQ7cawV/D/bg== X-Received: by 2002:a81:c213:0:b0:56c:ea52:a9fc with SMTP id z19-20020a81c213000000b0056cea52a9fcmr13911538ywc.12.1687281971388; Tue, 20 Jun 2023 10:26:11 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 08/23] xen/pt: determine the legacy/PCIe mode for a passed through device Date: Tue, 20 Jun 2023 13:24:42 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Even if we have some real PCIe device being passed through to a guest, there are situations when we cannot use its PCIe features, primarily allowing to access extended (>256) config space. Basically, we can allow reading PCIe extended config space only if both the device and emulated system are PCIe-capable. So it's a combination of checks: - PCI Express capability presence - pci_is_express(device) - pci_bus_is_express(device bus) The AND-product of these checks is stored to pcie_enabled_dev flag in XenPCIPassthroughState for later use in functions like xen_pt_pci_config_access_check. This way we get consistent behavior when the same PCIe device being passed through to either i440 domain or Q35 one. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt.c | 28 ++++++++++++++++++++++++++-- hw/xen/xen_pt.h | 1 + 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index a540149639..65c5516ef4 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -701,6 +701,21 @@ static const MemoryListener xen_pt_io_listener = { .priority = 10, }; +static inline bool xen_pt_dev_is_pcie_mode(PCIDevice *d) +{ + XenPCIPassthroughState *s = XEN_PT_DEVICE(d); + PCIBus *bus = pci_get_bus(d); + + if (bus != NULL) { + if (pci_is_express(d) && pci_bus_is_express(bus) && + xen_host_pci_find_next_cap(&s->real_device, 0, PCI_CAP_ID_EXP)) { + return true; + } + } + + return false; +} + /* destroy. */ static void xen_pt_destroy(PCIDevice *d) { @@ -787,8 +802,17 @@ static void xen_pt_realize(PCIDevice *d, Error **errp) s->real_device.dev, s->real_device.func); } - /* Initialize virtualized PCI configuration (Extended 256 Bytes) */ - memset(d->config, 0, PCI_CONFIG_SPACE_SIZE); + s->pcie_enabled_dev = xen_pt_dev_is_pcie_mode(d); + if (s->pcie_enabled_dev) { + XEN_PT_LOG(d, "Host device %04x:%02x:%02x.%d passed thru " + "in PCIe mode\n", s->real_device.domain, + s->real_device.bus, s->real_device.dev, + s->real_device.func); + } + + /* Initialize virtualized PCI configuration space (256/4K bytes) */ + memset(d->config, 0, pci_is_express(d) ? PCIE_CONFIG_SPACE_SIZE + : PCI_CONFIG_SPACE_SIZE); s->memory_listener = xen_pt_memory_listener; s->io_listener = xen_pt_io_listener; diff --git a/hw/xen/xen_pt.h b/hw/xen/xen_pt.h index b20744f7c7..1c9cd6b615 100644 --- a/hw/xen/xen_pt.h +++ b/hw/xen/xen_pt.h @@ -234,6 +234,7 @@ struct XenPCIPassthroughState { PCIHostDeviceAddress hostaddr; bool is_virtfn; + bool pcie_enabled_dev; bool permissive; bool permissive_warned; XenHostPCIDevice real_device; From patchwork Tue Jun 20 17:24:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 358F2EB64D7 for ; Tue, 20 Jun 2023 17:26:27 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552069.861958 (Exim 4.92) (envelope-from ) id 1qBf7R-0006di-Qj; Tue, 20 Jun 2023 17:26:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552069.861958; Tue, 20 Jun 2023 17:26:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7R-0006dT-LG; Tue, 20 Jun 2023 17:26:17 +0000 Received: by outflank-mailman (input) for mailman id 552069; Tue, 20 Jun 2023 17:26:16 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7Q-0004q1-NY for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:16 +0000 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [2607:f8b0:4864:20::1135]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8e86d7eb-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:14 +0200 (CEST) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-57083a06b71so41788197b3.1 for ; Tue, 20 Jun 2023 10:26:14 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:13 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8e86d7eb-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281973; x=1689873973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LEqoX2AjLX1PvGaIbFaWG2CMKLGJZqfTIlTMaOcN4vw=; b=syngMxkIZTeaQp9EqDCXblsZMbin0jsNke7dSjaot9xphnUmSuAruqjdyZe2WoweR3 eH3f1NX9hHczy6+4cCHq6/IvVWjiWN90p0y8rp4ZLL6+B6jbVkJDavnPNLdcDi0Dvsy0 GE88YUqM8DFZxDlftNsQp3Gcd2G3Q7VTV+HkrymeuvD2SjJpuHSV7W6isdnwiNQAnpbM l1sFy6VCqaTSSUiaqaeqTHMNKZC3TX3/pKRDESIrsX6VUJCPqDB0rGSJutFzEW+1FDYO mX2KmAlrMDsAn96lm/5k8KIKcHRj61qU1g/swUa5rHHKp57FAUtvjaBQ0q+0JXrryVWZ I9PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281973; x=1689873973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LEqoX2AjLX1PvGaIbFaWG2CMKLGJZqfTIlTMaOcN4vw=; b=d2JLtfhy7C5IuyehNJOHUeeQuoKPS9JMqg2aujEFA6na9jvQBdN+h098SMvLnxmOI1 xDZi1JJoo3ckaqsiKADw97uKg6g75xKxjumf0+GIbfV0orB4CHjgNEfPr2pJOAFDD8vd QGf72wZ6dUnP9EBFJ7DdltaVZGuQoNOAmc2grXNtAKmhIU7Y/74/kv8DzkMZTEx1qTkt MidqAregkNKCzwxrIX0tmKc9tcoPVIRpog5j2Yd5bBT2IHk1FO1j40qzkWgNAaUXMRkO JLMh0FrtS3lE1JeKanmjrt3qIat0E+DFMd/oBOBWwyU44Gj+ap2NUZUqgHNB1hUM2Q5K IVEA== X-Gm-Message-State: AC+VfDxViylTsMaamSl0l7RswJGXD0bqwCFDsEk9mXLOQ9W0jtUHfBLc 1UebMYirpo3HJ46cGVOnmL0KsVNOAp7ZPA== X-Google-Smtp-Source: ACHHUZ7kcV+4jXM1nvoQ+9LnCTBkDnjZwwW17Mts+DPH57RjuKWgSyke9VCfEcmOzSYeFt76ut4FPg== X-Received: by 2002:a81:7dd5:0:b0:56f:fa56:4c32 with SMTP id y204-20020a817dd5000000b0056ffa564c32mr9995963ywc.52.1687281973507; Tue, 20 Jun 2023 10:26:13 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 09/23] xen/pt: Xen PCIe passthrough support for Q35: bypass PCIe topology check Date: Tue, 20 Jun 2023 13:24:43 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Compared to legacy i440 system, there are certain difficulties while passing through PCIe devices to guest OSes like Windows 7 and above on platforms with native support of PCIe bus (in our case Q35). This problem is not applicable to older OSes like Windows XP -- PCIe passthrough on such OSes can be used normally as these OSes have no support for PCIe-specific features and treat all PCIe devices as legacy PCI ones. The problem manifests itself as "Code 10" error for a passed thru PCIe device in Windows Device Manager (along with exclamation mark on it). The device with such error do not function no matter the fact that Windows successfully booted while actually using this device, ex. as a primary VGA card with VBE features, LFB, etc. working properly during boot time. It doesn't matter which PCI class the device have -- the problem is common to GPUs, NIC cards, USB controllers, etc. In the same time, all these devices can be passed thru successfully using i440 emulation on same Windows 7+ OSes. The actual root cause of the problem lies in the fact that Windows kernel (PnP manager particularly) while processing StartDevice IRP refuses to continue to start the device and control flow actually doesn't even reach the IRP handler in the device driver at all. The real reason for this typically does not appear at the time PnP manager tries to start the device, but happens much earlier -- during the Windows boot stage, while enumerating devices on a PCI/PCIe bus in the Windows pci.sys driver. There is a set of checks for every discovered device on the PCIe bus. Failing some of them leads to marking the discovered PCIe device as 'invalid' by setting the flag. Later on, StartDevice attempt will fail due to this flag, finally resulting in Code 10 error. The actual check in pci.sys which results in the PCIe device being marked as 'invalid' in our case is a validation of upstream PCIe bus hierarchy to which passed through device belongs. Basically, pci.sys checks if the PCIe device has parent devices, such as PCIe Root Port or upstream PCIe switch. In our case the PCIe device has no parents and resides on bus 0 without eg. corresponding Root Port. Therefore, in order to resolve this problem in a architecturally correct way, we need to introduce to Xen some support of at least trivial non-flat PCI bus hierarchy. In very simplest case - just one virtual Root Port, on secondary bus of which all physical functions of the real passed thru device will reside, eg. GPU and its HDAudio function. This solution is not hard to implement technically, but there are multiple affecting limitations present in Xen (many related to each other) currently: - in many places the code is limited to use bus 0 only. This applicable to both hypervisor and supplemental modules like hvmloader. This limitation is enforced on API level -- many functions and interfaces allow to specify only devfn argument while bus 0 being implied. - lot of code assumes Type0 PCI config space layout only, while we need to handle Type1 PCI devices as well - currently there no way to assign to a guest domain even a simplest linked hierarchy of passed thru PCI devices. In some cases we might need to passthrough a real PCIe Switch/Root Port with his downstream child devices. - in a similar way Xen/hvmloader lacks the concept of IO/MMIO space nesting. Both code which does MMIO hole sizing and code which allocates BARs to MMIO hole have no idea of MMIO ranges nesting and their relations. In case of virtual Root Port we have basically an emulated PCI-PCI bridge with some parts of its MMIO range used for real MMIO ranges of passed through device(s). So, adding to Xen multiple PCI buses support will require a bit of effort and discussions regarding the actual design of the feature. Nevertheless, this task is crucial for PCI/GPU passthrough features of Xen to work properly. To summarize, we need to implement following things in the future: 1) Get rid of PCI bus 0 limitation everywhere. This could've been a simplest of subtasks but in reality this will require to change interfaces as well - AFAIR even adding a PCI device via QMP only allows to specify a device slot while we need to have some way to place the device on an arbitrary bus. 2) Fully or partially emulated PCI-PCI bridge which will provide a secondary bus for PCIe device placement - there might be a possibility to reuse some existing emulation QEMU provides. This also includes Type1 devices support. The task will become more complicated if there arise necessity, for example, to control the PCIe link for a passed through PCIe device. As PT device reset is mandatory in most cases, there might be a chance to encounter a situation when we need to retrain the PCIe link to restore PCIe link speed after the reset. In this case there will be a need to selectively translate accesses to certain registers of emulated PCIe Switch/Root Port to the corresponding physical upstream PCIe Switch/RootPort. This will require some interaction with Dom0, hopefully extending xen-pciback will be enough. 3) The concept of I/O and MMIO ranges nesting, for tasks like sizing MMIO hole or PCI BAR allocation. This one should be pretty simple. The actual implementation still is a matter to discuss of course. In the meantime there can be used a very simple workaround which allows to bypass pci.sys limitation for PCIe topology check - there exist one good exception to "must have upstream PCIe parent" rule of pci.sys. It's chipset-integrated devices. How pci.sys can tell if it deals with a chipset built-in device? It checks one of PCI Express Capability fields in the device PCI conf space. For chipset built-in devices this field will state "root complex integrated device" while in our case for a normal passed thru PCIe device there will be a "PCIe endpoint" type. So that's what the workaround does - it intercepts reading of this particular field for passed through devices and returns the "root complex integrated device" value for PCIe endpoints. This makes pci.sys happy and allows Windows 7 and above to use PT device on PCIe-capable system normally. So far no negative side effects were encountered while using this approach, so it's a good temporary solution until multiple PCI bus support will be added to Xen. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 49 +++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 47c8482f32..757a035aad 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -907,6 +907,55 @@ static int xen_pt_linkctrl2_reg_init(XenPCIPassthroughState *s, return 0; } +/* initialize PCI Express Capabilities register */ +static int xen_pt_pcie_capabilities_reg_init(XenPCIPassthroughState *s, + XenPTRegInfo *reg, + uint32_t real_offset, + uint32_t *data) +{ + uint8_t dev_type = get_pcie_device_type(s); + uint16_t reg_field; + + if (xen_host_pci_get_word(&s->real_device, + real_offset - reg->offset + PCI_EXP_FLAGS, + ®_field)) { + XEN_PT_ERR(&s->dev, "Error reading PCIe Capabilities reg\n"); + *data = 0; + return 0; + } + /* + * Q35 workaround for Win7+ pci.sys PCIe topology check. + * As our PT device currently located on a bus 0, fake the + * device/port type field to the "Root Complex integrated device" + * value to bypass the check + */ + switch (dev_type) { + case PCI_EXP_TYPE_ENDPOINT: + case PCI_EXP_TYPE_LEG_END: + XEN_PT_LOG(&s->dev, "Original PCIe Capabilities reg is 0x%04X\n", + reg_field); + reg_field &= ~PCI_EXP_FLAGS_TYPE; + reg_field |= ((PCI_EXP_TYPE_RC_END /*9*/ << 4) & PCI_EXP_FLAGS_TYPE); + XEN_PT_LOG(&s->dev, "Q35 PCIe topology check workaround: " + "faking Capabilities reg to 0x%04X\n", reg_field); + break; + + case PCI_EXP_TYPE_ROOT_PORT: + case PCI_EXP_TYPE_UPSTREAM: + case PCI_EXP_TYPE_DOWNSTREAM: + case PCI_EXP_TYPE_PCI_BRIDGE: + case PCI_EXP_TYPE_PCIE_BRIDGE: + case PCI_EXP_TYPE_RC_END: + case PCI_EXP_TYPE_RC_EC: + default: + /* do nothing, return as is */ + break; + } + + *data = reg_field; + return 0; +} + /* PCI Express Capability Structure reg static information table */ static XenPTRegInfo xen_pt_emu_reg_pcie[] = { /* Next Pointer reg */ From patchwork Tue Jun 20 17:24:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D750BEB64D8 for ; Tue, 20 Jun 2023 17:26:36 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552070.861967 (Exim 4.92) (envelope-from ) id 1qBf7U-00071T-7b; Tue, 20 Jun 2023 17:26:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552070.861967; Tue, 20 Jun 2023 17:26:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7U-00071D-4N; Tue, 20 Jun 2023 17:26:20 +0000 Received: by outflank-mailman (input) for mailman id 552070; Tue, 20 Jun 2023 17:26:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7T-0004q1-9l for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:19 +0000 Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [2607:f8b0:4864:20::1136]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8f95edf0-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:16 +0200 (CEST) Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-56ff9cc91b4so56501427b3.0 for ; Tue, 20 Jun 2023 10:26:16 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:15 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8f95edf0-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281975; x=1689873975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5oficzjs5xcBYSHcpYrMTcfh+AwyMWQx/k5j2kJ4DGo=; b=O2gmxwgPbbW0ycNLE6VtVZ2j4XzYDPo99Wh8EBMogWPXV+Kpoo9l2hXiGgJTL+3UWy AV3HvB2nSQNIdHMZ3w/fcuOS3koy85JWsyMEA7Yff+X+tkkHnwQAUrI+P7vHfcJew7Ug Xm/I3seDw2cBPzJ85DqXi1QSKXudjAHLntTZvTgkwyQs8QodwBBGFOhFsBa9wdc+0HRV HC+t05bwvGK34OqN7tNbN9nZ4fHb12xARVgPdXFx+ua4ql/xGu5zg7kk2qyn2rHk3n0l 9IaX5IlOAWeqfzqzljQHBqfCYEO5CzgHpqTA/ZiuAQed4J4/mx+PjRPgfZHTOe0RL5mE DujQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281975; x=1689873975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5oficzjs5xcBYSHcpYrMTcfh+AwyMWQx/k5j2kJ4DGo=; b=EJVJY8eWd9JJWyqHFcMDyvdMQ27K5DuILLi9pNmXp6Nv/Xk+tZ3sHOm1YX+vxI1sBD MjlFGRvSlGKepiySkziYNPPNtzJa6C9b/Cb/G9fWvXFGYmj6M9ayiVd+Nz3e/+tVq9Vd 8xX24D1ZzEIH5Ev5hWMKZdzloEPmxJxlCK9U7p/0jc1nGAE/KVNqCEuXZy64L178NV2U mrG2g4hPE9xQP9SE1hMZmmAVnP0P1vlC87yyZk6ZAECGlhkL5bFIPfXyeqFB1zN/ajVJ OGz4S4kcQJj+T1a5JgrFeTkqJzcJMtNUX43wq7ql2OpFiiED3WhkqWKldTm29KuLRZOw vzEw== X-Gm-Message-State: AC+VfDzT0r3S5e5xRfrKBF/niXqsb0F0BexwlCVmXzEEfirU5SJejhxx 1xjCk6e4QbOpVJrSJHBFhMA= X-Google-Smtp-Source: ACHHUZ6XnSFKFEmZ4JRuuQ8RG5O/zfoeupAWUrMFLcGCPcnHuHkd+hGqCOhfxRag80J/Ukwvt5V6wQ== X-Received: by 2002:a81:6ac6:0:b0:56d:ccf:19a9 with SMTP id f189-20020a816ac6000000b0056d0ccf19a9mr16316679ywc.34.1687281975343; Tue, 20 Jun 2023 10:26:15 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 10/23] xen/pt: add support for PCIe Extended Capabilities and larger config space Date: Tue, 20 Jun 2023 13:24:44 -0400 Message-Id: <775c2acb6ac819990e28d881167fcdfcd58b9886.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 This patch provides basic facilities for PCIe Extended Capabilities and support for controlled (via s->pcie_enabled_dev flag) access to PCIe config space (>256). PCIe Extended Capabilities make use of 16-bit capability ID. Also, a capability size might exceed 8-bit width. So as the very first step we need to increase type size for grp_id, grp_size, etc -- they were limited to 8-bit. The only troublesome issue with PCIe Extended Capability IDs is that their value range is actually same as for basic PCI capabilities. Eg. capability ID 3 means VPD Capability for PCI and at the same time Device Serial Number Capability for PCIe Extended caps. This adds a bit of inconvenience. In order to distinguish between two sets of same capability IDs, the patch introduces a set of macros to mark a capability ID as PCIe Extended one (or check if it is basic/extended + get a raw ID value): - PCIE_EXT_CAP_ID(cap_id) - IS_PCIE_EXT_CAP_ID(grp_id) - GET_PCIE_EXT_CAP_ID(grp_id) Here is how it's used: /* Intel IGD Opregion group */ { .grp_id = XEN_PCI_INTEL_OPREGION, /* no change */ .grp_type = XEN_PT_GRP_TYPE_EMU, .grp_size = 0x4, .size_init = xen_pt_reg_grp_size_init, .emu_regs = xen_pt_emu_reg_igd_opregion, }, /* Vendor-specific Extended Capability reg group */ { .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VNDR), .grp_type = XEN_PT_GRP_TYPE_EMU, .grp_size = 0xFF, .size_init = xen_pt_ext_cap_vendor_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_vendor, }, By using the PCIE_EXT_CAP_ID() macro it is possible to reuse existing header files with already defined PCIe Extended Capability ID values. find_cap_offset() receive capabily ID and checks if it's an Extended one by using IS_PCIE_EXT_CAP_ID(cap) macro, passing the real capabiliy ID value to either xen_host_pci_find_next_ext_cap or xen_host_pci_find_next_cap. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt.c | 10 ++++- hw/xen/xen_pt.h | 13 ++++-- hw/xen/xen_pt_config_init.c | 90 ++++++++++++++++++++++++++----------- 3 files changed, 83 insertions(+), 30 deletions(-) diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index 65c5516ef4..f757978800 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -96,8 +96,16 @@ void xen_pt_log(const PCIDevice *d, const char *f, ...) static int xen_pt_pci_config_access_check(PCIDevice *d, uint32_t addr, int len) { + XenPCIPassthroughState *s = XEN_PT_DEVICE(d); /* check offset range */ - if (addr > 0xFF) { + if (s->pcie_enabled_dev) { + if (addr >= PCIE_CONFIG_SPACE_SIZE) { + XEN_PT_ERR(d, "Failed to access register with offset " + "exceeding 0xFFF. (addr: 0x%02x, len: %d)\n", + addr, len); + return -1; + } + } else if (addr >= PCI_CONFIG_SPACE_SIZE) { XEN_PT_ERR(d, "Failed to access register with offset exceeding 0xFF. " "(addr: 0x%02x, len: %d)\n", addr, len); return -1; diff --git a/hw/xen/xen_pt.h b/hw/xen/xen_pt.h index 1c9cd6b615..eb062be3f4 100644 --- a/hw/xen/xen_pt.h +++ b/hw/xen/xen_pt.h @@ -33,6 +33,11 @@ void xen_pt_log(const PCIDevice *d, const char *f, ...) G_GNUC_PRINTF(2, 3); /* Helper */ #define XEN_PFN(x) ((x) >> XC_PAGE_SHIFT) +/* Macro's for PCIe Extended Capabilities */ +#define PCIE_EXT_CAP_ID(cap_id) ((cap_id) | (1U << 16)) +#define IS_PCIE_EXT_CAP_ID(grp_id) ((grp_id) & (1U << 16)) +#define GET_PCIE_EXT_CAP_ID(grp_id) ((grp_id) & 0xFFFF) + typedef const struct XenPTRegInfo XenPTRegInfo; typedef struct XenPTReg XenPTReg; @@ -174,13 +179,13 @@ typedef const struct XenPTRegGroupInfo XenPTRegGroupInfo; /* emul reg group size initialize method */ typedef int (*xen_pt_reg_size_init_fn) (XenPCIPassthroughState *, XenPTRegGroupInfo *, - uint32_t base_offset, uint8_t *size); + uint32_t base_offset, uint32_t *size); /* emulated register group information */ struct XenPTRegGroupInfo { - uint8_t grp_id; + uint32_t grp_id; XenPTRegisterGroupType grp_type; - uint8_t grp_size; + uint32_t grp_size; xen_pt_reg_size_init_fn size_init; XenPTRegInfo *emu_regs; }; @@ -190,7 +195,7 @@ typedef struct XenPTRegGroup { QLIST_ENTRY(XenPTRegGroup) entries; XenPTRegGroupInfo *reg_grp; uint32_t base_offset; - uint8_t size; + uint32_t size; QLIST_HEAD(, XenPTReg) reg_tbl_list; } XenPTRegGroup; diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 757a035aad..34ed9c25c5 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -32,28 +32,40 @@ static int xen_pt_ptr_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, /* helper */ /* A return value of 1 means the capability should NOT be exposed to guest. */ -static int xen_pt_hide_dev_cap(const XenHostPCIDevice *d, uint8_t grp_id) +static int xen_pt_hide_dev_cap(const XenHostPCIDevice *d, uint32_t grp_id) { - switch (grp_id) { - case PCI_CAP_ID_EXP: - /* The PCI Express Capability Structure of the VF of Intel 82599 10GbE - * Controller looks trivial, e.g., the PCI Express Capabilities - * Register is 0. We should not try to expose it to guest. - * - * The datasheet is available at - * http://download.intel.com/design/network/datashts/82599_datasheet.pdf - * - * See 'Table 9.7. VF PCIe Configuration Space' of the datasheet, the - * PCI Express Capability Structure of the VF of Intel 82599 10GbE - * Controller looks trivial, e.g., the PCI Express Capabilities - * Register is 0, so the Capability Version is 0 and - * xen_pt_pcie_size_init() would fail. - */ - if (d->vendor_id == PCI_VENDOR_ID_INTEL && - d->device_id == PCI_DEVICE_ID_INTEL_82599_SFP_VF) { - return 1; + if (IS_PCIE_EXT_CAP_ID(grp_id)) { + switch (GET_PCIE_EXT_CAP_ID(grp_id)) { + /* Here can be added device-specific filtering + * for PCIe Extended capabilities (those with offset >= 0x100). + * This is simply a placeholder as no filtering needed for now. + */ + default: + break; + } + } else { + /* basic PCI capability */ + switch (grp_id) { + case PCI_CAP_ID_EXP: + /* The PCI Express Capability Structure of the VF of Intel 82599 10GbE + * Controller looks trivial, e.g., the PCI Express Capabilities + * Register is 0. We should not try to expose it to guest. + * + * The datasheet is available at + * http://download.intel.com/design/network/datashts/82599_datasheet.pdf + * + * See 'Table 9.7. VF PCIe Configuration Space' of the datasheet, the + * PCI Express Capability Structure of the VF of Intel 82599 10GbE + * Controller looks trivial, e.g., the PCI Express Capabilities + * Register is 0, so the Capability Version is 0 and + * xen_pt_pcie_size_init() would fail. + */ + if (d->vendor_id == PCI_VENDOR_ID_INTEL && + d->device_id == PCI_DEVICE_ID_INTEL_82599_SFP_VF) { + return 1; + } + break; } - break; } return 0; } @@ -1615,7 +1627,7 @@ static XenPTRegInfo xen_pt_emu_reg_igd_opregion[] = { static int xen_pt_reg_grp_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg, - uint32_t base_offset, uint8_t *size) + uint32_t base_offset, uint32_t *size) { *size = grp_reg->grp_size; return 0; @@ -1623,14 +1635,22 @@ static int xen_pt_reg_grp_size_init(XenPCIPassthroughState *s, /* get Vendor Specific Capability Structure register group size */ static int xen_pt_vendor_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg, - uint32_t base_offset, uint8_t *size) + uint32_t base_offset, uint32_t *size) +{ + uint8_t sz = 0; + int ret = xen_host_pci_get_byte(&s->real_device, base_offset + 0x02, &sz); + + *size = sz; + return ret; +} + { return xen_host_pci_get_byte(&s->real_device, base_offset + 0x02, size); } /* get PCI Express Capability Structure register group size */ static int xen_pt_pcie_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg, - uint32_t base_offset, uint8_t *size) + uint32_t base_offset, uint32_t *size) { PCIDevice *d = PCI_DEVICE(s); uint8_t version = get_pcie_capability_version(s); @@ -1702,7 +1722,7 @@ static int xen_pt_pcie_size_init(XenPCIPassthroughState *s, /* get MSI Capability Structure register group size */ static int xen_pt_msi_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg, - uint32_t base_offset, uint8_t *size) + uint32_t base_offset, uint32_t *size) { uint16_t msg_ctrl = 0; uint8_t msi_size = 0xa; @@ -1730,7 +1750,7 @@ static int xen_pt_msi_size_init(XenPCIPassthroughState *s, /* get MSI-X Capability Structure register group size */ static int xen_pt_msix_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg, - uint32_t base_offset, uint8_t *size) + uint32_t base_offset, uint32_t *size) { int rc = 0; @@ -1953,6 +1973,26 @@ static uint8_t find_cap_offset(XenPCIPassthroughState *s, uint8_t cap) return 0; } +/************* + * Main + */ + +static uint32_t find_cap_offset(XenPCIPassthroughState *s, uint32_t cap) +{ + uint32_t retval = 0; + + if (IS_PCIE_EXT_CAP_ID(cap)) { + if (s->pcie_enabled_dev) { + retval = xen_host_pci_find_next_ext_cap(&s->real_device, 0, + GET_PCIE_EXT_CAP_ID(cap)); + } + + } else { + retval = xen_host_pci_find_next_cap(&s->real_device, 0, cap); + } + return retval; +} + static void xen_pt_config_reg_init(XenPCIPassthroughState *s, XenPTRegGroup *reg_grp, XenPTRegInfo *reg, Error **errp) From patchwork Tue Jun 20 17:24:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4406BEB64DB for ; Tue, 20 Jun 2023 17:26:31 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552075.861977 (Exim 4.92) (envelope-from ) id 1qBf7V-0007Ho-Ir; Tue, 20 Jun 2023 17:26:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552075.861977; Tue, 20 Jun 2023 17:26:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7V-0007HO-DT; Tue, 20 Jun 2023 17:26:21 +0000 Received: by outflank-mailman (input) for mailman id 552075; Tue, 20 Jun 2023 17:26:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7U-0004q1-Gr for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:20 +0000 Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [2607:f8b0:4864:20::32d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 90d758ef-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:18 +0200 (CEST) Received: by mail-ot1-x32d.google.com with SMTP id 46e09a7af769-6b5853a140cso1750519a34.2 for ; Tue, 20 Jun 2023 10:26:18 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:17 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 90d758ef-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281977; x=1689873977; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6HvrCL8rOJsuiwOG3muIoVRllaEm248y0LD47Kgrft0=; b=nmgU4y9Ln0rtIz80PAR3vTp7t61K68cxtW9WeRAqn7eaE3uMj1tya6GoffQApOwPAD kwbpLr9u2Y90w7sOrUMa+ArGM8Ep49feEFdPQONaGxHoMfoLK1N63oRqHGJpxj4lCw2C eIX4N0M/EYtGsqpl9NfqIMVk+rbdMPaJbFUp8SNYa5mPbJB2q99z+qXsrJQXNFsxMyB1 NP+HYGwDfrCjlZ9JQZC6nHfjzf++zIthNMA0BoriKULZyJhp7bWZlO2SJbBUy/CXTPpV urJ/7Hw4lJxU78QQ1byX2mi/JGovBTW7NtftPygrxBG8yIV8IPUBkt4HtpgLWeu9wQdm 3Oyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281977; x=1689873977; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6HvrCL8rOJsuiwOG3muIoVRllaEm248y0LD47Kgrft0=; b=Oi5FKrVTmpSxxFKlfA40ZibIQfGHAEhD/SuXb1CjYi7LLpiK/491SH47q14BlCsFmR 9wJor+u64ezeY8D+LrPabUuMszeFlrhqyBbugCVWUiZohMRQwwwbkHWLJ2hDjvzrFaY2 eydNvNMrp+F6dUNYFWK0onDmPQbonkpwKUwTHqG6R5Gbty1mv91iCka473rzE2ZJi+ob LkNHFnQ7OA9tvqqNjaHU3rIcpcEx0zZRO+sSymCH6hc7mJKW4xVCI8pgVKBQ56lYTfbm us96r7L9E9gwlIKHY03YJpKSxi4QulUvsRW+3ZSKTJRKnAZ+GT0B7K3eJ501Bvcy497Z Ex2A== X-Gm-Message-State: AC+VfDzoSWnV5T1xyenCqEJ4OCUOlRPj4VCxI379SmfWlYqKEzfAdFIE VLn2o6/h5IRqIGubxqL4WX0= X-Google-Smtp-Source: ACHHUZ6Qt7O71pC+krywoR4rbBwRp9dAu+gyJXT+ubbJS5lT0aUS37r2zV6kUeMl6FXGYYhiuV77iQ== X-Received: by 2002:a05:6358:455:b0:12b:e9fd:4b53 with SMTP id 21-20020a056358045500b0012be9fd4b53mr8661292rwe.22.1687281977366; Tue, 20 Jun 2023 10:26:17 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 11/23] xen/pt: handle PCIe Extended Capabilities Next register Date: Tue, 20 Jun 2023 13:24:45 -0400 Message-Id: <8687512cd728cd0ad00d9a7d35cd3ce5c3a2c2ca.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 The patch adds new xen_pt_ext_cap_ptr_reg_init function which is used to initialize the emulated next pcie extended capability pointer. Primary mission of this function is to have a method to selectively hide some extended capabilities from the capability linked list, skipping them by altering the Next capability pointer value. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 87 +++++++++++++++++++++++-------------- 1 file changed, 55 insertions(+), 32 deletions(-) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 34ed9c25c5..ed36edbc4a 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -27,7 +27,10 @@ static int xen_pt_ptr_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data); - +static int xen_pt_ext_cap_ptr_reg_init(XenPCIPassthroughState *s, + XenPTRegInfo *reg, + uint32_t real_offset, + uint32_t *data); /* helper */ @@ -1928,48 +1931,68 @@ out: return 0; } +#define PCIE_EXT_CAP_NEXT_SHIFT 4 +#define PCIE_EXT_CAP_VER_MASK 0xF -/************* - * Main - */ - -static uint8_t find_cap_offset(XenPCIPassthroughState *s, uint8_t cap) +static int xen_pt_ext_cap_ptr_reg_init(XenPCIPassthroughState *s, + XenPTRegInfo *reg, + uint32_t real_offset, + uint32_t *data) { - uint8_t id; - unsigned max_cap = XEN_PCI_CAP_MAX; - uint8_t pos = PCI_CAPABILITY_LIST; - uint8_t status = 0; + int i, rc; + XenHostPCIDevice *d = &s->real_device; + uint16_t reg_field; + uint16_t cur_offset, version, cap_id; + uint32_t header; - if (xen_host_pci_get_byte(&s->real_device, PCI_STATUS, &status)) { - return 0; - } - if ((status & PCI_STATUS_CAP_LIST) == 0) { - return 0; + if (real_offset < 0x0010) { + XEN_PT_ERR(&s->dev, "Incorrect PCIe extended capability offset " + "encountered: 0x%04x\n", real_offset); + return -EINVAL; } - while (max_cap--) { - if (xen_host_pci_get_byte(&s->real_device, pos, &pos)) { - break; - } - if (pos < PCI_CONFIG_HEADER_SIZE) { - break; - } + rc = xen_host_pci_get_word(d, real_offset, ®_field); + if (rc) + return rc; - pos &= ~3; - if (xen_host_pci_get_byte(&s->real_device, - pos + PCI_CAP_LIST_ID, &id)) { - break; - } + /* preserve version field */ + version = reg_field & PCIE_EXT_CAP_VER_MASK; + cur_offset = reg_field >> PCIE_EXT_CAP_NEXT_SHIFT; - if (id == 0xff) { - break; + while (cur_offset && cur_offset != 0xFFF) { + rc = xen_host_pci_get_long(d, cur_offset, &header); + if (rc) { + XEN_PT_ERR(&s->dev, "Failed to read PCIe extended capability " + "@0x%x (rc:%d)\n", cur_offset, rc); + return rc; } - if (id == cap) { - return pos; + + cap_id = PCI_EXT_CAP_ID(header); + + for (i = 0; xen_pt_emu_reg_grps[i].grp_size != 0; i++) { + uint32_t cur_grp_id = xen_pt_emu_reg_grps[i].grp_id; + + if (!IS_PCIE_EXT_CAP_ID(cur_grp_id)) + continue; + + if (xen_pt_hide_dev_cap(d, cur_grp_id)) + continue; + + if (GET_PCIE_EXT_CAP_ID(cur_grp_id) == cap_id) { + if (xen_pt_emu_reg_grps[i].grp_type == XEN_PT_GRP_TYPE_EMU) + goto out; + + /* skip TYPE_HARDWIRED capability, move the ptr to next one */ + break; + } } - pos += PCI_CAP_LIST_NEXT; + /* next capability */ + cur_offset = PCI_EXT_CAP_NEXT(header); } + +out: + *data = (cur_offset << PCIE_EXT_CAP_NEXT_SHIFT) | version; return 0; } From patchwork Tue Jun 20 17:24:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0256EB64D7 for ; Tue, 20 Jun 2023 17:26:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552076.861986 (Exim 4.92) (envelope-from ) id 1qBf7W-0007ag-SE; Tue, 20 Jun 2023 17:26:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552076.861986; Tue, 20 Jun 2023 17:26:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7W-0007Zz-O3; Tue, 20 Jun 2023 17:26:22 +0000 Received: by outflank-mailman (input) for mailman id 552076; Tue, 20 Jun 2023 17:26:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7V-0005Q4-C5 for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:21 +0000 Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [2607:f8b0:4864:20::112a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 91e49442-0f8f-11ee-b234-6b7b168915f2; Tue, 20 Jun 2023 19:26:20 +0200 (CEST) Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-5700401acbeso54070667b3.0 for ; Tue, 20 Jun 2023 10:26:20 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:18 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 91e49442-0f8f-11ee-b234-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281979; x=1689873979; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9ketisDiBHbRbLfAtb4owX6CirSCgZ560hstWukEHPU=; b=JV7mVW9o+DEfW49UhY6Ry0LlDSYnubxQvhH/85RSPJPv5iznjafVvKwT0WTw80NIHH ZzVJZ4uxnp6Hy9ouHIKN/+dUXz8k7PgDunBleGVEfPS4AGMqIIn0cN0bzuYyHyjXJt9S c4qjaIg4M+xIfnZlwwTV4j2h2oPsNsgCab9w5IYYB7RNikhwoAkZ7HPMhFE+Y/J6GhHn vL5uSRHMD7mxu/Qe0WvnyIcUII9xycI5IylaW7/BSTgK9RIqaXATKHTUBh9mjIBw2MUY jqGFkFdOXhXlpkyJu+qAZTjJsClrW1bOrRSiDbVFLiEPjs4JJdZo6525f4dH3VxIWXDR yNSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281979; x=1689873979; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9ketisDiBHbRbLfAtb4owX6CirSCgZ560hstWukEHPU=; b=BoOvfSqPVu6KDIrMOKYsQYtZL4zf5Lu+PCJ03BKA5SiiXGqGDEcK7z4lOk7FNCh7i6 c1JPwYdScUexfvxfI+ooB+86dEN6zAoGyemXlBjCCThPe5aH5FZ8pJBqZgLG/YDhjWnP ax4mXA/xYJsEbmCPDvN3XbOj3TLGv/nk5LSuxz8rVERJOb2h93fIqwwOkn/O15ERf/fl laNiWDs2pAWHyMUT9f5veAVlbo9lQvnciZDzDw+SCa/TX1R//tH9q9wjlteP18JVXkFW JuIk3NUahpbNPApB8kT6O2vwhKqNQsMP5uv5AswdoTsanYW/mm/5FxSBo9rfI+wiraYa r8kQ== X-Gm-Message-State: AC+VfDx1ySQZOAEbUp1reIjHOX1r5qJ28VdmsIf8I1/rwLzf6cT4FVTD GM0tztKXNFHkBsmggXYuS5Y= X-Google-Smtp-Source: ACHHUZ7Sy8MslonVrUjxUdWKSpd6ukF5+zxv5os2k+6NK3tmmIpKUWxcbVfweJ393p1b0WLmoKTnJw== X-Received: by 2002:a0d:e447:0:b0:565:99c0:a690 with SMTP id n68-20020a0de447000000b0056599c0a690mr12747083ywe.36.1687281979267; Tue, 20 Jun 2023 10:26:19 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 12/23] xen/pt: allow to hide PCIe Extended Capabilities Date: Tue, 20 Jun 2023 13:24:46 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 We need to hide some unwanted PCI/PCIe capabilities for passed through devices. Normally we do this by marking the capability register group as XEN_PT_GRP_TYPE_HARDWIRED which exclude this capability from the capability list and returns zeroes on attempts to read capability body. Skipping the capability in the linked list of capabilities can be done by changing Next Capability register to skip one or many unwanted capabilities. One difference between PCI and PCIe Extended capabilities is that we don't have the list head field anymore. PCIe Extended capabilities always start at offset 0x100 if they're present. Unfortunately, there are typically only few PCIe extended capabilities present which means there is a chance that some capability we want to hide will reside at offset 0x100 in PCIe config space. The simplest way to hide such capabilities from guest OS or drivers is faking their capability ID value. This patch adds the Capability ID register handler which checks - if the capability to which this register belong starts at offset 0x100 in PCIe config space - if this capability is marked as XEN_PT_GRP_TYPE_HARDWIRED If it is the case, then a fake Capability ID value is returned. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt.c | 11 ++++++++++- hw/xen/xen_pt.h | 4 ++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index f757978800..2399fabb2b 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -164,7 +164,16 @@ static uint32_t xen_pt_pci_read_config(PCIDevice *d, uint32_t addr, int len) reg_grp_entry = xen_pt_find_reg_grp(s, addr); if (reg_grp_entry) { /* check 0-Hardwired register group */ - if (reg_grp_entry->reg_grp->grp_type == XEN_PT_GRP_TYPE_HARDWIRED) { + if (reg_grp_entry->reg_grp->grp_type == XEN_PT_GRP_TYPE_HARDWIRED && + /* + * For PCIe Extended Capabilities we need to emulate + * CapabilityID and NextCapability/Version registers for a + * hardwired reg group located at the offset 0x100 in PCIe + * config space. This allows us to hide the first extended + * capability as well. + */ + !(reg_grp_entry->base_offset == PCI_CONFIG_SPACE_SIZE && + ranges_overlap(addr, len, 0x100, 4))) { /* no need to emulate, just return 0 */ val = 0; goto exit; diff --git a/hw/xen/xen_pt.h b/hw/xen/xen_pt.h index eb062be3f4..9a191cbc8f 100644 --- a/hw/xen/xen_pt.h +++ b/hw/xen/xen_pt.h @@ -93,6 +93,10 @@ typedef int (*xen_pt_conf_byte_read) #define XEN_PCI_INTEL_OPREGION 0xfc +#define XEN_PCIE_CAP_ID 0 +#define XEN_PCIE_CAP_LIST_NEXT 2 +#define XEN_PCIE_FAKE_CAP_ID_BASE 0xFE00 + #define XEN_PCI_IGD_DOMAIN 0 #define XEN_PCI_IGD_BUS 0 #define XEN_PCI_IGD_DEV 2 From patchwork Tue Jun 20 17:24:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20876EB64D8 for ; Tue, 20 Jun 2023 17:29:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552101.862006 (Exim 4.92) (envelope-from ) id 1qBfAS-0002DX-Sv; Tue, 20 Jun 2023 17:29:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552101.862006; Tue, 20 Jun 2023 17:29:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAS-0002DQ-QM; Tue, 20 Jun 2023 17:29:24 +0000 Received: by outflank-mailman (input) for mailman id 552101; Tue, 20 Jun 2023 17:29:24 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7Y-0004q1-MX for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:24 +0000 Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com [2607:f8b0:4864:20::32b]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9329ca32-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:22 +0200 (CEST) Received: by mail-ot1-x32b.google.com with SMTP id 46e09a7af769-6b57d9043e4so1892872a34.2 for ; Tue, 20 Jun 2023 10:26:22 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:20 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9329ca32-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281981; x=1689873981; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K61wLAMvxduhXajLJmXKkCD2KgQmA24lyEh8ZOauW3g=; b=IX6OWGyMDuquswGxYoFJvDL0Ts6f749+YDHQGq/VK7v6MzucLGn3mJLNRgyBXD73nh I/uap9v+a/tpM5eiSkd2OkS0iQARyHW6tg8ULcTLPED98ouQUyuWH2UQR8ZlORK0Cr6B BDgLegkcEOk635hmIGl+Bwtfz5e1hT96JE/JDmGP0Bqw0e7cIkLr/RU266DuT5tbGceb zRUNYzhZb93MyWsSOjfGJeeGl5aBmP4hUNJ3PbRSOE6eRw3nAOQED29dYWIme6OHtNXm 5ulmEwaOXRsnulp5JjUk1Y9/q/G60rjQqvx5duoVQR4CjtOtUdcENi3WHS3YKVj8zhvS WFwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281981; x=1689873981; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K61wLAMvxduhXajLJmXKkCD2KgQmA24lyEh8ZOauW3g=; b=djD44MFWJm0v3nw6RQEDi/69ILS2qwgrWYKx6HZRor/NOJHXsut8FtzKC4xP1Dldv4 cBqk4kHLZZuxhh8BJKN9mQx77OS5KD2gZ5cAKSXocXYyNsq+rmgxetLxAbYOICshjATd VYT6MntJQILChifhpusoF9Z1mACNLltcrH2yIuOQh3R4UUWWbEulZFihuFzIvWbQJAJz IAyqOz4JyPJkni9h1pTIl9WrlCM6adGfsH2IogsSUwxjmCvCs6+i/PNOJ7xQotnOAKmV iLfQP0f8HluwbV30cXvRD8eRGZVJMMM0GvpjrHJJMpJnuwrgLqkSJGVtnh8XFnn7XCTo wqiQ== X-Gm-Message-State: AC+VfDyudPLOCvEykoZ3moXIQSeG+S/r7CXY1p+FdN3W5gOkru4qlfcd FLLHsC6ZHW89fveF+jNS4oo= X-Google-Smtp-Source: ACHHUZ5spqw7iFHqYn8GRj32Nl0UI37k62ovrhkdOFMHxlCYXYMjF7pxPN/co2hG4xEewZ4G5pFXTg== X-Received: by 2002:a05:6358:1a84:b0:130:e675:3e8c with SMTP id gm4-20020a0563581a8400b00130e6753e8cmr4571923rwb.28.1687281981240; Tue, 20 Jun 2023 10:26:21 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 13/23] xen/pt: add Vendor-specific PCIe Extended Capability descriptor and sizing Date: Tue, 20 Jun 2023 13:24:47 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 The patch provides Vendor-specific PCIe Extended Capability description structure and corresponding sizing function. In this particular case the size of the Vendor capability is available in the VSEC Length field. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 71 ++++++++++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index ed36edbc4a..20b5561d25 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -124,6 +124,17 @@ static uint32_t get_throughable_mask(const XenPCIPassthroughState *s, return throughable_mask & valid_mask; } +static void log_pcie_extended_cap(XenPCIPassthroughState *s, + const char *cap_name, + uint32_t base_offset, uint32_t size) +{ + if (size) { + XEN_PT_LOG(&s->dev, "Found PCIe Extended Capability: %s at 0x%04x, " + "size 0x%x bytes\n", cap_name, + (uint16_t) base_offset, size); + } +} + /**************** * general register functions */ @@ -1622,6 +1633,42 @@ static XenPTRegInfo xen_pt_emu_reg_igd_opregion[] = { }, }; +/* Vendor-specific Ext Capability Structure reg static information table */ +static XenPTRegInfo xen_pt_ext_cap_emu_reg_vendor[] = { + { + .offset = XEN_PCIE_CAP_ID, + .size = 2, + .init_val = 0x0000, + .ro_mask = 0xFFFF, + .emu_mask = 0xFFFF, + .init = xen_pt_ext_cap_capid_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .offset = XEN_PCIE_CAP_LIST_NEXT, + .size = 2, + .init_val = 0x0000, + .ro_mask = 0xFFFF, + .emu_mask = 0xFFFF, + .init = xen_pt_ext_cap_ptr_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .offset = PCI_VNDR_HEADER, + .size = 4, + .init_val = 0x00000000, + .ro_mask = 0xFFFFFFFF, + .emu_mask = 0x00000000, + .init = xen_pt_common_reg_init, + .u.dw.read = xen_pt_long_reg_read, + .u.dw.write = xen_pt_long_reg_write, + }, + { + .size = 0, + }, +}; /**************************** * Capabilities */ @@ -1647,9 +1694,23 @@ static int xen_pt_vendor_size_init(XenPCIPassthroughState *s, return ret; } +static int xen_pt_ext_cap_vendor_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) { - return xen_host_pci_get_byte(&s->real_device, base_offset + 0x02, size); + uint32_t vsec_hdr = 0; + int ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_VNDR_HEADER, + &vsec_hdr); + + *size = PCI_VNDR_HEADER_LEN(vsec_hdr); + + log_pcie_extended_cap(s, "Vendor-specific", base_offset, *size); + + return ret; } + /* get PCI Express Capability Structure register group size */ static int xen_pt_pcie_size_init(XenPCIPassthroughState *s, const XenPTRegGroupInfo *grp_reg, @@ -1876,6 +1937,14 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_reg_grp_size_init, .emu_regs = xen_pt_emu_reg_igd_opregion, }, + /* Vendor-specific Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VNDR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vendor_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_vendor, + }, { .grp_size = 0, }, From patchwork Tue Jun 20 17:24:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EAA87EB64DB for ; Tue, 20 Jun 2023 17:26:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552079.861997 (Exim 4.92) (envelope-from ) id 1qBf7b-00088y-7I; Tue, 20 Jun 2023 17:26:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552079.861997; Tue, 20 Jun 2023 17:26:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7b-00088i-2h; Tue, 20 Jun 2023 17:26:27 +0000 Received: by outflank-mailman (input) for mailman id 552079; Tue, 20 Jun 2023 17:26:25 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7Z-0005Q4-G5 for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:25 +0000 Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [2607:f8b0:4864:20::1136]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 947a6d3e-0f8f-11ee-b234-6b7b168915f2; Tue, 20 Jun 2023 19:26:24 +0200 (CEST) Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-5728df0a7d9so39840357b3.1 for ; Tue, 20 Jun 2023 10:26:24 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:23 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 947a6d3e-0f8f-11ee-b234-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281983; x=1689873983; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aUH0LrDNCluOY2g7IZF9uazabmu57c7aIkRqhWuUoW4=; b=iwRwRTaFQJfYCfdFgHs5hBrVVJyyLJCQKXLzDjc2uw1+9/aIo5cnUv+/xiPHbxjuh2 XFwA15wc+YNcwlq8WwcGhVyB1lz9VsRgzw4+XvIpPjE8ExRcuEvjNOBNE0RQ57VEEDNJ 64EPpQNaQEYzxkQ3c/wV95dN8zfhpngMP2nU9OZ/liVkq+4MPfn3nPH+QPU4ginQEKvn U8V7lDKGEZ9gP08J4zgUGhm2JE9iozhSVNTDMielRBCFN0rxFigIZOc94CQIhyYeuQPR hxGZm6uHroeQ+mGmjc3xBa9l0n2oo9PXJfgI65Ux8poxJl/JDEtJvcHnMCpECNXdCZx2 LiXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281983; x=1689873983; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aUH0LrDNCluOY2g7IZF9uazabmu57c7aIkRqhWuUoW4=; b=Pz6i8/RVg5DWZMZ4Phw45JRcjHkY74lq+SWcK4tOKSd4ozeYDuMda6AcDw+uptmlVF o+cR9hjN3vVEVDbNjKWVSdYkhxtziOTXHuljWpl9ss01HhRdXWfsBv8MItBvYeDpXg9+ 8bvKuveMEIAkA4LAePzI0GKOmwxDJvWBnKFRswJZtfab2RMLtnG8ARflGOF6nmxIPQH1 jIMlbc1Z8FlXfMraWN6Oxps8BW3U/FbFxUr+4sO3CwlsCGOPTIV5pvJW54P7EiCNFyAi LhsCeFMDSOgdIyDcv024eSgluuMUVnzJq4No+iiuPZSOx0HZwgsHsiAQWe4DqnWZgxfg odYQ== X-Gm-Message-State: AC+VfDz/kHTzUOEL8X6mVf1/jXWpbbskZGAoCkOVyJNZCjrsETqDFlo9 xr7GhYoSWMz1Wf04F0DqfRs= X-Google-Smtp-Source: ACHHUZ7H14tCE3HKsTlD/gtaiOjoMH3zAfVEsd3ThHpqpefM/jk8OkdgqNWhx9f/7UfbiTzox58o8A== X-Received: by 2002:a81:5215:0:b0:56c:e2c1:6695 with SMTP id g21-20020a815215000000b0056ce2c16695mr12352572ywb.50.1687281983595; Tue, 20 Jun 2023 10:26:23 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 14/23] xen/pt: add fixed-size PCIe Extended Capabilities descriptors Date: Tue, 20 Jun 2023 13:24:48 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 This adds description structures for all fixed-size PCIe Extended Capabilities. For every capability register group, only 2 registers are emulated currently: Capability ID (16 bit) and Next Capability Offset/Version (16 bit). Both needed to implement selective capability hiding. All other registers are passed through at the moment (unless they belong to a "hardwired" capability which is hidden) Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 183 ++++++++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 20b5561d25..69d8857c66 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -1669,6 +1669,37 @@ static XenPTRegInfo xen_pt_ext_cap_emu_reg_vendor[] = { .size = 0, }, }; + +/* Common reg static information table for all passthru-type + * PCIe Extended Capabilities. Only Extended Cap ID and + * Next pointer are handled (to support capability hiding). + */ +static XenPTRegInfo xen_pt_ext_cap_emu_reg_dummy[] = { + { + .offset = XEN_PCIE_CAP_ID, + .size = 2, + .init_val = 0x0000, + .ro_mask = 0xFFFF, + .emu_mask = 0xFFFF, + .init = xen_pt_ext_cap_capid_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .offset = XEN_PCIE_CAP_LIST_NEXT, + .size = 2, + .init_val = 0x0000, + .ro_mask = 0xFFFF, + .emu_mask = 0xFFFF, + .init = xen_pt_ext_cap_ptr_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, + { + .size = 0, + }, +}; + /**************************** * Capabilities */ @@ -1945,6 +1976,158 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_ext_cap_vendor_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_vendor, }, + /* Device Serial Number Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_DSN), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_DSN_SIZEOF, /*0x0C*/ + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Power Budgeting Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PWR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_PWR_SIZEOF, /*0x10*/ + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Root Complex Internal Link Control Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCILC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x0C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Root Complex Event Collector Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCEC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x08, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Root Complex Register Block Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCRB), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x14, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Configuration Access Correlation Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_CAC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x08, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Alternate Routing ID Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ARI), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_ARI_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Address Translation Services Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ATS), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_ATS_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Single Root I/O Virtualization Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_SRIOV), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_SRIOV_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Page Request Interface Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PRI), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_PRI_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Latency Tolerance Reporting Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_LTR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_LTR_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Secondary PCIe Capability Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_SECPCI), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x10, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Process Address Space ID Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PASID), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = PCI_EXT_CAP_PASID_SIZEOF, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* L1 PM Substates Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_L1SS), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x10, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Precision Time Measurement Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PTM), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x0C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* M-PCIe Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x20), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x1C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* LN Requester (LNR) Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x1C), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x08, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Function Readiness Status (FRS) Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x21), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x10, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Readiness Time Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(0x22), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0x0C, + .size_init = xen_pt_reg_grp_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, }, From patchwork Tue Jun 20 17:24:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45BD3EB64D7 for ; Tue, 20 Jun 2023 17:29:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552105.862027 (Exim 4.92) (envelope-from ) id 1qBfAW-0002kY-Cs; Tue, 20 Jun 2023 17:29:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552105.862027; Tue, 20 Jun 2023 17:29:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAW-0002kN-8r; Tue, 20 Jun 2023 17:29:28 +0000 Received: by outflank-mailman (input) for mailman id 552105; Tue, 20 Jun 2023 17:29:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7c-0004q1-Nw for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:28 +0000 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [2607:f8b0:4864:20::1133]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 95a531e3-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:26 +0200 (CEST) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-5707b429540so62318627b3.1 for ; Tue, 20 Jun 2023 10:26:26 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:25 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 95a531e3-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281985; x=1689873985; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/ZtR49Wj+BsujqlKtwlIvLXrUvLyGZ/7NrGzguEb9A0=; b=rxOo0tXcKfS1IYKEXuRJnHwkAChp/cBkhBvjJfVCWGOzcsHGlCZ6l3tNOl0XO8izkf 9Ua5For1WY8+nDPZPjKubfsBDBU/BzDNa6nmgGSAIshSBmPHoYrQPtjQl0y1Xheg+pE8 mM18gyjsk2UTfcWTS2tbtC/0QL/fxAXqEYd0vMsCcXesrSzdEw4xpnL1uHCir3pVNz7k fO0iYsRcdwiWniF2hmpqKPZiwJcE4Ct8XbLtn4XH9Vidk+E4PIB7BHmUoGSLo1GRK+Pl BVW1BZbGXKhPuDybfQm9FQoZV3aTssCTg85XrL/1IVvdxnV3aGq8D/bcziQJXFRksMgz 9cxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281985; x=1689873985; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/ZtR49Wj+BsujqlKtwlIvLXrUvLyGZ/7NrGzguEb9A0=; b=b939cd+6hQl2RJcO0q6JzHnlMJ9rVoZTCy2OGWkNqdtBzXu1hPNAp83n+PpB8SoMcG QXLxbONHWFuCxeFPgxKJmHcd8mCSvuoc2zqsKnZ0Nq0etd25AmNtQQAcCp53KerMlIdx Tg8TlDlfxTkJLFFxchoqLlyK9o3rWqqJdRWEMyRzzH4OQqHremepAHWaC8Wirm808lAL GF7qguPHZeo7TkFI/N5fS+fMM3ljiUTyNswcSxdYJrQiF7SE6kvq74t8SBfFWPbwSkSr V3JrEN17dyExF+tyUS9O57uAm9CgXWAQRhvMLv1PPNwQExYzKPLGJvtqrJ96+dy41MRk 56cA== X-Gm-Message-State: AC+VfDzdQArY9IWEOrPNvQUTf3DIEKYKn293NjY2Rr4Bbj/hM2yFTsqZ Wagx4jYzz0f4wDszkUT2iO0= X-Google-Smtp-Source: ACHHUZ7vnof2FXuOFDNyhaBtlcGPG7xrWzUFudElGDOK5WYidGkK2Dmf/59suWaFDE/d1Yxi9aEXng== X-Received: by 2002:a0d:cbcb:0:b0:56c:e5a3:3e09 with SMTP id n194-20020a0dcbcb000000b0056ce5a33e09mr21057302ywd.15.1687281985609; Tue, 20 Jun 2023 10:26:25 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 15/23] xen/pt: add AER PCIe Extended Capability descriptor and sizing Date: Tue, 20 Jun 2023 13:24:49 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 The patch provides Advanced Error Reporting PCIe Extended Capability description structure and corresponding capability sizing function. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 72 +++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 69d8857c66..9fd0531bc4 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -1861,6 +1861,70 @@ static int xen_pt_msix_size_init(XenPCIPassthroughState *s, } +/* get Advanced Error Reporting Extended Capability register group size */ +#define PCI_ERR_CAP_TLP_PREFIX_LOG (1U << 11) +#define PCI_DEVCAP2_END_END_TLP_PREFIX (1U << 21) +static int xen_pt_ext_cap_aer_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint8_t dev_type = get_pcie_device_type(s); + uint32_t aer_caps = 0; + uint32_t sz = 0; + int pcie_cap_pos; + uint32_t devcaps2; + int ret = 0; + + pcie_cap_pos = xen_host_pci_find_next_cap(&s->real_device, 0, + PCI_CAP_ID_EXP); + if (!pcie_cap_pos) { + XEN_PT_ERR(&s->dev, + "Cannot find a required PCI Express Capability\n"); + return -1; + } + + if (get_pcie_capability_version(s) > 1) { + ret = xen_host_pci_get_long(&s->real_device, + pcie_cap_pos + PCI_EXP_DEVCAP2, + &devcaps2); + if (ret) { + XEN_PT_ERR(&s->dev, "Error while reading Device " + "Capabilities 2 Register \n"); + return -1; + } + } + + if (devcaps2 & PCI_DEVCAP2_END_END_TLP_PREFIX) { + ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_ERR_CAP, + &aer_caps); + if (ret) { + XEN_PT_ERR(&s->dev, + "Error while reading AER Extended Capability\n"); + return -1; + } + + if (aer_caps & PCI_ERR_CAP_TLP_PREFIX_LOG) { + sz = 0x48; + } + } + + if (!sz) { + if (dev_type == PCI_EXP_TYPE_ROOT_PORT || + dev_type == PCI_EXP_TYPE_RC_EC) { + sz = 0x38; + } else { + sz = 0x2C; + } + } + + *size = sz; + + log_pcie_extended_cap(s, "AER", base_offset, *size); + return ret; +} + static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { /* Header Type0 reg group */ { @@ -2128,6 +2192,14 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_reg_grp_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_dummy, }, + /* Advanced Error Reporting Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ERR), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_aer_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, }, From patchwork Tue Jun 20 17:24:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70751EB64DB for ; Tue, 20 Jun 2023 17:29:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552119.862051 (Exim 4.92) (envelope-from ) id 1qBfAd-0003Ut-JG; Tue, 20 Jun 2023 17:29:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552119.862051; Tue, 20 Jun 2023 17:29:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAd-0003UM-CN; Tue, 20 Jun 2023 17:29:35 +0000 Received: by outflank-mailman (input) for mailman id 552119; Tue, 20 Jun 2023 17:29:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7e-0004q1-Nr for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:30 +0000 Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [2607:f8b0:4864:20::336]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 96d93a47-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:28 +0200 (CEST) Received: by mail-ot1-x336.google.com with SMTP id 46e09a7af769-6b2a875b1bfso4234793a34.0 for ; Tue, 20 Jun 2023 10:26:28 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:27 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 96d93a47-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281987; x=1689873987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wmwAG6Md0CC+XJeE9pl5kl+Q1W6gqDq1BcbgL+QyAho=; b=ojdkKj4dQV1QyhCxug8WoquBpw815OckKFl3hwP7HkoAoa2X4qJP17fjliqnsFTNMW ewK+iOGLtIsH/DmfNZ2Nmgz1yNLZvSJRxPH1raGXnZtXltujtz6NOZh2IAbOvEoUB5RU Gef26ixeDHQUyCt+MdLqn/Y8AtSbViaFTJQ+edeid3dtdVzA1HOG9Xu+ji71nnMHxcOs NiXNBpeYmlhi00nJEXWyOi5N165VFfwgDpzGJ1QRZ21tEhmepre01i/XdLlnPAs+F/T6 ZPlOPJHormUgTF3gvYnZ4CIoouBD37NlVFXoNd9ezrTSxY1KxlZnNCwo8EeL7E2dz4fK ciWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281987; x=1689873987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wmwAG6Md0CC+XJeE9pl5kl+Q1W6gqDq1BcbgL+QyAho=; b=PlnmObrfxuhFshFD+YHfXf2NlELwRlzzbkobzUaWoNJLWRKOe90irOeYsVFtQOXvQJ qZN0TfiPXBgp0apGFdVQ/GQntHJGikW+37vRCeqXy29s0dT2l8BJ56l2GmrhXri6HUa6 7m7KAYaDQ5uNlsuszUXRKvg7+bLVgBuasGq1JblYhZgJONGm0Dd/S9NtkJC2AgDltrnA UwQQ2lu1P4bJzHA8hdf1HmjPsMXYHdOqsdHbzlbLh2UKlnF55zxAeaRZTJMSaLWda61G /91pb4XreEyIEW2dZ0kno9+0EVSM4iYKX6FUMcHOhjKGft1Vi4hYuPJk01S92Fhfa13F 9eWw== X-Gm-Message-State: AC+VfDxRc48slUsYH9YF8Yu8meSfZxcNNidPu/vUDiPVDYZ3Amo+j2r/ KF+XyRqJrlq/8fpVnTdyCYJtyKcd1db3ow== X-Google-Smtp-Source: ACHHUZ7OZCfrEKRiy1peK/z+7dOVkohj0K6JF3B4EUqAq6V/RhqJ229VE/MynC5WaUeVXsnaYNyELQ== X-Received: by 2002:a05:6359:a12:b0:130:5951:b268 with SMTP id el18-20020a0563590a1200b001305951b268mr5880443rwb.31.1687281987530; Tue, 20 Jun 2023 10:26:27 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 16/23] xen/pt: add descriptors and size calculation for RCLD/ACS/PMUX/DPA/MCAST/TPH/DPC PCIe Extended Capabilities Date: Tue, 20 Jun 2023 13:24:50 -0400 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Add few more PCIe Extended Capabilities entries to the xen_pt_emu_reg_grps[] array along with their corresponding *_size_init() functions. All these capabilities have non-fixed size but their size calculation is very simple, hence adding them in a single batch. For every capability register group, only 2 registers are emulated currently: Capability ID (16 bit) and Next Capability Offset/Version (16 bit). Both needed to implement the selective capability hiding. All other registers are passed through at the moment (unless they belong to a capability marked as "hardwired" which is hidden) Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 224 ++++++++++++++++++++++++++++++++++++ 1 file changed, 224 insertions(+) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 9fd0531bc4..1fba0b9d6c 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -1925,6 +1925,174 @@ static int xen_pt_ext_cap_aer_size_init(XenPCIPassthroughState *s, return ret; } +/* get Root Complex Link Declaration Extended Capability register group size */ +#define RCLD_GET_NUM_ENTRIES(x) (((x) >> 8) & 0xFF) +static int xen_pt_ext_cap_rcld_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint32_t elem_self_descr = 0; + + int ret = xen_host_pci_get_long(&s->real_device, + base_offset + 4, + &elem_self_descr); + + *size = 0x10 + RCLD_GET_NUM_ENTRIES(elem_self_descr) * 0x10; + + log_pcie_extended_cap(s, "Root Complex Link Declaration", + base_offset, *size); + return ret; +} + +/* get Access Control Services Extended Capability register group size */ +#define ACS_VECTOR_SIZE_BITS(x) ((((x) >> 8) & 0xFF) ?: 256) +static int xen_pt_ext_cap_acs_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint16_t acs_caps = 0; + + int ret = xen_host_pci_get_word(&s->real_device, + base_offset + PCI_ACS_CAP, + &acs_caps); + + if (acs_caps & PCI_ACS_EC) { + uint32_t vector_sz = ACS_VECTOR_SIZE_BITS(acs_caps); + + *size = PCI_ACS_EGRESS_CTL_V + ((vector_sz + 7) & ~7) / 8; + } else { + *size = PCI_ACS_EGRESS_CTL_V; + } + + log_pcie_extended_cap(s, "ACS", base_offset, *size); + return ret; +} + +/* get Multicast Extended Capability register group size */ +static int xen_pt_ext_cap_multicast_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint8_t dev_type = get_pcie_device_type(s); + + switch (dev_type) { + case PCI_EXP_TYPE_ENDPOINT: + case PCI_EXP_TYPE_LEG_END: + case PCI_EXP_TYPE_RC_END: + case PCI_EXP_TYPE_RC_EC: + default: + *size = PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF; + break; + + case PCI_EXP_TYPE_ROOT_PORT: + case PCI_EXP_TYPE_UPSTREAM: + case PCI_EXP_TYPE_DOWNSTREAM: + *size = 0x30; + break; + } + + log_pcie_extended_cap(s, "Multicast", base_offset, *size); + return 0; +} + +/* get Dynamic Power Allocation Extended Capability register group size */ +static int xen_pt_ext_cap_dpa_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint32_t dpa_caps = 0; + uint32_t num_entries; + + int ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_DPA_CAP, + &dpa_caps); + + num_entries = (dpa_caps & PCI_DPA_CAP_SUBSTATE_MASK) + 1; + + *size = PCI_DPA_BASE_SIZEOF + num_entries /*byte-size registers*/; + + log_pcie_extended_cap(s, "Dynamic Power Allocation", base_offset, *size); + return ret; +} + +/* get TPH Requester Extended Capability register group size */ +static int xen_pt_ext_cap_tph_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint32_t tph_caps = 0; + uint32_t num_entries; + + int ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_TPH_CAP, + &tph_caps); + + switch(tph_caps & PCI_TPH_CAP_LOC_MASK) { + case PCI_TPH_LOC_CAP: + num_entries = (tph_caps & PCI_TPH_CAP_ST_MASK) >> PCI_TPH_CAP_ST_SHIFT; + num_entries++; + break; + + case PCI_TPH_LOC_NONE: + case PCI_TPH_LOC_MSIX: + default: + /* not in the capability */ + num_entries = 0; + } + + *size = PCI_TPH_BASE_SIZEOF + num_entries * 2; + + log_pcie_extended_cap(s, "TPH Requester", base_offset, *size); + return ret; +} + +/* get Downstream Port Containment Extended Capability register group size */ +static int xen_pt_ext_cap_dpc_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint16_t dpc_caps = 0; + + int ret = xen_host_pci_get_word(&s->real_device, + base_offset + PCI_EXP_DPC_CAP, + &dpc_caps); + + if (dpc_caps & PCI_EXP_DPC_CAP_RP_EXT) { + *size = 0x20 + ((dpc_caps & PCI_EXP_DPC_RP_PIO_LOG_SIZE) >> 8) * 4; + } else { + *size = 0xC; + } + + log_pcie_extended_cap(s, "Downstream Port Containment", + base_offset, *size); + return ret; +} + +/* get Protocol Multiplexing Extended Capability register group size */ +#define PMUX_GET_NUM_ENTRIES(x) ((x) & 0x3F) +static int xen_pt_ext_cap_pmux_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint32_t pmux_caps = 0; + + int ret = xen_host_pci_get_long(&s->real_device, + base_offset + 4, + &pmux_caps); + + *size = 0x10 + PMUX_GET_NUM_ENTRIES(pmux_caps) * 4; + + log_pcie_extended_cap(s, "PMUX", base_offset, *size); + return ret; +} + static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { /* Header Type0 reg group */ { @@ -2200,6 +2368,62 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_ext_cap_aer_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_dummy, }, + /* Root Complex Link Declaration Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_RCLD), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_rcld_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Access Control Services Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_ACS), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_acs_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Multicast Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_MCAST), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_multicast_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Dynamic Power Allocation Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_DPA), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_dpa_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* TPH Requester Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_TPH), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_tph_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Protocol Multiplexing Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_PMUX), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_pmux_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Downstream Port Containment Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_DPC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_dpc_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, }, From patchwork Tue Jun 20 17:24:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8297DEB64D8 for ; Tue, 20 Jun 2023 17:29:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552118.862047 (Exim 4.92) (envelope-from ) id 1qBfAd-0003Qo-5w; Tue, 20 Jun 2023 17:29:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552118.862047; Tue, 20 Jun 2023 17:29:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAd-0003QZ-1v; Tue, 20 Jun 2023 17:29:35 +0000 Received: by outflank-mailman (input) for mailman id 552118; Tue, 20 Jun 2023 17:29:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7g-0004q1-L1 for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:32 +0000 Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [2607:f8b0:4864:20::1129]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 980c656c-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:30 +0200 (CEST) Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-5701eaf0d04so52045437b3.2 for ; Tue, 20 Jun 2023 10:26:30 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:28 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 980c656c-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281989; x=1689873989; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A524XPsKBXV8yIUUkONAn98uAFCiKVTYyMvCRnHwi2c=; b=iQ/RkOUENSnolhvtv9AkWVvC4+UoqgOn8Qysgb9xM3IofX21Tzz6Tk0oEZuJOA/e/3 4b4tNVmMbG7HVhgB2DPb/TeYAHtGMylmnD3PJGacsE6/UpjcGwS+jXQ9UYlvdfzS+y8W +Vu4tu3igfZ700VaRbou+aFFluHvMdiysHJurJ1+8YlyD5mkRIgzLOMNBKfFmqyq/QtV e4/X4zvaJw2dDt7kr+97WsS0/n2+F2Co2wdODsrDd0BtitsYUXbJ6XJ4MLgdQtK4MhsU JFdQFOyU4xBA1kNZjcfwHK2JxiV8LkG0adb9oUxGBPDx1/yQrCCu2Mhlu2/Rgp5Tt9wY uRIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281989; x=1689873989; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A524XPsKBXV8yIUUkONAn98uAFCiKVTYyMvCRnHwi2c=; b=JDWdEL5p0O4RX9n3yG9K6mNndOj+47mLRXuqEAD1eyAVvylFJCFAu5kfO/q5XMojq+ xKfyonHOwQoAy2+KDyV+jhBkZE8U4uPIeX1Ky3nbdnutaROf0tG6MeEEQdTK2ik5bzNF qr8YQqFZdEDmxjEEXdQLtza4lu6tFizzeWNtib445GBKCC2eNfvC/HpUz64o9KMu7hBZ hog9kjRPx0ErL3BjRoKxINXJksbSik6YkLKDKW6j1mQN93BEbLIUWHdLrFEwkCPjDU1z wVLtdhW7FsMakWabdOi3GH6vsEjkIuSfaQcmFcy6Bu81hIRbcYJzTXKiHbYIgi/MfCJh gCjA== X-Gm-Message-State: AC+VfDzEpb+kOp5Ji1nNFiL5kLmYB+1FWl+lyZLHaOgNeuU/wEyDoAMQ 6EBi7+VId4OllOuNkCHdEck= X-Google-Smtp-Source: ACHHUZ6S4m8fu+H0OgrlYLgV+H0vy/atkpSHK7ZVonYsWljbCF1QmDliwrTxL0EO6NGWYcflU0wPUw== X-Received: by 2002:a81:6684:0:b0:568:f2c:ee40 with SMTP id a126-20020a816684000000b005680f2cee40mr12259954ywc.3.1687281989562; Tue, 20 Jun 2023 10:26:29 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 17/23] xen/pt: add Resizable BAR PCIe Extended Capability descriptor and sizing Date: Tue, 20 Jun 2023 13:24:51 -0400 Message-Id: <30ddc5858c9a309ed4ab812ab93b3e930b7bfd68.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Unlike other PCIe Extended Capabilities, we currently cannot allow attempts to use Resizable BAR Capability. Without specifically handling BAR resizing we're likely end up with corrupted MMIO hole layout if guest OS will attempt to use this feature. Actually, recent Windows versions started to understand and use the Resizable BAR Capability (see [1]). For now, we need to hide the Resizable BAR Capability from guest OS until BAR resizing emulation support will be implemented in Xen. This support is a pretty much mandatory todo-feature as the effect of writing to Resizable BAR control registers can be considered similar to reprogramming normal BAR registers -- i.e. this needs to be handled explicitly, resulting in corresponding MMIO BAR range(s) remapping. Until then, we mark the Resizable BAR Capability as XEN_PT_GRP_TYPE_HARDWIRED. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 1fba0b9d6c..c5157ee3ee 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -2093,6 +2093,27 @@ static int xen_pt_ext_cap_pmux_size_init(XenPCIPassthroughState *s, return ret; } +/* get Resizable BAR Extended Capability register group size */ +static int xen_pt_ext_cap_rebar_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint32_t rebar_ctl = 0; + uint32_t num_entries; + + int ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_REBAR_CTRL, + &rebar_ctl); + num_entries = + (rebar_ctl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + *size = num_entries*8 + 4; + + log_pcie_extended_cap(s, "Resizable BAR", base_offset, *size); + return ret; +} + static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { /* Header Type0 reg group */ { @@ -2424,6 +2445,13 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .size_init = xen_pt_ext_cap_dpc_size_init, .emu_regs = xen_pt_ext_cap_emu_reg_dummy, }, + /* Resizable BAR Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_REBAR), + .grp_type = XEN_PT_GRP_TYPE_HARDWIRED, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_rebar_size_init, + }, { .grp_size = 0, }, From patchwork Tue Jun 20 17:24:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C74C2EB64D7 for ; Tue, 20 Jun 2023 17:29:55 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552121.862067 (Exim 4.92) (envelope-from ) id 1qBfAf-00045n-Rq; Tue, 20 Jun 2023 17:29:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552121.862067; Tue, 20 Jun 2023 17:29:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAf-00044W-OE; Tue, 20 Jun 2023 17:29:37 +0000 Received: by outflank-mailman (input) for mailman id 552121; Tue, 20 Jun 2023 17:29:35 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7j-0004q1-5N for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:35 +0000 Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [2607:f8b0:4864:20::112d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9927badf-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:32 +0200 (CEST) Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-561b7729a12so88482187b3.1 for ; Tue, 20 Jun 2023 10:26:32 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:31 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9927badf-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281991; x=1689873991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dtxBS4t7Cp6zwlgH6yFPLtcHEBS+l3bJvetWgivlgAc=; b=DXr1jJKf7Pnlc0opub8mlP8txDB++zKhqESKRyN5fEU/XmhzGk0Ki4Pgl0faNoVtIp 9MVCQzOaaYq9Hafn00UmZw0xdLJv/Y+B/4KkvWPiSqIWM0G6n20EJX52qbK2hpf6aX8m VDfU8rjLroEhv8iPcEaV3/cfwx+epSNs1PBG8UUWPukiReaCYRd3fhbHAr++IpPrRHEx 7XjvEz+gRa4OveWFoJTqEg3KncaTM1bX0Za+tf05gn17TvkPoZ9JxORSmhvGEn2O+VCL v7HKZOb9XKZJVXJoLLIOF1wZH8LH4n4ZvvfDWHj/tMJ9JW/g1OCOcEbQMQTxeS5XPML7 TsUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281991; x=1689873991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dtxBS4t7Cp6zwlgH6yFPLtcHEBS+l3bJvetWgivlgAc=; b=P7Ou51Q+Yt6yN+xL2X9AIV+UlG9JxxI86fAexHH7N4ZdM/fdgzHjfkDRLPOlS4U5sC a3t/nACXbj+CDaQgPVynp6uuJEgT//FA09ehIERR0MgVJfvzKhbNP96DBd7b3wJ9UdAj cgy1BNg/4ju8UhoXVvw3WmcF0puQiORsZkLmoPUNo+ke2kwatupt1J3qt5LJom+VnW2b YX8wjxxUjkbIPuaP3dNfnOz8h1jadsdAUsbl3fRi5DNFKRv/4iBIGadMMCDAHe/lXpN0 TDENpV9M2PKFh5AicjBZCAqQ/a9DcSKxvYjZHTiR2JLiMZj57reyqyf9LTglIuedRp+3 7llw== X-Gm-Message-State: AC+VfDy3FZ3wVSfgO9CYJHoR+0QFOmLKNYyNTqyWZo7JbCiS1rmXWXF9 3qrux4gFnHMIZX+RY2E1M7A= X-Google-Smtp-Source: ACHHUZ5bilXSKSkByz0RaH7wiwsEk1kQjzttcfCjKeTvwtAuolFbt15zpQmnjABm1+bdn7RNwdCaMQ== X-Received: by 2002:a0d:ccc3:0:b0:56c:e480:2b2b with SMTP id o186-20020a0dccc3000000b0056ce4802b2bmr21843674ywd.12.1687281991477; Tue, 20 Jun 2023 10:26:31 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 18/23] xen/pt: add VC/VC9/MFVC PCIe Extended Capabilities descriptors and sizing Date: Tue, 20 Jun 2023 13:24:52 -0400 Message-Id: <3cc4807a182443d70c560090828f780719a531cf.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Virtual Channel/MFVC capabilities are relatively useless for emulation (passing through accesses to them should be enough in most cases) yet they have hardest format of all PCIe Extended Capabilities, mostly because VC capability format allows the sparse config space layout with gaps between the parts which make up the VC capability. We have the main capability body followed by variable number of entries where each entry may additionally reference the arbitration table outside main capability body. There are no constrains on these arbitration table offsets -- in theory, they may reside outside the VC capability range anywhere in PCIe extended config space. Also, every arbitration table size is not fixed - it depends on current VC/Port Arbitration Select field value. To simplify things, this patch assume that changing VC/Port Arbitration Select value (i.e. resizing arbitration tables) do not cause arbitration table offsets to change. Normally the device must place arbitration tables considering their maximum size, not current one. Maximum arbitration table size depends on VC/Port Arbitration Capability bitmask -- this is what actually used to calculate the arbitration table size. Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 191 ++++++++++++++++++++++++++++++++++++ 1 file changed, 191 insertions(+) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index c5157ee3ee..4e14adf2b2 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -2114,6 +2114,173 @@ static int xen_pt_ext_cap_rebar_size_init(XenPCIPassthroughState *s, return ret; } +/* get VC/VC9/MFVC Extended Capability register group size */ +static uint32_t get_arb_table_len_max(XenPCIPassthroughState *s, + uint32_t max_bit_supported, + uint32_t arb_cap) +{ + int n_bit; + uint32_t table_max_size = 0; + + if (!arb_cap) { + return 0; + } + + for (n_bit = 7; n_bit >= 0 && !(arb_cap & (1 << n_bit)); n_bit--); + + if (n_bit > max_bit_supported) { + XEN_PT_ERR(&s->dev, "Warning: encountered unknown VC arbitration " + "capability supported: 0x%02x\n", (uint8_t) arb_cap); + } + + switch (n_bit) { + case 0: break; + case 1: return 32; + case 2: return 64; + case 3: /*128 too*/ + case 4: return 128; + default: + table_max_size = 8 << n_bit; + } + + return table_max_size; +} + +#define GET_ARB_TABLE_OFFSET(x) (((x) >> 24) * 0x10) +#define GET_VC_ARB_CAPABILITY(x) ((x) & 0xFF) +#define ARB_TABLE_ENTRY_SIZE_BITS(x) (1 << (((x) & PCI_VC_CAP1_ARB_SIZE)\ + >> 10)) +static int xen_pt_ext_cap_vchan_size_init(XenPCIPassthroughState *s, + const XenPTRegGroupInfo *grp_reg, + uint32_t base_offset, + uint32_t *size) +{ + uint32_t header; + uint32_t vc_cap_max_size = PCIE_CONFIG_SPACE_SIZE - base_offset; + uint32_t next_ptr; + uint32_t arb_table_start_max = 0, arb_table_end_max = 0; + uint32_t port_vc_cap1, port_vc_cap2, vc_rsrc_cap; + uint32_t ext_vc_count = 0; + uint32_t arb_table_entry_size; /* in bits */ + const char *cap_name; + int ret; + int i; + + ret = xen_host_pci_get_long(&s->real_device, base_offset, &header); + if (ret) { + goto err_read; + } + + next_ptr = PCI_EXT_CAP_NEXT(header); + + switch (PCI_EXT_CAP_ID(header)) { + case PCI_EXT_CAP_ID_VC: + case PCI_EXT_CAP_ID_VC9: + cap_name = "Virtual Channel"; + break; + case PCI_EXT_CAP_ID_MFVC: + cap_name = "Multi-Function VC"; + break; + default: + XEN_PT_ERR(&s->dev, "Unknown VC Extended Capability ID " + "encountered: 0x%04x\n", PCI_EXT_CAP_ID(header)); + return -1; + } + + if (next_ptr && next_ptr > base_offset) { + vc_cap_max_size = next_ptr - base_offset; + } + + ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_VC_PORT_CAP1, + &port_vc_cap1); + if (ret) { + goto err_read; + } + + ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_VC_PORT_CAP2, + &port_vc_cap2); + if (ret) { + goto err_read; + } + + ext_vc_count = port_vc_cap1 & PCI_VC_CAP1_EVCC; + + arb_table_start_max = GET_ARB_TABLE_OFFSET(port_vc_cap2); + + /* check arbitration table offset for validity */ + if (arb_table_start_max >= vc_cap_max_size) { + XEN_PT_ERR(&s->dev, "Warning: VC arbitration table offset points " + "outside the expected range: %#04x\n", + (uint16_t) arb_table_start_max); + /* skip this arbitration table */ + arb_table_start_max = 0; + } + + if (arb_table_start_max) { + uint32_t vc_arb_cap = GET_VC_ARB_CAPABILITY(port_vc_cap2); + uint32_t num_phases = get_arb_table_len_max(s, 3, vc_arb_cap); + uint32_t arb_tbl_sz = QEMU_ALIGN_UP(num_phases * 4, 32) / 8; + + arb_table_end_max = base_offset + arb_table_start_max + arb_tbl_sz; + } + + /* get Function/Port Arbitration Table Entry size */ + arb_table_entry_size = ARB_TABLE_ENTRY_SIZE_BITS(port_vc_cap1); + + /* process all VC Resource entries */ + for (i = 0; i < ext_vc_count; i++) { + uint32_t arb_table_offset; + + /* read VC Resource Capability */ + ret = xen_host_pci_get_long(&s->real_device, + base_offset + PCI_VC_RES_CAP + i * PCI_CAP_VC_PER_VC_SIZEOF, + &vc_rsrc_cap); + if (ret) { + goto err_read; + } + + arb_table_offset = GET_ARB_TABLE_OFFSET(vc_rsrc_cap); + + if (arb_table_offset > arb_table_start_max) { + /* check arbitration table offset for validity */ + if (arb_table_offset >= vc_cap_max_size) { + XEN_PT_ERR(&s->dev, "Warning: Port/Function arbitration table " + "offset points outside the expected range: %#04x\n", + (uint16_t) arb_table_offset); + /* skip this arbitration table */ + arb_table_offset = 0; + } else { + arb_table_start_max = arb_table_offset; + } + + if (arb_table_offset) { + uint32_t vc_arb_cap = GET_VC_ARB_CAPABILITY(vc_rsrc_cap); + uint32_t num_phases = get_arb_table_len_max(s, 5, vc_arb_cap); + uint32_t arb_tbl_sz = + QEMU_ALIGN_UP(num_phases * arb_table_entry_size, 32) / 8; + + arb_table_end_max = base_offset + arb_table_offset + arb_tbl_sz; + } + } + } + + if (arb_table_end_max) { + *size = arb_table_end_max - base_offset; + } else { + *size = PCI_CAP_VC_BASE_SIZEOF + + ext_vc_count * PCI_CAP_VC_PER_VC_SIZEOF; + } + + log_pcie_extended_cap(s, cap_name, base_offset, *size); + return 0; + +err_read: + XEN_PT_ERR(&s->dev, "Error while reading VC Extended Capability\n"); + return ret; +} + static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { /* Header Type0 reg group */ { @@ -2452,6 +2619,30 @@ static const XenPTRegGroupInfo xen_pt_emu_reg_grps[] = { .grp_size = 0xFF, .size_init = xen_pt_ext_cap_rebar_size_init, }, + /* Virtual Channel Extended Capability reg group (2) */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vchan_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Virtual Channel Extended Capability reg group (9) */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_VC9), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vchan_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, + /* Multi-Function Virtual Channel Extended Capability reg group */ + { + .grp_id = PCIE_EXT_CAP_ID(PCI_EXT_CAP_ID_MFVC), + .grp_type = XEN_PT_GRP_TYPE_EMU, + .grp_size = 0xFF, + .size_init = xen_pt_ext_cap_vchan_size_init, + .emu_regs = xen_pt_ext_cap_emu_reg_dummy, + }, { .grp_size = 0, }, From patchwork Tue Jun 20 17:24:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6A7D5EB64D7 for ; Tue, 20 Jun 2023 17:29:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552103.862013 (Exim 4.92) (envelope-from ) id 1qBfAT-0002Gu-6p; Tue, 20 Jun 2023 17:29:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552103.862013; Tue, 20 Jun 2023 17:29:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAT-0002Fu-1N; Tue, 20 Jun 2023 17:29:25 +0000 Received: by outflank-mailman (input) for mailman id 552103; Tue, 20 Jun 2023 17:29:24 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7k-0004q1-9f for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:36 +0000 Received: from mail-vk1-xa36.google.com (mail-vk1-xa36.google.com [2607:f8b0:4864:20::a36]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9a4881d5-0f8f-11ee-8611-37d641c3527e; Tue, 20 Jun 2023 19:26:34 +0200 (CEST) Received: by mail-vk1-xa36.google.com with SMTP id 71dfb90a1353d-4715758343aso1420761e0c.1 for ; Tue, 20 Jun 2023 10:26:34 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:32 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9a4881d5-0f8f-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281993; x=1689873993; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OHujXKoC6+pvKbdfEDtYmXpzoxjEyy+YGxjQhJLOU0Y=; b=GM+CG3fWRPKOG4chiQRmboIRPep8REP/M8D+0GeYUmtTHRWf/d2klrmsavq5Cp+SFY mXAFc+c517Cql+9sL95wm3ozozHbV5djHykxm4q5KD0NDvXGtC1ci5gDMQklVHI43Hgx t+oy7zCw3J2vA2BJ3qzL6GWWzcGm2cLxOLwt/v1wZy1LBC7ya0d8jYzgjvAluAc/XA02 apVxuVVCddQCVq++rywBUAd7/un/VWrrvZUlvF7NqPObTXgXUSwO+3yOHWRyjSzYNT7Q 7TdzzijFwWvqNgZMM2RnbbVcIe3wCN87Fx5KMzW6GW2mXeAIuTNl0fB/g5pBSAYVpVj1 /TBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281993; x=1689873993; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OHujXKoC6+pvKbdfEDtYmXpzoxjEyy+YGxjQhJLOU0Y=; b=QcStLAMYwZnx7kmRBXauEH2NtL2pSbqN86nZTCM3OdH/UIpxPZEe2MikbvxGGWOEfo SUHByn6FzEea+jRZ4tlug6+cDW5h5MCnXoL7Sz2zTxB3pDHxhRbXdtWVeGmR84TKwS2M ycFK8SU0pQBHbWGDJo6smXZ4kgrpXCFaP8pvO87W1YQqivMD1QkFMYmzABsJMzcoZq5c wPN/1X5Y6WCOlxdE0dngAOixUxEDtBbaehrxfwYxlUvqhL9/9W7XOtu6eoLxSDQaLz5W 7CardVHvWz9CdWYl9d2J8/1EQRwu3s6GwcvgQ01PMTTMYqhsOMgCeRX+p/KtHbtelILb bzMQ== X-Gm-Message-State: AC+VfDyRfPvbvhleWetcjyqsCQk8jn5zFlnsGMs4KbjgiecUxTZ8Xyqh l//aE0A1k6Lb4D7zGygNxdE= X-Google-Smtp-Source: ACHHUZ5QiJc1oXfSp4p9cZik4NYdz6sPHMrvS71pbTKO/RMTg1uRBBt5TM62D7CH5DnCPiKqY10dDg== X-Received: by 2002:a1f:5c93:0:b0:464:7682:3506 with SMTP id q141-20020a1f5c93000000b0046476823506mr4435739vkb.12.1687281993305; Tue, 20 Jun 2023 10:26:33 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 19/23] xen/pt: Fake capability id Date: Tue, 20 Jun 2023 13:24:53 -0400 Message-Id: <65b91216c1337b560548f39978d41c988b498517.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Some PCIe capabilities needed to be faked for the xen implementation to work. This is the situation when we were asked to hide (aka "hardwire to 0") some PCIe ext capability, but it was located at offset 0x100 in PCIe config space. In this case we can't simply exclude it from the linked list of capabilities (as it is the first entry in the list), so we must fake its Capability ID in PCIe Extended Capability header, leaving the Next Ptr field intact while returning zeroes on attempts to read capability body (writes are ignored). Signed-off-by: Alexey Gerasimenko Signed-off-by: Joel Upham --- hw/xen/xen_pt_config_init.c | 72 ++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c index 4e14adf2b2..41b43b9445 100644 --- a/hw/xen/xen_pt_config_init.c +++ b/hw/xen/xen_pt_config_init.c @@ -16,6 +16,7 @@ #include "qapi/error.h" #include "qemu/timer.h" #include "xen_pt.h" +#include "xen-host-pci-device.h" #include "hw/xen/xen-legacy-backend.h" #define XEN_PT_MERGE_VALUE(value, data, val_mask) \ @@ -31,6 +32,10 @@ static int xen_pt_ext_cap_ptr_reg_init(XenPCIPassthroughState *s, XenPTRegInfo *reg, uint32_t real_offset, uint32_t *data); +static int xen_pt_ext_cap_capid_reg_init(XenPCIPassthroughState *s, + XenPTRegInfo *reg, + uint32_t real_offset, + uint32_t *data); /* helper */ @@ -995,6 +1000,17 @@ static XenPTRegInfo xen_pt_emu_reg_pcie[] = { .u.b.read = xen_pt_byte_reg_read, .u.b.write = xen_pt_byte_reg_write, }, + /* PCI Express Capabilities Register */ + { + .offset = PCI_EXP_FLAGS, + .size = 2, + .init_val = 0x0000, + .ro_mask = 0xFFFF, + .emu_mask = 0xFFFF, + .init = xen_pt_pcie_capabilities_reg_init, + .u.w.read = xen_pt_word_reg_read, + .u.w.write = xen_pt_word_reg_write, + }, /* Device Capabilities reg */ { .offset = PCI_EXP_DEVCAP, @@ -1633,6 +1649,54 @@ static XenPTRegInfo xen_pt_emu_reg_igd_opregion[] = { }, }; +/**************************** + * Emulated registers for + * PCIe Extended Capabilities + */ + +static uint16_t fake_cap_id = XEN_PCIE_FAKE_CAP_ID_BASE; + +/* PCIe Extended Capability ID reg */ +static int xen_pt_ext_cap_capid_reg_init(XenPCIPassthroughState *s, + XenPTRegInfo *reg, + uint32_t real_offset, + uint32_t *data) +{ + uint16_t reg_field; + int rc; + XenPTRegGroup *reg_grp_entry = NULL; + + /* use real device register's value as initial value */ + rc = xen_host_pci_get_word(&s->real_device, real_offset, ®_field); + if (rc) { + return rc; + } + + reg_grp_entry = xen_pt_find_reg_grp(s, real_offset); + + if (reg_grp_entry) { + if (reg_grp_entry->reg_grp->grp_type == XEN_PT_GRP_TYPE_HARDWIRED && + reg_grp_entry->base_offset == PCI_CONFIG_SPACE_SIZE) { + /* + * This is the situation when we were asked to hide (aka + * "hardwire to 0") some PCIe ext capability, but it was located + * at offset 0x100 in PCIe config space. In this case we can't + * simply exclude it from the linked list of capabilities + * (as it is the first entry in the list), so we must fake its + * Capability ID in PCIe Extended Capability header, leaving + * the Next Ptr field intact while returning zeroes on attempts + * to read capability body (writes are ignored). + */ + reg_field = fake_cap_id; + /* increment the value in order to have unique Capability IDs */ + fake_cap_id++; + } + } + + *data = reg_field; + return 0; +} + /* Vendor-specific Ext Capability Structure reg static information table */ static XenPTRegInfo xen_pt_ext_cap_emu_reg_vendor[] = { { @@ -2938,7 +3002,13 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp) } } - if (xen_pt_emu_reg_grps[i].grp_type == XEN_PT_GRP_TYPE_EMU) { + if (xen_pt_emu_reg_grps[i].grp_type == XEN_PT_GRP_TYPE_EMU || + /* + * We need to always emulate the PCIe Extended Capability + * header for a hidden capability which starts at offset 0x100 + */ + (xen_pt_emu_reg_grps[i].grp_type == XEN_PT_GRP_TYPE_HARDWIRED && + reg_grp_offset == 0x100)) { if (xen_pt_emu_reg_grps[i].emu_regs) { int j = 0; XenPTRegInfo *regs = xen_pt_emu_reg_grps[i].emu_regs; From patchwork Tue Jun 20 17:24:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Upham X-Patchwork-Id: 13286259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 558BEEB64DB for ; Tue, 20 Jun 2023 17:29:41 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.552108.862037 (Exim 4.92) (envelope-from ) id 1qBfAa-00036L-Mq; Tue, 20 Jun 2023 17:29:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 552108.862037; Tue, 20 Jun 2023 17:29:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBfAa-00036A-Iz; Tue, 20 Jun 2023 17:29:32 +0000 Received: by outflank-mailman (input) for mailman id 552108; Tue, 20 Jun 2023 17:29:30 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qBf7k-0005Q4-62 for xen-devel@lists.xenproject.org; Tue, 20 Jun 2023 17:26:36 +0000 Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [2607:f8b0:4864:20::112a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9aff712f-0f8f-11ee-b234-6b7b168915f2; Tue, 20 Jun 2023 19:26:35 +0200 (CEST) Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-5702415be17so42877067b3.2 for ; Tue, 20 Jun 2023 10:26:35 -0700 (PDT) Received: from joel-Precision-7920-Tower.. ([24.53.71.1]) by smtp.gmail.com with ESMTPSA id e65-20020a0dc244000000b0056cffe97a11sm604604ywd.13.2023.06.20.10.26.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 10:26:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9aff712f-0f8f-11ee-b234-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687281994; x=1689873994; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UQQT2DevBFB3oDkqTww+bCq8Z+NcubvcMEbZ9kPn1/I=; b=fstv2AOOVoUWAQIuOJ3m6gjiU4cTxA4w47YkcRP1Rz7FBW/kBHuRMpLSFwMaTEe5b1 BlytFKX3BaEwTZKAkeW11PgTa8oze9Czt/+yknEl/OEBfvwDwXn7SW3PG/2w8PqeoPvm av8TBcXKkS0ciL2D5leUNWfuF+rTprm3u/HkWbPk/mwfGv0uVvN2t3ZSUpWiiVE5OWrW icnEMAZo2HF7BoErM6qix14wzc9gUbIHbkcx4ZwzBkJIr1KsPb4CDkdR5gy+OyiVn6w8 Gh4q7qXxcQBsSJwTex+JKFslShnCOfxKn4FB052N1hJQvrzjk8BGqRy3FVkARemL4CKt I4yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687281994; x=1689873994; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UQQT2DevBFB3oDkqTww+bCq8Z+NcubvcMEbZ9kPn1/I=; b=CaaQBfduo8Mdv4SF19E/xLVyqjOukwcQd7Hl5riK1Hy/O8obsn81d2MrfGi+owudN1 bWcOJ/GBSF5N4st0kGUaRKOI6tfa5ZJOov5o4SIC/bMWhxmlb7Jm/E9oyBfNAA08UMJL O5951wN/1592u2ZGVefm0Er52Wkg8IMxO/9ezKtc+Qit7jvAysl2EaiCWyKyVy3NyHoq fd0H/Su8x+cRbPAPP5NPFLXOhdEkjD6doezhfntu/gHuO3cGNmdyTPPYhNCSX15f2Tfv 3oIEBZDe08DDhzzNIm8dXsamOyNBNC7ix0qpKpvw5pVm+VcArl6qv8vYiw8wbH9YC+qe Ri2A== X-Gm-Message-State: AC+VfDyFPjIlbW20NCdBJoMUKLr3eguiPLS3aYHE1UoqnCoykIt/OD8k /BdnJwpMposu53SJnXu2Me0= X-Google-Smtp-Source: ACHHUZ5TvogazAaiBwei1eL5eky/5rmlzL7gOMXYPR2q+pv034XN8iCHLrs6IQjYOJgREMAVTq8NZw== X-Received: by 2002:a0d:ea01:0:b0:56c:e5a3:3e1b with SMTP id t1-20020a0dea01000000b0056ce5a33e1bmr12551422ywe.22.1687281994508; Tue, 20 Jun 2023 10:26:34 -0700 (PDT) From: Joel Upham To: qemu-devel@nongnu.org Cc: Joel Upham , Stefano Stabellini , Anthony Perard , Paul Durrant , "Michael S. Tsirkin" , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v1 20/23] xen platform: unplug ahci object Date: Tue, 20 Jun 2023 13:24:54 -0400 Message-Id: <9b8183903cbf20db4e2f0dafda9e0ed271a86a8e.1687278381.git.jupham125@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 This will unplug the ahci device when the Xen driver calls for an unplug. This has been tested to work in linux and Windows guests. When q35 is detected, we will remove the ahci controller with the hard disks. In the libxl config, cdrom devices are put on a seperate ahci controller. This allows for 6 cdrom devices to be added, and 6 qemu hard disks. Signed-off-by: Joel Upham Signed-off-by: David Woodhouse --- hw/i386/xen/xen_platform.c | 19 ++++++++++++++++++- hw/pci/pci.c | 17 +++++++++++++++++ include/hw/pci/pci.h | 3 +++ 3 files changed, 38 insertions(+), 1 deletion(-) diff --git a/hw/i386/xen/xen_platform.c b/hw/i386/xen/xen_platform.c index 57f1d742c1..0375337222 100644 --- a/hw/i386/xen/xen_platform.c +++ b/hw/i386/xen/xen_platform.c @@ -34,6 +34,7 @@ #include "sysemu/block-backend.h" #include "qemu/error-report.h" #include "qemu/module.h" +#include "include/hw/i386/pc.h" #include "qom/object.h" #ifdef CONFIG_XEN @@ -223,6 +224,12 @@ static void unplug_disks(PCIBus *b, PCIDevice *d, void *opaque) if (flags & UNPLUG_NVME_DISKS) { object_unparent(OBJECT(d)); } + break; + + case PCI_CLASS_STORAGE_SATA: + if (!aux) { + object_unparent(OBJECT(d)); + } default: break; @@ -231,7 +238,17 @@ static void unplug_disks(PCIBus *b, PCIDevice *d, void *opaque) static void pci_unplug_disks(PCIBus *bus, uint32_t flags) { - pci_for_each_device(bus, 0, unplug_disks, &flags); + PCIBus *q35 = find_q35(); + if (q35) { + /* When q35 is detected, we will remove the ahci controller + * with the hard disks. In the libxl config, cdrom devices + * are put on a seperate ahci controller. This allows for 6 cdrom + * devices to be added, and 6 qemu hard disks. + */ + pci_function_for_one_bus(bus, unplug_disks, &flags); + } else { + pci_for_each_device(bus, 0, unplug_disks, &flags); + } } static void platform_fixed_ioport_writew(void *opaque, uint32_t addr, uint32_t val) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 1cc7c89036..8eac3d751a 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -1815,6 +1815,23 @@ void pci_for_each_device_reverse(PCIBus *bus, int bus_num, } } +void pci_function_for_one_bus(PCIBus *bus, + void (*fn)(PCIBus *b, PCIDevice *d, void *opaque), + void *opaque) +{ + bus = pci_find_bus_nr(bus, 0); + + if (bus) { + PCIDevice *d; + + d = bus->devices[PCI_DEVFN(4,0)]; + if (d) { + fn(bus, d, opaque); + return; + } + } +} + void pci_for_each_device_under_bus(PCIBus *bus, pci_bus_dev_fn fn, void *opaque) { diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index e6d0574a29..c53e21082a 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -343,6 +343,9 @@ void pci_for_each_device_under_bus(PCIBus *bus, void pci_for_each_device_under_bus_reverse(PCIBus *bus, pci_bus_dev_fn fn, void *opaque); +void pci_function_for_one_bus(PCIBus *bus, + void (*fn)(PCIBus *bus, PCIDevice *d, void *opaque), + void *opaque); void pci_for_each_bus_depth_first(PCIBus *bus, pci_bus_ret_fn begin, pci_bus_fn end, void *parent_state); PCIDevice *pci_get_function_0(PCIDevice *pci_dev);