From patchwork Wed Mar 27 03:05:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13605514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02CCECD1283 for ; Wed, 27 Mar 2024 03:06:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpJcU-0001wz-GP; Tue, 26 Mar 2024 23:06:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpJcS-0001wS-Ex for qemu-devel@nongnu.org; Tue, 26 Mar 2024 23:06:28 -0400 Received: from fout3-smtp.messagingengine.com ([103.168.172.146]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpJcR-0004Wz-2L for qemu-devel@nongnu.org; Tue, 26 Mar 2024 23:06:28 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.nyi.internal (Postfix) with ESMTP id 73D7713800CD; Tue, 26 Mar 2024 23:06:26 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 26 Mar 2024 23:06:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1711508786; x=1711595186; bh=mH7qGKH3fb CivyC7O2Hbv2OPHkXyD7Im69azzuDMjbE=; b=ZOf9lyc6/HjY/wuJnLbLX2/GwE qfPsmb3WdAfan/JK7nNyrDQP5q6PnCDsFHyaLmzJ3zbXLXxnEqMa6lVrfZ3elMjg fekxlhfHdxlT7X5ik8oOmBMG702yFK3Fq0aa7MBaFxSgqxAYzqgHaDm25+HwTxON LwccVvhRoHocKX622oE87/9bINdRO8OuFUn0FuK4zp9OoTiLqUO2HmJbV7m560vF AArUt37PO3QVH6Ucixr34asDAOLv/rqPHDttCsN2O3lWl7aJ83/0Tl0DISGlSQ/G vfooRoNQn76qUTWFQSk95LWXcrU13DxyITj/i0ab7kUmZr0pKkAfe9X50T/g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1711508786; x= 1711595186; bh=mH7qGKH3fbCivyC7O2Hbv2OPHkXyD7Im69azzuDMjbE=; b=n e3hzTjafh0pQimzREXXjk4LxZqrkNunjuhCA9UwpwTfpDFt6BYsC7eRNo3Pm2gVU T9Gl6zemCY5luFdxvXaehc862Ik/eEflHH0VF4trhtAmDu16iW8zaCVGI6LCUVy8 l0ViZKriTCHtgKx1b2JZP62y0jzuxadjgAdNViWY8g+CnInOYw3ZbCNZH9tSoUJI /b10Lx2SXEdFaC5JwU5VDXK5LcOIrTnmK8DI56mKTmjDcolnsQxl0QWoh+5N9sl5 YevkOOcJsBXcZ08zIaGuFvFhKMuxHYxmsauyyA5a8N/UfbzwDqMr/FfKG8zwgQRX TKvapsNXH5GuDIZkhuy6A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddugedgheeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Mar 2024 23:06:24 -0400 (EDT) From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Stefano Stabellini , Anthony Perard , Paul Durrant , Paolo Bonzini , Richard Henderson , Eduardo Habkost , "Michael S. Tsirkin" , Marcel Apfelbaum , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v3 1/2] hw/xen: detect when running inside stubdomain Date: Wed, 27 Mar 2024 04:05:14 +0100 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: none client-ip=103.168.172.146; envelope-from=marmarek@invisiblethingslab.com; helo=fout3-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Introduce global xen_is_stubdomain variable when qemu is running inside a stubdomain instead of dom0. This will be relevant for subsequent patches, as few things like accessing PCI config space need to be done differently. Signed-off-by: Marek Marczykowski-Górecki Reviewed-by: Anthony PERARD --- Changes in v3: - move to xen_hvm_init_pc() - coding style Changes in v2: - use sigend int for domid to match xenstore_read_int() types - fix code style --- hw/i386/xen/xen-hvm.c | 22 ++++++++++++++++++++++ include/hw/xen/xen.h | 1 + system/globals.c | 1 + 3 files changed, 24 insertions(+) diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c index 7745cb3..3291c17 100644 --- a/hw/i386/xen/xen-hvm.c +++ b/hw/i386/xen/xen-hvm.c @@ -583,6 +583,26 @@ static void xen_wakeup_notifier(Notifier *notifier, void *data) xc_set_hvm_param(xen_xc, xen_domid, HVM_PARAM_ACPI_S_STATE, 0); } +static bool xen_check_stubdomain(struct xs_handle *xsh) +{ + char *dm_path = g_strdup_printf( + "/local/domain/%d/image/device-model-domid", xen_domid); + char *val; + int32_t dm_domid; + bool is_stubdom = false; + + val = xs_read(xsh, 0, dm_path, NULL); + if (val) { + if (sscanf(val, "%d", &dm_domid) == 1) { + is_stubdom = dm_domid != 0; + } + free(val); + } + + g_free(dm_path); + return is_stubdom; +} + void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory) { MachineState *ms = MACHINE(pcms); @@ -595,6 +615,8 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory) xen_register_ioreq(state, max_cpus, &xen_memory_listener); + xen_is_stubdomain = xen_check_stubdomain(state->xenstore); + QLIST_INIT(&xen_physmap); xen_read_physmap(state); diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h index 37ecc91..ecb89ec 100644 --- a/include/hw/xen/xen.h +++ b/include/hw/xen/xen.h @@ -36,6 +36,7 @@ enum xen_mode { extern uint32_t xen_domid; extern enum xen_mode xen_mode; extern bool xen_domid_restrict; +extern bool xen_is_stubdomain; int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num); int xen_set_pci_link_route(uint8_t link, uint8_t irq); diff --git a/system/globals.c b/system/globals.c index e353584..d602a04 100644 --- a/system/globals.c +++ b/system/globals.c @@ -60,6 +60,7 @@ bool qemu_uuid_set; uint32_t xen_domid; enum xen_mode xen_mode = XEN_DISABLED; bool xen_domid_restrict; +bool xen_is_stubdomain; struct evtchn_backend_ops *xen_evtchn_ops; struct gnttab_backend_ops *xen_gnttab_ops; struct foreignmem_backend_ops *xen_foreignmem_ops; From patchwork Wed Mar 27 03:05:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13605515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AB0AC6FD1F for ; Wed, 27 Mar 2024 03:07:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpJcc-00027x-97; Tue, 26 Mar 2024 23:06:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpJcV-00022V-Nj for qemu-devel@nongnu.org; Tue, 26 Mar 2024 23:06:31 -0400 Received: from fout3-smtp.messagingengine.com ([103.168.172.146]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpJcT-0004XI-5t for qemu-devel@nongnu.org; Tue, 26 Mar 2024 23:06:30 -0400 Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id C74C81380165; Tue, 26 Mar 2024 23:06:28 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Tue, 26 Mar 2024 23:06:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1711508788; x=1711595188; bh=MbWNAocmC2 tXBNyeSris83Z/pJQ9bxIJsFcp7jvH5XI=; b=B9XiLJ1Neg+S24LlG51xvdKE3F 09Eso1tQ6eMCxdX8xfVx09URS4QDwDXRVh0fyn7FIxabsDhRjzljbNXrF0Lq2rTG cPi2Xo6LK3zo9eaBCWpFzZVIpvSTNQ65CN69nb8TfJXc/V9uJYnTOLjLWAk8iphm IHDf6Ep3ZzOWL+rwAKMeQBR0cDJ7QPw/Cg8axiXh7jb6cfLN1n3ZF0jrEKikMYCg Fgni1Mrx7OgrHfiOHwKcoi7z4gAbtX6pAsuCE3qlxdvXp1ew0m6HY6bmMEI6Mgj2 nvJrKwskZcdUm0fhlo96QhwCPz09U0RvGj2+VF8ooZNuIOfg136ADQP87I6w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1711508788; x= 1711595188; bh=MbWNAocmC2tXBNyeSris83Z/pJQ9bxIJsFcp7jvH5XI=; b=x rn9Iqc7z00/ieKDGMeEHf1xO9LKYNU0XBssT6keiTb7RQ9Hws4K0T5mPmvSejNv/ UXTuHacyb1TwViEI0fJ0LLH/7qJxA0aVOmxvN03bxmQgPWvAMIAYBcmmNtRC2d8s zksfLwnyJDoYRjl/5UeEpBCPuzYCS0FdYMTqZUOBd6gf5j9iAGQdwV0gYdqX+kVl L2yHAJJt4cyCM7XKkW4gATx7UOSnPXwNjjA6MVafYQUVfUQJd42VMB1GW4jS85wL awikQIvHhpI/gObFsUMzSqsvgopC+osgLzTtFcWNSPvWE0nzLQZVGTDhyvwAng7N xkQguMh4PqLAY/npi3PpA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddugedgheeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Mar 2024 23:06:27 -0400 (EDT) From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Stefano Stabellini , Anthony Perard , Paul Durrant , xen-devel@lists.xenproject.org (open list:X86 Xen CPUs) Subject: [PATCH v3 2/2] xen: fix stubdom PCI addr Date: Wed, 27 Mar 2024 04:05:15 +0100 Message-ID: <35049e99da634a74578a1ff2cb3ae4cc436ede33.1711506237.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: none client-ip=103.168.172.146; envelope-from=marmarek@invisiblethingslab.com; helo=fout3-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When running in a stubdomain, the config space access via sysfs needs to use BDF as seen inside stubdomain (connected via xen-pcifront), which is different from the real BDF. For other purposes (hypercall parameters etc), the real BDF needs to be used. Get the in-stubdomain BDF by looking up relevant PV PCI xenstore entries. Signed-off-by: Marek Marczykowski-Górecki Reviewed-by: Anthony PERARD --- Changes in v3: - reduce 'path' size - add two missing error_setg() calls - coding style Changes in v2: - use xs_node_scanf - use %d instead of %u to read values written as %d - add a comment from another iteration of this patch by Jason Andryuk --- hw/xen/xen-host-pci-device.c | 76 ++++++++++++++++++++++++++++++++++++- hw/xen/xen-host-pci-device.h | 6 +++- 2 files changed, 81 insertions(+), 1 deletion(-) diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c index 8c6e9a1..eaf32f2 100644 --- a/hw/xen/xen-host-pci-device.c +++ b/hw/xen/xen-host-pci-device.c @@ -9,6 +9,8 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "qemu/cutils.h" +#include "hw/xen/xen-legacy-backend.h" +#include "hw/xen/xen-bus-helper.h" #include "xen-host-pci-device.h" #define XEN_HOST_PCI_MAX_EXT_CAP \ @@ -33,13 +35,73 @@ #define IORESOURCE_PREFETCH 0x00001000 /* No side effects */ #define IORESOURCE_MEM_64 0x00100000 +/* + * Non-passthrough (dom0) accesses are local PCI devices and use the given BDF + * Passthough (stubdom) accesses are through PV frontend PCI device. Those + * either have a BDF identical to the backend's BDF (xen-backend.passthrough=1) + * or a local virtual BDF (xen-backend.passthrough=0) + * + * We are always given the backend's BDF and need to lookup the appropriate + * local BDF for sysfs access. + */ +static void xen_host_pci_fill_local_addr(XenHostPCIDevice *d, Error **errp) +{ + unsigned int num_devs, len, i; + unsigned int domain, bus, dev, func; + char *be_path = NULL; + char path[16]; + + be_path = qemu_xen_xs_read(xenstore, 0, "device/pci/0/backend", &len); + if (!be_path) { + error_setg(errp, "Failed to read device/pci/0/backend"); + goto out; + } + + if (xs_node_scanf(xenstore, 0, be_path, "num_devs", NULL, + "%d", &num_devs) != 1) { + error_setg(errp, "Failed to read or parse %s/num_devs", be_path); + goto out; + } + + for (i = 0; i < num_devs; i++) { + snprintf(path, sizeof(path), "dev-%d", i); + if (xs_node_scanf(xenstore, 0, be_path, path, NULL, + "%x:%x:%x.%x", &domain, &bus, &dev, &func) != 4) { + error_setg(errp, "Failed to read or parse %s/%s", be_path, path); + goto out; + } + if (domain != d->domain || + bus != d->bus || + dev != d->dev || + func != d->func) + continue; + snprintf(path, sizeof(path), "vdev-%d", i); + if (xs_node_scanf(xenstore, 0, be_path, path, NULL, + "%x:%x:%x.%x", &domain, &bus, &dev, &func) != 4) { + error_setg(errp, "Failed to read or parse %s/%s", be_path, path); + goto out; + } + d->local_domain = domain; + d->local_bus = bus; + d->local_dev = dev; + d->local_func = func; + goto out; + } + error_setg(errp, "Failed to find PCI device %x:%x:%x.%x in xenstore", + d->domain, d->bus, d->dev, d->func); + +out: + free(be_path); +} + static void xen_host_pci_sysfs_path(const XenHostPCIDevice *d, const char *name, char *buf, ssize_t size) { int rc; rc = snprintf(buf, size, "/sys/bus/pci/devices/%04x:%02x:%02x.%d/%s", - d->domain, d->bus, d->dev, d->func, name); + d->local_domain, d->local_bus, d->local_dev, d->local_func, + name); assert(rc >= 0 && rc < size); } @@ -342,6 +404,18 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain, d->dev = dev; d->func = func; + if (xen_is_stubdomain) { + xen_host_pci_fill_local_addr(d, errp); + if (*errp) { + goto error; + } + } else { + d->local_domain = d->domain; + d->local_bus = d->bus; + d->local_dev = d->dev; + d->local_func = d->func; + } + xen_host_pci_config_open(d, errp); if (*errp) { goto error; diff --git a/hw/xen/xen-host-pci-device.h b/hw/xen/xen-host-pci-device.h index 4d8d34e..270dcb2 100644 --- a/hw/xen/xen-host-pci-device.h +++ b/hw/xen/xen-host-pci-device.h @@ -23,6 +23,12 @@ typedef struct XenHostPCIDevice { uint8_t dev; uint8_t func; + /* different from the above in case of stubdomain */ + uint16_t local_domain; + uint8_t local_bus; + uint8_t local_dev; + uint8_t local_func; + uint16_t vendor_id; uint16_t device_id; uint32_t class_code;