From patchwork Fri Nov 24 01:47:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13466873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2D0DC61DF7 for ; Fri, 24 Nov 2023 01:48:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640072.997874 (Exim 4.92) (envelope-from ) id 1r6LJ4-0001mQ-SF; Fri, 24 Nov 2023 01:48:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640072.997874; Fri, 24 Nov 2023 01:48:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ4-0001mG-KN; Fri, 24 Nov 2023 01:48:34 +0000 Received: by outflank-mailman (input) for mailman id 640072; Fri, 24 Nov 2023 01:48:32 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ2-0001id-OI for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 01:48:32 +0000 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 8fd474e0-8a6b-11ee-98e2-6d05b1d4d9a1; Fri, 24 Nov 2023 02:48:28 +0100 (CET) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 33CEE3200A4C; Thu, 23 Nov 2023 20:48:26 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 23 Nov 2023 20:48:27 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Nov 2023 20:48:24 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8fd474e0-8a6b-11ee-98e2-6d05b1d4d9a1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1700790505; x=1700876905; bh=3J KKOIM2Nz4Vj4VaBllPeeub5rgQRGqwurOAqkObw3w=; b=sB+ENd5v0/NOttHP2V 9MBQ3FuxRsST6OScRMKrRtQ0ZciacTM5hBw+s9CfNuB8NSM8ioIlLa6j8Y11lh7c G5KZH3vowVq0IuIto6FRUwnwstNdXjFNF1Is2Kj+MgfBpjYiNQIBrc+kVrFW/cnE hoMJ3gpMsCkF+Yrrg8FTLzwx+15w+Z5vW5zewy23fD2zjYcY9uW5xVITVFZ/X/IV WCDVeUVBLe/UYjJcecu6cPNmLFNdr6/dYkwpzkPOg33yQeQSPN8VxAXxtXAHmJRO LykLovvFRmXmBOZpfTSBwpcrvoB+aC733dk1T2cno3UHbPR6WUabogMcaFZ+jj4U XRxQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1700790505; x=1700876905; bh=3JKKOIM2Nz4Vj4VaBllPeeub5rgQRGqwurO AqkObw3w=; b=RtStN/0xsNETKwmkUsv7uyWt6o3dlO455ML68m1IkDEj97ayjZe qFF89JchV7eoZj8xQ9YQNaOQFpHqrI99LW93whwbmYIov6qISpk7vKi6wM6H2P/6 GchrwAlNWT5MKrmR9dCO9IyvYDVXUd75PEuIg9hO979JnbHEhJgh33oKDvyefLDy ZwebOgXZbS/zmAkP8pmNqA/vjSuR/OzfqmbpJaiFt4LGzfFtmfcXibLr+tviEY4W YrEiOSIEgX0QRDtSs0m7iOkqppo5ZKq31hIZp8oVs6TSwZn5IuQ8+gBOWsawtqTV YzHoy9KBUbMNzF5M+p+D8tioWIQovS+MUKQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehgedgfeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v4 1/6] x86/msi: passthrough all MSI-X vector ctrl writes to device model Date: Fri, 24 Nov 2023 02:47:05 +0100 Message-ID: <6c030c703fcd37d7dfae3c488c21b5bca540e352.1700790421.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 QEMU needs to know whether clearing maskbit of a vector is really clearing, or was already cleared before. Currently Xen sends only clearing that bit to the device model, but not setting it, so QEMU cannot detect it. Because of that, QEMU is working this around by checking via /dev/mem, but that isn't the proper approach. Give all necessary information to QEMU by passing all ctrl writes, including masking a vector. Advertise the new behavior via XENVER_get_features, so QEMU can know it doesn't need to access /dev/mem anymore. While this commit doesn't move the whole maskbit handling to QEMU (as discussed on xen-devel as one of the possibilities), it is a necessary first step anyway. Including telling QEMU it will get all the required information to do so. The actual implementation would need to include: - a hypercall for QEMU to control just maskbit (without (re)binding the interrupt again - a methor for QEMU to tell Xen it will actually do the work Those are not part of this series. Signed-off-by: Marek Marczykowski-Górecki --- I did not added any control to enable/disable this new behavior (as Roger have suggested for possible non-QEMU ioreqs). I don't see how the new behavior could be problematic for some existing ioreq server (they already received writes to those addresses, just not all of them), but if that's really necessary, I can probably add a command line option to restore previous behavior system-wide. Changes in v4: - ignore unaligned writes with X86EMUL_OKAY - restructure the code to forward all writes in _msixtbl_write() instead of manipulating return value of msixtbl_write() - this makes WRITE_LEN4_COMPLETION special case unnecessary - advertise the changed behavior via XENVER_get_features instead of DMOP v3: - advertise changed behavior in XEN_DMOP_get_ioreq_server_info - make "flags" parameter IN/OUT - move len check back to msixtbl_write() - will be needed there anyway in a later patch v2: - passthrough quad writes to emulator too (Jan) - (ab)use len==0 for write len=4 completion (Jan), but add descriptive #define for this magic value --- xen/arch/x86/hvm/vmsi.c | 19 ++++++++++++++----- xen/common/kernel.c | 1 + xen/include/public/features.h | 8 ++++++++ 3 files changed, 23 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 128f23636279..2436154c40b6 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -283,8 +283,8 @@ static int msixtbl_write(struct vcpu *v, unsigned long address, unsigned long flags; struct irq_desc *desc; - if ( (len != 4 && len != 8) || (address & (len - 1)) ) - return r; + if ( !IS_ALIGNED(address, len) ) + return X86EMUL_OKAY; rcu_read_lock(&msixtbl_rcu_lock); @@ -345,8 +345,7 @@ static int msixtbl_write(struct vcpu *v, unsigned long address, unlock: spin_unlock_irqrestore(&desc->lock, flags); - if ( len == 4 ) - r = X86EMUL_OKAY; + r = X86EMUL_OKAY; out: rcu_read_unlock(&msixtbl_rcu_lock); @@ -357,7 +356,17 @@ static int cf_check _msixtbl_write( const struct hvm_io_handler *handler, uint64_t address, uint32_t len, uint64_t val) { - return msixtbl_write(current, address, len, val); + /* ignore invalid length or unaligned writes */ + if ( len != 4 && len != 8 || !IS_ALIGNED(address, len) ) + return X86EMUL_OKAY; + + /* + * This function returns X86EMUL_UNHANDLEABLE even if write is properly + * handled, to propagate it to the device model (so it can keep its + * internal state in sync). + */ + msixtbl_write(current, address, len, val); + return X86EMUL_UNHANDLEABLE; } static bool cf_check msixtbl_range( diff --git a/xen/common/kernel.c b/xen/common/kernel.c index 08dbaa2a054c..229784c6ce52 100644 --- a/xen/common/kernel.c +++ b/xen/common/kernel.c @@ -642,6 +642,7 @@ long do_xen_version(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) fi.submap |= (1U << XENFEAT_direct_mapped); else fi.submap |= (1U << XENFEAT_not_direct_mapped); + fi.submap |= (1U << XENFEAT_dm_msix_all_writes); break; default: return -EINVAL; diff --git a/xen/include/public/features.h b/xen/include/public/features.h index 36936f6a4ee0..634534827d43 100644 --- a/xen/include/public/features.h +++ b/xen/include/public/features.h @@ -120,6 +120,14 @@ #define XENFEAT_runstate_phys_area 18 #define XENFEAT_vcpu_time_phys_area 19 +/* + * If set, Xen will passthrough all MSI-X vector ctrl writes to device model, + * not only those unmasking an entry. This allows device model to properly keep + * track of the MSI-X table without having to read it from the device behind + * Xen's backs. This information is relevant only for device models. + */ +#define XENFEAT_dm_msix_all_writes 20 + #define XENFEAT_NR_SUBMAPS 1 #endif /* __XEN_PUBLIC_FEATURES_H__ */ From patchwork Fri Nov 24 01:47:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13466877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BE5FC61D85 for ; Fri, 24 Nov 2023 01:49:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640073.997879 (Exim 4.92) (envelope-from ) id 1r6LJ5-0001tE-2v; Fri, 24 Nov 2023 01:48:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640073.997879; Fri, 24 Nov 2023 01:48:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ4-0001s7-Th; Fri, 24 Nov 2023 01:48:34 +0000 Received: by outflank-mailman (input) for mailman id 640073; Fri, 24 Nov 2023 01:48:33 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ2-0001id-Vg for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 01:48:33 +0000 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 917abd95-8a6b-11ee-98e2-6d05b1d4d9a1; Fri, 24 Nov 2023 02:48:31 +0100 (CET) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 3F8883200A55; Thu, 23 Nov 2023 20:48:29 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Thu, 23 Nov 2023 20:48:29 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Nov 2023 20:48:27 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 917abd95-8a6b-11ee-98e2-6d05b1d4d9a1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1700790508; x=1700876908; bh=uB O3SarZT20DQtUTXpVyOmSMQyGWgALrG2Y7qimUROA=; b=rVzUuAHaPZDELVy1KG U1lWLVxVnc0dEo6e4TUaNwFxTXVb282s6mTQd9oKgNfcQotkH8IUGhCiMfeolbYD iXNhp8kRd4gacX18+OHety+Y6gxkrx3lp+M+9q53M17+Jq/itnWLRYdDyDqZVf4z OcfELG/Mt/Hz2eMeJJ33HSOoxtwrbVlFwJv0U03JFURFz2zIKty0DzMnlhHgg2YO uuJ5yqNsBpEgr7VY0dhM1ItNJzseeqae409Pzz9nTH4K4FI/RuMDDYcYJYYXMzqC A1iyRRn62Rh3FxLBJf7iW59thqXWL1nsQ6OzhsRI0MD0FIC+lKXTRAsoeg8eC+Zl JEvQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1700790508; x=1700876908; bh=uBO3SarZT20DQtUTXpVyOmSMQyGWgALrG2Y 7qimUROA=; b=aEMo9Ou8iX6KPKFy+BGsG5uWwIpWAsYKWhQ8w3twaG7TlrnyKSY eR0qOS3s/GLAG0XxmkVLFKXabL0oF1OeFNOa4lxLdnFOa6JN6M+Mwy64TQzYXV9P xN0s8mCPteRU240tJAg3vmE/+jkVWIZa+V5gePIkwUOe2r7W8yXik+idB3ctuive TcxKdL0V0S4MoMppURvULYdjsl88PfK6EQMFO6neVFVBmLf6vZqgzo0swq6gbkL8 AFiwhGK3LIRV266yNQwDa/9BqTuBGlgZ/yn1XmkSq88BtqFAKuDx5I6rAEgf56Oq q3G1xmDXQEyPjYSagXBE7ba67hNQcZmshaQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehgedgfeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefh iefhgfehgfekteeigfdtiedvgfevhffhfedvudeltedvgfegtdekteeffeegheenucffoh hmrghinhepmhhsihigthgslhgplhhishhtrdhnvgigthenucevlhhushhtvghrufhiiigv pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisg hlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v4 2/6] x86/hvm: Allow access to registers on the same page as MSI-X table Date: Fri, 24 Nov 2023 02:47:06 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Some devices (notably Intel Wifi 6 AX210 card) keep auxiliary registers on the same page as MSI-X table. Device model (especially one in stubdomain) cannot really handle those, as direct writes to that page is refused (page is on the mmio_ro_ranges list). Instead, extend msixtbl_mmio_ops to handle such accesses too. Doing this, requires correlating read/write location with guest of MSI-X table address. Since QEMU doesn't map MSI-X table to the guest, it requires msixtbl_entry->gtable, which is HVM-only. Similar feature for PV would need to be done separately. This will be also used to read Pending Bit Array, if it lives on the same page, making QEMU not needing /dev/mem access at all (especially helpful with lockdown enabled in dom0). If PBA lives on another page, QEMU will map it to the guest directly. If PBA lives on the same page, discard writes and log a message. Technically, writes outside of PBA could be allowed, but at this moment the precise location of PBA isn't saved, and also no known device abuses the spec in this way (at least yet). To access those registers, msixtbl_mmio_ops need the relevant page mapped. MSI handling already has infrastructure for that, using fixmap, so try to map first/last page of the MSI-X table (if necessary) and save their fixmap indexes. Note that msix_get_fixmap() does reference counting and reuses existing mapping, so just call it directly, even if the page was mapped before. Also, it uses a specific range of fixmap indexes which doesn't include 0, so use 0 as default ("not mapped") value - which simplifies code a bit. GCC gets confused about 'desc' variable: arch/x86/hvm/vmsi.c: In function ‘msixtbl_range’: arch/x86/hvm/vmsi.c:553:8: error: ‘desc’ may be used uninitialized [-Werror=maybe-uninitialized] 553 | if ( desc ) | ^ arch/x86/hvm/vmsi.c:537:28: note: ‘desc’ was declared here 537 | const struct msi_desc *desc; | ^~~~ It's conditional initialization is actually correct (in the case where it isn't initialized, function returns early), but to avoid build failure initialize it explicitly to NULL anyway. Signed-off-by: Marek Marczykowski-Górecki --- Changes in v4: - drop same_page parameter of msixtbl_find_entry(), distinguish two cases in relevant callers - rename adj_access_table_idx to adj_access_idx - code style fixes - drop alignment check in adjacent_{read,write}() - all callers already have it earlier - delay mapping first/last MSI-X pages until preparing device for a passthrough v3: - merge handling into msixtbl_mmio_ops - extend commit message v2: - adjust commit message - pass struct domain to msixtbl_page_handler_get_hwaddr() - reduce local variables used only once - log a warning if write is forbidden if MSI-X and PBA lives on the same page - do not passthrough unaligned accesses - handle accesses both before and after MSI-X table --- xen/arch/x86/hvm/vmsi.c | 191 ++++++++++++++++++++++++++++++++-- xen/arch/x86/include/asm/msi.h | 5 +- xen/arch/x86/msi.c | 40 +++++++- 3 files changed, 225 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 2436154c40b6..d1e8cb1e30f6 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -180,6 +180,10 @@ static bool msixtbl_initialised(const struct domain *d) return d->arch.hvm.msixtbl_list.next; } +/* + * Lookup an msixtbl_entry on the same page as given addr. It's up to the + * caller to check if address is strictly part of the table - if relevant. + */ static struct msixtbl_entry *msixtbl_find_entry( struct vcpu *v, unsigned long addr) { @@ -187,8 +191,8 @@ static struct msixtbl_entry *msixtbl_find_entry( struct domain *d = v->domain; list_for_each_entry( entry, &d->arch.hvm.msixtbl_list, list ) - if ( addr >= entry->gtable && - addr < entry->gtable + entry->table_len ) + if ( PFN_DOWN(addr) >= PFN_DOWN(entry->gtable) && + PFN_DOWN(addr) <= PFN_DOWN(entry->gtable + entry->table_len - 1) ) return entry; return NULL; @@ -213,6 +217,131 @@ static struct msi_desc *msixtbl_addr_to_desc( return NULL; } +/* + * Returns: + * - UINT_MAX if no handling should be done + * - UINT_MAX-1 if write should be discarded + * - a fixmap idx to use for handling + */ +#define ADJACENT_DONT_HANDLE UINT_MAX +#define ADJACENT_DISCARD_WRITE (UINT_MAX - 1) +static unsigned int adjacent_handle( + const struct msixtbl_entry *entry, unsigned long addr, bool write) +{ + unsigned int adj_type; + const struct arch_msix *msix; + + if ( !entry || !entry->pdev ) + return ADJACENT_DONT_HANDLE; + + if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable) && addr < entry->gtable ) + adj_type = ADJ_IDX_FIRST; + else if ( PFN_DOWN(addr) == PFN_DOWN(entry->gtable + entry->table_len - 1) && + addr >= entry->gtable + entry->table_len ) + adj_type = ADJ_IDX_LAST; + else + return ADJACENT_DONT_HANDLE; + + msix = entry->pdev->msix; + ASSERT(msix); + + if ( !msix->adj_access_idx[adj_type] ) + { + gprintk(XENLOG_WARNING, + "Page for adjacent(%d) MSI-X table access not initialized for %pp (addr %#lx, gtable %#lx\n", + adj_type, &entry->pdev->sbdf, addr, entry->gtable); + + return ADJACENT_DONT_HANDLE; + } + + /* If PBA lives on the same page too, discard writes. */ + if ( write && + ((adj_type == ADJ_IDX_LAST && + msix->table.last == msix->pba.first) || + (adj_type == ADJ_IDX_FIRST && + msix->table.first == msix->pba.last)) ) + { + gprintk(XENLOG_WARNING, + "MSI-X table and PBA of %pp live on the same page, " + "writing to other registers there is not implemented\n", + &entry->pdev->sbdf); + return ADJACENT_DISCARD_WRITE; + } + + return msix->adj_access_idx[adj_type]; +} + +static int adjacent_read( + unsigned int fixmap_idx, + paddr_t address, unsigned int len, uint64_t *pval) +{ + const void __iomem *hwaddr; + + *pval = ~0UL; + + ASSERT(fixmap_idx != ADJACENT_DISCARD_WRITE); + + hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address); + + switch ( len ) + { + case 1: + *pval = readb(hwaddr); + break; + + case 2: + *pval = readw(hwaddr); + break; + + case 4: + *pval = readl(hwaddr); + break; + + case 8: + *pval = readq(hwaddr); + break; + + default: + ASSERT_UNREACHABLE(); + } + return X86EMUL_OKAY; +} + +static int adjacent_write( + unsigned int fixmap_idx, + uint64_t address, uint32_t len, uint64_t val) +{ + void __iomem *hwaddr; + + if ( fixmap_idx == ADJACENT_DISCARD_WRITE ) + return X86EMUL_OKAY; + + hwaddr = fix_to_virt(fixmap_idx) + PAGE_OFFSET(address); + + switch ( len ) + { + case 1: + writeb(val, hwaddr); + break; + + case 2: + writew(val, hwaddr); + break; + + case 4: + writel(val, hwaddr); + break; + + case 8: + writeq(val, hwaddr); + break; + + default: + ASSERT_UNREACHABLE(); + } + return X86EMUL_OKAY; +} + static int cf_check msixtbl_read( const struct hvm_io_handler *handler, uint64_t address, uint32_t len, uint64_t *pval) @@ -220,16 +349,31 @@ static int cf_check msixtbl_read( unsigned long offset; struct msixtbl_entry *entry; unsigned int nr_entry, index; + unsigned int adjacent_fixmap; int r = X86EMUL_UNHANDLEABLE; - if ( (len != 4 && len != 8) || (address & (len - 1)) ) + if ( !IS_ALIGNED(address, len) ) return r; rcu_read_lock(&msixtbl_rcu_lock); - entry = msixtbl_find_entry(current, address); if ( !entry ) goto out; + + adjacent_fixmap = adjacent_handle(entry, address, false); + if ( adjacent_fixmap != ADJACENT_DONT_HANDLE ) + { + r = adjacent_read(adjacent_fixmap, address, len, pval); + goto out; + } + + if ( address < entry->gtable || + address >= entry->gtable + entry->table_len ) + goto out; + + if ( len != 4 && len != 8 ) + goto out; + offset = address & (PCI_MSIX_ENTRY_SIZE - 1); if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET ) @@ -282,6 +426,7 @@ static int msixtbl_write(struct vcpu *v, unsigned long address, int r = X86EMUL_UNHANDLEABLE; unsigned long flags; struct irq_desc *desc; + unsigned int adjacent_fixmap; if ( !IS_ALIGNED(address, len) ) return X86EMUL_OKAY; @@ -291,6 +436,19 @@ static int msixtbl_write(struct vcpu *v, unsigned long address, entry = msixtbl_find_entry(v, address); if ( !entry ) goto out; + + adjacent_fixmap = adjacent_handle(entry, address, true); + if ( adjacent_fixmap != ADJACENT_DONT_HANDLE ) + { + r = adjacent_write(adjacent_fixmap, address, len, val); + goto out; + } + if ( address < entry->gtable || + address >= entry->gtable + entry->table_len ) + goto out; + if ( len != 4 && len != 8 ) + goto out; + nr_entry = array_index_nospec(((address - entry->gtable) / PCI_MSIX_ENTRY_SIZE), MAX_MSIX_TABLE_ENTRIES); @@ -356,8 +514,8 @@ static int cf_check _msixtbl_write( const struct hvm_io_handler *handler, uint64_t address, uint32_t len, uint64_t val) { - /* ignore invalid length or unaligned writes */ - if ( len != 4 && len != 8 || !IS_ALIGNED(address, len) ) + /* ignore unaligned writes */ + if ( !IS_ALIGNED(address, len) ) return X86EMUL_OKAY; /* @@ -374,14 +532,22 @@ static bool cf_check msixtbl_range( { struct vcpu *curr = current; unsigned long addr = r->addr; - const struct msi_desc *desc; + const struct msixtbl_entry *entry; + const struct msi_desc *desc = NULL; + unsigned int adjacent_fixmap; ASSERT(r->type == IOREQ_TYPE_COPY); rcu_read_lock(&msixtbl_rcu_lock); - desc = msixtbl_addr_to_desc(msixtbl_find_entry(curr, addr), addr); + entry = msixtbl_find_entry(curr, addr); + adjacent_fixmap = adjacent_handle(entry, addr, false); + if ( adjacent_fixmap == ADJACENT_DONT_HANDLE ) + desc = msixtbl_addr_to_desc(entry, addr); rcu_read_unlock(&msixtbl_rcu_lock); + if ( adjacent_fixmap != ADJACENT_DONT_HANDLE ) + return 1; + if ( desc ) return 1; @@ -622,12 +788,15 @@ void msix_write_completion(struct vcpu *v) v->arch.hvm.hvm_io.msix_snoop_gpa ) { unsigned int token = hvmemul_cache_disable(v); - const struct msi_desc *desc; + const struct msi_desc *desc = NULL; + const struct msixtbl_entry *entry; uint32_t data; rcu_read_lock(&msixtbl_rcu_lock); - desc = msixtbl_addr_to_desc(msixtbl_find_entry(v, snoop_addr), - snoop_addr); + entry = msixtbl_find_entry(v, snoop_addr); + if ( entry && snoop_addr >= entry->gtable && + snoop_addr < entry->gtable + entry->table_len ) + desc = msixtbl_addr_to_desc(entry, snoop_addr); rcu_read_unlock(&msixtbl_rcu_lock); if ( desc && diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h index c1ece2786e01..6d12f9dac0db 100644 --- a/xen/arch/x86/include/asm/msi.h +++ b/xen/arch/x86/include/asm/msi.h @@ -207,6 +207,10 @@ struct msg_address { PCI_MSIX_ENTRY_SIZE + \ (~PCI_MSIX_BIRMASK & (PAGE_SIZE - 1))) +/* indexes in adj_access_idx[] below */ +#define ADJ_IDX_FIRST 0 +#define ADJ_IDX_LAST 1 + struct arch_msix { unsigned int nr_entries, used_entries; struct { @@ -214,6 +218,7 @@ struct arch_msix { } table, pba; int table_refcnt[MAX_MSIX_TABLE_PAGES]; int table_idx[MAX_MSIX_TABLE_PAGES]; + unsigned int adj_access_idx[2]; spinlock_t table_lock; bool host_maskall, guest_maskall; domid_t warned; diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index 7f8e79425452..86fbe7f2e143 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -916,6 +916,36 @@ static int msix_capability_init(struct pci_dev *dev, list_add_tail(&entry->list, &dev->msi_list); *desc = entry; } + else + { + /* + * If the MSI-X table doesn't start at the page boundary, map the first page for + * passthrough accesses. + */ + if ( PAGE_OFFSET(table_paddr) ) + { + int idx = msix_get_fixmap(msix, table_paddr, table_paddr); + + if ( idx > 0 ) + msix->adj_access_idx[ADJ_IDX_FIRST] = idx; + else + gprintk(XENLOG_ERR, "Failed to map first MSI-X table page: %d\n", idx); + } + /* + * If the MSI-X table doesn't end on the page boundary, map the last page + * for passthrough accesses. + */ + if ( PAGE_OFFSET(table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE) ) + { + uint64_t entry_paddr = table_paddr + msix->nr_entries * PCI_MSIX_ENTRY_SIZE; + int idx = msix_get_fixmap(msix, table_paddr, entry_paddr); + + if ( idx > 0 ) + msix->adj_access_idx[ADJ_IDX_LAST] = idx; + else + gprintk(XENLOG_ERR, "Failed to map last MSI-X table page: %d\n", idx); + } + } if ( !msix->used_entries ) { @@ -1078,6 +1108,16 @@ static void _pci_cleanup_msix(struct arch_msix *msix) WARN(); msix->table.first = 0; msix->table.last = 0; + if ( msix->adj_access_idx[ADJ_IDX_FIRST] ) + { + msix_put_fixmap(msix, msix->adj_access_idx[ADJ_IDX_FIRST]); + msix->adj_access_idx[ADJ_IDX_FIRST] = 0; + } + if ( msix->adj_access_idx[ADJ_IDX_LAST] ) + { + msix_put_fixmap(msix, msix->adj_access_idx[ADJ_IDX_LAST]); + msix->adj_access_idx[ADJ_IDX_LAST] = 0; + } if ( rangeset_remove_range(mmio_ro_ranges, msix->pba.first, msix->pba.last) ) From patchwork Fri Nov 24 01:47:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13466875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 771B5C636D0 for ; Fri, 24 Nov 2023 01:48:56 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640075.997902 (Exim 4.92) (envelope-from ) id 1r6LJ8-0002V6-Py; Fri, 24 Nov 2023 01:48:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640075.997902; Fri, 24 Nov 2023 01:48:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ8-0002UQ-K1; Fri, 24 Nov 2023 01:48:38 +0000 Received: by outflank-mailman (input) for mailman id 640075; Fri, 24 Nov 2023 01:48:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ7-0002RQ-No for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 01:48:37 +0000 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 92e91973-8a6b-11ee-9b0e-b553b5be7939; Fri, 24 Nov 2023 02:48:34 +0100 (CET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id C8BCC3200A49; Thu, 23 Nov 2023 20:48:31 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 23 Nov 2023 20:48:32 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Nov 2023 20:48:30 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 92e91973-8a6b-11ee-9b0e-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1700790511; x=1700876911; bh=LX yAOwjjde0ByZ9X1DEhA7dY4ff5w8COeOtbRqSP8vg=; b=Pdug+HOGAg7VS85J4z BNN594jq/XE/8+nYpETemTpPhbQDUo7R9Vl3D2sZh8qGPDEGE2kMRj+my8W9ZjPX OiTtAaPSAdtp/8JcIwZAK7xEmT3ts0tUF5zwayUmbwuGk48/9fG2rqnpaLHCSlR4 I/tbZ38ooYCDO3LgmDADS/441aJYHU5mB/e+WbTNwwYyCPlDYFIJGHGxvnPzWKoT oQLUa8lJs53nFwzwp1d+8mnxFO0KbDyWUry5TCq2uo92WkJOmqRnjhgXr0foEyAO 03JzV5fT+seBNl4wDLqo17cXrUCZuMIeY4/B+IeKsmebWo8zuAjsjMYBsV+He5as iVbw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1700790511; x=1700876911; bh=LXyAOwjjde0ByZ9X1DEhA7dY4ff5w8COeOt bRqSP8vg=; b=jYGZRnb3loPFsO7oIZa6c+7DpFES3Hkd7fWkz5bKhkh6p+GZBrE 8i/nVMl8eItIqi486wZgJxdbaJXrD7xtiptOcFjC4vW7t2PPFBNrScfD6220o01g VZTPRqdmVk4Zte45ww2kVEm6SdltYt2bJSnQp5dOVJ2ulCuw/AuKHaIMLWiNIXfy BVELY3H1AJoDHzM3FrfnYTToAlJzXmN2/Kjph31aZt5Dh3t+mlidMcHp0omx5aI5 kHF9yEkbXN7fDAxuE4y+R/voMWx1duOJAqsTqx4C9AKQ4ZUrJYQTjVTJfbexPopZ bZp75RCKiygMFKrpgZGCx3mURi7CB7cT8+Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehgedggedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Doug Goldstein , Stefano Stabellini Subject: [PATCH v4 3/6] automation: prevent QEMU access to /dev/mem in PCI passthrough tests Date: Fri, 24 Nov 2023 02:47:07 +0100 Message-ID: <5c7cf4c680200c37187b90eebc49f600865c65ec.1700790421.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 /dev/mem access doesn't work in dom0 in lockdown and in stubdomain. Simulate this environment with removing /dev/mem device node. Full test for lockdown and stubdomain will come later, when all requirements will be in place. Signed-off-by: Marek Marczykowski-Górecki Acked-by: Stefano Stabellini --- This can be applied only after QEMU change is committed. Otherwise the test will fail. --- automation/scripts/qubes-x86-64.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh index d81ed7b931cf..7eabc1bd6ad4 100755 --- a/automation/scripts/qubes-x86-64.sh +++ b/automation/scripts/qubes-x86-64.sh @@ -163,6 +163,8 @@ ifconfig eth0 up ifconfig xenbr0 up ifconfig xenbr0 192.168.0.1 +# ensure QEMU wont have access /dev/mem +rm -f /dev/mem # get domU console content into test log tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" & xl create /etc/xen/domU.cfg From patchwork Fri Nov 24 01:47:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13466872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84E9CC61D85 for ; Fri, 24 Nov 2023 01:48:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640074.997897 (Exim 4.92) (envelope-from ) id 1r6LJ8-0002SI-F7; Fri, 24 Nov 2023 01:48:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640074.997897; Fri, 24 Nov 2023 01:48:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ8-0002SB-Bg; Fri, 24 Nov 2023 01:48:38 +0000 Received: by outflank-mailman (input) for mailman id 640074; Fri, 24 Nov 2023 01:48:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJ6-0001id-I2 for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 01:48:36 +0000 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 943652b4-8a6b-11ee-98e2-6d05b1d4d9a1; Fri, 24 Nov 2023 02:48:35 +0100 (CET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 04B9E3200A48; Thu, 23 Nov 2023 20:48:33 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 23 Nov 2023 20:48:34 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Nov 2023 20:48:32 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 943652b4-8a6b-11ee-98e2-6d05b1d4d9a1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1700790513; x=1700876913; bh=eq 3idnNj7apk53TrjyNs3MN+0xTHXPGEghyRmeuxtK4=; b=pixeXlnLfcoyGZAYeF ko+mkN4lEy+OTyNialH5Nql+TFXiVQyuFa3NuKcq218M68g1VUKl9a21l/Hz8o3n sjHAYRMR64q6jC8JOnUnro0CLkqqaG3MR19WSeHVJNvlLCNqJQqguSesU5o6oz3j Y5MpwEzvZOgoAAlTBzgqA0KpvI9XDtgmTy5vow6Z4szlZ67Qwwz+buu48R1s8SBz 6CQ2RbzNvRWCP0L86KOZWDdvO90luhNg77CBBPPR9csG4RGCnreTmqhqCELgA4PW HjTTpvGsWnleoJSClwczt0opwRdfJKRcAz499Fp3AgLlI8qBgC1b9/f6KKaNnGpp ESgA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1700790513; x=1700876913; bh=eq3idnNj7apk53TrjyNs3MN+0xTHXPGEghy RmeuxtK4=; b=1IrHat9KKHv8B5r7izegXxv0l51hOzSWM0hqKGjxKrusIAJwU7l +kbUvIKJdjJXLLIsdCQM8eMYRPgagOWUVMzA3vTs0mXiaYTEsxD5SZLXWrSqMWnX 3iPtEKwN+A8bEuOmX5y/wi1MC5NpQV6aa+uMXJeYZZBMEJ9yKcMdamhfNmW0eju8 k7Qh+a+niUyGc1bDjg1eDAsQ9+Q5DdED5JL1OxggAa5VR7OrPSWrkHXkj7ajDEDB 6Qug3CfzbHVGPZYhLBfwS5GIckHxC3XHFxZidPilrB/r5+v9G4CBlYlj5vR4Cgpo xFaNRP7wKgnCiuMYysuQaLGmFpEja2GD3lg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehgedggedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeff iedtgeevffetledvgefhhfevgffhfeekleehueejjeegvddvgfffjeeutddvleenucffoh hmrghinhepkhgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhngh hslhgrsgdrtghomh X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Doug Goldstein , Stefano Stabellini Subject: [PATCH v4 4/6] automation: switch to a wifi card on ADL system Date: Fri, 24 Nov 2023 02:47:08 +0100 Message-ID: <9f18111c2d48b0715837789fc905d9e7b2a1ef0c.1700790421.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Switch to a wifi card that has registers on a MSI-X page. This tests the "x86/hvm: Allow writes to registers on the same page as MSI-X table" feature. Switch it only for HVM test, because MSI-X adjacent write is not supported on PV. This requires also including drivers and firmware in system for tests. Remove firmware unrelated to the test, to not increase initrd size too much (all firmware takes over 100MB compressed). And finally adjusts test script to handle not only eth0 as a test device, but also wlan0 and connect it to the wifi network. Signed-off-by: Marek Marczykowski-Górecki Reviewed-by: Stefano Stabellini --- This needs two new gitlab variables: WIFI_HW2_SSID and WIFI_HW2_PSK. I'll provide them in private. This change requires rebuilding test containers. This can be applied only after QEMU change is committed. Otherwise the test will fail. --- automation/gitlab-ci/test.yaml | 4 ++++ automation/scripts/qubes-x86-64.sh | 7 +++++++ automation/tests-artifacts/alpine/3.18.dockerfile | 7 +++++++ automation/tests-artifacts/kernel/6.1.19.dockerfile | 2 ++ 4 files changed, 20 insertions(+) diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index 6aabdb9d156f..931a8fb28e1d 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -195,6 +195,10 @@ adl-pci-pv-x86-64-gcc-debug: adl-pci-hvm-x86-64-gcc-debug: extends: .adl-x86-64 + variables: + PCIDEV: "00:14.3" + WIFI_SSID: "$WIFI_HW2_SSID" + WIFI_PSK: "$WIFI_HW2_PSK" script: - ./automation/scripts/qubes-x86-64.sh pci-hvm 2>&1 | tee ${LOGFILE} needs: diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh index 7eabc1bd6ad4..60498ef1e89a 100755 --- a/automation/scripts/qubes-x86-64.sh +++ b/automation/scripts/qubes-x86-64.sh @@ -94,6 +94,13 @@ on_reboot = "destroy" domU_check=" set -x -e interface=eth0 +if [ -e /sys/class/net/wlan0 ]; then + interface=wlan0 + set +x + wpa_passphrase "$WIFI_SSID" "$WIFI_PSK" > /etc/wpa_supplicant.conf + set -x + wpa_supplicant -B -iwlan0 -c /etc/wpa_supplicant.conf +fi ip link set \"\$interface\" up timeout 30s udhcpc -i \"\$interface\" pingip=\$(ip -o -4 r show default|cut -f 3 -d ' ') diff --git a/automation/tests-artifacts/alpine/3.18.dockerfile b/automation/tests-artifacts/alpine/3.18.dockerfile index f1b4a8b7a191..b821a291fed3 100644 --- a/automation/tests-artifacts/alpine/3.18.dockerfile +++ b/automation/tests-artifacts/alpine/3.18.dockerfile @@ -34,6 +34,13 @@ RUN \ apk add curl && \ apk add udev && \ apk add pciutils && \ + apk add wpa_supplicant && \ + # Select firmware for hardware tests + apk add linux-firmware-other && \ + mkdir /lib/firmware-preserve && \ + mv /lib/firmware/iwlwifi-so-a0-gf-a0* /lib/firmware-preserve/ && \ + rm -rf /lib/firmware && \ + mv /lib/firmware-preserve /lib/firmware && \ \ # Xen cd / && \ diff --git a/automation/tests-artifacts/kernel/6.1.19.dockerfile b/automation/tests-artifacts/kernel/6.1.19.dockerfile index 3a4096780d20..84ed5dff23ae 100644 --- a/automation/tests-artifacts/kernel/6.1.19.dockerfile +++ b/automation/tests-artifacts/kernel/6.1.19.dockerfile @@ -32,6 +32,8 @@ RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSI make xen.config && \ scripts/config --enable BRIDGE && \ scripts/config --enable IGC && \ + scripts/config --enable IWLWIFI && \ + scripts/config --enable IWLMVM && \ cp .config .config.orig && \ cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \ make -j$(nproc) bzImage && \ From patchwork Fri Nov 24 01:47:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13466871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72EAEC61D97 for ; Fri, 24 Nov 2023 01:48:52 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640076.997916 (Exim 4.92) (envelope-from ) id 1r6LJC-00031A-4A; Fri, 24 Nov 2023 01:48:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640076.997916; Fri, 24 Nov 2023 01:48:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJC-000313-0t; Fri, 24 Nov 2023 01:48:42 +0000 Received: by outflank-mailman (input) for mailman id 640076; Fri, 24 Nov 2023 01:48:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJA-0002RQ-GO for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 01:48:40 +0000 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9609b4b5-8a6b-11ee-9b0e-b553b5be7939; Fri, 24 Nov 2023 02:48:38 +0100 (CET) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id B0BF13200A48; Thu, 23 Nov 2023 20:48:36 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 23 Nov 2023 20:48:37 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Nov 2023 20:48:34 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9609b4b5-8a6b-11ee-9b0e-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1700790516; x=1700876916; bh=vpLMgvoJof R2y12hqIaKomgeg+9OwWixKJMvmnvwMU0=; b=0saaUeqCjfcRk1zzI83y/8Esez QgzgBJY3IvqCvRd2Trg7lQtfXYLxfo+wNqViHFjlN6QvRw5frgIwJwP8QaRztNVY 5J+vfQ0Kl1GSRkvPx2OdGRvgILWOT67/4QKNemlJNPy2/XCQhwV+D8B8KZe8kkRD 3Jk5eL90Lut/rjpwBMZnLTnDl9zqqdv9xEr1GpyizY9GevbB9IxCFM2wdlaeWjKe ioXHRLfgrjEZfFwnlmJLfsjnrEJG2bEOGdVV9+QIuYcjYjgGKjzTKNLFmNxlE1Kb Ycl6IEPnTAhdmDXYFbuKllX1mc996zT9keRAexjghEklE/2o5d+PEzuPaEJA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1700790516; x= 1700876916; bh=vpLMgvoJofR2y12hqIaKomgeg+9OwWixKJMvmnvwMU0=; b=E iIQUonKQZ9r/WIVdN6uVj5C7QZ6r/ZL8sJVBTxMLNjDiL61ZPvi6BQZMr4EzVh+e vFdwiNIZqRhFxhvmHnVHcPd1a2ouhlTykReqdolPnZGKn/rWWENuWFMHlwHdf97r FY8qws65QDhIUbuwmY+KphdSLyV2kXf7hbqA6D4187mVA66PU9OCCzZiubV8Psnl vlIYTVyZdSklM0nUJbZH9t4UCTvdYrCgN2R0icE4iHf+H45D8h4bC19oGOIX7x8X pZea6Nlei6/COX+jkHS61yAXqXCLoCGr2ifOxE11z2Vf2lFic1OkNZv/0tRAtQsV PbH1gmNI6fxXt936kXOtg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehgedggedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgeet tdeigefhkefhfeegvdeuiedvtdffkeeltdeivdfgteelueeuhfetleevueegnecuffhomh grihhnpeigvghnrdhorhhgpdhgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigv pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisg hlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v4 5/6] [DO NOT APPLY] switch to qemu fork Date: Fri, 24 Nov 2023 02:47:09 +0100 Message-ID: <96b370aee16ee0429db38009a5d694cdf8ba34b7.1700790421.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 This makes tests to use patched QEMU, to actually test the new behavior. --- Config.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Config.mk b/Config.mk index 2c43702958eb..dd2687c0e9e7 100644 --- a/Config.mk +++ b/Config.mk @@ -222,8 +222,8 @@ endif OVMF_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/ovmf.git OVMF_UPSTREAM_REVISION ?= ba91d0292e593df8528b66f99c1b0b14fadc8e16 -QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git -QEMU_UPSTREAM_REVISION ?= master +QEMU_UPSTREAM_URL ?= https://github.com/marmarek/qemu +QEMU_UPSTREAM_REVISION ?= origin/msix MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git MINIOS_UPSTREAM_REVISION ?= b08019f0b2fbc30c75169a160acb9fd9af5d68f4 From patchwork Fri Nov 24 01:47:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 13466876 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57150C61D97 for ; Fri, 24 Nov 2023 01:48:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640077.997927 (Exim 4.92) (envelope-from ) id 1r6LJE-0003LN-FP; Fri, 24 Nov 2023 01:48:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640077.997927; Fri, 24 Nov 2023 01:48:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJE-0003LC-Al; Fri, 24 Nov 2023 01:48:44 +0000 Received: by outflank-mailman (input) for mailman id 640077; Fri, 24 Nov 2023 01:48:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6LJC-0002RQ-OL for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 01:48:42 +0000 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 975a60df-8a6b-11ee-9b0e-b553b5be7939; Fri, 24 Nov 2023 02:48:41 +0100 (CET) Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 487873200A49; Thu, 23 Nov 2023 20:48:39 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Thu, 23 Nov 2023 20:48:39 -0500 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Nov 2023 20:48:37 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 975a60df-8a6b-11ee-9b0e-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1700790518; x=1700876918; bh=YR4SrntrkR 9h95RVyu4/vR1Bp6CZaXa+a+sAHNBe1pw=; b=5Nuj4LMMKdO58yXKRIAtvoDnPE JfHUU6qTzl/I/HUBkzeEJ2PAkVE0i7lhYatqPUbzKwEAQt8w5+ARm4I53gXnKMpB fCKNVR3I4F0124bK362/WTB6DKthrf6rYsdBNO8r/zrSFOMUi2bxFmeyG38C1Jcc NkID4CN3AtnF6vvIgmP0/QGjQ4wiffkRdZzF2ficl0vz8VXxrNFKr0KjEn7ssSZh WwpiXkMfY4squ3HH54OtaDO162pbX7eCYby8/qvH7TH+l6bzCKdV/MLnlMkKr2eg OSvstZC1syIgTYAQJ86fr3AD7Llc0Y9Qy0Y2TICOAzTp6u0gHA0zibl1b3iA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1700790518; x= 1700876918; bh=YR4SrntrkR9h95RVyu4/vR1Bp6CZaXa+a+sAHNBe1pw=; b=u ljuZ0hyCLTwE+0FV7o8cunHauFzgCqgDXtNQuIExT6p1zDOss76+iTyoIFOzU/8i zKAQq/jek/4RIZVDziNzf+kwtq2GNX1apxJbxFK3/MtLxZ/gCbrAUkFCG5Mjm/by o0Nn3J8aFchSicd4BUq4+/A1TNmX+KLMKUUiLrckTNft8y9khSR9Tcy9mpWGQfYa xbqUkeXG0ETp3ErH1nqQy6pbLVpuwwwkbv7I6RgPKZm2AmTSTjAMQ5jlWS1xkvhw bqLmp/i2eAeS6m3JtN+cKUIr22rjD3G1r7eXDDHt2oDMZEye/PwoZXsapz0r1/Nd R5aPUTodtRw6FI15NaNtQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehgedggedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredtjeenucfhrhhomhepofgrrhgv khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepueet hefgieehheeftdeuleduhffgieelueejhffggfeiieevtefhfeffffeftdehnecuffhomh grihhnpehgihhtlhgrsgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthhhinhhgsh hlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Doug Goldstein , Stefano Stabellini Subject: [PATCH v4 6/6] [DO NOT APPLY] switch to alternative artifact repo Date: Fri, 24 Nov 2023 02:47:10 +0100 Message-ID: <7e7977bc5630a40f522b45da891689cdf11da29c.1700790421.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 For testing, switch to my containers registry that includes containers rebuilt with changes in this series. --- automation/gitlab-ci/build.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml index 32af30ccedc9..52abb12bce48 100644 --- a/automation/gitlab-ci/build.yaml +++ b/automation/gitlab-ci/build.yaml @@ -324,7 +324,7 @@ qemu-system-ppc64-8.1.0-ppc64-export: alpine-3.18-rootfs-export: extends: .test-jobs-artifact-common - image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18 + image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/alpine:3.18 script: - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz artifacts: @@ -335,7 +335,7 @@ alpine-3.18-rootfs-export: kernel-6.1.19-export: extends: .test-jobs-artifact-common - image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19 + image: registry.gitlab.com/xen-project/people/marmarek/xen/tests-artifacts/kernel:6.1.19 script: - mkdir binaries && cp /bzImage binaries/bzImage artifacts: