From patchwork Sun Jan 15 22:14:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 13102466 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56765C3DA78 for ; Sun, 15 Jan 2023 22:14:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.478174.741202 (Exim 4.92) (envelope-from ) id 1pHBGy-0003ac-KM; Sun, 15 Jan 2023 22:14:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 478174.741202; Sun, 15 Jan 2023 22:14:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pHBGy-0003aV-HC; Sun, 15 Jan 2023 22:14:40 +0000 Received: by outflank-mailman (input) for mailman id 478174; Sun, 15 Jan 2023 22:14:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pHBGv-0003aN-Cc for xen-devel@lists.xenproject.org; Sun, 15 Jan 2023 22:14:38 +0000 Received: from casper.infradead.org (casper.infradead.org [2001:8b0:10b:1236::1]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id fe05a741-9521-11ed-91b6-6bf2151ebd3b; Sun, 15 Jan 2023 23:14:35 +0100 (CET) Received: from [2001:8b0:10b:5::bb3] (helo=u3832b3a9db3152.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHBGr-008Am9-UN; Sun, 15 Jan 2023 22:14:34 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fe05a741-9521-11ed-91b6-6bf2151ebd3b DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:References: In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=USOJ+cxlk1PyqjpZN81t13zkmrDPpT6u+SQIj14OMZE=; b=BsRaAFzKtc/sGee4NxKo+j1BWV mECSjylTlTXAhsO1+Rg9dRLzLeN9yhebdXuaQJVEdFgb7pdCn9eE347Ou2vseswC/OobK6Czyaquy ROqu/NxtbcGiKueFDffZQ7HUqQPoUIfzqXzy4SEzg5pUkUoGaUHLxAtCU88lQRmQn6g9lzNP7SFpv UfRq1mRUI7fr9ntUgBtbXIu6StNGgLcSUJSuAZtMjM+DPlG2X8z0YtzM5OSsBr7IRAWpjT3hxwst/ wDEjRp+/TkQGeUNn9wwkvvXuT4lVRteAnWcfAExHEpZ3xjPa0003P2GhdKUsGqUnG1InNZ5XXxBk8 5e0y1eFQ==; Message-ID: <4bffa69a949bfdc92c4a18e5a1c3cbb3b94a0d32.camel@infradead.org> Subject: [PATCH] x86/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain From: David Woodhouse To: Thomas Gleixner , LKML , xen-devel , Juergen Gross Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , Michael Ellerman , Christophe Leroy , linuxppc-dev@lists.ozlabs.org, "Ahmed S. Darwish" , Reinette Chatre Date: Sun, 15 Jan 2023 22:14:19 +0000 In-Reply-To: <20221111122015.631728309@linutronix.de> References: <20221111120501.026511281@linutronix.de> <20221111122015.631728309@linutronix.de> User-Agent: Evolution 3.44.4-0ubuntu1 MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html The Xen MSI → PIRQ magic does support MSI-X, so advertise it. (In fact it's better off with MSI-X than MSI, because it's actually broken by design for 32-bit MSI, since it puts the high bits of the PIRQ# into the high 32 bits of the MSI message address, instead of the Extended Destination ID field which is in bits 4-11. Strictly speaking, this really fixes a much older commit 2e4386eba0c0 ("x86/xen: Wrap XEN MSI management into irqdomain") which failed to set the flag. But that never really mattered until __pci_enable_msix_range() started to check and bail out early. So in 6.2-rc we see failures e.g. to bring up networking on an Amazon EC2 m4.16xlarge instance: [ 41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524 [ 41.498705] ena 0000:00:03.0: Can not reserve msix vectors [ 41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts Side note: This is the first bug found, and first patch tested, by running Xen guests under QEMU/KVM instead of running under actual Xen. Fixes: 99f3d2797657 ("PCI/MSI: Reject MSI-X early") Signed-off-by: David Woodhouse --- arch/x86/pci/xen.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index b94f727251b6..790550479831 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -433,6 +433,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = { }; static struct msi_domain_info xen_pci_msi_domain_info = { + .flags = MSI_FLAG_PCI_MSIX, .ops = &xen_pci_msi_domain_ops, };