From patchwork Sun Jan 15 22:14:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 13102465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B3ADC46467 for ; Sun, 15 Jan 2023 22:14:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231488AbjAOWOg (ORCPT ); Sun, 15 Jan 2023 17:14:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231318AbjAOWOf (ORCPT ); Sun, 15 Jan 2023 17:14:35 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C3B21B54F; Sun, 15 Jan 2023 14:14:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:References: In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=USOJ+cxlk1PyqjpZN81t13zkmrDPpT6u+SQIj14OMZE=; b=BsRaAFzKtc/sGee4NxKo+j1BWV mECSjylTlTXAhsO1+Rg9dRLzLeN9yhebdXuaQJVEdFgb7pdCn9eE347Ou2vseswC/OobK6Czyaquy ROqu/NxtbcGiKueFDffZQ7HUqQPoUIfzqXzy4SEzg5pUkUoGaUHLxAtCU88lQRmQn6g9lzNP7SFpv UfRq1mRUI7fr9ntUgBtbXIu6StNGgLcSUJSuAZtMjM+DPlG2X8z0YtzM5OSsBr7IRAWpjT3hxwst/ wDEjRp+/TkQGeUNn9wwkvvXuT4lVRteAnWcfAExHEpZ3xjPa0003P2GhdKUsGqUnG1InNZ5XXxBk8 5e0y1eFQ==; Received: from [2001:8b0:10b:5::bb3] (helo=u3832b3a9db3152.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHBGr-008Am9-UN; Sun, 15 Jan 2023 22:14:34 +0000 Message-ID: <4bffa69a949bfdc92c4a18e5a1c3cbb3b94a0d32.camel@infradead.org> Subject: [PATCH] x86/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain From: David Woodhouse To: Thomas Gleixner , LKML , xen-devel , Juergen Gross Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , Michael Ellerman , Christophe Leroy , linuxppc-dev@lists.ozlabs.org, "Ahmed S. Darwish" , Reinette Chatre Date: Sun, 15 Jan 2023 22:14:19 +0000 In-Reply-To: <20221111122015.631728309@linutronix.de> References: <20221111120501.026511281@linutronix.de> <20221111122015.631728309@linutronix.de> User-Agent: Evolution 3.44.4-0ubuntu1 MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The Xen MSI → PIRQ magic does support MSI-X, so advertise it. (In fact it's better off with MSI-X than MSI, because it's actually broken by design for 32-bit MSI, since it puts the high bits of the PIRQ# into the high 32 bits of the MSI message address, instead of the Extended Destination ID field which is in bits 4-11. Strictly speaking, this really fixes a much older commit 2e4386eba0c0 ("x86/xen: Wrap XEN MSI management into irqdomain") which failed to set the flag. But that never really mattered until __pci_enable_msix_range() started to check and bail out early. So in 6.2-rc we see failures e.g. to bring up networking on an Amazon EC2 m4.16xlarge instance: [ 41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524 [ 41.498705] ena 0000:00:03.0: Can not reserve msix vectors [ 41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts Side note: This is the first bug found, and first patch tested, by running Xen guests under QEMU/KVM instead of running under actual Xen. Fixes: 99f3d2797657 ("PCI/MSI: Reject MSI-X early") Signed-off-by: David Woodhouse --- arch/x86/pci/xen.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index b94f727251b6..790550479831 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -433,6 +433,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = { }; static struct msi_domain_info xen_pci_msi_domain_info = { + .flags = MSI_FLAG_PCI_MSIX, .ops = &xen_pci_msi_domain_ops, };