From patchwork Fri Aug 16 11:08:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13765975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BE108C531DE for ; Fri, 16 Aug 2024 11:09:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.778488.1188532 (Exim 4.92) (envelope-from ) id 1seup8-0001u6-PS; Fri, 16 Aug 2024 11:08:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 778488.1188532; Fri, 16 Aug 2024 11:08:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seup8-0001tx-MM; Fri, 16 Aug 2024 11:08:50 +0000 Received: by outflank-mailman (input) for mailman id 778488; Fri, 16 Aug 2024 11:08:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seup7-0001dU-8l for xen-devel@lists.xenproject.org; Fri, 16 Aug 2024 11:08:49 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20618.outbound.protection.outlook.com [2a01:111:f403:2414::618]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e80a6c5e-5bbf-11ef-a505-bb4a2ccca743; Fri, 16 Aug 2024 13:08:47 +0200 (CEST) Received: from BL1PR13CA0002.namprd13.prod.outlook.com (2603:10b6:208:256::7) by CYYPR12MB8732.namprd12.prod.outlook.com (2603:10b6:930:c8::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22; Fri, 16 Aug 2024 11:08:41 +0000 Received: from BN3PEPF0000B06B.namprd21.prod.outlook.com (2603:10b6:208:256:cafe::90) by BL1PR13CA0002.outlook.office365.com (2603:10b6:208:256::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.17 via Frontend Transport; Fri, 16 Aug 2024 11:08:41 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B06B.mail.protection.outlook.com (10.167.243.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7897.4 via Frontend Transport; Fri, 16 Aug 2024 11:08:41 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 16 Aug 2024 06:08:37 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e80a6c5e-5bbf-11ef-a505-bb4a2ccca743 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xHu3I9i469r2xga5Dumc3PV8+yxHIvLwK5eAg9Oi7B49rB4aFT/LcjEt7Pu7LGtu3OS/Z+Oe/TruRhX2YJ45LhR0qugs42eu7LpwfELRwAwVzsuG47KBRPs8NIat3lJTW7Nm0qQCzJb6kWbArlYSUUws7dx4PMta/Kh6aoAI1RnxEqClw/w2PiGT6ZeElgvPYnvdfGcoJ4dK6oiCt/otjHCwso3Bhq9RZLv0gU1ebQ8n0RKhFgBIhmfPyaHWBX5TK+uVCRbLyU9nBNBmu3L7fWpm6FnK+bD8cXqiu0jIsTa9+gEgcIXonuk+LadvHtLFtDA7jS2hC7pTpVYkK51ayg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XyXanb/U7Jt+W/lUhXCnanELwNqOqGwqpo+XIrtjkes=; b=Sdw9hRZTThmZkSr+AQuNSSZ9mxSXmZVwBu20SruJceCVGoRF2HYDzObIARt/PMf8khq03LmIqJYsGcM5O8mgyjNBD0fYu26jeZw6D4EOK2Sv9JCoGCogQ2YN0sqdvSRuUiHwjMLzO6Ktf4UmOwqGCw2naRWDnXR/zy45FkvNi/nZI2EEMAPXVbolyM3RBiJWeINQZdjlqspcwQXRale744UsLxbrXiH1ye5KqNBTgQTjOYpDqIgkR833dAJbJyKXxhuZQMeAdIJWsTDHCEqa+qKEQuq62QqXCWDsKn9I3fVcR0xZ8Z0/5v4O1Ei47JcxfrsKh0YR7PYpdzKh4xwOMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XyXanb/U7Jt+W/lUhXCnanELwNqOqGwqpo+XIrtjkes=; b=XUkl0Glodxx/ishz0iK7lFJ+F2M93lC3NGioghtZv8kRs+eWPZopWeM5oQrT+tdJ0p6aFB/9VXaVQ4OQ+L7t9+6R5YaVhg8VlnEtTn7flDo9VQ9Uuiez1Cmn8HLv0vAml1liRPH7j6UqsRRmrKnCF/xTpeVDcR+4g/4TKhFOglY= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v13 1/6] xen/pci: Add hypercall to support reset of pcidev Date: Fri, 16 Aug 2024 19:08:15 +0800 Message-ID: <20240816110820.75672-2-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240816110820.75672-1-Jiqian.Chen@amd.com> References: <20240816110820.75672-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B06B:EE_|CYYPR12MB8732:EE_ X-MS-Office365-Filtering-Correlation-Id: 301c08eb-8243-44f4-f4e6-08dcbde3c95e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|376014|7416014; X-Microsoft-Antispam-Message-Info: daac0cXhmaJOBBkeO8RgZhNCkqQW00sqqVtKk2nuqJp8c3vA8jzSXn1w5qKNlVn2H3aKK/yDhzu2zSnfKdmj5ut4qmfN0898xJ+s9mitma/H4fNTGzR44NL/IS5m1LoAevKrLTS0XNL0SzQYhl0aKRn7e4uMPe2219nB5aNVwffG94AUC/YNjj8+8SEWsOl7V30uQe7zADL//TRqDxp9BSrV/0WwDPihDoFYyxXQYdadGEKtODQfMF8MFafZCINlunBZ5yRZ0Sm69rhwF6LnKzZFGE9g/6OOrXeGGn8lXBFOY24fp5//oWOwbl4phOG1+KS+mFAHfRUtvyR2PT1zLTAy1Qy9zpJCyALdWkDQYUVV+DnLWyuX4Hx82waNLEqmip3Z7ZPeLzc3P4HlazOwWxrd+pFBBBv1L9wSiJxfZSKUNWERzd7Lq0ngJ6zCQyIZPPnukqbW+Po1hAmtBgRuO6UJ6o1LA/t9w+E/yYqd9VD5j9FYDdv/xr971R9lCA0anjyr8c3CdgMEZFbNNB4e8qzolrMSlQalck8QluK5Gc4Iwjd1QjLsBZwwZD+HSu77L5gwr3LZj4N7f9sRORdOczFflc3+1LVVLNSOu5/fJariWGs25bCKbMM25LNDfTtaoeCOnmftrOri0fU17lQrX68VE+r3saSMugqTeDoGkqtHL6ES5H/39rnkRxScYkw6bOWld8ZyBzr3i/DGLG+c2Ao3y4/IUnRS8rSblQ3e/rSOJiNiNs0QsXp6AE1Q39HgAdS+aSCpuObwlxl9u7525oCpwQ3OIdhqXG8ZJEX3SuQsVendmy5ZB7fl4fR1V3wttPk1z5pBFEgC3DckBD9iVKzsMZmeCYlZAoAAriGu4IDJMuvh63NJgTFYaQDN1v4jXjiTOYVODqtx53pjiTMlONpnm6zZ5XceZko3Geij62WhEKIhXJ7YEV5xtN4mtqujTS8gaXdbnnLj2x8o8cZ8E6AJlFJO2tKcWpGP62cmimcdKc69MpM54AtD21G48QCnTzrotRCVXhKe66XDPYW46U+uT8kYmQJQ51q8ry+Y7LfJ0U30G0WA9n9DBBhmbd/UPh6o88I+1Q9LJZhP8nl1ZDLz7JW55XYWxzKJwR4KDP6y67Qvt+fY3ppb+8pdGm+oe6Mc02QIyYYg31wDvr6IxKjzv4H7O2zQr05CVA3IXkWlRFHoUV2EvKg/y6R9TvpjWzN0BmIu4Av81dJbBP7ryaZh5jZkY+U0L++G9X0DVJ7Z6FiFxrKXfPv2WuEcA9FEJvsk8Q25/kfwIdQG/flMmMD0duR9Ssi/3swbL4Jg9Yvg2KA0v8+2oSAjk4CFmNLhAc1KVkhlnjuerEDfXYpQnSEub3E2pmVOJF9SNo/bef6BEot1BLtaiNDuOHkkvM78PZ3dX2Gym31+2x1b2/aXqWl5J5ZGRRMRNnW/uMbPAJQJJhY3kV92nX5mKuS8+Xa5 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2024 11:08:41.2655 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 301c08eb-8243-44f4-f4e6-08dcbde3c95e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B06B.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8732 When a device has been reset on dom0 side, the Xen hypervisor doesn't get notification, so the cached state in vpci is all out of date compare with the real device state. To solve that problem, add a new hypercall to support the reset of pcidev and clear the vpci state of device. So that once the state of device is reset on dom0 side, dom0 can call this hypercall to notify hypervisor. The behavior of different reset types may be different in the future, so divide them now so that they can be easily modified in the future without affecting the hypercall interface. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen --- v12->v13 changes: Deleted all "state" words in new code, because it is not necessary. Deleted unnecessary parameter reset_type of function vpci_reset_device, and changed this function to inline function Added description to commit message to indicate that the classification of reset types is for possible different behaviors in the future Renamed reset_type of struct pci_device_reset to flags, and modified the value of macro definition of reset, let them occupy two lowest bits. Change the function vpci_reset_device to an inline function and delete the ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); because this call exists in subsequent functions and it accesses domain and pci_lock, which will affect the compilation process. v11->v12 changes: Change the title of this patch(Add hypercall to support reset of pcidev). Remove unnecessary notes, erroneous stamps, and #define. v10->v11 changes: Move the curly braces of "case PHYSDEVOP_pci_device_state_reset" to the next line. Delete unnecessary local variables "struct physdev_pci_device *dev". Downgrade printk to dprintk. Moved struct pci_device_state_reset to the public header file. Delete enum pci_device_state_reset_type, and use macro definitions to represent different reset types. Delete pci_device_state_reset_method, and add switch cases in PHYSDEVOP_pci_device_state_reset to handle different reset functions. Add reset type as a function parameter for vpci_reset_device_state for possible future use. v9->v10 changes: Nothing. v8->v9 changes: Move pcidevs_unlock below write_lock, and remove "ASSERT(pcidevs_locked());" from vpci_reset_device_state; Add pci_device_state_reset_type to distinguish the reset types. v7->v8 changes: Nothing. v6->v7 changes: Nothing. v5->v6 changes: Rebase code and change old function vpci_remove_device, vpci_add_handlers to vpci_deassign_device, vpci_assign_device. v4->v5 changes: Add pci_lock wrap function vpci_reset_device_state. v3->v4 changes: Change the comment of PHYSDEVOP_pci_device_state_reset; Move printings behind pcidevs_unlock. v2->v3 changes: Move the content out of pci_reset_device_state and delete pci_reset_device_state; Add xsm_resource_setup_pci check for PHYSDEVOP_pci_device_state_reset; Add description for PHYSDEVOP_pci_device_state_reset; --- xen/arch/x86/hvm/hypercall.c | 1 + xen/drivers/pci/physdev.c | 52 ++++++++++++++++++++++++++++++++++++ xen/include/public/physdev.h | 17 ++++++++++++ xen/include/xen/vpci.h | 6 +++++ 4 files changed, 76 insertions(+) diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c index c1bd17571e47..68815b03eb25 100644 --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -83,6 +83,7 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) case PHYSDEVOP_pci_mmcfg_reserved: case PHYSDEVOP_pci_device_add: case PHYSDEVOP_pci_device_remove: + case PHYSDEVOP_pci_device_reset: case PHYSDEVOP_dbgp_op: if ( !is_hardware_domain(currd) ) return -ENOSYS; diff --git a/xen/drivers/pci/physdev.c b/xen/drivers/pci/physdev.c index 42db3e6d133c..980ff1ba3d07 100644 --- a/xen/drivers/pci/physdev.c +++ b/xen/drivers/pci/physdev.c @@ -2,6 +2,7 @@ #include #include #include +#include #ifndef COMPAT typedef long ret_t; @@ -67,6 +68,57 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) break; } + case PHYSDEVOP_pci_device_reset: + { + struct pci_device_reset dev_reset; + struct pci_dev *pdev; + pci_sbdf_t sbdf; + + ret = -EOPNOTSUPP; + if ( !is_pci_passthrough_enabled() ) + break; + + ret = -EFAULT; + if ( copy_from_guest(&dev_reset, arg, 1) != 0 ) + break; + + sbdf = PCI_SBDF(dev_reset.dev.seg, + dev_reset.dev.bus, + dev_reset.dev.devfn); + + ret = xsm_resource_setup_pci(XSM_PRIV, sbdf.sbdf); + if ( ret ) + break; + + pcidevs_lock(); + pdev = pci_get_pdev(NULL, sbdf); + if ( !pdev ) + { + pcidevs_unlock(); + ret = -ENODEV; + break; + } + + write_lock(&pdev->domain->pci_lock); + pcidevs_unlock(); + switch ( dev_reset.flags & PCI_DEVICE_RESET_MASK ) + { + case PCI_DEVICE_RESET_COLD: + case PCI_DEVICE_RESET_WARM: + case PCI_DEVICE_RESET_HOT: + case PCI_DEVICE_RESET_FLR: + ret = vpci_reset_device(pdev); + break; + + default: + ret = -EOPNOTSUPP; + break; + } + write_unlock(&pdev->domain->pci_lock); + + break; + } + default: ret = -ENOSYS; break; diff --git a/xen/include/public/physdev.h b/xen/include/public/physdev.h index f0c0d4727c0b..3902723ce1db 100644 --- a/xen/include/public/physdev.h +++ b/xen/include/public/physdev.h @@ -296,6 +296,13 @@ DEFINE_XEN_GUEST_HANDLE(physdev_pci_device_add_t); */ #define PHYSDEVOP_prepare_msix 30 #define PHYSDEVOP_release_msix 31 +/* + * Notify the hypervisor that a PCI device has been reset, so that any + * internally cached state is regenerated. Should be called after any + * device reset performed by the hardware domain. + */ +#define PHYSDEVOP_pci_device_reset 32 + struct physdev_pci_device { /* IN */ uint16_t seg; @@ -305,6 +312,16 @@ struct physdev_pci_device { typedef struct physdev_pci_device physdev_pci_device_t; DEFINE_XEN_GUEST_HANDLE(physdev_pci_device_t); +struct pci_device_reset { + physdev_pci_device_t dev; +#define PCI_DEVICE_RESET_COLD 0x0 +#define PCI_DEVICE_RESET_WARM 0x1 +#define PCI_DEVICE_RESET_HOT 0x2 +#define PCI_DEVICE_RESET_FLR 0x3 +#define PCI_DEVICE_RESET_MASK 0x3 + uint32_t flags; +}; + #define PHYSDEVOP_DBGP_RESET_PREPARE 1 #define PHYSDEVOP_DBGP_RESET_DONE 2 diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index da8d0f41e6f4..41e7c3bc2791 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -304,6 +304,12 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v) } #endif +static inline int __must_check vpci_reset_device(struct pci_dev *pdev) +{ + vpci_deassign_device(pdev); + return vpci_assign_device(pdev); +} + #endif /* From patchwork Fri Aug 16 11:08:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13765974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D1FCC531DF for ; Fri, 16 Aug 2024 11:09:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.778489.1188542 (Exim 4.92) (envelope-from ) id 1seupA-00029A-3I; Fri, 16 Aug 2024 11:08:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 778489.1188542; Fri, 16 Aug 2024 11:08:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seup9-00028z-VG; Fri, 16 Aug 2024 11:08:51 +0000 Received: by outflank-mailman (input) for mailman id 778489; Fri, 16 Aug 2024 11:08:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seup8-0001dU-8m for xen-devel@lists.xenproject.org; Fri, 16 Aug 2024 11:08:50 +0000 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2060b.outbound.protection.outlook.com [2a01:111:f403:240a::60b]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e939a468-5bbf-11ef-a505-bb4a2ccca743; Fri, 16 Aug 2024 13:08:49 +0200 (CEST) Received: from BL1PR13CA0026.namprd13.prod.outlook.com (2603:10b6:208:256::31) by CH0PR12MB8552.namprd12.prod.outlook.com (2603:10b6:610:18e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.17; Fri, 16 Aug 2024 11:08:44 +0000 Received: from BN3PEPF0000B06B.namprd21.prod.outlook.com (2603:10b6:208:256:cafe::c8) by BL1PR13CA0026.outlook.office365.com (2603:10b6:208:256::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.10 via Frontend Transport; Fri, 16 Aug 2024 11:08:44 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B06B.mail.protection.outlook.com (10.167.243.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7897.4 via Frontend Transport; Fri, 16 Aug 2024 11:08:44 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 16 Aug 2024 06:08:41 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e939a468-5bbf-11ef-a505-bb4a2ccca743 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TimHHGckCdCnqkPOd+819TWZqyRgz7zp7zDGwKU5wBLivYHtBKKsV7U4ouN5FjvBH4vbofwUaQCkKi8ny86RDbn9Kn2l7R2y//LjfjJj9ZIqYRk77lczXGfXjAADQ1ogQtr2fCwOPSZ3r+vHZrs57MQUBQJRXiMxIq0XL+VffsG7crMvm/ZK2KJ/rss+P9/rEtd7hbl4PbzMZfVcvi00dVa4gttshVyeNzHLpAVE7gtyv/lK4jaefbMW4wlGvASka+GnqMSf201lRPezokQn8BI2jDiSOzwdVSh2WxBmQjibt5OWOEemjVWuWubDJqOuyM8ltYfm2/iQSG/lEfuVHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P2Kh0oS3w0hJmzs1gUO8zu7HPWkBa8kExEr9EreyX/M=; b=AB+a08wkhc8qgNm41yZh6TtKM3qLjTNd/jFsSz6kS1CWbMBilgwGQtfhS/rNFxba4N9oFEpX4Ptzyn49Peqrk4FocgbSzBRKq1CCFmNWy94yZpw3SeRMx3c9RGHFZkwcZ4fbehCKWVv+LnWNu4YLtuVBba9ieD30B7aPKTaetztkbms3RdAPXCBjya7n46nKwwToNCMK4vZs1YQlUe70t6cWWwmGgIkp6a1n2/lnkXq7NPS8rR/4bqfsZ0rMDe2BgaJsNoS4r6Je9WHJggYAyzqBSFDkjBXE8f1d6vHfYKbKqW8DSTY1DRb+eyODrkQyU9D4jplcKVjtcUP979hduA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P2Kh0oS3w0hJmzs1gUO8zu7HPWkBa8kExEr9EreyX/M=; b=x8FDEDL0oYUF5xAVSjUa/RSWgbFLpJz42koa7I5sxQXhyobQszPc39kkQE1oB+3MBlev+q6D65mEcZEUerNJzt21j+zDLOQ/Zk+QFM0/OAt7OiBirVLcx0IGTFA/C+IBT7ScpCpR9YLwgiAyQVXDUknyL1Jf6qnEoD6KpTO3S8k= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v13 2/6] x86/pvh: Allow (un)map_pirq when dom0 is PVH Date: Fri, 16 Aug 2024 19:08:16 +0800 Message-ID: <20240816110820.75672-3-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240816110820.75672-1-Jiqian.Chen@amd.com> References: <20240816110820.75672-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B06B:EE_|CH0PR12MB8552:EE_ X-MS-Office365-Filtering-Correlation-Id: 54c187cd-78f4-4756-c9bb-08dcbde3cb5a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|7416014|376014; X-Microsoft-Antispam-Message-Info: tu3Jm9nEg3LtFbx8a/fOOLx2jKTNmqAzAPM60wO4wrmvwKi1yv2Nx/yoUiIasC1qw+4G3TtxlwgRD3XJ/6rKEDf9JshQhCrgn5OmJROz+/pRMYsHuqbcrqk9EHjF8Cl4fH82C6MD7iSqkQlBADJfBEGsq61+rc4Xeccm5fz76yzPSVev40q9CfjbbyvpIZ5ITs1J/iyy97WToMVmu9dp/UkW3AsYWwam1DH3aof8AxdlOeCLhjPNBVE7gNf/NlH+9jAiyqxH2v0IoGizuEMXVpTHFoAcMyW3hR0w+htPTcaGwVWWzFV/eQohYMWlExGMRz5RdN3JAOa3vNJWFzSbCoHU6MzHmWRDRMKAuWIJcOquZswRPoiCv7WprsA59njBsgXOQx6Acsle30q+KvLjNRHvgapkSwnfHIDgcteSFSd87cCoIj7o656H/sMjiH13kE3ewmBVbYygPxJzZ+NaElB1lNPwoo6e96YrLWrErYLqy+RVggiJrau4pDj+c/pcahMQVihU1nYCQqWTcfn3UGqOV2VtsM6Xf3ItRMGM1Bv5hw4Bh65AeInlgerwxPafNKlevodtuGcMwMHdAs+f6Hy+/ENmXA0iByiyxVvduV0cnZeil+i5xLA5tv8ZA5upq8KeV9Ltx11/J1FS+dBqDpVaayA9KXnSuIXbbXw7A/GtoqtI25UVHrCYdppoIu/YC3+sFptwlxKhAtWkq5xkTuf1RHGhXufSFuA3pb3E8THjRauOZN1WfLU4FLAqxdMcsG4Q2pDaO1KKlssplSnN/ZvtigDnkwqm61TuuxQ4floBmKLKecM0YqQpU7ko57Dhr2xKSf5wqUzaq8XRnE9nhA32Aytr6QEVokVfpXSjPua35FEQjqDcJHtL/eN1p7HVDOJWeao2A6L1w/X5ASgTtpBtrj1yjkX55jHdFRbEk6hsgQ5Tcm/CZzhShKSUiCk5E7RKmr6B+pA8aEjul8NcCke6mbDGGZjvWCIlZ2z6eBBxt/BSjk/GOfMDCW1Hx9No7ukbB6atdBH8cNcieH3UZl16BXAJcoGoN98kq821yV59nLzBPhcBDs2lN9wCvH4SkGezF3m91CbGUNruTW+7P/63ZDpoaaBMFmaTVpd3u+i+F6HOGzxRDqoZ4kILHNcQIep+2P1tfzpFK9i8HRzg8cvYlmhe/Oi0bmVdOGa/Ztr/oWPMLx7qW/kkMcF7T1Pxh/StJHoCYKwohejAruqgH+J7uAybedzQX2bUcw8jl+v9XRby+uG9FVYRPocPA2+YBcTQvdwyABdUCcCbwwa1LYRdbl/ZsUjXsh4gdBH912JGB3xTu9u3R3KKrO4QxzYDAx0MvpGyhuzZZTfCqA9r42tOomZsqvL/Ba3l9r9IzKzCKxjbtmED/IETdAxDUF7GpVyBD/aqjyiBB/YaEPrKhPO08VslpAiFmyR8dcYXwITmkbpoe6FVCn8Q4DgceFzM X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(7416014)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2024 11:08:44.5937 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 54c187cd-78f4-4756-c9bb-08dcbde3cb5a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B06B.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB8552 If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for a passthrough device by using gsi, see qemu code xen_pt_realize->xc_physdev_map_pirq and libxl code pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq is not allowed because currd is PVH dom0 and PVH has no X86_EMU_USE_PIRQ flag, it will fail at has_pirq check. So, allow PHYSDEVOP_map_pirq when dom0 is PVH and also allow iPHYSDEVOP_unmap_pirq for the removal device path to unmap pirq. So that the interrupt of a passthrough device can be successfully mapped to pirq for domU with a notion of PIRQ when dom0 is PVH. To exposing the functionality to wider than (presently) necessary audience(like PVH domU), so it doesn't add any futher restrictions. And there already are some senarios for domains without X86_EMU_USE_PIRQ to use these functions. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen --- v12->v13 changes: Removed the PHYSDEVOP_(un)map_pirq restriction check for pvh domU and added a corresponding description in the commit message. v11->v12 changes: Avoid using return, set error code instead when (un)map is not allowed. v10->v11 changes: Delete the judgment of "d==currd", so that we can prevent physdev_(un)map_pirq from being executed when domU has no pirq, instead of just preventing self-mapping. And modify the description of the commit message accordingly. v9->v10 changes: Indent the comments above PHYSDEVOP_map_pirq according to the code style. v8->v9 changes: Add a comment above PHYSDEVOP_map_pirq to describe why need this hypercall. Change "!is_pv_domain(d)" to "is_hvm_domain(d)", and "map.domid == DOMID_SELF" to "d == current->domian". v7->v8 changes: Add the domid check(domid == DOMID_SELF) to prevent self map when guest doesn't use pirq. That check was missed in the previous version. v6->v7 changes: Nothing. v5->v6 changes: Nothing. v4->v5 changes: Move the check of self map_pirq to physdev.c, and change to check if the caller has PIRQ flag, and just break for PHYSDEVOP_(un)map_pirq in hvm_physdev_op. v3->v4 changes: add check to prevent PVH self map. v2->v3 changes: Du to changes in the implementation of the second patch on kernel side(that it will do setup_gsi and map_pirq when assigning a device to passthrough), add PHYSDEVOP_setup_gsi for PVH dom0, and we need to support self mapping. --- xen/arch/x86/hvm/hypercall.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c index 68815b03eb25..0b7fc060b4e2 100644 --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -73,6 +73,8 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) { case PHYSDEVOP_map_pirq: case PHYSDEVOP_unmap_pirq: + break; + case PHYSDEVOP_eoi: case PHYSDEVOP_irq_status_query: case PHYSDEVOP_get_free_pirq: From patchwork Fri Aug 16 11:08:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13765971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48591C531DC for ; Fri, 16 Aug 2024 11:09:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.778490.1188552 (Exim 4.92) (envelope-from ) id 1seupG-0002WG-CJ; Fri, 16 Aug 2024 11:08:58 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 778490.1188552; Fri, 16 Aug 2024 11:08:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupG-0002W3-97; Fri, 16 Aug 2024 11:08:58 +0000 Received: by outflank-mailman (input) for mailman id 778490; Fri, 16 Aug 2024 11:08:57 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupF-0001Oc-92 for xen-devel@lists.xenproject.org; Fri, 16 Aug 2024 11:08:57 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20610.outbound.protection.outlook.com [2a01:111:f403:200a::610]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ec9afd98-5bbf-11ef-8776-851b0ebba9a2; Fri, 16 Aug 2024 13:08:55 +0200 (CEST) Received: from BN9P221CA0005.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::28) by CYYPR12MB8750.namprd12.prod.outlook.com (2603:10b6:930:be::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.20; Fri, 16 Aug 2024 11:08:49 +0000 Received: from BN3PEPF0000B069.namprd21.prod.outlook.com (2603:10b6:408:10a:cafe::4a) by BN9P221CA0005.outlook.office365.com (2603:10b6:408:10a::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23 via Frontend Transport; Fri, 16 Aug 2024 11:08:49 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B069.mail.protection.outlook.com (10.167.243.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7897.4 via Frontend Transport; Fri, 16 Aug 2024 11:08:48 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 16 Aug 2024 06:08:44 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec9afd98-5bbf-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Fp/dq7IrhwiDsx/qUTdgRxH4Nqmdm8q/w5WTnN+n3sJgQw95LC9Y25ZkDZ6gH3TgGTUtCXmgVZkqw82lCA6EaGrYMJCnQzELO0njQxFVmP1hK+nM4ydLmvR6Vi/ay1kh596EcfLZOSUiCz7u0riQZq5O34SDdiR4rOA41HvWE4PZDRKUoXNS2HpOxsEnb8muf8aIZR3fk+oFrPX0IRplUTDIXDesDteY4i6+2C6mGuGMxf5M7olfTq8j6t+2tjLomNyyvEwfoJS/UfQ2wkFVTAeAIARKnhZFQ3dPd0qKfvHXCz2hLrEEIOEAcauGe0dsXOo9FjNC8jajCwMt0mJGcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QEmijQKrvaaonY0ryNFeoICShxLiMC7FhSWe8KUC0mg=; b=VPcTrTBgQ2G+7E0KiDIhL2kIfTx+0L7KW4snaC5FYsLhMsEF8ioJ5NahXjktUkefZlo0Mej1wgg96FUjIPoo9gGqp3ZHsXPcomGxfzj9PfFnIiM9XwMHNixvAfQSaJfQl9Y8X/hi11GU87NXUS/uv4RPjFt3xW2vAGxcxpL7vonLTKRZQaEubBHr8ljHztsuicsGCv7vrdd0/i+AfNUj+/MaMdo+u0PXeFFaGdkvufOtYf3OZzxPQM7QtMhO2MBEtNScd5J9OXZ6LX1NTajQbi9ljXM6LIf/QHbTYRF8//4bzFHj56+MzlvlBvAaT1D76nNcACWfPrKLr3i+iLRv5w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QEmijQKrvaaonY0ryNFeoICShxLiMC7FhSWe8KUC0mg=; b=zeNgXRolwhjky5XuvT27UfM8gPkhwPgxHlJDW1CT0K8DKO43/zCcl26/lcHq1xehdhc5avKkEWPQkgX3Kg2wRHcnivwCW+dpduEb8C6SsEOvp7z8fYvtTK+fLspfpIrGlhDG4OoDhsJakBojUfY8x5NGgcPbKNX51xVebesDVk4= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v13 3/6] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0 Date: Fri, 16 Aug 2024 19:08:17 +0800 Message-ID: <20240816110820.75672-4-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240816110820.75672-1-Jiqian.Chen@amd.com> References: <20240816110820.75672-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B069:EE_|CYYPR12MB8750:EE_ X-MS-Office365-Filtering-Correlation-Id: 91041b6b-1b0e-4d14-68d8-08dcbde3cd64 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: wL+0lLg6AKsqXM1d+3yJcRlTxYamDeEiureb6JxOJTEtELTaMrOudw5NOunJ5gcs5iimbCrovCEGXjm7+6RKLpBcgA2HLUDDNnATASs3ADN80zEykhVimZXyxNTZZ1jAqZ+KIekExBaDF+WeHXVfXQ2BE/DDahmaNaFkI3bFCBNyEsGQbrRZ/Ki2BRY2Lcu0pi2KrcYUlE/yU4cy6c3bbbyY0EpFAMP+mzZg5qcW8FXFXqH0zYx0TThTCA3XuInlxRuuLiYprNvODz/7RkZtUj/TGqjwJW5X1BNrcSn/ss/GpO/EblPb2RCH8GlfLwNZqcbC3dfRO//saMM/Onkgg5ViwFtPo31bH+XZWYKOOjRvhV0ClV6l62xwnmqI+VhijwATQwKDR/A/2yl3UoqWzbLD5xLO5m7cpiZ7OQrMRbGD9D171BrZEIojI+XkgzJqurqlYU51QZPvDCNhXa9pcayZtfn6wQ4aGwAXle/5djZHjWvP40Awvi2t7m1I56GxUPdmutDwUo/RGYI4s60FT6ucXsrrumDE3h+Bdzrnzl0v+HMKDpQq5q1k3N8rhbFcFMjUSxdjtFylvH6HbWf1qk5GZtFjlyDws3Ni5rv4gnZTh3ipqWnxzYa31903ir7npVbbuhpchohghLxegJgcD9um/jSmzlCnS1cfKHjXd6z7tf35XsXgI9BC/SFZmTXRbRDyt4P2jD7pigJmt19yJHqBTYXA+gRnaylD+v2Ouoq2/bp8eN0tZq51tRC/BBgNPXkZSveC+0z6ghRruoAgSsIwDm+VqLxEWshO0NAsg0ZhUn/dGGILcJPVI2d2A7KabXlVT4zxs9LO1vKgS4aGWcMnp0yuI6w/7remxtQwH2B/kYZdeVrkDmDRFrFvdmTI0j7WInoZvVX3/MyopDdySujRhKHN4XgVu1w7ty0SdFz2+sk+1Sl4SL8T+XoYG2vJBEEiO0c2DqQWrXbqq1f+0PS1ZeCgeEBUvblvP71FcHIxbi8Q5+PKNQz2By+uIaUGYLEMi/zTyouj4icWmAGi5d/GMKj+TDCcdbQWnFgZmGlb/+gr1XPrmh88FP1NPZDUWhWD3rHCZARdbU6uRDVAFY/W5idjDEiDAGYzdDZQtsUxeH5ZX606FmZZf8UHzVEjk+LER1p/5Emug5RE4KoYOD4hAbSrpg0eb9AennGPulN/5XPrcQK0w2M3v259jKBjH4dyfM3SZPb6KRlVZIgN2ttpk2CYa3W5b1YTj+Qv3WLsFINxustum0pmApA5gQs10OKsCgAQYhxPeapZwGxrpnwhFCFX7pkJtq0sF8a+JSsLIe06SrJ7yy3yyT2BnondfMhXw/9T63yNDyrK/uFDCeAeslLO4HjwLFFCP0psOzY50FgD7C6bPr3Nfw8eT5whPIa4tM22054BXg3Gqc64NQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2024 11:08:48.0122 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 91041b6b-1b0e-4d14-68d8-08dcbde3cd64 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B069.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8750 The gsi of a passthrough device must be configured for it to be able to be mapped into a hvm domU. But When dom0 is PVH, the gsis may not get registered(see below clarification), it causes the info of apic, pin and irq not be added into irq_2_pin list, and the handler of irq_desc is not set, then when passthrough a device, setting ioapic affinity and vector will fail. To fix above problem, on Linux kernel side, a new code will need to call PHYSDEVOP_setup_gsi for passthrough devices to register gsi when dom0 is PVH. So, add PHYSDEVOP_setup_gsi into hvm_physdev_op for above purpose. Clarify two questions: First, why the gsi of devices belong to PVH dom0 can work? Because when probe a driver to a normal device, it uses the normal probe function of pci device, in its callstack, it requests irq and unmask corresponding ioapic of gsi, then trap into xen and register gsi finally. Callstack is(on linux kernel side) pci_device_probe-> request_threaded_irq-> irq_startup-> __unmask_ioapic-> io_apic_write, then trap into xen hvmemul_do_io-> hvm_io_intercept-> hvm_process_io_intercept-> vioapic_write_indirect-> vioapic_hwdom_map_gsi-> mp_register_gsi. So that the gsi can be registered. Second, why the gsi of passthrough device can't work when dom0 is PVH? Because when assign a device to passthrough, it uses the specific probe function of pciback, in its callstack, it doesn't install a fake irq handler due to the ISR is not running. So that mp_register_gsi on Xen side is never called, then the gsi is not registered. Callstack is(on linux kernel side) pcistub_probe->pcistub_seize-> pcistub_init_device-> xen_pcibk_reset_device-> xen_pcibk_control_isr->isr_on==0. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen Acked-by: Jan Beulich --- xen/arch/x86/hvm/hypercall.c | 1 + 1 file changed, 1 insertion(+) diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c index 0b7fc060b4e2..81883c8d4f60 100644 --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -82,6 +82,7 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) return -ENOSYS; break; + case PHYSDEVOP_setup_gsi: case PHYSDEVOP_pci_mmcfg_reserved: case PHYSDEVOP_pci_device_add: case PHYSDEVOP_pci_device_remove: From patchwork Fri Aug 16 11:08:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13765977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59167C531DE for ; Fri, 16 Aug 2024 11:09:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.778493.1188562 (Exim 4.92) (envelope-from ) id 1seupJ-0002st-Qp; Fri, 16 Aug 2024 11:09:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 778493.1188562; Fri, 16 Aug 2024 11:09:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupJ-0002sk-My; Fri, 16 Aug 2024 11:09:01 +0000 Received: by outflank-mailman (input) for mailman id 778493; Fri, 16 Aug 2024 11:09:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupI-0001Oc-Cl for xen-devel@lists.xenproject.org; Fri, 16 Aug 2024 11:09:00 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20618.outbound.protection.outlook.com [2a01:111:f403:2414::618]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id eee73b25-5bbf-11ef-8776-851b0ebba9a2; Fri, 16 Aug 2024 13:08:58 +0200 (CEST) Received: from BN9P221CA0005.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::28) by SA0PR12MB4493.namprd12.prod.outlook.com (2603:10b6:806:72::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.20; Fri, 16 Aug 2024 11:08:52 +0000 Received: from BN3PEPF0000B069.namprd21.prod.outlook.com (2603:10b6:408:10a:cafe::42) by BN9P221CA0005.outlook.office365.com (2603:10b6:408:10a::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23 via Frontend Transport; Fri, 16 Aug 2024 11:08:52 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B069.mail.protection.outlook.com (10.167.243.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7897.4 via Frontend Transport; Fri, 16 Aug 2024 11:08:51 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 16 Aug 2024 06:08:48 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: eee73b25-5bbf-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Qgd8BIPxS3E37iV3X9XYhiLJShr4gyCvYg5c0s9tJacPQqA7KXJ161Payx6202qJK619f/Y+42FGLAj0tJf9iccRpeK5JoxtPEbMdOWK3qC1rYO5bdmRm146WwCKaJLpLYG9Tbd9ALL1iqEsyMtsyycYjRGWzRSlZOTqNCWVGdt6KOZ1jCMmlyHivtV9YvIJabdV6tdx+RIAWOdX/glHH2nkrXyrjjG9GA9XcsD4irtFbi60M/PisccfRtOLQ8KldcIKCBywABhCQT554YVNXhtXtx4Qmq3NBRig+hegb8daqM5fy5Q6Y8cwUOdRuJ8WGLVKVqHdr1VTDmjGmkogOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Kbo5f+6yaKRR0HDxkNq97Qgqo1a26fSHS6Hkqz1R/2w=; b=yZqmgSWzcPk0/se63zYlgeLunsT6Puji+RW52cw7uTDwBKAWYdOcj7rdyMAkPZ/OwtfS+6/uU5lVYcLANy2QVbWvT8BwtG+Xa33ylUGfKx0nljfbmT2G0WETmNWuWqoLXNf5axWU4dvtvTwvZhAVfs3QpeJ9nzSWpGeyWBwfjKJEY7HpEm/G0l+LLFyZgv51WJjVt1UbTPAsULoXgl74rBYVRKHRVkzXdMkt9vezYY2jFVBay/TfjykwLrGJHyyu51ItT0TZdQR8rr9KTAuksbtl5MAktGx4VE/AFyeidQW29zTf6YVWa1iJvKw6Lx8viQ+qPw21+vQ1eL+E2+OATA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Kbo5f+6yaKRR0HDxkNq97Qgqo1a26fSHS6Hkqz1R/2w=; b=nJVB8lIdkqzdsKoPY0A3+ft27xMOaJLJCdUirUkVn85VKGMrmfipGgGhW/t2NzJQ8bp3qHJe81mztVwM5pWQVoIXlZChy3y1U9uuoNIK37X3w/ThuAwp4p679DzXkkN83Hi/m/tc2/EeyZ6Q7k13hwyVu03vjiFLbqhGwiNvyAk= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v13 4/6] x86/domctl: Add hypercall to set the access of x86 gsi Date: Fri, 16 Aug 2024 19:08:18 +0800 Message-ID: <20240816110820.75672-5-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240816110820.75672-1-Jiqian.Chen@amd.com> References: <20240816110820.75672-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B069:EE_|SA0PR12MB4493:EE_ X-MS-Office365-Filtering-Correlation-Id: 64dbaae4-2342-4785-d4c1-08dcbde3cfba X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|7416014|82310400026; X-Microsoft-Antispam-Message-Info: 3+RRkz2tvkLib2uIdS5eL+Iv+tDdgJs++78C8ohwD5mwqOwI0OguDefoHbB4cQv9eodf6AOv6Yp1YqXS2RzWn4l+Ujb5X031bO9tG0FqWSDEBAODQR7KCmpsmjf9zn/v55ed1dRqo8qW45GMGpo5tI/6M1cq8HYYWaGu9vXaM2Jx7dLrLRDrelmMgONXgUoK3JaIoi1OmI9pg48YfnDaTCYz2hfCJ+D795hOcUjP11bjeqcTiTuUXi6qR/NDK/B9BmI68+N66XxGubjORyES+gpZNW632yhRFflGzStVLIuwZg0dhnVbiq+bFUOIXxGC1/JlEGHMAP6za6eVH8sIEARTib8la1QwcEEVByaF0ctxrvXBhYfiDdoGORUBvtwvXWFQ5i8IimAjVcA8BoiPvWuqfLNRo7EXufR7SrQnMLUjjjN3r7CAXsTj+ng0+NjwyMPEN+Eedr1YDrQqqmEBDxO98x5QyvTv7Q9hEr0kIxNcfQO7tLEeIVXysswLvU6YLAvLA9flCW4kEcfx2N9uRxe5lSsMotWS7vbuLrPGA1+QzNwGPKTPKvDKoG5PbRT0O1PAX7QXvlqTEfC1YhKjoS69GV13J+RfJi1zXNf+SX4GqneXfrzeYjcMPPPsncjNm3ARi2pjL0klNLn3ruYPcRh48wLo3h8eT6vpaybn8/lP+w4s++bIecl0W6kbJA3oP0bRbtA8w4DRUfwEvh5Ozhot/a3K+gtCvDqeLLF8vdFkbP+UQt28COUhJXmL3JYxAU31cbqmUgIIBfRguWhQ7uTDj4AWT6sDBKb8c8WCvRDQ7RzTdnYwpOSwJkzq3DYccjrDFB1xJFcXIkVYxpmpw7r65EJwymlE1X1vfdLv/IO3+IekjfC/07AsLxz1xr/2yA/G3V98UYlcjSYBtltcD4rDqumU1K2kew9ORp8E5x95su/R5sHJ2A03fBlxe0mjdF6bhS++yMmVYjz3glqVZnRrQp1EWxkdvDNcS/lQ/6IhMCp2PkqyeZQ7d2g3tJVbr8fOCXsedeIcLeu8SxKem7djoWOQjksS+mDxkr/l/60M8eC3NvxkYgddKrraLuOunV1p3zFx386Mmi45dTFilga8NWpWQLDGnr7vMYVPjBoxjt1/Lw+YvE2Nglpfprn2Tq8UzEUhJN6/B9nA1M4dR/T5IrnafzlcSRUPzMewYsaaEqHcxXjH0m0bxMpuoB8JhvdkoQ1ziZDHSDJu6J/qVoccDxTViBnu6J2qapnyDAXsAg2s71CxzdNb7VxsOLw/l69bOJRveMiyv6nzi+Odqs4UNXVNoEiDcHupu3x/MHM+kBAumbglw1BMsownIo7v94cgMLSenfODTDcR5LN4mrEeGdWytNfsaV+f7VR3uffWM/vmmKeqeBDIr+JKBeNkQ2c8aBzmO2KcOuJ2sJqzRw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(7416014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2024 11:08:51.9496 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 64dbaae4-2342-4785-d4c1-08dcbde3cfba X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B069.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4493 Some type of domains don't have PIRQs, like PVH, it doesn't do PHYSDEVOP_map_pirq for each gsi. When passthrough a device to guest base on PVH dom0, callstack pci_add_dm_done->XEN_DOMCTL_irq_permission will fail at function domain_pirq_to_irq, because PVH has no mapping of gsi, pirq and irq on Xen side. What's more, current hypercall XEN_DOMCTL_irq_permission requires passing in pirq to set the access of irq, it is not suitable for dom0 that doesn't have PIRQs. So, add a new hypercall XEN_DOMCTL_gsi_permission to grant/revoke the permission of irq (translated from x86 gsi) to dumU when dom0 has no PIRQs. Regarding the translation from gsi to irq, it is that if there are ACPI overrides entries then get translation from them, if not gsi are identity mapped into irq. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen --- CC: Daniel P . Smith Remaining comment @Daniel P . Smith: + ret = -EPERM; + if ( !irq_access_permitted(currd, irq) || + xsm_irq_permission(XSM_HOOK, d, irq, flags) ) + break; Is it okay to issue the XSM check using the translated value(irq), not the one(gsi) that was originally passed into the hypercall? --- v12->v13 changes: For struct xen_domctl_gsi_permission, rename "access_flag" to "flags", change its type from uint8_t to uint32_t, delete "pad", add XEN_DOMCTL_GSI_REVOKE and XEN_DOMCTL_GSI_GRANT macros. Move "gsi > highest_gsi()" into function gsi_2_irq. Modify parameter gsi in function gsi_2_irq and mp_find_ioapic to unsigned int type. Delete unnecessary spaces and brackets around "~XEN_DOMCTL_GSI_ACTION_MASK". Delete unnecessary goto statements and change to direct break. Add description in commit message to explain how gsi to irq isconverted. v11->v12 changes: Change nr_irqs_gsi to highest_gsi() to check gsi boundary, then need to remove "__init" of highest_gsi function. Change the check of irq boundary from <0 to <=0, and remove unnecessary space. Add #define XEN_DOMCTL_GSI_PERMISSION_MASK 1 to get lowest bit. v10->v11 changes: Extracted from patch#5 of v10 into a separate patch. Add non-zero judgment for other bits of allow_access. Delete unnecessary judgment "if ( is_pv_domain(currd) || has_pirq(currd) )". Change the error exit path identifier "out" to "gsi_permission_out". Use ARRAY_SIZE() instead of open coed. v9->v10 changes: Modified the commit message to further describe the purpose of adding XEN_DOMCTL_gsi_permission. Added a check for all zeros in the padding field in XEN_DOMCTL_gsi_permission, and used currd instead of current->domain. In the function gsi_2_irq, apic_pin_2_gsi_irq was used instead of the original new code, and error handling for irq0 was added. Deleted the extra spaces in the upper and lower lines of the struct xen_domctl_gsi_permission definition. v8->v9 changes: Change the commit message to describe more why we need this new hypercall. Add comment above "if ( is_pv_domain(current->domain) || has_pirq(current->domain) )" to explain why we need this check. Add gsi_2_irq to transform gsi to irq, instead of considering gsi == irq. Add explicit padding to struct xen_domctl_gsi_permission. v5->v8 changes: Nothing. v4->v5 changes: New implementation to add new hypercall XEN_DOMCTL_gsi_permission to grant gsi. --- xen/arch/x86/domctl.c | 29 +++++++++++++++++++++++++++++ xen/arch/x86/include/asm/io_apic.h | 2 ++ xen/arch/x86/io_apic.c | 21 +++++++++++++++++++++ xen/arch/x86/mpparse.c | 7 +++---- xen/include/public/domctl.h | 10 ++++++++++ xen/xsm/flask/hooks.c | 1 + 6 files changed, 66 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 68b5b46d1a83..60b5578c47f8 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -36,6 +36,7 @@ #include #include #include +#include static int update_domain_cpu_policy(struct domain *d, xen_domctl_cpu_policy_t *xdpc) @@ -237,6 +238,34 @@ long arch_do_domctl( break; } + case XEN_DOMCTL_gsi_permission: + { + int irq; + unsigned int gsi = domctl->u.gsi_permission.gsi; + uint32_t flags = domctl->u.gsi_permission.flags; + + /* Check all bits are zero except lowest bit */ + ret = -EINVAL; + if ( flags & ~XEN_DOMCTL_GSI_ACTION_MASK ) + break; + + ret = irq = gsi_2_irq(gsi); + if ( ret <= 0 ) + break; + + ret = -EPERM; + if ( !irq_access_permitted(currd, irq) || + xsm_irq_permission(XSM_HOOK, d, irq, flags) ) + break; + + if ( flags ) + ret = irq_permit_access(d, irq); + else + ret = irq_deny_access(d, irq); + + break; + } + case XEN_DOMCTL_getpageframeinfo3: { unsigned int num = domctl->u.getpageframeinfo3.num; diff --git a/xen/arch/x86/include/asm/io_apic.h b/xen/arch/x86/include/asm/io_apic.h index 78268ea8f666..62456806c7af 100644 --- a/xen/arch/x86/include/asm/io_apic.h +++ b/xen/arch/x86/include/asm/io_apic.h @@ -213,5 +213,7 @@ unsigned highest_gsi(void); int ioapic_guest_read( unsigned long physbase, unsigned int reg, u32 *pval); int ioapic_guest_write(unsigned long physbase, unsigned int reg, u32 val); +int mp_find_ioapic(unsigned int gsi); +int gsi_2_irq(unsigned int gsi); #endif diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index 772700584639..5859484875cc 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -955,6 +955,27 @@ static int pin_2_irq(int idx, int apic, int pin) return irq; } +int gsi_2_irq(unsigned int gsi) +{ + int ioapic, irq; + unsigned int pin; + + if ( gsi > highest_gsi() ) + return -ERANGE; + + ioapic = mp_find_ioapic(gsi); + if ( ioapic < 0 ) + return -EINVAL; + + pin = gsi - io_apic_gsi_base(ioapic); + + irq = apic_pin_2_gsi_irq(ioapic, pin); + if ( irq <= 0 ) + return -EINVAL; + + return irq; +} + static inline int IO_APIC_irq_trigger(int irq) { int apic, idx, pin; diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c index 306d8ed97a83..e13b83bbe9dd 100644 --- a/xen/arch/x86/mpparse.c +++ b/xen/arch/x86/mpparse.c @@ -842,8 +842,7 @@ static struct mp_ioapic_routing { } mp_ioapic_routing[MAX_IO_APICS]; -static int mp_find_ioapic ( - int gsi) +int mp_find_ioapic(unsigned int gsi) { unsigned int i; @@ -854,7 +853,7 @@ static int mp_find_ioapic ( return i; } - printk(KERN_ERR "ERROR: Unable to locate IOAPIC for GSI %d\n", gsi); + printk(KERN_ERR "ERROR: Unable to locate IOAPIC for GSI %u\n", gsi); return -1; } @@ -915,7 +914,7 @@ void __init mp_register_ioapic ( return; } -unsigned __init highest_gsi(void) +unsigned highest_gsi(void) { unsigned x, res = 0; for (x = 0; x < nr_ioapics; x++) diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index 2a49fe46ce25..e1028fc524cf 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -464,6 +464,14 @@ struct xen_domctl_irq_permission { uint8_t pad[3]; }; +/* XEN_DOMCTL_gsi_permission */ +struct xen_domctl_gsi_permission { + uint32_t gsi; +#define XEN_DOMCTL_GSI_REVOKE 0 +#define XEN_DOMCTL_GSI_GRANT 1 +#define XEN_DOMCTL_GSI_ACTION_MASK 1 + uint32_t flags; +}; /* XEN_DOMCTL_iomem_permission */ struct xen_domctl_iomem_permission { @@ -1306,6 +1314,7 @@ struct xen_domctl { #define XEN_DOMCTL_get_paging_mempool_size 85 #define XEN_DOMCTL_set_paging_mempool_size 86 #define XEN_DOMCTL_dt_overlay 87 +#define XEN_DOMCTL_gsi_permission 88 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -1328,6 +1337,7 @@ struct xen_domctl { struct xen_domctl_setdomainhandle setdomainhandle; struct xen_domctl_setdebugging setdebugging; struct xen_domctl_irq_permission irq_permission; + struct xen_domctl_gsi_permission gsi_permission; struct xen_domctl_iomem_permission iomem_permission; struct xen_domctl_ioport_permission ioport_permission; struct xen_domctl_hypercall_init hypercall_init; diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index 278ad38c2af3..dfa23738cd8a 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -695,6 +695,7 @@ static int cf_check flask_domctl(struct domain *d, unsigned int cmd, case XEN_DOMCTL_shadow_op: case XEN_DOMCTL_ioport_permission: case XEN_DOMCTL_ioport_mapping: + case XEN_DOMCTL_gsi_permission: #endif #ifdef CONFIG_HAS_PASSTHROUGH /* From patchwork Fri Aug 16 11:08:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13765976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2B8DC3DA4A for ; Fri, 16 Aug 2024 11:09:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.778495.1188572 (Exim 4.92) (envelope-from ) id 1seupM-0003Eb-56; Fri, 16 Aug 2024 11:09:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 778495.1188572; Fri, 16 Aug 2024 11:09:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupM-0003Dn-1J; Fri, 16 Aug 2024 11:09:04 +0000 Received: by outflank-mailman (input) for mailman id 778495; Fri, 16 Aug 2024 11:09:02 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupK-0001Oc-FU for xen-devel@lists.xenproject.org; Fri, 16 Aug 2024 11:09:02 +0000 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on20629.outbound.protection.outlook.com [2a01:111:f403:2408::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f05456ec-5bbf-11ef-8776-851b0ebba9a2; Fri, 16 Aug 2024 13:09:00 +0200 (CEST) Received: from BN9P221CA0014.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::19) by LV2PR12MB5822.namprd12.prod.outlook.com (2603:10b6:408:179::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.19; Fri, 16 Aug 2024 11:08:55 +0000 Received: from BN3PEPF0000B069.namprd21.prod.outlook.com (2603:10b6:408:10a:cafe::e0) by BN9P221CA0014.outlook.office365.com (2603:10b6:408:10a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23 via Frontend Transport; Fri, 16 Aug 2024 11:08:55 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B069.mail.protection.outlook.com (10.167.243.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7897.4 via Frontend Transport; Fri, 16 Aug 2024 11:08:54 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 16 Aug 2024 06:08:51 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f05456ec-5bbf-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rf6NK+uDqciD4+FVL0kooHFRBOgOFCPHsTQ0w4glt1Rod0fPPtw30fh5wCGFRUDCnSBr01jTAsSXZMPCoPzZYBHW5Jt1Q4sgUrL7UfnQ9v5DQKv776Tl6HKB3+ZO3tSbF837IuJhO755GE2A7RCqn9iwWAEbaPxW7YnuCB9sWFC+teYXy3RPT1tau6hKnV+69fK0vVEBvxvqsWIWKTKKMOAWCUNClkKT6mPDpPvZoLAAeau2dqc20NPvnCkd/UTMjKjVAYIOQCxzzgtqdhEpcWQQZ3VGmoAg+4cc8wi2KWpl3aBMBx2kVtjklDWyBxe19kmUK+weDq36ncI8ai74ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2q0e1DukKo3wUcVPM7NjkIVTk+D+YR5G+Cd5OTV+f3Y=; b=DvrtlV4Gu+u2T/hbw9Og+iDtlSf4wby+YX6uu3zFTnsocCBCKpdzLsrUAzPHPacgzvseBVgOxOkAgiilvw6jfQ/lxrfUtucU5AkhgMdJAhBI3PQq1cqnNdqp+x1kx2wTNC+2jvPismISXLnptJaBhYbmPGQ26JOIvQyIH5gVB7UcDEf6HXVKft7qLwYiB4pMkz+d/p8jYEmKI7Sa/63R/LMCTHquL7kvwZ6oFZysMXXsWefxcADxvbPcNWT+viVVkG5WqvqRGSjWNZUQMUHYEiQs9TPJdG37E625KD1gr9HlByfjh9TVewMZjm8Nam5TznAfsdhj/rmw+wfUrKOAcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2q0e1DukKo3wUcVPM7NjkIVTk+D+YR5G+Cd5OTV+f3Y=; b=VYxp4SReaBBWfUBmr2CqDnHDpvAGRrRbbuP+i4lvXoHBh65BRzp+HBzJXRP1sQRnh/TWl6lWQQPRJF7p9JTS4fwsjrW/Ad4Y9vufRZbRshxwyek849XPTOsrKrJKnUaECuuBQ4viFhvOVh0qLLbxEn2EFMiYw+ecqRCmwnQ7tUo= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [RFC XEN PATCH v13 5/6] tools: Add new function to get gsi from dev Date: Fri, 16 Aug 2024 19:08:19 +0800 Message-ID: <20240816110820.75672-6-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240816110820.75672-1-Jiqian.Chen@amd.com> References: <20240816110820.75672-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B069:EE_|LV2PR12MB5822:EE_ X-MS-Office365-Filtering-Correlation-Id: 71663f5a-01f0-4762-bd44-08dcbde3d18b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|7416014|36860700013|1800799024|376014; X-Microsoft-Antispam-Message-Info: /+9tS5bJ6J0aMXjrRgww9AFJDeV8hexjKotOPXiKNnzRwMlHuZWtRE7g3ADFcfvA4mmmOw/oAt7Q2CAQ99Vrv7O+r/DXrINHxQyjT9Yuoq0jO4PdVorsrnWD7e6yVBp35xQEh9TOKxF0Ci0JmWwHDCtQF7XwN4mt01IMt+X649IL1eHiTDtxeFa2NWNh8jzWBCEpgq9gf6GQDGVXfXhrfV8jFgNOua9voTSZ147g/qzmbjHyMtekZOy2V52ohhUY3ElJzke0tmfWD/ndFcgItBpG0iWCfoikajY+v92x4TTUaksFSsr/sZGZ02gvc1YeN2MW9lUP1DMXqfaCblXQWBaU+Y9eD5gQAO4zZf685X/H8KlwjaPv8ZBxvJIHjb605yaADtSC3uYsnO5DcNC+NvdnACUXERZH8OFWjUFLV52VO8Ca9C7vq1lnRIpqv1emDAJbJehjCHXyDRwvv7hiPqgK5N+JXUvLt9S42p1dcoUVYTxu4ctI23mYdR7N8ID2MgaRzQ0by6QF2IzHGjzjuso28UatksDIvjjIEObsCELweF2zzLKwjLg88Ve+WWNVlJSDJbGPRAK3r3eljAuyqerEzyFvPMNcH6HNnrritg8N/gpltzCmhnm+aK2GZmY5GNq7RcMb/Ulgv4pNMfrJYoLX9isUcSGYFY1E5lxEMotpBm/gy989UNd3PIJX8jtRz3I401gEcOwVtAjKslhUmeYuPxKS4wEoU3ulo8/MuKYktx6WS8wjmYxRBZK7TSfenC2Ymu8IzBF+Ct2aildqiN0SCJg4kpQNb1wQVVJkdCsBAnWds3sri+daN3DGoiLuNHEnLE2+n4iaoG7SzEMsxeXTdrY4hJMNpIQnr5Cu26+n4BpDX0IsE/4tiL1FKgjj/e6p926DG/Nyf8eSn71DY78BEmFnH8P6DLOu447QLY/2qO455gZsevDvjO011pCv340AgNG3kiz+sJm0IjzXfEhhMjqYwJi/v6hbQ4qBxUuySx3U7aUdJrp1hiKEh0vymmv/liyxEU7ZKywSnnI/mK90XVGJvwoR+At2BhTO4BullNlk4sQZ147XGqzdvm/KidH7HzxUZPHNqmxUfFHdv7EUavwe8pW0d0J29ZYcJDykpV2C0R0XIR7BHkg6HNg7x3apHmrX3Qxk5O6Q0166kgZQxZBWoGXjkVDXtlkHa3yltM7bxXhgOfiLmlRRG7aZeOYb0EhaZ28qw+CyLYxj6sXbHcZbxFJhuf9oC0y7Bssivwi6Nu6t9phroJLR/MkMXzqo8CrNrxZZGC8KcPLZNyaJFK+DFuNMdtnqbS9V9L5mOMKnkkY1HRYr9wbGqMoAZHXzx1T2xiePwaOF8wugl5AMAZZSKWvNL9lIOExF4s4= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(7416014)(36860700013)(1800799024)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2024 11:08:54.9964 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 71663f5a-01f0-4762-bd44-08dcbde3d18b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B069.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5822 When passthrough a device to domU, QEMU and xl tools use its gsi number to do pirq mapping, see QEMU code xen_pt_realize->xc_physdev_map_pirq, and xl code pci_add_dm_done->xc_physdev_map_pirq, but the gsi number is got from file /sys/bus/pci/devices//irq, that is wrong, because irq is not equal with gsi, they are in different spaces, so pirq mapping fails. And in current codes, there is no method to get gsi for userspace. For above purpose, add new function to get gsi, and the corresponding ioctl is implemented on linux kernel side. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Chen Jiqian --- RFC: it needs to wait for the corresponding third patch on linux kernel side to be merged. https://lore.kernel.org/xen-devel/20240607075109.126277-4-Jiqian.Chen@amd.com/ --- v12->v13 changes: Rename the function xc_physdev_gsi_from_pcidev to xc_pcidev_get_gsi to avoid confusion with physdev namesapce. Move the implementation of xc_pcidev_get_gsi into xc_linux.c. Directly use xencall_fd(xch->xcall) in the function xc_pcidev_get_gsi instead of opening "privcmd". v11->v12 changes: Nothing. v10->v11 changes: Patch#4 of v10, directly open "/dev/xen/privcmd" in the function xc_physdev_gsi_from_dev instead of adding unnecessary functions to libxencall. Change the type of gsi in the structure privcmd_gsi_from_dev from int to u32. v9->v10 changes: Extract the implementation of xc_physdev_gsi_from_dev to be a new patch. --- tools/include/xen-sys/Linux/privcmd.h | 7 +++++++ tools/include/xenctrl.h | 2 ++ tools/libs/ctrl/xc_freebsd.c | 6 ++++++ tools/libs/ctrl/xc_linux.c | 20 ++++++++++++++++++++ tools/libs/ctrl/xc_minios.c | 6 ++++++ tools/libs/ctrl/xc_netbsd.c | 6 ++++++ tools/libs/ctrl/xc_solaris.c | 6 ++++++ 7 files changed, 53 insertions(+) diff --git a/tools/include/xen-sys/Linux/privcmd.h b/tools/include/xen-sys/Linux/privcmd.h index bc60e8fd55eb..607dfa2287bc 100644 --- a/tools/include/xen-sys/Linux/privcmd.h +++ b/tools/include/xen-sys/Linux/privcmd.h @@ -95,6 +95,11 @@ typedef struct privcmd_mmap_resource { __u64 addr; } privcmd_mmap_resource_t; +typedef struct privcmd_pcidev_get_gsi { + __u32 sbdf; + __u32 gsi; +} privcmd_pcidev_get_gsi_t; + /* * @cmd: IOCTL_PRIVCMD_HYPERCALL * @arg: &privcmd_hypercall_t @@ -114,6 +119,8 @@ typedef struct privcmd_mmap_resource { _IOC(_IOC_NONE, 'P', 6, sizeof(domid_t)) #define IOCTL_PRIVCMD_MMAP_RESOURCE \ _IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t)) +#define IOCTL_PRIVCMD_PCIDEV_GET_GSI \ + _IOC(_IOC_NONE, 'P', 10, sizeof(privcmd_pcidev_get_gsi_t)) #define IOCTL_PRIVCMD_UNIMPLEMENTED \ _IOC(_IOC_NONE, 'P', 0xFF, 0) diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 9ceca0cffc2f..82de6748f7a7 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -1641,6 +1641,8 @@ int xc_physdev_unmap_pirq(xc_interface *xch, uint32_t domid, int pirq); +int xc_pcidev_get_gsi(xc_interface *xch, uint32_t sbdf); + /* * LOGGING AND ERROR REPORTING */ diff --git a/tools/libs/ctrl/xc_freebsd.c b/tools/libs/ctrl/xc_freebsd.c index 9dd48a3a08bb..9019fc663361 100644 --- a/tools/libs/ctrl/xc_freebsd.c +++ b/tools/libs/ctrl/xc_freebsd.c @@ -60,6 +60,12 @@ void *xc_memalign(xc_interface *xch, size_t alignment, size_t size) return ptr; } +int xc_pcidev_get_gsi(xc_interface *xch, uint32_t sbdf) +{ + errno = ENOSYS; + return -1; +} + /* * Local variables: * mode: C diff --git a/tools/libs/ctrl/xc_linux.c b/tools/libs/ctrl/xc_linux.c index c67c71c08be3..92591e49a1c8 100644 --- a/tools/libs/ctrl/xc_linux.c +++ b/tools/libs/ctrl/xc_linux.c @@ -66,6 +66,26 @@ void *xc_memalign(xc_interface *xch, size_t alignment, size_t size) return ptr; } +int xc_pcidev_get_gsi(xc_interface *xch, uint32_t sbdf) +{ + int ret; + privcmd_pcidev_get_gsi_t dev_gsi = { + .sbdf = sbdf, + .gsi = 0, + }; + + ret = ioctl(xencall_fd(xch->xcall), + IOCTL_PRIVCMD_PCIDEV_GET_GSI, &dev_gsi); + + if (ret < 0) { + PERROR("Failed to get gsi from dev"); + } else { + ret = dev_gsi.gsi; + } + + return ret; +} + /* * Local variables: * mode: C diff --git a/tools/libs/ctrl/xc_minios.c b/tools/libs/ctrl/xc_minios.c index 3dea7a78a576..462af827b33c 100644 --- a/tools/libs/ctrl/xc_minios.c +++ b/tools/libs/ctrl/xc_minios.c @@ -47,6 +47,12 @@ void *xc_memalign(xc_interface *xch, size_t alignment, size_t size) return memalign(alignment, size); } +int xc_pcidev_get_gsi(xc_interface *xch, uint32_t sbdf) +{ + errno = ENOSYS; + return -1; +} + /* * Local variables: * mode: C diff --git a/tools/libs/ctrl/xc_netbsd.c b/tools/libs/ctrl/xc_netbsd.c index 31979937621e..1318d4d90608 100644 --- a/tools/libs/ctrl/xc_netbsd.c +++ b/tools/libs/ctrl/xc_netbsd.c @@ -63,6 +63,12 @@ void *xc_memalign(xc_interface *xch, size_t alignment, size_t size) return valloc(size); } +int xc_pcidev_get_gsi(xc_interface *xch, uint32_t sbdf) +{ + errno = ENOSYS; + return -1; +} + /* * Local variables: * mode: C diff --git a/tools/libs/ctrl/xc_solaris.c b/tools/libs/ctrl/xc_solaris.c index 5128f3f0f490..049e28d55ccd 100644 --- a/tools/libs/ctrl/xc_solaris.c +++ b/tools/libs/ctrl/xc_solaris.c @@ -32,6 +32,12 @@ void *xc_memalign(xc_interface *xch, size_t alignment, size_t size) return memalign(alignment, size); } +int xc_pcidev_get_gsi(xc_interface *xch, uint32_t sbdf) +{ + errno = ENOSYS; + return -1; +} + /* * Local variables: * mode: C From patchwork Fri Aug 16 11:08:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13765978 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 07774C531DC for ; Fri, 16 Aug 2024 11:09:20 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.778509.1188582 (Exim 4.92) (envelope-from ) id 1seupS-0003zM-Mg; Fri, 16 Aug 2024 11:09:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 778509.1188582; Fri, 16 Aug 2024 11:09:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupS-0003zD-I5; Fri, 16 Aug 2024 11:09:10 +0000 Received: by outflank-mailman (input) for mailman id 778509; Fri, 16 Aug 2024 11:09:09 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1seupR-0001dU-FU for xen-devel@lists.xenproject.org; Fri, 16 Aug 2024 11:09:09 +0000 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2060a.outbound.protection.outlook.com [2a01:111:f403:2417::60a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f3e4befa-5bbf-11ef-a505-bb4a2ccca743; Fri, 16 Aug 2024 13:09:07 +0200 (CEST) Received: from BL0PR02CA0053.namprd02.prod.outlook.com (2603:10b6:207:3d::30) by CH3PR12MB8233.namprd12.prod.outlook.com (2603:10b6:610:129::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.19; Fri, 16 Aug 2024 11:08:59 +0000 Received: from BN3PEPF0000B06A.namprd21.prod.outlook.com (2603:10b6:207:3d:cafe::94) by BL0PR02CA0053.outlook.office365.com (2603:10b6:207:3d::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22 via Frontend Transport; Fri, 16 Aug 2024 11:08:59 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B06A.mail.protection.outlook.com (10.167.243.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7897.4 via Frontend Transport; Fri, 16 Aug 2024 11:08:59 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 16 Aug 2024 06:08:55 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f3e4befa-5bbf-11ef-a505-bb4a2ccca743 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eH6BuZ+ExgZK/hER10eJY0XhegRz4fyygNk/Daj0ZfnGJPkzxyHKA0lFfaVDW7sfBj5ObM+RjIfRPIVI8ip4COTN2Dy2WD5tU57BsIhci9rGxL6mT3++hzhFpT8yx/cSZIdxTyErFwrl3/Se2+0rTQetmCc4bXu+VzZER3NqbuRI3JZ3t7MKGPkdWzspWzSf67x7Basy11ipBlXIaWCCSlZoZ7uximcj2W9+E/EpVYIV4dCii+eUf5Tvmdw78vUT+VN0gGtuXArnhFD6LtERzQHCTt/LU0hVj8ia+4f15U4JCMGCaE9sWIoZrpWbRp3rDaK1qN+l8mo+uboaNI4kjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bu5Qz9WFGKymDn26KEbzd9h2r23enFrhojTi0ypkn/U=; b=VdBbbRreQBIcvqImzQKLNiUBY2DH4jCVzRFTre8hX6SyIDqUCqXtv8JyNzFlHy2x/Tl5ZdhLBQ2iBpNU9dYfWv973kwsBoOiZM7bijJYIs92EuTqUS7wVspTC5lldhj4wxvXDdIRtX+3xBqEl3EI1JpagnTMszidzJWIWyAZO5kppdktTgtUPE0xqSlzBergfYTC41WVYmhCq0JrYNhiMPMqGfhaEqilj4pMCdpuOgfBWJ5MnViq58wdoDNH2v6qZu0OcTEQ6gfqVbsyRpIYWPXxq6SZITmDBQjJc6xclxRejA3NoxhBe2WvRG4EoQcG+AmSnFm4AjBKAlrGfRHFZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bu5Qz9WFGKymDn26KEbzd9h2r23enFrhojTi0ypkn/U=; b=aOookuMGajrnUHkXV4V1bAsL5V3q/BLR4J3WwUcePNpX7e/yVr04NJ+KrPp84ZJW/NMcuhNI5LnHEgJ9mDYoLcjdtPJLZJywp4jzV4ILMIUDW83FVCeO/P4TPDpGsSWSlwz5LlzsiAKUIm0T06WZMU4ScpcWxP6flUWWm+Wr4IE= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [RFC XEN PATCH v13 6/6] tools: Add new function to do PIRQ (un)map on PVH dom0 Date: Fri, 16 Aug 2024 19:08:20 +0800 Message-ID: <20240816110820.75672-7-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240816110820.75672-1-Jiqian.Chen@amd.com> References: <20240816110820.75672-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B06A:EE_|CH3PR12MB8233:EE_ X-MS-Office365-Filtering-Correlation-Id: e4a13c71-18f9-43dc-f35c-08dcbde3d41a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|376014|7416014; X-Microsoft-Antispam-Message-Info: M3RMgYltroc2QiHlmkQv2U29IYla6ChUOs3AcZXiSivwXo8Tv0RF7C/qWT3VtaCiN9i1nH9CPCVt2VCPOaKH9ZH3TJ2cEKb3RjUz8aALGZGlMy4CiMomOq4NaMX6J3/caPNtvk7HiuOXhVq0bjq9QaYZNQMv73pU3T5c6CV/XlrTW8vb7f/+HQClRurjvFmPFTg2EZKEPrkPBV2qBOgaH5vFrZWd7UUHRiCS5VIra5KtY0n/ECSDMEOfqGDdGTzltzRXJcUP3wx3jP33iuclj8u/+P6KNVnNPG4Dd0GdXk18SKPoFeiiHAdnGv9JfrNsoAh07F0kWq86V0OJzqNa0THR7czYZPrpDnNLjk00I49AKLqh8uElcmzFVwwyZZLxdi6GKl99INDT1ceUlcOYzMzo05Dbw++KhoYxl1nImMfmSMRux8psucQYDkQTqP6PXtvUO+CCpQHEWk5N2czNn6pMqfR91upapzBWpLPHSGGNAfzeS1spw4KWzSz0gEAmqbj9p451C41fOiqz1k+0dAFp5lgIKf91vyRIsgiG5ndGx+CNZPXAszWoiOswlDGkObCMJzD+GYBzrsw6RvbFQLCBH0XEVk/Y4Pk3tsYGaaKnlFb53fuYvuXfvhaS21s6hUsf7uXp0yiAhlq7RfpiFGh4WSfG4UFQl58Fcas72ZT5cNeRhFQLhsnujQ0MW4Fbs8XkiTC2qx8MBJ6+dxEcXthzsYzmAJGc2XcxFXt9ZdFdhizXdtOr8hrgYx6+KoFpig1n1zPl75PBL3gkds1/+TeLRZ2wZypPAGtkZDqwn4WpnoRU82gHaFIzWXUDUxP9qL4LnM5U5dUZisN/cLE83WVZh3QUC2ZKbOfWzMJr5hQKIjgDWe13QFEBpQ0AywnCpnRk7AriQ1BHDrSdlE2Kt9OaiX1Im/PU1KkCnYGGcZ180Ul/OvHPtYP8HAhRG41XPa0KRvPmj7r/3jEA2SyhIqnuzmGkByMjM1hmTFVTvmAC+2QY0H7yuKOPKJCigPX7DkiINSGoXN4LJNx0x8H/bY8y9dIs3XRoxeqYKPgXLfRvOZPOg028SmLD19MPa+VrmrJOf+YSCCBzXgyhWxexfok16Jow8S3n30HO58ziuoHVhqCDBgHoUp2NS21gkJJCb+/xu+EnDeK4PLuA3Y6I1dTHkU/qDZ7VTw3MHjtLq45kkg3o3cEG9WurOqWJyTptkO+nRVd0MoQmI+BfPBTRVVd9Hxj7agJ4RoLQHRzDP5Nk3afz8fQCBdQRpD1GNzLf6r30x3ijxqOiHvm4ubHQp5NgIoBg0eP99M19N+X+wJCJ9FN1Go82snhhnWXOTDu3CX1YyKTAkkiMfGimH+5P8w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2024 11:08:59.2904 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e4a13c71-18f9-43dc-f35c-08dcbde3d41a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B06A.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8233 When dom0 is PVH, and passthrough a device to dumU, xl will use the gsi number of device to do a pirq mapping, see pci_add_dm_done->xc_physdev_map_pirq, but the gsi number is got from file /sys/bus/pci/devices//irq, that confuses irq and gsi, they are in different space and are not equal, so it will fail when mapping. To solve this issue, use xc_physdev_gsi_from_dev to get the real gsi and add a new function xc_physdev_map_pirq_gsi to get a free pirq for gsi(why not use current function xc_physdev_map_pirq, because it doesn't support to allocate a free pirq, what's more, to prevent changing it and affecting its callers, so add xc_physdev_map_pirq_gsi). Besides, PVH dom0 doesn't have PIRQ flag, it doesn't do PHYSDEVOP_map_pirq for each gsi. So grant function callstack pci_add_dm_done->XEN_DOMCTL_irq_permission will fail at function domain_pirq_to_irq. And old hypercall XEN_DOMCTL_irq_permission requires passing in pirq, it is not suitable for dom0 that doesn't have PIRQs to grant irq permission. To solve this issue, use the new hypercall XEN_DOMCTL_gsi_permission to grant the permission of irq( translate from gsi) to dumU when dom0 has no PIRQs. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Chen Jiqian --- RFC: it needs to wait for the corresponding third patch on linux kernel side to be merged. https://lore.kernel.org/xen-devel/20240607075109.126277-4-Jiqian.Chen@amd.com/ --- v12->v13 changes: Deleted patch #6 of v12, and added function xc_physdev_map_pirq_gsi to map pirq for gsi. For functions that generate libxl error, changed the return value from -1 to ERROR_*. Instead of declaring "ctx", use the macro "CTX". Add the function libxl__arch_local_romain_ has_pirq_notion to determine if there is a concept of pirq in the domain where xl is located. In the function libxl__arch_hvm_unmap_gsi, before unmap_pirq, use map_pirq to obtain the pirq corresponding to gsi. v11->v12 changes: Nothing. v10->v11 changes: New patch Modification of the tools part of patches#4 and #5 of v10, use privcmd_gsi_from_dev to get gsi, and use XEN_DOMCTL_gsi_permission to grant gsi. Change the hard-coded 0 to use LIBXL_TOOLSTACK_DOMID. Add libxl__arch_hvm_map_gsi to distinguish x86 related implementations. Add a list pcidev_pirq_list to record the relationship between sbdf and pirq, which can be used to obtain the corresponding pirq when unmap PIRQ. --- tools/include/xenctrl.h | 10 +++ tools/libs/ctrl/xc_domain.c | 15 +++++ tools/libs/ctrl/xc_physdev.c | 27 ++++++++ tools/libs/light/libxl_arch.h | 6 ++ tools/libs/light/libxl_arm.c | 15 +++++ tools/libs/light/libxl_pci.c | 112 ++++++++++++++++++++-------------- tools/libs/light/libxl_x86.c | 72 ++++++++++++++++++++++ 7 files changed, 212 insertions(+), 45 deletions(-) diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 82de6748f7a7..c798472995f7 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -1382,6 +1382,11 @@ int xc_domain_irq_permission(xc_interface *xch, uint32_t pirq, bool allow_access); +int xc_domain_gsi_permission(xc_interface *xch, + uint32_t domid, + uint32_t gsi, + uint32_t flags); + int xc_domain_iomem_permission(xc_interface *xch, uint32_t domid, unsigned long first_mfn, @@ -1637,6 +1642,11 @@ int xc_physdev_map_pirq_msi(xc_interface *xch, int entry_nr, uint64_t table_base); +int xc_physdev_map_pirq_gsi(xc_interface *xch, + uint32_t domid, + int gsi, + int *pirq); + int xc_physdev_unmap_pirq(xc_interface *xch, uint32_t domid, int pirq); diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c index f2d9d14b4d9f..e3538ec0ba80 100644 --- a/tools/libs/ctrl/xc_domain.c +++ b/tools/libs/ctrl/xc_domain.c @@ -1394,6 +1394,21 @@ int xc_domain_irq_permission(xc_interface *xch, return do_domctl(xch, &domctl); } +int xc_domain_gsi_permission(xc_interface *xch, + uint32_t domid, + uint32_t gsi, + uint32_t flags) +{ + struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_gsi_permission, + .domain = domid, + .u.gsi_permission.gsi = gsi, + .u.gsi_permission.flags = flags, + }; + + return do_domctl(xch, &domctl); +} + int xc_domain_iomem_permission(xc_interface *xch, uint32_t domid, unsigned long first_mfn, diff --git a/tools/libs/ctrl/xc_physdev.c b/tools/libs/ctrl/xc_physdev.c index 460a8e779ce8..c752cd1f4410 100644 --- a/tools/libs/ctrl/xc_physdev.c +++ b/tools/libs/ctrl/xc_physdev.c @@ -95,6 +95,33 @@ int xc_physdev_map_pirq_msi(xc_interface *xch, return rc; } +int xc_physdev_map_pirq_gsi(xc_interface *xch, + uint32_t domid, + int gsi, + int *pirq) +{ + int rc; + struct physdev_map_pirq map; + + if ( !pirq ) + { + errno = EINVAL; + return -1; + } + memset(&map, 0, sizeof(struct physdev_map_pirq)); + map.domid = domid; + map.type = MAP_PIRQ_TYPE_GSI; + map.index = gsi; + map.pirq = *pirq; + + rc = do_physdev_op(xch, PHYSDEVOP_map_pirq, &map, sizeof(map)); + + if ( !rc ) + *pirq = map.pirq; + + return rc; +} + int xc_physdev_unmap_pirq(xc_interface *xch, uint32_t domid, int pirq) diff --git a/tools/libs/light/libxl_arch.h b/tools/libs/light/libxl_arch.h index f88f11d6de1d..c8ef52ddbe7f 100644 --- a/tools/libs/light/libxl_arch.h +++ b/tools/libs/light/libxl_arch.h @@ -91,6 +91,12 @@ void libxl__arch_update_domain_config(libxl__gc *gc, libxl_domain_config *dst, const libxl_domain_config *src); +_hidden +int libxl__arch_hvm_map_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid); +_hidden +int libxl__arch_hvm_unmap_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid); +_hidden +bool libxl__arch_local_domain_has_pirq_notion(libxl__gc *gc); #if defined(__i386__) || defined(__x86_64__) #define LAPIC_BASE_ADDRESS 0xfee00000 diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index a4029e3ac810..5a9db5e85f6f 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -1774,6 +1774,21 @@ void libxl__arch_update_domain_config(libxl__gc *gc, { } +int libxl__arch_hvm_map_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + return ERROR_INVAL; +} + +int libxl__arch_hvm_unmap_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + return ERROR_INVAL; +} + +bool libxl__arch_local_domain_has_pirq_notion(libxl__gc *gc) +{ + return true; +} + /* * Local variables: * mode: C diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c index 96cb4da0794e..2014a67e6e56 100644 --- a/tools/libs/light/libxl_pci.c +++ b/tools/libs/light/libxl_pci.c @@ -17,6 +17,7 @@ #include "libxl_osdeps.h" /* must come before any other headers */ #include "libxl_internal.h" +#include "libxl_arch.h" #define PCI_BDF "%04x:%02x:%02x.%01x" #define PCI_BDF_SHORT "%02x:%02x.%01x" @@ -1478,32 +1479,43 @@ static void pci_add_dm_done(libxl__egc *egc, fclose(f); if (!pci_supp_legacy_irq()) goto out_no_irq; - sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain, - pci->bus, pci->dev, pci->func); - f = fopen(sysfs_path, "r"); - if (f == NULL) { - LOGED(ERROR, domainid, "Couldn't open %s", sysfs_path); - goto out_no_irq; - } - if ((fscanf(f, "%u", &irq) == 1) && irq) { - r = xc_physdev_map_pirq(ctx->xch, domid, irq, &irq); - if (r < 0) { - LOGED(ERROR, domainid, "xc_physdev_map_pirq irq=%d (error=%d)", - irq, r); - fclose(f); - rc = ERROR_FAIL; + + /* When dom0 is PVH, should use gsi to map pirq and grant permission */ + rc = libxl__arch_local_domain_has_pirq_notion(gc); + if (!rc) { + rc = libxl__arch_hvm_map_gsi(gc, pci_encode_bdf(pci), domid); + if (rc) { + LOGED(ERROR, domainid, "libxl__arch_hvm_map_gsi failed"); goto out; } - r = xc_domain_irq_permission(ctx->xch, domid, irq, 1); - if (r < 0) { - LOGED(ERROR, domainid, - "xc_domain_irq_permission irq=%d (error=%d)", irq, r); - fclose(f); - rc = ERROR_FAIL; - goto out; + } else { + sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain, + pci->bus, pci->dev, pci->func); + f = fopen(sysfs_path, "r"); + if (f == NULL) { + LOGED(ERROR, domainid, "Couldn't open %s", sysfs_path); + goto out_no_irq; + } + if ((fscanf(f, "%u", &irq) == 1) && irq) { + r = xc_physdev_map_pirq(ctx->xch, domid, irq, &irq); + if (r < 0) { + LOGED(ERROR, domainid, "xc_physdev_map_pirq irq=%d (error=%d)", + irq, r); + fclose(f); + rc = ERROR_FAIL; + goto out; + } + r = xc_domain_irq_permission(ctx->xch, domid, irq, 1); + if (r < 0) { + LOGED(ERROR, domainid, + "xc_domain_irq_permission irq=%d (error=%d)", irq, r); + fclose(f); + rc = ERROR_FAIL; + goto out; + } } + fclose(f); } - fclose(f); /* Don't restrict writes to the PCI config space from this VM */ if (pci->permissive) { @@ -2229,33 +2241,43 @@ skip_bar: if (!pci_supp_legacy_irq()) goto skip_legacy_irq; - sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain, - pci->bus, pci->dev, pci->func); - - f = fopen(sysfs_path, "r"); - if (f == NULL) { - LOGED(ERROR, domid, "Couldn't open %s", sysfs_path); - goto skip_legacy_irq; - } + /* When dom0 is PVH, should use gsi to unmap pirq and deny permission */ + rc = libxl__arch_local_domain_has_pirq_notion(gc); + if (!rc) { + rc = libxl__arch_hvm_unmap_gsi(gc, pci_encode_bdf(pci), domid); + if (rc) { + LOGED(ERROR, domid, "libxl__arch_hvm_unmap_gsi failed"); + goto out; + } + } else { + sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain, + pci->bus, pci->dev, pci->func); - if ((fscanf(f, "%u", &irq) == 1) && irq) { - rc = xc_physdev_unmap_pirq(ctx->xch, domid, irq); - if (rc < 0) { - /* - * QEMU may have already unmapped the IRQ. So the error - * may be spurious. For now, still print an error message as - * it is not easy to distinguished between valid and - * spurious error. - */ - LOGED(ERROR, domid, "xc_physdev_unmap_pirq irq=%d", irq); + f = fopen(sysfs_path, "r"); + if (f == NULL) { + LOGED(ERROR, domid, "Couldn't open %s", sysfs_path); + goto skip_legacy_irq; } - rc = xc_domain_irq_permission(ctx->xch, domid, irq, 0); - if (rc < 0) { - LOGED(ERROR, domid, "xc_domain_irq_permission irq=%d", irq); + + if ((fscanf(f, "%u", &irq) == 1) && irq) { + rc = xc_physdev_unmap_pirq(ctx->xch, domid, irq); + if (rc < 0) { + /* + * QEMU may have already unmapped the IRQ. So the error + * may be spurious. For now, still print an error message as + * it is not easy to distinguished between valid and + * spurious error. + */ + LOGED(ERROR, domid, "xc_physdev_unmap_pirq irq=%d", irq); + } + rc = xc_domain_irq_permission(ctx->xch, domid, irq, 0); + if (rc < 0) { + LOGED(ERROR, domid, "xc_domain_irq_permission irq=%d", irq); + } } - } - fclose(f); + fclose(f); + } skip_legacy_irq: diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c index 60643d6f5376..20e3956f09b8 100644 --- a/tools/libs/light/libxl_x86.c +++ b/tools/libs/light/libxl_x86.c @@ -879,6 +879,78 @@ void libxl__arch_update_domain_config(libxl__gc *gc, libxl_defbool_val(src->b_info.u.hvm.pirq)); } +bool libxl__arch_local_domain_has_pirq_notion(libxl__gc *gc) +{ + int r; + xc_domaininfo_t info; + + r = xc_domain_getinfo_single(CTX->xch, LIBXL_TOOLSTACK_DOMID, &info); + if (r == 0) { + if (!(info.flags & XEN_DOMINF_hvm_guest) || + (info.arch_config.emulation_flags & XEN_X86_EMU_USE_PIRQ)) + return true; + } else { + LOGE(ERROR, "getdomaininfo failed ret=%d", r); + } + + return false; +} + +int libxl__arch_hvm_map_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + int pirq = -1, gsi, r; + + gsi = xc_pcidev_get_gsi(CTX->xch, sbdf); + if (gsi < 0) { + return ERROR_FAIL; + } + + r = xc_physdev_map_pirq_gsi(CTX->xch, domid, gsi, &pirq); + if (r < 0) { + LOGED(ERROR, domid, "xc_physdev_map_pirq_gsi gsi=%d ret=%d", gsi, r); + return ERROR_FAIL; + } + + r = xc_domain_gsi_permission(CTX->xch, domid, gsi, XEN_DOMCTL_GSI_GRANT); + if (r < 0) { + LOGED(ERROR, domid, "xc_domain_gsi_permission gsi=%d ret=%d", gsi, r); + return ERROR_FAIL; + } + + return 0; +} + +int libxl__arch_hvm_unmap_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + int pirq = -1, gsi, r; + + gsi = xc_pcidev_get_gsi(CTX->xch, sbdf); + if (gsi < 0) { + return ERROR_FAIL; + } + + /* Before unmapping, use mapping to get the already mapped pirq first */ + r = xc_physdev_map_pirq_gsi(CTX->xch, domid, gsi, &pirq); + if (r < 0) { + LOGED(ERROR, domid, "xc_physdev_map_pirq_gsi gsi=%d ret=%d", gsi, r); + return ERROR_FAIL; + } + + r = xc_physdev_unmap_pirq(CTX->xch, domid, pirq); + if (r < 0) { + LOGED(ERROR, domid, "xc_physdev_unmap_pirq gsi=%d ret=%d", gsi, r); + return ERROR_FAIL; + } + + r = xc_domain_gsi_permission(CTX->xch, domid, gsi, XEN_DOMCTL_GSI_REVOKE); + if (r < 0) { + LOGED(ERROR, domid, "xc_domain_gsi_permission gsi=%d ret=%d", gsi, r); + return ERROR_FAIL; + } + + return 0; +} + /* * Local variables: * mode: C