From patchwork Fri Nov 24 10:31:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13467486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95571C61DF7 for ; Fri, 24 Nov 2023 10:33:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640386.998593 (Exim 4.92) (envelope-from ) id 1r6TUh-0000OO-Gj; Fri, 24 Nov 2023 10:33:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640386.998593; Fri, 24 Nov 2023 10:33:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6TUh-0000OB-DZ; Fri, 24 Nov 2023 10:33:07 +0000 Received: by outflank-mailman (input) for mailman id 640386; Fri, 24 Nov 2023 10:33:06 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6TUg-00005b-At for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 10:33:06 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2061e.outbound.protection.outlook.com [2a01:111:f400:7ea9::61e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d923369d-8ab4-11ee-9b0e-b553b5be7939; Fri, 24 Nov 2023 11:33:04 +0100 (CET) Received: from CY5PR15CA0043.namprd15.prod.outlook.com (2603:10b6:930:1b::12) by PH8PR12MB7448.namprd12.prod.outlook.com (2603:10b6:510:214::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.20; Fri, 24 Nov 2023 10:32:59 +0000 Received: from CY4PEPF0000EDD6.namprd03.prod.outlook.com (2603:10b6:930:1b:cafe::cd) by CY5PR15CA0043.outlook.office365.com (2603:10b6:930:1b::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.21 via Frontend Transport; Fri, 24 Nov 2023 10:32:59 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD6.mail.protection.outlook.com (10.167.241.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7025.20 via Frontend Transport; Fri, 24 Nov 2023 10:32:59 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 24 Nov 2023 04:32:53 -0600 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d923369d-8ab4-11ee-9b0e-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H+sdh8KvLMHd1LBsYdO7e0DdSYgTkga6gXdEoiwQayu9xWr/WppouJZQEpbXQdItM9/ziZCctN0AHC+lxtEOCUZN4PFRDcFQUtKtEcSm93lvutkQo+yTHjj/qKEB0CkMx8nuvwEwdpIQiMKgAvZmv1r22EbczO3pfrQY8YbhxZbWfim0IKf9VFFmXS9H3pK2cm7gNJomwXerzIWAcX2J7Y4HGmopkHBM2EpaKK9iKhg8RpJDjKXgHAeNTGXU0ApFcsjsbpZYfh1dHQpgd0wQL/pEibcCPd5oMDWAs9hCog0aJLzX9K11JRVURoKT7QsQV9afKzCvJcOqgn0q0aXlaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4aH+2SVcQn/nsVjZ45IvLi1aXjfOjVFR+f+ua3swKd8=; b=avvWCSA/Jy380gIoNZZWA2sTNRtJZGiBMSKRf0j4CQaWtkBiEVT557QIbIquhWhVZoZi1hi9z9LY4hWZrl3RHNZ+KDjlva+hFRy1cPVimWp0Db2InOg0er+8ls2vA9krnEVEIuQSt7S9rcqm8Dp9mDpkZaZnmcc6XgOkBEPVZu8DoO9eCpHeEcpyGFLWihHGMiOzte1/lQXcrhPdpRKtw34zETeKpIUIZeP6lgW/kvuj8nuhdQTu7G8gJwB/AjoINF9Efs+B91g0QoljUzgSZkVSuEI1XbitCAjTkmBx4t89AkE6MvykmTvLgkMRxQ3kBXkMGjbWJm8MOloObIQEUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4aH+2SVcQn/nsVjZ45IvLi1aXjfOjVFR+f+ua3swKd8=; b=B7uZvhUxln1N+q4YTG01eiRGtk8iyKwiP1eAcTdyH+JVdKXPdunGlatWK3cE1cktLW9zPrCBlo93qRiM3YiyenRFM0JmV2eoK9R6+QQp33nh5FkVzLJr53JQKvZVK9KY+uhtEplOHbwsKbAZ0Otf0PLmio2DbGR2zGmd2d5Zovg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Thomas Gleixner , "Boris Ostrovsky" , "Rafael J . Wysocki" , Len Brown , Bjorn Helgaas , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , , , CC: Stefano Stabellini , Alex Deucher , Christian Koenig , Stewart Hildebrand , Xenia Ragiadakou , Honglei Huang , "Julia Zhang" , Huang Rui , Jiqian Chen , Huang Rui Subject: [RFC KERNEL PATCH v2 1/3] xen/pci: Add xen_reset_device_state function Date: Fri, 24 Nov 2023 18:31:21 +0800 Message-ID: <20231124103123.3263471-2-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231124103123.3263471-1-Jiqian.Chen@amd.com> References: <20231124103123.3263471-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD6:EE_|PH8PR12MB7448:EE_ X-MS-Office365-Filtering-Correlation-Id: b93f64d7-346c-4ffb-31d8-08dbecd8babb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Y25TUEANezV16ofpBWfoq00mLowrkjE9spwmym00POnUVjNmXCCV6FqWLa1qBMau6eNokvjlIgZfnrVYNfvmFEzKv21u5NVx+MUSlXjwjKpXK0Imz72hv0olTZOoUneqtO/kgQukrdJuUgbycDfD+VHnY0A9a4O2rly6vVbXJzw492gl756imh/0Kpw82q3wV9nwTCTH/yfPZfLoR6oz3wRBXOQmVUgsK2THRvIAh1KwSDuXQ2iE77Fn6nSJH5gHQ8jqsC60Zidm+wi6wZpvQK4THT52rCnQW0VAF52wsxRCil3Nj54d6HIgCuMPQ2qF1D4DZRlujoyqh1jwhkYG8HQ2mS2zC56yyoOULMMvZaCAQ9l47IR/hdVql8bSZ5Cd3PdUJVHON1uDxKzXyGdgkPi59GKufOCeAEFHSF/6Cb0dbmsqs6lRzNYjT5BqHFSADUsfMGYT1RYCL13TwsjQkBCUoZ0xQaO6aA2smp78KJfmOkf9kpP2kKCvJd12fWvKyBQJXCl8V/4wJBmT8zkouxCD3/LmaQC62TpS3V0DRGpgEhuQHcTSOtBHpozc2arX0vdCZGbydRKpVJ8OtrjByYVnlDRSNDzNPyRsW9AZ4I+k9Z4Z/avwobAJPmf5dMFheyaaYFiMBXbXr4vOYqbbbydCNdZDrcAHOmFYODhDII4ehwhwKT4imnl9YCDidQU0LIqZadpjCDytLmnhW33dvanadnQR5ucXrBuUkoLyKLJnOrLKpWwbRsdYjqconrfj4K/M0yjmXfnUdLFfki+0ExCqRzcDk8Y4IbVbzZkWlPE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(39860400002)(346002)(396003)(230922051799003)(451199024)(64100799003)(186009)(82310400011)(1800799012)(40470700004)(36840700001)(46966006)(26005)(16526019)(83380400001)(336012)(426003)(47076005)(82740400003)(86362001)(356005)(81166007)(40480700001)(36860700001)(41300700001)(921008)(2616005)(1076003)(36756003)(7416002)(7696005)(6666004)(2906002)(40460700003)(5660300002)(316002)(54906003)(70206006)(70586007)(110136005)(478600001)(8936002)(4326008)(8676002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2023 10:32:59.2040 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b93f64d7-346c-4ffb-31d8-08dbecd8babb X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD6.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7448 When device on dom0 side has been reset, the vpci on Xen side won't get notification, so that the cached state in vpci is all out of date with the real device state. To solve that problem, this patch add a function to clear all vpci device state when device is reset on dom0 side. And call that function in pcistub_init_device. Because when we use "pci-assignable-add" to assign a passthrough device in Xen, it will reset passthrough device and the vpci state will out of date, and then device will fail to restore bar state. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen --- drivers/xen/pci.c | 12 ++++++++++++ drivers/xen/xen-pciback/pci_stub.c | 3 +++ include/xen/interface/physdev.h | 2 ++ include/xen/pci.h | 6 ++++++ 4 files changed, 23 insertions(+) diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c index 72d4e3f193af..e9b30bc09139 100644 --- a/drivers/xen/pci.c +++ b/drivers/xen/pci.c @@ -177,6 +177,18 @@ static int xen_remove_device(struct device *dev) return r; } +int xen_reset_device_state(const struct pci_dev *dev) +{ + struct physdev_pci_device device = { + .seg = pci_domain_nr(dev->bus), + .bus = dev->bus->number, + .devfn = dev->devfn + }; + + return HYPERVISOR_physdev_op(PHYSDEVOP_pci_device_state_reset, &device); +} +EXPORT_SYMBOL_GPL(xen_reset_device_state); + static int xen_pci_notifier(struct notifier_block *nb, unsigned long action, void *data) { diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c index e34b623e4b41..5a96b6c66c07 100644 --- a/drivers/xen/xen-pciback/pci_stub.c +++ b/drivers/xen/xen-pciback/pci_stub.c @@ -421,6 +421,9 @@ static int pcistub_init_device(struct pci_dev *dev) else { dev_dbg(&dev->dev, "resetting (FLR, D3, etc) the device\n"); __pci_reset_function_locked(dev); + err = xen_reset_device_state(dev); + if (err) + goto config_release; pci_restore_state(dev); } /* Now disable the device (this also ensures some private device diff --git a/include/xen/interface/physdev.h b/include/xen/interface/physdev.h index a237af867873..231526f80f6c 100644 --- a/include/xen/interface/physdev.h +++ b/include/xen/interface/physdev.h @@ -263,6 +263,8 @@ struct physdev_pci_device { uint8_t devfn; }; +#define PHYSDEVOP_pci_device_state_reset 32 + #define PHYSDEVOP_DBGP_RESET_PREPARE 1 #define PHYSDEVOP_DBGP_RESET_DONE 2 diff --git a/include/xen/pci.h b/include/xen/pci.h index b8337cf85fd1..b2e2e856efd6 100644 --- a/include/xen/pci.h +++ b/include/xen/pci.h @@ -4,10 +4,16 @@ #define __XEN_PCI_H__ #if defined(CONFIG_XEN_DOM0) +int xen_reset_device_state(const struct pci_dev *dev); int xen_find_device_domain_owner(struct pci_dev *dev); int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain); int xen_unregister_device_domain_owner(struct pci_dev *dev); #else +static inline int xen_reset_device_state(const struct pci_dev *dev) +{ + return -1; +} + static inline int xen_find_device_domain_owner(struct pci_dev *dev) { return -1; From patchwork Fri Nov 24 10:31:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13467487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0CCBBC61DF4 for ; Fri, 24 Nov 2023 10:33:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640387.998603 (Exim 4.92) (envelope-from ) id 1r6TUm-0000i0-Oa; Fri, 24 Nov 2023 10:33:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640387.998603; Fri, 24 Nov 2023 10:33:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6TUm-0000ht-Lk; Fri, 24 Nov 2023 10:33:12 +0000 Received: by outflank-mailman (input) for mailman id 640387; Fri, 24 Nov 2023 10:33:11 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6TUl-0000g8-8j for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 10:33:11 +0000 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2060f.outbound.protection.outlook.com [2a01:111:f400:7eaa::60f]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id dc1f4be5-8ab4-11ee-98e2-6d05b1d4d9a1; Fri, 24 Nov 2023 11:33:09 +0100 (CET) Received: from CY8PR10CA0022.namprd10.prod.outlook.com (2603:10b6:930:4f::23) by BL1PR12MB5207.namprd12.prod.outlook.com (2603:10b6:208:318::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.21; Fri, 24 Nov 2023 10:33:05 +0000 Received: from CY4PEPF0000EDD5.namprd03.prod.outlook.com (2603:10b6:930:4f:cafe::ad) by CY8PR10CA0022.outlook.office365.com (2603:10b6:930:4f::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.21 via Frontend Transport; Fri, 24 Nov 2023 10:33:05 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD5.mail.protection.outlook.com (10.167.241.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7025.12 via Frontend Transport; Fri, 24 Nov 2023 10:33:04 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 24 Nov 2023 04:32:59 -0600 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: dc1f4be5-8ab4-11ee-98e2-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oKAxpiYbBpjb0NMpmXcUsFWlgX0ZKfgvTMvn6wL998PfOsItsUYp4OU/8rEKgyBNAVQLIgVJ6TzJJxT+dJrMY9GnQJb6Jy1I+PHrw0c5kIBOwH77UPOkfcmcesN0aydoIRGaKXfSKP5tZeFPCm7rDaVDInww99qXgiQPn0jNrlrM0nPZ4HIgiQmdEogkaotd2gSfeCz3/ImZcxpfoVihTbTKUGwl+TpT8E6EUVcq66gB0YuVMcGYEAsOvP6vpx/K7DwlI0K21SbaoSdMhfiJOfIDLe2EyPKelB/JnK/zb+WAMogG2SDyC99lAtE5TqLVkdCCoj1A5V2Nbh2e8GY1Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xfANHFWcou1FeEN+wZbXol2ku2zP/OewOlGHwHnCwuw=; b=hE3RPWUXgDqv7ZyJWjpkXhO8iwJ6Auuhifoc8J7dTpjMb4VDxaHAMkfsxh0Avl5i3oNTu60LeT2OltpCANC2kYWiJ9hdODw2TkjliRwW72NJLlWFwyau810blzbiQGSyVVPAeaSbGyiuyc7EdPdBp3KsQQl37lWgv/kcwesJH7+k1jyy14H14l3+X/rC/w0hhEkvG3LOS2sFnaJH5+y4VxTWdcwhsf+MiMYtuiuinmhXVCq46nh9O0qXFjh9D2cW1CWoEPSxf9vgUsglhjcQtKBtX8XDCIUdHnwj/0o+XhEW0i4gutbTXwNR42eTQEWF7xPRJz+JIjxpNYZV6K4S5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xfANHFWcou1FeEN+wZbXol2ku2zP/OewOlGHwHnCwuw=; b=JRwAzIrleJpxCNrADTyIsfYRH5mp2WI0lNSvz/+OsuKxwXGLmvp95RePHv7P8+X3w5mVj9HpOUm4vaSwVdr1PnTMJUd7A7AUtCDeZ8LlO7IuIdmHvSAPzGtRsYFjYLFvSk+GCdGqhl3h9+y8eSDKCnAsrBfvPEtYjO5Z6NLKvaM= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Thomas Gleixner , "Boris Ostrovsky" , "Rafael J . Wysocki" , Len Brown , Bjorn Helgaas , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , , , CC: Stefano Stabellini , Alex Deucher , Christian Koenig , Stewart Hildebrand , Xenia Ragiadakou , Honglei Huang , "Julia Zhang" , Huang Rui , Jiqian Chen , Huang Rui Subject: [RFC KERNEL PATCH v2 2/3] xen/pvh: Unmask irq for passthrough device in PVH dom0 Date: Fri, 24 Nov 2023 18:31:22 +0800 Message-ID: <20231124103123.3263471-3-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231124103123.3263471-1-Jiqian.Chen@amd.com> References: <20231124103123.3263471-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD5:EE_|BL1PR12MB5207:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f2d2249-14f0-4ab2-43a2-08dbecd8be21 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HbOO3cPBX404OwUvQTq+zwaugFWDj/7ufDz1LIQJQkBOpEbJC0ctMgL5dOI7VEDnSwnoh9WlzvE5F/fibO6EGOeHVo+9mNJ91aeA63O6Q0uhmDqoNHV6fajHqfafx+0oZ+h6wLvjvK8HZLsIVuSlvEQWFsua5WWYFIX7m8XmTcLQiC+LNsiVFM63/PFU9dz7/dezHcQE9GVV1udengbgI5Px9sUSrD8Lt/qVOo0BOpW2t2wd+3Rxc7E7yQ7fQDUmBgYtsNuk0A/yCNqxIsfd2Mr4Z9VlUUjpC8GomY5kR+wTPPWsTQeOsp/w45fFS1lBJh6DiLBKq3RIqn68eSY371FTops2eL+9OUjP+29FzPNTFNGtOUETNUztCoyRF1a4aQXmZigGg3V1aOZEQG1EoY2XScfKpfknDGF5C2rfVfwy1C0f50yn17jZdntdofJVUEzwO4P366xb/PJAcq7KgQSJ+eE6NeTfJUSm9IxjvNZgHZzt15c6xIW4jiP6UcT+MCpiiifvqHdZIuAIdQMGkHXH3u8DzJaQM6UJQ2MHgXJcQzR33JfmDPpFVj78ml/tAN17rm5TpA9aeV22mKa0/NHv+tZF0+Zk2cjry8E9exT4pK0FcZu8y4gT6bRZl2N5Iiy+jtaGjtL7creQ6bmHwbGFHXf0wIwizC41kC202+Fct2qh5bGw+1HFVMUDcNkY+v0iRkHJGCIcFeshN4GAiIPnQ6o+P34RrmYzSgTz7KyzzCgYsg6mD9hE7aTz10cE+SQ+I9igFlW+jljEkaj1WHVMIhEAmGtqcp74RgIxNE0= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(396003)(376002)(346002)(136003)(230922051799003)(451199024)(82310400011)(64100799003)(186009)(1800799012)(40470700004)(46966006)(36840700001)(478600001)(40460700003)(81166007)(8936002)(4326008)(8676002)(110136005)(70586007)(316002)(54906003)(70206006)(356005)(36756003)(16526019)(26005)(1076003)(2906002)(336012)(426003)(36860700001)(921008)(47076005)(40480700001)(2616005)(83380400001)(86362001)(41300700001)(5660300002)(7416002)(6666004)(82740400003)(7696005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2023 10:33:04.8592 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2f2d2249-14f0-4ab2-43a2-08dbecd8be21 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD5.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5207 This patch is to solve two problems we encountered when we try to passthrough a device to hvm domU base on Xen PVH dom0. First, hvm guest will alloc a pirq and irq for a passthrough device by using gsi, before that, the gsi must first has a mapping in dom0, see Xen code pci_add_dm_done->xc_domain_irq_permission, it will call into Xen and check whether dom0 has the mapping. See XEN_DOMCTL_irq_permission->pirq_access_permitted, "current" is PVH dom0 and it return irq is 0, and then return -EPERM. This is because the passthrough device doesn't do PHYSDEVOP_map_pirq when thay are enabled. Second, in PVH dom0, the gsi of a passthrough device doesn't get registered, but gsi must be configured for it to be able to be mapped into a domU. After searching codes, we can find map_pirq and register_gsi will be done in function vioapic_write_redirent->vioapic_hwdom_map_gsi when the gsi(aka ioapic's pin) is unmasked in PVH dom0. So the problems can be conclude to that the gsi of a passthrough device doesn't be unmasked. To solve the unmaske problem, this patch call the unmask_irq when we assign a device to be passthrough. So that the gsi can get registered and mapped in PVH dom0. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui --- drivers/xen/xen-pciback/pci_stub.c | 7 +++++++ include/linux/irq.h | 1 + kernel/irq/chip.c | 1 + kernel/irq/internals.h | 1 - kernel/irq/irqdesc.c | 2 +- 5 files changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c index 5a96b6c66c07..b83d02bcc76c 100644 --- a/drivers/xen/xen-pciback/pci_stub.c +++ b/drivers/xen/xen-pciback/pci_stub.c @@ -357,6 +357,7 @@ static int pcistub_match(struct pci_dev *dev) static int pcistub_init_device(struct pci_dev *dev) { struct xen_pcibk_dev_data *dev_data; + struct irq_desc *desc = NULL; int err = 0; dev_dbg(&dev->dev, "initializing...\n"); @@ -399,6 +400,12 @@ static int pcistub_init_device(struct pci_dev *dev) if (err) goto config_release; + if (xen_initial_domain() && xen_pvh_domain()) { + if (dev->irq <= 0 || !(desc = irq_to_desc(dev->irq))) + goto config_release; + unmask_irq(desc); + } + if (dev->msix_cap) { struct physdev_pci_device ppdev = { .seg = pci_domain_nr(dev->bus), diff --git a/include/linux/irq.h b/include/linux/irq.h index 90081afa10ce..44650ca178d9 100644 --- a/include/linux/irq.h +++ b/include/linux/irq.h @@ -659,6 +659,7 @@ extern void handle_percpu_irq(struct irq_desc *desc); extern void handle_percpu_devid_irq(struct irq_desc *desc); extern void handle_bad_irq(struct irq_desc *desc); extern void handle_nested_irq(unsigned int irq); +extern void unmask_irq(struct irq_desc *desc); extern void handle_fasteoi_nmi(struct irq_desc *desc); extern void handle_percpu_devid_fasteoi_nmi(struct irq_desc *desc); diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c index dc94e0bf2c94..fd67b40b678d 100644 --- a/kernel/irq/chip.c +++ b/kernel/irq/chip.c @@ -439,6 +439,7 @@ void unmask_irq(struct irq_desc *desc) irq_state_clr_masked(desc); } } +EXPORT_SYMBOL_GPL(unmask_irq); void unmask_threaded_irq(struct irq_desc *desc) { diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h index bcc7f21db9ee..d08e3e7b2819 100644 --- a/kernel/irq/internals.h +++ b/kernel/irq/internals.h @@ -95,7 +95,6 @@ extern void irq_disable(struct irq_desc *desc); extern void irq_percpu_enable(struct irq_desc *desc, unsigned int cpu); extern void irq_percpu_disable(struct irq_desc *desc, unsigned int cpu); extern void mask_irq(struct irq_desc *desc); -extern void unmask_irq(struct irq_desc *desc); extern void unmask_threaded_irq(struct irq_desc *desc); #ifdef CONFIG_SPARSE_IRQ diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c index 27ca1c866f29..5977efed31b5 100644 --- a/kernel/irq/irqdesc.c +++ b/kernel/irq/irqdesc.c @@ -380,7 +380,7 @@ struct irq_desc *irq_to_desc(unsigned int irq) { return mtree_load(&sparse_irqs, irq); } -#ifdef CONFIG_KVM_BOOK3S_64_HV_MODULE +#if defined CONFIG_KVM_BOOK3S_64_HV_MODULE || defined CONFIG_XEN_PVH EXPORT_SYMBOL_GPL(irq_to_desc); #endif From patchwork Fri Nov 24 10:31:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13467488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2820FC61D97 for ; Fri, 24 Nov 2023 10:33:27 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.640388.998613 (Exim 4.92) (envelope-from ) id 1r6TUu-00019F-0j; Fri, 24 Nov 2023 10:33:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 640388.998613; Fri, 24 Nov 2023 10:33:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6TUt-000193-Tu; Fri, 24 Nov 2023 10:33:19 +0000 Received: by outflank-mailman (input) for mailman id 640388; Fri, 24 Nov 2023 10:33:18 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r6TUr-0000g8-Vk for xen-devel@lists.xenproject.org; Fri, 24 Nov 2023 10:33:18 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on20623.outbound.protection.outlook.com [2a01:111:f400:7ea9::623]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e06c7123-8ab4-11ee-98e2-6d05b1d4d9a1; Fri, 24 Nov 2023 11:33:17 +0100 (CET) Received: from DM6PR03CA0011.namprd03.prod.outlook.com (2603:10b6:5:40::24) by CH3PR12MB9196.namprd12.prod.outlook.com (2603:10b6:610:197::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.21; Fri, 24 Nov 2023 10:33:11 +0000 Received: from CY4PEPF0000EDD1.namprd03.prod.outlook.com (2603:10b6:5:40:cafe::1d) by DM6PR03CA0011.outlook.office365.com (2603:10b6:5:40::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.21 via Frontend Transport; Fri, 24 Nov 2023 10:33:11 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD1.mail.protection.outlook.com (10.167.241.205) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7025.12 via Frontend Transport; Fri, 24 Nov 2023 10:33:11 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 24 Nov 2023 04:33:04 -0600 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e06c7123-8ab4-11ee-98e2-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mgTjMI1DDRivQcdWBDMnin9cWH1Si2VOfk06dmQa35bskeTB5/FlvQ1RT9tTQNrHycU2fSDmk3nHFEf2zed6FMUDON8FxQq3Tey7nupl34bVpTlTCwak/csHALGGv4sgeJIrb6AsPOv0Z8Q/zrCHUjh2yFRYhZpdjfM1zpysk+okaZSTtgt62BQmbTRfB+UExepKcNuP0c+HAAfGNUAv5WZBBTKiBFoGRCmDn5LrkawoVLsgdAcfDZxDBqhwBXrvgjCrsmOK9qNlOcl5hBSRCCuYY5hSjew+p8ZyjXCDG2llp3DX3f7K0eakvYSi/tb2BB5cDBR3OeK246KR0ge8NA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CxiFRDtp5qqk6Bo0ku32SWQukLFTF8Compi0RCGR9bw=; b=KB9YWq/9lGG2S6be55uerEVZLcnL5tLTdb1swsh+B6Thdr0r+sn3BCgE+9e/Gz1BVREz+O1Lf6MDfGdmdOrjRt4ZiUXWQhxXqf2G7kxoqc9qD/Cz6seM/F5mJmWDtinLM5ba35i9o2J96B58ljGqPHf1T26HadKuAqCm6KDh7YsgpH+xXU2/9MEXLkOF+MNkbWoIyiIqXcalYdT1R8dkF2KcKZNV/iSeBkj64ws91O6H7Nxq3kNpNBS7ILzHEhJFVTraW/U4l+SMA1WOjU6aEwBlBvHExZPlpmMjeW4wgZzw4oq1IbnkkC3ctBrYoNkei1t1jW4JcmF/ODewpfWulg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CxiFRDtp5qqk6Bo0ku32SWQukLFTF8Compi0RCGR9bw=; b=msTqFvnRpN8xjA7liweI1Ed94Es5KNBUKc1CKyr4iSeG4zW85JbUYetwPKqz1NI6r/x8UlHZLuLWkeZtxQuGUAfktNApZnbztFXgKXDXpHTdwvocOVGcaFg12vETgAVyP/ztrasBt+WQg5jlrDkYE+SqncGDnTIzS3/z5Aus6E0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Thomas Gleixner , "Boris Ostrovsky" , "Rafael J . Wysocki" , Len Brown , Bjorn Helgaas , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , , , CC: Stefano Stabellini , Alex Deucher , Christian Koenig , Stewart Hildebrand , Xenia Ragiadakou , Honglei Huang , "Julia Zhang" , Huang Rui , Jiqian Chen , Huang Rui Subject: [RFC KERNEL PATCH v2 3/3] xen/privcmd: Add new syscall to get gsi from irq Date: Fri, 24 Nov 2023 18:31:23 +0800 Message-ID: <20231124103123.3263471-4-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231124103123.3263471-1-Jiqian.Chen@amd.com> References: <20231124103123.3263471-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD1:EE_|CH3PR12MB9196:EE_ X-MS-Office365-Filtering-Correlation-Id: 80ea1451-a9bb-427e-cf1a-08dbecd8c1e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qNlkV3KqrC0RcB1olsj+8U2+ITAJfIdDp+h2cntHYuQMWOmB+0cxyXJ+14LBLnvlJrIvGAv5IVYEf8PXuuzQnj9B1jGYso0DxwYAFLnBMykDWt9p7cQVB6W9sCBxsC0pyCJLQkBC5YjHmBJzBEPx7pXFPbKiK5+weT2SskjYhPY31DrwEXrCzNbVRMcbkxsLyr9g422xYvxUhcduWLHGhGcfxh8Gp5K7Hue+9ETbvLCiddAmodXERWNO4QC3E1BhZI4EtJfyyqyL+3+SkmOicm1TwHwEMV+zehY3b7aAoj24/LfqCZni2oLoyshCzp2vjb7tpWgGoPz+4iZQ4wME+Opl1a59MmpmulPMhWG1jbyv+bApVDnJb58HGH4jzUAPmUljfJ0xGufEF35KT4ZoEcv3RkgxuLxIG3tAuWV1fb4Ky6Lk3k0WSCUyc2L+jbakE27Uy+53WCEzVp+U1aE0hcOzLg93WeM4FJDeDCZ9OeUR2QU77p/neNVXxIzJT0hnmgrPenbMOFEgqwtGzjCK0pzu3uSyLg4Dwr+bxwOdVJP+dTMbeYxUoUuYW1OYuiu3pOn98o3svHDYYeLHlW5DUK3UidG1TsbuIgnt5GQyuJzcjW6nZVYabHQ1QuinxoPHI070uaeHGBjB3glGYETXQhvHudGS3OLDaRBMSxmTQs3/KN/8PCsKW/PpksjznNiE8VbvXXz5t12gmHhN7W88L5gGloncafAY6QVcIbpmgpB2KAwHPVsyXDhsCJGH0Dd2bGriAD+pbf0EGVZRQKhj9tT/2JljvRc1860UlRQynpQ= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(346002)(396003)(39860400002)(376002)(230922051799003)(1800799012)(64100799003)(186009)(451199024)(82310400011)(36840700001)(40470700004)(46966006)(36756003)(41300700001)(36860700001)(2616005)(356005)(47076005)(16526019)(426003)(336012)(26005)(40480700001)(7696005)(81166007)(1076003)(83380400001)(82740400003)(40460700003)(478600001)(6666004)(110136005)(316002)(70206006)(70586007)(54906003)(2906002)(921008)(8936002)(8676002)(4326008)(86362001)(7416002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2023 10:33:11.2117 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 80ea1451-a9bb-427e-cf1a-08dbecd8c1e5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD1.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9196 In PVH dom0, it uses the linux local interrupt mechanism, when it allocs irq for a gsi, it is dynamic, and follow the principle of applying first, distributing first. And if you debug the codes, you will find the irq number is alloced from small to large, but the applying gsi number is not, may gsi 38 comes before gsi 28, it causes the irq number is not equal with the gsi number. And when we passthrough a device, QEMU will use device's gsi number to do mapping actions, see xen_pt_realize-> xc_physdev_map_pirq, but the gsi number is got from file /sys/bus/pci/devices/xxxx:xx:xx.x/irq, irq!= gsi, so it will fail when mapping. And in current linux codes, there is no method to translate irq to gsi for userspace. For above purpose, this patch record the relationship of gsi and irq when PVH dom0 do acpi_register_gsi_ioapic for devices and adds a new syscall into privcmd to let userspace can get that translation when they have a need. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui --- arch/x86/include/asm/apic.h | 7 ++++++ arch/x86/include/asm/xen/pci.h | 5 ++++ arch/x86/kernel/acpi/boot.c | 2 +- arch/x86/pci/xen.c | 21 +++++++++++++++++ drivers/xen/events/events_base.c | 39 ++++++++++++++++++++++++++++++++ drivers/xen/privcmd.c | 20 ++++++++++++++++ include/uapi/xen/privcmd.h | 7 ++++++ include/xen/events.h | 5 ++++ 8 files changed, 105 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index d21f48f1c242..5646444285ac 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -169,6 +169,8 @@ extern bool apic_needs_pit(void); extern void apic_send_IPI_allbutself(unsigned int vector); +extern int acpi_register_gsi_ioapic(struct device *dev, u32 gsi, + int trigger, int polarity); #else /* !CONFIG_X86_LOCAL_APIC */ static inline void lapic_shutdown(void) { } #define local_apic_timer_c2_ok 1 @@ -183,6 +185,11 @@ static inline void apic_intr_mode_init(void) { } static inline void lapic_assign_system_vectors(void) { } static inline void lapic_assign_legacy_vector(unsigned int i, bool r) { } static inline bool apic_needs_pit(void) { return true; } +static inline int acpi_register_gsi_ioapic(struct device *dev, u32 gsi, + int trigger, int polarity) +{ + return (int)gsi; +} #endif /* !CONFIG_X86_LOCAL_APIC */ #ifdef CONFIG_X86_X2APIC diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h index 9015b888edd6..aa8ded61fc2d 100644 --- a/arch/x86/include/asm/xen/pci.h +++ b/arch/x86/include/asm/xen/pci.h @@ -5,6 +5,7 @@ #if defined(CONFIG_PCI_XEN) extern int __init pci_xen_init(void); extern int __init pci_xen_hvm_init(void); +extern int __init pci_xen_pvh_init(void); #define pci_xen 1 #else #define pci_xen 0 @@ -13,6 +14,10 @@ static inline int pci_xen_hvm_init(void) { return -1; } +static inline int pci_xen_pvh_init(void) +{ + return -1; +} #endif #ifdef CONFIG_XEN_PV_DOM0 int __init pci_xen_initial_domain(void); diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c index d0918a75cb00..45b157e18c0b 100644 --- a/arch/x86/kernel/acpi/boot.c +++ b/arch/x86/kernel/acpi/boot.c @@ -739,7 +739,7 @@ static int acpi_register_gsi_pic(struct device *dev, u32 gsi, } #ifdef CONFIG_X86_LOCAL_APIC -static int acpi_register_gsi_ioapic(struct device *dev, u32 gsi, +int acpi_register_gsi_ioapic(struct device *dev, u32 gsi, int trigger, int polarity) { int irq = gsi; diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index 652cd53e77f6..f056ab5c0a06 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -114,6 +114,21 @@ static int acpi_register_gsi_xen_hvm(struct device *dev, u32 gsi, false /* no mapping of GSI to PIRQ */); } +static int acpi_register_gsi_xen_pvh(struct device *dev, u32 gsi, + int trigger, int polarity) +{ + int irq; + + irq = acpi_register_gsi_ioapic(dev, gsi, trigger, polarity); + if (irq < 0) + return irq; + + if (xen_pvh_add_gsi_irq_map(gsi, irq) == -EEXIST) + printk(KERN_INFO "Already map the GSI :%u and IRQ: %d\n", gsi, irq); + + return irq; +} + #ifdef CONFIG_XEN_PV_DOM0 static int xen_register_gsi(u32 gsi, int triggering, int polarity) { @@ -558,6 +573,12 @@ int __init pci_xen_hvm_init(void) return 0; } +int __init pci_xen_pvh_init(void) +{ + __acpi_register_gsi = acpi_register_gsi_xen_pvh; + return 0; +} + #ifdef CONFIG_XEN_PV_DOM0 int __init pci_xen_initial_domain(void) { diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 6de6b084ea60..a02d62955509 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -957,6 +957,43 @@ int xen_irq_from_gsi(unsigned gsi) } EXPORT_SYMBOL_GPL(xen_irq_from_gsi); +int xen_gsi_from_irq(unsigned irq) +{ + struct irq_info *info; + + list_for_each_entry(info, &xen_irq_list_head, list) { + if (info->type != IRQT_PIRQ) + continue; + + if (info->irq == irq) + return info->u.pirq.gsi; + } + + return -1; +} +EXPORT_SYMBOL_GPL(xen_gsi_from_irq); + +int xen_pvh_add_gsi_irq_map(unsigned gsi, unsigned irq) +{ + int tmp_irq; + struct irq_info *info; + + tmp_irq = xen_irq_from_gsi(gsi); + if (tmp_irq != -1) + return -EEXIST; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (info == NULL) + panic("Unable to allocate metadata for GSI%d\n", gsi); + + info->type = IRQT_PIRQ; + info->irq = irq; + info->u.pirq.gsi = gsi; + list_add_tail(&info->list, &xen_irq_list_head); + + return 0; +} + static void __unbind_from_irq(unsigned int irq) { evtchn_port_t evtchn = evtchn_from_irq(irq); @@ -2303,6 +2340,8 @@ void __init xen_init_IRQ(void) xen_init_setup_upcall_vector(); xen_alloc_callback_vector(); + if (xen_pvh_domain()) + pci_xen_pvh_init(); if (xen_hvm_domain()) { native_init_IRQ(); diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c index 1ce7f3c7a950..6fa8a01d8ae6 100644 --- a/drivers/xen/privcmd.c +++ b/drivers/xen/privcmd.c @@ -45,6 +45,7 @@ #include #include #include +#include #include "privcmd.h" @@ -842,6 +843,21 @@ static long privcmd_ioctl_mmap_resource(struct file *file, return rc; } +static long privcmd_ioctl_gsi_from_irq(struct file *file, void __user *udata) +{ + struct privcmd_gsi_from_irq kdata; + + if (copy_from_user(&kdata, udata, sizeof(kdata))) + return -EFAULT; + + kdata.gsi = xen_gsi_from_irq(kdata.irq); + + if (copy_to_user(udata, &kdata, sizeof(kdata))) + return -EFAULT; + + return 0; +} + #ifdef CONFIG_XEN_PRIVCMD_EVENTFD /* Irqfd support */ static struct workqueue_struct *irqfd_cleanup_wq; @@ -1534,6 +1550,10 @@ static long privcmd_ioctl(struct file *file, ret = privcmd_ioctl_ioeventfd(file, udata); break; + case IOCTL_PRIVCMD_GSI_FROM_IRQ: + ret = privcmd_ioctl_gsi_from_irq(file, udata); + break; + default: break; } diff --git a/include/uapi/xen/privcmd.h b/include/uapi/xen/privcmd.h index 8b8c5d1420fe..61f0ffbec077 100644 --- a/include/uapi/xen/privcmd.h +++ b/include/uapi/xen/privcmd.h @@ -126,6 +126,11 @@ struct privcmd_ioeventfd { __u8 pad[2]; }; +struct privcmd_gsi_from_irq { + __u32 irq; + __u32 gsi; +}; + /* * @cmd: IOCTL_PRIVCMD_HYPERCALL * @arg: &privcmd_hypercall_t @@ -157,5 +162,7 @@ struct privcmd_ioeventfd { _IOW('P', 8, struct privcmd_irqfd) #define IOCTL_PRIVCMD_IOEVENTFD \ _IOW('P', 9, struct privcmd_ioeventfd) +#define IOCTL_PRIVCMD_GSI_FROM_IRQ \ + _IOC(_IOC_NONE, 'P', 10, sizeof(struct privcmd_gsi_from_irq)) #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */ diff --git a/include/xen/events.h b/include/xen/events.h index 23932b0673dc..78377c936efe 100644 --- a/include/xen/events.h +++ b/include/xen/events.h @@ -131,6 +131,11 @@ int xen_pirq_from_irq(unsigned irq); /* Return the irq allocated to the gsi */ int xen_irq_from_gsi(unsigned gsi); +/* Return the gsi from irq */ +int xen_gsi_from_irq(unsigned irq); + +int xen_pvh_add_gsi_irq_map(unsigned gsi, unsigned irq); + /* Determine whether to ignore this IRQ if it is passed to a guest. */ int xen_test_irq_shared(int irq);