From patchwork Sat Dec 2 01:27:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D5A9C10F09 for ; Sat, 2 Dec 2023 01:27:39 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645834.1008282 (Exim 4.92) (envelope-from ) id 1r9Emw-0003DA-Pw; Sat, 02 Dec 2023 01:27:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645834.1008282; Sat, 02 Dec 2023 01:27:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emw-0003A8-K0; Sat, 02 Dec 2023 01:27:22 +0000 Received: by outflank-mailman (input) for mailman id 645834; Sat, 02 Dec 2023 01:27:20 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emu-000272-Cp for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:20 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id eec6a01c-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:19 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGuu008549; Sat, 2 Dec 2023 01:27:09 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:08 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:03 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:03 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: eec6a01c-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DslJoTQFHmjSct0YQDC2OWdL/+ojFYhj50ygnLXpLNQkPqhLvS7MgjVyHoZZN60d7wABqDUxELiDd0vxybFBAHi3Y6W0R1avuDYL6ROEPLcWFL9T0M1voeZGo/s1j8WTWwwOpEID2QlkIKXQ/GUEX7ZWAc+Mo9e0l1EQqAnkAtZJNmnWqyRWbv7olqssLTYIub1SOSfhzXgmTQyrSmT6aNlLRWHZGdMdlF9MAl71K+OTMkISX4DO4/hLDlO+ZnNOH92zRXc0XcE7ZiObfQkuOjcfPBB6qTKk9gChMIsz1JFjHKzG4ZR3+rI384EA9cjw19Pf/UyAM2V5PqpDVbCU0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4G495MYzvuMQHdmRAHaXxuT/rdye2JvaiP/jgpIr1+U=; b=jMLzmViePie6sO8v8VzBFTePUfPA5N24JKZ+JXANlJP2tX1DyLJrXrrA5QBKFjxIc5hszrjAs/IxIMTeVCrCwQJptagBXScMliC4n57vr0UgGFb044tH+W43D5Bnl7NvRHT3GjEua1mJtsjwQDIg8gmIS61WKxfIyyJ4K1LyPBIj1uwYlfbEuPGHGbJpo21BLSS1ZJljxnPovvBGuDuU38ZAx3NHdK0iA1gIpmrip6VXZpJHJ5XRuMxmeWmTUxJz5cpDyzDpsgnl7L3WdjLdPNmNaI4F3vAJcuZv81qcauSrImIxR+VQNlXQrvMNkudRCzEt5XKTM1hmQp6ZmLl8YA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4G495MYzvuMQHdmRAHaXxuT/rdye2JvaiP/jgpIr1+U=; b=IIr6XWQPw1Hrcb/D6C5jn8eWkaa6+NCHKdZmvcJG380Q8zKEl2revyJhNkfnoInRWtrbxZn4ELTAHphnTva6Tx+7aaxVm940XfngN3jK+iVDM7OW9EJ9Awxlf3wxhtNgfELFDNisfMthVE53AbFt+VQQFkYp2rkFKCX4G1B/eDdDS3eRUJP8LPE6EGcnRxo0lYcmeRVEcJqZShJ7VZ4o08SaK95B7tpI0Bf7vWG0OaoKbTgOOAPs0Pr8Vx2z3V19ciT2J3fDdCMYj/jmnZctCzXOiUaCNWcCdGDPKb6jvSJc9Yks5+LY/01XliWfsUhOfSmkK4Bx8SRfxFQVSs+wkQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Volodymyr Babchuk , Jan Beulich , Andrew Cooper , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v11 01/17] pci: msi: pass pdev to pci_enable_msi() function Thread-Topic: [PATCH v11 01/17] pci: msi: pass pdev to pci_enable_msi() function Thread-Index: AQHaJL6ns+Jw4l4utUGjbwsXvB/LmQ== Date: Sat, 2 Dec 2023 01:27:02 +0000 Message-ID: <20231202012556.2012281-2-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: d518bee5-c3ca-48e4-6d3f-08dbf2d5c9ed x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: chqwC0f3SIFfrzc24kdrEDAdU4FEZMZosOlprBGYeT2H5yQmW9RlgRcez5oDbISKg6VDgkupDnQ+Fso9BcI4l5umcGdch6XRQMxnKQAlzh/AOMhihB4nIv+CZxPpdskjIl13eHrZsKVz6NmoegSCSCNMcIMxrbgYSNvIvzT7O6gnb6DKv5YOqVyTvYpdebLa3qEIF7rxf9TnPdEAVivEDvgRbxZxvd9c9F4cQzi3RhqcPx3RJu33apmhkOHcEnfT4Iodw9MG3VeCMteNXHk+F/aXsI0jdfDJ5Bl7i9rpPIl5BoGdzwfs99KrE8brrdUFqg3bVJYrgvVcJFOrrlvUjry+oehykAO9dHz2q9kGi5b1w6WepPz2tRyievD13nwH9FBqvhjbTH1cJVxevoqe96V6BhzoRI0iKav1TukWNcmkHkMz6myLJOEFrCAcxl5QsGW0vDHTkqoQeOCrWN4mLGkrhIEdjCcqUh4oB6GS0fnC+40R/6mv3fziu3UTQNjDt7F+Ku/ta3nW+P4Fkz4mC6kr/4khlFPO/3XUhNe9OCTxbgnRn3UUeg9agjUCjUKSphWT2Ip5o/xqmzQvKVMVMs0lwc9RdtWcabbLtZJ1YihoWzZlXSJL11+iEvyuJ3YK x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?zIDxl/XMTKd7aTsYYVC/t5G?= =?iso-8859-1?q?YEgM0tLTB7BJGmzvLPZ/mgFY163A5F2/GgyxnKBsTi9RkkAx3eXiY1zJ1o/W?= =?iso-8859-1?q?jHKDBtsVmVKv58AhPHbrxZ8hg7xYlQsAssJvpzNzzUG1jdHoYNMmR+hnTSEJ?= =?iso-8859-1?q?OEsXgG9fR8SP2kx480Y9aKIGmc/MjKZg0kxVYnPx/Ho/0A1bnA2kZQrRw4c+?= =?iso-8859-1?q?CTuF8n3oqSSBiSxHgulHB/VaAerI5RQedl4TJdFy3DcxHGxVvCkFauqOOFZE?= =?iso-8859-1?q?DotvmK7o0sg6eokUcORZtunWuWSKrMD13hbfkJigE8XeuC/6BYefX9TD4+rI?= =?iso-8859-1?q?OAhA46HUjo3s6jGoFM+fsOsxHm5/jqonXZAh0CZI5rnQ50wHysW97ruBJpRd?= =?iso-8859-1?q?nLWGLWZr3+2bYCNtz/Rdm7Hh+aO6uROKE3ZDa0aEj1vnv7UdzQ5mn7oZG5Ft?= =?iso-8859-1?q?PXIfOtGJ2FRZfwLf55LQl8IwBjQGPChkyPX9JDJXv+yxy6M3gyfS6KebumsC?= =?iso-8859-1?q?ug3vhyLEvVv3YPlE7j7DSoZ+TkaVi5xXPBztI4XF+3USZ8ge9Jw+P1o+sBmM?= =?iso-8859-1?q?1FCRorCEdXGQxMfOFlLN5Yoscad3cIiRMBBcMDvdg/TFD9ZRYkVxQRyaHTeP?= =?iso-8859-1?q?qp7lKBTirDz5UeuQdc0yG7aYv/sUrbAxe9H0rxbHJITuwXLndjPy9/XoCxYl?= =?iso-8859-1?q?EJuZbXs7Z3naw5LEQIWwlXN/ieE4j9h7U7JvHi3cumH6ruAAjOI4WALWLUMk?= =?iso-8859-1?q?Yj4WeDeBWYDp92Y0FgT2GiRdh2UE2ZgmZDhslhu0bIxgZzYR47wE2Zv+1Xqe?= =?iso-8859-1?q?7LsZY1w1yYTXQ09cW6xCka1Y8zJwwA/ZH4hTHBltnknlpjivC4qF2t0Boqfh?= =?iso-8859-1?q?4LeV66UV9jF+IEkYcnO0A0oqfAxPEUzhRw6xspwm1AexGaRXm9Qz0LvAQS7O?= =?iso-8859-1?q?+u5R7KNyshNOwNuRJT5J3TktvnbGoVAZgj7rclkG4bxJC/3h3v64m7RoD3Uu?= =?iso-8859-1?q?nFedkIG6UPupY8ZhkhvNqK7eU5LCmz046AVgQm7NAttTKVNroMy6qn0nv5Eh?= =?iso-8859-1?q?Pcg5UMxtNoP2ZsyS/KobXFDm6MAY6z6UELJeYCHCgk0/3K8OtJ4WNaAsX4ch?= =?iso-8859-1?q?itqZKfe7AfohrFTPgIxkxhu4bNu73YtpaCuwqwo6YdytcMEiCzc2Ti5cVuPA?= =?iso-8859-1?q?mfjq+tg6TQJPI3l+HhbOsZNu2LHmcY+mr43CjQVRJIeDSy0fJRnXcsq2oN6V?= =?iso-8859-1?q?hixidzob3JHKOpHuB+WXMD0l0mRdr5FrMJdJsaCbj0Y5H8ZPCq5obUfLJWF8?= =?iso-8859-1?q?WTtRahsS/cQvXX45c5LtYtBG7YTRrvWXkjJU3EgTlA2G+IQXb+1q80p9pjr3?= =?iso-8859-1?q?SCKBa1Bt5lyGe1aOroggdcagJnca28nS+HJpeuOZoPOkwG+W3Pe0tUQUCYAb?= =?iso-8859-1?q?C38qX2VUmu4/D4Aicoc0mTMdg0p3SJxiuL81/LVAEHkZ4eiwQgoivJotPI/I?= =?iso-8859-1?q?oK/ZKzEqdsIMf1tzKse1GIpxDvlMsnFIelMNKEyeC7F4u1zlEk5nyGo9EW7w?= =?iso-8859-1?q?Tikggr/066qpNENIXjOoXswC64mcB2mr6CzXwcXhJq8Y+eDh4MHnw0Jw8/ei?= =?iso-8859-1?q?Pt6n25GREmSnDZwMl4RK2ae32EZFpMgkiqAvFCQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d518bee5-c3ca-48e4-6d3f-08dbf2d5c9ed X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:02.5985 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8fTZhMk9O60ROip8imXIkZbsfZv6URjP/oouUUWSXhonpxYcA8WlB/63TL7vcPT4aIgoHMxKXbW5FbzcPMP4gnS5e2ket0AdIhUaT/CKjow= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: VsgJcOmBAdO4fmSzjT8sVc2vOJro-q6x X-Proofpoint-ORIG-GUID: VsgJcOmBAdO4fmSzjT8sVc2vOJro-q6x X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=441 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 Previously pci_enable_msi() function obtained pdev pointer by itself, but taking into account upcoming changes to PCI locking, it is better when caller passes already acquired pdev pointer to the function, because caller knows better how to obtain the pointer and which locks are needed to be used. Also, in most cases caller already has pointer to pdev, so we can avoid an extra list walk. Signed-off-by: Volodymyr Babchuk Reviewed-by: Stefano Stabellini --- In v11: - Made pdev parameter very first in pci_enable_msi() and friends. - Extended the commit message - Added check for pdev into ns16550 driver - Replaced hard tabs with spaces Changes in v10: - New in v10. This is the result of discussion in "vpci: add initial support for virtual PCI bus topology" --- xen/arch/x86/include/asm/msi.h | 5 +++-- xen/arch/x86/irq.c | 2 +- xen/arch/x86/msi.c | 19 ++++++++++--------- xen/drivers/char/ns16550.c | 28 ++++++++++++++++++---------- 4 files changed, 32 insertions(+), 22 deletions(-) diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h index c1ece2786e..07b3ee55e9 100644 --- a/xen/arch/x86/include/asm/msi.h +++ b/xen/arch/x86/include/asm/msi.h @@ -81,8 +81,9 @@ struct irq_desc; struct hw_interrupt_type; struct msi_desc; /* Helper functions */ -extern int pci_enable_msi(struct msi_info *msi, struct msi_desc **desc); -extern void pci_disable_msi(struct msi_desc *msi_desc); +extern int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi, + struct msi_desc **desc); +extern void pci_disable_msi(struct msi_desc *desc); extern int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off); extern void pci_cleanup_msi(struct pci_dev *pdev); extern int setup_msi_irq(struct irq_desc *desc, struct msi_desc *msidesc); diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index 6e668b1b4f..50e49e1a4b 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2176,7 +2176,7 @@ int map_domain_pirq( if ( !pdev ) goto done; - ret = pci_enable_msi(msi, &msi_desc); + ret = pci_enable_msi(pdev, msi, &msi_desc); if ( ret ) { if ( ret > 0 ) diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index 7f8e794254..335c0868a2 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -983,13 +983,13 @@ static int msix_capability_init(struct pci_dev *dev, * irq or non-zero for otherwise. **/ -static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) +static int __pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi, + struct msi_desc **desc) { - struct pci_dev *pdev; struct msi_desc *old_desc; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(NULL, msi->sbdf); + if ( !pdev ) return -ENODEV; @@ -1038,13 +1038,13 @@ static void __pci_disable_msi(struct msi_desc *entry) * of irqs available. Driver should use the returned value to re-send * its request. **/ -static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) +static int __pci_enable_msix(struct pci_dev *pdev, struct msi_info *msi, + struct msi_desc **desc) { - struct pci_dev *pdev; struct msi_desc *old_desc; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(NULL, msi->sbdf); + if ( !pdev || !pdev->msix ) return -ENODEV; @@ -1151,15 +1151,16 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) * Notice: only construct the msi_desc * no change to irq_desc here, and the interrupt is masked */ -int pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) +int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi, + struct msi_desc **desc) { ASSERT(pcidevs_locked()); if ( !use_msi ) return -EPERM; - return msi->table_base ? __pci_enable_msix(msi, desc) : - __pci_enable_msi(msi, desc); + return msi->table_base ? __pci_enable_msix(pdev, msi, desc) : + __pci_enable_msi(pdev, msi, desc); } /* diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c index ddf2a48be6..cfe9ff8d2a 100644 --- a/xen/drivers/char/ns16550.c +++ b/xen/drivers/char/ns16550.c @@ -452,21 +452,29 @@ static void __init cf_check ns16550_init_postirq(struct serial_port *port) if ( rc > 0 ) { struct msi_desc *msi_desc = NULL; + struct pci_dev *pdev; pcidevs_lock(); - rc = pci_enable_msi(&msi, &msi_desc); - if ( !rc ) + pdev = pci_get_pdev(NULL, msi.sbdf); + if ( pdev ) { - struct irq_desc *desc = irq_to_desc(msi.irq); - unsigned long flags; - - spin_lock_irqsave(&desc->lock, flags); - rc = setup_msi_irq(desc, msi_desc); - spin_unlock_irqrestore(&desc->lock, flags); - if ( rc ) - pci_disable_msi(msi_desc); + rc = pci_enable_msi(pdev, &msi, &msi_desc); + + if ( !rc ) + { + struct irq_desc *desc = irq_to_desc(msi.irq); + unsigned long flags; + + spin_lock_irqsave(&desc->lock, flags); + rc = setup_msi_irq(desc, msi_desc); + spin_unlock_irqrestore(&desc->lock, flags); + if ( rc ) + pci_disable_msi(msi_desc); + } } + else + rc = -ENODEV; pcidevs_unlock(); From patchwork Sat Dec 2 01:27:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2D1EC4167B for ; Sat, 2 Dec 2023 01:27:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645844.1008378 (Exim 4.92) (envelope-from ) id 1r9En9-00069F-9S; Sat, 02 Dec 2023 01:27:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645844.1008378; Sat, 02 Dec 2023 01:27:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En8-00067B-BM; Sat, 02 Dec 2023 01:27:34 +0000 Received: by outflank-mailman (input) for mailman id 645844; Sat, 02 Dec 2023 01:27:30 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En3-000272-KR for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:29 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f28e4afa-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:25 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGuv008549; Sat, 2 Dec 2023 01:27:10 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:09 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:05 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:04 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f28e4afa-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EtsSzTTFKfMxr9RT3u7c667tfDkp40sFebXgbVBZl2pfkgxTcOQhVvsVtqSqgJMnsiEYM25PHvLkUR9u7lHnYd2Fsfm1mHQNiHmMTPRv9XaGFypUvJ0N1v2Q3cfBaNCIjFDxgl+D70MMlwzIHIZSEXPi1Qgw+uqLazQUYQkVQ4x3zHoB9yZIMa//ftyevRG7SK5xBMJQY3fjVrHZfECtBj/ykbjhjfZC6UCec63e5DmeLVBIyW3V5WOrK+HDZ8OvhP7Tsqo0/sE2LBij1D3Qja+fItcnsiC7u8AJs21Q518A4nnX+Zl8Z6v9tJKGiYyOggPx7lFBhCMYFTUXnyIsrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IICNCcNAN0CxumVVzE/EyahOFwl5SBHiAYz4cBvI7SY=; b=KPSiaLdszceynAJ4498AxfHYvxQSkaCsJu4sfh5jAinQIWoF8l4QLtXMJQUsmkFsqu74d6GwjW3ipyYoVL3vVr7RSQsPS9Zv42p892Eq0VHzmegIEMawguZ0bxOWatubmo2IVe+uMMHMWRt7GLE8EP/wqrDM3JjTFSagmgRH5Vkz6cwrXEdOxcckavX/9hj4d2+s8VRRwpKiAdDOeoR6JHN9X/jnVnNXNEIaZ7scuowcoTcXYwkJyF2avO1ZntZJQu5IO5Vl2y08Yp3eHcc5L8S9KyDjNo3Bavl9ZRS6ovKjb7N0KD3W1NlioH5SwmCfvQszE6qKEahRP54ZdJHsvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IICNCcNAN0CxumVVzE/EyahOFwl5SBHiAYz4cBvI7SY=; b=cy94FqguVr4bbMwcACAJYI7hwhmvWhith8Pdx0FlH9CyPXyatOFwGQEgZeWIY7cGxCAlLYDIKzI+gVRNEdu/BZohljtDs9wRxB3iDCYHAo1o0PyTF8tXEaVl6lt3fQ8njw7ToySLgbWRl93TvpqmymSfSv3txKZu20pUtzfZhOUkqrFwm6OCfFVJ3wBZX2K00fFX/tk655yUzMdREjkf2hVKqVdeJOBxv0EUOz4I9rnXLLgCW3eLiT5a6G7PfxsbuPA5PgFoZ7mOySpQUqI8SG30Algr825S1NsPjKJL/nFG5W8c+CW2k/ZCvRJczKXXQph0wAqzFPp7gHeQ38jeiQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Paul Durrant , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= , Kevin Tian Subject: [PATCH v11 02/17] pci: introduce per-domain PCI rwlock Thread-Topic: [PATCH v11 02/17] pci: introduce per-domain PCI rwlock Thread-Index: AQHaJL6n2Oa1PKslt0Wo3rzC+KkVjQ== Date: Sat, 2 Dec 2023 01:27:02 +0000 Message-ID: <20231202012556.2012281-3-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 9001b78d-5e3f-4a6d-8e86-08dbf2d5caea x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: e0GE2z6cPR+Kk8pM706jF6sVXMbpqpThb0Q4GOjknq/TdSc8kmcS1fpUYUVDJVH7PJIDF0CVuLDrXWGH5CbdLt3Zek0DJavc8A4cDI5fSzqxhjBAqzggXk98cigogP2lF3yGQXRKP3IeZrRt0X83dD/3DyQ6/SE2QOkT/QrwaTAahOh1qm3uovi+P93rX7aRJsGUH8CxoZ0W66xDgSf9s9FoX4enWjGMUEZjbf2886FWYeoVc3jgg5zDgRktR7k2zw707lpvPMQyUkYHP3erVWqvTCNo11ENaYMlqfk7lBKma1SlFVcX053RkiODCtsaQrrfQbAfOEIZKWvqABesQIlqRFqqLx57QsGRu0viM8IYdFej8BW5a5jSBqYk/yvMPX9qx9wgLa3KjDZKVgmLJeQ6jhWNLVMn8OCiKcuSsB9+KY5NllrOKMxAxGVtU9qqKA3ykWeoX6Jvr5tmIfxxUU70o2dA/xWXDHbc8F0neBPgdT6+2TTkWYTQ1mEhaxo9ue8ULwJLzNVlnCYafxKYmpgm1n4qd7/ghN0jjuJ9S6HIZ75jNIXduybNBFjcH1exZITpGyu7Tr/0UtxH0HSLDKMv2D0tXPxb16cGbW1BDod9ceEaYzmms5SS3jW6eDbM x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(7416002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?P5BO2oYAzU5fkC7pj/zfZYlgZ/aL?= =?utf-8?q?XLzpUw3CaPXjuQgfb1Dnm4VX7w2RD8ElND/7XzG0OmoNv4cCffEqJ8hmr5JpZbq/i?= =?utf-8?q?yv3KzEJinxjkooGpYC76vIREq3fBuiGD68ZxlSveEV2+YbaYNi3y5/SrNxCKGQ2l9?= =?utf-8?q?MxZEnqNDjY6YcVmmQGuHY+WOdFmazZFNTLDZavhOjEav+2aUZ7b5eUtGpX7rZm5PL?= =?utf-8?q?8dj0XyrXNalot/vRhyXk5ufhO3UYzygK/qE8sRVE0RFevvW5c98+JDlJOoHTFqqcN?= =?utf-8?q?FWyMcgXCzvogWD/KUXly1YM5kLGdyyB169LRwnrSXyYR5sh9GJ7omajrDDwI7ST/4?= =?utf-8?q?B1iiel/q4KaAtLWAXg1rPjr45o2szh9i6KzzuFGo6tM4IBVnuvkOPan5vtoc6ojCb?= =?utf-8?q?HrX1ihNMe8Z2Dgkf9KMa8mSMLAROWIvS9cvzDi5lxoLM2x3JqVECCxFTGsFdhHDld?= =?utf-8?q?Dmai7bAHugnqOVNEpBqlVzujnlxA9cQJlDcN8r9ajXWyMqr2blS9A8zGaKi6UbQI+?= =?utf-8?q?OEDQM0i41KUBCjWv4aWnJq3BDwpNIi2r7P68xKVoOZ6g+e0B7HogLZB8Sdu3EY75f?= =?utf-8?q?EiBx1YtsjdWjsQXlkO/iM6MA8e7b0BrClUIdz3LIoFiMKoA669dZNThmzrLI9c68j?= =?utf-8?q?FPh7sK+uIz3CjUMi4wc0tB1iKfuC6uGdMWuCyRoMWi/wnVxk//+MQ7sQ8yWO+aNTt?= =?utf-8?q?zETgRZJWzPw9J3P27TqYO9CViDudse/uy7vl1KzC9Hl5s+o2HNlYbiap5b4554VYl?= =?utf-8?q?F7AfRiruqS0sPxvmS8Go5ClJnhRqCwy0t27DFAHJc91pem0WiYzfTRljTkfIt+Fez?= =?utf-8?q?CE1Zar0yHqq6lDFdTNre07X4hE87BsM5QN40V2uhOiqH8UmW4FWfSfloBZTgjE7L7?= =?utf-8?q?dtZZGePYMiTShCRvLOo9OgXowkJZwWf9DfdtheQUi/48P3c5e9VzYGkgXEvHp/V5v?= =?utf-8?q?Bfs+Dab3xDHDRpHWdncCxH1N7+W8hdXb4RT6IyFTqUL9lM0criScPCYy+TlFUNOf3?= =?utf-8?q?ktrp0pqDl8Xzlu6q3El0bjsHyCUAsK9Bwj7hHbM0uZQBI4qTt5JINZ3Eau7kV41Oy?= =?utf-8?q?v8SxJ94LoOWlnQnJcSqh2qXzndeDwtSKqLvkRA5Rm0HDSLJu5jYyi+cIoD8mXDsi8?= =?utf-8?q?nLrkAfh9G2ShozEd7Zb1oWFj/OqgbJvTTAPi4wrC5q30fqt6QIzV9mFHR+bEqsI/e?= =?utf-8?q?nOy5tzccgwTnB06SEOS7+/M+UOsnh/zvAPCak3qI/ms8dMi6es5cK9wZa54oxG3kv?= =?utf-8?q?tnjFu2gaIqMF8uHu1sKXbq+g1KDZK0P8p6qR5b5xc6XgVZD8ZCob2JVBQaOc/4Ow9?= =?utf-8?q?D1XI9hNltWaPsDIL52SVK1fcQgFYyJl1QcUxWtvwwi3se8LLSjMhLGFR+Oix76d+X?= =?utf-8?q?eqFAm4amtp1kDtaucUZLPQ0ZBBucPJkg7hsQX4ImARPFetRFlduVIfu+FMRnRTNxs?= =?utf-8?q?wdM/vL6i1E2AdfsTYLhztISRa3VsCQooDuxBt3z6pjhk6pJLqgxwrP0HWGkH/CDdE?= =?utf-8?q?gAwFmUVGEBGr4FbRKKuHkw4m82jcA8utXQ=3D=3D?= Content-ID: <4992D0DFB71AB74A97E397CF7B083A59@eurprd03.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9001b78d-5e3f-4a6d-8e86-08dbf2d5caea X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:02.9281 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Heozpc5Ctl5PMKuY2ty3OCNbKXtAt4Qu/zOR/YGyq4jN65ILqtQEnUgNfse9debQS5zDQgVWjlyvH1lsFKs6rVE578AKchShrQNv/FgKkwg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: aR7fQrPtL88xBAhVHPNd2McuMJ03MFdI X-Proofpoint-ORIG-GUID: aR7fQrPtL88xBAhVHPNd2McuMJ03MFdI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 Add per-domain d->pci_lock that protects access to d->pdev_list. Purpose of this lock is to give guarantees to VPCI code that underlying pdev will not disappear under feet. This is a rw-lock, but this patch adds only write_lock()s. There will be read_lock() users in the next patches. This lock should be taken in write mode every time d->pdev_list is altered. All write accesses also should be protected by pcidevs_lock() as well. Idea is that any user that wants read access to the list or to the devices stored in the list should use either this new d->pci_lock or old pcidevs_lock(). Usage of any of this two locks will ensure only that pdev of interest will not disappear from under feet and that the pdev still will be assigned to the same domain. Of course, any new users should use pcidevs_lock() when it is appropriate (e.g. when accessing any other state that is protected by the said lock). In case both the newly introduced per-domain rwlock and the pcidevs lock is taken, the latter must be acquired first. Suggested-by: Roger Pau Monné Suggested-by: Jan Beulich Signed-off-by: Volodymyr Babchuk Reviewed-by: Roger Pau Monné Acked-by: Stefano Stabellini --- Changes in v10: - pdev->domain is assigned after removing from source domain but before adding to target domain in reassign_device() functions. Changes in v9: - returned back "pdev->domain = target;" in AMD IOMMU code - used "source" instead of pdev->domain in IOMMU functions - added comment about lock ordering in the commit message - reduced locked regions - minor changes non-functional changes in various places Changes in v8: - New patch Changes in v8 vs RFC: - Removed all read_locks after discussion with Roger in #xendevel - pci_release_devices() now returns the first error code - extended commit message - added missing lock in pci_remove_device() - extended locked region in pci_add_device() to protect list_del() calls --- xen/common/domain.c | 1 + xen/drivers/passthrough/amd/pci_amd_iommu.c | 9 ++- xen/drivers/passthrough/pci.c | 71 +++++++++++++++++---- xen/drivers/passthrough/vtd/iommu.c | 9 ++- xen/include/xen/sched.h | 1 + 5 files changed, 78 insertions(+), 13 deletions(-) diff --git a/xen/common/domain.c b/xen/common/domain.c index cd2ca6d49a..9b8902daa3 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -651,6 +651,7 @@ struct domain *domain_create(domid_t domid, #ifdef CONFIG_HAS_PCI INIT_LIST_HEAD(&d->pdev_list); + rwlock_init(&d->pci_lock); #endif /* All error paths can depend on the above setup. */ diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c index 6bc73dc210..5cd208bbef 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -481,8 +481,15 @@ static int cf_check reassign_device( if ( devfn == pdev->devfn && pdev->domain != target ) { - list_move(&pdev->domain_list, &target->pdev_list); + write_lock(&source->pci_lock); + list_del(&pdev->domain_list); + write_unlock(&source->pci_lock); + pdev->domain = target; + + write_lock(&target->pci_lock); + list_add(&pdev->domain_list, &target->pdev_list); + write_unlock(&target->pci_lock); } /* diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 04d00c7c37..b8ad4fa07c 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -453,7 +453,9 @@ static void __init _pci_hide_device(struct pci_dev *pdev) if ( pdev->domain ) return; pdev->domain = dom_xen; + write_lock(&dom_xen->pci_lock); list_add(&pdev->domain_list, &dom_xen->pdev_list); + write_unlock(&dom_xen->pci_lock); } int __init pci_hide_device(unsigned int seg, unsigned int bus, @@ -746,7 +748,9 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( !pdev->domain ) { pdev->domain = hardware_domain; + write_lock(&hardware_domain->pci_lock); list_add(&pdev->domain_list, &hardware_domain->pdev_list); + write_unlock(&hardware_domain->pci_lock); /* * For devices not discovered by Xen during boot, add vPCI handlers @@ -756,7 +760,9 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( ret ) { printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret); + write_lock(&hardware_domain->pci_lock); list_del(&pdev->domain_list); + write_unlock(&hardware_domain->pci_lock); pdev->domain = NULL; goto out; } @@ -764,7 +770,9 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( ret ) { vpci_remove_device(pdev); + write_lock(&hardware_domain->pci_lock); list_del(&pdev->domain_list); + write_unlock(&hardware_domain->pci_lock); pdev->domain = NULL; goto out; } @@ -814,7 +822,11 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) pci_cleanup_msi(pdev); ret = iommu_remove_device(pdev); if ( pdev->domain ) + { + write_lock(&pdev->domain->pci_lock); list_del(&pdev->domain_list); + write_unlock(&pdev->domain->pci_lock); + } printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf); free_pdev(pseg, pdev); break; @@ -885,26 +897,61 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, int pci_release_devices(struct domain *d) { - struct pci_dev *pdev, *tmp; - u8 bus, devfn; - int ret; + int combined_ret; + LIST_HEAD(failed_pdevs); pcidevs_lock(); - ret = arch_pci_clean_pirqs(d); - if ( ret ) + + combined_ret = arch_pci_clean_pirqs(d); + if ( combined_ret ) { pcidevs_unlock(); - return ret; + return combined_ret; } - list_for_each_entry_safe ( pdev, tmp, &d->pdev_list, domain_list ) + + write_lock(&d->pci_lock); + + while ( !list_empty(&d->pdev_list) ) { - bus = pdev->bus; - devfn = pdev->devfn; - ret = deassign_device(d, pdev->seg, bus, devfn) ?: ret; + struct pci_dev *pdev = list_first_entry(&d->pdev_list, + struct pci_dev, + domain_list); + uint16_t seg = pdev->seg; + uint8_t bus = pdev->bus; + uint8_t devfn = pdev->devfn; + int ret; + + write_unlock(&d->pci_lock); + ret = deassign_device(d, seg, bus, devfn); + write_lock(&d->pci_lock); + if ( ret ) + { + const struct pci_dev *tmp; + + /* + * We need to check if deassign_device() left our pdev in + * domain's list. As we dropped the lock, we can't be sure + * that list wasn't permutated in some random way, so we + * need to traverse the whole list. + */ + for_each_pdev ( d, tmp ) + { + if ( tmp == pdev ) + { + list_move_tail(&pdev->domain_list, &failed_pdevs); + break; + } + } + + combined_ret = combined_ret ?: ret; + } } + + list_splice(&failed_pdevs, &d->pdev_list); + write_unlock(&d->pci_lock); pcidevs_unlock(); - return ret; + return combined_ret; } #define PCI_CLASS_BRIDGE_HOST 0x0600 @@ -1124,7 +1171,9 @@ static int __hwdom_init cf_check _setup_hwdom_pci_devices( if ( !pdev->domain ) { pdev->domain = ctxt->d; + write_lock(&ctxt->d->pci_lock); list_add(&pdev->domain_list, &ctxt->d->pdev_list); + write_unlock(&ctxt->d->pci_lock); setup_one_hwdom_device(ctxt, pdev); } else if ( pdev->domain == dom_xen ) diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index e13b7d99db..d3dfab679e 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -2816,8 +2816,15 @@ static int cf_check reassign_device_ownership( if ( devfn == pdev->devfn && pdev->domain != target ) { - list_move(&pdev->domain_list, &target->pdev_list); + write_lock(&source->pci_lock); + list_del(&pdev->domain_list); + write_unlock(&source->pci_lock); + pdev->domain = target; + + write_lock(&target->pci_lock); + list_add(&pdev->domain_list, &target->pdev_list); + write_unlock(&target->pci_lock); } if ( !has_arch_pdevs(source) ) diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 3609ef88c4..57391e74b6 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -461,6 +461,7 @@ struct domain #ifdef CONFIG_HAS_PCI struct list_head pdev_list; + rwlock_t pci_lock; #endif #ifdef CONFIG_HAS_PASSTHROUGH From patchwork Sat Dec 2 01:27:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F2BE4C46CA3 for ; Sat, 2 Dec 2023 01:27:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645840.1008343 (Exim 4.92) (envelope-from ) id 1r9En3-00051D-GC; Sat, 02 Dec 2023 01:27:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645840.1008343; Sat, 02 Dec 2023 01:27:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En3-0004yv-4f; Sat, 02 Dec 2023 01:27:29 +0000 Received: by outflank-mailman (input) for mailman id 645840; Sat, 02 Dec 2023 01:27:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En0-000272-Jr for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:26 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f15d6cb9-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:23 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGuw008549; Sat, 2 Dec 2023 01:27:11 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:10 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:05 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:05 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f15d6cb9-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UXEU4vleLxXE8/i07IX4ceyEdWgyWp/0G9+2oRT1IESSOHHqBA+O8qWd488MEwxriDwwHSXj332M9/e6JWNAtIE5LG7Ssuy/bprGydff1SNXVrdXQDdSJ7x1LhBm2+SVE5Z3bp9xRtVUiy2AikNACki4M5CypgWaUxbrqBCXCllZQ50WB4P8QQc3sUw42mhP9hvSRYEtKzQciOZV5uJ1tuTxsKDVQAea5n4/9i0cxuPYES/DB1CsV2JklJH5VpaHm4OuwQOlBmyY6g6iHVXrn1RnJ1m9plWtnTg94WmZ7OcqniUFi+kPhlwKkGoNYHKUTBLsCuu8eRLK0g5g8+BrAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cRAy4fTV/DZJicZixDZOgYxrMqpCiX7LtB/4lX/CcM8=; b=SQN8AHUmTVXRXdH+SU36FGzL3lCAHXk6K1B4sAa4HWMfkLif2WYC0XGvT4z9gLoNRuKglSKnfRTsclHmnk0g8vjKUmCxpG6MEOLNAa66S1Om6WIQLvsAdMM58pKG8Qt3Y1g45q+5AEMtxxMCuWKWlizRfLM5yGGaNjc5LS+QXkdgCf1fhZlcT2BvKAk93DNkKD1qa/shJxSNZ4rNPPo6T7AxbxYFUwQ7JpQwRiEaCa/jRMMrfiG7QPS/AjwnFIgSdGhOgOHana+h7DQg1WYZIrzw19MgX4nsf5+R340swrMTHs2YDhZ+J/iTgaY+DdHJzg2SGnyIOma7ezyARdXSMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cRAy4fTV/DZJicZixDZOgYxrMqpCiX7LtB/4lX/CcM8=; b=nrrxBwGNYvFrQxZhvUdcf2Bq/zvs1nD0+UkrtjpXUnqnxVPe9E6gqeejmxn81ZWfMR9EBL0wi8FPUYZXY81O34vyr3bBPpJSnUAedxsv7EUoPaa5rVse6uG53UeULULdBJO5jObvn3LHQn6AGEPCQwwLt1yJq+GxeITkXqssQkRlM67gNX64VlX8AjMBJxTlFjzNeNoMzXzLkF/wz3Y3yWvf/FD63rDXtXqk247zA3q/Wonw8Fkbx2ROPHpySNhYCh88W/pumCbOZ6Di0c7V8nXTDi1rKZwjZpLo4PG9wmc2t5befpkaCPBmKNqxnE1s6al1BUh02yW+r54hY6hu+g== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Jun Nakajima , Kevin Tian , Paul Durrant , Volodymyr Babchuk Subject: [PATCH v11 03/17] vpci: use per-domain PCI lock to protect vpci structure Thread-Topic: [PATCH v11 03/17] vpci: use per-domain PCI lock to protect vpci structure Thread-Index: AQHaJL6nSmaa9yrPGkyMNHyYLaQzMQ== Date: Sat, 2 Dec 2023 01:27:03 +0000 Message-ID: <20231202012556.2012281-4-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 596e1aac-28f6-4160-f525-08dbf2d5cb25 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: kjJoz97MBr8uR75DiSdp70VH6hOV1SszBvaZyTS69122yiusj7IjlYkQZSa4yxSWA48R5fl7mYpLgUoz5q1Fwuqk+lc0+PAix2TSYwHdYIoeIbtrbQmuGHOFS2dNKVTGeILOdMKWwjlWfvezGg6cFNIplvBcg/ezquhHUFMHUr1vJ9WbtyGjXpEVdYxkbkOjsJG0459otIdskHyjsx9HK3P3a770fZaAHDtlHJhXnwakya+I1aaTo6NaH7VI13JV0YoGIJEtla+0F2qai7zTeDvzaBSAa5b/FScjue5E2fg4mrEyfgURnkbFRIG2JjDTYPaUCxmDxHqKyumIPGNfkszSZmsHKLg4/P104RaQN07QnMFQvH+yTs9O/opK+1N7cSJscVnHe/d+qYHp8V36/v16aeN8crpDq9srdP3qhvX6pcfkhMi06UzApp5eG5GJoanDbhZATED+H/S775KDZyT1wl1Kp5aY/a9FeQiEs5frD2t1eVT98fIZIOeKS18Sa00xMOPI/zWa45tV9laCs6TUIbRFPCYoaxl7r803bHbXgx7ALmyOe/O2vvvlKCwGcy3xJmthAUwNaaCtzbasCCum6yvlneOs/Iujwm2N7/Dv+y05INXachOPVTk7guubQgsFB8RxFA8ehp5xH3VWaQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(30864003)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007)(559001)(579004)(309714004);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?/nHirWfh2nqIocNDHrVSgcQwhxcn?= =?utf-8?q?Zal77Fdseme2ZqayEBUBPpJg38+4SCWV6jsQudQ0GFGgBoJ+qG+FQRw8K0zUMJNuN?= =?utf-8?q?dTnLF6/kSnhoVAJtcM86YAYmReb57JD66h7P/prZLbNozyDePv+vfKu2zAe2cf0+e?= =?utf-8?q?Z4ctbJgOmf/Rr+Cg2BoYYoUfTiZ8jSo5q8+x+DLa02KpTw2JzVxNHuk2Tk/wt0H5A?= =?utf-8?q?foUvK/3qSR5/s+BuclYXPSca2BN0JssrrEP/7KBa8DB6O5SStUNKGLNhW6ee7fZp+?= =?utf-8?q?WAllQIytV6p/vYedsYo+ZzCtcSNYEx8mqIYNpeSCfY61IuFK7gw5T6vUJoBK8pogI?= =?utf-8?q?1CJ1qp7ZZ4iDIETK3wIi5Mb7ezU+frlWw+LERg0fYFpEaj25aBVoMAtaKFfRLvZFH?= =?utf-8?q?55qmdifYXTHG7EDvZfr96F0v5P0jwO8Ik5+VWzIMNQfdbLWU6731aa0vLpv32Km8D?= =?utf-8?q?uchGoto+S6RblMKs0KOesoiobREJVFE1F2YxJ8QNBEPS9mfvCbyNCilT9QbfSansS?= =?utf-8?q?Z/m6tbmR27/bvPM7homo+gUPebX57gUOEJGFurBRRx1x+6u9JpFQ18BResXoNveXZ?= =?utf-8?q?AuEQOOcocWsh/m9THe5cwXvF59G1MKpDSncGKT0+Nwm0WV7bY5xZNUkZdt2Pa/ELb?= =?utf-8?q?34XMPPDO0V90pc4jwlzpapghnLVM+kIFDlUGiffvYGyaHRv66GBpfprlbgPoy0iYb?= =?utf-8?q?Abym1iQvdE2WDwK9TfU1sFNANT25Ez3O2xzFS5nEG5hvbAhxengdHQdIwuPnwWJwd?= =?utf-8?q?e4rlBnusPsOoyO+I4FqANQHML3RDBHaLANnO20hTK8gFKWRv66eidwunb1VnI+6DQ?= =?utf-8?q?txAV+mRBYc/oAh2mKgTcDvApCv1Lk8209Ogther4SZUz/N3Y2FtVaj7RM4Ed+tGQ8?= =?utf-8?q?mtk1BnDbQqXIsSwidXdRnShAZ7bjRKe+qM/QM3JHeDHqXdlX754spVhLHsgqJLhZ/?= =?utf-8?q?KypOqPNiRNx7uZ5CrYYEptqxjKTQLw/C/vzBvEy5PeFpQRIP2eYEjFKGRVNaP4mmC?= =?utf-8?q?Pp4IpUKZ9cZa6OkPVk0PK4qDcFbYQcDUjmrI019hmmYvO92wiN/+1nWMp+dBLLaz/?= =?utf-8?q?KPs3KUer31QddfofotgoNEEAx0ya7djzxhU3KxV8FtxJoJ5izGQcPgGr2HlxZpyrE?= =?utf-8?q?mBxkzdcr+z9+/USZSa3eIYTYUD5FaJslCM5skirA7jM2EIp5Lofxib9w+udGK9p1h?= =?utf-8?q?CkkKYRn6p0TSXsk5/kqcmSp46vIypTy3Pv7Rxm9J+fGTA5Zl36A6iT6A2x0hTFyQP?= =?utf-8?q?rYA7sBINl2hPvVOK/8xKkyvWgqHpsfgKEchECVTrp0aKSEXXdga1H8MeKtNBki37O?= =?utf-8?q?6IgVuVk84Zja81nBSpf1hxM6MX0832YJrBLOPrBvXqD/W9PQhT3KoMapePYl1sQXt?= =?utf-8?q?9BVdK5g+TAI66Z4MRRJs6Gd/IprcB+3C4TmXRW2mYJSKYL48/CnRdhErxh3TXS8K/?= =?utf-8?q?opnxslcPS79BkRh4NjS4GDnFnq9iikdn6e3L8BZtNCunYAlXwqMEBf/DrjegLOMym?= =?utf-8?q?MFU1f5Httful7a8x/BbpkxeYNGSxY7GcJw=3D=3D?= Content-ID: <0B1D1D1E00A5804E95A9220661BC4728@eurprd03.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 596e1aac-28f6-4160-f525-08dbf2d5cb25 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:03.2638 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 80R1qTd0ATc99ZB3zFH4rQSzXqTk8vJdgzfVfetuu4iO+Nw1P+YPnqvAM/RwYNRCzLjiM6YVg5Nu9A556j/Mz3xFAnqg7VbmAhRCycFoJQU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: nXYKR62mWrGcTEkuz1d59TIQrkR_v0Mv X-Proofpoint-ORIG-GUID: nXYKR62mWrGcTEkuz1d59TIQrkR_v0Mv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko Use a previously introduced per-domain read/write lock to check whether vpci is present, so we are sure there are no accesses to the contents of the vpci struct if not. This lock can be used (and in a few cases is used right away) so that vpci removal can be performed while holding the lock in write mode. Previously such removal could race with vpci_read for example. When taking both d->pci_lock and pdev->vpci->lock, they should be taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid possible deadlock situations. 1. Per-domain's pci_rwlock is used to protect pdev->vpci structure from being removed. 2. Writing the command register and ROM BAR register may trigger modify_bars to run, which in turn may access multiple pdevs while checking for the existing BAR's overlap. The overlapping check, if done under the read lock, requires vpci->lock to be acquired on both devices being compared, which may produce a deadlock. It is not possible to upgrade read lock to write lock in such a case. So, in order to prevent the deadlock, use d->pci_lock instead. To prevent deadlock while locking both hwdom->pci_lock and dom_xen->pci_lock, always lock hwdom first. All other code, which doesn't lead to pdev->vpci destruction and does not access multiple pdevs at the same time, can still use a combination of the read lock and pdev->vpci->lock. 3. Drop const qualifier where the new rwlock is used and this is appropriate. 4. Do not call process_pending_softirqs with any locks held. For that unlock prior the call and re-acquire the locks after. After re-acquiring the lock there is no need to check if pdev->vpci exists: - in apply_map because of the context it is called (no race condition possible) - for MSI/MSI-X debug code because it is called at the end of pdev->vpci access and no further access to pdev->vpci is made 5. Use d->pci_lock around for_each_pdev and pci_get_pdev_by_domain while accessing pdevs in vpci code. 6. We are removing multiple ASSERT(pcidevs_locked()) instances because they are too strict now: they should be corrected to ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock)), but problem is that mentioned instances does not have access to the domain pointer and it is not feasible to pass a domain pointer to a function just for debugging purposes. Suggested-by: Roger Pau Monné Suggested-by: Jan Beulich Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk --- Changes in v11: - Fixed commit message regarding possible spinlocks - Removed parameter from allocate_and_map_msi_pirq(), which was added in the prev version. Now we are taking pcidevs_lock in physdev_map_pirq() - Returned ASSERT to pci_enable_msi - Fixed case when we took read lock instead of write one - Fixed label indentation Changes in v10: - Moved printk pas locked area - Returned back ASSERTs - Added new parameter to allocate_and_map_msi_pirq() so it knows if it should take the global pci lock - Added comment about possible improvement in vpci_write - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in appropriate places - Renamed release_domain_locks() to release_domain_write_locks() - moved domain_done label in vpci_dump_msi() to correct place Changes in v9: - extended locked region to protect vpci_remove_device and vpci_add_handlers() calls - vpci_write() takes lock in the write mode to protect potential call to modify_bars() - renamed lock releasing function - removed ASSERT()s from msi code - added trylock in vpci_dump_msi Changes in v8: - changed d->vpci_lock to d->pci_lock - introducing d->pci_lock in a separate patch - extended locked region in vpci_process_pending - removed pcidevs_lockis vpci_dump_msi() - removed some changes as they are not needed with the new locking scheme - added handling for hwdom && dom_xen case --- xen/arch/x86/hvm/vmsi.c | 22 +++++++-------- xen/arch/x86/hvm/vmx/vmx.c | 2 -- xen/arch/x86/irq.c | 8 +++--- xen/arch/x86/msi.c | 10 ++----- xen/arch/x86/physdev.c | 2 ++ xen/drivers/passthrough/pci.c | 9 +++--- xen/drivers/vpci/header.c | 18 ++++++++++++ xen/drivers/vpci/msi.c | 28 +++++++++++++++++-- xen/drivers/vpci/msix.c | 52 ++++++++++++++++++++++++++++++----- xen/drivers/vpci/vpci.c | 50 +++++++++++++++++++++++++++++++-- 10 files changed, 160 insertions(+), 41 deletions(-) diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 128f236362..03caf91bee 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable) struct msixtbl_entry *entry, *new_entry; int r = -EINVAL; - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock)); ASSERT(rw_is_write_locked(&d->event_lock)); if ( !msixtbl_initialised(d) ) @@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq) struct pci_dev *pdev; struct msixtbl_entry *entry; - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock)); ASSERT(rw_is_write_locked(&d->event_lock)); if ( !msixtbl_initialised(d) ) @@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data, { unsigned int i; - ASSERT(pcidevs_locked()); + ASSERT(rw_is_locked(&pdev->domain->pci_lock)); if ( (address & MSI_ADDR_BASE_MASK) != MSI_ADDR_HEADER ) { @@ -725,8 +725,8 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev) int rc; ASSERT(msi->arch.pirq != INVALID_PIRQ); + ASSERT(rw_is_locked(&pdev->domain->pci_lock)); - pcidevs_lock(); for ( i = 0; i < msi->vectors && msi->arch.bound; i++ ) { struct xen_domctl_bind_pt_irq unbind = { @@ -745,7 +745,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev) msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, msi->vectors, msi->arch.pirq, msi->mask); - pcidevs_unlock(); } static int vpci_msi_enable(const struct pci_dev *pdev, unsigned int nr, @@ -778,15 +777,14 @@ int vpci_msi_arch_enable(struct vpci_msi *msi, const struct pci_dev *pdev, int rc; ASSERT(msi->arch.pirq == INVALID_PIRQ); + ASSERT(rw_is_locked(&pdev->domain->pci_lock)); rc = vpci_msi_enable(pdev, vectors, 0); if ( rc < 0 ) return rc; msi->arch.pirq = rc; - pcidevs_lock(); msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, vectors, msi->arch.pirq, msi->mask); - pcidevs_unlock(); return 0; } @@ -797,8 +795,8 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq, unsigned int i; ASSERT(pirq != INVALID_PIRQ); + ASSERT(rw_is_locked(&pdev->domain->pci_lock)); - pcidevs_lock(); for ( i = 0; i < nr && bound; i++ ) { struct xen_domctl_bind_pt_irq bind = { @@ -814,7 +812,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq, write_lock(&pdev->domain->event_lock); unmap_domain_pirq(pdev->domain, pirq); write_unlock(&pdev->domain->event_lock); - pcidevs_unlock(); } void vpci_msi_arch_disable(struct vpci_msi *msi, const struct pci_dev *pdev) @@ -854,6 +851,7 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry, int rc; ASSERT(entry->arch.pirq == INVALID_PIRQ); + ASSERT(rw_is_locked(&pdev->domain->pci_lock)); rc = vpci_msi_enable(pdev, vmsix_entry_nr(pdev->vpci->msix, entry), table_base); if ( rc < 0 ) @@ -861,7 +859,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry, entry->arch.pirq = rc; - pcidevs_lock(); rc = vpci_msi_update(pdev, entry->data, entry->addr, 1, entry->arch.pirq, entry->masked); if ( rc ) @@ -869,7 +866,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry, vpci_msi_disable(pdev, entry->arch.pirq, 1, false); entry->arch.pirq = INVALID_PIRQ; } - pcidevs_unlock(); return rc; } @@ -895,6 +891,8 @@ int vpci_msix_arch_print(const struct vpci_msix *msix) { unsigned int i; + ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock)); + for ( i = 0; i < msix->max_entries; i++ ) { const struct vpci_msix_entry *entry = &msix->entries[i]; @@ -913,7 +911,9 @@ int vpci_msix_arch_print(const struct vpci_msix *msix) struct pci_dev *pdev = msix->pdev; spin_unlock(&msix->pdev->vpci->lock); + read_unlock(&pdev->domain->pci_lock); process_pending_softirqs(); + read_lock(&pdev->domain->pci_lock); /* NB: we assume that pdev cannot go away for an alive domain. */ if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) ) return -EBUSY; diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 5663bc0178..dd836e49f3 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -413,8 +413,6 @@ static int cf_check vmx_pi_update_irte(const struct vcpu *v, spin_unlock_irq(&desc->lock); - ASSERT(pcidevs_locked()); - return iommu_update_ire_from_msi(msi_desc, &msi_desc->msg); unlock_out: diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index 50e49e1a4b..4d89d9af99 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2166,7 +2166,7 @@ int map_domain_pirq( struct pci_dev *pdev; unsigned int nr = 0; - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock)); ret = -ENODEV; if ( !cpu_has_apic ) @@ -2323,7 +2323,7 @@ int unmap_domain_pirq(struct domain *d, int pirq) if ( (pirq < 0) || (pirq >= d->nr_pirqs) ) return -EINVAL; - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock)); ASSERT(rw_is_write_locked(&d->event_lock)); info = pirq_info(d, pirq); @@ -2888,6 +2888,8 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p, { int irq, pirq, ret; + ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock)); + switch ( type ) { case MAP_PIRQ_TYPE_MSI: @@ -2917,7 +2919,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p, msi->irq = irq; - pcidevs_lock(); /* Verify or get pirq. */ write_lock(&d->event_lock); pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr); @@ -2933,7 +2934,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p, done: write_unlock(&d->event_lock); - pcidevs_unlock(); if ( ret ) { switch ( type ) diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index 335c0868a2..6a548611a5 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -602,7 +602,7 @@ static int msi_capability_init(struct pci_dev *dev, unsigned int i, mpos; uint16_t control; - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&dev->domain->pci_lock)); pos = pci_find_cap_offset(dev->sbdf, PCI_CAP_ID_MSI); if ( !pos ) return -ENODEV; @@ -771,7 +771,7 @@ static int msix_capability_init(struct pci_dev *dev, if ( !pos ) return -ENODEV; - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&dev->domain->pci_lock)); control = pci_conf_read16(dev->sbdf, msix_control_reg(pos)); /* @@ -988,8 +988,6 @@ static int __pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi, { struct msi_desc *old_desc; - ASSERT(pcidevs_locked()); - if ( !pdev ) return -ENODEV; @@ -1043,8 +1041,6 @@ static int __pci_enable_msix(struct pci_dev *pdev, struct msi_info *msi, { struct msi_desc *old_desc; - ASSERT(pcidevs_locked()); - if ( !pdev || !pdev->msix ) return -ENODEV; @@ -1154,7 +1150,7 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi, struct msi_desc **desc) { - ASSERT(pcidevs_locked()); + ASSERT(pcidevs_locked() || rw_is_locked(&pdev->domain->pci_lock)); if ( !use_msi ) return -EPERM; diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 47c4da0af7..369c9e788c 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p, case MAP_PIRQ_TYPE_MSI: case MAP_PIRQ_TYPE_MULTI_MSI: + pcidevs_lock(); ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi); + pcidevs_unlock(); break; default: diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index b8ad4fa07c..182da45acb 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -750,7 +750,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, pdev->domain = hardware_domain; write_lock(&hardware_domain->pci_lock); list_add(&pdev->domain_list, &hardware_domain->pdev_list); - write_unlock(&hardware_domain->pci_lock); /* * For devices not discovered by Xen during boot, add vPCI handlers @@ -759,18 +758,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, ret = vpci_add_handlers(pdev); if ( ret ) { - printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret); - write_lock(&hardware_domain->pci_lock); list_del(&pdev->domain_list); write_unlock(&hardware_domain->pci_lock); pdev->domain = NULL; + printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret); goto out; } + write_unlock(&hardware_domain->pci_lock); ret = iommu_add_device(pdev); if ( ret ) { - vpci_remove_device(pdev); write_lock(&hardware_domain->pci_lock); + vpci_remove_device(pdev); list_del(&pdev->domain_list); write_unlock(&hardware_domain->pci_lock); pdev->domain = NULL; @@ -1146,7 +1145,9 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt, } while ( devfn != pdev->devfn && PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) ); + write_lock(&ctxt->d->pci_lock); err = vpci_add_handlers(pdev); + write_unlock(&ctxt->d->pci_lock); if ( err ) printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n", ctxt->d->domain_id, err); diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 767c1ba718..c86d33d0cd 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -172,6 +172,7 @@ bool vpci_process_pending(struct vcpu *v) if ( rc == -ERESTART ) return true; + write_lock(&v->domain->pci_lock); spin_lock(&v->vpci.pdev->vpci->lock); /* Disable memory decoding unconditionally on failure. */ modify_decoding(v->vpci.pdev, @@ -190,6 +191,7 @@ bool vpci_process_pending(struct vcpu *v) * failure. */ vpci_remove_device(v->vpci.pdev); + write_unlock(&v->domain->pci_lock); } return false; @@ -201,8 +203,20 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev, struct map_data data = { .d = d, .map = true }; int rc; + ASSERT(rw_is_write_locked(&d->pci_lock)); + while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART ) + { + /* + * It's safe to drop and reacquire the lock in this context + * without risking pdev disappearing because devices cannot be + * removed until the initial domain has been started. + */ + write_unlock(&d->pci_lock); process_pending_softirqs(); + write_lock(&d->pci_lock); + } + rangeset_destroy(mem); if ( !rc ) modify_decoding(pdev, cmd, false); @@ -243,6 +257,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) unsigned int i; int rc; + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); + if ( !mem ) return -ENOMEM; @@ -522,6 +538,8 @@ static int cf_check init_bars(struct pci_dev *pdev) struct vpci_bar *bars = header->bars; int rc; + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); + switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f ) { case PCI_HEADER_TYPE_NORMAL: diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c index a253ccbd7d..6ff71e5f9a 100644 --- a/xen/drivers/vpci/msi.c +++ b/xen/drivers/vpci/msi.c @@ -263,7 +263,7 @@ REGISTER_VPCI_INIT(init_msi, VPCI_PRIORITY_LOW); void vpci_dump_msi(void) { - const struct domain *d; + struct domain *d; rcu_read_lock(&domlist_read_lock); for_each_domain ( d ) @@ -275,6 +275,9 @@ void vpci_dump_msi(void) printk("vPCI MSI/MSI-X d%d\n", d->domain_id); + if ( !read_trylock(&d->pci_lock) ) + continue; + for_each_pdev ( d, pdev ) { const struct vpci_msi *msi; @@ -316,14 +319,33 @@ void vpci_dump_msi(void) * holding the lock. */ printk("unable to print all MSI-X entries: %d\n", rc); - process_pending_softirqs(); - continue; + goto pdev_done; } } spin_unlock(&pdev->vpci->lock); + pdev_done: + /* + * Unlock lock to process pending softirqs. This is + * potentially unsafe, as d->pdev_list can be changed in + * meantime. + */ + read_unlock(&d->pci_lock); process_pending_softirqs(); + if ( !read_trylock(&d->pci_lock) ) + { + printk("unable to access other devices for the domain\n"); + goto domain_done; + } } + read_unlock(&d->pci_lock); + domain_done: + /* + * We need this label at the end of the loop, but some + * compilers might not be happy about label at the end of the + * compound statement so we adding an empty statement here. + */ + ; } rcu_read_unlock(&domlist_read_lock); } diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c index d1126a417d..b6abab47ef 100644 --- a/xen/drivers/vpci/msix.c +++ b/xen/drivers/vpci/msix.c @@ -147,6 +147,8 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr) { struct vpci_msix *msix; + ASSERT(rw_is_locked(&d->pci_lock)); + list_for_each_entry ( msix, &d->arch.hvm.msix_tables, next ) { const struct vpci_bar *bars = msix->pdev->vpci->header.bars; @@ -163,7 +165,13 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr) static int cf_check msix_accept(struct vcpu *v, unsigned long addr) { - return !!msix_find(v->domain, addr); + int rc; + + read_lock(&v->domain->pci_lock); + rc = !!msix_find(v->domain, addr); + read_unlock(&v->domain->pci_lock); + + return rc; } static bool access_allowed(const struct pci_dev *pdev, unsigned long addr, @@ -358,21 +366,35 @@ static int adjacent_read(const struct domain *d, const struct vpci_msix *msix, static int cf_check msix_read( struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data) { - const struct domain *d = v->domain; - struct vpci_msix *msix = msix_find(d, addr); + struct domain *d = v->domain; + struct vpci_msix *msix; const struct vpci_msix_entry *entry; unsigned int offset; *data = ~0UL; + read_lock(&d->pci_lock); + + msix = msix_find(d, addr); if ( !msix ) + { + read_unlock(&d->pci_lock); return X86EMUL_RETRY; + } if ( adjacent_handle(msix, addr) ) - return adjacent_read(d, msix, addr, len, data); + { + int rc = adjacent_read(d, msix, addr, len, data); + + read_unlock(&d->pci_lock); + return rc; + } if ( !access_allowed(msix->pdev, addr, len) ) + { + read_unlock(&d->pci_lock); return X86EMUL_OKAY; + } spin_lock(&msix->pdev->vpci->lock); entry = get_entry(msix, addr); @@ -404,6 +426,7 @@ static int cf_check msix_read( break; } spin_unlock(&msix->pdev->vpci->lock); + read_unlock(&d->pci_lock); return X86EMUL_OKAY; } @@ -491,19 +514,33 @@ static int adjacent_write(const struct domain *d, const struct vpci_msix *msix, static int cf_check msix_write( struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data) { - const struct domain *d = v->domain; - struct vpci_msix *msix = msix_find(d, addr); + struct domain *d = v->domain; + struct vpci_msix *msix; struct vpci_msix_entry *entry; unsigned int offset; + read_lock(&d->pci_lock); + + msix = msix_find(d, addr); if ( !msix ) + { + read_unlock(&d->pci_lock); return X86EMUL_RETRY; + } if ( adjacent_handle(msix, addr) ) - return adjacent_write(d, msix, addr, len, data); + { + int rc = adjacent_write(d, msix, addr, len, data); + + read_unlock(&d->pci_lock); + return rc; + } if ( !access_allowed(msix->pdev, addr, len) ) + { + read_unlock(&d->pci_lock); return X86EMUL_OKAY; + } spin_lock(&msix->pdev->vpci->lock); entry = get_entry(msix, addr); @@ -579,6 +616,7 @@ static int cf_check msix_write( break; } spin_unlock(&msix->pdev->vpci->lock); + read_unlock(&d->pci_lock); return X86EMUL_OKAY; } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 3bec9a4153..0b694beadf 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -38,6 +38,8 @@ extern vpci_register_init_t *const __end_vpci_array[]; void vpci_remove_device(struct pci_dev *pdev) { + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); + if ( !has_vpci(pdev->domain) || !pdev->vpci ) return; @@ -73,6 +75,8 @@ int vpci_add_handlers(struct pci_dev *pdev) const unsigned long *ro_map; int rc = 0; + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); + if ( !has_vpci(pdev->domain) ) return 0; @@ -326,11 +330,12 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size, uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) { - const struct domain *d = current->domain; + struct domain *d = current->domain; const struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; uint32_t data = ~(uint32_t)0; + rwlock_t *lock; if ( !size ) { @@ -342,11 +347,21 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) * Find the PCI dev matching the address, which for hwdom also requires * consulting DomXEN. Passthrough everything that's not trapped. */ + lock = &d->pci_lock; + read_lock(lock); pdev = pci_get_pdev(d, sbdf); if ( !pdev && is_hardware_domain(d) ) + { + read_unlock(lock); + lock = &dom_xen->pci_lock; + read_lock(lock); pdev = pci_get_pdev(dom_xen, sbdf); + } if ( !pdev || !pdev->vpci ) + { + read_unlock(lock); return vpci_read_hw(sbdf, reg, size); + } spin_lock(&pdev->vpci->lock); @@ -392,6 +407,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) ASSERT(data_offset < size); } spin_unlock(&pdev->vpci->lock); + read_unlock(lock); if ( data_offset < size ) { @@ -431,10 +447,23 @@ static void vpci_write_helper(const struct pci_dev *pdev, r->private); } +/* Helper function to unlock locks taken by vpci_write in proper order */ +static void release_domain_write_locks(struct domain *d) +{ + ASSERT(rw_is_write_locked(&d->pci_lock)); + + if ( is_hardware_domain(d) ) + { + ASSERT(rw_is_write_locked(&dom_xen->pci_lock)); + write_unlock(&dom_xen->pci_lock); + } + write_unlock(&d->pci_lock); +} + void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, uint32_t data) { - const struct domain *d = current->domain; + struct domain *d = current->domain; const struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; @@ -447,8 +476,20 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, /* * Find the PCI dev matching the address, which for hwdom also requires - * consulting DomXEN. Passthrough everything that's not trapped. + * consulting DomXEN. Passthrough everything that's not trapped. + * If this is hwdom, we need to hold locks for both domain in case if + * modify_bars() is called */ + /* + * TODO: We need to take pci_locks in exclusive mode only if we + * are modifying BARs, so there is a room for improvement. + */ + write_lock(&d->pci_lock); + + /* dom_xen->pci_lock always should be taken second to prevent deadlock */ + if ( is_hardware_domain(d) ) + write_lock(&dom_xen->pci_lock); + pdev = pci_get_pdev(d, sbdf); if ( !pdev && is_hardware_domain(d) ) pdev = pci_get_pdev(dom_xen, sbdf); @@ -457,6 +498,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, /* Ignore writes to read-only devices, which have no ->vpci. */ const unsigned long *ro_map = pci_get_ro_map(sbdf.seg); + release_domain_write_locks(d); + if ( !ro_map || !test_bit(sbdf.bdf, ro_map) ) vpci_write_hw(sbdf, reg, size, data); return; @@ -498,6 +541,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, ASSERT(data_offset < size); } spin_unlock(&pdev->vpci->lock); + release_domain_write_locks(d); if ( data_offset < size ) /* Tailing gap, write the remaining. */ From patchwork Sat Dec 2 01:27:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AAE6BC10DC1 for ; Sat, 2 Dec 2023 01:27:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645829.1008237 (Exim 4.92) (envelope-from ) id 1r9Emt-0002Ay-Ts; Sat, 02 Dec 2023 01:27:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645829.1008237; Sat, 02 Dec 2023 01:27:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emt-0002Ao-Qk; Sat, 02 Dec 2023 01:27:19 +0000 Received: by outflank-mailman (input) for mailman id 645829; Sat, 02 Dec 2023 01:27:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Ems-000271-Gj for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:18 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id eb9763e6-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:14 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWY004068; Sat, 2 Dec 2023 01:27:12 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:11 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:06 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:06 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: eb9763e6-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PVV67vpxBffXxsYMlmmK8gJaR/DMybyhELr2tdhCVNw3hW0O0kXW68wkcapv0Ncfg0Rj2E+9jNidiIgfx/0+aq8YqreY6rzRqvnNJArH2F72q+JFkiIJtR0Cw2Axg+aKGfBnVyrqgcCjYKaxcDwRqXDM6xjvS6HSePLHDa/h8Ui8GTbHCAevSCNPEi5q8aCiHryReFM7aqoZz+KVOYu2uKuNgPNpipnGcM3N18ogPxlhmLcI5xRL2IgucwL69jiSCXJ9Hbc2OuQzbRIKGJJrpQnIZ4oiHvQA8G/wss7fwVZlkojgxq1gRr4ikGlfbB12PwyxwzyzGSYzWA/4RclyEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kEcu45xdaIpfXtUGO2Sqnh0DJtv9WM0RX9NJ+e+vxXs=; b=nZej2pAQdMYTWnY1bn+qOynGXYYu4KqAS7k8GvFFypdaOPzH4Velgtkd/8pJJWu54w6uNuKCNQW2oFDoq7eE70Pre4H/LDLqidGJlArMXSWGteW7ekg5V8oKyynKAY32MC3PJHZ/42jBX5frh4ueqqfzX1sg5lqsPJJEk+pbq+QfzUHmpZjhXhhP1DScs1qywzi3rELpHnB22J4SUzN1gmmQuX7fbogVDmY1criJ48Uma/9kt2EW0lpiOUXJtK5vTsxtnpnKtFaa6eY7gYAwQtD8q7uQJzp+L3mVu00Nxdi9um1k1oAnZyCh+EyjuHsrfUqppmCbA7dZyQ0BBgyShw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kEcu45xdaIpfXtUGO2Sqnh0DJtv9WM0RX9NJ+e+vxXs=; b=UODhRrK7CCn3wvuVnh6kgfZTEVVI8/4u9ribXWt9c4560GyJmKMnCiOElfC6dIr5tX2dxPKguL0AUNCS8bIG1wSiCPGCy5ho3fH8cWKqYZ2kBsonSTookbLLydbPbjJes62k+gU0gkCV/BWO/L+IIsqpmr8S406g/BuQyeebsJWmJfOZHOodPVKA4QzIsESuvQMR3Qgn/ewq+p4V3yvE0kCQmoXQRYYNgNbtX8aUNZbPxXVPrUIqBTR8zMo9Xtgf7Al2CjIFs2OeK+5BPc63DLeGXYFja/w4NE2PHLBfqyoOaoQcK9Qi1zspqVTbGFwXHO5VCjHyAP5aB/i4ey0Hyg== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , =?utf-8?q?Roger?= =?utf-8?q?_Pau_Monn=C3=A9?= , Volodymyr Babchuk Subject: [PATCH v11 04/17] vpci: restrict unhandled read/write operations for guests Thread-Topic: [PATCH v11 04/17] vpci: restrict unhandled read/write operations for guests Thread-Index: AQHaJL6nE24AjhfTmkCFnkV7ezIkjw== Date: Sat, 2 Dec 2023 01:27:03 +0000 Message-ID: <20231202012556.2012281-5-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 80ec34e3-a240-4e66-ebcc-08dbf2d5cbae x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 2N/TNAahW4L73JX+HAXfzCTMdpr1gcTQhVzVOy+PLMysr87Ho5bS1WZ4sUwqYLwqkDxkmPOMTS3fcS0R14AZzEcxqvjUYOhi2bFuhggx9DmsICGIvRWWxBbwwKbDTjcf9iQ4B7XhZ0M30Z4zXR8Wf1ebgKClp+wK0p7vbk2uPYwg8L2C6Yy/KW+TXmBJgYBGhh4KPeSBzBNk8NFFYOcEH8Kr6xAxI6/RAMmEK1pMZ66FJxk2gNua8zVitz9Wf8nSunb8v/KqJ21bdi/Ltqw4y2/GzCur5r9DaJaWi5E6U7bJ5bpeWYtB6Yl6znjPnFcwFITSzMyK+XFwdwRiyYoSHyATLd6zlURGOVosnHsoceeuP90duHQGCUgxlX7oBIDzpFYHdOdPM4ngYQJn+ZHfte42dGSAggDfsZjyIWLLzjiLb0L1gDVfdZa78JxExxJG0TLvRLkyVykANsm2FTo33gHhzqKYNQdPGrCtQ+H6PIfe7ie2PFVdphjy7GZgDXPNRZ46crvdZ4HDfb1Y1I1cW1rRRurHCADnGLPSWTlh5SRRdvEFijVFExeDxtA5eY9xovoMgQtwoN+1tOnQJWU9r0QIKfz0dD7Sl0Ov8LNQdudLcfnyIVJaEngpChDl80K0 x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?t8ekKUdTnBnid36GLRx9ifLuEgkK?= =?utf-8?q?4uzUNCZSIfWgPK+reqtq2Grm1FuPgbF6jXEewoFiA45Ut/dvqX7PtK9iSG4cTXdnP?= =?utf-8?q?t4LyBOzZm5n4mSfKpd6VGjGaZQJTucNAlDl5iFebfNBDmVPjU5wRziu8J2rtW2uhG?= =?utf-8?q?53DgcukeWv2IzGkSrT8Hcd998i2orC2x+p7zpjBYam5wVsaVJl86XoeeYVrpodOz2?= =?utf-8?q?i3eE+0tA+16wAEwgJiBjS93RpuLZsEXtyk23AXPipNjfbts75n2/bx3vdhiLE/RSv?= =?utf-8?q?UBptkqhD8ffAEBnf+u8Fzjeq/qigkkvI09wlHm4i9slLu87MrWGBXgIt0T2vOFV2y?= =?utf-8?q?jTIT/jlS4RpMs1bjs/S1qlPJQiW0aTNEb564NA4feSWHnUonBzyI5QODX3tn8p3Gw?= =?utf-8?q?cRI5HhjMsvzeSZwmJnOh79iJpVu1qhyNxf467JLqgZDMj1T0KfriuM2knDrietESf?= =?utf-8?q?OsX9K6f+GzP1molhwOjM7Wul5pYStI1r8iRPQpzM2Nd7O7BLAOMw+wQ0RD4yBZxyC?= =?utf-8?q?bi/u4AqruIQUzyMNIsMzvyiM4AwVCIx5zV1bynKLBVmvgZBKxkW+DOj8Iq3/lr8xd?= =?utf-8?q?9+RTh4+vMnmINgrfHI/06eC/jL1dkidsfIyLfaUxG/ty1Vh/yVWRCYaWcP0wUDy5b?= =?utf-8?q?WbBddA3fwFpbzaL72MehwNnt34dtDc+muTUp58EZ6XNMc7AeJubdRxquUFNQ1JnJn?= =?utf-8?q?FEESBcqTwSO7BrsQtQ9QDC6CGkdfduG5uP2o6nL+EblTmBVAVUDquLunrypMlIUwj?= =?utf-8?q?Z+10sgYIv9nbls6sp4PeaTwES0L5u7PvGhJ/0sSEzPiIi3pgGRvxOUnsz8KgTL6WZ?= =?utf-8?q?UuDnPeAvi/j+D2a0iKyUsgf4NJW1odHSM6aG3h/AHwEjWp/qYWwxkWJ+v6bN71kOU?= =?utf-8?q?jG+U3mVUi5CwGZTdpfXEvLxQW2QVrGIMS3k9YtqD8tMbl0xjPGgmeYjxzfhRgSde1?= =?utf-8?q?+P3Ug7873f/Koc3KzU1THCZjd7c7PR37PWedTI6xxaFdbm2QAA4K/ewI/HRi90oZ9?= =?utf-8?q?pXJP2ibb3Y+qKkQDv9VjEQ2U19aCZLFTseTnQSZL4lSgsYR86JRmx3H3U7tE+8vxn?= =?utf-8?q?jUpy4Sho+J9012wW1V9/zhDYyCDIjHwqapHd3dce/toBe9w5TiCJ3P2T4DExelSui?= =?utf-8?q?80I7vahyVXZtDiMPQOEbWUt54dBXq2mSuHxeu0zOEBLvWA4PXzhdoAHoJaU/1g7Jv?= =?utf-8?q?sq39pCwZzc3kyHQX2x6B4p32kh0HXSyNP7aqchIjCEQjznt2NIroV/KxW9y0ci5Kc?= =?utf-8?q?uyeI3u42q17QCmn/sSjxGroPnrJlUCg0HxHH4V/X7QTpc4sV+YZoPkyBIcrjfto5N?= =?utf-8?q?VxvPYVz66b/hfLC4T5oofKn3/npC/prF19Q0xIQIwQ2909OcVeoPDFn7EMC2vX0aD?= =?utf-8?q?F3oeRiRw/4LpyE4+p2H+z/3NFntLhoKYypprOCnHrHDUQlCjKfZrOZ+Inugk7g7P3?= =?utf-8?q?LCYI50MtC1mvTDfh9+zfhOvcb1R2/PVYHWX+zgI3VPu69zxFIjvH2NRO8b4sUTriX?= =?utf-8?q?ReEZu1iO6YwkU83RBCBdAfgyWGw2Ao+oNA=3D=3D?= Content-ID: <0602D929B1A7F342A0843CA4FA2B004F@eurprd03.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 80ec34e3-a240-4e66-ebcc-08dbf2d5cbae X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:03.5224 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Vww/+lC8a1TlEoTF72cbIYT1y9XqExC3c/h9NufnURYzjLVp4D0kOePyFjXMc7b/acBK5+/+ZoZZxjDz98W2Iw1n78y13+M4VRaQAkS1guU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: NWLUW9YaROk7wzB6Wg9Jh9JOx0Yo-LuI X-Proofpoint-GUID: NWLUW9YaROk7wzB6Wg9Jh9JOx0Yo-LuI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko A guest would be able to read and write those registers which are not emulated and have no respective vPCI handlers, so it will be possible for it to access the hardware directly. In order to prevent a guest from reads and writes from/to the unhandled registers make sure only hardware domain can access the hardware directly and restrict guests from doing so. Suggested-by: Roger Pau Monné Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk Reviewed-by: Roger Pau Monné --- Since v9: - removed stray formatting change - added Roger's R-b tag Since v6: - do not use is_hwdom parameter for vpci_{read|write}_hw and use current->domain internally - update commit message New in v6 --- xen/drivers/vpci/vpci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 0b694beadf..4fec4b26d9 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -233,6 +233,10 @@ static uint32_t vpci_read_hw(pci_sbdf_t sbdf, unsigned int reg, { uint32_t data; + /* Guest domains are not allowed to read real hardware. */ + if ( !is_hardware_domain(current->domain) ) + return ~(uint32_t)0; + switch ( size ) { case 4: @@ -276,6 +280,10 @@ static uint32_t vpci_read_hw(pci_sbdf_t sbdf, unsigned int reg, static void vpci_write_hw(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, uint32_t data) { + /* Guest domains are not allowed to write real hardware. */ + if ( !is_hardware_domain(current->domain) ) + return; + switch ( size ) { case 4: From patchwork Sat Dec 2 01:27:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 52C48C10DC1 for ; Sat, 2 Dec 2023 01:27:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645831.1008251 (Exim 4.92) (envelope-from ) id 1r9Emu-0002Pg-O9; Sat, 02 Dec 2023 01:27:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645831.1008251; Sat, 02 Dec 2023 01:27:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emu-0002NY-FM; Sat, 02 Dec 2023 01:27:20 +0000 Received: by outflank-mailman (input) for mailman id 645831; Sat, 02 Dec 2023 01:27:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Ems-000271-Uk for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:18 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ec023917-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:15 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGux008549; Sat, 2 Dec 2023 01:27:12 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:11 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:06 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:06 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec023917-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mM8yHmad3IcII5Zqb2UQEHPu7SwkbZGTuQSS6nbF2dK0DMeoL2k4eZyKCBzD0auCTQguH9frS6HmZ6hJ+E8xr5Vfg2/fdRdISec5U+kgs9PxST0XX5q68aZSZx6bL9QscgkPf0sc0Zkb9K9xVlJrvCxjzEYGyB1eATQPxKoQaWD8NtGe2U7YN121SK13u4Jk5C03wQkFnHNPUZ6W4pNdkuxkrVpKLu3sPo36C6vfDMjFZalb9CCL+lMPfWw50dTWaPcOdIBS8/TL6BiuXmpUm5ffHsDbsq8WW3rk/Yj6i/9dM2OlMY+yuvXWw2ijPwccZ02fQ/0aICxWzNgBHgzsIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+ufRyLrmUQweuEkuIRkuhaAtvncGzXThnIHC9E3OfTc=; b=JriZjHaqC94pObkSE5KRdpBUtSpFtvGQkDTBybBwrsx1MzrOdpvD6MqsjHsGgPYooQLX1kYmHaU3ar4TjzxeAchdIFbsDbj8NU7bXr02NbdaFrthBZUWOFGWQjIp3DLpI8I4nybpI4gVxs0nM5yph/nphZZmspKKrnd43E7Y06rREshif5KkdBa1qv/oUs/BLH1RoPVA5sIWtMwNxeMg0gG+JQtslwhK/Es83WMG7S1WxHIAEs80TJodnh8msfIp40ygVO6NcuKW3vMqzZDuvrOPsO8O/aQYU+1+rRLqRGG86ZkKrvSgqRmb0t4mlkPJwEjNne+CGVuVU9Wu7wT0vg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+ufRyLrmUQweuEkuIRkuhaAtvncGzXThnIHC9E3OfTc=; b=H0Gtq134SqUatHUkpFlx1/snqF5tK46UDPsq5CHvEqMkiMQFGAvtdcYhoBv+8uusXWv8lxWyDs8hzaEN4d6Guu4E1tycR3QOuxCOis5lo46U9qYl9xiW+sUPcr3eqsDKW/+Xwz0P3PW9MByfBsmVsKqd7D7uzKG61GPNoWarxQqMufsTyWuKtl6tAsqgAbsF2wzDREcbhudenY+A//jL3GkVBN+71Rk86lHQlqHTg76RwBcNBDs6jQzw+DYQ7pdO/tmKNANu1aKMupCu/S0Ku4rWXC3JaysZOglCgiU7YZ3CV5S32wWPnvV0K1eq/SijxriCPOECZ+FJ6rYXxXIQkQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Volodymyr Babchuk Subject: [PATCH v11 05/17] vpci: add hooks for PCI device assign/de-assign Thread-Topic: [PATCH v11 05/17] vpci: add hooks for PCI device assign/de-assign Thread-Index: AQHaJL6ni6LuV+9huE67e0lasEO6Tw== Date: Sat, 2 Dec 2023 01:27:03 +0000 Message-ID: <20231202012556.2012281-6-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 4ac59772-3fe1-43cf-2a7f-08dbf2d5cbf5 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: T1NXKGGBLZ1YwzegKp3eCX/AU7B4El6e4EuVC5d0aVazkim0vi1AJqqzryZ7lT5glNUOBY9yCfLlL4ZeO8iISXlsT4CLbxw3lWM3dBXuNQ631bFW4zdOdsDE0k78URgpi7Ht0Io/g9nx9d0gRh8xWkf/Gb4uogR2tFP1L1shtQsHbNMx9d9HHA6FbFdJHV4BDk2NeAPFQSoaGakHInJj3k99ipvPBWR6C5CNN8XSEge0iJzl6ltuvrkiezbj/acUTt0JCM2yaAmnIiRVCT7nR+wJffKaTw7v0Jl0XwiV9cRf+zlGK95KyR2s/Jrk8hVR55CHoO65AQXcS6qAMe5/uW7H3QdZWdlwQth9BLkUC27Ve3ApnaeBdDFs6Dz0wYxsCtzJkV8x6m0rGycOKEaUuS6UzqdaxZocjlPp/4In5qX2lh2v3r46+TxTx6A9jM/gGjja43TUgmoL25SCuSdWlxLaXHrDLyq7J3ncvH/TWPfzJkarWVU47tlbROfN3AoEcLUgHNPsQ1VCu9S+0ZDAm4RM2UzcM3gkqwMNxXHoORsseLt7n6NFAxUql23muNbLT+X1nizrMguyJUKfFFg2PzYmv+2SnY4nyMQTA72SKBnhpex4i3xIXw04iyXDVqdyZufW56Rh3dThET7I48JwQ3SAOZTSpwaRTIDoJObTPPQ= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(230273577357003)(230173577357003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?ukb3y7IMqtxe1PfIdSc2J0W?= =?iso-8859-1?q?us/HZyRol4omHP+bsNBUOzAXbVQn0c6KuJLtd5vtGVrGHb6QWqMRYYTvEw7X?= =?iso-8859-1?q?AOZJTP5AN2Tys0KE82iOdk1ko3QLuZO99hSdeFqgtNnk6BpUCvLpCD7Dn5xk?= =?iso-8859-1?q?ktf2MJAjQo148V/md0pxcfj4yyAxudpZu6bKoEOC4fsgLKIhA3Ome9NmbHnR?= =?iso-8859-1?q?Vyvs7tVl/rlMUkD2NEn5OV24daP9nIh3OiXFZOZ6r7CIZNjX4KSH7BDJR5B8?= =?iso-8859-1?q?3oX6kMUZHZaDc4VvfxsbBYYprpludqxaVZD2l8DXaJ8HQGQsB9dJwgr706/S?= =?iso-8859-1?q?fuQf9A0wwdzBDFQmdFLmbA/GZKWRhcT/Z8C1S88+gzItMoJ64QIgWAOWA9uU?= =?iso-8859-1?q?Kwv6KfgpUfMrvZcg/iPl8+KNAM7IvFPVrP3oPUQs2hzxyF+k3m8Rxnm518h7?= =?iso-8859-1?q?CbddZqN06NkiCOAy5Y/y1498iD2rku07ZgcVhYAwtAVGWqEeBPWm46bD28TW?= =?iso-8859-1?q?Z5NbMYx9lba3bf3wYlRBfdpGdbYa3FLSn9o1dl4OpQ7d5nyvNphXl3iTyhX5?= =?iso-8859-1?q?bb+sgc/C1anF7VwCuY5d0zSEbjdkCsPovk6iQRroCd48d5QtO7ooQT7V2Zqk?= =?iso-8859-1?q?zpqf+HO2wf5QbXyw1SdF22dhuXh40B0OGIkxcYVGP1+5FkmK7SIKR+vgyHkp?= =?iso-8859-1?q?YZ5NEtaFi0j5PpLhkZ7VrF1Wn2we9y8syEvw6uTB9EEWWnef3ZB6lrqYrUW3?= =?iso-8859-1?q?Q0ijVqA0U5QiSjPcfcwbwOoZwRBFGpJFztxiYNc2k5SYuTkHN0W+mRzSfVvU?= =?iso-8859-1?q?HOC7iJsRzsr2/Vseqz6Aeo/skIYJJgPcGpIZdIKS9bYhblJnHiEVTdS9K4OI?= =?iso-8859-1?q?PfGPSbciNmTMIrIhTHHzcsEZW4ZfUp70V9kSCHxBqIseiVUQJrXvd1jH/M2d?= =?iso-8859-1?q?tKoO+m/kf0GO6K+7Cv7R2FZGoAWG5iah+C47/uwKQMrOQHehLOnE7PlQYxW9?= =?iso-8859-1?q?tq+ceoonTFzbsgm7oTf5QKpmfZJq13veMzRLfI6zyN8iKR/bdtGjweDHaXcP?= =?iso-8859-1?q?MnP28MYeutTxmu+qCsKnC/ETMsTWQTtnxmaB+gv8YM34dhG7vbzQaSaH+zgY?= =?iso-8859-1?q?1UkRFKKa9hYr0n5ZDINIYLsOAEXR3Gl5eexMD6egSnIRoZhNp8pLNUuhltTu?= =?iso-8859-1?q?SCVeDUQQJsObprfM9DCFmPGs7jrhCXfN5nJV0Z+kgOemlH13aLCMMlgmR8HL?= =?iso-8859-1?q?lmFxvDOnugieD54y9RMBesrecu5gFKE/74zp8aHyhQtcSQLZPu5LK47ocPI1?= =?iso-8859-1?q?3H2rXMwxDmVN2hJXDWnv4vIghbPWffdTmjsdaAbLA8D7inhPabDpwZTPZAz/?= =?iso-8859-1?q?C7g/pT01IZjJxKF97/Gn1dTmwJa/z2rxTj7YV0ulc5m+OHnCRbeG3zOGoYNb?= =?iso-8859-1?q?R0aD4g2Aeas2AbIf0CvP577TBm1Z8diULAgq60PZ0QMR3T24Fha2olcM+IEK?= =?iso-8859-1?q?NebMBfVfh8VXJRR888VXQD7H30aLW0NIX7ySlKVPf54NAajcDQaekiqqViyW?= =?iso-8859-1?q?COqPoEIl9VRnP1n4IcU6rG2zICrffciSdYBPEFxf34Amvt3ppBphBrhahKJe?= =?iso-8859-1?q?hgW6HWB33JV3Rho0dqM24jxe3Ti1GWSXEpNl9pg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4ac59772-3fe1-43cf-2a7f-08dbf2d5cbf5 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:03.8312 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: apciIFBOVQw2VZTUe+CQTqJ59dVXwGcHVhf1gl8IkNeTxC4PA+dnxBidAuoS5UBSkZwfMKnmtcVerRJdbzoZjLlmJYteEO7sTTY8zha1v00= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: w0xi6zycblUHWwVUxpM0FEYZ4N3Kodon X-Proofpoint-ORIG-GUID: w0xi6zycblUHWwVUxpM0FEYZ4N3Kodon X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=753 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko When a PCI device gets assigned/de-assigned we need to initialize/de-initialize vPCI state for the device. Also, rename vpci_add_handlers() to vpci_assign_device() and vpci_remove_device() to vpci_deassign_device() to better reflect role of the functions. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk Reviewed-by: Roger Pau Monné --- In v11: - Call vpci_assign_device() in "deassign_device" if IOMMU call "reassign_device" was successful. In v10: - removed HAS_VPCI_GUEST_SUPPORT checks - HAS_VPCI_GUEST_SUPPORT config option (in Kconfig) as it is not used anywhere In v9: - removed previous vpci_[de]assign_device function and renamed existing handlers - dropped attempts to handle errors in assign_device() function - do not call vpci_assign_device for dom_io - use d instead of pdev->domain - use IS_ENABLED macro In v8: - removed vpci_deassign_device In v6: - do not pass struct domain to vpci_{assign|deassign}_device as pdev->domain can be used - do not leave the device assigned (pdev->domain == new domain) in case vpci_assign_device fails: try to de-assign and if this also fails, then crash the domain In v5: - do not split code into run_vpci_init - do not check for is_system_domain in vpci_{de}assign_device - do not use vpci_remove_device_handlers_locked and re-allocate pdev->vpci completely - make vpci_deassign_device void In v4: - de-assign vPCI from the previous domain on device assignment - do not remove handlers in vpci_assign_device as those must not exist at that point In v3: - remove toolstack roll-back description from the commit message as error are to be handled with proper cleanup in Xen itself - remove __must_check - remove redundant rc check while assigning devices - fix redundant CONFIG_HAS_VPCI check for CONFIG_HAS_VPCI_GUEST_SUPPORT - use REGISTER_VPCI_INIT machinery to run required steps on device init/assign: add run_vpci_init helper In v2: - define CONFIG_HAS_VPCI_GUEST_SUPPORT so dead code is not compiled for x86 In v1: - constify struct pci_dev where possible - do not open code is_system_domain() - extended the commit message --- xen/drivers/passthrough/pci.c | 24 ++++++++++++++++++++---- xen/drivers/vpci/header.c | 2 +- xen/drivers/vpci/vpci.c | 6 +++--- xen/include/xen/vpci.h | 10 +++++----- 4 files changed, 29 insertions(+), 13 deletions(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 182da45acb..a3312fdab2 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -755,7 +755,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, * For devices not discovered by Xen during boot, add vPCI handlers * when Dom0 first informs Xen about such devices. */ - ret = vpci_add_handlers(pdev); + ret = vpci_assign_device(pdev); if ( ret ) { list_del(&pdev->domain_list); @@ -769,7 +769,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, if ( ret ) { write_lock(&hardware_domain->pci_lock); - vpci_remove_device(pdev); + vpci_deassign_device(pdev); list_del(&pdev->domain_list); write_unlock(&hardware_domain->pci_lock); pdev->domain = NULL; @@ -817,7 +817,7 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->bus == bus && pdev->devfn == devfn ) { - vpci_remove_device(pdev); + vpci_deassign_device(pdev); pci_cleanup_msi(pdev); ret = iommu_remove_device(pdev); if ( pdev->domain ) @@ -875,6 +875,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, goto out; } + write_lock(&d->pci_lock); + vpci_deassign_device(pdev); + write_unlock(&d->pci_lock); + devfn = pdev->devfn; ret = iommu_call(hd->platform_ops, reassign_device, d, target, devfn, pci_to_dev(pdev)); @@ -886,6 +890,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, pdev->fault.count = 0; + write_lock(&target->pci_lock); + ret = vpci_assign_device(pdev); + write_unlock(&target->pci_lock); + out: if ( ret ) printk(XENLOG_G_ERR "%pd: deassign (%pp) failed (%d)\n", @@ -1146,7 +1154,7 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt, PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) ); write_lock(&ctxt->d->pci_lock); - err = vpci_add_handlers(pdev); + err = vpci_assign_device(pdev); write_unlock(&ctxt->d->pci_lock); if ( err ) printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n", @@ -1476,6 +1484,10 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) if ( pdev->broken && d != hardware_domain && d != dom_io ) goto done; + write_lock(&pdev->domain->pci_lock); + vpci_deassign_device(pdev); + write_unlock(&pdev->domain->pci_lock); + rc = pdev_msix_assign(d, pdev); if ( rc ) goto done; @@ -1502,6 +1514,10 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) pci_to_dev(pdev), flag); } + write_lock(&d->pci_lock); + rc = vpci_assign_device(pdev); + write_unlock(&d->pci_lock); + done: if ( rc ) printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n", diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index c86d33d0cd..ec6c93eef6 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -190,7 +190,7 @@ bool vpci_process_pending(struct vcpu *v) * killed in order to avoid leaking stale p2m mappings on * failure. */ - vpci_remove_device(v->vpci.pdev); + vpci_deassign_device(v->vpci.pdev); write_unlock(&v->domain->pci_lock); } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 4fec4b26d9..9dacbcf958 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -36,7 +36,7 @@ extern vpci_register_init_t *const __start_vpci_array[]; extern vpci_register_init_t *const __end_vpci_array[]; #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array) -void vpci_remove_device(struct pci_dev *pdev) +void vpci_deassign_device(struct pci_dev *pdev) { ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); @@ -69,7 +69,7 @@ void vpci_remove_device(struct pci_dev *pdev) pdev->vpci = NULL; } -int vpci_add_handlers(struct pci_dev *pdev) +int vpci_assign_device(struct pci_dev *pdev) { unsigned int i; const unsigned long *ro_map; @@ -103,7 +103,7 @@ int vpci_add_handlers(struct pci_dev *pdev) } if ( rc ) - vpci_remove_device(pdev); + vpci_deassign_device(pdev); return rc; } diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index d743d96a10..75cfb532ee 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -25,11 +25,11 @@ typedef int vpci_register_init_t(struct pci_dev *dev); static vpci_register_init_t *const x##_entry \ __used_section(".data.vpci." p) = x -/* Add vPCI handlers to device. */ -int __must_check vpci_add_handlers(struct pci_dev *pdev); +/* Assign vPCI to device by adding handlers to device. */ +int __must_check vpci_assign_device(struct pci_dev *pdev); /* Remove all handlers and free vpci related structures. */ -void vpci_remove_device(struct pci_dev *pdev); +void vpci_deassign_device(struct pci_dev *pdev); /* Add/remove a register handler. */ int __must_check vpci_add_register(struct vpci *vpci, @@ -235,12 +235,12 @@ bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, #else /* !CONFIG_HAS_VPCI */ struct vpci_vcpu {}; -static inline int vpci_add_handlers(struct pci_dev *pdev) +static inline int vpci_assign_device(struct pci_dev *pdev) { return 0; } -static inline void vpci_remove_device(struct pci_dev *pdev) { } +static inline void vpci_deassign_device(struct pci_dev *pdev) { } static inline void vpci_dump_msi(void) { } From patchwork Sat Dec 2 01:27:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6933C4167B for ; Sat, 2 Dec 2023 01:27:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645830.1008244 (Exim 4.92) (envelope-from ) id 1r9Emu-0002IG-An; Sat, 02 Dec 2023 01:27:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645830.1008244; Sat, 02 Dec 2023 01:27:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emu-0002HK-5C; Sat, 02 Dec 2023 01:27:20 +0000 Received: by outflank-mailman (input) for mailman id 645830; Sat, 02 Dec 2023 01:27:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Ems-000271-Nj for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:18 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ebfa4255-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:14 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWZ004068; Sat, 2 Dec 2023 01:27:12 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:12 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:07 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:07 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ebfa4255-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hP60KBmfNMFnjfgi1NsdMQEhxrg1U9g4M9jo8A6E9qNb9kDA9lmalLgcqOL77+sm4L1FyYepuJiiMlqbGh8oGPY7NHfTpDpbmGmdKxXgPM9pZ503bdwRHi4ntmmhfSHrqDlCVFS9mRKXag7RmMqHShJavjjDb6MSdNOJsk8NKs3iW36He4ugQLtEjcT9U1SUfAJ11sl2xAvvxC6U/4nHaCFn/vsphczfKc1KB0VjM1aINSuoqZxYoFlufUBedqC9k/mZxAf2W5GQm6HUDl6pKTMF3EQok/G/3Q+f4mYAfWQfnTgClnJAdRUPkrKDUzfUjzi3PDfsf4PZ0zjAuiqjbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SFwYHxU24LM47Wx7eKT7t91aTd5L+oJ//Tk/2jLWySA=; b=euaBkwaH4JkwfYUsiOJbB8lyo2DSGT3Pp19b0KD/ttzezvKzta83irEPVvdLDNDqfmUqZfeSeQGo3E7DBkHmoxmBltOtBuSU/IyDVpg/qVtFmCRC7Pj4vT70ri/7IX63PB/igZNrdU6zvg3Ig7deQBDy4r2WG/gWa99y0KJROavN2EbR6ToHKjUhlYs32DQD72egn8nCurnj4V5Kx0Raho1mkVKsHqMP/ciUuHWYlYeI0j9qnOSwpQxUKPZookA1a33aeiH6KRgjDMDXlil74II88BfSVyI7vI+4upXZ5qbMs24aMMFIHE+c/fpYEelPdtCpTWfZ84ItW8+XeXVHCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SFwYHxU24LM47Wx7eKT7t91aTd5L+oJ//Tk/2jLWySA=; b=P6GtjoKrKttcLr4AYJm9tfR4Cm7oSOmuKtCFLa/x0kIjrtkE7EcKdWpRkA74IRIVmfxLp6pXm6usY3Ic8RtEGBJcbEb1NwV03BTl1b2keVyktQFKdRwaiksjRiabCqDwfnjqyyP1Z8pcb6CV6AGewjtj5V+ypD48PdvBXxXZyHmD394Fu642H2+29r/+STMyZP5RGJplhGn1DgrWNCvz+SalpB3fQGLjKEUI/olq7J6H7dLYsyuOoaKBZqv4RQ34kHpuMae3Tdj9VLY6O4qczxibzyXjEdOhfNrbchXkaYV9pFDeNntq5G72pyWmoduGaD6+r0tqW5vl/JCp0Dsk1w== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Subject: [PATCH v11 06/17] vpci/header: rework exit path in init_bars Thread-Topic: [PATCH v11 06/17] vpci/header: rework exit path in init_bars Thread-Index: AQHaJL6n1kPp1+DEXkyG3DA4rY4zlA== Date: Sat, 2 Dec 2023 01:27:04 +0000 Message-ID: <20231202012556.2012281-7-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 9198ba8e-0535-4f38-dfc3-08dbf2d5cc2b x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: is4K3x+RaHsNn5pOgEiOOEKDVwXh7Y6YF4KcfKgo6iXJObxSNcTYyiJ9sVsXS8i1Cxf2/fXhY/RtoQy/SbEMwqxJ/oTgC6vug6PkMkMbO2iqwJYIwT4HnM1JVtH89AxWEViW0oFgnNOd9VJ0r4ow/yqxelLdo8d0QIxEt58nUK6XuWW2dRMCrItiIbu9pbgs6M0PAtMjmgqi/yh1kiAN8ACdP/bR0Lj5mtWX1J3NGD0+JsB0Gqfk91qmC/NEmtbk55ibPASYvs4RsW/nHlURoHdryK2ndeUGrB7x8rTb4c5aRZ/eVjE8SK6imTa+JqGxi0Q85fLQwAsvVzI75wOl/S7VXO49jnq/0AohqriaV4a2s5n6Lvao6v4S+RqVo5vPDbNiygULFZMKOplflS06RR5ciyXtKfGLfQ24Vi/6a3dz/guc72oBe1w28aDGgJCNYwqoWVooRxIldxQWhXFRKcTpVgiADaGw7he6ndFLpgrrZbf1d1hFsXYUw4YI7bcPD0qB/W3ghS+pywiekB95EITkA/V4afqWH119yTyquF6gwPp7jjvDvalgMBOJJpcidKG0DBMtb/zBYNXBazfv7Rnb3Ix88v5G8fwg1z7tVvyI5QHSdj2fUMC1usApD8nG x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?7hcuBPTzhabFIoqgXe/WK+Nu0gHc?= =?utf-8?q?i7CZKIZT6K2eeHf1ayqpBROIMhXjtdlkFdyPNR1OlLNFVUfIEUYv6bks2kny+ktIx?= =?utf-8?q?QclxFzvoKb68Mmd1hb1Km8UL8LbbE0ZwkcMGpseb+q+V99Fa7sU0XKAMgyALEz7+s?= =?utf-8?q?L5j10gGxsjAd74FbyI+oqsUp65Pr45gNLNQSw6cfi2fhDi+AgiO3WX+6f2WR3Wo3C?= =?utf-8?q?Aha49NSAj9PU5FNOQmLXO+tQNcG0wh2XtC2avLfxiJ9zIHfcOjeNfGkEHCCz3Sd8J?= =?utf-8?q?zwGbLAQ9DiKVLoU1InZvsz/Ndz8B1WsLj9kkwO5br+d6cAnMavv/GvogcDkz61bCI?= =?utf-8?q?2Kan1dfzPVeHGCI/wmfUYX6TQogxOWBOLRwUmnqaMQXHFCBYloO+pPQu9oJ+3c9CY?= =?utf-8?q?agOVIGcRqog1iapR4itYlPt7icl19VBJy8GaHS32PBTRtWEEPXPA/Sz0DwoTK2Yu2?= =?utf-8?q?vOpNY1l67tckBacilTC+vnoTij3HcVv4euJPuhA8xubs+E0aKawivR7U/1449wXAN?= =?utf-8?q?OBFZmHKfMK1T8ODfbLK6EBXlBtho+OPYzxv62hQQ6vvBUQgA/G/nvKSB5g9BZu9cL?= =?utf-8?q?QdsUDV79L8EIddjy5SPHRLDJOKc08/AoTRSq6p4nJLuvnL8qFiTDy+mvdEV/H2uTS?= =?utf-8?q?bfATG6lOvJoJl9Yzi0KO96ng835/F0MwLstfbYhQVt/y+2kyyLSXjvoVEzaC3PHcd?= =?utf-8?q?bUpxZ9WbigOAYpDfUYmOD17SuVWahk+LlElyAC6eSyOl2yFTqyiv8wC1cR/10GYD+?= =?utf-8?q?iLl64lwg6ho9ZQKfzho3aPnnDk9qpDju/XbXoEX6qmOznL21HB0nXBNFKQXjBkIAr?= =?utf-8?q?CS3mhw44NO8SNU//N++SwewKaCh4b+7jdbxNVNrkOTT7eLLlaDxx9GkOWbU4Tot62?= =?utf-8?q?K1LOmih3q7yHEdgq61dAggX/N2mbNVOI8Flo+lHOunziEqvjAqVdo1qtGakzqVIvK?= =?utf-8?q?1u1Hg3j4fows1ZJsWktlnXiZnNkhnD/Nh1dumpbX537qT/V6c1LcN2jc/OqWWrcLZ?= =?utf-8?q?02Vtyl1smw17KTz5y7asIlASBMwAtZCl1EQ8yWir/aQRsgjWo0MoRhFznfChEOWOJ?= =?utf-8?q?kE+uJ6urJ0Dnmjn1DEidy5YFjUbcdjlFle1uHcuhYKsYvV2jJMVfwaLPkdRflLW4K?= =?utf-8?q?tzhyfrqj7tHJUyznr+ZU1CJkEXvYobCz/wT0Kf1pxk8Y37SodtuTJFZvxkixiKfm2?= =?utf-8?q?Jzaf1YCjkVAinF6jqgFRm2mhRopJ/ts0LG1CkNGZuw6MEMTXJ1TIvVetwb3OShlLe?= =?utf-8?q?k6AtIqRnV0xov3lDfz38BkcvfsdqM/wRbT5vR4mn55xgBKMU0yE2KkSlXhXFhhpBG?= =?utf-8?q?hDXu2lrAg2CEQpRdT2ucdFFJ3HGp8gLRPXwO/4KJbWn5w8cyZR/e404ngFr3UENcq?= =?utf-8?q?vtz/BO+y8j/uSTxIZEYNiKiGQKlt8+NvZeRCXEN7abPjScrtGmz1yUrhlEkCL6vLl?= =?utf-8?q?aV/JawBekWHXtsSoWlSosej/8aUnXhY+x2PF0iUPsOZZ+eVWYLOBs12khHSKcu3cz?= =?utf-8?q?UIyIJIlTHNJU/CWmntAPSzY/KB/1w/op+Q=3D=3D?= Content-ID: MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9198ba8e-0535-4f38-dfc3-08dbf2d5cc2b X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:04.1397 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bPrDjI8ZhUaHNyVjvH0tWsCZwlrQWOQKz+4lQKh6DTveeedSLI32RgMW7zprGVlSmtTLnSHIE4L4zWu8MJXey8LeBehgPHEbKIlrCCIKYsk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: T730irxzVb2dsD5u96ynCg5bhDM3i3Kk X-Proofpoint-GUID: T730irxzVb2dsD5u96ynCg5bhDM3i3Kk X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=930 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 Introduce "fail" label in init_bars() function to have the centralized error return path. This is the pre-requirement for the future changes in this function. This patch does not introduce functional changes. Signed-off-by: Volodymyr Babchuk Suggested-by: Roger Pau Monné Acked-by: Roger Pau Monné --- In v11: - Do not remove empty line between "goto fail;" and "continue;" In v10: - Added Roger's A-b tag. In v9: - New in v9 --- xen/drivers/vpci/header.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index ec6c93eef6..e6a1d58c42 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -581,10 +581,7 @@ static int cf_check init_bars(struct pci_dev *pdev) rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4, &bars[i]); if ( rc ) - { - pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); - return rc; - } + goto fail; continue; } @@ -604,10 +601,7 @@ static int cf_check init_bars(struct pci_dev *pdev) rc = pci_size_mem_bar(pdev->sbdf, reg, &addr, &size, (i == num_bars - 1) ? PCI_BAR_LAST : 0); if ( rc < 0 ) - { - pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); - return rc; - } + goto fail; if ( size == 0 ) { @@ -622,10 +616,7 @@ static int cf_check init_bars(struct pci_dev *pdev) rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4, &bars[i]); if ( rc ) - { - pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); - return rc; - } + goto fail; } /* Check expansion ROM. */ @@ -647,6 +638,10 @@ static int cf_check init_bars(struct pci_dev *pdev) } return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0; + + fail: + pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); + return rc; } REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE); From patchwork Sat Dec 2 01:27:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D6F04C10F07 for ; Sat, 2 Dec 2023 01:27:36 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645828.1008232 (Exim 4.92) (envelope-from ) id 1r9Emt-00027Z-LP; Sat, 02 Dec 2023 01:27:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645828.1008232; Sat, 02 Dec 2023 01:27:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emt-00027S-IS; Sat, 02 Dec 2023 01:27:19 +0000 Received: by outflank-mailman (input) for mailman id 645828; Sat, 02 Dec 2023 01:27:18 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Ems-000272-8U for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:18 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ec92fc62-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:16 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGv0008549; Sat, 2 Dec 2023 01:27:12 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:12 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:07 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:07 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec92fc62-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fNOrBjonGpoY9DN1gCwVwR2d6NwrLJyAara6IXSLX1N5WHxSxrCfy7ZuXI25PS5qO30S8xvMYOEc6g9hc3O41mg91O047itZA8NFTLdxay0HpFAlX8mH2azn7fGnI2DE9biI9Rniu8SyVTo3Cqbu1SSZCtsPY9N2JecLL400Pqk42hK4p/GtB8WTM7cp3insIxEd2tIdx6+I4PGRyXpR8qBJkIlu5lA8d2u1yKR22jnQIAJvbObSaKq+RGUOUpHxLtesg59Dv5DdejCz4EU7RPAgmt9m6QmLxv+SPGXChkkcwMapMTMs3c+gUokWDKAjVDzUA48k8aw5V75KnV0H0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fCAtQkOh1LlRj3VEXjXrBI42f2r4sRFKlSbnUPHqVJA=; b=BH+U/AlC8xu908Yx+05uXx85l7StqHYXsUWum4OGSKeYcPTF0qQzAs1TbbZACbZQHTOgQZ3z1EJBnBlpVw38cGuDMvI000QHsLorwp55sdHq4NF21OExLgssXuBsF8Zc32oDVAjZE1FktPBzIWNApfjRrXP0Agoles9EJulBBs4vUPI6Fq88PzJfOw1WrTB6NSaplEvMbZaQwWBYsOfRRqVreUBGD+b5LFdalNWM1LVbKldJRJM4kvcURtLFOLtSxfWQ7UoMATBZLMhGqEL8cdNbjOxiswLd826oZyHLqLzgsDLLA4+2OoTl+fvyeSkbyb22D4R83pMdR6b/BgdEcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fCAtQkOh1LlRj3VEXjXrBI42f2r4sRFKlSbnUPHqVJA=; b=J/LuqckK9XcohCSzJGtJvEEQFCu3cPWCBA9fg6MreZWdkXJh71qKm2u1sNXZg0IpCUV9c8HCd5nlrjbzgXSIqKAPZF9LX9Kg+FjfBjWqRmLR4uAyU++adIOrdrbftMhkG8m6Fi/v2Gz0U00SjbdP4X4dZwY+aAxsBbYDJicu1ZLNrNRSAMeeE2+zftUWLdhKMdaHlxWyPV7Ex6kNOTwl07Eqerq31dfWADXJkrCUuX5JDgolcSOwUYBYMzqRAAczqck0AvOgIeGrTjs19WqS3ekXuWpwLj0tiw7XWjcbGt0I8yUPk82u3d7cXZEiPx3po4jj27FLDrubQrXqmO6e4w== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Volodymyr Babchuk Subject: [PATCH v11 07/17] vpci/header: implement guest BAR register handlers Thread-Topic: [PATCH v11 07/17] vpci/header: implement guest BAR register handlers Thread-Index: AQHaJL6oqjV1cLv4e0GmK4Y233LkQA== Date: Sat, 2 Dec 2023 01:27:04 +0000 Message-ID: <20231202012556.2012281-8-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 4096a790-67a7-4017-985c-08dbf2d5cc65 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: bZ/mq0D5DD02n0TzDj21sjIrPTrc5BYS8fSHsWsO/PBomx1RvJstm182hO1w/+JDII4pXOJlM2XINppKZQxEezv+wbTSDKhxqx1vvpPEq7t4QaeghGUylzetrILD9eKf07x9nNMF2WO15UP87MOTwMYEdElLCtQw9iAhxlvsxt4n3OeVT3j7krqTSS3MM6sUUz2Vn34FU4npSCMCXAf8Lz7AGehQeg/cC47evwLyOIjlpiU736OyTmCqBGe3R6xnMJhO2sg9nfU4a2MbVUlGXTXU8PJ87+XeMi3zVZV3scfJ6dENAFYriwj8kSPeNC2Mev+vWqgSulBCcFyJgHcMmsOxW6l7L8mimQFVL+LScUQNHvEIxF3vKa3nHcIgQjySSdjrrMQ+mmcrmt2yH7IhymE1c3nOmz9c2Dga5IGqjV+8TXRC9jfIRmrIj+Wzd6WT2FLwcohrEbe9mBqGyxcOd8H1HLZMFX//QLot34Z5oJr7Ke/2yJUJ7Of1Tmrv5Me6BpogTYkmflKpQxnH6L+uqxlX3SN8994d+03jikhgLsVzthNONokwJ+K+EVn1tM+0AHrtdtQYWdtt49WRlag/NKK8aTs7hmvnnEEooHmMu6eKgHObeIYlmRPfipQ17a7u9CIO4YI7DDobKeFYCuxFxwBxFOS1SVCdXPW83JEQsrQ= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(230273577357003)(230173577357003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?Htiq+dBpyQVuNr6YzmL/BOo?= =?iso-8859-1?q?bwv5XoY6X8QAQQm3GTyYklzxwMl4QitTh8gpnQzFVCXOOmjf7jxAUASsQaqW?= =?iso-8859-1?q?SbPMM63nvufSu/OJmsQo8XXP/kCuL796Wymhrskc38JiUMfeGtnV5kBy4hRm?= =?iso-8859-1?q?1cVOW37iyCiwNWytLx2zBbbyM88IOgLjT7NWN+6s4gf6+VEbvX++4atJbcdN?= =?iso-8859-1?q?Qh+zucmmPcSX1+P9J2AtuH4APFhMfedIUBfpQuUyNHiqNbsTf8Rg5dfkWfBa?= =?iso-8859-1?q?BAlTyamkEB1Oa1QRh/CeqriMcvgd5fJIkgZeJufkUsgr8sDplces7RZdqJ1Z?= =?iso-8859-1?q?7HTXhJUOHrvhsbTy5tMvDawXyXAmilu+xhbwtxDRb75VlyIUVxblSx4JgrG7?= =?iso-8859-1?q?hqOwQAfqtbINRATlxqj+coICijCAYqWadwzfTaKbzakCH4HfHu5+BGIWlw5P?= =?iso-8859-1?q?g2qljB+JS01fHzL4WtvupLv965uIvMEyW4R0ewRalGVTvsLq9PLycww5apkQ?= =?iso-8859-1?q?zId76DKo6kk8hMUJY8D0B4qOGIWuIXAcFctuw1TqDTU7AUO08/D28bJNHiqF?= =?iso-8859-1?q?+72PXK9fbW9S1NfHDePLUHlIlyk3kgZhFtqoo230pn4jHMRme4RUncQu3/Ou?= =?iso-8859-1?q?fS/CKrqAtIqygUMWssLtGCcJXzFiabygOK6YuXX79g2H6AngZO7UuZfsbwq2?= =?iso-8859-1?q?zGdM+IU4h73NY8uS7bBteH7zdHq3OOEe79k16bwKn5pNxsmR3637rHeJKGFP?= =?iso-8859-1?q?rD0qJYMdHg0Tq1Khx85TGNyxRS6LnZ0T4R5IiwtQkirbhms0BhzVUu8TC0aj?= =?iso-8859-1?q?qlBPfu03fuJJp/Tg9qekbN36YQSgzddvFzkZuMv79DKqsQShhj7OB3Pv4SfU?= =?iso-8859-1?q?1zagho+tJ7kTirJCWeJZ0fdrfEG45BLCRfFq2a125GfHCtKbs0e1tJKei1fT?= =?iso-8859-1?q?Hh6iV9IhmmSVD66+UDGzqBRypy1n/AEfxycJagudOS/OuThSOYwX1XxmwJFd?= =?iso-8859-1?q?oaTSnzee0pLZdEo/AQYuefvFkv+tEQ/dJQy1HOU6zllCwmPZubEpXBJf/rhA?= =?iso-8859-1?q?CYgtrAF6WkG5E33+yVc4njFzwhvK21+VvoEGSdZO7oUvA+boAV/fT9Azunjr?= =?iso-8859-1?q?IDkXnhhSfsRt21kR/v6VE5O181o5k3lD1KbTBoH+q6S6LVABfxLN2tKDb0NE?= =?iso-8859-1?q?yD4vSu4LkTtWC9YrPEFQy29c6MtMJ5yvbMtg9qRgWXi0FFdTk0W4amJY9m+1?= =?iso-8859-1?q?y93OyQ7WcxLrmplnvwRa+Wm39OQX1P4wmKI/bBlwJfquT/olAPhtGEoNRKr8?= =?iso-8859-1?q?TwKlTnL1QvoYhhPZyE72N4OnOQnMk9tiip+cjn6koZGZBEOGSm6A0K3+WX5m?= =?iso-8859-1?q?10dfa8hSdubMZVNpwfQWlwGzWbM0eC0f6PJdKXeRXgBqyvBx3m5bLPxfrjbW?= =?iso-8859-1?q?PG4IlnaQkO9Xnej3xQXOQ/LRj1GWIWiU1ZSc7tJvOUOxJ2HWQyyAkZ5KQrjd?= =?iso-8859-1?q?BdXYHkvyAs1daYv9oV0ECh3eExMwdeuU3aJUqKC4O9GxEqd885aMBE5OV0Ol?= =?iso-8859-1?q?+5Sun3IQx4jFY1BOiLLjWggcdM/7Vgd1YGZlUH4HA/h+GdbKdJno9uPzb2LW?= =?iso-8859-1?q?pakutau0INxEd6OlckkSvTGLOelA/mHDWpHLJqQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4096a790-67a7-4017-985c-08dbf2d5cc65 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:04.5039 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: zdp+vZbn8xdrJiaDy1p0te9QVmS4piHM514fR+wL1/RJpEnFhuhSkt4mE1xrzFSNmVYp6/HDd1ao7krFSNOrr7iZaGYN8FSLMiSbweM/c9g= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: sCeXfsJgOFqBdcKh-tnc0IaIldpWNxpk X-Proofpoint-ORIG-GUID: sCeXfsJgOFqBdcKh-tnc0IaIldpWNxpk X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko Add relevant vpci register handlers when assigning PCI device to a domain and remove those when de-assigning. This allows having different handlers for different domains, e.g. hwdom and other guests. Emulate guest BAR register values: this allows creating a guest view of the registers and emulates size and properties probe as it is done during PCI device enumeration by the guest. All empty, IO and ROM BARs for guests are emulated by returning 0 on reads and ignoring writes: this BARs are special with this respect as their lower bits have special meaning, so returning default ~0 on read may confuse guest OS. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk Reviewed-by: Roger Pau Monné --- In v11: - Access guest_addr after adjusting for MEM64_HI bar in guest_bar_write() - guest bar handlers renamed and now _mem_ part to denote that they are handling only memory BARs - refuse to update guest BAR address if BAR is enabled In v10: - ull -> ULL to be MISRA-compatbile - Use PAGE_OFFSET() instead of combining with ~PAGE_MASK - Set type of empty bars to VPCI_BAR_EMPTY In v9: - factored-out "fail" label introduction in init_bars() - replaced #ifdef CONFIG_X86 with IS_ENABLED() - do not pass bars[i] to empty_bar_read() handler - store guest's BAR address instead of guests BAR register view Since v6: - unify the writing of the PCI_COMMAND register on the error path into a label - do not introduce bar_ignore_access helper and open code - s/guest_bar_ignore_read/empty_bar_read - update error message in guest_bar_write - only setup empty_bar_read for IO if !x86 Since v5: - make sure that the guest set address has the same page offset as the physical address on the host - remove guest_rom_{read|write} as those just implement the default behaviour of the registers not being handled - adjusted comment for struct vpci.addr field - add guest handlers for BARs which are not handled and will otherwise return ~0 on read and ignore writes. The BARs are special with this respect as their lower bits have special meaning, so returning ~0 doesn't seem to be right Since v4: - updated commit message - s/guest_addr/guest_reg Since v3: - squashed two patches: dynamic add/remove handlers and guest BAR handler implementation - fix guest BAR read of the high part of a 64bit BAR (Roger) - add error handling to vpci_assign_device - s/dom%pd/%pd - blank line before return Since v2: - remove unneeded ifdefs for CONFIG_HAS_VPCI_GUEST_SUPPORT as more code has been eliminated from being built on x86 Since v1: - constify struct pci_dev where possible - do not open code is_system_domain() - simplify some code3. simplify - use gdprintk + error code instead of gprintk - gate vpci_bar_{add|remove}_handlers with CONFIG_HAS_VPCI_GUEST_SUPPORT, so these do not get compiled for x86 - removed unneeded is_system_domain check - re-work guest read/write to be much simpler and do more work on write than read which is expected to be called more frequently - removed one too obvious comment --- xen/drivers/vpci/header.c | 135 +++++++++++++++++++++++++++++++++----- xen/include/xen/vpci.h | 3 + 2 files changed, 122 insertions(+), 16 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index e6a1d58c42..43216429d9 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -477,6 +477,75 @@ static void cf_check bar_write( pci_conf_write32(pdev->sbdf, reg, val); } +static void cf_check guest_mem_bar_write(const struct pci_dev *pdev, + unsigned int reg, uint32_t val, + void *data) +{ + struct vpci_bar *bar = data; + bool hi = false; + uint64_t guest_addr; + + if ( bar->type == VPCI_BAR_MEM64_HI ) + { + ASSERT(reg > PCI_BASE_ADDRESS_0); + bar--; + hi = true; + } + else + { + val &= PCI_BASE_ADDRESS_MEM_MASK; + } + + guest_addr = bar->guest_addr; + guest_addr &= ~(0xffffffffULL << (hi ? 32 : 0)); + guest_addr |= (uint64_t)val << (hi ? 32 : 0); + + /* Allow guest to size BAR correctly */ + guest_addr &= ~(bar->size - 1); + + /* + * Xen only cares whether the BAR is mapped into the p2m, so allow BAR + * writes as long as the BAR is not mapped into the p2m. + */ + if ( bar->enabled ) + { + /* If the value written is the current one avoid printing a warning. */ + if ( guest_addr != bar->guest_addr ) + gprintk(XENLOG_WARNING, + "%pp: ignored guest BAR %zu write while mapped\n", + &pdev->sbdf, bar - pdev->vpci->header.bars + hi); + return; + } + bar->guest_addr = guest_addr; +} + +static uint32_t cf_check guest_mem_bar_read(const struct pci_dev *pdev, + unsigned int reg, void *data) +{ + const struct vpci_bar *bar = data; + uint32_t reg_val; + + if ( bar->type == VPCI_BAR_MEM64_HI ) + { + ASSERT(reg > PCI_BASE_ADDRESS_0); + bar--; + return bar->guest_addr >> 32; + } + + reg_val = bar->guest_addr; + reg_val |= bar->type == VPCI_BAR_MEM32 ? PCI_BASE_ADDRESS_MEM_TYPE_32 : + PCI_BASE_ADDRESS_MEM_TYPE_64; + reg_val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0; + + return reg_val; +} + +static uint32_t cf_check empty_bar_read(const struct pci_dev *pdev, + unsigned int reg, void *data) +{ + return 0; +} + static void cf_check rom_write( const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data) { @@ -537,6 +606,7 @@ static int cf_check init_bars(struct pci_dev *pdev) struct vpci_header *header = &pdev->vpci->header; struct vpci_bar *bars = header->bars; int rc; + bool is_hwdom = is_hardware_domain(pdev->domain); ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); @@ -578,8 +648,11 @@ static int cf_check init_bars(struct pci_dev *pdev) if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO ) { bars[i].type = VPCI_BAR_MEM64_HI; - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, - 4, &bars[i]); + rc = vpci_add_register(pdev->vpci, + is_hwdom ? vpci_hw_read32 : + guest_mem_bar_read, + is_hwdom ? bar_write : guest_mem_bar_write, + reg, 4, &bars[i]); if ( rc ) goto fail; @@ -590,6 +663,14 @@ static int cf_check init_bars(struct pci_dev *pdev) if ( (val & PCI_BASE_ADDRESS_SPACE) == PCI_BASE_ADDRESS_SPACE_IO ) { bars[i].type = VPCI_BAR_IO; + if ( !IS_ENABLED(CONFIG_X86) && !is_hwdom ) + { + rc = vpci_add_register(pdev->vpci, empty_bar_read, NULL, + reg, 4, NULL); + if ( rc ) + goto fail; + } + continue; } if ( (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == @@ -606,6 +687,15 @@ static int cf_check init_bars(struct pci_dev *pdev) if ( size == 0 ) { bars[i].type = VPCI_BAR_EMPTY; + + if ( !is_hwdom ) + { + rc = vpci_add_register(pdev->vpci, empty_bar_read, NULL, + reg, 4, NULL); + if ( rc ) + goto fail; + } + continue; } @@ -613,28 +703,41 @@ static int cf_check init_bars(struct pci_dev *pdev) bars[i].size = size; bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH; - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4, - &bars[i]); + rc = vpci_add_register(pdev->vpci, + is_hwdom ? vpci_hw_read32 : guest_mem_bar_read, + is_hwdom ? bar_write : guest_mem_bar_write, + reg, 4, &bars[i]); if ( rc ) goto fail; } - /* Check expansion ROM. */ - rc = pci_size_mem_bar(pdev->sbdf, rom_reg, &addr, &size, PCI_BAR_ROM); - if ( rc > 0 && size ) + if ( is_hwdom ) { - struct vpci_bar *rom = &header->bars[num_bars]; + rc = pci_size_mem_bar(pdev->sbdf, rom_reg, &addr, &size, PCI_BAR_ROM); + if ( rc > 0 && size ) + { + struct vpci_bar *rom = &header->bars[num_bars]; - rom->type = VPCI_BAR_ROM; - rom->size = size; - rom->addr = addr; - header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) & - PCI_ROM_ADDRESS_ENABLE; + rom->type = VPCI_BAR_ROM; + rom->size = size; + rom->addr = addr; + header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) & + PCI_ROM_ADDRESS_ENABLE; - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, rom_reg, - 4, rom); + rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, + rom_reg, 4, rom); + if ( rc ) + rom->type = VPCI_BAR_EMPTY; + } + } + else + { + /* TODO: Check expansion ROM, we do not handle ROM for guests for now */ + header->bars[num_bars].type = VPCI_BAR_EMPTY; + rc = vpci_add_register(pdev->vpci, empty_bar_read, NULL, + rom_reg, 4, NULL); if ( rc ) - rom->type = VPCI_BAR_EMPTY; + goto fail; } return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0; diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 75cfb532ee..2028f2151f 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -67,7 +67,10 @@ struct vpci { struct vpci_header { /* Information about the PCI BARs of this device. */ struct vpci_bar { + /* Physical (host) address. */ uint64_t addr; + /* Guest address. */ + uint64_t guest_addr; uint64_t size; enum { VPCI_BAR_EMPTY, From patchwork Sat Dec 2 01:27:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4E3B6C10F05 for ; Sat, 2 Dec 2023 01:27:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645838.1008324 (Exim 4.92) (envelope-from ) id 1r9En1-0004V0-Fi; Sat, 02 Dec 2023 01:27:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645838.1008324; Sat, 02 Dec 2023 01:27:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En1-0004Pb-2N; Sat, 02 Dec 2023 01:27:27 +0000 Received: by outflank-mailman (input) for mailman id 645838; Sat, 02 Dec 2023 01:27:25 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emz-000272-JR for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:25 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f1b1c5c7-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:24 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWa004068; Sat, 2 Dec 2023 01:27:13 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:13 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:08 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:07 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f1b1c5c7-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WsTiag/Nyo8pa7Cp19bBLPah6EcvNwpz9qqJWOKb6wSn6n/lTDKOzxee37RRRXIoYVLuxCTz4HEYUmA8sL8QzfANiGU6hkgwB+wtKpQh15KL672PUnTIqoJwM7aiO1qnvBFdWFeHUHu8myHqV7GZgqjjHpcuDVAp/YhOY3OOnNI5gtiFxQVwJI86VRhuvFK4ivdl3z/64ntNRhPQ9HbUww+qirnuTwUSMa4aeYkhN2EvQ6snpeJbWVZy1HoC2tcxVHma+iMvxwtcXKnBz5bZdeOsErsS7P34C1sSgjB9Kc4wP9WH5yWpe2LRfzTR0MSesE/5Bj6nHwoe93rq3oJrxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cjuf4AqZ+uhh774QaAreURssqr1arEGYp9N8KZKWT+c=; b=N/0vUNF85upo7GnfXXQPL07ngpb9gzQPeLlv+SYOXV3l4RQZAGfUPly/ldYqM4McwCBsp7ZgxFFr+fcArYrQMzYaVQ2ZDWkoOO1BQ02HjJC0VaGrNexf5Z1JBk5ujjqjqMODvUCu/xdzZIqA2QKkP+jaVfJNQGc5euqltpXqw0PGL4lbUCyCqJg3koJ+Djn3abcCg0bFzyw6sH+TQqDwY5Pe8Of1D6oZz9mXcMcGuxB0l1ad9KyiAWoJ3SsqXzi0RFrbNmyOAnGP3UJCRSDCA2TiAI2inZ53LsbYAIS1nJeTL3eT9IFM1V8MfZHwAcvLTwZywnSuIK5HBpAaOun6Xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cjuf4AqZ+uhh774QaAreURssqr1arEGYp9N8KZKWT+c=; b=pvDY/UgvjFb130NHJBqlyDf818MizzZBGe4VaNObNhzWFw0Fbs0EWkJyRKbL1srBPOyT+sbXvaLVvhnxwNXuxMI8DOVo2g1eZJ9RixdKM71SqAl0BpXfQn3tnUpKQFwcIMS8tn1ygYwj43EA6+LNisvbuK1rgCXbHeNHjzggu43h8pwejkTKGLnkOixJGtRXTq1quTwJaFnTv+gW71SN/gdeaq8dYJQRdlLmVTyY92fGVAaah/yLHJcfaNbihqp6SbAUnaaNic3CeM5dcamPBwdOsr7P7mouqisq/h7oR6AsGfPBZnDH9YZDYo9SOmR53ntn4SNVIjgUxx8PB2xqzQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Volodymyr Babchuk Subject: [PATCH v11 08/17] rangeset: add RANGESETF_no_print flag Thread-Topic: [PATCH v11 08/17] rangeset: add RANGESETF_no_print flag Thread-Index: AQHaJL6oAOK+50zr5Ea10ElGDBGzrQ== Date: Sat, 2 Dec 2023 01:27:04 +0000 Message-ID: <20231202012556.2012281-9-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: d63231e4-0d73-4807-e57d-08dbf2d5ccb0 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: bfujD/gXSTsGuZxF+1V9JRKOHyLwRq8rQjOHQQ180quws1TZu/x0opCGLWdmu/0quYEaS3svecZwyufC0uZQna5lpjmUApQukhzmjTzJniAbK/ssE6ahCMZlxA1/i3vtYRuFip1MvU5Z2GDQ7+atk0sh3883ZDTSGjFnRmbGdisLoMX+bLNz+f8+CJfk+WaWjHeiUKKsV1jOfOfEOtlp7XCzzXcQZ1XbeJlBKj7SsGRu2eCpDKB0ozwa+Mw33KRKRJ1Nk/odRTUgGMC7u5fhhHL24J6ki3yEqRnxknxV5i2z4NXA6rh9mxXU61YnxSn9MHcDZ/qtzZ2JbnhBK3fp+c9XJk1kcJUsxjonS73zMuGzRknsQUCmUn6W20HPA2k/yVu33LIoTbSMSUdQlSqHfSeOSiwTxVkuxkTSJje7i7WtjNBcnvjuBy/6DTw3ppvCnkHsiz3jW7ztvQjbtDDzDHezbQt3VwnfOzmqMZulBTClQ7yVFw5+7Jl1TiNFZP8Lvw5Kv3qDEv123ecQt9VkjnrnOhT2G2ZwadnS+8nG8HHx8LXGySoEqo0L/JRQiUyqml9ICY1hqu8eRC/nT0VUqyRnOmz2l5dknbPTeoKrJeztvOaVufeQsIoAMIEw38Vd x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?VaCPbk407zia+di7zjrgr8/?= =?iso-8859-1?q?yQ7gcnxIMTlnigGXqh+dQClnR8ON/RGzYDijkzaM/Jevlw17Ek+19IRkJAOW?= =?iso-8859-1?q?fXiLoBM38NTpIw2QkQX5Sr/10c7u3ayeazIutHENRI9p2kqSIADqJniYYr30?= =?iso-8859-1?q?nE4sGOSpzlbpZFkVbaayytuNPLvOQTdZ34097aFpdMdYxGoRGW7ZyJTI2DC2?= =?iso-8859-1?q?uOU+uX+x3kl9VSAPpmxYkbs3JJayYtMgrT7Ua57GI6m+Qjr785P9RPHiTRbZ?= =?iso-8859-1?q?nuP+2dD+GPJKaBvc1ehCHGY9IZq93kx2UPhPkQC1H8L16QEQE8fhOXSohqwL?= =?iso-8859-1?q?596W6+1W3kn0Wx04JIbitkO/wO5POAbIS7gNahhbhZW/jmolNAHoHoSN+XO5?= =?iso-8859-1?q?jkkYnPOkUKHRSuMMePBPQeJYT88TNY3bDtuZXTdq+ZYdebYDHtMT4fLts54t?= =?iso-8859-1?q?ZV0mWzqxTIuYRzAKImEcoJfbnaiX0WammfBZylgfloFwCNpXOgZaBDyMupdM?= =?iso-8859-1?q?59dZ/Gskc7APnjgsSMRzRMYK2Zavh2lcaJEHUiKI9lqYIhmkoZv/R9V5hjxb?= =?iso-8859-1?q?yAGUSsRAOSPrbz0xOJgd7VMXAyl0I2+9KzT/58ooLpRLlKz9mxolzb0coHpN?= =?iso-8859-1?q?ehyq/ESFltYVNp/J6Cxez+Rkvu6aw4/cTLegyNyu0wDYeOWKSDLitKHdCfbe?= =?iso-8859-1?q?/3RLSL2PBW1WsS+2pGZ7CFgOczDtUOGjujXSNyqUxZOqbW5evPQ3BRZFR8w0?= =?iso-8859-1?q?vtUuX4zD/pv2rkbHT7kXKXMo+NmjyFB9Ts1Sy98rQOHsXinJXUhcbhKk19s3?= =?iso-8859-1?q?W97ovbCPYdqTblt1uq6Vocn1VGhOFFr/0rQsN8dKT1UYAmb88Zc/Ai4hYQKA?= =?iso-8859-1?q?EEfOnGITsiiHTh3VX68hqClPyqbjk+IdX0MD7B8ppYHVwtCrg7pDnYKV2g6O?= =?iso-8859-1?q?YowQvh4UF/FXoYG/skJn8Lw7nI2uaKSoxz5cmVxp5b56A7C2ohe0r0x9fOwq?= =?iso-8859-1?q?Y8WdAAfhLuVV6D3YrudgwBq7y/a3RALuvAcuBVBCV17X+GdN8TvIt7AeaTG0?= =?iso-8859-1?q?Ba2k5XIPgelng8MdE1WodU7Rh/WSZ4XO3iFiThz874gjJqddAi8tPxL+ShFF?= =?iso-8859-1?q?+huqfe2vJP8eLEjCcy3HI/XwMnvtQR3zLimaRgHqkktLoCyr791uTJovxSlc?= =?iso-8859-1?q?EL2oR7R8oBAsREi8uvYZW6HIbUJYhi9oOnGHPcL5jAEJl56AE9OveKBJswLf?= =?iso-8859-1?q?c4/wiSRJxK92whpnv+kliJF52kDk3xFavHl2LGqBoCrC35krwzMP1thVIOHC?= =?iso-8859-1?q?DvakUgos4R16K2+KqDf4vjW1Qs5y+ydtsP9sjbKzhIO8cVj190lhJ2a6ODum?= =?iso-8859-1?q?X7aL7mxBI6RCytDf016oTPFBlZExLfCn9o69C0NxMA0daRLNKkdGzLYKrINm?= =?iso-8859-1?q?21/D4qCr+QmTqF21CJIrhDdjZwqMgnr9fIjrtKV183DmGMhlewLGRSAua1Ht?= =?iso-8859-1?q?HuI0FHtLB2IQIZjGeEXua9cSMyyeaASDW+H3JpAQ3oAqXOFFXWXZLZqTQVEz?= =?iso-8859-1?q?NovTWFJSjBTKCu60RkkgT3aW06Qwia6xFoGowld+p59wtnwv4xdiIjxCz+uI?= =?iso-8859-1?q?RFIZtD56LOXLiH5T3ZSbD8Kwn5dFepBSwfWLtZg=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d63231e4-0d73-4807-e57d-08dbf2d5ccb0 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:04.7658 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: qZefivLQ10rNr+I3opfWEfd/HHOvkfZVL6SBoP1WbOaOJBXBGDaaE9PgjNi9j8Ih3GFhH1fD4fzyEkm6MS3A2o+iHmMgw0vLysiUyCE7+wg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: M-rz3I0bjEPDnKG-omNko7Zd3fxZyvrC X-Proofpoint-GUID: M-rz3I0bjEPDnKG-omNko7Zd3fxZyvrC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko There are range sets which should not be printed, so introduce a flag which allows marking those as such. Implement relevant logic to skip such entries while printing. While at it also simplify the definition of the flags by directly defining those without helpers. Suggested-by: Jan Beulich Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk Reviewed-by: Jan Beulich --- Since v5: - comment indentation (Jan) Since v1: - update BUG_ON with new flag - simplify the definition of the flags --- xen/common/rangeset.c | 5 ++++- xen/include/xen/rangeset.h | 5 +++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/xen/common/rangeset.c b/xen/common/rangeset.c index 16a4c3b842..0ccd53caac 100644 --- a/xen/common/rangeset.c +++ b/xen/common/rangeset.c @@ -433,7 +433,7 @@ struct rangeset *rangeset_new( INIT_LIST_HEAD(&r->range_list); r->nr_ranges = -1; - BUG_ON(flags & ~RANGESETF_prettyprint_hex); + BUG_ON(flags & ~(RANGESETF_prettyprint_hex | RANGESETF_no_print)); r->flags = flags; safe_strcpy(r->name, name ?: "(no name)"); @@ -575,6 +575,9 @@ void rangeset_domain_printk( list_for_each_entry ( r, &d->rangesets, rangeset_list ) { + if ( r->flags & RANGESETF_no_print ) + continue; + printk(" "); rangeset_printk(r); printk("\n"); diff --git a/xen/include/xen/rangeset.h b/xen/include/xen/rangeset.h index 8be0722787..87bd956962 100644 --- a/xen/include/xen/rangeset.h +++ b/xen/include/xen/rangeset.h @@ -49,8 +49,9 @@ void rangeset_limit( /* Flags for passing to rangeset_new(). */ /* Pretty-print range limits in hexadecimal. */ -#define _RANGESETF_prettyprint_hex 0 -#define RANGESETF_prettyprint_hex (1U << _RANGESETF_prettyprint_hex) +#define RANGESETF_prettyprint_hex (1U << 0) + /* Do not print entries marked with this flag. */ +#define RANGESETF_no_print (1U << 1) bool __must_check rangeset_is_empty( const struct rangeset *r); From patchwork Sat Dec 2 01:27:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2C96C07E97 for ; Sat, 2 Dec 2023 01:27:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645837.1008316 (Exim 4.92) (envelope-from ) id 1r9En0-0004CX-5T; Sat, 02 Dec 2023 01:27:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645837.1008316; Sat, 02 Dec 2023 01:27:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emz-0004Az-Sx; Sat, 02 Dec 2023 01:27:25 +0000 Received: by outflank-mailman (input) for mailman id 645837; Sat, 02 Dec 2023 01:27:24 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emy-000272-JS for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:24 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f1555a11-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:23 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGv1008549; Sat, 2 Dec 2023 01:27:13 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:13 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:08 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:08 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f1555a11-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UznBWdFVrRl3jnC+ikULpAj8Vkr8LmSDh/3KmsehG7mQRgp4iv36SqjfMMQfadtw3m1sdsZk03o84JHjJKRIgVgZN7NCnIDSzsoNTtcvG9AAYz+6NzXnMxEG8oJw2ggYxKy//NA5wpDunPXwFt6vFEK6tUw8D/TJ/vAEUJrZ1GbgYOh+r6G4Fx5j/1Ylq8FwxQ9pLV6jH04vQl8Rl7SMlVkll20rVRhU3+pZGHEBKbJyMVAIFAds/d6/pbHy2u5i25qn5H4YVfAWCaENWeh6MHJt8oanlkcvSY9uLxCqGTnYp2zxY4GKa0wYdswEGzXwG7yyNq7YC0eaTrvsjih8hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IwB9N34h98HO0xQgLgRJxTUI/bKMCMF1l0/De+vjKko=; b=DF4Ln2xBkwLagblpkJzmUoDoLr/QftQXNYhmbIwY0E4zNk6hwZgv6KP+CLmMTVWygbKPdNPj6cP5vOvKJvdGgpc4Rza1FAqn16zPCZNH+MP6xY9QduKIFVe4lVGidPXtqwuzM3eXMj/2bhTeQ27XbVM2oIS3HNVChx07Hn9YQ9YpI5ZEDtHA/rrGEaFmvCf2/8YoLSHWYNFZRGjwh1XcdKocUiwnU5z9dKsvfCNbCHFxdIqPhztI/vY/gna6HKbIcCRxfqeszSCkhQEyWBO3s2XwmcbOue6ZmFH6zO4e07i/GvEFB413QYJv+xmNrv6b9bZ7+oo7n/8QlGQUfpkNQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IwB9N34h98HO0xQgLgRJxTUI/bKMCMF1l0/De+vjKko=; b=J98vTK6i3gAk9MzMQDm/TvOTrl5PG5bAqPAbeVzwGQzBdsKbeJ74kOXH1dxtUtaFAlJrAAIgIinbq4XB+ww1CvH05ogiSDa3F3vprysYuOMsaoMRmDQ7f2mfetfsfdQIiHDPHJgG2BKjpo0rbh7li+pXc2dfy983BVtcUqLMTJTpOsiTQqrIoAtc/FIVlv2mBEEpBbeCgGc0JYHP811E1FNgkNik5knHC2tuFJWC+msLMLU6kopvo17XJizrxRNEztlqqmIFG0R+ctrZiMB8fEy0+v09Acg6jyhX3InzQwPtZqoEb7VK4+V1wIAfyY7F7Vx6Sd3HYjPMDuYhp1jOZQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v11 09/17] rangeset: add rangeset_empty() function Thread-Topic: [PATCH v11 09/17] rangeset: add rangeset_empty() function Thread-Index: AQHaJL6oz4wJTetDeUydVFKkkhEQHg== Date: Sat, 2 Dec 2023 01:27:04 +0000 Message-ID: <20231202012556.2012281-10-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: aadcf4ec-9550-43e7-ab63-08dbf2d5cceb x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: nhlQ1WsmfK9OsXn13NI4hVSXNbVK25Vm2Tpy1/MRS4IsGSMe4BE8pVywbvj0zFWOMpjE7unxHsAj7xn6gWQn47+UREjxdWPnsIZwqVUfiv5TxT1uDhVlV3uaNzHbEvJFz1NCNxVUmlAJ/jJpXYP9XVBenoh7HsyKPJGm3pxSH2UhK4nfynDq45E5ja7VbIfu4z6P+YM0z0bLoJI8hzN16bRt816i3GnyTo66NjT6ADWLIero9pZBbzb5q/RlsW0LybaAO8Nk9qE60nAdDw4dVOhxx70CmYCC0AU5fyXVo8jojrOzVqIy85jFo9vBD/Z4xn4ZmfRhyzQVSimsDiaP6EYi+F/Xz4DPBEt1bK7TFYmnnLrY2GN4oJ/PYxMzZYlmNYBvE6VNLKtUhPhFxYPG5R2Vys9+V1/NKHdTh4g/vKREfXwRTbvNwSEMVbK0I46knYE5kwIt/9PrkZr0VkY9u7v0I8ufAhPBjUDw+NTDJAzQS7MCrJQ85S2OjQRyzPhjteVWYap+cECqu8fpC96bf5ExWVpnaoV7z2T+Z61Ni1p3wSqmlmPWi0uu8vc/9iZaeBFsVysx3FSWZrHHt2Xm0fMrSOSZPaDL5D2zh9pwn8yZXazUrFs9yOuOV24zjeME x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?WLvcgjsvBHsMqky0zQwQcM4?= =?iso-8859-1?q?XeuRjS0Qw095cQpfIaQUWwMAC028rmCOL2PJA1O2zalPEpt4h+dtUMkiMy8B?= =?iso-8859-1?q?Haxo2t5Xie2uKqk9SDQAVvpa4hvXUkfSbvbTOPc5hwvfg5DV29/MbumzOZWL?= =?iso-8859-1?q?WrLCDuxoKdDPfsMUPgyCe9ZZ/DMZNy+MV3LxiUv+FKzJr5zCd2Cq4pD8PnAT?= =?iso-8859-1?q?wYHrFAcxg5TPfT3WLVnsulkquqxXFghcsUmRqvS/r9OBAKWKm2xf4yZh4djK?= =?iso-8859-1?q?teP9nXxFXRR8srTrs3gngVAwI2eppTpyzB/wqqBI6oLqtPW2ucDr83UxtU+q?= =?iso-8859-1?q?8Tmb5Pa0MxoA8CUwqrMZ11kn952czd9Cs8AA/P2XIy/ab2pqQ/39C3g0bZxa?= =?iso-8859-1?q?EZzb6MQH2seVwzNvq/NaXjbKn70caFR4/2hiHd4vJKczP+7Z9QT86L5Q+tM1?= =?iso-8859-1?q?oDBDzqg/nrAybSEJLpmtsek9EMOVlQaIv+ucPfkUSGK5VWeLOi/rcI//ZL3W?= =?iso-8859-1?q?PpCL5UNETWEkjQ2Q6nZdnC0LbUvkzd8DD5xLWjaGV9A1slttx3L+hHweu8sX?= =?iso-8859-1?q?40tRjaIpxgeDqgc58nU1ZbvPLL0M9V4ev/sp86WGcBzjFIzAc7SWLomUoRhp?= =?iso-8859-1?q?BaN2EWYmy+Si4iaEW8aE1UnaKiVGBQOYQdL7SUU1GTct52gy7r9uTU30giRX?= =?iso-8859-1?q?r9M9bX3nSVnHG0fzhyq/MsUj0k7llUpfLic4chwzInHhQLYwr8R4tIzbZvmI?= =?iso-8859-1?q?9YmnEq2Un03TX2dLRpXHZvhLYMMkE9byahXNcl5zPgG6ovjAJxqb8v/6+HOT?= =?iso-8859-1?q?93LKg9q2g3qsSTDrmEtXD552hyH23lUZvnkw73QuS0ps/mairbP+NWYm5ojc?= =?iso-8859-1?q?2ICcTAnIJBRdYlf8OnwmBtxwEmiod5seVupHZHJy/SUx6m6buQ7vDNBvRmI+?= =?iso-8859-1?q?onEoc2qs1QUzntA1Bf3rDlLxh5hgxTBcPtMXyGweieCqZYhkJO000Q0lnlld?= =?iso-8859-1?q?PsH1uzbZWIvLxX2DGuU66S8lq9f+op3/HKMkC/4fMfYjlgPZurkmh2ctVlJT?= =?iso-8859-1?q?Ra23MqGHyapXeL5XjVN9cxJO+I/GkUh/sgafXusBdpUGfjhZcbN7q78wDcsc?= =?iso-8859-1?q?uBBTefuPyuPzlIpWAfQHg2ujOOVRKe4vFY7L8jD1ImjZXc93bGdWleE5yNzG?= =?iso-8859-1?q?IB2X8PcpPeknsnbUdVGoykCgSkd5ceM7bxUJqM2FOReh3ykQCsrJkmwcf7C4?= =?iso-8859-1?q?GlivH/Q+6JvfhFwcBbr4NQkllMxzljEWgGJD2Amc5MIlKDcoGCRe/0vn1uf7?= =?iso-8859-1?q?c5jKl1YCMW+pAtExzTWEgtU99Dhg1ktcJ46A/CVCdh9TJXKSDNHlKxEQDS7q?= =?iso-8859-1?q?cy4XRDv1NVznTAPjRZMeB/Uk16/KDTXxjQPEqWLLG3bK5F4n/k30GSm99dgC?= =?iso-8859-1?q?NHOx+UuypfnCcjH9MyrqDfvEvzGq7Lwo/qXMRYxNtEVJddvFoMJ/nRhH9YaO?= =?iso-8859-1?q?zyCuK2hRs/aiL3x/rQRrVa/ipilh3MKOtYOoZIgShJ+22rUYumC6J6xGOvKB?= =?iso-8859-1?q?ELHKPvm/IJ+bW2nnNNOvU0cF0zHQgaprUKIM53yqSMgadOWajyr+2yxH9qLp?= =?iso-8859-1?q?I76YLUJuAHbi3kpFBoqKJzjv1yI/dFMjqUAXUpQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: aadcf4ec-9550-43e7-ab63-08dbf2d5cceb X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:05.0030 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jUN888k2aiTDk+PrJARDtnLVfiwHu932wFOQ4QlOF/+aaKfJfDJFV1NxyME7ZQRi+nqt33fNx95kfE7Xj09WGeDTdteWAjEki1uxXStkk5U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: EhZ-FRdFIeSCDyhSlkw-nvwWhrmdKWxX X-Proofpoint-ORIG-GUID: EhZ-FRdFIeSCDyhSlkw-nvwWhrmdKWxX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=706 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 This function can be used when user wants to remove all rangeset entries but do not want to destroy rangeset itself. Signed-off-by: Volodymyr Babchuk --- Changes in v11: - Now the function only empties rangeset, without removing it from domain's list Changes in v10: - New in v10. The function is used in "vpci/header: handle p2m range sets per BAR" --- xen/common/rangeset.c | 16 ++++++++++++---- xen/include/xen/rangeset.h | 3 ++- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/xen/common/rangeset.c b/xen/common/rangeset.c index 0ccd53caac..d0c525cb50 100644 --- a/xen/common/rangeset.c +++ b/xen/common/rangeset.c @@ -448,11 +448,20 @@ struct rangeset *rangeset_new( return r; } -void rangeset_destroy( - struct rangeset *r) +void rangeset_empty(struct rangeset *r) { struct range *x; + if ( r == NULL ) + return; + + while ( (x = first_range(r)) != NULL ) + destroy_range(r, x); +} + +void rangeset_destroy( + struct rangeset *r) +{ if ( r == NULL ) return; @@ -463,8 +472,7 @@ void rangeset_destroy( spin_unlock(&r->domain->rangesets_lock); } - while ( (x = first_range(r)) != NULL ) - destroy_range(r, x); + rangeset_empty(r); xfree(r); } diff --git a/xen/include/xen/rangeset.h b/xen/include/xen/rangeset.h index 87bd956962..62cb67b49b 100644 --- a/xen/include/xen/rangeset.h +++ b/xen/include/xen/rangeset.h @@ -56,7 +56,7 @@ void rangeset_limit( bool __must_check rangeset_is_empty( const struct rangeset *r); -/* Add/claim/remove/query a numeric range. */ +/* Add/claim/remove/query/empty a numeric range. */ int __must_check rangeset_add_range( struct rangeset *r, unsigned long s, unsigned long e); int __must_check rangeset_claim_range(struct rangeset *r, unsigned long size, @@ -70,6 +70,7 @@ bool __must_check rangeset_overlaps_range( int rangeset_report_ranges( struct rangeset *r, unsigned long s, unsigned long e, int (*cb)(unsigned long s, unsigned long e, void *data), void *ctxt); +void rangeset_empty(struct rangeset *r); /* * Note that the consume function can return an error value apart from From patchwork Sat Dec 2 01:27:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81A67C4167B for ; Sat, 2 Dec 2023 01:27:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645833.1008270 (Exim 4.92) (envelope-from ) id 1r9Emw-0002wT-0H; Sat, 02 Dec 2023 01:27:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645833.1008270; Sat, 02 Dec 2023 01:27:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emv-0002uL-Ri; Sat, 02 Dec 2023 01:27:21 +0000 Received: by outflank-mailman (input) for mailman id 645833; Sat, 02 Dec 2023 01:27:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emt-000271-O8 for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:19 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ed06d412-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:16 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWb004068; Sat, 2 Dec 2023 01:27:14 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:14 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:09 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:08 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ed06d412-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LmNB+vq+TOgQkcGi81io7BXU1Y49/WmBS+YUC0NuOXDV6Yzk2lNrVuOoPfZZ4AY2g4WPSiNx8GI1/l/86DLzzHkKTKXW1nrpWF1EYgu+mZ5VGyzBRc8Pjm+kaF+67k/NqvVIpKXa8tMFedEBq1XEQlGzsJ+4b922DlPDLt639pC04KvR7b5BXvQ/8Y/fYJet7AKSo16tSGMYvnD5uC+DGy3g5OLl5f3w0f8F707PVv/Hs+wIYovUyodQ+5W1St2Fzn+22qAYlAG+VwDkn5paApHlbd0wheZ8cnG7LYirglGLVFrJT71BMIT12iURasru0Zfl7/rPnOlEZtqfDLegVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HU3w/MVRT7OnffWTfx0/oqZxB5j1V8feoRNAk0O5Ezo=; b=jq8IEYCGGOrhw2u32VsxwnUSe9zGje8Hg+hE4V7vAs+T49Sbxgtjz5v4szGRUPlS/44u/94qtGgI2YmaWq8TlNgRcACThbsq/gtTkeQKYP6lgLTnD2w9Tyzebay+Q7tYhAmA9Ch05zo4a1mkBhkcupmMSXIAyFrIsmxvYieLiSiDCKDw/5oW7P2QeAdHd3czatRjFXaoiFGAWFPZKgouesF5VHVMGlBbhsXH8fEw7OEXaWnMwltahJzoXutgs0DhSFZcUCi9u12gXYVR6FcI6twfx5wzfpcqlLYw3+yAeeZvdGemF9Yvmvo+9/O0trLgIHXHgytTJG284i+dOC7fiA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HU3w/MVRT7OnffWTfx0/oqZxB5j1V8feoRNAk0O5Ezo=; b=LHdYUO2H4ADVu2K5xxb9f9z95unZ3poZ9Y4mndVEwMJNmuhf6dlyUWNcilrV2R6kB3DLUj11xictAhe99x0/a3z722GdVQ2oLEgi08bTp+QL84bN0Al2+yVqnMEmMBpnTIpar4eAYSoPttm+FrkwPDAOpV7nhrX8ipd17TqRXMp4fCz1ExgklBYIBI6zXvcST0YXSyL5t2tV2HD5OqgDZJ0TPqvdWgUAvx/k+nN/rPOXJbbtw2fricXkhJnTaS3ay9SPxYl/80cILpPE+nG1M7exs5cKL7jXc5Q9kTaGhgcG7MjbShTz+p2ovMfD2hJciS+5f+i6cWytuAyNULnhVA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , =?utf-8?q?Roger?= =?utf-8?q?_Pau_Monn=C3=A9?= , Volodymyr Babchuk Subject: [PATCH v11 10/17] vpci/header: handle p2m range sets per BAR Thread-Topic: [PATCH v11 10/17] vpci/header: handle p2m range sets per BAR Thread-Index: AQHaJL6ovuONmUhRl0Ozo/OYp0VBgw== Date: Sat, 2 Dec 2023 01:27:05 +0000 Message-ID: <20231202012556.2012281-11-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 8c0d6dd1-22ac-4fdf-8f1a-08dbf2d5cd2c x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ACc05rBYttSwdH0bhs5E9hNr/W3ulRZmqIMNWEWDv0GCUayoROtm5kKes8ORIyHI/3pdc33pdRKzrXEeBAbHZBOaBE3A4qxG+tadd1++YLaKn7pyPlq1DBC0l7UefgMD3lmb0NtI6qF2M5iJoKHZHdnP0r/BdNiAAYtR1kEnlP08R8jTdmmu5cyIvkiiw9kQyUfN20nbldXCnKEVIthybf+PfGFSP0DEV6dMxOhWxuIp1nfLOnpKyh3IroN6XrcxNxVMpIjlUFuO4BdaZKuSPwhGHi2J2DpwBJkdsjt40cj/CU4SJxN2ZgHPcFoHJTAxVA8+W2709iZCHzvfzR31rcVCZV/eVD3s/JDehTHUM3NgITLWJv+Fd6eV6RUZmiYnf+k2Pah38MbCypllkEhbLQYusy/NWKAIfphbOpdRapdtks8FLz5dOrZ9MGTQDWeUijCdncuuCV15RWyo+yBx+45/xE595MRNOvNDpXdqGWYgcVWlXezl2isx4fYww7PS/5mInujOTnHQqO/fNzM29BGm/lrCzwKXr2SfqUcPoAIyEUgWGBW1SIpjpj4hB/L9melAjtv28CZSfORoW9Eev5Nr0VfVOisl1idwI3V7Q9+cFNrm4yqjHXgAATL8aI1+JBQ9OGWmzT9zeQEheiTll2rr38WckUCIuoamW2Q0lZI= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(230273577357003)(230173577357003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(30864003)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?OPT9tZLGLIguhAs+mWeiFApyMMBQ?= =?utf-8?q?Bv65pcAprDEEjAPDicARR6WTUo+5xyIYdCLjVdhbhtAShxBYGjcPe5j2p6LsO1RmR?= =?utf-8?q?z2qGfdx6hi6+Kh/CYe/mPWx562REj+jX65Pst9IdT0JiCkW5QrDtFyheA2QLCgqao?= =?utf-8?q?y5c8VRvmqVA/qJ/cdnwFmkWZoNrthy3X4V95Ur3xJvBW6WA1dPnpwxzwA4RJ7AYqi?= =?utf-8?q?OmCK9vTmbxG4zmiNgBhC6/jy2VyqiUuNBWsEcGJ5HFUqsMuV1cmkw4X8KO3ycQLy0?= =?utf-8?q?qz6T9/MzFFpX1c5kNQdsMIoQ9BucpVFRHuVw8eB6hr6ynB06wd+KnDDdDK6ujopwS?= =?utf-8?q?UoaqPogGQ1XArzv2cDXcX2wW+k+Oo5RGu8H8m0uN0NHWA60s8T6FtIH4Yf9NkCfjL?= =?utf-8?q?obigeniPAjdx+68RqL5/J47dmGFCUQQdSXIzVqOtBalZuAnUPpZOMDR2WCPMwJAG5?= =?utf-8?q?WCOTu+PZWu6IyBeQukmtcmTuPNOkjFC20rQ6EO3EfKtFwFtmZOs1dNzXXYGoxm6qf?= =?utf-8?q?nT58fRdAzU5mxNS39fNWNptehurMo/uWXUwX/n5LrYLk88xTy5V/XbDhkK7Ywpxx3?= =?utf-8?q?oba77vRh+hzxbbJvpT9frO0nUdfNpZ5ofiHoi8Jf0ggGnrpkfb3aBy/14w9jAYr7/?= =?utf-8?q?JODqgOnvaEos4UZGZsOdLoddmbem8w5Ze7RSQcNlYm7hdPfw7+Y8m0CD5IAP3fMfO?= =?utf-8?q?Ik1sbH3kFqpjnQ1aOQG8nLkFi3Sjhoel2v5ryUDo+cMIX8xq9K1pcmfU97xu103/g?= =?utf-8?q?teFcGTmcPxVMFTs7elbOAOLJXSI+e7QDEM916B4t7lPHX2H1/MGdZoXoe0skT4mLe?= =?utf-8?q?jh3EZ61UW9tpxQMAhfY4u7Y/9akiWaZtyvja8cjMLcznlLF8t13k/s8N+P2a9W/Qd?= =?utf-8?q?QdhTYgBG/XnLqT3gTx/lfxUnKK2QmeCiwwkJ5gddSdMsqg9PR48BH0Hq7wbsEjrAu?= =?utf-8?q?Yn5qeTJEmyWEUnE7ctpC6lkFBHDU7bcJ/R3evSgDB/i+3DcUbJ5MMQsoCrYpJS8qa?= =?utf-8?q?OQZhoD0BIPeFhru/J6n+GL7jkz4Af9XKqLotgHQi5TA6dAztfjgDGPEIu+GFaql+2?= =?utf-8?q?rxr48/hajQ8SM1viol4+p/LFUX3/BZwRXkH4dXjPEsMvVk3D7VeR/pL35szH81fdp?= =?utf-8?q?d172OJCrj3aLnP8vrtrSAXSoyFOazcvefW0ZS/6kRA4dpztMhEEcystb3d+xZ0AgD?= =?utf-8?q?uPQYQ2mS8ob42g4VZBXJAMifwJUy0ms/x9w4gkZu1Rduhs3zzhvEAkPV08kKWr6Oc?= =?utf-8?q?aBwSEJ9nAg+O4NHINq5gL6jeQoikrwegKHBD09DGl6Z+9mDqP+yhWxPZ3zYI9eJSH?= =?utf-8?q?QkpFI95vQNHzMBtjG6qke2uDWavzHyyflU1IWxECN1OchCksUCiblUFJrDfNVWVWZ?= =?utf-8?q?dRbc0Jcl3k3AduC2J54vkkv6yp4kJZNfKIqqj1Xf0wX8fpx83ZgHBleh2XZMvJNl/?= =?utf-8?q?Or8OZBobyLJkQN2ykK/87tH/tjbz9lWUT0tT3zZ4RGI7a/338MJLzZoyVDDSmS+y6?= =?utf-8?q?cqjrXI+cZvruOgoMltOV0Fdecaow7jdgAA=3D=3D?= Content-ID: <17F77D55BA824E4184BB3E1AAC4D11DB@eurprd03.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8c0d6dd1-22ac-4fdf-8f1a-08dbf2d5cd2c X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:05.4098 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: qXvwjJSqm2E1ESW1zU1Ri18Umjfw4XchjKLd5GHJJwKzNP5oQujUKwrmV7jy+C2xZPRh2mzUdPEGukXbSXJQ7H2UxUQnLmoMheeHEm5di/o= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: 4Swmap5dXgCDj_U_mu-8DJ8dqBQi8fJV X-Proofpoint-GUID: 4Swmap5dXgCDj_U_mu-8DJ8dqBQi8fJV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko Instead of handling a single range set, that contains all the memory regions of all the BARs and ROM, have them per BAR. As the range sets are now created when a PCI device is added and destroyed when it is removed so make them named and accounted. Note that rangesets were chosen here despite there being only up to 3 separate ranges in each set (typically just 1). But rangeset per BAR was chosen for the ease of implementation and existing code re-usability. Also note that error handling of vpci_process_pending() is slightly modified, and that vPCI handlers are no longer removed if the creation of the mappings in vpci_process_pending() fails, as that's unlikely to lead to a functional device in any case. This is in preparation of making non-identity mappings in p2m for the MMIOs. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk Reviewed-by: Roger Pau Monné --- In v11: - Modified commit message to note changes in error handling in vpci_process_pending() - Removed redundant ASSERT() in defer_map. There is no reason to introduce it in this patch and there is no other patch where introducing that ASSERT() was appropriate. - Fixed formatting - vpci_process_pending() clears v->vpci.pdev if it failed checks at the beginning - Added Roger's R-B tag In v10: - Added additional checks to vpci_process_pending() - vpci_process_pending() now clears rangeset in case of failure - Fixed locks in vpci_process_pending() - Fixed coding style issues - Fixed error handling in init_bars In v9: - removed d->vpci.map_pending in favor of checking v->vpci.pdev != NULL - printk -> gprintk - renamed bar variable to fix shadowing - fixed bug with iterating on remote device's BARs - relaxed lock in vpci_process_pending - removed stale comment Since v6: - update according to the new locking scheme - remove odd fail label in modify_bars Since v5: - fix comments - move rangeset allocation to init_bars and only allocate for MAPPABLE BARs - check for overlap with the already setup BAR ranges Since v4: - use named range sets for BARs (Jan) - changes required by the new locking scheme - updated commit message (Jan) Since v3: - re-work vpci_cancel_pending accordingly to the per-BAR handling - s/num_mem_ranges/map_pending and s/uint8_t/bool - ASSERT(bar->mem) in modify_bars - create and destroy the rangesets on add/remove --- xen/drivers/vpci/header.c | 257 ++++++++++++++++++++++++++------------ xen/drivers/vpci/vpci.c | 6 + xen/include/xen/vpci.h | 2 +- 3 files changed, 185 insertions(+), 80 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 43216429d9..7c84cee5d1 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -161,63 +161,107 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd, bool vpci_process_pending(struct vcpu *v) { - if ( v->vpci.mem ) + struct pci_dev *pdev = v->vpci.pdev; + struct map_data data = { + .d = v->domain, + .map = v->vpci.cmd & PCI_COMMAND_MEMORY, + }; + struct vpci_header *header = NULL; + unsigned int i; + + if ( !pdev ) + return false; + + read_lock(&v->domain->pci_lock); + + if ( !pdev->vpci || (v->domain != pdev->domain) ) + { + v->vpci.pdev = NULL; + read_unlock(&v->domain->pci_lock); + return false; + } + + header = &pdev->vpci->header; + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { - struct map_data data = { - .d = v->domain, - .map = v->vpci.cmd & PCI_COMMAND_MEMORY, - }; - int rc = rangeset_consume_ranges(v->vpci.mem, map_range, &data); + struct vpci_bar *bar = &header->bars[i]; + int rc; + + if ( rangeset_is_empty(bar->mem) ) + continue; + + rc = rangeset_consume_ranges(bar->mem, map_range, &data); if ( rc == -ERESTART ) + { + read_unlock(&v->domain->pci_lock); return true; + } - write_lock(&v->domain->pci_lock); - spin_lock(&v->vpci.pdev->vpci->lock); - /* Disable memory decoding unconditionally on failure. */ - modify_decoding(v->vpci.pdev, - rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd, - !rc && v->vpci.rom_only); - spin_unlock(&v->vpci.pdev->vpci->lock); - - rangeset_destroy(v->vpci.mem); - v->vpci.mem = NULL; if ( rc ) - /* - * FIXME: in case of failure remove the device from the domain. - * Note that there might still be leftover mappings. While this is - * safe for Dom0, for DomUs the domain will likely need to be - * killed in order to avoid leaking stale p2m mappings on - * failure. - */ - vpci_deassign_device(v->vpci.pdev); - write_unlock(&v->domain->pci_lock); + { + spin_lock(&pdev->vpci->lock); + /* Disable memory decoding unconditionally on failure. */ + modify_decoding(pdev, v->vpci.cmd & ~PCI_COMMAND_MEMORY, + false); + spin_unlock(&pdev->vpci->lock); + + /* Clean all the rangesets */ + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) + if ( !rangeset_is_empty(header->bars[i].mem) ) + rangeset_empty(header->bars[i].mem); + + v->vpci.pdev = NULL; + + read_unlock(&v->domain->pci_lock); + + if ( !is_hardware_domain(v->domain) ) + domain_crash(v->domain); + + return false; + } } + v->vpci.pdev = NULL; + + spin_lock(&pdev->vpci->lock); + modify_decoding(pdev, v->vpci.cmd, v->vpci.rom_only); + spin_unlock(&pdev->vpci->lock); + + read_unlock(&v->domain->pci_lock); return false; } static int __init apply_map(struct domain *d, const struct pci_dev *pdev, - struct rangeset *mem, uint16_t cmd) + uint16_t cmd) { struct map_data data = { .d = d, .map = true }; - int rc; + struct vpci_header *header = &pdev->vpci->header; + int rc = 0; + unsigned int i; ASSERT(rw_is_write_locked(&d->pci_lock)); - while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART ) + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { - /* - * It's safe to drop and reacquire the lock in this context - * without risking pdev disappearing because devices cannot be - * removed until the initial domain has been started. - */ - write_unlock(&d->pci_lock); - process_pending_softirqs(); - write_lock(&d->pci_lock); - } + struct vpci_bar *bar = &header->bars[i]; + + if ( rangeset_is_empty(bar->mem) ) + continue; - rangeset_destroy(mem); + while ( (rc = rangeset_consume_ranges(bar->mem, map_range, + &data)) == -ERESTART ) + { + /* + * It's safe to drop and reacquire the lock in this context + * without risking pdev disappearing because devices cannot be + * removed until the initial domain has been started. + */ + write_unlock(&d->pci_lock); + process_pending_softirqs(); + write_lock(&d->pci_lock); + } + } if ( !rc ) modify_decoding(pdev, cmd, false); @@ -225,7 +269,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev, } static void defer_map(struct domain *d, struct pci_dev *pdev, - struct rangeset *mem, uint16_t cmd, bool rom_only) + uint16_t cmd, bool rom_only) { struct vcpu *curr = current; @@ -236,7 +280,6 @@ static void defer_map(struct domain *d, struct pci_dev *pdev, * started for the same device if the domain is not well-behaved. */ curr->vpci.pdev = pdev; - curr->vpci.mem = mem; curr->vpci.cmd = cmd; curr->vpci.rom_only = rom_only; /* @@ -250,33 +293,33 @@ static void defer_map(struct domain *d, struct pci_dev *pdev, static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) { struct vpci_header *header = &pdev->vpci->header; - struct rangeset *mem = rangeset_new(NULL, NULL, 0); struct pci_dev *tmp, *dev = NULL; const struct domain *d; const struct vpci_msix *msix = pdev->vpci->msix; - unsigned int i; + unsigned int i, j; int rc; ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); - if ( !mem ) - return -ENOMEM; - /* - * Create a rangeset that represents the current device BARs memory region - * and compare it against all the currently active BAR memory regions. If - * an overlap is found, subtract it from the region to be mapped/unmapped. + * Create a rangeset per BAR that represents the current device memory + * region and compare it against all the currently active BAR memory + * regions. If an overlap is found, subtract it from the region to be + * mapped/unmapped. * - * First fill the rangeset with all the BARs of this device or with the ROM + * First fill the rangesets with the BAR of this device or with the ROM * BAR only, depending on whether the guest is toggling the memory decode * bit of the command register, or the enable bit of the ROM BAR register. */ for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { - const struct vpci_bar *bar = &header->bars[i]; + struct vpci_bar *bar = &header->bars[i]; unsigned long start = PFN_DOWN(bar->addr); unsigned long end = PFN_DOWN(bar->addr + bar->size - 1); + if ( !bar->mem ) + continue; + if ( !MAPPABLE_BAR(bar) || (rom_only ? bar->type != VPCI_BAR_ROM : (bar->type == VPCI_BAR_ROM && !header->rom_enabled)) || @@ -292,14 +335,31 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) continue; } - rc = rangeset_add_range(mem, start, end); + rc = rangeset_add_range(bar->mem, start, end); if ( rc ) { printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n", start, end, rc); - rangeset_destroy(mem); return rc; } + + /* Check for overlap with the already setup BAR ranges. */ + for ( j = 0; j < i; j++ ) + { + struct vpci_bar *prev_bar = &header->bars[j]; + + if ( rangeset_is_empty(prev_bar->mem) ) + continue; + + rc = rangeset_remove_range(prev_bar->mem, start, end); + if ( rc ) + { + gprintk(XENLOG_WARNING, + "%pp: failed to remove overlapping range [%lx, %lx]: %d\n", + &pdev->sbdf, start, end, rc); + return rc; + } + } } /* Remove any MSIX regions if present. */ @@ -309,14 +369,21 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) + vmsix_table_size(pdev->vpci, i) - 1); - rc = rangeset_remove_range(mem, start, end); - if ( rc ) + for ( j = 0; j < ARRAY_SIZE(header->bars); j++ ) { - printk(XENLOG_G_WARNING - "Failed to remove MSIX table [%lx, %lx]: %d\n", - start, end, rc); - rangeset_destroy(mem); - return rc; + const struct vpci_bar *bar = &header->bars[j]; + + if ( rangeset_is_empty(bar->mem) ) + continue; + + rc = rangeset_remove_range(bar->mem, start, end); + if ( rc ) + { + gprintk(XENLOG_WARNING, + "%pp: failed to remove MSIX table [%lx, %lx]: %d\n", + &pdev->sbdf, start, end, rc); + return rc; + } } } @@ -356,27 +423,37 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) for ( i = 0; i < ARRAY_SIZE(tmp->vpci->header.bars); i++ ) { - const struct vpci_bar *bar = &tmp->vpci->header.bars[i]; - unsigned long start = PFN_DOWN(bar->addr); - unsigned long end = PFN_DOWN(bar->addr + bar->size - 1); - - if ( !bar->enabled || - !rangeset_overlaps_range(mem, start, end) || - /* - * If only the ROM enable bit is toggled check against - * other BARs in the same device for overlaps, but not - * against the same ROM BAR. - */ - (rom_only && tmp == pdev && bar->type == VPCI_BAR_ROM) ) + const struct vpci_bar *remote_bar = &tmp->vpci->header.bars[i]; + unsigned long start = PFN_DOWN(remote_bar->addr); + unsigned long end = PFN_DOWN(remote_bar->addr + + remote_bar->size - 1); + + if ( !remote_bar->enabled ) continue; - rc = rangeset_remove_range(mem, start, end); - if ( rc ) + for ( j = 0; j < ARRAY_SIZE(header->bars); j++) { - printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n", - start, end, rc); - rangeset_destroy(mem); - return rc; + const struct vpci_bar *bar = &header->bars[j]; + + if ( !rangeset_overlaps_range(bar->mem, start, end) || + /* + * If only the ROM enable bit is toggled check against + * other BARs in the same device for overlaps, but not + * against the same ROM BAR. + */ + (rom_only && + tmp == pdev && + bar->type == VPCI_BAR_ROM) ) + continue; + + rc = rangeset_remove_range(bar->mem, start, end); + if ( rc ) + { + gprintk(XENLOG_WARNING, + "%pp: failed to remove [%lx, %lx]: %d\n", + &pdev->sbdf, start, end, rc); + return rc; + } } } } @@ -400,10 +477,10 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) * will always be to establish mappings and process all the BARs. */ ASSERT((cmd & PCI_COMMAND_MEMORY) && !rom_only); - return apply_map(pdev->domain, pdev, mem, cmd); + return apply_map(pdev->domain, pdev, cmd); } - defer_map(dev->domain, dev, mem, cmd, rom_only); + defer_map(dev->domain, dev, cmd, rom_only); return 0; } @@ -598,6 +675,18 @@ static void cf_check rom_write( rom->addr = val & PCI_ROM_ADDRESS_MASK; } +static int bar_add_rangeset(const struct pci_dev *pdev, struct vpci_bar *bar, + unsigned int i) +{ + char str[32]; + + snprintf(str, sizeof(str), "%pp:BAR%u", &pdev->sbdf, i); + + bar->mem = rangeset_new(pdev->domain, str, RANGESETF_no_print); + + return !bar->mem ? -ENOMEM : 0; +} + static int cf_check init_bars(struct pci_dev *pdev) { uint16_t cmd; @@ -679,6 +768,10 @@ static int cf_check init_bars(struct pci_dev *pdev) else bars[i].type = VPCI_BAR_MEM32; + rc = bar_add_rangeset(pdev, &bars[i], i); + if ( rc ) + goto fail; + rc = pci_size_mem_bar(pdev->sbdf, reg, &addr, &size, (i == num_bars - 1) ? PCI_BAR_LAST : 0); if ( rc < 0 ) @@ -728,6 +821,12 @@ static int cf_check init_bars(struct pci_dev *pdev) rom_reg, 4, rom); if ( rc ) rom->type = VPCI_BAR_EMPTY; + else + { + rc = bar_add_rangeset(pdev, rom, i); + if ( rc ) + goto fail; + } } } else diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 9dacbcf958..8865c1580e 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -38,6 +38,8 @@ extern vpci_register_init_t *const __end_vpci_array[]; void vpci_deassign_device(struct pci_dev *pdev) { + unsigned int i; + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); if ( !has_vpci(pdev->domain) || !pdev->vpci ) @@ -63,6 +65,10 @@ void vpci_deassign_device(struct pci_dev *pdev) if ( pdev->vpci->msix->table[i] ) iounmap(pdev->vpci->msix->table[i]); } + + for ( i = 0; i < ARRAY_SIZE(pdev->vpci->header.bars); i++ ) + rangeset_destroy(pdev->vpci->header.bars[i].mem); + xfree(pdev->vpci->msix); xfree(pdev->vpci->msi); xfree(pdev->vpci); diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 2028f2151f..18a0eca3da 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -72,6 +72,7 @@ struct vpci { /* Guest address. */ uint64_t guest_addr; uint64_t size; + struct rangeset *mem; enum { VPCI_BAR_EMPTY, VPCI_BAR_IO, @@ -156,7 +157,6 @@ struct vpci { struct vpci_vcpu { /* Per-vcpu structure to store state while {un}mapping of PCI BARs. */ - struct rangeset *mem; struct pci_dev *pdev; uint16_t cmd; bool rom_only : 1; From patchwork Sat Dec 2 01:27:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7C3FCC07E97 for ; Sat, 2 Dec 2023 01:27:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645832.1008257 (Exim 4.92) (envelope-from ) id 1r9Emv-0002ZH-1u; Sat, 02 Dec 2023 01:27:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645832.1008257; Sat, 02 Dec 2023 01:27:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emu-0002Wk-TN; Sat, 02 Dec 2023 01:27:20 +0000 Received: by outflank-mailman (input) for mailman id 645832; Sat, 02 Dec 2023 01:27:19 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Ems-000272-T2 for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:19 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id edc4a75c-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:17 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGv2008549; Sat, 2 Dec 2023 01:27:14 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:14 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:09 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:09 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: edc4a75c-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MJ8P1LR0sHZ0c63MAEMiae1onWOKXr68uT2ZXgD/7RSnk/3zwyF/DD/aBZH1CsGHEBJzdPPVwH9hdONRJ73lp/vgYO1GoZ0wylmwMq1axear86DTDrjaMBZ1CQv70imOC8OcXq4j6CAWzzUt1lc+n/XEW3HMpMHAUAqUNE7hNHoqW8K4YKp1PPiGtocp4a8PUiwSAEFXq7aG4aTZmHOAxnWurqcRu3JRK3U/q07MefbhMPIS/Hz7oX9VBDSMi4OatzdZDalI2NPLZqwkozOJZpfU64hDff5PKpLbQh0XR8aVt48DOtcnDPxIJycNQEL/1eDX1/C3h8RLxlgi8Ga2bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6JZJ4KNkIkIzlijRLVq+YijCSEjROoN8dREC9RVL4YU=; b=eU5RxjpI8ooCAk0V2nT0qfntZrlj9CFAcWJ1Fmp49Ffp7EoDVGs1+QqCX7rJtFkResp3oMRGXQ1HSLjz/5oyNrKuvHEXDlCcEFre5KquO5Iv/XGtCPWmgovRkuRTsUr9BnSDFUs+ER6ocpMsSbt4UY++GRKvmpQjz/LlLfoH5uT01wu9IJqlDu9I8yZTTzci9muE85WZTYT9uayjC5pOuRCO29wG2CWJK6//PNIHIyHgMNO97x8BoHuEqqXFGBbnvGoD/qh9pc9URcj8G3SFDbXvXA8T2o5T5wMx8VaFD5ZwJQoQIidLRlkJWn1luoUi2GZOHP5VgiWGvdb3plwWSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6JZJ4KNkIkIzlijRLVq+YijCSEjROoN8dREC9RVL4YU=; b=Gnrv63yCISztxCQ2XP19Dnfrz17fRPZWzYohEHGTfL/bbgEG0loSmBHO+U8/TABph3rSF9kAr5LSsZKNJ2hedCG2qzOWSwz8kZizaL+RTU5bAUWM9r+G4TStBCdgwnHfdg0e4QRzJZJ+3bfpfFGT4sDETnku1sFj4jMq2RXoFkLYTGEyUVgDM0XrSYFYGLbPqO7ZAB4gcOQs8d2iQKSi0SB7G6boZO3wki+S5rfcv8z33AWR/m1jSSCnxXsL0Etiw+yplrxJ5R6yPxJz9B01BtLoIgQkRwpFN591av7C23cB5RKZA1h63tyqskmq0vsCqI8S1OpWeQbiNpLU5TYZ+g== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , "Daniel P. Smith" , Volodymyr Babchuk Subject: [PATCH v11 11/17] vpci/header: program p2m with guest BAR view Thread-Topic: [PATCH v11 11/17] vpci/header: program p2m with guest BAR view Thread-Index: AQHaJL6om72B3RYnP0aRq1inaCmShA== Date: Sat, 2 Dec 2023 01:27:05 +0000 Message-ID: <20231202012556.2012281-12-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 28c3a8a2-7ce8-453b-af25-08dbf2d5cd8e x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: peh66Gl38lFbeQ7++fR9s6DvKkkNl42E8JKcVxQX55SpMMRCmyQys8KdRUNqIbJMgJW+mFJ41RTGl08XcXEzOpOnsp/hUYs4kTg75K7RlT9e3sROz6yedFVIhcQ2ONou4z8hAxQ20XadoqlytwobXees7G2mKwt0+IrKMcNQhXzpBbUR7EbnlhkEBwD/R9QL4SlU5k99QRiBuyTUa/RMncDmIaQxLw4ehrR9kSE1WAXkwYmFHAIFIhwGKQR4t6AefpmC8J75UfKBqvazEzy4uTVem1up437hk1okHEy8yeV8TvgOLAkW5JMqnG7E5SQ5yM0Oh2AgkuhTYMMLoZlDX6Fv6/EjQeGukGQ7TapZy+1bRiv3ITF/YMlr6eo5gTjII72vkinqbh43NcTiR//QxjOSn12ssgrTDNjyMvAsUrY6mD/COpodsMl5skRTnl6HuXHkQ92THfuYnvs406lTQCA15SzgnRHYbwScHrjQJZbCRy5b0CYg/DXAkoY3q5evaOpCiQrkpQo4yUR8X89ktUlj0QJXEUAj2thGVSKJy1yyTjKo74182kssPXLkT5XwkkeLSAmXBtkVhgs34v8kDjLrngCWT3XvduMsupTKWbbxhtm97EjMuTXqOl7+cmXxvz4S7hOS4snm7jEbGHvXBM2Kn6SFD+4sRj1swUkw0FA= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(230273577357003)(230173577357003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(30864003)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?dF9536ehMnXKgtonKO0GR6m?= =?iso-8859-1?q?nGuk8C055hgxSQXsq/2mGy1biysrurt/UpWLWSc3EQxeTM/m9Wh3cfCjsOiq?= =?iso-8859-1?q?yKrz7kJ576zx5pX+jeLOfITZ/jIQAJG607DxsjaV9HVhCBAOT5IkxNRdI6f5?= =?iso-8859-1?q?NFUx9h2jY4Ld+wmKuv5GP8oNZ3D+pFfBlLudxoWPN3EMkvTcTiTt8et51gus?= =?iso-8859-1?q?STJwTdedX45glc8FzG7n3ZDTAet22FF6Mb5wFWiS7R7o8reRlcJfFzLgEfsI?= =?iso-8859-1?q?SO9Ddfix1ujuwJifb2IOoN51otVPKUrmyjvuiiRKOHLRZNluM6e09oYHyUVw?= =?iso-8859-1?q?RG43Ck1ZBPbJul/o7pf6w2XsIP2nWHN3bpDusKTTCc2XbqWXn6abEqk7AzNw?= =?iso-8859-1?q?ly8F2FOxy3AS/Ai0B8RWXFcA4Y2dLMUxQCob2NJxt79MlwHWGEHyMuIoFdwB?= =?iso-8859-1?q?8vV2dO0+mapafyuJWM3lnPwG1bYU3BjKIfA9ZGtuU5lg1DA6zvtbMib2EFPm?= =?iso-8859-1?q?rjSfkToqsmhSg3YwpdURD3OtXwwllR4J55epmdMQqEz9X4q2FZ1jKnvZunpa?= =?iso-8859-1?q?X8cwbKO4jvoD34q0WipU3z1kdI/mTDRZVDU4tGLfwcGu8UqiteI8zpd4KYri?= =?iso-8859-1?q?hdtJrPtUdyhemshTCKwxVZI9Taf0LZf11PFs0N3Fj5iIzZMSGCpLth02PK1B?= =?iso-8859-1?q?LnxPWduHH2RU9FUKh4uCm7jrvxCl1d9cHllHXT3Hnz49QWzlBBxmq214H8RV?= =?iso-8859-1?q?MUQaL7X+sUgE7tsUtO577JqsU/zQvx06jmJsXZX33EikrKmgR1WAUUZmGkdz?= =?iso-8859-1?q?Qe58gnz2Sz6Yig8InGi3YeYvbXCpjpPkU5zK09GxAQw/PeZJVvl0pZrvXz2v?= =?iso-8859-1?q?WUmqEoVzVNyvtyrsC8aY8byY7dpxdrzy7ENUQqoh81nm8FXTnK5JbxGENLwV?= =?iso-8859-1?q?/U9kDPzXgf1Xxom1ds/DhzVkjRsPz4sGOFlyEo0OHx+OMEoOOG1FrfGwTuJc?= =?iso-8859-1?q?PtN6GvYrG6EMbssbSx8oUbyl0W+tie6NZsuKGeSFXS9g4iEtKD0w+HTxGW0x?= =?iso-8859-1?q?BA08t68C0eMPfD7qRbHf5W4npX2TzNOHqeiwnJC6gXtOsLgq56B/4TATV9av?= =?iso-8859-1?q?I4U6jsasKfCwixql6/HU0sPmKnWybysSg23lUN4N0mwL4gbIVlloGusHTZCv?= =?iso-8859-1?q?XqVmMrzl8dITjxZ3LfQCslp8xKXgUOyaid2UkP/AOkFzfw+n7An0PCkoF+1D?= =?iso-8859-1?q?5MOsNywc35aXGfUeToIvoGnJiAF8BIwR50qT4Ci7hy8Kzh2+kpgNC1ugn5Qp?= =?iso-8859-1?q?JgtHXlMOlB/SO8Wk0eWaHtrkruNXN0OhHpME+vXj9B2OkM2j0rAWIo0ReCgE?= =?iso-8859-1?q?EKwtLrjcPwzLwWsQ9i5Rv4e0UmMMPAZGlmBSJnC1Zst1Ii4+YpNC1T7qFTZC?= =?iso-8859-1?q?GtSBAwQKH3HiCwJcaDUKbDQin77X8W1DG+AYH0MhkNdoJlkug10evq3vv7RX?= =?iso-8859-1?q?WuXJb7c7NyIJ/rH4XKTkBscO6ClxYmvsiyt30BnB2gt9bJpPx79QUnvQlW9h?= =?iso-8859-1?q?Q8paF+o9iNGiZ1hh8nr4R3qIsdIxc2o6/ZcyEQzzWt6jy4GgtNVelgm7mhy/?= =?iso-8859-1?q?ubGLq8zADg8SNBTnNZdXFrdVmozqPvOMXeqV0Hw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 28c3a8a2-7ce8-453b-af25-08dbf2d5cd8e X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:05.6553 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: c45RotjQVgWwH7ivQzjwrjyQeVc+xLGpUI+d5Rc3CgKUgOwSHYeNbp9OvckWzAzoh8i4wGBRjbFKWtg1jUvuZfYLupRPQDSNxG556BMkK84= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: zCpSr2_hKR2E1wjHDHosQKpIg6EeqP3W X-Proofpoint-ORIG-GUID: zCpSr2_hKR2E1wjHDHosQKpIg6EeqP3W X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko Take into account guest's BAR view and program its p2m accordingly: gfn is guest's view of the BAR and mfn is the physical BAR value. This way hardware domain sees physical BAR values and guest sees emulated ones. Hardware domain continues getting the BARs identity mapped, while for domUs the BARs are mapped at the requested guest address without modifying the BAR address in the device PCI config space. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk --- In v11: - Add vmsix_guest_table_addr() and vmsix_guest_table_base() functions to access guest's view of the VMSIx tables. - Use MFN (not GFN) to check access permissions - Move page offset check to this patch - Call rangeset_remove_range() with correct parameters In v10: - Moved GFN variable definition outside the loop in map_range() - Updated printk error message in map_range() - Now BAR address is always stored in bar->guest_addr, even for HW dom, this removes bunch of ugly is_hwdom() checks in modify_bars() - vmsix_table_base() now uses .guest_addr instead of .addr In v9: - Extended the commit message - Use bar->guest_addr in modify_bars - Extended printk error message in map_range - Moved map_data initialization so .bar can be initialized during declaration Since v5: - remove debug print in map_range callback - remove "identity" from the debug print Since v4: - moved start_{gfn|mfn} calculation into map_range - pass vpci_bar in the map_data instead of start_{gfn|mfn} - s/guest_addr/guest_reg Since v3: - updated comment (Roger) - removed gfn_add(map->start_gfn, rc); which is wrong - use v->domain instead of v->vpci.pdev->domain - removed odd e.g. in comment - s/d%d/%pd in altered code - use gdprintk for map/unmap logs Since v2: - improve readability for data.start_gfn and restructure ?: construct Since v1: - s/MSI/MSI-X in comments --- xen/drivers/vpci/header.c | 79 +++++++++++++++++++++++++++++---------- xen/include/xen/vpci.h | 13 +++++++ 2 files changed, 73 insertions(+), 19 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 7c84cee5d1..21b3fb5579 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -33,6 +33,7 @@ struct map_data { struct domain *d; + const struct vpci_bar *bar; bool map; }; @@ -40,13 +41,24 @@ static int cf_check map_range( unsigned long s, unsigned long e, void *data, unsigned long *c) { const struct map_data *map = data; + /* Start address of the BAR as seen by the guest. */ + unsigned long start_gfn = PFN_DOWN(map->bar->guest_addr); + /* Physical start address of the BAR. */ + mfn_t start_mfn = _mfn(PFN_DOWN(map->bar->addr)); int rc; for ( ; ; ) { unsigned long size = e - s + 1; + /* + * Ranges to be mapped don't always start at the BAR start address, as + * there can be holes or partially consumed ranges. Account for the + * offset of the current address from the BAR start. + */ + mfn_t map_mfn = mfn_add(start_mfn, s - start_gfn); + unsigned long m_end = mfn_x(map_mfn) + size - 1; - if ( !iomem_access_permitted(map->d, s, e) ) + if ( !iomem_access_permitted(map->d, mfn_x(map_mfn), m_end) ) { printk(XENLOG_G_WARNING "%pd denied access to MMIO range [%#lx, %#lx]\n", @@ -54,7 +66,8 @@ static int cf_check map_range( return -EPERM; } - rc = xsm_iomem_mapping(XSM_HOOK, map->d, s, e, map->map); + rc = xsm_iomem_mapping(XSM_HOOK, map->d, mfn_x(map_mfn), m_end, + map->map); if ( rc ) { printk(XENLOG_G_WARNING @@ -72,8 +85,8 @@ static int cf_check map_range( * - {un}map_mmio_regions doesn't support preemption. */ - rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s)) - : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s)); + rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, map_mfn) + : unmap_mmio_regions(map->d, _gfn(s), size, map_mfn); if ( rc == 0 ) { *c += size; @@ -82,8 +95,9 @@ static int cf_check map_range( if ( rc < 0 ) { printk(XENLOG_G_WARNING - "Failed to identity %smap [%lx, %lx] for d%d: %d\n", - map->map ? "" : "un", s, e, map->d->domain_id, rc); + "Failed to %smap [%lx %lx] -> [%lx %lx] for %pd: %d\n", + map->map ? "" : "un", s, e, mfn_x(map_mfn), + mfn_x(map_mfn) + size, map->d, rc); break; } ASSERT(rc < size); @@ -162,10 +176,6 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd, bool vpci_process_pending(struct vcpu *v) { struct pci_dev *pdev = v->vpci.pdev; - struct map_data data = { - .d = v->domain, - .map = v->vpci.cmd & PCI_COMMAND_MEMORY, - }; struct vpci_header *header = NULL; unsigned int i; @@ -185,6 +195,11 @@ bool vpci_process_pending(struct vcpu *v) for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { struct vpci_bar *bar = &header->bars[i]; + struct map_data data = { + .d = v->domain, + .map = v->vpci.cmd & PCI_COMMAND_MEMORY, + .bar = bar, + }; int rc; if ( rangeset_is_empty(bar->mem) ) @@ -235,7 +250,6 @@ bool vpci_process_pending(struct vcpu *v) static int __init apply_map(struct domain *d, const struct pci_dev *pdev, uint16_t cmd) { - struct map_data data = { .d = d, .map = true }; struct vpci_header *header = &pdev->vpci->header; int rc = 0; unsigned int i; @@ -245,6 +259,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev, for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { struct vpci_bar *bar = &header->bars[i]; + struct map_data data = { .d = d, .map = true, .bar = bar }; if ( rangeset_is_empty(bar->mem) ) continue; @@ -310,12 +325,16 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) * First fill the rangesets with the BAR of this device or with the ROM * BAR only, depending on whether the guest is toggling the memory decode * bit of the command register, or the enable bit of the ROM BAR register. + * + * For non-hardware domain we use guest physical addresses. */ for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { struct vpci_bar *bar = &header->bars[i]; unsigned long start = PFN_DOWN(bar->addr); unsigned long end = PFN_DOWN(bar->addr + bar->size - 1); + unsigned long start_guest = PFN_DOWN(bar->guest_addr); + unsigned long end_guest = PFN_DOWN(bar->guest_addr + bar->size - 1); if ( !bar->mem ) continue; @@ -335,11 +354,25 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) continue; } - rc = rangeset_add_range(bar->mem, start, end); + /* + * Make sure that the guest set address has the same page offset + * as the physical address on the host or otherwise things won't work as + * expected. + */ + if ( PAGE_OFFSET(bar->guest_addr) != PAGE_OFFSET(bar->addr) ) + { + gprintk(XENLOG_G_WARNING, + "%pp: Can't map BAR%d because of page offset mismatch: %lx vs %lx\n", + &pdev->sbdf, i, PAGE_OFFSET(bar->guest_addr), + PAGE_OFFSET(bar->addr)); + return -EINVAL; + } + + rc = rangeset_add_range(bar->mem, start_guest, end_guest); if ( rc ) { printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n", - start, end, rc); + start_guest, end_guest, rc); return rc; } @@ -351,12 +384,12 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) if ( rangeset_is_empty(prev_bar->mem) ) continue; - rc = rangeset_remove_range(prev_bar->mem, start, end); + rc = rangeset_remove_range(prev_bar->mem, start_guest, end_guest); if ( rc ) { gprintk(XENLOG_WARNING, "%pp: failed to remove overlapping range [%lx, %lx]: %d\n", - &pdev->sbdf, start, end, rc); + &pdev->sbdf, start_guest, end_guest, rc); return rc; } } @@ -365,8 +398,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) /* Remove any MSIX regions if present. */ for ( i = 0; msix && i < ARRAY_SIZE(msix->tables); i++ ) { - unsigned long start = PFN_DOWN(vmsix_table_addr(pdev->vpci, i)); - unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) + + unsigned long start = PFN_DOWN(vmsix_guest_table_addr(pdev->vpci, i)); + unsigned long end = PFN_DOWN(vmsix_guest_table_addr(pdev->vpci, i) + vmsix_table_size(pdev->vpci, i) - 1); for ( j = 0; j < ARRAY_SIZE(header->bars); j++ ) @@ -424,8 +457,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) for ( i = 0; i < ARRAY_SIZE(tmp->vpci->header.bars); i++ ) { const struct vpci_bar *remote_bar = &tmp->vpci->header.bars[i]; - unsigned long start = PFN_DOWN(remote_bar->addr); - unsigned long end = PFN_DOWN(remote_bar->addr + + unsigned long start = PFN_DOWN(remote_bar->guest_addr); + unsigned long end = PFN_DOWN(remote_bar->guest_addr + remote_bar->size - 1); if ( !remote_bar->enabled ) @@ -512,6 +545,8 @@ static void cf_check bar_write( struct vpci_bar *bar = data; bool hi = false; + ASSERT(is_hardware_domain(pdev->domain)); + if ( bar->type == VPCI_BAR_MEM64_HI ) { ASSERT(reg > PCI_BASE_ADDRESS_0); @@ -542,6 +577,10 @@ static void cf_check bar_write( */ bar->addr &= ~(0xffffffffULL << (hi ? 32 : 0)); bar->addr |= (uint64_t)val << (hi ? 32 : 0); + /* + * Update guest address as well, so hardware domain sees BAR identity mapped + */ + bar->guest_addr = bar->addr; /* Make sure Xen writes back the same value for the BAR RO bits. */ if ( !hi ) @@ -793,6 +832,7 @@ static int cf_check init_bars(struct pci_dev *pdev) } bars[i].addr = addr; + bars[i].guest_addr = addr; bars[i].size = size; bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH; @@ -814,6 +854,7 @@ static int cf_check init_bars(struct pci_dev *pdev) rom->type = VPCI_BAR_ROM; rom->size = size; rom->addr = addr; + rom->guest_addr = addr; header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) & PCI_ROM_ADDRESS_ENABLE; diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 18a0eca3da..c39fab4832 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -205,6 +205,19 @@ static inline paddr_t vmsix_table_addr(const struct vpci *vpci, unsigned int nr) (vpci->msix->tables[nr] & ~PCI_MSIX_BIRMASK); } +static inline paddr_t vmsix_guest_table_base(const struct vpci *vpci, + unsigned int nr) +{ + return (vpci->header.bars[vpci->msix->tables[nr] & + PCI_MSIX_BIRMASK].guest_addr & PCI_BASE_ADDRESS_MEM_MASK); +} + +static inline paddr_t vmsix_guest_table_addr(const struct vpci *vpci, + unsigned int nr) +{ + return vmsix_guest_table_base(vpci, nr) + + (vpci->msix->tables[nr] & ~PCI_MSIX_BIRMASK); +} /* * Note regarding the size calculation of the PBA: the spec mentions "The last * QWORD will not necessarily be fully populated", so it implies that the PBA From patchwork Sat Dec 2 01:27:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C4CDC10F05 for ; Sat, 2 Dec 2023 01:27:36 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645835.1008291 (Exim 4.92) (envelope-from ) id 1r9Emx-0003LF-Gh; Sat, 02 Dec 2023 01:27:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645835.1008291; Sat, 02 Dec 2023 01:27:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emx-0003K7-46; Sat, 02 Dec 2023 01:27:23 +0000 Received: by outflank-mailman (input) for mailman id 645835; Sat, 02 Dec 2023 01:27:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emu-000271-OH for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:20 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ed77f349-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:17 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWc004068; Sat, 2 Dec 2023 01:27:15 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:14 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:09 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:09 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ed77f349-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bCQtJj4C3OyLvyP07sjtYz/XMAIavCGvgfPgqI078Uw4ifiMe2Ks+WDclK3kAA8fmW54UUckTOzQdf0l8lzcfjY89SmN8z6sl+pgCd9HRA/6UDrpwsC6tUwgaeyXOXq8mVWtMuvkL7wIsnh8I1GablCWsxfDc1+CRUlGOyKWhag6t9QVi5zmFPjPMpoQ2c2Qp77v1sIPhGeRsXH2+z7M2r67U04qM/VseKQl2gZt2Tsch5pSlVioaFbWMbeaT7cMixrDxe/wZ4QipHV9V6NN4Jwd4/FymYiEn/EXyKu5OO+QhgJDvhrwBefjXbi9ElqsulJMsSjakmS9pW/GiVj2gA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q3pAsw+LQMW37ya9xY/fsEtMgle4eyEFHTErMj7PTwA=; b=UjyMO+dt5QnyuFZ1My3/vv+m+fj4UzGRzGlkr1lOiaKOPtJguC9oKSMvVZBJhMCG8yL6GSyFq81QgJKMdsFpcesFeSQNkmB74gqsjmchQyl0C3eDpcOfm6RHWhm95JDUfB53y1Ogr30MWRncLI0EFchySU3O9nl7xUNUD98ZU6l35O1WwoGiHC8GTOiFnzdwYAuUeFKNuiwFERL6imOm7J94CxNYtuyQRrF6QdL/J9Kh+AJDthNcoa+cs7xKWvkWV7+jRk9kW9kVsFtqO8B6uk4J4tqZYNvdjyppEpptPm7bBpvsLW/p4duIdx9iWGJnSme1jQeoRAzTWruFEbih0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q3pAsw+LQMW37ya9xY/fsEtMgle4eyEFHTErMj7PTwA=; b=lNrTyzTZs3cOeRQ8SMNKIowGFqODO2etAm86cg3flk9Tz/BppgSo271B3C5/42D3YCCD5JG5edmBFfrkeIlh6eHZlh1NrENsCj2Na8tbYZE+9lPuMXKrMicF8/tjx5zb/Tpo1xiL0m1yQDKHiVjh5ewP6ssyBZ8mRAXQajGjiJaiTxn0XhkiNm8Pi90FGwpo3d2tqOEpfQi/5mw7GseIHs2Lp2BAmmIEhXHwTe2rlOxvujPMEs5g78+PcT8U5X3xUK0Pmwyj6ecXCTvu+3KkCntcTpzTNwzaN65HGDMdp5PT9FjIV1jehRdoqlb/dvbeVhpekjWHBRsk7ntJ8kmLHQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Volodymyr Babchuk Subject: [PATCH v11 12/17] vpci/header: emulate PCI_COMMAND register for guests Thread-Topic: [PATCH v11 12/17] vpci/header: emulate PCI_COMMAND register for guests Thread-Index: AQHaJL6pbF2piO+GXk2gsbpguvwtEw== Date: Sat, 2 Dec 2023 01:27:05 +0000 Message-ID: <20231202012556.2012281-13-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 7eeb0784-d548-49f3-edaa-08dbf2d5cdcd x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3E8QrFBpn4d2JTFqbXNlmc+7Y5PmnvBKTQWcUZG/y7btkyH54A+cFcPtVBXSJR6yOImji76Nn/miJN2JMoeMWOfT1T3ZWXdUEjai/hbFhT9hbKKaYrDneFHtgVP16JQxTqU2SMKPpmrFV5plS1z8y1YgYR1F6zS2PR3SV8Xe8dsA/LXULHA+fNgkYlXiokvxA0pA+00Z68xsmGdhg1p9F7rcpo6jvjaWMOsUHK3+z3mOd0Lh3PcjC24fjLq9p9bhoRGZTvuPIbn/V9JsSyOGvUbzHZem+zYAZae0kf9/0gObSWNq+fnEqYwzdL5E/tI11NBmHHOrL+q487cTTbKycNilwRjj4QI3HENyCOMYz8CCBfUU18R/yzFyShNG/jQH+S+JAda8Zc+IjnrGtGh+U7UWMFJftFYcuJqhiBs92JniKzWS0uZln0D6WM90aCexM0KIX0tQhdkbObct4GOd/dmZAvrGx6mrWHRwJ2tL5uuf9Ca2VSNwuWL2v8lq5ZmLK9GVrSbksb/KjEyPJf/rr+U9hYOVt2bjX2HqTG3QbAijr1Sm8WykrHTWtcx4k1SHg9GE6WArNIZhyEeZWurih54D1QJKM4HogbfI8b8SH9DrwSl8jSB0LS5cdv0gykde+O62AQAN8+RELsZacs4DGWrsGn9DOdcp1VcejnkGOpo= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(230273577357003)(230173577357003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?LFbhjfAeQtPViVq9hvudyKu?= =?iso-8859-1?q?byzH56K+b0vUaMLbp6E/HmsGzeOFPs8MMj7IIJ2F225wxlWQCe5O7XCyXMTJ?= =?iso-8859-1?q?GsU9POIElFhT77Y52xE40AenBP0ViYLwsssAjFEQvn7cM1xO2W7xzrE19UN+?= =?iso-8859-1?q?HwoS32fx+czCcRQjH4CpowhTb71wcOWE6+HdvXb2YLN+kLkZv1GqsXcKshEO?= =?iso-8859-1?q?KgveQkuCRtAQsyRXDRXDkdTcVEpuc2Qdg0pRE0YK9cie5kkTttseYAQUhjjw?= =?iso-8859-1?q?LPSnKrXqqRdUEa/b/YXBsQzNKmknbZmPjXsx94Vla62IlXjQGcDXDvf7kCKm?= =?iso-8859-1?q?2e6Oe21POX0GepJ5LP4Zr9BQ7G5u6760ZfdKk2I1pHicGcw6wUl5s3rc2/oe?= =?iso-8859-1?q?ukRbzXKZdSB7JwbgG/LwVNzJAf4t4e/ZjEWZ7ZaABCq4e1ZpJAgdsqKuCiuZ?= =?iso-8859-1?q?mzKjFOnyo7Mmf1fHTTTQ8u4kn/LJ7qoN5AIT3heXE2z1SFnfRapTKqCm0yYP?= =?iso-8859-1?q?c2cTNMwqFbJUEG1xxIckFN00OM1vF3/cEMaxpxDBoKXsLtxjAKR7ZrWi8EcL?= =?iso-8859-1?q?xyLHbAYkTio5moNErwL5QmADXX9CKVdP18zibJWpKDRk1KkLBD/vbsn1u0v8?= =?iso-8859-1?q?HIvq7+jyP9xOsHWPubVNABm0Han+JeLJPUBl+1HJLYZIDvohnv0xg0gT/PBN?= =?iso-8859-1?q?LqD/f/nW6QA3zOvteMKUGwMuYhFPDVeViFPm0ZpRRp/WjYmMrFvR6+6jZSMk?= =?iso-8859-1?q?bseVlKjpABB490ZX5ueTcxGCqfeAyzmYVSgg/PGMrigtecbMGJ6MNoOmK75Y?= =?iso-8859-1?q?A5FOBk1iWEpL5IeYNbD0X2hL8yniQ1lpxBARF96mKjgGRPNCD66toh/iVe5T?= =?iso-8859-1?q?7l/iWQzZ41+h46nksvMdSAkIP7OHzQLAMFLJ4VLwz8abQ1AYr1cX6h14olU1?= =?iso-8859-1?q?B2IGLfXvbnCR8pE2w4HEowo4RdXIfg1XQIQDzKXF9jFYt+RjppvbWPkg+1lz?= =?iso-8859-1?q?05fniA5KVF51CV+tu/olh6pE44AazDozPxz5130LqmbbMBwNV/qGhsEBGZ9U?= =?iso-8859-1?q?paafHypnhUW68yycvlIl3hrcmVdPUiOtchv4rhDdxrPQEefZFzO/ac1s2Mht?= =?iso-8859-1?q?jrs8hj6lA9B+baeBRATDXLjB09XZcGj8dF2GXqIb7jSI90/iUTVxVdER7MGd?= =?iso-8859-1?q?ll/ttJqss7tvNME8G6rigZDiKNaK6lJXMpvQc9d10UYdEb225asbwyLA8dRT?= =?iso-8859-1?q?mSIeyvg4aNdyvXyGFWbx7dZE60dsa07CKfOQegAsVr41pYk1bWZb0/CzI84q?= =?iso-8859-1?q?FBvQonnDnv2AdxMfw7t19U95kx/wbNBWYzxRqs5VGS6u2WrT6xDUT6bgFGr5?= =?iso-8859-1?q?Fp7U4xp2WUPfUyseMnHIdB/1EhlBag5HrpIuhpHTs3/aq9UcZQakadqKipHV?= =?iso-8859-1?q?pnP5NNcyUFGz+fDTetW8i48UeZiJlNy7E7HhqUeEBRQ/tf20DocKDi8AkiYr?= =?iso-8859-1?q?N6ruqmz0L9OSl3rF36ubpGf15ufZBtoU69XrRSfc/n85AeV/KIivTMh3XwLc?= =?iso-8859-1?q?5JHZmH1ijGc7z7cQOuURJGCTikkiYle1cCp0hGH56L0gTYPrTCgQszvDhNv1?= =?iso-8859-1?q?rUtL8FmfXMe2IRKJUx5Ml+8NRK0nL7GfTrWUDUA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7eeb0784-d548-49f3-edaa-08dbf2d5cdcd X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:05.9424 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: scQYzCULk6geVBzzqAFkPuaJciwpisnhaDop1ed+LL5jDiOFuXHcwMu4DtPJHVmPju1nOD9Jo9WIp2VexLbdZ64bsV/hhZutt5FQMJV4v3g= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: w3MQhgRcfDnR5a8Ep2nAdKiHh-HfIuWo X-Proofpoint-GUID: w3MQhgRcfDnR5a8Ep2nAdKiHh-HfIuWo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko Xen and/or Dom0 may have put values in PCI_COMMAND which they expect to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the guest's view of this will want to be zero initially, the host having set it to 1 may not easily be overwritten with 0, or else we'd effectively imply giving the guest control of the bit. Thus, PCI_COMMAND register needs proper emulation in order to honor host's settings. According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2 Device Control" the reset state of the command register is typically 0, so when assigning a PCI device use 0 as the initial state for the guest's view of the command register. Here is the full list of command register bits with notes about emulation, along with QEMU behavior in the same situation: PCI_COMMAND_IO - QEMU does not allow a guest to change value of this bit in real device. Instead it is always set to 1. A guest can write to this register, but writes are ignored. PCI_COMMAND_MEMORY - QEMU behaves exactly as with PCI_COMMAND_IO. In Xen case, we handle writes to this bit by mapping/unmapping BAR regions. For devices assigned to DomUs, memory decoding will be disabled at the initialization. PCI_COMMAND_MASTER - Allow guest to control it. QEMU passes through writes to this bit. PCI_COMMAND_SPECIAL - Guest can generate special cycles only if it has access to host bridge that supports software generation of special cycles. In our case guest has no access to host bridges at all. Value after reset is 0. QEMU passes through writes of this bit, we will do the same. PCI_COMMAND_INVALIDATE - Allows "Memory Write and Invalidate" commands to be generated. It requires additional configuration via Cacheline Size register. We are not emulating this register right now and we can't expect guest to properly configure it. QEMU "emulates" access to Cachline Size register by ignoring all writes to it. QEMU passes through writes of PCI_COMMAND_INVALIDATE bit, we will do the same. PCI_COMMAND_VGA_PALETTE - Enable VGA palette snooping. QEMU passes through writes of this bit, we will do the same. PCI_COMMAND_PARITY - Controls how device response to parity errors. QEMU ignores writes to this bit, we will do the same. PCI_COMMAND_WAIT - Reserved. Should be 0, but QEMU passes through writes of this bit, so we will do the same. PCI_COMMAND_SERR - Controls if device can assert SERR. QEMU ignores writes to this bit, we will do the same. PCI_COMMAND_FAST_BACK - Optional bit that allows fast back-to-back transactions. It is configured by firmware, so we don't want guest to control it. QEMU ignores writes to this bit, we will do the same. PCI_COMMAND_INTX_DISABLE - Disables INTx signals. If MSI(X) is enabled, device is prohibited from asserting INTx as per specification. Value after reset is 0. In QEMU case, it checks of INTx was mapped for a device. If it is not, then guest can't control PCI_COMMAND_INTX_DISABLE bit. In our case, we prohibit a guest to change value of this bit if MSI(X) is enabled. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk --- It is better to rework this patch using new register handling tools that Stewart Hildenbrand upstreaming right now. In v11: - Fix copy-paste mistake: vpci->msi should be vpci->msix - Handle PCI_COMMAND_IO - Fix condition for disabling INTx in the MSI-X code - Show domU changes to only allowed bits - Show PCI_COMMAND_MEMORY write only after P2M was altered - Update comments in the code In v10: - Added cf_check attribute to guest_cmd_read - Removed warning about non-zero cmd - Updated comment MSI code regarding disabling INTX - Used ternary operator in vpci_add_register() call - Disable memory decoding for DomUs in init_bars() In v9: - Reworked guest_cmd_read - Added handling for more bits Since v6: - fold guest's logic into cmd_write - implement cmd_read, so we can report emulated INTx state to guests - introduce header->guest_cmd to hold the emulated state of the PCI_COMMAND register for guests Since v5: - add additional check for MSI-X enabled while altering INTX bit - make sure INTx disabled while guests enable MSI/MSI-X Since v3: - gate more code on CONFIG_HAS_MSI - removed logic for the case when MSI/MSI-X not enabled --- xen/drivers/vpci/header.c | 53 ++++++++++++++++++++++++++++++++++++--- xen/drivers/vpci/msi.c | 6 +++++ xen/drivers/vpci/msix.c | 5 ++++ xen/include/xen/vpci.h | 3 +++ 4 files changed, 64 insertions(+), 3 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 21b3fb5579..bc2ebe125b 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -167,6 +167,9 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd, if ( !rom_only ) { pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); + /* Show DomU that we updated P2M */ + header->guest_cmd &= ~PCI_COMMAND_MEMORY; + header->guest_cmd = (val & PCI_COMMAND_MEMORY); header->bars_mapped = map; } else @@ -518,14 +521,40 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) return 0; } +/* TODO: Add proper emulation for all bits of the command register. */ static void cf_check cmd_write( const struct pci_dev *pdev, unsigned int reg, uint32_t cmd, void *data) { struct vpci_header *header = data; + if ( !is_hardware_domain(pdev->domain) ) + { + const struct vpci *vpci = pdev->vpci; + uint16_t excluded = PCI_COMMAND_PARITY | PCI_COMMAND_SERR | + PCI_COMMAND_FAST_BACK | PCI_COMMAND_IO; + + if ( (vpci->msi && vpci->msi->enabled) || + (vpci->msix && vpci->msix->enabled) ) + excluded |= PCI_COMMAND_INTX_DISABLE; + + cmd &= ~excluded; + + /* + * Show guest that we allowed it to change bits that are not + * immediately excluded. Do not show change to + * PCI_COMMAND_MEMORY bit till we finish with P2M + */ + header->guest_cmd = (header->guest_cmd & + (excluded | PCI_COMMAND_MEMORY)) | + (cmd & ~PCI_COMMAND_MEMORY); + + cmd |= pci_conf_read16(pdev->sbdf, reg) & excluded; + } + /* * Let Dom0 play with all the bits directly except for the memory - * decoding one. + * decoding one. Bits that are not allowed for DomU are already + * handled above. */ if ( header->bars_mapped != !!(cmd & PCI_COMMAND_MEMORY) ) /* @@ -539,6 +568,14 @@ static void cf_check cmd_write( pci_conf_write16(pdev->sbdf, reg, cmd); } +static uint32_t cf_check guest_cmd_read( + const struct pci_dev *pdev, unsigned int reg, void *data) +{ + const struct vpci_header *header = data; + + return header->guest_cmd; +} + static void cf_check bar_write( const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data) { @@ -755,8 +792,9 @@ static int cf_check init_bars(struct pci_dev *pdev) } /* Setup a handler for the command register. */ - rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND, - 2, header); + rc = vpci_add_register(pdev->vpci, + is_hwdom ? vpci_hw_read16 : guest_cmd_read, + cmd_write, PCI_COMMAND, 2, header); if ( rc ) return rc; @@ -768,6 +806,15 @@ static int cf_check init_bars(struct pci_dev *pdev) if ( cmd & PCI_COMMAND_MEMORY ) pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY); + /* + * Clear PCI_COMMAND_MEMORY for DomUs, so they will always start with + * memory decoding disabled and to ensure that we will not call modify_bars() + * at the end of this function. + */ + if ( !is_hwdom ) + cmd &= ~ (PCI_COMMAND_MEMORY | PCI_COMMAND_IO); + header->guest_cmd = cmd; + for ( i = 0; i < num_bars; i++ ) { uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4; diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c index 6ff71e5f9a..6cfa9aea00 100644 --- a/xen/drivers/vpci/msi.c +++ b/xen/drivers/vpci/msi.c @@ -70,6 +70,12 @@ static void cf_check control_write( if ( vpci_msi_arch_enable(msi, pdev, vectors) ) return; + + /* + * Make sure domU doesn't enable INTx while enabling MSI. + */ + if ( !is_hardware_domain(pdev->domain) ) + pci_intx(pdev, false); } else vpci_msi_arch_disable(msi, pdev); diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c index b6abab47ef..2aa2833160 100644 --- a/xen/drivers/vpci/msix.c +++ b/xen/drivers/vpci/msix.c @@ -97,6 +97,7 @@ static void cf_check control_write( for ( i = 0; i < msix->max_entries; i++ ) if ( !msix->entries[i].masked && msix->entries[i].updated ) update_entry(&msix->entries[i], pdev, i); + } else if ( !new_enabled && msix->enabled ) { @@ -135,6 +136,10 @@ static void cf_check control_write( } } + /* Make sure domU doesn't enable INTx while enabling MSI-X. */ + if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) ) + pci_intx(pdev, false); + msix->masked = new_masked; msix->enabled = new_enabled; diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index c39fab4832..a50dad6968 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -87,6 +87,9 @@ struct vpci { } bars[PCI_HEADER_NORMAL_NR_BARS + 1]; /* At most 6 BARS + 1 expansion ROM BAR. */ + /* Guest (domU only) view of the PCI_COMMAND register. */ + uint16_t guest_cmd; + /* * Store whether the ROM enable bit is set (doesn't imply ROM BAR * is mapped into guest p2m) if there's a ROM BAR on the device. From patchwork Sat Dec 2 01:27:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D665C4167B for ; Sat, 2 Dec 2023 01:27:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645836.1008301 (Exim 4.92) (envelope-from ) id 1r9Emy-0003gY-LZ; Sat, 02 Dec 2023 01:27:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645836.1008301; Sat, 02 Dec 2023 01:27:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emy-0003eZ-2g; Sat, 02 Dec 2023 01:27:24 +0000 Received: by outflank-mailman (input) for mailman id 645836; Sat, 02 Dec 2023 01:27:22 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9Emv-000272-Rw for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:21 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ef9d9905-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:20 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGv3008549; Sat, 2 Dec 2023 01:27:15 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:15 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:10 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:10 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ef9d9905-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OvyuZRmdhw+QyvsQKkf4y618fBcDryLJKYr+YkBBxcpgN4e8nYCowiY3YaUfMPKQ4qK7KqEkJEl/5dXkCJuSKeUyVmfPNg86vFEXZg3pvxmKutREuq4oPVNISaHTmkVy0r/j60BkPYbVbBsQkOZpGsBpWcoPSpjxyUWnuAd1X84okxsb2ek54BLo6iaZk7VE4KMdmHMUYsrfnFL693bdhM3bnFk2uWPAg+SMiaUMkLzLyN94jl3d9vVsXU3J/B6fcfyIahbwCZlH9gcLZO6613bVgIo49tJu84uPnOBffkHFtjWw2tGYT+ehCqBS5On8dKe6zpCDyU+yQkyxNeyAPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nvwRYk1bAN4JyImy8c8Ch0HVDpPVv8XFVfUhLtLPGZo=; b=N0ghPJuqaihdkMY5ANVKJgaApu2BwpGeX+3PoSeA4iWcyLKu3kh41XcwKUoBQD+IBbAi+C7f/8JQ0Cu+WVAEmV1HQvbMlBpkLb6bq7o2DxSThzG44URwmbKG2LltzPEz5a0yNPpq8s3Ap5G/gClvqIzAtZ58FzJ3dc4Qhlk8suD9BY3tmjZNYUv262cd6cwj83Hk0hjo4Qb1nf/fmL1/PqQpqtdkLWZdz+PWMt6zRit30+bpLShWZdoU85zOAiGzD0zcPhs8RBWgEuNWhwD/qxmvAHkWp40TvoOnhIun6b1swTYpK2lGOOD/xy51suhTng0CoboeLVXREBMUisMniw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nvwRYk1bAN4JyImy8c8Ch0HVDpPVv8XFVfUhLtLPGZo=; b=rJWgxohtDRtBX7h1BrpXHOJmdg27x+FIkzuU/FO8vnYYogUJ0ubiOQ5eujW5PdvNRYvoZBqdYI71DiEf+eGEEo4R2rxnarCbC2Ph6P6rNbGEknkdSm6K2vf8Gi5C77e2CWbaKz3QtBD9DYimZc56CmpAK60WzSr+vfovCA2Ub6koynpBAL3UjRR9VGGrhrQEwTlCueF5HCjuX2gFwab4pQKuR2alcN0zlKd5PlloYi5LsD5DIQ3h6GcCKadJ8MNpgCVXbn+v/+opmC9xNUH2Lh5C46nbsQF87SPJqm9qMbSBsApvgMAgWDh3AyQyEH/o0P+7MG0V+w/irLIOb6EBcA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Volodymyr Babchuk Subject: [PATCH v11 13/17] vpci: add initial support for virtual PCI bus topology Thread-Topic: [PATCH v11 13/17] vpci: add initial support for virtual PCI bus topology Thread-Index: AQHaJL6puZ1COBY1lkyH/O/4FVoj/g== Date: Sat, 2 Dec 2023 01:27:06 +0000 Message-ID: <20231202012556.2012281-14-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: e9759de7-86f5-4474-9989-08dbf2d5ce14 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZOhTyIlegJlSD1VDS6nSnSk0omMBd+/gus4217HcRldMHDbaZI0MMFcaSFGKffzubV5PNFs7OP2RI2P8+JOoUGweDytdonDY+S5S/DXmdd7Yl68hQpwItp0p0ueZ/X3n3YZh1r0gTc40ElQPbtPiouNOXbC5SuzkycVsdH38bLXcPCmbJCKcr3BzGquKZCG6UYtUfvSGkIw4Fpoodz3wcIp7nwCgaULZb8sv26mT8YbtJQuo+CBOVqGJkE7FiN3PBqMJpv3YdZjIHepjnlOd9VOb914ia74GLWNW/8eaAICUNu48BCJwdYztqMV9B5yNX4z9zEBXRYbX7dviDNPtnA8DUkYaJP52kSuGPB1c9RRiLBTkJs0Z49rCKlPYUy0kiF2+FMZFEwAG92Fe7DKXtFD8qn+Kb6A0fZs86PaLqU8szf3BL1c236/ZbAEJ5te2EJNjwycWpnZYubdNCDvUX3HzIaG6LfVWceeHkBJc71nlwHboVxNhZUVrU4vRng/fkq494fVuezw0cghv0QDDtLrLBiz063cMSgDSEKFBwCZEYWAU1Ns2W8L8AQLIFp7cKPso0qU/XzTHAgMWRRc0mPMqxMtAeZT3nDK6AqLYBbPCy5DbN0qcGhlmyPeo20Jl x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?27JXlJA5kQiY1dOUVSa7gw5?= =?iso-8859-1?q?Ne8UFGK41GmuafqHi7DCJELezXt0CjgkTDa6lW8cXibcMdqbllSaaWE6/P9L?= =?iso-8859-1?q?MZLyyixCg4TlNKDc8a0+L57QIio17hYQRFrVmFHLonzF1n89dlh4tfHlQl4p?= =?iso-8859-1?q?P9DWwufDHWnfrzk/pR/enwOFPzWi1WTBBXy0emx1UZhZVzCzUDMZALxEKipF?= =?iso-8859-1?q?uc/W/1hNIkMe06iLw0QGmfkGN+AAcKyRu1gZ6fUIQ5V4wR1peTyMoDHGWbjF?= =?iso-8859-1?q?KyNvlI4hT7s7aSx+T8UalAfySBfE61UgCweeAY0mUH8UGFaFJ2B5Cs/Y27qP?= =?iso-8859-1?q?NlxUQzetZfEHQkaZfLtDFS+UDWoSfUdAQCAoK94c4n/CVmb9NJJhh5VJYQxp?= =?iso-8859-1?q?RI/fE2tZ1zovea5KUJqhYYAomGPPEWPk2qtoJaD1WPfqqi+PaFL7WGpHI832?= =?iso-8859-1?q?y5DEce8Xtwf3n64gobEeLDOYi/LNJVS+Rz1/amyUFK4M4e3vDELZGtk/iqFn?= =?iso-8859-1?q?BbL7lI8rZJEI2afIDEzOCx1qPikyRvAzNGY9ZUfEnBxGOBJgdMQyTOCLM0GE?= =?iso-8859-1?q?B6LgXa7k+GGT8a2/g+gDMiPMotFPdGN86i7VzM2xZQKzPUIw5v6Kq0XzQ67L?= =?iso-8859-1?q?0axDLzgZX5Glvd4ws/4QeZdOVuPcAAh++8FyQvR/X626og7Myg79PpKtUvbJ?= =?iso-8859-1?q?Fsxx0fePaeBEq3IXjbiEgSd+JiqoNMP0FEtbQB4UICiTc2YlFCDVqvdb4BgJ?= =?iso-8859-1?q?kGa2kzWaglGqdZoFxryC/Bqvo/+Y47Flv0Tg+Mps338I+ZF0NZ4AmcPyNTDU?= =?iso-8859-1?q?An7Q170OQ7W43mOyJzaXcKOBDtAbE7CkClWczMX7uj2q/+hDOvKbopBxtbsx?= =?iso-8859-1?q?h4ASRbeN54Ad/7xxq867SRpoNXgAG/GaXzh5TOHhnyvUTXCjpuWge2/RGOxz?= =?iso-8859-1?q?+oHm+seU2yvaHfNqJ1Gn/ck5Ta/E7zYmj5Rilx/skxnRCBvYDtcwcotE9Oxk?= =?iso-8859-1?q?Vif9YAee4Rx8+NqUF7Vw7W66H9G01T1U8WSA/lU2aF8JQZfm45vjpE+mySwn?= =?iso-8859-1?q?CRdFl9L0BtmWyEGTOxGd5soWzACKVcI/ENCoB2/XpOnZrH+4OHjJ1H4+KI5t?= =?iso-8859-1?q?RYQ2gCSw7qN4LlV9Tsluh0nubY8MKs5rIG6w+UB6GVN8Bdgm1hTizm85y7Mh?= =?iso-8859-1?q?eNcsU9Z9PRoMtqCRAZ4fNcbT/GnqwbZnuEPssDRlrbjfE0DO2ogkBf84O5lJ?= =?iso-8859-1?q?4AwULP9G7ewYQCdjAPK6VCPkCosqzeWZyvCJ2JtUwgSc5lsR6EMiXr36CUri?= =?iso-8859-1?q?q9JkUmZj6IIL1ByE5rb4Rj7LJW3/A6pwLGugMOsSQYha2JqqcifEH0qdBlHW?= =?iso-8859-1?q?DlGsBt7aHPGEGpw2AmQ0uTMlXLG+JH7HAfgrG53SVUXzTw7vk0ZTRB6CJ+be?= =?iso-8859-1?q?317fZFvgEBz/dJmtfgN4LtYP/aBinwwXib+AGsLlod0XVGZqGDq5rpq/tKnw?= =?iso-8859-1?q?ednD/Hmc8LxRuyoYDRJD5uNNnkf8dD0IRKhPLGWctNgYSbYcF7FDr3ThSF76?= =?iso-8859-1?q?SB24R9b/YkaaSF7JpU8U/GkebHal0uAnSQ2ogMXN2ltrX9nSeix3tBvusb2q?= =?iso-8859-1?q?N3n8kLc9mbnOJllPCyPkWn87BJ/dxrvv+22stBQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e9759de7-86f5-4474-9989-08dbf2d5ce14 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:06.2020 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yk+rGzhWAGivMeF6OYzcCrZ+u8ebLTEu2HIM/mvTsStgFPOKrL2vvuJEHH1bJtlgxlWjcv4uzZmsWX9euIg44NXC+egOBi0eOiqr6l7x1fA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: WTwQsomRv3Z7QJzM_Y3WrHyo2UHSYsYP X-Proofpoint-ORIG-GUID: WTwQsomRv3Z7QJzM_Y3WrHyo2UHSYsYP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko Assign SBDF to the PCI devices being passed through with bus 0. The resulting topology is where PCIe devices reside on the bus 0 of the root complex itself (embedded endpoints). This implementation is limited to 32 devices which are allowed on a single PCI bus. Please note, that at the moment only function 0 of a multifunction device can be passed through. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk --- In v11: - Fixed code formatting - Removed bogus write_unlock() call - Fixed type for new_dev_number In v10: - Removed ASSERT(pcidevs_locked()) - Removed redundant code (local sbdf variable, clearing sbdf during device removal, etc) - Added __maybe_unused attribute to "out:" label - Introduced HAS_VPCI_GUEST_SUPPORT Kconfig option, as this is the first patch where it is used (previously was in "vpci: add hooks for PCI device assign/de-assign") In v9: - Lock in add_virtual_device() replaced with ASSERT (thanks, Stewart) In v8: - Added write lock in add_virtual_device Since v6: - re-work wrt new locking scheme - OT: add ASSERT(pcidevs_write_locked()); to add_virtual_device() Since v5: - s/vpci_add_virtual_device/add_virtual_device and make it static - call add_virtual_device from vpci_assign_device and do not use REGISTER_VPCI_INIT machinery - add pcidevs_locked ASSERT - use DECLARE_BITMAP for vpci_dev_assigned_map Since v4: - moved and re-worked guest sbdf initializers - s/set_bit/__set_bit - s/clear_bit/__clear_bit - minor comment fix s/Virtual/Guest/ - added VPCI_MAX_VIRT_DEV constant (PCI_SLOT(~0) + 1) which will be used later for counting the number of MMIO handlers required for a guest (Julien) Since v3: - make use of VPCI_INIT - moved all new code to vpci.c which belongs to it - changed open-coded 31 to PCI_SLOT(~0) - added comments and code to reject multifunction devices with functions other than 0 - updated comment about vpci_dev_next and made it unsigned int - implement roll back in case of error while assigning/deassigning devices - s/dom%pd/%pd Since v2: - remove casts that are (a) malformed and (b) unnecessary - add new line for better readability - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI functions are now completely gated with this config - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT New in v2 --- xen/drivers/Kconfig | 4 +++ xen/drivers/vpci/vpci.c | 57 +++++++++++++++++++++++++++++++++++++++++ xen/include/xen/sched.h | 8 ++++++ xen/include/xen/vpci.h | 11 ++++++++ 4 files changed, 80 insertions(+) diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig index db94393f47..780490cf8e 100644 --- a/xen/drivers/Kconfig +++ b/xen/drivers/Kconfig @@ -15,4 +15,8 @@ source "drivers/video/Kconfig" config HAS_VPCI bool +config HAS_VPCI_GUEST_SUPPORT + bool + depends on HAS_VPCI + endmenu diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 8865c1580e..c92f2d7bc3 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -36,6 +36,49 @@ extern vpci_register_init_t *const __start_vpci_array[]; extern vpci_register_init_t *const __end_vpci_array[]; #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array) +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT +static int add_virtual_device(struct pci_dev *pdev) +{ + struct domain *d = pdev->domain; + unsigned int new_dev_number; + + if ( is_hardware_domain(d) ) + return 0; + + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); + + /* + * Each PCI bus supports 32 devices/slots at max or up to 256 when + * there are multi-function ones which are not yet supported. + */ + if ( pdev->info.is_extfn && !pdev->info.is_virtfn ) + { + gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n", + &pdev->sbdf); + return -EOPNOTSUPP; + } + new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map, + VPCI_MAX_VIRT_DEV); + if ( new_dev_number == VPCI_MAX_VIRT_DEV ) + return -ENOSPC; + + __set_bit(new_dev_number, &d->vpci_dev_assigned_map); + + /* + * Both segment and bus number are 0: + * - we emulate a single host bridge for the guest, e.g. segment 0 + * - with bus 0 the virtual devices are seen as embedded + * endpoints behind the root complex + * + * TODO: add support for multi-function devices. + */ + pdev->vpci->guest_sbdf = PCI_SBDF(0, 0, new_dev_number, 0); + + return 0; +} + +#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */ + void vpci_deassign_device(struct pci_dev *pdev) { unsigned int i; @@ -45,6 +88,12 @@ void vpci_deassign_device(struct pci_dev *pdev) if ( !has_vpci(pdev->domain) || !pdev->vpci ) return; +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT + if ( pdev->vpci->guest_sbdf.sbdf != ~0 ) + __clear_bit(pdev->vpci->guest_sbdf.dev, + &pdev->domain->vpci_dev_assigned_map); +#endif + spin_lock(&pdev->vpci->lock); while ( !list_empty(&pdev->vpci->handlers) ) { @@ -101,6 +150,13 @@ int vpci_assign_device(struct pci_dev *pdev) INIT_LIST_HEAD(&pdev->vpci->handlers); spin_lock_init(&pdev->vpci->lock); +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT + pdev->vpci->guest_sbdf.sbdf = ~0; + rc = add_virtual_device(pdev); + if ( rc ) + goto out; +#endif + for ( i = 0; i < NUM_VPCI_INIT; i++ ) { rc = __start_vpci_array[i](pdev); @@ -108,6 +164,7 @@ int vpci_assign_device(struct pci_dev *pdev) break; } + out: __maybe_unused; if ( rc ) vpci_deassign_device(pdev); diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 57391e74b6..84e608f670 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -462,6 +462,14 @@ struct domain #ifdef CONFIG_HAS_PCI struct list_head pdev_list; rwlock_t pci_lock; +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT + /* + * The bitmap which shows which device numbers are already used by the + * virtual PCI bus topology and is used to assign a unique SBDF to the + * next passed through virtual PCI device. + */ + DECLARE_BITMAP(vpci_dev_assigned_map, VPCI_MAX_VIRT_DEV); +#endif #endif #ifdef CONFIG_HAS_PASSTHROUGH diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index a50dad6968..6a06267ef2 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -21,6 +21,13 @@ typedef int vpci_register_init_t(struct pci_dev *dev); #define VPCI_ECAM_BDF(addr) (((addr) & 0x0ffff000) >> 12) +/* + * Maximum number of devices supported by the virtual bus topology: + * each PCI bus supports 32 devices/slots at max or up to 256 when + * there are multi-function ones which are not yet supported. + */ +#define VPCI_MAX_VIRT_DEV (PCI_SLOT(~0) + 1) + #define REGISTER_VPCI_INIT(x, p) \ static vpci_register_init_t *const x##_entry \ __used_section(".data.vpci." p) = x @@ -155,6 +162,10 @@ struct vpci { struct vpci_arch_msix_entry arch; } entries[]; } *msix; +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT + /* Guest SBDF of the device. */ + pci_sbdf_t guest_sbdf; +#endif #endif }; From patchwork Sat Dec 2 01:27:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8FA4C07E97 for ; Sat, 2 Dec 2023 01:27:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645843.1008370 (Exim 4.92) (envelope-from ) id 1r9En7-0005up-Nq; Sat, 02 Dec 2023 01:27:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645843.1008370; Sat, 02 Dec 2023 01:27:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En7-0005t8-Ef; Sat, 02 Dec 2023 01:27:33 +0000 Received: by outflank-mailman (input) for mailman id 645843; Sat, 02 Dec 2023 01:27:29 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En3-000271-C4 for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:29 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f3c1fae9-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:27 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWd004068; Sat, 2 Dec 2023 01:27:16 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:15 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:10 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:10 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f3c1fae9-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gcz2fapa26LFQAmG1I8hVtDYK9jALeWrZB4DOZdt0BzwvqgvCrF8Olrnm/zYHQJYdQavkfEKsjUl8nD69BlsGUmoRpz6fPaU8zJbI4LENuIxaQw2Y5FauB4qrDIN6+YAIK2rC3K255dwKplyuwO7DADfST4HjfrlOAsiAHKIWPIx3UW/y+SpkEJRajVshOAclC3uqNZ9XMhbZXaHdy7srIisr4EsTSpOsi1mYT7uBpvJMr4cluX8/3SZHfuUZTYuJeRUKXEleUs5uOwZGvCQEFTT2R52Vo4cL3RB40Bdi+mCmah3sz6iJEZniLfRVczHIx+I5L46JL8b21KObiLcQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PbQ7MUIQ2IkedJNUm+tfwIF8ti84wzB5RJbUQkThnyY=; b=ZOaHy6KqIy72HdGYkLM32Ielp1BvC4Tsm75c6fEywoLdFK+NRPTQUORF80D9Fo8aQoaNQvnOneKI7+Fa/wJnvST/UwY9T3CXObSI9qkPfMnklyCAd2Fk7INwYRCUVmTvr9n9/KNfI8VBCcfPB8gDk0S5QaJ+o71aBUbowbovUdLBz8V6w8hP+lm8OIXysqzfbPwO2FgqiEeXdiyHokoN5MfyIC4gB9hP3uqbcMCf0OZpP/muKaftGESsPFupathe3EMFOXvg7bJxCvrtQoH8dmrtaFQxqPBkLkp8McXFV7sOmdrx6TfrEQJ1dDrSjfO83xvsvGWBoIjH6ai/MBPg+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PbQ7MUIQ2IkedJNUm+tfwIF8ti84wzB5RJbUQkThnyY=; b=dVVclVNkDLNX0e+t0qzaDqyjPucNs51ccJWSxjvYg9EQDB+XZMTNc+GfPiKNyD9hzzZc14CReKXHxXFQ86vYj+j4wWWWviV3tfTUdG/7Soif2aDp99wFtPTWAVIT0o7IjNjbeRpt19v30Kqukz0HpQ1BQuxSlaj3na47W3sKi3jZjY+Iowl969AZQT9zccpd9v6dDww2Jkl37DR8P19/Zb/p2uDF2IsBDe2dQkfo0kVHCUFqaCKZzo/It4ZAfY8uApjgcIYfV92o23CR0mA3mfZ/8rrUbDa+9gRui71EaibxzV5XZkVgyvz5fHIeb2YewOoMPwFeRG84Z+opv657uw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , =?iso-8859-1?q?Roger_Pau_Mo?= =?iso-8859-1?q?nn=E9?= , Volodymyr Babchuk Subject: [PATCH v11 14/17] xen/arm: translate virtual PCI bus topology for guests Thread-Topic: [PATCH v11 14/17] xen/arm: translate virtual PCI bus topology for guests Thread-Index: AQHaJL6pGDTUzUqjcE21xonM3gZpjA== Date: Sat, 2 Dec 2023 01:27:06 +0000 Message-ID: <20231202012556.2012281-15-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 1b72d35d-ceaf-452c-688b-08dbf2d5ce5b x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: BhuAiKPS14c6DzpW49cSHpSn4xRJopWjveOMY2wYYQpBy/o+AL/sVgoiE/5U2Ii90JDQyG7KyBfOcAZrMSkU2IIRPOm8yFalVv1JXU421cBpoi7OAPIaCgPiLDutPbQDo4Dmot7UT0EBEYTP0MEfdMT+0tvkrbrrM1OZ36zPsBSSshvEpkVHPNk19aYjg3zaAGxGBSwiDAYZNAedf3SA8m9CsU3PCY1QAI7e6knB6A+LFLMLUvoNkyiisZKoUC4TmX0zyrabVUqDnT3ZX1PKUZa4qNecPMoJVyLN/XJJU2tXuEmt9JijJq8u5EpoOGQwJdiLaYOqgHdgkmOlFtlFB3W43fw1bQFHTWCjP/dnOlDK2iDVkAEOQtcJ7GZAJBHC3siwr/Da43Cx0EUg4t+bkr7ZrHg+VhsHOqayryidOrBOjlQrpxM46ptM36p3o/0tAD1+CfIU8+E/iCL/Lm1BKyhbHfAMRumhJSe6K89ob/3QmsQS4pOpCYgY8Bf9wH8uMn+uhO64shFUTcQtLfkqg7ceSSzROjQ2gdD5U8DFxS6A7zw5Wh9x92DSfbs2hCCG+M3GGj4V12dL/OASQ2xyaV9ZdD6XG5P+uvOXTJJ2Se28L3Dij11DzfOlqtWZOGHg x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?it+ETn2n+fRtJW4vDkVaBTA?= =?iso-8859-1?q?vTPCMdx8wT7Im1tREvpvA+6LEeo3rDWyhvLqFh4FhFrotTRr5bIZrsKThjtt?= =?iso-8859-1?q?wpYKiXCrUzZr/aJxDZmi3QMW0+UuN4c1np3DDmxO09TEuiHR6MwM8f9UqRhK?= =?iso-8859-1?q?NYJgH1r78+OXfklKSBsYP+0z597I+TkUfNM6r0CFlSjF6Y6t3kIdR3Sw4PF5?= =?iso-8859-1?q?MnSrpa2sUoWMOzWYY+eCm4FraacxcoMRW38Wxo+KxIizRyrnQtmrfGTd50v6?= =?iso-8859-1?q?4pdPrF8oj9tjGQ/8hU5N8m6rD7uSI1PoBa9YpfLyaVFJshfcnlQD9cI7w2Qq?= =?iso-8859-1?q?6lwsqfodtme6UaZ1j4ZDdKLfaHKd+tDxyXq6DtU/ed20Fijv0nwyVG2GUNfb?= =?iso-8859-1?q?jS4WRdz4KctUW2cpOKQ99hcK5eBKTLJLciahnBPHlW6SvtBKPo/QpCp0grEP?= =?iso-8859-1?q?BThOr50dbYqavtUTxU6TyElT/ABzb1fRP7S/AwJZTHu5bXH5D+BRsTiJnxzf?= =?iso-8859-1?q?Qus3cUJckkE/pAhchi/VQa4XwpJmK+qagYXFZjU7pJGwr0+qrWJug7Ok/iNh?= =?iso-8859-1?q?1AzjrI/iZueUWoZxoLjFKgh5lFVaelyKAa8vl8ThUOHjyio4E132SWCgtzfh?= =?iso-8859-1?q?+YeIKuBa5dWd/qIAah1fVtcy5jEcMHma1K6tG6ZayJQiKA0iD0vKDjJxqNVh?= =?iso-8859-1?q?O9ZWZbjlLBRrlN1qWJPFP4OVXJgjqUO0fOgbqnOIvFAsX0wA6sXBm5PFzbMC?= =?iso-8859-1?q?ezZ9GW/kUbs9GsgSx/XB8P8U5HRLufPdxbxkBZGhxIQSsWinSuAKS9bgzMgO?= =?iso-8859-1?q?vcYU2Qu4vu5ofBi0olre+fcSos+fkR0hYtmNLJu18XDZwruv4l4Rwj2d/eW1?= =?iso-8859-1?q?pemhi3OLO0QyItrcKYdDGRybJ91zM9jD/b3TMn/gtcmIpFuuYnZZKp7DjuwZ?= =?iso-8859-1?q?CXmSUETs1zW7W2OD4FWzTgsV1h4f1Zvnwntayl2HbKde54eC6uY7Fy9yOjLx?= =?iso-8859-1?q?2CwB80M/2JaX/F1faZacsNs4EcvZsjxEGmlRX7QSy2IQOUi22RXw9bjotFxF?= =?iso-8859-1?q?L6UQmMMgaAMgqF5L7QqRHGQKzXPodyrZEgIAdGiCUrrtL65Dfxh1jh01qMxT?= =?iso-8859-1?q?Uu3IzqyG/hVrxE0KOHPQC/5cf3srHX09O7G0Pd07nZBnw943KPOVQuTMbYY6?= =?iso-8859-1?q?HbVFtFHPWjjArLtR4sISfakAoBC7EAMGCRsHCtWvkhZpaXn6sTNDxUzuucUX?= =?iso-8859-1?q?0e/qAQxtz1aiJRgrD4Bl+KX65/Iq8zItNyLp3OWOHxIEWsFp0wPGEFuA716A?= =?iso-8859-1?q?RwBUfAIz5uxUDYciNPvBHehkuDvCzQfwCbzmpCC+gOdR6+0IUsNG87ce8ZFH?= =?iso-8859-1?q?Ieh8r7CeM0FkjMRWc0tXO2FVRtMhBP7lk5On6U/kMLL28Mh9NIpWP9whH+5I?= =?iso-8859-1?q?2/QU+ge99gxSfHKwz44lNCqngsvi5eHUhxdcX517gwm3cub0vVZ1sMq9ZFRd?= =?iso-8859-1?q?fbztoJBJEUjn0Hc3LySJRI6uZDatSpbwMRDwM1r8+xtBgOmSDP8mhFYEa0nW?= =?iso-8859-1?q?eJdvXlt82kss3PI8jBvPgYLaMUJ7x1EtUt4VCGd+JCFtEG6amar+ZTTg+mq7?= =?iso-8859-1?q?RmQnAG6GIBqp+P/kUk3zr7aKsyte+XYE7F3tNwA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1b72d35d-ceaf-452c-688b-08dbf2d5ce5b X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:06.4593 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: sfLpyYgFxmli5SNiqjaGvaNs4tATJklytmbXrCI3WYMkzJHot/fv8gf8IOpIbBOnuA8lWAGCafTQmNIcLvjAW3tMQIujyGTIFt30dcphYjQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: HMiYs3pK8hazm7EO0Zwij9s3iT339YZz X-Proofpoint-GUID: HMiYs3pK8hazm7EO0Zwij9s3iT339YZz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko There are three originators for the PCI configuration space access: 1. The domain that owns physical host bridge: MMIO handlers are there so we can update vPCI register handlers with the values written by the hardware domain, e.g. physical view of the registers vs guest's view on the configuration space. 2. Guest access to the passed through PCI devices: we need to properly map virtual bus topology to the physical one, e.g. pass the configuration space access to the corresponding physical devices. 3. Emulated host PCI bridge access. It doesn't exist in the physical topology, e.g. it can't be mapped to some physical host bridge. So, all access to the host bridge itself needs to be trapped and emulated. Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Volodymyr Babchuk --- In v11: - Fixed format issues - Added ASSERT_UNREACHABLE() to the dummy implementation of vpci_translate_virtual_device() - Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow the logic in the function Since v9: - Commend about required lock replaced with ASSERT() - Style fixes - call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa Since v8: - locks moved out of vpci_translate_virtual_device() Since v6: - add pcidevs locking to vpci_translate_virtual_device - update wrt to the new locking scheme Since v5: - add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT case to simplify ifdefery - add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device - reset output register on failed virtual SBDF translation Since v4: - indentation fixes - constify struct domain - updated commit message - updates to the new locking scheme (pdev->vpci_lock) Since v3: - revisit locking - move code to vpci.c Since v2: - pass struct domain instead of struct vcpu - constify arguments where possible - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT New in v2 --- xen/arch/arm/vpci.c | 47 +++++++++++++++++++++++++++++++---------- xen/drivers/vpci/vpci.c | 24 +++++++++++++++++++++ xen/include/xen/vpci.h | 12 +++++++++++ 3 files changed, 72 insertions(+), 11 deletions(-) diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index 3bc4bb5508..7a6a0017d1 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -7,31 +7,51 @@ #include -static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge, - paddr_t gpa) +static bool vpci_sbdf_from_gpa(struct domain *d, + const struct pci_host_bridge *bridge, + paddr_t gpa, pci_sbdf_t *sbdf) { - pci_sbdf_t sbdf; + bool translated = true; + + ASSERT(sbdf); if ( bridge ) { - sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr); - sbdf.seg = bridge->segment; - sbdf.bus += bridge->cfg->busn_start; + sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr); + sbdf->seg = bridge->segment; + sbdf->bus += bridge->cfg->busn_start; } else - sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE); + { + /* + * For the passed through devices we need to map their virtual SBDF + * to the physical PCI device being passed through. + */ + sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE); + read_lock(&d->pci_lock); + translated = vpci_translate_virtual_device(d, sbdf); + read_unlock(&d->pci_lock); + } - return sbdf; + return translated; } static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, register_t *r, void *p) { struct pci_host_bridge *bridge = p; - pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa); + pci_sbdf_t sbdf; /* data is needed to prevent a pointer cast on 32bit */ unsigned long data; + ASSERT(!bridge == !is_hardware_domain(v->domain)); + + if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) ) + { + *r = ~0UL; + return 1; + } + if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa), 1U << info->dabt.size, &data) ) { @@ -39,7 +59,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, return 1; } - *r = ~0ul; + *r = ~0UL; return 0; } @@ -48,7 +68,12 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, register_t r, void *p) { struct pci_host_bridge *bridge = p; - pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa); + pci_sbdf_t sbdf; + + ASSERT(!bridge == !is_hardware_domain(v->domain)); + + if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) ) + return 1; return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa), 1U << info->dabt.size, r); diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index c92f2d7bc3..7c0b610ccc 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -77,6 +77,30 @@ static int add_virtual_device(struct pci_dev *pdev) return 0; } +/* + * Find the physical device which is mapped to the virtual device + * and translate virtual SBDF to the physical one. + */ +bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf) +{ + const struct pci_dev *pdev; + + ASSERT(!is_hardware_domain(d)); + ASSERT(rw_is_locked(&d->pci_lock)); + + for_each_pdev ( d, pdev ) + { + if ( pdev->vpci && (pdev->vpci->guest_sbdf.sbdf == sbdf->sbdf) ) + { + /* Replace guest SBDF with the physical one. */ + *sbdf = pdev->sbdf; + return true; + } + } + + return false; +} + #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */ void vpci_deassign_device(struct pci_dev *pdev) diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 6a06267ef2..ec6598237b 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -294,6 +294,18 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v) } #endif +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT +bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf); +#else +static inline bool vpci_translate_virtual_device(const struct domain *d, + pci_sbdf_t *sbdf) +{ + ASSERT_UNREACHABLE(); + + return false; +} +#endif + #endif /* From patchwork Sat Dec 2 01:27:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF423C10DC1 for ; Sat, 2 Dec 2023 01:27:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645841.1008350 (Exim 4.92) (envelope-from ) id 1r9En4-0005Am-GZ; Sat, 02 Dec 2023 01:27:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645841.1008350; Sat, 02 Dec 2023 01:27:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En3-00058M-Sz; Sat, 02 Dec 2023 01:27:29 +0000 Received: by outflank-mailman (input) for mailman id 645841; Sat, 02 Dec 2023 01:27:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En1-000272-K5 for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:27 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f243ff3b-90b1-11ee-98e5-6d05b1d4d9a1; Sat, 02 Dec 2023 02:27:25 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGv4008549; Sat, 2 Dec 2023 01:27:16 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-10 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:15 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:11 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:11 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f243ff3b-90b1-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eB4bd+G5qUD75eQXMR/lOkFVQEOzoQ5eKGuoPzBsWir2qx7GHtukzxUwADNZtGDG+UmF+FWK3daHfJCuXfn+HiseQ3n/0EU4P3dIyeL95VoL+gGwgmhpyjEvW8k2W6hUdcK5O12/hjqewdi3m/tnn4Lix+WLlNF5wG3o35s4QZ3p93TKqBzhqjsCo7tIvw0aZLKlXoEzanULa5wXnvkJ8hWGwtCrbjDdzjXo/zRrYHk4jC7KsYK1NByJ9vKnmgFzh/no3/yg23U97acQWh92pYwihksmVWeQ94zBo3oqxVB34Df5MQKvohw7lUuEf6PDWuNkIkN2q5MFYdMkVWXAEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=22ltegJ1p7Qot1PJ9ShhvDH8ZaJL6Qo1XKi3VZAv5vo=; b=a00GlAtIWWE1HPHzPyWKQHaqSTEc+DeDbtSdWUpYNPgofTixJMAKF8n+qYpoEU2keqQRPcJDjzxMjMwTvWM5xt6n3CREwQ6VwP4u6bxdo7aovD9XL+3WvVxYQXueLR7Yxx3fSbsqSkQ3VrJ5PdAM8c0sEUHoFLK0GDixgnAVjmY7fLwCx3YArYMW7wqFZeQcBYIKD19/txSUFb2zndPb2yHXLyBwpmM/+yETGH0T0ijpH4OKAOr0VknWytxGMBUqVr/uZAnipjm+7dEkjU9FGs5bOWoWdq8An6OqAqDvSDP4jg+biCWZeOHkDorGMirvBDUPX/cPELv2wRMJ1RJIRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=22ltegJ1p7Qot1PJ9ShhvDH8ZaJL6Qo1XKi3VZAv5vo=; b=K/a4cXricBvqbwuH5aFCGNR/d5bLAS5NHDy/mULx7lwCWK3aluRhm9ZlQFRcceEWBKrfjz5VTbx4bbDAc9xYhLEsADL8HnLJDUiMWTuAkgY9YkSP1sceVTAVdkiiNlh97ma1RWvEmU7txdVVcou9/XCgG0GjTyjBGCBaUPQpszD0XspmI+ifgRUaLWoYcYYQ35Hn6oTidY20DAW0fYfJf8fQMln2G9TZF+lyD2GjptWDUTyJLubJCiIrhcqYZg5xsMgxkdl1jrCdHX8XgG/4JDoI2NlJWHQoRaEc9xZXtQdBQaH7+YDgGTvU0Lf0wOMX1mxyq/UHMNi4KSKJgq/5bA== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Oleksandr Andrushchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Julien Grall , Volodymyr Babchuk Subject: [PATCH v11 15/17] xen/arm: account IO handlers for emulated PCI MSI-X Thread-Topic: [PATCH v11 15/17] xen/arm: account IO handlers for emulated PCI MSI-X Thread-Index: AQHaJL6p9eGZ4W17RUGnV1K8Sgg+4w== Date: Sat, 2 Dec 2023 01:27:07 +0000 Message-ID: <20231202012556.2012281-16-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 28609c78-400e-4876-6c4d-08dbf2d5ce9b x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: BLmX9YE5PDVxL2tS9RtMDUPBEInubQi8eTN8Rlw2p83ZtYKxLnh2rtpIteDIG+Y+ACXeBs7Bg3pFI1Tq19P43g4TIiBHzCBxixmo5cCOcB8Zi83N0FL+jzsNRJQ7sVvrXKsbgItdmx+/L/L6620yU9l2SqJVTckbvut+QVKfU2+b7rJoUdMVkilszsfwXiOcP2F4yXkcHcVYGOQQwAMBOW9wrlFj0Eqt//CCorAvSMe/tk8MFZ9D6jOEvWaHWYEaEvW46zt68yVIIgj9L7C7Vv36Oy4d7eKlVfwqFoVseE2SfFUzdQ/bBXS0vupi6B2wrjiRKTpzYaACYiHczuvWYrcDLULZHBNzetQMQE+rLFSKRzuuq7xzvJqdJKcRittxxeP8gMBfktbEp4DldqYA7ebsxxS9m4ayVAjWZHsuEFNsDQGJM0ZM/D5nMprjMi9ld3vN1uSSI+3pDvjTt8HMZbLTj5v9qU9Wsm9Ef/84XNkyddLFI677Q7RXt5fwxOXQ4N/KA0QDDooU1qKUoHzmMh1BrEAdvipEeLLeD5BAybRJUOkbrR/nvK+OJAxydqiM2/ASMl6ejoUuhic6LOqdCZ05H/937BGV68knZij64BCvkDOsIKCJToRfBirJlR8T x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(66899024)(4326008)(8676002)(8936002)(2906002)(15650500001)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?WcsZYxn3QwWqbDQbQvvMMAJ?= =?iso-8859-1?q?1lCmdikKYzToszdgjOzmLZjUnckXb6iLpUJoXiYOChFeQDTELUO0CpM6Khe1?= =?iso-8859-1?q?wflXa4MnQP+KsM0/f5T/FM/1GUtsR4Ohry9DciP7fahj0jTk/ufRG6vyldkU?= =?iso-8859-1?q?e/DcBJaI5PG1vVrz0ESoXGS0V/wQKE+emF1NuKORw+c29GGMPTJ7+zcoXt1G?= =?iso-8859-1?q?imp7Eku4Kel8JukEs3aiJoAP0Gt3kXw5zjgyVnoWgca689XjdiRh3lUjovBz?= =?iso-8859-1?q?eZsThtDLAhi3OXyO+Apl8veoa6/LV6l0r/h4PzJ/NOwwPNdYDMz4/izur1yg?= =?iso-8859-1?q?XFbNs0nAeZ2xOL6LhK8rYA00eoUVxZZxjeL7UdZXxUS897vFM80w5jmZ9FQF?= =?iso-8859-1?q?uNSz33yTpfIe3X/3C5VYq95UCXwgw0eAyawGkdqVAX601ahSjUCb54+hpmZd?= =?iso-8859-1?q?FQhBvLRi38a/G3CKDeTMA/vH9AG/ND0x3+WYFMZUf+m59zO1+fZZvsk/LrUa?= =?iso-8859-1?q?MK/+YgghuE4/LE8qPdDRDeEZkXlvYHCy/DNuxDx4JUFEekfYX2QesTHNbuqQ?= =?iso-8859-1?q?iTlYPGWTfMgXt558VfaEnFz118vAeUJrvGwParkv57f8QeSieGnd/u8XqLSi?= =?iso-8859-1?q?WsfHx6NtgnL4NbSjTXijVv1MIfGo0yBeVlTQgowInLW0i1o9kLm+YGchogxF?= =?iso-8859-1?q?a8WZc5xpvI0yFuTRviuh9MtXTt+YEL3WlenVkoFkVeg5kK+oRk0rS+kv/NWJ?= =?iso-8859-1?q?Ob+mL9EEKEMbqdACHPaDYrUbTaTqXJkv38O4te7MpQEREvkqsmzmM1Ud5hyc?= =?iso-8859-1?q?4bm8Q+s4JkzevR4hzlOI7v3CExz1+9xFVCdwkWYk10NOb8oFikg/NRAmMs4o?= =?iso-8859-1?q?SItszcLbiUaKFh5EmFP3Rys57+FfLyJDPRiJG1I8hprfLTysReN+sjiQZunv?= =?iso-8859-1?q?82wH9QfsPORBEoz7DgAFuVRbdeDoaNXgiU3zh7r6X+q1q0VY8N8I3Aa/7fWQ?= =?iso-8859-1?q?Mr3VJA6eOJEzCFijUrmQmHpdUY0nqkhtctSbXt8Po1XikR/MWhCoZGv9rihP?= =?iso-8859-1?q?uZ5Rn9AB7b+Z9CjeDl+X8Y1d2DwgZhBR2DfFZCSd752vj3khwbLHxPiP+PLR?= =?iso-8859-1?q?0+DtWKBkIe1JgLlNGHVYebwbRtlvHLBGGbWtOes6xbAlaWNWIXkz+lh83EbB?= =?iso-8859-1?q?896mPwPgguej4yv1da2MqA1DG/wI/X3JAXFpTzkT+nXsvu1teyN97VmSotWk?= =?iso-8859-1?q?vAJgEuiXZ5E55DHvULJ06mgs6L9dTIFcw/UjAKBNGF8VkRwEGCE9zhaje3O+?= =?iso-8859-1?q?//cGXimtGJRRLW1AMxJdTrWhGZnWDWwMSAGnQvyA/Hndiuf6M3AKCYX4wj1E?= =?iso-8859-1?q?pXBkQ+Sb/CFKp4Wq9d6oJ4NKDTrOzJgKHy57iI9GPJDbwdLCxjzrBGPJqOgW?= =?iso-8859-1?q?dzr4PcMXeQ3cYJBRPKn0P8C49FS3PgnsIYsQxUTPeHWv57Fw/Ml9yiqZlknU?= =?iso-8859-1?q?XIzRvRkkZ2iv3gxacwcRBNDScRy+5gjSJp1YGZp5p2c1++ouDNyZqLIOdDqa?= =?iso-8859-1?q?3F/sxhWkGGMLqHZ7oLKFZp/I2V9z3F2kkBHkfCar2fKXCeDVHoSrD5LRyV89?= =?iso-8859-1?q?be/nSPgAPQlP5AgzqlDP/G2pN+ens9B2vv53Pyw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 28609c78-400e-4876-6c4d-08dbf2d5ce9b X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:07.1866 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8HnUlaeCbebl0Qk5tIhOKyMGqB2hKDCA1eLKpRlYNDuftGtfHM3s6XMSTXqQ/DBw1OOzmaJtvTyCqwk63/wz8UsLYpwto+Ac2y+VjuqoHus= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: _pKyBxUJvXrRlOj-gNFJXZ5EHgRpb6yz X-Proofpoint-ORIG-GUID: _pKyBxUJvXrRlOj-gNFJXZ5EHgRpb6yz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1011 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=404 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Oleksandr Andrushchenko At the moment, we always allocate an extra 16 slots for IO handlers (see MAX_IO_HANDLER). So while adding IO trap handlers for the emulated MSI-X registers we need to explicitly tell that we have additional IO handlers, so those are accounted. Signed-off-by: Oleksandr Andrushchenko Acked-by: Julien Grall Signed-off-by: Volodymyr Babchuk --- This actually moved here from the part 2 of the prep work for PCI passthrough on Arm as it seems to be the proper place for it. Since v5: - optimize with IS_ENABLED(CONFIG_HAS_PCI_MSI) since VPCI_MAX_VIRT_DEV is defined unconditionally New in v5 --- xen/arch/arm/vpci.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index 7a6a0017d1..348ba0fbc8 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -130,6 +130,8 @@ static int vpci_get_num_handlers_cb(struct domain *d, unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d) { + unsigned int count; + if ( !has_vpci(d) ) return 0; @@ -150,7 +152,17 @@ unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d) * For guests each host bridge requires one region to cover the * configuration space. At the moment, we only expose a single host bridge. */ - return 1; + count = 1; + + /* + * There's a single MSI-X MMIO handler that deals with both PBA + * and MSI-X tables per each PCI device being passed through. + * Maximum number of emulated virtual devices is VPCI_MAX_VIRT_DEV. + */ + if ( IS_ENABLED(CONFIG_HAS_PCI_MSI) ) + count += VPCI_MAX_VIRT_DEV; + + return count; } /* From patchwork Sat Dec 2 01:27:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CD954C46CA0 for ; Sat, 2 Dec 2023 01:27:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645839.1008338 (Exim 4.92) (envelope-from ) id 1r9En2-0004vH-U0; Sat, 02 Dec 2023 01:27:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645839.1008338; Sat, 02 Dec 2023 01:27:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En2-0004s8-Is; Sat, 02 Dec 2023 01:27:28 +0000 Received: by outflank-mailman (input) for mailman id 645839; Sat, 02 Dec 2023 01:27:26 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En0-000271-FY for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:26 +0000 Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f1d96726-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:24 +0100 (CET) Received: from pps.filterd (m0174678.ppops.net [127.0.0.1]) by mx0a-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B1NLGv5008549; Sat, 2 Dec 2023 01:27:17 GMT Received: from eur03-dba-obe.outbound.protection.outlook.com (mail-dbaeur03lp2169.outbound.protection.outlook.com [104.47.51.169]) by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3uqfjgj05q-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:16 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:11 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:11 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f1d96726-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LqwzjfVbpuSbADigDKB3q6s/+4Ysl4kv9K1b1mlW2jPLkPg+2IdgeCtSwFrGA788BBfs82E/2RoEUefWNFdczzKy+K42djELnHHrMAc8AOfalgfPhiH4dl0XvfxT8mGIHn32uQtdHNWcpAqp9ivhwu/jSyI+X+7UzNVsqjnseVWe9dPLAEr+yJA68ljINOnq4/prp80gAZbXt3LzyMQwEjkw7DKXHSXgN+7QabEu6khWfU1vOjHAJ208go4yg1ZfEnn7ughdCpYQcCqDOIo3lXYVsRSjzRI8q3OPPslOPOnHQ4vIjM/lDPs1pmae7EplKrPSnUBiprkIUCr8Juxncw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hdZ0jIGkzRKTDO0d37Jm8MydN5O2zv8obceLXDno5fI=; b=NkHMRIbbbIG/oXJff0ijwMmlAED6egLPtwuxyZGI9PYvf0zx9LzE3iLDOBfzF8njxAW1CJiOXSX25nbJX9eZd3/kjeV/Pfk7n77fe3Rz4/gaK1LOGyemDGq2UEoa4CdeGur/LcNbCd2ZdyQNfELfbnnEoz97wkPReLfNmPcTxIDT2OO9wQkrmUHB1/eGKycafJ6fRDp18bmyRD+YSEGRGCP4BHWmUEkph7KAjaZvZvkn91Oj1hNp5ohetTzPNnOqI36exYGZ0DcTAEgxTR3K8TsyErHbF0/HElwTm/pSxPjw00/PgitWs2hGgJm6yIl3xK9SMXOcCcopL9eIuHz4/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hdZ0jIGkzRKTDO0d37Jm8MydN5O2zv8obceLXDno5fI=; b=IYfK9x9TGxqAs0x86VS3DmPrsSwIYQOJZqDDAZTlLMexzplhUksPdYI1ANszeXdn4fgKvWxIMqm8w7B71emdWTIlxoic33WQuQ/ZABaP6tmD6u2MMJIf3RUsIQc9CYbqXUJnO0wq9rHarGF7abN12th4frc+8fWEYdPEQ5tNHvPhr8JbIS+WgRHSU6TgQKDpa4ikt8J34od1UTMnO7ZcAqWbZo1KsWRolj9z4+WblpBrp+4BfQw3Mqhc4O8azGzZPd2Pra+XdGWULjhn+1koE/OyNR8ICIYLGQ1m3kuITpFY6AeAbvSmdKFNDe+05hrUuv78GbgOnjYa3Ko3sLYZlQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu , Volodymyr Babchuk Subject: [PATCH v11 16/17] xen/arm: vpci: permit access to guest vpci space Thread-Topic: [PATCH v11 16/17] xen/arm: vpci: permit access to guest vpci space Thread-Index: AQHaJL6psXyo5g/AOESuTV51rbTuPw== Date: Sat, 2 Dec 2023 01:27:07 +0000 Message-ID: <20231202012556.2012281-17-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: f88aa59d-d7ae-41e0-9770-08dbf2d5ced5 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: TFoblPe7cdkIG+CF31m0fHdzEKcUHB+nVJ4ql8yuHAwHWdKtng4Kg2gTA76H2qj3cEsLbbOGB/UW9Z3sRtwnoEUruR5AhrsDLe9+9SOvDvcheCCQW4IVcOdWB7WQPpRPCTbEgNf21/djYPe1i9bQ1Bg6xTJiewxQ/2OkG3GIqbY0souisgtiWE1Ng0V+Asn/GDuU8l3szdzJ/62YrfhViU2yA8vVdc2ySY2O9OKaAbzmR2hh3hnb83L5y6y6hNTSoFCvAbGkOjLAjf+B3SPJoj3ATLKFvU7vxOSHgWkQDlggiOkRT4bd6HZ4/i9eVWCB6/0iWuo9I6gt6aCgWPH6GjcqpovBNH+4tlMsq3Na0nz7MzcMtNkY2XaxC/rnntKechf1NPHqPAxLlqhWvyKOUUaA6HSwIGdkNuVm4GPHPX0MsKwjd7NnxCNhxG0EU0R+roc953jRjLG/vSZf2c9zI+ZJrOSEaQRCgnybRDyAnJ3m05wxeASEJEA6LH9tYdRPJnEGf8hqedy3OP4+pnTuTP2lcwA08J1kg8VMQ3fu0i391a0euXqQv3lk57dVFfJU4xdjniPmiwNwH9azBJfb0corzopa6lpmYf85LXQHILSksicvD1pp4/wDi95b2jZg x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(7416002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?Fwz1+ysKbbDmT8z+qXOD4Tr?= =?iso-8859-1?q?kiQkMe6nSowDqR1CBUNB1L+DDZvhterRx7zYuqMTQZmh0xf12dFrY3mtoAkY?= =?iso-8859-1?q?0X4ewf5yBtryKNjlrnqk7GV7d9NZ504X637NwU+/fZmbMEIwqAEh7NzHMGe3?= =?iso-8859-1?q?oCWuWlaHyULD72+axK4S7kliXVvGdrpwogUT9KTQKLgWh2MhJTuggLEj+faB?= =?iso-8859-1?q?TxqN1q7mw1gC5bLSJ9AS7mDFgFg0cP5yn+rfJ07+T5m7aaBkoabIdsDJn6ih?= =?iso-8859-1?q?qcLJ//28k34/OHI2eOQad7ra7vK/FJRtFWKS/XRUTwczo1TdisijcDAmUcvd?= =?iso-8859-1?q?3PEK9V2iur7NJJeAG4VmeAFguTbe1WDJ+Acab76ke3M5F/ODUdOnreDxkahu?= =?iso-8859-1?q?ZEz5ekEzsv2GX+rncnlmwGD9I+RgfmC+dGy3grbMOK5B15AjR2FjxLHDyNZd?= =?iso-8859-1?q?YeNfrMWo7g9b2jUn1IOUYsVfpnkZ+dlRTVN/CSSS6Etlp7SKvAstJfumFBT4?= =?iso-8859-1?q?H7N9L/9QQDIJRhjE2vsthOWQQxeRvhhfAoG9SFbQMfPnR+0sGfw6a9sxp9Mc?= =?iso-8859-1?q?OAjI+I8YX8SgyH6VLhpHK0h7dg6+osooGGpyXWBQ7aLwQ/SIi+OHsz2LeTHj?= =?iso-8859-1?q?tEEtF4ddjhI19yQvjvG4rMw/qhXOW/tXUYtnG4sooz5aQyZz4ZNOf08gnBR5?= =?iso-8859-1?q?Qb8f3XzzVw1X9wtGcR7z04YC+s66FkYV3myf4Rn1/zvb/1iMaTozIfEblR6v?= =?iso-8859-1?q?qfwdRN/Y5YNhIaILCjepKzNfE0sTlOa2yC05A2hXAUzXu2NwRMf6sAyohZ4c?= =?iso-8859-1?q?Xf8a3vAuChoeg1w4gPeiB8f77rYgIij6STtgonRCjF5QhLI4e3wMNQfGmyLQ?= =?iso-8859-1?q?q+/uikJ4jjCQYvtZIrp8vFM6yAG2J2RaFDOIrv39r9WyG6xzLQd6O5YT1v9L?= =?iso-8859-1?q?o5s6ySq/Hj+ONlirl35IyjrQolHKuw4LucV0ZyBGvJzwvKXepRqgnnmyg4zI?= =?iso-8859-1?q?6UFg4vdvuktU13u13TU8TGOWmC0AWsqqopqI8qQ1Yb70XiTq06W4qZE6jWDI?= =?iso-8859-1?q?doKWIr+6ACaPu5aJy+bFow8O7gQ6vItr2xPMY3FltAR8K1Jsq+d4xhZo66WR?= =?iso-8859-1?q?aKgBDaDmf3a1JzXU3KnMQBdyCt17Ov+XkD2813LMw3JfFC284B/UrGuesJsj?= =?iso-8859-1?q?+UE2i1QrL4J9awiMpPGkNO6i2sNi5w2WJOK4fFK2B0HN62S2fklXNoS+H2xh?= =?iso-8859-1?q?2Bt+IWQmMKFrfm5uvUe+w2fLZu+qIGpR6SbA7VEYi/ygU6nfIol5xNwdR0KX?= =?iso-8859-1?q?A27OtCQqtne/T2LV+4xzttShbrDeGKTCh2PEzEwPDoKfUp1QjSFhpZ3YelmN?= =?iso-8859-1?q?Cb2tW5NH3JzsIbKFFsE0rsVDUL6wpKqdblmcNVWmBjcADjzeeZhCJQitSf8A?= =?iso-8859-1?q?AKYmi0zYAt+3//PcZRM6ZAul7tieAQFRsJ8Mhk4/tEZet9ruPd2C53fclkBY?= =?iso-8859-1?q?shV/ivF43r5q7t2YW2NNy96wETSpPCKHPcdS5SMpLlnj24efjxHdDcJS97nT?= =?iso-8859-1?q?rC4Lqc4eea9/9NNqqLRhELM/qgmwNDRnks/YIfxP2U6hSJZmNpKWbitxSiZX?= =?iso-8859-1?q?g5es7uohUyF4RhspcHVsbw3ydcE1/llW1RHlr4Q=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f88aa59d-d7ae-41e0-9770-08dbf2d5ced5 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:07.4675 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: vCsHc3wz1+dVzVYuVNuZ0AbUHPSC500soctpl6D42mEKaUB9cvY8P+fsB3XyrZL1ksJLz6jMJw9PLHQpi39doupGc6Nh6CNMGpTe24oo/Bg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-GUID: dGrDLNxqCYl5a5Kf7fTKg2uqQU53M4Ue X-Proofpoint-ORIG-GUID: dGrDLNxqCYl5a5Kf7fTKg2uqQU53M4Ue X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 suspectscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=720 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 From: Stewart Hildebrand Move iomem_caps initialization earlier (before arch_domain_create()). Signed-off-by: Stewart Hildebrand Signed-off-by: Volodymyr Babchuk --- Changes in v10: * fix off-by-one * also permit access to GUEST_VPCI_PREFETCH_MEM_ADDR Changes in v9: * new patch This is sort of a follow-up to: baa6ea700386 ("vpci: add permission checks to map_range()") I don't believe we need a fixes tag since this depends on the vPCI p2m BAR patches. --- xen/arch/arm/vpci.c | 9 +++++++++ xen/common/domain.c | 4 +++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index 348ba0fbc8..b6ef440f17 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -2,6 +2,7 @@ /* * xen/arch/arm/vpci.c */ +#include #include #include @@ -115,8 +116,16 @@ int domain_vpci_init(struct domain *d) return ret; } else + { register_mmio_handler(d, &vpci_mmio_handler, GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL); + iomem_permit_access(d, paddr_to_pfn(GUEST_VPCI_MEM_ADDR), + paddr_to_pfn(GUEST_VPCI_MEM_ADDR + + GUEST_VPCI_MEM_SIZE - 1)); + iomem_permit_access(d, paddr_to_pfn(GUEST_VPCI_PREFETCH_MEM_ADDR), + paddr_to_pfn(GUEST_VPCI_PREFETCH_MEM_ADDR + + GUEST_VPCI_PREFETCH_MEM_SIZE - 1)); + } return 0; } diff --git a/xen/common/domain.c b/xen/common/domain.c index 9b8902daa3..dccd272533 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -695,6 +695,9 @@ struct domain *domain_create(domid_t domid, radix_tree_init(&d->pirq_tree); } + if ( !is_idle_domain(d) ) + d->iomem_caps = rangeset_new(d, "I/O Memory", RANGESETF_prettyprint_hex); + if ( (err = arch_domain_create(d, config, flags)) != 0 ) goto fail; init_status |= INIT_arch; @@ -711,7 +714,6 @@ struct domain *domain_create(domid_t domid, watchdog_domain_init(d); init_status |= INIT_watchdog; - d->iomem_caps = rangeset_new(d, "I/O Memory", RANGESETF_prettyprint_hex); d->irq_caps = rangeset_new(d, "Interrupts", 0); if ( !d->iomem_caps || !d->irq_caps ) goto fail; From patchwork Sat Dec 2 01:27:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13476719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F02A5C10DC1 for ; Sat, 2 Dec 2023 01:27:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.645845.1008384 (Exim 4.92) (envelope-from ) id 1r9EnA-0006Js-2f; Sat, 02 Dec 2023 01:27:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 645845.1008384; Sat, 02 Dec 2023 01:27:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En9-0006HX-7B; Sat, 02 Dec 2023 01:27:35 +0000 Received: by outflank-mailman (input) for mailman id 645845; Sat, 02 Dec 2023 01:27:30 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1r9En4-000271-1m for xen-devel@lists.xenproject.org; Sat, 02 Dec 2023 01:27:30 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f43a1941-90b1-11ee-9b0f-b553b5be7939; Sat, 02 Dec 2023 02:27:28 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3B20exWe004068; Sat, 2 Dec 2023 01:27:17 GMT Received: from eur02-vi1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2040.outbound.protection.outlook.com [104.47.11.40]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3uqt8mg1hm-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 02 Dec 2023 01:27:16 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by DU0PR03MB8599.eurprd03.prod.outlook.com (2603:10a6:10:3e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Sat, 2 Dec 2023 01:27:12 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::8e03:368:1fd7:1822%6]) with mapi id 15.20.7046.027; Sat, 2 Dec 2023 01:27:11 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f43a1941-90b1-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a7JNZrgUj2PpAz83t3yqhrFYTPLO4rXqg7kb8Q4pBN4HbrufLP4SJ4iLVUCxVJN3ZfVut9dNYGQrLxa8q2WtAXdWsJzpO7LMO8aXVJ/SdfvgYjnfSL+WBev0qngdPf+1lBR9FPM3tDXAtRAR+Ee7XPFaSqxON05xEjftqc1hT7U6SbxfIf2roWgFSAH+7C0keMoBu3SN5wUizyG1/9g/9a8YlyIDFtqAixqm07Qc8rmPwtuHPK4gCFzhz+PGWol5rS6apY67mAQxQTmsA777YWnwZ95PVIbRNt8c7/duo9p0ClkWMGUE2bsLATb9eg8afvBSiH/hSGxf6hMfd+iAog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XatSDoliS1e0RbDS2pE/RuspFe++g/oP6bCvkIjzp1E=; b=abI954K3Uifyls8k+aJGfmF0dmXIPoWk2DxTsFFSIYdNCO2gPCMFKWDG8KVZEUybqP65HqnwvjGEXCESFHwBngCgwRpqt6OPtFzzEAfm18IBH1MhsCMNhfi0y2+9GHRxLjyqfDB/0D/vrZxYwe2V3Ksqs76f3yfntNi/p5Vl8545kZJkm/jck2Ww1IeIdYV9Vpu3NOtFMxIvcvoA8s2HPGco73Pg61TaUSBn68wv0DKDRmZ8PwIvu8sy8gYccliAT65FkorlYR7Sy+TnqRlRwL7wL1Hyt5Mc/JoErNJFh1EQ7g4+yE3HUhGX5JjRVpAxiG9al0j4Yx9TaU6vlJTFNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XatSDoliS1e0RbDS2pE/RuspFe++g/oP6bCvkIjzp1E=; b=D+OJHAAGRkqvdCWVlTPVNB+fNN15PrS5EPAtV7rbdZvjfOaXilPaRLZGN0c0jdPwl2s7P0+cie9fEe4cPT3W99R+xF0jPDKEJN1GEBdmThXtoRmMUPOxjzFn38vPtTSjsYnYLCpeLPANhEkgK7CEmY69uwBW9ppFSm3vipIJQysCsTs05v70Kp5FzR0AH8wGecAzOvoAHR+UZK3vq6lYwJN8zFnQRF4V4J/HeeEyMGmX4PwJ6RqFDVRxVW2hadBU572eEbW8zjp/8siU7zjn2dkpTo8OBzVHV6GLZuWrXmz18ESY1OYo9ZqkcfttvxWO0BfqOl6UKTrFXi15g5zsxw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , Volodymyr Babchuk , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v11 17/17] arm/vpci: honor access size when returning an error Thread-Topic: [PATCH v11 17/17] arm/vpci: honor access size when returning an error Thread-Index: AQHaJL6qzrPLCvUkHkyTdIPsJJSlOA== Date: Sat, 2 Dec 2023 01:27:07 +0000 Message-ID: <20231202012556.2012281-18-volodymyr_babchuk@epam.com> References: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> In-Reply-To: <20231202012556.2012281-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.42.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|DU0PR03MB8599:EE_ x-ms-office365-filtering-correlation-id: 790126c9-ae6d-448f-9d16-08dbf2d5cf17 x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: C4StpRTQe8cgl5GLJroaSGZQ6WbA4kMVH1CAH1Uy7+0GXoUQk79KPVPtfPZmpgiGjSy+R6vZUpeaY7iwc/5JPsreQFy8Ru0QLYb/XXaYbyx3wnwqCe5xOMFkQLQKXRL9byZHbVoiOljnXEUQ2rmaBUT/lLdE27pW+TwuOfpv/r2rnxcOuLyIytC/7Pdoy7U4AUsoA2yj90NUrkqTe8EGWL5RdQwPuiyhSf9YSAj8m+cypu/SqQRGvAw43P6jO9dpVXtB+KVEFO4CXml+wkjoxaf4a+mABsRAdou7MW3q6yHPGOUbI6wlx5qFEgZ/MYJIoPwMmbF85KO4CeUUrq8D8RZixt9lUM10prrFqjaeK7NxzGCWoVit/ARCv82n9BhPzbmEQMBOusBGIGHxoiLwNg75XAXfBIyHLEjpZ/EY8wZeGV8PuafTmqtBkc3mWg56PeYYOa98BHi+1W9dp096QFLMdkrmeIEKMuL+VVQO06HvnL6D21R5N9FXsjDga76zsyiPNfcicnjurQbcmnBsrgvmOGiOL9w6ACWn7fFNPTzmB2lRFjvMFELyrKUGILImM1DnSIz8493kPmMZTTfQLXkf4QoEzop5nGLzAfkYVu+7WplHG9oBZW9zPrbD1vyX x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(39860400002)(346002)(376002)(366004)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(5660300002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(36756003)(2616005)(107886003)(1076003)(6512007)(6506007)(55236004)(83380400001)(6486002)(478600001)(26005)(71200400001)(6666004)(38100700002)(122000001)(316002)(91956017)(54906003)(64756008)(66446008)(6916009)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?+E7IAylV0uyncK6YgKmckpX?= =?iso-8859-1?q?LsfqpzaiALM40OTSFAnjy4uEb54ptwhvqCJXpr0Iv3o2BsPsBKbMRQ6BbyX/?= =?iso-8859-1?q?h73b7csYJcCqXpHz4Zt5BpDq2+JwsYi6OmwwH7y/iBNwvAcX4Oi627U51rO1?= =?iso-8859-1?q?O8lqdU2oRJxFPWcnx7/7JQPXx6ul2MnokU9UrBaQve50DdCuEOu2qDT+UOUc?= =?iso-8859-1?q?OT7b8d5jswbi6sVLScAsnIMD+zAcUcBv9FnfRVrWKTSut4VZgmWXeormoMV4?= =?iso-8859-1?q?0pYfuraQTkVWUtxnRI72Xevy571ZHo+IDrhTkcP4dFlFVRwt16o0ST85j/01?= =?iso-8859-1?q?cGKp/IY7BnMhJOzdquvbynthxUF6PkqaHWoBn+WIlLdkTx1nZJ7i8WpKwwuG?= =?iso-8859-1?q?r9e2yiX1jt04zQLIOQB+cuyVwlUz2UF+20qKeOtUhllXgH/QD1DIpvKt4cta?= =?iso-8859-1?q?N/KiGEbC0welmbjqFoz7Q63YxFNM7dxiRnTQH7r+JFGNVeU61ReKnvmEHl03?= =?iso-8859-1?q?w3eh5AY2RJDp5gKaQCKagAQdMcdfsLoVj5D3kTqjdMYGwSQ3oPYzlGl9Ab8B?= =?iso-8859-1?q?1z4ReQujQkUMiKtmcbbEFot3Vnxroth0S1vLs98zPnrnx95qeRZnlfDf7Oxj?= =?iso-8859-1?q?iDRYGPogxBAPmLkuLZRCVthDQRz8naSirFBYAd8AYeYpMUZAaRZAdBymUHll?= =?iso-8859-1?q?kT0evAGd82lOUtWm2m++XQRmO65y0MM5J3+9X5nj4NMhLMrW6mwrv/cvDjsN?= =?iso-8859-1?q?e5kiI3vfxcEjabKhoi3QVO4Z3uFEPK5XXpTPN4ChG+mJtCsUw9tCu1kqwBpv?= =?iso-8859-1?q?Umg8b6cn1YmC9UkwC09bIRY2M6vQJ+AHWXdITzgVC7+BlEJNdudsSb15/CJb?= =?iso-8859-1?q?0jxSJXto1k9Zx5mFqEExMODd4j4qVrAChCJ+z1SXKR7zHrxWN27NrqnyY44v?= =?iso-8859-1?q?GxyaMQWtig6mRKN5i0FzgpDsQOS9WHfB4QHUydVvu31PMG9hqLMSOHe/f3zy?= =?iso-8859-1?q?8qF0sCYGSVDWjiUl9j4EXF5u1XI/TMaYjBlNulDDGAAM9t/5WTQvr1OrdmQ5?= =?iso-8859-1?q?0aM/qHGfKvkFV4zVE6dLOJITAIKYOL+p2NVXCxsH+Qnt/lje3zDgKbNPSj5f?= =?iso-8859-1?q?tMDu3wT/blm4Wyg+OZ+Pa+NTUJiVnsWQ0rHHhQJHlPIMJVXgYKH5KLZgf6w8?= =?iso-8859-1?q?vm6lvApa+OI3Wwt44OGeCfUC3VPflHZkBN5QQiT85Qab7THxwdNytLlLsUEl?= =?iso-8859-1?q?ZHjHwosbf5ydNHfWfgZWL7h9ch/EDAaPwnxJ/wj2wqOBiefMbqXZTe65N/s1?= =?iso-8859-1?q?N84Purq6nkojIa91JJh3A4dNtk4dh2hsvn0Ey8DSdmmziWUs8hCPcPNRQpZb?= =?iso-8859-1?q?uBVeXFBr0dxvwccwB59AA9lFZQ7WcjAReijZoZnoEu+Ud3GV4ZGNkJYPaNN4?= =?iso-8859-1?q?GXJINfRT5R+peJedtXSn8+j873OJkCiq+PV5kEII5FwfKy+6opjzdhppY6yZ?= =?iso-8859-1?q?MRR8f1N/ctcz7nq/YD9zwpTdoePl658ej2lju2AlQzlo308Bc9UyAU3q2Pbx?= =?iso-8859-1?q?yZ77OcDxgc5iHDDuZzgEINJoj7OrdjMa/uUJtP2KG9gWwI3XWtgMIEJMPLmU?= =?iso-8859-1?q?xoTpbHyqn1pargQvzEZFHilAkXTX/Jc0x/E5otQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 790126c9-ae6d-448f-9d16-08dbf2d5cf17 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2023 01:27:07.6995 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: FCKmHkfcEAC9BPZLtu1lgyIBg1B9zq6mz9O28MTUqGKQbSJ3apN0gDh3ZCMZuce75KhEh+g9r5w9OjFxRFimJZbO7yqdnrIDoG44IcoZKmg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR03MB8599 X-Proofpoint-ORIG-GUID: SIiNy1jAmIhYwptOJd8C2gwuStlBMK-X X-Proofpoint-GUID: SIiNy1jAmIhYwptOJd8C2gwuStlBMK-X X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-01_24,2023-11-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=824 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2312020008 Guest can try to read config space using different access sizes: 8, 16, 32, 64 bits. We need to take this into account when we are returning an error back to MMIO handler, otherwise it is possible to provide more data than requested: i.e. guest issues LDRB instruction to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target register. Signed-off-by: Volodymyr Babchuk --- xen/arch/arm/vpci.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index b6ef440f17..05a479096e 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -42,6 +42,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, { struct pci_host_bridge *bridge = p; pci_sbdf_t sbdf; + const uint8_t access_size = (1 << info->dabt.size) * 8; + const uint64_t access_mask = GENMASK_ULL(access_size - 1, 0); /* data is needed to prevent a pointer cast on 32bit */ unsigned long data; @@ -49,7 +51,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) ) { - *r = ~0UL; + *r = access_mask; return 1; } @@ -60,7 +62,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, return 1; } - *r = ~0UL; + *r = access_mask; return 0; }