From patchwork Fri Feb 25 11:46:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 12760152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4B37FC433F5 for ; Fri, 25 Feb 2022 11:47:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.279033.476542 (Exim 4.92) (envelope-from ) id 1nNZ46-0002AH-6n; Fri, 25 Feb 2022 11:47:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 279033.476542; Fri, 25 Feb 2022 11:47:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nNZ46-0002AA-3Q; Fri, 25 Feb 2022 11:47:14 +0000 Received: by outflank-mailman (input) for mailman id 279033; Fri, 25 Feb 2022 11:47:13 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nNZ45-0002A4-1Y for xen-devel@lists.xenproject.org; Fri, 25 Feb 2022 11:47:13 +0000 Received: from esa2.hc3370-68.iphmx.com (esa2.hc3370-68.iphmx.com [216.71.145.153]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a9cc1772-9630-11ec-8eb8-a37418f5ba1a; Fri, 25 Feb 2022 12:47:11 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a9cc1772-9630-11ec-8eb8-a37418f5ba1a DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1645789631; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=0Ktb92adsVkmWTCw5E6AmJZ00+bq5/0UXmMSrAVMpSU=; b=iAkN2Rb8CbX6QWRUHgYhm+QedTpwhgWLorpRlvdLATG8udSYLqFXnwPu 4PyIHwWlIZJQicdi9DabFbLIvcxQEkPOQgvkvzUAopXsV2ySoSj+oeKFE 6V//NAlckxggAVmBxaoTG5KPQEzra7O923wJdwyZZ5WQjBzlkq4j0T7DJ s=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com X-SBRS: 5.1 X-MesageID: 64985351 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-Data: A9a23:O3EBhKKs4Al757lWFE+R05UlxSXFcZb7ZxGr2PjKsXjdYENShGdTn DFKDG7QbKveamX0c90jYdzk808GvcXQm4VkHFFlqX01Q3x08seUXt7xwmUcns+xwm8vaGo9s q3yv/GZdJhcokf0/0vrav67xZVF/fngqoDUUYYoAQgsA148IMsdoUg7wbRh2NQ12YLR7z6l4 rseneWOYDdJ5BYsWo4kw/rrRMRH5amaVJsw5zTSVNgT1LPsvyB94KE3fMldG0DQUIhMdtNWc s6YpF2PEsE1yD92Yj+tuu6TnkTn2dc+NyDW4pZdc/DKbhSvOkXee0v0XRYRQR4/ttmHozx+4 OpLm6ybYwgzBOrRhO03fB1BIyZzAZQTrdcrIVDn2SCS50jPcn+qyPRyFkAme4Yf/46bA0kXq 6ZecmpUKEne2aTmm9pXScE17ignBNPsM44F/Glp0BnSDOo8QICFSKLPjTNd9Gls15ATRquHD yYfQSg0agnpbiQeBnEGUo4TvP+4vXfUVgQN/Tp5ooJoujOOnWSdyoPFMtDYZ9iLTsV9hVuDq yTN+GGRKgEXMpmTxCSI9lqoh/TThmXrVYQKDrq6+/V2xlqJyQQu5AY+DAXh56Pj0wjnBowZe xd8FjcSQbYay0qFfvLXUAeC52OYtzQzX8NgFOAG0VTYokbL2DqxCm8BRz9HTdUpss4qWDAnv mO0c8PV6S9H6+PMFy/EnluAhXbrYHVOczdeDcMRZVZdu7HeTJcPYgUjpzqJOIq8lZXLFD752 FhmRwBu1uxI3abnO0hWlG0rYg5ARLCVH2bZBS2NBwpJCz+Vgqb/NuREDnCBsJ59wH6xFAXpg ZT9s5H2ABoyJZ+MjjeRZ+4GAauk4f2IWBWF3wIyQMV4q2n1oyb7FWy13N2YDB04WirjUWW0C HI/RCsLvMMDVJdURfUfj32N5zQCkvG7SIWNugH8ZdtSeJlhHDJrDwk1DXN8K1vFyRB2+YlmY M/zWZ/1UR4yVPQ2pBLrFrx1+eJ6mUgDKZb7GMmTI+KPiuHFOhZ4iN4tbTOzUwzOxPjc8VWNr o0GbJPiJtc2eLSWXxQ7OLU7dDgiBXM6GYr3u4pQcOuCKRBhA2YvF7naxrZJRmCvt/09ejvgl p1lZnJl9Q== IronPort-HdrOrdr: A9a23:HNVGa60VKQdNEG4iYLxt8AqjBLYkLtp133Aq2lEZdPUzSL3+qy nOpoV+6faQsl0ssR4b9exoVJPufZq+z/5ICOsqU4tKNTOO0AHEEGgI1+rf6gylNyri9vNMkY dMGpIObeEY1GIK7voSNjPIceod/A== X-IronPort-AV: E=Sophos;i="5.90,136,1643691600"; d="scan'208";a="64985351" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RQ7B3ukD7yRi8vctcmCEY/+ppIS1yXl5Ew9TypzlYcorf73Zft2gzwEGAr2TNj1XOVBprx+fw+rmMA6wUpSAeMkzwwLcFi88/Ri3NRkfS5Ikz6rWaoPHx6Oykaq3CGHkVGgCgzXgTmD1WGABxbC73pNd9QAD+APcHp5H8ERclYZNJxJrgX7t2Zg8JwmccefXii5ceA1ps0IA3MOcWPvR4n+dKI6E0y6ppbAn0R+pubTJSeFdePQx01X2Jo70qhhY90Y0hH0oBhPHIXitd/YcBYJ6zZdCScH3Nfqnw/eucQdbObB5zgQ3dvPwTrrdMwlOs8C0cP7L9vg3BF0QV5zQRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yx4r1lSZIl1pe2SrW52IVtMxwtt6xWeMduyLUDwhOFc=; b=ZycM94aGaz/GDb7qsf3y60aUp6NrJPKDTjYQbG4o3sneLguMlLzr7GxSxz7P99s+NietpOAnEAkPcFpTqzptNRdazSihZsjWjIsoptK5vA9RrDkJrEhQ8n4aWALKQw2XjNpbxk/dz+m6X7nWog9WALzY/kEanTY1wg6aHn+N7t6llwcrx1LyyTjg2h2aTXNy0u8kpOb0QGAZcPA9DqL5S2WduD68BAmiMp4CiNuD+qK415CdMpAlAQ98hyhDduUHzuqYYZBh4QB4YI1PHFPZn6kHBRQ3skKdEAhwFKi31FTxm+Y6uqOns4U2xCTZqnM42AG9Tj/tbiB72JxilPk33g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yx4r1lSZIl1pe2SrW52IVtMxwtt6xWeMduyLUDwhOFc=; b=oTfOuiyKztwsUZFy0DaBMeZ7RUO21MgPKaSdTSOPR5O5dUGJqK+WkTnKpGQdfVzHE+BzlGhnQbWwlbTeExqTfpqxiAUogLqY36LO852rcTP3H/wE0qhkNBNPu4EtGJ0BpNFLsSOJ4jkRMFXBr12RtxypZ2TtlK1boofqPxlxd5Y= From: Roger Pau Monne To: CC: Roger Pau Monne , Jan Beulich Subject: [PATCH] vpci/msix: fix PBA accesses Date: Fri, 25 Feb 2022 12:46:47 +0100 Message-ID: <20220225114647.93879-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.34.1 X-ClientProxiedBy: LO4P265CA0009.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2ad::12) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2c966e0b-d76e-4596-8c9f-08d9f854887f X-MS-TrafficTypeDiagnostic: MWHPR03MB2896:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q2A8oXFuEFOHiK8sk47CNAPGjjONGP9c6cj26B4XwTADZPV1WKJejTizWTOjsn7uTwr0aPg/0SWKVEirOuN/iEyAVa4PPCNL917T/2iWXJy2l7YgRH1Qr1752ZGb7YokwK6TBQ7yvJ6h+YLAFeua8ELkrMbZykx9MU0S4jSfcBsR/7p/HoavihIOFfCkpq6QnYJ/H5MCr76kVI4Dx/sB1vvU8JpeeTbtm0w5Eo/VSoBPtO4LLNjAEBuN9LK/+K4hy6Z9bIMB+tVt5y6iePlSsca19YE89/p/6nughkckqFtddQ6PZy+2QKh21ldld8ccw+osAMOpIgB2fsEc/o9hlgC7dVAzRdl3fx7yU/E1Q1+xnJ4YzFl7bEtxS/Xzk2vkJvx3Y9mPyjuBV8lYHo9rvhQvnMp2deM2H18LcIaucqn40XbleXRRiHfRIbZ9Z1UjY1+MWjUmYJyx195Su3einjrECkemSyhVG4sTwHiqDDyswfL1Jr52/elSpM9ycs4jH1Erxx2ywxCL/K2DzEgShcC+KeAsc5UCoWqiHwVNS2uDI7ve7Wv8ISvvJVuRxBfIyfomKVHbSqqzRSeOUOj4NlV0TFyPMa2+wKN1w+QYdYe7GuQFu3ssT4wWvEjIjmo8b53JThA7FmcPf6VG6VSEQg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(83380400001)(8936002)(6916009)(508600001)(6486002)(54906003)(38100700002)(316002)(82960400001)(1076003)(2616005)(4326008)(2906002)(8676002)(6666004)(26005)(6512007)(6506007)(86362001)(5660300002)(66476007)(66556008)(186003)(66946007)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?bz/50VQ880wYrU+1WqT6FMEo+dC5?= =?utf-8?q?nr3EhRfGqL1qWPGGIZ0P2STduAo9gZ96GhpqCI4/9DMr12C0PTrVEMFR2YZu2P/W8?= =?utf-8?q?YQ9eI3TUIsAv3yb+3SD4A5jjNHZ4Csfvk0r42PuRsMytmbvIVEjBwrTMQTS/dHzbl?= =?utf-8?q?nIrS1aab2V8Agc6reGW1ckyKroj5mWTPxLflwAT+Og5ZPxQ7C5cbyQHVC3UpbK8Is?= =?utf-8?q?0HHJA56fU5zDL4VC8lWbC6rKb/GGTqnq/ukKBGRXm5a/CFqwrNO1MA9MGjmE8XILw?= =?utf-8?q?1FNXH7BdpN1qpANU16YXinuxWDwR67j//+vF4NFBIBDIR2UrewJXwyihYe+Y77LP4?= =?utf-8?q?PnMibSg3DJ7rxhrYAuCcFCowwu/I7KXzxCvWJ+DSzy4uRhjZDJmWj9LsvAWUhqqQX?= =?utf-8?q?guRh5LKs44sqnBpau+lun7ngsIknd2zzboFfaw4EpemQ9uN3J4dOkZgFawF9+zu/Z?= =?utf-8?q?bQ3nNjCuIB8vg9uCnStFuKjgTfHwKKcUX3b8rUAQ1Xpujv0p5q/lf2xcmJ+uCg26x?= =?utf-8?q?atZU7X5cEeQiB+TXqIvFsmw5qSG42QA5jnw5XOzOmeXT8SiBl2gdy1s4x8iipseZw?= =?utf-8?q?ZRbphgeM2q3MC2W4G5OM/SsCEOmHZ4KbaxQOPdpz/F6Wn66goBOeNmwrczt2V8gZe?= =?utf-8?q?zpDO5q1929e5ZBRaCh86rQ+7izVFRR6QdxaGFaWkYaz04crG5BAUvsBG5fxARCQa7?= =?utf-8?q?tovklRyVtSc9Pdm5A+hIKqPzq4qJ99RJdTUTwbKaEVruh1AiMA+ymBVj0xo2ctLFI?= =?utf-8?q?lyKbs1KdeKI62zZaO2ZqK4jyYR9wX0MM6TOWrQPkF4sqIw5z9eDnHEGnLWFAbQCed?= =?utf-8?q?n71iCpNXH+gq7GZ5+xRGJdWLZl7CDoog4HcYc2G8pKWI15nYWojZPnednj0bq/Jt2?= =?utf-8?q?d9E0pOFUgYw4OgOPMpZRnuRvXEoN4MKSlxXfVc9yoAOE0mPRvEP3RuuxcVPFbUVqb?= =?utf-8?q?A/QqEbz+Rn+oZgAPXAN4OcyePkLAAiiOBgE4s3YpChFQ7Z7MmJTmQa0qf3+pVBF/8?= =?utf-8?q?rFvg6Ri+bf8Ss9J6ay6mXOkdOzD7RNYa7cm3m4iOT4EEbO/qwacD1+I74DsL5pfzL?= =?utf-8?q?HIumu9yu1k13DJ0LXM2ZYdHjimaXyJXvFPUzHSzqwM4vbhfLMNLaFziSGvn81/bOz?= =?utf-8?q?YjxXG2h1PY8I59G6Cc3HMb31WIgewTwAboqGL4Y2bliBEmyTqNxfsbZCQMBLEYu16?= =?utf-8?q?Ob79fgNTyqRB2wZk4eJCkChCVtxCYsrIZCQGLu8mNrso44TUiRGTNFVq+JpxCAlsd?= =?utf-8?q?IwrSbTk7FtgoRfkPcaL/2iorkMEXdET9lZRXnLI+Jh9NJjUwqcPL/DLH/3ZtBoofY?= =?utf-8?q?TEq1WGGbRymdKroUut14lM9+F7jCMAHSyX5QCYDKQWGKRvsvJCm+HE4OrDhABcgzr?= =?utf-8?q?XqQasOkXnBfHl+a/Zi4vA/s6v4o5k8Dj1m7N8alSSjclpHrVpR5BnVT6UhrFNSSVx?= =?utf-8?q?OCrPN//rUkZMZKf084YlgEtCcSMhdUl+4gUxs3RlwnqukmMgt9tIJb2thYlU4scFd?= =?utf-8?q?DBEQYkHM06TfF7FBnaERxxt0r05eenWUZbL74SHs9vD2L3blOg+y+K0=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 2c966e0b-d76e-4596-8c9f-08d9f854887f X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 11:47:00.2043 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vIYiGkilxNq9B27A8fPHOtBuXCOLKga7NYxACYg5cz/1d/viMssS3mgSxjmu0AO59UetFIzuMqIU2eNdUMq1xA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2896 X-OriginatorOrg: citrix.com Map the PBA in order to access it from the MSI-X read handler. Note that previously the handler will pass the physical host address into the read{l,q} handlers, which is wrong as those expect a linear address. Map the PBA using ioremap when the first access is performed. Note that 32bit arches might want to abstract the call to ioremap into a vPCI arch handler, so they can use a fixmap range to map the PBA. Reported-by: Jan Beulich Signed-off-by: Roger Pau Monné --- I don't seem to have a box with a driver that will try to access the PBA, so I would consider this specific code path only build tested. At least it doesn't seem to regress the current state of vPCI. --- xen/drivers/vpci/msix.c | 28 +++++++++++++++++++++++++--- xen/drivers/vpci/vpci.c | 2 ++ xen/include/xen/vpci.h | 2 ++ 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c index 846f1b8d70..70085e98b9 100644 --- a/xen/drivers/vpci/msix.c +++ b/xen/drivers/vpci/msix.c @@ -198,8 +198,13 @@ static int msix_read(struct vcpu *v, unsigned long addr, unsigned int len, if ( !access_allowed(msix->pdev, addr, len) ) return X86EMUL_OKAY; + spin_lock(&msix->pdev->vpci->lock); if ( VMSIX_ADDR_IN_RANGE(addr, msix->pdev->vpci, VPCI_MSIX_PBA) ) { + struct vpci *vpci = msix->pdev->vpci; + paddr_t base = vmsix_table_addr(vpci, VPCI_MSIX_PBA); + unsigned int idx = addr - base; + /* * Access to PBA. * @@ -207,25 +212,42 @@ static int msix_read(struct vcpu *v, unsigned long addr, unsigned int len, * guest address space. If this changes the address will need to be * translated. */ + + if ( !msix->pba ) + { + msix->pba = ioremap(base, vmsix_table_size(vpci, VPCI_MSIX_PBA)); + if ( !msix->pba ) + { + /* + * If unable to map the PBA return all 1s (all pending): it's + * likely better to trigger spurious events than drop them. + */ + spin_unlock(&vpci->lock); + gprintk(XENLOG_WARNING, "%pp: unable to map MSI-X PBA\n", + msix->pdev); + return X86EMUL_OKAY; + } + } + switch ( len ) { case 4: - *data = readl(addr); + *data = readl(msix->pba + idx); break; case 8: - *data = readq(addr); + *data = readq(msix->pba + idx); break; default: ASSERT_UNREACHABLE(); break; } + spin_unlock(&vpci->lock); return X86EMUL_OKAY; } - spin_lock(&msix->pdev->vpci->lock); entry = get_entry(msix, addr); offset = addr & (PCI_MSIX_ENTRY_SIZE - 1); diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index fb0947179b..f8674f3ea8 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -51,6 +51,8 @@ void vpci_remove_device(struct pci_dev *pdev) xfree(r); } spin_unlock(&pdev->vpci->lock); + if ( pdev->vpci->msix && pdev->vpci->msix->pba ) + iounmap(pdev->vpci->msix->pba); xfree(pdev->vpci->msix); xfree(pdev->vpci->msi); xfree(pdev->vpci); diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index e8ac1eb395..bf93fc8d22 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -127,6 +127,8 @@ struct vpci { bool enabled : 1; /* Masked? */ bool masked : 1; + /* PBA map */ + void *pba; /* Entries. */ struct vpci_msix_entry { uint64_t addr;