From patchwork Mon Mar 4 23:44:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13581396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DBBACC54E55 for ; Tue, 5 Mar 2024 00:51:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vFyAR2KQv/JJeJi6ULyEwS7+7KwfEqy9zqaFuvP88cI=; b=2GcFBFMk8zmMv1 Hqw7/d9bslbeITyGHQyagYSKLQsLTIDTKxdDpNeVFFO7mqg87o5i0ixhJa7gn0o6A6xIN4uYtxjeT UZJuSLgoZwuWbCIXx4HS5lUcPAzWIekFwc1jAddYzwEoNtBRqqw2S4K8OUhGGA9g6f5BfwxQTISam lhJbwog50ojv1U5v7RBzA7mBUAMg8AeoihCxdN0Ty7ZaWNsgNseznEBezBUj6yrW3M7mk/kP4YtKQ d5+Onuu/GI+XroVohhNT5uYSnnlR0u1mXFklQty/9YQ8cS3yU0X7QCpxrB68Zq2sG+l1jzG/dYVUa gFQWygDTebst4kxhmJ5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rhJ19-0000000BHVN-46Hm; Tue, 05 Mar 2024 00:50:51 +0000 Received: from mail-mw2nam10on20601.outbound.protection.outlook.com ([2a01:111:f403:2412::601] helo=NAM10-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rhHz0-0000000B7EH-3ti2 for linux-arm-kernel@lists.infradead.org; Mon, 04 Mar 2024 23:44:41 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NSIeBxgouGpAbyqx+I9ym3RBEjlrA3vwHGPlRQOP2rAuudt52A43h1+HGh7Y/xtDsOSZcckYLC8L0X7rDs0h+1AJzXhBz9BztJIWjyjKT/XAl8Gm6drYS+8WXGTNx2Ml9kGNBqtla212vZoCNOFvCp+bo6Y5Zp1e+m0xFYjJK8rbej9SdEffUHf7QQA0BQnYPlhUqoZ+kPKFN5hnPJTaBWCWCzWHyk+CoBAqLdv7beJqqETsLk7jftme+45XtbhZYxvg4t9biY5xtJ1XdcQSv520PNdFGKd5FgHcIWQgVP7HZ4xDhik03hrik8kJGViTVpF1I+2QCd8eI98vWUYfdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LJG6ge+sxaoQOokliAV/M4mbEj0DyQrb+A1UnJUG6Rw=; b=X8t2A9Q8W0uet4dYV+JkooZI7/L+mdgYa8XGwSOiriPHuxdbrV+DvW2RIhPdus5/jKTNIHZxSTwnkEex3HogbQFALRa0YI4IVNiAeWW0Cyvy0Z0CYSERawJlstCv2FKx6W1Cjy/HfnJSyretZ8dJUTK7+i9ojd8GrsModU3A1V5d1rxpm4NX1laep3FI2SXr9d7xAygcfwPpTWwvbUqVdvaMEaeBs87TlCeyb0qdls25UudMdoqoslWRRxjDJQFRDFDGqrUPOVfyeXRbt5DRuTiHMx5UzyPxsu4KHFTTsTR0aX6mnPtohTvsgahCnq1QZv77LVcBTrC1xk+6Pl0qXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LJG6ge+sxaoQOokliAV/M4mbEj0DyQrb+A1UnJUG6Rw=; b=PZhwUC6KDeKKlhWLNxNgwEP6jUilLusyCk6rrbGKOWy1jdhe68VXb75t9NRQ8+VX0SF4Bvr6lPLy3IKH85UsdaKq9v8rSEJ7HC8BfTX79dhPWcCfK2b/bVP2UmnJXUt5GrIlbBWPRyXn1dbTWurYcIZ9CsfOZmXLqYCWYVPh7f/BfQOenwyPorU83QvgSV+HZCzOGV3BbR/vh3D7fdJ52bED6IbZWTm+QkPdEvWkFcQImhPbHjDiIbthp4u8cqcTOftk/kRGLjoRMN1p/4VkgoV/pSKv9QHClWOiHSdVIPp/APEnSmpjMyksfvGlR/q9IE+Re7+QAZQ7sIVSj5RIdQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by MW6PR12MB8949.namprd12.prod.outlook.com (2603:10b6:303:248::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.38; Mon, 4 Mar 2024 23:44:21 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c33c:18db:c570:33b3]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c33c:18db:c570:33b3%5]) with mapi id 15.20.7339.035; Mon, 4 Mar 2024 23:44:21 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v5 27/27] iommu/arm-smmu-v3: Allow setting a S1 domain to a PASID Date: Mon, 4 Mar 2024 19:44:15 -0400 Message-ID: <27-v5-9a37e0c884ce+31e3-smmuv3_newapi_p2_jgg@nvidia.com> In-Reply-To: <0-v5-9a37e0c884ce+31e3-smmuv3_newapi_p2_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR1501CA0030.namprd15.prod.outlook.com (2603:10b6:207:17::43) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|MW6PR12MB8949:EE_ X-MS-Office365-Filtering-Correlation-Id: 7f1b9904-46d0-4977-2b97-08dc3ca50133 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N4LecfmD1ss1+MciqoHiZ5H/clLeuG2z/glHU96vqqM6TqUSF2J4t6kQnJ5KyWgFvRmeSzLRntZ3L6b6qn62IgwCTlkh5D/cZANXtCQ05TccDmtFxWPZpNeH4cZmY4A5eoDTMLEB3L6Kukpjtrfh855uH90fSLQyxU2eaFZE1qJvPV/4YTwwcBDAclvmlgsu5qlaujDyjTEewET2ke+V0mPz1mavSY1Nhj52cxqlMHwgnFGfJzRl5IkdQmHpjyClfcpAfg1e+d5oiwf33aXcdgUWZKlN7PQZalvLHdGP5FQDzfaCJq/KdVif++Zfil6odolPNUhberoM5IH85y4VbJ4rcJ+j70PdL8FaCBAVjT65TXOuZEX/a9d1bZ9hpGg4ieR2ctL+lYBOArxonTKgZKh8RceU9lqjQ60B1wMriD1xXSRXjrh5sphCTyzafzTqe7pDSKgQbjQFb2q85AAsC+iRGZBVqBmnMrzdGs9MUBwn2MVxDrv0AuOI292NuD73wOk20jTuzAx/dO6TdQosxn0HVN1sb/cy9loXtBKsCL30W2Bt9M/TP1Plu0wiVuKJBLDWVjeT+ZgoZDTTRh2srOpJMmcsb8CdlBCO3WPZMXKViTyXOzHJYPdpGL/N19nzfxHpoodqfPrCt0nRmqM9P+8iERahOvztJyUHatFN1SA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: XAAUT6DXdopVWFhS7x3Bh5aQ3k279BgU++nvEEsRHJyzX/ufKD40DVUIjaH1ROBl/YLSh9Z6d3ZzCqB9GDAXsnMKp+9/AKtlCt1/JKsudDoK+cEqy9Jg/xN+xxyeTgbei9uTC9yJdIclz/UFcfbECvf+FJdrDCknwStgyDI97papS1LwqX/aTRHLWHysCiMA6uhWtFBzW2koUGf4NHg2WOE+Xvx/GYfdruMTK7bKDjgJMcNItq8ZTZbCmxgxzixJiZup2xte/w8SA/xiMGx8YMUMnHcdnTq4cJ576AHPPZ5lY+Hp5wC1dS3qmOvw5YI8wdXnEFiEpOAdMnOTtVP0ZMaOu+SNXlL19qNQly0fN3JtRtZ+KNNPtL8+nPDP9wmv8XzCT5DG6RUFmJQIr5Ze5LlWGKdzCyOzz0y8x03IjpT4H/lz79CfMWFI/unUzE7mO7d3MxCXkWMghoFteCea02US+GIQb8GrplbfdICTwqUYJhKTVEDzgZwiRd9gmL0OQow5l45E4LD1tVXODyu8bv8REenRmB0D5sUlabUwnB/N3Eg5yY9L+raTezbsXoO6aB/JQgd4wFhPv1cQewcH+2j/Gep9MvsDkqRxs5e5g+mSbRsFVK0lL2UKDpVBt1O0CitBQzkSBbKfxokS6fds3T3oau7+lTxkyn+MvhZ7A9QeVYTRHXvlezNDn493oRn69ge3mVjYXv8KGpQwHDaOQYWXrZzsW5W3pmqvB9jjPf3KTRt+0FLMR4CSVzc6xQJ0m3WoDOPtZgDbpY4L5W/+TEHLgFDeFGbP1RBgYANMD0P/8OhT26h9pfUZIFLfDSweO2PfePOBkcUzyWMvir9/EBRL0qG4F/K/3F1e7m2Vf2IXilFY3dvkplbg2j8iFuuNQIQiP6+V6LAP0UHlzA9VrAzrFgNrdELZp+wjrPsQeCRlKMog7DTjNqDrcrFm3AgePZt+dxFWgoXSl/gQH2BCOhNetmQXg4kxMpvX5utT0+CgJjTF1i3QN8k7NDnsPMwlcDJLcE1R2petsQVnlrmjSxUYQpEqkuzcPRnbJwllo3gpsxyW0fXqiE8PHlSmrwyhzU0h/kQ7CmiOXs9UID4yE9IBWnCy6Sod57SHc6TN9OYDsq+WN5UcnpKtA5uxjJsw7AezmZ2VN0C2qhebAQx5WB1KiruGLFlaoPjqqTKrTsns+BJQ62so+Z9XisyLuqs29sQFs4MlJ6vxoi7OQzLS/XFFtfXyhBH84Uolj3uxWyBtqm2myfkt/XJ4Fo8Ftsi0s4rk50lrI5SRoxNyG93VmqUk7x3HyRUesHVTwpQZ9QAuEfdJ2GC9sHQCs627Ks1AOKqL6yQiXHUMxPTzk8KE2AZuuOtoKCiB4vGhDDqXl4hsXhlXQdHUYcYhJWkIpm+nlG6QByFYBZmTzHBIAg/732zz6Gol5RfPDcDJeteFCt+Ay0m6wxH5TTMqzbC4kaq0ba2tPakkJeRjwvJp+DsST3ufz7oG3QJmqkSu1j+H3492Rhd/09/oRBYk90mF1fenC9pSlLMCnGh5vUAaN+iZ3uLGzF+WQo9P7jRPP+wc19fzc2MxUST9ldVyBjc3Hh8B X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7f1b9904-46d0-4977-2b97-08dc3ca50133 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2024 23:44:16.8572 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: E7f+9W5OJ5++vBX9IXzVIDr/lLK218trRvtkpP29wfXjnHjD4E5jRbpT6yMi83g5 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8949 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240304_154435_292180_B0FCA15C X-CRM114-Status: GOOD ( 18.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The SVA cleanup made the SSID logic entirely general so all we need to do is call it with the correct cd table entry for a S1 domain. This is slightly tricky because of the ASID and how the locking works, the simple fix is to just update the ASID once we get the right locks. Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 45 +++++++++++++++++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 2 files changed, 42 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 535b28d13da5c4..5788998025c89d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1322,8 +1322,6 @@ void arm_smmu_make_s1_cd(struct arm_smmu_cd *target, typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - lockdep_assert_held(&master->smmu->asid_lock); - memset(target, 0, sizeof(*target)); target->data[0] = cpu_to_le64( @@ -2791,6 +2789,36 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; } +static int arm_smmu_s1_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_cd target_cd; + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) + ret = arm_smmu_domain_finalise(smmu_domain, smmu); + else if (smmu_domain->smmu != smmu) + ret = -EINVAL; + mutex_unlock(&smmu_domain->init_mutex); + if (ret) + return ret; + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) + return -EINVAL; + + /* + * We can read cd.asid outside the lock because arm_smmu_set_pasid() + * will fix it + */ + arm_smmu_make_s1_cd(&target_cd, master, smmu_domain); + return arm_smmu_set_pasid(master, to_smmu_domain(domain), id, + &target_cd); +} + static void arm_smmu_update_ste(struct arm_smmu_master *master, struct iommu_domain *sid_domain, bool want_ats) @@ -2818,7 +2846,7 @@ static void arm_smmu_update_ste(struct arm_smmu_master *master, int arm_smmu_set_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid, - const struct arm_smmu_cd *cd) + struct arm_smmu_cd *cd) { struct iommu_domain *sid_domain = iommu_get_domain_for_dev(master->dev); struct attach_state state = {.ssid = pasid}; @@ -2842,6 +2870,14 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, if (ret) goto out_unlock; + /* + * We don't want to obtain to the asid_lock too early, so fix up the + * caller set ASID under the lock in case it changed. + */ + cd->data[0] &= ~cpu_to_le64(CTXDESC_CD_0_ASID); + cd->data[0] |= cpu_to_le64( + FIELD_PREP(CTXDESC_CD_0_ASID, smmu_domain->cd.asid)); + arm_smmu_write_cd_entry(master, pasid, cdptr, cd); arm_smmu_update_ste(master, sid_domain, state.want_ats); @@ -2859,7 +2895,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) struct iommu_domain *domain; bool last_ssid = master->cd_table.used_ssids == 1; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; @@ -3365,6 +3401,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_s1_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 92bff64eef644b..1acafd5c494971 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -771,7 +771,7 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, int arm_smmu_set_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid, - const struct arm_smmu_cd *cd); + struct arm_smmu_cd *cd); void arm_smmu_remove_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid);