From patchwork Tue Feb 6 15:12:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13547502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70294C4828D for ; Tue, 6 Feb 2024 15:14:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LzQhlYtNsF1wOqbSqCPMP9cDc+KtFZsgbMA2Xu07AGc=; b=lGmq6ADGWFLlXA a5DTjnzWXE2nYL9afcqC4Hnl/+biqOaiZwRXvO0J0WT7QRJrPpginIsNOLmbR1Ze6TiEDC6FJ1t7O B138wdmxDMgDeiScTtE5P5JLHOQjWMNyL4GJwQyEDpKsvj2lYdW2PUOnTUb0+u6I9OCVINf2JtzIV 56C4kgjp30r1q4V2Ll2FWwm50NHFoFLoZ8Bi4FOEts0W4INVn1aDtHSI5ZuidypcGW7nzftEtSAbq I3ccvFtf85BHfkTotTqtXGcvwmqf0fGY8zu+UWDR758E1qFWm38xgNgassgjhXlV/4s9EN6w+EKdY R3suXLCEFg+iDmJQusiw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rXN92-000000081tw-2YoO; Tue, 06 Feb 2024 15:13:56 +0000 Received: from mail-bn8nam12on20601.outbound.protection.outlook.com ([2a01:111:f403:2418::601] helo=NAM12-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rXN8P-000000081Kg-1qEk for linux-arm-kernel@lists.infradead.org; Tue, 06 Feb 2024 15:13:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YSXN21rL2WS9sQ5ySB7OZ+4Qz53h+4VEG3SLynwa3gew+gxmr5w+0XooAbsyazZ/nNmzJ4ZS7EDenzIXRSPpadRRcTkpTbTvp1w/+GS1zqNnb5FBeje5EukXaWQhSj/Q9Fql9a/Rfiu6XkKULZQKAiPmQoDZTuQozSRALFDw0C/YE8OUdy4GfQ2Qgpwzis7wh1fCYK5Dc8HMNEG4OEsHKCUWTzHxU0YgDL/lZaRrHFhGdxmnsfpLReIImwzwHgSq4yRirYnxVggU5m9bkf+V09a+gcEOGa+67f0UEQusUkfGPVUzDIQsnREOFOifcGcrwMGTfHnElxR8Vn6y0nwAAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jDaL8iCsrN3cG8xsMKrhBAKxDr5uGda1SYKKuRZihrQ=; b=Au5+p0WeM9elHaERu0c7Z3kvvgqJ0NWj9/oenZrNjVNUAdRXBRmv5mImG81JILz1wuOFhxCC5J+u2uEMNmnPUAUHWrF8P5bIIQH2pfyBqfeSbaoepwsN0W+2tOsahbQfASy9lktxD8E9g7SjPmRMgI5VKqgTdtt1IYSf+pi1qKwTVeLt3RAPckgHtACiwrdlA00YTdFpzmYGVwtCSwvdGsNzYK9s45Nkqgt/TARG/H2dXI64SxK4gYsilRnvzgtYIir7uikc6uQK0iLA6YnFiwHsKIJV4HDp49VABkbkStW7C5qc7krIV1lBFCpZ/R8ZkhDC9/Uq0+D5xHkiMJ3r1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jDaL8iCsrN3cG8xsMKrhBAKxDr5uGda1SYKKuRZihrQ=; b=jRGEjs9Sphkpp6OuUUmkvpyTQzN++yCEdFXLG9rQ4fwVWYLQbJZY9TsPBUuiUeMsO508rRGB1Hdbldhe0Zi9A8MctKkQsWyBxHHBUlFVupKZxqJbBq5zGaq5X8vEurq9pqEyhBpVCjLdDBbGxFvhKEpjqK/xjY6w5RmFU1SANWQHKjbM6Sv3dNwwwem9Bb/dcWji3D4rHAvMeaOBQjH/x2i1x3jA8LP3TSnsVUNKqXm8poHr29bVCyC99f/hbCRwoFFLzpnnE1owJs8wvQFNLnO/fALXo8apWeamtyjRIMrTBMmeZKS1dqGg1BK6Qi8t3CYpsq8GslQqtPDRRus70w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by SJ0PR12MB6733.namprd12.prod.outlook.com (2603:10b6:a03:477::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.16; Tue, 6 Feb 2024 15:13:04 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::96dd:1160:6472:9873]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::96dd:1160:6472:9873%6]) with mapi id 15.20.7270.012; Tue, 6 Feb 2024 15:13:04 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Lu Baolu , Jean-Philippe Brucker , Joerg Roedel , Moritz Fischer , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameer Kolothum , Mostafa Saleh , Zhangfei Gao Subject: [PATCH v5 07/17] iommu/arm-smmu-v3: Compute the STE only once for each master Date: Tue, 6 Feb 2024 11:12:44 -0400 Message-ID: <7-v5-cd1be8dd9c71+3fa-smmuv3_newapi_p1_jgg@nvidia.com> In-Reply-To: <0-v5-cd1be8dd9c71+3fa-smmuv3_newapi_p1_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR13CA0019.namprd13.prod.outlook.com (2603:10b6:208:160::32) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|SJ0PR12MB6733:EE_ X-MS-Office365-Filtering-Correlation-Id: 4a0c522d-9326-497d-8cb5-08dc272618ea X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VYL5HpLu60kjVF1gTK2wg9adJ6gKSyq2KcaPJDUy+E/yNrDnBCqwVno3HaXmbC70OHvFa0vlbrhZcrUTRvBHxZ6uir+X+fv6/U7In8o/RmFpg4z70EJz8HAWj9YT2xZLWeQYqy7p55gI9CjT+WMLUqBpRIkl6Z1evbWc+M7jqErt8aRAh4tQveXa+3X0zBmMnCMUFzLMltgwPHLLAuIunWvSDlF8wGJDmd9Py0dGbsuevzx84z/Jx5dBqLvl3sHHqaYnuGQ9k4rUixJO5vHnM8x28sYX58a2xn6VwH24Slr62zoH0uJN3RMdk+HVZQ5ywcIvnyf57OyH/zq7uP8oP9TAeWEN5cjeeb0naO42Lw93txEshi7JYAXC1aX70mlM4UccqCt93paMK3ffmtEP5Kr9XyeZu3zbHzrFvOW7HZtBC+VaQxFWEXN0kxiTbb5pIOeB6m06/Zjf9KCc5v21VvEVss6p3dSDRfz0R72ekeBxUIRcTUeTzR2ynPw0SdYzcySX25xx2IhnW8dJm24OPbOVyKPbEuKAzxobJWLPZJgUHlahNNci0NP3gjuFH6nqWETntO0zg//Qngjd1XE/Xqj36yebFKhu15xvGkDDDeM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(366004)(376002)(396003)(136003)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(54906003)(316002)(83380400001)(110136005)(38100700002)(2616005)(6506007)(6666004)(26005)(2906002)(7416002)(6512007)(6486002)(86362001)(5660300002)(478600001)(8676002)(66946007)(66556008)(4326008)(8936002)(36756003)(66476007)(41300700001)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NbZqARf8j54aSRQYtO4/p1RLdi/hYYI3M4MWAzbsfKSiNgi3bxiVd6g9sJcmPQDpEU7hpVVT18o3s2FB701WNufsGn9zEZicZcoeQOqQwaQKNUS2t+B66gvkREvA+39zj9oa2Ig/tN7jgARTYY5D+61qQ1JVhGGALGbRQZpecHd/idq/oWTbUN3y+Lfzdqsoa+CK1DAqlfyU5H0pMZDRsQn/Ku3RNEHyI6N52GNoJgkzer7EEEer6WGZ/P93XZgg9Glc9Cd/4Mll/qzXae7mhCJzLkqSeJwBSxFBEczxe7Qt71+8pJ2Y3a/6DasLmKk1v/bJBymb7t0U5WYRebjuo6gkBtMTViUdr6BOhqmepKrZvCmLkHsjt6T2E6oNiAjTcnB0dqirpyg9FpMoJraIiKkDGDjZGLt+edBCX0Wj828pfFxCT7dKL1bQGvEG4jzU4EPucj7T7FmyqR7eTHS/AYvEH2cV8LdnwIXDPNnsZBE7UKRPTqjMkzyX/H01ux82wo/RJiWE66+Njt1OfNdSssyAtVrNKNAiTkz0/xLGXWS+oMzQ1QMWlJSBJEUOX9p9pii6kXtqcb7y8YfdBz+sfLTHF8Xj1iXdreduULkIoO8u8sr7gBw3KIUJcDBx3WQhWGPTSnOSKPT1boaISLpg7FVqouQFGTkDsnOzzkyrntIEcbAcVl3i9UtWbuGS9mrPdAE7Oj2NDssu0o2GscbsI+IQPNOo4voc/sltmYokbXUdQgKj8fCZLrcFUcgFwzaVNcg7mnqa35d7TThlO62wUtcSENST29GppYwwdSXWqC2Aehgz+u5Tg64ieVKyUPUKe6OMck+yZD5C+PSF61Bnp67EE8eT2ejTwGCIGmu2Wvqy/B9vxD5uT5xKmN1qoOOlmyrrQ0STysXYpp9Og4hbILj/vVOkXQpEAEBfA832Zop4Gy8F50wV5BER4BMjU2qFQe1f2w1X+XtEM0HmvJs/PLhepy33lUhwjDpQmqgYq0At4BweUpms/tfcjURrqynwbHlYDqRp0vTWchAgbGp02ZLiO8lonnR99Lr45hlleuQiGnWLalWhxCuv3ZWQi+Zms/EIq00ezI8gH6EE3EuG01I738NVVUSprOtU5ulqnIkZg4LfKvddY/Vu0Yfr6ckZB9MEtWjVdWeTGzt5qQxoLQLQAMJEEgCSuAmFFlDnPltpf6UeoSHba0ylKLPGp7IpLpGKUpg7r+ubjCTu3kYBjNgrPR07dlEhr4/FG7KIFhTufQpbeBUeouoOCwWd/vh7/ZdXkRjlwQZitLELxsNXWNU2cOIjIDBtxHcsAwD9WBs2ZyJCDQbs+Lqjqq+HhDA5yWrDM9B41JwZVTXPXb0kt53WXHRTgH2y9R/GUuiEwyNHVloyEcHhknsSOTB1dVg1/U3P/7QOOKQ0Quebw+s54rqNTr21HcP41oPnpo4dGNu/BEze2JTWYeCYopV1s1jHw6Gb/twxg8Ytrg1ZgEJTqoOcVDRhCgNLxsNZtEiU5F/ibdakzJLEqURp2CL7t0ADqr04u7uyoIJ0B9uEW3QrINvb1LHPauM40q9FgbafBxltqglWXqMf1VxGxZjlrnez X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4a0c522d-9326-497d-8cb5-08dc272618ea X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2024 15:12:56.1101 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0n/KkvRUPK6kT2cIpD9eYalNEB8jOEU0SEb8ZtEEpKJhBdBChWnOCxadgv/HdGgk X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6733 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240206_071317_527986_5BD86505 X-CRM114-Status: GOOD ( 16.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently arm_smmu_install_ste_for_dev() iterates over every SID and computes from scratch an identical STE. Every SID should have the same STE contents. Turn this inside out so that the STE is supplied by the caller and arm_smmu_install_ste_for_dev() simply installs it to every SID. This is possible now that the STE generation does not inform what sequence should be used to program it. This allows splitting the STE calculation up according to the call site, which following patches will make use of, and removes the confusing NULL domain special case that only supported arm_smmu_detach_dev(). Reviewed-by: Michael Shavit Reviewed-by: Nicolin Chen Reviewed-by: Mostafa Saleh Tested-by: Shameer Kolothum Tested-by: Nicolin Chen Tested-by: Moritz Fischer Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 57 ++++++++------------- 1 file changed, 22 insertions(+), 35 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 1229545ae6db4e..1138e868c4d73e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1611,35 +1611,6 @@ static void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target, STRTAB_STE_3_S2TTB_MASK); } -static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, - struct arm_smmu_ste *dst) -{ - struct arm_smmu_domain *smmu_domain = master->domain; - struct arm_smmu_ste target = {}; - - if (!smmu_domain) { - if (disable_bypass) - arm_smmu_make_abort_ste(&target); - else - arm_smmu_make_bypass_ste(&target); - arm_smmu_write_ste(master, sid, dst, &target); - return; - } - - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - arm_smmu_make_cdtable_ste(&target, master); - break; - case ARM_SMMU_DOMAIN_S2: - arm_smmu_make_s2_domain_ste(&target, master, smmu_domain); - break; - case ARM_SMMU_DOMAIN_BYPASS: - arm_smmu_make_bypass_ste(&target); - break; - } - arm_smmu_write_ste(master, sid, dst, &target); -} - /* * This can safely directly manipulate the STE memory without a sync sequence * because the STE table has not been installed in the SMMU yet. @@ -2466,7 +2437,8 @@ arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) } } -static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) +static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master, + const struct arm_smmu_ste *target) { int i, j; struct arm_smmu_device *smmu = master->smmu; @@ -2483,7 +2455,7 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) if (j < i) continue; - arm_smmu_write_strtab_ent(master, sid, step); + arm_smmu_write_ste(master, sid, step, target); } } @@ -2590,6 +2562,7 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; + struct arm_smmu_ste target; struct arm_smmu_domain *smmu_domain = master->domain; if (!smmu_domain) @@ -2603,7 +2576,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; - arm_smmu_install_ste_for_dev(master); + if (disable_bypass) + arm_smmu_make_abort_ste(&target); + else + arm_smmu_make_bypass_ste(&target); + arm_smmu_install_ste_for_dev(master, &target); /* * Clearing the CD entry isn't strictly required to detach the domain * since the table is uninstalled anyway, but it helps avoid confusion @@ -2618,6 +2595,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; unsigned long flags; + struct arm_smmu_ste target; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -2679,7 +2657,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) list_add(&master->domain_head, &smmu_domain->devices); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + switch (smmu_domain->stage) { + case ARM_SMMU_DOMAIN_S1: if (!master->cd_table.cdtab) { ret = arm_smmu_alloc_cd_tables(master); if (ret) { @@ -2693,9 +2672,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->domain = NULL; goto out_list_del; } - } - arm_smmu_install_ste_for_dev(master); + arm_smmu_make_cdtable_ste(&target, master); + break; + case ARM_SMMU_DOMAIN_S2: + arm_smmu_make_s2_domain_ste(&target, master, smmu_domain); + break; + case ARM_SMMU_DOMAIN_BYPASS: + arm_smmu_make_bypass_ste(&target); + break; + } + arm_smmu_install_ste_for_dev(master, &target); arm_smmu_enable_ats(master); goto out_unlock;