From patchwork Mon Feb 26 17:07:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13572525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B019AC5478C for ; Mon, 26 Feb 2024 17:09:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tWJVJObUCW5UVrQEGuS+WoPbqPM88jin2so2jdpc61s=; b=4V+R09hcPhWAwk jz9SlAMmdXAyal5ZrXcrQXXU8aHd1rsO6YHQGk579H3N8H50J1GuLM16PSDP/YhJ7SEgv99pAVO9d IMYkHx7FbOrbSWq2vP8XMGjMwbJSavMlyWFs+10mjJXa0Hc4cRkbPgCjq+rvTHkmdXdHcnuSrKisJ 5LQXFQl8vhVUzWgXCMx+grKL8TOYvP4iL07vyKtbwzfLSCrbEAACo+zvL95hyAyj0xuGhOvfoVnzX 4ote64F7Xk3UtJ2HZhtG8giHkI2rhXuuotCVSmBCEHe3fQHQI7IMospjoXWS+2jODegTk+FgUxtVr c+2TU7Nn0tuajBBYGGEg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1reeT4-00000001qGX-2xSE; Mon, 26 Feb 2024 17:08:42 +0000 Received: from mail-sn1nam02on2061d.outbound.protection.outlook.com ([2a01:111:f400:7ea9::61d] helo=NAM02-SN1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1reeSD-00000001pkC-449M for linux-arm-kernel@lists.infradead.org; Mon, 26 Feb 2024 17:07:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DV2kHTsbTU5gOBk/DijmRaCHpGfxlJEWBSDgeECulBE6fRLfFFPSEwa40z+zg26jG6jorjIFzD8e4pNz8Xj2WaXlLBP3gZAqwZ9InnPt89qowsyz1B+kW2wLGegPLk74pDUS5e+tUUouxHYU6u74ZynvKgZiZDRcd0aFtMawR19bdodBpNsicDT9LjD0+a6PqucMmVIVKvBDL9Ixh/qwRXnImoYb+o71+Q02ZpMQCMcDc6+A+3Hf2pvTwIL1j92PhgSXG9/Fxb4FYLLzzZr2TFKTXiySyCrFgRkDVaxXWhwDhoW7TbbDoLCe/24emJ1F6qvRhj6RdpLY6KnCjYEgKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CD0+FHrIzPyJO85Q97dh3teCEo7kSfrLDHD10sjtyxI=; b=RSShp/B3l4mhBXN1/hFKmUwNKMJtO81F1km+pBftT2eDmvPJNQQ14XPlVd31IZW/qrl7AnY9qxU0OfsTyWUm4teSP8vB3gHStNEY7UnJEvq3bYbG59oAN0Q5TerIvefMNZ7juPzNBKhFWCWcG29nVrTePp6RGjUM5Hecor+y611AnkItikYDiAvAfGvxaXgA47apoOa/kuy9UjYyXoErgIpiZejBuvDCubbrNXLoUolctzVk8VyrKi++ZWqMpMsugoH/RefGuJuP0D1ciKKb7wKUTh8KDMCSOQ7PynZhePL3xkJ2UMPrPVAERFJ6y4/5cwLdMYR4cm6ENIa7kreXkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CD0+FHrIzPyJO85Q97dh3teCEo7kSfrLDHD10sjtyxI=; b=H+PBDRku3EMgb4mD2jqM/RgYct/2MKIotUys8bOnC/FmRjyxeYktPapLor9zM4XDe0Y7RUU/IF6lg0bi4Ztva2dnZSBsMuEzYWzz23796CVYxnwCCVe6uaDClOW0iL1QBGlU56wbZXuBb17oxWISYVnA8xfgOBeo/Pl6qd5346ioKZAuZb5+hmZYsIS32h7Lq9nxoT+wSaTzpWs8+peT4WWj8Pf5bYRqeqNnYGc+FcXPoVNceS4+oOg5t4AJ9l92AJLujg7AOEHQs3mk+9R032PphRpbE6nq6dwexmBYC03OaN0MH2PwDjuB4ei3PJ1NbLDx/54Ps3OEs/+ne/JWpg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by PH7PR12MB6905.namprd12.prod.outlook.com (2603:10b6:510:1b7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Mon, 26 Feb 2024 17:07:33 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::2e3e:c7c0:84da:3941]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::2e3e:c7c0:84da:3941%6]) with mapi id 15.20.7316.034; Mon, 26 Feb 2024 17:07:33 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Lu Baolu , Jean-Philippe Brucker , Joerg Roedel , Moritz Fischer , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi , Mostafa Saleh , Zhangfei Gao Subject: [PATCH v6 02/16] iommu/arm-smmu-v3: Consolidate the STE generation for abort/bypass Date: Mon, 26 Feb 2024 13:07:13 -0400 Message-ID: <2-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com> In-Reply-To: <0-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR03CA0020.namprd03.prod.outlook.com (2603:10b6:208:23a::25) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|PH7PR12MB6905:EE_ X-MS-Office365-Filtering-Correlation-Id: 8889dac3-5f55-4258-1997-08dc36ed6a5e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Pod089Eo3tsKlOCGtDEshuW3U0RD8KJr6WkK6/4UHuf7eu7xN4R7RTCdAy8rBF2UQdUNEOv0IPbEzEQAGxl1r9y3i6/+Q58EarCKW9DoKnGucHdBVAUEuziGDI46ZP5geno8TJL2zb945jFmGy8YmQTJtZ++ia1YLBZKJdixrIKUPo2zGTxc2YgQ8FbONYOAueR0MhEF2c+NhgdcH5wfleZ6simdDWqMGmvoFOgwjfTzFHZ9T/IFzsD6lFvq/8i6rpospRwrwi9DybTPgPV3C0p4o09cizUrQCGM/JzvQkb668H6Db8YnwpfEmfdYukATyecBLEIquoA7KKvX4oG4APNeT0dlDmrpJgOClt7LrvIKwSARN3dgS8K7e/Zl0qwH/ZbvijTM+wl15TzmU2NXKgWf6xZ5QjdJjblREKqbgpbURgk/POk4YOAB4zxyBhZJ9ZuZ8xhS0gPZVIMee2AT1ZqQ7D0wQJd+m5oI9H1Lxy6QjqUifbdgsFkT4jDRv8hZ9GTGfizyyrkOlR8kjZdbmoD/jpZvyX9mrg7a+vvspUEItm3DzMUH6PjvKcZZtQojPTapJqaG2f791fLwuGFFahPEb87WKgCrYLYkZLv9Z+BDaxFREbUptTB9FXGQzY+ocpAMQNVmnkb949qmrgQ9Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: M1suTowiZ5bMPht+LI0Ks6QKC0cr7faGQhaAkyqtf5exBsUJkmbVfLmLTpuEcttTR9i2WS4s94pRbVZ8qSLiGbnsIEVey5PGWFB/nv02MQA+27nnqqG6wIFNjuSYUo1e1uuTAYncJ4x7eaNcEckgGhM1Vl/c5dvxbK5F6yAiaP1N9kj78265RLxuTz2qWI6+iag8esGD9X9bBWmb5GgMaiHS3r0M5iXixz0PBlxadMd3ItG2+DuOn8wWn94h0y7nYFKxQCz2KXh6pfrA1SxV/nVH2BVtBWfqrfsCZRKvN8PhiJxRsPPhHe8gx7En+OM4/7j3C/2yAbwM1n0TRALojpyrQ4nnmLAZ+2RdFDe6WeT4zGDZNTPdyOAzvzNBPU+AHYdqHytVRjNu0aX8NltUFXi3DEu3Z5kfT1d8pRtXtPfobEharXDs+Eegf0TL4+YGDCYTVm0YYt2uXsahkLQNSRgRfuQTMRsRfUiB9S747tAyqBUZw9GEIsPzfxgHd1IayceA0QU+2FJIz+8r/bbs+t07T0pvjilmpMKgcH1s/uo2WUpIT5uEWmviIFiahK5Gs2l8Dr9+/tKobeNYVrwAIrkzQVrQXLHOqbCHSNcZ+n7A4Go1mKqM+n/r6ehrqV1oK7ufwkVtzoZTOsU7waPin32WhlrbpD1fflobQMBr0ENfHONw0C/DmvTaA/ME6LFykO4C+5ztZbYSwPMPPFx31wPBYP95x21BFLC6wuYUx5N37GiQGdZVRpQUJoXpX62CJP8vlt7K+ZdagBJ2nqbfESkCrUcoqFb07S4n+zIzq6MJ0W1JZej+di9jdiQt2isaePCPyKLsyixfZ8AwgfSfbmYpe3mAy8mm8WZ4psOq0iTyuYN6ACyhhWI8PgnApzisCQfEbBjhbcKenlrRpacxXGmZIM3Wj4DeZspqGcPqLxtYhDDWcejzYJOGrhaV8Yt90DgkWEesSOZcUjPWChOV6ybZcqTN9EP73SpGfUIFEcGtYpfms/pSYG6FRoFhT+g4TMyYm49A6cE94TQ5eCo4fXD6Lbg+HLo0IuJReuDexGpX2kOeFB1Ni76lHrBpkx4gH9v9JYlZEcNkm/ORWWdfzmGD6oaaoYb8EtKmrnbZPE3Y8gvgCdHS809UCkOJwJVueO/jUhj/uYcqI8txe59bcpLJnTy9LLqRYqCFeyaNwz4wKcfGTCyIO2wR5waE+FL/oilpxnfU8YTOQP5Xj4jTZp85xrNGmsixUDxzulfxjtKIC0yT5W15mUx6Wby9oneSL9q5R6++LnWaiWQQ9eopNtoKS3xfoYncL5LH5kjCkfGL4hGONZB50JfXXrTotoPTztAasY9HoI9Uq8VgV5S8y+efXEDPuOeQZ3Oq7hGg7/L3j5Fc4d0N8PDIeEahMxQ/8gxnxNXz4wpcNhgjTQwV8J7kyvgfqorvTd/NX5By2J2S08P0aKRZACOm2Gx8rndTIfapQr7BX0v5p4OwEGGffYkW80j4lQfXSGWLYtIFpPKiGW9MwylDj7i3zdzNKAQgoniaNJumc4sabKDCKuFmzHrc2xWYT6CRdu6iUOw0rj+KXDkGBnqWpMR7Lh6f3WI6 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8889dac3-5f55-4258-1997-08dc36ed6a5e X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 17:07:30.1113 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ty3zT2IZ20iHH6UP4pojYL//ob5tIZsj0NyNhEfxT2sTKcq/oJdU2+jCVNq5CiFE X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6905 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240226_090750_605317_7A806EA1 X-CRM114-Status: GOOD ( 16.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This allows writing the flow of arm_smmu_write_strtab_ent() around abort and bypass domains more naturally. Note that the core code no longer supplies NULL domains, though there is still a flow in the driver that end up in arm_smmu_write_strtab_ent() with NULL. A later patch will remove it. Remove the duplicate calculation of the STE in arm_smmu_init_bypass_stes() and remove the force parameter. arm_smmu_rmr_install_bypass_ste() can now simply invoke arm_smmu_make_bypass_ste() directly. Rename arm_smmu_init_bypass_stes() to arm_smmu_init_initial_stes() to better reflect its purpose. Reviewed-by: Michael Shavit Reviewed-by: Nicolin Chen Reviewed-by: Mostafa Saleh Tested-by: Shameer Kolothum Tested-by: Nicolin Chen Tested-by: Moritz Fischer Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 99 ++++++++++++--------- 1 file changed, 56 insertions(+), 43 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 9805d989dafd79..12ba1b97d696c9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1447,6 +1447,24 @@ static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } +static void arm_smmu_make_abort_ste(struct arm_smmu_ste *target) +{ + memset(target, 0, sizeof(*target)); + target->data[0] = cpu_to_le64( + STRTAB_STE_0_V | + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT)); +} + +static void arm_smmu_make_bypass_ste(struct arm_smmu_ste *target) +{ + memset(target, 0, sizeof(*target)); + target->data[0] = cpu_to_le64( + STRTAB_STE_0_V | + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS)); + target->data[1] = cpu_to_le64( + FIELD_PREP(STRTAB_STE_1_SHCFG, STRTAB_STE_1_SHCFG_INCOMING)); +} + static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, struct arm_smmu_ste *dst) { @@ -1457,37 +1475,31 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, struct arm_smmu_domain *smmu_domain = master->domain; struct arm_smmu_ste target = {}; - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - cd_table = &master->cd_table; - break; - case ARM_SMMU_DOMAIN_S2: - s2_cfg = &smmu_domain->s2_cfg; - break; - default: - break; - } + if (!smmu_domain) { + if (disable_bypass) + arm_smmu_make_abort_ste(&target); + else + arm_smmu_make_bypass_ste(&target); + arm_smmu_write_ste(master, sid, dst, &target); + return; + } + + switch (smmu_domain->stage) { + case ARM_SMMU_DOMAIN_S1: + cd_table = &master->cd_table; + break; + case ARM_SMMU_DOMAIN_S2: + s2_cfg = &smmu_domain->s2_cfg; + break; + case ARM_SMMU_DOMAIN_BYPASS: + arm_smmu_make_bypass_ste(&target); + arm_smmu_write_ste(master, sid, dst, &target); + return; } /* Nuke the existing STE_0 value, as we're going to rewrite it */ val = STRTAB_STE_0_V; - /* Bypass/fault */ - if (!smmu_domain || !(cd_table || s2_cfg)) { - if (!smmu_domain && disable_bypass) - val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); - else - val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); - - target.data[0] = cpu_to_le64(val); - target.data[1] = cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG, - STRTAB_STE_1_SHCFG_INCOMING)); - target.data[2] = 0; /* Nuke the VMID */ - arm_smmu_write_ste(master, sid, dst, &target); - return; - } - if (cd_table) { u64 strw = smmu->features & ARM_SMMU_FEAT_E2H ? STRTAB_STE_1_STRW_EL2 : STRTAB_STE_1_STRW_NSEL1; @@ -1534,22 +1546,20 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, arm_smmu_write_ste(master, sid, dst, &target); } -static void arm_smmu_init_bypass_stes(struct arm_smmu_ste *strtab, - unsigned int nent, bool force) +/* + * This can safely directly manipulate the STE memory without a sync sequence + * because the STE table has not been installed in the SMMU yet. + */ +static void arm_smmu_init_initial_stes(struct arm_smmu_ste *strtab, + unsigned int nent) { unsigned int i; - u64 val = STRTAB_STE_0_V; - - if (disable_bypass && !force) - val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); - else - val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); for (i = 0; i < nent; ++i) { - strtab->data[0] = cpu_to_le64(val); - strtab->data[1] = cpu_to_le64(FIELD_PREP( - STRTAB_STE_1_SHCFG, STRTAB_STE_1_SHCFG_INCOMING)); - strtab->data[2] = 0; + if (disable_bypass) + arm_smmu_make_abort_ste(strtab); + else + arm_smmu_make_bypass_ste(strtab); strtab++; } } @@ -1577,7 +1587,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return -ENOMEM; } - arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT, false); + arm_smmu_init_initial_stes(desc->l2ptr, 1 << STRTAB_SPLIT); arm_smmu_write_strtab_l1_desc(strtab, desc); return 0; } @@ -3196,7 +3206,7 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu) reg |= FIELD_PREP(STRTAB_BASE_CFG_LOG2SIZE, smmu->sid_bits); cfg->strtab_base_cfg = reg; - arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents, false); + arm_smmu_init_initial_stes(strtab, cfg->num_l1_ents); return 0; } @@ -3907,7 +3917,6 @@ static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu) iort_get_rmr_sids(dev_fwnode(smmu->dev), &rmr_list); list_for_each_entry(e, &rmr_list, list) { - struct arm_smmu_ste *step; struct iommu_iort_rmr_data *rmr; int ret, i; @@ -3920,8 +3929,12 @@ static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu) continue; } - step = arm_smmu_get_step_for_sid(smmu, rmr->sids[i]); - arm_smmu_init_bypass_stes(step, 1, true); + /* + * STE table is not programmed to HW, see + * arm_smmu_initial_bypass_stes() + */ + arm_smmu_make_bypass_ste( + arm_smmu_get_step_for_sid(smmu, rmr->sids[i])); } }