From patchwork Tue Aug 6 23:41:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13755509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70D85C49EA1 for ; Tue, 6 Aug 2024 23:44:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:Message-ID:Date:Subject:Cc:To:From:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=kQnOObQ2Tw+DASc46sbnZDsJpR8alMQbzKHA9EuyZLQ=; b=kzuUpwDmYH59pC44chn6h+jDZc u05wnXrqcbmLfSSI1nNgNbebdslQEq0eFx2FiYtVZwcxZnZQomKUNKo5XbqlIB0EqhkoezLT5IbEH vRO9GnIPT3rxrDbNS9hEfQc8cJu4nG8AlCo/PAxvJv7+NBFyWLkTq03N2wXJp4coCiQq1g5fEcmpt IPogJOHmZCEnXh77RIY7GAJ4N2NX+aZ37QVICTLcwL4KywPYmODrl/D7sAA9yx8b04Goh2inM9ECl VyYvDPZ4eKcWiuV6eU9xeTuTmusdExUFuD5PNwpnjvidx/r2M3Lyz5Ee7iEfNS8i5Nc30epA3G7QT DgF2sn+w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sbTqc-00000003ML4-0oRB; Tue, 06 Aug 2024 23:44:10 +0000 Received: from mail-mw2nam10on2061c.outbound.protection.outlook.com ([2a01:111:f403:2412::61c] helo=NAM10-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sbTo7-00000003LjQ-3zSc for linux-arm-kernel@lists.infradead.org; Tue, 06 Aug 2024 23:41:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Uud/dldgf82qbHzCn7nS88NlGMxEYYQjUhm2jayK0mfA5V6naFEgQ0GHmVwh7ijC4FXawRucVPLa0OUctEdJ6r9fW9gOtt13bD/gcTrD4JLkPBrTNOhcYZxvZqJPCO2ZVR2eRC+eUVjz+BNghUQfFWXQNL8wQadnaq2Sy04xRIGrogkvjC/sWAYcHJVRvLAspWjP4qoEsU3t296oR9YMr5GeSrSG+nRUvSajpbqY0f6oUqKoXtPY6qdRatnjyDiFAuJHf+JzuTWyl7mPYhnZEovXdUybFO9z4fq0+CaUfqy1hmklE7kmCxjrTafAxlpbjyWkh5b8nJ6nDnTRgo6UbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kQnOObQ2Tw+DASc46sbnZDsJpR8alMQbzKHA9EuyZLQ=; b=vyGLD+i/IkIVJG58TYcmkoBRzvdxsH+zvwv80U5OB3nsCG1vBpCEAVkou5VP8jTnnRFFwDbhw4VJGvbGEw+4osj3YopRgG3Ndgt6B1PxwsGcsLt7/g8Ugsu/4kl/UXuvUd9DxNZoukTIWTvhB8MfsPYPEzKXqFlmwkEhKlkpe0pyjBKKkvXoqMpc380u5WUjemJHAX4CEn5PLeFhSa6CP9DcgBfnbD1PmZxDOgphu8VomojjoQWWVq75JUEjzw/e2kRwDuwZSmELF5Wz99m+NJf9mkcfjir+aARM7KHoRuVCE/q+UKVadU3BsojMVPfXeA6HMR3uBs6blxKnAnpR8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kQnOObQ2Tw+DASc46sbnZDsJpR8alMQbzKHA9EuyZLQ=; b=qeQop9MyzYfn6bYVIZrgpnuKa2mVwy7bSkYbfZVPAixTXqMwZVlvlH0dgcLPomFU4zNXQjDTRNNrSWsPlYVfp0ZxbhIQtSHELp8PODv+WFC0zWobcR3Bvs4lOE5Nd5IRkT/+nYDGQgx9NIffYNHP5jIKdWvvIHniwklfAsoRks3H9YU91J73dS5nCm0AAPYUfgGrMQli3mBk1yOcIeEDP/bsi3HbjgfHSXJMuLpC9gR59nYwVzxfmEWf0NhslvL1VDsWkCZsyxEfLEe0spBlveInIPeaWx6ECZcNbZVg+ycwomeYAOrRBeL6QZdEzbHRa17pEmN7+1nSUd8CtEW+0g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by DS0PR12MB9422.namprd12.prod.outlook.com (2603:10b6:8:1bb::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.27; Tue, 6 Aug 2024 23:41:24 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7849.008; Tue, 6 Aug 2024 23:41:24 +0000 From: Jason Gunthorpe To: acpica-devel@lists.linux.dev, Alex Williamson , Hanjun Guo , iommu@lists.linux.dev, Joerg Roedel , Kevin Tian , kvm@vger.kernel.org, Len Brown , linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Lorenzo Pieralisi , "Rafael J. Wysocki" , Robert Moore , Robin Murphy , Sudeep Holla , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH 0/8] Initial support for SMMUv3 nested translation Date: Tue, 6 Aug 2024 20:41:13 -0300 Message-ID: <0-v1-54e734311a7f+14f72-smmuv3_nesting_jgg@nvidia.com> X-ClientProxiedBy: BN7PR06CA0048.namprd06.prod.outlook.com (2603:10b6:408:34::25) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|DS0PR12MB9422:EE_ X-MS-Office365-Filtering-Correlation-Id: 4a02c72c-d302-419b-41b6-08dcb6714804 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|366016|376014|921020; X-Microsoft-Antispam-Message-Info: G5xULkvCiGOhwA0en24Bf5nJWqRHZ0vE7YXvCgCriW6PR0AOzhpD0yyKI8JRAgcxvUQ8C32Q/+rSs1P9Z6BxvzITP73gXu6Shzc0UEu4u21HJHaEnBIv4zSHYUymfPTvmltsI1VSy9RcOtKrpBbmJiSZSeTedt9GBokoE0helixQV2gW3pGqG5bJNfOxr2pU1nLE12SbRLVR7+NM2OkBS/ybwrtCyBSSUHmwNuz1QHKq9juPdOH8dUi315mK6BpxjTiGXXHnF+YYRu3X4qCKUuS6yjeDiGuUS0vyQAY4414jIeAKlcFqkyi2Vn4Gd75CmNPMK3CY7XkcamgrCq6XZdYxfARhn1hhk6vj1is9VCpLApGzidPFM+lGv2VW6ZuvnOAh1SbKunYF0PVXVTLuCP8jkT7R7mVGlrga2dbXxCRFnisM+tUn9xRxQ+y2dABI1SU9MMnEA+51RYCFs5G1OZ7XHXr/j+s8SqYunZGwucCqYMqgBmm2MtSleYzUSuGwVmfw/x6oMCLmfR6sMovXLR846X4rgjywOZqM1dE1POby21fHR/5AT5gei0S00UoqKLCfBQ4Cl9gMaFI2v4mQLUwMP9G6xJDaZWfZmdfAyst7Ext8tdzeGauZsIcbCf6q6t7qh9MQf6sjwcGHR5fngWQwSNGhyA5b7KUt6u7QB3YdA2H0Poc5u2L+LXzcKtd4+eIcIpwWxdFOr99dgX+tXsMMkNdUzrhZmiJW5NCc84xfl8gW6dQc4AiMFeHPrFlB8gk2084VPc39vfCqs/8pOSxGeo5W1ppYLrzwnGBdy8xInE1+CP7KkQ36TQun0Ho6x41LxHPlZ8xI4mgXmXe2GERYFy0FxohgCPmC4O4V0e7K+m/ZadisPtJKSKW1neBs3bqOJrZYrLFnK7I2e+FH0kJg+joweDyAYfdAxK4IxZzt/5vOfg8BRk7wOupA4Xf6c3h0sF02HdqXwI9iziYXAtAT1/cDQB6H61qRKyYftb4/UL2FXzlaXrLrm9a54Fj+XEpU06euxeHmTrUTC3JTuxHugxlIGsKO/65vAr9CxJGKr9as9FkO70T/P+Be89oNHMOLh7K1yOKBoGehCRKRpy4xNKxVQELC/TDQY07JLvK6cZcaLnAUq0ZwfWZsCU/6Uc1Yj4I3FMkPamddRbIiL5xbzYpIDDh00EGdumYjnlj1hrK5BjVg4Y92o6uG9krm8PPfTGZVqbyizlhqZWDN4sbA0kP+w40f9LsVOg20suf005qlgM3p98kJKuVD69TVo0iE+nhbMRyL0KWNxMg3y1R8jtSk8l523aFspNrD6PU8ZgrkFRLZUTxIgpvgSTBXnhsfaMqD5xemM4WwapT6i1qX3DEDePaAVfhkq86L3P6/f71PMzoMuZLIjvr3z4EnKhM3vAMo39E28xY3mwbX3g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(366016)(376014)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: VbOQyB92vMJkJ574gBSR5wRYPnhooisn03LIKZeKs9DCHEsTgMQg76HMvsSMNDBMkLxrrxpcNpJThL0KRBhBbcNx3LfTLKTuondzq3cvrlIZLi8u7XdGETGvSP50+SKKxn2LY+K44tWte6etSv1poEmTcjJKS5q/dhdRw3f7p813kTE0Qz43teH0mB3wn0sN1TGNcMJZSvbt6qy8f0rrCsQGEI0pr7g7G8WaEGSgFoSK6O+Bq4TSTldxTC6TQmcs42ipmbo1eABWq6R/F1BhKIr3PsvO+O7bNgz514hBP5zGKAw8U0bA+lhDzD9KpS1AYxsR+LLq/3pT3RKjp5Fx8tQaA0/F35zNhcnNloMhbZwxFSIqtrBwj9ZKtX5MO7V7hqLs27/58aPXUIzzHTQb3mnOqwkOMUxutWNEL2XXvr72rFwmJ/u3PpejlKVDdONQKGIXYE1wmjO9woqJNvv+rDHV7/5dgf1aabwD0cRk8HXW4Dh0SbNpkB9quphLRtv/6YEkURDjQxj++MF29E7OgNfX84kjDjt6zjQ1KjIg/Hc7/eMVzffSxJPCKsJ8VN3S+wNQwxGr3mkgThwFmRVXyrxB1Ld00Mn8Bdt4yOB2OoHUtmAf7bPcgcEaenb1cV4uuPnoZ+Ts3W0T95ljQnAWTAziulO5X8gQKNZt2fxmLzkAzsKHO6h1Hu1+mN7UhjG9ZulpYxbJEs4K1iPBEM82VxmYvebtjEsPR+Sg5jWlQJzOKX51Ax6HdugTGxptnd8qyC0O39reOcq7DUh06pdQrpBe0Nxq34jBBoUVFN8dqwj1PQPCdJAWXKGoa66D6BxM4xEkac2viK44YTfDy7itxnz8viKLt6BayQP0GjYNXpICXrd8DP1hCHGQ2DcUfh+qNtXjJnmOTuWUUNaE16x3bgp5AOYbwmMAeSmHrvF1vUy+/n45nYvhGpZMSibC+rGhzVytOd/lfG278gWHla/uhsJukCEn8HAHcwbWrD78Idd6wNWoGkadreesQwSWYqU6jujR7NWOw9D5GZJLLqcZOdX6xSw2lCOhSUFvl5JQg5wFQabPy7Ux1kASQT8wBlMVSGSvpTVk3PP99k9df/eEYKOzPLXTqpy8kEIyA371S/kftoflIadRAFYhDdYyD9kd1LqNU0yBPnBCNHZgzKvIKQ6jVPgmIWKLVteUtzvO6Gtqg9fNQVFPII7AQ/XTyZpHTSrUZIrYjCdJfb/rv5upAV4i/zs60QzNMUWCIc4DG9w4E/pyLsacZjpX12gyQQgY7Sbafk2LAoA8UcJX6kLQtton1MSRPCveomvdObBmOXglcgDxwU2FWKMl5v+3S0/1UAqeTnvcQjFW2f5PDq2KSYjUUbvh1+vPpScY3tUbIKa9x1jBaC1F7JF6LBaFG1y7PVcaDDwwWMW3PKE1IofosP2NGwSO0aSLO+jUlNv+rxI9G7oUQ+AdaXBBIXQ2TDElclu9Xtnv4eaUvPr7b8kmakDMek+iXMGGheiJFPSdh07YaJ91Of2fUMraHOOzmXpl2Vsd7BVIpdlvO+3O6hXtQrT2+rYhbt2ZMNPn0kqSSgE= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4a02c72c-d302-419b-41b6-08dcb6714804 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Aug 2024 23:41:23.6382 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mDm1JHpOGPjhPRZqOXT0l3BTuSTXnc7+SM1r/3ydAm7j+KzLg7rxm0m2VeFCcDvQ X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB9422 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240806_164136_092600_2467EEE2 X-CRM114-Status: GOOD ( 15.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This brings support for the IOMMFD ioctls: - IOMMU_GET_HW_INFO - IOMMU_HWPT_ALLOC_NEST_PARENT - IOMMU_DOMAIN_NESTED - ops->enforce_cache_coherency() This is quite straightforward as the nested STE can just be built in the special NESTED domain op and fed through the generic update machinery. The design allows the user provided STE fragment to control several aspects of the translation, including putting the STE into a "virtual bypass" or a aborting state. This duplicates functionality available by other means, but it allows trivially preserving the VMID in the STE as we eventually move towards the VIOMMU owning the VMID. Nesting support requires the system to either support S2FWB or the stronger CANWBS ACPI flag. This is to ensure the VM cannot bypass the cache and view incoherent data, currently VFIO lacks any cache flushing that would make this safe. Yan has a series to add some of the needed infrastructure for VFIO cache flushing here: https://lore.kernel.org/linux-iommu/20240507061802.20184-1-yan.y.zhao@intel.com/ Which may someday allow relaxing this further. Remove VFIO_TYPE1_NESTING_IOMMU since it was never used and superseded by this. This is the first series in what will be several to complete nesting support. At least: - IOMMU_RESV_SW_MSI related fixups - VIOMMU object support to allow ATS invalidations - vCMDQ hypervisor support for direct invalidation queue assignment - KVM pinned VMID using VIOMMU for vBTM - Cross instance S2 sharing - Virtual Machine Structure using VIOMMU (for vMPAM?) - Fault forwarding support through IOMMUFD's fault fd for vSVA It is enough to allow significant amounts of qemu work to progress. This is on github: https://github.com/jgunthorpe/linux/commits/smmuv3_nesting Jason Gunthorpe (5): vfio: Remove VFIO_TYPE1_NESTING_IOMMU iommu/arm-smmu-v3: Use S2FWB when available iommu/arm-smmu-v3: Report IOMMU_CAP_ENFORCE_CACHE_COHERENCY for CANWBS iommu/arm-smmu-v3: Implement IOMMU_HWPT_ALLOC_NEST_PARENT iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED Nicolin Chen (3): ACPI/IORT: Support CANWBS memory access flag iommu/arm-smmu-v3: Support IOMMU_GET_HW_INFO via struct arm_smmu_hw_info iommu/arm-smmu-v3: Add arm_smmu_cache_invalidate_user drivers/acpi/arm64/iort.c | 13 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 398 ++++++++++++++++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 27 ++ drivers/iommu/arm/arm-smmu/arm-smmu.c | 16 - drivers/iommu/io-pgtable-arm.c | 24 +- drivers/iommu/iommu.c | 10 - drivers/iommu/iommufd/vfio_compat.c | 7 +- drivers/vfio/vfio_iommu_type1.c | 12 +- include/acpi/actbl2.h | 1 + include/linux/io-pgtable.h | 2 + include/linux/iommu.h | 54 ++- include/uapi/linux/iommufd.h | 79 ++++ include/uapi/linux/vfio.h | 2 +- 13 files changed, 572 insertions(+), 73 deletions(-) base-commit: e5e288d94186b266b062b3e44c82c285dfe68712