From patchwork Thu Oct 31 00:20:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13857358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57CAED5CCAC for ; Thu, 31 Oct 2024 00:44:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=rQRHfPftorME/58aDpfc57GcioAVlyZuojbIq3ySi6A=; b=jDVIy5oYyhKf3HDReLN44mkz6I zDCKbaeVz4FUf26iKUPmXdvXNITGkfe4B2tuJLgFuFUWtwsPLtUAPc1v8Xy7cCFJBeBsih4Mx7p3y Vjh9EYuYRPaXiKbHSejcR/9fJj/1Ch4vjr8BmeyDh+nRpwSQc1qG2lWg08spL0UsoRAYmW6Nizoba 9Wd6k23pHTua3bp1vRAerrhnb3+/93qfuvzRsryNXyBvsiS8pmOOPeal4gTAtFopZif2cpvzkTY+q iizmPWywRpmF8UF2/trIhJIM73U+MA9L1BAbdTDz+8uzXfF0u+HEqq25i1CH6FuaviA2RipoAgGmL uDcnNwiw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t6JIs-00000002AuP-2Tn9; Thu, 31 Oct 2024 00:44:46 +0000 Received: from mail-mw2nam04on20622.outbound.protection.outlook.com ([2a01:111:f403:240a::622] helo=NAM04-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t6Iw5-000000028V5-30q0 for linux-arm-kernel@lists.infradead.org; Thu, 31 Oct 2024 00:21:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r4V/tqfLXVbMgMy2j8qqogO5HArJdI1LDFSx9HdYmh8hdLC/HKlPnx+GnVc9g9xbxr9XPUesq30vU59HdmKPN5m7snVLokctSZ1bFYJsdMhz0JUQcaxrVXX10y8pQTlGsE8XmXhexzQiMgQNwyXffZW4y/O+Zdw4STk9gzVwfbiz/wWoYPGwHSOLEPtvW86btWf4jVx9425jAzcRRthbpkQ0UH9+k4IKrPxQO8fNIiNdd6bAsH6j0FJmIsKaOTt64it+Sg/dMD8X21MX0QqNNxc+M4qEX989naFMmA6D9Zj+QKa1tamc5pbLTaU70s7LwvvYwUbVBg3atdt5lcamPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rQRHfPftorME/58aDpfc57GcioAVlyZuojbIq3ySi6A=; b=QAyfUxqCZ9/Y5Y4hF2V8hbw2EeUHrEhHbz0lw3kdaGSIDVEWJ2+bV/qC7sZEmkTPIkyBz/gN0CknYeFS7kNxNeQoCzGH8U/7dz0z6xUflYKRYey/bLuwt0ZfgAKUZBIhriqvSdYxYhQxNR4JDjwzZFgDQ1q40APmSpIXyvDPKFEf/b/YHL/zwruZU9Zvie3mwy6YRQgoHBunP1+JjWq9UCQgTD+KnOPTkTl+fmAM60jgx8zytOYbQ+AS3ZWbG9U/kE+KOzx8KiplEf8dK2zIMJodsTcCq3Iu3YYkvOcdgDNkZvpcvItU2Ppg5Q0byYuR7CGlf7aR86GJvFtRdJaaJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rQRHfPftorME/58aDpfc57GcioAVlyZuojbIq3ySi6A=; b=Vp6rLy6EOFVL0T6GJBGNSPfSWUs3CnE9/3swO76sRuAxI+OBMnwCIxPiJzXG03fk97oUGXDCjW4N6f9xUm+LI3aw0Yp6KmeObX+JWrBfVSXSGljTo626dFuxz5ZJLcbn9ITFUFDhf/3cH02alFpoLsPF2ubeThL0sv/NZ6/MdITcyIp3k3WngxKkW7iAM+k1YWhejxXcvZaoAgyqQ1iPmCIPC6wYD7buOE1PfSoDbofl+iGtoBeRtysOaJD2O36rx22Dsz2W4wQZc2qf4JixPy84kS/tB6N6G+DG0xI6N7sUW3hszeAJuMxfuzCpIilcAbHqLgCmeTmMK1LTizhjCg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by DM4PR12MB7573.namprd12.prod.outlook.com (2603:10b6:8:10f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.32; Thu, 31 Oct 2024 00:20:58 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8093.018; Thu, 31 Oct 2024 00:20:58 +0000 From: Jason Gunthorpe To: acpica-devel@lists.linux.dev, iommu@lists.linux.dev, Joerg Roedel , Kevin Tian , kvm@vger.kernel.org, Len Brown , linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Lorenzo Pieralisi , "Rafael J. Wysocki" , Robert Moore , Robin Murphy , Sudeep Holla , Will Deacon Cc: Alex Williamson , Donald Dutile , Eric Auger , Hanjun Guo , Jean-Philippe Brucker , Jerry Snitselaar , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, "Rafael J. Wysocki" , Shameerali Kolothum Thodi , Mostafa Saleh Subject: [PATCH v4 12/12] iommu/arm-smmu-v3: Support IOMMU_HWPT_INVALIDATE using a VIOMMU object Date: Wed, 30 Oct 2024 21:20:56 -0300 Message-ID: <12-v4-9e99b76f3518+3a8-smmuv3_nesting_jgg@nvidia.com> In-Reply-To: <0-v4-9e99b76f3518+3a8-smmuv3_nesting_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0275.namprd03.prod.outlook.com (2603:10b6:408:f5::10) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|DM4PR12MB7573:EE_ X-MS-Office365-Filtering-Correlation-Id: a250627b-ff5b-48f0-2132-08dcf941e3f1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024|921020; X-Microsoft-Antispam-Message-Info: 6VNd+NcAg3V9iuuAzlqVpUdLfuzq/27/3YVyjizotvJ0G5JJWizge9TzBy5SL7YCHOx+RiGUvMLYsG979LQAVKTfUXw8qpVVvMUjWSZbx0JZYk9FlAljE3rdwxTT124r0LGprHITt4/nKFasT1El9Elnq38AwWjH4cx/p/fvL918XAhBqX3cMLkQVGvdpO0HtS3jYlP3ygbNI4YIDMkjcL8+HtHPvVZGLdxtVTEXOlOZBGwT//6IA/wT2u6ZOYu4TQRozAXft7BcxZMvup2aleZwnb7y25zjivk+Dqy4c90NDp2WGLHtQWt1cTdG67VaJWcN65lB7PoZsT02c7A8bdfiTscM2SmPUokmr2KwSFlMQvhp6b37g69E7THGfAL7VTS5z5cjfffBzLAhOK22sESswvBoSwa0ZUvO1Av7a/hhasjzONHqMdzEf97M0cmHH1sCtk3FdtJAWyyox6u5DBBljfKCiNCfGO3qQMet0EJhVB0UnopEfpCvZhFcGCp/u3uULMV44NvKk1Z9CnbZ1KXXO3l4naU2a4lclHRaSHgLYffsGv2LR/PfoNJCOryLuIYkPJhaQ6Ye184EyTd8Au7S/3zbjIcNJQZV0eQWbNF/7TmDbNqls+4QVMGvb+D059cVKvNhTG17bKi94EztsHDFFW/qPHsSkT0iVZL3do4Bqmof98n3omVWjI8Ak2a/F/vpXBL7k2HHNSRwhw6VWtq4b5XNMxSkstt2kDIjbO4NeCqUM6fR81lcRVAed3bsYyVY4HWBAt5jNcaAh5wa/O9A4xChT+vbsPV/OdJvc3aPk9OsA1QA42inauDulLgGREBSwkhC9bWMYCyP3j5PEI0QxIIjV1buw1XU35z/L8Pdyvrwz7QOLFvvNF4mBhDPeMvtLpvlgnpvH2zay6pPZbN6rLIkYdGOCXuE0bdojQjkAoRGB8K0Ri0aZIIR+Lia/XBheDqXViertCPa2IDWc0jDPcgIq64FZ4V3BdL5pNqK4idUD5TsS+fLCd1Wigc2ltZHUU8rNucd+mIuwQkfKvaLx3++WbrLgIahgYkyTqzEFxujywwFAyVm86wBgMzWEn9O/UbWvAXrU5EjHwSchlk7mto3LGKqScQ5GrJFomQKYsxesTaNzYJHF+bP81QwHnBp2eymLcJa+2o7oTws/gsusEEdkOLjgJqiP/x6vl/YwyNfrjTob/00TrdOK9Lrm2ZLA2qaTYsWkaNiUjANbwXqU0AqrCUQI/UZIPshcyY/d4OVSEYHzd5e15YCtJQapmRUZpAa34cBGg7NEuJzuU9p2QPJk2A0qzO312zOWG/xsMtvkSYvQ0Yr3sxiuVvDqtS6kUeMDPn5zi7MroLDu+W+wMOXHn4GjpJaTETC8ws= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: U/AIEd0S8lI69ZQU4yErwElz9YbvdIJa2F1KYh6eBe9EbA7Aot3tbMGt/PsxqBaLXkD5lk4IMrXmwykA1a3pA0eaj3j2m5bTajVoU4calz6EsrVN9ctAeGVAEzgn6A7qFCFTqPA7AqaDhc6MH+MiWEm/PNcpkkQhUPbPeoBE0+DYAekHZTW9CZdcIzICkX1039poBRs1J8Zg78OxuA7W/1CpOClkI9QsOjT9hlSpzgEa1CnexhxvV89aW4fbJ44t+UuF2CRyFxgT071d37BzyWN9toUfuWc3PDtH8BOVbMHPSrVM5Iim0TJ7Qxf4DT7d07J/hHL8o2WuJIKTfFmL2tkr/pY4hvdsCxf7aWowqnV8RojDpqEa6Z6Kj6JqjH8rK0E7mQOEWzkrEZ8LcivjW3+AWmTaorDfqt20K2vu/mNd9JlPIkO/wAxbybKlFljiZWfUfY9eLP1chlACGJpCh6b/lNoJ2PNxLDlg+xytTBRXC3CdCr71mta1yqRFuZMPELbwddaXK6C8/qhgGBCS+OZ+36goSlU0h6Z1+3WWGstJlypvHoIwv3a74gjBaFS4tfhUmSbvh0EebFvAggwM62Ss3DQRHuarAeEqSR3VgQGnlda/TkFHuJvR82JnXM4fTAnNRkuycZ03N3SDI5KIlIUfrfhr/JrKyI2PB4XIKKINKT7B4/f0BiOxhow2cdyiRU6wTwSSR1DKcxxbQucbZBsLW/hHIFJTQRAzgum48VXcumD2Dto+pK9AP6bfgUQx7VfidABp9CN3oI2Q9uEpviCfNX7bWTaTdPuJSkRHk1PhkPjP21qypcwf6ZMQTUECQGXKWXU5pHVmkGfMVRyQtR5tF/PN9xPCLlnIGPLJiO2an2e92atr3kcvo0Xde/FS4C1MU16jLbKPVLRxcJ/67E0J+MD87uzvcwenIikvHRcLUmwVLLxB/u1e/oCN3F8A/rkLg101cwJR8K2BJsWOMN2rQ3MGHYztStQLwigpBKrbBIwJohmzxclF3/tMv2gdF61AijCqIUit6vYM41+4Ig1unEfhJXv/qoBOG2XfhOuInLO0Lq0OVIRpLSrBQg5r1dDhgyzjxuSSzfmAdtkXgA8Vdag7g93bV3wO2SuAXhvWVFsdvOevvu8u/sigda+IirxTZP9sMsA4QOv9jGYemSpdaejSSDuGdck6Z3xqt8nCNRMJtcMDP21jqnSzvPTwjkuFVKWTGM45gf6jK1B3jo+5Mc3KCvjPOaQ61Z11PFc4Gj9SRzq1DPKR0pgKnO/QDms1yr5RSwORd5DiIOwBuB/qQKxlL9CEfX3wXl7WZRCaax11HztgmQQt3tVe4y28mtK3LeU9XIdhLQg75mX/V0RRC8T3NDvxJ18PXZh6bRmiw/+pAtMPvZoVg4qgOCro7wXwVlSYPIeeKkiX0jhpv1D0kHfWPSDacOfreNgsfPZm4fUN++BGHi/6Ir91ifyIxZubJlqkA0JRYCj3X3Np1kq9LePm+zcM/fdSV1QDFxtnHB4JnA1PW00umTO5oqYCuaoIkDyfSjgoo00kGDPFWVWyPBWoi6S+f91xx6YR6ws= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a250627b-ff5b-48f0-2132-08dcf941e3f1 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2024 00:20:57.4587 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XaeoloZJaFe+WphM+iJsnCyVF+T5m64kxEgkJTER4ORtxlDgu2QDEtTAoPKKU3S+ X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7573 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241030_172113_806381_571144E5 X-CRM114-Status: GOOD ( 21.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Nicolin Chen Implement the vIOMMU's cache_invalidate op for user space to invalidate the IOTLB entries, Device ATS and CD entries that are cached by hardware. Add struct iommu_viommu_arm_smmuv3_invalidate defining invalidation entries that are simply in the native format of a 128-bit TLBI command. Scan those commands against the permitted command list and fix their VMID/SID fields to match what is stored in the vIOMMU. Co-developed-by: Eric Auger Signed-off-by: Eric Auger Co-developed-by: Jason Gunthorpe Signed-off-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 134 ++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 6 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 + include/uapi/linux/iommufd.h | 24 ++++ 4 files changed, 166 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c index ab515706d48463..2cfa1557817bc1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c @@ -209,8 +209,134 @@ arm_vsmmu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags, return &nested_domain->domain; } +static int arm_vsmmu_vsid_to_sid(struct arm_vsmmu *vsmmu, u32 vsid, u32 *sid) +{ + struct arm_smmu_master *master; + struct device *dev; + int ret = 0; + + xa_lock(&vsmmu->core.vdevs); + dev = iommufd_viommu_find_dev(&vsmmu->core, (unsigned long)vsid); + if (!dev) { + ret = -EIO; + goto unlock; + } + master = dev_iommu_priv_get(dev); + + /* At this moment, iommufd only supports PCI device that has one SID */ + if (sid) + *sid = master->streams[0].id; +unlock: + xa_unlock(&vsmmu->core.vdevs); + return ret; +} + +/* This is basically iommu_viommu_arm_smmuv3_invalidate in u64 for conversion */ +struct arm_vsmmu_invalidation_cmd { + union { + u64 cmd[2]; + struct iommu_viommu_arm_smmuv3_invalidate ucmd; + }; +}; + +/* + * Convert, in place, the raw invalidation command into an internal format that + * can be passed to arm_smmu_cmdq_issue_cmdlist(). Internally commands are + * stored in CPU endian. + * + * Enforce the VMID or SID on the command. + */ +static int arm_vsmmu_convert_user_cmd(struct arm_vsmmu *vsmmu, + struct arm_vsmmu_invalidation_cmd *cmd) +{ + /* Commands are le64 stored in u64 */ + cmd->cmd[0] = le64_to_cpu(cmd->ucmd.cmd[0]); + cmd->cmd[1] = le64_to_cpu(cmd->ucmd.cmd[1]); + + switch (cmd->cmd[0] & CMDQ_0_OP) { + case CMDQ_OP_TLBI_NSNH_ALL: + /* Convert to NH_ALL */ + cmd->cmd[0] = CMDQ_OP_TLBI_NH_ALL | + FIELD_PREP(CMDQ_TLBI_0_VMID, vsmmu->vmid); + cmd->cmd[1] = 0; + break; + case CMDQ_OP_TLBI_NH_VA: + case CMDQ_OP_TLBI_NH_VAA: + case CMDQ_OP_TLBI_NH_ALL: + case CMDQ_OP_TLBI_NH_ASID: + cmd->cmd[0] &= ~CMDQ_TLBI_0_VMID; + cmd->cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, vsmmu->vmid); + break; + case CMDQ_OP_ATC_INV: + case CMDQ_OP_CFGI_CD: + case CMDQ_OP_CFGI_CD_ALL: { + u32 sid, vsid = FIELD_GET(CMDQ_CFGI_0_SID, cmd->cmd[0]); + + if (arm_vsmmu_vsid_to_sid(vsmmu, vsid, &sid)) + return -EIO; + cmd->cmd[0] &= ~CMDQ_CFGI_0_SID; + cmd->cmd[0] |= FIELD_PREP(CMDQ_CFGI_0_SID, sid); + break; + } + default: + return -EIO; + } + return 0; +} + +static int arm_vsmmu_cache_invalidate(struct iommufd_viommu *viommu, + struct iommu_user_data_array *array) +{ + struct arm_vsmmu *vsmmu = container_of(viommu, struct arm_vsmmu, core); + struct arm_smmu_device *smmu = vsmmu->smmu; + struct arm_vsmmu_invalidation_cmd *last; + struct arm_vsmmu_invalidation_cmd *cmds; + struct arm_vsmmu_invalidation_cmd *cur; + struct arm_vsmmu_invalidation_cmd *end; + int ret; + + cmds = kcalloc(array->entry_num, sizeof(*cmds), GFP_KERNEL); + if (!cmds) + return -ENOMEM; + cur = cmds; + end = cmds + array->entry_num; + + static_assert(sizeof(*cmds) == 2 * sizeof(u64)); + ret = iommu_copy_struct_from_full_user_array( + cmds, sizeof(*cmds), array, + IOMMU_VIOMMU_INVALIDATE_DATA_ARM_SMMUV3); + if (ret) + goto out; + + last = cmds; + while (cur != end) { + ret = arm_vsmmu_convert_user_cmd(vsmmu, cur); + if (ret) + goto out; + + /* FIXME work in blocks of CMDQ_BATCH_ENTRIES and copy each block? */ + cur++; + if (cur != end && (cur - last) != CMDQ_BATCH_ENTRIES - 1) + continue; + + /* FIXME always uses the main cmdq rather than trying to group by type */ + ret = arm_smmu_cmdq_issue_cmdlist(smmu, &smmu->cmdq, last->cmd, + cur - last, true); + if (ret) { + cur--; + goto out; + } + last = cur; + } +out: + array->entry_num = cur - cmds; + kfree(cmds); + return ret; +} + static const struct iommufd_viommu_ops arm_vsmmu_ops = { .alloc_domain_nested = arm_vsmmu_alloc_domain_nested, + .cache_invalidate = arm_vsmmu_cache_invalidate, }; struct iommufd_viommu *arm_vsmmu_alloc(struct device *dev, @@ -233,6 +359,14 @@ struct iommufd_viommu *arm_vsmmu_alloc(struct device *dev, if (s2_parent->smmu != master->smmu) return ERR_PTR(-EINVAL); + /* + * FORCE_SYNC is not set with FEAT_NESTING. Some study of the exact HW + * defect is needed to determine if arm_vsmmu_cache_invalidate() needs + * any change to remove this. + */ + if (WARN_ON(smmu->options & ARM_SMMU_OPT_CMDQ_FORCE_SYNC)) + return ERR_PTR(-EOPNOTSUPP); + /* * Must support some way to prevent the VM from bypassing the cache * because VFIO currently does not do any cache maintenance. canwbs diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b47f80224781ba..2a9f2d1d3ed910 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -766,9 +766,9 @@ static void arm_smmu_cmdq_write_entries(struct arm_smmu_cmdq *cmdq, u64 *cmds, * insert their own list of commands then all of the commands from one * CPU will appear before any of the commands from the other CPU. */ -static int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, - struct arm_smmu_cmdq *cmdq, - u64 *cmds, int n, bool sync) +int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq *cmdq, u64 *cmds, int n, + bool sync) { u64 cmd_sync[CMDQ_ENT_DWORDS]; u32 prod; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 01c1d16dc0c81a..af25f092303f10 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -529,6 +529,7 @@ struct arm_smmu_cmdq_ent { #define CMDQ_OP_TLBI_NH_ALL 0x10 #define CMDQ_OP_TLBI_NH_ASID 0x11 #define CMDQ_OP_TLBI_NH_VA 0x12 + #define CMDQ_OP_TLBI_NH_VAA 0x13 #define CMDQ_OP_TLBI_EL2_ALL 0x20 #define CMDQ_OP_TLBI_EL2_ASID 0x21 #define CMDQ_OP_TLBI_EL2_VA 0x22 @@ -951,6 +952,10 @@ void arm_smmu_attach_commit(struct arm_smmu_attach_state *state); void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master, const struct arm_smmu_ste *target); +int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq *cmdq, u64 *cmds, int n, + bool sync); + #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); bool arm_smmu_master_sva_supported(struct arm_smmu_master *master); diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 125b51b78ad8f9..2a492e054fb7c9 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -688,9 +688,11 @@ struct iommu_hwpt_get_dirty_bitmap { * enum iommu_hwpt_invalidate_data_type - IOMMU HWPT Cache Invalidation * Data Type * @IOMMU_HWPT_INVALIDATE_DATA_VTD_S1: Invalidation data for VTD_S1 + * @IOMMU_VIOMMU_INVALIDATE_DATA_ARM_SMMUV3: Invalidation data for ARM SMMUv3 */ enum iommu_hwpt_invalidate_data_type { IOMMU_HWPT_INVALIDATE_DATA_VTD_S1 = 0, + IOMMU_VIOMMU_INVALIDATE_DATA_ARM_SMMUV3 = 1, }; /** @@ -729,6 +731,28 @@ struct iommu_hwpt_vtd_s1_invalidate { __u32 __reserved; }; +/** + * struct iommu_viommu_arm_smmuv3_invalidate - ARM SMMUv3 cahce invalidation + * (IOMMU_VIOMMU_INVALIDATE_DATA_ARM_SMMUV3) + * @cmd: 128-bit cache invalidation command that runs in SMMU CMDQ. + * Must be little-endian. + * + * Supported command list only when passing in a vIOMMU via @hwpt_id: + * CMDQ_OP_TLBI_NSNH_ALL + * CMDQ_OP_TLBI_NH_VA + * CMDQ_OP_TLBI_NH_VAA + * CMDQ_OP_TLBI_NH_ALL + * CMDQ_OP_TLBI_NH_ASID + * CMDQ_OP_ATC_INV + * CMDQ_OP_CFGI_CD + * CMDQ_OP_CFGI_CD_ALL + * + * -EIO will be returned if the command is not supported. + */ +struct iommu_viommu_arm_smmuv3_invalidate { + __aligned_le64 cmd[2]; +}; + /** * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) * @size: sizeof(struct iommu_hwpt_invalidate)