From patchwork Wed May 8 18:57:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9F59C04FFE for ; Wed, 8 May 2024 18:58:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cia5qKBna4G4Uljrn6DYiymyzefFaqirgnHEHQCP8iA=; b=BCd7HPljyu92zl +kDMSB2lZR7pi/wlQrzNu+TodtCOcwFzuxnT5ZvFertprF3G+8SKHoh/WoUYNo3ov54QWiZNRt6dM zfczp6ugsGoexOkR14ux9Cpy3I5oVhwV64FnLwJczimsVltzr4QKF7MkYXB9L46ewD5FPY3YHOsvB KNUo40i1BYnrEvbj02RksPoH+U22UP5kRUVb9uRPWwex5rsb3i0HRwtdTZK/J7Z25hlFuiI+NeizQ 3JVrpFu/xBjq0DeI5Zrt2j2/BKtMog5z0eK5QBZD6/48nEa09TlyJaesV/lKJ8JUV2Hn6ZxubNMm9 JvfvbZQDBISdoEDFwNqQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUW-0000000GcVs-3oaw; Wed, 08 May 2024 18:58:12 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU6-0000000Gc8t-0lqp for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CZ0Xe0Kz2cgxKEmyXlF9SCNM7k2hkMghGi0lKR3+634d7oxR3EfXHJNiZV9VZQ73CuBDf+OCFwbFTaSnD1hWKg56fFJggqVhJ3eoDBhlDfW6xmCV8Bd3SPsP3kw0kYU/R+6GXe8DzJLMiXQjDGbuqKEZ6jkyZ163lYuN4JkO8zhVOnsHLB2DBF/VBMC/TRw8ydt5b/ekqdXnWlmfF8hN0y7EYqHxmpQRwAqZwCI3DoJkkCsSLzLpPzRfHHrUk+cxEJKo1nbhEM9xEIIiSHl84NCSP+74+9b/bxWmd/QcWSMnLfsJTCYMYGuQiDZii4fx112Ssb/k+bcJytb8j1AhWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O4YOnaEiRj6hBwAdgo+aFd8H2MqvXXS+W9MsIEdNjqQ=; b=IrJlDlJmKDm0oxBHaK1rP0o1rSwA0mIMepwplc4XtjGKR1U5+sCT+sHUoXRQwDXaEt1UtQJmhbn9GFyYwe+6SLuG93tLnmZpYVi9hyZatSuCruu9l6SM5XnhTFc4VHGDtbQkY58VwVOgtIyNJOnu6AdGkvUS9NATSNjq+u7ahrFLBIjASns953sMxr/2nzHgIitdQ0BmK4qqbJQd4yQe/p8emEvb6AEyJ+Pe0VlQlzm6lrM+GJTtuXM/35YbusLBaTtkow3PHIVOJGX/fd0EJLmr0XQysKau0xxg/CxY8HCxcSn7sDRl0kEsiLqBLqKffrqGrLjJdN7twEGUua/+tw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O4YOnaEiRj6hBwAdgo+aFd8H2MqvXXS+W9MsIEdNjqQ=; b=pv5yALMMioFId1XxgZvQuN8dx5pm75PTEf3Uy3PyY6D/GnPV1vExPPkFZRHZkzNo2AtiCOBuyOb+d549Kum5ob8/4hg1x27W3Z3Di8ILnrvIkVKvpuDMet0tOglKiMMveFPXtIpJjLIsXEBTxIvqfmEbdGq6k3aBohfLO7ur/74R10sQbqz22rxa2NFcn9+R8qA4Y4/SfyTUNHICc80w6S9iZUnzov/3h8TTKwVRsN6CIBZFNm+8c8JMpqhcGQ72ewlc/MfZRA31i9dwCrBGs1gS+ad1L24s3LfKXsI9dXzQpuZ2PiUcQ045lSs7AfIInaJSM3aRv3z6P6XrU2XSxA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:27 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:27 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 05/14] iommu/arm-smmu-v3: Add ssid to struct arm_smmu_master_domain Date: Wed, 8 May 2024 15:57:13 -0300 Message-ID: <5-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P222CA0020.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:2c7::25) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 4b727ee0-f716-4485-1694-08dc6f90b37d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: jS60I++WJloUYcN3RYPrjFSRiTS7hrjqswWTzd7B7GLjoBDeXpgUVFnQxWrS0I1axOyTYAMwjQJlC1Ebqum6FTZlqfw93e1xd0dNUBdPHSLJazozCndglga7vkGPeOMzGW3BjcqLusNI9BjU0bl1ayYx8sSN+4xFRj4488aGxn/KSyQzQJmmbN+/xMdkdq1CchapY7nhEvuk1MSE22kW7X3tLWlt577W+4dCUe2RAdzh6IXTElOaVm8No9UoYFk4wcsRvBPdByYU4eAvCc3d5v5N9+2XjGX/GKSNaJVUnV8UK2ir7MqA6gS2Y+OZZYX+CaDVsJMrEkxtYzHU/gIhrU3p/ePIjiAjeumYoB4y3PwPJhynnl0MCjiP5CzxZYmeCXvT2RytjGsyamhkkAfaCYYme0fKjxfRSvGWyPI/+mMqzlgBohs9FmuuZ3m0ocvrzMWHSs8yLECQKNuTNUvG4jvkkxAoRZFlI9/Qv1eFZX0RZOHprd/8O+XUantrs5jvOauf7PJ7pqJaH/pTZ7Jrp0abOPIEbQ84y2oEO8Kqu3EW6Fq4DsCPPBVITgzUFbZ029FW5A+Fh98Wo+aDSHcfxMVgtt6NsMiJdynUTfc4sItkCq08NI3ApVBjZusaoFt5ypvx1Cv3ojzrOPW56lvfa95VnsfhpGndKyTgMW84QJP5nIPdHRIXAhPKEjVo+RpQa84lvdNlKkrje2aOfbnZPT9CUnOMsXhnyNQvYB5SuMBqt12tgMgSyz+nUyZmIHreNO+af3hmRWmuRbvzXCsWdWcjR6ILm+V52CqPWMQI3EZ4Z7eq4e2jAnU2mmZTANtMwGrVzSSeLHIujQH7pWtTNxqRZlSa3rFX7m/Ujascn6lJozvVS5LZlkI4tM8MDYiMPUI2a9iMoQWEyKBogrT71CqRYcE+eHN5OS5BBxU4a79gp2k2fHYb5G+cSOCFoTIFTgFLUWqvMhZY/mVdjDXBghuxCm7f/FidfMiqlZR22KyUjXF1Z9EoPABFBACU6WXxzwYg04E7ipxfTiRQjYtk04/6HrIGh67PVsg0rnEoKKoaUdK8W1XBcbBCKcj5EXgAqE4rhr32wMzB8q6ZWQ7bLGWiuX7JW8jNqozxplAn/oJTo7D/xotNHZigB0ssXYXhR4OYMxgxBd+UbJVLvd2yY/DyFPXM/z7tRu98JzXlbySeb3vTLOyMMvmbO+ysqr9U5gToiGJFUDsWxYJP0CvVYDJtsUe1pqWEEI1XUMoEJ8y0/KFtO6NpEKVe5m6yR/IqUX5SXFUv6Uw3VVnG4So+CQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7NBstXE2rWY9XfxOpMLc5Gokhp7azghRMZySfpvwmTBoDc7vH5Opcs8j6OsQYZbwXWDJkZAmo48DempSP8GpFkL6eV7w22aynLOmafrZ0GK4iFdF9g/01p6URTNibshcPx2m+dPjGHeH14ixVth4fYEkymNUWIm1Th7vtRyX+O13CF/w8ojtlnC8PqvEUmuQzDo7CmJGYI3th8c0tPK1cqiZk2IWLE+j6hplaUbLOSeuOW+kYqz9cgajxpUGqCJjIq+MaNbbthr35bqt3SjR5PKH7ONhU7TIS4EBUMTO4epmfiwo5762heM2L23dOvxSrwnLEiOrfkOxVspXf9EO4PFXzc5e57gGFkcZPeJI54eK12Ndzs17OPuhsdsrd3lr6P3MFx39ZyPgGVCHHu+8NcKJwTcso1AyCeKq45iG8sexWFxB/qVTYvzt2kx5re4dvTzQrq3ccuks+1XADffCfA5FKLAV/cg+H/KF0mz+S/z83RbcVz4isKnf67fhIfyJSOktGCf601neHgUsPQvm+uh3eGRdPkeWP7JUfe26oaZDGp38SetuUumqgJRG62dkc6UJufB0P+VrnULoPo80OuyL5Pva2A/wtBEPogvNlshxPqccsABd3RrB2RB88HYqSELDqb45RiPUCw//G5m7tP66tQsCE5AXaSCbSNm/nHcLafixiYR2397FYB4TK73zodETmhwTeUPiVq7g3NCMSIpuC7NjivqdLKmjqqTnbvAbOCsbeT/kW+l1t8XXGiBFxEm//LkcPdpl3jVzp8fEUeDHRMvOuc4+WYoe1mGYf9CAxaE5V4mod/HbiuV3C4sqVY19JF+26yoFUK2TYp1ecfDrt5J93LFmIewFea+4PTAmwJqQO9XugXjBG3DNE/ipAi63ibFQuxXj47FZqB5og5v/odMv11+41fiZFGwWLDtj4/B3TIv/9tqAujz6TqE5pKAdmmXKlutumrqOwOsfqKZQPkISoGqDEH9uOlelWiBtL1oF+zcF3Y4NgGCWVY+C09Z7osgFsUQr2l6HGOxPM6Gf6aJ7yxirw1tkSUSPaKQXNiFTWS2MQOSC+dcTZTXJxSYNeB0XNtTe6p+b5x0JFIAzKe/zfOCIIAuBIesX3Fltuf4wgRuhoYiSG+aNtaK3971/+c9a2kT47OCquMSsVHCq5NS0wVAlT0RlGKEW29iFVVK1UPLd3rROEWelmzcxFBKzDIyV85DmkYOKm6S49YOAfinX5XaLU8TkCOcWzI8OTGvqjH0kXlfmRIfKgcT6YnvigYJpZazwvfW4cz9lneX+lKOXgZTiI8a4Czg09cQ75fhMSOrzbNfTMLKCkoY2hBtqlIMysKup1Bj71kFUB8Q/6nEx0oaUikuntHILk7M+H1GBLODPMGg8TTjkjQ8FuNaxYZebPqTf/KPv3M+oKStrFwV4TR3z6gsalsaZvasDVq8SQIXgzZBqIiDScOIRPSkQvNcIStxZ5LyLiSv6sh1HkjkW1uxqovzu3KTjjrNJNUgQUmhOc79Z1xLS4tRQRTtIL01J3XzjSJeCk+5pJCjt7PR13LS5ifG0z9Hea/zelVWcg2Iz8ZXpC/vACZra X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b727ee0-f716-4485-1694-08dc6f90b37d X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:25.8719 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jg4AatnrH6c6ISUFoenZhWtq8uR84fS+7BzpwhIU2rOiO86l/xPgUhi/WFoUUrl3 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115746_435818_87B7BCDB X-CRM114-Status: GOOD ( 15.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare to allow a S1 domain to be attached to a PASID as well. Keep track of the SSID the domain is using on each master in the arm_smmu_master_domain. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Reviewed-by: Michael Shavit Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 15 ++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 42 +++++++++++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 ++- 3 files changed, 43 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index cb3a0e4143c84a..d31caceb584984 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -47,13 +47,12 @@ arm_smmu_update_s1_domain_cd_entry(struct arm_smmu_domain *smmu_domain) struct arm_smmu_master *master = master_domain->master; struct arm_smmu_cd *cdptr; - /* S1 domains only support RID attachment right now */ - cdptr = arm_smmu_get_cd_ptr(master, IOMMU_NO_PASID); + cdptr = arm_smmu_get_cd_ptr(master, master_domain->ssid); if (WARN_ON(!cdptr)) continue; arm_smmu_make_s1_cd(&target_cd, master, smmu_domain); - arm_smmu_write_cd_entry(master, IOMMU_NO_PASID, cdptr, + arm_smmu_write_cd_entry(master, master_domain->ssid, cdptr, &target_cd); } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); @@ -294,8 +293,8 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn, smmu_domain); } - arm_smmu_atc_inv_domain(smmu_domain, mm_get_enqcmd_pasid(mm), start, - size); + arm_smmu_atc_inv_domain_sva(smmu_domain, mm_get_enqcmd_pasid(mm), start, + size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -332,7 +331,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm_get_enqcmd_pasid(mm), 0, 0); + arm_smmu_atc_inv_domain_sva(smmu_domain, mm_get_enqcmd_pasid(mm), 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -411,8 +410,8 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm_get_enqcmd_pasid(mm), 0, - 0); + arm_smmu_atc_inv_domain_sva(smmu_domain, + mm_get_enqcmd_pasid(mm), 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 01e970f6ee4363..3a7ef73994b57b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2013,8 +2013,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +static int __arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + ioasid_t ssid, unsigned long iova, size_t size) { struct arm_smmu_master_domain *master_domain; int i; @@ -2042,8 +2042,6 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); - cmds.num = 0; spin_lock_irqsave(&smmu_domain->devices_lock, flags); @@ -2054,6 +2052,16 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!master->ats_enabled) continue; + /* + * Non-zero ssid means SVA is co-opting the S1 domain to issue + * invalidations for SVA PASIDs. + */ + if (ssid != IOMMU_NO_PASID) + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + else + arm_smmu_atc_inv_to_cmd(master_domain->ssid, iova, size, + &cmd); + for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); @@ -2064,6 +2072,19 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return __arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, + size); +} + +int arm_smmu_atc_inv_domain_sva(struct arm_smmu_domain *smmu_domain, + ioasid_t ssid, unsigned long iova, size_t size) +{ + return __arm_smmu_atc_inv_domain(smmu_domain, ssid, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -2085,7 +2106,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -2183,7 +2204,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2518,7 +2539,8 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static struct arm_smmu_master_domain * arm_smmu_find_master_domain(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) + struct arm_smmu_master *master, + ioasid_t ssid) { struct arm_smmu_master_domain *master_domain; @@ -2526,7 +2548,8 @@ arm_smmu_find_master_domain(struct arm_smmu_domain *smmu_domain, list_for_each_entry(master_domain, &smmu_domain->devices, devices_elm) { - if (master_domain->master == master) + if (master_domain->master == master && + master_domain->ssid == ssid) return master_domain; } return NULL; @@ -2559,7 +2582,8 @@ static void arm_smmu_remove_master_domain(struct arm_smmu_master *master, return; spin_lock_irqsave(&smmu_domain->devices_lock, flags); - master_domain = arm_smmu_find_master_domain(smmu_domain, master); + master_domain = arm_smmu_find_master_domain(smmu_domain, master, + IOMMU_NO_PASID); if (master_domain) { list_del(&master_domain->devices_elm); kfree(master_domain); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index ce01d3fc768ad6..23d8a5b7912069 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -772,6 +772,7 @@ void arm_smmu_make_sva_cd(struct arm_smmu_cd *target, struct arm_smmu_master_domain { struct list_head devices_elm; struct arm_smmu_master *master; + ioasid_t ssid; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) @@ -803,8 +804,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_sva(struct arm_smmu_domain *smmu_domain, + ioasid_t ssid, unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu);