From patchwork Tue Jun 25 12:37:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13711093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5DD31C2BBCA for ; Tue, 25 Jun 2024 12:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CrTXGyZLQ5FtW0AFYk6OXLwD0FCE7zEIQ9P+Er4Ywho=; b=eB9q6blqHKA+0W3lsgSQ+V99Ji fetkhEcSxwZ29to8MkT+QOMpa8tvhsfVH/cCuMvVNE88iF+7l+N1eUyrDGbeIlbb08fGu3p9JtKA6 LvOkuXrqnP06uQ62/aaIkt+dcn5ap09PkynQiJbz+oInh6AcnOdb3LofBYjDDfPbIEiJImnj7vidd gpvPdNLPqsWcssHeNB3qGgsxyX3TwOcZIlcH61OB+bgxv0y5b0VhrHTK9b/Xy0HZRpJjOmpaj9jXP SjU73HpkhCMrkx62YrUxf2uKno4wJQiRqoCEyFGpGTwXEhDAkdd32ybh3eLDawj+sAnl31EKMYslO /z2RZzAQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sM5eX-00000002r85-3Hkt; Tue, 25 Jun 2024 12:52:05 +0000 Received: from mail-dm6nam11on20601.outbound.protection.outlook.com ([2a01:111:f403:2415::601] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sM5eS-00000002r77-2LnD for linux-arm-kernel@lists.infradead.org; Tue, 25 Jun 2024 12:52:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EgB4Bn4kDe7hBBUvKICIDNaRoB0EF4ly8OgWTfpkF0kf+HyEZhtcCOMfMMExgSF99wG9abGHCc61mAJ/Fl6sTirQOwM6/jE6jAE9uohMuMK7tWKIE4Yx4XPq5Ys1NnyhxlxPJhdJ41J/3fA1fjiC165gsONZYGr3HAHzO+8EIrobl3WECTiEPJ1/tSThsYCJc020f3pa8ArWFKw1pvjxdeNp2fygfY2I+QkY0AoklgmrbbnX5oyjMLz/YmyRT70YMK1iB4fiEFBKUSJk1VUYQb0pinG4uYbYkVE4IjV8ddFA0dRRzCz/hTZkarKi6THWtb8RBBgJ66hII0UKiy0K/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CrTXGyZLQ5FtW0AFYk6OXLwD0FCE7zEIQ9P+Er4Ywho=; b=ilpnnwa0R544jxeEHQ18lH2ogLiC0J8cNMOEsGkoDU+QGDKlA97v47N6CM4JtWCofOWs/B+XgpnyStzjLsafHLpXr3t2iCHYtXlt8gnVByIqM/mjL17T79SOPxEWeZQjbykwdqIZXSJaXlvt3ES9nyi7vN7m35E+RnjMHUgymf7zkhGwTzYJAmXtJE8K1fBF9B7seOlPlqCUc5oNDGnX59LDHOnN5zgt6kwu56zsUFa3BOL3u3gZoPOY4fN8gUzJmEwMKyHRkIQ3NRt0wR8YMab/4pBVg7nh3QewH+drGjcZm/RaH5O27r/Qyl0/rOFNQdOuyS6qswgf44c7GktOfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CrTXGyZLQ5FtW0AFYk6OXLwD0FCE7zEIQ9P+Er4Ywho=; b=g2GWgdaDB1jX0Yvk6F+/zx+N6jWsw0qkRR3Uy8CQh9iR3Ibo0eAjKFx8UCygkSVOkGHHMDzHznbyuZwgZ33hl2S8YOjtn8dWJll/se9kKlS8LGmtV5IZFY1cgD6LCZVTXX3Ct5rdcqXgQMNO4WHggvVqXlR2kVcB5r9pnVDWQTSc81HFRMvyQdBYfJvE4xsQqcGBsVBUjZa50gjQRnnSedYQ7+wzfCAi7xw3IutJaA8K6tsyoYDyoWh4dPPmwO3NiB0C+tSgie4TFI5xxwe+pkmBWwHIdbDzXJMkukHXyNly4WDPGHUQed2CbW+DXHtC2ezSG8F+6pnB9uphHlkl7g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by SA1PR12MB8946.namprd12.prod.outlook.com (2603:10b6:806:375::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7698.26; Tue, 25 Jun 2024 12:37:46 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%6]) with mapi id 15.20.7698.025; Tue, 25 Jun 2024 12:37:46 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Jerry Snitselaar , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v9 06/14] iommu/arm-smmu-v3: Do not use master->sva_enable to restrict attaches Date: Tue, 25 Jun 2024 09:37:37 -0300 Message-ID: <6-v9-5cd718286059+79186-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v9-5cd718286059+79186-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR05CA0045.namprd05.prod.outlook.com (2603:10b6:208:335::25) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|SA1PR12MB8946:EE_ X-MS-Office365-Filtering-Correlation-Id: b0699e82-eaa0-4319-564b-08dc95139d92 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|7416011|366013|1800799021|376011; X-Microsoft-Antispam-Message-Info: mWAtimW9ZYLezF3TlbHoxGKfa6tl/BaTB7LS4fa8znbscE/b45gesX3L+4IIXHs+LdyffNpV4Srzm5C3fcAx+PC/xdJyJSfhgSLaWr+Nv7tfRETZlEWoALZt61dAW69LiAyY7NEvU2VKcu30ahsZQycOR+UciSK4NiV9DAfGTWfuzaj1X1kjuhv9JEsfoDJwsXWxy5lFKXALKA7/sJ74KoDTg6O8fTqX+BT2xF0YoDD/bxI160vCfuZ5qngMZGa2j4FtqcpzO9RIhFJmTEDXsb6dsFhaiAe8o2U/k6egiQ5cfk+IIptY5TES49XKnLorbQjEAPLipnHLCXvZ6vfrWQzrsn/62KiAvFyQyHD7qbz1KMMxoyEG1DJFfXAo3q+BbY4Y0fHKoFBvCwnctEYiiIGE+R0PbAfNnouxDZ/tciGk6rBXq3Yf19pyElsUVllhpv+xtA38M/qmjDc1Xl8tEX6hzRkQHuxVLivZUPljGcGk4wwNwSESDPtGAm2+OO9HDYE9pd3d5StLOhcx3FiOx2XYj8IcjkUr0jf0coOFkmqDFvZ6QpANCar6REp7U+APox/xuTLCR1Tg+KQJMFsvJvRXGdfLjh9a/G4tiyURXnbLgGEI2BSN/pfye80OGMUMS8elnZT+kHe2UWHVYusL2Xy7CkrilLZXzIGk7FkdB4BxwKmDXQvM/Z5DWfmujVFfW2tkEKk45McrNHhrRby09F0gWAIFwCIZzZE7tMHDTtdlckkxmJbIxwwb44MWAvzI60Y3JURcRlavGKyDDdsOSiy3ZE8yqQ6334H1q4dSMZ11X+Jvm/0qfjlb+UME1szZLvzx4854W8mK5N3Tagg+KY4av+QK/oCUXNmIrtfA/BLcKogzlffqYdWVA2ZpTsXTqpWr+G9On/VfbubqdWp79/TBtHA5gbrcMwaJRvaugp87vCq/XH/spHXMD3dL2XyNFAutffEDpPe5dSMQQRfPsvfUkss9Tr3KDhEnZk6wCuSeQ4blmKWeGaPUFDbuMejVQjr7xrsIjSTLWRJ0gPIQzET26LZGOobVzBDKFhtRxOFQSoY6dmHmBbHLBkuXYRwloatvGnANmnkWra8zV111Klno+mx+/MRIV4KKchd9YmeS/ihNEERhDeAA7mHRC4EVVzERRbuXZG7Jzk7aYkIqw5df9e3nowH1WIJBI4n22HO1Nrje1SHZktIr5JcsprMH54PWPcjzLLoltiVbzgttdOjVIts9D+GA4X+APWkYWC+VKbk3iLgS4xI1/GrGIsQD3zeGifstxWGP9w2RS/rRmqz9+VLrvU1Ttv0yNVM9mEE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230037)(7416011)(366013)(1800799021)(376011);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: q7TaxrlNgHfvtcxWZEpDNKtKOtVMWATzq9YddGITonua2YVRbS48ixrtNxIFGrTaM970hvUXJfQNj6ykb7Noff1GuyjiDMoHr6BcZfahoW/wGhSnyG8iO6jKt3rAJBRJvh6q7s9ZoiKEvdZeRDiHtOIUvVVlfnAeON1z8pMsoUfm1MqB5Ur2zM3IdHW0ZtjP+36VIL4QGAnyhnmW7gwmSVOUtdVVzkZBXEQFHJGKbpSSod88AC5mRSSJcEVGsVKscPQY+TWjXKr0HSLx4o0WlbBkslhf6hx9xkh/xy4kGwgETZUSbAuwhOLnGMSOiCspHSlyAVgnEFZQzZD9D2ErTwN8+8mE9mgTMjaCqx1fvjStObh+QNjzogc9WOtj4Dy7uBTLeNhf384gFLKZjpkeJvMKcyajDn2TW79I2GAmyf98ElTGSTmiPDQRqtMybeXBWmZFAO5rJyqqAw9KTSFza3kJISJfU3hE8fnX6ouCKjpGhdRoItOMT75u82tLVHHUqoC4DaS+ng8KaOySX/Mly0vy3ft24IA4XPHEjjWAYJlDjE2cjGCwPZNsMcElAnL79BG97o8JMmGuUaRem3G2X/SvHsYSnoFcMkMrDZSXDokIY5ChnV+AjCSU2bvvbvx9u+joAvLfv4EFR9BOBxn4QkT8JzZn3QEZzNRppMqRbWrPqqxt6VqCOZYgcImhOKVKdtSns5C185eMoejs/OL74dKs4HakfNjlJ3DkS7LUAtuFvMcLEpbVa5+tekubhbf855sT3BDDF2VpiMwecdahQzz81Wwy49ekPQdprDHuPKAMHiLXrPeainiDfhCRVEMJU9vuQPXdqjIU6aqIwsxNLgvkI1DYypOYF6QZenjJqjogz/5pEgHeH8uFgYdk0nay++vmL1lXO2KuNTt7Q8AuFp6GwZWzMiKbf5507dbim0i7wKF/+GXIHEE+dLQO6CDpD0KfjXpZKwxmBf/Cys+SRDx9fOSl8hfgBYKmMk+vAhBLBDzY/r0VBbJ/I0lzm69FAicphmT52uyBFTpKnnio6OrG/nhzcSeg5WlfCV8VBxIlTu/Ks28Zo5oQ1GJHb+/rBTybTuB3UlvxuKYCaPuYtvFnoIR5ioSKMg8rjSkTD/NzIm7kZkTTD5wBlwSqZhP+Q2RKVhjSQVlI3MXvmI6M5xU4flGepGvhfna02c3tf408cpqLWKDDPKEftKRn01HO+rBwESszv4e0zvFJWotLREE3sSj3pBfVdVlz2QINnomV3C51fVi4rqVLtMZquOLWwddqnBpr7g5XwMUTH80T/uT7nqbkMTiRAjk3yhjTVYjfnPYTvisSWyQxJvlQ1uIfWmIpDscbaYk1oZlH5Mu+wExjpmk38dC7o0UDB7vBNyO8WTmdAycF8czRfJlRH9AtDKiZ+ekPpwogCHZTU4tQsylJ8tMLi6wez9wLr6RCv86F458QsN5QG4Gs8lMtjFQHegQ4fDWZKUpdrQKkyQyz1MIXnoYQ12V/WmbQzovlqlAa2dup3MntE56KdVrZAjuKrWFEIw8TbfaknU2Bm39X0bbPahfURV6TpUjIDa3dituDlrB62KfRIw16IgRX8zYM X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b0699e82-eaa0-4319-564b-08dc95139d92 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2024 12:37:46.1502 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: itlpA6hHhizlF5h3LEQrktS33wjI4/fNXICGG6N1ATzPmOHEIB4H93t3+rl3wGFu X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8946 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240625_055200_610554_D7AA6315 X-CRM114-Status: GOOD ( 20.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We no longer need a master->sva_enable to control what attaches are allowed. Instead we can tell if the attach is legal based on the current configuration of the master. Keep track of the number of valid CD entries for SSID's in the cd_table and if the cd_table has been installed in the STE directly so we know what the configuration is. The attach logic is then made into: - SVA bind, check if the CD is installed - RID attach of S2, block if SSIDs are used - RID attach of IDENTITY/BLOCKING, block if SSIDs are used arm_smmu_set_pasid() is already checking if it is possible to setup a CD entry, at this patch it means the RID path already set a STE pointing at the CD table. Tested-by: Nicolin Chen Reviewed-by: Nicolin Chen Reviewed-by: Jerry Snitselaar Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 24 ++++++++++----------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 7 ++++++ 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index f563f2ee6fd76d..38d1465a44b479 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1289,6 +1289,8 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, struct arm_smmu_cd *cdptr, const struct arm_smmu_cd *target) { + bool target_valid = target->data[0] & cpu_to_le64(CTXDESC_CD_0_V); + bool cur_valid = cdptr->data[0] & cpu_to_le64(CTXDESC_CD_0_V); struct arm_smmu_cd_writer cd_writer = { .writer = { .ops = &arm_smmu_cd_writer_ops, @@ -1297,6 +1299,13 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, .ssid = ssid, }; + if (ssid != IOMMU_NO_PASID && cur_valid != target_valid) { + if (cur_valid) + master->cd_table.used_ssids--; + else + master->cd_table.used_ssids++; + } + arm_smmu_write_entry(&cd_writer.writer, cdptr->data, target->data); } @@ -2733,16 +2742,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) state.master = master = dev_iommu_priv_get(dev); smmu = master->smmu; - /* - * Checking that SVA is disabled ensures that this device isn't bound to - * any mm, and can be safely detached from its old domain. Bonds cannot - * be removed concurrently since we're holding the group mutex. - */ - if (arm_smmu_master_sva_enabled(master)) { - dev_err(dev, "cannot attach - SVA enabled\n"); - return -EBUSY; - } - mutex_lock(&smmu_domain->init_mutex); if (!smmu_domain->smmu) { @@ -2758,7 +2757,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) cdptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID); if (!cdptr) return -ENOMEM; - } + } else if (arm_smmu_ssids_in_use(&master->cd_table)) + return -EBUSY; /* * Prevent arm_smmu_share_asid() from trying to change the ASID @@ -2831,7 +2831,7 @@ static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, .old_domain = iommu_get_domain_for_dev(dev), }; - if (arm_smmu_master_sva_enabled(master)) + if (arm_smmu_ssids_in_use(&master->cd_table)) return -EBUSY; /* diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index f4061ffc1e612d..65b75dbfd15914 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -602,12 +602,19 @@ struct arm_smmu_ctx_desc_cfg { dma_addr_t cdtab_dma; struct arm_smmu_l1_ctx_desc *l1_desc; unsigned int num_l1_ents; + unsigned int used_ssids; u8 in_ste; u8 s1fmt; /* log2 of the maximum number of CDs supported by this table */ u8 s1cdmax; }; +/* True if the cd table has SSIDS > 0 in use. */ +static inline bool arm_smmu_ssids_in_use(struct arm_smmu_ctx_desc_cfg *cd_table) +{ + return cd_table->used_ssids; +} + struct arm_smmu_s2_cfg { u16 vmid; };