From patchwork Thu Aug 17 18:16:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13356863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E89FC2FC0F for ; Thu, 17 Aug 2023 18:21:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=hcqfNZ//Hm+ByMm6patOW4EtDLAq4qhNnZhjlbEjaiA=; b=yJb8FzZJdiUj4cTgzh2O/+MPGE e6D3/AKma2T4hmDPeycAhJWkthi9qxDuJ5b0F/TTw5VRxssw1vyj9bb1Ae++ZRfQvaWb664qQz8vE /+B+eiG/TNyGAv0VsFQ1q7wK2Lm6QoLSoe/8k53k3YiDbTODHiNtQ7o1Llx96CpJaBkwx9MeJsm1X 0QF2wHMBiqvjm2fAaQPGoY+RN6R/LOkT9wDpKXp/vmEh7FGKBL8sN8XZPoq4dO0yPeIb8ti+8bSVZ 9cVBIGnO+u9QHUYZDmDR6H6Wh8My8yMQD4mADJl9IJmLrnguDxG+rtKl7YnpBtOU+kWwA3Dxe86aq R6iuXeTQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qWhcZ-006xMq-0I; Thu, 17 Aug 2023 18:21:23 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qWhcS-006xGz-1P for linux-arm-kernel@lists.infradead.org; Thu, 17 Aug 2023 18:21:19 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d730a22484aso2206197276.0 for ; Thu, 17 Aug 2023 11:21:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692296472; x=1692901272; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+2D70KRXNWhhcPVOCTxbPCzMxKwpJ9kMCl7BCRH1CBA=; b=gJr4ZsyQvZcjc2Pt/pveOp9R4MwHTIcB7oarl0HOsTGMYprA4U+50iabB0HHjnq7EU ycFLSKx6rqWi49wrijynWCNdiavuy2qDlPHvXNCWDfWHFqRINtDveCEXp6wcY9PKt/hV m0tCvaRJno6xBpQYyzb9DR8cAnM1njLUpHTn3ub/6IqMX+FJTN/ZBeEFsSJp5+M+Tzzq aWVlPPfUsS9KLy0CoTG33Imb7YpEKVL3dBKntBKNB7u0QAwZsl9eAdylIrcwwrK9ICDm Qs2bcTk0KpLxc/xGMZenHbjn5BToWLj/qAYPYI7Fo/t/30tte08VXPxPAtPGjjlHJJWd f68w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692296472; x=1692901272; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+2D70KRXNWhhcPVOCTxbPCzMxKwpJ9kMCl7BCRH1CBA=; b=EKSXKft22ca7dO5NtprQxfIV6csuZG5+3hO73bNBkR9masJm9m1kQMQGX9y2iidW0b r9zHmT/HFe9dyB03p3o0l9pTqU5txy+oJc16rzhvoHEKjYzOpvTDKqSQAE37VbnLPtw0 QtPI69XavQ0TyeRS+bwei/cX/XE6Jt0sVkODN7eriXEan9JqRTHJTE2oXSubrfwHNYpv BAyXuj8sgJ1qsMfwMaUw24jhHqCYnIbzQ1roYh+pQGDAVEb/OyobDAOJzXGkxdF7sap2 waxBemXiENIwBH/62zIPMuWsxr/ZCPXG++XgUdTYcoMYqJ/qHFQpPTsGv4BvUXeFDT0E Iomg== X-Gm-Message-State: AOJu0YxvL4wpq9y/dxBTrWhNdmf2Ey68JkMk/pHftu2jbUblE9BUkmPY wOMPH7RQ25BxfVrHLBxYD/dx2vKwJkwz X-Google-Smtp-Source: AGHT+IEfGIdi50eVGb7u0h+TWz7zRSXVqh1WuPmcOmzYvjkcKgmVMU4WPccj8sV7aIFWIbA0pRIoE9IkWWP4 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:4a77:fd20:7069:bdf9]) (user=mshavit job=sendgmr) by 2002:a25:d152:0:b0:ca3:3341:6315 with SMTP id i79-20020a25d152000000b00ca333416315mr4578ybg.0.1692296472695; Thu, 17 Aug 2023 11:21:12 -0700 (PDT) Date: Fri, 18 Aug 2023 02:16:23 +0800 In-Reply-To: <20230817182055.1770180-1-mshavit@google.com> Mime-Version: 1.0 References: <20230817182055.1770180-1-mshavit@google.com> X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog Message-ID: <20230818021629.RFC.v1.1.I9df3dec9e33165276eba8e2dbf7025bfee286d90@changeid> Subject: [RFC PATCH v1 1/8] iommu/arm-smmu-v3: Add list of installed_smmus From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, jgg@nvidia.com, nicolinc@nvidia.com, tina.zhang@intel.com, jean-philippe@linaro.org, robin.murphy@arm.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230817_112116_499947_B1ACF38A X-CRM114-Status: GOOD ( 21.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a new arm_smmu_installed_smmu class to aggregate masters belonging to the same SMMU that a domain is attached to. Update usages of the domain->devices list to first iterate over this parent installed_smmus list. This allows functions that batch commands to an SMMU to first iterate over the list of installed SMMUs before preparing the batched command from the set of attached masters. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 28 ++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 109 ++++++++++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 14 ++- 3 files changed, 118 insertions(+), 33 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 238ede8368d10..a4e235b4f1c4b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -48,21 +48,37 @@ static int arm_smmu_write_ctx_desc_devices(struct arm_smmu_domain *smmu_domain, int ssid, struct arm_smmu_ctx_desc *cd) { + struct arm_smmu_installed_smmu *installed_smmu; struct arm_smmu_master *master; unsigned long flags; int ret; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - ret = arm_smmu_write_ctx_desc(master, ssid, cd); + spin_lock_irqsave(&smmu_domain->installed_smmus_lock, flags); + list_for_each_entry(installed_smmu, &smmu_domain->installed_smmus, list) { + list_for_each_entry(master, &installed_smmu->devices, list) { + ret = arm_smmu_write_ctx_desc(master, ssid, cd); + if (ret) { + list_for_each_entry_from_reverse( + master, &installed_smmu->devices, list) + arm_smmu_write_ctx_desc(master, ssid, + NULL); + break; + } + } if (ret) { - list_for_each_entry_from_reverse(master, &smmu_domain->devices, domain_head) - arm_smmu_write_ctx_desc(master, ssid, NULL); + list_for_each_entry_continue_reverse( + installed_smmu, &smmu_domain->installed_smmus, + list) { + list_for_each_entry( + master, &installed_smmu->devices, list) + arm_smmu_write_ctx_desc(master, ssid, + NULL); + } break; } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->installed_smmus_lock, flags); return ret; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index f17704c35858d..cb4bf0c7c3dd6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1811,10 +1811,12 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size) { int i; + int ret; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; + struct arm_smmu_installed_smmu *installed_smmu; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) return 0; @@ -1838,21 +1840,26 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); - cmds.num = 0; - - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - if (!master->ats_enabled) - continue; + spin_lock_irqsave(&smmu_domain->installed_smmus_lock, flags); + list_for_each_entry(installed_smmu, &smmu_domain->installed_smmus, list) { + cmds.num = 0; + list_for_each_entry(master, &installed_smmu->devices, list) { + if (!master->ats_enabled) + continue; - for (i = 0; i < master->num_streams; i++) { - cmd.atc.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); + for (i = 0; i < master->num_streams; i++) { + cmd.atc.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(installed_smmu->smmu, + &cmds, &cmd); + } } + ret = arm_smmu_cmdq_batch_submit(installed_smmu->smmu, &cmds); + if (ret) + break; } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->installed_smmus_lock, flags); - return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); + return ret; } /* IO_PGTABLE API */ @@ -2049,8 +2056,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->installed_smmus); + spin_lock_init(&smmu_domain->installed_smmus_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2353,9 +2360,66 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) pci_disable_pasid(pdev); } -static void arm_smmu_detach_dev(struct arm_smmu_master *master) +static void arm_smmu_installed_smmus_remove_device( + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) { + struct arm_smmu_installed_smmu *installed_smmu; + struct arm_smmu_device *smmu; unsigned long flags; + + spin_lock_irqsave(&smmu_domain->installed_smmus_lock, flags); + list_for_each_entry(installed_smmu, &smmu_domain->installed_smmus, + list) { + smmu = installed_smmu->smmu; + if (smmu != master->smmu) + continue; + list_del(&master->list); + if (list_empty(&installed_smmu->devices)) { + list_del(&installed_smmu->list); + kfree(installed_smmu); + } + break; + } + spin_unlock_irqrestore(&smmu_domain->installed_smmus_lock, flags); +} + +static int +arm_smmu_installed_smmus_add_device(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) +{ + struct arm_smmu_installed_smmu *installed_smmu; + struct arm_smmu_device *smmu = master->smmu; + bool list_entry_found = false; + unsigned long flags; + int ret = 0; + + spin_lock_irqsave(&smmu_domain->installed_smmus_lock, flags); + list_for_each_entry(installed_smmu, &smmu_domain->installed_smmus, + list) { + if (installed_smmu->smmu == smmu) { + list_entry_found = true; + break; + } + } + if (!list_entry_found) { + installed_smmu = kzalloc(sizeof(*installed_smmu), GFP_KERNEL); + if (!installed_smmu) { + ret = -ENOMEM; + goto unlock; + } + INIT_LIST_HEAD(&installed_smmu->devices); + installed_smmu->smmu = smmu; + list_add(&installed_smmu->list, &smmu_domain->installed_smmus); + } + list_add(&master->list, &installed_smmu->devices); +unlock: + spin_unlock_irqrestore(&smmu_domain->installed_smmus_lock, flags); + return ret; +} + +static void arm_smmu_detach_dev(struct arm_smmu_master *master) +{ struct arm_smmu_domain *smmu_domain = master->domain; if (!smmu_domain) @@ -2363,9 +2427,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_disable_ats(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + arm_smmu_installed_smmus_remove_device(smmu_domain, master); master->domain = NULL; master->ats_enabled = false; @@ -2385,7 +2447,6 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; - unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -2435,9 +2496,11 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + ret = arm_smmu_installed_smmus_add_device(smmu_domain, master); + if (ret) { + master->domain = NULL; + return ret; + } if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { if (!master->cd_table.cdtab) { @@ -2467,9 +2530,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; out_list_del: - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + arm_smmu_installed_smmus_remove_device(smmu_domain, master); return ret; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 83d2790b701e7..a9202d2045537 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -690,12 +690,20 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of smmu devices that a domain is installed to */ +struct arm_smmu_installed_smmu { + struct list_head list; + /* List of masters that the domain is attached to*/ + struct list_head devices; + struct arm_smmu_device *smmu; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; struct arm_smmu_domain *domain; - struct list_head domain_head; + struct list_head list; struct arm_smmu_stream *streams; /* Locked by the iommu core using the group mutex */ struct arm_smmu_ctx_desc_cfg cd_table; @@ -731,8 +739,8 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head installed_smmus; + spinlock_t installed_smmus_lock; struct list_head mmu_notifiers; };