From patchwork Thu Aug 3 10:12:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13339652 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17320C0015E for ; Thu, 3 Aug 2023 10:14:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=YHTxmNohxPfSaxqUpA2XVICRgTGqUqjAD5kcI+97PW8=; b=AusB8fjCnZ0o08XJxtztrI3ZiY BZRXOg8vw7+VYhaMGyLGUCSH8OO31oV/bMv4ydTq4x1gnP17VCpOcPEBaZUANiJW9m8/iO4iH6ejR +PYG01dl9b6o6KWaPegrQnKqgufyjht+mzkL/v2e33HjRC0GRIRxITLMtywqcVKQgVIN7jJ/oWPzO 5YwexTTRJfqamHszikgDNawE0h2sxPy6leCZXWHg+nYO7QmbXOAIOOGqYAh5ErUaZUoZ9asdvEzuq RluepzzgS6bQP6Bp9uTMDpFoa6L1NkKJDSpMmIInJrEOXji17t4kRWLr2mx5JYthUNKa/RxyPS2hp a41pe0dA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLS-007Jda-00; Thu, 03 Aug 2023 10:14:14 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLP-007JZx-1g for linux-arm-kernel@lists.infradead.org; Thu, 03 Aug 2023 10:14:12 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-583b256faf5so13560557b3.1 for ; Thu, 03 Aug 2023 03:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057647; x=1691662447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Y1wRhIRrggJAZYs6e7aV6RdIU6iSXzW0ONi/CWI4qUk=; b=v8M8EpvgykyvNj5+VFK4+5VS1GCx+4thN93X5qGdv2Q6mDg+Q9BprkKi/ZqVY1cMB5 hgughi4UN1aYu/EXNqY13NPNEQutZvvu2GYwDAvr8QOh2o+EQF3P9BYp4pftEx0ox7R7 H293Y2C2VTDWZWO6xMfqIKxlFoQitde1l6TRNNpWpcVfHJP20iQVEqHd2/LLHggfZnBa ZYsMt0hplLT0HqFF4jX+LVen+NA6yXSnT9/tIhVW1iyXGrZn3ULvFc83mH9XfV5ftRLp iOKnNntwlT0uTtypEVze4RGIkmvLieWvFRULXEHMTA02TJWP0zXal6LCAjTnDjAfnCr4 3weA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057647; x=1691662447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Y1wRhIRrggJAZYs6e7aV6RdIU6iSXzW0ONi/CWI4qUk=; b=jcUfMZDRolNztZNHOy7yI2osbJmTDpgO4PG/vd8njZZL2xPXz/WTqR/nTIznG0O546 qEFnAbRNIpUHAdxXsmAOC1jQprvXjWs4Z9mNIcbIEdI6yczW3IyDQMYwuFP6Xe9OrovZ ah6GqNbbLwBTgr8IGdcRAk3/lQeQZj2fZ/QSPY7Yh5LPk0KnXnFiukM8VBWN0V58cK17 xjVOodvhA9o0hX3AzljZO+CjQ2s3tYjc/7/fvUP7e8DgW8tEOXi6v/QIPERFt7wv4zUH 91G1p5D7W2Hp3RO2x7TkvtvkTzh/Qax2jBKdd2X9ZgWIAVMyg+MBKtSckeyzbgek8kr9 WOUg== X-Gm-Message-State: ABy/qLYlmeybNiZJKGVMgJzaG5FvKzv1Ic46H6a1R5J4i1Qa8CS6gGW3 keelLM7ACliAtzuYt9vlXUsV1zU9OVe1 X-Google-Smtp-Source: APBJJlGq7z9/Y1wQFTi6d5yV5b9fzok7EZ8JGI+3DYlCHleYWyG2dZOEveRtsUS2kTL+q5mmdG0WEODi8+D8 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:b301:0:b0:573:54d8:e5be with SMTP id r1-20020a81b301000000b0057354d8e5bemr268745ywh.3.1691057647550; Thu, 03 Aug 2023 03:14:07 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:21 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.1.I18d69f4908b8a9f801239117d2a6c3f5aa1e2e76@changeid> Subject: [PATCH v5 1/6] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230803_031411_582232_E3165F9D X-CRM114-Status: GOOD ( 12.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- (no changes since v2) Changes in v2: - Fix commit message wrapping drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 61de66d17a5d5..4df335424b266 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2305,7 +2305,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Thu Aug 3 10:12:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13339653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4AAD9C001DF for ; Thu, 3 Aug 2023 10:14:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=6xAtMTa93RJhFJVVM0AZHdNCS/dzwmpE0eg8wCEArwo=; b=daCo2ZDBGuZd8QCVU2g3xx8Bjj dUThZPlnb8OZoAMwKSw0AREajQveXD8QXs/cIrcsLsxW1BbY0iF5e6TFOdEwk0kb+72WvTtAQSXZa qLuDCkljOSMSSc970GBRN48Aw83R4PV0NPqYUj/IBN/wshfFyCjBKThYpxyADuPhLaghAhuImLt02 0AywXqXstk8JEJLQRbo+Y1i32yhqbhE9p2Ya122vesk3W1CMbfL3g1/5USixHSTDWRRH9i0+85sYC jMM/aWA9LEm53Bx6d9gBwtis9jzkozfZziSJxzqIl/g3dIfvSdUB71zBNwuHLa+eHcEh8x/fcP0tC 1jJDz0QQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLX-007JgP-2B; Thu, 03 Aug 2023 10:14:19 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLT-007Jck-2H for linux-arm-kernel@lists.infradead.org; Thu, 03 Aug 2023 10:14:17 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d326cb4c97fso851306276.0 for ; Thu, 03 Aug 2023 03:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057652; x=1691662452; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IgE52kHcfRPOfED/lVwbKZpQuW1B4++XZftllscpWsM=; b=j2OVAQZZ7MDjPx3io3qsYNX350Y8kbR8uyyFo4vpsvU1CTrDPyy339XvDoTtkCfLKl hjuPhFUVdHds84l/I3DUxRLC11T1vIyAbXfh7nF0cLGUVaaLcssV1ZPLXJgRWTlUXiiS sAPOWKRpC4CWBpE5Tosmj7zGbyCzgLSkhayxDyzvH6arF2iY/dL3u2xXVqZfXf/BeTpc 3RHFbmb6uv9SkPCVhpZcis7e3ORDX//7PRXqKm2A1GBAsk3u9SrbHtdu+VCbDrNZeWpe JLQ2yz0xlOAwyFQGIXMYDDWqpsno2z24Vp47ND8adkdHa2UB5BBeMjjKzlqDfGOkFwSt Rsxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057652; x=1691662452; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IgE52kHcfRPOfED/lVwbKZpQuW1B4++XZftllscpWsM=; b=lo2Oa6QgvwHQ+5fxZqkIMnB6VfWjJN/9UtwvIBbyCPraSBd83KblHMQsDOpQUvOpCO rhGh78LXJm5aRqMp0JbGgsPJnA7QgwuaT2Nc95unM0vHKvjiBAY/h9RBEjCWEE/UEFJu Ub2nisHHi9648/qDmmD4wTQFlc/5IULuBEDEW1x1uAneX5bNLIHZGzWRvonn+m2qzRK7 ar9YK2UOoN8rudpWxyU+8x9CKubXSHfb9b4LGLCxvq/989X8QElXtz8Q9LO7CDrBBjBg sy6XUtv42V6xfiEY1uh0Z1mgBXAC7RXJKkh79nEjAk2+ao1FRdbljXNLD+0/f594VYy+ j8YQ== X-Gm-Message-State: ABy/qLbRH7aygqaLoAhYHzM0HSTjRhpz5gjGmnLZigfiiRNVsEHmZiQh ky7dCKyVa0v4objLjtz1JF16U9iSml6E X-Google-Smtp-Source: APBJJlEgEds+gT/fNLvAp8/c4zfzgVx9Ij/FBCk6jJtg1pjmXeqiokAEkZmsWVgHFgnyT7rhESChAa4lqPAv X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:48a:0:b0:d37:353:b7eb with SMTP id 132-20020a25048a000000b00d370353b7ebmr74807ybe.11.1691057651922; Thu, 03 Aug 2023 03:14:11 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:22 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.2.I8db07b9eaef3bd5ef9bfc5c8c6d44768a4d95293@changeid> Subject: [PATCH v5 2/6] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230803_031415_767332_339DC068 X-CRM114-Status: GOOD ( 30.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- Changes in v5: - Renamed domain_head to list for consistency with other lists - Renamed attached_domains to attached_ssids to avoid confusion. This is a list of master/ssid pairs the domain is attached to, not a list of other domains. Changes in v4: - Remove reference to the master's domain accidentally re-introduced during a rebase. Make arm_smmu_atc_inv_domain static. Changes in v2: - Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 28 ++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 99 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 22 ++++- 3 files changed, 98 insertions(+), 51 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index e3992a0c16377..d80c39e7e2fb5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -37,21 +37,35 @@ struct arm_smmu_bond { static DEFINE_MUTEX(sva_lock); +/* + * When ssid is 0, update all the CD entries that this domain is attached to. + * When ssid is non-zero, write the CD into all the masters where this domain is + * the primary domain, with the provided SSID. This is used because SVA still + * piggybacks over the primary domain. + */ static int arm_smmu_write_ctx_desc_devices(struct arm_smmu_domain *smmu_domain, int ssid, struct arm_smmu_ctx_desc *cd) { + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; unsigned long flags; int ret; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - ret = arm_smmu_write_ctx_desc(master, ssid, cd); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_ssids, + list) { + master = attached_domain->master; + if (ssid && attached_domain->ssid == 0) { + ret = arm_smmu_write_ctx_desc(master, ssid, cd); + } else { + ret = arm_smmu_write_ctx_desc( + master, attached_domain->ssid, cd); + } if (ret) break; } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); return ret; } @@ -222,7 +236,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -243,7 +257,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) arm_smmu_write_ctx_desc_devices(smmu_domain, mm->pasid, &quiet_cd); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -333,7 +347,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 4df335424b266..6e614ad12fb48 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1282,7 +1282,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }; if (master) { - smmu_domain = master->domain; + smmu_domain = master->non_pasid_domain.domain; smmu = master->smmu; } @@ -1725,7 +1725,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1750,8 +1757,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1797,8 +1804,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1808,13 +1814,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1837,25 +1849,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_ssids, + list) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1877,7 +1901,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1970,7 +1994,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2050,8 +2074,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_ssids); + spin_lock_init(&smmu_domain->attached_ssids_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2289,12 +2313,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2310,10 +2334,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2377,19 +2400,19 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_del(&master->non_pasid_domain.list); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); - master->domain = NULL; master->ats_enabled = false; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2434,8 +2457,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (ret) return ret; - master->domain = smmu_domain; - /* * The SMMU does not support enabling ATS with bypass. When the STE is * in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests and @@ -2449,26 +2470,26 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { if (!master->cd_table.cdtab) { ret = arm_smmu_alloc_cd_tables(master); - if (ret) { - master->domain = NULL; + if (ret) return ret; - } } ret = arm_smmu_write_ctx_desc(master, 0, &smmu_domain->cd); - if (ret) { - master->domain = NULL; + if (ret) return ret; - } } + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_add(&master->non_pasid_domain.list, + &smmu_domain->attached_ssids); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); return 0; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index e76452e735a04..66a492cafe2e8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,11 +689,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head list; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; /* Locked by the iommu core using the group mutex */ @@ -730,8 +738,12 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + /* + * List of arm_smmu_attached_domain nodes used to track all the + * {master, ssid} pairs that this domain is attached to. + */ + struct list_head attached_ssids; + spinlock_t attached_ssids_lock; struct list_head mmu_notifiers; }; @@ -752,8 +764,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Thu Aug 3 10:12:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13339654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 056A6C0015E for ; Thu, 3 Aug 2023 10:14:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=NlK/LAC1/LnyfTWCvr1Z86uLErTNsbr59MjwWMHPjqI=; b=FMQkrGwKEuO+IiBC6txifEnHss VyZHn0TKeppSBmiN5EcyoPEp18VnKdnOexnZemDI30Zpw0FOA6fKRBpOBojcHkseuvh4adAOqfqzn 0PP4y7HaryO3UWkTgoGTaRYK3NENiL8xWEpKmpbnsn8ZOTVJHm3BdzqS4HaRq5U5jjbYOCpa1bTOQ Negn1VrOPmRnn4MWwUdjXvFM7LhWEU5uHjlTMRjkcTpf5MahU86kcdXGen+coV0/9ZLaUPHXkZE2V h5dIZmAYA+KyjpiB3+/fS6tR8vLC+QFFIKwAO8ryRguNLRGJxpaW5c2BiOrgvwl0ggHe5dtgk9FUQ r/d4vghg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLa-007JhQ-1N; Thu, 03 Aug 2023 10:14:22 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLV-007JfD-2n for linux-arm-kernel@lists.infradead.org; Thu, 03 Aug 2023 10:14:19 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-57320c10635so8482147b3.3 for ; Thu, 03 Aug 2023 03:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057656; x=1691662456; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rOifn/Wuhm3Lbc3GJRKlq49fGVZtVjA5QOC6h6hHPVo=; b=zJO63gTN+QV8Tfx+6gzEyj5dUonNQXVuyOR/4CqHTz6PD/3cyRUXHXVcphE0fpyuCR ZKz98+w5g8mFMMzzvIJ747TDQzawpo9yAa/1f6z+hOOvccv3tyru4rVG/SNKz9FZCl5s uIn2qnCLLDiJNac2CKJCfNSreSw5An3t/Kx+wsM8Pcw1M5SxyvLmfICMMkWalT4pT/fH IEFOJ549A2Nn1ee23hdlvSi7Y3WyIy4Tdve79X6p9c1NumDYR1CfcPF3sBfoSZXRe5Qr WrgHPCqRl2usHE3RPid/EudzlKgU3D4crbQvFz+8xt4Mn708pcq+X+TmyEB36fv7aHaX l0Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057656; x=1691662456; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rOifn/Wuhm3Lbc3GJRKlq49fGVZtVjA5QOC6h6hHPVo=; b=LMdvp3thp/hlV6zqMzcckKUYl3o+ZEgA6TSnZvvHirVo+Z5qh2atdkD0zt/ZfY1/ua QAaRyTcxhnyaVxjM76nfvAnH/7kYCLxXNUgE5QGK9uuezRgeg84iGGH2RFk79PKOrF5S vrHiymSENCtVTkj9bAVRx2NsL+C/NZK3mMPnKcxlLv2jOnAeC64iP47QKyB6rxsA1fKu jUHQqwzZNqsUtdBD/GY7ZbYePnOZGGknI+F8eUQGCSGpXh1miiUHLb6qKsEvGxL4hxEN mNrOG0Jcfu9e+m9HaiV20XhJjhGEEcx3IrrQH3bujneWnrulLHaEnZMpItBKrpaHW/8G mtdQ== X-Gm-Message-State: ABy/qLaqzBBRSGSllkEhzuhuV8odSv0OKGQD2IvGZqUJJy4B2nEhJrap yaQ8LRmFVKlJZKY9yhvDJV2yTbtS/Ggq X-Google-Smtp-Source: APBJJlEww5GzZLOemvoHR0jnjoJTpv9UvSTAuAgQXGJ5jhZ7oiz42irlxltf7zXwxFAEDzkY/BIZ2zeS4ZW5 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:ab0b:0:b0:d0e:e780:81b3 with SMTP id u11-20020a25ab0b000000b00d0ee78081b3mr118271ybi.2.1691057656667; Thu, 03 Aug 2023 03:14:16 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:23 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.3.I374b2826d8b01f3a0110fd42a01208d2e8adb0a2@changeid> Subject: [PATCH v5 3/6] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230803_031417_916787_FE3F5E56 X-CRM114-Status: GOOD ( 14.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- (no changes since v2) Changes in v2: - Make use of arm_smmu_atc_inv_cmd_set_ssid drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 6e614ad12fb48..e0565c644ffdb 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1798,13 +1798,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1814,6 +1816,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Thu Aug 3 10:12:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13339655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4DC1BC04A6A for ; Thu, 3 Aug 2023 10:14:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=iVvZST/M9oQaQHAkUIPkmQP6E6C+UESPENRMGOnAm+I=; b=yHfEl4dKMU/1EcCJS8rXhmagTY dgardulwgtyVR8qy3Bec0wJDzzMRl7AkIHg0mYdqO4wSR8apDbpK3Q5P4PV/uuvehFU7tWEG36Vrs O8viWfmtn2YZu51S63Gnfed5SnIGmv7CfA1lF6zDD0SwQqAKkKsq4oreyx6e5W3gh1sirJDBAYTJe ya6OnNdHeuxokDSxDfDxWmfG8Pidjf0Hq+AFbyh1nuV0Cxgj2yGFP7BOfCBL19b8eKbNdmDnhF/pG 9Wb6oF4+rl4Ntlt4wG4FUTN2kRZaFYtl1nlxjFRE0ZluSHe44wswhgzm2xKLqYxKajsHQTURdaUZ+ jYXtxLtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLg-007JkE-0A; Thu, 03 Aug 2023 10:14:28 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLb-007Jgy-27 for linux-arm-kernel@lists.infradead.org; Thu, 03 Aug 2023 10:14:25 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d390abf3319so875945276.0 for ; Thu, 03 Aug 2023 03:14:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057661; x=1691662461; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7z5OBENHOr2eyPDXE+tfWeU2HnsC9/l+M3HstrRsv94=; b=4QHFgQUz86psObn8rqXE96jintVeFyZR35xzg9vOmflXpYO0PFbxuqsmlweMhUyr32 vvFq/X22VGeQUffOj2LowuNN14kNeh1exPxyIHK6iorLiAJaUvsTfmR54Ja3uWSYoj+K wHwk8QvAu93nEOl9iwvn2H9dvkc74xR7xbg3RtyEn4XCPooe2xQxuXh0RSIscAWlq0Sl IzhGppqwyVK3eZFtOSbFmx/hgQL7NAhApxNRK/g5uJhvrqX1D2Mskmt507nzwfbxGjuP AnsyYIfVraineTEMavWQs8qcmr1vAiZpzI1xLzKZeB0QitKz0Uqxb1nB+OxX2ujF9nFv NtgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057661; x=1691662461; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7z5OBENHOr2eyPDXE+tfWeU2HnsC9/l+M3HstrRsv94=; b=OkUzS6BZp2jlxz3/rRE0Gu2iljlP79J06ilFgI3EN0ry9iYc5KaozCTBfd9tVwBmg3 wLFWHhADIlxbzNs1jLkhAsoeEIDci8wIJd4xe9l5SVlMK9y1sxfC2skPz3o8FPKj+cPA RSv63YQDEaF3pwn/gYHbBYgzdCQz8aRhrQIWF3uW1rAMC1yFUHYwVaVC227Ts+jK/h8H m14hhLPLPfeFX1RsXBOLbfwHdc+aO1ZrlPKm1VxyqAWsg+QunRRQoCcmvNGum4NCCCYK HoiYDgxyUJig5VcuMAKN1d0nUKr35X1yxcCPzCYHQaZwhhEn7u48czAfleHnOFqkKei+ 0/uA== X-Gm-Message-State: ABy/qLbyGZUff2yRiadnTf5veKKp0FPhsQqeciAIyZgbGI5UeUKPHsTW NNpKurd1hVPFsW6QNs2QTTBtg+19EdU9 X-Google-Smtp-Source: APBJJlGCHddp3RdaF6WFwHVXgiwX+tBKMPbRCk0pc6RCj+MlyNyRm7wsquSL5VW9/VzNUGGLERFhfnIyncME X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:db82:0:b0:d0f:15a4:5a53 with SMTP id g124-20020a25db82000000b00d0f15a45a53mr121006ybf.2.1691057661160; Thu, 03 Aug 2023 03:14:21 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:24 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.4.I4ba46c0f7d599f43094d6ba1113c0b4fe49bd908@changeid> Subject: [PATCH v5 4/6] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230803_031423_699589_BC4F0F59 X-CRM114-Status: GOOD ( 28.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- Changes in v5: - Fix missing error value return in set_dev_pasid - Fix issue where nr_attached_pasid_domains isn't updated when arm_smmu_write_ctx_desc fails - Fix missing free of the attached_domain node - Split off the CD table refactor to separate patch series: https://lore.kernel.org/all/20230802163328.2623773-1-mshavit@google.com/ - Link to v4: https://lore.kernel.org/all/20230621063825.268890-1-mshavit@google.com/ - Remove districting change where a NULL master is passed to arm_smmu_prepare_domain_for_smmu Changes in v4: - Fix build warning and error on patch 07. The error was introduced during a v1->v2 rebase and hidden by patch 09 which removed the offending line. - Link to v3: https://lore.kernel.org/all/20230614154304.2860121-1-mshavit@google.com/ Changes in v3: - Dropped the bulk of the SVA refactoring to re-work as a follow-up series. - Reworded cover letter to omit dropped changes. - Rebased on 6.4 tip - Link to v2: https://lore.kernel.org/all/20230606120854.4170244-1-mshavit@google.com/ Changes in v2: - Reworded cover letter and commits based on v1 feedback. - Split and reworked `iommu/arm-smmu-v3: Move cdtable to arm_smmu_master` - Added SVA clean-up and refactor. - A few other small bug fixes and cosmetics. - Link to v1: https://lore.kernel.org/all/20230510205054.2667898-1-mshavit@google.com/ - Add missing atc invalidation when detaching with pasid drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 156 ++++++++++++++++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 141 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e0565c644ffdb..7b296458dafec 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2388,6 +2388,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2423,6 +2428,25 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, master); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) + ret = -EINVAL; + + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2438,6 +2462,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2448,21 +2476,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) - smmu_domain->smmu = NULL; - } else if (smmu_domain->smmu != smmu) - ret = -EINVAL; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; + } - mutex_unlock(&smmu_domain->init_mutex); - if (ret) - return ret; + arm_smmu_detach_dev(master); /* * The SMMU does not support enabling ATS with bypass. When the STE is @@ -2500,6 +2524,72 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->cd_table.cdtab) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master, pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + return ret; + } + + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_add(&attached_domain->list, &smmu_domain->attached_ssids); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + master->nr_attached_pasid_domains += 1; + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2738,6 +2828,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); @@ -2874,12 +2973,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_ssids, list) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->list); + master->nr_attached_pasid_domains -= 1; + kfree(attached_domain); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); + arm_smmu_write_ctx_desc(master, pasid, NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2899,6 +3022,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 66a492cafe2e8..433f58bd99dd2 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -713,6 +713,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Thu Aug 3 10:12:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13339656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 804D3C001DF for ; Thu, 3 Aug 2023 10:15:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=HqLDRBuA1T9Efav/KjDhvms3GKaXWoWKV6CZRTw34SA=; b=mDh5QFfKH9Eq227+A+uDaJIl4W Di40qHl9Y9UcFk8sFqiOwIlNBKW/II1h2zfqHOl/FPBby3xXyQ+/YWOqABr+Lsvyi+zSu4xgjKCSg dCaQCw+H0cuyuLqfVr/9H32F8KOtrKi+mFc/HEFqaR/bc7w3fFjc36VxYLLr6SjIo5ZfmHWD3+1O/ 9TZza2trv+h+BpKRCJGiKsSXTZ7fgd8OSBUbGTEV7dEyVsg9kbekZEpOm2T+8/J2jjVHE1bFABBGg 1VyF07NZfE7ApXWshomRvN8V0r+r/xOojw7nfioXMZ6lNz5dj38uoZvWgKYDy2v43N8GN+GdX4caO 4bfYdJeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLn-007Jog-19; Thu, 03 Aug 2023 10:14:35 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLh-007Jji-2h for linux-arm-kernel@lists.infradead.org; Thu, 03 Aug 2023 10:14:33 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d41fa6e7103so33406276.0 for ; Thu, 03 Aug 2023 03:14:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057666; x=1691662466; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xOWJ0lqxCqyUuPaPvlunqjXsmwJAIBAp4YS6BHDu6Oc=; b=q8ilSnaU6t9HzGSP25lQgwS3HkEavJ1kpDRxanKDeMxMFpXMt79mwz5/LyAooJlDvA ZFkcNzkxOdJkn+mQlAQajTVdFv7/wktHRMWl9+jtmycAuPJQ443dGFHW9mHfYob5d0dR 5gCKHa5Mr+kyEbJAgDyNiqFHQG8H2exc8qDgBJSYDg23RNV0TbdqhxbIdbAAfRYb1jMH hgM5mz2RHJacrXX1YxLeqEnyi0WI3oxPjjDRLSX5c3zy/nPep2zhTSl0jYzYmZaboMSF Yc8Hk93qM/6eAhy29HnpU9HJeSdlYmBnHfXx9w9kfu8bDRKPI6LjT6pzgjlkK/ioJoFh nk/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057666; x=1691662466; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xOWJ0lqxCqyUuPaPvlunqjXsmwJAIBAp4YS6BHDu6Oc=; b=AVJS7gSNpyEf6D50qYWN6APiX5pSuzg7YqL/4wzZIbRsvBOPfLDqWX+6vQr2nkvFHL pQVmxNll++dQPMaTAJbKNA4pyTu+DIm4ucMPxlA+E9YVYfFYtbAQ8LzVFQp0ogTaYuO9 olkojkGjHnoIBytaje1ckzXh6/Ggxjyz5+wP/mO3u06U3fWdt0MiDAPa3T/QwtoD3PVL dPW0bq1YiUGRRIRX6uS+Ov92ksL82QAlwybF1asH8e66yiGQG8pGu7saSdZFp9iJnu5+ a2Lw1D2dklzJzIkhPzL4KQbWAemjPfeckYd7h3UBI81r1dKBB2fPn2zmYu0clm/snd9j 3oTQ== X-Gm-Message-State: ABy/qLaDSTp9ZXIxtoAtFWIWfyp9wxzWDO8YBLHsyG8X8MncDq4RUsY8 7NhNnPgHE6JdoYAbdKSEW9HwHEQH5UxQ X-Google-Smtp-Source: APBJJlFvhgXCXZP9pdYDhKfOUVlYdwH/SmwuHsXAcZhU925R9DHZabK6vjqUogAiaEDk0wFB/8skMjsyTC09 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:2042:0:b0:cb6:6c22:d0f8 with SMTP id g63-20020a252042000000b00cb66c22d0f8mr142117ybg.4.1691057665928; Thu, 03 Aug 2023 03:14:25 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:25 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.5.I6f3fb0734ef5ef746ae7c9b27f632f506197eb30@changeid> Subject: [PATCH v5 5/6] iommu/arm-smmu-v3: Free pasid domains on iommu release From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230803_031429_917587_E4CFFD9B X-CRM114-Status: GOOD ( 18.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The iommu core doesn't guarantee that pasid domains will be detached before the device is released. Track the list of domains that a master is attached to with PASID, so that they can be freed when the iommu is released. Signed-off-by: Michael Shavit --- Changes in v5: - New commit: Free attached pasid domains on release_device() call drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 22 +++++++++++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 10 +++++++++- 2 files changed, 25 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 7b296458dafec..5fd6c4d4f0ae4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2587,6 +2587,9 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, mutex_unlock(&arm_smmu_asid_lock); master->nr_attached_pasid_domains += 1; + list_add(&attached_domain->list_in_master, + &master->attached_domains); + return 0; } @@ -2786,6 +2789,7 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; INIT_LIST_HEAD(&master->bonds); + INIT_LIST_HEAD(&master->attached_domains); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); @@ -2825,16 +2829,21 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) static void arm_smmu_release_device(struct device *dev) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_domain *smmu_domain; + unsigned long flags; if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); if (WARN_ON(master->nr_attached_pasid_domains != 0)) { - /* - * TODO: Do we need to handle this case? - * This requires a mechanism to obtain all the pasid domains - * that this master is attached to so that we can clean up the - * domain's attached_domain list. - */ + list_for_each_entry(attached_domain, &master->attached_domains, list_in_master) { + smmu_domain = attached_domain->domain; + spin_lock_irqsave(&smmu_domain->attached_ssids_lock, flags); + list_del(&attached_domain->list); + list_del(&attached_domain->list_in_master); + kfree(&attached_domain->list_in_master); + spin_unlock_irqrestore(&smmu_domain->attached_ssids_lock, flags); + } } arm_smmu_detach_dev(master); @@ -2995,6 +3004,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) attached_domain->ssid != pasid) continue; list_del(&attached_domain->list); + list_del(&attached_domain->list_in_master); master->nr_attached_pasid_domains -= 1; kfree(attached_domain); break; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 433f58bd99dd2..efa428629f4d9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,9 +689,15 @@ struct arm_smmu_stream { struct rb_node node; }; -/* List of {masters, ssid} that a domain is attached to */ +/* + * List of {masters, ssid} that a domain is attached to, and conversely of + * domains that a master is attached to. + */ struct arm_smmu_attached_domain { + /* List node arm_smmu_domain*/ struct list_head list; + /* List node in arm_smmu_master*/ + struct list_head list_in_master; struct arm_smmu_domain *domain; struct arm_smmu_master *master; int ssid; @@ -714,6 +720,8 @@ struct arm_smmu_master { struct list_head bonds; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; + /* Locked by the iommu core using the group mutex */ + struct list_head attached_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Thu Aug 3 10:12:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13339657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D54A0C0015E for ; Thu, 3 Aug 2023 10:15:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=eORrEC7vasehgLHnVXH8Nb3FpvPxBBuGJlPZB6KRHLU=; b=1Kjkv9idEPNRHMGnNeZnEkDBBL 12gngsso5l34DZvyR3wlm097gkXH7iiHRsYQ/pYKuyEClnFlflYJCrEpoGHFojFHAhSSo1/EviPUC k8Skcvu3bOvJUbQqd1r4xglfCbh8c8NvKs8kV5M25v70///I1mDYxknbCAATKuIz7iyRW3srKTgW+ nr4vs62IL2wUwShAmi0PnwliHO8ZaIMaNsJjOkr9Hfa0vE5cZNx28n7NjX1bIRI2D+xSo2BknhcVt 8ksqPGEiFPCpg3Rtvs5eCiqfzN+ieuhKjiMwZT/zdyioqnesOgHosv6niEmm0IVnpmEKACj2VTVpj K41tAK9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLq-007JqT-0q; Thu, 03 Aug 2023 10:14:38 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRVLj-007JmO-2n for linux-arm-kernel@lists.infradead.org; Thu, 03 Aug 2023 10:14:34 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-583c49018c6so8433057b3.0 for ; Thu, 03 Aug 2023 03:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691057670; x=1691662470; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DLa2HUSCTWFTI0X2o3f102hvfCfHBN0tK5dUlzxCyIc=; b=a6jeCmc4bN/A2ChgfTwyYgnoSOjzynwD0hKdab97fxFiQfSeGipLdBD8vrco7vSA/g QOt22+Zq6NX8xu96ciZpkKp4t5eOm+ZaDlxM6cwB96C4VYR8qRCGUhy2F8aLjmrZX7BK lgPFNpSbNKKYhMqQtdmKUOe2mtgb5tKgGOgDmq9z4UUNQSfYVDQitc/nmlYflTh8r71x WHXjiURsl1a7dnvRWS7u0ZIp9NkZQ7glmppXYAaIMaXGtS+Nd08ghmzEF5M9LzrJXJmc KqvH0cWbBfcQls+aRNNfkuVmAcdzik2ZO7MoKwsaPcJ6aKQIAFn69fAb4koX4ECdcQQ0 esCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691057670; x=1691662470; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DLa2HUSCTWFTI0X2o3f102hvfCfHBN0tK5dUlzxCyIc=; b=IV1SVoOpgsiJyX6Sg6JdL8CstuYtDoDWsPBASPQL3R8Jst3wqvm/xwgbGZ0dAEZtY4 GEAIAxEg+xsEGXKDjCMS2cmiVP5JgMggg+7lZgK/JnJ78ySLZmOjLxWCGml68vPCTjzx 6DMuTepHZxXOrLphGqsBzQNoTj/4Ta8wcGLB83kIoweCz2gUW7LFxPVp3KFeYtP8hh6e TZr+xKnB+5fDGynYcIG2ATwBwD7FFVW4KoO8A3/Fus8MgkN1PLUBydkTD7wNHiRq/dbW mKXEONp4h1Y6PzqLYyRTnTdVGv0YrZ2pPQh6JtShbkpsgeBKD4gYgxZJcK6PmNR+f8aF KXGw== X-Gm-Message-State: ABy/qLYAWGnN+gBZop4MJ5isNRskFmc2EKSI8soGeR9XsnlmAshtuECs eZUYrn556GOzlA9ERA43Ulgp3VBjwnMB X-Google-Smtp-Source: APBJJlH6RK3TEtZXk1xM04qItMlyJrFzxnsOEH42ZDuBzo1V5QaUS9nSNVNbePa1fe9X8ZQRJ7o7+n5wLnH3 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:b625:0:b0:586:50cf:e13f with SMTP id u37-20020a81b625000000b0058650cfe13fmr100164ywh.1.1691057670282; Thu, 03 Aug 2023 03:14:30 -0700 (PDT) Date: Thu, 3 Aug 2023 18:12:26 +0800 In-Reply-To: <20230803101351.1561031-1-mshavit@google.com> Mime-Version: 1.0 References: <20230803101351.1561031-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803181225.v5.6.Iff18df41564b9df82bf40b3ec7af26b87f08ef6e@changeid> Subject: [PATCH v5 6/6] iommu/arm-smmu-v3: Cleanup arm_smmu_domain_finalise From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: baolu.lu@linux.intel.com, will@kernel.org, jgg@nvidia.com, jean-philippe@linaro.org, robin.murphy@arm.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230803_031431_901740_521F8E8B X-CRM114-Status: GOOD ( 14.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove unused master parameter now that the CD table is allocated elsewhere. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe --- (no changes since v1) --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 5fd6c4d4f0ae4..db8fd4b3591b5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2129,7 +2129,6 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) } static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; @@ -2167,7 +2166,6 @@ static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, } static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int vmid; @@ -2192,8 +2190,7 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } -static int arm_smmu_domain_finalise(struct iommu_domain *domain, - struct arm_smmu_master *master) +static int arm_smmu_domain_finalise(struct iommu_domain *domain) { int ret; unsigned long ias, oas; @@ -2201,7 +2198,6 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct io_pgtable_cfg pgtbl_cfg; struct io_pgtable_ops *pgtbl_ops; int (*finalise_stage_fn)(struct arm_smmu_domain *, - struct arm_smmu_master *, struct io_pgtable_cfg *); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; @@ -2253,7 +2249,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, domain->geometry.aperture_end = (1UL << pgtbl_cfg.ias) - 1; domain->geometry.force_aperture = true; - ret = finalise_stage_fn(smmu_domain, master, &pgtbl_cfg); + ret = finalise_stage_fn(smmu_domain, &pgtbl_cfg); if (ret < 0) { free_io_pgtable_ops(pgtbl_ops); return ret; @@ -2429,15 +2425,14 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) } static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, - struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) + struct arm_smmu_domain *smmu_domain) { int ret = 0; mutex_lock(&smmu_domain->init_mutex); if (!smmu_domain->smmu) { smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(&smmu_domain->domain, master); + ret = arm_smmu_domain_finalise(&smmu_domain->domain); if (ret) smmu_domain->smmu = NULL; } else if (smmu_domain->smmu != smmu) @@ -2462,7 +2457,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; - ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); if (ret) return ret; @@ -2541,7 +2536,7 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, master = dev_iommu_priv_get(dev); smmu = master->smmu; - ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain, master); + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); if (ret) return ret;