From patchwork Tue Jun 6 12:07:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9AFB5C77B7A for ; Tue, 6 Jun 2023 12:09:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=xrXjr4bg9z8VyBISOOyL6gEbUbEIwiynI5kex1/+/PU=; b=pNKJT50lf34H69qG2D5BchyILt Ln92hgEUmoU677mhTlsMl22mUWQ1iYyoLuf8ex1ymVeoURrTDDgkFCnzRZN+HzZv83TIgrlHxaMTg OFs6rKhpzmK34loTcU/Qi6fDCrMKVUKXqTfItgB7yazNJ7AoBBuQkOialXDXJCtOwUFIaaHOlK0D0 elS5uZSWg+ujQ0BnKs41g6uKqmiKGHFn4F84Oh96qkYpdRNBQqpchM8UuPrebUZR+IWhYORpfIiX9 COadE7CgKm1canmSOkzvOVyGBcskWrqnJfa9Ve4ci6PN/xiDHqABSBzTZXwA7eseJVbkyg9zkL8Nh ImxAx6/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVC-001YZO-2X; Tue, 06 Jun 2023 12:09:30 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VV9-001YXT-2w for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:09:29 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-babb78a3daaso8051168276.1 for ; Tue, 06 Jun 2023 05:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053365; x=1688645365; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lJ3cILiWBl6sHwkWb+xA2FBDWFhU70WEk+uPmzetVys=; b=waAz3jZdXTtDO8YssmYN5G/omZlhLk3AOZz+YlKxfJTmIQFgsouKNpICkvLnj5PC/H bZqWVLF3XcBri+fZh1WmYCnXCapku6Wrt/uBr9E/faVNwT/kYzGDxtQq94OVjLaHSWXT lXowvoVL5l6wuhwdpgDhR3x7IXZKL7LtKM68ekc5whSTKAPi9JRVV7858iLvKqE+rAgQ V2z91OUSOPBgHt4rZ0RPDqTgQR0hkZpkD9w6Hgeww7FbN7MUsMroas3NWhEUMDgh1Un4 LEaUgWdNTiF8qSbnloEwbSGHXijMkKZgdWQFvALN51/9Hcu+UqMrEXjYbAHZLscdB6Ev xhmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053365; x=1688645365; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lJ3cILiWBl6sHwkWb+xA2FBDWFhU70WEk+uPmzetVys=; b=XXSufNJHbye8Z0gIJsdNhuxRgwnynSWWxsVQweB4/XOS3zXoXnNZ0Us5poX5uou8ur u5TBWB4zPfYSLkX+yD2BIjQqa9iHJHrTDx+xzrkBLmGRyyX5QMHASPilt5MegzajbtdV O3cBw99j6AXqK8B9KYq1+64jjIIovLm78TwiqodETNrc+PZRom2hu9WWuay11Gzs9R1U KP/JwjrU+0T90tjIKyMJTkcDWv/UZhP+gRD2X2ihfVJT7fa3Qp0K/UR98AsigPt/Tqq/ KP5v5iQfJdmoHMySIiph1GQoIkaOpXeuMXd5i0CC03akOxak/sw4HqAjyJOlMetjbjz1 irPg== X-Gm-Message-State: AC+VfDw1zVJ0jQ3csZ9d9PNfaZBYzfSRpnFkPdGk6oZ06C5Vb6WFIzJ8 wC6pZ5PZCYAcNsSN0y9diQ2F1qtbrxsb X-Google-Smtp-Source: ACHHUZ5Ybk7tUiRypHKa3i9pd62JKC+ZZIVel3EJWkEbpOHcFdGs75+3Ha1gLuNWDNdErEjkenDKSQuQDFjq X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a25:e794:0:b0:ba8:4ff5:4671 with SMTP id e142-20020a25e794000000b00ba84ff54671mr1024741ybh.9.1686053365471; Tue, 06 Jun 2023 05:09:25 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:37 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-2-mshavit@google.com> Subject: [PATCH v2 01/18] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_050927_947724_A2F2D414 X-CRM114-Status: GOOD ( 19.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 ++++++++++--------- 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947eb..968559d625c40 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } - smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain = container_of(cd, struct arm_smmu_domain, cd); smmu = smmu_domain->smmu; ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb757227..beff04b897718 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1863,7 +1863,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1946,7 +1946,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid = smmu_domain->cd.asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2077,7 +2077,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2096,13 +2096,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2115,23 +2116,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_free_asid; - cfg->cd.asid = (u16)asid; - cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid = (u16)asid; + cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; @@ -2141,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index b574c58a34876..68d519f21dbd8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -593,7 +593,6 @@ struct arm_smmu_ctx_desc_cfg { struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -707,25 +706,28 @@ enum arm_smmu_domain_stage { }; struct arm_smmu_domain { - struct arm_smmu_device *smmu; - struct mutex init_mutex; /* Protects smmu pointer */ + struct arm_smmu_device *smmu; + struct mutex init_mutex; /* Protects smmu pointer */ - struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; - atomic_t nr_ats_masters; + struct io_pgtable_ops *pgtbl_ops; + bool stall_enabled; + atomic_t nr_ats_masters; - enum arm_smmu_domain_stage stage; + enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; + struct arm_smmu_s2_cfg s2_cfg; }; - struct iommu_domain domain; + struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head devices; + spinlock_t devices_lock; - struct list_head mmu_notifiers; + struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Tue Jun 6 12:07:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C8784C7EE24 for ; Tue, 6 Jun 2023 12:10:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+QDprvRqiRPH83UUBExQROoDlKuyPxVDfBkEF5rOWCM=; b=n6cvvvje2UxOwUg1yWuWyMl4PE IM3ZbVxtdhW4dSepv3KeMrLUOnHdWIMaVqJHtYeixKONtHy4nK+EcXj1N5zalMHZGzBzp7N4R6WjN +i5J4ey0hO9WIuhMWVdKzybA0M0v5U/heaG3EJ8jFBouYusJ2qa4kJvlJvh7rtsPawWl/E5D8DCyG sbualOwQfBz6sq9bkluwrLM4ZGnpUtEPemHRkzuIYEr+1sSXddx7PSRVO6WnVlade4fJjQbUf8eBn l+uQFFGusJPRKM+PHU/pWkEulQBT2a+PaB75OWEzY9eaOr6RjXyofEa7F0KhfNm2+ZHNxJRHX0m/L Kb2o9cjw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVL-001Ydd-0g; Tue, 06 Jun 2023 12:09:39 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVI-001YaZ-2r for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:09:38 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-568fa455b8fso89457137b3.3 for ; Tue, 06 Jun 2023 05:09:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053372; x=1688645372; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0tYLIcPaSiiyZHuB/nJJEFPBa3OnAACeS9baH6iUupo=; b=22Yh0QrehWwX8i9ZY6o2wrXkvcH70GK73SeCmuwBZwKKIBgf4Y/t35VFAUZIJ/leF9 ConLKIZYFWTFR9V/OVafYZ02wvG46i5Vcp14Qf81Xm5C2Hs2dD7L/2ySTOOyN1a8I9os 4KkwnWaTT0rZuSQh804kQxJBizMU3MoLg1EUBgrAqB0y5FsizZiprrl0hrnBZdZijww+ D5N9/JQY4pwY47dUeXjE6rWyX9nbabwmJ92YYT9x/UhrcwjJqa7OCH1M1+uppz53YBcW j19D9te7IclkE5OlyAyd0o1t6I3Pab7vT1wTIv76Y26ZCgau5H3C7rRmBE3W4ahxmoaC L1DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053372; x=1688645372; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0tYLIcPaSiiyZHuB/nJJEFPBa3OnAACeS9baH6iUupo=; b=FcuWK5NCFATWeGyJqfKolcNkkWARD5bmM0YcaVHBW8wuz4nPBWJSGncuyen0Q02isl 7fCGxiscKOOFGO0Je6Go0K20yy3outDY877eJhVFDHdTXt3XqZj1rQfT04UOBtl1OYoI EgJkD/eHdbi/5Be/3ofVYt1SRTSonvWTlg2ZlvAhFAytHiIYLqpmNCwaZRVdOaeFFSWn l6LLbQgask3HFyx/DQzsj2KyneBLTU+ZyisvLnmDp+C8x4u2ZPgLYpM8SA974PyFx4as jPBCptT+fjEzoYx3MQy5JbfXUiHSfPeas+aGkpzdkZuZd5Snzf15IJtm+eyHSES+b1at IQwA== X-Gm-Message-State: AC+VfDxKlCQA0Ckj1yokPpAO7c82WJRdnYGTFisXBpc5m70R96vrrvff Dpi5OxsLpYi6h4shi/M+Mj5FyiLTJzpQ X-Google-Smtp-Source: ACHHUZ5p0pzhcVx3L9ve0S1GT3stb9FRLoqqHMUTFR2YglDAkPr5FcqBbOdhuAE/cItzugQg9zWvZoV8vR6i X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:8d4d:0:b0:561:81b:7369 with SMTP id w13-20020a818d4d000000b00561081b7369mr949415ywj.8.1686053372788; Tue, 06 Jun 2023 05:09:32 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:38 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-3-mshavit@google.com> Subject: [PATCH v2 02/18] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_050936_929212_7F461B89 X-CRM114-Status: GOOD ( 18.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Except for Nested domains, arm_smmu_master will own the STEs that are inserted into the arm_smmu_device's STE table. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index beff04b897718..023769f5ca79a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1126,15 +1126,16 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, + struct arm_smmu_s1_cfg *cfg) { int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; + cfg->s1cdmax = master->ssid_bits; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -1175,12 +1176,11 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu, + struct arm_smmu_ctx_desc_cfg *cdcfg) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -2076,7 +2076,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2108,11 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cfg->s1cdmax = master->ssid_bits; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain); + ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2140,7 +2138,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, return 0; out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); out_free_asid: arm_smmu_free_asid(cd); out_unlock: @@ -2704,6 +2702,13 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled = true; + ret = arm_smmu_init_s1_cfg(master, &master->owned_s1_cfg); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } + return &smmu->iommu; err_free_master: @@ -2719,6 +2724,7 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); + arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); kfree(master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 68d519f21dbd8..053cc14c23969 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -688,6 +688,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_s1_cfg owned_s1_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Tue Jun 6 12:07:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 15503C7EE24 for ; Tue, 6 Jun 2023 12:10:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7uFFuh13wzhbINRIgqvxLpehTgmS0M6yHzLJ+mfa7Yc=; b=zq4BRgPuWbSzBiJylxA7Bxbkt+ muHy0mNIyBCdx0hjVtRX6egIaCgWeEiHdQ1RmnyaWCpSfGp07cyIIZnWsSak2LCia7qrFMgs9xmYP 0x+MwxjXaYwzV5eXWz9ifmFBqpK0qMnRxRXD2HbuuFQBrGvQeNptBYmN32qVp4B/2+7e9kX3uRrNe emVGq72DXC0dhHHuOnZjcBmKsPhZr7BF1t8DUomDoWgG3fw4vBjrfF5MtMrd89jxWvSrnz15IEk/9 yDzqaX0O4TOXMtW7baI3H7efryrw87AN3FR/KAdwLxmvQaUeDt41pGNpuzHJRwJJm9H9icLEgyTHb EIbkvL/g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVP-001Yg5-2v; Tue, 06 Jun 2023 12:09:43 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVN-001YeQ-1A for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:09:42 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-56942442eb0so88672297b3.1 for ; Tue, 06 Jun 2023 05:09:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053380; x=1688645380; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hRUuglN2mFNqzu8aGhjOdpRI6fbxHMR7AUFnYMKbqRg=; b=kFD8qk5nz8nEkhEGuavEr1Filv5HOrp6syWncQpUGLaJi3DYvQwXC5BFVUTs6ShZMw h2Y95axOqxnM9XTmFJ6HM/LFDny8rzY6qFSMC/+3xIlGp/7hPbzW7cKJtEwfD1qJdZ7/ zyK5eZGzR/IklUms3hb6ynufw6ajbdu3QxaJ7Jfuq+CZShyStZGF4+K+2RlTw70PAT9j KenSIraTK1+dvYOx1AWhSiCb+EOCSpPRtSBeuDYUWj3jT31ikuHdVpsQd4VxIg0AQeVX Rc82mEmNmqsBHLrx5rXhOEoFNFvIeWbCvbY6ADIAislq33tv4hCanDFstI6fN4sCfrsT hQAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053380; x=1688645380; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hRUuglN2mFNqzu8aGhjOdpRI6fbxHMR7AUFnYMKbqRg=; b=aashDrJX1XFsyExJRfJAgWMqFxLlMOolrI3tavkVkCj558B8S1mANjbIdLUkHva2hg T12kZJkXs1vlT4nePyou6dHJWznNvrzhG34pYZe9bszAJU5IpmeHk8KXD3ExDlsmg4UX y1bx59Qa1CdPEijxyUqkbDXikb6XhezPnXP25qIwFFeevKrDzgknQ1vNJuZbrkt4Hk1r J/CHn8oqkA/lnj9nxMH1BoTehk4SqwdWNzp+HoI5qBPN0azqZCUpmxcDkbSec8SBO8kr AbI2k2PPZ0lgmUCQGSdaAHLbIrWz89yh4nheLCgsMorandjs7Oe4Dy39LMm31rqwk32J BYSA== X-Gm-Message-State: AC+VfDx7h2FtudQHNcmBo9+Rh745mI7I/xtYgcr/jwcI4+ONEwfqUwFs 20vxG2q9aLFUcgJ56jz7t0JmMxITUUqY X-Google-Smtp-Source: ACHHUZ548G9N8HPAkr+f/occvz3qoizKM4Via0ZKMso3g/f4jIYftDXWFivpLcTLw0OV/yYIOtXOKOVjnR3G X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:a7c3:0:b0:568:ee83:d87d with SMTP id e186-20020a81a7c3000000b00568ee83d87dmr903702ywh.5.1686053380117; Tue, 06 Jun 2023 05:09:40 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:39 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-4-mshavit@google.com> Subject: [PATCH v2 03/18] iommu/arm-smmu-v3: Refactor write_strtab_ent From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_050941_404196_25DD08DE X-CRM114-Status: GOOD ( 16.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Explicity keep track of the s1_cfg and s2_cfg that are attached to a master in arm_smmu_master, regardless of whether they are owned by arm_smmu_master, arm_smmu_domain or userspace. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 37 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 023769f5ca79a..d79c6ef5d6ed4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1269,10 +1269,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; - struct arm_smmu_device *smmu = NULL; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_s1_cfg *s1_cfg = NULL; struct arm_smmu_s2_cfg *s2_cfg = NULL; - struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1280,24 +1279,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }, }; - if (master) { - smmu_domain = master->domain; - smmu = master->smmu; - } - - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - s1_cfg = &smmu_domain->s1_cfg; - break; - case ARM_SMMU_DOMAIN_S2: - case ARM_SMMU_DOMAIN_NESTED: - s2_cfg = &smmu_domain->s2_cfg; - break; - default: - break; - } - } + if (master->s1_cfg) + s1_cfg = master->s1_cfg; + else if (master->s2_cfg) + s2_cfg = master->s2_cfg; if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { @@ -1319,8 +1304,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { - if (!smmu_domain && disable_bypass) + if (!(s1_cfg || s2_cfg)) { + if (disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -2401,6 +2386,8 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + master->s1_cfg = NULL; + master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2454,6 +2441,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } master->domain = smmu_domain; + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + master->s1_cfg = &smmu_domain->s1_cfg; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || + smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + master->s2_cfg = &smmu_domain->s2_cfg; + } /* * The SMMU does not support enabling ATS with bypass. When the STE is diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 053cc14c23969..3c614fbe2b8b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,6 +689,8 @@ struct arm_smmu_master { struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_s2_cfg *s2_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Tue Jun 6 12:07:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0FAE0C77B7A for ; Tue, 6 Jun 2023 12:10:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+9pxBpmexphj0dTSHrVDeN6w0IMl7j/gn8YPOEJZLBc=; b=FSh7JIu+txVbBeceH7QhoEc4mT 2uI3/YZCyufzGV/o0II9pHDd112EVu0VPrhIerZfYZxaO7dVodPL8eCYEVgtO5KtiEIGnpd3hbnhx CwSb3rQAjOTfI963j/Z8IMqx2jqoZBIxoqC+h8gleBQJP1lgQGXNWNIPmTXtSbOWchG0/rkymZIoX gClhfm9SBEhde2PW7ZuFJisyM64oSfJcwQNzswa887F1CE476eRp1uNDTUq6ms3JHXQjf1oGllfNp CdJInGQTLoQonBPY7PidKiWPzBV3xNMhgnQBXbo06BA3Pxiqe/jsAKgfrHtZTSJP7Gh/JS43JFiIs QS+enZFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVY-001Ykx-2H; Tue, 06 Jun 2023 12:09:52 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVU-001Yi2-1X for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:09:50 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-ba88ec544ddso9725748276.1 for ; Tue, 06 Jun 2023 05:09:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053387; x=1688645387; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2jn7i2fkADg5o5hw9/ST6xj0QVfVf89sxP9WIoaewAo=; b=3aV/qxiznuH8GTuMuHr9HvpNdOF3Mj6M/f2Q90XcVu/Pc4u1FsruIXD/E8piJxf1Ld 9qy4d0v7FNE6cU6qEPuO/5fPtlmSHiyOIyexir61uAQrLoGwxi8p5OWNfUmek3+ZSMJ+ 8VPc1vnq7RsBifEkgFpZO+d7+QTxQRcbpYWGXSlwrajLLp5U9n8Gnd93vgQ9OOu2Lr9W bBFvwx0mgkOk8iy4p/loN0TDrX7pUoseXwjcH7b3tWzgwPb7EDDT3PCipUxmbvw64A5a Sp0Cx0m47oHm0wYXpWuCeC1Z9Q6nACUySk/VAqRlc1n5WzufL2dQI3uqSemcShORn4D9 lQKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053387; x=1688645387; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2jn7i2fkADg5o5hw9/ST6xj0QVfVf89sxP9WIoaewAo=; b=eM73gkL7vcad3TxGEmI04qOoTFCjglDaEZ0AxnEsKpuQpmAyCkOL2bjdXQnuOEOwV/ AGzN3VWWqCZAsfwYT8OwUKUizlDKBY3mpegmZrhydeqGOgpKgv+ejkAdzGsERV/BK84R b8/gCK8Bz1Wk+WEVuK2OfiImQh9SPhRkBIJse0Mhka6+zKvHjp0U9Ks+kbLXzogu0HLr oG7H3BFAPqqwMxgAvzRlVzUy/5HN3Ynz7ccTLEq7vriHKty6lKCELK1D5qEC6nHu5RIT Nv32dh/VESerKIoKr2AOoSX5Dww3XM3x1VXJEbzzR1u1DzmzX7tSAbepNROjHhA0+CVa pkWA== X-Gm-Message-State: AC+VfDy1kGf1yaaarI+4WIHpSQCWgOA9KOZ6TIMe5CYmjl539Bfa28ZF /4blsCY+xJTTVmNxHgvuLJuuZuJH4qSP X-Google-Smtp-Source: ACHHUZ4Dujxdnj8KPuw64qiRHO+6CEzrG6IUkz4lybxA+DxAoTdYihnjq/d8TCCtjGrwlDEd6rq+6WSBbgj+ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:1204:b0:bac:f582:ef18 with SMTP id s4-20020a056902120400b00bacf582ef18mr1089911ybu.5.1686053387100; Tue, 06 Jun 2023 05:09:47 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:40 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-5-mshavit@google.com> Subject: [PATCH v2 04/18] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_050948_517340_1AD97190 X-CRM114-Status: GOOD ( 30.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Update arm_smmu_write_ctx_desc and downstream functions to be agnostic of the CD table's ownership. Note that in practice, it will only be called in cases where the table is owned by arm_smmu_master. Whether arm_smmu_write_ctx_desc will trigger a sync is also made part of the API. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. Note that the next commit ceases this sharing of the s1_cfg which makes that loop necessary. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 ++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 80 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 81 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c40..48fa8eb271a45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -45,10 +45,12 @@ static struct arm_smmu_ctx_desc * arm_smmu_share_asid(struct mm_struct *mm, u16 asid) { int ret; + unsigned long flags; u32 new_asid; struct arm_smmu_ctx_desc *cd; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; + struct arm_smmu_master *master; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -80,7 +82,11 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -211,6 +217,8 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_master *master; + unsigned long flags; mutex_lock(&sva_lock); if (smmu_mn->cleared) { @@ -222,7 +230,12 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, &quiet_cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -248,8 +261,10 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; + unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; + struct arm_smmu_master *master; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -279,7 +294,12 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - ret = arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (ret) goto err_put_notifier; @@ -296,15 +316,23 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { + unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, NULL); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d79c6ef5d6ed4..b6f7cf60f8f3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,14 +965,13 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, .cfgi = { @@ -981,16 +980,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, }, }; - cmds.num = 0; + if (!master) + return; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_streams; i++) { - cmd.cfgi.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + smmu = master->smmu; + cmds.num = 0; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1020,16 +1018,18 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &s1_cfg->cdcfg; - if (smmu_domain->s1_cfg.s1fmt == STRTAB_STE_0_S1FMT_LINEAR) + if (s1_cfg->s1fmt == STRTAB_STE_0_S1FMT_LINEAR) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; @@ -1041,13 +1041,21 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx = ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +/* + * master must be provided if a CD sync is required but may be null otherwise + * (such as when the CD table isn't inserted into the STE yet, or is about to + * be detached. + */ +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, + int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1065,10 +1073,10 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; - if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >= (1 << s1_cfg->s1cdmax))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr = arm_smmu_get_cd_ptr(smmu, s1_cfg, master, ssid); if (!cdptr) return -ENOMEM; @@ -1092,11 +1100,11 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, cdptr[3] = cpu_to_le64(cd->mair); /* - * STE is live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=1. + * STE may be live, and the SMMU might read dwords of this CD + * in any order. Ensure that it observes valid values before + * reading V=1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); val = cd->tcr | #ifdef __BIG_ENDIAN @@ -1108,7 +1116,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->stall_enabled) + if (s1_cfg->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1122,7 +1130,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } @@ -1136,6 +1144,7 @@ static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; cfg->s1cdmax = master->ssid_bits; + cfg->stall_enabled = master->stall_enabled; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2093,8 +2102,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2110,12 +2117,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret = arm_smmu_write_ctx_desc(smmu, cfg, + NULL /*Not attached to a master yet */, + 0, cd); if (ret) goto out_free_cd_tables; @@ -2386,6 +2390,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + if (master->s1_cfg) + arm_smmu_write_ctx_desc( + master->smmu, master->s1_cfg, + NULL /* Skip sync since we detach the CD table next*/, + 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); @@ -2435,7 +2444,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled != master->stall_enabled) { + smmu_domain->s1_cfg.stall_enabled != + master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3c614fbe2b8b9..00a493442d6f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,6 +595,7 @@ struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; u8 s1fmt; u8 s1cdmax; + bool stall_enabled; }; struct arm_smmu_s2_cfg { @@ -713,7 +714,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -742,7 +742,9 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, From patchwork Tue Jun 6 12:07:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC912C77B73 for ; Tue, 6 Jun 2023 13:16:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FwesB2yLxjTALgowYni/Fbs/PcD0HSfyxAhgYqjf9/k=; b=Kbqjo3yr6BcgPhBPX5wu2asZlZ GcG+ECGRHSX/o/bE4ItjqIJNpYYtGtE4cPdYXI35H4AWsKDRX7RoeHeyuCB44T5Biqg7SYVV/A7VH W2Hg+LBTOLxK81W/P03RXXDWdIK9V+W+x8ZBxgPS9iDowCApUHzUxIY/l2wemddk1nKeK5N/LleEv 1kyCM6o7E4llDC/O36omB8ZOng3kym79PNcVTJf4d94kUWitfHSEHpH53/JbLKo9ikAcVhLQinr9s KdDhqfnKcb+6bMGd4h5l10Vkh5uYO8+rwUy7DMh1yMUUcUxKOiGJBENHPRRXHp7g9WzlYky3+5U81 tor9Lw2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6WX2-001oDG-1g; Tue, 06 Jun 2023 13:15:28 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVe-001YmO-05 for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:09:59 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1b02730e417so55826155ad.2 for ; Tue, 06 Jun 2023 05:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053394; x=1688645394; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Wf1RkoS1TdpddcvtRBe1GBIueZ5xLUPAAm9eoAp49Lo=; b=TtzN+i4Nv9gGr4efu4ItozcQGiq9M+zfsaBG6ZPDEmf81aQoXMt7jAp6xIL5vWASUy 3JdC1llGmcgJD2YE7TPR/xcb6ID6WzMEjTW4IHN5nSUGXpQVxCJg0LmIKvdtJIn1rMz8 nNNY5NNwnoHArYIpzkOPscZ6R0OkHXOXMVxldeeYAaksx1lr3e53zrPlL256ai4CIH1f zGOuYxToD9xJMrDHMB1iLVaxVc5NERQPjpPr+2JH3aqH2MJQE97/yB9SZbpHuiJhE3a9 QDSw6EFpOJ3nqmmYIKiSlidtEnlVEixjyTGMqYLMAZA9Rv7DXt+j6E8FhEmAHXV3WIXy ZIGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053394; x=1688645394; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wf1RkoS1TdpddcvtRBe1GBIueZ5xLUPAAm9eoAp49Lo=; b=ab5vy22KrvJAYoFhZZLo94/luFPs47a07OMrsFxH2YuqNSG9tSVboSc95Jurx75hcA gF8sAEAF5/vldOLV6UuyJ5nnvdyLqftow0o0A0sOHxik/ObReV32mOtm8oDO+CHsWSiV pT7uLxG5wYfvvz9UKqq5C9n1B2JyPvkWgSOWKcWVNC6wA9cjWJBK/ykgf2dbM791vZde Eq4v6C/0+uvx6KiFLtFcwpJegIvbYf673y5V/OFnnshx785tm9A+6zu1+1RNssfOJ0tg WDdaxid0nxj+7c+4LKUvdcncQhH43RLaKH+gcaQNi1Ye0Tdqggu8Fv9pfMz2fk0YOQTt wMrg== X-Gm-Message-State: AC+VfDxzBcuyXxBQq9jgeKUQRXip8gSVTZhaNqXI6UtvzCKpAkziztds ATDVdtJ5psMXhoYCD4Zb+fxGmx/0IsK0 X-Google-Smtp-Source: ACHHUZ4lwhgJHo/kAZj8thICGBCDPxKgi+/dMCNxEJPPDpLMYK30s7X7w7DyaKAUhvc7tyxV3avIIaVelmSw X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a17:903:32cf:b0:1b0:6a10:1ba1 with SMTP id i15-20020a17090332cf00b001b06a101ba1mr582137plr.13.1686053394148; Tue, 06 Jun 2023 05:09:54 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:41 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-6-mshavit@google.com> Subject: [PATCH v2 05/18] iommu/arm-smmu-v3: Use the master-owned s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_050958_073570_683D218E X-CRM114-Status: GOOD ( 17.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Insert CDs for STAGE_1 domains into a CD table owned by the arm_smmu_master. Remove the CD table that was owned by arm_smmu_domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 -- 2 files changed, 12 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b6f7cf60f8f3d..08f440fe1da6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2065,12 +2065,8 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2082,14 +2078,13 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2102,10 +2097,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_init_s1_cfg(master, cfg); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2117,19 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(smmu, cfg, - NULL /*Not attached to a master yet */, - 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2192,7 +2173,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2439,20 +2420,20 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->s1_cfg.stall_enabled != - master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - master->s1_cfg = &smmu_domain->s1_cfg; + master->s1_cfg = &master->owned_s1_cfg; + ret = arm_smmu_write_ctx_desc( + smmu, + master->s1_cfg, NULL /*Not attached to a master yet */, + 0, &smmu_domain->cd); + if (ret) { + master->s1_cfg = NULL; + master->domain = NULL; + goto out_unlock; + } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { master->s2_cfg = &smmu_domain->s2_cfg; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 00a493442d6f9..dff0fa8345462 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -718,10 +718,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; - }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Tue Jun 6 12:07:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7020EC7EE24 for ; Tue, 6 Jun 2023 12:10:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=h3OreILsMLhwqFatPU1CKs6YUGXd2V/taXD1bYhi5f4=; b=uQ2p95x1nYFJjDWS9rj0SR67wE 9+z4djv0lKz9tTIiwwCGfomLAcT5BvC2ywEWuX7CqOWw4ct0WkSPFXwKr+NGZaxGg2taEkwExN54J EdbMlGHO+T5UslY0UBkiyfpgdp4eNjCGbk8bxo3koAMr50nol0/Xu4vqUnHCimYoHDL84SIvZzVdG /bGs/En4ZTpSw6Hplp0z/xzC+lPMJDyMFpxraJkM52AfpqIksyo4PkEznzfvpVqb4H39sOdEI6nPB prnprAXEszkFF/RZBcmkJy/qMrgBQw54NYieOu7/Yj8Hm3h3a8kHJLLijvIDY9plkHZVe9YpyrOXi 03aMhm+A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVl-001YsZ-1Y; Tue, 06 Jun 2023 12:10:05 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVi-001Yqk-1Q for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:03 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-56561689700so98858057b3.2 for ; Tue, 06 Jun 2023 05:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053400; x=1688645400; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=q3oiLY+ykJwBM19GqpOciDwpUp4BC6bUpIyUkrOw/lU=; b=3hPQpvgL+wDxU0OjZQfIfZ4PvVC+sRTH9qlAAPJqZw1xiM2Q3iNaeKDiXdelb0sVio D6xhz8ENSzVt3nnIIlVKdYJfctHXIAa0AVbONky2oo5QejO9Xh6VEvVppH5VVD/R0DTg Rz+4hNgCNRDD7UxQBbLKaWHBeXwxqhF9abdIbDSldq7wOSbdk8rtegg1ed7wG1S+DX4F 9gDbr52SBPiRvOlnYTCDE5jSJQThXBeNqvfSEcx2+4zLLOYwZ5zlaQVcRWkWS8JjZRaJ XFM2GVTbPxXllz89+WNgxWz51t+GrQPq+ZkabafS5wNcTuWm2e98+FU9XPrgilTH4onp qvPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053400; x=1688645400; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=q3oiLY+ykJwBM19GqpOciDwpUp4BC6bUpIyUkrOw/lU=; b=FDDRQr6m9FfsSc9E2r58YiFTSu/KldGWxsz5cI2uuLKCTSSbJa/UvzGnW+eAk2txH0 UdT0ZKwtrdUqT6ruUQUs+jg96tyebxQF4iNdN7YzTdFFkwkO5fmPC3Qi0GIG61RqxMSa dXqSp+NKoa6XSfV7ktapUGI3HasNkY6g9joi1SlgS5CahLil+PQK3m640XB/4wtSjsaI BLcZGny03moNaPuT/zeZOHZRUvrXZJ+LCwCVvnTMGGEPOR4e0DBCsIhzZ1D8dWo5eRY0 XBZYC/VYSXBoovT0NmyjcBklsqJzKy1et1Uq6tfnfLfCGtuX3UzdSEzSNJ46uoT4eKMS L8iA== X-Gm-Message-State: AC+VfDyIR/0PaAtU4gu2MygfBwK3MxtBWHi0Skge8miMvkJjVNJRSZ5R mBCIL8zdIp0lv9+AW+EN8g9FX/9jBwbL X-Google-Smtp-Source: ACHHUZ4SbzdX0vrWGGzX8FEpBXrbLqYVs8p8cs/JwhdTrNJ2i2fH3FCcYmqUDP7FZD+MtLPKPWmNhTPp6j14 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:10c8:b0:ba8:797c:9bc7 with SMTP id w8-20020a05690210c800b00ba8797c9bc7mr1064725ybu.11.1686053400830; Tue, 06 Jun 2023 05:10:00 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:42 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-7-mshavit@google.com> Subject: [PATCH v2 06/18] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051002_494344_346D62E9 X-CRM114-Status: GOOD ( 12.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- v1->v2: Fix commit message wrapping --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 08f440fe1da6d..dc7a59e87a2b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2286,7 +2286,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Tue Jun 6 12:07:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7840AC7EE24 for ; Tue, 6 Jun 2023 12:10:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7xzF5E4+CLcZG/n3Z7UzjZ+nSXAVo7Je84UHpcxpRck=; b=LSQSlV2VOnzevsXYDG4+PXlqhI P14+qNWJapm0oxDoaKNWDCbC24Eg8MmRq2Y0beR6U9zDO8x29pv0KjXyCcgwoU42qi5K9VLTfBavv qTLalua39qKALfvosTUXkIEZBtDquV9rSwARpBDg07tGVwJIzmDyAQDDMu5vbAziLaQWUawbZ92qh ykxXmjymxIW8+MwvXLkAWHu4rP8AOAHLTDOhLXNPDpxdgYDRO6GtCl398MK17j8ko8xLwSyU1979r LpNXGoajqv4TTDF6fcmGslgmxGFx+RJSTl1ZS0K+lKCxv31Dk9iiuhhpTi/gIajKpC+FQu9RME4/+ kPSVRGmw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVt-001Yyi-1C; Tue, 06 Jun 2023 12:10:13 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVo-001YuO-2z for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:12 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bad06cc7fb7so9692688276.3 for ; Tue, 06 Jun 2023 05:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053407; x=1688645407; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xd2apE9I9Ql686K5E2cINv1GKkymHr8pAPZ24057YLo=; b=dUvBUGJ2uTFCpl8ZzYeV2M0aI5C148+z57kDrwYIZVrDrrDOTlwtK/k6da1JB47kly YHAETWdmnBmikZon7aFfK/ZKCgh4Enp6nkCmTtT5GvNqpLkDH2PgvgWfMhbmn6mgG7Kw An5suNQm3aoYqek50TCwOmQWe00ojcep/USlSNuwDmRCo9E8r0d6RG4bQu4fcFT6/OzN 8RUmKOHjiuRYxpF1976rRclQJRTy6zUNQJjRUEDHb/5iXfGsp33uXLP9ViCApsZtuT26 YCEq7EKWHDuJioM1w3nhhLDclVMfd+l5MQuVPq2qMPNzfworQYfG1hk0tMuJ40k2YsfJ NgiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053407; x=1688645407; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xd2apE9I9Ql686K5E2cINv1GKkymHr8pAPZ24057YLo=; b=a+qI2MNWl8vq6M2Q9nOTSGl36z2PKUfG/2+aboVdtV+eFvCIw0ToWcIAD4H06lDAL2 ITRBLPeV3k1FZsBWC9fku9+tTwKdemlMLAkoA+ybOKna5XL/PJUuOZeoBLtaok6kJmnl pMsI1OOATeWt7Hb306LGWVkdFbwUxbpvoweZXqkcbRYU8rs66nFPdUwlrpQq7Rknj7jm xI3o1okeMuKWgPR6nnP7oFyyqdGq0QZcP+nTstFeiZOOBaPByGM0jvuHHYF7OT5drcO/ YWFi5k9Apl6Pr/HCSABOUpnWYsVhS8gkoB7/o4ca6YqymmRMKMlyG0sUGQu6NZbb2Tam H7cw== X-Gm-Message-State: AC+VfDyW+NFP0PYceY4DvU8LvtcpoLhCPYs5AJRntjpEXxNND2THDpR8 /+06WogXPhBAJtP6rK2bpxAIimOVjMhr X-Google-Smtp-Source: ACHHUZ42r0IDYsLBXWt9XxFV7szayDqyBwH+j4RbI/1aUPDRFaWDF/S14ZoUu60ryXiFEZUoFZvsGSvsWE8Q X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:10c8:b0:ba8:797c:9bc7 with SMTP id w8-20020a05690210c800b00ba8797c9bc7mr1064923ybu.11.1686053407816; Tue, 06 Jun 2023 05:10:07 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:43 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-8-mshavit@google.com> Subject: [PATCH v2 07/18] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051008_969201_BF91EB46 X-CRM114-Status: GOOD ( 26.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- The arm_smmu_atc_inv_domain_ssid function is only temporarily introduced to make these changes atomic, but is eventually removed in latter SVA refactoring patches. v1->v2: Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 53 +++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 88 ++++++++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 ++-- 3 files changed, 105 insertions(+), 54 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 48fa8eb271a45..d07c08b53c5cf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -51,6 +51,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -82,11 +83,14 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, + attached_domain->ssid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -210,7 +214,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -218,6 +222,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; unsigned long flags; mutex_lock(&sva_lock); @@ -230,15 +235,21 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + /* + * SVA domains piggyback on the attached_domain with SSID 0. + */ + if (attached_domain->ssid == 0) + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, &quiet_cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -265,6 +276,7 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,12 +306,14 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); if (ret) goto err_put_notifier; @@ -319,6 +333,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; @@ -327,12 +342,14 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, NULL); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* * If we went through clear(), we've already invalidated, and no @@ -340,7 +357,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index dc7a59e87a2b4..70580ba7065dc 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1711,7 +1711,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1736,8 +1743,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1783,8 +1790,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1794,13 +1800,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1823,25 +1835,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1863,7 +1887,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1951,7 +1975,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2031,8 +2055,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2270,12 +2294,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2291,10 +2315,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2358,18 +2381,17 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_del(&master->non_pasid_domain.domain_head); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - master->domain = NULL; master->ats_enabled = false; if (master->s1_cfg) arm_smmu_write_ctx_desc( @@ -2378,6 +2400,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2422,7 +2445,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2449,13 +2471,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&master->non_pasid_domain.domain_head, + &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); out_unlock: mutex_unlock(&smmu_domain->init_mutex); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dff0fa8345462..6929590530367 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -682,11 +682,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head domain_head; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; @@ -724,8 +732,8 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head attached_domains; + spinlock_t attached_domains_lock; struct list_head mmu_notifiers; }; @@ -748,8 +756,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Tue Jun 6 12:07:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A2777C7EE24 for ; Tue, 6 Jun 2023 12:10:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=lX2GjonPW6PIbW2W0iK82aAt5nAQtGp7EGaFf6GIn+Q=; b=Pal8SglOAAtElvSL2IOUQy3Fi2 5P5AViGqWd3PBZJk0o32tEWwiI3FKNANz+R7zAqYUQjnhO9WFHcKkSTtoKnd83bxGKHOKtcxA6EVB XkgM+6aeZQs45h6yy0WxQfPYFQV6AQd0lOtOXvNaOP3DIY4jin6DmJspoO0NWP7QIzM3CYKKjLpXv AlTLzPq5pU8Iajx8rPN3gzXNL7CEJs8k4saBIhZF5ey454a/Gsc3J5yRkAgiGVvoneiQxzDTI0ugi VCPDA77ml5ViSVP9RCnPMCtFiJ3rMdBcsLcrI+5NtWkXHkceodEySENj6PRAG9Z6sha2kpU43ywho REbURTcA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VW1-001Z5d-0c; Tue, 06 Jun 2023 12:10:21 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VVx-001Z0T-2V for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:19 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bb38ef6410fso1168431276.0 for ; Tue, 06 Jun 2023 05:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053415; x=1688645415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7Q0Iu14cVZzOpWwfTMTOhF5XX3kNov0fzYqifn+D0GY=; b=Qj9FF129nefazp7rsEoZgyUDmzsW8Dl4oEfnoMt+BCvxl+NTWVy4OR8Ok5LqOWKaUx OH7Ra2Dk8NTJ1kmkTIH+GHrVJ6MooFUGr5V25M/4AriM4Cz8/19OmBMafzikZWYs40Qm Fm13HyQurOg1VgQWnI/LXtDi8eOJhMzFYZLUgVv4ltWLJ/xFS52hTWzsa+AUMGoRw701 UWMML1rZU2decc6dYLad29aIonKRI1Ygob0M9Vxe7Oe5pnNn9zl3Yxvx8/CaB3mrwLhR DUfVGLtQJ2r9UdgtgOlHuFp9zLZ1wVr8e0hzTvXgagApF3nwANZv4uRM+s4bJO8+8g/o gfAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053415; x=1688645415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7Q0Iu14cVZzOpWwfTMTOhF5XX3kNov0fzYqifn+D0GY=; b=Nw6YcxRZjOlFdgj6Fb65siLweD//SDVdHip1Xd6odfNZt3/E3IZxhmoogi5/yd6f2Q C5KzgVDnwFGJouJhRUeVrOahuk2dDdjZg9SodZCurtQrooNkmXc03ayB0Fs3E8opN7s/ WMCajGyCuHOBxGsrNBroolPsL2W635fe1n0Z99lSc3z3TCB5zrR66bRTOwxgDoH547Ks aly800lwvxJBHZ1U+B8ILqeIzjFQdEd8Q+angVzNIXNLqUf5S82Wzzya1fG1GwDnRy/T iz4WXxiOqCJ3OKG7BLcxmm0wSMULfIH5Xg24mhIx13cOE6VI6mwmsQlrlQYtltVovami HR/w== X-Gm-Message-State: AC+VfDzl8nu8cKyIV9uWgB82Rld7q2HrNVBoSAhYJ+hxxzlaWovaaGE4 zs4eH0Pri+SbBG7+1tYYXCw9ni5cWSNu X-Google-Smtp-Source: ACHHUZ7jK5Jjz1CVCGvoACxb+sx/xnsAJ9T0sjJSsJfmybQ9sFj4pmgNmu8oIVfJWJ4QZm4rznUCA24jkhkf X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a25:7d02:0:b0:ba8:1e72:2c88 with SMTP id y2-20020a257d02000000b00ba81e722c88mr941602ybc.13.1686053414834; Tue, 06 Jun 2023 05:10:14 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:44 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-9-mshavit@google.com> Subject: [PATCH v2 08/18] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051017_826801_8901B86D X-CRM114-Status: GOOD ( 14.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- v1->v2: Make use of arm_smmu_atc_inv_cmd_set_ssid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 70580ba7065dc..176013bb974b8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1784,13 +1784,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1800,6 +1802,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Tue Jun 6 12:07:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2440EC7EE24 for ; Tue, 6 Jun 2023 12:10:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=OGB/gFzMa7wNKJKlwxBvlTQPTsfYvo/L8/H7LI4Lc7M=; b=W1oGlLKBaAC8tphWdIULGzPHZs einxQILlprHBXOqOfiYfStbILAyweFMqDE1XXwDt9yniFARIe261tEwiYh/GHZ67SXPjooVwihJXm 5TeRNGj3/s3hxPeYdSVM5hOqIJXvIBrfMYb9LkWnN3bTEgcWOwMjdTPdtAmlKu+51QEv8db0/VTq1 SWzfavGk/nHgnsIoUtyF0ogguxMuEFXgvc9hnLIo/WkcVA1+seT172ltRpQgwHJYxXWFRnfLLSDJV /WLW8/3bAox5J+pSb8CtpHt+PEysjd3cN/8zvdKB9Rfg0LOr2WCtK7IUx0sZZab1ZUW/kEsp/oBFH OGt3NhGQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VW7-001ZAR-0V; Tue, 06 Jun 2023 12:10:27 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VW4-001Z7Y-0N for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:26 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-56552a72cfbso98735587b3.3 for ; Tue, 06 Jun 2023 05:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053422; x=1688645422; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3LYjw95yD0sZH0rb5+NoUTso90gxnubtDPkmj/Mi1ew=; b=bZ86/wFTbj2wtUpoCgOw4xZy8dz24Kk0iZH7f3z3LZSNyHenlrwf03yuk8k4lY8GhZ y1ArvYVhRcMz1IOi04omxYB0mcrRZI5V09XLi+Sa8QGGBr6VyEjj2P52WgwNdQQpIDLp nDqySVm3Rebb+dFrdx1Xw7MsWcJz0Q8EW3QqgA9BQUlw/qyzisMLaUBYsB+Rt9CUiT+y 4cEYoghynadDQo5J3wLuTsDPNNGXwnpsUhMWqgm+3i6dZ6/Y0Ok5gDdXY8KyIijuVa8o ZL98lanUHXZy3CyBf3yE134mXtHwlcsYxsA9IjwBUkMQpmTv+Plz3nJMC+BTxI/sUXN5 g3dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053422; x=1688645422; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3LYjw95yD0sZH0rb5+NoUTso90gxnubtDPkmj/Mi1ew=; b=Z3fta+DtTfi12c1EW1L47B7+YD3m7yN33V6rdXYQPdYenwDkPNvEqYNSjt+5nHzJgp RdgQKA/DFFpPaiugHGkC6K8a+Xkg6dB8vEpUOlb1G3l6gVkMDwyyu/4rECHIONbvDrBv tqjxKjnlZgCAaIMg2JPsLpbF+HY/n4bc/mykvtX5uIcJT6AImSd6hsXeTM23sEfl1SVr F1sLxInHLUER7N2nXNB94QoI752ggqw8/Gq7JiGKrAv5zvPKFBzDjlOcDnhbsbv2osrO ACuhz7pmcffjtV+p0A1MYtalBOZarmfz6OvywR11xn9FtATEXirQr8YZTEsf3+XLkrHP 7kKg== X-Gm-Message-State: AC+VfDxUFK2C8L1af8eGJJtNil8TThhjA7THQ4v2A0+nPOMrefUtArbe FgVI9YQAVHb6FeZfbeej9Jv/NhVeF/MP X-Google-Smtp-Source: ACHHUZ5N6kK1PpV8gIUdN8Ivermc0+vkL3GnbPorEBhBumNs25TqclqvPwtC+LqzRwPQHVAzqJj9VdH+bg5k X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:161e:b0:ba8:456a:fac7 with SMTP id bw30-20020a056902161e00b00ba8456afac7mr1108645ybb.0.1686053422637; Tue, 06 Jun 2023 05:10:22 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:45 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-10-mshavit@google.com> Subject: [PATCH v2 09/18] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051024_166847_2801E546 X-CRM114-Status: GOOD ( 26.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- v1->v2: Add missing atc invalidation when detaching with pasid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 168 +++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 149 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 176013bb974b8..a6fa56585c219 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,6 +2173,10 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } +/* + * master may be null for domain types that are finalized before being attached + * to a master. + */ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct arm_smmu_master *master) { @@ -2369,6 +2373,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2411,6 +2420,28 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +/* + * Once attached for the first time, a domain can no longer be attached to any + * master with a distinct upstream SMMU. + */ +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, NULL); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) { + ret = -EINVAL; + } + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2426,6 +2457,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2436,22 +2471,18 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) { - smmu_domain->smmu = NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu != smmu) { - ret = -EINVAL; - goto out_unlock; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; } + arm_smmu_detach_dev(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2460,8 +2491,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg = NULL; - master->domain = NULL; - goto out_unlock; + return ret; } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { @@ -2490,11 +2520,75 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_enable_ats(master, smmu_domain); -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); return ret; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->s1_cfg || master->s2_cfg) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + master->nr_attached_pasid_domains += 1; + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + } + + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&attached_domain->domain_head, &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2740,6 +2834,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); @@ -2875,12 +2978,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->domain_head); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, pasid, + NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + master->nr_attached_pasid_domains -= 1; + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2900,6 +3027,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6929590530367..48795a7287b69 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -707,6 +707,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Tue Jun 6 12:07:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3FB9C77B7A for ; Tue, 6 Jun 2023 12:10:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FiHDFcc2Z6Jq6ii7W3l3LTKTWwbaND7jEzApRXzCy3k=; b=xYaDVS9EoRffZIEbiqLZ1RflWg pTu+wKuHj52dGeBZ8Cr8lngWp1zrMQ/mK5ZHxRTI6FSCON9AHO0FnLZbuJl9migFE95sjxT0QxfqL ZDgpmEhsJhCR99yBpcmrCtzBHAapRJ7Ut3QUlW630HSpesQptSFFY09m0XBK1SZTnCYjtCVfCD0b9 8UWx0k8yYuO9WCWnXCm9H7+Q5YO9cVksUvFr7TpiwjUFOqRM/eC/4bN019u13QsCjo8Lt830EBk+d p0jQO9Nl/Avqn0e1z7H7tSKL61QlpSXFTPSjS5TMqJpeXPRaA1rnTC02E8PfM6K5BFCUXIIbuZlAJ 3q7R2AgQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWE-001ZFc-06; Tue, 06 Jun 2023 12:10:34 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWB-001ZCt-0s for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:32 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-568ab5c813eso97967937b3.2 for ; Tue, 06 Jun 2023 05:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053430; x=1688645430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=83dlZFDhtSz2rg2AR6ggZ2Mh/ASzqdpRvSd02rbzaCk=; b=IoFA55ulrjtnJDTW5pbuUTS9XeM0qmUYpaAJ3OnF3fjh404oOtNySp8WzgeTzaaiiO mVRlBZ2obwVwh4OUFmiQJt7UUnDxoDMkllhaIBQAmBhFfw48M+MKZQjL4e//71A1xQ+M ZQe0Lx0j/uNjanZ997DNS7XHK7FjkoWOzElSxea08x+rZug6VZyihB4gEnokoKygwiFy oohF6tU936F+ATOKuIyJj4FuSck9QyYVTKPs6mJWVCMGD4SJTG4PI7zHHboXxVFaZsG2 nDS7Gx45dkGyCy6J2pqiwfXPIBXnbTfVR9nkfcP+mnZCb8QJyIFTWcKIELfCjtVWDZgK oB0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053430; x=1688645430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=83dlZFDhtSz2rg2AR6ggZ2Mh/ASzqdpRvSd02rbzaCk=; b=ccreRhR1ZQ9PagnzXq2INRz2H3GCQVesX6Rs/EkWR7BhJKsfIF+mQPkzg0xWBICl+q fJWF4g7npDUyohH4r1XeyMTefhx20xP13zB4Bde+gmGWzMRrh0yejew+wocV2Gv1omXl D+7hrsyWsndvl0dvJXmx8fZ2qcSzml9glqSPt+eNQXCA7tE208H6h2Y5stObGSwJBDAc WOVDPcUUSqEVEzA7XcCSKW3lD0cOPGk45FYAz1ttgRjkorwVLZHnaM1aR7BuqjFrJAMJ 0A9WR59jb4rc+BMv/c7O2n6JrVTa3WgiTFPN7Vj4VnHLIACuZod3Mo8hWbIAOYYZ6E7x RS3w== X-Gm-Message-State: AC+VfDwperUVVlXkjJjdz3XiNcUJ1hurZR9qxXtxog/85nPfjJ9/JQKs inoa21PvMjNPo/3O9gD3C2Yyg0qGxI/8 X-Google-Smtp-Source: ACHHUZ64pvQvCxVufeRzNt0VKrbwAwJNjh5MfrOYdfJtgOmahfaQ951Oi+PY3aJvHU4m52siOi/iL4Zbsjnl X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:b289:0:b0:562:837:122f with SMTP id q131-20020a81b289000000b005620837122fmr942001ywh.9.1686053430214; Tue, 06 Jun 2023 05:10:30 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:46 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-11-mshavit@google.com> Subject: [PATCH v2 10/18] iommu/arm-smmu-v3-sva: Remove bond refcount From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051031_307054_ADC8AF2F X-CRM114-Status: GOOD ( 13.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The iommu-sva framework checks if a bond between a device and mm already exists and handles refcounting at the iommu_domain level. __arm_smmu_sva_bind is therefore only called once for a device/mm pair. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d07c08b53c5cf..20301d0a2c0b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -29,7 +29,6 @@ struct arm_smmu_bond { struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; - refcount_t refs; }; #define sva_to_bond(handle) \ @@ -377,21 +376,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return ERR_PTR(-ENODEV); - /* If bind() was already called for this {dev, mm} pair, reuse it. */ - list_for_each_entry(bond, &master->bonds, list) { - if (bond->mm == mm) { - refcount_inc(&bond->refs); - return &bond->sva; - } - } - bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) return ERR_PTR(-ENOMEM); bond->mm = mm; bond->sva.dev = dev; - refcount_set(&bond->refs, 1); bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -570,7 +560,7 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, } } - if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + if (!WARN_ON(!bond)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); kfree(bond); From patchwork Tue Jun 6 12:07:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3DADDC7EE24 for ; Tue, 6 Jun 2023 12:11:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=DXtBtpzgSks6IR07CK3v+3mFDj0BEGXiwN+UT0U/19E=; b=lFI+Rloam1BHs/xUoPRhPaZuWX 581vZiiLJGRCZN3yYy8jd6xtrGb8FfuJsFLbYtE7khwci0MHojn4MGeudCeKazIWNvoY8J8A0Dswn UH0igb9MI4K3LNs1cYk7nm9OpN9OzXERveaRiFl0kMfaoWgX9rsFcdWZPqMJ5JQpve2t53q21Mosh ehXIrvZI1o5RzOGJhMlhgFnkwzKpP53Pi3jEPkqo9rGDZvzOEsL3i2vqRUdGWAtMRpDwyU2APprVU 3hNh8GKX0WFsC2PMmjxExDSZsG0OJxY6BMc0ruJVFus5evE7U8qHdEzb5mCIwdWri+/b4hcIIv406 +P2M9QZw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWK-001ZKr-22; Tue, 06 Jun 2023 12:10:40 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWI-001ZII-0f for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:39 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-babb78a3daaso8052840276.1 for ; Tue, 06 Jun 2023 05:10:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053437; x=1688645437; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0QGlHe+zKTAk3+6sKXLWwU9ZaaAOqSw/Vt2fUhgy2WA=; b=bS4nn54XlFs3QFnaLf26ObLJelQ97e9PnCVNdve77ggjCR3JfNlgF6+o8d1vUqjrUM wvbXSNOisrmhnBwK5+9Sazm5rU2Zgirf/L3AYbVVxxno+05JhYVLwDDz3CWGUB4CgacW 7FI2dvkLiNurNExQW8FFZfErrWB4Og5cGkpTlMRWPQdtpgidl8RlYM87aELCqrHicqXT zpsqKEpKie2x+xoq6gnaxrUin3F+7UedVAY8bBpVCRRrbKzzvtt973F6bc+QBgFcfs/w A85oh7tf3LR1d+Jaic4iI6vwBIZt8daoHhctT54OQIG6Li3F73eKCdyQ6LRcKn5ogmAj Q1uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053437; x=1688645437; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0QGlHe+zKTAk3+6sKXLWwU9ZaaAOqSw/Vt2fUhgy2WA=; b=cEGbho0gWJ4X8/oAk1DCIqp9LeQ0iYsRkNpjZPu6UtfcYJjP4D0SSdWVS2EgaBfCb7 qKBt71O7N+pFEo2kYPzjwukK+vc8XrK4KMaRVstB1OyI0qlws2eWIY6IVYGcR1K13BbV JtGwdgtTqHHzKO4sAX1XRPArSt342xkQIk4z7XUWV8n4j2Lphs63FO87R6uYM597VmaI T0ITEloCtT+V3VJyFzGQouppeoVfh6SSFg2YxjM8l5y1qXzXluMWj2+UUJNbIc9iTnvG J8q3Rr4uwPXCl1BjJ6tnwq97KUP2HCZk2fduWClVdLIWeYsuqYG5WSwHKgUr078MQZeF uZVQ== X-Gm-Message-State: AC+VfDzpyVWXtVfGEqwo3/WW5vpnqIZhD88Cs0Y4prUQwT27PwmVQta/ jhbo+k7d+WWEKpwBny+v6t3a9hYB496d X-Google-Smtp-Source: ACHHUZ6jnEzs4FzKbc1eucPM21c1GLeALy+/7ZK+Ce8gDScxpZwrM/NUIO/Lv/HW4xE6mHRkm8wTRbjilPxN X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:703:b0:bac:f608:7113 with SMTP id k3-20020a056902070300b00bacf6087113mr1100722ybt.4.1686053437004; Tue, 06 Jun 2023 05:10:37 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:47 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-12-mshavit@google.com> Subject: [PATCH v2 11/18] iommu/arm-smmu-v3-sva: Clean unused iommu_sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051038_243350_42E78C18 X-CRM114-Status: GOOD ( 13.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The __arm_smmu_sva_bind function returned an unused iommu_sva handle that can be removed. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 20301d0a2c0b0..650c9c9ad52f1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -25,7 +25,6 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) struct arm_smmu_bond { - struct iommu_sva sva; struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; @@ -364,8 +363,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static struct iommu_sva * -__arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { int ret; struct arm_smmu_bond *bond; @@ -374,14 +372,13 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); + return -ENODEV; bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return ERR_PTR(-ENOMEM); + return -ENOMEM; bond->mm = mm; - bond->sva.dev = dev; bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -390,11 +387,11 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) } list_add(&bond->list, &master->bonds); - return &bond->sva; + return 0; err_free_bond: kfree(bond); - return ERR_PTR(ret); + return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -572,13 +569,10 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { int ret = 0; - struct iommu_sva *handle; struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - handle = __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(handle)) - ret = PTR_ERR(handle); + ret = __arm_smmu_sva_bind(dev, mm); mutex_unlock(&sva_lock); return ret; From patchwork Tue Jun 6 12:07:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 724C8C7EE24 for ; Tue, 6 Jun 2023 12:11:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=AiZpvWC0TNlWbTr/4QRgXB7YViZWioaZ04a3ziZFOms=; b=NGKBn26/+6AEcr0SXft9VfgXVD 2h3a/ZZDJGdNMe5Xw2DHT9NI+NCdooBO5ZYrToNrVcjKruLlfX3x+sYPUCaVnaViRTWOm96thgjeX xCX7NZ7RcQwKFE+4+AYWZ4+oN+BfqdbVT4ARN8ZLaaoSnZ/RtUFZlWXPU7AFoM1fUSMpmRH0qsVV2 OgZwPWkfcMJRQNjcEyxiTSQYI5IGTHcQQsIcvq2HNcvQ8+Dg/AUcTsW4Vq4OukwsGLEAJJ3VsOuGX OWHJkDXozKLeFtP9zt5Jim8oN7S9v2e+egzw50xE55J7PHL4uf5IIyFiQtUnSZXkvxVUCaMseuhOT nozeq+Eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWT-001ZRf-20; Tue, 06 Jun 2023 12:10:49 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWQ-001ZO6-1M for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:48 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1b03f9dfd52so23975905ad.3 for ; Tue, 06 Jun 2023 05:10:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053444; x=1688645444; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FnWnvbHmIF1ltrhpko3AmatpWzsBYOi2I16/Atgd3Oo=; b=OT+iGAb66chgU7NBKy4tFHkZVVSeUEqENw2JBDh6NO0OJc11YO1CWkwgNkZjHtYtDP lZ7NxCAGg5dgVzWFUtOEhk2CWrx1Pfiqw0NYIcjLZSSHU16TLJ0GKWraeJX9Q0twcSP6 3YgWeiLUY98PryAP+mhZtWKw3M6BYA3n/XIGSGx3+UrSbGweWJffi8+xWt2FfV3NThzo XBXaKTbXD4ydtH9xv2nOk9efQS5Rjmd49C5Tsn1GOsK8wtmeQmZU8CsM5vSV2dmEDWv+ GKDEFYPFJzpH0Rq9Q4iEdy1A3yyzmEYO02gaGn+H03RDX5nInQ1OlVuD38G4glG62cFv r7RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053444; x=1688645444; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FnWnvbHmIF1ltrhpko3AmatpWzsBYOi2I16/Atgd3Oo=; b=OzVMI/i3kJlX8DD30ane1BHd/9ANkAlcmjXnSbcVEwf+hg0K+2MA4v+Mza87sETMvm 9Utm1m2lhcjTakuPz58NVChNT3FAL+cWyClo5sHWEoTC66q1NEu1HVUrksaX7Lwltdm3 XQkFKgrs1efcHfJbPFnFHxfweVHnVlTQUS8HStM2axsZw+LCy7hK9ZBodIklS1lwc496 mBjITR60kAftsELbrYg6ugF+LMTr7IoEr42y0yglCuBDmLg06U4dy4XVLsYU4WNlZVHn C+bNEUTJHdNyFp/l2o3oJUbN2GbOdGGgPq5Nn5TtQKz52bxnfj8Vsk3TW6fRQ7O8rJAX GICw== X-Gm-Message-State: AC+VfDyClpk/Bb8IlUvJLc911r6PfBYzceWiA8Dw6dN/UERV4ZVBDTVh F2TTmfzdLhSLaU6fCiosr6jTGQZknbcP X-Google-Smtp-Source: ACHHUZ4S3Q+ySPLlvi9D4XGLvO/Q6X4CMd1K3T6dbkp23pUhANdGW1N8k+Ri4zirY2cDlUlKNOpRe2XT+uiU X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a17:902:8693:b0:1af:ff00:dfc2 with SMTP id g19-20020a170902869300b001afff00dfc2mr602005plo.5.1686053443904; Tue, 06 Jun 2023 05:10:43 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:48 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-13-mshavit@google.com> Subject: [PATCH v2 12/18] iommu/arm-smmu-v3-sva: Remove arm_smmu_bond From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051046_467231_FF78B72A X-CRM114-Status: GOOD ( 19.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's a 1:1 relationship between arm_smmu_bond and the iommu_domain used in set_dev_pasid/remove_dev_pasid. arm_smmu_bond has become an unnecessary complication. It's more natural to store any needed information at the iommu_domain container level. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 24 insertions(+), 48 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 650c9c9ad52f1..b615a85e6a54e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -24,14 +24,13 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) -struct arm_smmu_bond { - struct mm_struct *mm; +struct arm_smmu_sva_domain { + struct iommu_domain iommu_domain; struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; }; -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) +#define to_sva_domain(domain) \ + container_of(domain, struct arm_smmu_sva_domain, iommu_domain) static DEFINE_MUTEX(sva_lock); @@ -363,10 +362,10 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, + struct arm_smmu_sva_domain *sva_domain, + struct mm_struct *mm) { - int ret; - struct arm_smmu_bond *bond; struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -374,24 +373,14 @@ static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return -ENODEV; - bond = kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return -ENOMEM; - - bond->mm = mm; - - bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret = PTR_ERR(bond->smmu_mn); - goto err_free_bond; + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + mm); + if (IS_ERR(sva_domain->smmu_mn)) { + sva_domain->smmu_mn = NULL; + return PTR_ERR(sva_domain->smmu_mn); } - - list_add(&bond->list, &master->bonds); + master->nr_attached_sva_domains += 1; return 0; - -err_free_bond: - kfree(bond); - return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -521,7 +510,7 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { + if (master->nr_attached_sva_domains != 0) { dev_err(master->dev, "cannot disable SVA, device is bound\n"); mutex_unlock(&sva_lock); return -EBUSY; @@ -545,23 +534,12 @@ void arm_smmu_sva_notifier_synchronize(void) void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond = NULL, *t; + struct arm_smmu_sva_domain *sva_domain = to_sva_domain(domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm == mm) { - bond = t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } + master->nr_attached_sva_domains -= 1; + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); mutex_unlock(&sva_lock); } @@ -572,7 +550,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); mutex_unlock(&sva_lock); return ret; @@ -590,12 +568,11 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { struct iommu_domain *arm_smmu_sva_domain_alloc(void) { - struct iommu_domain *domain; + struct arm_smmu_sva_domain *sva_domain; - domain = kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) + sva_domain = kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) return NULL; - domain->ops = &arm_smmu_sva_domain_ops; - - return domain; + sva_domain->iommu_domain.ops = &arm_smmu_sva_domain_ops; + return &sva_domain->iommu_domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a6fa56585c219..b7f834dde85d1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2784,7 +2784,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 48795a7287b69..3525d60668c23 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,7 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; + unsigned int nr_attached_sva_domains; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; }; From patchwork Tue Jun 6 12:07:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 161EAC77B7A for ; Tue, 6 Jun 2023 12:11:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=itz/HEtdE90toX6nOsnTfTTcWg83Y74TTyTwILuM1Qk=; b=Qnc2PzeH46Yd0DOxYs3bVlufot i6724xruecXrVu+Ah3rUIukn+aFuADTsWVZTmTj+4vUhRcnWf4IyNpjSR0Bzhj0c6aSql20qK/8oh hh85bUdB0IfmZC8zriQ9GbalLa6v/0dJaTOV+eJ8X+LhYoCiBynxNsYsJSu/HZ8uJAR4HUy18IlIZ IgE/Lslj1i6fkQBfeWiNMUiDr+oQjX61jr2YEUPCntcehF6ttZ/3UxLDndGFfTLDQLW5/ThcEq4by N8DL/6BcNF2TOId1pFveLhJ2rLgfV19aggNxcSx9EZqMjJrkhOxCjJRQy/C2HbhT9ImiZmzm4zKf+ 6/wy8Upg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWa-001ZXJ-1r; Tue, 06 Jun 2023 12:10:56 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWX-001ZSi-1p for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:10:54 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-256647a6fadso2017423a91.3 for ; Tue, 06 Jun 2023 05:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053451; x=1688645451; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=C0PSyQeXC7Iz/P3YpxIMlG3+vOjTAssvVZqtnA/BFhw=; b=2YvtQkhtruPyaU6tyIiOFVtpwOKVajkps/SV+rmgxl5WcQGFxK06oATnOVAhefKsb/ /+tX6arfJ/Za9QYTa80Ei6E11onSKYS7FIz7iInWoRFPOWEY5aPsmfcRL0zuObD3kAq0 teU/b922dsyZ//kQGYgXVvkHMSBnz3ieADNsR8xmAE0B3/N50Bl+vA1z7orEKcxD09Na bdbw4y7eF6C+HCfUimAIszfp3GX30HXWkqD/lwSejlJbb9AM/4g8brT1EpgR9qNhnEbM IypWkQ/P7yNKwLZhGySBmvzzhXMm7ZZFnsb2tJniozWmxdNQhN6ba8XMv52XGQjbrdS9 De3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053451; x=1688645451; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=C0PSyQeXC7Iz/P3YpxIMlG3+vOjTAssvVZqtnA/BFhw=; b=QHoQUJK4an+kLjZsSEZ/mboBR2P124y8uVXEbdet2fvUgk3+zexd/SFjmYU9ccFGuC wVe0/2VmtbRef5E9gPNtLcIQ34xkQANPNun9BO1ym5ouwTeSABmiDfwIVgyzNz58LTpo KUy8eJndhvhdZSmesV9cv3VUlo75ZLSIPwC0xqqMnWe+WFPqugxlwj1ebbCTimP3GTBJ vWOfWcstuWqAXw/BehDuAxtmfZxKS2BmZ5fyKN2egKmsf1el0o4lUMDnW1g+B5dJtoXx VE/StxFiXUXlF/cmNB2HB/usunhJIN0NNo9PaJ9mt3m/stduENZaA4AQ0GFuyfjmNgNE fyNg== X-Gm-Message-State: AC+VfDwg9g+r34EDa1UoRZSCgFHGGWqPI2e+WBa60zS6UYz8tnTVo6Qp geD7o0Lk8V+mn48BpFgOKGaPQO2Z/a+Z X-Google-Smtp-Source: ACHHUZ4x7W0FDltSlEvI3muVSMQ0fdaK5g12vYIpYYXAKI4ob4o42ABzQjjOpYbLtirWzKE/HT0Jn0tZ7kY+ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a17:902:d483:b0:1ae:485e:d192 with SMTP id c3-20020a170902d48300b001ae485ed192mr629661plg.8.1686053451002; Tue, 06 Jun 2023 05:10:51 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:49 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-14-mshavit@google.com> Subject: [PATCH v2 13/18] iommu/arm-smmu-v3-sva: Add check when enabling sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051053_622052_AFE01E1A X-CRM114-Status: GOOD ( 10.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SVA domains can only be attached when the master's STEs have a stage 1 domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index b615a85e6a54e..e2a91f20f0906 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -499,9 +499,15 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int ret; mutex_lock(&sva_lock); + + if (!master->s1_cfg) { + ret = -EBUSY; + goto unlock; + } ret = arm_smmu_master_sva_enable_iopf(master); if (!ret) master->sva_enabled = true; +unlock: mutex_unlock(&sva_lock); return ret; From patchwork Tue Jun 6 12:07:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0CB6FC77B7A for ; Tue, 6 Jun 2023 12:11:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=m3iVVhJf6FbNyosEA6yGuZ8dmzh+EXNONqmaR1NJpO0=; b=mqsdM7+DAMUqpslQytfPJkbd/T ICjOljbeAdU2cDbbvYwv/ppDep7rdEMe+eIYgSoRvix6fz7oqye7Za1OXmfF9ZwpnH/So7NHGV6ls y3c2M+Knjz6GmTYXLFy7Zep0m7fNX17fYStMQeK7zG6voqH+R956vlPO0HLQVBaRn/scF87CLZFyi 2Ck+qRm1JMInSADscF2RxAQ9I5cumwpzfgH/bJpp+BStPHKTpLxpaDqrSQuTjvlEmgQnxq9p3/pj+ zTDjuDGUioOFyKAs90ng1ZRazXLkSpH7U4k5c9DDiHL9C8mZctG8EVa8xNg6MbbeUWHapsezbqVKJ vVj0etSg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWg-001ZbF-0u; Tue, 06 Jun 2023 12:11:02 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWd-001ZYo-1b for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:11:00 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-ba8337ade1cso9622573276.2 for ; Tue, 06 Jun 2023 05:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053458; x=1688645458; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=e/IPHrVLjUq2s1o0Kk8jlOX8iem0l8MY258dkWJhSSA=; b=VZ4aVUegMrv+4Q6yvYwHunY/3cwuUlcr0h58VcDyy+whkFdBNoq1bmH2cIlw3NPTGf 2xwwiPuSrWUGDBls3d7jNYpe94klnR35TS5OkpOjjZPOxAO8MIWR85bzh29QZGu9r6vq Uhrp4oqPT2o6ewJtPDlh64Ybaq2kCogm4tvMgZDNn5hii7e3Sp7nkxS2CPQ+QqPKUqO2 jr7HF72nwtGIZwbU2rVIDb/0fXNIWezbdzedeFrmGi0/fABER6IwQMRfR22nwYnhBjrI RYCBaFffmg4sgiF4vWOkI7rmUWylAkhG7nKO4jBF3BV258AG4UrAU6eDZogjLom8Cb8J fe7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053458; x=1688645458; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=e/IPHrVLjUq2s1o0Kk8jlOX8iem0l8MY258dkWJhSSA=; b=gj7o9nFISbVwcqs+IzyFTetQ83xaQuu9q/hW91yraw8AKYqQ3UpWagb6wSEZuB5Rfe 1bzr+kYtRY7kSEtwveKtKtLp8LQaa662N4vVQ0zzDPLhwR4v21PvyviewdnHyyZ/xPNZ WbGXYjFRchbAhlNRIR/wdRLum2jIFAQeX+pnbAC22Uim8L6CXDQ9d0p5H4jFAOo8RxH4 6CWtW9JDuMt2Nc3InJINJFmDeTFNTTFG4cE76D8Kqeta006VOmD5mhlYBhXl6xMlNa8u 9WLac39A+btK5O1f2gSeg6OwpnWtH/nj88ntcEK1l8UW45/AICNKUISfjjEww0jtUV4r HAMA== X-Gm-Message-State: AC+VfDxyFcMHLaRMxAZDOVrzr66I3HbMXhJAEFFZ4vGe5g/HdYhS/381 FCZuxRwNzZtWgnOHbJHxOvV28BSe4yDf X-Google-Smtp-Source: ACHHUZ7wQ2cnghC4sSj2hPvQ7d1hwPGvVMm7khlu49LtLrhV/uxm381LyvtDF8uhJajK7hUiXOUQxpnGItxh X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:1183:b0:ba7:7478:195f with SMTP id m3-20020a056902118300b00ba77478195fmr620081ybu.13.1686053458202; Tue, 06 Jun 2023 05:10:58 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:50 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-15-mshavit@google.com> Subject: [PATCH v2 14/18] iommu/arm-smmu-v3: Support domains with shared CDs From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051059_537440_02B3851A X-CRM114-Status: GOOD ( 16.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SVA may attach a CD to masters that have different upstream SMMU devices. The arm_smmu_domain structure can only be attached to a single upstream SMMU device however. To work around this limitation, we propose an ARM_SMMU_DOMAIN_S1_SHARED domain type for domains that attach a CD shared across with arm_smmu_domains (each attached to a different upstream SMMU device). Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 ++++++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b7f834dde85d1..69b1d09fd0284 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,6 +965,20 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } +static struct arm_smmu_ctx_desc *arm_smmu_get_cd(struct arm_smmu_domain *domain) +{ + if (domain->stage == ARM_SMMU_DOMAIN_S1_SHARED_CD) + return domain->shared_cd; + else + return &domain->cd; +} + +static bool arm_smmu_is_s1_domain(struct arm_smmu_domain *domain) +{ + return domain->stage == ARM_SMMU_DOMAIN_S1_SHARED_CD || + domain->stage == ARM_SMMU_DOMAIN_S1; +} + /* master may be null */ static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) @@ -1887,8 +1901,8 @@ static void arm_smmu_tlb_inv_context(void *cookie) * insertion to guarantee those are observed before the TLBI. Do be * careful, 007. */ - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); + if (arm_smmu_is_s1_domain(smmu_domain)) { + arm_smmu_tlb_inv_asid(smmu, arm_smmu_get_cd(smmu_domain)->asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1968,10 +1982,10 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, }, }; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + if (arm_smmu_is_s1_domain(smmu_domain)) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->cd.asid; + cmd.tlbi.asid = arm_smmu_get_cd(smmu_domain)->asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2549,7 +2563,7 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, return -ENODEV; } - if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + if (!arm_smmu_is_s1_domain(smmu_domain)) { dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); return -EINVAL; } @@ -2575,7 +2589,7 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, */ mutex_lock(&arm_smmu_asid_lock); ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - pasid, &smmu_domain->cd); + pasid, arm_smmu_get_cd(smmu_domain)); if (ret) { mutex_unlock(&arm_smmu_asid_lock); kfree(attached_domain); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3525d60668c23..4ac69427abf1c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -713,6 +713,7 @@ struct arm_smmu_master { /* SMMU private data for an IOMMU domain */ enum arm_smmu_domain_stage { ARM_SMMU_DOMAIN_S1 = 0, + ARM_SMMU_DOMAIN_S1_SHARED_CD, ARM_SMMU_DOMAIN_S2, ARM_SMMU_DOMAIN_NESTED, ARM_SMMU_DOMAIN_BYPASS, @@ -728,6 +729,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { struct arm_smmu_ctx_desc cd; + struct arm_smmu_ctx_desc *shared_cd; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Tue Jun 6 12:07:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5DFC6C7EE24 for ; Tue, 6 Jun 2023 12:11:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ttCby0PbaVfRI95oU3JizGBRvIZlb9JNvIos5JKilJs=; b=WS9DMzWzUlpXsWSZMz929WaoYv GkvV4gDAQByh2sLyv1Gw9Un/1RPjC/pRov1ZCQ5TYhy5XawYkQ06+pjqwKPrdBXsiXYZM08tW9c/e U1Q0HXPLQVHr5UmKzYXsBSKOcvkvmJZBrezDrj0gL+cc9vLAb7AV0i5LaulEAY8Y6cxaJE/9PXakw yDrMR+k3HlPuFHUsnK4N2JvIjuYnQ3a9YWz6Cbw9nGDnIIdTTYNavOO5T5mDClUK9ZsMmPSCnBnG0 ZHNnZYSx3Te6E5AQayseTvicEVuoNTkZyodN/+6yi9Pf7Nui70DX5LNw6xHo93OQ+uvOj76TbfpUf Y0byDwCw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWm-001Zh2-2b; Tue, 06 Jun 2023 12:11:08 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWk-001Zdu-14 for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:11:08 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bb05f0e6ef9so8996471276.1 for ; Tue, 06 Jun 2023 05:11:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053465; x=1688645465; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KjUOU0DPHGHGmTP8anm43RkbEqkADC+hmHbnOauQw70=; b=udNKb9hwr0Kk9+BMB5itHj3lPHTSQqr87Qcscot19+Y4rmBCzb19uWrHkx/Yg/muYr MGnuG3BikPGCJJpxU6JP94pix6vlJnCu6PPp2wr+vKwfs8xuLQYo48h5eTdLrLdFCsDq lDUqnvfRISjHbk7HBIJh4zxYeGfUKG+Uh6XEG8xJGMiCYyuh31GHhpeObSF6rfLCWKMQ hSF6T73IH9AXGpQgRV9XBJv7cE02pjOCpymzblrPCUrYRuRkPT2C8bQDiH+m09tXB/7G mK8/B294f8Jybhs9wu6cVKkJO0K5faZneZGSNqaCMhqKBxcMW5E+WPg+FsoEqeUXUccd OJpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053465; x=1688645465; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KjUOU0DPHGHGmTP8anm43RkbEqkADC+hmHbnOauQw70=; b=CCxO5nn/aFRda+cAZXbJYv+cW6lyZecDhg8wbcJaxdETfewNOzFTVSMYl0kDLBQlxo CNxzGLm7d69Ez3g2ZwaD97BTqObdAd35zAsBTaRNQ4DWTuNMNYuAsV/IGQa8Xof8YpgM oC4MK8XYqT7O/S4mBRbxqDFHVeCEDCe/ufItQUYlRobaSwMKiJ+uahzmuA8THotLPWBH 2LArFhzGPuf6m68KGdtz3q2hOSC/ey4F5DHNTF5BpJxfaAQac+i1gnNpPiXzIf655qtr R/GmGbBYYFxLxlMydGU/T+LlZQnGVv7jJbRTTTxvJwDxNzqOrETninIU7jDuaCFvgLRk 11Wg== X-Gm-Message-State: AC+VfDwXxovIXEC72D6ohTDwj9jGWAU2HdQvWdyrHT8EC5HoWvvM/keJ 8I0Iu5hrysxj+zzympr7JVDL2W1xysIw X-Google-Smtp-Source: ACHHUZ7AhoR4YEqAAGrC5wUivVYgQiraCSATW/GS+5HPDvrEOFS63RxN2Qa/uS1DsH75Bn8LomG8TlzYehmQ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a25:ac4e:0:b0:ba8:5bdf:2e84 with SMTP id r14-20020a25ac4e000000b00ba85bdf2e84mr953711ybd.10.1686053465052; Tue, 06 Jun 2023 05:11:05 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:51 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-16-mshavit@google.com> Subject: [PATCH v2 15/18] iommu/arm-smmu-v3: Allow more re-use for SVA From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051106_382417_7410DACE X-CRM114-Status: GOOD ( 18.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that arm-smmu-v3.c supports attaching domains with pasid, SVA can also re-use much of the same logic. This change allows SVA to allocate arm_smmu_domains with a shared CD and attach them using the arm-smmu-v3 set_dev_pasid implementation. Because these domains aren't backed by an iommu_domain we must make sure that an arm_smmu_domain's backing iommu_domain isn't accessed on functions used by SVA. A good rule here would be to allow domain_finalize to access it. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 63 ++++++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 11 ++++ 2 files changed, 60 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 69b1d09fd0284..3c5ff4f58934a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1926,7 +1926,7 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { /* Get the leaf page size */ - tg = __ffs(smmu_domain->domain.pgsize_bitmap); + tg = __ffs(smmu_domain->smmu->pgsize_bitmap); /* Convert page size of 12,14,16 (log2) to 1,2,3 */ cmd->tlbi.tg = (tg - 10) / 2; @@ -2053,6 +2053,14 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap) } } +static void arm_smmu_init_smmu_domain(struct arm_smmu_domain *smmu_domain) +{ + mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); + INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); +} + static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) { struct arm_smmu_domain *smmu_domain; @@ -2075,14 +2083,22 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) if (!smmu_domain) return NULL; - mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->attached_domains); - spin_lock_init(&smmu_domain->attached_domains_lock); - INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); - + arm_smmu_init_smmu_domain(smmu_domain); return &smmu_domain->domain; } +struct arm_smmu_domain * +arm_smmu_init_shared_cd_domain(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_ctx_desc *cd) +{ + arm_smmu_init_smmu_domain(smmu_domain); + smmu_domain->smmu = smmu; + smmu_domain->stage = ARM_SMMU_DOMAIN_S1_SHARED_CD; + smmu_domain->shared_cd = cd; + return smmu_domain; +} + static int arm_smmu_bitmap_alloc(unsigned long *map, int span) { int idx, size = 1 << span; @@ -2541,11 +2557,9 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { int ret = 0; - unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; if (!fwspec) @@ -2558,6 +2572,18 @@ static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, if (ret) return ret; + return arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, pasid); +} + +int arm_smmu_domain_set_dev_pasid(struct device *dev, + struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid) +{ + unsigned long flags; + struct arm_smmu_attached_domain *attached_domain; + int ret; + if (pasid == 0) { dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); return -ENODEV; @@ -2991,12 +3017,8 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; - struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct arm_smmu_domain *smmu_domain; - struct arm_smmu_attached_domain *attached_domain; - unsigned long flags; - if (!master || pasid == 0) + if (pasid == 0) return; domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); @@ -3005,7 +3027,20 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) if (domain->type == IOMMU_DOMAIN_SVA) return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - smmu_domain = to_smmu_domain(domain); + arm_smmu_domain_remove_dev_pasid(dev, to_smmu_domain(domain), pasid); +} + +void arm_smmu_domain_remove_dev_pasid(struct device *dev, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid) +{ + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + unsigned long flags; + + if (!master) + return; + mutex_lock(&arm_smmu_asid_lock); spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 4ac69427abf1c..2c33c0461036d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -761,6 +761,17 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size); +int arm_smmu_domain_set_dev_pasid(struct device *dev, + struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid); +void arm_smmu_domain_remove_dev_pasid(struct device *dev, + struct arm_smmu_domain *smmu_domain, + ioasid_t pasid); +struct arm_smmu_domain * +arm_smmu_init_shared_cd_domain(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, + struct arm_smmu_ctx_desc *cd); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Tue Jun 6 12:07:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29F52C7EE24 for ; Tue, 6 Jun 2023 12:11:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=GrAl/dALxjn+DlGK7+nKzYF3/+nfCCeOlC07teiUVqs=; b=uGC6RjLV1DNTOca1Sd9/4qDBZF LZWSMuNTF5YlRpD6mDLOHPCWfbOUzBYfNh5fsuGGCqoWSPGcgHTmqT62ze7o5wCuWdd9YHW2jkFYJ 9jkwbTbJaoHQ3ATmDbJmtdGtLDTKWjDBHIuaHwI2+lJqq6p3vKrticXTO6LzIfyY+vNAPgit5Bx3o VA4ZBpvyqPdGWZy08asgd2ZKG+8C7HDatl+6DKGBkETh3q+roe4pYETt6hOPpvkOF78uA+dQ0QWxH r2IHa+ccsPm/HHvkGQRocE4R51DkSd8TPbRHxyfXaHcKMhdCn3ec67G7CKjkI/9D0PDYIzj9Z+UwR iCAOE3Jg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWx-001Zq4-1P; Tue, 06 Jun 2023 12:11:19 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWr-001ZkE-2Y for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:11:16 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-565a33c35b1so89796107b3.0 for ; Tue, 06 Jun 2023 05:11:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053472; x=1688645472; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=k8coPEvUwDlKI+G80MftH9xxXZgTa0epa7omkndSMHA=; b=vWzHFe+f6b0SO9ZjCJqoUru52xdUuNnZIq6LnMnCtriPlDLRJZ18LiSlZY3NpYKKCf nMe/iV6ctP3Oz5SwfbarLiRIxyCChHxJbaf7V04aWyS1M/t6tk7jjopXLvNz/UIoBA5O CDebWGr2/1v3E1fW0PClyPNl068DVDA6EOjD4ZobvbmF2BTjWvCkGSao48mifbZOWmNb /QyrEUnZXV/JMNSnzX++RYhSbPNvVocemSoVMQNs4El0N9p5AXTml140OsqVQ7c5W4kG b1q7xBiEzKY39mS8kM2Bpbgxd/iihYyrRoZV2ieeGptRuyuSzNFCxplOVUU1visxhY6O kFPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053472; x=1688645472; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=k8coPEvUwDlKI+G80MftH9xxXZgTa0epa7omkndSMHA=; b=OWCJ9W/HzJg8ZqlV2cjw5MJH5As5PlHCQ6mmD7KyJqlVoJwFfvzfpJImsK0iAGNUGM sNSnzznzAECmTzyAogvKmbDmPBP++bxgIGqxE2V9CuTtBzx3MGeQk01hzUQ41cFQK4Db jt7a99R95Q+j4A2msqB4e+MbXdjdW99HLWpfmnsaLo7qWXUvKKu1QCYpeuASDxqPbjvb aIj3FKG3aq1LsIpsYFsQ9Ers0KWux3y3DVgV1bpTPDMTPEm7gxKsSeUWEEwmIubNA3yh Qy3q62JFudywConUlpkFVAevLywynafjVozTrNk/sBSYtCgLyWNnGTDqS3ysxm9k/sFc E7fw== X-Gm-Message-State: AC+VfDzWdktseZ99OGxMeWSH0GHC/yo7/PZbHGlrmPLGt3fYVBvXNSbq ehaCKDkP9Se91XuxgChRVNxUZ9Q+Ni0c X-Google-Smtp-Source: ACHHUZ4yVw8kxBQaZNeJJwBixm7xdLTIiPMzfDmRrqmi1UCOSbaEkRgUbl8QYue0ANyIEacYietJ03+RyloM X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:ad1c:0:b0:55d:6af3:1e2c with SMTP id l28-20020a81ad1c000000b0055d6af31e2cmr959734ywh.3.1686053472400; Tue, 06 Jun 2023 05:11:12 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:52 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-17-mshavit@google.com> Subject: [PATCH v2 16/18] iommu/arm-smmu-v3-sva: Attach S1_SHARED_CD domain From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051113_837221_B96062DF X-CRM114-Status: GOOD ( 17.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare an smmu domain of type S1_SHARED_CD per smmu_mmu_notifier. Attach that domain using the common arm_smmu_domain_set_dev_pasid implementation when attaching an SVA domain. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 67 ++++++------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index e2a91f20f0906..9a2da579c3563 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -19,7 +19,7 @@ struct arm_smmu_mmu_notifier { bool cleared; refcount_t refs; struct list_head list; - struct arm_smmu_domain *domain; + struct arm_smmu_domain domain; }; #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) @@ -198,7 +198,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, unsigned long start, unsigned long end) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; size_t size; /* @@ -217,7 +217,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; struct arm_smmu_master *master; struct arm_smmu_attached_domain *attached_domain; unsigned long flags; @@ -233,15 +233,10 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * but disable translation. */ spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); - list_for_each_entry(attached_domain, &smmu_domain->attached_domains, - domain_head) { + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { master = attached_domain->master; - /* - * SVA domains piggyback on the attached_domain with SSID 0. - */ - if (attached_domain->ssid == 0) - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, - master, mm->pasid, &quiet_cd); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + attached_domain->ssid, &quiet_cd); } spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); @@ -265,15 +260,13 @@ static const struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = { /* Allocate or get existing MMU notifier for this {domain, mm} pair */ static struct arm_smmu_mmu_notifier * -arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, +arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; - unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; - struct arm_smmu_master *master; - struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,7 +287,6 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, refcount_set(&smmu_mn->refs, 1); smmu_mn->cd = cd; - smmu_mn->domain = smmu_domain; smmu_mn->mn.ops = &arm_smmu_mmu_notifier_ops; ret = mmu_notifier_register(&smmu_mn->mn, mm); @@ -302,24 +294,11 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, kfree(smmu_mn); goto err_free_cd; } - - spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); - list_for_each_entry(attached_domain, &smmu_domain->attached_domains, - domain_head) { - master = attached_domain->master; - ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, - master, mm->pasid, cd); - } - spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - if (ret) - goto err_put_notifier; + arm_smmu_init_shared_cd_domain(smmu, &smmu_mn->domain, cd); list_add(&smmu_mn->list, &smmu_domain->mmu_notifiers); return smmu_mn; -err_put_notifier: - /* Frees smmu_mn */ - mmu_notifier_put(&smmu_mn->mn); err_free_cd: arm_smmu_free_shared_cd(cd); return ERR_PTR(ret); @@ -327,27 +306,15 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { - unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; - struct arm_smmu_attached_domain *attached_domain; - struct arm_smmu_master *master; - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); - list_for_each_entry(attached_domain, &smmu_domain->attached_domains, - domain_head) { - master = attached_domain->master; - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, NULL); - } - spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - /* * If we went through clear(), we've already invalidated, and no * new TLB entry can have been formed. @@ -369,17 +336,26 @@ static int __arm_smmu_sva_bind(struct device *dev, struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + int ret; if (!master || !master->sva_enabled) return -ENODEV; - sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(master->smmu, + smmu_domain, mm); if (IS_ERR(sva_domain->smmu_mn)) { sva_domain->smmu_mn = NULL; return PTR_ERR(sva_domain->smmu_mn); } + master->nr_attached_sva_domains += 1; + smmu_domain = &sva_domain->smmu_mn->domain; + ret = arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, mm->pasid); + if (ret) { + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); + return ret; + } return 0; } @@ -544,8 +520,9 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - master->nr_attached_sva_domains -= 1; + arm_smmu_domain_remove_dev_pasid(dev, &sva_domain->smmu_mn->domain, id); arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); + master->nr_attached_sva_domains -= 1; mutex_unlock(&sva_lock); } From patchwork Tue Jun 6 12:07:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 41068C7EE24 for ; Tue, 6 Jun 2023 12:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=8LgdqqrVyVtPrOMlk/KXYvZVlFss9V3C4G8hobELxO8=; b=x2/BDz1mG4ohySg32NK5/3KxPj iBzvPRvDWgEVyRw7o0ljH9PgcWaas3D/7SwPbDDQHkIaGWwTotR/ttvKswdawszOKZb3hvy9Ta2B2 BjAC/sRzBWJv4I5BoE0vzIHXTRYP4hAhflvcGdrpRt5iCQyJwS4/PxcYG5G/6qmw3++dEG5mq2I4+ TTCYhJPpOc7c9rJRZWe2HvqVFR2063cuHrphETwgcosIptHMsclDsEGGVWpIwM9XpEIrjBO19r2TF 7W/8CShDribIooxn86QqZ4/Oi6i5phwl2BQpoLpwe5/3YKXnlMs9qNhMHMnqOP8ZRqwzVgeWYTqJF dzoHE/ww==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VX4-001ZwP-25; Tue, 06 Jun 2023 12:11:26 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VWy-001Zr3-2p for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:11:22 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-564fb1018bcso98283097b3.0 for ; Tue, 06 Jun 2023 05:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053479; x=1688645479; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=t4Hj/7++tZGwCPYZCwYpw7ebOJKZDYQm/8wAZtGxvBQ=; b=tvcKBuXmjcLSD6iUqEdjKkd3pjcW4Bgh2u7k7FhSOJ9MGZTQnIaSWKko/P99xTYVVN iQnaTLb8zj7e/VoY15bc6Mw1W9iPTt0hZMjbPFyPEYnptWJCScPiJPt4tWdXsyXHsO+P gp36FkV0kxNfBjzogTXR9DDW9oucqrdXHcWBZLfDLsvddtSL990dMhINvFRRLE7S0+MX ZynWgj6jxtuOzKbhLTqSqz3k6XpUEVduVIFfHenqn5whjfD/DaqykP3VEdW3yujAx8OO aoDD2uEAol7D8se7oYPQAf+WTlEmk3xSuhFYNAY0mS+9pV4pFZDuvyzqmGrDYSDktahz d0/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053479; x=1688645479; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=t4Hj/7++tZGwCPYZCwYpw7ebOJKZDYQm/8wAZtGxvBQ=; b=aIteysg82ojCJ980N3fXrGDlIVHRMGEBGgzwOynYpUSkTSSUFf6CfIqoWqIqrNcA8c nxvMSw8aizL0+NCmFtrVFnTS3T6zhUpUJDLNCfvPQmIvSqNNjBr13bKIdAjAVDQjdH/l nUBdQxI9PzbDksAuBgm+P4uHMOut1B3bQgFZPyPRKKgOlsp0BQ/YVvzossXTKK1ywspB QYwE7D/joiS/NArYZv72LqqRBCiGX1JWKyXl0udxAGT+bS1yYLFyPJbJWCErTNxpMnTz nlu7Iv4dxhQBdBSGMAchdejGymS1Vgasvc8UH4V1fla72d4NwPTLHwLO0HuOyXzuhRFM PnAw== X-Gm-Message-State: AC+VfDx6jyGJY2bP5+bkYF8yASBgzox0WIKbvys6Gu4hIzSHqh/SG8H+ +2orQ/SADrezX/dhKTbS1Ks6V5P6OuJb X-Google-Smtp-Source: ACHHUZ5K9xZ7uAxAugHtHkCI/mVJfPep/YguDaCB7GWP0kSNToN9Vq9i1EmKFOq+y4/UV6U9gPdF7J95aSTk X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a81:aa12:0:b0:55a:3133:86fa with SMTP id i18-20020a81aa12000000b0055a313386famr966118ywh.3.1686053479602; Tue, 06 Jun 2023 05:11:19 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:53 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-18-mshavit@google.com> Subject: [PATCH v2 17/18] iommu/arm-smmu-v3-sva: Alloc notifier for {smmu,mn} From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051120_914891_49073639 X-CRM114-Status: GOOD ( 16.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The arm_smmu_nofitier for an mn can be shared across all devices with the same upstream smmu. This breaks the last remaining explicit dependency on the device's primary domain in arm-smmu-v3-sva. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 +++++++----------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 ++-- 3 files changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 9a2da579c3563..3e49838e4f55c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -258,17 +258,16 @@ static const struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = { .free_notifier = arm_smmu_mmu_notifier_free, }; -/* Allocate or get existing MMU notifier for this {domain, mm} pair */ +/* Allocate or get existing MMU notifier for this {smmu, mm} pair */ static struct arm_smmu_mmu_notifier * arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, - struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; - list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { + list_for_each_entry(smmu_mn, &smmu->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { refcount_inc(&smmu_mn->refs); return smmu_mn; @@ -296,9 +295,8 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, } arm_smmu_init_shared_cd_domain(smmu, &smmu_mn->domain, cd); - list_add(&smmu_mn->list, &smmu_domain->mmu_notifiers); + list_add(&smmu_mn->list, &smmu->mmu_notifiers); return smmu_mn; - err_free_cd: arm_smmu_free_shared_cd(cd); return ERR_PTR(ret); @@ -314,7 +312,6 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) return; list_del(&smmu_mn->list); - /* * If we went through clear(), we've already invalidated, and no * new TLB entry can have been formed. @@ -331,18 +328,17 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) static int __arm_smmu_sva_bind(struct device *dev, struct arm_smmu_sva_domain *sva_domain, - struct mm_struct *mm) + struct mm_struct *mm, + ioasid_t id) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct iommu_domain *domain = iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_domain *smmu_domain; int ret; if (!master || !master->sva_enabled) return -ENODEV; sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(master->smmu, - smmu_domain, mm); if (IS_ERR(sva_domain->smmu_mn)) { sva_domain->smmu_mn = NULL; @@ -533,7 +529,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm, id); mutex_unlock(&sva_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3c5ff4f58934a..e68c5264c6171 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2058,7 +2058,6 @@ static void arm_smmu_init_smmu_domain(struct arm_smmu_domain *smmu_domain) mutex_init(&smmu_domain->init_mutex); INIT_LIST_HEAD(&smmu_domain->attached_domains); spin_lock_init(&smmu_domain->attached_domains_lock); - INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); } static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) @@ -2859,6 +2858,7 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) goto err_free_master; } + INIT_LIST_HEAD(&smmu->mmu_notifiers); return &smmu->iommu; err_free_master: diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 2c33c0461036d..041b0e532ac3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -674,6 +674,8 @@ struct arm_smmu_device { struct rb_root streams; struct mutex streams_mutex; + + struct list_head mmu_notifiers; }; struct arm_smmu_stream { @@ -737,8 +739,6 @@ struct arm_smmu_domain { struct list_head attached_domains; spinlock_t attached_domains_lock; - - struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Tue Jun 6 12:07:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13269066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7E59C7EE24 for ; Tue, 6 Jun 2023 12:11:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4u2GtBK4QdyaeAmk5SbKUddpjq/qAnOsKlpJt2WYNvQ=; b=4xqNxQUrk7KvbAM2+KkoBVh6MT CHaQk6teX5kQaSTO1j98WO6SuHHFkx/Rc6FHMFJ6qAxsovhjqR+v0gt87J7x5exyXDxpP9POs2XQl jTXV3eLI0XpHYLgouvwXIu3uKWmKs73fT224qcNudSOsyIlpKLw/O0P0bGKyGc308D7048vH4kfgp idTa2nxMUfEzn3k1kb/umeqkue3xo6kkUuZaWAVyTUfrGesxss7c8XuaJ+8a32TQYFPEKVz4ubkKQ btfy0rznuEBg+lCRMsRBiXVsPXsC3DNGWED3JLbkdm+wtPgqvPVOl3q2Nrv1Rj8WaTSRCBQSQDCi1 D/FGOfIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6VXB-001a1b-1I; Tue, 06 Jun 2023 12:11:33 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6VX5-001Zdu-2x for linux-arm-kernel@lists.infradead.org; Tue, 06 Jun 2023 12:11:31 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bb05f0e6ef9so8997104276.1 for ; Tue, 06 Jun 2023 05:11:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686053486; x=1688645486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=690IhMaVKQOheawjxO/ZDJYW5nna84UvkxSb29OMpJY=; b=eW2xKOUzDLrIwpVZxF6w1AffFH+eybFfzAor2EtdG3o/dhStpX5h+hko1PkiB6GL7y mmS6o7zPFxcUQRSRX5DG/d0PY60IviKLBvNBEtNsYPu/n5rjxNTwETco8yESLc1KVOOd wk+YW7cQw9e9JkvXKhVdJLpbLQn3z9yJ/KLwX3YpX8iS5O24ALewvrvR0jWyO9RzWlM8 oBsNOOWQlcWJkiplzzMhLA+1nJr7ffwmXaxT30h7zIprl7PUkqEBHdk4bIQdyCKLuL2P Z/jOGUaNT82GRH9Ra+nCHqqwczwcKPcwLQGL1DiptDxhlo6Fn6zbRcEFclgC4GlKyr69 EAqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686053486; x=1688645486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=690IhMaVKQOheawjxO/ZDJYW5nna84UvkxSb29OMpJY=; b=DdpOuVQoa5StuohQlGk5hIHtXSbYXWVcoHJghHlTAUZkj4Oco/mdMVnytlaAZfiYwk C0B+CYr4GPPFLWvOd05+Gf1hsGXOlWKPRmoqo1ndbzTQAwgcMM430sHZgRlfjdRBMqzN JUZurK02pljYtFJ9DfRbuVaH8nJNoj4h5/Jbra8Z9Acv/EXxAw9c+2HhYOkR4uo3ZNXK U6I8fWKf0nimO/Qxs1J7i4RngmJh9Vkc0i2S+JipFv0YtUKCE6HHh+aXT1BHdDtKJRUk JhhssbQjmRighv5x+VvuHjGFMpGwQ3YPW3HdON9TJCkEHtHHzXalz3bBgX9rOPBhn4ME Yi9A== X-Gm-Message-State: AC+VfDwc91bJ8ggvvEYaXkaL807DEETBZFo8XNDsrQWGH2KKaNwpak0p FpyESkF562XhMgq6RTsU4m/RzQokDHJr X-Google-Smtp-Source: ACHHUZ4AJPXtcMUNlSjrot0SIGIe1VeYT3TxZRBKLGtRsB13dquq7jBw5UDlIYSeITtYdyny8gcWeIoq4RF3 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a615:63d5:b54e:6919]) (user=mshavit job=sendgmr) by 2002:a05:6902:100b:b0:bad:600:1833 with SMTP id w11-20020a056902100b00b00bad06001833mr1035863ybt.0.1686053486705; Tue, 06 Jun 2023 05:11:26 -0700 (PDT) Date: Tue, 6 Jun 2023 20:07:54 +0800 In-Reply-To: <20230606120854.4170244-1-mshavit@google.com> Mime-Version: 1.0 References: <20230606120854.4170244-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230606120854.4170244-19-mshavit@google.com> Subject: [PATCH v2 18/18] iommu/arm-smmu-v3-sva: Remove atc_inv_domain_ssid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_051127_971730_5AA2091C X-CRM114-Status: GOOD ( 17.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org arm_smmu_atc_inv_domain is sufficient in all cases now that arm_smmu_domain always tracks all the master/ssids that it is attached to. Also remove the last usage of mm->pasid in arm-smmu-v3-sva. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 9 ++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 21 +++---------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 ++-- 3 files changed, 9 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 3e49838e4f55c..5a124281bbef6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -211,7 +211,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain(smmu_domain, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -241,7 +241,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -304,7 +304,6 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_device *smmu, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { - struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; struct arm_smmu_domain *smmu_domain = &smmu_mn->domain; @@ -318,7 +317,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } /* Frees smmu_mn */ @@ -347,7 +346,7 @@ static int __arm_smmu_sva_bind(struct device *dev, master->nr_attached_sva_domains += 1; smmu_domain = &sva_domain->smmu_mn->domain; - ret = arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, mm->pasid); + ret = arm_smmu_domain_set_dev_pasid(dev, master, smmu_domain, id); if (ret) { arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e68c5264c6171..1e02e73e586f7 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1821,13 +1821,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_atc_inv_master_ssid(master, 0); } -/* - * If ssid is non-zero, issue atc invalidations with the given ssid instead of - * the one the domain is attached to. This is used by SVA since it's pasid - * attachments aren't recorded in smmu_domain yet. - */ -int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) { int i; unsigned long flags; @@ -1866,11 +1861,7 @@ int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, master = attached_domain->master; if (!master->ats_enabled) continue; - if (ssid != 0) - arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); - else - arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); - + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); @@ -1881,12 +1872,6 @@ int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, - unsigned long iova, size_t size) -{ - return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); -} - /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 041b0e532ac3d..9c382bc8c0549 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -759,8 +759,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size); int arm_smmu_domain_set_dev_pasid(struct device *dev, struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain,