From patchwork Wed Aug 2 16:32:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21538C04E69 for ; Wed, 2 Aug 2023 16:34:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=495fheEkbpMkHyhnFZH2azNArVsunYRBclNyLAbM9Lk=; b=ojTUKNQF9MfHztB/o6I20qfrjz 3m70ekIb9BNs/K810eLAJ1vA/hdi6Yz1SAB4ld3SgzlAWjPsqey03yxn7BVafVhzoIUXAcypxnPc4 EBV96LgPBvNVK9v6+pwLvBo9BAAIKKFlBxU/kQqj/ww0F46y7FIf+xSFz5ANtL9x0g8S05qZYK3gB 8m/6Nfm0ekhYJYz7+mCko86M6YXuXx2wNRy/LMOMJoYEwiWf8GtOMBu6208luA8///e2fu5EZRUCR RlF7thU6LKhl7pWkaiD8u0UzGDvH2O9nF2c2BwwS4BG3Dxf8SrI84SY7hQl/SAsGsLSoGbl6NgiBv 5ADG2bvQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnA-005MJf-1d; Wed, 02 Aug 2023 16:33:44 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREn7-005MIO-10 for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:33:42 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d087ffcc43cso7095142276.3 for ; Wed, 02 Aug 2023 09:33:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994020; x=1691598820; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FGhZFYnHXRSeNZzMPSJMjfhqZmzQC/L7Opahs9wW1cY=; b=qbAiZYTHU/BZPRpEk6nJ6g5qVVdJKLX6nPtkdkC4XfeuGUhWBI+dO+3QEtHojy3qGy Wu8mwDFpDzE5m7m+mWKIHRZPJKiQg4mqSSKngYIUZ/eq2nmhoLhvAQ7zLmSBWN50L0x+ 0zBP5MI9g8JmUVCuQiHLNquf61mUnORjSkDTx5dKApzwBTIFvadWhcBp5Ymv7EvdkQ4a 9D5JVsbAFcttUaOxTbShccfSulmLAHikd2SSBJilr2ot+xuJ3dABG/Yy0FgRgEOGdm/s 8I8w1lcAMk8EVlPEuxqnDCrydmzwy1LP0ZjDC5UlTAT30AJrx27Ys1EfXgS0uIMbWCN3 UqFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994020; x=1691598820; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FGhZFYnHXRSeNZzMPSJMjfhqZmzQC/L7Opahs9wW1cY=; b=Yb1wWAa7wc+qB8sztxWL7Zdx6+ZoVJ3JJdwy0b42Dk5xjd3YkNE7obTvDqXpZLHOql TlZQzWuEO93CtWLEEqJ4CKvBBkO5pwiqT1yvYYO3aEt9khkGoQMscR3eRW9BavM+oMe7 LFhAskm0iwInrxkdDnOJc4j1j351wo+SovBHvPQHPZsb1yLwOH9GHE+9W6X1iks9rmmO Mmwm3/1vrH+ahkvDDg+r1QRP9H5XFM6O3rzuc24gmDSo1Jzm67vxXz27rNYRxwTJiZRL sFVj+gRDJw6xqcaSeK6d3PTdGLDUJnwN4iaLvJmVEtICzz3SfA+/9FpF83IQTSMlKsNv ds0A== X-Gm-Message-State: ABy/qLZyLBlgatS+Tu1TcFnjw0Iuu7p+V1oLXabZaIboEEsu73xaPcDe Wz0Fk2w6uKXz5vJRLcze5B6Yhmb93qMp X-Google-Smtp-Source: APBJJlHWFUWWHsYw58rm6CBPt9s/MG2WLzrm5C/8K8y7EA3IHdFMgUVM/XfPKs+0tYYNzqnfBnpDLuFWFS7q X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:ac43:0:b0:d0a:5674:c4f6 with SMTP id r3-20020a25ac43000000b00d0a5674c4f6mr115325ybd.3.1690994019921; Wed, 02 Aug 2023 09:33:39 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:29 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.1.I67ab103c18d882aedc8a08985af1fba70bca084e@changeid> Subject: [PATCH v4 1/8] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093341_346240_07A49B73 X-CRM114-Status: GOOD ( 18.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v2) Changes in v2: - Undo over-reaching column alignment change .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 ++++++++++--------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +++-- 3 files changed, 17 insertions(+), 14 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947e..968559d625c4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } - smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain = container_of(cd, struct arm_smmu_domain, cd); smmu = smmu_domain->smmu; ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 9b0dc3505601..bb277ff86f65 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1869,7 +1869,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1957,7 +1957,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid = smmu_domain->cd.asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2088,7 +2088,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2107,13 +2107,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2126,23 +2127,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_free_asid; - cfg->cd.asid = (u16)asid; - cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid = (u16)asid; + cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; @@ -2152,7 +2153,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dcab85698a4e..f841383a55a3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -599,7 +599,6 @@ struct arm_smmu_ctx_desc_cfg { struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -724,7 +723,10 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Wed Aug 2 16:32:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3EDDEC04E69 for ; Wed, 2 Aug 2023 16:34:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PbjVbmOPLRYktG5o/Jpz4QIL5rsgegIUxl4/eEqBOTY=; b=LMFjGJKCi4n3eiQRwn9hU1gant XfvynWDM98gaRoekQ0hY87dZD5Frx8+11nzMxmFdlKwOrK3gA1eJCDesz6jI9Lk9BOEc9Kvs/yjdh jRCCJS7oveSU3MtiC+McJ9L/dBV2kJ7jE8Ec696VxcE9Px3tbO6Ci9tphkENX1ApwUrFUUHM3B2xP SF1jI4QGrrsfLEoM4z2kA7M27uJgvDB2u9TTC6nu7ClFG6JTgGw3E715JPQg2yYllUgWfNsl8viMb KalyAUR8CUYMJ8v0k9yYjWwYlolHq/TioVGMuDVMovbbkQJuqgQrvPxi1NbzeK/DiTSvdsOQuSGud 7S1GLfmA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnH-005MN9-2P; Wed, 02 Aug 2023 16:33:51 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnE-005MJx-2H for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:33:50 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-583312344e7so84575577b3.1 for ; Wed, 02 Aug 2023 09:33:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994025; x=1691598825; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=r3+yIt5RymsTwdmi2GYawedbQ7ADkg2oqEcGa/FOhO4=; b=7NfRVq4Idu++ILngYRgZBFEO+HTiQCZCmbx/2UL+ljXsZfjQhl48ik+oDiGj4WaNye JZql0cWzJrMvjTCXbJwCNsZTtIgCGFEftqjWcvDsRr4gLQ39lw99YUhKyUPrm+IR/TmH pSqfU9ZWM0AmwTGSyEITP7sup0vFzfF3xKV2tRk58CVllYuCybMJgQGpStU9eMXwg/L1 SGh76hpI7+bkwUe4/3YCAptaeSZj2OsihEshnMZ4l+QyE+X1YxI5fdKN3gH8ftTBgDXi CRDzjEW4GeNkIAQ5usnpQ3S85FxPrriZ+Pgftpyg4Bpr0OF1LAt32aJISvVspD1VNteW bxZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994025; x=1691598825; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=r3+yIt5RymsTwdmi2GYawedbQ7ADkg2oqEcGa/FOhO4=; b=a6Rx/FEbDopoufPE/Zr/28Hvyp/WCS5s+5mSgxN7AiDt4ijd2lKEw6qUhGDZT8ISAf 1s65fkomTFQupGuNakMVCjrwnkg7JDXFrBvLWHCIlgtKeBh4CB9MIdAwkEx8s1g4uISt u+sUYz7VM8+F18b+4nsuLKjXyp9v7aq9K+ElqpXBaqqJExU0oiFh9i/jzXGSJI45ZbzD 9YH6aFgtTPKpmwkG64onkcMKWDNN9MZyrfUTzXrpv/OylK7IOKDPSA2oqfuAY54vjY6C SMgFlnewnXKJu/60K79OknA2Y6iYaewgVts6B+oGygUodAowWjszEut2oyYjWPh0WUgh RFCw== X-Gm-Message-State: ABy/qLbEpWkg7z6LYN8eseYABFhVbI8DYabkLBwfuJ5OXo47vR/MSdo6 +xt83JcfreUEXe5xr/eWcdP9vSquzHw+ X-Google-Smtp-Source: APBJJlFC6Oy0FtutmhAiFR3UtXa0rRExc7WVClsorb4LF6gLLLjkdXyFQjf0UhnZSSlw3hxYzj1CbJFPFnDC X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:ad27:0:b0:581:3899:91bc with SMTP id l39-20020a81ad27000000b00581389991bcmr152757ywh.6.1690994024824; Wed, 02 Aug 2023 09:33:44 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:30 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.2.I1ef1ed19d7786c8176a0d05820c869e650c8d68f@changeid> Subject: [PATCH v4 2/8] iommu/arm-smmu-v3: Replace s1_cfg with cdtab_cfg From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093348_743730_3572E0F0 X-CRM114-Status: GOOD ( 22.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove struct arm_smmu_s1_cfg. This is really just a CD table with a bit of extra information. Enhance the existing CD table structure, struct arm_smmu_ctx_desc_cfg, with max_cds_bits and replace all usages of arm_smmu_s1_cfg with arm_smmu_ctx_desc_cfg. Compute the other values that were stored in s1cfg directly from existing arm_smmu_ctx_desc_cfg. For clarity, use the name "cd_table" for the variables pointing to arm_smmu_ctx_desc_cfg in the new code instead of cdcfg. A later patch will make this fully consistent. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v3) Changes in v3: - Updated commit messages again - Replace more usages of cdcfg with cdtable (lines that were already touched by this commit anyways). Changes in v2: - Updated commit message drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 45 +++++++++++---------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 10 ++--- 2 files changed, 26 insertions(+), 29 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index bb277ff86f65..ded613aedbb0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1033,9 +1033,9 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; - if (smmu_domain->s1_cfg.s1fmt == STRTAB_STE_0_S1FMT_LINEAR) + if (!cdcfg->l1_desc) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; @@ -1071,7 +1071,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; - if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >= (1 << smmu_domain->cd_table.max_cds_bits))) return -E2BIG; cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); @@ -1138,19 +1138,16 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) size_t l1size; size_t max_contexts; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; - max_contexts = 1 << cfg->s1cdmax; + max_contexts = 1 << cdcfg->max_cds_bits; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || max_contexts <= CTXDESC_L2_ENTRIES) { - cfg->s1fmt = STRTAB_STE_0_S1FMT_LINEAR; cdcfg->num_l1_ents = max_contexts; l1size = max_contexts * (CTXDESC_CD_DWORDS << 3); } else { - cfg->s1fmt = STRTAB_STE_0_S1FMT_64K_L2; cdcfg->num_l1_ents = DIV_ROUND_UP(max_contexts, CTXDESC_L2_ENTRIES); @@ -1186,7 +1183,7 @@ static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) int i; size_t size, l1size; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -1276,7 +1273,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, u64 val = le64_to_cpu(dst[0]); bool ste_live = false; struct arm_smmu_device *smmu = NULL; - struct arm_smmu_s1_cfg *s1_cfg = NULL; + struct arm_smmu_ctx_desc_cfg *cd_table = NULL; struct arm_smmu_s2_cfg *s2_cfg = NULL; struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { @@ -1294,7 +1291,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, if (smmu_domain) { switch (smmu_domain->stage) { case ARM_SMMU_DOMAIN_S1: - s1_cfg = &smmu_domain->s1_cfg; + cd_table = &smmu_domain->cd_table; break; case ARM_SMMU_DOMAIN_S2: case ARM_SMMU_DOMAIN_NESTED: @@ -1325,7 +1322,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { + if (!smmu_domain || !(cd_table || s2_cfg)) { if (!smmu_domain && disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else @@ -1344,7 +1341,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, return; } - if (s1_cfg) { + if (cd_table) { u64 strw = smmu->features & ARM_SMMU_FEAT_E2H ? STRTAB_STE_1_STRW_EL2 : STRTAB_STE_1_STRW_NSEL1; @@ -1360,10 +1357,14 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, !master->stall_enabled) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (s1_cfg->cdcfg.cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) | - FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) | - FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) | - FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt); + val |= (cd_table->cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) | + FIELD_PREP(STRTAB_STE_0_S1CDMAX, + cd_table->max_cds_bits) | + FIELD_PREP(STRTAB_STE_0_S1FMT, + cd_table->l1_desc ? + STRTAB_STE_0_S1FMT_64K_L2 : + STRTAB_STE_0_S1FMT_LINEAR); } if (s2_cfg) { @@ -2082,11 +2083,11 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc_cfg *cd_table = &smmu_domain->cd_table; /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) + if (cd_table->cdtab) arm_smmu_free_cd_tables(smmu_domain); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); @@ -2106,7 +2107,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc_cfg *cd_table = &smmu_domain->cd_table; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2119,7 +2120,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cfg->s1cdmax = master->ssid_bits; + cd_table->max_cds_bits = master->ssid_bits; smmu_domain->stall_enabled = master->stall_enabled; @@ -2457,7 +2458,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { + master->ssid_bits != smmu_domain->cd_table.max_cds_bits) { ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index f841383a55a3..35a93e885887 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,12 +595,8 @@ struct arm_smmu_ctx_desc_cfg { dma_addr_t cdtab_dma; struct arm_smmu_l1_ctx_desc *l1_desc; unsigned int num_l1_ents; -}; - -struct arm_smmu_s1_cfg { - struct arm_smmu_ctx_desc_cfg cdcfg; - u8 s1fmt; - u8 s1cdmax; + /* log2 of the maximum number of CDs supported by this table */ + u8 max_cds_bits; }; struct arm_smmu_s2_cfg { @@ -725,7 +721,7 @@ struct arm_smmu_domain { union { struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; + struct arm_smmu_ctx_desc_cfg cd_table; }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Wed Aug 2 16:32:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C4C13C04A6A for ; Wed, 2 Aug 2023 16:34:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=KuKpsTnk4OTG9juzpZbXPZ1vZ7BD3teCfRbhbg7r45M=; b=l/bAA/1mBGWG3++CUOULHg8kmS ReyIViHQkAARVpiVWQl7O64NLWosdS3VlDBqyyshIKUahHHfUyclpsjgnn4EZAbLwzZSfY6KKt/3T hiNTevzq6v/rMsGQFoCvF8v0QfOwjFF1DEK3UdkOw4pA0FGUEJJhUGv2fqtFFWSYP2qwGpz+WrW/B WbFW9YxP6SdUHRAgup+EJ/qzCXF795PDo5czLWHpnty+MIJXCaMXCOkvpJ8YFV7GmbrgyLQUnXL/K iyj4jS3HC8ANhTDTkS8PosX9DbA24y/46J3C+OCvGjIhd9CzogGVSj0KhXfM+pfIiisbJ65a+xrA0 sW2eqrWw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnJ-005MOf-32; Wed, 02 Aug 2023 16:33:53 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnG-005MLC-0T for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:33:51 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5840614b13cso981477b3.0 for ; Wed, 02 Aug 2023 09:33:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994029; x=1691598829; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=bGhWSbzc4cZej0FeJJbc3VXFTtBzmMATq60yYbQRCzk=; b=Wfmjss+UWmZsCytY4z2zpjHfRx8D5zoF9UEpfx4Rmioi27engfbfmtGjsJKQWBcyVf QS5jrHEj985TIIPJU2Mfx45hFU3uZrL0e52dgYK7tVtA9xaCfnfD2ihgwvbDsnbWO4Ag So0UGVMoWSVnDk+FESZFRihVwOytovkaGPzW4T8N0gSCoM2jXawxIf3LpwlN5Khp+pah yjcreTw1ONoQ8cMSKG2QPznIeGjCQIWll17RNVuYyACk1GcDPSczUDF/RDsO9l6A46K0 0Lcty3jmaLU2xZl9Y76nI9/56Xbhl0emopedb/DqA/C06TZARGBtPULdRn5u2XTvcmw/ kOJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994029; x=1691598829; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bGhWSbzc4cZej0FeJJbc3VXFTtBzmMATq60yYbQRCzk=; b=jCLa2lO5pM9bnUs6maYN6RwzStDBCGw1lxwLCMJT3swVOUG3nrV6OR80MvseEyR8k+ y/gm6ekUOG1GsYiMfaiso+jHLmE/ExWPI0o0+AocEz6qxWBEffuq+cnpZy10IfbYFvCb 5bJfhcaLEYagBP5F6XKnb4so6QAYkB0V6MW88tDTMrZ/Xmj7g77WLBmgbgTeyqTZSypz F7/rVxBXNYJhegzOiiOex3hy6Miz04sOF9Pmt/1QfCK4jzr9Mo7aeyuBmB7HLvNqCPU7 hl0xzyHl0Z3LCXdlvbaVw8gFUH2rRMGS22RGj5Vy9ZkZQL9dIozL+WiHy3pyZduQIYPr ImwA== X-Gm-Message-State: ABy/qLbdn+SNhcqgRE506dUAMkN7i4oTqEUu9Zp9kBriGX7QA7Q5XoG/ QQdH8Vn2tfLBFZxEsAe1un/lmhx4amr9 X-Google-Smtp-Source: APBJJlFrdCMuLTcNvB4PEmrjrdw1E2K0mx6VOTk63TBd4jAAKLgWBwBxXteZbNG1vw2PQrNREyOTvEazvG+L X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:ad5b:0:b0:56c:e9fe:3cb4 with SMTP id l27-20020a81ad5b000000b0056ce9fe3cb4mr227897ywk.1.1690994028873; Wed, 02 Aug 2023 09:33:48 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:31 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.3.I875254464d044a8ce8b3a2ad6beb655a4a006456@changeid> Subject: [PATCH v4 3/8] iommu/arm-smmu-v3: Encapsulate ctx_desc_cfg init in alloc_cd_tables From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093350_186232_C0774FFA X-CRM114-Status: GOOD ( 12.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This is slighlty cleaner: arm_smmu_ctx_desc_cfg is initialized in a single function instead of having pieces set ahead-of time by its caller. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v1) drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index ded613aedbb0..fe4b19c3b8de 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1132,7 +1132,8 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) { int ret; size_t l1size; @@ -1140,6 +1141,7 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + cdcfg->max_cds_bits = master->ssid_bits; max_contexts = 1 << cdcfg->max_cds_bits; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2107,7 +2109,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cd_table = &smmu_domain->cd_table; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2120,11 +2121,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cd_table->max_cds_bits = master->ssid_bits; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain); + ret = arm_smmu_alloc_cd_tables(smmu_domain, master); if (ret) goto out_free_asid; From patchwork Wed Aug 2 16:32:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61869C04FDF for ; Wed, 2 Aug 2023 16:34:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=50SQViGLMWHpSQjjaOpf7Q6p6xBcCidulgyl9xqq7BU=; b=QcLagFan/rB7E57N66RmofaZ3K 78c0t3K71xMRRRKga/cOVR7YkpOVsozJRWXbbihKm8pHbpFySxyRdLrShEblT+QHGEqNVxXwMrAF2 8zOC/3jq39+2M6n1qsDK9DeDU5bUXepPzkhVTQeBp9TGP8CZgbtxYRQZo2DqcS6O8DFbdTtA0qIm+ 4IzRC8Magc+dwmyxctAw+u34qDk/8QQAFDK208X+KIq5VDzPjpwe4YIiAbC0LAS733But/E3L6+3o pW0SZ/1Q78kULS4aoceVeRjIzAjTMUAffiPq+jVoSwrLuVCNtkfgBHr54yxFQwRW/lKRzEqFk/lLH 06f2xb7A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnN-005MQ7-1c; Wed, 02 Aug 2023 16:33:57 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnK-005MOc-1U for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:33:55 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5867fe87d16so5511817b3.2 for ; Wed, 02 Aug 2023 09:33:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994033; x=1691598833; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JkSkXQKYJ56ysrmlEDQExeOLrrR+KHQKd0NE5Ad+0tQ=; b=Sx4NgzTXcAqonwe3umZclUZGuCSTNOJdjHZfhQsHFzrrGgvIBu+PVtD/2bbThkc2kg O/sl/fsv5azhqF6rFWZaIoCHnP/dsGxMk80XJekswYjv3IYw+0unms2NiOxnWe9mtMed d+vYRYCK+LmW/8uHomdrDGzkjSrKkMZ98LCels1IDh735su3QiiRYMMiZPsdWP6xMEMn WhaPiXb3DC8n86HOkDb1expULa/KNgZQMmuevHGecWWfRrBbMHtE8vm19ZWUoU8x2a/c IlcM9zeUNhiB99UW7QV8MuCdv5p4/3k5j4PcrytWgqaoLwkYtmZLqLUYQ0ylGG3xlVub +rbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994033; x=1691598833; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JkSkXQKYJ56ysrmlEDQExeOLrrR+KHQKd0NE5Ad+0tQ=; b=lR3zuCejI/2ZFIfXuyliXFDtpz9dbEAo6CIO2CrBYa9LekH4o5GsTAkHCoYDVI9gX8 XeUj6dpCVeDnkEP5yvvSMQQ8d9GwXeTaggkEOi32TJaHhsPXL5vnaagjmK8t9tOZbtkW j1G4Cwc/rLUjwjkWOOxGtAH2yj8OxiWTpOLhtgVP9agc1QhTNF8IedS5fRXT2X4W3mav B5T00dNwWKC3yQtCyulhD2buanxtET6hExmCGaY2PXP+TvY1v0TNuumsNaEre9BXX7d9 PmEEtj9qU251MXsY1fTjhJbcZBI4nlMc4Afs69LZsPjOBeOC1Fht2B6Q8NuYorSIQUCK V2HQ== X-Gm-Message-State: ABy/qLaxbzo15Shxnd4Sn92dpCeXxAOTb1lXoCkmq/4ajzROI+HPrlk+ sKjb6nfXvCVt8KHFsLwr+sTnaZEimLaj X-Google-Smtp-Source: APBJJlEJkxL7n5t/WP1xXbG+m0y+BaZM5+ewq4E8ARRYBnMdtfnw5rA+asX1x1w8C+XChKwEP8BbnPzCQZIC X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:48a:0:b0:d37:353:b7eb with SMTP id 132-20020a25048a000000b00d370353b7ebmr56014ybe.11.1690994033344; Wed, 02 Aug 2023 09:33:53 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:32 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.4.I5aa89c849228794a64146cfe86df21fb71629384@changeid> Subject: [PATCH v4 4/8] iommu/arm-smmu-v3: move stall_enabled to the cd table From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093354_496178_C3390512 X-CRM114-Status: GOOD ( 18.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This controls whether CD entries will have the stall bit set when writing entries into the table. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v2) Changes in v2: - Use a bitfield instead of a bool for stall_enabled drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 ++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 ++- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index fe4b19c3b8de..c01023404c26 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1114,7 +1114,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->stall_enabled) + if (smmu_domain->cd_table.stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1141,6 +1141,7 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + cdcfg->stall_enabled = master->stall_enabled; cdcfg->max_cds_bits = master->ssid_bits; max_contexts = 1 << cdcfg->max_cds_bits; @@ -2121,8 +2122,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain, master); if (ret) goto out_free_asid; @@ -2461,7 +2460,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled != master->stall_enabled) { + smmu_domain->cd_table.stall_enabled != + master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 35a93e885887..05b1f0ee6080 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -597,6 +597,8 @@ struct arm_smmu_ctx_desc_cfg { unsigned int num_l1_ents; /* log2 of the maximum number of CDs supported by this table */ u8 max_cds_bits; + /* Whether CD entries in this table have the stall bit set. */ + u8 stall_enabled:1; }; struct arm_smmu_s2_cfg { @@ -714,7 +716,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; From patchwork Wed Aug 2 16:32:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CE503C001E0 for ; Wed, 2 Aug 2023 16:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Lr+y06L/WXPLv/M46sDsfzqYI5buiA4Op69V06QdEH8=; b=0JqdQEKkaf+CXyA/jsyS5wr2ve 0bc7hDA+OjQhSnP07TYTIN6Hs9iaBlTKFuytiwDUC+eDUV61jGc9bvnTzjdZFm45w3X/1vFCO0bI3 TiaFTL5RYmzjw6DLjS4KeK1wqOQ2iVXWmzepOC5okheKgy/EJI1rGWg+WpFXb8W8dh1xFiZrqTIz/ lD6fo3bthBkFfBbXf5MG7ID9tVooEKqyhVGfn/Y84JZBNDuYhgw0Ir8ghUQf+T4KiiHay4S5EOZ6B ickwZfAFSvSPJWxOEe2WgeSghoBB9bQwgyZmA6sNaGNJNEavzdD8JSsl0WfS+LHqSzAOy9WUqdcc2 JyxqudnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnS-005MSa-0Q; Wed, 02 Aug 2023 16:34:02 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnP-005MQs-0w for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:34:01 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5840614b107so84488617b3.1 for ; Wed, 02 Aug 2023 09:33:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994038; x=1691598838; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Nsrqxec2unoGbX9KQJr31nnyfTckBujoXSXq19+cQZI=; b=taFPg4CtXS6wQkzNpsgzmsouWdIXqXLq8mdwTKz4sWNpCRb6E/eVmyTize8AOaZZ7p 8X9ZZYRU4ucutlE6Zw/olc2LiTrW9LagJHGu6A/COc0FCt3EtaddlnVFqWjXyQgvhBgQ EfrQQuumwOHTcmfRWI09mpGOXhxITl8DAVelGC2Ujx0K7Zh8SnwXMdIiK7aaA/XeVK3q slTafS5X3XUzTA0sgoMqjydf7aBreEe0/eGCuyLdCbfSlyvBF4HLkdpG1JoPG/KX+kzr bmvj3MgOAo8X4BK9kNRrwf1U7KJcx0oCrTBtkhCH1/Hhg2eu8A1/LXZXLfCp6h5vpYi8 PBEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994038; x=1691598838; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Nsrqxec2unoGbX9KQJr31nnyfTckBujoXSXq19+cQZI=; b=Q9epKKjxbCSr3jCRAu7tHcIJqux26n/n1P3ObFUlQ1lZAPXHu6Htvhqm4njeIg9rqg RzQx+ratQ5upzDu6bAFQyrz6357NKt4zBjq8prfK/NAIiT9GW3I+ZXr1Of1evBvGBLPz XSsxUGsglCCgwsX5WR1zDiJiuKm3G7fB5fH253JWupE6adXIHrLcZgiJWPVC5yZkdO8X HoVt3ouRCot6h3Pz6/5jVMRtRptH+Z7a6Wc948wRHkTyw/M87Bpxiewceh78xG6p9ZsM 6VkeUAMzbPWyEjx9PJniZANDXy7vYzti9kGpiRwHR5uhnuFW9wwCMBc+eBU/lujE19WE qJ5w== X-Gm-Message-State: ABy/qLbSnB2tLs/+Rf3fjvSRmKy80clSg4Yy8KZB603ieMklexnDBuce 85wvZQTigN9WITOMtxlLGYP83VjWcym/ X-Google-Smtp-Source: APBJJlHB2nMjYwaaODRCjwzrktJ+gOtl/VadKPEXlLhCk+WCKpltfQG1DgzRA8oBRZvT6az0EVvebvxSrSJc X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:ce47:0:b0:d20:7752:e384 with SMTP id x68-20020a25ce47000000b00d207752e384mr116092ybe.3.1690994038147; Wed, 02 Aug 2023 09:33:58 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:33 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.5.I219054a6cf538df5bb22f4ada2d9933155d6058c@changeid> Subject: [PATCH v4 5/8] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093359_339757_C97C3EA0 X-CRM114-Status: GOOD ( 29.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Update arm_smmu_write_ctx_desc and downstream functions to operate on a master instead of an smmu domain. We expect arm_smmu_write_ctx_desc() to only be called to write a CD entry into a CD table owned by the master. Under the hood, arm_smmu_write_ctx_desc still fetches the CD table from the domain that is attached to the master, but a subsequent commit will move that table's ownership to the master. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. This loop may look weird but becomes necessary when the CD table becomes per-master in a subsequent commit. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v3) Changes in v3: - Add a helper to write a CD to all masters that a domain is attached to. - Fixed an issue where an arm_smmu_write_ctx_desc error return wasn't correctly handled by its caller. Changes in v2: - minor style fixes Changes in v1: - arm_smmu_write_ctx_desc now get's the CD table to write to from the master parameter instead of a distinct parameter. This works well because the CD table being written to should always be owned by the master by the end of this series. This version no longer allows master to be NULL. .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 31 +++++++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 51 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 46 insertions(+), 38 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c4..e3992a0c1637 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -37,6 +37,24 @@ struct arm_smmu_bond { static DEFINE_MUTEX(sva_lock); +static int arm_smmu_write_ctx_desc_devices(struct arm_smmu_domain *smmu_domain, + int ssid, + struct arm_smmu_ctx_desc *cd) +{ + struct arm_smmu_master *master; + unsigned long flags; + int ret; + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret = arm_smmu_write_ctx_desc(master, ssid, cd); + if (ret) + break; + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + return ret; +} + /* * Check if the CPU ASID is available on the SMMU side. If a private context * descriptor is using it, try to replace it. @@ -80,7 +98,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + arm_smmu_write_ctx_desc_devices(smmu_domain, 0, cd); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -222,7 +240,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + arm_smmu_write_ctx_desc_devices(smmu_domain, mm->pasid, &quiet_cd); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -279,9 +297,11 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - ret = arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); - if (ret) + ret = arm_smmu_write_ctx_desc_devices(smmu_domain, mm->pasid, cd); + if (ret) { + arm_smmu_write_ctx_desc_devices(smmu_domain, mm->pasid, NULL); goto err_put_notifier; + } list_add(&smmu_mn->list, &smmu_domain->mmu_notifiers); return smmu_mn; @@ -304,7 +324,8 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) return; list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + arm_smmu_write_ctx_desc_devices(smmu_domain, mm->pasid, NULL); /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index c01023404c26..34bd7815aeb8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -971,14 +971,12 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, .cfgi = { @@ -988,15 +986,10 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, }; cmds.num = 0; - - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_streams; i++) { - cmd.cfgi.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1026,14 +1019,13 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, - u32 ssid) +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->domain->cd_table; if (!cdcfg->l1_desc) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; @@ -1047,13 +1039,13 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx = ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1070,11 +1062,12 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, u64 val; bool cd_live; __le64 *cdptr; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->domain->cd_table; - if (WARN_ON(ssid >= (1 << smmu_domain->cd_table.max_cds_bits))) + if (WARN_ON(ssid >= (1 << cd_table->max_cds_bits))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr = arm_smmu_get_cd_ptr(master, ssid); if (!cdptr) return -ENOMEM; @@ -1102,7 +1095,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * order. Ensure that it observes valid values before reading * V=1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); val = cd->tcr | #ifdef __BIG_ENDIAN @@ -1114,7 +1107,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->cd_table.stall_enabled) + if (cd_table->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1128,7 +1121,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } @@ -1138,7 +1131,7 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; cdcfg->stall_enabled = master->stall_enabled; @@ -2137,12 +2130,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret = arm_smmu_write_ctx_desc(master, 0, cd); if (ret) goto out_free_cd_tables; @@ -2460,8 +2448,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->cd_table.stall_enabled != - master->stall_enabled) { + smmu_domain->cd_table.stall_enabled != master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 05b1f0ee6080..6066a09c0199 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -744,7 +744,7 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, From patchwork Wed Aug 2 16:32:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1EA8C001E0 for ; Wed, 2 Aug 2023 16:34:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=keprB3FgfpGda8V7VCxhMR9mQKRFvxG80MOwySXbaIw=; b=qpwCp+N3nW5XOikC6NnFbOXiMX /oyZMbElK4V/q6WA6lprapgfzv14U3QCU7A79j97BKw3nR1v7rGBj1rFKX3i1BLGOxHA424pWWRv4 bxGaaVOH6UJMoX3RKvlEawq/CDE83gVHZ4HfFUeO5KYYZQU6KmmXBf1QV3XxhlCnNmyiJUKLqT2Tz +hemSdBfvlqGe+8Q9rHCJ7AlZmdoLzn57TvpSsrSc8puRsbIy/AEIH5PJpEhyKZsg1MSxwgo+hNeY 5RpTE7Jq7w61soBd2xeV0CbM0HUfgg2SWleVg5x9WZFlz2dkWVuxThaiKiu4boXOU2l/WqPo/xmZU XuZaX7bQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnX-005MVY-1N; Wed, 02 Aug 2023 16:34:07 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnU-005MTE-0H for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:34:05 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d2082104a44so6405665276.2 for ; Wed, 02 Aug 2023 09:34:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994043; x=1691598843; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CzAFxyJDXJsx0hZ+hbsCVkJECtcH8fWndodY3u5zpmw=; b=cEDNdOI8JNpc2QWjId+mul5PAIF12jDtATLJGhmIRCTsPeUo6cGzEcWxs46simEVZ2 HK7L8OdNkwNNAh9A/cMVFKC5hH+qISXl5IWtI5ueMmqEP+1Zi9tR2zeF8I/CEoxcsX1L 3v1z8gGpNg9SUYBA7DNdYXx4v7DHN5lh4v7e60ZaGaVCpm2/Pta6lB6QlYmzFnzkpMFF 26oNUBPAI5nbvLD9kfWhKyX6SsUHv3sbf2a4MjbTlV3F+CdUWI/waDwa3bnTkU4qSbnC 6yvAv7E76YYabnZSzVEBTQHSdN7tmcu2ofH4fFb9fk/7gHn0Kj41CpycgMjrYrZvo4Bw V1ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994043; x=1691598843; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CzAFxyJDXJsx0hZ+hbsCVkJECtcH8fWndodY3u5zpmw=; b=K44JzaUG0e/oWEY8W2+Y/lq0e9wgX6U2E7lFbmojQd4223/VoW/8vFZwTjfgKzGs3u qVczR6w0r7mK+5gIH2mAAiqvhAPtu+iYJimY7URJDVczRZVWG+F4lzjiTzAAsTRuSZxs a9jwEmCP79qhee/BNuLmxQIzmyZATT5HSc+hgyl5XUfByagD77zchLajWVV9fvKgIWBj mXja60XKCFX1cEQw4zTFz+jQRIS9ewNfLn8K2nR4By/riUbghC4p3F25Xr6OunyCK5bq NU0UFPU5mxNACqhwN/qKlZ+Zq1rRYLY1DhZKDWi3Xi9p2IWktvMnVPaI4Fc5pQI+5Ouc CYzQ== X-Gm-Message-State: ABy/qLayYNDa8+juJcAqkiInnExGwtzcTYmESrzoQOlPIBS+3IPseQ7O Ui3uOy/7IttCr1sYnsLI7mpaBLlWGEzE X-Google-Smtp-Source: APBJJlHSZVoJ0Zt/UGM3qKViSNreWEl+6b6i/oXi858Hw8Zgz3igrPhVPKME4uqykzkxufvxGPYDY/gFZ82o X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a25:2311:0:b0:d07:e80c:412e with SMTP id j17-20020a252311000000b00d07e80c412emr111717ybj.12.1690994043019; Wed, 02 Aug 2023 09:34:03 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:34 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.6.Ice063dcf87d1b777a72e008d9e3406d2bcf6d876@changeid> Subject: [PATCH v4 6/8] iommu/arm-smmu-v3: Move CD table to arm_smmu_master From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093404_131740_2EC5E33B X-CRM114-Status: GOOD ( 25.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With this change, each master will now own its own CD table instead of sharing one with other masters attached to the same domain. Attaching a stage 1 domain installs CD entries into the master's CD table. SVA writes its CD entries into each master's CD table if the domain is shared across masters. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe --- Changes in v4: - Added comment about the cd_table's dependency on the iommu core's group mutex. - Narrowed the range of code for which the domain's init_mutex is held on attach since it now only protects the arm_smmu_domain_finalise call. Changes in v2: - Allocate CD table when it's first needed instead of on probe. Changes in v1: - The master's CD table allocation was previously split to a different commit. This change now atomically allocates the new CD table, uses it, and removes the old one. drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 82 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 7 +- 2 files changed, 39 insertions(+), 50 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 34bd7815aeb8..e2c33024ad85 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1025,7 +1025,7 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid) unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &master->domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; if (!cdcfg->l1_desc) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; @@ -1062,7 +1062,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, u64 val; bool cd_live; __le64 *cdptr; - struct arm_smmu_ctx_desc_cfg *cd_table = &master->domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; if (WARN_ON(ssid >= (1 << cd_table->max_cds_bits))) return -E2BIG; @@ -1125,14 +1125,13 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) +static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master) { int ret; size_t l1size; size_t max_contexts; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; cdcfg->stall_enabled = master->stall_enabled; cdcfg->max_cds_bits = master->ssid_bits; @@ -1174,12 +1173,12 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_master *master) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -1287,7 +1286,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, if (smmu_domain) { switch (smmu_domain->stage) { case ARM_SMMU_DOMAIN_S1: - cd_table = &smmu_domain->cd_table; + cd_table = &master->cd_table; break; case ARM_SMMU_DOMAIN_S2: case ARM_SMMU_DOMAIN_NESTED: @@ -2077,14 +2076,10 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) free_io_pgtable_ops(smmu_domain->pgtbl_ops); - /* Free the CD and ASID, if we allocated them */ + /* Free the ASID or VMID */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_ctx_desc_cfg *cd_table = &smmu_domain->cd_table; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cd_table->cdtab) - arm_smmu_free_cd_tables(smmu_domain); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2096,7 +2091,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { @@ -2115,10 +2110,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_alloc_cd_tables(smmu_domain, master); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2130,17 +2121,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(master, 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2203,7 +2186,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2436,22 +2419,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (!smmu_domain->smmu) { smmu_domain->smmu = smmu; ret = arm_smmu_domain_finalise(domain, master); - if (ret) { + if (ret) smmu_domain->smmu = NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu != smmu) { + } else if (smmu_domain->smmu != smmu) ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->cd_table.max_cds_bits) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->cd_table.stall_enabled != master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; - } + + mutex_unlock(&smmu_domain->init_mutex); + if (ret) + return ret; master->domain = smmu_domain; @@ -2465,6 +2440,22 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + if (!master->cd_table.cdtab) { + ret = arm_smmu_alloc_cd_tables(master); + if (ret) { + master->domain = NULL; + return ret; + } + } + + ret = arm_smmu_write_ctx_desc(master, 0, &smmu_domain->cd); + if (ret) { + master->domain = NULL; + return ret; + } + } + arm_smmu_install_ste_for_dev(master); spin_lock_irqsave(&smmu_domain->devices_lock, flags); @@ -2472,10 +2463,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_enable_ats(master); - -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); - return ret; + return 0; } static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, @@ -2719,6 +2707,8 @@ static void arm_smmu_release_device(struct device *dev) arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); + if (master->cd_table.cdtab_dma) + arm_smmu_free_cd_tables(master); kfree(master); } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6066a09c0199..1f3b37025777 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -694,6 +694,8 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + /* Locked by the iommu core using the group mutex */ + struct arm_smmu_ctx_desc_cfg cd_table; unsigned int num_streams; bool ats_enabled; bool stall_enabled; @@ -720,11 +722,8 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_ctx_desc_cfg cd_table; - }; - struct arm_smmu_s2_cfg s2_cfg; + struct arm_smmu_s2_cfg s2_cfg; }; struct iommu_domain domain; From patchwork Wed Aug 2 16:32:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BB5CC001E0 for ; Wed, 2 Aug 2023 16:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=/IjjDvDlc8a0RPmCre0VPUbrl6N4I+TYnNG6gDNWv+U=; b=EodhV68bqHDOqgIxAoylaJ19I8 WqE5CFa10RpWgjdoMK2FwxUNxWSnwEPvHIpzqKNy5ehy1DZJkVRsKg7S6Rkmk4eXi4hMbAuAvR3+O g2pzP7Os4Ei/sGbD6I08xvdf0PenmRCcTzQikdUaNKPe1b2+Yvcvt731lmRsY+pUNZgHq2GUOY0g/ RtzC/8oUzfwOwhj1f6pgt46zb7kFDKgTSSi/3D28sW250obUW/AeWitQDfSLRjCb5qG9jpxfbTO+j jqMyHXi1WNSga+7KEmWyODFgUcRfzogpTRM25GQH6JOXPq8xriMsY2aiHAcJGRmEOWW2J0NCvDC2B pxDdjOTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREne-005Ma3-0l; Wed, 02 Aug 2023 16:34:14 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnY-005MW1-39 for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:34:11 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d087ffcc43cso7095410276.3 for ; Wed, 02 Aug 2023 09:34:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994047; x=1691598847; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yNDoyQCVv3of8CbzD7UrBZ/J064WuiNsrQAu+4gPoJw=; b=COmmzr/ZrzYU67GXsaAkeCXcFTbuJ4N8bt6e/ulY4MJB1st5jFNkRFhT/96mRMHlz+ nDJ0CKW6+HGNxd2G2NDaa+3+cbydrawx8InWmB8xrO4E/yMJBV42TKmhqE/Ydulfwzbc 5f7EkEk6lU0H1/tArBx2LsVPEdl45hoKc0tDzDQTGPcORkHWZw3ESp1OUVk4MGPkpZ17 xuz4LlVY+3IgMv9G21YFJ66iibwnU+0jp2Le7P3y3olqcOEoYLoh/sWwhQ/C5wY0w5wu reRZS2ocDr1HijC+xabhRJy5L71w9aKwcLgE52nPKM9ig6A8G87EtLRFx5j9s10DvhTM OdFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994047; x=1691598847; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yNDoyQCVv3of8CbzD7UrBZ/J064WuiNsrQAu+4gPoJw=; b=h1/uM506L8lhIruFtVT2yBYa3gKhelfW2y+JTMdtKK4nVnJDLfpjb6mWjJGsVJWq0V rZuwdOdibLpzse5F/TYBjluBblByLQKQ2JUsTo8sL20YpgeZqoF6L2EqmCEaC73hGaTN IStYAp1+yKqmaP66e0Kjo1iIgz7gYgvvE7AJBYZ9+at4nN4pz71L1qrT4eKaPBsiQoKM fAn9NyoHgSvrNjEh7z/Cy5kny6+eIFkvwHGZe+tMb5I4Hlt3oAbVofb8co/DzBedELuM WnSmGV98g/nagaG9vGRlngAzwKYVHEch+JmoRnOMchUmbhZhugZb9bXbTDwoph+VfgfR s8Ww== X-Gm-Message-State: ABy/qLa+KG4yMl1hwj65Hv0whXfC/s6BOqTy+erbZ+v/Jd0T7upYi2xv zLjg1X9HNkJ1SPX+uS/JtRtmxI1SmaiY X-Google-Smtp-Source: APBJJlEsy+/m4IjjcjYGAp6bgcwwwvW6CNiqjOP+V/RjmxBEbdQ/ZzssDS1E1hx9GXz5q6Yk4d1bzfTzej4b X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a05:6902:160e:b0:d09:6ba9:69ec with SMTP id bw14-20020a056902160e00b00d096ba969ecmr129159ybb.4.1690994047546; Wed, 02 Aug 2023 09:34:07 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:35 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.7.Idedc0f496231e2faab3df057219c5e2d937bbfe4@changeid> Subject: [PATCH v4 7/8] iommu/arm-smmu-v3: Skip cd sync if CD table isn't active From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093409_032334_5E3D7D8A X-CRM114-Status: GOOD ( 19.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This commit explicitly keeps track of whether a CD table is installed in an STE so that arm_smmu_sync_cd can skip the sync when unnecessary. This was previously achieved through the domain->devices list, but we are moving to a model where arm_smmu_sync_cd directly operates on a master and the master's CD table instead of a domain. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v3) Changes in v3: - Flip the cd_table.installed bit back off when table is detached - re-order the commit later in the series since flipping the installed bit to off isn't obvious when the cd_table is still shared by multiple masters. Changes in v2: - Store field as a bit instead of a bool. Fix comment about STE being live before the sync in write_ctx_desc(). drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 +++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e2c33024ad85..00b602ae9636 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -985,6 +985,9 @@ static void arm_smmu_sync_cd(struct arm_smmu_master *master, }, }; + if (!master->cd_table.installed) + return; + cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.cfgi.sid = master->streams[i].id; @@ -1091,7 +1094,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, cdptr[3] = cpu_to_le64(cd->mair); /* - * STE is live, and the SMMU might read dwords of this CD in any + * STE may be live, and the SMMU might read dwords of this CD in any * order. Ensure that it observes valid values before reading * V=1. */ @@ -1360,6 +1363,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, cd_table->l1_desc ? STRTAB_STE_0_S1FMT_64K_L2 : STRTAB_STE_0_S1FMT_LINEAR); + cd_table->installed = true; + } else { + master->cd_table.installed = false; } if (s2_cfg) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 1f3b37025777..e76452e735a0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -599,6 +599,8 @@ struct arm_smmu_ctx_desc_cfg { u8 max_cds_bits; /* Whether CD entries in this table have the stall bit set. */ u8 stall_enabled:1; + /* Whether this CD table is installed in any STE */ + u8 installed:1; }; struct arm_smmu_s2_cfg { From patchwork Wed Aug 2 16:32:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13338520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B7F70C001E0 for ; Wed, 2 Aug 2023 16:34:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=BqcyKl9IjCq2nV6w83pq6X/6zpyNSkJGTRutDBPdIng=; b=XtsC6jpbMMshuUIGVS/eC0Mcmo r2g9YJa9gqWAS1i5GS4j8zaPJjZog5NUPGGM+LKhexk/7FIkL+V61JbAumXgQsR+n5NuxZWOq4hwV f34RYPssVhrMBQtoC9MDs8uOu4sROwOcXF2VRJP7U9gIJvGFgZzphFOjc6RWamcwNhpplH65bfcj3 FeTgspe0/Y5OFKnVRcsXuyGRqQDJLBQ0dw9G+YOwd9pp7SDueuAj7CrJ+uEkI69+Ke2T8syfv8ckD NRx/c222aDL5TuqB1a0uIEZhoZUPlOy2Ep/c/BvVw8Dr66kYZsLssSmHIeMI7jH60hST5iy6D5ksw GIPBfUgA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qREnf-005Mar-2g; Wed, 02 Aug 2023 16:34:15 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qREnc-005MYs-3A for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 16:34:14 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-583a89cccf6so64602457b3.1 for ; Wed, 02 Aug 2023 09:34:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690994051; x=1691598851; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SX+2RilXbghvKGSDcU4l4pSTUg8nkB5V07mKjGHD9zU=; b=fmu/wq+j9XJamWtxak49+VY4i4wA2bpndlx+SuvJLUzuyQMgduoha1xv5yDw7EjQmH hYwFC5Dp3ghD80jI55zMLNMB300EShX/vDYkeOuVlO+tWwRzAL0G8vPFXjxcoJvx90Nw +oDyX1su5nDPZPhGL4RondzIp9Rpb0Np2iOgsBBcV8RZSQVg+cn33X3Hhw/noDCPo0Sd pGvmMrQvN9ULN4OBjNPxWoErEXF7uRE1GSrkkoM14g7hSbJl7ABYgIHvjjnbe9KXNmwA 6pZ/zqPfh8eUhUH0SPxSKkpMGTMURuUb48mxPoXxPo4feGyHztBWBWZPYNw0sV2TOVhR 3MAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690994051; x=1691598851; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SX+2RilXbghvKGSDcU4l4pSTUg8nkB5V07mKjGHD9zU=; b=JCSiyfrbyziCj8u7EuhkFlsoyF9A4EHJtreS/mFV0OHqxrew/ekP8kHdK+UscqydtH tDPtNgrdf3N4BS6XYQ0xH1q9Of/sZA/6BeFUWy+hWTBWheOBt/rEjd8IA53ZJXLiMDqn 54wf4S5Cch5yKKSbb4xyPQ0eAlDeaoSF0YZvp7QUW57w2TWmr9iagB1JBRB3GdZHTUI/ NmN5ukriKSqtNQY72GbaouIjCKKwpG3YFflwtE/MqgL4vogvLAWVexLKXj9x+4waX3oi TRUwcTR1S4wakvU7j4/+pq15ZaDKhtPf47XeQve0tY+uSp4KhM48DDfqwWjbP4fjaChP Mptg== X-Gm-Message-State: ABy/qLZXh1ggaGHbNDu2Sr9Daih5gCid/LKTHZ30mhRzCspihxPRkIHU 6Dc1AJTxhbdcXtGvZB5pzkgX8Pb1Cws4 X-Google-Smtp-Source: APBJJlH+p33kyoFFDof3hVEcZBsvq//wMVZT//zNeOrqEMZM0ygVocCT4AvqB+44uOX7f+sfkJFHZDM7OKl4 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:39d1:8774:b733:6210]) (user=mshavit job=sendgmr) by 2002:a81:af0e:0:b0:579:fa4c:1f22 with SMTP id n14-20020a81af0e000000b00579fa4c1f22mr137586ywh.6.1690994051697; Wed, 02 Aug 2023 09:34:11 -0700 (PDT) Date: Thu, 3 Aug 2023 00:32:36 +0800 In-Reply-To: <20230802163328.2623773-1-mshavit@google.com> Mime-Version: 1.0 References: <20230802163328.2623773-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230803003234.v4.8.I5ee79793b444ddb933e8bc1eb7b77e728d7f8350@changeid> Subject: [PATCH v4 8/8] iommu/arm-smmu-v3: Rename cdcfg to cd_table From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: will@kernel.org, robin.murphy@arm.com, nicolinc@nvidia.com, jgg@nvidia.com, jean-philippe@linaro.org, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230802_093413_022217_7B91654D X-CRM114-Status: GOOD ( 15.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org cdcfg is a confusing name, especially given other variables with the cfg suffix in this driver. cd_table more clearly describes what is being operated on. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- (no changes since v3) Changes in v3: - Commit message update Changes in v2: - New commit drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 66 ++++++++++----------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 00b602ae9636..61de66d17a5d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1028,18 +1028,18 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid) unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; - if (!cdcfg->l1_desc) - return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; + if (!cd_table->l1_desc) + return cd_table->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; - l1_desc = &cdcfg->l1_desc[idx]; + l1_desc = &cd_table->l1_desc[idx]; if (!l1_desc->l2ptr) { if (arm_smmu_alloc_cd_leaf_table(smmu, l1_desc)) return NULL; - l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; + l1ptr = cd_table->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ arm_smmu_sync_cd(master, ssid, false); @@ -1134,33 +1134,33 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master) size_t l1size; size_t max_contexts; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; - cdcfg->stall_enabled = master->stall_enabled; - cdcfg->max_cds_bits = master->ssid_bits; - max_contexts = 1 << cdcfg->max_cds_bits; + cd_table->stall_enabled = master->stall_enabled; + cd_table->max_cds_bits = master->ssid_bits; + max_contexts = 1 << cd_table->max_cds_bits; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || max_contexts <= CTXDESC_L2_ENTRIES) { - cdcfg->num_l1_ents = max_contexts; + cd_table->num_l1_ents = max_contexts; l1size = max_contexts * (CTXDESC_CD_DWORDS << 3); } else { - cdcfg->num_l1_ents = DIV_ROUND_UP(max_contexts, + cd_table->num_l1_ents = DIV_ROUND_UP(max_contexts, CTXDESC_L2_ENTRIES); - cdcfg->l1_desc = devm_kcalloc(smmu->dev, cdcfg->num_l1_ents, - sizeof(*cdcfg->l1_desc), + cd_table->l1_desc = devm_kcalloc(smmu->dev, cd_table->num_l1_ents, + sizeof(*cd_table->l1_desc), GFP_KERNEL); - if (!cdcfg->l1_desc) + if (!cd_table->l1_desc) return -ENOMEM; - l1size = cdcfg->num_l1_ents * (CTXDESC_L1_DESC_DWORDS << 3); + l1size = cd_table->num_l1_ents * (CTXDESC_L1_DESC_DWORDS << 3); } - cdcfg->cdtab = dmam_alloc_coherent(smmu->dev, l1size, &cdcfg->cdtab_dma, + cd_table->cdtab = dmam_alloc_coherent(smmu->dev, l1size, &cd_table->cdtab_dma, GFP_KERNEL); - if (!cdcfg->cdtab) { + if (!cd_table->cdtab) { dev_warn(smmu->dev, "failed to allocate context descriptor\n"); ret = -ENOMEM; goto err_free_l1; @@ -1169,9 +1169,9 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master) return 0; err_free_l1: - if (cdcfg->l1_desc) { - devm_kfree(smmu->dev, cdcfg->l1_desc); - cdcfg->l1_desc = NULL; + if (cd_table->l1_desc) { + devm_kfree(smmu->dev, cd_table->l1_desc); + cd_table->l1_desc = NULL; } return ret; } @@ -1181,30 +1181,30 @@ static void arm_smmu_free_cd_tables(struct arm_smmu_master *master) int i; size_t size, l1size; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; - if (cdcfg->l1_desc) { + if (cd_table->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); - for (i = 0; i < cdcfg->num_l1_ents; i++) { - if (!cdcfg->l1_desc[i].l2ptr) + for (i = 0; i < cd_table->num_l1_ents; i++) { + if (!cd_table->l1_desc[i].l2ptr) continue; dmam_free_coherent(smmu->dev, size, - cdcfg->l1_desc[i].l2ptr, - cdcfg->l1_desc[i].l2ptr_dma); + cd_table->l1_desc[i].l2ptr, + cd_table->l1_desc[i].l2ptr_dma); } - devm_kfree(smmu->dev, cdcfg->l1_desc); - cdcfg->l1_desc = NULL; + devm_kfree(smmu->dev, cd_table->l1_desc); + cd_table->l1_desc = NULL; - l1size = cdcfg->num_l1_ents * (CTXDESC_L1_DESC_DWORDS << 3); + l1size = cd_table->num_l1_ents * (CTXDESC_L1_DESC_DWORDS << 3); } else { - l1size = cdcfg->num_l1_ents * (CTXDESC_CD_DWORDS << 3); + l1size = cd_table->num_l1_ents * (CTXDESC_CD_DWORDS << 3); } - dmam_free_coherent(smmu->dev, l1size, cdcfg->cdtab, cdcfg->cdtab_dma); - cdcfg->cdtab_dma = 0; - cdcfg->cdtab = NULL; + dmam_free_coherent(smmu->dev, l1size, cd_table->cdtab, cd_table->cdtab_dma); + cd_table->cdtab_dma = 0; + cd_table->cdtab = NULL; } bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd)