From patchwork Tue Aug 1 18:35:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13337164 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BBA8C00528 for ; Tue, 1 Aug 2023 18:39:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=KQ7nMFEqP442dRW1DBoWCKSdP+LpVRZIbjF/8yB2tus=; b=ijD3ec0fmpZm3E2LBpj6TJrj2a wEJqn3h695lg3RYd3T/kaJFGJaIBrBoHQOZtSuCORz7rEP4IMzGFyZAurx+XDRidONhJhnwEYIJyb LXNllFMx1+XRibE8HeqHvcb2ACcqHF7pZMGUxqzhP5mIe22yZd2VfCGSm5Ovx85vboRDe/q0Nyq9t jr+wr9xfJGC+N4OTH9sIRZKceI8vDp8O8ZZ/DbcDrOBKG0qRcCM4F13G8Xtkr42lhrtiNxuD8ujQK HVMsztCGmI46WX6uYJs7FksLwSOP8ar1DWX0l4jGrA+X04hfq5PYYomXVGOTwRZEZvMzUF5VcmQOR lfeIkhRA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qQuHF-00331j-1l; Tue, 01 Aug 2023 18:39:25 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qQuHB-0032xZ-19 for linux-arm-kernel@lists.infradead.org; Tue, 01 Aug 2023 18:39:24 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d1efa597303so208088276.0 for ; Tue, 01 Aug 2023 11:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690915160; x=1691519960; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+4BLZZeGhG40EcbLuFOBYtdkNq0Rnp36mGGQJXKhqHg=; b=giB1hGxDlMSE5unC+CgFqybePP3FRLQ3UTMIB/Lm9Au07qT5AJnLiDgMQro9DByt7H H9hkyrUp7SAxF04p+3pIrtSkoOg90qLXfLooiJG0HgGocxJZUMAcJ/JacG4hXgLjizx7 akACRhjWunOio6OLZsR/9Wih0Y45Tk6Orv4jJLCZvVKyjDjFkHFfl7tCRO8LLcH1xdMv 8Xk0T2hj41QDoxlDYIWw+J3hnweUqpeu8l/I2YW/TGMqfUbS++J5kRl5Y1rHzLlsifU3 XaYMK8WiQsu14t+4wWZWDt5E+zm1ESd/v39kvrd08xmqr4a65ilNzqc9BSzbALTDHPmv gWeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690915160; x=1691519960; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+4BLZZeGhG40EcbLuFOBYtdkNq0Rnp36mGGQJXKhqHg=; b=JiDv9XwpTYDKchnweeikjftgN0tGMlT7Yb1KC43TZ4M7pO+9UPT8syaWsy7YB4h7/m 9V69z+6GoFT297NwOkDjLnTFeapaIYZRbMDPnR/8IvueZhqGQQ5k5RmbpyZuiZL50etd MUjjGaSKFE+pLVTqvMMtxbGU0yoitfATurENDtey382BYtwQoeCqiKbwA6IEDgVlQr6b vGGafUVM+8X0F5WrgyeTwuHpTvJYdAB1C/GuaijpA38JcIgr03LuN3EfpY+ktodPiOpY 6mX0xiTkFqoD5AMHXbGQpw9ZaZYU2b6kKUS9R1l5veSy/QO6yDL+dfciYaFviaD2UWgE zUkw== X-Gm-Message-State: ABy/qLaZ0Di+cCDNuYfLOcHTEe7cnVCxvp1OIzf/9QHTp/mjuntMdJL2 0LHymVDp8wNK2VkAq1Nq6GS77ZxpgoHU X-Google-Smtp-Source: APBJJlF1fqQecIcDPlsgvc3fvbv+86ydAutfRz0YQYPV0qbDrPfVZ9ahNyS3E9NkY1dnqq62bKNmKaGPtm5R X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:a54:d53d:50e4:b5b8]) (user=mshavit job=sendgmr) by 2002:a05:6902:11cb:b0:d16:7ccc:b406 with SMTP id n11-20020a05690211cb00b00d167cccb406mr154700ybu.5.1690915159874; Tue, 01 Aug 2023 11:39:19 -0700 (PDT) Date: Wed, 2 Aug 2023 02:35:23 +0800 In-Reply-To: <20230801183845.4026101-1-mshavit@google.com> Mime-Version: 1.0 References: <20230801183845.4026101-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230802023524.v3.6.Ice063dcf87d1b777a72e008d9e3406d2bcf6d876@changeid> Subject: [PATCH v3 6/8] iommu/arm-smmu-v3: Move CD table to arm_smmu_master From: Michael Shavit To: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: robin.murphy@arm.com, will@kernel.org, jean-philippe@linaro.org, jgg@nvidia.com, nicolinc@nvidia.com, Michael Shavit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230801_113921_398655_79658D06 X-CRM114-Status: GOOD ( 22.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With this change, each master will now own its own CD table instead of sharing one with other masters attached to the same domain. Attaching a stage 1 domain installs CD entries into the master's CD table. SVA writes its CD entries into each master's CD table if the domain is shared across masters. Signed-off-by: Michael Shavit Reviewed-by: Jason Gunthorpe --- (no changes since v2) Changes in v2: - Allocate CD table when it's first needed instead of on probe. Changes in v1: - The master's CD table allocation was previously split to a different commit. This change now atomically allocates the new CD table, uses it, and removes the old one. drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 65 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 2 files changed, 31 insertions(+), 40 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 34bd7815aeb8e..b05963cd4e5b5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1025,7 +1025,7 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid) unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &master->domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; if (!cdcfg->l1_desc) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; @@ -1062,7 +1062,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, u64 val; bool cd_live; __le64 *cdptr; - struct arm_smmu_ctx_desc_cfg *cd_table = &master->domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; if (WARN_ON(ssid >= (1 << cd_table->max_cds_bits))) return -E2BIG; @@ -1125,14 +1125,13 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) +static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master) { int ret; size_t l1size; size_t max_contexts; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; cdcfg->stall_enabled = master->stall_enabled; cdcfg->max_cds_bits = master->ssid_bits; @@ -1174,12 +1173,12 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_master *master) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->cd_table; + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_ctx_desc_cfg *cdcfg = &master->cd_table; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -1287,7 +1286,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, if (smmu_domain) { switch (smmu_domain->stage) { case ARM_SMMU_DOMAIN_S1: - cd_table = &smmu_domain->cd_table; + cd_table = &master->cd_table; break; case ARM_SMMU_DOMAIN_S2: case ARM_SMMU_DOMAIN_NESTED: @@ -2077,14 +2076,10 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) free_io_pgtable_ops(smmu_domain->pgtbl_ops); - /* Free the CD and ASID, if we allocated them */ + /* Free the ASID or VMID */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_ctx_desc_cfg *cd_table = &smmu_domain->cd_table; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cd_table->cdtab) - arm_smmu_free_cd_tables(smmu_domain); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2096,7 +2091,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { @@ -2115,10 +2110,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_alloc_cd_tables(smmu_domain, master); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2130,17 +2121,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(master, 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2203,7 +2186,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2443,14 +2426,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->cd_table.max_cds_bits) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->cd_table.stall_enabled != master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } master->domain = smmu_domain; @@ -2465,6 +2440,22 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + if (!master->cd_table.cdtab) { + ret = arm_smmu_alloc_cd_tables(master); + if (ret) { + master->domain = NULL; + goto out_unlock; + } + } + + ret = arm_smmu_write_ctx_desc(master, 0, &smmu_domain->cd); + if (ret) { + master->domain = NULL; + goto out_unlock; + } + } + arm_smmu_install_ste_for_dev(master); spin_lock_irqsave(&smmu_domain->devices_lock, flags); @@ -2719,6 +2710,8 @@ static void arm_smmu_release_device(struct device *dev) arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); + if (master->cd_table.cdtab_dma) + arm_smmu_free_cd_tables(master); kfree(master); } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6066a09c01996..f2acfcc1af925 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -694,6 +694,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_ctx_desc_cfg cd_table; unsigned int num_streams; bool ats_enabled; bool stall_enabled; @@ -720,11 +721,8 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_ctx_desc_cfg cd_table; - }; - struct arm_smmu_s2_cfg s2_cfg; + struct arm_smmu_s2_cfg s2_cfg; }; struct iommu_domain domain;