From patchwork Wed Jun 14 15:41:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79F32EB64D9 for ; Wed, 14 Jun 2023 15:43:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=nTl9cQP1aWn7AXsIY95ZEOnRVhCxUAxUnUlWptbBFNU=; b=TmZO+YLUVw3AVnoIB4PIH9Q8Rz nTvYByh0Lnj4ISsOZ+ihdTB5UTk2tmk3hZHBhn9CrrjUmclXOxaLZYybm3QXgkmjPdPA17wJoeX4w R3HPp4XFAGFWtlqjoLCLCIeVNdegzL7pRzKoT0Bs4QphHHb7IOOv3X9Yqh9BKy2+njTqIr6kOSTVo +FeHbxAmLoGik9Njld52NQCIwXkUsVs4VujPy7Ec6LYOfJHmayMLYZkhu+QdUuy+LTYdwVUn73Kqp lgPxg+ajGfFV/KPEsLSU8iSW9dAQqxckdToOZ4ljGATyOTP/G7MWW4mqOWPDHlO86vYv+5ThC6Sc9 lav864EA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sep-00C6AU-1s; Wed, 14 Jun 2023 15:43:39 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sen-00C69j-1O for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:43:38 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-56ff697b86cso12028947b3.1 for ; Wed, 14 Jun 2023 08:43:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757415; x=1689349415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=XcbpIeR43JDZfn4lLCsVf0JdsLi6nRjRM50ZNMNNgoiXVaiw+/ZGHnE5KTJjMCs9aO vnK/+7uf6oxkg7kl2xEDoB5F4B2QtCLAzLVEn4t/E3os9JAaCcYrN3AO74r8GCTCI1ai CNeyG9s42Dtj6TN/jMZhQ7UQlbA6doeYtm3JtoHKSi/95yADdM887JCovkOxifmKCPA2 z9m+8SiMXacLLqG5IobUr532pG3VZDDME93RDKR+p2TzHi9YlbOhVRmhqLQz3NpdaYUk zE23NqAZ3c+vCO3XhN19QdpZYVPa2PS0tkE5LTgBL+aO1vkz+gKWJS2k5UoUETqYjKW/ Dl/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757415; x=1689349415; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=JlAgZVkCWHYAflNa0Gq38E4xyCtnhCJvxEXKUc+GcRgyKTtQu5bEofqkPB+Qf4VyJS DZpsTusAB3j6Afy6y4L7Plgo9tBfW9SBxgaKZT0o6prDLG+V4A/OHhwL0N8COcs9BzXT ue/xgezE7C23CpUyEY24LZTGNNknEuQQZ3v20lV8FwmbNCn4/XDSsYSYVlftxQCUy1qq yvh3QW6J6WCl5yLeVWoN88sbGQHtqKV/lvFkH4snVN/YQHdp3tAC9fJRvFMTDfILzyQ5 6i6cVcfrm8hXFEjT9ZavuWxWm8/l7S1iSJbsX0I8GVFGz53ALAMMvbADOCWi9hWSFUg8 ZGAA== X-Gm-Message-State: AC+VfDyGuhROyB4PhEc2JRFb47sU0j/k2+wtjFpw2IsmPlktlADlzf2h OdRYqP+k5AKlQn1k0oIQhjGjEaHaeWDj X-Google-Smtp-Source: ACHHUZ6p8for2Qz1f423vXRFsQW/AZzRMixrBtrLwCsmjpkKSoWyMi0rfNyVEkzE8Fw0M2x5Q3BMJBOn9mnK X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a81:ac49:0:b0:561:c9c0:98d9 with SMTP id z9-20020a81ac49000000b00561c9c098d9mr1044537ywj.4.1686757414962; Wed, 14 Jun 2023 08:43:34 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:53 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-2-mshavit@google.com> Subject: [PATCH v3 01/13] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084337_468812_A9757ABC X-CRM114-Status: GOOD ( 19.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 ++++++++++--------- 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947eb..968559d625c40 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } - smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain = container_of(cd, struct arm_smmu_domain, cd); smmu = smmu_domain->smmu; ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb757227..beff04b897718 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1863,7 +1863,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1946,7 +1946,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid = smmu_domain->cd.asid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -2077,7 +2077,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2096,13 +2096,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2115,23 +2116,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_free_asid; - cfg->cd.asid = (u16)asid; - cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid = (u16)asid; + cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; @@ -2141,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index b574c58a34876..68d519f21dbd8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -593,7 +593,6 @@ struct arm_smmu_ctx_desc_cfg { struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -707,25 +706,28 @@ enum arm_smmu_domain_stage { }; struct arm_smmu_domain { - struct arm_smmu_device *smmu; - struct mutex init_mutex; /* Protects smmu pointer */ + struct arm_smmu_device *smmu; + struct mutex init_mutex; /* Protects smmu pointer */ - struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; - atomic_t nr_ats_masters; + struct io_pgtable_ops *pgtbl_ops; + bool stall_enabled; + atomic_t nr_ats_masters; - enum arm_smmu_domain_stage stage; + enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; + struct arm_smmu_s2_cfg s2_cfg; }; - struct iommu_domain domain; + struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head devices; + spinlock_t devices_lock; - struct list_head mmu_notifiers; + struct list_head mmu_notifiers; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) From patchwork Wed Jun 14 15:41:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96FCCEB64D9 for ; Wed, 14 Jun 2023 15:44:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=JtyNsFqPMgRMR/4IX6kE2UHyrUNxxoZ7F2pXCAOKgp4=; b=jlXTaH0rrvJ+I5XZb44xNZbtrP BYV39z5vb//oyDRyI+WGIOx+eHiBp6ze1rFG+kiTs7i/ye/DHG9Mh/+XvYAdWa/p2EXePEVuS2+pg QoErq/2Dxt1fmYq0/SPAziIw6ebFYZ5PazeJ78bIWAf38wun1nT1z02mVuUZT6YE+gAZd/F8xlTxe TwWEPVnW+kZ8zlYe5bEPQPbWTvVZ0oJ6Fmdb5ORPREioR05mDo4OuWgtRzFjrBT3jtD7d7TvMHVcC POG6AoSvlzNHWUjfmZ+kXhd2c7o6DXO2vGoMyHJOf1qIIpWa59uXK6Bd+hnRnXyEEr67PKGEILNHd Wh4mvM0Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sex-00C6Cj-0w; Wed, 14 Jun 2023 15:43:47 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Seu-00C6Bh-24 for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:43:46 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bd5059e5347so1729251276.0 for ; Wed, 14 Jun 2023 08:43:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757423; x=1689349423; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=qhhtV69WKH2JluNtYemjQLj4d4hfClGTE//m6T+O7FE46Wo3W/vrmENSH/WtXfrT48 8a5S07rhQ59AKc8+BgfzUy5Bj2swQlHUnRk3ucu69XFSSsZ+AomWUEn1sB396DApLgAe 0adprdBEbSVUxQQfd4dT6RDzrQc6KTQN6bsgOYjw9CUc9dhFHYC2qWnzn+HRn1UPA9aH kjf90LWp0yxILw0IiPJOuiriOeSoOMmi9XkZKcrRXV97WwWsR1pNdEew2aR6B2qrQLUm 3F+WX1+FcFiI+9BtyS27Rf8odZ0hD84gSWCl+MLkesx9GDJcjrjG+T0HGly7ZGRIuVWx 6KSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757423; x=1689349423; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=GtBxIjWB7hRM5hf2PEhobrLZbbTp5dNO0lhLS8ObppnTcmkDQSbmqR3ngUmKKiF08j 4idqpHJFWPQ5erg9LWDwZBNt5aDO/pYJsbXLLtcMSLBzoMnW6mJmOfofC+pyVF90uo4E 6UwgiQCqVUO77mgHqQPiIZe+/eTTn+SXdDwqVib6W4uLLU43JBXOwawGE5VqNc5w4hQ9 CCLtWULqRXB6Gb+OWJyOdozkt3qtkKeaepDUa5DHGhtOVMJnMwALKTOhhB3uvssZdjRQ C6XRDM0kwUPE5XAr2sM3mNwqncMMssMR5qd+O1VMTzWIXKuxd8A4HlZnx3vkBHbnTFzv YJ0A== X-Gm-Message-State: AC+VfDz0AMCFuueU9wlr743p8suIf0H98XZgDd0YNq1HyLFtoao4XSCG 6MUkKpiD618RBukTWjzB87/FI2azyqXD X-Google-Smtp-Source: ACHHUZ7dqpkyX9QIV+ua1DgHJ+4Gpjcw5QzgmZIv8lQZIXKZNkPkBQS62jzmn2be9tvs4Kd2TECXDYFsL+z8 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:e6c4:0:b0:bc7:f6af:8cfe with SMTP id d187-20020a25e6c4000000b00bc7f6af8cfemr880044ybh.4.1686757422932; Wed, 14 Jun 2023 08:43:42 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:54 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-3-mshavit@google.com> Subject: [PATCH v3 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084344_676730_212FE2FF X-CRM114-Status: GOOD ( 18.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Except for Nested domains, arm_smmu_master will own the STEs that are inserted into the arm_smmu_device's STE table. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index beff04b897718..023769f5ca79a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1126,15 +1126,16 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, return 0; } -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, + struct arm_smmu_s1_cfg *cfg) { int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; + cfg->s1cdmax = master->ssid_bits; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -1175,12 +1176,11 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) return ret; } -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu, + struct arm_smmu_ctx_desc_cfg *cdcfg) { int i; size_t size, l1size; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; if (cdcfg->l1_desc) { size = CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -2076,7 +2076,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2108,11 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - cfg->s1cdmax = master->ssid_bits; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_alloc_cd_tables(smmu_domain); + ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2140,7 +2138,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, return 0; out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); out_free_asid: arm_smmu_free_asid(cd); out_unlock: @@ -2704,6 +2702,13 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled = true; + ret = arm_smmu_init_s1_cfg(master, &master->owned_s1_cfg); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } + return &smmu->iommu; err_free_master: @@ -2719,6 +2724,7 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); + arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); kfree(master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 68d519f21dbd8..053cc14c23969 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -688,6 +688,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_s1_cfg owned_s1_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Wed Jun 14 15:41:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3AE64EB64D9 for ; Wed, 14 Jun 2023 15:44:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=xcXLA4qvVwdY9xNm98oHzV9Ji6w7LjxGzKvI+3GHKq8=; b=Zv3m7h9/eI+tr9jxPVft9U1V9/ zw84xRnXy85HHpsIDzEC2bfY8RlT23jCh4fl7CrAXfTTYaUZ68mmut4Qdklosb7D6qKE9I/+il1/A Q8CzcUmRrkOH2NsSeBiCSSJzOjzSYQOf2vtUNhH1rWD9S7uh5n/WM+g/SyJjuicbeGkuPVjoKPR8b sOCKsGhKMV3koIaHvaEv/E/L5XObkPuqs/FTCm90vsOVDO5fal8xumhYnxbrgWGBG8n0NfqzsFQ+B WGwvgPNghxB/RJ8CPFw+B0Ezjq0+5v/YFLpYaxNC8w7ugmeI/jVOBOLxyg1X7Lt+ybkwN3OVbZOd2 WsvIUT5w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sf5-00C6G9-01; Wed, 14 Jun 2023 15:43:55 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sf2-00C6El-18 for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:43:53 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5651d8acfe2so12042677b3.2 for ; Wed, 14 Jun 2023 08:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757431; x=1689349431; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=JXTplwSsE5VOimvzwSxoOBAXrvM0DkSnTFs5atKOgzNUFE/C6yugZQCGi8Ug+Q8za4 5m/tPpgqerRmaFNQv3MeyUdn3zLMeQPVwdgidcdiQSni224I4J4BLjawCnAL5dEuCm4B GttF1B/Zse1OdabCBCXmYiTJKA+xHsGxGmBk9ODJn0kHpM9qNz7aVzxYApxkUb654QRM ZbG1mWIJsW+aW9HEn9KYHX9jCB1Y3C20S8I1DWRQBR0nMQHa2kwM/UwkgmmL2shc3mGR 6bxqrYSA4iicQ8gtGX2d5ts0FDp2lDELfAQh+Q2Venkpo5DaS6bweeY9O1W8mf1klzN7 iDyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757431; x=1689349431; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=l1KdFmq3u9BtSD7MWY/KRx3qKPzZvUYGTx0ZFvhaqSzcjoAq1FjcVlTWNlHwrO+yzk BercAYDnaVZTzDURfBE/8f+YugEUu6XNizjt7/Ob7YRbcqVAL/Vx3PD3e5NTObxYHzPq j4DL5XAvX2hrtIKlxomTrxkTrss1yHvlXo7fot+SNg9GbuVY3bnQSSqE8o5jtyxD5UB5 PTYNFY6l6Xnn/h5LwykjcMen+nxYmgGCbOH9EInWKXuvINqeE+R6i3A4wxxXlzVbu4A7 TGNHdSrq1eziKoUKK2H+SIo1h8+MAwLt70i9aGGgiBAQslNV+bc8H0jiaqBnlZ5m6/nm guhw== X-Gm-Message-State: AC+VfDyNT9MGAOImPzMZ5aEfq+oIg9YaSfFFgVHoBzuv4T0grd0A9mWR Zpr921sYTtI1sHNqgsfGWQNSGNQg4Jkh X-Google-Smtp-Source: ACHHUZ4sh+CBjo+i26d91zindBZbilBHxQfue0Y4MR1TnNvJRRBgFBF/DXyPE1IT5owTTggbKbSZ6ISdN3jZ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:abeb:0:b0:bc8:c749:eede with SMTP id v98-20020a25abeb000000b00bc8c749eedemr1129224ybi.7.1686757431181; Wed, 14 Jun 2023 08:43:51 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:55 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-4-mshavit@google.com> Subject: [PATCH v3 03/13] iommu/arm-smmu-v3: Refactor write_strtab_ent From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084352_395874_651A6A3D X-CRM114-Status: GOOD ( 16.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Explicity keep track of the s1_cfg and s2_cfg that are attached to a master in arm_smmu_master, regardless of whether they are owned by arm_smmu_master, arm_smmu_domain or userspace. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 37 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 023769f5ca79a..d79c6ef5d6ed4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1269,10 +1269,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; - struct arm_smmu_device *smmu = NULL; + struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_s1_cfg *s1_cfg = NULL; struct arm_smmu_s2_cfg *s2_cfg = NULL; - struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1280,24 +1279,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, }, }; - if (master) { - smmu_domain = master->domain; - smmu = master->smmu; - } - - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - s1_cfg = &smmu_domain->s1_cfg; - break; - case ARM_SMMU_DOMAIN_S2: - case ARM_SMMU_DOMAIN_NESTED: - s2_cfg = &smmu_domain->s2_cfg; - break; - default: - break; - } - } + if (master->s1_cfg) + s1_cfg = master->s1_cfg; + else if (master->s2_cfg) + s2_cfg = master->s2_cfg; if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { @@ -1319,8 +1304,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { - if (!smmu_domain && disable_bypass) + if (!(s1_cfg || s2_cfg)) { + if (disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -2401,6 +2386,8 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + master->s1_cfg = NULL; + master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2454,6 +2441,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } master->domain = smmu_domain; + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { + master->s1_cfg = &smmu_domain->s1_cfg; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || + smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + master->s2_cfg = &smmu_domain->s2_cfg; + } /* * The SMMU does not support enabling ATS with bypass. When the STE is diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 053cc14c23969..3c614fbe2b8b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,6 +689,8 @@ struct arm_smmu_master { struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_s2_cfg *s2_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; From patchwork Wed Jun 14 15:41:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCB1CEB64D9 for ; Wed, 14 Jun 2023 15:44:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PEmdYWktDf4Ok6OoR9mBsFqM8BaO56E/qeeJJcpZBPE=; b=XoclEXeQ2P0G04S7hvd8WSEEtZ LDwKaAb0nYQtJuVEVBY0SSvlWAN37shx4gkfgGwDDpI6Ach6/rcNuNZgrBDYvkw9XrRgIEZff7QKU b779L32PtBQ3nFeDx3onFZp9QlAwH0oteYYsvJAlmCSehM2ADOa435t6QNtTmx64Go6E3jFH73Fmv 60aiEKufzFUMBcTf+u+0ovqHjUYMfeqXEsrX+/21Oe9PvVnv0FKTIF4AOXIfAwRNof1B4VTNxH5M/ EBs1XwXw8n8M1SIKHtgnC9rtTQ1a9CswTPH1T4mIy0yOmfSi0vC91Fd8Pp+/mb/PdV2B2ya6xHcgk a+u6Db+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfE-00C6K7-1Y; Wed, 14 Jun 2023 15:44:04 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfA-00C6I7-1L for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:02 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-57003dac4a8so16254467b3.1 for ; Wed, 14 Jun 2023 08:43:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757439; x=1689349439; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=yTgaJKyhLJN0Sp5lo50YLdxEaUm3GgA7aXol24wNrGXS9BmaZCSXsBzcwf4Qnp+cxw MlQpJOuDL9kZUj/a5rrSo8t1KtW6798cWJvnCfnrfPiJbxg2BRihO+S5JisosY1Tqp58 yIGYg9aw8YKoSYorcITNOl8mk05e/qbpqsKk+hFZOkQ5ufs0Q6KiWNeexHmceSWU2WQK Dqf5E9lb4jGnDahmR9g4vcKZuxX76Jk2uA+xNW4FmWQJnFHkM1e1iREGtDSxGiH1DxEn 6c+PtrzgFhbj6sFtPYCjfglMkJQSVX6bEJdWxnPbAvYSq+LJSgSsOfzjwqdAqEdYQwTX uONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757439; x=1689349439; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=BL40NQxZIW0EhOhRvJrqq5yKKENcU9ONYeIrGdDO7aE5UR+UrU6fQiMhlAS/+ZYzY/ uqbu9YQgOZ6cB3X96UKy+YXzElsBXf9uqUyQbIeNbEfnRKIRhrBbc9OtboLiXA/juxaX PczdcAPJHD1TVdWG1uzvlSdZOBp/qj85w+sdOyUITUo3X1tKvY18Uau9iyZlsN8yzGgy JoFW/mMQetMD+pZ4EM21xkoC1YZGHbdfhwL/zFldNkxbg6uQq6aBZKGd8iXxeNYQf9F0 yyoT6pmk0+6YmvRd7Zf9JM6G2QYlKZ+PdkhqHzpfc+yizv3g53Jb9BJAvft+PXDX/XV3 hnGA== X-Gm-Message-State: AC+VfDy6slL1BB87zvw2qDV8CjuAulbg3D5qxDQrtDQhim+RDfCCITjW BB5wYIhz9IjkKxHZs7u/wyQxH5Jr/ajw X-Google-Smtp-Source: ACHHUZ7o7RRQszFm1HXVdMdfa0qJvyR3iunWQYPPUhpuoM93jMBJspNdX6+1qOLtnIHGseC2dPP6/sg/rHO4 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6902:1185:b0:ba8:381b:f764 with SMTP id m5-20020a056902118500b00ba8381bf764mr887496ybu.3.1686757438966; Wed, 14 Jun 2023 08:43:58 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:56 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-5-mshavit@google.com> Subject: [PATCH v3 04/13] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084400_457689_88A2E45C X-CRM114-Status: GOOD ( 30.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Update arm_smmu_write_ctx_desc and downstream functions to be agnostic of the CD table's ownership. Note that in practice, it will only be called in cases where the table is owned by arm_smmu_master. Whether arm_smmu_write_ctx_desc will trigger a sync is also made part of the API. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. Note that the next commit ceases this sharing of the s1_cfg which makes that loop necessary. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 ++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 80 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 81 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c40..48fa8eb271a45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -45,10 +45,12 @@ static struct arm_smmu_ctx_desc * arm_smmu_share_asid(struct mm_struct *mm, u16 asid) { int ret; + unsigned long flags; u32 new_asid; struct arm_smmu_ctx_desc *cd; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; + struct arm_smmu_master *master; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -80,7 +82,11 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -211,6 +217,8 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_master *master; + unsigned long flags; mutex_lock(&sva_lock); if (smmu_mn->cleared) { @@ -222,7 +230,12 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, &quiet_cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -248,8 +261,10 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct mm_struct *mm) { int ret; + unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; + struct arm_smmu_master *master; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -279,7 +294,12 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - ret = arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (ret) goto err_put_notifier; @@ -296,15 +316,23 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) { + unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; if (!refcount_dec_and_test(&smmu_mn->refs)) return; list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, NULL); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d79c6ef5d6ed4..b6f7cf60f8f3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,14 +965,13 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_device *smmu; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_CFGI_CD, .cfgi = { @@ -981,16 +980,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, }, }; - cmds.num = 0; + if (!master) + return; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_streams; i++) { - cmd.cfgi.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + smmu = master->smmu; + cmds.num = 0; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1020,16 +1018,18 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg = &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg = &s1_cfg->cdcfg; - if (smmu_domain->s1_cfg.s1fmt == STRTAB_STE_0_S1FMT_LINEAR) + if (s1_cfg->s1fmt == STRTAB_STE_0_S1FMT_LINEAR) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; idx = ssid >> CTXDESC_SPLIT; @@ -1041,13 +1041,21 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, l1ptr = cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx = ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +/* + * master must be provided if a CD sync is required but may be null otherwise + * (such as when the CD table isn't inserted into the STE yet, or is about to + * be detached. + */ +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, + int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1065,10 +1073,10 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; - if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >= (1 << s1_cfg->s1cdmax))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr = arm_smmu_get_cd_ptr(smmu, s1_cfg, master, ssid); if (!cdptr) return -ENOMEM; @@ -1092,11 +1100,11 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, cdptr[3] = cpu_to_le64(cd->mair); /* - * STE is live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=1. + * STE may be live, and the SMMU might read dwords of this CD + * in any order. Ensure that it observes valid values before + * reading V=1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); val = cd->tcr | #ifdef __BIG_ENDIAN @@ -1108,7 +1116,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - if (smmu_domain->stall_enabled) + if (s1_cfg->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1122,7 +1130,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } @@ -1136,6 +1144,7 @@ static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, struct arm_smmu_ctx_desc_cfg *cdcfg = &cfg->cdcfg; cfg->s1cdmax = master->ssid_bits; + cfg->stall_enabled = master->stall_enabled; max_contexts = 1 << cfg->s1cdmax; if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2093,8 +2102,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - smmu_domain->stall_enabled = master->stall_enabled; - ret = arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2110,12 +2117,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret = arm_smmu_write_ctx_desc(smmu, cfg, + NULL /*Not attached to a master yet */, + 0, cd); if (ret) goto out_free_cd_tables; @@ -2386,6 +2390,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; master->ats_enabled = false; + if (master->s1_cfg) + arm_smmu_write_ctx_desc( + master->smmu, master->s1_cfg, + NULL /* Skip sync since we detach the CD table next*/, + 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; arm_smmu_install_ste_for_dev(master); @@ -2435,7 +2444,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = -EINVAL; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled != master->stall_enabled) { + smmu_domain->s1_cfg.stall_enabled != + master->stall_enabled) { ret = -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3c614fbe2b8b9..00a493442d6f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,6 +595,7 @@ struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; u8 s1fmt; u8 s1cdmax; + bool stall_enabled; }; struct arm_smmu_s2_cfg { @@ -713,7 +714,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -742,7 +742,9 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, From patchwork Wed Jun 14 15:41:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68D22EB64D8 for ; Wed, 14 Jun 2023 15:44:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=r6zyHZMWqV28JGiWluTi+luUdkSMLZhSz/aH0ROVJbE=; b=uyEw9WJmAVHh6KtNse+1Kwjmdr cF4gaIGdC+vfk4xMqiWzOhMZbyjjmZhPBkbYs4ny2Ejy/7nSewc87JBtoOoerd379s+GDd+vxDkuH Gj7mMhT5bRxt5oh1Tv/iRa2REKoRQsQmli7C0PvWAczMsMlFaHcGXNN/vWzykiI2J+7oELMbHlu+c v3Idc6dmOPjrR9doAmOFVUNK+O/+z1T0xFHwLzZMnO8Rw4LQKKd4U9uhULK9K7XyZwYmgRsaa9oJt ToQFJ/ESiGpwouB2eu1BC/HspORC1+3LRWAtuNTrLJ0pZqRK4VrRj08oMjgA/+XtElOX6jtmkOusJ SHpZ82qg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfM-00C6NM-1G; Wed, 14 Jun 2023 15:44:12 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfJ-00C6LB-0u for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:10 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc4c832c501so875450276.0 for ; Wed, 14 Jun 2023 08:44:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757447; x=1689349447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=tKbqq+SaTI8WzYXCxXw85rHSkOoIOr6eh4ii5g7z7oVm96FiPbCikxVlsDi1F+HwC2 dn1mWxIi51qsEvtY5sEZwXIhwhpWDJFjaT9RH2MzSRlK3QV+ElSp1EHRBAYKVhsynkLx iPVmHZjb2ZJzh6cYEahzlpwRheLpFuNbypKJcMzn2fsSBC53RC9koPOVxYRRQt57PIiU EK5hKMvKJc774Zr79U33ubKpVacDJqsAdVE/UJ70e3MNg8mtmbtuj4qIoQI/QoMhH6v3 JJV6hSq7MUNSZ7giHoLoLKjz57HKq8Z8nTIDRkVAF3+KsJV9QKIqSJkruMChmA6orr95 GT+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757447; x=1689349447; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=D6m/oLzCA4WEDQQxQYsF8et0+sN33zwNDnu6iVIdTFkjghzMVBeNzPKD0YzOpGuub6 1mfhQafstSZ5NyMer0SxKCph/unjLSBMryaSRVhGZ73GCejCjpRf2izAEJJgXj4WAJ2H lPDgZo7flE581i8nALVEs/+JNwaNDBfTVOp+la7XIK+AVEefWxBU/Ef99LxiXtNkNPa9 BN7Os8fKKUOnMl188otjr9gMv7/wleIdvu6gvx6Q9B42tve7jTSkikVf/eY0GxdljLey tx5ikNt/PIWRDBHVtVo0vaZCyWHwWiP3woEeckvSf26oqWtHfN4MI+KOHkMiX/oaDBg6 8TUQ== X-Gm-Message-State: AC+VfDyIcyUps0bS/UgG3JeBks7QrgiIVHebmexwzoRxWe3vub8iRWJq 9BXbPL/fo+NoCctM0sN0sS1G64t215+1 X-Google-Smtp-Source: ACHHUZ5EtGEW3+47VP69eKKpHTh+jEmHvp2V4beoSm1a7xc04JVeMxxNJra+AufYViBxpnByyMaU/fRQocWi X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:d001:0:b0:bcf:7f0b:e1d0 with SMTP id h1-20020a25d001000000b00bcf7f0be1d0mr328965ybg.0.1686757447045; Wed, 14 Jun 2023 08:44:07 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:57 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-6-mshavit@google.com> Subject: [PATCH v3 05/13] iommu/arm-smmu-v3: Use the master-owned s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084409_323745_E040BC22 X-CRM114-Status: GOOD ( 17.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Insert CDs for STAGE_1 domains into a CD table owned by the arm_smmu_master. Remove the CD table that was owned by arm_smmu_domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 -- 2 files changed, 12 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b6f7cf60f8f3d..08f440fe1da6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2065,12 +2065,8 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2082,14 +2078,13 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; u32 asid; struct arm_smmu_device *smmu = smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr = &pgtbl_cfg->arm_lpae_s1_cfg.tcr; @@ -2102,10 +2097,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (ret) goto out_unlock; - ret = arm_smmu_init_s1_cfg(master, cfg); - if (ret) - goto out_free_asid; - cd->asid = (u16)asid; cd->ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2117,19 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; - ret = arm_smmu_write_ctx_desc(smmu, cfg, - NULL /*Not attached to a master yet */, - 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2192,7 +2173,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + finalise_stage_fn = arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2439,20 +2420,20 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } else if (smmu_domain->smmu != smmu) { ret = -EINVAL; goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { - ret = -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && - smmu_domain->s1_cfg.stall_enabled != - master->stall_enabled) { - ret = -EINVAL; - goto out_unlock; } master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - master->s1_cfg = &smmu_domain->s1_cfg; + master->s1_cfg = &master->owned_s1_cfg; + ret = arm_smmu_write_ctx_desc( + smmu, + master->s1_cfg, NULL /*Not attached to a master yet */, + 0, &smmu_domain->cd); + if (ret) { + master->s1_cfg = NULL; + master->domain = NULL; + goto out_unlock; + } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { master->s2_cfg = &smmu_domain->s2_cfg; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 00a493442d6f9..dff0fa8345462 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -718,10 +718,7 @@ struct arm_smmu_domain { enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; - }; struct arm_smmu_s2_cfg s2_cfg; }; From patchwork Wed Jun 14 15:41:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1A14EB64D9 for ; Wed, 14 Jun 2023 15:44:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=nu9LBXnbxFHMhKav0NhMnxvs5X5YFBLA83ZGBy2Wu18=; b=Mu6hEl5PZNs7PZUk/gMPOV6lvU qo+b5gHUxmrjFNYjMLugEdSGtjtfWdjW79LKIkGcCsDAmcf3szaUgHiKM5Bttwjk/T3Q/R8vP/azo rpMLa2Pi8D2yyIG5/hMbBbn10rGe2rw5mowBozwWDoJfbN3DFR3TcVV9cj0wImKrDX8lzJ9qkmmXI y3S89UnJGFv5+SpYSAmxMCYOQ2trN5UT/9XFVG2XjxtPo8+hupyw9SnCcoWm5K7Utu3tdtOOpnvcN PRD+GpH3AzvdG8oWqJNhODqoBJmDZdxHK51umghkvriF7A47XmqlY9s+P6MLRysHLM5bba60xpSQ6 buhyHVEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfT-00C6Q0-0h; Wed, 14 Jun 2023 15:44:19 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfQ-00C6OU-2H for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:17 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-666693a71acso501850b3a.3 for ; Wed, 14 Jun 2023 08:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757455; x=1689349455; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=ZreyidDpJbP1PBXH0/670/kPY2SsUh00/efeS6mDpY0HajYSURNuwl2jgv15mHAY+8 BQnjoS82KZhUCP+ZW8JKdnnpOoil/pxglVP0s2csLwMNXQxHU51EaIYCRwcdKqquCh0q 3bWW4wrsmrxasyohwzBecrXf6yKsHWQhn0bySkpCPo4xUAgaHO41cs19iiK6VFDnmGcL 1z3TlJElBou8TAuRPKhYnjDJwUUqq75L0bf3w6VNMdMUoFGls32SxQ9iHK9xTnV32fN7 YIabDW9594lEZ4404D02Bxq7KeKnTkNz75qnKx2pKgIU+oWiPHE0kF1iXqTgciI2k5V0 H35Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757455; x=1689349455; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=YvwmRHKS4mFSqzMdWVMR4NxUEJDQe+f/1OryAUkF0Zr/q+pF4iHBScTnCiIOYnnA/Z vI6hZ9NqIl8wz4RN/ScsSe0KC5myEJOrtGBeOJ0TCp9eb8ZUArc8mne+Dr6rFTg+96UM NRrlg2MNpbnhbI7Q4GKJmFuMvA9Kp85+tSdUxtQZIsQD9XmTTaUAgI4BqtufbUO82gEY zxFtccid/vN9UHQtIcB/3aajDU2MHAJYRXDEXNDs14JtPZSUF3fr7/WQj4FyB4qoHdXC zxU+9eruOdwebkT6iOMbSLPZfVx+PnZUQnbCR693ws6QbJhwFeyXsz6/6/G2Mq7gdOsz KgGQ== X-Gm-Message-State: AC+VfDwaBXH2n9UcIxBbb1cIfRh3C2RZGCZVc+BMeuWla+sHsTxyUE0Z dO4nG3F/0yLCp79i2/1iqVylekZ49pxC X-Google-Smtp-Source: ACHHUZ5hvf161DqxDEUjciWGgh2uDlHkYPKEsVFxC66gt2tXnPwcs58oZC95f7+2kSAq6UzNy1BZjwtxJnV7 X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6a00:2e99:b0:654:424b:17d2 with SMTP id fd25-20020a056a002e9900b00654424b17d2mr664816pfb.2.1686757454862; Wed, 14 Jun 2023 08:44:14 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:58 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-7-mshavit@google.com> Subject: [PATCH v3 06/13] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084416_744675_80D6CD06 X-CRM114-Status: GOOD ( 12.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- v1->v2: Fix commit message wrapping --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 08f440fe1da6d..dc7a59e87a2b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2286,7 +2286,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) pdev = to_pci_dev(master->dev); atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } From patchwork Wed Jun 14 15:41:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9673FEB64D8 for ; Wed, 14 Jun 2023 15:44:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=0C5B7jl5++L4KKyr7klQuEq04tz9ZKvy+SYR0DL6/fE=; b=HyotOUq32JCIfE53rWTvTwldcT 5yHSs8JQMEnAXGISCjJIfRQIAXv6tCe9lCGLkno8ICsU15xdVVYL9xWOFTotzyYtDEvhzNQT+dxhB O/ialXw1suj4P+ybv4h2TJz88fBHD3YRSoLwuGE84m0lowfLpGZ+F9Hj7NtdS2sQky2InVVBJ2Zw+ PkmX7n+0S/tsg5Lvl9yN22w1snRhW/q8sLMYV+Bu2enJmuoBxbfwWAVl/4CjgI6OVaRrX2MoC2FZe 7BHk3Hvuvk0zzFXdTIIVrdylUAdEFLAXXhn5e3LCDfvqFczRRl3I7W/uP1d+oeKhyaJepqnAp4jJT dICY3JBg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfa-00C6Tc-32; Wed, 14 Jun 2023 15:44:26 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9SfY-00C6S6-0G for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:26 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5695f6ebd85so9930047b3.3 for ; Wed, 14 Jun 2023 08:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757462; x=1689349462; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nXG6hIyHVJ6VKFy81TWuK5UaUuQM8CF9N4uqi7zpqBY=; b=R7FYyPY8TwvB20rCtxYPY/uQ5V5mDRkEHmhZfOb9KmTXHRM7eCTlVaQLZTxHDxNfqQ NmcTodCbjXklTQtaW9342WzLO/qDXql2oLQDMgBHUYeCESt+DxwMmAfoRk5gB5eHnSN6 VNGNlCaRnEjOw7yP6Q7EfZPKHD8LfWOhNKARyLC2efpqVESxw7rxqoSYKKlEystXOSoG 9zY7HcKK7aUsKlwsobKQjbGSkVDpSqiBA1ykeBjIrXXiTSGJZLGbiGlL2z9pJ5yWw9tQ PUyzvANopNaCfONGGpI3ju5/ofPKh0JCJ82lCwIX+Foujvvhvrmb1qY7e8P6edZ6x9Mc t1lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757462; x=1689349462; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nXG6hIyHVJ6VKFy81TWuK5UaUuQM8CF9N4uqi7zpqBY=; b=KQt4E+XdtO5WqqkcJPVGAJFOM+9GzvdVWP3NGbPI5t/w5mx7rujUxugKziSsNPSQw8 D8VHYbGAzcAWoSEnsWLzxzOyokhLXpeL4lBS6njzkUpDg/tOA2pC9MUKySkF2A/ldk8F WFXyCRa7o01lRZ25Yrr4kMR1m1RupiYsD56w/QIHj01mHHJ6EsFD8N+LanG33mwPLZ2b xlYT6Eo5r4kJ5EirCSh/bxPugAedBYx01F5GFjmMJq4AKI5773jOEzMkBIVqJ+hfap1R i0WP93h5KaTF6YYc7mM/K8OD2GSTWc6FzOdIf7PXMFhWVX7NXEHnsNwCDD7nk5HruHQI c8zg== X-Gm-Message-State: AC+VfDx0Lh1jM9uLCbv+mxWaZ54Ng3WErzcEAl4zwiUtX4LDNT3RB/uF XDlnoHApLfZpGcEfVteQmg4ZnGoeS1rD X-Google-Smtp-Source: ACHHUZ4TGsqLNiR6hBlfswNfYTLUZad9nG7u8NFLZlv9iskT4PA7Nu2Z3tNlM0v8J1A0IZywoYJb6DlEVGAN X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:690c:711:b0:56d:1f95:fde8 with SMTP id bs17-20020a05690c071100b0056d1f95fde8mr996359ywb.4.1686757462821; Wed, 14 Jun 2023 08:44:22 -0700 (PDT) Date: Wed, 14 Jun 2023 23:41:59 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-8-mshavit@google.com> Subject: [PATCH v3 07/13] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084424_124269_E852FF8B X-CRM114-Status: GOOD ( 26.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- v1->v2: Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 53 +++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 88 ++++++++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 ++-- 3 files changed, 105 insertions(+), 54 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 48fa8eb271a45..d07c08b53c5cf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -51,6 +51,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; cd = xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -82,11 +83,14 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, + attached_domain->ssid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -210,7 +214,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -218,6 +222,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; unsigned long flags; mutex_lock(&sva_lock); @@ -230,15 +235,21 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; + /* + * SVA domains piggyback on the attached_domain with SSID 0. + */ + if (attached_domain->ssid == 0) + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, &quiet_cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -265,6 +276,7 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm == mm) { @@ -294,12 +306,14 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, goto err_free_cd; } - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); if (ret) goto err_put_notifier; @@ -319,6 +333,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) unsigned long flags; struct mm_struct *mm = smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd = smmu_mn->cd; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain = smmu_mn->domain; @@ -327,12 +342,14 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) list_del(&smmu_mn->list); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, NULL); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); /* * If we went through clear(), we've already invalidated, and no @@ -340,7 +357,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index dc7a59e87a2b4..70580ba7065dc 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1711,7 +1711,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) } static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid = !!ssid; + cmd->atc.ssid = ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1736,8 +1743,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, */ *cmd = (struct arm_smmu_cmdq_ent) { .opcode = CMDQ_OP_ATC_INV, - .substream_valid = !!ssid, - .atc.ssid = ssid, + .substream_valid = false, + .atc.ssid = 0, }; if (!size) { @@ -1783,8 +1790,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1794,13 +1800,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instead of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1823,25 +1835,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); cmds.num = 0; - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master = attached_domain->master; if (!master->ats_enabled) continue; + if (ssid != 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1863,7 +1887,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1951,7 +1975,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2031,8 +2055,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return NULL; mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return &smmu_domain->domain; @@ -2270,12 +2294,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - struct arm_smmu_domain *smmu_domain = master->domain; /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2291,10 +2315,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain = master->domain; - if (!master->ats_enabled) return; @@ -2358,18 +2381,17 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain = master->domain; + struct arm_smmu_domain *smmu_domain = master->non_pasid_domain.domain; if (!smmu_domain) return; - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_del(&master->non_pasid_domain.domain_head); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - master->domain = NULL; master->ats_enabled = false; if (master->s1_cfg) arm_smmu_write_ctx_desc( @@ -2378,6 +2400,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) 0, NULL); master->s1_cfg = NULL; master->s2_cfg = NULL; + master->non_pasid_domain.domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -2422,7 +2445,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - master->domain = smmu_domain; if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2449,13 +2471,17 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled = arm_smmu_ats_supported(master); + master->non_pasid_domain.master = master; + master->non_pasid_domain.domain = smmu_domain; + master->non_pasid_domain.ssid = 0; arm_smmu_install_ste_for_dev(master); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&master->non_pasid_domain.domain_head, + &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); out_unlock: mutex_unlock(&smmu_domain->init_mutex); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dff0fa8345462..6929590530367 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -682,11 +682,19 @@ struct arm_smmu_stream { struct rb_node node; }; +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head domain_head; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; @@ -724,8 +732,8 @@ struct arm_smmu_domain { struct iommu_domain domain; - struct list_head devices; - spinlock_t devices_lock; + struct list_head attached_domains; + spinlock_t attached_domains_lock; struct list_head mmu_notifiers; }; @@ -748,8 +756,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int ssid, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Wed Jun 14 15:42:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81658EB64D9 for ; Wed, 14 Jun 2023 15:44:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=26Ujbaow6bIa/GIJ5AX/YW9ezaQgHnx9p1xwYKzOPRY=; b=GDmx0YX4FU4akRrn9AbJestkJt Ajlm/Hee/lVs1h8z2A1JhujqcbycYXFpCSH+sn+1zZyvJmC+uxH9USmKuPLfUtY6gL0iPwXYNha0+ 6ByW5uyJmr0+JS5ukQve10PNO2Lr7thYHdlMPP9I0i3ue4VDQvszubAF6oiem2qOdU2LB2m6ibkD3 6ZETqMaV6Yk57byfANFaOfUDptI0iMNzjKV0w1KrPp4p7vqBXfn9now3P0twaLKgayXXX9U8q8cg6 DM4iRve9bV+j3H5msV7E/17YFAVeZkJrpCfJnmdS7VSWz/vV5T3YaPTk43oojHOgcsvAndNugO7yj 9Rvd/+Hg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfj-00C6Xf-2c; Wed, 14 Jun 2023 15:44:35 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfh-00C6VP-0Q for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:34 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-565d1b86a64so11168867b3.3 for ; Wed, 14 Jun 2023 08:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757471; x=1689349471; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=omVhI9NBPuwnuDBL2dAuwy0d7VVlvqTslf6ohlhvBoo=; b=jMlnk/MKenoGoe+vncInH3bNLbjsOY0Xrv5py/yEVhJC0Voa9L6CSXUuAVOi6fmYCJ Tgf0uByedAMjl4kDugva5Z7ex59EM1z4IazeM7oUL8ziiQDhW3iZaQ0JG9Qm40ngiaZL +M5Dg5le9q5NHjjTl7k9QZk59/KC3EuCDrBCsL0bF4lycPrvzpsN0P/LhUKYz/tUyV0x isJj8aPRjzh8meDpJOrtJNeFlx8yvD55tCSLvcLBEnDtcXfsEbJZtuB1BwtuBcopqymr /Rvz8IKzzGTFog5Cr0gOu/QV8fyEzYweTi1tUgmgmlB4M3eES0bWGS4bsAPec9Qi8ALG 4M8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757471; x=1689349471; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=omVhI9NBPuwnuDBL2dAuwy0d7VVlvqTslf6ohlhvBoo=; b=RSZGTa0hO7tjA8o5OdyiUI/L07P10rOCfif/LJfJ7yXLy+LWpnDrnK9vhGnwOnrJbI WZeACOCIY9+EVh92fRzM8XzNV3jfC42bNo85qOGoM0CVhUSrZrd3tPoppXZ+gCiOq69O ccLUJSYwB5TjugrU+smtJ3RJ4Bgotr0YibIHshXfGXwX2xojpYxW308vyFX1bl44N6/0 i/FAtv6tsdzMEe4n8y7J+aTZ9mgiujyLySy9OWNSHGoFjyaFpBhAT0hbj+5w5ff6cr7f Cg+xjC/dNuHFm6l7Kkbv78/frd6Gx7D0ZulgKjX/2cng4H6yX46l7eGgznNRdXQRGCvM Ajkw== X-Gm-Message-State: AC+VfDwE6sxugQe6i6yqQlCT22tbLuxT3k5FOrAf8MG9b2WWqcTaMDXb imOhBXIugCqpfdY25zigXhFh1eSl4WVE X-Google-Smtp-Source: ACHHUZ6qWrQixYgUn/5HtahtJSL3blxJF6JctRnT69qFj5a4lL50PstwAlMR8K47HQ216a2/pWujMLt858mE X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a81:4426:0:b0:568:c4ea:ce66 with SMTP id r38-20020a814426000000b00568c4eace66mr971778ywa.5.1686757470875; Wed, 14 Jun 2023 08:44:30 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:00 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-9-mshavit@google.com> Subject: [PATCH v3 08/13] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084433_170811_511AFEDF X-CRM114-Status: GOOD ( 14.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- v1->v2: Make use of arm_smmu_atc_inv_cmd_set_ssid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 70580ba7065dc..176013bb974b8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1784,13 +1784,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -1800,6 +1802,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instead of * the one the domain is attached to. This is used by SVA since it's pasid From patchwork Wed Jun 14 15:42:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 872D6EB64D9 for ; Wed, 14 Jun 2023 15:45:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=z5l+qnRWHbcac1dzRjfxJy+msX+0lAgmYJQu+M+2XDA=; b=fICMe4h553rEnjAMHkcRv1nTCP foDEHLjfDlOeMpSSW48btORw3fG39bGmuiMH0JMHSV4ZQaN0y6lZzH+cnSB9vxI1RtE/JBg4n5cB9 6hH7BwXJjGlGAqF0mQc+InYWTmZtwjW8yphkNXOjsIS8JM/yUb1J/WvFkyp3N+90iHvoEof/TBiEr 8fSwv6DO13vwGmz5kl2cltocyKB2wHOnupcbiMRi0WpHgA2SQ32ESIgGbCPJ+djv0R3bXq6hQJTiu 6FU38dlv8pi4v5Q906r/TQ0oSuzsyCmJUR/JDU2iY23Z9eo3SL3kSm3PDRpc3Y4oalccrSYX1JlBk oWd48MSA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfr-00C6bv-1s; Wed, 14 Jun 2023 15:44:43 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfn-00C6ZT-2o for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:42 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-653496acd69so6215352b3a.2 for ; Wed, 14 Jun 2023 08:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757478; x=1689349478; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=zp8WWvGOizfWW9ZsurLF24HEAfBobFq4bOmSr8VWFOI=; b=P33CVxX4esw3dBSK5S17bbxhmwwXbDXFc5BNId/SskkIfaAcUMv/viU+8T/DGWsJKX W9bktuoWVfSWVcE8R8uDRfAvQ4J8cBGtPOqgyU2/iC9WsioU9cykfvXPVsg70c2BTtRO Gn53yG3SbQkxPaXuBcq5feCokxIsGkfeR1AnKvxlwKGyvIJf18/P/bs0RKpu0Zo9BCVG RFqiWZSuLLIGNb+GzEW5JjCnRYL4iGTuffbe32LPPmJP0d7z/jH5V0+foHmxOBBmhJ2e B2976zpIyp5dSuDIvhNrhiaSnDq4OVl9P5ifH6jbPCuE4HSZXUETXYqnZp7CStHkqLdk cEKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757478; x=1689349478; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zp8WWvGOizfWW9ZsurLF24HEAfBobFq4bOmSr8VWFOI=; b=Ugs+kBsKxCSKg+Ol4fq9hOhF1edjiXs3aAhEd/DyQRjJAqUviUtH1VKavQbAVeZFYA u0FhoJyIY/H+1OZkIZPb2eZPvN3yMhNmszYU6fOsnNiN/SzLqTI1yyRge81iQ0ag7mRk EP/3aZDOdz3C5fibznwEj79+UYiGhWgz5rJPzdacG4QgUAkCNafhfn2ykn2e3SAHmj3u HvPRdV4sKU1vmxldZE3J4klR7G9mrm6qSZtCitvJ5AmB7pO9QxeKhunCXso34wqQ0Wcg Rb9DcpMH7wqiWaIuO5yg2rjQSiwMbZ1Oo1jJ+IIufTWC5L2eIo9+FQ7qtaoxrRIyvsO8 IC0w== X-Gm-Message-State: AC+VfDyHfit6fWwUn86JpdjsHJfLOJMUDBlekJLhjedOoWuXrqPZ+/5o 93p4m73aj+AawWwl1Ev97afOYpVw3I/r X-Google-Smtp-Source: ACHHUZ7oWcQmm2W8o0FHztY9pz4yhQmPGX8vnYdXZ3QyFdJYkZqNWjsWL87VFkksLupGG9fw4PhBms4uMnVQ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6a00:1513:b0:64f:c0b1:6967 with SMTP id q19-20020a056a00151300b0064fc0b16967mr682372pfu.1.1686757478685; Wed, 14 Jun 2023 08:44:38 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:01 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-10-mshavit@google.com> Subject: [PATCH v3 09/13] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084440_199082_4BC3250A X-CRM114-Status: GOOD ( 26.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- v1->v2: Add missing atc invalidation when detaching with pasid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 168 +++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 149 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 176013bb974b8..a6fa56585c219 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,6 +2173,10 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } +/* + * master may be null for domain types that are finalized before being attached + * to a master. + */ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct arm_smmu_master *master) { @@ -2369,6 +2373,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master) return 0; } +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *master) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2411,6 +2420,28 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_install_ste_for_dev(master); } +/* + * Once attached for the first time, a domain can no longer be attached to any + * master with a distinct upstream SMMU. + */ +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain) +{ + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu = smmu; + ret = arm_smmu_domain_finalise(&smmu_domain->domain, NULL); + if (ret) + smmu_domain->smmu = NULL; + } else if (smmu_domain->smmu != smmu) { + ret = -EINVAL; + } + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; @@ -2426,6 +2457,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2436,22 +2471,18 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain, master); - if (ret) { - smmu_domain->smmu = NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu != smmu) { - ret = -EINVAL; - goto out_unlock; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; } + arm_smmu_detach_dev(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { master->s1_cfg = &master->owned_s1_cfg; ret = arm_smmu_write_ctx_desc( @@ -2460,8 +2491,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg = NULL; - master->domain = NULL; - goto out_unlock; + return ret; } } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 || smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { @@ -2490,11 +2520,75 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_enable_ats(master, smmu_domain); -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); return ret; } +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret = 0; + unsigned long flags; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master = dev_iommu_priv_get(dev); + smmu = master->smmu; + + ret = arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + + if (pasid == 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->s1_cfg || master->s2_cfg) + return -EBUSY; + + attached_domain = kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master = master; + attached_domain->domain = smmu_domain; + attached_domain->ssid = pasid; + + master->nr_attached_pasid_domains += 1; + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + } + + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&attached_domain->domain_head, &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2740,6 +2834,15 @@ static void arm_smmu_release_device(struct device *dev) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains != 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); @@ -2875,12 +2978,36 @@ static int arm_smmu_def_domain_type(struct device *dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; - domain = iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid == 0) + return; + + domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type == IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, domain_head) { + if (attached_domain->master != master || + attached_domain->ssid != pasid) + continue; + list_del(&attached_domain->domain_head); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, pasid, + NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + master->nr_attached_pasid_domains -= 1; + mutex_unlock(&arm_smmu_asid_lock); } static struct iommu_ops arm_smmu_ops = { @@ -2900,6 +3027,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6929590530367..48795a7287b69 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -707,6 +707,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; /* SMMU private data for an IOMMU domain */ From patchwork Wed Jun 14 15:42:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280162 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 159AFEB64D8 for ; Wed, 14 Jun 2023 15:45:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FUDd86Bbj/aoLrC28fhzeglulWpVPOmujdqZLoA4HpU=; b=PeQo3sLhyCT9jvsRGAxLkvPBKz lAdUraOyx8mSyu/+szM5dxEJMfQ6nH3udTWiR2dNasgit15tiY48YuFMd6PAlJ08JzzVVQLxZuau5 7El1EhzlJsPks/RKZOO28hglJW1kStqmpo2Wauzv2maaQSjK4IFjP1WlocbzwdQKg25n9irNo1lLn bhnxzpExE7LXCY0FhPEj6Xck93IiFk4abM25HFL8xJLJR5E/rZei28jeKSYamIxBXtM0J5+VKpVh0 +vEea3aqPbFFMD06gJVugfQwGvMQ8B1bVC/Ntp4CreL3KbGCnc49477KY2VTN87iVBo1nj3SgCZCq tJd6qjkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfx-00C6fg-39; Wed, 14 Jun 2023 15:44:49 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sfv-00C6e6-2Z for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:49 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-561ceb5b584so12010717b3.3 for ; Wed, 14 Jun 2023 08:44:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757486; x=1689349486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=vuL1XrQpU5Uq0e2ygjk0DfQm99VPq2z9YNT+avIv27XZ5urDEgwpmKbViVzjQ8Ors2 T0rraZDZVM0ODSgkQTw6g6hdTWSyZHC/ZwHObRZETsh7v9KsqI7mfjRVjTddqwb75Yd6 PkKi9NkGSgecoW5IThOwn0gGyOxQIy0qaPS8Ued69ldHUGXC3bEZuqzTL/jNRXCrcJi3 uAxDiKxWDeKXd8Icj/EZsPQK5O6GRp7wJwAZct0hM+oyxo3a3VP2+SBs18GWULEH6AcA +Q3vGN3bOvN8pQVCjBFeK+N8axcpPszN+TMygeq+12fShtLMkNw5EOWGCZf8F5q4Py43 k0CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757486; x=1689349486; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=hqz9SbE1Y1B1nbuLB1X1aYZTy1MfzB1FweWaP2RRrAihw4AZ4brk7GdQmevTCqugE6 QqJebOpZuyrDL2sofu9wuL6zEjbm8BMDfw8iXCfDIXJ6I4dTIXiKbe6D64FTTY0cl0KQ N9vH6h7IiX7hV7xH6/Y0xkoXOwH91CATUHeAq9gn2I7Bdg8hO+kKCwqkJwwiiYS2C+RD dwv0HGYxY0hESUFNj8Z6szJNtbcKpOvm9JCiEvd0zHNsMW7IStu3+PgB5lBjgKbNP5LU jFy5eNHeMZRsJEjIAdk1UiVr/LPU1q5To1P8Ppcf9lfP1NwB3DrRFX9BHvBnFhqev8Ra 8HvA== X-Gm-Message-State: AC+VfDyznxcItNkMxV8yrBDNcQBbfZNnhcRgmtgFC1brPs2vccGCUdk3 oiMlFIV6ZahM0FCwHQ+0kUrXz1avA6Zg X-Google-Smtp-Source: ACHHUZ6PWgAWHwE/x5mre24ZvBjK+WBUn1h6Ifenl3MHyaJYnWLa0SO3LQyqPPJbjhH7MP8WEYsBnUmWbutp X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a81:440c:0:b0:56d:791:d1a4 with SMTP id r12-20020a81440c000000b0056d0791d1a4mr1051771ywa.7.1686757486567; Wed, 14 Jun 2023 08:44:46 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:02 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-11-mshavit@google.com> Subject: [PATCH v3 10/13] iommu/arm-smmu-v3-sva: Remove bond refcount From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084447_833321_75DC1D73 X-CRM114-Status: GOOD ( 13.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The iommu-sva framework checks if a bond between a device and mm already exists and handles refcounting at the iommu_domain level. __arm_smmu_sva_bind is therefore only called once for a device/mm pair. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d07c08b53c5cf..20301d0a2c0b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -29,7 +29,6 @@ struct arm_smmu_bond { struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; - refcount_t refs; }; #define sva_to_bond(handle) \ @@ -377,21 +376,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return ERR_PTR(-ENODEV); - /* If bind() was already called for this {dev, mm} pair, reuse it. */ - list_for_each_entry(bond, &master->bonds, list) { - if (bond->mm == mm) { - refcount_inc(&bond->refs); - return &bond->sva; - } - } - bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) return ERR_PTR(-ENOMEM); bond->mm = mm; bond->sva.dev = dev; - refcount_set(&bond->refs, 1); bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -570,7 +560,7 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, } } - if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + if (!WARN_ON(!bond)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); kfree(bond); From patchwork Wed Jun 14 15:42:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60C87EB64D9 for ; Wed, 14 Jun 2023 15:45:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Wp1cI66UKYVPWSunLOP/qahTo4eKMI5Xwo57UeR3b2k=; b=UdAvjerS55pWdf6azvD2go0OiQ HbzEOH0f2jZSdBXtfmsSRRd97vILyq1JR8b0pmk3fdXcMW/TqnaSFQCdYP48a8+am6mMnQJIZ7pU1 a29lUyd7VdhhJT/JH10bWJcbo+gcOMQLBYIIfdseR5MbPFUc1OQddRR59G+C+6UCmkQRf47oH8uTe 04zRpHoHxVScFqYXpox2UzPjg1SPUu6ewoJDTTxKriavq38EzHAj1SmUFrHU1Anp894nM6Xp+6p0D oJOu8Chu5SK2S1k3R0BwKoeyjRUWx06wldC2l/LUOimkEYKDwSnvArwDmbRK+KusNn1ifrEZaLyva IIybCNmA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sg6-00C6l1-39; Wed, 14 Jun 2023 15:44:58 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9Sg3-00C6iC-2d for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:44:57 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bc8ea14f4eeso1003449276.0 for ; Wed, 14 Jun 2023 08:44:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757494; x=1689349494; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=miKKuVOqy3O5VFZtsBJw0DWAr4nJ6ea8nlIylue8vHaIsdYyKs5VFM0KTJacu51Qzm +DH3Zl9jsvtSJPHee4tayDVENGPTKWf9gWXrcL4hC1Sc2GgiVQBtqQ7FnZG8+YwqkBPh yA8Et/DjbQCMxnJvenLQiNf7A7GFBipkyxcRcbX25mPlSQAORH+f6lepCF7STCNnbSjz QEVoAOVVQy/KvCuZreeHmBUtlj6a0/QT8lJ7Ls6W5a6m4+Xc6bUdeRYUvj91Jh2RQGIL 1UcOZtIRMZQKKjjJou6pQavpNlTolKJ9ovsaIENZITg6nYLeQ/1+rErxMkqfwI2pqIqu 2vrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757494; x=1689349494; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=EglSCBsK9Wv21RGsOW6v5cc6WpsEHUBAY5RY+Nq9CGLgh2VepxVDgbZseSRJRAAisM qn2NCzRLHQocPnxs7jiUUXvJXl+G9rVaa7zv4dALubb0FqAi5Eyz9IY8tm+cJM5uvyGe ZTYAjw9+bsFSE4R3MYf39kzx/Z1XUmWdl7vYvWhwMenC6U57HhLzLRgC88+CO2dxajDj LMNTiB6SE2P9hA2GlGuqJWwwZaumoKf6IdMnAGVJS6mncAH+6/br3ZuU9P+ghDzEBweK WsBne8d+CNz3oZs/UoS29z9L68puvUyOxXYCf9YJOSjX/dFgA6GrIJLFFEL/WyJVMk1+ 63yQ== X-Gm-Message-State: AC+VfDyQBzUKJns4k/IIzJfsYCGAMSC8ebTQgoI9KiGQv1cB4lPo4mpo RnAO4lKDFvuE2JwZEgLzZ372OiBcvIpu X-Google-Smtp-Source: ACHHUZ78kKfpFZreseZ4ojfWuvi9sDnPV/0PlekSvb+T1K6vSEDreVSyrSqoVe2hWBnfSGsU5TgB/jwAdAGz X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:7356:0:b0:bc7:f6af:8cff with SMTP id o83-20020a257356000000b00bc7f6af8cffmr1176382ybc.2.1686757494488; Wed, 14 Jun 2023 08:44:54 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:03 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-12-mshavit@google.com> Subject: [PATCH v3 11/13] iommu/arm-smmu-v3-sva: Clean unused iommu_sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084455_860505_0094754F X-CRM114-Status: GOOD ( 13.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The __arm_smmu_sva_bind function returned an unused iommu_sva handle that can be removed. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 20301d0a2c0b0..650c9c9ad52f1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -25,7 +25,6 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) struct arm_smmu_bond { - struct iommu_sva sva; struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; @@ -364,8 +363,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static struct iommu_sva * -__arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { int ret; struct arm_smmu_bond *bond; @@ -374,14 +372,13 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); + return -ENODEV; bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return ERR_PTR(-ENOMEM); + return -ENOMEM; bond->mm = mm; - bond->sva.dev = dev; bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -390,11 +387,11 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) } list_add(&bond->list, &master->bonds); - return &bond->sva; + return 0; err_free_bond: kfree(bond); - return ERR_PTR(ret); + return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -572,13 +569,10 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { int ret = 0; - struct iommu_sva *handle; struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - handle = __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(handle)) - ret = PTR_ERR(handle); + ret = __arm_smmu_sva_bind(dev, mm); mutex_unlock(&sva_lock); return ret; From patchwork Wed Jun 14 15:42:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280164 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2014CEB64D8 for ; Wed, 14 Jun 2023 15:45:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=BnN9luZBC+der3Py2JmbOgRrAxSLYHCiduqBokkbWoM=; b=Mlyg3G4L6dInc1zcYGBQ+O5muu /hYsw9J96Kn1ITw3JoqR7iSRCFVnbPpRdBS4eNx0JdQIA+XQ/hpaGSZU3IL/Xx6PyGL/AoBOgO3s8 UTsc7R9nKFw4t7VZxM4Y/0Un0yENkZqSo3MoXzpwaIqCzJQ8HC5SAKZB0Iyw0OobuWuCp5psyAfmA /BmcExkV5VQpWZ6Pwh2wZCLNKr2U8QdOo2I/RKxSIQu0AZ4WVXeJLhSKhYVmhORi/pFt6k2FfRtlb CARTcE3pw9YLOX3WkQQ2BWRY1+qUPQJos45lBWrJvAIq1gPe93eUI4S+tzidohdp0cePgrFsJ6kC6 /PHyB5KQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9SgE-00C6oq-05; Wed, 14 Jun 2023 15:45:06 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9SgB-00C6nQ-23 for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:45:05 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-570022400b9so10162687b3.0 for ; Wed, 14 Jun 2023 08:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757502; x=1689349502; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rIYz1+kUM82lpQwp18pWI6n2YGQKaG9udPtQlpxJvZA=; b=ITLTLTaLCsc9HGbm9o39sZh1FY+PCkOK3DqheGkhRPBC/5m4sMot8zJeqp6fvfWE1r t3T//BbToTEl9NchoFS4gdxcOle/PsfltFonLoaAgA2DAOWZERwJ7kzdxMSY2fW6ZZ8y pY+tDn0S1BbfL62uKyPVDPJuYg7OXqEm7XeJ9+DUWbpMulT67bfgJEo9dgHiDVDe4FGN rscahF06lVNVc5DgWu9mjCNiD2g7elxaGjWG9gVInQls6qTkbGhdTgpJbQc97OTMvGT9 oNdhtFouxQm4yOzmlFFhwPGSPEOZr2p+ZrgnJzHBF4mAXsbdtpPsvzu8r/CxQgvc5R+4 6r/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757502; x=1689349502; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rIYz1+kUM82lpQwp18pWI6n2YGQKaG9udPtQlpxJvZA=; b=W4qB3nwLwUwQpxq06U17LIcfJQYEbmYJb2LeUW/XkdZZG0MH17fs2+cL7jVM+7/zNI 7WkHseI7k+bo46EcoIbZ/0NyBH72qpR0/EWPWWou7a9BOkS+3JaudEV1FXSWHpScURl6 IcUjbcRRyOSG1ML4NpcEfyLHNV40aoYANT66cfOPBcaQwABF4BNW7aD8rtxF4GxHCz22 IhC104PHml5idAxnE7uToEzaaSXnkiFwLmlfc5kRLJY2hlhcm8fDmBNQRNGDRthxhCNl chsvKMd5t81XnZfgU85jslLVxrNtJf+bGJDpTP+u08IDh+17AKMQifXKQARz0Zg0LiJQ 8uYQ== X-Gm-Message-State: AC+VfDyi1ROfe1JGn4GwPXNcTu7A8GTYGFXWIZNb/K+EzkoItLtD+PSN nLO3FIZKE7Khnmom28r/4F0JX8GnKiPI X-Google-Smtp-Source: ACHHUZ7a8fpXcjdm9ERgwJcGejxZs5tL9yslpX+iW/oP0DV08UZci/UMc92nS9z9HRLv/d8ZvhGulclwoYLQ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a25:ab85:0:b0:ba8:4ba3:5b54 with SMTP id v5-20020a25ab85000000b00ba84ba35b54mr311468ybi.11.1686757502482; Wed, 14 Jun 2023 08:45:02 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:04 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-13-mshavit@google.com> Subject: [PATCH v3 12/13] iommu/arm-smmu-v3-sva: Remove arm_smmu_bond From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084503_693443_41AEFF17 X-CRM114-Status: GOOD ( 19.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There's a 1:1 relationship between arm_smmu_bond and the iommu_domain used in set_dev_pasid/remove_dev_pasid. arm_smmu_bond has become an unnecessary complication. It's more natural to store any needed information at the iommu_domain container level. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 24 insertions(+), 48 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 650c9c9ad52f1..b615a85e6a54e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -24,14 +24,13 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) -struct arm_smmu_bond { - struct mm_struct *mm; +struct arm_smmu_sva_domain { + struct iommu_domain iommu_domain; struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; }; -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) +#define to_sva_domain(domain) \ + container_of(domain, struct arm_smmu_sva_domain, iommu_domain) static DEFINE_MUTEX(sva_lock); @@ -363,10 +362,10 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, + struct arm_smmu_sva_domain *sva_domain, + struct mm_struct *mm) { - int ret; - struct arm_smmu_bond *bond; struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -374,24 +373,14 @@ static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) if (!master || !master->sva_enabled) return -ENODEV; - bond = kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return -ENOMEM; - - bond->mm = mm; - - bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret = PTR_ERR(bond->smmu_mn); - goto err_free_bond; + sva_domain->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, + mm); + if (IS_ERR(sva_domain->smmu_mn)) { + sva_domain->smmu_mn = NULL; + return PTR_ERR(sva_domain->smmu_mn); } - - list_add(&bond->list, &master->bonds); + master->nr_attached_sva_domains += 1; return 0; - -err_free_bond: - kfree(bond); - return ret; } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -521,7 +510,7 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { + if (master->nr_attached_sva_domains != 0) { dev_err(master->dev, "cannot disable SVA, device is bound\n"); mutex_unlock(&sva_lock); return -EBUSY; @@ -545,23 +534,12 @@ void arm_smmu_sva_notifier_synchronize(void) void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond = NULL, *t; + struct arm_smmu_sva_domain *sva_domain = to_sva_domain(domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm == mm) { - bond = t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } + master->nr_attached_sva_domains -= 1; + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); mutex_unlock(&sva_lock); } @@ -572,7 +550,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct mm_struct *mm = domain->mm; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, mm); + ret = __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); mutex_unlock(&sva_lock); return ret; @@ -590,12 +568,11 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { struct iommu_domain *arm_smmu_sva_domain_alloc(void) { - struct iommu_domain *domain; + struct arm_smmu_sva_domain *sva_domain; - domain = kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) + sva_domain = kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) return NULL; - domain->ops = &arm_smmu_sva_domain_ops; - - return domain; + sva_domain->iommu_domain.ops = &arm_smmu_sva_domain_ops; + return &sva_domain->iommu_domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a6fa56585c219..b7f834dde85d1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2784,7 +2784,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 48795a7287b69..3525d60668c23 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,7 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; + unsigned int nr_attached_sva_domains; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; }; From patchwork Wed Jun 14 15:42:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13280165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74FF8EB64D9 for ; Wed, 14 Jun 2023 15:45:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=QJzQTWUf4ETAcNewICQ8BGhOx/bBRDxY3rhQivICJsA=; b=Uw4qvHlcsSh6tkhcGCbUg0rQ9A DNn0VoYKSQc9bOvVjz6MIPHRpuKb3tOJRie3kTR4HdDlpnsN5JzEb8JzFpgFJIvt5lsD8kI2Ma6yz 4cIhGf4kJBuDj9rPNT5Pi9bHWBnrs5n6tw6fmn241IsHXZF5hHoTzAecdPSfCOTEYTEggnj7VRLME lggnMtpLei/w0F0Ne757X9SY0kcacpezxPd0JihnbFPrOGkArXFzd0CnYEgoMJDIty0Ee88VP/a+A fDFOW8pxCuYyX8sYs7UnUxIRbfV4KpDvQ9LsSblTMufVny5LpSuceBkKJGniWbamMok9Wnn4a0VQm NLZdhzog==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9SgM-00C6tO-2Y; Wed, 14 Jun 2023 15:45:14 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9SgJ-00C6rT-1m for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 15:45:12 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bd5059e5347so1733308276.0 for ; Wed, 14 Jun 2023 08:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686757510; x=1689349510; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=vp6Nlsbz0oC9g4ftIwLdhomK6sQnklY1ZxeEkHnIXfEXrOB32EsbD/Sxec0iRhhQHg FG7D8M3PpWeaM8iqTIew0USWOhnUT0FjBe9eXnTTSV/LhmO3ZmKELA9cij4LrrquyyEX xN4kZsa+CrEDpQ+9SCj337Iz/uwRx2q05yM0XfIncqMYOokbXzbVERwKppl5BO38z5eP hHmqL5KhrByMtgmNr4kVUihXIw6qeRCfAOy3kJEM2Xe0aOvPnzQXKdDIUXJE6sDAuRHp s5nPeYy80N984gbWZ3p5ANSaT8JevwNNBkebJ7ua9ynBAmNYrnU+pSstx30pek9GSY+/ b8wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686757510; x=1689349510; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=KNpmspGyRMGfMM2C3eR9+55vphpcwRyWjXoOzCdIQ/ZqAMQ0rLUK+ZJg6mfSBVaWs2 lenhdDMfshjVxvN86EikKDM1XGc7i3hIrbz89alTh8+DvVw6+zaJ5K9wNowK3nkUjAXR DTFmQ4QAsTTDAomO6esWTO922f8LxOUyc1u7QqD/8PDVALqgDbXEjDBO3kT/Ob30sQTz EDQUuN4j2ZD4nhFqhDgiVb+O0V+y4fhl6uLMtdAglBpu5BDVkxMw9oE5p+UDBKTmGKHf ZY78aY+MSK48IvAevZhkqOq98sCdYQUM37wxrehrfM7UAFRb6HRKVXOntnZyQSLRwObE xDYg== X-Gm-Message-State: AC+VfDxdBCBwq12xs+s/8IdOxJTs6GiMjpEL66uIEUj6jDcVHsXaL+xF hh1eYdH/fZya1dO4qSErChTJ/t19ZKKw X-Google-Smtp-Source: ACHHUZ54OC1oThkrdI5m1JIqCvJD/OCLMDFClBfZ1oOPcGRnJCDeBJWfnhAFwtQXg3YtdduEf0o8Lo03wyNZ X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:c6e6:49bf:5c44:5965]) (user=mshavit job=sendgmr) by 2002:a05:6902:1185:b0:ba8:381b:f764 with SMTP id m5-20020a056902118500b00ba8381bf764mr888385ybu.3.1686757510234; Wed, 14 Jun 2023 08:45:10 -0700 (PDT) Date: Wed, 14 Jun 2023 23:42:05 +0800 In-Reply-To: <20230614154304.2860121-1-mshavit@google.com> Mime-Version: 1.0 References: <20230614154304.2860121-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614154304.2860121-14-mshavit@google.com> Subject: [PATCH v3 13/13] iommu/arm-smmu-v3-sva: Add check when enabling sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_084511_596495_A3680F0E X-CRM114-Status: GOOD ( 10.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SVA domains can only be attached when the master's STEs have a stage 1 domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index b615a85e6a54e..e2a91f20f0906 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -499,9 +499,15 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int ret; mutex_lock(&sva_lock); + + if (!master->s1_cfg) { + ret = -EBUSY; + goto unlock; + } ret = arm_smmu_master_sva_enable_iopf(master); if (!ret) master->sva_enabled = true; +unlock: mutex_unlock(&sva_lock); return ret;