From patchwork Mon Feb 12 18:33:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10214529 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7617C60153 for ; Mon, 12 Feb 2018 21:36:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 600F928C75 for ; Mon, 12 Feb 2018 21:36:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5468328D1B; Mon, 12 Feb 2018 21:36:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B276E28C75 for ; Mon, 12 Feb 2018 21:36:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iLVet4kU+FrqzNxsSxgODGd1gRw5I4TVdGi9Qe82tLo=; b=YFZ7PY20Gy1GUwwWThVLpYX+2Y cbebwCWOwsSd3DRn8+dxEeLXp5hqCjsiEFSJScOEkG6zmffnFONpTEbOea5ZpTmqP78SHxu3nj4ix KvlpvuoBeJ2PRU3LhqxTnrJFn1fdeExwJdE9DEob6UnssaCPzuA3sD2d7s2vkELte6tbb6HACmAOk M4oZ2BBPOvjZsYHG4Q+DsCKtI3Thhpxb2W2LW6tzWHf27KUW+f+LK41AvDhLDsF7ll+i5+tFoYKZD 0AzDGrI3H94H34r1Nv01BwIgvPSPlxTwE1sp7SR8CbUqrvaOolTZV0GZkRP3IZ997dgf+TKNKI7/k dp8TG8rg==; Received: from [198.137.202.133] (helo=bombadil.infradead.org) by casper.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1elLld-0005HB-P5; Mon, 12 Feb 2018 21:36:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iLVet4kU+FrqzNxsSxgODGd1gRw5I4TVdGi9Qe82tLo=; b=higc9HpcHeZbav1aUxy8ceQ5rW 9XPf3HWpsY+PEVV/Y6Rk0qGvj9imkdxHtpkZpwMnsLyW37mrezh1f+eKK8b0AMUlXPHmLv0Oko/+a oezDNR8jRCqyJJdI/cg7JYMv6TCzwsVhk30/F0mThrIfl3X1/kbMptjI9t5tg/hw8cibQ3wsg+lvU HRPR17npz3L5EqHy3pId6e79vb2vzaQNW/mN13L9Z+zI6OQYzGTudDfaEFF+Ra1Ehp4qEDEib+dV2 eYNrK+Jl02R7ecfFLGDM5cd+t6XdP0UGVrooGXiG8R4MEJoQopi6L2GMdxz8MtVNczvsEODnaQxEw 6VxgYl0Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1elJ00-00079z-Df; Mon, 12 Feb 2018 18:38:44 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1elIuK-0004Zx-Co for linux-arm-kernel@lists.infradead.org; Mon, 12 Feb 2018 18:32:56 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ED889164F; Mon, 12 Feb 2018 10:32:47 -0800 (PST) Received: from e106794-lin.cambridge.arm.com (e106794-lin.cambridge.arm.com [10.1.210.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1B0AD3F24D; Mon, 12 Feb 2018 10:32:41 -0800 (PST) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org Subject: [PATCH 19/37] iommu/arm-smmu-v3: Add second level of context descriptor table Date: Mon, 12 Feb 2018 18:33:34 +0000 Message-Id: <20180212183352.22730-20-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180212183352.22730-1-jean-philippe.brucker@arm.com> References: <20180212183352.22730-1-jean-philippe.brucker@arm.com> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, xieyisheng1@huawei.com, ilias.apalodimas@linaro.org, catalin.marinas@arm.com, xuzaibo@huawei.com, jonathan.cameron@huawei.com, will.deacon@arm.com, okaya@codeaurora.org, yi.l.liu@intel.com, lorenzo.pieralisi@arm.com, ashok.raj@intel.com, tn@semihalf.com, joro@8bytes.org, bharatku@xilinx.com, rfranz@cavium.com, lenb@kernel.org, jacob.jun.pan@linux.intel.com, alex.williamson@redhat.com, robh+dt@kernel.org, thunder.leizhen@huawei.com, bhelgaas@google.com, shunyong.yang@hxt-semitech.com, dwmw2@infradead.org, liubo95@huawei.com, rjw@rjwysocki.net, jcrouse@codeaurora.org, robdclark@gmail.com, hanjun.guo@linaro.org, sudeep.holla@arm.com, robin.murphy@arm.com, christian.koenig@amd.com, nwatters@codeaurora.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The SMMU can support up to 20 bits of SSID. Add a second level of page tables to accommodate this. Devices that support more than 1024 SSIDs now have a table of 1024 L1 entries (8kB), pointing to tables of 1024 context descriptors (64kB), allocated on demand. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3-context.c | 137 ++++++++++++++++++++++++++++++++++-- 1 file changed, 130 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3-context.c b/drivers/iommu/arm-smmu-v3-context.c index 3b0bb9475dea..aaffc2071966 100644 --- a/drivers/iommu/arm-smmu-v3-context.c +++ b/drivers/iommu/arm-smmu-v3-context.c @@ -14,6 +14,19 @@ #include "iommu-pasid.h" +/* + * Linear: when less than 1024 SSIDs are supported + * 2lvl: at most 1024 L1 entrie, + * 1024 lazy entries per table. + */ +#define CTXDESC_SPLIT 10 +#define CTXDESC_NUM_L2_ENTRIES (1 << CTXDESC_SPLIT) + +#define CTXDESC_L1_DESC_DWORD 1 +#define CTXDESC_L1_DESC_VALID 1 +#define CTXDESC_L1_DESC_L2PTR_SHIFT 12 +#define CTXDESC_L1_DESC_L2PTR_MASK 0xfffffffffUL + #define CTXDESC_CD_DWORDS 8 #define CTXDESC_CD_0_TCR_T0SZ_SHIFT 0 #define ARM64_TCR_T0SZ_SHIFT 0 @@ -86,7 +99,17 @@ struct arm_smmu_cd_table { struct arm_smmu_cd_tables { struct iommu_pasid_table pasid; - struct arm_smmu_cd_table table; + bool linear; + union { + struct arm_smmu_cd_table table; + struct { + __le64 *ptr; + dma_addr_t ptr_dma; + size_t num_entries; + + struct arm_smmu_cd_table *tables; + } l1; + }; }; #define pasid_to_cd_tables(pasid_table) \ @@ -122,9 +145,44 @@ static void arm_smmu_free_cd_leaf_table(struct device *dev, dmam_free_coherent(dev, size, desc->ptr, desc->ptr_dma); } +static void arm_smmu_write_cd_l1_desc(__le64 *dst, + struct arm_smmu_cd_table *desc) +{ + u64 val = (desc->ptr_dma & CTXDESC_L1_DESC_L2PTR_MASK << + CTXDESC_L1_DESC_L2PTR_SHIFT) | CTXDESC_L1_DESC_VALID; + + *dst = cpu_to_le64(val); +} + static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_cd_tables *tbl, u32 ssid) { - return tbl->table.ptr + ssid * CTXDESC_CD_DWORDS; + unsigned long idx; + struct arm_smmu_cd_table *l1_desc; + struct iommu_pasid_table_cfg *cfg = &tbl->pasid.cfg; + + if (tbl->linear) + return tbl->table.ptr + ssid * CTXDESC_CD_DWORDS; + + idx = ssid >> CTXDESC_SPLIT; + if (idx >= tbl->l1.num_entries) + return NULL; + + l1_desc = &tbl->l1.tables[idx]; + if (!l1_desc->ptr) { + __le64 *l1ptr = tbl->l1.ptr + idx * CTXDESC_L1_DESC_DWORD; + + if (arm_smmu_alloc_cd_leaf_table(cfg->iommu_dev, l1_desc, + CTXDESC_NUM_L2_ENTRIES)) + return NULL; + + arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); + /* An invalid L1 entry is allowed to be cached */ + iommu_pasid_flush(&tbl->pasid, idx << CTXDESC_SPLIT, false); + } + + idx = ssid & (CTXDESC_NUM_L2_ENTRIES - 1); + + return l1_desc->ptr + idx * CTXDESC_CD_DWORDS; } static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) @@ -307,16 +365,51 @@ static struct iommu_pasid_table * arm_smmu_alloc_cd_tables(struct iommu_pasid_table_cfg *cfg, void *cookie) { int ret; + size_t size = 0; struct arm_smmu_cd_tables *tbl; struct device *dev = cfg->iommu_dev; + struct arm_smmu_cd_table *leaf_table; + size_t num_contexts, num_leaf_entries; tbl = devm_kzalloc(dev, sizeof(*tbl), GFP_KERNEL); if (!tbl) return NULL; - ret = arm_smmu_alloc_cd_leaf_table(dev, &tbl->table, 1 << cfg->order); + num_contexts = 1 << cfg->order; + if (num_contexts <= CTXDESC_NUM_L2_ENTRIES) { + /* Fits in a single table */ + tbl->linear = true; + num_leaf_entries = num_contexts; + leaf_table = &tbl->table; + } else { + /* + * SSID[S1CDmax-1:10] indexes 1st-level table, SSID[9:0] indexes + * 2nd-level + */ + tbl->l1.num_entries = num_contexts / CTXDESC_NUM_L2_ENTRIES; + + tbl->l1.tables = devm_kzalloc(dev, + sizeof(struct arm_smmu_cd_table) * + tbl->l1.num_entries, GFP_KERNEL); + if (!tbl->l1.tables) + goto err_free_tbl; + + size = tbl->l1.num_entries * (CTXDESC_L1_DESC_DWORD << 3); + tbl->l1.ptr = dmam_alloc_coherent(dev, size, &tbl->l1.ptr_dma, + GFP_KERNEL | __GFP_ZERO); + if (!tbl->l1.ptr) { + dev_warn(dev, "failed to allocate L1 context table\n"); + devm_kfree(dev, tbl->l1.tables); + goto err_free_tbl; + } + + num_leaf_entries = CTXDESC_NUM_L2_ENTRIES; + leaf_table = tbl->l1.tables; + } + + ret = arm_smmu_alloc_cd_leaf_table(dev, leaf_table, num_leaf_entries); if (ret) - goto err_free_tbl; + goto err_free_l1; tbl->pasid.ops = (struct iommu_pasid_table_ops) { .alloc_priv_entry = arm_smmu_alloc_priv_cd, @@ -326,11 +419,22 @@ arm_smmu_alloc_cd_tables(struct iommu_pasid_table_cfg *cfg, void *cookie) .clear_entry = arm_smmu_clear_cd, }; - cfg->base = tbl->table.ptr_dma; - cfg->arm_smmu.s1fmt = ARM_SMMU_S1FMT_LINEAR; + if (tbl->linear) { + cfg->base = leaf_table->ptr_dma; + cfg->arm_smmu.s1fmt = ARM_SMMU_S1FMT_LINEAR; + } else { + cfg->base = tbl->l1.ptr_dma; + cfg->arm_smmu.s1fmt = ARM_SMMU_S1FMT_64K_L2; + arm_smmu_write_cd_l1_desc(tbl->l1.ptr, leaf_table); + } return &tbl->pasid; +err_free_l1: + if (!tbl->linear) { + dmam_free_coherent(dev, size, tbl->l1.ptr, tbl->l1.ptr_dma); + devm_kfree(dev, tbl->l1.tables); + } err_free_tbl: devm_kfree(dev, tbl); @@ -343,7 +447,26 @@ static void arm_smmu_free_cd_tables(struct iommu_pasid_table *pasid_table) struct device *dev = cfg->iommu_dev; struct arm_smmu_cd_tables *tbl = pasid_to_cd_tables(pasid_table); - arm_smmu_free_cd_leaf_table(dev, &tbl->table, 1 << cfg->order); + if (tbl->linear) { + arm_smmu_free_cd_leaf_table(dev, &tbl->table, 1 << cfg->order); + } else { + size_t i, size; + + for (i = 0; i < tbl->l1.num_entries; i++) { + struct arm_smmu_cd_table *table = &tbl->l1.tables[i]; + + if (!table->ptr) + continue; + + arm_smmu_free_cd_leaf_table(dev, table, + CTXDESC_NUM_L2_ENTRIES); + } + + size = tbl->l1.num_entries * (CTXDESC_L1_DESC_DWORD << 3); + dmam_free_coherent(dev, size, tbl->l1.ptr, tbl->l1.ptr_dma); + devm_kfree(dev, tbl->l1.tables); + } + devm_kfree(dev, tbl); }