From patchwork Thu May 12 14:45:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: yf.wang@mediatek.com X-Patchwork-Id: 12847815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 491C2C433EF for ; Thu, 12 May 2022 14:59:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fQEFBcLXgvjxJwLLkXf7UlnpAeQqrRVivFn9psiYTD4=; b=Gx7hn2Ccnr/j+F A2CksJkE/A3L/UGD6d9m61SASujBWpKO2Cp1TfD5jWBka5DT6GXuHYG5nWanllp+PG1vY5q+HNpVH ERWxa59v84CTYkgdwru32ayk0DIzVWZBU2NIPpBGKmUIwULKEKiembbx3KWt88ny8BbX1DRBMGoae LLILeqAuR5/mJe0BocV4Mm4NIoV0dncJognfGfzMVUFQD46qbUuyAIXeDZYxGe+TfWeXFisZ4xuH6 nK0N3KpfK9mrrcTVQcyGVKz+qPVrKkiVNT1Og4r6cuaF7j4pqubV99MaX8MwCb/HDE2Vzmp+M3zrz UyhFKDkNI4lezVUJr1VA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npAHw-00CXOr-El; Thu, 12 May 2022 14:59:36 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npAHX-00CXHK-8Z; Thu, 12 May 2022 14:59:12 +0000 X-UUID: 0ce8f2f487574fc7a06928622a07b77b-20220512 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.4, REQID:d95a2b7c-d89c-43a9-a0d9-e8f04e2f291f, OB:0, LO B:0,IP:0,URL:5,TC:0,Content:0,EDM:0,RT:0,SF:0,FILE:0,RULE:Release_Ham,ACTI ON:release,TS:5 X-CID-META: VersionHash:faefae9, CLOUDID:f0d314a7-eab7-4b74-a74d-5359964535a9, C OID:IGNORED,Recheck:0,SF:nil,TC:nil,Content:0,EDM:-3,File:nil,QS:0,BEC:nil X-UUID: 0ce8f2f487574fc7a06928622a07b77b-20220512 Received: from mtkcas66.mediatek.inc [(172.29.193.44)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 640077869; Thu, 12 May 2022 07:59:05 -0700 Received: from mtkmbs10n2.mediatek.inc (172.21.101.183) by MTKMBS62N2.mediatek.inc (172.29.193.42) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 12 May 2022 07:52:51 -0700 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs10n2.mediatek.inc (172.21.101.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Thu, 12 May 2022 22:52:49 +0800 Received: from mbjsdccf07.mediatek.inc (10.15.20.246) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 12 May 2022 22:52:48 +0800 From: To: Will Deacon , Robin Murphy , "Joerg Roedel" , Matthias Brugger , "Isaac J. Manjarres" , Georgi Djakov , Yunfei Wang , Sven Peter , "moderated list:ARM SMMU DRIVERS" , "open list:IOMMU DRIVERS" , open list , "moderated list:ARM/Mediatek SoC support" CC: , Libo Kang , Yong Wu , Ning Li Subject: [PATCH v3 1/2] iommu/io-pgtable-arm-v7s: Add a quirk to allow pgtable PA up to 35bit Date: Thu, 12 May 2022 22:45:55 +0800 Message-ID: <20220512144557.18025-2-yf.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20220512144557.18025-1-yf.wang@mediatek.com> References: <20220512144557.18025-1-yf.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220512_075911_339306_6E8F339E X-CRM114-Status: GOOD ( 21.78 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org From: Yunfei Wang The calling to kmem_cache_alloc for level 2 pgtable allocation may run in atomic context, and it fails sometimes when DMA32 zone runs out of memory. Since Mediatek IOMMU hardware support at most 35bit PA in pgtable, so add a quirk to allow the PA of pgtables support up to bit35. Signed-off-by: Ning Li Signed-off-by: Yunfei Wang Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/iommu/io-pgtable-arm-v7s.c | 56 ++++++++++++++++++++++-------- include/linux/io-pgtable.h | 15 +++++--- 2 files changed, 52 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c index be066c1503d3..57455ae052ac 100644 --- a/drivers/iommu/io-pgtable-arm-v7s.c +++ b/drivers/iommu/io-pgtable-arm-v7s.c @@ -149,6 +149,10 @@ #define ARM_V7S_TTBR_IRGN_ATTR(attr) \ ((((attr) & 0x1) << 6) | (((attr) & 0x2) >> 1)) +/* Mediatek extend ttbr bits[2:0] for PA bits[34:32] */ +#define ARM_V7S_TTBR_35BIT_PA(ttbr, pa) \ + ((ttbr & ((u32)(~0U << 3))) | ((pa & GENMASK(34, 32)) >> 32)) + #ifdef CONFIG_ZONE_DMA32 #define ARM_V7S_TABLE_GFP_DMA GFP_DMA32 #define ARM_V7S_TABLE_SLAB_FLAGS SLAB_CACHE_DMA32 @@ -182,14 +186,8 @@ static bool arm_v7s_is_mtk_enabled(struct io_pgtable_cfg *cfg) (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_EXT); } -static arm_v7s_iopte paddr_to_iopte(phys_addr_t paddr, int lvl, - struct io_pgtable_cfg *cfg) +static arm_v7s_iopte to_iopte_mtk(phys_addr_t paddr, arm_v7s_iopte pte) { - arm_v7s_iopte pte = paddr & ARM_V7S_LVL_MASK(lvl); - - if (!arm_v7s_is_mtk_enabled(cfg)) - return pte; - if (paddr & BIT_ULL(32)) pte |= ARM_V7S_ATTR_MTK_PA_BIT32; if (paddr & BIT_ULL(33)) @@ -199,6 +197,17 @@ static arm_v7s_iopte paddr_to_iopte(phys_addr_t paddr, int lvl, return pte; } +static arm_v7s_iopte paddr_to_iopte(phys_addr_t paddr, int lvl, + struct io_pgtable_cfg *cfg) +{ + arm_v7s_iopte pte = paddr & ARM_V7S_LVL_MASK(lvl); + + if (!arm_v7s_is_mtk_enabled(cfg)) + return pte; + + return to_iopte_mtk(paddr, pte); +} + static phys_addr_t iopte_to_paddr(arm_v7s_iopte pte, int lvl, struct io_pgtable_cfg *cfg) { @@ -234,6 +243,7 @@ static arm_v7s_iopte *iopte_deref(arm_v7s_iopte pte, int lvl, static void *__arm_v7s_alloc_table(int lvl, gfp_t gfp, struct arm_v7s_io_pgtable *data) { + gfp_t gfp_l1 = __GFP_ZERO | ARM_V7S_TABLE_GFP_DMA; struct io_pgtable_cfg *cfg = &data->iop.cfg; struct device *dev = cfg->iommu_dev; phys_addr_t phys; @@ -241,9 +251,11 @@ static void *__arm_v7s_alloc_table(int lvl, gfp_t gfp, size_t size = ARM_V7S_TABLE_SIZE(lvl, cfg); void *table = NULL; + if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT) + gfp_l1 = __GFP_ZERO; + if (lvl == 1) - table = (void *)__get_free_pages( - __GFP_ZERO | ARM_V7S_TABLE_GFP_DMA, get_order(size)); + table = (void *)__get_free_pages(gfp_l1, get_order(size)); else if (lvl == 2) table = kmem_cache_zalloc(data->l2_tables, gfp); @@ -251,7 +263,8 @@ static void *__arm_v7s_alloc_table(int lvl, gfp_t gfp, return NULL; phys = virt_to_phys(table); - if (phys != (arm_v7s_iopte)phys) { + if (phys != (arm_v7s_iopte)phys && + !(cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT)) { /* Doesn't fit in PTE */ dev_err(dev, "Page table does not fit in PTE: %pa", &phys); goto out_free; @@ -457,9 +470,14 @@ static arm_v7s_iopte arm_v7s_install_table(arm_v7s_iopte *table, arm_v7s_iopte curr, struct io_pgtable_cfg *cfg) { + phys_addr_t phys = virt_to_phys(table); arm_v7s_iopte old, new; - new = virt_to_phys(table) | ARM_V7S_PTE_TYPE_TABLE; + new = phys | ARM_V7S_PTE_TYPE_TABLE; + + if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT) + new = to_iopte_mtk(phys, new); + if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_NS) new |= ARM_V7S_ATTR_NS_TABLE; @@ -778,7 +796,9 @@ static phys_addr_t arm_v7s_iova_to_phys(struct io_pgtable_ops *ops, static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie) { + slab_flags_t slab_flag = ARM_V7S_TABLE_SLAB_FLAGS; struct arm_v7s_io_pgtable *data; + phys_addr_t paddr; if (cfg->ias > (arm_v7s_is_mtk_enabled(cfg) ? 34 : ARM_V7S_ADDR_BITS)) return NULL; @@ -788,7 +808,8 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg, if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS | IO_PGTABLE_QUIRK_NO_PERMS | - IO_PGTABLE_QUIRK_ARM_MTK_EXT)) + IO_PGTABLE_QUIRK_ARM_MTK_EXT | + IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT)) return NULL; /* If ARM_MTK_4GB is enabled, the NO_PERMS is also expected. */ @@ -801,10 +822,12 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg, return NULL; spin_lock_init(&data->split_lock); + if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT) + slab_flag = 0; data->l2_tables = kmem_cache_create("io-pgtable_armv7s_l2", ARM_V7S_TABLE_SIZE(2, cfg), ARM_V7S_TABLE_SIZE(2, cfg), - ARM_V7S_TABLE_SLAB_FLAGS, NULL); + slab_flag, NULL); if (!data->l2_tables) goto out_free_data; @@ -850,12 +873,17 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg, wmb(); /* TTBR */ - cfg->arm_v7s_cfg.ttbr = virt_to_phys(data->pgd) | ARM_V7S_TTBR_S | + paddr = virt_to_phys(data->pgd); + cfg->arm_v7s_cfg.ttbr = paddr | ARM_V7S_TTBR_S | (cfg->coherent_walk ? (ARM_V7S_TTBR_NOS | ARM_V7S_TTBR_IRGN_ATTR(ARM_V7S_RGN_WBWA) | ARM_V7S_TTBR_ORGN_ATTR(ARM_V7S_RGN_WBWA)) : (ARM_V7S_TTBR_IRGN_ATTR(ARM_V7S_RGN_NC) | ARM_V7S_TTBR_ORGN_ATTR(ARM_V7S_RGN_NC))); + + if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT) + cfg->arm_v7s_cfg.ttbr = + ARM_V7S_TTBR_35BIT_PA(cfg->arm_v7s_cfg.ttbr, paddr); return &data->iop; out_free_data: diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h index 86af6f0a00a2..7ed15ad4710c 100644 --- a/include/linux/io-pgtable.h +++ b/include/linux/io-pgtable.h @@ -74,17 +74,22 @@ struct io_pgtable_cfg { * to support up to 35 bits PA where the bit32, bit33 and bit34 are * encoded in the bit9, bit4 and bit5 of the PTE respectively. * + * IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT: (ARM v7s format) MediaTek IOMMUs + * extend the translation table support up to 35 bits PA, the + * encoding format is same with IO_PGTABLE_QUIRK_ARM_MTK_EXT. + * * IO_PGTABLE_QUIRK_ARM_TTBR1: (ARM LPAE format) Configure the table * for use in the upper half of a split address space. * * IO_PGTABLE_QUIRK_ARM_OUTER_WBWA: Override the outer-cacheability * attributes set in the TCR for a non-coherent page-table walker. */ - #define IO_PGTABLE_QUIRK_ARM_NS BIT(0) - #define IO_PGTABLE_QUIRK_NO_PERMS BIT(1) - #define IO_PGTABLE_QUIRK_ARM_MTK_EXT BIT(3) - #define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5) - #define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6) + #define IO_PGTABLE_QUIRK_ARM_NS BIT(0) + #define IO_PGTABLE_QUIRK_NO_PERMS BIT(1) + #define IO_PGTABLE_QUIRK_ARM_MTK_EXT BIT(3) + #define IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT BIT(4) + #define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5) + #define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6) unsigned long quirks; unsigned long pgsize_bitmap; unsigned int ias; From patchwork Thu May 12 14:45:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: yf.wang@mediatek.com X-Patchwork-Id: 12847814 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 969DDC433EF for ; Thu, 12 May 2022 14:59:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ct6KLQp1kRBx8F9uQUXsE0l7dZaevrViXQRcCG+kshs=; b=M47Z8IrqD0Xclt KJg0T59oshGJDtkDlqH2hwbtZ/LnrUC73nxsbNvUfzUwg1UG5pJMsnoyJMKUfNC5iZa864pxogaVQ iOWts8Rwzw4rdS7p5VQAB75Qq6YwE/9L3cMm8gN3XrGwatoSeT0PSZ0RMlm9j4k1t7vkCCa60nsY8 6uT8B3orEmnc0LcEfc2kEpr55V6jBB/sK2Bg/ISvt2ICoMrU4XHSMSqsxLOV/23YfK/Y4PIvkAt8Q HddNoWgGF++Z3/dkHLjdpWDG9JNKRnf7VN0PEYZu7Z7lBqcGidgp9ArhD36f+5PW/L8zlnFrMX+kV L8uOnpw6Neie+GnhDBLA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npAHZ-00CXI1-1f; Thu, 12 May 2022 14:59:13 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npAHW-00CXGw-Q0; Thu, 12 May 2022 14:59:12 +0000 X-UUID: 6c2ec7fdc9c04be9ad7592d7cf082cf5-20220512 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.4, REQID:dc37053d-7418-4ee1-9c4f-c5979d8ef499, OB:0, LO B:0,IP:0,URL:5,TC:0,Content:0,EDM:0,RT:0,SF:0,FILE:0,RULE:Release_Ham,ACTI ON:release,TS:5 X-CID-META: VersionHash:faefae9, CLOUDID:d2d314a7-eab7-4b74-a74d-5359964535a9, C OID:IGNORED,Recheck:0,SF:nil,TC:nil,Content:0,EDM:-3,File:nil,QS:0,BEC:nil X-UUID: 6c2ec7fdc9c04be9ad7592d7cf082cf5-20220512 Received: from mtkcas66.mediatek.inc [(172.29.193.44)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1620724226; Thu, 12 May 2022 07:59:05 -0700 Received: from mtkmbs10n1.mediatek.inc (172.21.101.34) by MTKMBS62N2.mediatek.inc (172.29.193.42) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 12 May 2022 07:52:54 -0700 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.15; Thu, 12 May 2022 22:52:52 +0800 Received: from mbjsdccf07.mediatek.inc (10.15.20.246) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 12 May 2022 22:52:50 +0800 From: To: Yong Wu , Joerg Roedel , "Will Deacon" , Matthias Brugger , "open list:MEDIATEK IOMMU DRIVER" , "moderated list:MEDIATEK IOMMU DRIVER" , "moderated list:ARM/Mediatek SoC support" , "open list" CC: , Libo Kang , Yong Wu , Yunfei Wang , Ning Li Subject: [PATCH v3 2/2] iomm/mediatek: Allow page table PA up to 35bit Date: Thu, 12 May 2022 22:45:56 +0800 Message-ID: <20220512144557.18025-3-yf.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20220512144557.18025-1-yf.wang@mediatek.com> References: <20220512144557.18025-1-yf.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220512_075910_874485_A082FB26 X-CRM114-Status: GOOD ( 14.65 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org From: Yunfei Wang Add the quirk IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT support, so that allows page table PA up to 35bit, not only in ZONE_DMA32. Signed-off-by: Ning Li Signed-off-by: Yunfei Wang --- drivers/iommu/mtk_iommu.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c index 6fd75a60abd6..1b9a876ef271 100644 --- a/drivers/iommu/mtk_iommu.c +++ b/drivers/iommu/mtk_iommu.c @@ -33,6 +33,7 @@ #define REG_MMU_PT_BASE_ADDR 0x000 #define MMU_PT_ADDR_MASK GENMASK(31, 7) +#define MMU_PT_ADDR_2_0_MASK GENMASK(2, 0) #define REG_MMU_INVALIDATE 0x020 #define F_ALL_INVLD 0x2 @@ -118,6 +119,7 @@ #define WR_THROT_EN BIT(6) #define HAS_LEGACY_IVRP_PADDR BIT(7) #define IOVA_34_EN BIT(8) +#define PGTABLE_PA_35_EN BIT(9) #define MTK_IOMMU_HAS_FLAG(pdata, _x) \ ((((pdata)->flags) & (_x)) == (_x)) @@ -401,6 +403,9 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom, .iommu_dev = data->dev, }; + if (MTK_IOMMU_HAS_FLAG(data->plat_data, PGTABLE_PA_35_EN)) + dom->cfg.quirks |= IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT; + if (MTK_IOMMU_HAS_FLAG(data->plat_data, HAS_4GB_MODE)) dom->cfg.oas = data->enable_4GB ? 33 : 32; else @@ -450,6 +455,7 @@ static int mtk_iommu_attach_device(struct iommu_domain *domain, struct mtk_iommu_domain *dom = to_mtk_domain(domain); struct device *m4udev = data->dev; int ret, domid; + u32 regval; domid = mtk_iommu_get_domain_id(dev, data->plat_data); if (domid < 0) @@ -472,8 +478,14 @@ static int mtk_iommu_attach_device(struct iommu_domain *domain, return ret; } data->m4u_dom = dom; - writel(dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_MASK, - data->base + REG_MMU_PT_BASE_ADDR); + + /* Bits[6:3] are invalid for mediatek platform */ + if (MTK_IOMMU_HAS_FLAG(data->plat_data, PGTABLE_PA_35_EN)) + regval = (dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_MASK) | + (dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_2_0_MASK); + else + regval = dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_MASK; + writel(regval, data->base + REG_MMU_PT_BASE_ADDR); pm_runtime_put(m4udev); } @@ -987,6 +999,7 @@ static int __maybe_unused mtk_iommu_runtime_resume(struct device *dev) struct mtk_iommu_suspend_reg *reg = &data->reg; struct mtk_iommu_domain *m4u_dom = data->m4u_dom; void __iomem *base = data->base; + u32 regval; int ret; ret = clk_prepare_enable(data->bclk); @@ -1010,7 +1023,14 @@ static int __maybe_unused mtk_iommu_runtime_resume(struct device *dev) writel_relaxed(reg->int_main_control, base + REG_MMU_INT_MAIN_CONTROL); writel_relaxed(reg->ivrp_paddr, base + REG_MMU_IVRP_PADDR); writel_relaxed(reg->vld_pa_rng, base + REG_MMU_VLD_PA_RNG); - writel(m4u_dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_MASK, base + REG_MMU_PT_BASE_ADDR); + + /* Bits[6:3] are invalid for mediatek platform */ + if (MTK_IOMMU_HAS_FLAG(data->plat_data, PGTABLE_PA_35_EN)) + regval = (m4u_dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_MASK) | + (m4u_dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_2_0_MASK); + else + regval = m4u_dom->cfg.arm_v7s_cfg.ttbr & MMU_PT_ADDR_MASK; + writel(regval, base + REG_MMU_PT_BASE_ADDR); /* * Users may allocate dma buffer before they call pm_runtime_get, @@ -1038,7 +1058,8 @@ static const struct mtk_iommu_plat_data mt2712_data = { static const struct mtk_iommu_plat_data mt6779_data = { .m4u_plat = M4U_MT6779, - .flags = HAS_SUB_COMM | OUT_ORDER_WR_EN | WR_THROT_EN, + .flags = HAS_SUB_COMM | OUT_ORDER_WR_EN | WR_THROT_EN | + PGTABLE_PA_35_EN, .inv_sel_reg = REG_MMU_INV_SEL_GEN2, .iova_region = single_domain, .iova_region_nr = ARRAY_SIZE(single_domain),