From patchwork Fri Mar 14 05:02:54 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cho KyongHo X-Patchwork-Id: 3830291 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 340149F2BB for ; Fri, 14 Mar 2014 05:05:17 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 498B12016C for ; Fri, 14 Mar 2014 05:05:16 +0000 (UTC) Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 479ED202B8 for ; Fri, 14 Mar 2014 05:05:15 +0000 (UTC) Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WOKHo-00083P-ED; Fri, 14 Mar 2014 05:04:01 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WOKHT-0008Kf-GD; Fri, 14 Mar 2014 05:03:39 +0000 Received: from mailout2.samsung.com ([203.254.224.25]) by merlin.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WOKH7-0008EH-0O for linux-arm-kernel@lists.infradead.org; Fri, 14 Mar 2014 05:03:18 +0000 Received: from epcpsbgr3.samsung.com (u143.gpu120.samsung.co.kr [203.254.230.143]) by mailout2.samsung.com (Oracle Communications Messaging Server 7u4-24.01 (7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTP id <0N2E00F3CU0VW530@mailout2.samsung.com> for linux-arm-kernel@lists.infradead.org; Fri, 14 Mar 2014 14:02:55 +0900 (KST) Received: from epcpsbgm2.samsung.com ( [203.254.230.51]) by epcpsbgr3.samsung.com (EPCPMTA) with SMTP id A1.86.10092.F7D82235; Fri, 14 Mar 2014 14:02:55 +0900 (KST) X-AuditID: cbfee68f-b7f156d00000276c-40-53228d7f6223 Received: from epmmp1.local.host ( [203.254.227.16]) by epcpsbgm2.samsung.com (EPCPMTA) with SMTP id F8.11.28157.F7D82235; Fri, 14 Mar 2014 14:02:55 +0900 (KST) Received: from DO-PULLIP-CHO07.dsn.sec.samsung.com ([12.36.165.149]) by mmp1.samsung.com (Oracle Communications Messaging Server 7u4-24.01(7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTPA id <0N2E004COU0UEWB2@mmp1.samsung.com>; Fri, 14 Mar 2014 14:02:54 +0900 (KST) Date: Fri, 14 Mar 2014 14:02:54 +0900 From: Cho KyongHo To: Linux ARM Kernel , Linux DeviceTree , Linux IOMMU , Linux Kernel , Linux Samsung SOC Subject: [PATCH v11 04/27] iommu/exynos: fix L2TLB invalidation Message-id: <20140314140254.32dbc0cd8cde8f4ffcf71c9f@samsung.com> X-Mailer: Sylpheed 3.3.0 (GTK+ 2.10.14; i686-pc-mingw32) MIME-version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrEIsWRmVeSWpSXmKPExsVy+t8zY936XqVgg56NfBZ37p5jtZh/BEi8 OvKDyWLBfmuLztkb2C16F1xls9j0+BqrxeVdc9gsZpzfx2RxYcVGdospiw6zWhx+085qcfJP L6PF+hmvWSxm3lrD4sDv8eTgPCaP2Q0XWTz+He5n8rhzbQ+bx+Yl9R6Tbyxn9OjbsorR4/Mm OY8rR88wBXBGcdmkpOZklqUW6dslcGVsWDKZqeCHRMW8x1fYGxh/CHcxcnJICJhIHDk6mQ3C FpO4cG89kM3FISSwjFFi7c1PTDBF3193sEMkFjFKvF3dyAjhTGaS+LblHyNIFYuAqsSkFc/Y QWw2AS2J1XOPgxWJCLQxSXxtPMQC4jALzGWWeLb2KStIlbCAvcTtvl1g3bwCjhKT+35C7bOQ uNDUwQ4RF5T4MfkeC4jNDDR187YmVghbXmLzmrfMEPW9HBJ7ZnJDXCEg8W0yyDIOoLisxKYD UCWSEgdX3GCZwCgyC8nUWUimzkIydQEj8ypG0dSC5ILipPQiY73ixNzi0rx0veT83E2MkPjt 38F494D1IcZkoJUTmaVEk/OB8Z9XEm9obGZkYWpiamxkbmlGmrCSOO/9h0lBQgLpiSWp2amp BalF8UWlOanFhxiZODilGhj35rb47nG+9Ha34ceG6qvvNBrk70ukq+SnHz4vIVTKbb3mgtSx rRbLGzVcd7Ek6iZ+tehYYPP3Qvfm2lXTop9F3FnfITzFce73b0+Kfz/1vWjHl3Dy+D3Z3evl hB+fWf1OyGxh/qr4uOVPpSar6ZrZfsrROcIhaLcryUDw7A0ncb9Hn3gKVa4osRRnJBpqMRcV JwIAtsJ9g/UCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrAKsWRmVeSWpSXmKPExsVy+t9jAd36XqVgg9cdHBZ37p5jtZh/BEi8 OvKDyWLBfmuLztkb2C16F1xls9j0+BqrxeVdc9gsZpzfx2RxYcVGdospiw6zWhx+085qcfJP L6PF+hmvWSxm3lrD4sDv8eTgPCaP2Q0XWTz+He5n8rhzbQ+bx+Yl9R6Tbyxn9OjbsorR4/Mm OY8rR88wBXBGNTDaZKQmpqQWKaTmJeenZOal2yp5B8c7x5uaGRjqGlpamCsp5CXmptoqufgE 6Lpl5gB9oKRQlphTChQKSCwuVtK3wzQhNMRN1wKmMULXNyQIrsfIAA0krGPM2LBkMlPBD4mK eY+vsDcw/hDuYuTkkBAwkfj+uoMdwhaTuHBvPVsXIxeHkMAiRom3qxsZIZzJTBLftvxjBKli EVCVmLTiGVgHm4CWxOq5x8GKRATamCS+Nh5iAXGYBeYySzxb+5QVpEpYwF7idt8usG5eAUeJ yX0/mSD2WUhcaILYzSsgKPFj8j0WEJsZaOrmbU2sELa8xOY1b5knMPLNQlI2C0nZLCRlCxiZ VzGKphYkFxQnpeca6RUn5haX5qXrJefnbmIEp4dn0jsYVzVYHGIU4GBU4uGdcVQxWIg1say4 MvcQowQHs5IIb1WEUrAQb0piZVVqUX58UWlOavEhxmSgvycyS4km5wNTV15JvKGxiZmRpZGZ hZGJuTlpwkrivAdbrQOFBNITS1KzU1MLUotgtjBxcEo1MK7YH3NaaM/dS/v5/2n+Xt7xX//D LBGPS3LRQotOSn66/+/2DD3x/pjgrPB+6QKBJ9ucUuqOXj/fU2b+lz1P+mxNN9/3b//e/Lo7 90CPf2WNAZ9xZFXXTP5Vq9xt1iy9rf//mm7Ap3+K/Lphc3SWvFo5/bSaefZ5v8dxnyOCFiTp RVk5G+k631ZiKc5INNRiLipOBADeCRjsUwMAAA== DLP-Filter: Pass X-MTR: 20000000000000000@CPGS X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140314_010317_323688_5A0224F4 X-CRM114-Status: GOOD ( 13.06 ) X-Spam-Score: -6.9 (------) Cc: Kukjin Kim , Prathyush , Grant Grundler , Joerg Roedel , Sachin Kamat , Sylwester Nawrocki , Varun Sethi , Antonios Motakis , Tomasz Figa , Rahul Sharma X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP L2TLB is 8-way set-associative TLB with 512 entries. The number of sets is 64. A single 4KB(small page) translation information is cached only to a set whose index is the same with the lower 6 bits of the page frame number. A single 64KB(large page) translation information can be cached to any 16 sets whose top two bits of their indices are the same with the bit [5:4] of the page frame number. A single 1MB(section) or larger translation information can be cached to any set in the TLB. It is required to invalidate entire sets that may cache the target translation information to guarantee that the L2TLB has no stale data. Signed-off-by: Cho KyongHo --- drivers/iommu/exynos-iommu.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c index 4a74ed8..0d26aeb 100644 --- a/drivers/iommu/exynos-iommu.c +++ b/drivers/iommu/exynos-iommu.c @@ -225,9 +225,14 @@ static void __sysmmu_tlb_invalidate(void __iomem *sfrbase) } static void __sysmmu_tlb_invalidate_entry(void __iomem *sfrbase, - unsigned long iova) + unsigned long iova, unsigned int num_inv) { - __raw_writel((iova & SPAGE_MASK) | 1, sfrbase + REG_MMU_FLUSH_ENTRY); + unsigned int i; + for (i = 0; i < num_inv; i++) { + __raw_writel((iova & SPAGE_MASK) | 1, + sfrbase + REG_MMU_FLUSH_ENTRY); + iova += SPAGE_SIZE; + } } static void __sysmmu_set_ptbase(void __iomem *sfrbase, @@ -477,7 +482,8 @@ static bool exynos_sysmmu_disable(struct device *dev) return disabled; } -static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova) +static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova, + size_t size) { unsigned long flags; struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu); @@ -487,9 +493,24 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova) if (is_sysmmu_active(data)) { int i; for (i = 0; i < data->nsfrs; i++) { + unsigned int maj; + unsigned int num_inv = 1; + maj = __raw_readl(data->sfrbases[i] + REG_MMU_VERSION); + /* + * L2TLB invalidation required + * 4KB page: 1 invalidation + * 64KB page: 16 invalidation + * 1MB page: 64 invalidation + * because it is set-associative TLB + * with 8-way and 64 sets. + * 1MB page can be cached in one of all sets. + * 64KB page can be one of 16 consecutive sets. + */ + if ((maj >> 28) == 2) /* major version number */ + num_inv = min_t(unsigned int, size / PAGE_SIZE, 64); if (sysmmu_block(data->sfrbases[i])) { __sysmmu_tlb_invalidate_entry( - data->sfrbases[i], iova); + data->sfrbases[i], iova, num_inv); sysmmu_unblock(data->sfrbases[i]); } } @@ -999,7 +1020,7 @@ done: spin_lock_irqsave(&priv->lock, flags); list_for_each_entry(data, &priv->clients, node) - sysmmu_tlb_invalidate_entry(data->dev, iova); + sysmmu_tlb_invalidate_entry(data->dev, iova, size); spin_unlock_irqrestore(&priv->lock, flags); return size;