From patchwork Tue Jan 26 18:06:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 8125671 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 088A79F859 for ; Tue, 26 Jan 2016 18:09:05 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 24D942024F for ; Tue, 26 Jan 2016 18:09:04 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 366D8202BE for ; Tue, 26 Jan 2016 18:09:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aO81j-0004ez-SJ; Tue, 26 Jan 2016 18:07:39 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aO81A-0004TK-Hf for linux-arm-kernel@lists.infradead.org; Tue, 26 Jan 2016 18:07:06 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9C87D5E2; Tue, 26 Jan 2016 10:06:06 -0800 (PST) Received: from e104324-lin.cambridge.arm.com (e104324-lin.cambridge.arm.com [10.1.205.42]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9C5B13F246; Tue, 26 Jan 2016 10:06:46 -0800 (PST) From: Robin Murphy To: will.deacon@arm.com Subject: [PATCH 4/4] iommu/arm-smmu: Use per-context TLB sync as appropriate Date: Tue, 26 Jan 2016 18:06:37 +0000 Message-Id: X-Mailer: git-send-email 2.7.0.25.gfc10eb5.dirty In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160126_100704_748349_806E6952 X-CRM114-Status: GOOD ( 12.09 ) X-Spam-Score: -6.9 (------) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: iommu@lists.linux-foundation.org, mitchelh@codeaurora.org, joro@8bytes.org, linux-arm-kernel@lists.infradead.org, tchalamarla@caviumnetworks.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP TLB synchronisation is a mighty big hammmer to bring down on the transaction stream, typically stalling all in-flight transactions until the sync completes. Since in most cases (except at stage 2 on SMMUv1) a per-context sync operation is available, prefer that over the global operation when performing TLB maintenance for a single domain, to avoid unecessarily disrupting ongoing traffic in other contexts. Signed-off-by: Robin Murphy --- drivers/iommu/arm-smmu.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 18e0e10..bf1895c 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -219,6 +219,8 @@ #define ARM_SMMU_CB_S1_TLBIVAL 0x620 #define ARM_SMMU_CB_S2_TLBIIPAS2 0x630 #define ARM_SMMU_CB_S2_TLBIIPAS2L 0x638 +#define ARM_SMMU_CB_TLBSYNC 0x7f0 +#define ARM_SMMU_CB_TLBSTATUS 0x7f4 #define ARM_SMMU_CB_ATS1PR 0x800 #define ARM_SMMU_CB_ATSR 0x8f0 @@ -546,14 +548,22 @@ static void __arm_smmu_free_bitmap(unsigned long *map, int idx) } /* Wait for any pending TLB invalidations to complete */ -static void __arm_smmu_tlb_sync(struct arm_smmu_device *smmu) +static void __arm_smmu_tlb_sync(struct arm_smmu_device *smmu, int cbndx) { int count = 0; - void __iomem *gr0_base = ARM_SMMU_GR0(smmu); + void __iomem *base, __iomem *status; - writel_relaxed(0, gr0_base + ARM_SMMU_GR0_sTLBGSYNC); - while (readl_relaxed(gr0_base + ARM_SMMU_GR0_sTLBGSTATUS) - & sTLBGSTATUS_GSACTIVE) { + if (cbndx < 0) { + base = ARM_SMMU_GR0(smmu); + status = base + ARM_SMMU_GR0_sTLBGSTATUS; + writel_relaxed(0, base + ARM_SMMU_GR0_sTLBGSYNC); + } else { + base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cbndx); + status = base + ARM_SMMU_CB_TLBSTATUS; + writel_relaxed(0, base + ARM_SMMU_CB_TLBSYNC); + } + + while (readl_relaxed(status) & sTLBGSTATUS_GSACTIVE) { cpu_relax(); if (++count == TLB_LOOP_TIMEOUT) { dev_err_ratelimited(smmu->dev, @@ -567,7 +577,13 @@ static void __arm_smmu_tlb_sync(struct arm_smmu_device *smmu) static void arm_smmu_tlb_sync(void *cookie) { struct arm_smmu_domain *smmu_domain = cookie; - __arm_smmu_tlb_sync(smmu_domain->smmu); + int cbndx = smmu_domain->cfg.cbndx; + + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2 && + smmu_domain->smmu->version < ARM_SMMU_V2) + cbndx = -1; + + __arm_smmu_tlb_sync(smmu_domain->smmu, cbndx); } static void arm_smmu_tlb_inv_context(void *cookie) @@ -588,7 +604,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) base + ARM_SMMU_GR0_TLBIVMID); } - __arm_smmu_tlb_sync(smmu); + __arm_smmu_tlb_sync(smmu, cfg->cbndx); } static void arm_smmu_tlb_inv_range_nosync(unsigned long iova, size_t size, @@ -1534,7 +1550,7 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu) reg &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT); /* Push the button */ - __arm_smmu_tlb_sync(smmu); + __arm_smmu_tlb_sync(smmu, -1); writel(reg, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0); }