From patchwork Wed Aug 13 00:51:39 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mitchel Humpherys X-Patchwork-Id: 4715671 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 90564C0338 for ; Wed, 13 Aug 2014 00:54:51 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 9352220172 for ; Wed, 13 Aug 2014 00:54:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8BF1220160 for ; Wed, 13 Aug 2014 00:54:49 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XHMoU-0007K5-0a; Wed, 13 Aug 2014 00:53:14 +0000 Received: from smtp.codeaurora.org ([198.145.11.231]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XHMnv-0006YA-H7 for linux-arm-kernel@lists.infradead.org; Wed, 13 Aug 2014 00:52:40 +0000 Received: from smtp.codeaurora.org (localhost [127.0.0.1]) by smtp.codeaurora.org (Postfix) with ESMTP id 6F30213F897; Wed, 13 Aug 2014 00:52:18 +0000 (UTC) Received: by smtp.codeaurora.org (Postfix, from userid 486) id 6289013F8C8; Wed, 13 Aug 2014 00:52:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from mitchelh-linux.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mitchelh@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id BB30013F8C6; Wed, 13 Aug 2014 00:52:17 +0000 (UTC) From: Mitchel Humpherys To: linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, Will Deacon , devicetree@vger.kernel.org Subject: [PATCH 6/6] iommu/arm-smmu: add .domain_{set, get}_attr for coherent walk control Date: Tue, 12 Aug 2014 17:51:39 -0700 Message-Id: <1407891099-24641-7-git-send-email-mitchelh@codeaurora.org> X-Mailer: git-send-email 2.0.4 In-Reply-To: <1407891099-24641-1-git-send-email-mitchelh@codeaurora.org> References: <1407891099-24641-1-git-send-email-mitchelh@codeaurora.org> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140812_175239_649111_5A699B41 X-CRM114-Status: GOOD ( 19.81 ) X-Spam-Score: -0.7 (/) Cc: Mitchel Humpherys X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Under certain conditions coherent hardware translation table walks can result in degraded performance. Add a new domain attribute to disable/enable this feature in generic code along with the domain attribute setter and getter to handle it in the ARM SMMU driver. Signed-off-by: Mitchel Humpherys --- drivers/iommu/arm-smmu.c | 57 +++++++++++++++++++++++++++++++----------------- include/linux/iommu.h | 1 + 2 files changed, 38 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 73d056668b..11672a8371 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -426,6 +426,7 @@ struct arm_smmu_cfg { u8 irptndx; u32 cbar; pgd_t *pgd; + bool htw_disable; }; #define INVALID_IRPTNDX 0xff @@ -833,14 +834,17 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev) return IRQ_HANDLED; } -static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr, - size_t size) +static void arm_smmu_flush_pgtable(struct arm_smmu_domain *smmu_domain, + void *addr, size_t size) { + struct arm_smmu_cfg *cfg = &smmu_domain->cfg; + struct arm_smmu_device *smmu = smmu_domain->smmu; unsigned long offset = (unsigned long)addr & ~PAGE_MASK; /* Ensure new page tables are visible to the hardware walker */ - if (smmu->features & ARM_SMMU_FEAT_COHERENT_WALK) { + if ((smmu->features & ARM_SMMU_FEAT_COHERENT_WALK) && + !cfg->htw_disable) { dsb(ishst); } else { /* @@ -943,7 +947,7 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) } /* TTBR0 */ - arm_smmu_flush_pgtable(smmu, cfg->pgd, + arm_smmu_flush_pgtable(smmu_domain, cfg->pgd, PTRS_PER_PGD * sizeof(pgd_t)); reg = __pa(cfg->pgd); writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO); @@ -1468,7 +1472,8 @@ static bool arm_smmu_pte_is_contiguous_range(unsigned long addr, (addr + ARM_SMMU_PTE_CONT_SIZE <= end); } -static int arm_smmu_alloc_init_pte(struct arm_smmu_device *smmu, pmd_t *pmd, +static int arm_smmu_alloc_init_pte(struct arm_smmu_domain *smmu_domain, + pmd_t *pmd, unsigned long addr, unsigned long end, unsigned long pfn, int prot, int stage) { @@ -1482,9 +1487,10 @@ static int arm_smmu_alloc_init_pte(struct arm_smmu_device *smmu, pmd_t *pmd, if (!table) return -ENOMEM; - arm_smmu_flush_pgtable(smmu, page_address(table), PAGE_SIZE); + arm_smmu_flush_pgtable(smmu_domain, page_address(table), + PAGE_SIZE); pmd_populate(NULL, pmd, table); - arm_smmu_flush_pgtable(smmu, pmd, sizeof(*pmd)); + arm_smmu_flush_pgtable(smmu_domain, pmd, sizeof(*pmd)); } if (stage == 1) { @@ -1558,7 +1564,7 @@ static int arm_smmu_alloc_init_pte(struct arm_smmu_device *smmu, pmd_t *pmd, pte_val(*(cont_start + j)) &= ~ARM_SMMU_PTE_CONT; - arm_smmu_flush_pgtable(smmu, cont_start, + arm_smmu_flush_pgtable(smmu_domain, cont_start, sizeof(*pte) * ARM_SMMU_PTE_CONT_ENTRIES); } @@ -1568,11 +1574,13 @@ static int arm_smmu_alloc_init_pte(struct arm_smmu_device *smmu, pmd_t *pmd, } while (pte++, pfn++, addr += PAGE_SIZE, --i); } while (addr != end); - arm_smmu_flush_pgtable(smmu, start, sizeof(*pte) * (pte - start)); + arm_smmu_flush_pgtable(smmu_domain, start, + sizeof(*pte) * (pte - start)); return 0; } -static int arm_smmu_alloc_init_pmd(struct arm_smmu_device *smmu, pud_t *pud, +static int arm_smmu_alloc_init_pmd(struct arm_smmu_domain *smmu_domain, + pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys, int prot, int stage) { @@ -1586,9 +1594,9 @@ static int arm_smmu_alloc_init_pmd(struct arm_smmu_device *smmu, pud_t *pud, if (!pmd) return -ENOMEM; - arm_smmu_flush_pgtable(smmu, pmd, PAGE_SIZE); + arm_smmu_flush_pgtable(smmu_domain, pmd, PAGE_SIZE); pud_populate(NULL, pud, pmd); - arm_smmu_flush_pgtable(smmu, pud, sizeof(*pud)); + arm_smmu_flush_pgtable(smmu_domain, pud, sizeof(*pud)); pmd += pmd_index(addr); } else @@ -1597,7 +1605,7 @@ static int arm_smmu_alloc_init_pmd(struct arm_smmu_device *smmu, pud_t *pud, do { next = pmd_addr_end(addr, end); - ret = arm_smmu_alloc_init_pte(smmu, pmd, addr, next, pfn, + ret = arm_smmu_alloc_init_pte(smmu_domain, pmd, addr, next, pfn, prot, stage); phys += next - addr; } while (pmd++, addr = next, addr < end); @@ -1605,7 +1613,8 @@ static int arm_smmu_alloc_init_pmd(struct arm_smmu_device *smmu, pud_t *pud, return ret; } -static int arm_smmu_alloc_init_pud(struct arm_smmu_device *smmu, pgd_t *pgd, +static int arm_smmu_alloc_init_pud(struct arm_smmu_domain *smmu_domain, + pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys, int prot, int stage) { @@ -1619,9 +1628,9 @@ static int arm_smmu_alloc_init_pud(struct arm_smmu_device *smmu, pgd_t *pgd, if (!pud) return -ENOMEM; - arm_smmu_flush_pgtable(smmu, pud, PAGE_SIZE); + arm_smmu_flush_pgtable(smmu_domain, pud, PAGE_SIZE); pgd_populate(NULL, pgd, pud); - arm_smmu_flush_pgtable(smmu, pgd, sizeof(*pgd)); + arm_smmu_flush_pgtable(smmu_domain, pgd, sizeof(*pgd)); pud += pud_index(addr); } else @@ -1630,8 +1639,8 @@ static int arm_smmu_alloc_init_pud(struct arm_smmu_device *smmu, pgd_t *pgd, do { next = pud_addr_end(addr, end); - ret = arm_smmu_alloc_init_pmd(smmu, pud, addr, next, phys, - prot, stage); + ret = arm_smmu_alloc_init_pmd(smmu_domain, pud, addr, next, + phys, prot, stage); phys += next - addr; } while (pud++, addr = next, addr < end); @@ -1677,8 +1686,8 @@ static int arm_smmu_handle_mapping(struct arm_smmu_domain *smmu_domain, do { unsigned long next = pgd_addr_end(iova, end); - ret = arm_smmu_alloc_init_pud(smmu, pgd, iova, next, paddr, - prot, stage); + ret = arm_smmu_alloc_init_pud(smmu_domain, pgd, iova, next, + paddr, prot, stage); if (ret) goto out_unlock; @@ -1908,11 +1917,15 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr attr, void *data) { struct arm_smmu_domain *smmu_domain = domain->priv; + struct arm_smmu_cfg *cfg = &smmu_domain->cfg; switch (attr) { case DOMAIN_ATTR_NESTING: *(int *)data = (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED); return 0; + case DOMAIN_ATTR_COHERENT_HTW_DISABLE: + *((bool *)data) = cfg->htw_disable; + return 0; default: return -ENODEV; } @@ -1922,6 +1935,7 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr attr, void *data) { struct arm_smmu_domain *smmu_domain = domain->priv; + struct arm_smmu_cfg *cfg = &smmu_domain->cfg; switch (attr) { case DOMAIN_ATTR_NESTING: @@ -1933,6 +1947,9 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain, smmu_domain->stage = ARM_SMMU_DOMAIN_S1; return 0; + case DOMAIN_ATTR_COHERENT_HTW_DISABLE: + cfg->htw_disable = *((bool *)data); + return 0; default: return -ENODEV; } diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 0550286df4..8a6449857a 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -81,6 +81,7 @@ enum iommu_attr { DOMAIN_ATTR_FSL_PAMU_ENABLE, DOMAIN_ATTR_FSL_PAMUV1, DOMAIN_ATTR_NESTING, /* two stages of translation */ + DOMAIN_ATTR_COHERENT_HTW_DISABLE, DOMAIN_ATTR_MAX, };