From patchwork Fri May 11 19:06:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10395013 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 425B16053A for ; Fri, 11 May 2018 19:10:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 311E328F8B for ; Fri, 11 May 2018 19:10:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 24CB328F95; Fri, 11 May 2018 19:10:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 876FE28F8E for ; Fri, 11 May 2018 19:10:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 473716B06B6; Fri, 11 May 2018 15:10:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 41FFE6B06B8; Fri, 11 May 2018 15:10:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C3DE6B06B9; Fri, 11 May 2018 15:10:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi0-f70.google.com (mail-oi0-f70.google.com [209.85.218.70]) by kanga.kvack.org (Postfix) with ESMTP id F23526B06B6 for ; Fri, 11 May 2018 15:10:13 -0400 (EDT) Received: by mail-oi0-f70.google.com with SMTP id k136-v6so3492816oih.4 for ; Fri, 11 May 2018 12:10:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=2GSbL2bzKzcfRwPg/E1gkMptRdi1so+Ior4EC9nb3zY=; b=PpY8uqBGcyoIpCmPplJl5lyyxzo1pRng1FODKZhILj92IVX9JoB9r2Boavna/kHkek YOUHKROCTGSvNVGGSJeTtjz21qbQ/RCISXgAfj6wE9yylbTdlQ7GmfJ8fr1tlf347REt /dtezzBY1rJy/SVDdFFKkRLCphhxx0Tl6MBBkOA2BOpUkWgC1+cIUGvjiGOQa4ZrclIN aErD2igB7MB8rfHDWKEpFhOYQDdQFdyQnCYqT9ih2FZ9V8VNhGcmcNFp+15twQywfz7W A+HzzWwTWDhJuKW1mqDZrXrimy/mt7qlEfOjjI24THrT5d7LSPt4v0oHsyA96PCyVJHF wTGg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=jean-philippe.brucker@arm.com X-Gm-Message-State: ALKqPwdMdsipKgarjRwqAVDgECx2ySizbU8w8/EdchUweLd9YDBjws5n ea5xeUKU9O/2uExOaFDRKQl2593jAtxHoP0VIRYBgpLd1sIGGytAordmJ3awOWOCJcToMcoCSCB SKPdrj5qUIhR/ml5sQlzetVUCGrsigY19y0NhCBoZqVKUb0M9t93GifEYs5LT006t/Q== X-Received: by 2002:aca:e4d1:: with SMTP id b200-v6mr32519oih.242.1526065813773; Fri, 11 May 2018 12:10:13 -0700 (PDT) X-Google-Smtp-Source: AB8JxZq3J9aii3raR4miYUnnWEKb6AjBz1piTyyLQo31z6pTUIJMJOVTP5FEQgD4aoKhweBPwzy+ X-Received: by 2002:aca:e4d1:: with SMTP id b200-v6mr32488oih.242.1526065813020; Fri, 11 May 2018 12:10:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526065812; cv=none; d=google.com; s=arc-20160816; b=N/RKhEbdAji9gn0qFN8jCJWlzmXkkt9Q7a2C2q9hHyjTpNUUlG73kT8AevxPc2as8s YBbDUTTzuMmm/G0Nheh9Pj3VO2H9B7Kzm+uXaWMdRuY4Nx4dicGpG159enOvNghSUoS8 W1IpSZMbTLIvH7w5NsqTDsZAcKYtuzRF6QMK/TiFCVQwH2EMJruhYZ86L5nQ21Yo76I4 bhZInvIqIPlqqbUoXMeinEDiCRoVbqVxn3Fa6hYc59D05TvGfPCkEI5JbV7zzdKfZZl7 Kx14SXDwG2jfwJJYdx7JSXeDqJEustHXwuOQ2m7Xk4L+yiLcYcZ81xtS3ry2JJ2E1uv3 Y+9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=2GSbL2bzKzcfRwPg/E1gkMptRdi1so+Ior4EC9nb3zY=; b=IlW+mD5s1dEgNw6Iwp4VOjSCvDQ6Pu/X+Goo8DrJiqmzLqNbY3SWvYe6XNz9S8Pb5q ZV4k0vjzswFO/YdjTbNSwEpaGqte1PqLngb2gxTBHBptDXFK/xw1FbZmEV1bKgTNI1mu ntT+31PrkxC3ryAFbpHX781VuQcgl/r9G3uTdGpfxpewTNz95nVNKAJOVngiQ4gjTF4m IyS/TJIJoByR0nEQSMnQ4cr0nsNdgcE76/YETyIkkgnDWSIwv86Zmv0UOYwFP23lY7af 1XrQY8koud3HhIB8dRI2LI9M/FvUp9vSMoodrhIJya3OVplMfNm244/tqPasNjP36lkK rnRA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=jean-philippe.brucker@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id t32-v6si1396630oti.338.2018.05.11.12.10.12 for ; Fri, 11 May 2018 12:10:12 -0700 (PDT) Received-SPF: pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jean-philippe.brucker@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=jean-philippe.brucker@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 949DC1713; Fri, 11 May 2018 12:10:12 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.210.33]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 56B793F23C; Fri, 11 May 2018 12:10:07 -0700 (PDT) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-mm@kvack.org Cc: joro@8bytes.org, will.deacon@arm.com, robin.murphy@arm.com, alex.williamson@redhat.com, tn@semihalf.com, liubo95@huawei.com, thunder.leizhen@huawei.com, xieyisheng1@huawei.com, xuzaibo@huawei.com, ilias.apalodimas@linaro.org, jonathan.cameron@huawei.com, liudongdong3@huawei.com, shunyong.yang@hxt-semitech.com, nwatters@codeaurora.org, okaya@codeaurora.org, jcrouse@codeaurora.org, rfranz@cavium.com, dwmw2@infradead.org, jacob.jun.pan@linux.intel.com, yi.l.liu@intel.com, ashok.raj@intel.com, kevin.tian@intel.com, baolu.lu@linux.intel.com, robdclark@gmail.com, christian.koenig@amd.com, bharatku@xilinx.com, rgummal@xilinx.com Subject: [PATCH v2 29/40] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update Date: Fri, 11 May 2018 20:06:30 +0100 Message-Id: <20180511190641.23008-30-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker@arm.com> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP If the SMMU supports it and the kernel was built with HTTU support, enable hardware update of access and dirty flags. This is essential for shared page tables, to reduce the number of access faults on the fault queue. We can enable HTTU even if CPUs don't support it, because the kernel always checks for HW dirty bit and updates the PTE flags atomically. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3-context.c | 16 ++++++++++++++-- drivers/iommu/arm-smmu-v3.c | 12 ++++++++++++ drivers/iommu/iommu-pasid-table.h | 4 ++++ 3 files changed, 30 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3-context.c b/drivers/iommu/arm-smmu-v3-context.c index 0e12f6804e16..bdc9bfd1f35d 100644 --- a/drivers/iommu/arm-smmu-v3-context.c +++ b/drivers/iommu/arm-smmu-v3-context.c @@ -52,6 +52,11 @@ #define CTXDESC_CD_0_TCR_TBI0 (1ULL << 38) #define ARM64_TCR_TBI0 (1ULL << 37) +#define CTXDESC_CD_0_TCR_HA (1UL << 43) +#define ARM64_TCR_HA (1ULL << 39) +#define CTXDESC_CD_0_TCR_HD (1UL << 42) +#define ARM64_TCR_HD (1ULL << 40) + #define CTXDESC_CD_0_AA64 (1UL << 41) #define CTXDESC_CD_0_S (1UL << 44) #define CTXDESC_CD_0_R (1UL << 45) @@ -182,7 +187,7 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_cd_tables *tbl, u32 ssid) return l1_desc->ptr + idx * CTXDESC_CD_DWORDS; } -static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) +static u64 arm_smmu_cpu_tcr_to_cd(struct arm_smmu_context_cfg *cfg, u64 tcr) { u64 val = 0; @@ -197,6 +202,12 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) val |= ARM_SMMU_TCR2CD(tcr, IPS); val |= ARM_SMMU_TCR2CD(tcr, TBI0); + if (cfg->hw_access) + val |= ARM_SMMU_TCR2CD(tcr, HA); + + if (cfg->hw_dirty) + val |= ARM_SMMU_TCR2CD(tcr, HD); + return val; } @@ -250,7 +261,7 @@ static int __arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid, iommu_pasid_flush(&tbl->pasid, ssid, true); - val = arm_smmu_cpu_tcr_to_cd(cd->tcr) | + val = arm_smmu_cpu_tcr_to_cd(cfg, cd->tcr) | #ifdef __BIG_ENDIAN CTXDESC_CD_0_ENDI | #endif @@ -455,6 +466,7 @@ arm_smmu_alloc_shared_cd(struct iommu_pasid_table_ops *ops, struct mm_struct *mm reg = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1); par = cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR0_PARANGE_SHIFT); tcr |= par << ARM_LPAE_TCR_IPS_SHIFT; + tcr |= TCR_HA | TCR_HD; cd->ttbr = virt_to_phys(mm->pgd); cd->tcr = tcr; diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index c2c96025ac3b..7c839d305d97 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -66,6 +66,8 @@ #define IDR0_ASID16 (1 << 12) #define IDR0_ATS (1 << 10) #define IDR0_HYP (1 << 9) +#define IDR0_HD (1 << 7) +#define IDR0_HA (1 << 6) #define IDR0_BTM (1 << 5) #define IDR0_COHACC (1 << 4) #define IDR0_TTF GENMASK(3, 2) @@ -528,6 +530,8 @@ struct arm_smmu_device { #define ARM_SMMU_FEAT_E2H (1 << 15) #define ARM_SMMU_FEAT_BTM (1 << 16) #define ARM_SMMU_FEAT_SVA (1 << 17) +#define ARM_SMMU_FEAT_HA (1 << 18) +#define ARM_SMMU_FEAT_HD (1 << 19) u32 features; #define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0) @@ -1567,6 +1571,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, .arm_smmu = { .stall = !!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE), .asid_bits = smmu->asid_bits, + .hw_access = !!(smmu->features & ARM_SMMU_FEAT_HA), + .hw_dirty = !!(smmu->features & ARM_SMMU_FEAT_HD), }, }; @@ -2818,6 +2824,12 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) smmu->features |= ARM_SMMU_FEAT_E2H; } + if (reg & (IDR0_HA | IDR0_HD)) { + smmu->features |= ARM_SMMU_FEAT_HA; + if (reg & IDR0_HD) + smmu->features |= ARM_SMMU_FEAT_HD; + } + /* * If the CPU is using VHE, but the SMMU doesn't support it, the SMMU * will create TLB entries for NH-EL1 world and will miss the diff --git a/drivers/iommu/iommu-pasid-table.h b/drivers/iommu/iommu-pasid-table.h index b84709e297bc..a7243579a4cb 100644 --- a/drivers/iommu/iommu-pasid-table.h +++ b/drivers/iommu/iommu-pasid-table.h @@ -78,12 +78,16 @@ struct iommu_pasid_sync_ops { * SMMU properties: * @stall: devices attached to the domain are allowed to stall. * @asid_bits: number of ASID bits supported by the SMMU + * @hw_dirty: hardware may update dirty flag + * @hw_access: hardware may update access flag * * @s1fmt: PASID table format, chosen by the allocator. */ struct arm_smmu_context_cfg { u8 stall:1; u8 asid_bits; + u8 hw_dirty:1; + u8 hw_access:1; #define ARM_SMMU_S1FMT_LINEAR 0x0 #define ARM_SMMU_S1FMT_4K_L2 0x1