From patchwork Mon Feb 12 18:33:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10214397 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AF99F60467 for ; Mon, 12 Feb 2018 20:40:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0BF028A18 for ; Mon, 12 Feb 2018 20:40:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 90CAF28D1F; Mon, 12 Feb 2018 20:40:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 80E1F28A18 for ; Mon, 12 Feb 2018 20:40:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NeHm0LjdqwXi+QGmeTimC8/QmGIvxWLXzIDFUA9hM8s=; b=kV9QAbiNLW9zqELL4HgS5ufMMK 3ZF6kxPapvoZ24nIugbxq5PY2SJw/eEvC99coK8x9RNFsOO2gifkGlWgRv3XLEzpOLvBGaQEJwC9+ 5oa6RplRJscEjf/MFOxo7Xlf3PWJkxGoRCNams9Nn8OCYC+OOCxYwe50dvhePCiPnkOqNmjTGFqxC c23Yazeu4qNP4g023XQQoi0RxW6fITan0144gKwX0/Io0aUODd0/KAEiiJE4NRm+hrnp4OiXklyX/ yNkcUKmu9IIuCdxx1N1Wsz8awl2UdtCL/ijYSN1+DMqt81YNB8a9Wy8ixh1QTVHoLhAudj/wUjitK cfpuA51w==; Received: from [198.137.202.133] (helo=bombadil.infradead.org) by casper.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1elKtb-0000Ng-7i; Mon, 12 Feb 2018 20:40:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NeHm0LjdqwXi+QGmeTimC8/QmGIvxWLXzIDFUA9hM8s=; b=ViMu4STDTtb2UsLqy9AwblK5IM R+1oqDVfOda0xWU4wMsHtJcNcAIgNjuF2pomALclg/0PSEL9v5QD78D60/UhPGgYG+/AD83S1Smzq i9rvYBgtFpOe3WjnVnhPd/b+BNDnrrDZHylABJIhyIHOSgNR8+yR3EqXD9/fCk44fNFCNR0tOReen j3TCh5VepUsGtDLIbLvK4icvcW7pGQlMJxvLDveIKtvew+888xjMGI0G9TTXm3I/z9/fdRLekQ5X+ SQcUOftIhmK87PxLKR/NhBv/Ac2csJmFimdCXAaBBvABPLfJWNnWA7B5Rp6o1jVRBmN0pRP1j38oa 1gHYvmCg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1elJ27-0003oM-N4; Mon, 12 Feb 2018 18:40:55 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1elIv7-0005Q0-Fu for linux-arm-kernel@lists.infradead.org; Mon, 12 Feb 2018 18:33:43 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B08F0165C; Mon, 12 Feb 2018 10:33:30 -0800 (PST) Received: from e106794-lin.cambridge.arm.com (e106794-lin.cambridge.arm.com [10.1.210.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D20B73F24D; Mon, 12 Feb 2018 10:33:24 -0800 (PST) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org Subject: [PATCH 26/37] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update Date: Mon, 12 Feb 2018 18:33:41 +0000 Message-Id: <20180212183352.22730-27-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180212183352.22730-1-jean-philippe.brucker@arm.com> References: <20180212183352.22730-1-jean-philippe.brucker@arm.com> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, xieyisheng1@huawei.com, ilias.apalodimas@linaro.org, catalin.marinas@arm.com, xuzaibo@huawei.com, jonathan.cameron@huawei.com, will.deacon@arm.com, okaya@codeaurora.org, yi.l.liu@intel.com, lorenzo.pieralisi@arm.com, ashok.raj@intel.com, tn@semihalf.com, joro@8bytes.org, bharatku@xilinx.com, rfranz@cavium.com, lenb@kernel.org, jacob.jun.pan@linux.intel.com, alex.williamson@redhat.com, robh+dt@kernel.org, thunder.leizhen@huawei.com, bhelgaas@google.com, shunyong.yang@hxt-semitech.com, dwmw2@infradead.org, liubo95@huawei.com, rjw@rjwysocki.net, jcrouse@codeaurora.org, robdclark@gmail.com, hanjun.guo@linaro.org, sudeep.holla@arm.com, robin.murphy@arm.com, christian.koenig@amd.com, nwatters@codeaurora.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP If the SMMU supports it and the kernel was built with HTTU support, enable hardware update of access and dirty flags. This is essential for shared page tables, to reduce the number of access faults on the fault queue. We can still enable HTTU if CPUs don't support it, because the kernel always checks for HW dirty bit and updates the PTE flags atomically. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3-context.c | 20 ++++++++++++++++++-- drivers/iommu/arm-smmu-v3.c | 12 ++++++++++++ drivers/iommu/iommu-pasid.h | 4 ++++ 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3-context.c b/drivers/iommu/arm-smmu-v3-context.c index 5b8c5875e0d9..eaeba1bec2e9 100644 --- a/drivers/iommu/arm-smmu-v3-context.c +++ b/drivers/iommu/arm-smmu-v3-context.c @@ -62,7 +62,16 @@ #define ARM64_TCR_TBI0_SHIFT 37 #define ARM64_TCR_TBI0_MASK 0x1UL +#define ARM64_TCR_HA_SHIFT 39 +#define ARM64_TCR_HA_MASK 0x1UL +#define ARM64_TCR_HD_SHIFT 40 +#define ARM64_TCR_HD_MASK 0x1UL + #define CTXDESC_CD_0_AA64 (1UL << 41) +#define CTXDESC_CD_0_TCR_HD_SHIFT 42 +#define CTXDESC_CD_0_TCR_HA_SHIFT 43 +#define CTXDESC_CD_0_HD (1UL << CTXDESC_CD_0_TCR_HD_SHIFT) +#define CTXDESC_CD_0_HA (1UL << CTXDESC_CD_0_TCR_HA_SHIFT) #define CTXDESC_CD_0_S (1UL << 44) #define CTXDESC_CD_0_R (1UL << 45) #define CTXDESC_CD_0_A (1UL << 46) @@ -199,7 +208,7 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_cd_tables *tbl, u32 ssid) return l1_desc->ptr + idx * CTXDESC_CD_DWORDS; } -static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) +static u64 arm_smmu_cpu_tcr_to_cd(struct arm_smmu_context_cfg *cfg, u64 tcr) { u64 val = 0; @@ -214,6 +223,12 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) val |= ARM_SMMU_TCR2CD(tcr, IPS); val |= ARM_SMMU_TCR2CD(tcr, TBI0); + if (cfg->hw_access) + val |= ARM_SMMU_TCR2CD(tcr, HA); + + if (cfg->hw_dirty) + val |= ARM_SMMU_TCR2CD(tcr, HD); + return val; } @@ -269,7 +284,7 @@ static int __arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid, iommu_pasid_flush(&tbl->pasid, ssid, true); - val = arm_smmu_cpu_tcr_to_cd(cd->tcr) | + val = arm_smmu_cpu_tcr_to_cd(cfg, cd->tcr) | #ifdef __BIG_ENDIAN CTXDESC_CD_0_ENDI | #endif @@ -460,6 +475,7 @@ arm_smmu_alloc_shared_cd(struct iommu_pasid_table_ops *ops, struct mm_struct *mm reg = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1); par = cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR0_PARANGE_SHIFT); tcr |= par << ARM_LPAE_TCR_IPS_SHIFT; + tcr |= TCR_HA | TCR_HD; cd->ttbr = virt_to_phys(mm->pgd); cd->tcr = tcr; diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 1cdeea7e22cb..8528704627b5 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -67,6 +67,8 @@ #define IDR0_ASID16 (1 << 12) #define IDR0_ATS (1 << 10) #define IDR0_HYP (1 << 9) +#define IDR0_HD (1 << 7) +#define IDR0_HA (1 << 6) #define IDR0_BTM (1 << 5) #define IDR0_COHACC (1 << 4) #define IDR0_TTF_SHIFT 2 @@ -573,6 +575,8 @@ struct arm_smmu_device { #define ARM_SMMU_FEAT_E2H (1 << 14) #define ARM_SMMU_FEAT_BTM (1 << 15) #define ARM_SMMU_FEAT_SVA (1 << 16) +#define ARM_SMMU_FEAT_HA (1 << 17) +#define ARM_SMMU_FEAT_HD (1 << 18) u32 features; #define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0) @@ -1631,6 +1635,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, .arm_smmu = { .stall = !!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE), .asid_bits = smmu->asid_bits, + .hw_access = !!(smmu->features & ARM_SMMU_FEAT_HA), + .hw_dirty = !!(smmu->features & ARM_SMMU_FEAT_HD), }, }; @@ -2865,6 +2871,12 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) smmu->features |= ARM_SMMU_FEAT_E2H; } + if (reg & (IDR0_HA | IDR0_HD)) { + smmu->features |= ARM_SMMU_FEAT_HA; + if (reg & IDR0_HD) + smmu->features |= ARM_SMMU_FEAT_HD; + } + /* * If the CPU is using VHE, but the SMMU doesn't support it, the SMMU * will create TLB entries for NH-EL1 world and will miss the diff --git a/drivers/iommu/iommu-pasid.h b/drivers/iommu/iommu-pasid.h index 77e449a1655b..46fd44e7f4f1 100644 --- a/drivers/iommu/iommu-pasid.h +++ b/drivers/iommu/iommu-pasid.h @@ -79,6 +79,8 @@ struct iommu_pasid_sync_ops { * * SMMU properties: * @stall: devices attached to the domain are allowed to stall. + * @hw_dirty: hardware may update dirty flag + * @hw_access: hardware may update access flag * @asid_bits: number of ASID bits supported by the SMMU * * @s1fmt: PASID table format, chosen by the allocator. @@ -86,6 +88,8 @@ struct iommu_pasid_sync_ops { struct arm_smmu_context_cfg { u8 stall:1; u8 asid_bits; + u8 hw_dirty:1; + u8 hw_access:1; #define ARM_SMMU_S1FMT_LINEAR 0x0 #define ARM_SMMU_S1FMT_4K_L2 0x1