From patchwork Wed Aug 14 09:23:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yicong Yang X-Patchwork-Id: 13763171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08D97C52D7F for ; Wed, 14 Aug 2024 09:35:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eWc+JuAKAf4gYw2f6rIC0rDmWietzcFUsSFnp5+VcbM=; b=tB+Wy+jbieTo3NnxxrFGkdX/Uz L5GLpmlwFSIip1spyfoMTqB5jMJHRFNpkWonybIh1k8g/pugwiMUrR+00LLKNJ0jAdHHd1U7rBKDw TrO2/tMoMn243bqsQRYgjbicfdem0gy7SWfw+ogTm56wu9shSr0t6rBS+kJ7zNCCG3FZjxzo3zLgt Fj58Q5hx90BSzDMvtINjYFf5ABUl7y/f3kCrWhT5J0DCq2/mkeRvSzOfJBgxtRDR+5EO60SF+8WwS H8ed5TLtLvpB3ftNR2MoTgjCd+xdI6DiNg/yGLyrlMUX0UkdGAxKmuCw+bA7fC322vvr3pRA4Xt2K veQJAy7A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1seAPB-00000006Q0e-1M8M; Wed, 14 Aug 2024 09:34:57 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1seAKQ-00000006OMp-03zH for linux-arm-kernel@lists.infradead.org; Wed, 14 Aug 2024 09:30:04 +0000 Received: from mail.maildlp.com (unknown [172.19.162.254]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4WkNHg5SKBzyPmt; Wed, 14 Aug 2024 17:29:19 +0800 (CST) Received: from kwepemd200014.china.huawei.com (unknown [7.221.188.8]) by mail.maildlp.com (Postfix) with ESMTPS id B1DDB1800D0; Wed, 14 Aug 2024 17:29:49 +0800 (CST) Received: from localhost.localdomain (10.50.165.33) by kwepemd200014.china.huawei.com (7.221.188.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.34; Wed, 14 Aug 2024 17:29:49 +0800 From: Yicong Yang To: , , , , CC: , , , , , , , , , Subject: [PATCH v2 1/2] arm64: Add support for FEAT_HAFT Date: Wed, 14 Aug 2024 17:23:32 +0800 Message-ID: <20240814092333.7727-2-yangyicong@huawei.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20240814092333.7727-1-yangyicong@huawei.com> References: <20240814092333.7727-1-yangyicong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.165.33] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemd200014.china.huawei.com (7.221.188.8) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240814_023002_799666_3866706B X-CRM114-Status: GOOD ( 18.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Yicong Yang Armv8.9/v9.4 introduces the feature Hardware managed Access Flag for Table descriptors (FEAT_HAFT). The feature is indicated by ID_AA64MMFR1_EL1.HAFDBS == 0b0011 and can be enabled by TCR2_EL1.HAFT so it has a dependency on FEAT_TCR2. This patch adds the Kconfig for FEAT_HAFT and support detecting and enabling the feature. Signed-off-by: Yicong Yang --- arch/arm64/Kconfig | 19 +++++++++++++++++++ arch/arm64/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + arch/arm64/tools/sysreg | 1 + 4 files changed, 47 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a2f8ff354ca6..869792458a23 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2137,6 +2137,25 @@ config ARM64_EPAN if the cpu does not implement the feature. endmenu # "ARMv8.7 architectural features" +menu "ARMv8.9 architectural features" + +config ARM64_HAFT + bool "Support for Hardware managed Access Flag for Table Descriptor" + depends on ARM64_HW_AFDBM + default y + help + The ARMv8.9/ARMv9.5 introduces the feature Hardware managed Access + Flag for Table descriptors. When enabled an architectural executed + memory access will update the Access Flag in each Table descriptor + which is accessed during the translation table walk and for which + the Access Flag is 0. The Access Flag of the Table descriptor use + the same bit of PTE_AF. + + The feature will only be enabled if all the CPUs in the system + support this feature. If unsure, say Y. + +endmenu # "ARMv8.9 architectural features" + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 646ecd3069fd..ed4c968be935 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2044,6 +2044,17 @@ static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap, #endif +#if CONFIG_ARM64_HAFT + +static void cpu_enable_haft(struct arm64_cpu_capabilities const *cap) +{ + sysreg_clear_set_s(SYS_TCR2_EL1, 0, TCR2_EL1x_HAFT); + isb(); + local_flush_tlb_all(); +} + +#endif + #ifdef CONFIG_ARM64_AMU_EXTN /* @@ -2580,6 +2591,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpus = &dbm_cpus, ARM64_CPUID_FIELDS(ID_AA64MMFR1_EL1, HAFDBS, DBM) }, +#endif +#ifdef CONFIG_ARM64_HAFT + { + .desc = "Hardware managed Access Flag for Table Descriptor", + /* + * Contrary to the page/block access flag, the table access flag + * cannot be emulated in software (no access fault will occur). + * Therefore mandate that all CPUs have FEAT_HAFT. + */ + .type = ARM64_CPUCAP_BOOT_CPU_FEATURE, + .capability = ARM64_HAFT, + .matches = has_cpuid_feature, + .cpu_enable = cpu_enable_haft, + ARM64_CPUID_FIELDS(ID_AA64MMFR1_EL1, HAFDBS, HAFT) + }, #endif { .desc = "CRC32 instructions", diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index ac3429d892b9..0b7a3a237e5d 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -55,6 +55,7 @@ HAS_TLB_RANGE HAS_VA52 HAS_VIRT_HOST_EXTN HAS_WFXT +HAFT HW_DBM KVM_HVHE KVM_PROTECTED_MODE diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 7ceaa1e0b4bc..9b3d15ea8a63 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -1688,6 +1688,7 @@ UnsignedEnum 3:0 HAFDBS 0b0000 NI 0b0001 AF 0b0010 DBM + 0b0011 HAFT EndEnum EndSysreg From patchwork Wed Aug 14 09:23:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yicong Yang X-Patchwork-Id: 13763172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8C5E0C3DA4A for ; Wed, 14 Aug 2024 09:35:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jRaEcT3Oy5C4IMsvoDIjzpydaYwoz1bAZFusjcAvFSA=; b=XU5JXhJ47m5KXC9OQ7wnPfgKzi XE060GAHejs5q7McE6PHQ7cS4HzjjPBG/QIXmgPsIvPAIT1Pz2jnFQiog4ZtwNbvcK78WoVkbvs7C Z8nNHv116Hxe19yGe5AeXzuA0XUacrDOacilvpoIAkwx+wIpwZUlId00HNL86x4V+nQN54mpVsRXf /qlx8HtIxA3NuGowgsiXcD8aEaZo74S3mqE5ZpaYLoMe6pV89UU8iaBpFlrA/kBCX+XVSX9uTYQIB BMtEQislL2erovNEJv++mM1kusz7zrALbKIIfA9pdet+aCIs6SrKAeqqUV88lJ77B/JLDIb3txgsD TRWoamEQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1seAPk-00000006Q65-3gA6; Wed, 14 Aug 2024 09:35:32 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1seAKb-00000006OUR-1iy3 for linux-arm-kernel@bombadil.infradead.org; Wed, 14 Aug 2024 09:30:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Content-Transfer-Encoding :MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From: Sender:Reply-To:Content-ID:Content-Description; bh=jRaEcT3Oy5C4IMsvoDIjzpydaYwoz1bAZFusjcAvFSA=; b=SHrZM72d1/t1QIKJTR+Khtmvoe lrgtmp3Y0l6q/AeXvsDUCEHPkhrGvWA/fFZQ56UVZOuxfJg8V5oVF+ubtd1pEnsLO/8AYVPHP4IHD YQ8MTjZ4JM4/bzxnJ+6N5U22hfqY513p5oS8/+wsmJsC4T1xOMSdjn1gItblYlXpV1xhPfY+biPRD AiBLxy5o+HVQuTWGtih6eR9uQX1CBRlj3VrWSokUU8QkiKDgwnyFfOYCgz0NoD/0J308X50QYB+hS BZlDvBSPhKS1MIQAPWXwCx9GbTErvanhGcS6ELziEpK1b1HRoHT4wUgmGoUuMfZEioXmUzmJWwKVf ElA7BM8Q==; Received: from szxga05-in.huawei.com ([45.249.212.191]) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1seAKW-000000080U7-3lw9 for linux-arm-kernel@lists.infradead.org; Wed, 14 Aug 2024 09:30:12 +0000 Received: from mail.maildlp.com (unknown [172.19.88.214]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4WkNBf3rN4z1j6Rq; Wed, 14 Aug 2024 17:24:58 +0800 (CST) Received: from kwepemd200014.china.huawei.com (unknown [7.221.188.8]) by mail.maildlp.com (Postfix) with ESMTPS id 47EBB1A016C; Wed, 14 Aug 2024 17:29:50 +0800 (CST) Received: from localhost.localdomain (10.50.165.33) by kwepemd200014.china.huawei.com (7.221.188.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.34; Wed, 14 Aug 2024 17:29:49 +0800 From: Yicong Yang To: , , , , CC: , , , , , , , , , Subject: [PATCH v2 2/2] arm64: Enable ARCH_HAS_NONLEAF_PMD_YOUNG Date: Wed, 14 Aug 2024 17:23:33 +0800 Message-ID: <20240814092333.7727-3-yangyicong@huawei.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20240814092333.7727-1-yangyicong@huawei.com> References: <20240814092333.7727-1-yangyicong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.165.33] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemd200014.china.huawei.com (7.221.188.8) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240814_103010_303982_7E9832C9 X-CRM114-Status: GOOD ( 11.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Yicong Yang With the support of FEAT_HAFT, the NONLEAF_PMD_YOUNG can be enabled on arm64 since the hardware is capable of updating the AF flag for PMD table descriptor. Since the AF bit of the table descriptor shares the same bit position in block descriptors, we only need to implement arch_has_hw_nonleaf_pmd_young() and select related configs. The related pmd_young test/update operations keeps the same with and already implemented for transparent page support. Currently ARCH_HAS_NONLEAF_PMD_YOUNG is used to improve the efficiency of lru-gen aging. Signed-off-by: Yicong Yang --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable.h | 14 ++++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 869792458a23..74451a7ccc4f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -36,6 +36,7 @@ config ARM64 select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_NMI_SAFE_THIS_CPU_OPS select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE + select ARCH_HAS_NONLEAF_PMD_YOUNG if ARM64_HAFT select ARCH_HAS_PTE_DEVMAP select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_HW_PTE_YOUNG diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7a4f5604be3f..077bea37867e 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1205,7 +1205,7 @@ static inline int __ptep_clear_flush_young(struct vm_area_struct *vma, return young; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG) #define __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, @@ -1213,7 +1213,7 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma, { return __ptep_test_and_clear_young(vma, address, (pte_t *)pmdp); } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG */ static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, unsigned long address, pte_t *ptep) @@ -1448,6 +1448,16 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf, */ #define arch_has_hw_pte_young cpu_has_hw_af +#ifdef CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG + +#define arch_has_hw_nonleaf_pmd_young arch_has_hw_nonleaf_pmd_young +static inline bool arch_has_hw_nonleaf_pmd_young(void) +{ + return cpus_have_final_cap(ARM64_HAFT); +} + +#endif + /* * Experimentally, it's cheap to set the access flag in hardware and we * benefit from prefaulting mappings as 'old' to start with.