From patchwork Thu Nov 24 12:39:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13054923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 979C7C43217 for ; Thu, 24 Nov 2022 12:41:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=9kiaeZn5RvrxBjrh07qYIOmRKnzlFIDWOgKBZr068yk=; b=qjG2aC+I97KaUI snAmZGFb8fyOgoFLSzHgKjkWHldMgCv8lMogK549GwF+gpusHMrQ+6UgtRayuGzSC81r2UJXMHXAF zhR+roNs9wfcHh1L1wsRWKG2QhE31KUKnZnQuFzMhmjAz6NNJRzhKMFS+k8WzIratcEg8+9ArOU8U CcqWr04VOelXtOnGFyrf/UPltRvVvpQx4Nrx9jWrwY0V3sgQiy9/q+sVXBnPn5TBaFGpLlQ9PUvX7 IxCDIdbiAYr6CiBAdjtsutrVgNfkoQte3unMbr8K3R+5Xn0vv8wz+HV3r8KFDWYsBr20nmlOWCYk/ lOyfcg4nDCh9nzEoqRwA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oyBWF-008OIQ-ME; Thu, 24 Nov 2022 12:39:55 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oyBWB-008OGr-Pd for linux-arm-kernel@lists.infradead.org; Thu, 24 Nov 2022 12:39:53 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5A1F462119; Thu, 24 Nov 2022 12:39:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 230F4C433D6; Thu, 24 Nov 2022 12:39:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669293590; bh=KirZTFUiK4kTziLKSiVWQHct2b0fmASRj+ItTrc2ra8=; h=From:To:Cc:Subject:Date:From; b=m+dIRbG20mrdOZce0CLfidhT1M70hzHvvIBgIve/eKiV1MNRh08oUUm/5kmK48EAH FMDuCO4Pz2ZtqbbCSfVcLbyB4GDiTyDRLEAJ59s50rdPLgebZDOWR3mgRohvUwQWmt PvSOzKsvVgnDqznR1hAmOHgj3NQXGdultjWIQ8TYcnDigAaU2Ng24XPF9oBpmO5o95 8xnI5oHxJtvPnsn0dFPjB122xsk0cpM3Oy39AClnd3IsjLz/aPv5Ivg4L9NVgNS7xh rzyqA9XTJScxEfVBO+NBhE6KiK4mECsStShwlHSpqkNQbXWTgDl/bB/6eE11WN7LJP djPe70hyVNF3Q== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown , Anshuman Khandual , Richard Henderson , Ryan Roberts Subject: [PATCH v2 00/19] arm64: Enable LPA2 support for 4k and 16k pages Date: Thu, 24 Nov 2022 13:39:13 +0100 Message-Id: <20221124123932.2648991-1-ardb@kernel.org> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221124_043951_928958_901182D0 X-CRM114-Status: GOOD ( 20.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Enable support for LPA2 when running with 4k or 16k pages. In the former case, this requires 5 level paging with a runtime fallback to 4 on non-LPA2 hardware. For consistency, the same approach is adopted for 16k pages, where we fall back to 3 level paging (47 bit virtual addressing) on non-LPA2 configurations. (Falling back to 48 bits would involve finding a workaround for the fact that we cannot construct a level 0 table covering 52 bits of VA space that appears aligned to its size in memory, and has the top 2 entries that represent the 48-bit region appearing at an alignment of 64 bytes, which is required by the architecture for TTBR address values. Also, using an additional level of paging to translate a single VA bit is wasteful in terms of TLB efficiency) This means support for falling back to 3 levels of paging at runtime when configured for 4 is also needed. Another thing worth to note is that the repurposed physical address bits in the page table descriptors were not RES0 before, and so there is now a big global switch (called TCR.DS) which controls how all page table descriptors are interpreted. This requires some extra care in the PTE conversion helpers, and additional handling in the boot code to ensure that we set TCR.DS safely if supported (and not overridden) Note that this series is mostly orthogonal to work by Anshuman done last year: this series assumes that 52-bit physical addressing is never needed to map the kernel image itself, and therefore that we never need ID map range extension to cover the kernel with a 5th level when running with 4. And given that the LPA2 architectural feature covers both the virtual and physical range extensions, where enabling the latter is required to enable the former, we can simplify things further by only enabling them as a pair. (I.e., 52-bit physical addressing cannot be enabled for 48-bit VA space or smaller) This series applies onto some of my previous work that is still in flight, so these patches will not apply in isolation. Complete branch can be found here: https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=arm64-4k-lpa2 It supersedes the RFC v1 I sent out last week, which covered 16k pages only. It also supersedes some related work I sent out in isolation before: [PATCH] arm64: mm: Enable KASAN for 16k/48-bit VA configurations [PATCH 0/3] arm64: mm: Model LVA support as a CPU feature Tested on QEMU with -cpu max and lpa2 both off and on, as well as using the arm64.nolva override kernel command line parameter. Note that this requires a QEMU built from the latest sources. Cc: Marc Zyngier Cc: Will Deacon Cc: Mark Rutland Cc: Kees Cook Cc: Catalin Marinas Cc: Mark Brown Cc: Anshuman Khandual Cc: Richard Henderson Cc: Ryan Roberts Anshuman Khandual (3): arm64/mm: Simplify and document pte_to_phys() for 52 bit addresses arm64/mm: Add FEAT_LPA2 specific TCR_EL1.DS field arm64/mm: Add FEAT_LPA2 specific ID_AA64MMFR0.TGRAN[2] Ard Biesheuvel (16): arm64: kaslr: Adjust randomization range dynamically arm64: mm: get rid of kimage_vaddr global variable arm64: head: remove order argument from early mapping routine arm64: mm: Handle LVA support as a CPU feature arm64: mm: Deal with potential ID map extension if VA_BITS > VA_BITS_MIN arm64: mm: Add feature override support for LVA arm64: mm: Wire up TCR.DS bit to PTE shareability fields arm64: mm: Add LPA2 support to phys<->pte conversion routines arm64: mm: Add definitions to support 5 levels of paging arm64: mm: add 5 level paging support to G-to-nG conversion routine arm64: Enable LPA2 at boot if supported by the system arm64: mm: Add 5 level paging support to fixmap and swapper handling arm64: kasan: Reduce minimum shadow alignment and enable 5 level paging arm64: mm: Add support for folding PUDs at runtime arm64: ptdump: Disregard unaddressable VA space arm64: Enable 52-bit virtual addressing for 4k and 16k granule configs arch/arm64/Kconfig | 23 ++- arch/arm64/include/asm/assembler.h | 42 ++--- arch/arm64/include/asm/cpufeature.h | 2 + arch/arm64/include/asm/fixmap.h | 1 + arch/arm64/include/asm/kernel-pgtable.h | 27 ++- arch/arm64/include/asm/memory.h | 23 ++- arch/arm64/include/asm/pgalloc.h | 53 +++++- arch/arm64/include/asm/pgtable-hwdef.h | 34 +++- arch/arm64/include/asm/pgtable-prot.h | 18 +- arch/arm64/include/asm/pgtable-types.h | 6 + arch/arm64/include/asm/pgtable.h | 197 ++++++++++++++++++-- arch/arm64/include/asm/sysreg.h | 2 + arch/arm64/include/asm/tlb.h | 3 +- arch/arm64/kernel/cpufeature.c | 46 ++++- arch/arm64/kernel/head.S | 99 +++++----- arch/arm64/kernel/image-vars.h | 4 + arch/arm64/kernel/pi/idreg-override.c | 29 ++- arch/arm64/kernel/pi/kaslr_early.c | 23 ++- arch/arm64/kernel/pi/map_kernel.c | 115 +++++++++++- arch/arm64/kernel/sleep.S | 3 - arch/arm64/mm/init.c | 2 +- arch/arm64/mm/kasan_init.c | 124 ++++++++++-- arch/arm64/mm/mmap.c | 4 + arch/arm64/mm/mmu.c | 138 ++++++++++---- arch/arm64/mm/pgd.c | 17 +- arch/arm64/mm/proc.S | 76 +++++++- arch/arm64/mm/ptdump.c | 4 +- arch/arm64/tools/cpucaps | 1 + 28 files changed, 907 insertions(+), 209 deletions(-)