From patchwork Fri Nov 11 17:11:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13040650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2759AC433FE for ; Fri, 11 Nov 2022 17:36:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vq+VJYmokM/LVWybEfGeWPtVvxg6zqAquWsOidle9zs=; b=VIt8Gzyw7V7bq2 5uS/D5niJuwYokDf4+LkIsYkpHt6bddSkx22jCmmHjAmbCjNY2iu5VJ6II6EqpcnmWz0a6xTPCy0P Vwx2DD0fcXdmUbGdJs8w/vrMsTomsHFyNFPplilJbMpUkT93dOfKTJ2VbKWYWuUCvE7mhXkh5wg5d 4x2uNwQO7LloPvvn6mdhTXb8qleBw3T6TggJCGrEQkYnP0WtBTVEEaREbpE9XnKD0k2lVcJi2CdVK VP3NY9a6sY18+KWSTnthgZd+Xb5Ev9vu6a3NAjURGaDYlnH11u8RjzVcjsSke7CDyCTpFNHol4gD4 Nii4SwwYBo6NxXYbGWjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otXvg-00HJT1-1D; Fri, 11 Nov 2022 17:35:00 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otXas-00H88u-Tb for linux-arm-kernel@lists.infradead.org; Fri, 11 Nov 2022 17:13:34 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 67A686205A; Fri, 11 Nov 2022 17:13:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F12EBC43145; Fri, 11 Nov 2022 17:13:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668186809; bh=tgmlFnpiF9RkwBN3vxx8H4aJOPNe1ta4QCFRQlJGuzE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QNkxN5DDp0wuCtf7Lk6fafJHNbhYMtqwTWkD+bx0RXVERMYdaGpYlDqbg+fzk3A1O 8YxEryhAtwA8ueAvgW19iCEJeXQFHOt8dF/Von9uL0iCUkMvho5y+VWOxs2OtHccyl ZQovWBamim1iGnX4KE4TbFD3yxtvZKjgtF16x1mMVlxP/tbbUNiL07dhfYEcndeN2+ WWs9i1rRipv8xH3+zCMkXRkmA4jPHJf+RM+NlLtDmYwBrR7FIhb1vUSKoYhktPzYSO QZoajESSQCEhSMwAe5KRFZObiDiHtCiapFV1FHKP/Jf6QbaqMJ5avj5efaYs8KG7Ww XWieGBtbBlzsw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown , Anshuman Khandual Subject: [PATCH v7 31/33] arm64: mmu: Retire SWAPPER_BLOCK_xxx and related constants Date: Fri, 11 Nov 2022 18:11:59 +0100 Message-Id: <20221111171201.2088501-32-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221111171201.2088501-1-ardb@kernel.org> References: <20221111171201.2088501-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=8726; i=ardb@kernel.org; h=from:subject; bh=tgmlFnpiF9RkwBN3vxx8H4aJOPNe1ta4QCFRQlJGuzE=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjboJchiyWNJHejgMy5FKdUplSDIf3CTF4B1QP8xyX CVYidBWJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY26CXAAKCRDDTyI5ktmPJCmFC/ 93qMvN/RgcKls3bgdPJ6CG4yR7czP+p7Tg/sYeM2YgKw0w7SVinOomsWapOe55gwQl1ZOoSQKrYdmX zLMk1AYKZP0MLX6M9/hWUeZ65wgZCMzYzZkLnNnKHeoDkizpeQSGzbOCiOb3bPyf+V8L5ztp7IKNY0 JXZ/b/UnbTnWZF1x2gYeEic//z8PUSbPFRJ0UMT6coLK7cUeP1R2lNK0KyE1F/s7bK3Uv4/j5ExFYw euot/fqlFSSDGTPZ8j8IVxrbbQToKOmoYggpdBzA0T25rRn5OykfR7ZPKs57qtRuaTG5JFH1XuEm1O fLoZjjooK0MdBUox9r2pRYhkDYDwo0xmrQoMs4WFArPfNCbs6w9Ysk7q9OFdXBiAcAQo32bfXesexD 9kKcnhZ2/JdaU5WI5Vc31v52lDQHtXmNe0O0ldANwBnydc4Yfbn2/1xYIDqiT2KXdZOm6wKfuZHsJF P4d/wxDAYI1V7k4BHCfyJIUOHJcK0B6fow3nZ7np4Gwoo= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221111_091331_088464_96811981 X-CRM114-Status: GOOD ( 21.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Initially, the kernel mapping as well as the ID map used block mappings on 4k pagesize configurations, but this hasn't been the case for a long time. Currently, only the initial ID map uses the larger granularity, to simplify the early mapping code, which is implemented in assembler. The permanent ID map as well as the kernel mapping (which is now created only once) always map the kernel down to pages. This means the SWAPPER_BLOCK_xxx and related constants are no longer named appropriately, so let's rename them to INIT_IDMAP_BLOCK_xxx instead. Get rid of a stale comment while at it. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/kernel-pgtable.h | 60 +++++++------------- arch/arm64/kernel/head.S | 36 ++++++------ arch/arm64/mm/proc.S | 2 +- 3 files changed, 40 insertions(+), 58 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index ed0db7fc0022d34e..4278cd088347fefd 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -19,28 +19,13 @@ * 64K (section size = 512M). */ #ifdef CONFIG_ARM64_4K_PAGES -#define ARM64_KERNEL_USES_PMD_MAPS 1 +#define INIT_IDMAP_USES_PMD_MAPS 1 +#define INIT_IDMAP_TABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1) #else -#define ARM64_KERNEL_USES_PMD_MAPS 0 +#define INIT_IDMAP_USES_PMD_MAPS 0 +#define INIT_IDMAP_TABLE_LEVELS (CONFIG_PGTABLE_LEVELS) #endif -/* - * The idmap and swapper page tables need some space reserved in the kernel - * image. Both require pgd, pud (4 levels only) and pmd tables to (section) - * map the kernel. With the 64K page configuration, swapper and idmap need to - * map to pte level. The swapper also maps the FDT (see __create_page_tables - * for more information). Note that the number of ID map translation levels - * could be increased on the fly if system RAM is out of reach for the default - * VA range, so pages required to map highest possible PA are reserved in all - * cases. - */ -#if ARM64_KERNEL_USES_PMD_MAPS -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1) -#else -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS) -#endif - - /* * If KASLR is enabled, then an offset K is added to the kernel address * space. The bottom 21 bits of this offset are zero to guarantee 2MB @@ -69,14 +54,14 @@ #define EARLY_PGDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, PGDIR_SHIFT, add)) -#if SWAPPER_PGTABLE_LEVELS > 3 +#if INIT_IDMAP_TABLE_LEVELS > 3 #define EARLY_PUDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, PUD_SHIFT, add)) #else #define EARLY_PUDS(vstart, vend, add) (0) #endif -#if SWAPPER_PGTABLE_LEVELS > 2 -#define EARLY_PMDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, SWAPPER_TABLE_SHIFT, add)) +#if INIT_IDMAP_TABLE_LEVELS > 2 +#define EARLY_PMDS(vstart, vend, add) (EARLY_ENTRIES(vstart, vend, INIT_IDMAP_TABLE_SHIFT, add)) #else #define EARLY_PMDS(vstart, vend, add) (0) #endif @@ -93,23 +78,23 @@ #else #define INIT_IDMAP_DIR_SIZE (INIT_IDMAP_DIR_PAGES * PAGE_SIZE) #endif -#define INIT_IDMAP_DIR_PAGES EARLY_PAGES(KIMAGE_VADDR, _end + MAX_FDT_SIZE + SWAPPER_BLOCK_SIZE, 1) +#define INIT_IDMAP_DIR_PAGES EARLY_PAGES(KIMAGE_VADDR, _end + MAX_FDT_SIZE + INIT_IDMAP_BLOCK_SIZE, 1) /* Initial memory map size */ -#if ARM64_KERNEL_USES_PMD_MAPS -#define SWAPPER_BLOCK_SHIFT PMD_SHIFT -#define SWAPPER_BLOCK_SIZE PMD_SIZE -#define SWAPPER_TABLE_SHIFT PUD_SHIFT +#if INIT_IDMAP_USES_PMD_MAPS +#define INIT_IDMAP_BLOCK_SHIFT PMD_SHIFT +#define INIT_IDMAP_BLOCK_SIZE PMD_SIZE +#define INIT_IDMAP_TABLE_SHIFT PUD_SHIFT #else -#define SWAPPER_BLOCK_SHIFT PAGE_SHIFT -#define SWAPPER_BLOCK_SIZE PAGE_SIZE -#define SWAPPER_TABLE_SHIFT PMD_SHIFT +#define INIT_IDMAP_BLOCK_SHIFT PAGE_SHIFT +#define INIT_IDMAP_BLOCK_SIZE PAGE_SIZE +#define INIT_IDMAP_TABLE_SHIFT PMD_SHIFT #endif /* The number of segments in the kernel image (text, rodata, inittext, initdata, data+bss) */ #define KERNEL_SEGMENT_COUNT 5 -#if SWAPPER_BLOCK_SIZE > SEGMENT_ALIGN +#if INIT_IDMAP_BLOCK_SIZE > SEGMENT_ALIGN #define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 1) #else #define EARLY_SEGMENT_EXTRA_PAGES 0 @@ -118,15 +103,12 @@ /* * Initial memory map attributes. */ -#define SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) -#define SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) - -#if ARM64_KERNEL_USES_PMD_MAPS -#define SWAPPER_RW_MMUFLAGS (PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS) -#define SWAPPER_RX_MMUFLAGS (SWAPPER_RW_MMUFLAGS | PMD_SECT_RDONLY) +#if INIT_IDMAP_USES_PMD_MAPS +#define INIT_IDMAP_RW_MMUFLAGS (PMD_ATTRINDX(MT_NORMAL) | PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) +#define INIT_IDMAP_RX_MMUFLAGS (INIT_IDMAP_RW_MMUFLAGS | PMD_SECT_RDONLY) #else -#define SWAPPER_RW_MMUFLAGS (PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS) -#define SWAPPER_RX_MMUFLAGS (SWAPPER_RW_MMUFLAGS | PTE_RDONLY) +#define INIT_IDMAP_RW_MMUFLAGS (PTE_ATTRINDX(MT_NORMAL) | PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) +#define INIT_IDMAP_RX_MMUFLAGS (INIT_IDMAP_RW_MMUFLAGS | PTE_RDONLY) #endif /* diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 6e730a0be1e8196d..3bc96ef82f0f74e4 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -214,23 +214,23 @@ SYM_CODE_END(preserve_boot_args) populate_entries \tbl, \rtbl, \istart, \iend, #PMD_TYPE_TABLE, #PAGE_SIZE, \tmp mov \tbl, \sv -#if SWAPPER_PGTABLE_LEVELS > 3 +#if INIT_IDMAP_TABLE_LEVELS > 3 compute_indices \vstart, \vend, #PUD_SHIFT, #(PAGE_SHIFT - 3), \istart, \iend, \count mov \sv, \rtbl populate_entries \tbl, \rtbl, \istart, \iend, #PMD_TYPE_TABLE, #PAGE_SIZE, \tmp mov \tbl, \sv #endif -#if SWAPPER_PGTABLE_LEVELS > 2 - compute_indices \vstart, \vend, #SWAPPER_TABLE_SHIFT, #(PAGE_SHIFT - 3), \istart, \iend, \count +#if INIT_IDMAP_TABLE_LEVELS > 2 + compute_indices \vstart, \vend, #INIT_IDMAP_TABLE_SHIFT, #(PAGE_SHIFT - 3), \istart, \iend, \count mov \sv, \rtbl populate_entries \tbl, \rtbl, \istart, \iend, #PMD_TYPE_TABLE, #PAGE_SIZE, \tmp mov \tbl, \sv #endif - compute_indices \vstart, \vend, #SWAPPER_BLOCK_SHIFT, #(PAGE_SHIFT - 3), \istart, \iend, \count - bic \rtbl, \phys, #SWAPPER_BLOCK_SIZE - 1 - populate_entries \tbl, \rtbl, \istart, \iend, \flags, #SWAPPER_BLOCK_SIZE, \tmp + compute_indices \vstart, \vend, #INIT_IDMAP_BLOCK_SHIFT, #(PAGE_SHIFT - 3), \istart, \iend, \count + bic \rtbl, \phys, #INIT_IDMAP_BLOCK_SIZE - 1 + populate_entries \tbl, \rtbl, \istart, \iend, \flags, #INIT_IDMAP_BLOCK_SIZE, \tmp .endm /* @@ -317,8 +317,8 @@ SYM_FUNC_START_LOCAL(create_idmap) #endif adrp x0, init_idmap_pg_dir adrp x3, _text - adrp x6, _end + MAX_FDT_SIZE + SWAPPER_BLOCK_SIZE - mov x7, SWAPPER_RX_MMUFLAGS + adrp x6, _end + MAX_FDT_SIZE + INIT_IDMAP_BLOCK_SIZE + mov x7, INIT_IDMAP_RX_MMUFLAGS map_memory x0, x1, x3, x6, x7, x3, IDMAP_PGD_ORDER, x10, x11, x12, x13, x14, EXTRA_SHIFT @@ -326,20 +326,20 @@ SYM_FUNC_START_LOCAL(create_idmap) adrp x1, _text adrp x2, __bss_start adrp x3, _end - bic x4, x2, #SWAPPER_BLOCK_SIZE - 1 - mov x5, SWAPPER_RW_MMUFLAGS - mov x6, #SWAPPER_BLOCK_SHIFT + bic x4, x2, #INIT_IDMAP_BLOCK_SIZE - 1 + mov x5, INIT_IDMAP_RW_MMUFLAGS + mov x6, #INIT_IDMAP_BLOCK_SHIFT bl remap_region /* Remap the FDT after the kernel image */ adrp x1, _text - adrp x22, _end + SWAPPER_BLOCK_SIZE - bic x2, x22, #SWAPPER_BLOCK_SIZE - 1 - bfi x22, x21, #0, #SWAPPER_BLOCK_SHIFT // remapped FDT address - add x3, x2, #MAX_FDT_SIZE + SWAPPER_BLOCK_SIZE - bic x4, x21, #SWAPPER_BLOCK_SIZE - 1 - mov x5, SWAPPER_RW_MMUFLAGS - mov x6, #SWAPPER_BLOCK_SHIFT + adrp x22, _end + INIT_IDMAP_BLOCK_SIZE + bic x2, x22, #INIT_IDMAP_BLOCK_SIZE - 1 + bfi x22, x21, #0, #INIT_IDMAP_BLOCK_SHIFT // remapped FDT address + add x3, x2, #MAX_FDT_SIZE + INIT_IDMAP_BLOCK_SIZE + bic x4, x21, #INIT_IDMAP_BLOCK_SIZE - 1 + mov x5, INIT_IDMAP_RW_MMUFLAGS + mov x6, #INIT_IDMAP_BLOCK_SHIFT bl remap_region /* diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index f0db2c05e797aeed..b596a39394ba5363 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -206,7 +206,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -#define KPTI_NG_PTE_FLAGS (PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS) +#define KPTI_NG_PTE_FLAGS (PTE_ATTRINDX(MT_NORMAL) | PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) .pushsection ".idmap.text", "awx"