From patchwork Thu Jun 30 14:42:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9430AC43334 for ; Thu, 30 Jun 2022 14:43:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hGvele9SRxDB9iF+yqQ/2+8kZc19+4wP2b2zjg1kKVY=; b=3v6to5+uGfegmQ teThC7WzJK0oob1lcOqz2wh4bw2DJ3Q3Af3iSP77i3Apf9lEfM1xaUZZ+oh/scHh8/iFQWRCE5WNv k5E6V7ID14+SMgGTeKXuv1uHtsXvwod/FuUGKWEy18zmc+tXZV7W42QpCl7cDBiWg8KwLxnt2ii8E L4OaJgR2ii1MojcA4pQBRh8nmZFDnsd7IrEyG8Fdi7256R/IbfTFv1JDgptkbCbTgfoeLCxbRjzl/ vDHyxMeXILrBICJKjzV8pJMop7lBvy/7ym6cJfESYqi1F1vrhPCh9ZQkd4Dxxb7D9Dk3F+LNG/OCA BPsSgFDLC42F6KN6gDSg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNa-0007he-Bo; Thu, 30 Jun 2022 14:42:50 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNO-0007dQ-J5 for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:40 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id ED9CD61FC0; Thu, 30 Jun 2022 14:42:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D037EC341CD; Thu, 30 Jun 2022 14:42:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600157; bh=yKzAtmQqDHKeriNUxaPWTpMM8mu/8wkHAf5dF48Z5iM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RWHoVILu8uM7iDd3R3qopqNNal/EQBqzj6km3aYRLWBveEdYe72TQJSMco8WAqYqJ kbhcGb2/p6vRPFN0WdkWLfmGWErytT+Bw6ih4DfYV/uWhG5DI1phU/1W5fF3Hy9Ayp 4+igaNtCW0gYJmgYFHF0FF0ziDZS4Vlm15MPQsyyLE+QRBrrvdcGZbEi1RTUFrpTJZ /Wz7ZTIvyFW/y1qbrH49ndS5aMCyQ9PZxQYNzeg/CxwhdzHbGwIclwwdVns4xgHeFQ WdW0vaLAkOTtAZsRJqqbrGjzF7erbO5ULWLJ+2DpylumlR94/8xa3uGSmL1vbBMYD2 +CEkxseGd8BEA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 1/6] arm64: lds: reduce effective minimum image alignment to 64k Date: Thu, 30 Jun 2022 16:42:25 +0200 Message-Id: <20220630144230.2332555-2-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5090; h=from:subject; bh=yKzAtmQqDHKeriNUxaPWTpMM8mu/8wkHAf5dF48Z5iM=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZLvJEbt9YqMqyjqFpG7xGYk/dxiM/H0t0K1UEi hKMpzm+JAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22SwAKCRDDTyI5ktmPJHgxC/ 9YZbu5GVqsv3VSgoxBpBSbepW36bX6nU4EdlpYFgtLBMD+DvNStzYYsoGbKqKDTYTYIevbLAFUxPYK ngnK9k7BOEectcZWhgSAyGS3XyFwLS6TNAzExEGxn9PU+MfLwo4J/Xl6o+uxT5ty9/aAd+sjeLjYiF mTsVOBu7moD8NA32Nstf235r+gI4eC5FL9L0Xupjo/zGbELfnXwsLlt7SjZUx/lJA4BkQp2665hFiT zY0NTcv7bZcKzb5ox4qjCML/sZBpwd9nFRebvznkGPAvbvUo/dBOWLuwXUcoFraYcDgTeqQf86XL7N n374MMB1tQ8/ESvPUXHRp6/+UCXCeX6B0nmryQ+w8uo4+uObWdPD+KJkFh3EBdoyfW3LCHFC95KSp8 NVHkc5ul6CrEQe5hry0XrtB2FenH4/7smZPeiKli54iKTzeaOSfGp9Gc9REZygguKymAl2S45YqoDs VrE2uhOhWb17YHQyktUezo/Sxtj0oWolgGP3yDBRhrNEc= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074238_752598_C604ED9C X-CRM114-Status: GOOD ( 23.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Our segment alignment is 64k for all configurations, and coincidentally, this is the largest alignment supported by the PE/COFF executable format used by EFI. This means that generally, there is no need to move the image around in memory after it has been loaded by the firmware, which can be advantageous as it also permits us to rely on the memory attributes set by the firmware (R-X for [_text, __inittext_end] and RW- for [__initdata_begin, _end]. This means we can jump right from the EFI stub into the image with the MMU and caches enabled. However, the minimum alignment of the image is actually 128k on 64k pages configurations with CONFIG_VMAP_STACK=y, due to the existence of a single 128k aligned object in the image, which is the stack of the init task. Let's work around this by adding some padding before the init stack allocation, so we can round down the stack pointer to a suitably aligned value if the image is not aligned to 128k in memory. Note that this does not affect the boot protocol, which still requires 2 MiB alignment for bare metal boot. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/efi.h | 7 ------- arch/arm64/kernel/head.S | 3 +++ arch/arm64/kernel/vmlinux.lds.S | 11 ++++++++++- drivers/firmware/efi/libstub/arm64-stub.c | 2 +- include/linux/efi.h | 6 +----- 5 files changed, 15 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index ad55079abe47..3be3efee8fac 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -57,13 +57,6 @@ efi_status_t __efi_rt_asm_wrapper(void *, const char *, ...); /* arch specific definitions used by the stub code */ -/* - * In some configurations (e.g. VMAP_STACK && 64K pages), stacks built into the - * kernel need greater alignment than we require the segments to be padded to. - */ -#define EFI_KIMG_ALIGN \ - (SEGMENT_ALIGN > THREAD_ALIGN ? SEGMENT_ALIGN : THREAD_ALIGN) - /* * On arm64, we have to ensure that the initrd ends up in the linear region, * which is a 1 GB aligned region of size '1UL << (VA_BITS_MIN - 1)' that is diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 5089660788fd..09b0cddf2161 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -394,6 +394,9 @@ SYM_FUNC_END(create_kernel_mapping) msr sp_el0, \tsk ldr \tmp1, [\tsk, #TSK_STACK] +#if THREAD_ALIGN > SEGMENT_ALIGN + bic \tmp1, \tmp1, #THREAD_ALIGN - 1 +#endif add sp, \tmp1, #THREAD_SIZE sub sp, sp, #PT_REGS_SIZE diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 45131e354e27..0efccdf52be2 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -274,7 +274,16 @@ SECTIONS _data = .; _sdata = .; - RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) +#if THREAD_ALIGN > SEGMENT_ALIGN + /* + * Add some padding for the init stack so we can fix up any potential + * misalignment at runtime. In practice, this can only occur on 64k + * pages configurations with CONFIG_VMAP_STACK=y. + */ + . += THREAD_ALIGN - SEGMENT_ALIGN; + ASSERT(. == init_stack, "init_stack not at start of RW_DATA as expected") +#endif + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, SEGMENT_ALIGN) /* * Data written with the MMU off but read with the MMU on requires diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c index 577173ee1f83..ad7392e6c200 100644 --- a/drivers/firmware/efi/libstub/arm64-stub.c +++ b/drivers/firmware/efi/libstub/arm64-stub.c @@ -98,7 +98,7 @@ efi_status_t handle_kernel_image(unsigned long *image_addr, * 2M alignment if KASLR was explicitly disabled, even if it was not * going to be activated to begin with. */ - u64 min_kimg_align = efi_nokaslr ? MIN_KIMG_ALIGN : EFI_KIMG_ALIGN; + u64 min_kimg_align = efi_nokaslr ? MIN_KIMG_ALIGN : SEGMENT_ALIGN; if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { efi_guid_t li_fixed_proto = LINUX_EFI_LOADED_IMAGE_FIXED_GUID; diff --git a/include/linux/efi.h b/include/linux/efi.h index 7d9b0bb47eb3..492497054a5a 100644 --- a/include/linux/efi.h +++ b/include/linux/efi.h @@ -416,11 +416,7 @@ void efi_native_runtime_setup(void); /* * This GUID may be installed onto the kernel image's handle as a NULL protocol * to signal to the stub that the placement of the image should be respected, - * and moving the image in physical memory is undesirable. To ensure - * compatibility with 64k pages kernels with virtually mapped stacks, and to - * avoid defeating physical randomization, this protocol should only be - * installed if the image was placed at a randomized 128k aligned address in - * memory. + * and moving the image in physical memory is undesirable. */ #define LINUX_EFI_LOADED_IMAGE_FIXED_GUID EFI_GUID(0xf5a37b6d, 0x3344, 0x42a5, 0xb6, 0xbb, 0x97, 0x86, 0x48, 0xc1, 0x89, 0x0a) From patchwork Thu Jun 30 14:42:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40071C433EF for ; Thu, 30 Jun 2022 14:44:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=E25jvtRJFUsMyQlBSPTQz2TxB3LnwjtF2VHRhjjDYi4=; b=UKPQBLVvDHLHP/ jifpqq6DP9EhwJVfrvdfvYMBZKfRrmTFEGgnodcbZTpyfBZGlOBSucNL4QyfNn1l2byos1HbIqHdq /VdF7SVv6bbtbjm/5nW1jkZ9NRxrkhd+VZ1fckuWTX48pr7VwfuZisUWIYCMJImo3vNRMOhGkrVUF +oKhbIrZJmAd0LnoFlJm0pw+Z7SWoG1dmDndYaHgs12sTY+B/5tcfek5HVdi6FeWfeBO/42E0wbRe WSy9MeEIJZ1Nv0iV+GrzbxVXqerCcI2/++yzGKnoqHJDDBKsJ4OC3UH0bwpdhp0DqF6seg3Fpg8e7 CG9MvtmeNkbkttPrNDzw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNo-0007lf-2k; Thu, 30 Jun 2022 14:43:04 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNQ-0007eB-Ge for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:41 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DE4C76231F; Thu, 30 Jun 2022 14:42:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C3A13C3411E; Thu, 30 Jun 2022 14:42:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600159; bh=N1GtE6NvYykzcTIJ4gqDcO1sHS3/meS1CkMcN6NWlh8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oVg7gPxT5a9EL+87JspAk29cuWgAYN+qU+MQEOwv+XkTg/Z/3z4PhnxhW9I7aUiF4 r309M5kxli3/6YMCzsndanPNIt9oZM4CJwYAXah89F4XoOXRsriRaNho+zulbREBgM GhmuAmO/RsxiNO0y5yX6ICeYe4k4XTe6ThVjwUDRxiDMkIbst6zZGr1QPTWxd0wBmD Br9avkDFtBMkad+kfoOGtrIqUTlMgZNFHjrNWr477I3QqDqeRvEIcr0vjFVTvq9NN5 r2RInU7Xs4y2EnnH3Gn/cik4kC1fs9LPgh3i01KEaDWFgbncaHP9C2mLs7AD3nAPH5 GjTO7328dFhvQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 2/6] arm64: kernel: move ID map out of .text mapping Date: Thu, 30 Jun 2022 16:42:26 +0200 Message-Id: <20220630144230.2332555-3-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2791; h=from:subject; bh=N1GtE6NvYykzcTIJ4gqDcO1sHS3/meS1CkMcN6NWlh8=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZM6hXt5zZOdSwlKQMQBAgQ49jBTPzOSvcIWV0Q w2CZON+JAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22TAAKCRDDTyI5ktmPJDVBC/ 9tWnGoKPxljrvY1zaSUjWb7wA1GqNtS1bG3w+3pk3XlMgLEzHuOLBzGPFDtT4dDxffaqKQ7GFIGZ8u pF1IrTTI3oNAwZDrWlW1wgLDiUnYAe9qTtjztXpioqylj3DSzqCMQoJKKPRJZee380XwVl6UDcsCJg egbsglF21+cNRbgookKyypsRKcMALRDx4zUsKuexkqkoVYcez8ZGzZZUenCdnwqID2guLFaVRf98Cs LuBW/L2Hvo7OtyN5xQzLNpuTnjFw/509vHa2wsagEUvtvued0gMNWsek+sGlWWOY27YRd5sYkQ5z2v gL9uwbVnZ83hLfgJycUsmtQ0Rmteo5zsoXs/P260d61BmdFc6grLFEMJZ8a0JDX6D0nLZt1y2RBKMX Fy2RIb1ozoP6cPy9j9ISlvcxu5Rnrr6AAZtG/7c8WOcIWHIKWoIKq+Bf5sX2lVnlA9vFXpyFrqSsYx afQrxKDJAVJFDqCGy8C/A329vOZSmOaAJauMEy+WDU6A4= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074240_660914_B6C4506A X-CRM114-Status: GOOD ( 14.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Reorganize the ID map slightly so that only code that is executed via the 1:1 mapping remains. This allows us to move the ID map out of the .text segment, as it will no longer need executable permissions via the kernel mapping. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 5 ++++- arch/arm64/kernel/vmlinux.lds.S | 2 +- arch/arm64/mm/proc.S | 2 -- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 09b0cddf2161..2210bbd13cf9 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -464,7 +464,7 @@ SYM_FUNC_END(__primary_switched) * end early head section, begin head code that is also used for * hotplug and needs to have the same protections as the text region */ - .section ".idmap.text","awx" + .text /* * Starting from EL2 or EL1, configure the CPU to execute at the highest @@ -556,6 +556,7 @@ SYM_FUNC_START_LOCAL(set_cpu_boot_mode_flag) ret SYM_FUNC_END(set_cpu_boot_mode_flag) + .section ".idmap.text","awx" /* * This provides a "holding pen" for platforms to hold all secondary * cores are held until we're ready for them to initialise. @@ -600,6 +601,7 @@ SYM_FUNC_START_LOCAL(secondary_startup) br x8 SYM_FUNC_END(secondary_startup) + .text SYM_FUNC_START_LOCAL(__secondary_switched) mov x0, x20 bl set_cpu_boot_mode_flag @@ -659,6 +661,7 @@ SYM_FUNC_END(__secondary_too_slow) * Checks if the selected granule size is supported by the CPU. * If it isn't, park the CPU */ + .section ".idmap.text","awx" SYM_FUNC_START(__enable_mmu) mrs x3, ID_AA64MMFR0_EL1 ubfx x3, x3, #ID_AA64MMFR0_TGRAN_SHIFT, 4 diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 0efccdf52be2..5002d869fa7f 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -168,7 +168,6 @@ SECTIONS LOCK_TEXT KPROBES_TEXT HYPERVISOR_TEXT - IDMAP_TEXT *(.gnu.warning) . = ALIGN(16); *(.got) /* Global offset table */ @@ -195,6 +194,7 @@ SECTIONS TRAMP_TEXT HIBERNATE_TEXT KEXEC_TEXT + IDMAP_TEXT . = ALIGN(PAGE_SIZE); } diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 7837a69524c5..113a4fedf5b8 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -107,7 +107,6 @@ SYM_FUNC_END(cpu_do_suspend) * * x0: Address of context pointer */ - .pushsection ".idmap.text", "awx" SYM_FUNC_START(cpu_do_resume) ldp x2, x3, [x0] ldp x4, x5, [x0, #16] @@ -163,7 +162,6 @@ alternative_else_nop_endif isb ret SYM_FUNC_END(cpu_do_resume) - .popsection #endif .pushsection ".idmap.text", "awx" From patchwork Thu Jun 30 14:42:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E86E2C433EF for ; Thu, 30 Jun 2022 14:44:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mjpNad34rSAsHqJOdfg6lCpLSB0gdT6couNFaIBnUIU=; b=W1rrYRXjsREjsd RQqVPNfKbcxrX2TSTtNi290yCwdEWxfec9j02E9pPTuudgHZmjOvgdAJq0mGTO3plX8qIvWmcxQkT tKG1KYk3OqbgAIhKq00F5ORCElizBVpk9mVO8ydH5Ofb8U85jsbJEGHc3hylJ9AS3knJdFBya9pxS 4WHIEDtYJfu9TGjthOOn7u4t0RwwONcNUJxzxjR9MHdMnUr5L/nnPhp/Nsv1zlekg1LPHcY3TWzTI tYcKV3+RN6fqrNJIWbVKRx8cf8AzMKB9pArDwdouSoKi49uy1IjoImkaa8DJF+ONnqbvJNEYCTDmY S9f3tRByHsrWwek86YWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNz-0007oh-4N; Thu, 30 Jun 2022 14:43:15 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNS-0007fM-8W for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:43 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D2A626232A; Thu, 30 Jun 2022 14:42:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6A10C341CC; Thu, 30 Jun 2022 14:42:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600161; bh=NncCjvt4+51bdakgsRg19upHSCRxlcIVp95M4/z6w4w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iEf9R1jzcATeDkN4g6pB1Io4gSrFnJbBMNN6gt1PTSxqSmfz1+C7dcN/QCmWzZMZn g79TD19LZNWXcLjSds1sv57nZTJTi8j3grBKxjtMVNZWQtQjv947JsH8RvqLIu7bDp qNnonVXGjBx5s3NWai5ikDp6G8ZrHpzWXKy9PqpZlAb6Yr42VKwzDRm1An0tKc557Q BYPS8r579YliHzRgpuVt/DkiZzdcrzzVKjw2AdAnNUIiOB12NwCN3nvV+O46LisRYb Wun4Tvp7ztl9rZzimB2JAF6uXjusul1Vq7jWsFAjJnOODWRZLiAVWn5ecMhbRJcNzr b2dAABPXMu4IQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 3/6] arm64: head: record the MMU state at primary entry Date: Thu, 30 Jun 2022 16:42:27 +0200 Message-Id: <20220630144230.2332555-4-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2408; h=from:subject; bh=NncCjvt4+51bdakgsRg19upHSCRxlcIVp95M4/z6w4w=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZOROjfUk2KvV9kdvYVpuikW8TiFeJ3VqL36aF6 Gw9BCteJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22TgAKCRDDTyI5ktmPJGX4C/ 4kZcgz4l8Z+rJ+33sdxreXrLv+4XIAYuHHU9bVORx7o3Bc3FZM1i5RtslRDkn/J5ucu2y92+DwMdT1 rh3iwrNmGnePqQQr/OoTI59XCXaP2dLOatiNKYO0VtpblGz7iwmfY7o3F3oh88pwhq14xNwH4cRGW2 kyukeMTnIcMkUPo6/ci9Xv0NG6dG2Jnc6bKlHLbeQgzyGQC5/qjHZ7yNddP7hdUdHg0MIk7KJHrKCm HHTfWb2qYDCtgLPWQeeeoU0a1Go8oiVomjOej7Qpx4Xcz0h0khIRAvqPWa9ZsBx3+YOi6XtvzB5Bf5 aUseB9M9w3D4tOEONfsIN4054MehKhDrSx2JnRXwIHCS1TXTurn6uI7pIFCk6J4PoFVCVQjxjhsaDh 9szwvjOASmb3jxAhmsTxgn5iDAJbE8iyQqlw4n0B4Xrnmmvu0n76Vqf/p9utnZJLgU3RPvTqxKDxjM PobhqxnL6BV3bfe5QJ8rXNVDz6BRgp1N9Sok9MnNpkQEk= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074242_422104_B392E841 X-CRM114-Status: GOOD ( 10.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare for being able to deal with primary entry with the MMU and caches enabled, by recording whether or not we entered with the MMU on in register x19. While at it, add disable_mmu_workaround macro invocations to init_kernel_el, as its manipulation of SCTLR_ELx may amount to disabling of the MMU after subsequent patches. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 2210bbd13cf9..a79c842395ee 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -77,6 +77,7 @@ * primary lowlevel boot path: * * Register Scope Purpose + * x19 primary_entry() .. start_kernel() whether we entered with the MMU on * x20 primary_entry() .. __primary_switch() CPU boot mode * x21 primary_entry() .. start_kernel() FDT pointer passed at boot in x0 * x22 create_idmap() .. start_kernel() ID map VA of the DT blob @@ -85,6 +86,7 @@ * x28 create_idmap() callee preserved temp register */ SYM_CODE_START(primary_entry) + bl record_mmu_state bl preserve_boot_args bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 @@ -107,6 +109,17 @@ SYM_CODE_START(primary_entry) b __primary_switch SYM_CODE_END(primary_entry) +SYM_CODE_START_LOCAL(record_mmu_state) + mrs x19, CurrentEL + cmp x19, #CurrentEL_EL2 + mrs x19, sctlr_el1 + b.ne 0f + mrs x19, sctlr_el2 +0: tst x19, #SCTLR_ELx_M + cset w19, ne + ret +SYM_CODE_END(record_mmu_state) + /* * Preserve the arguments passed by the bootloader in x0 .. x3 */ @@ -484,6 +497,7 @@ SYM_FUNC_START(init_kernel_el) SYM_INNER_LABEL(init_el1, SYM_L_LOCAL) mov_q x0, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround msr sctlr_el1, x0 isb mov_q x0, INIT_PSTATE_EL1 @@ -515,6 +529,7 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) /* Switching to VHE requires a sane SCTLR_EL1 as a start */ mov_q x0, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x0 /* @@ -530,6 +545,7 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) 1: mov_q x0, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround msr sctlr_el1, x0 msr elr_el2, lr From patchwork Thu Jun 30 14:42:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20CF2C43334 for ; Thu, 30 Jun 2022 14:44:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LT9+xsxwNvxx+5uMp9T7WX+yF3f8BvgFl6/dA+veXDE=; b=PoD5shKRvk/D7I BCStXg1W0dVBw5QVIMFee5c9WFyrhFGZUe3YgVEi+NT0n1yb6f4K7QaBPrEcJEfoTx8PdzwPPaHTG /ID1ittYI0uvxhzggOKGpXE59aaYDO7xfZN6WYdU2Q2/W2qLCu+GOzAauToFzh3uYp5VzVBIbGmHH duOoFFZpr+OEYPYILUQr8oF+xoaRPZuRfnOUl+rcwG3AO1CkmgL7qXZtESs5k/rlj6X4B34JjscwL UjB6Gu4a7Br7DFMMRgAjMAZUBmuF82IE/4xN42h22KMqBsIF/u3qHzGqTF9nY0Hk5fg5DeoXJBn4H un6z2FxPQ4VzcjZV5LHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vOB-0007tI-NC; Thu, 30 Jun 2022 14:43:27 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNU-0007g1-63 for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:45 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C68A06231F; Thu, 30 Jun 2022 14:42:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AAA31C341CB; Thu, 30 Jun 2022 14:42:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600163; bh=TwP65EAjFILSHRHy/ydIST6zDHt+Ko5p7xzLC5G/PHw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kNL9UgnMF8Sk9alm+LLDllU3dqGAI3YSfkVSBIz6Ju1TK/ApfcsgGR2/sziFfo/SD /LsNonZJTCIYz2bC5+FR6mk4btN94mrzbzpiNt8JX88OiE66aJNiPwsnm9k73IaqtS 4lhE2nsTqFRZK0t/rZVMvSvdOQ0PavZmeaj6RxvZSvut5TdR8Zqy0iouQ6c8Cp+Duc zUyFMGh6rwRIRfKjtzXhOzSQ7WtFixiYCiF81+8G7dvPYriwkYNgsvjiYQUnIm6NA2 CNkJjAdqAmctHQlx2uz7OJONryvNVV9DztV6IgTbx53eCQY2v7ZAiHYfRliuIq9FQV EP8q57BNlte7A== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 4/6] arm64: head: avoid cache invalidation when entering with the MMU on Date: Thu, 30 Jun 2022 16:42:28 +0200 Message-Id: <20220630144230.2332555-5-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1782; h=from:subject; bh=TwP65EAjFILSHRHy/ydIST6zDHt+Ko5p7xzLC5G/PHw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZQNCLRZcOyp7euLrckrC6C93tSZUwKV/WoulmF EtOUeuuJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22UAAKCRDDTyI5ktmPJEBhDA C5ROrsu0Sz/8VTjBUeoEaUILbFs+BtEexMTVIWbIhmqwC6WG4HMvQ0SnGgbooQoyCsCeCpQvkb3R1U HRh4VpKJNBGxq+2OTYdL6oyhDNfnRDzI3WZW9Eozp3eFP27/42KICcfF5yILJVFFYVevZ+QHxxKgh0 UZPUOxJDErT02lUK5LVfKTMvm43f0jOswMadhkYQOSL6qwSkuKZz4XWK1mROlA1H6stGXneorW3f8T e74qIoiumyxs9HGpXdC0uHpJpL4Hy34zH6twBER/sxxU/3CwuQzlTLEU/jyTlIQwvM89fHrtc2iitf 6tP85EvoFQ3YE+Wy+iOGSr1YDu95pZv/C8kvjPHl/TjGMTEcAPAYqEXla+Hcvm0D6sq3FAn0Tb/PjS pOwCD7j8K5Nx77rXQb53NunwYftNhEbnnKZp4O/y0Skn2dS7SPjC7LtjkHC4e5WenzhVS5NQ6rGjNa yjWmEvqi7PyPidHst2MLCOb1A+XHwMPE/ioecGF1o7lEc= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074244_341134_CB65C305 X-CRM114-Status: GOOD ( 12.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we enter with the MMU on, there is no need for explicit cache invalidation for stores to memory, as they will be coherent with the caches. Let's take advantage of this, and create the ID map with the MMU still enabled if that is how we entered, and avoid any cache invalidation calls in that case. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index a79c842395ee..42fc7e980b35 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -88,9 +88,9 @@ SYM_CODE_START(primary_entry) bl record_mmu_state bl preserve_boot_args + bl create_idmap bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 - bl create_idmap /* * The following calls CPU setup code, see arch/arm64/mm/proc.S for @@ -130,11 +130,13 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: ret SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(clear_page_tables) @@ -371,12 +373,13 @@ SYM_FUNC_START_LOCAL(create_idmap) * accesses (MMU disabled), invalidate those tables again to * remove any speculatively loaded cache lines. */ + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy adrp x0, init_idmap_pg_dir adrp x1, init_idmap_pg_end bl dcache_inval_poc - ret x28 +0: ret x28 SYM_FUNC_END(create_idmap) SYM_FUNC_START_LOCAL(create_kernel_mapping) From patchwork Thu Jun 30 14:42:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C5C4C43334 for ; Thu, 30 Jun 2022 14:45:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eO1hu8t3KX4GjBmvIy0qVV3fvhRLQSR6uB9r+RKpr7Q=; b=MuYwxSv/Ltfbvp V1wQJm//X7yYo3WLw7qo8+t3vjlM2Rnjn4QjXmpXoPaVJm5zx4bbdjaxbgV2dKIlBFLKjOarp4RaN yGhpL5y4eV0Rsz22KPM+EBOFexTqf7dlKosUqxm5eNnodVBznoDAZNGXjOImWjribf6HaargUO0ta U+512SK+vQGGV2/mWC8afsQ7EFcmikGiiADI2TB4FYj58CIxt1rTSfZbTolkt9S/PXWEa5y7XpEEE jLRI9ofpBC+3RR7niAfa7hPAe98zBLP5tjT1zWTOlqbIJBZDymLen7haoUsX5OwW+/oVTV4d6v39y gTBKdGw1oudolE7yoqyg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vOi-00085h-Vw; Thu, 30 Jun 2022 14:44:01 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNW-0007gX-4m for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:47 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BC9F66232A; Thu, 30 Jun 2022 14:42:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E039C341CD; Thu, 30 Jun 2022 14:42:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600165; bh=eLBbBu42Eait2R2zKHPQ835md1AWZ1nnjes6vvOLsBw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kCmh/vzM9NMISZkzCeyPGvHCc7Xdtnu/+qvWTRjFlh6lEWvQ2PZpR9lVGIwHB6nzV f3pMAWqE1gSiJ4wW9bTB8leOiCzTUdC6+N+Uv908jd30ByqR6r/iEcgkj4zcZNMC7T BHkQFvL3PZwu5w6i/k2cTaPr9RXaeoj708/PiVPk2ijDgJrLmON31SVeBsbKq1z0BO 4XOdZDfqUDAoa7hDm+nSlddnSbSPkIGKxjF92G5rryB2v2hPHs2yYiEybf8C1Hcuro 7giQ/Ec8X6y7/aAwI89C6R3HnxmOt46CkolBJ13qqUIsu4ERCs97r1Jwv2pYel4cHl ff0QGIObmYFqg== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 5/6] arm64: head: clean the ID map page to the PoC Date: Thu, 30 Jun 2022 16:42:29 +0200 Message-Id: <20220630144230.2332555-6-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1571; h=from:subject; bh=eLBbBu42Eait2R2zKHPQ835md1AWZ1nnjes6vvOLsBw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZR8GLSCJM4bY9wu1I1X6SWBFAudWgpLwK0tor4 myoC44CJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22UQAKCRDDTyI5ktmPJJNqC/ 0Zz0YIqADhEaDGi0kLMok1DsZjuWiYg1l46T/gOdnROnZUMsEaXv/TbjYFoXYGpRd2rnSInn9K74wl vy16UEt5CPZVPC4LhgaNPJesmwCKSz0Xb0UxQiRwQVPR2ZY6lJoBCDr5achwvkaEIOj46U6m0E7ZcB rQkDbmMySE3tOyvyRcLT9VUfhnFOSGxWsdU/QMZnbvAulClqiKojSJjKzgvgH9YxIWITyqoh6Sh2GN pS6dWWU/10SZ/az057wpKed3DNKHGy14FBmGyCM2lYa9SLi5qnOBLe5ttl/IFlZWzsfPnGW1rOHPNI CPCUCVOWa3qxgYZMJvUHxvhHP2dDjG2oUtYGgTPvwp4SR71Ph7f48TsDHu9kQCjOHrWGEWKERxqXbG x1e/SbxCK1mhRKgY/2eRRhwripER6GhmV0mm1Nf3cpLOC0JEb27KqtBX+xkLmM8oJFO+XVCiDbEwl9 jH57/nKZnLRnDK7FdEr1RoLpClMQecxw3R6v8PE5QGmpg= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074246_305071_F8F8EF04 X-CRM114-Status: GOOD ( 15.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we enter with the MMU and caches enabled, the caller may not have performed any cache maintenance. So clean the ID mapped page to the PoC, to ensure that instruction and data accesses with the MMU off see the correct data. Note that this means primary_entry() itself needs to be moved into the ID map as well, as we will return from init_kernel_el() with the MMU and caches off. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 42fc7e980b35..4ca4d66b418f 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -70,7 +70,7 @@ __EFI_PE_HEADER - __INIT + .section ".idmap.text","awx" /* * The following callee saved general purpose registers are used on the @@ -89,6 +89,17 @@ SYM_CODE_START(primary_entry) bl record_mmu_state bl preserve_boot_args bl create_idmap + + /* + * If we entered with the MMU and caches on, clean the ID mapped part + * of the primary boot code to the PoC so we can safely execute it with + * the MMU off. + */ + cbz x19, 0f + adrp x0, __idmap_text_start + adr_l x1, __idmap_text_end + bl dcache_clean_poc +0: bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 @@ -109,6 +120,7 @@ SYM_CODE_START(primary_entry) b __primary_switch SYM_CODE_END(primary_entry) + __INIT SYM_CODE_START_LOCAL(record_mmu_state) mrs x19, CurrentEL cmp x19, #CurrentEL_EL2 From patchwork Thu Jun 30 14:42:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901952 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 175FAC433EF for ; Thu, 30 Jun 2022 14:45:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l8GBFX1gDOh+Oa4BEzlRWX1XRa18Azy4kDLSVp3N+oU=; b=306HWT/K++rmoN +5lGOno6Gy8JuOX5qyTaau+c9xiEdAOd9ofSz3xc7O2RM8k64xlLuzNu8a2d+96ls0Eig9LNEJW9b v05xXcmf5ylc7i8j/BgZRBaHhNpTZ8sGy/pp53CIgkN7P6beJZeH5DySbv/3rA2sOY3MEsU0Jrw6P WS9of6bGaH1AXIgXHsipx2w7hzDIolmW0lcXJ/QcNBwnjOnmIY5dJlAdWj7RukKJymcvNgVy2gqbB fBoAqlH/hq98JmFfdAv0qmdj32g/Zvfn3YBR736oB0UUi2MzbG+W5R31IFctAdyfuu9ZeRZIjHCFP K+izieDCeHnt96jf0UjA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vP3-0008FH-BI; Thu, 30 Jun 2022 14:44:21 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNY-0007h2-55 for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:49 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id ABB856232A; Thu, 30 Jun 2022 14:42:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9135BC341CB; Thu, 30 Jun 2022 14:42:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600167; bh=9lLFI1Q7GpU0jIzBfscXwQ5dctUoxP7KVFxnfsmujWw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rSEukn4CtKHKiQJz53rVI/6GSAeo+eu4h+abG+KIdun852XuJawptIUbYqMs/O4L4 wFYr/QUJUFWuBB5APiot8HBUGY5OpxUiRlupzSy+4/5ql7eTHhDMgsoNaWLXmUGqxV NvgTRPJJh/QYB94OqHyPIDDUSuYp+GZwIgzh3Us3pCuB/Zzy6eM2J2wHu/gmW7EFkY rSeduuSgUuNgyhADUqHtcgKsCHbjEMAddDk6+EYtYZriVLgTmKga+5Ek3Ay9Ozz43Z iFJts7/J7ttH7sbbDWH3P7p+oA/qLnu5rzJv30Iye3ZP//SFg6+goFU33lGtnwYgXs 9CLOjw8r4zLHw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 6/6] arm64: efi/libstub: enter with the MMU on if executing in place Date: Thu, 30 Jun 2022 16:42:30 +0200 Message-Id: <20220630144230.2332555-7-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=986; h=from:subject; bh=9lLFI1Q7GpU0jIzBfscXwQ5dctUoxP7KVFxnfsmujWw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZTY5VvtnR3y49Yx07L2TkwYt2h0Bdce/jw1gF0 afm0+nqJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22UwAKCRDDTyI5ktmPJP04C/ oDZIc29O7WwnIt4t08BxmNLzpa7WBmiwKAjD900F1kD/Eh26IAgwNfWI1x8OFKa4TuFYg3e0Ihp/H1 PgkKj+7m1kN+9cHthAp5VbXDYqpYKVMHffL4P+X6bU3PgGWKdORudZQxTbDUwy5Rvu9Wh983qE8xkX Er16LWO1OkInpqcWwBJutGyAN07DDXQrrFV5lgLcKtGKefWN3SN0v7KZVmrnrWiGUZm2XYNThJPf1H 0Ho+OE6+aq9KBWhS2Ad8wIuxPx46GpMysqeRjAn68dmaarq/i00EusWxs3q8aMzkMkaprTFs/2dEee H7UZ5Q9ur14sTYQvl64xSOz9VtpgrrWSWxQx/V+Y+RcZP7k3zv2iAVSppTGFW5O73ecyf2DfHx9ycD 8/s9w7eZS7J7XmVNRD7qJm3fNx7TCRsHwhkHg6UE8sb7dNwEs1SU028VjENo7AD6hyQqJJsIUSR/pf 5gb6OzHd95RVPchE8ekfk19A8SPnN0UiByPJOZDcER6xk= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074248_276096_260F16E8 X-CRM114-Status: GOOD ( 12.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the kernel image has not been moved from the place where it was loaded by the firmware, just call the kernel entrypoint directly, and keep the MMU and caches enabled. This removes the need for any cache invalidation in the entry path. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/efi-entry.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 61a87fa1c305..0da0b373cf32 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -23,6 +23,10 @@ SYM_CODE_START(efi_enter_kernel) add x19, x0, x2 // relocated Image entrypoint mov x20, x1 // DTB address + adrp x3, _text // just call the entrypoint + cmp x0, x3 // directly if the image was + b.eq 2f // not moved around in memory + /* * Clean the copied Image to the PoC, and ensure it is not shadowed by * stale icache entries from before relocation.