From patchwork Tue Nov 29 16:14:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C85FC433FE for ; Tue, 29 Nov 2022 16:17:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yd1fuPcMg1lsiFjtCuAti6p3qtcm7wAhonkgXouVhak=; b=j8epWs4gnMSlMx 56fp4CQ9FAXB91FqXOo5EkZF8/fl5uJ9WDiMf0H9AcE8BMtNV/5RVB5/dJ1gi5nUFUu3BG/1nbtrd TCMnaj4PLUUPha6UtKmnMG/JDxgMUNuDd1OsYqiV4XtCzQu54B40Da6saB35ewmeUJvPYYe45A+ss 1Zh4u0cvwq43cap8NJs+hE3w6eV7xIm/NjjmEuv35C/53SfL3lMZa0SSUS8O/QqU9o2J7rwW5tYDn kj3FwxSz/vVqJtJCqTKWJS6TGkHmKmlAwZ5lyuxLn0P7lsx/IoU4ACVibCGfC/CLTG+sfuJnfkzC1 5XwnhZmPpSQ4Sr0M/sbQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p03Hf-00A1Ag-CP; Tue, 29 Nov 2022 16:16:35 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p03Fd-00A0Y1-Vh for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 16:14:31 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 56CB7617BA; Tue, 29 Nov 2022 16:14:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CF271C433D7; Tue, 29 Nov 2022 16:14:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669738469; bh=2Olt+kmxs4NdbHYZ6IbTcuFcKJShiSGgmZUFx4t4lTU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CMnxJfnpVc2+2sTR0JxTSypfzWIKxDuBe6eoU0qTjLnAMur2MG8ZlFGdEzgFm/dWz ZkYx6p4KajYXSNtCDxmXGECvca1eqkk5CdzW/jjvmvxCvt5cNkz7RQ8B5DHg1Cz9wX B49L01HmOIL+sZO7XnOb/DLoSFhpjwEYGsLBP7ffGla9RIz9fkdZ1HhqKw6LuXvN8T n8EdopOlETOzpzNUdT8XT9OvDJ5qwchF9cUAYe4arOyQ0vo4agmKPwxv9jqtfoGMk0 IxszDgQxkeR8VBwGdVZqhp7zjIH1pYFf6xH+EiQ+Vj2DOo75u4HzBN1tAlZPYI6h5a 46pde5b0jrmtg== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Catalin Marinas , Marc Zyngier , Mark Rutland Subject: [PATCH v6 3/6] arm64: head: record the MMU state at primary entry Date: Tue, 29 Nov 2022 17:14:15 +0100 Message-Id: <20221129161418.1968319-4-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129161418.1968319-1-ardb@kernel.org> References: <20221129161418.1968319-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5596; i=ardb@kernel.org; h=from:subject; bh=2Olt+kmxs4NdbHYZ6IbTcuFcKJShiSGgmZUFx4t4lTU=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhi/TG7WYkzHsQnU4InkgB+IbQqcf8WBJJeyroATV JKD51RmJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4Yv0wAKCRDDTyI5ktmPJBEEC/ 0SuhhpExln4DX5B4JAgLwoCQCo0P06tpyaO7jaqlcDhORkcxH6VfZOBF4cNgZoBQxdhRK4P3+f2HSR FJAGQfLoJunUtbKT6lpRHSsfExvZ6VkzR8oXqflVFWEcfuVxvGt6VggJQxX0h5d1PdP3h48xQye3Es n9HYgJYbhROXvr8AqtbvUCs6p/brQGK3UdeN8xwfL4eL9XyxVaNmrcAdUjtr60LDsj3suJS+xydKjQ J6gH1FVtfc961aU+61Dlyws3iO5BOVjUYJ0DfQDCQ8tvEy5ao4yhHPW8rq8Xpn6eRjppA3Ia+BNsjM meXINLi+bAWM6wXU7wLPtg9FlD95KAvyFtu/c+xKhcqaxZKsYdH+c9gG6Nwb8xvlZyXlaM9VcjPEWx p3RuQhT66r24YjiXN8ZYR3cVHeBmVHzbKa/V6DnSkug2fWg9rsMghIZfC6i0UbMt+/4IoiMaMvMj/W SKjGHkWXQCfmbyY/x2OA6EBSrnQmDFavz1BW1mOwo6o84= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_081430_160197_B80F29EB X-CRM114-Status: GOOD ( 19.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare for being able to deal with primary entry with the MMU and caches enabled, by recording whether or not we entered with the MMU on in register x19 and in a global variable. (Note that writing to this variable does not require cache invalidation, nor is it required for storing the bootargs in that case, so omit the cache maintenance). Since boot with the MMU enabled is not permitted by the bare metal boot protocol, ensure that a diagnostic is emitted and a taint bit set if the MMU was found to be enabled on a non-EFI boot. We will make an exception for EFI boot later, which has strict requirements for the mapping of system memory, permitting us to relax the boot protocol and hand over from the EFI stub to the core kernel with MMU and caches left enabled. To reduce the likelihood that bare metal boot will violate this requirement, introduce a separate entry point for EFI boot, which is different from the one that is invoked by branching to offset #0 in the image. While at it, add 'pre_disable_mmu_workaround' macro invocations to init_kernel_el, as its manipulation of SCTLR_ELx may amount to disabling of the MMU after subsequent patches. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 28 ++++++++++++++++++-- arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kernel/setup.c | 9 +++++-- 3 files changed, 34 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index bec97aad092c2b43..c3b97f4ae6d769f7 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -77,6 +77,7 @@ * primary lowlevel boot path: * * Register Scope Purpose + * x19 primary_entry() .. start_kernel() whether we entered with the MMU on * x20 primary_entry() .. __primary_switch() CPU boot mode * x21 primary_entry() .. start_kernel() FDT pointer passed at boot in x0 * x22 create_idmap() .. start_kernel() ID map VA of the DT blob @@ -85,8 +86,13 @@ * x25 primary_entry() .. start_kernel() supported VA size * x28 create_idmap() callee preserved temp register */ -SYM_CODE_START(primary_entry) - bl preserve_boot_args +SYM_CODE_START(efi_primary_entry) + bl record_mmu_state + b 0f + +SYM_INNER_LABEL(primary_entry, SYM_L_LOCAL) + mov x19, xzr // MMU must be off on bare metal boot +0: bl preserve_boot_args bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 bl create_idmap @@ -109,6 +115,18 @@ SYM_CODE_START(primary_entry) b __primary_switch SYM_CODE_END(primary_entry) +SYM_CODE_START_LOCAL(record_mmu_state) + mrs x19, CurrentEL + cmp x19, #CurrentEL_EL2 + mrs x19, sctlr_el1 + b.ne 0f + mrs x19, sctlr_el2 +0: tst x19, #SCTLR_ELx_C // Z := (C == 0) + and x19, x19, #SCTLR_ELx_M // isolate M bit + csel x19, xzr, x19, eq // clear x19 if Z + ret +SYM_CODE_END(record_mmu_state) + /* * Preserve the arguments passed by the bootloader in x0 .. x3 */ @@ -119,11 +137,14 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: str_l x19, mmu_enabled_at_boot, x0 + ret SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(clear_page_tables) @@ -497,6 +518,7 @@ SYM_FUNC_START(init_kernel_el) SYM_INNER_LABEL(init_el1, SYM_L_LOCAL) mov_q x0, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround msr sctlr_el1, x0 isb mov_q x0, INIT_PSTATE_EL1 @@ -529,11 +551,13 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) cbz x0, 1f /* Set a sane SCTLR_EL1, the VHE way */ + pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x1 mov x2, #BOOT_CPU_FLAG_E2H b 2f 1: + pre_disable_mmu_workaround msr sctlr_el1, x1 mov x2, xzr 2: diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 8151412653de209c..b925094692983734 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -11,7 +11,7 @@ #endif PROVIDE(__efistub_kernel_size = _edata - _text); -PROVIDE(__efistub_primary_entry_offset = primary_entry - _text); +PROVIDE(__efistub_primary_entry_offset = efi_primary_entry - _text); /* * The EFI stub has its own symbol namespace prefixed by __efistub_, to diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 12cfe9d0d3fac10d..0ac8605a8efe00a5 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -58,6 +58,7 @@ static int num_standard_resources; static struct resource *standard_resources; phys_addr_t __fdt_pointer __initdata; +u64 mmu_enabled_at_boot __initdata; /* * Standard memory resources @@ -332,8 +333,12 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p) xen_early_init(); efi_init(); - if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0) - pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!"); + if (!efi_enabled(EFI_BOOT)) { + if ((u64)_text % MIN_KIMG_ALIGN) + pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!"); + WARN_TAINT(mmu_enabled_at_boot, TAINT_FIRMWARE_WORKAROUND, + FW_BUG "Booted with MMU enabled!"); + } arm64_memblock_init();