From patchwork Fri Nov 11 17:11:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13040639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A580C4332F for ; Fri, 11 Nov 2022 17:26:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cP0N586VK5xIHik6oEEBZ/l3qIIPiO2/EDBN+IweqSw=; b=kgbyLNCj515pFT 17cXXLpWYJI7uuUkz6yysqa5lAKzgg+J97G+xWehDk5sOWrGkl0R/ozOpcRsRCPPKD67ED1/sMOjL ivAQUXf/JbdNIHBVPkjnzMjeHKKrehP0jAJuKn3nuPaTx/yivet+/BtCuDvgQGYEBaMiFKu+mw7Yg PRS4wX1GsMsRKkc3YS7f+/DBGOpfqWfO2Qqg7+HTDAzFKqDlbWsWTpxUw00rba1iers68Zz32aTfp yZP4SAAw3O1KhGsp5h/XRQ2twCQLPAKQ+HkBQeHGo+3H0e5De8ooVLsWXHjzbzT3ICreQ7oFAvuay 1t5Xe8St25Jj0TdHwBHQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otXmR-00HEU9-4C; Fri, 11 Nov 2022 17:25:27 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otXaR-00H7sL-7M for linux-arm-kernel@lists.infradead.org; Fri, 11 Nov 2022 17:13:05 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B199262068; Fri, 11 Nov 2022 17:13:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3F09DC433D6; Fri, 11 Nov 2022 17:13:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1668186782; bh=47ONy3JhJC4HIAvQ6skMJjbwiDZp25Gx2Jf027x3lRU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XDUilcY6LwqxCAXAv9GPNEdxiOlbmSzMSM+1I/ThVNai4uYjoTaZ8l90yziVWjvPK S8miw0YThFwnxxs63BvWo2wXGedzP4EOOmkHVSdxxHS4heoJH2b2F0PWCzR4v3efN3 7YO402Xxrhps+aucTTwPrcFZtelgiKrfwCZRp3V9j+gkNq71iH95e5JSPeGNOVolj+ YqKvhIG7Aq/1UpShAoe5oftK0OnXFTp0NkLWSTKQe4md5Hfvk9XzzFoSU6eHS/tVZX 2ynyJva6fXmOJGTaxQ1s3vw/Dd8Als81bAiy/B+rMHeJOOS9OJNalPKG4v2eti4cW8 Bqb9n8Y8MPueQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown , Anshuman Khandual Subject: [PATCH v7 19/33] arm64: head: Clear BSS and the kernel page tables in one go Date: Fri, 11 Nov 2022 18:11:47 +0100 Message-Id: <20221111171201.2088501-20-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221111171201.2088501-1-ardb@kernel.org> References: <20221111171201.2088501-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2739; i=ardb@kernel.org; h=from:subject; bh=47ONy3JhJC4HIAvQ6skMJjbwiDZp25Gx2Jf027x3lRU=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjboJIg8xKEi1l187QYJI3mHvpUWHw8wbNqdDl7HXs vzkZ75KJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY26CSAAKCRDDTyI5ktmPJK5IC/ 4owhyNvqdGEKmANvkvY6KRORJRmPqN6Hw2E7QUvCYEKcYdemUCmBO3Q5EtQKVqj1VXU5FzDhDWTtJE J6fhXnbMVjxLQ8oMt6es9vz9J23M/RZuUReoOxVN/Yr1o77lbZXXwikvxxAOesbKuRZ3u9lfgwvo4p zIT+niqtIsND2y3/AzfqmKwxwn4BLon6N2zQi9tW2+hy0R3BT5avFX3Qi0vSUMistMqjhNWWOtNdOL Po4A2gAm1kLNrZgSFDtXv75z6kXotsZ1SHBPYP02po+hbaMoJgRZDvm1ORb9glGydHa6KC+1FNJYWP UDzXvXyv8J9UJ8KOu9aPrRvD4T6x5DsjHrt5OUIJUxODI23yk+fpWg/pm/lFvFRhxEgL/e9AGWfsQ4 dhC3d7FE360u9t3c3oI8GKz+GD4TWNFoTk3sApvn9VySF6C64MCK9Q3hlRB6rlWItNhwiuui447PCz kMuRhj2cINOHIJmMSgCG1wnZoTlWOv95JLosxgXNA9FXU= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221111_091303_463995_AA320DB3 X-CRM114-Status: GOOD ( 14.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will move the CPU feature overrides into BSS in a subsequent patch, and this requires that BSS is zeroed before the feature override detection code runs. So let's map BSS read-write in the ID map, and zero it via this mapping. Since the kernel page tables are right next to it, and also zeroed via the ID map, let's drop the separate clear_page_tables() function, and just zero everything in one go. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 33 +++++++------------- 1 file changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 786b7bd79a4026e9..0e7aaa65ea174efc 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -126,17 +126,6 @@ SYM_CODE_START_LOCAL(preserve_boot_args) b dcache_inval_poc // tail call SYM_CODE_END(preserve_boot_args) -SYM_FUNC_START_LOCAL(clear_page_tables) - /* - * Clear the init page tables. - */ - adrp x0, init_pg_dir - adrp x1, init_pg_end - sub x2, x1, x0 - mov x1, xzr - b __pi_memset // tail call -SYM_FUNC_END(clear_page_tables) - /* * Macro to populate page table entries, these entries can be pointers to the next level * or last level entries pointing to physical memory. @@ -335,9 +324,9 @@ SYM_FUNC_START_LOCAL(create_idmap) map_memory x0, x1, x3, x6, x7, x3, IDMAP_PGD_ORDER, x10, x11, x12, x13, x14, EXTRA_SHIFT - /* Remap the kernel page tables r/w in the ID map */ + /* Remap BSS and the kernel page tables r/w in the ID map */ adrp x1, _text - adrp x2, init_pg_dir + adrp x2, __bss_start adrp x3, _end bic x4, x2, #SWAPPER_BLOCK_SIZE - 1 mov x5, SWAPPER_RW_MMUFLAGS @@ -437,14 +426,6 @@ SYM_FUNC_START_LOCAL(__primary_switched) mov x0, x20 bl set_cpu_boot_mode_flag - // Clear BSS - adr_l x0, __bss_start - mov x1, xzr - adr_l x2, __bss_stop - sub x2, x2, x0 - bl __pi_memset - dsb ishst // Make zero page visible to PTW - #if VA_BITS > 48 adr_l x8, vabits_actual // Set this early so KASAN early init str x25, [x8] // ... observes the correct value @@ -708,6 +689,15 @@ SYM_FUNC_START_LOCAL(__primary_switch) adrp x1, reserved_pg_dir adrp x2, init_idmap_pg_dir bl __enable_mmu + + // Clear BSS + adrp x0, __bss_start + mov x1, xzr + adrp x2, init_pg_end + sub x2, x2, x0 + bl __pi_memset + dsb ishst // Make zero page visible to PTW + #ifdef CONFIG_RELOCATABLE adrp x23, KERNEL_START and x23, x23, MIN_KIMG_ALIGN - 1 @@ -722,7 +712,6 @@ SYM_FUNC_START_LOCAL(__primary_switch) orr x23, x23, x0 // record kernel offset #endif #endif - bl clear_page_tables bl create_kernel_mapping adrp x1, init_pg_dir