From patchwork Tue Mar 7 14:04:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13163756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 04E63C678D5 for ; Tue, 7 Mar 2023 14:32:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=v/RtFc5/wSTSrK25QKI55r9RVB+M8Uzl1w40PELFbeo=; b=fO/GXE/x8G7lhj XG2WvbXMNnJ3uPdsJtcnwRH42B4JtIhrL40TGg/leuBtnHv9ETXnTIsbgyCqs2YCstGijj1BU44Kj 5XvxswkQex4JRmfXrFVjKXAFPzIvxS8ufOL0CYaLljBRzwTXBh0k5U9FpINrvOMaYbW3LClEa3Aen i8peYcMwYDjIrurEGpDuUVF1xu0r/UVKe0NgiPz9Vu6OFbtZytqXP+v6LAZQqJkZd1k7dzNY/Md07 oC56w5RD/m/YtTMXDdPy0pQRxj1wB+VtTP6GzSSkmdGH8x7UQ8TzxtV8VJ/zh2bhxOUwiZ8WZxYoP KmjqtS8W5/OlSJ3KxpXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pZYL2-000oTI-W3; Tue, 07 Mar 2023 14:30:49 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pZXyf-000eOM-2e for linux-arm-kernel@lists.infradead.org; Tue, 07 Mar 2023 14:07:42 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A7557B818F7; Tue, 7 Mar 2023 14:07:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 72DE3C433A0; Tue, 7 Mar 2023 14:07:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678198058; bh=WFiiboUNlXGaUmkdqX5fmc34WsQ0VoU7MZXGSi+88TY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZaLcnLPEg0R+H7zkhz3RRod76KFXo1jHQJ+Z2pIZw8gBUIaW920u2MWBAmmWFdtKP 8dvvK9g9F+GKwTaN5prLbCyhc8eVemUZCWJq1ZGQkECClqtwEUjo+zWRE5nRBkgiXY fzJbMXpG0hH+B2/0kM7PEcSKjnIttGefaKzzJRxe5sXlOT4jyYbnp1QJOpXyu/sme2 tL/ldM3Ry/gF4tHQN2bHSIXOAZDGDRq42ujT/Gg+kWZVWZRe39bY4Bdka6rUSICEoQ lNtKXSQLKwpr157uiwgiwPtjo+A14yZGy3ZXJvZw97A2zMhhtbSfu6vgepvECvLty6 vxtK5iidqEnPw== From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook Subject: [PATCH v3 25/60] arm64: head: Clear BSS and the kernel page tables in one go Date: Tue, 7 Mar 2023 15:04:47 +0100 Message-Id: <20230307140522.2311461-26-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230307140522.2311461-1-ardb@kernel.org> References: <20230307140522.2311461-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2710; i=ardb@kernel.org; h=from:subject; bh=WFiiboUNlXGaUmkdqX5fmc34WsQ0VoU7MZXGSi+88TY=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIYXdxa4nKujXcpU7UUunp7dPm2V91+ioYHtD9PcSgc0Kl Q6WK4I7SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwESu3GP4zSJrGhGr3XMr3WO3 MePS4n1MhiGmidrP95xlZ0tJs/26hZGhYcsnfq7aKU21ytETP+vEmM9WnKO85YPQ7uiwN60LBcV ZAA== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230307_060741_467448_BF5A5BDB X-CRM114-Status: GOOD ( 15.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will move the CPU feature overrides into BSS in a subsequent patch, and this requires that BSS is zeroed before the feature override detection code runs. So let's map BSS read-write in the ID map, and zero it via this mapping. Since the kernel page tables are right next to it, and also zeroed via the ID map, let's drop the separate clear_page_tables() function, and just zero everything in one go. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 33 +++++++------------- 1 file changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 0fa44b3188c1e204..ade0cb99c8a83a3d 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -177,17 +177,6 @@ SYM_CODE_START_LOCAL(preserve_boot_args) ret SYM_CODE_END(preserve_boot_args) -SYM_FUNC_START_LOCAL(clear_page_tables) - /* - * Clear the init page tables. - */ - adrp x0, init_pg_dir - adrp x1, init_pg_end - sub x2, x1, x0 - mov x1, xzr - b __pi_memset // tail call -SYM_FUNC_END(clear_page_tables) - /* * Macro to populate page table entries, these entries can be pointers to the next level * or last level entries pointing to physical memory. @@ -386,9 +375,9 @@ SYM_FUNC_START_LOCAL(create_idmap) map_memory x0, x1, x3, x6, x7, x3, IDMAP_PGD_ORDER, x10, x11, x12, x13, x14, EXTRA_SHIFT - /* Remap the kernel page tables r/w in the ID map */ + /* Remap BSS and the kernel page tables r/w in the ID map */ adrp x1, _text - adrp x2, init_pg_dir + adrp x2, __bss_start adrp x3, _end bic x4, x2, #SWAPPER_BLOCK_SIZE - 1 mov x5, SWAPPER_RW_MMUFLAGS @@ -489,14 +478,6 @@ SYM_FUNC_START_LOCAL(__primary_switched) mov x0, x20 bl set_cpu_boot_mode_flag - // Clear BSS - adr_l x0, __bss_start - mov x1, xzr - adr_l x2, __bss_stop - sub x2, x2, x0 - bl __pi_memset - dsb ishst // Make zero page visible to PTW - #if VA_BITS > 48 adr_l x8, vabits_actual // Set this early so KASAN early init str x25, [x8] // ... observes the correct value @@ -780,6 +761,15 @@ SYM_FUNC_START_LOCAL(__primary_switch) adrp x1, reserved_pg_dir adrp x2, init_idmap_pg_dir bl __enable_mmu + + // Clear BSS + adrp x0, __bss_start + mov x1, xzr + adrp x2, init_pg_end + sub x2, x2, x0 + bl __pi_memset + dsb ishst // Make zero page visible to PTW + #ifdef CONFIG_RELOCATABLE adrp x23, KERNEL_START and x23, x23, MIN_KIMG_ALIGN - 1 @@ -794,7 +784,6 @@ SYM_FUNC_START_LOCAL(__primary_switch) orr x23, x23, x0 // record kernel offset #endif #endif - bl clear_page_tables bl create_kernel_mapping adrp x1, init_pg_dir