From patchwork Fri Nov 24 10:19:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13467572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2363C636CB for ; Fri, 24 Nov 2023 11:31:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=KQa/ljbJ48ppHIMthzqbM3RhLhiBc0c5h3xfDRWz8Qs=; b=EJDJ93K0IjWxjCqM3I+KPse4hz eZOEwHcna0K+vrBQqlS4GwQN2QvlG68sRBPXfVZPPY1UVrplc7CF4lJlZUFDZsUk9RkW2AAL1Yedh xg/ua6gZHzhIYJIysmKu4EB/kz4itYuKbh8gKjIVtIXtNMHvpyOnhFufLdEyAOfaH9YR9mDG51g7N J57fjA9t26p60m5rwj0B6Eg63dCejchJxrgSWEW+QbSO7ZimJwc/p5eXGlp59CgW7Za1J0GtGW6gJ fww1jUcN3pAEtxSDfY7ZBmcQWdDBCobDYxrNOTTJOJaXyygWSkZ/F+ei0k0uO+FhxQo9YZKu/+S2d PRj1zIkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r6UPY-006yVq-2I; Fri, 24 Nov 2023 11:31:52 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r6TJ4-006otv-0Q for linux-arm-kernel@lists.infradead.org; Fri, 24 Nov 2023 10:21:17 +0000 Received: by mail-wm1-x349.google.com with SMTP id 5b1f17b1804b1-40b2977d6baso8983825e9.2 for ; Fri, 24 Nov 2023 02:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1700821264; x=1701426064; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=zO78n1C+IUUd4qQDduar+mDDdO70FT4CA4aVaSqaf3o=; b=pxsXNw8cpaYHd/KuLKIvW+/BUQYEnjGNA1bP19SwPWRJvU4x+Pf9HTjjZO6SHhViYb Xtuhs1l4PCY+LUP7aRrajDFaeK0Pgwv/ejhxFGGfehd7/rb1lLpxMGGoQ4TKgFuWlDp7 oWKKUSoG2W2QuV7gTZp4gdbvpyRxe9j6m2kYdaqQO37hkAyfNM5eqKv7vgedroVVGUOo NF6pKgTECpKKkg5bEiVfASYQzRJBbbDFr/Tg++9M7CVkjleEs/UJ5J/9eBeAG/JWUNLe T0kWjz/0+gz3EyOaQ49jUk9YE0ycA+XCMD0S2r9aCxNee6epjlwZGr81jfSWjoYU2kBQ iHsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700821264; x=1701426064; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zO78n1C+IUUd4qQDduar+mDDdO70FT4CA4aVaSqaf3o=; b=UnEygCut4OdrZBppxtS6JXLHQLZZxqdN/ugwtizy0dAQCMvF8HL7wB90U+Fj7AeVW4 SiV234VhMBh6Qeupz2rXdayJObYxQn1UpotqVy660xqEZ05XOv2jtElBH7wYDjg5i6wr tafEkhzAJtR2LDMm13MAViJd58pmpyGOF5crykZs1lry5vC4B8oYRM2xY3jzvrIgLDQI 9V2Fs3W6O5dPBXilhs/Sl7FE2s800hu4p2Yi1yRjdc+p3khA7s7ZJBVXFhG3MvIqMW9w 8DmnUnTwR40/SJx4pndq/0UkleSbhnCauKVGeygFLryC3L4fbAtt61XoWdzJpELhPzKg 3tCA== X-Gm-Message-State: AOJu0YyS3cehsipXHc6yYZBCNFnq7Z6sUF7y2DTQoC4d1iivL84SokLM dj2aDTrEbArr1FvNRMOuEotpltKXMbrHylAw//iK1Ej1jryWb8uxD8uCmmWx/f1/KkucUKESwY6 gx5jge7EJTc+OBZ/wBu+5CSYyq6mbpU4N+yBgWyXROJVBAEBE4th3luDRp6gD87YKzOxHNFI0lI A= X-Google-Smtp-Source: AGHT+IFds6lDNxC+wfxNjjQ4BJ0IOb3Lhz4+mu1JWEbRpip6yVBtW9wTPhlgHd/mpJ5KZ+rN2yoADRiF X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:458d:b0:409:687f:3419 with SMTP id r13-20020a05600c458d00b00409687f3419mr45473wmo.0.1700821263775; Fri, 24 Nov 2023 02:21:03 -0800 (PST) Date: Fri, 24 Nov 2023 11:19:02 +0100 In-Reply-To: <20231124101840.944737-41-ardb@google.com> Mime-Version: 1.0 References: <20231124101840.944737-41-ardb@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3393; i=ardb@kernel.org; h=from:subject; bh=nYRIcwIYlsV304zLeGlXyA6jIf75Zhb+sT/wec3N1GQ=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JITWhYj4/24Ypr9ZJbud4t7vd+0aRXNH+M5+PfskzELu9V WDK58XRHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAi55YxMjzom9167kOz5uQd vy8WeNpt+/zyaEh9S/iNW2kGd3lXWM9j+B+yvtnuePqOsj5Dh4/XFvp9SQ45cIf1/JoHR1pWn2h I2soGAA== X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog Message-ID: <20231124101840.944737-63-ardb@google.com> Subject: [PATCH v5 22/39] arm64: head: Clear BSS and the kernel page tables in one go From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231124_022106_258480_6B81137D X-CRM114-Status: GOOD ( 15.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel We will move the CPU feature overrides into BSS in a subsequent patch, and this requires that BSS is zeroed before the feature override detection code runs. So let's map BSS read-write in the ID map, and zero it via this mapping. Since the kernel page tables are right next to it, and also zeroed via the ID map, let's drop the separate clear_page_tables() function, and just zero everything in one go. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 33 +++++++------------- arch/arm64/kernel/vmlinux.lds.S | 3 ++ 2 files changed, 14 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index ca5e5fbefcd3..2af518161f3a 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -177,17 +177,6 @@ SYM_CODE_START_LOCAL(preserve_boot_args) ret SYM_CODE_END(preserve_boot_args) -SYM_FUNC_START_LOCAL(clear_page_tables) - /* - * Clear the init page tables. - */ - adrp x0, init_pg_dir - adrp x1, init_pg_end - sub x2, x1, x0 - mov x1, xzr - b __pi_memset // tail call -SYM_FUNC_END(clear_page_tables) - /* * Macro to populate page table entries, these entries can be pointers to the next level * or last level entries pointing to physical memory. @@ -386,9 +375,9 @@ SYM_FUNC_START_LOCAL(create_idmap) map_memory x0, x1, x3, x6, x7, x3, IDMAP_PGD_ORDER, x10, x11, x12, x13, x14, EXTRA_SHIFT - /* Remap the kernel page tables r/w in the ID map */ + /* Remap BSS and the kernel page tables r/w in the ID map */ adrp x1, _text - adrp x2, init_pg_dir + adrp x2, __bss_start adrp x3, _end bic x4, x2, #SWAPPER_BLOCK_SIZE - 1 mov_q x5, SWAPPER_RW_MMUFLAGS @@ -489,14 +478,6 @@ SYM_FUNC_START_LOCAL(__primary_switched) mov x0, x20 bl set_cpu_boot_mode_flag - // Clear BSS - adr_l x0, __bss_start - mov x1, xzr - adr_l x2, __bss_stop - sub x2, x2, x0 - bl __pi_memset - dsb ishst // Make zero page visible to PTW - #if VA_BITS > 48 adr_l x8, vabits_actual // Set this early so KASAN early init str x25, [x8] // ... observes the correct value @@ -782,6 +763,15 @@ SYM_FUNC_START_LOCAL(__primary_switch) adrp x1, reserved_pg_dir adrp x2, init_idmap_pg_dir bl __enable_mmu + + // Clear BSS + adrp x0, __bss_start + mov x1, xzr + adrp x2, init_pg_end + sub x2, x2, x0 + bl __pi_memset + dsb ishst // Make zero page visible to PTW + #ifdef CONFIG_RELOCATABLE adrp x23, KERNEL_START and x23, x23, MIN_KIMG_ALIGN - 1 @@ -796,7 +786,6 @@ SYM_FUNC_START_LOCAL(__primary_switch) orr x23, x23, x0 // record kernel offset #endif #endif - bl clear_page_tables bl create_kernel_mapping adrp x1, init_pg_dir diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 8dd5dda66f7c..8a3c6aacc355 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -311,12 +311,15 @@ SECTIONS __pecoff_data_rawsize = ABSOLUTE(. - __initdata_begin); _edata = .; + /* start of zero-init region */ BSS_SECTION(SBSS_ALIGN, 0, 0) . = ALIGN(PAGE_SIZE); init_pg_dir = .; . += INIT_DIR_SIZE; init_pg_end = .; + /* end of zero-init region */ + #ifdef CONFIG_RELOCATABLE . += SZ_4K; /* stack for the early relocation code */ early_init_stack = .;