From patchwork Mon Apr 11 09:48:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12808909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6310EC433F5 for ; Mon, 11 Apr 2022 10:12:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/NnUkjw6C3gE5kYd4WA5T0pM2Y8GngjYbXhKBeS66eU=; b=Ked7A6rosg3fgo 4mjCC4gf03NlD11l81xY6cP6PxpdRICBkg1cPXVNPg0dmokPUtZ4h0UeEqs6Q8BZlaRMdVXIYdG+z Eig/0i/PnQ9Z/hbgxSnfU+N9O1V1NeWmPLKU/GQEUxux7DFrB++EmzzlbmYGKMDarEmduxblDxPv8 weftcIC8Cuic5nudCS/T45VVkaVMBWnSckVHWf6f9BYRKusR83286IkJblNe80HQ90e3YQbMu6RIf zWkbjpX4WX2s3d5nutcNoYFJLMEJpoO7TgQDTjK3K9VYOPOIdz6Db0MUkYDlqFpxP7qzzGVPQrYcH XqRywhNiqTvJEgWWEP5Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ndr0v-008Ck3-K3; Mon, 11 Apr 2022 10:11:18 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ndqfy-0082al-LV for linux-arm-kernel@lists.infradead.org; Mon, 11 Apr 2022 09:49:41 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2791F611B3; Mon, 11 Apr 2022 09:49:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 099B8C385AD; Mon, 11 Apr 2022 09:49:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1649670578; bh=YtSuSXiDuWch666ErRH9dPdBAf8I6jd+Df9mu/gOaZw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DzLw3aWBy8WmRMoMjWoqIcOelOmc7L1YCVZO87zpXRuv7AQfJhAoKh9XgmWGLXbQg 1s0Rxku6fCCVr1cfEvyngjAwWXap3S5+R25Xow4Lr+s5VZjvURR1ODfMrRL+iC3QAT tXRDd67rvKibOmOkwksJ2IpCVDMw0/cR98e+KiXCptX3ON4UwIdzo+K7defpnD351c SK9PO65fFW9VN6tKyEUnaA8xO9qwEfJcnaYQwqNfk4A6+USAdnZTAji07U6Wsgqj0l AKQ0/bJVAKkWdfPG1Rs506goNeoBKp67y8Y6iuIQxLVt4S+6M+d2DGASlYVYRyWDnz yYKHIphhJH8OA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-efi@vger.kernel.org, Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH v3 23/30] arm64: mm: move ro_after_init section into the data segment Date: Mon, 11 Apr 2022 11:48:17 +0200 Message-Id: <20220411094824.4176877-24-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220411094824.4176877-1-ardb@kernel.org> References: <20220411094824.4176877-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5001; h=from:subject; bh=YtSuSXiDuWch666ErRH9dPdBAf8I6jd+Df9mu/gOaZw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBiU/lbsbTLg/fLmyhtxio90Ih3mCllrNJrjvzr0JA3 YnjysQGJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYlP5WwAKCRDDTyI5ktmPJO44C/ 99Mc64RsrOpbZiSXzUz24edLR0Dx3KSU/rQpBTVvLaCGh4xktmqYkCltVxTlYkDcb8ncSYOBvgN/Wu Zviqaxhfm5iWgQHVwcL8b0cVieNBTLgud6/hPQWcHPwfnW+Cny4oCnpbwQCj4iayX1wI+xG7tFgoab jHLDtbnBxtKRy0uKNfuFlKHKmOFxQDReXQQlz2FzRlw5/x2gcxddiouIAMpBvvWML0Zwlkp1aTPkLG NUw3u5aOFA/BmAEG+WjT3ZjymCSpRMuBjB1eBFN5EDs5P0vBQAp+OSzzmwpl3vQBQvOSVSeuAI2epo SJZ1fyZEfWXMH2eAGUvjVyCs7umdHxmYSe7HpILabyL/WpvpTm9W1XnREeHcKg1Vz5Eii18TgTycNG 1//gYVgoXlQrsYUUo/FeBRhXTdpZF27/HCh43x/sgPB6E0TtDKa7hE5zeCd+Cs5ms1UuDgSjDVZJSp KJ04aIiU+zC3rm99GbxEtHoc3VfKxfrOKGRMQt0wH46l8= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220411_024938_845950_BAFDB5DD X-CRM114-Status: GOOD ( 16.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, the ro_after_init sections sits right in the middle of the text/rodata/inittext segment, making it difficult to map any of those non-writable during early boot. So instead, move it to the start of .data, and update the init sequences so that the section is remapped read-only once startup completes. Note that this moves the entire HYP data section into .data as well - this likely needs to remain as a single block for now, but could perhaps split into a .rodata and .data..ro_after_init section later. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/vmlinux.lds.S | 41 ++++++++++++-------- arch/arm64/mm/mmu.c | 29 ++++++++------ 2 files changed, 42 insertions(+), 28 deletions(-) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index cb4821c411f4..5b465295335a 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -59,6 +59,7 @@ #define RO_EXCEPTION_TABLE_ALIGN 4 #define RUNTIME_DISCARD_EXIT +#define RO_AFTER_INIT_DATA #include #include @@ -192,22 +193,6 @@ SECTIONS /* everything from this point to __init_begin will be marked RO NX */ RO_DATA(PAGE_SIZE) - HYPERVISOR_DATA_SECTIONS - - idmap_pg_dir = .; - . += PAGE_SIZE; - -#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - tramp_pg_dir = .; - . += PAGE_SIZE; -#endif - - reserved_pg_dir = .; - . += PAGE_SIZE; - - swapper_pg_dir = .; - . += PAGE_SIZE; - . = ALIGN(SEGMENT_ALIGN); __init_begin = .; __inittext_begin = .; @@ -270,6 +255,30 @@ SECTIONS _data = .; _sdata = .; + + __start_ro_after_init = .; + idmap_pg_dir = .; + . += PAGE_SIZE; + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_pg_dir = .; + . += PAGE_SIZE; +#endif + reserved_pg_dir = .; + . += PAGE_SIZE; + + swapper_pg_dir = .; + . += PAGE_SIZE; + + HYPERVISOR_DATA_SECTIONS + + .data.ro_after_init : { + *(.data..ro_after_init) + JUMP_TABLE_DATA + . = ALIGN(SEGMENT_ALIGN); + __end_ro_after_init = .; + } + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) /* diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index e7145f0281be..ef1f01da387d 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -488,11 +488,17 @@ static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, void __init mark_linear_text_alias_ro(void) { /* - * Remove the write permissions from the linear alias of .text/.rodata + * Remove the write permissions from the linear alias of .text/.rodata/ro_after_init */ update_mapping_prot(__pa_symbol(_stext), (unsigned long)lm_alias(_stext), (unsigned long)__init_begin - (unsigned long)_stext, PAGE_KERNEL_RO); + + update_mapping_prot(__pa_symbol(__start_ro_after_init), + (unsigned long)lm_alias(__start_ro_after_init), + (unsigned long)__end_ro_after_init - + (unsigned long)__start_ro_after_init, + PAGE_KERNEL_RO); } static bool crash_mem_map __initdata; @@ -601,12 +607,10 @@ void mark_rodata_ro(void) { unsigned long section_size; - /* - * mark .rodata as read only. Use __init_begin rather than __end_rodata - * to cover NOTES and EXCEPTION_TABLE. - */ - section_size = (unsigned long)__init_begin - (unsigned long)__start_rodata; - update_mapping_prot(__pa_symbol(__start_rodata), (unsigned long)__start_rodata, + section_size = (unsigned long)__end_ro_after_init - + (unsigned long)__start_ro_after_init; + update_mapping_prot(__pa_symbol(__start_ro_after_init), + (unsigned long)__start_ro_after_init, section_size, PAGE_KERNEL_RO); debug_checkwx(); @@ -730,18 +734,19 @@ static void __init map_kernel(pgd_t *pgdp) text_prot = __pgprot_modify(text_prot, PTE_GP, PTE_GP); /* - * Only rodata will be remapped with different permissions later on, - * all other segments are allowed to use contiguous mappings. + * Only data will be partially remapped with different permissions + * later on, all other segments are allowed to use contiguous mappings. */ map_kernel_segment(pgdp, _stext, _etext, text_prot, &vmlinux_text, 0, VM_NO_GUARD); - map_kernel_segment(pgdp, __start_rodata, __inittext_begin, PAGE_KERNEL, - &vmlinux_rodata, NO_CONT_MAPPINGS, VM_NO_GUARD); + map_kernel_segment(pgdp, __start_rodata, __inittext_begin, PAGE_KERNEL_RO, + &vmlinux_rodata, 0, VM_NO_GUARD); map_kernel_segment(pgdp, __inittext_begin, __inittext_end, text_prot, &vmlinux_inittext, 0, VM_NO_GUARD); map_kernel_segment(pgdp, __initdata_begin, __initdata_end, PAGE_KERNEL, &vmlinux_initdata, 0, VM_NO_GUARD); - map_kernel_segment(pgdp, _data, _end, PAGE_KERNEL, &vmlinux_data, 0, 0); + map_kernel_segment(pgdp, _data, _end, PAGE_KERNEL, &vmlinux_data, + NO_CONT_MAPPINGS | NO_BLOCK_MAPPINGS, 0); if (!READ_ONCE(pgd_val(*pgd_offset_pgd(pgdp, FIXADDR_START)))) { /*