From patchwork Mon Jun 13 14:45:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12879702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3572BC433EF for ; Mon, 13 Jun 2022 14:59:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7PBIFnszIARRmapL8yRmkvX8h5Pjt0UMBLmba+pvJH8=; b=fJd8yiey37WBap eTCRpNTFYyz+zNx/sE3UREjlQJBqveSDtygbrw+ZVwp6xJM87rGqG5RXMSOnqnwfmu/TmJwUmlHM7 LEgs1Taf016zZ+y9uDCxCY24uc6Hve4GyIMf+gPgAPbmXa7GRJlt+ZgJlVUw8QnSCUfFkY/6KSI6/ tMZVZ+t2S/9TLdvIxxRrd5CBE8tgN/iCjN3bRMpZF6ksQi0zHNPhYKAiBxkDdlNhd193Q2iDmQNfC dtwYRE5SmxH1IL2LBYU4vIx1qjRXK/yKQlnL0rc57l0aCHcYN3TX65Ga1IsQCKLvTez9yKG1qHg9c 5y02C9ejBgBClmMoKXzg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o0lWe-004MQd-TS; Mon, 13 Jun 2022 14:58:45 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o0lLH-004GjI-3r for linux-arm-kernel@lists.infradead.org; Mon, 13 Jun 2022 14:47:00 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id ABB6F612DF; Mon, 13 Jun 2022 14:46:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E253C341C0; Mon, 13 Jun 2022 14:46:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655131618; bh=E2NfwwMf/mlkhPj0/Dm+COiDScQU3SGHYa/X8m18eYE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I4N9qiwFfoR62Ybu7y9csLyCKKSZH3LDFRn+r4JQ3hSD3m8dV11KTd51+eghMwh/O Eq+z/+tff4f26wkAQqbYso2GidBJkYL0YC0LIu8F4mry6+PXydwPZA8IvvScu5FDKr 4Y3q8XChOdlZBf4zK5ik4CMy9F3qnMgV8MzTXwQXl5MFNIhldgCdgwBJgci0p8VJ7N ERyJb/0CNUJFOibPDX6asVqLc+zizx+I70GiQOIhXGhGUQSv5zbioSfVbuaJX5C/tT 7tt5w9mGvM9imoHSOaGE7YGwOHQV+GnRd+sdt16hr40C8GXPkngxI2ZXtgDlahfJaF RHGJbSYZcVEag== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-hardening@vger.kernel.org, Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown , Anshuman Khandual Subject: [PATCH v4 22/26] arm64: mm: move ro_after_init section into the data segment Date: Mon, 13 Jun 2022 16:45:46 +0200 Message-Id: <20220613144550.3760857-23-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220613144550.3760857-1-ardb@kernel.org> References: <20220613144550.3760857-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5175; h=from:subject; bh=E2NfwwMf/mlkhPj0/Dm+COiDScQU3SGHYa/X8m18eYE=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBip02WLqn4rwEiXk3ewFAo3CfcRzxVyaG2WpgZTFy+ 8seXWiaJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYqdNlgAKCRDDTyI5ktmPJBJUC/ 9L3WpKh5i+ZvRIl6yH1JT4eqIWI8jmrXBewTgWLpOa0Cxtdyz0bkBabh6KNVIxaVLL1vxmdXYGIXNl x28T9EdIakKEZ4ZlQ4V4wDve+KRcW8GOP+tLMwRGAUWKYDhAjj/Ox46eM8J4VG5bEBVqSyOVZuRVCq F2NjYa/NKrFsmwrSw0lYgZ0lPQbaT6tacCsL77IYBa1nVNT6Q+waDPv5VbVWShRCyBkFK+dMzE08T6 ScVIjheOje+TROF9XBe+PD9jOrCXbbJF9JovohSomFoKqwXLZjYtc1JJmtK8lCArLKta0rtD8xcLrD GO01KcH/b2CcfVIl3uVpa2kL2tM7oXyOkMZb109oWPeRMCdlllh0qNvdRh9lcQFD/u0WjPdm8+F0aA 44/0vtBCx6Q+f4+CBMQhemoh5qxYAHG4kU33NV7eAYA3uK0ZKezYTq+RBLPMBlX2AiVR9NdE3+JpOX 08xME3mQcIi5NkKyPBWhctI2Og7vmsO+AbFeLD/ObDSl0= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220613_074659_263436_7761DABD X-CRM114-Status: GOOD ( 17.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, the ro_after_init sections sits right in the middle of the text/rodata/inittext segment, making it difficult to map any of those non-writable during early boot. So instead, move it to the start of .data, and update the init sequences so that the section is remapped read-only once startup completes. Note that this moves the entire HYP data section into .data as well - this likely needs to remain as a single block for now, but could perhaps split into a .rodata and .data..ro_after_init section later. Signed-off-by: Ard Biesheuvel Reviewed-by: Kees Cook --- arch/arm64/kernel/vmlinux.lds.S | 42 ++++++++++++-------- arch/arm64/mm/mmu.c | 29 ++++++++------ 2 files changed, 42 insertions(+), 29 deletions(-) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 45131e354e27..736aca63dad1 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -59,6 +59,7 @@ #define RO_EXCEPTION_TABLE_ALIGN 4 #define RUNTIME_DISCARD_EXIT +#define RO_AFTER_INIT_DATA #include #include @@ -188,30 +189,13 @@ SECTIONS /* everything from this point to __init_begin will be marked RO NX */ RO_DATA(PAGE_SIZE) - HYPERVISOR_DATA_SECTIONS - /* code sections that are never executed via the kernel mapping */ .rodata.text : { TRAMP_TEXT HIBERNATE_TEXT KEXEC_TEXT - . = ALIGN(PAGE_SIZE); } - idmap_pg_dir = .; - . += PAGE_SIZE; - -#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - tramp_pg_dir = .; - . += PAGE_SIZE; -#endif - - reserved_pg_dir = .; - . += PAGE_SIZE; - - swapper_pg_dir = .; - . += PAGE_SIZE; - . = ALIGN(SEGMENT_ALIGN); __init_begin = .; __inittext_begin = .; @@ -274,6 +258,30 @@ SECTIONS _data = .; _sdata = .; + + __start_ro_after_init = .; + idmap_pg_dir = .; + . += PAGE_SIZE; + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_pg_dir = .; + . += PAGE_SIZE; +#endif + reserved_pg_dir = .; + . += PAGE_SIZE; + + swapper_pg_dir = .; + . += PAGE_SIZE; + + HYPERVISOR_DATA_SECTIONS + + .data.ro_after_init : { + *(.data..ro_after_init) + JUMP_TABLE_DATA + . = ALIGN(SEGMENT_ALIGN); + __end_ro_after_init = .; + } + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) /* diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9828ad826837..e9b074ffc768 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -495,11 +495,17 @@ static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, void __init mark_linear_text_alias_ro(void) { /* - * Remove the write permissions from the linear alias of .text/.rodata + * Remove the write permissions from the linear alias of .text/.rodata/ro_after_init */ update_mapping_prot(__pa_symbol(_stext), (unsigned long)lm_alias(_stext), (unsigned long)__init_begin - (unsigned long)_stext, PAGE_KERNEL_RO); + + update_mapping_prot(__pa_symbol(__start_ro_after_init), + (unsigned long)lm_alias(__start_ro_after_init), + (unsigned long)__end_ro_after_init - + (unsigned long)__start_ro_after_init, + PAGE_KERNEL_RO); } static bool crash_mem_map __initdata; @@ -608,12 +614,10 @@ void mark_rodata_ro(void) { unsigned long section_size; - /* - * mark .rodata as read only. Use __init_begin rather than __end_rodata - * to cover NOTES and EXCEPTION_TABLE. - */ - section_size = (unsigned long)__init_begin - (unsigned long)__start_rodata; - update_mapping_prot(__pa_symbol(__start_rodata), (unsigned long)__start_rodata, + section_size = (unsigned long)__end_ro_after_init - + (unsigned long)__start_ro_after_init; + update_mapping_prot(__pa_symbol(__start_ro_after_init), + (unsigned long)__start_ro_after_init, section_size, PAGE_KERNEL_RO); debug_checkwx(); @@ -733,18 +737,19 @@ static void __init map_kernel(pgd_t *pgdp) text_prot = __pgprot_modify(text_prot, PTE_GP, PTE_GP); /* - * Only rodata will be remapped with different permissions later on, - * all other segments are allowed to use contiguous mappings. + * Only data will be partially remapped with different permissions + * later on, all other segments are allowed to use contiguous mappings. */ map_kernel_segment(pgdp, _stext, _etext, text_prot, &vmlinux_text, 0, VM_NO_GUARD); - map_kernel_segment(pgdp, __start_rodata, __inittext_begin, PAGE_KERNEL, - &vmlinux_rodata, NO_CONT_MAPPINGS, VM_NO_GUARD); + map_kernel_segment(pgdp, __start_rodata, __inittext_begin, PAGE_KERNEL_RO, + &vmlinux_rodata, 0, VM_NO_GUARD); map_kernel_segment(pgdp, __inittext_begin, __inittext_end, text_prot, &vmlinux_inittext, 0, VM_NO_GUARD); map_kernel_segment(pgdp, __initdata_begin, __initdata_end, PAGE_KERNEL, &vmlinux_initdata, 0, VM_NO_GUARD); - map_kernel_segment(pgdp, _data, _end, PAGE_KERNEL, &vmlinux_data, 0, 0); + map_kernel_segment(pgdp, _data, _end, PAGE_KERNEL, &vmlinux_data, + NO_CONT_MAPPINGS | NO_BLOCK_MAPPINGS, 0); if (!READ_ONCE(pgd_val(*pgd_offset_pgd(pgdp, FIXADDR_START)))) { /*