From patchwork Wed Jun 3 08:00:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 11585195 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FABD739 for ; Wed, 3 Jun 2020 08:01:38 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1897F20734 for ; Wed, 3 Jun 2020 08:01:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="rczD6kJj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1897F20734 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BnlEjCymVECv4dad6JjvIIhG4TtZ85V4mUclr67m+tg=; b=rczD6kJj29dSyi yONczX2z5wTVm4vvG2ZeKn7GV+yB+bt547m1DPpRv1xq/yKfbtVoNEaTs8dv5dX97Pf/3Bu4b0CRb 0kcb0E5+3blGRk+AFTlxgPc31QwrkQRpxdb+K9ajZXK4JT4wP8UnduJeH5q4ePRNbKqfwwY/IHBxw FQM+old+pvtlcAAh1+jWHA4z3ZouQPIpd1IPUuN3whX59NrOUZ1szGTs/JnDkahE2ycvbnQqndUdk dsFsVB4MJuQRXepcg0cJRq29KuDPauOI2DLwzrftthxjEmJO7xxLLM4xsn2d6b99MGq59ldh0BR5m 0bRac6ECpefgCUdi7YiA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgOL9-0008Jp-OC; Wed, 03 Jun 2020 08:01:35 +0000 Received: from relay1-d.mail.gandi.net ([217.70.183.193]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgOL5-0008IW-PC for linux-riscv@lists.infradead.org; Wed, 03 Jun 2020 08:01:34 +0000 X-Originating-IP: 90.112.45.105 Received: from debian.home (lfbn-gre-1-325-105.w90-112.abo.wanadoo.fr [90.112.45.105]) (Authenticated sender: alex@ghiti.fr) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 387A5240004; Wed, 3 Jun 2020 08:01:19 +0000 (UTC) From: Alexandre Ghiti To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Atish Patra , Zong Li , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org Subject: [PATCH v4 1/4] riscv: Move kernel mapping to vmalloc zone Date: Wed, 3 Jun 2020 04:00:07 -0400 Message-Id: <20200603080010.13366-2-alex@ghiti.fr> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200603080010.13366-1-alex@ghiti.fr> References: <20200603080010.13366-1-alex@ghiti.fr> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200603_010132_096532_FB00006A X-CRM114-Status: GOOD ( 19.53 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.193 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.193 listed in wl.mailspike.net] 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexandre Ghiti Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org This is a preparatory patch for relocatable kernel. The kernel used to be linked at PAGE_OFFSET address and used to be loaded physically at the beginning of the main memory. Therefore, we could use the linear mapping for the kernel mapping. But the relocated kernel base address will be different from PAGE_OFFSET and since in the linear mapping, two different virtual addresses cannot point to the same physical address, the kernel mapping needs to lie outside the linear mapping. In addition, because modules and BPF must be close to the kernel (inside +-2GB window), the kernel is placed at the end of the vmalloc zone minus 2GB, which leaves room for modules and BPF. The kernel could not be placed at the beginning of the vmalloc zone since other vmalloc allocations from the kernel could get all the +-2GB window around the kernel which would prevent new modules and BPF programs to be loaded. Signed-off-by: Alexandre Ghiti Reviewed-by: Zong Li --- arch/riscv/boot/loader.lds.S | 3 +- arch/riscv/include/asm/page.h | 10 +++++- arch/riscv/include/asm/pgtable.h | 38 ++++++++++++++------- arch/riscv/kernel/head.S | 3 +- arch/riscv/kernel/module.c | 4 +-- arch/riscv/kernel/vmlinux.lds.S | 3 +- arch/riscv/mm/init.c | 58 +++++++++++++++++++++++++------- arch/riscv/mm/physaddr.c | 2 +- 8 files changed, 88 insertions(+), 33 deletions(-) diff --git a/arch/riscv/boot/loader.lds.S b/arch/riscv/boot/loader.lds.S index 47a5003c2e28..62d94696a19c 100644 --- a/arch/riscv/boot/loader.lds.S +++ b/arch/riscv/boot/loader.lds.S @@ -1,13 +1,14 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include +#include OUTPUT_ARCH(riscv) ENTRY(_start) SECTIONS { - . = PAGE_OFFSET; + . = KERNEL_LINK_ADDR; .payload : { *(.payload) diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 2d50f76efe48..48bb09b6a9b7 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -90,18 +90,26 @@ typedef struct page *pgtable_t; #ifdef CONFIG_MMU extern unsigned long va_pa_offset; +extern unsigned long va_kernel_pa_offset; extern unsigned long pfn_base; #define ARCH_PFN_OFFSET (pfn_base) #else #define va_pa_offset 0 +#define va_kernel_pa_offset 0 #define ARCH_PFN_OFFSET (PAGE_OFFSET >> PAGE_SHIFT) #endif /* CONFIG_MMU */ extern unsigned long max_low_pfn; extern unsigned long min_low_pfn; +extern unsigned long kernel_virt_addr; #define __pa_to_va_nodebug(x) ((void *)((unsigned long) (x) + va_pa_offset)) -#define __va_to_pa_nodebug(x) ((unsigned long)(x) - va_pa_offset) +#define linear_mapping_va_to_pa(x) ((unsigned long)(x) - va_pa_offset) +#define kernel_mapping_va_to_pa(x) \ + ((unsigned long)(x) - va_kernel_pa_offset) +#define __va_to_pa_nodebug(x) \ + (((x) >= PAGE_OFFSET) ? \ + linear_mapping_va_to_pa(x) : kernel_mapping_va_to_pa(x)) #ifdef CONFIG_DEBUG_VIRTUAL extern phys_addr_t __virt_to_phys(unsigned long x); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 35b60035b6b0..94ef3b49dfb6 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -11,23 +11,29 @@ #include -#ifndef __ASSEMBLY__ - -/* Page Upper Directory not used in RISC-V */ -#include -#include -#include -#include - -#ifdef CONFIG_MMU +#ifndef CONFIG_MMU +#define KERNEL_VIRT_ADDR PAGE_OFFSET +#define KERNEL_LINK_ADDR PAGE_OFFSET +#else +/* + * Leave 2GB for modules and BPF that must lie within a 2GB range around + * the kernel. + */ +#define KERNEL_VIRT_ADDR (VMALLOC_END - SZ_2G + 1) +#define KERNEL_LINK_ADDR KERNEL_VIRT_ADDR #define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) #define VMALLOC_END (PAGE_OFFSET - 1) #define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) #define BPF_JIT_REGION_SIZE (SZ_128M) -#define BPF_JIT_REGION_START (PAGE_OFFSET - BPF_JIT_REGION_SIZE) -#define BPF_JIT_REGION_END (VMALLOC_END) +#define BPF_JIT_REGION_START PFN_ALIGN((unsigned long)&_end) +#define BPF_JIT_REGION_END (BPF_JIT_REGION_START + BPF_JIT_REGION_SIZE) + +#ifdef CONFIG_64BIT +#define VMALLOC_MODULE_START BPF_JIT_REGION_END +#define VMALLOC_MODULE_END (((unsigned long)&_start & PAGE_MASK) + SZ_2G) +#endif /* * Roughly size the vmemmap space to be large enough to fit enough @@ -57,9 +63,16 @@ #define FIXADDR_SIZE PGDIR_SIZE #endif #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) - #endif +#ifndef __ASSEMBLY__ + +/* Page Upper Directory not used in RISC-V */ +#include +#include +#include +#include + #ifdef CONFIG_64BIT #include #else @@ -483,6 +496,7 @@ static inline void __kernel_map_pages(struct page *page, int numpages, int enabl #define kern_addr_valid(addr) (1) /* FIXME */ +extern char _start[]; extern void *dtb_early_va; void setup_bootmem(void); void paging_init(void); diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index 98a406474e7d..8f5bb7731327 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -49,7 +49,8 @@ ENTRY(_start) #ifdef CONFIG_MMU relocate: /* Relocate return address */ - li a1, PAGE_OFFSET + la a1, kernel_virt_addr + REG_L a1, 0(a1) la a2, _start sub a1, a1, a2 add ra, ra, a1 diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c index 8bbe5dbe1341..1a8fbe05accf 100644 --- a/arch/riscv/kernel/module.c +++ b/arch/riscv/kernel/module.c @@ -392,12 +392,10 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, } #if defined(CONFIG_MMU) && defined(CONFIG_64BIT) -#define VMALLOC_MODULE_START \ - max(PFN_ALIGN((unsigned long)&_end - SZ_2G), VMALLOC_START) void *module_alloc(unsigned long size) { return __vmalloc_node_range(size, 1, VMALLOC_MODULE_START, - VMALLOC_END, GFP_KERNEL, + VMALLOC_MODULE_END, GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, __builtin_return_address(0)); } diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S index 0339b6bbe11a..a9abde62909f 100644 --- a/arch/riscv/kernel/vmlinux.lds.S +++ b/arch/riscv/kernel/vmlinux.lds.S @@ -4,7 +4,8 @@ * Copyright (C) 2017 SiFive */ -#define LOAD_OFFSET PAGE_OFFSET +#include +#define LOAD_OFFSET KERNEL_LINK_ADDR #include #include #include diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 736de6c8739f..37be2eb45e58 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -22,6 +22,9 @@ #include "../kernel/head.h" +unsigned long kernel_virt_addr = KERNEL_VIRT_ADDR; +EXPORT_SYMBOL(kernel_virt_addr); + unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] __page_aligned_bss; EXPORT_SYMBOL(empty_zero_page); @@ -178,8 +181,12 @@ void __init setup_bootmem(void) } #ifdef CONFIG_MMU +/* Offset between linear mapping virtual address and kernel load address */ unsigned long va_pa_offset; EXPORT_SYMBOL(va_pa_offset); +/* Offset between kernel mapping virtual address and kernel load address */ +unsigned long va_kernel_pa_offset; +EXPORT_SYMBOL(va_kernel_pa_offset); unsigned long pfn_base; EXPORT_SYMBOL(pfn_base); @@ -271,7 +278,7 @@ static phys_addr_t __init alloc_pmd(uintptr_t va) if (mmu_enabled) return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE); - pmd_num = (va - PAGE_OFFSET) >> PGDIR_SHIFT; + pmd_num = (va - kernel_virt_addr) >> PGDIR_SHIFT; BUG_ON(pmd_num >= NUM_EARLY_PMDS); return (uintptr_t)&early_pmd[pmd_num * PTRS_PER_PMD]; } @@ -372,14 +379,30 @@ static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) #error "setup_vm() is called from head.S before relocate so it should not use absolute addressing." #endif +static uintptr_t load_pa, load_sz; + +void create_kernel_page_table(pgd_t *pgdir, uintptr_t map_size) +{ + uintptr_t va, end_va; + + end_va = kernel_virt_addr + load_sz; + for (va = kernel_virt_addr; va < end_va; va += map_size) + create_pgd_mapping(pgdir, va, + load_pa + (va - kernel_virt_addr), + map_size, PAGE_KERNEL_EXEC); +} + asmlinkage void __init setup_vm(uintptr_t dtb_pa) { uintptr_t va, end_va; - uintptr_t load_pa = (uintptr_t)(&_start); - uintptr_t load_sz = (uintptr_t)(&_end) - load_pa; uintptr_t map_size = best_map_size(load_pa, MAX_EARLY_MAPPING_SIZE); + load_pa = (uintptr_t)(&_start); + load_sz = (uintptr_t)(&_end) - load_pa; + va_pa_offset = PAGE_OFFSET - load_pa; + va_kernel_pa_offset = kernel_virt_addr - load_pa; + pfn_base = PFN_DOWN(load_pa); /* @@ -402,26 +425,22 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) create_pmd_mapping(fixmap_pmd, FIXADDR_START, (uintptr_t)fixmap_pte, PMD_SIZE, PAGE_TABLE); /* Setup trampoline PGD and PMD */ - create_pgd_mapping(trampoline_pg_dir, PAGE_OFFSET, + create_pgd_mapping(trampoline_pg_dir, kernel_virt_addr, (uintptr_t)trampoline_pmd, PGDIR_SIZE, PAGE_TABLE); - create_pmd_mapping(trampoline_pmd, PAGE_OFFSET, + create_pmd_mapping(trampoline_pmd, kernel_virt_addr, load_pa, PMD_SIZE, PAGE_KERNEL_EXEC); #else /* Setup trampoline PGD */ - create_pgd_mapping(trampoline_pg_dir, PAGE_OFFSET, + create_pgd_mapping(trampoline_pg_dir, kernel_virt_addr, load_pa, PGDIR_SIZE, PAGE_KERNEL_EXEC); #endif /* - * Setup early PGD covering entire kernel which will allows + * Setup early PGD covering entire kernel which will allow * us to reach paging_init(). We map all memory banks later * in setup_vm_final() below. */ - end_va = PAGE_OFFSET + load_sz; - for (va = PAGE_OFFSET; va < end_va; va += map_size) - create_pgd_mapping(early_pg_dir, va, - load_pa + (va - PAGE_OFFSET), - map_size, PAGE_KERNEL_EXEC); + create_kernel_page_table(early_pg_dir, map_size); /* Create fixed mapping for early FDT parsing */ end_va = __fix_to_virt(FIX_FDT) + FIX_FDT_SIZE; @@ -441,6 +460,7 @@ static void __init setup_vm_final(void) uintptr_t va, map_size; phys_addr_t pa, start, end; struct memblock_region *reg; + static struct vm_struct vm_kernel = { 0 }; /* Set mmu_enabled flag */ mmu_enabled = true; @@ -467,10 +487,22 @@ static void __init setup_vm_final(void) for (pa = start; pa < end; pa += map_size) { va = (uintptr_t)__va(pa); create_pgd_mapping(swapper_pg_dir, va, pa, - map_size, PAGE_KERNEL_EXEC); + map_size, PAGE_KERNEL); } } + /* Map the kernel */ + create_kernel_page_table(swapper_pg_dir, PMD_SIZE); + + /* Reserve the vmalloc area occupied by the kernel */ + vm_kernel.addr = (void *)kernel_virt_addr; + vm_kernel.phys_addr = load_pa; + vm_kernel.size = (load_sz + PMD_SIZE - 1) & ~(PMD_SIZE - 1); + vm_kernel.flags = VM_MAP | VM_NO_GUARD; + vm_kernel.caller = __builtin_return_address(0); + + vm_area_add_early(&vm_kernel); + /* Clear fixmap PTE and PMD mappings */ clear_fixmap(FIX_PTE); clear_fixmap(FIX_PMD); diff --git a/arch/riscv/mm/physaddr.c b/arch/riscv/mm/physaddr.c index e8e4dcd39fed..35703d5ef5fd 100644 --- a/arch/riscv/mm/physaddr.c +++ b/arch/riscv/mm/physaddr.c @@ -23,7 +23,7 @@ EXPORT_SYMBOL(__virt_to_phys); phys_addr_t __phys_addr_symbol(unsigned long x) { - unsigned long kernel_start = (unsigned long)PAGE_OFFSET; + unsigned long kernel_start = (unsigned long)kernel_virt_addr; unsigned long kernel_end = (unsigned long)_end; /* From patchwork Wed Jun 3 08:00:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 11585197 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15CBE739 for ; Wed, 3 Jun 2020 08:02:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8636206C3 for ; Wed, 3 Jun 2020 08:02:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="K5FG8jrO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8636206C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=29+8z/xa7h9hmzayif50gTtiY4Uf3e6o2dXUVmzQCUg=; b=K5FG8jrOBcZBzt DZ4RMcoMaMlklJAvxhAHJ2On4f8FfvblublbP6yOB59Wzdfz5g4PVXVca0HSk2wgaqBjtO6KUMOuO U13++NlHHj2q8d3deHwdI9ZK1AM9m1J+tjcui5JlWbUxRs0G7zskZqoSRN9orHy5wzlpS1dmCCVEm iJQjNK8GPDrt3OPJnz/tmoFYHrL6QhQa1Y5ZaqV5SvVkigvFTtsEI/9CAxE5UK/1ABbGHE+7hL9aO Sjh5YO8/adub0SubpwmxDv9mJxlElf+KqUOnshYn9aaXjr4fuFdDagJ8XkpbMzgcWcOT1wl74zx0O EyQQOa3rBT20A/dds3Bw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgOMD-0008QR-UH; Wed, 03 Jun 2020 08:02:41 +0000 Received: from relay12.mail.gandi.net ([217.70.178.232]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgOMA-0008Pt-ON for linux-riscv@lists.infradead.org; Wed, 03 Jun 2020 08:02:40 +0000 Received: from debian.home (lfbn-gre-1-325-105.w90-112.abo.wanadoo.fr [90.112.45.105]) (Authenticated sender: alex@ghiti.fr) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 05B7E20000E; Wed, 3 Jun 2020 08:02:28 +0000 (UTC) From: Alexandre Ghiti To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Atish Patra , Zong Li , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org Subject: [PATCH v4 2/4] riscv: Introduce CONFIG_RELOCATABLE Date: Wed, 3 Jun 2020 04:00:08 -0400 Message-Id: <20200603080010.13366-3-alex@ghiti.fr> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200603080010.13366-1-alex@ghiti.fr> References: <20200603080010.13366-1-alex@ghiti.fr> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200603_010239_068222_61A21261 X-CRM114-Status: GOOD ( 16.81 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.178.232 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.178.232 listed in wl.mailspike.net] 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , Alexandre Ghiti Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org This config allows to compile the kernel as PIE and to relocate it at any virtual address at runtime: this paves the way to KASLR and to 4-level page table folding at runtime. Runtime relocation is possible since relocation metadata are embedded into the kernel. Note that relocating at runtime introduces an overhead even if the kernel is loaded at the same address it was linked at and that the compiler options are those used in arm64 which uses the same RELA relocation format. Signed-off-by: Alexandre Ghiti Reviewed-by: Zong Li Reviewed-by: Anup Patel --- arch/riscv/Kconfig | 12 +++++++ arch/riscv/Makefile | 5 ++- arch/riscv/kernel/vmlinux.lds.S | 6 ++-- arch/riscv/mm/Makefile | 4 +++ arch/riscv/mm/init.c | 63 +++++++++++++++++++++++++++++++++ 5 files changed, 87 insertions(+), 3 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index a31e1a41913a..93127d5913fe 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -170,6 +170,18 @@ config PGTABLE_LEVELS default 3 if 64BIT default 2 +config RELOCATABLE + bool + depends on MMU + help + This builds a kernel as a Position Independent Executable (PIE), + which retains all relocation metadata required to relocate the + kernel binary at runtime to a different virtual address than the + address it was linked at. + Since RISCV uses the RELA relocation format, this requires a + relocation pass at runtime even if the kernel is loaded at the + same address it was linked at. + source "arch/riscv/Kconfig.socs" menu "Platform type" diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index fb6e37db836d..1406416ea743 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -9,7 +9,10 @@ # OBJCOPYFLAGS := -O binary -LDFLAGS_vmlinux := +ifeq ($(CONFIG_RELOCATABLE),y) +LDFLAGS_vmlinux := -shared -Bsymbolic -z notext -z norelro +KBUILD_CFLAGS += -fPIE +endif ifeq ($(CONFIG_DYNAMIC_FTRACE),y) LDFLAGS_vmlinux := --no-relax endif diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S index a9abde62909f..e8ffba8c2044 100644 --- a/arch/riscv/kernel/vmlinux.lds.S +++ b/arch/riscv/kernel/vmlinux.lds.S @@ -85,8 +85,10 @@ SECTIONS BSS_SECTION(PAGE_SIZE, PAGE_SIZE, 0) - .rel.dyn : { - *(.rel.dyn*) + .rela.dyn : ALIGN(8) { + __rela_dyn_start = .; + *(.rela .rela*) + __rela_dyn_end = .; } _end = .; diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 363ef01c30b1..dc5cdaa80bc1 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -1,6 +1,10 @@ # SPDX-License-Identifier: GPL-2.0-only CFLAGS_init.o := -mcmodel=medany +ifdef CONFIG_RELOCATABLE +CFLAGS_init.o += -fno-pie +endif + ifdef CONFIG_FTRACE CFLAGS_REMOVE_init.o = -pg endif diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 37be2eb45e58..e63ea5b6b6cf 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -13,6 +13,9 @@ #include #include #include +#ifdef CONFIG_RELOCATABLE +#include +#endif #include #include @@ -379,6 +382,53 @@ static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) #error "setup_vm() is called from head.S before relocate so it should not use absolute addressing." #endif +#ifdef CONFIG_RELOCATABLE +extern unsigned long __rela_dyn_start, __rela_dyn_end; + +#ifdef CONFIG_64BIT +#define Elf_Rela Elf64_Rela +#define Elf_Addr Elf64_Addr +#else +#define Elf_Rela Elf32_Rela +#define Elf_Addr Elf32_Addr +#endif + +void __init relocate_kernel(uintptr_t load_pa) +{ + Elf_Rela *rela = (Elf_Rela *)&__rela_dyn_start; + /* + * This holds the offset between the linked virtual address and the + * relocated virtual address. + */ + uintptr_t reloc_offset = kernel_virt_addr - KERNEL_LINK_ADDR; + /* + * This holds the offset between kernel linked virtual address and + * physical address. + */ + uintptr_t va_kernel_link_pa_offset = KERNEL_LINK_ADDR - load_pa; + + for ( ; rela < (Elf_Rela *)&__rela_dyn_end; rela++) { + Elf_Addr addr = (rela->r_offset - va_kernel_link_pa_offset); + Elf_Addr relocated_addr = rela->r_addend; + + if (rela->r_info != R_RISCV_RELATIVE) + continue; + + /* + * Make sure to not relocate vdso symbols like rt_sigreturn + * which are linked from the address 0 in vmlinux since + * vdso symbol addresses are actually used as an offset from + * mm->context.vdso in VDSO_OFFSET macro. + */ + if (relocated_addr >= KERNEL_LINK_ADDR) + relocated_addr += reloc_offset; + + *(Elf_Addr *)addr = relocated_addr; + } +} + +#endif + static uintptr_t load_pa, load_sz; void create_kernel_page_table(pgd_t *pgdir, uintptr_t map_size) @@ -405,6 +455,19 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) pfn_base = PFN_DOWN(load_pa); +#ifdef CONFIG_RELOCATABLE +#ifdef CONFIG_64BIT + /* + * Early page table uses only one PGDIR, which makes it possible + * to map PGDIR_SIZE aligned on PGDIR_SIZE: if the relocation offset + * makes the kernel cross over a PGDIR_SIZE boundary, raise a bug + * since a part of the kernel would not get mapped. + * This cannot happen on rv32 as we use the entire page directory level. + */ + BUG_ON(PGDIR_SIZE - (kernel_virt_addr & (PGDIR_SIZE - 1)) < load_sz); +#endif + relocate_kernel(load_pa); +#endif /* * Enforce boot alignment requirements of RV32 and * RV64 by only allowing PMD or PGD mappings. From patchwork Wed Jun 3 08:00:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 11585199 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 465161667 for ; Wed, 3 Jun 2020 08:03:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 248DC207DF for ; Wed, 3 Jun 2020 08:03:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="HnerWoqk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 248DC207DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5xBowyd+D0SkmJ1xr5wrgpLBpQeKlH46Z8dJHQImyRQ=; b=HnerWoqkYpTVE2 d2awVeF8YUIFWdXPomqQlO2a5dsTOmRfCznnmXILMdAhe0m/oi2/Hjeb9dibzwbz0rNPA9JnMzYF7 RQHIoYnUNuWWyqiYnsYjgCroazlRdv7eIwnFchCgPxJkorky8UvRBHMWFb/+Ns8vOnMFcj8Ah2U28 0cCG/EMsiqetjFibThSX5NHJafdxlBseJMl6Hj9XbsNymSgaBfb7UQynl7Z6RsmrDdRur7d8yfIsR eXLwCOIHTba685Z6ysg/wvKCjuEXVb1vlXpX6ofpqvSJhezOVVdkTQm7Ant0n/DR/Kt9W3zaAB47u 2Ob/2rPuusSdu5qnEYLA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgONE-00007J-Ua; Wed, 03 Jun 2020 08:03:44 +0000 Received: from relay7-d.mail.gandi.net ([217.70.183.200]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgONB-00006Q-At for linux-riscv@lists.infradead.org; Wed, 03 Jun 2020 08:03:43 +0000 X-Originating-IP: 90.112.45.105 Received: from debian.home (lfbn-gre-1-325-105.w90-112.abo.wanadoo.fr [90.112.45.105]) (Authenticated sender: alex@ghiti.fr) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id A6CDC2000B; Wed, 3 Jun 2020 08:03:36 +0000 (UTC) From: Alexandre Ghiti To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Atish Patra , Zong Li , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org Subject: [PATCH v4 3/4] powerpc: Move script to check relocations at compile time in scripts/ Date: Wed, 3 Jun 2020 04:00:09 -0400 Message-Id: <20200603080010.13366-4-alex@ghiti.fr> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200603080010.13366-1-alex@ghiti.fr> References: <20200603080010.13366-1-alex@ghiti.fr> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200603_010341_513667_BA4B7DE7 X-CRM114-Status: GOOD ( 10.22 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.200 listed in wl.mailspike.net] X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , Alexandre Ghiti Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Relocating kernel at runtime is done very early in the boot process, so it is not convenient to check for relocations there and react in case a relocation was not expected. Powerpc architecture has a script that allows to check at compile time for such unexpected relocations: extract the common logic to scripts/ so that other architectures can take advantage of it. Signed-off-by: Alexandre Ghiti Reviewed-by: Anup Patel --- arch/powerpc/tools/relocs_check.sh | 18 ++---------------- scripts/relocs_check.sh | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+), 16 deletions(-) create mode 100755 scripts/relocs_check.sh diff --git a/arch/powerpc/tools/relocs_check.sh b/arch/powerpc/tools/relocs_check.sh index 014e00e74d2b..e367895941ae 100755 --- a/arch/powerpc/tools/relocs_check.sh +++ b/arch/powerpc/tools/relocs_check.sh @@ -15,21 +15,8 @@ if [ $# -lt 3 ]; then exit 1 fi -# Have Kbuild supply the path to objdump and nm so we handle cross compilation. -objdump="$1" -nm="$2" -vmlinux="$3" - -# Remove from the bad relocations those that match an undefined weak symbol -# which will result in an absolute relocation to 0. -# Weak unresolved symbols are of that form in nm output: -# " w _binary__btf_vmlinux_bin_end" -undef_weak_symbols=$($nm "$vmlinux" | awk '$1 ~ /w/ { print $2 }') - bad_relocs=$( -$objdump -R "$vmlinux" | - # Only look at relocation lines. - grep -E '\ X-Patchwork-Id: 11585201 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B52B739 for ; Wed, 3 Jun 2020 08:04:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 29C76206C3 for ; Wed, 3 Jun 2020 08:04:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="baJdBm+c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 29C76206C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rkEkw5QqPkETs3jN0pum2lLGzixVLfiVa01AvyrcEcA=; b=baJdBm+cim88Bd uO33+5/dESI9ySIPSqfz+wysw4u2VL0MMQKJbkwfWiHNC0Ca4WHl5eiTxDTSWhzZUnT6MdKUdQchP nD5YgGZDPFgRsYx4KD2fq+daL8Zqd5jQ8Mz2HesM9wI1N+Bfl0MGPy9weK1XXAXqye66VW46RibmI 0NXffxqKzXdWLbOMLuRS3XwnRuvUH74IeqKrk41yQGH31/tlnRdOXr+m2D6xekg87igWxW7gwHfqR ehPNfyYf3iK8mOoKmkaE2Uc2BPGxZ9BUzmzGcx+8vGxSZ38Ko4/M3jOh+qpFy/setysciwYb1+m33 PJ+bFbYGp4FBKajaz07Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgOOF-0000bw-S0; Wed, 03 Jun 2020 08:04:47 +0000 Received: from relay1-d.mail.gandi.net ([217.70.183.193]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgOOC-0000VD-ST for linux-riscv@lists.infradead.org; Wed, 03 Jun 2020 08:04:46 +0000 X-Originating-IP: 90.112.45.105 Received: from debian.home (lfbn-gre-1-325-105.w90-112.abo.wanadoo.fr [90.112.45.105]) (Authenticated sender: alex@ghiti.fr) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 18F37240005; Wed, 3 Jun 2020 08:04:39 +0000 (UTC) From: Alexandre Ghiti To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Atish Patra , Zong Li , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org Subject: [PATCH v4 4/4] riscv: Check relocations at compile time Date: Wed, 3 Jun 2020 04:00:10 -0400 Message-Id: <20200603080010.13366-5-alex@ghiti.fr> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200603080010.13366-1-alex@ghiti.fr> References: <20200603080010.13366-1-alex@ghiti.fr> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200603_010445_054537_510D3047 X-CRM114-Status: UNSURE ( 9.74 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.193 listed in list.dnswl.org] 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.193 listed in wl.mailspike.net] 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , Alexandre Ghiti Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Relocating kernel at runtime is done very early in the boot process, so it is not convenient to check for relocations there and react in case a relocation was not expected. There exists a script in scripts/ that extracts the relocations from vmlinux that is then used at postlink to check the relocations. Signed-off-by: Alexandre Ghiti Reviewed-by: Anup Patel --- arch/riscv/Makefile.postlink | 36 ++++++++++++++++++++++++++++++++ arch/riscv/tools/relocs_check.sh | 26 +++++++++++++++++++++++ 2 files changed, 62 insertions(+) create mode 100644 arch/riscv/Makefile.postlink create mode 100755 arch/riscv/tools/relocs_check.sh diff --git a/arch/riscv/Makefile.postlink b/arch/riscv/Makefile.postlink new file mode 100644 index 000000000000..bf2b2bca1845 --- /dev/null +++ b/arch/riscv/Makefile.postlink @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: GPL-2.0 +# =========================================================================== +# Post-link riscv pass +# =========================================================================== +# +# Check that vmlinux relocations look sane + +PHONY := __archpost +__archpost: + +-include include/config/auto.conf +include scripts/Kbuild.include + +quiet_cmd_relocs_check = CHKREL $@ +cmd_relocs_check = \ + $(CONFIG_SHELL) $(srctree)/arch/riscv/tools/relocs_check.sh "$(OBJDUMP)" "$(NM)" "$@" + +# `@true` prevents complaint when there is nothing to be done + +vmlinux: FORCE + @true +ifdef CONFIG_RELOCATABLE + $(call if_changed,relocs_check) +endif + +%.ko: FORCE + @true + +clean: + @true + +PHONY += FORCE clean + +FORCE: + +.PHONY: $(PHONY) diff --git a/arch/riscv/tools/relocs_check.sh b/arch/riscv/tools/relocs_check.sh new file mode 100755 index 000000000000..baeb2e7b2290 --- /dev/null +++ b/arch/riscv/tools/relocs_check.sh @@ -0,0 +1,26 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Based on powerpc relocs_check.sh + +# This script checks the relocations of a vmlinux for "suspicious" +# relocations. + +if [ $# -lt 3 ]; then + echo "$0 [path to objdump] [path to nm] [path to vmlinux]" 1>&2 + exit 1 +fi + +bad_relocs=$( +${srctree}/scripts/relocs_check.sh "$@" | + # These relocations are okay + # R_RISCV_RELATIVE + grep -F -w -v 'R_RISCV_RELATIVE' +) + +if [ -z "$bad_relocs" ]; then + exit 0 +fi + +num_bad=$(echo "$bad_relocs" | wc -l) +echo "WARNING: $num_bad bad relocations" +echo "$bad_relocs"