From patchwork Wed Feb 5 22:39:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 76C9213A4 for ; Wed, 5 Feb 2020 22:40:05 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id DBCB9217BA for ; Wed, 5 Feb 2020 22:40:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBCB9217BA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17671-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 9436 invoked by uid 550); 5 Feb 2020 22:39:49 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9276 invoked from network); 5 Feb 2020 22:39:48 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092435" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 01/11] modpost: Support >64K sections Date: Wed, 5 Feb 2020 14:39:40 -0800 Message-Id: <20200205223950.1212394-2-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 According to the ELF specification, if the value of st_shndx contains SH_XINDEX, the actual section header index is too large to fit in the st_shndx field and you should use the value out of the SHT_SYMTAB_SHNDX section instead. This table was already being parsed and saved into symtab_shndx_start, however it was not being used, causing segfaults when the number of sections is greater than 64K. Check the st_shndx field for SHN_XINDEX prior to using. Signed-off-by: Kristen Carlson Accardi Reviewed-by: Kees Cook --- scripts/mod/modpost.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 6e892c93d104..5ce7e9dc2f04 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -305,9 +305,23 @@ static const char *sec_name(struct elf_info *elf, int secindex) return sech_name(elf, &elf->sechdrs[secindex]); } +static int sym_index(const Elf_Sym *sym, const struct elf_info *info) +{ + unsigned long offset; + int index; + + if (sym->st_shndx != SHN_XINDEX) + return sym->st_shndx; + + offset = (unsigned long)sym - (unsigned long)info->symtab_start; + index = offset/(sizeof(*sym)); + + return TO_NATIVE(info->symtab_shndx_start[index]); +} + static void *sym_get_data(const struct elf_info *info, const Elf_Sym *sym) { - Elf_Shdr *sechdr = &info->sechdrs[sym->st_shndx]; + Elf_Shdr *sechdr = &info->sechdrs[sym_index(sym, info)]; unsigned long offset; offset = sym->st_value; From patchwork Wed Feb 5 22:39:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367271 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A903713A4 for ; Wed, 5 Feb 2020 22:40:11 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 1AB7F217BA for ; Wed, 5 Feb 2020 22:40:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1AB7F217BA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17672-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 9591 invoked by uid 550); 5 Feb 2020 22:39:50 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9437 invoked from network); 5 Feb 2020 22:39:49 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092438" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 02/11] x86: tools/relocs: Support >64K section headers Date: Wed, 5 Feb 2020 14:39:41 -0800 Message-Id: <20200205223950.1212394-3-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 While it is already supported to find the total number of section headers if we exceed 64K sections, we need to support the extended symbol table to get section header indexes for symbols when there are > 64K sections. Parse the elf file to read the extended symbol table info, and then replace all direct references to st_shndx with calls to sym_index(), which will determine whether we can read the value directly or whether we need to pull it out of the extended table. Signed-off-by: Kristen Carlson Accardi Reviewed-by: Kees Cook --- arch/x86/tools/relocs.c | 104 ++++++++++++++++++++++++++++++---------- 1 file changed, 78 insertions(+), 26 deletions(-) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index ce7188cbdae5..a00dc133f109 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -14,6 +14,10 @@ static Elf_Ehdr ehdr; static unsigned long shnum; static unsigned int shstrndx; +static unsigned int shsymtabndx; +static unsigned int shxsymtabndx; + +static int sym_index(Elf_Sym *sym); struct relocs { uint32_t *offset; @@ -32,6 +36,7 @@ struct section { Elf_Shdr shdr; struct section *link; Elf_Sym *symtab; + Elf32_Word *xsymtab; Elf_Rel *reltab; char *strtab; }; @@ -265,7 +270,7 @@ static const char *sym_name(const char *sym_strtab, Elf_Sym *sym) name = sym_strtab + sym->st_name; } else { - name = sec_name(sym->st_shndx); + name = sec_name(sym_index(sym)); } return name; } @@ -335,6 +340,23 @@ static uint64_t elf64_to_cpu(uint64_t val) #define elf_xword_to_cpu(x) elf32_to_cpu(x) #endif +static int sym_index(Elf_Sym *sym) +{ + Elf_Sym *symtab = secs[shsymtabndx].symtab; + Elf32_Word *xsymtab = secs[shxsymtabndx].xsymtab; + unsigned long offset; + int index; + + if (sym->st_shndx != SHN_XINDEX) + return sym->st_shndx; + + /* calculate offset of sym from head of table. */ + offset = (unsigned long) sym - (unsigned long) symtab; + index = offset/sizeof(*sym); + + return elf32_to_cpu(xsymtab[index]); +} + static void read_ehdr(FILE *fp) { if (fread(&ehdr, sizeof(ehdr), 1, fp) != 1) { @@ -468,31 +490,60 @@ static void read_strtabs(FILE *fp) static void read_symtabs(FILE *fp) { int i,j; + for (i = 0; i < shnum; i++) { struct section *sec = &secs[i]; - if (sec->shdr.sh_type != SHT_SYMTAB) { + int num_syms; + + switch (sec->shdr.sh_type) { + case SHT_SYMTAB_SHNDX: + sec->xsymtab = malloc(sec->shdr.sh_size); + if (!sec->xsymtab) { + die("malloc of %d bytes for xsymtab failed\n", + sec->shdr.sh_size); + } + if (fseek(fp, sec->shdr.sh_offset, SEEK_SET) < 0) { + die("Seek to %d failed: %s\n", + sec->shdr.sh_offset, strerror(errno)); + } + if (fread(sec->xsymtab, 1, sec->shdr.sh_size, fp) + != sec->shdr.sh_size) { + die("Cannot read extended symbol table: %s\n", + strerror(errno)); + } + shxsymtabndx = i; + continue; + + case SHT_SYMTAB: + num_syms = sec->shdr.sh_size/sizeof(Elf_Sym); + + sec->symtab = malloc(sec->shdr.sh_size); + if (!sec->symtab) { + die("malloc of %d bytes for symtab failed\n", + sec->shdr.sh_size); + } + if (fseek(fp, sec->shdr.sh_offset, SEEK_SET) < 0) { + die("Seek to %d failed: %s\n", + sec->shdr.sh_offset, strerror(errno)); + } + if (fread(sec->symtab, 1, sec->shdr.sh_size, fp) + != sec->shdr.sh_size) { + die("Cannot read symbol table: %s\n", + strerror(errno)); + } + for (j = 0; j < num_syms; j++) { + Elf_Sym *sym = &sec->symtab[j]; + + sym->st_name = elf_word_to_cpu(sym->st_name); + sym->st_value = elf_addr_to_cpu(sym->st_value); + sym->st_size = elf_xword_to_cpu(sym->st_size); + sym->st_shndx = elf_half_to_cpu(sym->st_shndx); + } + shsymtabndx = i; + continue; + + default: continue; - } - sec->symtab = malloc(sec->shdr.sh_size); - if (!sec->symtab) { - die("malloc of %d bytes for symtab failed\n", - sec->shdr.sh_size); - } - if (fseek(fp, sec->shdr.sh_offset, SEEK_SET) < 0) { - die("Seek to %d failed: %s\n", - sec->shdr.sh_offset, strerror(errno)); - } - if (fread(sec->symtab, 1, sec->shdr.sh_size, fp) - != sec->shdr.sh_size) { - die("Cannot read symbol table: %s\n", - strerror(errno)); - } - for (j = 0; j < sec->shdr.sh_size/sizeof(Elf_Sym); j++) { - Elf_Sym *sym = &sec->symtab[j]; - sym->st_name = elf_word_to_cpu(sym->st_name); - sym->st_value = elf_addr_to_cpu(sym->st_value); - sym->st_size = elf_xword_to_cpu(sym->st_size); - sym->st_shndx = elf_half_to_cpu(sym->st_shndx); } } } @@ -759,13 +810,14 @@ static void percpu_init(void) */ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) { - return (sym->st_shndx == per_cpu_shndx) && + int shndx = sym_index(sym); + + return (shndx == per_cpu_shndx) && strcmp(symname, "__init_begin") && strcmp(symname, "__per_cpu_load") && strncmp(symname, "init_per_cpu_", 13); } - static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, const char *symname) { @@ -1088,7 +1140,7 @@ static int do_reloc_info(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, sec_name(sec->shdr.sh_info), rel_type(ELF_R_TYPE(rel->r_info)), symname, - sec_name(sym->st_shndx)); + sec_name(sym_index(sym))); return 0; } From patchwork Wed Feb 5 22:39:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED618924 for ; Wed, 5 Feb 2020 22:40:18 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 5ED2B214AF for ; Wed, 5 Feb 2020 22:40:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5ED2B214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17673-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 9749 invoked by uid 550); 5 Feb 2020 22:39:52 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9592 invoked from network); 5 Feb 2020 22:39:51 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092441" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 03/11] x86/boot: Allow a "silent" kaslr random byte fetch Date: Wed, 5 Feb 2020 14:39:42 -0800 Message-Id: <20200205223950.1212394-4-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 From: Kees Cook Under earlyprintk, each RNG call produces a debug report line. When shuffling hundreds of functions, this is not useful information (each line is identical and tells us nothing new). Instead, allow for a NULL "purpose" to suppress the debug reporting. Signed-off-by: Kees Cook Signed-off-by: Kristen Carlson Accardi --- arch/x86/lib/kaslr.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/x86/lib/kaslr.c b/arch/x86/lib/kaslr.c index a53665116458..2b3eb8c948a3 100644 --- a/arch/x86/lib/kaslr.c +++ b/arch/x86/lib/kaslr.c @@ -56,11 +56,14 @@ unsigned long kaslr_get_random_long(const char *purpose) unsigned long raw, random = get_boot_seed(); bool use_i8254 = true; - debug_putstr(purpose); - debug_putstr(" KASLR using"); + if (purpose) { + debug_putstr(purpose); + debug_putstr(" KASLR using"); + } if (has_cpuflag(X86_FEATURE_RDRAND)) { - debug_putstr(" RDRAND"); + if (purpose) + debug_putstr(" RDRAND"); if (rdrand_long(&raw)) { random ^= raw; use_i8254 = false; @@ -68,7 +71,8 @@ unsigned long kaslr_get_random_long(const char *purpose) } if (has_cpuflag(X86_FEATURE_TSC)) { - debug_putstr(" RDTSC"); + if (purpose) + debug_putstr(" RDTSC"); raw = rdtsc(); random ^= raw; @@ -76,7 +80,8 @@ unsigned long kaslr_get_random_long(const char *purpose) } if (use_i8254) { - debug_putstr(" i8254"); + if (purpose) + debug_putstr(" i8254"); random ^= i8254(); } @@ -86,7 +91,8 @@ unsigned long kaslr_get_random_long(const char *purpose) : "a" (random), "rm" (mix_const)); random += raw; - debug_putstr("...\n"); + if (purpose) + debug_putstr("...\n"); return random; } From patchwork Wed Feb 5 22:39:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7408A924 for ; Wed, 5 Feb 2020 22:40:26 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id DB4F6214AF for ; Wed, 5 Feb 2020 22:40:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DB4F6214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17674-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 9830 invoked by uid 550); 5 Feb 2020 22:39:53 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9757 invoked from network); 5 Feb 2020 22:39:52 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092444" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 04/11] x86/boot/KASLR: Introduce PRNG for faster shuffling Date: Wed, 5 Feb 2020 14:39:43 -0800 Message-Id: <20200205223950.1212394-5-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 From: Kees Cook This *might* improve shuffling speed at boot. Possibly only marginally. This has not yet been tested, and would need to have some performance tests run to determine if it helps before merging. (notes from Kristen) - initial performance tests suggest that any improvement is indeed marginal. However, this code is useful for using a known seed. Signed-off-by: Kees Cook Signed-off-by: Kristen Carlson Accardi --- arch/x86/boot/compressed/kaslr.c | 4 +-- arch/x86/include/asm/kaslr.h | 3 +- arch/x86/lib/kaslr.c | 50 +++++++++++++++++++++++++++++++- arch/x86/mm/init.c | 2 +- arch/x86/mm/kaslr.c | 2 +- 5 files changed, 55 insertions(+), 6 deletions(-) diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index d7408af55738..ae4dce76a9bd 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -598,7 +598,7 @@ static unsigned long slots_fetch_random(void) if (slot_max == 0) return 0; - slot = kaslr_get_random_long("Physical") % slot_max; + slot = kaslr_get_random_seed("Physical") % slot_max; for (i = 0; i < slot_area_index; i++) { if (slot >= slot_areas[i].num) { @@ -880,7 +880,7 @@ static unsigned long find_random_virt_addr(unsigned long minimum, slots = (KERNEL_IMAGE_SIZE - minimum - image_size) / CONFIG_PHYSICAL_ALIGN + 1; - random_addr = kaslr_get_random_long("Virtual") % slots; + random_addr = kaslr_get_random_seed("Virtual") % slots; return random_addr * CONFIG_PHYSICAL_ALIGN + minimum; } diff --git a/arch/x86/include/asm/kaslr.h b/arch/x86/include/asm/kaslr.h index db7ba2feb947..47d5b25e5b11 100644 --- a/arch/x86/include/asm/kaslr.h +++ b/arch/x86/include/asm/kaslr.h @@ -2,7 +2,8 @@ #ifndef _ASM_KASLR_H_ #define _ASM_KASLR_H_ -unsigned long kaslr_get_random_long(const char *purpose); +unsigned long kaslr_get_random_seed(const char *purpose); +unsigned long kaslr_get_prandom_long(void); #ifdef CONFIG_RANDOMIZE_MEMORY void kernel_randomize_memory(void); diff --git a/arch/x86/lib/kaslr.c b/arch/x86/lib/kaslr.c index 2b3eb8c948a3..41b5610855a3 100644 --- a/arch/x86/lib/kaslr.c +++ b/arch/x86/lib/kaslr.c @@ -46,7 +46,7 @@ static inline u16 i8254(void) return timer; } -unsigned long kaslr_get_random_long(const char *purpose) +unsigned long kaslr_get_random_seed(const char *purpose) { #ifdef CONFIG_X86_64 const unsigned long mix_const = 0x5d6008cbf3848dd3UL; @@ -96,3 +96,51 @@ unsigned long kaslr_get_random_long(const char *purpose) return random; } + +/* + * 64bit variant of Bob Jenkins' public domain PRNG + * 256 bits of internal state + */ +struct prng_state { + u64 a, b, c, d; +}; + +static struct prng_state state; +static bool initialized; + +#define rot(x, k) (((x)<<(k))|((x)>>(64-(k)))) +static u64 prng_u64(struct prng_state *x) +{ + u64 e; + + e = x->a - rot(x->b, 7); + x->a = x->b ^ rot(x->c, 13); + x->b = x->c + rot(x->d, 37); + x->c = x->d + e; + x->d = e + x->a; + + return x->d; +} + +static void prng_init(struct prng_state *state) +{ + int i; + + state->a = kaslr_get_random_seed(NULL); + state->b = kaslr_get_random_seed(NULL); + state->c = kaslr_get_random_seed(NULL); + state->d = kaslr_get_random_seed(NULL); + + for (i = 0; i < 30; ++i) + (void)prng_u64(state); + + initialized = true; +} + +unsigned long kaslr_get_prandom_long(void) +{ + if (!initialized) + prng_init(&state); + + return prng_u64(&state); +} diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index e7bb483557c9..c974dbab2293 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -722,7 +722,7 @@ void __init poking_init(void) */ poking_addr = TASK_UNMAPPED_BASE; if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) - poking_addr += (kaslr_get_random_long("Poking") & PAGE_MASK) % + poking_addr += (kaslr_get_random_seed("Poking") & PAGE_MASK) % (TASK_SIZE - TASK_UNMAPPED_BASE - 3 * PAGE_SIZE); if (((poking_addr + PAGE_SIZE) & ~PMD_MASK) == 0) diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c index dc6182eecefa..b30bd1db7543 100644 --- a/arch/x86/mm/kaslr.c +++ b/arch/x86/mm/kaslr.c @@ -123,7 +123,7 @@ void __init kernel_randomize_memory(void) for (i = 0; i < ARRAY_SIZE(kaslr_regions); i++) remain_entropy -= get_padding(&kaslr_regions[i]); - prandom_seed_state(&rand_state, kaslr_get_random_long("Memory")); + prandom_seed_state(&rand_state, kaslr_get_random_seed("Memory")); for (i = 0; i < ARRAY_SIZE(kaslr_regions); i++) { unsigned long entropy; From patchwork Wed Feb 5 22:39:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B7CA13A4 for ; Wed, 5 Feb 2020 22:40:35 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id B0CFB214AF for ; Wed, 5 Feb 2020 22:40:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0CFB214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17675-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 9930 invoked by uid 550); 5 Feb 2020 22:39:54 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9832 invoked from network); 5 Feb 2020 22:39:53 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092448" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 05/11] x86: Makefile: Add build and config option for CONFIG_FG_KASLR Date: Wed, 5 Feb 2020 14:39:44 -0800 Message-Id: <20200205223950.1212394-6-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 Allow user to select CONFIG_FG_KASLR if dependencies are met. Change the make file to build with -ffunction-sections if CONFIG_FG_KASLR Signed-off-by: Kristen Carlson Accardi --- Makefile | 4 ++++ arch/x86/Kconfig | 13 +++++++++++++ 2 files changed, 17 insertions(+) diff --git a/Makefile b/Makefile index c50ef91f6136..41438a921666 100644 --- a/Makefile +++ b/Makefile @@ -846,6 +846,10 @@ ifdef CONFIG_LIVEPATCH KBUILD_CFLAGS += $(call cc-option, -flive-patching=inline-clone) endif +ifdef CONFIG_FG_KASLR +KBUILD_CFLAGS += -ffunction-sections +endif + # arch Makefile may override CC so keep this after arch Makefile is included NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5e8949953660..b4b1b17076a8 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2208,6 +2208,19 @@ config RANDOMIZE_BASE If unsure, say Y. +config FG_KASLR + bool "Finer grained Kernel Address Space Layout Randomization" + depends on $(cc-option, -ffunction-sections) + depends on RANDOMIZE_BASE && X86_64 + help + This option improves the randomness of the kernel text + over basic Kernel Address Space Layout Randomization (KASLR) + by reordering the kernel text at boot time. This feature + uses information generated at compile time to re-layout the + kernel text section at boot time at function level granularity. + + If unsure, say N. + # Relocation on x86 needs some additional build support config X86_NEED_RELOCS def_bool y From patchwork Wed Feb 5 22:39:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 035B713A4 for ; Wed, 5 Feb 2020 22:40:44 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 6B0D8214AF for ; Wed, 5 Feb 2020 22:40:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B0D8214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17676-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 9984 invoked by uid 550); 5 Feb 2020 22:39:56 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9931 invoked from network); 5 Feb 2020 22:39:54 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092452" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 06/11] x86: make sure _etext includes function sections Date: Wed, 5 Feb 2020 14:39:45 -0800 Message-Id: <20200205223950.1212394-7-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 We will be using -ffunction-sections to place each function in it's own text section so it can be randomized at load time. The linker considers these .text.* sections "orphaned sections", and will place them after the first similar section (.text). However, we need to move _etext so that it is after both .text and .text.* We also need to calculate text size to include .text AND .text.* Signed-off-by: Kristen Carlson Accardi --- arch/x86/kernel/vmlinux.lds.S | 18 +++++++++++++++++- include/asm-generic/vmlinux.lds.h | 2 +- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 3a1a819da137..e54e9ac5b429 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -146,8 +146,24 @@ SECTIONS #endif } :text =0xcccc - /* End of text section, which should occupy whole number of pages */ +#ifdef CONFIG_FG_KASLR + /* + * -ffunction-sections creates .text.* sections, which are considered + * "orphan sections" and added after the first similar section (.text). + * Adding this ALIGN statement causes the address of _etext + * to be below that of all the .text.* orphaned sections + */ + . = ALIGN(PAGE_SIZE); +#endif _etext = .; + + /* + * the size of the .text section is used to calculate the address + * range for orc lookups. If we just use SIZEOF(.text), we will + * miss all the .text.* sections. Calculate the size using _etext + * and _stext and save the value for later. + */ + text_size = _etext - _stext; . = ALIGN(PAGE_SIZE); X86_ALIGN_RODATA_BEGIN diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index e00f41aa8ec4..edf19f4296e2 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -798,7 +798,7 @@ . = ALIGN(4); \ .orc_lookup : AT(ADDR(.orc_lookup) - LOAD_OFFSET) { \ orc_lookup = .; \ - . += (((SIZEOF(.text) + LOOKUP_BLOCK_SIZE - 1) / \ + . += (((text_size + LOOKUP_BLOCK_SIZE - 1) / \ LOOKUP_BLOCK_SIZE) + 1) * 4; \ orc_lookup_end = .; \ } From patchwork Wed Feb 5 22:39:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52D3F14E3 for ; Wed, 5 Feb 2020 22:40:54 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 8E1DC217BA for ; Wed, 5 Feb 2020 22:40:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E1DC217BA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17677-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 10034 invoked by uid 550); 5 Feb 2020 22:39:57 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 9954 invoked from network); 5 Feb 2020 22:39:55 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092456" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 07/11] x86/tools: Adding relative relocs for randomized functions Date: Wed, 5 Feb 2020 14:39:46 -0800 Message-Id: <20200205223950.1212394-8-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 If we are randomizing function sections, we are going to need to recalculate relative offsets for relocs that are either in the randomized sections, or refer to the randomized sections. Add code to detect whether a reloc satisifies these cases, and if so, add them to the appropriate reloc list. Signed-off-by: Kristen Carlson Accardi Reviewed-by: Kees Cook --- arch/x86/boot/compressed/Makefile | 7 +++++- arch/x86/tools/relocs.c | 41 ++++++++++++++++++++++++++++--- arch/x86/tools/relocs.h | 4 +-- arch/x86/tools/relocs_common.c | 15 +++++++---- 4 files changed, 55 insertions(+), 12 deletions(-) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 1dac210f7d44..b7e5ea757ef4 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -119,6 +119,11 @@ $(obj)/vmlinux: $(vmlinux-objs-y) FORCE $(call if_changed,check-and-link-vmlinux) OBJCOPYFLAGS_vmlinux.bin := -R .comment -S + +ifdef CONFIG_FG_KASLR + RELOCS_ARGS += --fg-kaslr +endif + $(obj)/vmlinux.bin: vmlinux FORCE $(call if_changed,objcopy) @@ -126,7 +131,7 @@ targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relo CMD_RELOCS = arch/x86/tools/relocs quiet_cmd_relocs = RELOCS $@ - cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $< + cmd_relocs = $(CMD_RELOCS) $(RELOCS_ARGS) $< > $@;$(CMD_RELOCS) $(RELOCS_ARGS) --abs-relocs $< $(obj)/vmlinux.relocs: vmlinux FORCE $(call if_changed,relocs) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index a00dc133f109..2641eec5f2de 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -42,6 +42,8 @@ struct section { }; static struct section *secs; +static int fg_kaslr; + static const char * const sym_regex_kernel[S_NSYMTYPES] = { /* * Following symbols have been audited. There values are constant and do @@ -818,6 +820,32 @@ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) strncmp(symname, "init_per_cpu_", 13); } +static int is_function_section(struct section *sec) +{ + const char *name; + + if (!fg_kaslr) + return 0; + + name = sec_name(sec->shdr.sh_info); + + return(!strncmp(name, ".text.", 6)); +} + +static int is_randomized_sym(ElfW(Sym *sym)) +{ + const char *name; + + if (!fg_kaslr) + return 0; + + if (sym->st_shndx > shnum) + return 0; + + name = sec_name(sym_index(sym)); + return(!strncmp(name, ".text.", 6)); +} + static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, const char *symname) { @@ -842,13 +870,17 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, case R_X86_64_PC32: case R_X86_64_PLT32: /* - * PC relative relocations don't need to be adjusted unless - * referencing a percpu symbol. + * we need to keep pc relative relocations for sections which + * might be randomized, and for the percpu section. + * We also need to keep relocations for any offset which might + * reference an address in a section which has been randomized. * * NB: R_X86_64_PLT32 can be treated as R_X86_64_PC32. */ - if (is_percpu_sym(sym, symname)) + if (is_function_section(sec) || is_randomized_sym(sym) || + is_percpu_sym(sym, symname)) add_reloc(&relocs32neg, offset); + break; case R_X86_64_PC64: @@ -1158,8 +1190,9 @@ static void print_reloc_info(void) void process(FILE *fp, int use_real_mode, int as_text, int show_absolute_syms, int show_absolute_relocs, - int show_reloc_info) + int show_reloc_info, int fgkaslr) { + fg_kaslr = fgkaslr; regex_init(use_real_mode); read_ehdr(fp); read_shdrs(fp); diff --git a/arch/x86/tools/relocs.h b/arch/x86/tools/relocs.h index 43c83c0fd22c..05504052c846 100644 --- a/arch/x86/tools/relocs.h +++ b/arch/x86/tools/relocs.h @@ -31,8 +31,8 @@ enum symtype { void process_32(FILE *fp, int use_real_mode, int as_text, int show_absolute_syms, int show_absolute_relocs, - int show_reloc_info); + int show_reloc_info, int fg_kaslr); void process_64(FILE *fp, int use_real_mode, int as_text, int show_absolute_syms, int show_absolute_relocs, - int show_reloc_info); + int show_reloc_info, int fg_kaslr); #endif /* RELOCS_H */ diff --git a/arch/x86/tools/relocs_common.c b/arch/x86/tools/relocs_common.c index 6634352a20bc..f76cd6f53961 100644 --- a/arch/x86/tools/relocs_common.c +++ b/arch/x86/tools/relocs_common.c @@ -12,14 +12,14 @@ void die(char *fmt, ...) static void usage(void) { - die("relocs [--abs-syms|--abs-relocs|--reloc-info|--text|--realmode]" \ - " vmlinux\n"); + die("relocs [--abs-syms|--abs-relocs|--reloc-info|--text|--realmode" + "|--fg-kaslr] vmlinux\n"); } int main(int argc, char **argv) { int show_absolute_syms, show_absolute_relocs, show_reloc_info; - int as_text, use_real_mode; + int as_text, use_real_mode, fg_kaslr; const char *fname; FILE *fp; int i; @@ -30,6 +30,7 @@ int main(int argc, char **argv) show_reloc_info = 0; as_text = 0; use_real_mode = 0; + fg_kaslr = 0; fname = NULL; for (i = 1; i < argc; i++) { char *arg = argv[i]; @@ -54,6 +55,10 @@ int main(int argc, char **argv) use_real_mode = 1; continue; } + if (strcmp(arg, "--fg-kaslr") == 0) { + fg_kaslr = 1; + continue; + } } else if (!fname) { fname = arg; @@ -75,11 +80,11 @@ int main(int argc, char **argv) if (e_ident[EI_CLASS] == ELFCLASS64) process_64(fp, use_real_mode, as_text, show_absolute_syms, show_absolute_relocs, - show_reloc_info); + show_reloc_info, fg_kaslr); else process_32(fp, use_real_mode, as_text, show_absolute_syms, show_absolute_relocs, - show_reloc_info); + show_reloc_info, fg_kaslr); fclose(fp); return 0; } From patchwork Wed Feb 5 22:39:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7006292A for ; Wed, 5 Feb 2020 22:41:06 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 567D2214AF for ; Wed, 5 Feb 2020 22:41:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 567D2214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17678-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 10101 invoked by uid 550); 5 Feb 2020 22:39:59 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 10003 invoked from network); 5 Feb 2020 22:39:56 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092459" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 08/11] x86: Add support for finer grained KASLR Date: Wed, 5 Feb 2020 14:39:47 -0800 Message-Id: <20200205223950.1212394-9-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 At boot time, find all the function sections that have separate .text sections, shuffle them, and then copy them to new locations. Adjust any relocations accordingly. Signed-off-by: Kristen Carlson Accardi --- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/fgkaslr.c | 751 +++++++++++++++++++++++ arch/x86/boot/compressed/misc.c | 106 +++- arch/x86/boot/compressed/misc.h | 26 + arch/x86/boot/compressed/vmlinux.symbols | 15 + arch/x86/include/asm/boot.h | 15 +- arch/x86/include/asm/kaslr.h | 1 + arch/x86/lib/kaslr.c | 15 + scripts/kallsyms.c | 14 +- scripts/link-vmlinux.sh | 4 + 10 files changed, 939 insertions(+), 9 deletions(-) create mode 100644 arch/x86/boot/compressed/fgkaslr.c create mode 100644 arch/x86/boot/compressed/vmlinux.symbols diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index b7e5ea757ef4..60d4c4e59c05 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -122,6 +122,7 @@ OBJCOPYFLAGS_vmlinux.bin := -R .comment -S ifdef CONFIG_FG_KASLR RELOCS_ARGS += --fg-kaslr + OBJCOPYFLAGS += --keep-symbols=$(obj)/vmlinux.symbols endif $(obj)/vmlinux.bin: vmlinux FORCE diff --git a/arch/x86/boot/compressed/fgkaslr.c b/arch/x86/boot/compressed/fgkaslr.c new file mode 100644 index 000000000000..fa4e15488a6e --- /dev/null +++ b/arch/x86/boot/compressed/fgkaslr.c @@ -0,0 +1,751 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * fgkaslr.c + * + * This contains the routines needed to reorder the kernel text section + * at boot time. + */ +#define __DISABLE_EXPORTS +#define _LINUX_KPROBES_H +#define NOKPROBE_SYMBOL(fname) + +#include "misc.h" +#include "error.h" +#include "pgtable.h" +#include "../string.h" +#include "../voffset.h" +#include +#include +#include "../../include/asm/extable.h" + +/* Macros used by the included decompressor code below. */ +#define STATIC static + +/* + * Use normal definitions of mem*() from string.c. There are already + * included header files which expect a definition of memset() and by + * the time we define memset macro, it is too late. + */ +#undef memcpy +#undef memset +#define memzero(s, n) memset((s), 0, (n)) +#define memmove memmove + +/* Functions used by the included decompressor code below. */ +#include +void *memmove(void *dest, const void *src, size_t n); + +static unsigned long percpu_start; +static unsigned long percpu_end; + +static long kallsyms_names; +static long kallsyms_offsets; +static long kallsyms_num_syms; +static long kallsyms_relative_base; +static long kallsyms_markers; +static long __start___ex_table_addr; +static long __stop___ex_table_addr; +static long _stext; +static long _etext; +static long _sinittext; +static long _einittext; +static long fgkaslr_seed; + +/* addresses in mapped address space */ +static int *base; +static u8 *names; +static unsigned long relative_base; +static unsigned int *markers_addr; +static unsigned long long seed[4]; + +struct kallsyms_name { + u8 len; + u8 indecis[256]; +}; +struct kallsyms_name *names_table; + +/* + * This is an array of pointers to sections headers for randomized sections + */ +Elf64_Shdr **sections; + +/* + * This is an array of all section headers, randomized or otherwise. + */ +Elf64_Shdr *sechdrs; + +/* + * The number of elements in the randomized section header array (sections) + */ +int sections_size; + +static void parse_prandom_seed(void) +{ + char optstr[128], *options; + unsigned long long a, b, c, d; + + a = b = c = d = 0; + + /* + * passing the fgkaslr_seed option should only be used for + * debugging since the cmdline is exposed to the user + * via /proc/cmdline. + */ + if (cmdline_find_option("fgkaslr_seed", optstr, sizeof(optstr)) > 0) { + int retval; + + options = optstr; + + a = simple_strtoull(options, &options, 16); + + if (options && options[0] == ',') + b = simple_strtoull(options+1, &options, 16); + if (options && options[0] == ',') + c = simple_strtoull(options+1, &options, 16); + if (options && options[0] == ',') + d = simple_strtoull(options+1, &options, 16); + } + + if (a == 0) + a = kaslr_get_random_seed(NULL); + if (b == 0) + b = kaslr_get_random_seed(NULL); + if (c == 0) + c = kaslr_get_random_seed(NULL); + if (d == 0) + d = kaslr_get_random_seed(NULL); + + prng_init_seed(a, b, c, d); + seed[0] = a; + seed[1] = b; + seed[2] = c; + seed[3] = d; +} + +static bool is_text(long addr) +{ + if ((addr >= _stext && addr < _etext) || + (addr >= _sinittext && addr < _einittext)) + return true; + return false; +} + +bool is_percpu_addr(long pc, long offset) +{ + unsigned long ptr; + long address; + + address = pc + offset + 4; + + ptr = (unsigned long) address; + + if (ptr >= percpu_start && ptr < percpu_end) + return true; + + return false; +} + +static int cmp_section_addr(const void *a, const void *b) +{ + unsigned long ptr = (unsigned long)a; + Elf64_Shdr *s = *(Elf64_Shdr **)b; + unsigned long end = s->sh_addr + s->sh_size; + + if (ptr >= s->sh_addr && ptr < end) + return 0; + + if (ptr < s->sh_addr) + return -1; + + return 1; +} + +/* + * Discover if the address is in a randomized section and if so, adjust + * by the saved offset. + */ +Elf64_Shdr *adjust_address(long *address) +{ + Elf64_Shdr **s; + Elf64_Shdr *shdr; + + if (sections == NULL) { + debug_putstr("\nsections is null\n"); + return NULL; + } + + s = bsearch((const void *)*address, sections, sections_size, sizeof(*s), + cmp_section_addr); + if (s != NULL) { + shdr = *s; + *address += shdr->sh_offset; + return shdr; + } + + return NULL; +} + +void adjust_relative_offset(long pc, long *value, Elf64_Shdr *section) +{ + Elf64_Shdr *s; + long address; + + /* + * sometimes we are updating a relative offset that would + * normally be relative to the next instruction (such as a call). + * In this case to calculate the target, you need to add 32bits to + * the pc to get the next instruction value. However, sometimes + * targets are just data that was stored in a table such as ksymtab + * or cpu alternatives. In this case our target is not relative to + * the next instruction. + */ + + /* + * Calculate the address that this offset would call. + */ + if (!is_text(pc)) + address = pc + *value; + else + address = pc + *value + 4; + + /* + * if the address is in section that was randomized, + * we need to adjust the offset. + */ + s = adjust_address(&address); + if (s != NULL) + *value += s->sh_offset; + + /* + * If the PC that this offset was calculated for was in a section + * that has been randomized, the value needs to be adjusted by the + * same amount as the randomized section was adjusted from it's original + * location. + */ + if (section != NULL) + *value -= section->sh_offset; + +} + +static void kallsyms_swp(void *a, void *b, int size) +{ + int idx1, idx2; + int temp; + struct kallsyms_name name_a; + + /* determine our index into the array */ + idx1 = (int *)a - base; + idx2 = (int *)b - base; + temp = base[idx1]; + base[idx1] = base[idx2]; + base[idx2] = temp; + + /* also swap the names table */ + memcpy(&name_a, &names_table[idx1], sizeof(name_a)); + memcpy(&names_table[idx1], &names_table[idx2], + sizeof(struct kallsyms_name)); + memcpy(&names_table[idx2], &name_a, sizeof(struct kallsyms_name)); +} + +static int kallsyms_cmp(const void *a, const void *b) +{ + int addr_a, addr_b; + unsigned long uaddr_a, uaddr_b; + + addr_a = *(int *)a; + addr_b = *(int *)b; + + if (addr_a >= 0) + uaddr_a = addr_a; + if (addr_b >= 0) + uaddr_b = addr_b; + + if (addr_a < 0) + uaddr_a = relative_base - 1 - addr_a; + if (addr_b < 0) + uaddr_b = relative_base - 1 - addr_b; + + if (uaddr_b > uaddr_a) + return -1; + + return 0; +} + +static void deal_with_names(int num_syms) +{ + int num_bytes; + int i, j; + int offset; + + + /* we should have num_syms kallsyms_name entries */ + num_bytes = num_syms * sizeof(*names_table); + names_table = malloc(num_syms * sizeof(*names_table)); + if (names_table == NULL) { + debug_putstr("\nbytes requested: "); + debug_puthex(num_bytes); + error("\nunable to allocate space for names table\n"); + } + + /* read all the names entries */ + offset = 0; + for (i = 0; i < num_syms; i++) { + names_table[i].len = names[offset]; + offset++; + for (j = 0; j < names_table[i].len; j++) { + names_table[i].indecis[j] = names[offset]; + offset++; + } + } +} + +static void write_sorted_names(int num_syms) +{ + int i, j; + int offset = 0; + unsigned int *markers; + + /* + * we are going to need to regenerate the markers table, which is a + * table of offsets into the compressed stream every 256 symbols. + * this code copied almost directly from scripts/kallsyms.c + */ + markers = malloc(sizeof(unsigned int) * ((num_syms + 255) / 256)); + if (!markers) { + debug_putstr("\nfailed to allocate heap space of "); + debug_puthex(((num_syms + 255) / 256)); + debug_putstr(" bytes\n"); + error("Unable to allocate space for markers table"); + } + + for (i = 0; i < num_syms; i++) { + if ((i & 0xFF) == 0) + markers[i >> 8] = offset; + + names[offset] = (u8) names_table[i].len; + offset++; + for (j = 0; j < names_table[i].len; j++) { + names[offset] = (u8) names_table[i].indecis[j]; + offset++; + } + } + + /* write new markers table over old one */ + for (i = 0; i < ((num_syms + 255) >> 8); i++) + markers_addr[i] = markers[i]; + + free(markers); + free(names_table); +} + +static void sort_kallsyms(unsigned long map) +{ + int num_syms; + int i; + + debug_putstr("\nRe-sorting kallsyms ..."); + + num_syms = *(int *)(kallsyms_num_syms + map); + base = (int *)(kallsyms_offsets + map); + relative_base = *(unsigned long *)(kallsyms_relative_base + map); + markers_addr = (unsigned int *)(kallsyms_markers + map); + names = (u8 *)(kallsyms_names + map); + + /* + * the kallsyms table was generated prior to any randomization. + * it is a bunch of offsets from "relative base". In order for + * us to check if a symbol has an address that was in a randomized + * section, we need to reconstruct the address to it's original + * value prior to handle_relocations. + */ + for (i = 0; i < num_syms; i++) { + unsigned long addr; + int new_base; + + /* + * according to kernel/kallsyms.c, positive offsets are absolute + * values and negative offsets are relative to the base. + * + * TBD: I think we can just continue if positive value + * since that would be in the percpu section. + */ + if (base[i] >= 0) + addr = base[i]; + else + addr = relative_base - 1 - base[i]; + + if (adjust_address(&addr)) { + /* here we need to recalcuate the offset */ + new_base = relative_base - 1 - addr; + base[i] = new_base; + } + } + + /* + * here we need to read in all the kallsyms_names info + * so that we can regenerate it. + */ + deal_with_names(num_syms); + + sort(base, num_syms, sizeof(int), kallsyms_cmp, kallsyms_swp); + + /* write the newly sorted names table over the old one */ + write_sorted_names(num_syms); +} + +#define ARCH_HAS_SEARCH_EXTABLE +#include "../../../../lib/extable.c" + +static inline unsigned long +ex_fixup_handler(const struct exception_table_entry *x) +{ + return ((unsigned long)&x->handler + x->handler); +} + +static inline unsigned long +ex_fixup_addr(const struct exception_table_entry *x) +{ + return ((unsigned long)&x->fixup + x->fixup); +} + +static void update_ex_table(unsigned long map) +{ + struct exception_table_entry *start_ex_table = (struct exception_table_entry *) (__start___ex_table_addr + map); + struct exception_table_entry *stop_ex_table = (struct exception_table_entry *) (__stop___ex_table_addr + map); + int num_entries = ( __stop___ex_table_addr - __start___ex_table_addr ) / sizeof(struct exception_table_entry); + int i; + + debug_putstr("\nUpdating exception table...\n"); + for (i = 0; i < num_entries; i++) { + unsigned long insn = ex_to_insn(&start_ex_table[i]); + unsigned long fixup = ex_fixup_addr(&start_ex_table[i]); + unsigned long handler = ex_fixup_handler(&start_ex_table[i]); + unsigned long addr; + Elf64_Shdr *s; + + /* check each address to see if it needs adjusting */ + addr = insn - map; + s = adjust_address(&addr); + if (s != NULL) + start_ex_table[i].insn += s->sh_offset; + + addr = fixup - map; + s = adjust_address(&addr); + if (s != NULL) + start_ex_table[i].fixup += s->sh_offset; + + addr = handler - map; + s = adjust_address(&addr); + if (s != NULL) + start_ex_table[i].handler += s->sh_offset; + } +} + +static void sort_ex_table(unsigned long map) +{ + struct exception_table_entry *start_ex_table = (struct exception_table_entry *) (__start___ex_table_addr + map); + struct exception_table_entry *stop_ex_table = (struct exception_table_entry *) (__stop___ex_table_addr + map); + + debug_putstr("\nRe-sorting exception table...\n"); + + sort_extable(start_ex_table, stop_ex_table); +} + +static void write_seed(unsigned long map) +{ + unsigned long long *ptr; + int i; + + ptr = (unsigned long long *)(fgkaslr_seed + map); + for (i = 0; i < 4; i++) + ptr[i] = seed[i]; +} + +void post_relocations_cleanup(unsigned long map) +{ + update_ex_table(map); + sort_ex_table(map); + write_seed(map); + free(sections); + free(sechdrs); +} + +void pre_relocations_cleanup(unsigned long map) +{ + sort_kallsyms(map); +} + +static void shuffle_sections(int *list, int size) +{ + int i; + unsigned long j; + int temp; + + parse_prandom_seed(); + + for (i = size - 1; i > 0; i--) { + j = kaslr_get_prandom_long() % (i + 1); + + temp = list[i]; + list[i] = list[j]; + list[j] = temp; + } +} + +static void move_text(int num_sections, char *secstrings, Elf64_Shdr *text, + void *source, void *dest, Elf64_Phdr *phdr) +{ + unsigned long adjusted_addr; + int copy_bytes; + void *stash; + Elf64_Shdr **sorted_sections; + int *index_list; + + memmove(dest, source + text->sh_offset, text->sh_size); + copy_bytes = text->sh_size; + dest += text->sh_size; + adjusted_addr = text->sh_addr + text->sh_size; + + /* + * we leave the sections sorted in their original order + * by s->sh_addr, but shuffle the indexes in a random + * order for copying. + */ + index_list = malloc(sizeof(int) * num_sections); + if (!index_list) + error("Failed to allocate space for index list"); + + for (int i = 0; i < num_sections; i++) + index_list[i] = i; + + shuffle_sections(index_list, num_sections); + + /* + * to avoid overwriting earlier sections before they can get + * copied to dest, stash everything into a buffer first. + * this will cause our source address to be off by + * phdr->p_offset though, so we'll adjust s->sh_offset below. + * + * TBD: ideally we'd simply decompress higher up so that our + * copy wasn't in danger of overwriting anything important. + */ + stash = malloc(phdr->p_filesz); + if (!stash) + error("Failed to allocate space for text stash"); + + memcpy(stash, source + phdr->p_offset, phdr->p_filesz); + + /* now we'd walk through the sections. */ + for (int j = 0; j < num_sections; j++) { + unsigned long aligned_addr; + Elf64_Shdr *s; + const char *sname; + void *src; + + s = sections[index_list[j]]; + + sname = secstrings + s->sh_name; + + /* align addr for this section */ + aligned_addr = ALIGN(adjusted_addr, s->sh_addralign); + dest = (void *) ALIGN((unsigned long)dest, s->sh_addralign); + + /* + * copy out of stash, so adjust offset + */ + src = stash + s->sh_offset - phdr->p_offset; + + /* tbd - fill pad bytes with int3 */ + memmove(dest, src, s->sh_size); + + dest += s->sh_size; + copy_bytes += s->sh_size + aligned_addr - adjusted_addr; + adjusted_addr = aligned_addr + s->sh_size; + + /* we can blow away sh_offset for our own uses */ + s->sh_offset = aligned_addr - s->sh_addr; + } + + free(index_list); + + /* + * move remainder of text segment. Ok to just use original source + * here since this area is untouched. + */ + memmove(dest, source + text->sh_offset + copy_bytes, phdr->p_filesz - copy_bytes); + free(stash); +} + +#define GET_SYM(name) \ + if (strcmp(#name, strtab + sym->st_name) == 0) {\ + name = sym->st_value; \ + continue; \ + } + +static void parse_symtab(Elf64_Sym *symtab, char *strtab, long num_syms) +{ + Elf64_Sym *sym; + + if (symtab == NULL || strtab == NULL) + return; + + debug_putstr("\nLooking for symbols... "); + + /* + * walk through the symbol table looking for the symbols + * that we care about. + */ + for (sym = symtab; --num_syms >= 0; sym++) { + if (!sym->st_name) + continue; + + GET_SYM(kallsyms_num_syms); + GET_SYM(kallsyms_offsets); + GET_SYM(kallsyms_relative_base); + GET_SYM(kallsyms_names); + GET_SYM(kallsyms_markers); + GET_SYM(_stext); + GET_SYM(_etext); + GET_SYM(_sinittext); + GET_SYM(_einittext); + GET_SYM(fgkaslr_seed); + + /* these have to be renamed */ + if (strcmp("__start___ex_table", strtab + sym->st_name) == 0) { + __start___ex_table_addr = sym->st_value; + continue; + } + + if (strcmp("__stop___ex_table", strtab + sym->st_name) == 0) { + __stop___ex_table_addr = sym->st_value; + continue; + } + } +} + +void parse_sections_headers(void *output, Elf64_Ehdr *ehdr, Elf64_Phdr *phdrs) +{ + Elf64_Phdr *phdr; + Elf64_Shdr *s; + Elf64_Shdr *text = NULL; + Elf64_Shdr *percpu = NULL; + char *secstrings; + const char *sname; + int num_sections = 0; + Elf64_Sym *symtab = NULL; + char *strtab = NULL; + long num_syms = 0; + void *dest; + int i; + + debug_putstr("\nParsing ELF section headers... "); + + /* + * TBD: support more than 64K section headers. + */ + + /* we are going to need to allocate space for the section headers */ + sechdrs = malloc(sizeof(*sechdrs) * ehdr->e_shnum); + if (!sechdrs) + error("Failed to allocate space for shdrs"); + + sections = malloc(sizeof(*sections) * ehdr->e_shnum); + if (!sections) + error("Failed to allocate space for section pointers"); + + memcpy(sechdrs, output + ehdr->e_shoff, + sizeof(*sechdrs) * ehdr->e_shnum); + + /* we need to allocate space for the section string table */ + s = &sechdrs[ehdr->e_shstrndx]; + + secstrings = malloc(s->sh_size); + if (!secstrings) + error("Failed to allocate space for shstr"); + + memcpy(secstrings, output + s->sh_offset, s->sh_size); + + /* + * now we need to walk through the section headers and collect the + * sizes of the .text sections to be randomized. + */ + for (i = 0; i < ehdr->e_shnum; i++) { + s = &sechdrs[i]; + sname = secstrings + s->sh_name; + + if (s->sh_type == SHT_SYMTAB) { + /* only one symtab per image */ + symtab = malloc(s->sh_size); + if (!symtab) + error("Failed to allocate space for symtab"); + + memcpy(symtab, output + s->sh_offset, s->sh_size); + num_syms = s->sh_size/sizeof(*symtab); + continue; + } + + if (s->sh_type == SHT_STRTAB && (i != ehdr->e_shstrndx)) { + strtab = malloc(s->sh_size); + if (!strtab) + error("Failed to allocate space for strtab"); + + memcpy(strtab, output + s->sh_offset, s->sh_size); + } + + if (!strcmp(sname, ".text")) { + text = s; + continue; + } + + if (!strcmp(sname, ".data..percpu")) { + /* get start addr for later */ + percpu = s; + } + + if (!(s->sh_flags & SHF_ALLOC) || + !(s->sh_flags & SHF_EXECINSTR) || + !(strstarts(sname, ".text"))) + continue; + + sections[num_sections] = s; + + num_sections++; + } + sections[num_sections] = NULL; + sections_size = num_sections; + + parse_symtab(symtab, strtab, num_syms); + + for (i = 0; i < ehdr->e_phnum; i++) { + phdr = &phdrs[i]; + + switch (phdr->p_type) { + case PT_LOAD: + if ((phdr->p_align % 0x200000) != 0) + error("Alignment of LOAD segment isn't multiple of 2MB"); + dest = output; + dest += (phdr->p_paddr - LOAD_PHYSICAL_ADDR); + if (text && (phdr->p_offset == text->sh_offset)) { + move_text(num_sections, secstrings, text, output, dest, phdr); + } else { + if (percpu && (phdr->p_offset == percpu->sh_offset)) { + percpu_start = percpu->sh_addr; + percpu_end = percpu_start + phdr->p_filesz; + } + memmove(dest, output + phdr->p_offset, + phdr->p_filesz); + } + break; + default: /* Ignore other PT_* */ + break; + } + } + + /* we need to keep the section info to redo relocs */ + free(secstrings); + + free(phdrs); +} + +#include "../../../../lib/sort.c" +#include "../../../../lib/bsearch.c" diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 9652d5c2afda..977da0911ce7 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -203,10 +203,20 @@ static void handle_relocations(void *output, unsigned long output_len, if (IS_ENABLED(CONFIG_X86_64)) delta = virt_addr - LOAD_PHYSICAL_ADDR; - if (!delta) { - debug_putstr("No relocation needed... "); - return; + /* + * it is possible to have delta be zero and + * still have enabled fg kaslr. We need to perform relocations + * for fgkaslr regardless of whether the base address has moved. + */ + if (!IS_ENABLED(CONFIG_FG_KASLR)) { + if (!delta) { + debug_putstr("No relocation needed... "); + return; + } } + + pre_relocations_cleanup(map); + debug_putstr("Performing relocations... "); /* @@ -230,35 +240,106 @@ static void handle_relocations(void *output, unsigned long output_len, */ for (reloc = output + output_len - sizeof(*reloc); *reloc; reloc--) { long extended = *reloc; + long value; + + /* + * if using fgkaslr, we might have moved the address + * of the relocation. Check it to see if it needs adjusting + * from the original address. + */ + (void) adjust_address(&extended); + extended += map; ptr = (unsigned long)extended; if (ptr < min_addr || ptr > max_addr) error("32-bit relocation outside of kernel!\n"); - *(uint32_t *)ptr += delta; + value = *(int32_t *)ptr; + + /* + * If using fgkaslr, the value of the relocation + * might need to be changed because it referred + * to an address that has moved. + */ + adjust_address(&value); + + value += delta; + + *(uint32_t *)ptr = value; } #ifdef CONFIG_X86_64 while (*--reloc) { long extended = *reloc; + long value; + long oldvalue; + Elf64_Shdr *s; + + /* + * if using fgkaslr, we might have moved the address + * of the relocation. Check it to see if it needs adjusting + * from the original address. + */ + s = adjust_address(&extended); + extended += map; ptr = (unsigned long)extended; if (ptr < min_addr || ptr > max_addr) error("inverse 32-bit relocation outside of kernel!\n"); - *(int32_t *)ptr -= delta; + value = *(int32_t *)ptr; + oldvalue = value; + + /* + * If using fgkaslr, these relocs will contain + * relative offsets which might need to be + * changed because it referred + * to an address that has moved. + */ + adjust_relative_offset(*reloc, &value, s); + + /* + * only percpu symbols need to have their values adjusted for + * base address kaslr since relative offsets within the .text + * and .text.* sections are ok wrt each other. + */ + if (is_percpu_addr(*reloc, oldvalue)) + value -= delta; + + *(int32_t *)ptr = value; } for (reloc--; *reloc; reloc--) { long extended = *reloc; + long value; + + /* + * if using fgkaslr, we might have moved the address + * of the relocation. Check it to see if it needs adjusting + * from the original address. + */ + (void) adjust_address(&extended); + extended += map; ptr = (unsigned long)extended; if (ptr < min_addr || ptr > max_addr) error("64-bit relocation outside of kernel!\n"); - *(uint64_t *)ptr += delta; + value = *(int64_t *)ptr; + + /* + * If using fgkaslr, the value of the relocation + * might need to be changed because it referred + * to an address that has moved. + */ + (void) adjust_address(&value); + + value += delta; + + *(uint64_t *)ptr = value; } + post_relocations_cleanup(map); #endif } #else @@ -296,6 +377,11 @@ static void parse_elf(void *output) memcpy(phdrs, output + ehdr.e_phoff, sizeof(*phdrs) * ehdr.e_phnum); + if (IS_ENABLED(CONFIG_FG_KASLR)) { + parse_sections_headers(output, &ehdr, phdrs); + return; + } + for (i = 0; i < ehdr.e_phnum; i++) { phdr = &phdrs[i]; @@ -448,3 +534,11 @@ void fortify_panic(const char *name) { error("detected buffer overflow"); } + +/* + * TBD: why does including the .c file in this way work, but building + * a separate fgkaslr.o file cause memory reads to fail (garbage)? + */ +#ifdef CONFIG_FG_KASLR +#include "fgkaslr.c" +#endif diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index c8181392f70d..60ceb277596d 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -74,6 +74,32 @@ struct mem_vector { unsigned long long size; }; +#ifdef CONFIG_X86_64 +#define Elf_Ehdr Elf64_Ehdr +#define Elf_Phdr Elf64_Phdr +#define Elf_Shdr Elf64_Shdr +#else +#define Elf_Ehdr Elf32_Ehdr +#define Elf_Phdr Elf32_Phdr +#define Elf_Shdr Elf32_Shdr +#endif + +#if CONFIG_FG_KASLR +void parse_sections_headers(void *output, Elf_Ehdr *ehdr, Elf_Phdr *phdrs); +void pre_relocations_cleanup(unsigned long map); +void post_relocations_cleanup(unsigned long map); +Elf_Shdr *adjust_address(long *address); +void adjust_relative_offset(long pc, long *value, Elf_Shdr *section); +bool is_percpu_addr(long pc, long offset); +#else +static inline void parse_sections_headers(void *output, Elf_Ehdr *ehdr, Elf_Phdr *phdrs) { } +static inline void pre_relocations_cleanup(unsigned long map) { } +static inline void post_relocations_cleanup(unsigned long map) { } +static inline Elf_Shdr *adjust_address(long *address) { return NULL; } +static inline void adjust_relative_offset(long pc, long *value, Elf_Shdr *section) { } +static inline bool is_percpu_addr(long pc, long offset) { return true; } +#endif /* CONFIG_FG_KASLR */ + #if CONFIG_RANDOMIZE_BASE /* kaslr.c */ void choose_random_location(unsigned long input, diff --git a/arch/x86/boot/compressed/vmlinux.symbols b/arch/x86/boot/compressed/vmlinux.symbols new file mode 100644 index 000000000000..13ef31a0aaf2 --- /dev/null +++ b/arch/x86/boot/compressed/vmlinux.symbols @@ -0,0 +1,15 @@ +kallsyms_offsets +kallsyms_addresses +kallsyms_num_syms +kallsyms_relative_base +kallsyms_names +kallsyms_token_table +kallsyms_token_index +kallsyms_markers +__start___ex_table +__stop___ex_table +_sinittext +_einittext +_stext +_etext +fgkaslr_seed diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index 680c320363db..6918d33eb5ef 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -26,8 +26,19 @@ #ifdef CONFIG_KERNEL_BZIP2 # define BOOT_HEAP_SIZE 0x400000 -#else /* !CONFIG_KERNEL_BZIP2 */ -# define BOOT_HEAP_SIZE 0x10000 +#elif CONFIG_FG_KASLR +/* + * We need extra boot heap when using fgkaslr because we make a copy + * of the original decompressed kernel to avoid issues with writing + * over ourselves when shuffling the sections. We also need extra + * space for resorting kallsyms after shuffling. This value could + * be decreased if free() would release memory properly, or if we + * could avoid the kernel copy. It would need to be increased if we + * find additional tables that need to be resorted. + */ +# define BOOT_HEAP_SIZE 0x4000000 +#else /* !CONFIG_KERNEL_BZIP2 && !CONFIG_FG_KASLR */ +# define BOOT_HEAP_SIZE 0x10000 #endif #ifdef CONFIG_X86_64 diff --git a/arch/x86/include/asm/kaslr.h b/arch/x86/include/asm/kaslr.h index 47d5b25e5b11..f35a9831aaec 100644 --- a/arch/x86/include/asm/kaslr.h +++ b/arch/x86/include/asm/kaslr.h @@ -4,6 +4,7 @@ unsigned long kaslr_get_random_seed(const char *purpose); unsigned long kaslr_get_prandom_long(void); +void prng_init_seed(unsigned long a, unsigned long b, unsigned long c, unsigned long d); #ifdef CONFIG_RANDOMIZE_MEMORY void kernel_randomize_memory(void); diff --git a/arch/x86/lib/kaslr.c b/arch/x86/lib/kaslr.c index 41b5610855a3..950299d64e1e 100644 --- a/arch/x86/lib/kaslr.c +++ b/arch/x86/lib/kaslr.c @@ -144,3 +144,18 @@ unsigned long kaslr_get_prandom_long(void) return prng_u64(&state); } + +void prng_init_seed(unsigned long a, unsigned long b, unsigned long c, unsigned long d) +{ + int i; + + state.a = a; + state.b = b; + state.c = c; + state.d = d; + + for (i = 0; i < 30; ++i) + (void)prng_u64(&state); + + initialized = true; +} diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 94153732ec00..2f42c14df0f3 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -60,6 +60,7 @@ static unsigned int table_size, table_cnt; static int all_symbols; static int absolute_percpu; static int base_relative; +static int fg_kaslr; static int token_profit[0x10000]; @@ -71,7 +72,7 @@ static unsigned char best_table_len[256]; static void usage(void) { fprintf(stderr, "Usage: kallsyms [--all-symbols] " - "[--base-relative] < in.map > out.S\n"); + "[--base-relative] [--fg-kaslr] < in.map > out.S\n"); exit(1); } @@ -98,6 +99,7 @@ static bool is_ignored_symbol(const char *name, char type) "kallsyms_markers", "kallsyms_token_table", "kallsyms_token_index", + "fgkaslr_seed", /* Exclude linker generated symbols which vary between passes */ "_SDA_BASE_", /* ppc */ "_SDA2_BASE_", /* ppc */ @@ -466,6 +468,14 @@ static void write_src(void) output_label("kallsyms_token_index"); for (i = 0; i < 256; i++) printf("\t.short\t%d\n", best_idx[i]); + + if (fg_kaslr) { + output_label("fgkaslr_seed"); + printf("\t.quad\t%d\n", 0); + printf("\t.quad\t%d\n", 0); + printf("\t.quad\t%d\n", 0); + printf("\t.quad\t%d\n", 0); + } printf("\n"); } @@ -743,6 +753,8 @@ int main(int argc, char **argv) absolute_percpu = 1; else if (strcmp(argv[i], "--base-relative") == 0) base_relative = 1; + else if (strcmp(argv[i], "--fg-kaslr") == 0) + fg_kaslr = 1; else usage(); } diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 436379940356..33882bbf95cc 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -152,6 +152,10 @@ kallsyms() kallsymopt="${kallsymopt} --base-relative" fi + if [ -n "${CONFIG_FG_KASLR}" ]; then + kallsymopt="${kallsymopt} --fg-kaslr" + fi + local aflags="${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \ ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS}" From patchwork Wed Feb 5 22:39:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C69CF14E3 for ; Wed, 5 Feb 2020 22:41:19 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 39181214AF for ; Wed, 5 Feb 2020 22:41:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39181214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17679-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 10132 invoked by uid 550); 5 Feb 2020 22:39:59 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 10060 invoked from network); 5 Feb 2020 22:39:58 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092462" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 09/11] kallsyms: hide layout and expose seed Date: Wed, 5 Feb 2020 14:39:48 -0800 Message-Id: <20200205223950.1212394-10-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 To support finer grained kaslr (fgkaslr), we need to make a couple changes to kallsyms. Firstly, we need to hide our sorted list of symbols, since this will give away our new layout. Secondly, we will export the seed used for randomizing the layout so that it can be used to make a particular layout persist across boots for debug purposes. Signed-off-by: Kristen Carlson Accardi --- kernel/kallsyms.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 136ce049c4ad..432b13a3a033 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -698,6 +698,21 @@ const char *kdb_walk_kallsyms(loff_t *pos) } #endif /* CONFIG_KGDB_KDB */ +#ifdef CONFIG_FG_KASLR +extern const u64 fgkaslr_seed[] __weak; + +static int proc_fgkaslr_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%llx\n", fgkaslr_seed[0]); + seq_printf(m, "%llx\n", fgkaslr_seed[1]); + seq_printf(m, "%llx\n", fgkaslr_seed[2]); + seq_printf(m, "%llx\n", fgkaslr_seed[3]); + return 0; +} +#else +static inline int proc_fgkaslr_show(struct seq_file *m, void *v) { return 0; } +#endif + static const struct file_operations kallsyms_operations = { .open = kallsyms_open, .read = seq_read, @@ -707,7 +722,20 @@ static const struct file_operations kallsyms_operations = { static int __init kallsyms_init(void) { - proc_create("kallsyms", 0444, NULL, &kallsyms_operations); + /* + * When fine grained kaslr is enabled, we don't want to + * print out the symbols even with zero pointers because + * this reveals the randomization order. If fg kaslr is + * enabled, make kallsyms available only to privileged + * users. + */ + if (!IS_ENABLED(CONFIG_FG_KASLR)) + proc_create("kallsyms", 0444, NULL, &kallsyms_operations); + else { + proc_create_single("fgkaslr_seed", 0400, NULL, + proc_fgkaslr_show); + proc_create("kallsyms", 0400, NULL, &kallsyms_operations); + } return 0; } device_initcall(kallsyms_init); From patchwork Wed Feb 5 22:39:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3C5292A for ; Wed, 5 Feb 2020 22:41:31 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 36855214AF for ; Wed, 5 Feb 2020 22:41:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36855214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17680-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 10187 invoked by uid 550); 5 Feb 2020 22:40:00 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 10106 invoked from network); 5 Feb 2020 22:39:59 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092467" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 10/11] module: Reorder functions Date: Wed, 5 Feb 2020 14:39:49 -0800 Message-Id: <20200205223950.1212394-11-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 If a module has functions split out into separate text sections (i.e. compiled with the -ffunction-sections flag), reorder the functions to provide some code diversification to modules. Signed-off-by: Kristen Carlson Accardi Reviewed-by: Kees Cook --- kernel/module.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) diff --git a/kernel/module.c b/kernel/module.c index b56f3224b161..231563e95e61 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -53,6 +53,8 @@ #include #include #include +#include +#include #include #include "module-internal.h" @@ -3245,6 +3247,87 @@ static int find_module_sections(struct module *mod, struct load_info *info) return 0; } +/* + * shuffle_text_list() + * Use a Fisher Yates algorithm to shuffle a list of text sections. + */ +static void shuffle_text_list(Elf_Shdr **list, int size) +{ + int i; + unsigned int j; + Elf_Shdr *temp; + + for (i = size - 1; i > 0; i--) { + /* + * TBD - seed. We need to be able to use a known + * seed so that we can non-randomly randomize for + * debugging. + */ + + // pick a random index from 0 to i + get_random_bytes(&j, sizeof(j)); + j = j % (i + 1); + + temp = list[i]; + list[i] = list[j]; + list[j] = temp; + } +} + +/* + * randomize_text() + * Look through the core section looking for executable code sections. + * Store sections in an array and then shuffle the sections + * to reorder the functions. + */ +static void randomize_text(struct module *mod, struct load_info *info) +{ + int i; + int num_text_sections = 0; + Elf_Shdr **text_list; + int size = 0; + int max_sections = info->hdr->e_shnum; + unsigned int sec = find_sec(info, ".text"); + + if (!IS_ENABLED(CONFIG_FG_KASLR) || !kaslr_enabled()) + return; + + if (sec == 0) + return; + + text_list = kmalloc_array(max_sections, sizeof(*text_list), GFP_KERNEL); + if (text_list == NULL) + return; + + for (i = 0; i < max_sections; i++) { + Elf_Shdr *shdr = &info->sechdrs[i]; + const char *sname = info->secstrings + shdr->sh_name; + + if (!(shdr->sh_flags & SHF_ALLOC) || + !(shdr->sh_flags & SHF_EXECINSTR) || + strstarts(sname, ".init")) + continue; + + text_list[num_text_sections] = shdr; + num_text_sections++; + } + + shuffle_text_list(text_list, num_text_sections); + + for (i = 0; i < num_text_sections; i++) { + Elf_Shdr *shdr = text_list[i]; + unsigned int infosec; + const char *sname; + + sname = info->secstrings + shdr->sh_name; + infosec = shdr->sh_info; + + shdr->sh_entsize = get_offset(mod, &size, shdr, infosec); + } + + kfree(text_list); +} + static int move_module(struct module *mod, struct load_info *info) { int i; @@ -3282,6 +3365,8 @@ static int move_module(struct module *mod, struct load_info *info) } else mod->init_layout.base = NULL; + randomize_text(mod, info); + /* Transfer each section which specifies SHF_ALLOC */ pr_debug("final section addresses:\n"); for (i = 0; i < info->hdr->e_shnum; i++) { From patchwork Wed Feb 5 22:39:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kristen Carlson Accardi X-Patchwork-Id: 11367289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43C4A92A for ; Wed, 5 Feb 2020 22:41:41 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 81676214AF for ; Wed, 5 Feb 2020 22:41:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81676214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-17681-patchwork-kernel-hardening=patchwork.kernel.org@lists.openwall.com Received: (qmail 10231 invoked by uid 550); 5 Feb 2020 22:40:02 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 10136 invoked from network); 5 Feb 2020 22:39:59 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,407,1574150400"; d="scan'208";a="225092471" From: Kristen Carlson Accardi To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arjan@linux.intel.com, keescook@chromium.org Cc: rick.p.edgecombe@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Kristen Carlson Accardi Subject: [RFC PATCH 11/11] x86/boot: Move "boot heap" out of .bss Date: Wed, 5 Feb 2020 14:39:50 -0800 Message-Id: <20200205223950.1212394-12-kristen@linux.intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200205223950.1212394-1-kristen@linux.intel.com> References: <20200205223950.1212394-1-kristen@linux.intel.com> MIME-Version: 1.0 From: Kees Cook Currently the on-disk decompression image includes the "dynamic" heap region that is used for malloc() during kernel extraction, relocation, and decompression ("boot_heap" of BOOT_HEAP_SIZE bytes in the .bss section). It makes no sense to balloon the bzImage with "boot_heap" as it is zeroed at boot, and acts much more like a "brk" region. This seems to be a trivial change because head_{64,32}.S already only copies up to the start of the .bss section, so any growth in the .bss area was already not meaningful when placing the image in memory. The .bss size is, however, reflected in the boot params "init_size", so the memory range calculations included the "boot_heap" region. Instead of wasting the on-disk image size bytes, just account for this heap area when identifying the mem_avoid ranges, and leave it out of the .bss section entirely. For good measure, also zero initialize it, as this was already happening for when zeroing the entire .bss section. While the bzImage size is dominated by the compressed vmlinux, the difference removes 64KB for all compressors except bzip2, which removes 4MB. For example, this is less than 1% under CONFIG_KERNEL_XZ: -rw-r--r-- 1 kees kees 7813168 Feb 2 23:39 arch/x86/boot/bzImage.stock-xz -rw-r--r-- 1 kees kees 7747632 Feb 2 23:42 arch/x86/boot/bzImage.brk-xz but much more pronounced under CONFIG_KERNEL_BZIP2 (~27%): -rw-r--r-- 1 kees kees 15231024 Feb 2 23:44 arch/x86/boot/bzImage.stock-bzip2 -rw-r--r-- 1 kees kees 11036720 Feb 2 23:47 arch/x86/boot/bzImage.brk-bzip2 For the future fine-grain KASLR work, this will avoid significant pain, as the ELF section parser will use much more memory during boot and filling the bzImage with megabytes of zeros seemed like a poor idea. :) Signed-off-by: Kees Cook Signed-off-by: Kristen Carlson Accardi --- arch/x86/boot/compressed/head_32.S | 5 ++--- arch/x86/boot/compressed/head_64.S | 7 +++---- arch/x86/boot/compressed/kaslr.c | 2 +- arch/x86/boot/compressed/misc.c | 3 +++ arch/x86/boot/compressed/vmlinux.lds.S | 1 + 5 files changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index f2dfd6d083ef..1f3de8efd40e 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -59,6 +59,7 @@ .hidden _ebss .hidden _got .hidden _egot + .hidden _brk __HEAD SYM_FUNC_START(startup_32) @@ -249,7 +250,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) pushl $z_input_len /* input_len */ leal input_data(%ebx), %eax pushl %eax /* input_data */ - leal boot_heap(%ebx), %eax + leal _brk(%ebx), %eax pushl %eax /* heap area */ pushl %esi /* real mode pointer */ call extract_kernel /* returns kernel location in %eax */ @@ -276,8 +277,6 @@ efi32_config: */ .bss .balign 4 -boot_heap: - .fill BOOT_HEAP_SIZE, 1, 0 boot_stack: .fill BOOT_STACK_SIZE, 1, 0 boot_stack_end: diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index ee60b81944a7..850bc5220a8d 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -42,6 +42,7 @@ .hidden _ebss .hidden _got .hidden _egot + .hidden _brk __HEAD .code32 @@ -534,7 +535,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) */ pushq %rsi /* Save the real mode argument */ movq %rsi, %rdi /* real mode address */ - leaq boot_heap(%rip), %rsi /* malloc area for uncompression */ + leaq _brk(%rip), %rsi /* malloc area for uncompression */ leaq input_data(%rip), %rdx /* input_data */ movl $z_input_len, %ecx /* input_len */ movq %rbp, %r8 /* output target address */ @@ -701,12 +702,10 @@ SYM_DATA_END(efi64_config) #endif /* CONFIG_EFI_STUB */ /* - * Stack and heap for uncompression + * Stack for placement and uncompression */ .bss .balign 4 -SYM_DATA_LOCAL(boot_heap, .fill BOOT_HEAP_SIZE, 1, 0) - SYM_DATA_START_LOCAL(boot_stack) .fill BOOT_STACK_SIZE, 1, 0 SYM_DATA_END_LABEL(boot_stack, SYM_L_LOCAL, boot_stack_end) diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index ae4dce76a9bd..da64d2cdbb42 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -397,7 +397,7 @@ static void handle_mem_options(void) static void mem_avoid_init(unsigned long input, unsigned long input_size, unsigned long output) { - unsigned long init_size = boot_params->hdr.init_size; + unsigned long init_size = boot_params->hdr.init_size + BOOT_HEAP_SIZE; u64 initrd_start, initrd_size; u64 cmd_line, cmd_line_size; char *ptr; diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 977da0911ce7..cb12da264b59 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -463,6 +463,9 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, debug_putstr("early console in extract_kernel\n"); + /* Zero what is effectively our .brk section. */ + memset((void *)heap, 0, BOOT_HEAP_SIZE); + debug_putaddr(heap); free_mem_ptr = heap; /* Heap */ free_mem_end_ptr = heap + BOOT_HEAP_SIZE; diff --git a/arch/x86/boot/compressed/vmlinux.lds.S b/arch/x86/boot/compressed/vmlinux.lds.S index 508cfa6828c5..3ce690474940 100644 --- a/arch/x86/boot/compressed/vmlinux.lds.S +++ b/arch/x86/boot/compressed/vmlinux.lds.S @@ -73,4 +73,5 @@ SECTIONS #endif . = ALIGN(PAGE_SIZE); /* keep ZO size page aligned */ _end = .; + _brk = .; }