From patchwork Fri Apr 29 20:36:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12832768 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4539FC433F5 for ; Fri, 29 Apr 2022 20:42:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=OllBhxKcCtjw9XqaQIRPRYitI4y6CG/c5PvEcr2UcBw=; b=eB8FM1tlKDJpndbxeYlpTfEhCr jnZYV/W6mqSM+C2bqTFZzjv836dbjIw+ZYWuF27uSoN98LODqx+y28w0Z2Wh0AQLSVaTEmD1Pljin FuO+mfckmcWQIrpP49zgkja78OqFW1iNS1/Z4jkAObN28mHrp3Vg1ffGtA4oHXkqfRUvY1+oEuHVB 8xcXJ6N0wCaFt7rjUvuhvBq6x/xBp9vj/M0A49MdpeRIsl0vDvfQh6siFlKSFnL4UGPjNfVmKRV+s xinlPZSDalGDRdfDGaO8JTohoUi+8hYLs5DRnUEtelNj4nxAZ9FxLjZye9iO2uGhQwKmfM4s9FoLB DulnAOFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkXQD-00CYjv-Ox; Fri, 29 Apr 2022 20:41:02 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkXMN-00CX7y-VN for linux-arm-kernel@lists.infradead.org; Fri, 29 Apr 2022 20:37:08 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-2f4dfd09d7fso84976647b3.0 for ; Fri, 29 Apr 2022 13:37:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=vvdIaJf57HGaDpMnbrJ8giqLRUU+BTS8OjFrmaz2vTE=; b=pp5ZuqtMoctSU0CirU7eAqTG/TJOM0spnINt8Gjn3TLZaHntxQdntxMbG6AC9Ytn1A YbobImxlpkDKOtCc7p+HUcjkX6xeFPx/DEbzlgXaBCLyH85pOtUO5AjwtRLJ5v32iDgF GOxVmU9+KLnarIhguwlaAcJMXLCHb/CIIQn5HllZfWoEDWIMA9Dkq2G1E64CEeOFgP/U 8lRGqsLZUoDl9j6+7EG0KVET8D+nRVZ9/GAPoDTb0VvTkSklhvkSLnv3DwQ3HD+kRcY5 VrykuvxxspP2ULayXxAZ1mjo3k7ttqlSFX+HHy88dA/a3EhnXPGJv5NH9Tm80twX9v1Q sg6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=vvdIaJf57HGaDpMnbrJ8giqLRUU+BTS8OjFrmaz2vTE=; b=Y1Ne3bSNILhJ6lamKeqs5h9k66suiGJG1w/XabxJTcWCdzt0Tsm57yPIRwuIix0JWp QQJpQMOoXLqt5ETEA35GIzqY+FeVlbNZeO2Kj21POrMjmKjt72iz/zH/QMnWYRq8JR7U whOWOaQX6BxCWT4F4l/uGohF9LuPH/KJBbRQVV1zV9vOiKGRp7rnVwQfgi+XozJWZO5F Ufe9mkbYo0IM1eRRI5nLfo4fURVUFpQCRC/jsf38Qre0YZr+y8AnN2oK2GOacW9ZEbWW RaboAV1JIH/TXuN6rvp8fFA9BVQO1jnuclaMca+dN0+z3H7jWPPJIYEgZJ7GlIsk4a9L C98Q== X-Gm-Message-State: AOAM533X9qUStDqTSBV4CdpCy1oTT9YG+SYPbu8rJR/+lgamUsWqWJiL aDJW/DL5ryruJpx56RbbNTjx2WpOmuiyj6rVeeM= X-Google-Smtp-Source: ABdhPJzK+TM3mWgJX5Du5pYGBn42fe+Iz5sXydBvEX3gH6bzFZSppJYXAmv+vNiXUVCsvdPo/kBBgKk9tLQAOWDp42s= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:351:bea9:f158:1021]) (user=samitolvanen job=sendgmr) by 2002:a05:6902:390:b0:645:7d46:f1db with SMTP id f16-20020a056902039000b006457d46f1dbmr1280096ybs.85.1651264622333; Fri, 29 Apr 2022 13:37:02 -0700 (PDT) Date: Fri, 29 Apr 2022 13:36:29 -0700 In-Reply-To: <20220429203644.2868448-1-samitolvanen@google.com> Message-Id: <20220429203644.2868448-7-samitolvanen@google.com> Mime-Version: 1.0 References: <20220429203644.2868448-1-samitolvanen@google.com> X-Developer-Key: i=samitolvanen@google.com; a=openpgp; fpr=35CCFB63B283D6D3AEB783944CB5F6848BBC56EE X-Developer-Signature: v=1; a=openpgp-sha256; l=15228; h=from:subject; bh=0g7XP1LgryML35bYmY5P55DUledGJO075lwf/wPq5XY=; b=owEB7QES/pANAwAKAUy19oSLvFbuAcsmYgBibExVzS6QJPpNhwgvz4rLUMgG/eseoiNocaoGE0+w tX5LFGCJAbMEAAEKAB0WIQQ1zPtjsoPW0663g5RMtfaEi7xW7gUCYmxMVQAKCRBMtfaEi7xW7hkYC/ 4ubQXAmms/iLy/3p508tt7PKf5jjRHk61JliAxBIHRWk2AJ+/ApG33RlrxlUW3l1l97z84CdogaqSa jdo5kzTr5mVE8BojuUFoGj3o6JKuqeuKDXPluSgQkK9/n0l6BGyNpH5iMrTqjmTS5+qKjA7Z8yCk6N h2uNUbPmBJ5gerAmb+taQVdcEAcfdLsOmPIoRxKuuFj8hqFU8wwW7mbcQAQZNJ5U//BpPEbO0Ad299 aRh5GtCHpvSxLRL9t1egoSBtslKOt86N19OjWNXXnJybdVSRx5Mo8+RUbX33VIUz4hXbuUH7FJuyL9 JlZlvDFZwT5JKvRgZWVGKqKMEyyQuQfM5pn1P6Zx/CT5vuWlWZY3EUuVGjUZ/9wHZFNgzPJgCbRBoh ng2kyfVoFHuWLCYRgUx6LAoxF3YPUvGXy+ZGeUgSSYjaZyl2o+S9T4Gt8+YCoSNHiXDy5yGH3hYtX5 TMqHSVc2S30OxsckAvhcZzkLCdIj8XoQ0tfMFRpbJIeMU= X-Mailer: git-send-email 2.36.0.464.gb9c8b46e94-goog Subject: [RFC PATCH 06/21] cfi: Switch to -fsanitize=kcfi From: Sami Tolvanen To: linux-kernel@vger.kernel.org Cc: Kees Cook , Josh Poimboeuf , Peter Zijlstra , x86@kernel.org, Catalin Marinas , Will Deacon , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Joao Moreira , Sedat Dilek , Steven Rostedt , linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220429_133704_098937_5384EA4E X-CRM114-Status: GOOD ( 25.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Switch from Clang's original forward-edge control-flow integrity implementation to -fsanitize=kcfi, which is better suited for the kernel, as it doesn't require LTO, doesn't use a jump table that requires altering function references, and won't break cross-module function address equality. Signed-off-by: Sami Tolvanen --- Makefile | 13 +-- arch/Kconfig | 8 +- include/asm-generic/vmlinux.lds.h | 38 ++++----- include/linux/cfi.h | 24 +++++- include/linux/compiler-clang.h | 8 +- include/linux/module.h | 4 +- kernel/cfi.c | 129 ++++++++++++++++-------------- kernel/module.c | 34 +------- scripts/module.lds.S | 24 ++---- 9 files changed, 126 insertions(+), 156 deletions(-) diff --git a/Makefile b/Makefile index c3ec1ea42379..22a5d48f5fb4 100644 --- a/Makefile +++ b/Makefile @@ -915,18 +915,7 @@ export CC_FLAGS_LTO endif ifdef CONFIG_CFI_CLANG -CC_FLAGS_CFI := -fsanitize=cfi \ - -fsanitize-cfi-cross-dso \ - -fno-sanitize-cfi-canonical-jump-tables \ - -fno-sanitize-trap=cfi \ - -fno-sanitize-blacklist - -ifdef CONFIG_CFI_PERMISSIVE -CC_FLAGS_CFI += -fsanitize-recover=cfi -endif - -# If LTO flags are filtered out, we must also filter out CFI. -CC_FLAGS_LTO += $(CC_FLAGS_CFI) +CC_FLAGS_CFI := -fsanitize=kcfi -fno-sanitize-blacklist KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif diff --git a/arch/Kconfig b/arch/Kconfig index 625db6376726..601379a6173d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -722,12 +722,8 @@ config ARCH_SUPPORTS_CFI_CLANG config CFI_CLANG bool "Use Clang's Control Flow Integrity (CFI)" - depends on LTO_CLANG && ARCH_SUPPORTS_CFI_CLANG - # Clang >= 12: - # - https://bugs.llvm.org/show_bug.cgi?id=46258 - # - https://bugs.llvm.org/show_bug.cgi?id=47479 - depends on CLANG_VERSION >= 120000 - select KALLSYMS + depends on ARCH_SUPPORTS_CFI_CLANG + depends on $(cc-option,-fsanitize=kcfi) help This option enables Clang’s forward-edge Control Flow Integrity (CFI) checking, where the compiler injects a runtime check to each diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 69138e9db787..20bfd2f01d6f 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -421,6 +421,22 @@ __end_ro_after_init = .; #endif +/* + * .kcfi_traps contains a list KCFI trap locations. + */ +#ifndef KCFI_TRAPS +#ifdef CONFIG_CFI_CLANG +#define KCFI_TRAPS \ + __kcfi_traps : AT(ADDR(__kcfi_traps) - LOAD_OFFSET) { \ + __start___kcfi_traps = .; \ + KEEP(*(.kcfi_traps)) \ + __stop___kcfi_traps = .; \ + } +#else +#define KCFI_TRAPS +#endif +#endif + /* * Read only Data */ @@ -529,6 +545,8 @@ __stop___modver = .; \ } \ \ + KCFI_TRAPS \ + \ RO_EXCEPTION_TABLE \ NOTES \ BTF \ @@ -537,21 +555,6 @@ __end_rodata = .; -/* - * .text..L.cfi.jumptable.* contain Control-Flow Integrity (CFI) - * jump table entries. - */ -#ifdef CONFIG_CFI_CLANG -#define TEXT_CFI_JT \ - . = ALIGN(PMD_SIZE); \ - __cfi_jt_start = .; \ - *(.text..L.cfi.jumptable .text..L.cfi.jumptable.*) \ - . = ALIGN(PMD_SIZE); \ - __cfi_jt_end = .; -#else -#define TEXT_CFI_JT -#endif - /* * Non-instrumentable text section */ @@ -579,7 +582,6 @@ *(.text..refcount) \ *(.ref.text) \ *(.text.asan.* .text.tsan.*) \ - TEXT_CFI_JT \ MEM_KEEP(init.text*) \ MEM_KEEP(exit.text*) \ @@ -1008,8 +1010,7 @@ * keep any .init_array.* sections. * https://bugs.llvm.org/show_bug.cgi?id=46478 */ -#if defined(CONFIG_GCOV_KERNEL) || defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KCSAN) || \ - defined(CONFIG_CFI_CLANG) +#if defined(CONFIG_GCOV_KERNEL) || defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KCSAN) # ifdef CONFIG_CONSTRUCTORS # define SANITIZER_DISCARDS \ *(.eh_frame) @@ -1027,6 +1028,7 @@ *(.discard) \ *(.discard.*) \ *(.modinfo) \ + *(.kcfi_types) \ /* ld.bfd warns about .gnu.version* even when not emitted */ \ *(.gnu.version*) \ diff --git a/include/linux/cfi.h b/include/linux/cfi.h index 2cdbc0fbd0ab..9cbadfca7e01 100644 --- a/include/linux/cfi.h +++ b/include/linux/cfi.h @@ -2,17 +2,33 @@ /* * Clang Control Flow Integrity (CFI) support. * - * Copyright (C) 2021 Google LLC + * Copyright (C) 2022 Google LLC */ #ifndef _LINUX_CFI_H #define _LINUX_CFI_H +#include +#include + #ifdef CONFIG_CFI_CLANG -typedef void (*cfi_check_fn)(uint64_t id, void *ptr, void *diag); -/* Compiler-generated function in each module, and the kernel */ -extern void __cfi_check(uint64_t id, void *ptr, void *diag); +#ifdef CONFIG_MODULES +void module_cfi_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *mod); +#endif + +void *arch_get_cfi_target(unsigned long addr, struct pt_regs *regs); +enum bug_trap_type report_cfi(unsigned long addr, struct pt_regs *regs); +#else + +#ifdef CONFIG_MODULES +static inline void module_cfi_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, + struct module *mod) {} +#endif +static inline enum bug_trap_type report_cfi(unsigned long addr, struct pt_regs *regs) +{ + return BUG_TRAP_TYPE_NONE; +} #endif /* CONFIG_CFI_CLANG */ #endif /* _LINUX_CFI_H */ diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index babb1347148c..c4ff42859077 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -66,9 +66,6 @@ # define __noscs __attribute__((__no_sanitize__("shadow-call-stack"))) #endif -#define __nocfi __attribute__((__no_sanitize__("cfi"))) -#define __cficanonical __attribute__((__cfi_canonical_jump_table__)) - /* * Turn individual warnings and errors on and off locally, depending * on version. @@ -93,3 +90,8 @@ #define __diag_ignore_all(option, comment) \ __diag_clang(11, ignore, option) + +#if CONFIG_CFI_CLANG +/* Disable CFI checking inside a function. */ +#define __nocfi __attribute__((__no_sanitize__("kcfi"))) +#endif diff --git a/include/linux/module.h b/include/linux/module.h index 87857275c047..430ea19f14f6 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -27,7 +27,6 @@ #include #include #include -#include #include #include @@ -389,7 +388,8 @@ struct module { unsigned int num_syms; #ifdef CONFIG_CFI_CLANG - cfi_check_fn cfi_check; + unsigned long *kcfi_traps; + unsigned long *kcfi_traps_end; #endif /* Kernel parameters. */ diff --git a/kernel/cfi.c b/kernel/cfi.c index 2cc0d01ea980..d9907df6576e 100644 --- a/kernel/cfi.c +++ b/kernel/cfi.c @@ -1,94 +1,101 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Clang Control Flow Integrity (CFI) error and slowpath handling. + * Clang Control Flow Integrity (CFI) error handling. * - * Copyright (C) 2021 Google LLC + * Copyright (C) 2022 Google LLC */ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* Compiler-defined handler names */ -#ifdef CONFIG_CFI_PERMISSIVE -#define cfi_failure_handler __ubsan_handle_cfi_check_fail -#else -#define cfi_failure_handler __ubsan_handle_cfi_check_fail_abort -#endif - -static inline void handle_cfi_failure(void *ptr) +#include + +/* Returns the target of the indirect call that follows the trap in `addr`. */ +void * __weak arch_get_cfi_target(unsigned long addr, struct pt_regs *regs) { - if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) - WARN_RATELIMIT(1, "CFI failure (target: %pS):\n", ptr); - else - panic("CFI failure (target: %pS)\n", ptr); + return NULL; } #ifdef CONFIG_MODULES +/* Populates `kcfi_trap(_end)?` fields in `struct module`. */ +void module_cfi_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, + struct module *mod) +{ + char *secstrings; + unsigned int i; + + mod->kcfi_traps = NULL; + mod->kcfi_traps_end = NULL; + + secstrings = (char *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; + + for (i = 1; i < hdr->e_shnum; i++) { + if (strcmp(secstrings+sechdrs[i].sh_name, "__kcfi_traps")) + continue; -static inline cfi_check_fn find_module_check_fn(unsigned long ptr) + mod->kcfi_traps = (unsigned long *)sechdrs[i].sh_addr; + mod->kcfi_traps_end = (unsigned long *)(sechdrs[i].sh_addr + sechdrs[i].sh_size); + break; + } +} + +static bool is_module_cfi_trap(unsigned long addr) { - cfi_check_fn fn = NULL; + bool found = false; struct module *mod; + unsigned long *p; rcu_read_lock_sched_notrace(); - mod = __module_address(ptr); + + mod = __module_address(addr); if (mod) - fn = mod->cfi_check; + for (p = mod->kcfi_traps; !found && p < mod->kcfi_traps_end; ++p) + found = (*p == addr); + rcu_read_unlock_sched_notrace(); - return fn; + return found; } -static inline cfi_check_fn find_check_fn(unsigned long ptr) -{ - cfi_check_fn fn = NULL; +#else /* CONFIG_MODULES */ - if (is_kernel_text(ptr)) - return __cfi_check; +static inline bool is_module_cfi_trap(unsigned long addr) +{ + return false; +} - /* - * Indirect call checks can happen when RCU is not watching. Both - * the shadow and __module_address use RCU, so we need to wake it - * up if necessary. - */ - RCU_NONIDLE({ - fn = find_module_check_fn(ptr); - }); +#endif /* CONFIG_MODULES */ - return fn; -} +extern unsigned long __start___kcfi_traps[]; +extern unsigned long __stop___kcfi_traps[]; -void __cfi_slowpath_diag(uint64_t id, void *ptr, void *diag) +static bool is_cfi_trap(unsigned long addr) { - cfi_check_fn fn = find_check_fn((unsigned long)ptr); + unsigned long *p; - if (likely(fn)) - fn(id, ptr, diag); - else /* Don't allow unchecked modules */ - handle_cfi_failure(ptr); + for (p = __start___kcfi_traps; p < __stop___kcfi_traps; ++p) + if (*p == addr) + return true; + + return is_module_cfi_trap(addr); } -EXPORT_SYMBOL(__cfi_slowpath_diag); -#else /* !CONFIG_MODULES */ +#define __CFI_ERROR_FMT "CFI failure at %pS (target: %pS)\n" -void __cfi_slowpath_diag(uint64_t id, void *ptr, void *diag) +static enum bug_trap_type __report_cfi(void *addr, void *target, struct pt_regs *regs) { - handle_cfi_failure(ptr); /* No modules */ + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) { + pr_warn(__CFI_ERROR_FMT, addr, target); + __warn(NULL, 0, addr, 0, regs, NULL); + + return BUG_TRAP_TYPE_WARN; + } else { + pr_crit(__CFI_ERROR_FMT, addr, target); + return BUG_TRAP_TYPE_BUG; + } } -EXPORT_SYMBOL(__cfi_slowpath_diag); - -#endif /* CONFIG_MODULES */ -void cfi_failure_handler(void *data, void *ptr, void *vtable) +enum bug_trap_type report_cfi(unsigned long addr, struct pt_regs *regs) { - handle_cfi_failure(ptr); + if (!is_cfi_trap(addr)) + return BUG_TRAP_TYPE_NONE; + + return __report_cfi((void *)addr, arch_get_cfi_target(addr, regs), regs); } -EXPORT_SYMBOL(cfi_failure_handler); diff --git a/kernel/module.c b/kernel/module.c index 296fe02323e9..411ae8c358e6 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -57,6 +57,7 @@ #include #include #include +#include #include #include "module-internal.h" @@ -3871,8 +3872,9 @@ static int complete_formation(struct module *mod, struct load_info *info) if (err < 0) goto out; - /* This relies on module_mutex for list integrity. */ + /* These rely on module_mutex for list integrity. */ module_bug_finalize(info->hdr, info->sechdrs, mod); + module_cfi_finalize(info->hdr, info->sechdrs, mod); module_enable_ro(mod, false); module_enable_nx(mod); @@ -3928,8 +3930,6 @@ static int unknown_module_param_cb(char *param, char *val, const char *modname, return 0; } -static void cfi_init(struct module *mod); - /* * Allocate and load the module: note that size of section 0 is always * zero, and we rely on this for optional sections. @@ -4059,9 +4059,6 @@ static int load_module(struct load_info *info, const char __user *uargs, flush_module_icache(mod); - /* Setup CFI for the module. */ - cfi_init(mod); - /* Now copy in args */ mod->args = strndup_user(uargs, ~0UL >> 1); if (IS_ERR(mod->args)) { @@ -4502,31 +4499,6 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, #endif /* CONFIG_LIVEPATCH */ #endif /* CONFIG_KALLSYMS */ -static void cfi_init(struct module *mod) -{ -#ifdef CONFIG_CFI_CLANG - initcall_t *init; - exitcall_t *exit; - - rcu_read_lock_sched(); - mod->cfi_check = (cfi_check_fn) - find_kallsyms_symbol_value(mod, "__cfi_check"); - init = (initcall_t *) - find_kallsyms_symbol_value(mod, "__cfi_jt_init_module"); - exit = (exitcall_t *) - find_kallsyms_symbol_value(mod, "__cfi_jt_cleanup_module"); - rcu_read_unlock_sched(); - - /* Fix init/exit functions to point to the CFI jump table */ - if (init) - mod->init = *init; -#ifdef CONFIG_MODULE_UNLOAD - if (exit) - mod->exit = *exit; -#endif -#endif -} - /* Maximum number of characters written by module_flags() */ #define MODULE_FLAGS_BUF_SIZE (TAINT_FLAGS_COUNT + 4) diff --git a/scripts/module.lds.S b/scripts/module.lds.S index 1d0e1e4dc3d2..ccd75d283840 100644 --- a/scripts/module.lds.S +++ b/scripts/module.lds.S @@ -3,20 +3,11 @@ * Archs are free to supply their own linker scripts. ld will * combine them automatically. */ -#ifdef CONFIG_CFI_CLANG -# include -# define ALIGN_CFI ALIGN(PAGE_SIZE) -# define SANITIZER_DISCARDS *(.eh_frame) -#else -# define ALIGN_CFI -# define SANITIZER_DISCARDS -#endif - SECTIONS { /DISCARD/ : { *(.discard) *(.discard.*) - SANITIZER_DISCARDS + *(.kcfi_types) } __ksymtab 0 : { *(SORT(___ksymtab+*)) } @@ -31,6 +22,10 @@ SECTIONS { __patchable_function_entries : { *(__patchable_function_entries) } +#ifdef CONFIG_CFI_CLANG + __kcfi_traps : { KEEP(*(.kcfi_traps)) } +#endif + #ifdef CONFIG_LTO_CLANG /* * With CONFIG_LTO_CLANG, LLD always enables -fdata-sections and @@ -51,15 +46,6 @@ SECTIONS { *(.rodata .rodata.[0-9a-zA-Z_]*) *(.rodata..L*) } - - /* - * With CONFIG_CFI_CLANG, we assume __cfi_check is at the beginning - * of the .text section, and is aligned to PAGE_SIZE. - */ - .text : ALIGN_CFI { - *(.text.__cfi_check) - *(.text .text.[0-9a-zA-Z_]* .text..L.cfi*) - } #endif }