From patchwork Thu Apr 7 20:25:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7581C433F5 for ; Thu, 7 Apr 2022 20:28:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Awg9xlusl+o+UoQ/X27SLNopx+Hit5KyPsIyX5AB03E=; b=1DZjMPPO8FsJPI 647CjsVISAH2ZUknuMepmXv+VS1WYDVKlwrjModcoLqxi6Ab3b21JaXf4bH7KrDriedSzYSRsfN74 1xtfsadwHHM9e3qylsVQGE0J3A+Kft9ywZLCYIO5tC0Lsm+gzNKyhBqMON/4ZF+P9cA96wuzOgFsE hYwfhooqX8qlRfwuxobFlmbtso2vdPqn5Psb9soCX+VMpZRiK8bxNsWePD3dCk9/d50xSmH81Y5vZ TqnKXHWvnDMrxxSdhegthUCDat+Q/DsNqJVOryxG9yOFDOFatUaiSw6kOyFR9T3RXsFoLoL/rjPk9 igFbwvPl1Ko/4kqJyuxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYj3-00Ds5U-9h; Thu, 07 Apr 2022 20:27:29 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhe-00DrVS-I7 for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:09 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 808C5201CBCD; Thu, 7 Apr 2022 13:26:00 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 808C5201CBCD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363161; bh=1to1sOnNu6e2ju/VN5XiFAGxrunN1V9CrmUnEq1yJ+8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=I3CwEQbs5kuhPZ53n3cmqFQsX0zQ/lHPagKtgUJiMCU78lhCfUqX5xlwX2AaCVaJ8 wYUiyfF154o0ksIQ+OkRBkqZKaVp04XTfed1e9FLIovGJaO9WzfUW+gk0y1ovD0X78 ufjmPgKcZY4ioqMDA7MrPtJDfOhGI7d2Gb5/ouvs= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 1/9] objtool: Parse DWARF Call Frame Information in object files Date: Thu, 7 Apr 2022 15:25:10 -0500 Message-Id: <20220407202518.19780-2-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132602_802315_5DF4C240 X-CRM114-Status: GOOD ( 44.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" If CONFIG_DEBUG_INFO_DWARF* is enabled, the compiler generates DWARF Call Frame Information (CFI) for every object file and places it in a special section named ".debug_frame". The CFI information can be used for frame pointer validation. Implement a CFI parser in objtool. This can be invoked as follows: objtool dwarf generate The version of the DWARF standard supported in this work is version 4. The official documentation for this version is here: https://dwarfstd.org/doc/DWARF4.pdf Section 6.4 contains the description of the CFI. Initial implementation ====================== This initial work is for supporting frame pointer validation for ARM64. That said, it is generic enough to support other architectures in the future. CFI defines 7 different register rules to compute register values at every instruction address. These are described below. Of these, only the first 3 rules are generated by gcc and clang for the stack pointer (SP), the frame pointer (FP) and the return address (RA) for ARM64. As of this writing, the same is true for RISCV as well. So, in objtool, provide support for the first three rules and mark the rest as unsupported. Register rules ============== CFI defines the Canonical Frame Address (CFA) as the value of the SP when a call instruction is executed. For the called function, other register values are expressed relative to the CFA. CFI defines the following rules to obtain the value of a register in the previous frame, given a current frame: 1. Same_Value: The current and previous values of the register are the same. 2. Val_Offset(N): The previous value is (CFA + N) where N is a signed offset. 3. Offset(N): The previous value is saved at (CFA + N). 4. register(R): The previous value is saved in register R. 5. Val_Expression(E): The previous value is the value produced by evaluating a given DWARF expression. DWARF expressions are evaluated on a stack. That is, operands are pushed and popped on a stack, DWARF operators are applied on them and the result is obtained. 6. Expression(E): The previous value is stored at the address computed from a DWARF expression. 7. Architectural: The previous value is obtained in an architecture-specific way via an architecture-specific "augmentor". Augmentors are vendor specific and are not part of the DWARF standard. DWARF rule encoding =================== The CFI is defined in a very generic format to allow all of the above rules to be defined for different architectures. Since this work uses only a minimal subset of the rules, the supported CFI rules can be encoded in a more compact format for the kernel. Provide stubs for generating compact rule data from the CFI rules. In a future patch, the stubs will be filled with actual code and the rules will be written to a special section in the object file. And, in a future patch, the kernel will use the rule data. Unsupported rules ================= For arm64, this is not an issue. If the compiler generates unsupported rules for the SP, FP and RA for some other architecture, objtool will not generate any rule data for those code locations. These will be treated as unreliable PCs from an unwinder perspective. FP prolog, epilog and leaf functions ==================================== This implementation recognizes these cases in the DWARF CFI. It does not generate any rule data unless the frame is completely setup. So, if an interrupt or an exception were to happen in the prolog, epilog or a function where the frame has not been set up for whatever reason, the kernel will recognize that and consider the code unreliable from an unwinder perspective. Assembly functions ================== DWARF CFI is generated by the compiler only for C functions. So, objtool will not generate any rule data for assembly functions. By default, the kernel will consider all assembly functions as unreliable from an unwinder perspective. DWARF annotations for assembly code can be used so CFI can be generated for assembly functions as well. However, DWARF annotations are a pain to maintain. So, we should never go down that path. Now, there are certain points in assembly code that we would like to unwind through reliably. Like interrupt and exception handlers. For these, unwind hints can be defined and placed at strategic points in assembly code. This will be done in a futurep patch. Unwind hints are a lot simpler and a lot easier to maintain than DWARF annotations. Generated code ============== Generated code will not have any DWARF rules. The kernel will consider such code unreliable from an unwinder perspective. Architecture-specific part =========================== The following pieces in this implementation are architecture-specific. Code must be provided for each architecture spearately. - DWARF register number to architecture register mapping - Relocation handling for rule data (relocation types are processor-specific) - ABI-specific checking of rules parsed by objtool. Only a small amount of architecture-specific code is required. Other aspects such as endianness and address size are handled in the generic code. Signed-off-by: Madhavan T. Venkataraman --- tools/objtool/Build | 5 + tools/objtool/Makefile | 10 +- tools/objtool/arch/arm64/Build | 2 + tools/objtool/arch/arm64/dwarf_arch.c | 114 ++++ tools/objtool/arch/arm64/dwarf_clang.c | 53 ++ .../arch/arm64/include/arch/dwarf_reg.h | 17 + tools/objtool/builtin-dwarf.c | 57 ++ tools/objtool/dwarf_op.c | 560 ++++++++++++++++++ tools/objtool/dwarf_parse.c | 294 +++++++++ tools/objtool/dwarf_rules.c | 37 ++ tools/objtool/dwarf_util.c | 280 +++++++++ tools/objtool/elf.c | 2 +- tools/objtool/include/objtool/builtin.h | 1 + tools/objtool/include/objtool/dwarf_def.h | 438 ++++++++++++++ tools/objtool/include/objtool/elf.h | 1 + tools/objtool/include/objtool/objtool.h | 1 + tools/objtool/objtool.c | 1 + tools/objtool/weak.c | 27 + 18 files changed, 1898 insertions(+), 2 deletions(-) create mode 100644 tools/objtool/arch/arm64/Build create mode 100644 tools/objtool/arch/arm64/dwarf_arch.c create mode 100644 tools/objtool/arch/arm64/dwarf_clang.c create mode 100644 tools/objtool/arch/arm64/include/arch/dwarf_reg.h create mode 100644 tools/objtool/builtin-dwarf.c create mode 100644 tools/objtool/dwarf_op.c create mode 100644 tools/objtool/dwarf_parse.c create mode 100644 tools/objtool/dwarf_rules.c create mode 100644 tools/objtool/dwarf_util.c create mode 100644 tools/objtool/include/objtool/dwarf_def.h diff --git a/tools/objtool/Build b/tools/objtool/Build index b7222d5cc7bc..2ab5885398c1 100644 --- a/tools/objtool/Build +++ b/tools/objtool/Build @@ -7,9 +7,14 @@ objtool-$(SUBCMD_CHECK) += special.o objtool-$(SUBCMD_ORC) += check.o objtool-$(SUBCMD_ORC) += orc_gen.o objtool-$(SUBCMD_ORC) += orc_dump.o +objtool-$(SUBCMD_DWARF) += dwarf_parse.o +objtool-$(SUBCMD_DWARF) += dwarf_op.o +objtool-$(SUBCMD_DWARF) += dwarf_rules.o +objtool-$(SUBCMD_DWARF) += dwarf_util.o objtool-y += builtin-check.o objtool-y += builtin-orc.o +objtool-y += builtin-dwarf.o objtool-y += elf.o objtool-y += objtool.o diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile index 92ce4fce7bc7..2bc84ac5515f 100644 --- a/tools/objtool/Makefile +++ b/tools/objtool/Makefile @@ -41,13 +41,21 @@ AWK = awk SUBCMD_CHECK := n SUBCMD_ORC := n +SUBCMD_DWARF := n ifeq ($(SRCARCH),x86) SUBCMD_CHECK := y SUBCMD_ORC := y endif -export SUBCMD_CHECK SUBCMD_ORC +ifeq ($(SRCARCH),arm64) + SUBCMD_DWARF := y +ifneq ($(LLVM),) + SUBCMD_DWARF_CLANG := y +endif +endif + +export SUBCMD_CHECK SUBCMD_ORC SUBCMD_DWARF SUBCMD_DWARF_CLANG export srctree OUTPUT CFLAGS SRCARCH AWK include $(srctree)/tools/build/Makefile.include diff --git a/tools/objtool/arch/arm64/Build b/tools/objtool/arch/arm64/Build new file mode 100644 index 000000000000..e5710de8060f --- /dev/null +++ b/tools/objtool/arch/arm64/Build @@ -0,0 +1,2 @@ +objtool-$(SUBCMD_DWARF) += dwarf_arch.o +objtool-$(SUBCMD_DWARF_CLANG) += dwarf_clang.o diff --git a/tools/objtool/arch/arm64/dwarf_arch.c b/tools/objtool/arch/arm64/dwarf_arch.c new file mode 100644 index 000000000000..2607ec94a12e --- /dev/null +++ b/tools/objtool/arch/arm64/dwarf_arch.c @@ -0,0 +1,114 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * dwarf_arch.c - Architecture-specific support functions. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ +#include +#include + +#include +#include +#include +#include +#include +#include + +int arch_dwarf_fde_reloc(struct fde *fde) +{ + GElf_Rela rela; + struct symbol *rela_symbol; + int index; + + /* + * Find the code section, offset within the section and the symbol of + * the function for this FDE. We need the symbol for debugging purposes. + * We need the section and offset to set up relocation for the DWARF + * rules that we will create in a separate section. + */ + if (debug_frame->reloc) { + /* + * In this case, debug frame entries are relocatable. For every + * FDE, there is a pair of relocation entries - one for the FDE + * itself and one for the function it represents. So, we use + * the second entry in each pair to find the section, section + * offset and the symbol for the function. + */ + index = (fde->index * 2) + 1; + if (!gelf_getrela(debug_frame->reloc->data, index, &rela)) { + WARN_ELF("gelf_getrela"); + return -ENOENT; + } + rela_symbol = find_symbol_by_index(dwarf_file->elf, + GELF_R_SYM(rela.r_info)); + fde->section = find_section_by_name(dwarf_file->elf, + rela_symbol->name); + if (!fde->section) { + WARN("No section for FDE"); + return -ENOENT; + } + fde->offset = rela.r_addend; + fde->symbol = find_symbol_containing(fde->section, fde->offset); + } else { + /* + * In this case, the debug frame entries are not relocatable. + * In the normal build of the kernel, this code is not required + * because objtool will be run on relocatable objects. But + * this code can handle it if objtool is run on the vmlinux + * binary itself. This is for debugging purposes. + */ + struct section *sec; + GElf_Shdr *sh; + unsigned long addr = fde->start_pc; + unsigned long start_addr, end_addr; + + fde->section = NULL; + for_each_sec(dwarf_file, sec) { + sh = &sec->sh; + start_addr = sh->sh_addr; + end_addr = start_addr + sh->sh_size; + if (addr >= start_addr && addr < end_addr) { + fde->section = sec; + break; + } + } + if (!fde->section) { + WARN("No section for FDE"); + return -ENOENT; + } + fde->offset = 0; + fde->symbol = find_symbol_containing(fde->section, + fde->start_pc); + } + fde->start_pc += fde->offset; + fde->end_pc += fde->offset; + return 0; +} + +/* + * If the offsets are 0, it means that the frame is not fully set up at + * this point in the object code. E.g., in the frame pointer prolog or epilog + * or in a leaf function. Check for this. + * + * If the frame is properly set up, then the frame pointer and return + * address are saved adjacent to each other. Check for this. + */ +int arch_dwarf_check_rules(struct fde *fde, unsigned long pc, + struct rule *sp_rule, struct rule *fp_rule, + struct rule *ra_rule) +{ + if (!sp_rule->offset || sp_rule->saved || + !fp_rule->offset || !fp_rule->saved) + return -EINVAL; + + if (!ra_rule->offset || !ra_rule->saved || + (fp_rule->offset + 8) != ra_rule->offset) + return -EINVAL; + + if (fde) + arch_dwarf_clang_hack(fde, pc, sp_rule, fp_rule); + + return 0; +} diff --git a/tools/objtool/arch/arm64/dwarf_clang.c b/tools/objtool/arch/arm64/dwarf_clang.c new file mode 100644 index 000000000000..e07ccf9484cf --- /dev/null +++ b/tools/objtool/arch/arm64/dwarf_clang.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * dwarf_arch.c - Architecture-specific support functions. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ +#include +#include + +#include +#include +#include +#include +#include +#include + +void arch_dwarf_clang_hack(struct fde *fde, unsigned long pc, + struct rule *sp_rule, struct rule *fp_rule) +{ + struct section *sec = fde->section; + unsigned long start_pc = 0; + unsigned int instruction; + unsigned int opcode, rd, rn, imm; + + if (!fde) + return; + + if (!sec->reloc) + start_pc = sec->sh.sh_addr; + + instruction = *(unsigned int *)(sec->data->d_buf + pc - start_pc - 4); + rd = instruction & 0x1F; + rn = (instruction >> 5) & 0x1F; + imm = (instruction >> 10) & 0x3FFF; + opcode = instruction >> 24; + + if (opcode == 0x91 && rn == SP_REG && rd == FP_REG && imm && + sp_rule->offset == -fp_rule->offset) { + fde->sp_offset = imm; + } + + imm = (instruction >> 10) & 0xFFF; + opcode = instruction >> 22; + + if (opcode == 0x344 && rn == SP_REG && rd == SP_REG && imm && + sp_rule->offset == -fp_rule->offset) { + fde->sp_offset = imm; + } + + sp_rule->offset += fde->sp_offset; +} diff --git a/tools/objtool/arch/arm64/include/arch/dwarf_reg.h b/tools/objtool/arch/arm64/include/arch/dwarf_reg.h new file mode 100644 index 000000000000..b3c61060bb67 --- /dev/null +++ b/tools/objtool/arch/arm64/include/arch/dwarf_reg.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * dwarf_reg.h - DWARF register numbers for the stack pointer, frame pointer + * and return address. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (c) 2022 Microsoft Corporation + */ +#ifndef _ARM64_ARCH_DWARF_REG_H +#define _ARM64_ARCH_DWARF_REG_H + +#define FP_REG 29 +#define RA_REG 30 +#define SP_REG 31 + +#endif /* _ARM64_ARCH_DWARF_REG_H */ diff --git a/tools/objtool/builtin-dwarf.c b/tools/objtool/builtin-dwarf.c new file mode 100644 index 000000000000..f44b35eb3f55 --- /dev/null +++ b/tools/objtool/builtin-dwarf.c @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * builtin-dwarf.c - DWARF command invoked by "objtool dwarf ...". + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +/* + * objtool dwarf: + * + * This command analyzes a .o file and adds .dwarf_rules and .dwarf_offsets + * sections to it, which is used by the in-kernel reliable unwinder. + */ + +#include +#include +#include +#include + +static const char * const dwarf_usage[] = { + /* + * Generate DWARF rules for the kernel from DWARF Call Frame + * information. + */ + "objtool dwarf generate file", + + NULL, +}; + +const struct option dwarf_options[] = { + OPT_END(), +}; + +int cmd_dwarf(int argc, const char **argv) +{ + const char *object; + struct objtool_file *file; + + argc--; argv++; + if (argc != 2) + usage_with_options(dwarf_usage, dwarf_options); + + object = argv[1]; + + file = objtool_open_read(object); + if (!file) + return 1; + + if (!strncmp(argv[0], "gen", 3)) + return dwarf_parse(file); + + usage_with_options(dwarf_usage, dwarf_options); + + return 0; +} diff --git a/tools/objtool/dwarf_op.c b/tools/objtool/dwarf_op.c new file mode 100644 index 000000000000..31f9e0b4fd4b --- /dev/null +++ b/tools/objtool/dwarf_op.c @@ -0,0 +1,560 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * dwarf_op.c - Code to parse DWARF operations in object files. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +#include + +#include +#include +#include +#include +#include + +unsigned char op, operand; + +static unsigned long cur_address; +static struct rule sp_rule; +static struct rule fp_rule; +static struct rule ra_rule; +static bool unsupported; + +static inline bool dwarf_ignore_reg(unsigned int reg) +{ + /* + * We don't care about registers other than the stack pointer, frame + * pointer and the return address (if defined). + */ + return reg != SP_REG && reg != FP_REG && reg != RA_REG; +} + +static inline void dwarf_offset_rule(unsigned int reg, long offset, bool saved) +{ + if (dwarf_ignore_reg(reg)) + return; + + if (reg == FP_REG && offset > 0) { + /* This is for Clang. */ + reg = SP_REG; + } + + if (reg == SP_REG) { + sp_rule.offset = offset; + sp_rule.saved = saved; + } else if (reg == FP_REG) { + fp_rule.offset = offset; + fp_rule.saved = saved; + } else { + ra_rule.offset = offset; + ra_rule.saved = saved; + } +} + +static inline void dwarf_reg_rule(unsigned int reg, unsigned int other_reg) +{ + if (dwarf_ignore_reg(reg) && dwarf_ignore_reg(other_reg)) + return; + + /* + * We don't support the SP, FP and RA being saved in other registers + * and vice-versa. + */ + WARN("op=%d register rule for %d is not supported", op, reg); + unsupported = true; +} + +static inline void dwarf_restore_rule(unsigned int reg) +{ + if (dwarf_ignore_reg(reg)) + return; + + if (reg == SP_REG) + sp_rule = cur_cie->sp_rule; + else if (reg == FP_REG) + fp_rule = cur_cie->fp_rule; + else + ra_rule = cur_cie->ra_rule; +} + +static inline void dwarf_expression_rule(unsigned int reg) +{ + if (dwarf_ignore_reg(reg)) + return; + + /* + * We don't support expressions to compute the values for the SP, FP + * and RA. + */ + WARN("op=%d is not supported!", op); + unsupported = true; +} + +static inline void dwarf_undefined_rule(unsigned int reg) +{ + if (dwarf_ignore_reg(reg)) + return; + + /* + * We don't support the values for the SP, FP and RA being undefined. + */ + WARN("op=%d %d register undefined?", op, reg); + unsupported = true; +} + +/* + * Whenever the PC is changed, the SP and FP rules for the range (old PC to + * new PC) have to be written out in the form of a DWARF rule before + * changing the PC. + */ +static void dwarf_set_address(unsigned long address, bool *fail) +{ + bool skip = false; + + if (unsupported) { + /* + * We were not able to compute the rules. Reset the rules. + * Do not generate a DWARF rule for this code range. The + * kernel will therefore not be able to find the rules for + * the code range and will consider the range to be unreliable + * from an unwind perspective. + */ + unsupported = false; + dwarf_offset_rule(SP_REG, 0, false); + dwarf_offset_rule(FP_REG, 0, false); + dwarf_offset_rule(RA_REG, 0, false); + skip = true; + } + + if (arch_dwarf_check_rules(cur_fde, cur_address, + &sp_rule, &fp_rule, &ra_rule)) { + /* + * Ignore the rules for now. Do not generate a DWARF rule + * for this code range. The kernel will therefore not be able + * to find the rules for the code range and will consider the + * range to be unreliable from an unwind perspective. We do + * not reset the rules as they can get modified by further + * DWARF instruction processing to the point where they are + * not ignored anymore. + */ + skip = true; + } + + if (skip) + dwarf_rule_next(cur_fde, cur_address); + else if (dwarf_rule_add(cur_fde, cur_address, &sp_rule, &fp_rule)) + *fail = true; + cur_address = address; +} + +static unsigned char *dwarf_one_op(unsigned char *start, unsigned char *end) +{ + unsigned long length; + unsigned char byte; + unsigned int delta; + u64 pc, reg, other_reg; + s64 offset; + bool fail = false; + + byte = *start++; + + /* Primary op code in the high 2 bits */ + op = byte & 0xc0; + operand = byte & 0x3F; + if (op == DW_CFA_extended_op) { + /* Extended op is in the low 6 bits. */ + op = operand; + } + + switch (op) { + /* + * No-op instruction, used for padding. + */ + case DW_CFA_nop: + break; + + /* + * Instructions that advance the current PC. + */ + case DW_CFA_advance_loc: + /* + * The factored delta is the operand itself. Add the delta to + * the current instruction address to obtain the new + * instruction address. + */ + delta = (unsigned int) operand; + delta *= cur_cie->code_factor; + dwarf_set_address(cur_address + delta, &fail); + break; + case DW_CFA_set_loc: + /* + * Extract the target PC. Set the current instruction address + * to it. + */ + if (cur_cie->address_size == 64) + GET_VALUE(pc, start, end, 8); + else + GET_VALUE(pc, start, end, 4); + dwarf_set_address(pc, &fail); + break; + case DW_CFA_advance_loc1: + /* + * Extract the factored delta (unsigned byte) and add it to the + * current instruction address to obtain the new instruction + * address. + */ + GET_VALUE(delta, start, end, 1); + delta *= cur_cie->code_factor; + dwarf_set_address(cur_address + delta, &fail); + break; + case DW_CFA_advance_loc2: + /* + * Extract the factored delta (unsigned short) and add it to + * the current instruction address to obtain the new + * instruction address. + */ + GET_VALUE(delta, start, end, 2); + delta *= cur_cie->code_factor; + dwarf_set_address(cur_address + delta, &fail); + break; + case DW_CFA_advance_loc4: + /* + * Extract the factored delta (unsigned int) and add it to + * the current instruction address to obtain the new + * instruction address. + */ + GET_VALUE(delta, start, end, 4); + delta *= cur_cie->code_factor; + dwarf_set_address(cur_address + delta, &fail); + break; + + /* + * CFA definition instructions. + */ + case DW_CFA_def_cfa: + /* + * Extract the CFA register and the unfactored CFA offset. + * Define a val_offset(N) rule. + */ + READ_ULEB_128(reg, start, end, fail); + READ_ULEB_128(offset, start, end, fail); + cur_cie->cfa_reg = reg; + cur_cie->cfa_offset = offset; + dwarf_offset_rule(cur_cie->cfa_reg, cur_cie->cfa_offset, false); + break; + case DW_CFA_def_cfa_sf: + /* + * Same as DW_CFA_def_cfa except that the offset is signed + * and factored. + */ + READ_ULEB_128(reg, start, end, fail); + READ_SLEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + cur_cie->cfa_reg = reg; + cur_cie->cfa_offset = offset; + dwarf_offset_rule(cur_cie->cfa_reg, cur_cie->cfa_offset, false); + break; + case DW_CFA_def_cfa_register: + /* + * Same as DW_CFA_def_cfa except that the saved offset is used. + */ + READ_ULEB_128(reg, start, end, fail); + cur_cie->cfa_reg = reg; + dwarf_offset_rule(cur_cie->cfa_reg, cur_cie->cfa_offset, false); + break; + case DW_CFA_def_cfa_offset: + /* + * Same as DW_CFA_def_cfa except that the saved register is + * used. + */ + READ_ULEB_128(offset, start, end, fail); + cur_cie->cfa_offset = offset; + dwarf_offset_rule(cur_cie->cfa_reg, cur_cie->cfa_offset, false); + break; + case DW_CFA_def_cfa_offset_sf: + /* + * Same as DW_CFA_def_cfa_offset except that the offset is + * signed and factored. + */ + READ_SLEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + cur_cie->cfa_offset = offset; + dwarf_offset_rule(cur_cie->cfa_reg, cur_cie->cfa_offset, false); + break; + case DW_CFA_def_cfa_expression: + READ_ULEB_128(length, start, end, fail); + /* + * Skip the expression bytes. + */ + start += length; + if (start > end) + fail = true; + dwarf_expression_rule(SP_REG); + break; + + /* + * Register rule instructions. + */ + case DW_CFA_undefined: + READ_ULEB_128(reg, start, end, fail); + dwarf_undefined_rule(reg); + break; + case DW_CFA_same_value: + /* + * Set the register offset to be "same value". That is, it has + * not been modified by the callee. + */ + READ_ULEB_128(reg, start, end, fail); + dwarf_offset_rule(reg, 0, false); + break; + case DW_CFA_offset: + /* + * The register number that is encoded in the operand itself. + * Extract the factored offset. Define an offset(N) rule. + */ + reg = operand; + READ_ULEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + dwarf_offset_rule(reg, offset, true); + break; + case DW_CFA_offset_extended: + /* + * Same as DW_CFA_offset except for the encoding and size of + * the register operand. + */ + READ_ULEB_128(reg, start, end, fail); + READ_ULEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + dwarf_offset_rule(reg, offset, true); + break; + case DW_CFA_offset_extended_sf: + /* + * Same as DW_CFA_offset_extended except that the offset is + * signed and factored. + */ + READ_ULEB_128(reg, start, end, fail); + READ_SLEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + dwarf_offset_rule(reg, offset, true); + break; + case DW_CFA_val_offset: + /* + * Extract the register number and the factored offset. Define + * a val_offset(N) rule. + */ + READ_ULEB_128(reg, start, end, fail); + READ_ULEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + dwarf_offset_rule(reg, offset, false); + break; + case DW_CFA_val_offset_sf: + /* + * Same as DW_CFA_val_offset except that the offset is signed. + */ + READ_ULEB_128(reg, start, end, fail); + READ_SLEB_128(offset, start, end, fail); + offset *= cur_cie->data_factor; + dwarf_offset_rule(reg, offset, false); + break; + case DW_CFA_register: + READ_ULEB_128(reg, start, end, fail); + READ_ULEB_128(other_reg, start, end, fail); + dwarf_reg_rule(reg, other_reg); + break; + case DW_CFA_expression: + case DW_CFA_val_expression: + READ_ULEB_128(reg, start, end, fail); + READ_ULEB_128(length, start, end, fail); + /* + * Skip the expression bytes. + */ + start += length; + if (start > end) + fail = true; + dwarf_expression_rule(reg); + break; + case DW_CFA_restore: + /* + * Restore the rule for the register to the one specified in + * the CIE. + */ + reg = operand; + dwarf_restore_rule(reg); + break; + case DW_CFA_restore_extended: + /* + * Same as DW_CFA_restore except for the encoding and size of + * the register operand. + */ + READ_ULEB_128(reg, start, end, fail); + dwarf_restore_rule(reg); + break; + + /* + * Rule state instructions. + */ + case DW_CFA_remember_state: + cur_cie->saved_sp_rule = sp_rule; + cur_cie->saved_fp_rule = fp_rule; + cur_cie->saved_ra_rule = ra_rule; + break; + case DW_CFA_restore_state: + sp_rule = cur_cie->saved_sp_rule; + fp_rule = cur_cie->saved_fp_rule; + ra_rule = cur_cie->saved_ra_rule; + break; + default: + if (op >= DW_CFA_lo_user && op <= DW_CFA_hi_user) { + /* + * Ignore arch-specific or vendor-specific ops as they + * are irrelevant to the stack and frame pointers. + */ + } else { + WARN("Illegal CFA op %d", (int) op); + fail = true; + } + break; + } + return fail ? NULL : start; +} + +/* + * Run the DWARF instructions in a CIE or an FDE. + */ +static unsigned char *dwarf_op(unsigned char *start, unsigned char *end) +{ + bool fail = false; + + /* cur_fde is set if this is an FDE. */ + if (cur_fde) { + /* + * For an FDE, the rules are initialized from the rules + * computed for its CIE. + */ + sp_rule = cur_cie->sp_rule; + fp_rule = cur_cie->fp_rule; + ra_rule = cur_cie->ra_rule; + } else { + /* + * For a CIE, the rule is initialized to all zeroes. + */ + dwarf_offset_rule(SP_REG, 0, false); + dwarf_offset_rule(FP_REG, 0, false); + dwarf_offset_rule(RA_REG, 0, false); + } + + /* + * If an unsupported DWARF rule is discovered in dwarf_one_op(), + * this will be set to true. + */ + unsupported = false; + + while (start < end) { + start = dwarf_one_op(start, end); + if (!start) + return NULL; + } + + if (cur_fde) { + /* Generate the final rule for the FDE. */ + dwarf_set_address(cur_fde->end_pc, &fail); + } + return start; +} + +/* + * Parse DWARF instructions in CIEs and FDEs. + */ +void dwarf_parse_instructions(void) +{ + struct cie *cie; + struct fde *fde; + unsigned char *start, *end; + + cur_fde = NULL; + + for (cie = cies; cie != NULL; cie = cie->next) { + cur_cie = cie; + start = cie->instructions; + end = start + cie->instructions_size; + + /* + * Run the DWARF instructions in the CIE to compute the + * initial SP, FP and RA rules. + */ + if (!dwarf_op(start, end)) { + cur_cie->unusable = true; + continue; + } + + cie->sp_rule = sp_rule; + cie->fp_rule = fp_rule; + cie->ra_rule = ra_rule; + } + + for (fde = fdes; fde != NULL; fde = fde->next) { + /* + * If any problems are encountered below, simply skip the FDE. + * This means that no DWARF rules from this FDE will be + * included. So, the kernel will consider the FDE's code range + * to be unreliable from an unwinding perspective. + */ + + /* + * Find the CIE for this FDE using the section offset of the + * CIE. The CIE list is already in increasing section offset + * order. + */ + for (cie = cies; cie != NULL; cie = cie->next) { + if (cie->offset < fde->cie_offset) + continue; + if (cie->offset == fde->cie_offset) + fde->cie = cie; + break; + } + + cur_cie = fde->cie; + if (cur_cie == NULL || cur_cie->unusable) { + WARN("No CIE: Could not process FDE"); + continue; + } + + if (!fde->section) { + /* + * The section is needed to create relocation entries + * for the DWARF rules in the FDE. + */ + WARN("No section: Could not process FDE"); + continue; + } + cur_fde = fde; + + /* + * Run the DWARF instructions in the FDE to derive the rules + * for computing the SP and the FP within the FDE code range. + * Encode the rules in the form of DWARF rules for the benefit + * of the kernel. dwarf_op() will generate the rules as it runs + * the instructions. + */ + dwarf_rule_start(fde); + + cur_address = fde->start_pc; + start = fde->instructions; + end = start + fde->instructions_size; + + if (!dwarf_op(start, end)) { + /* + * Rollback the DWARF rules created in the above + * call. + */ + WARN("FDE instructions failed. Rolling back FDE."); + dwarf_rule_reset(fde); + continue; + } + + dwarf_rule_next(fde, fde->end_pc); + } +} diff --git a/tools/objtool/dwarf_parse.c b/tools/objtool/dwarf_parse.c new file mode 100644 index 000000000000..d5ac5630fbba --- /dev/null +++ b/tools/objtool/dwarf_parse.c @@ -0,0 +1,294 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * dwarf_parse.c - Code to parse DWARF information in object files. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +#include +#include + +#include +#include +#include +#include + +struct objtool_file *dwarf_file; +struct section *debug_frame; + +struct cie *cies, *cur_cie; +struct fde *fdes, *cur_fde; + +static struct cie *cies_tail; +static struct fde *fdes_tail; + +static u64 cie_id; +static int fde_index; +static int address_size; +static unsigned int offset_size; +static u64 entry_length; +static unsigned char *saved_start; + +/* + * Parse and create a new CIE. + */ +static unsigned char *dwarf_parse_cie(unsigned char *start, unsigned char *end) +{ + struct cie *cie; + bool fail = false; + + cie = dwarf_alloc(sizeof(*cie)); + if (!cie) { + WARN("%s: dwarf_alloc(cie) failed", __func__); + return NULL; + } + memset(cie, 0, sizeof(*cie)); + + /* Add CIE to global list. */ + if (cies_tail == NULL) + cies = cie; + else + cies_tail->next = cie; + cies_tail = cie; + cie->next = NULL; + + /* Section offset where this CIE resides. */ + cie->offset = saved_start - (unsigned char *) debug_frame->data->d_buf; + cie->length = entry_length; + cie->id = cie_id; + cie->unusable = false; + + /* + * Extract the DWARF CFI version. This is different from the DWARF + * version. + */ + cie->version = *start++; + if (cie->version > 4) { + /* + * This implementation does not support these versions. For + * instance, segment selectors are not supported. + */ + WARN("CIE version %d is not supported", cie->version); + return NULL; + } + + /* + * Store the size of an address in this architecture. + */ + if (cie->version == 4) { + GET_VALUE(cie->address_size, start, end, 1); + GET_VALUE(cie->segment_size, start, end, 1); + cie->address_size += cie->segment_size; + } else { + cie->address_size = address_size; + } + + /* + * This implementation does not support an augmentor. For instance, + * the address_size can be modified by the augmentor. Make sure that + * the augmentor is the null string. + */ + cie->augmentation = (char *) start; + if (*start++ != '\0') { + WARN("Augmentor is not supported"); + return NULL; + } + cie->segment_size = 0; + + /* Extract code alignment factor. */ + READ_ULEB_128(cie->code_factor, start, end, fail); + + /* Extract data alignment factor. */ + READ_SLEB_128(cie->data_factor, start, end, fail); + + if (cie->version == 1) + GET_VALUE(cie->return_address_reg, start, end, 1); + else + READ_ULEB_128(cie->return_address_reg, start, end, fail); + + /* The remaining bytes are DWARF instructions. */ + cie->instructions = start; + cie->instructions_size = end - start; + start = end; + + return fail ? NULL : start; +} + +/* + * Parse and create a new FDE. + */ +static unsigned char *dwarf_parse_fde(unsigned char *start, unsigned char *end) +{ + struct fde *fde; + unsigned long length; + + fde = dwarf_alloc(sizeof(*fde)); + if (!fde) { + WARN("%s: dwarf_alloc(fde) failed", __func__); + return NULL; + } + memset(fde, 0, sizeof(*fde)); + + /* Add FDE to global list. */ + if (fdes_tail == NULL) + fdes = fde; + else + fdes_tail->next = fde; + fdes_tail = fde; + fde->next = NULL; + + /* + * This is the index of the FDE record in the .debug_frame section. + * This is used to locate the symbol for the FDE in a relocatable + * object file. + */ + fde->index = fde_index++; + fde->length = entry_length; + + /* + * For an FDE, the CIE ID field actually contains the section offset + * of the CIE for the FDE. + */ + fde->cie_offset = cie_id; + + /* + * Set the CIE for this FDE to NULL for now. We will set this to the + * correct CIE later. + */ + fde->cie = NULL; + fde->segment_selector = 0; + + /* + * Extract the starting address of the code range to which this FDE + * applies. + */ + GET_VALUE(fde->start_pc, start, end, address_size); + + /* Extract the size of the code range to which this FDE applies. */ + GET_VALUE(length, start, end, address_size); + fde->end_pc = fde->start_pc + length; + + /* The remaining bytes are DWARF instructions. */ + fde->instructions = start; + fde->instructions_size = end - start; + start = end; + + /* Relocation is arch-specific. */ + if (arch_dwarf_fde_reloc(fde) == -EOPNOTSUPP) + return NULL; + + return start; +} + +/* + * Parse one entry for an FDE - either a CIE or an FDE. + */ +static unsigned char *dwarf_parse_one(unsigned char *start, unsigned char *end) +{ + bool is_cie; + + saved_start = start; + + /* + * The first value in an entry is the length field. + */ + GET_VALUE(entry_length, start, end, 4); + if (entry_length == 0) { + WARN("Illegal length in DWARF entry"); + return NULL; + } + + /* + * For 64 bit entries, the first 32 bits of the entry is all 1's. The + * actual length is after that. + */ + if (entry_length == 0xffffffff) { + GET_VALUE(entry_length, start, end, 8); + offset_size = 8; + } else { + offset_size = 4; + } + + if (entry_length > (size_t) (end - start)) { + WARN("DWARF entry is too big"); + return NULL; + } + end = start + entry_length; + + /* + * The CIE identifier field distinguishes between a CIE and an FDE. For + * a CIE, this field contains a specific ID value. For an FDE, it + * contains the section offset of the CIE used by the FDE. + */ + GET_VALUE(cie_id, start, end, offset_size); + + is_cie = (offset_size == 4 && cie_id == CIE_ID_32) || + (offset_size == 8 && cie_id == CIE_ID_64); + if (is_cie) + start = dwarf_parse_cie(start, end); + else + start = dwarf_parse_fde(start, end); + return start; +} + +/* + * Parse DWARF Call Frame Information. + */ +int dwarf_parse(struct objtool_file *file) +{ + unsigned char *start, *end; + + dwarf_file = file; + + /* + * Initialize the helper function based on endianness. This function + * is used to extract values from the DWARF section. + */ + switch (file->elf->ehdr.e_ident[EI_DATA]) { + default: + __fallthrough; + case ELFDATANONE: + __fallthrough; + case ELFDATA2LSB: + get_value = get_value_le; /* Little endian */ + break; + case ELFDATA2MSB: + get_value = get_value_be; /* Big endian */ + break; + } + + if (file->elf->ehdr.e_ident[EI_CLASS] == ELFCLASS64) + address_size = 64; + else + address_size = 32; + + /* + * DWARF Call Frame Information is contained in .debug_frame. + * NOTE: This implementation does not support .eh_frame. + */ + debug_frame = find_section_by_name(file->elf, ".debug_frame"); + if (!debug_frame) + return 0; + + dwarf_alloc_init(); + + /* + * Parse all the entries in .debug_frame and create CIEs and FDEs. + */ + start = debug_frame->data->d_buf; + end = start + debug_frame->data->d_size; + + while (start < end) { + start = dwarf_parse_one(start, end); + if (!start) + return -1; + } + + /* + * Run all the DWARF instructions in the CIEs and FDEs. + */ + dwarf_parse_instructions(); + return 0; +} diff --git a/tools/objtool/dwarf_rules.c b/tools/objtool/dwarf_rules.c new file mode 100644 index 000000000000..9cf201de392a --- /dev/null +++ b/tools/objtool/dwarf_rules.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * dwarf_rules.c - Allocation and management of DWARF rules. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ +#include + +#include +#include +#include +#include + +/* + * The following are stubs for now. Later, they will be filled to create + * DWARF rules that the kernel can use to compute the frame pointer at + * a given instruction address. + */ +void dwarf_rule_start(struct fde *fde) +{ +} + +int dwarf_rule_add(struct fde *fde, unsigned long addr, + struct rule *sp_rule, struct rule *fp_rule) +{ + return 0; +} + +void dwarf_rule_next(struct fde *fde, unsigned long addr) +{ +} + +void dwarf_rule_reset(struct fde *fde) +{ +} diff --git a/tools/objtool/dwarf_util.c b/tools/objtool/dwarf_util.c new file mode 100644 index 000000000000..77c70c54d26f --- /dev/null +++ b/tools/objtool/dwarf_util.c @@ -0,0 +1,280 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * dwarf_util.c - Support functions. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ +#include +#include +#include +#include + +void *free_space; +size_t free_size; + +/* Function to extract embedded values in the DWARF instruction stream. */ +u64 (*get_value)(unsigned char *field, unsigned int size); + +/* + * Little endian helper. Adapted from binutils. + */ +u64 get_value_le(unsigned char *field, unsigned int size) +{ + switch (size) { + case 1: + return *field; + + case 2: + return ((unsigned int) (field[0])) | + (((unsigned int) (field[1])) << 8); + + case 3: + return ((unsigned long) (field[0])) | + (((unsigned long) (field[1])) << 8) | + (((unsigned long) (field[2])) << 16); + + case 4: + return ((unsigned long) (field[0])) | + (((unsigned long) (field[1])) << 8) | + (((unsigned long) (field[2])) << 16) | + (((unsigned long) (field[3])) << 24); + + case 5: + if (sizeof(u64) >= 8) { + return ((u64) (field[0])) | + (((u64) (field[1])) << 8) | + (((u64) (field[2])) << 16) | + (((u64) (field[3])) << 24) | + (((u64) (field[4])) << 32); + } + __fallthrough; + + case 6: + if (sizeof(u64) >= 8) { + return ((u64) (field[0])) | + (((u64) (field[1])) << 8) | + (((u64) (field[2])) << 16) | + (((u64) (field[3])) << 24) | + (((u64) (field[4])) << 32) | + (((u64) (field[5])) << 40); + } + __fallthrough; + + case 7: + if (sizeof(u64) >= 8) { + return ((u64) (field[0])) | + (((u64) (field[1])) << 8) | + (((u64) (field[2])) << 16) | + (((u64) (field[3])) << 24) | + (((u64) (field[4])) << 32) | + (((u64) (field[5])) << 40) | + (((u64) (field[6])) << 48); + } + __fallthrough; + + case 8: + if (sizeof(u64) >= 8) { + return ((u64) (field[0])) | + (((u64) (field[1])) << 8) | + (((u64) (field[2])) << 16) | + (((u64) (field[3])) << 24) | + (((u64) (field[4])) << 32) | + (((u64) (field[5])) << 40) | + (((u64) (field[6])) << 48) | + (((u64) (field[7])) << 56); + } + __fallthrough; + + default: + WARN("%s: Unhandled data length: %d\n", __func__, size); + } + return 0; +} + +/* + * Big endian helper. Adapted from binutils. + */ +u64 get_value_be(unsigned char *field, unsigned int size) +{ + switch (size) { + case 1: + return *field; + + case 2: + return ((unsigned int) (field[1])) | + (((int) (field[0])) << 8); + + case 3: + return ((unsigned long) (field[2])) | + (((unsigned long) (field[1])) << 8) | + (((unsigned long) (field[0])) << 16); + + case 4: + return ((unsigned long) (field[3])) | + (((unsigned long) (field[2])) << 8) | + (((unsigned long) (field[1])) << 16) | + (((unsigned long) (field[0])) << 24); + + case 5: + if (sizeof(u64) >= 8) { + return ((u64) (field[4])) | + (((u64) (field[3])) << 8) | + (((u64) (field[2])) << 16) | + (((u64) (field[1])) << 24) | + (((u64) (field[0])) << 32); + } + __fallthrough; + + case 6: + if (sizeof(u64) >= 8) { + return ((u64) (field[5])) | + (((u64) (field[4])) << 8) | + (((u64) (field[3])) << 16) | + (((u64) (field[2])) << 24) | + (((u64) (field[1])) << 32) | + (((u64) (field[0])) << 40); + } + __fallthrough; + + case 7: + if (sizeof(u64) >= 8) { + return ((u64) (field[6])) | + (((u64) (field[5])) << 8) | + (((u64) (field[4])) << 16) | + (((u64) (field[3])) << 24) | + (((u64) (field[2])) << 32) | + (((u64) (field[1])) << 40) | + (((u64) (field[0])) << 48); + } + __fallthrough; + + case 8: + if (sizeof(u64) >= 8) { + return ((u64) (field[7])) | + (((u64) (field[6])) << 8) | + (((u64) (field[5])) << 16) | + (((u64) (field[4])) << 24) | + (((u64) (field[3])) << 32) | + (((u64) (field[2])) << 40) | + (((u64) (field[1])) << 48) | + (((u64) (field[0])) << 56); + } + __fallthrough; + + default: + WARN("%s: Unhandled data length: %d\n", __func__, size); + } + return 0; +} + +/* + * LEB 128 read functions adapted from LLVM code. + */ +u64 read_uleb_128(unsigned char *start, unsigned char *end, + unsigned int *num_read, bool *fail) +{ + unsigned char *cur = start, byte; + unsigned int shift = 0; + u64 value = 0; + u64 slice; + + do { + if (cur == end) { + WARN("%s: op=%d end of data", __func__, op); + *num_read = (unsigned int) (cur - start); + *fail = true; + return 0; + } + + byte = *cur++; + slice = byte & 0x7f; + + if ((shift >= 64 && slice != 0) || + (slice << shift >> shift) != slice) { + WARN("%s: op=%d value too large", __func__, op); + *num_read = (unsigned int) (cur - start); + *fail = true; + return 0; + } + + value += slice << shift; + shift += 7; + } while (byte >= 128); + + *num_read = (unsigned int) (cur - start); + + return value; +} + +s64 read_sleb_128(unsigned char *start, unsigned char *end, + unsigned int *num_read, bool *fail) +{ + unsigned char *cur = start, byte; + unsigned int shift = 0; + s64 value = 0; + u64 slice; + + do { + if (cur == end) { + WARN("%s: op=%d end of data", __func__, op); + *num_read = (unsigned int) (cur - start); + *fail = true; + return 0; + } + + byte = *cur++; + slice = byte & 0x7f; + + if ((shift >= 64 && slice != (value < 0 ? 0x7f : 0x00)) || + (shift == 63 && slice != 0 && slice != 0x7f)) { + WARN("%s: op=%d value too large", __func__, op); + *num_read = (unsigned int) (cur - start); + *fail = true; + return 0; + } + + value |= slice << shift; + shift += 7; + } while (byte >= 128); + + if (shift < 64 && (byte & 0x40)) { + /* Sign extend negative numbers if needed. */ + value |= (-1ULL) << shift; + } + + *num_read = (unsigned int) (cur - start); + + return value; +} + +void dwarf_alloc_init(void) +{ + /* + * Use the size of the .debug_frame section as an estimate of the + * memory we need. + */ + free_size = debug_frame->data->d_size * 4; + free_space = malloc(free_size); + if (!free_space) { + WARN("%s: Could not optimize allocations", __func__); + free_size = 0; + } +} + +void *dwarf_alloc(size_t size) +{ + void *buf; + + /* Round to 8 bytes. */ + size = (size + 7) & ~7UL; + + if (free_size >= size) { + buf = free_space; + free_space += size; + free_size -= size; + return buf; + } + return malloc(size); +} diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c index 4b384c907027..85606a19f633 100644 --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -108,7 +108,7 @@ static struct section *find_section_by_index(struct elf *elf, return NULL; } -static struct symbol *find_symbol_by_index(struct elf *elf, unsigned int idx) +struct symbol *find_symbol_by_index(struct elf *elf, unsigned int idx) { struct symbol *sym; diff --git a/tools/objtool/include/objtool/builtin.h b/tools/objtool/include/objtool/builtin.h index 15ac0b7d3d6a..02b18149cdd7 100644 --- a/tools/objtool/include/objtool/builtin.h +++ b/tools/objtool/include/objtool/builtin.h @@ -15,5 +15,6 @@ extern int cmd_parse_options(int argc, const char **argv, const char * const usa extern int cmd_check(int argc, const char **argv); extern int cmd_orc(int argc, const char **argv); +extern int cmd_dwarf(int argc, const char **argv); #endif /* _BUILTIN_H */ diff --git a/tools/objtool/include/objtool/dwarf_def.h b/tools/objtool/include/objtool/dwarf_def.h new file mode 100644 index 000000000000..7a0a18480d2b --- /dev/null +++ b/tools/objtool/include/objtool/dwarf_def.h @@ -0,0 +1,438 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * dwarf_def.h - DWARF definitions for parsing DWARF information. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (C) 2022 Microsoft Corporation + */ + +#ifndef _OBJTOOL_DWARF_DEF_H +#define _OBJTOOL_DWARF_DEF_H + +/* + * The DWARF Call Frame Information (CFI) is encoded in a self-contained + * section called .debug_frame. + * + * DWARF CFI defines the Canonical Frame Address (CFA) as the value of the + * stack pointer (SP) when a call instruction is executed. For the called + * function, other register values are expressed relative to the CFA. For + * a given code location within the function, one can compute things like: + * + * - what is the offset that must be added to the current stack pointer + * to get the CFA. + * + * - what offset must be subtracted from the CFA to obtain the current + * frame pointer (FP). + * + * - what offset must be subtracted from the CFA to obtain the location + * on the stack where a register value is saved. E.g., the return + * address (RA). + * + * - what register is saved in what register. + * + * - etc. + * + * This allows the unwinding of the stack. The unwinder starts at the top most + * frame and gets the value of the SP, FP and RA from the current register + * state. Using the DWARF CFI at the RA, the unwinder computes the values of + * the SP, FP and RA in the previous frame. This process continues until the + * SP hits the bottom of the stack and the unwinding terminates. + * + * In this work, the DWARF CFI is not used to build an unwinder. The existing + * frame pointer based unwinder is retained. But the CFI is used to compute + * an FP at every frame to validate the actual FP. If the computed and actual + * FPs match, then the stack frame is considered reliable. Otherwise, it is + * considered unreliable. If all of the frames in a stack trace are reliable, + * then the stack trace is reliable. + * + * Entries in a .debug_frame section are aligned on a multiple of the address + * size relative to the start of the section and come in two forms: + * + * - a Common Information Entry (CIE) and + * + * - a Frame Description Entry (FDE). + * + * A Common Information Entry holds information that is shared among many + * Frame Description Entries. So, it is like a header. There is at least one + * CIE in every non-empty .debug_frame section. + * + * The CIE contains information such as the size of an address in the + * architecture, the number of the return address register, etc. It also + * contains DWARF instructions to initialize unwind state at the start of + * an FDE. + * + * The FDE contains a code range to which it applies. It also contains DWARF + * instructions. These instructions are used to obtain the register rules + * at each code location. Using these rules, the SP, FP and RA are computed + * as mentioned above. + */ + +/* + * DWARF CFI defines the following rules to obtain the value of a register + * in the previous frame, given a current frame: + * + * 1. Same_Value: + * + * The current and previous values of the register are the same. + * + * 2. Val_Offset(N): + * + * The previous value is (CFA + N) where N is a signed offset. + * + * 3. Offset(N): + * + * The previous value is saved at (CFA + N). + * + * 4. register(R): + * + * The previous value is saved in register R. + * + * 5. Val_Expression(E): + * + * The previous value is the value produced by evaluating a given + * DWARF expression. DWARF expressions are evaluated on a stack. That + * is, operands are pushed and popped on a stack, operators are + * applied on them and the result is obtained. + * + * 6. Expression(E): + * + * The previous value is stored at the address computed from a DWARF + * expression. + * + * 7. Architectural: + * + * The previous value is obtained in an architecture-specific way via + * an architecture-specific "augmentor". The augmentors are vendor + * specific and are not part of the DWARF standard. + * + * Now, all of this is quite complicated. In this work, only (1), (2) and (3) + * will be supported. At the time of this writing, these are found to be + * sufficient for ARM64 and RISCV. Other architectures have not been checked. + * + * The code locations at which unsupported rules exist will be treated as + * unreliable from an unwinder perspective. In other words, if the RA of a + * stack frame is such a code location, then that frame is unreliable. + */ + +/* + * The following structure is used to encode the Same_Value, Offset(N) and + * the Val_Offset(N) rules. + * + * offset = 0, saved = true Never happens + * offset = 0, saved = false Same_Value + * offset = N, saved = true Offset(N) + * offset = N, saved = false Val_Offset(N) + */ +struct rule { + long offset; + bool saved; +}; + +/* + * Common Information Entry (CIE): + * + * next + * Next CIE in list. + * + * offset + * Section offset at which this CIE is found. + * + * length + * A constant that gives the number of bytes of the CIE structure, not + * including the length field itself. The size of the length field plus + * the value of length must be an integral multiple of the address size. + * + * id + * A constant that is used to distinguish CIEs from FDEs. + * + * version + * A version number. This number is specific to the call frame information + * and is independent of the DWARF version number. Versions 4 and above + * are not supported. + * + * address_size + * The size of a target address in this CIE and any FDEs that use it, + * in bytes. + * + * segment_size + * The size of a segment selector in this CIE and any FDEs that use it, + * in bytes. + * + * augmentation + * A null-terminated UTF-8 string that identifies the augmentation to + * this CIE or to the FDEs that use it. An augmentation is specified by + * an architecture to compute values in a way that is specific to that + * architecture. No augmentation is supported. + * + * code_factor + * A constant that is factored out of all advance location instructions. + * + * data_factor + * A constant that is factored out of certain offset instructions. + * + * return_address_reg + * The number of the return address register in the architecture. + * + * instructions + * DWARF instructions to initialize the unwind state at the start of + * an FDE. + * + * instructions_size + * Number of bytes of instructions. + * + * sp_rule + * Initial rule to compute the CFA derived from the above instructions. + * + * fp_rule + * Initial rule to compute the frame pointer derived from the above + * instructions. + * + * ra_rule + * Initial rule to compute the return address derived from the above + * instructions. + * + * saved_sp_rule, saved_fp_rule, saved_ra_rule + * Temporary storage to store and restore the CFA and frame pointer rules + * offsets while running instructions. + * + * cfa_reg, cfa_offset + * Temporary storage to remember the default CFA register and offset + * while running instructions. + * + * unusable + * Some error happened while processing CIE instructions. + */ +struct cie { + struct cie *next; + unsigned long offset; + unsigned long length; + unsigned long id; + unsigned char version; + unsigned char address_size; + unsigned char segment_size; + char *augmentation; + unsigned int code_factor; + int data_factor; + unsigned int return_address_reg; + unsigned char *instructions; + size_t instructions_size; + struct rule sp_rule; + struct rule fp_rule; + struct rule ra_rule; + struct rule saved_sp_rule; + struct rule saved_fp_rule; + struct rule saved_ra_rule; + unsigned int cfa_reg; + long cfa_offset; + bool unusable; +}; + +/* + * Frame Description Entry (FDE): + * + * next + * Next FDE in list. + * + * length + * A constant that gives the number of bytes of the header and instruction + * stream for this function, not including the length field itself. The + * size of the length field plus the value of length must be an integral + * multiple of the address size. + * + * cie_offset + * A constant offset into the .debug_frame section that denotes the CIE + * that is associated with this FDE. + * + * cie + * CIE for this FDE. + * + * segment_selector + * Segment selectors are not supported. + * + * start_pc, end_pc + * Range of code to which this FDE applies. + * + * instructions + * DWARF instructions to compute the register rules. + * + * instructions_size + * Number of bytes of instructions. + * + * index + * Index of the FDE record in the .debug_frame section. Used to obtain + * relocation information for the FDE. + * + * symbol + * Symbol information for the function for the code range. + * + * section + * Section information for the function. + * + * offset + * Offset within the section for the function. + * + * sp_offset + * Needed for clang hack. + */ +struct fde { + struct fde *next; + unsigned long length; + unsigned long cie_offset; + struct cie *cie; + unsigned long segment_selector; + unsigned long start_pc; + unsigned long end_pc; + unsigned char *instructions; + size_t instructions_size; + unsigned long index; + struct symbol *symbol; + struct section *section; + unsigned long offset; + unsigned long sp_offset; +}; + +/* + * These are identifiers for 32-bit and 64-bit CIE entries respectively. + * These identifiers distinguish a CIE from an FDE in .debug_frame. + */ +#define CIE_ID_32 0xFFFFFFFF +#define CIE_ID_64 0xFFFFFFFFFFFFFFFFUL + +/* + * DWARF instruction op codes. + */ + +/* Primary op codes in the high 2 bits */ + +#define DW_CFA_extended_op 0x00 +#define DW_CFA_advance_loc 0x40 +#define DW_CFA_offset 0x80 +#define DW_CFA_restore 0xc0 + +/* Extended op codes in the low 6 bits */ + +#define DW_CFA_nop 0x00 +#define DW_CFA_set_loc 0x01 +#define DW_CFA_advance_loc1 0x02 +#define DW_CFA_advance_loc2 0x03 +#define DW_CFA_advance_loc4 0x04 +#define DW_CFA_offset_extended 0x05 +#define DW_CFA_restore_extended 0x06 +#define DW_CFA_undefined 0x07 +#define DW_CFA_same_value 0x08 +#define DW_CFA_register 0x09 +#define DW_CFA_remember_state 0x0a +#define DW_CFA_restore_state 0x0b +#define DW_CFA_def_cfa 0x0c +#define DW_CFA_def_cfa_register 0x0d +#define DW_CFA_def_cfa_offset 0x0e +/* DWARF 3. */ +#define DW_CFA_def_cfa_expression 0x0f +#define DW_CFA_expression 0x10 +#define DW_CFA_offset_extended_sf 0x11 +#define DW_CFA_def_cfa_sf 0x12 +#define DW_CFA_def_cfa_offset_sf 0x13 +#define DW_CFA_val_offset 0x14 +#define DW_CFA_val_offset_sf 0x15 +#define DW_CFA_val_expression 0x16 +#define DW_CFA_lo_user 0x1c +#define DW_CFA_hi_user 0x3f + +/* + * Extract a value directly from the DWARF instruction stream. Must take into + * account endianness. + */ +#define GET_VALUE(value, start, end, count) \ + do { \ + size_t _size = (count); \ + size_t _avail = (end) - (start); \ + \ + if (sizeof(value) < _size) \ + _size = sizeof(value); \ + if ((start) > (end)) \ + _avail = 0; \ + if (_size > _avail) \ + _size = _avail; \ + if (_size == 0) \ + (value) = 0; \ + else \ + (value) = get_value((start), _size); \ + (start) += _size; \ + } while (0) + +/* + * Extract an unsigned integer expressed in LEB 128 format from the DWARF + * instruction stream. + */ +#define READ_ULEB_128(value, start, end, fail) \ + do { \ + u64 _val; \ + unsigned int _len; \ + bool _fail = false; \ + \ + _val = read_uleb_128(start, end, &_len, &_fail); \ + \ + start += _len; \ + (value) = _val; \ + if ((value) != _val) { \ + WARN("READ_ULEB_128: op=%d value mismatch", op);\ + _fail = true; \ + } \ + if (_fail) \ + (fail) = true; \ + } while (0) + +/* + * Extract a signed integer expressed in LEB 128 format from the DWARF + * instruction stream. + */ +#define READ_SLEB_128(value, start, end, fail) \ + do { \ + s64 _val; \ + unsigned int _len; \ + bool _fail = false; \ + \ + _val = read_sleb_128(start, end, &_len, &_fail); \ + \ + start += _len; \ + (value) = _val; \ + if ((value) != _val) { \ + WARN("READ_SLEB_128: op=%d value mismatch", op);\ + _fail = true; \ + } \ + if (_fail) \ + (fail) = true; \ + } while (0) + +extern struct objtool_file *dwarf_file; +extern struct section *debug_frame; +extern struct cie *cies, *cur_cie; +extern struct fde *fdes, *cur_fde; +extern unsigned char op, operand; +extern void *free_space; +extern size_t free_size; +extern u64 (*get_value)(unsigned char *field, unsigned int size); + +void dwarf_rule_start(struct fde *fde); +int dwarf_rule_add(struct fde *fde, unsigned long addr, + struct rule *sp_rule, struct rule *fp_rule); +void dwarf_rule_next(struct fde *fde, unsigned long addr); +void dwarf_rule_reset(struct fde *fde); +int arch_dwarf_fde_reloc(struct fde *fde); +void arch_dwarf_clang_hack(struct fde *fde, unsigned long pc, + struct rule *sp_rule, struct rule *fp_rule); +int arch_dwarf_check_rules(struct fde *fde, unsigned long pc, + struct rule *sp_rule, struct rule *fp_rule, + struct rule *ra_rule); +u64 get_value_le(unsigned char *field, unsigned int size); +u64 get_value_be(unsigned char *field, unsigned int size); +u64 read_uleb_128(unsigned char *start, unsigned char *end, + unsigned int *num_read, bool *fail); +s64 read_sleb_128(unsigned char *start, unsigned char *end, + unsigned int *num_read, bool *fail); +void dwarf_parse_instructions(void); +void dwarf_alloc_init(void); +void *dwarf_alloc(size_t size); + +#endif /* _OBJTOOL_DWARF_DEF_H */ diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h index cdc739fa9a6f..c133ece2cdcf 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -151,6 +151,7 @@ struct section *find_section_by_name(const struct elf *elf, const char *name); struct symbol *find_func_by_offset(struct section *sec, unsigned long offset); struct symbol *find_symbol_by_offset(struct section *sec, unsigned long offset); struct symbol *find_symbol_by_name(const struct elf *elf, const char *name); +struct symbol *find_symbol_by_index(struct elf *elf, unsigned int idx); struct symbol *find_symbol_containing(const struct section *sec, unsigned long offset); struct reloc *find_reloc_by_dest(const struct elf *elf, struct section *sec, unsigned long offset); struct reloc *find_reloc_by_dest_range(const struct elf *elf, struct section *sec, diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/include/objtool/objtool.h index f99fbc6078d5..0344e89a10e8 100644 --- a/tools/objtool/include/objtool/objtool.h +++ b/tools/objtool/include/objtool/objtool.h @@ -41,5 +41,6 @@ void objtool_pv_add(struct objtool_file *file, int idx, struct symbol *func); int check(struct objtool_file *file); int orc_dump(const char *objname); int orc_create(struct objtool_file *file); +int dwarf_parse(struct objtool_file *file); #endif /* _OBJTOOL_H */ diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c index bdf699f6552b..bfb9c5607cfc 100644 --- a/tools/objtool/objtool.c +++ b/tools/objtool/objtool.c @@ -38,6 +38,7 @@ static const char objtool_usage_string[] = static struct cmd_struct objtool_cmds[] = { {"check", cmd_check, "Perform stack metadata validation on an object file" }, {"orc", cmd_orc, "Generate in-place ORC unwind tables for an object file" }, + {"dwarf", cmd_dwarf, "Generate DWARF rules for object file"}, }; bool help; diff --git a/tools/objtool/weak.c b/tools/objtool/weak.c index 8314e824db4a..67b5016a8327 100644 --- a/tools/objtool/weak.c +++ b/tools/objtool/weak.c @@ -8,6 +8,7 @@ #include #include #include +#include #define UNSUPPORTED(name) \ ({ \ @@ -29,3 +30,29 @@ int __weak orc_create(struct objtool_file *file) { UNSUPPORTED("orc"); } + +int __weak dwarf_parse(struct objtool_file *file) + +{ + fprintf(stderr, "error: objtool: %s not implemented\n", __func__); + return -EOPNOTSUPP; +} + +int __weak arch_dwarf_fde_reloc(struct fde *fde) +{ + fprintf(stderr, "error: objtool: %s not implemented\n", __func__); + return -EOPNOTSUPP; +} + +void __weak arch_dwarf_clang_hack(struct fde *fde, unsigned long pc, + struct rule *sp_rule, struct rule *fp_rule) +{ +} + +int __weak arch_dwarf_check_rules(struct fde *fde, unsigned long pc, + struct rule *sp_rule, struct rule *fp_rule, + struct rule *ra_rule) +{ + fprintf(stderr, "error: objtool: %s not implemented\n", __func__); + return -EOPNOTSUPP; +} From patchwork Thu Apr 7 20:25:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8511EC433EF for ; Thu, 7 Apr 2022 20:27:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qGpwh7ArkliunKihM6m2+mqTAAwjNkreuqyWdrNEgHY=; b=XSuu9p3PSUfN0h T6v+3UJ7sHLREYgCutrnio7dMZcuGq7vyQGOoPN6sllBHUuJSg7QcWVAmjGs5TC39c+p3zpdPQ2ur Irxq0vk7kOXBCaI5447AsqhiWtLDYzHysWcJLtZkYy5CTfhj6i4LpqpJ8ewaLY0nUr+5eEebzhbk8 JgDJAfjf7V57rJZRsOY2IvJ5mlBcqsAiKLTMBeJ1dcbE/eQMiS+Y0QgatKe9lmQpFIpYZ9dF64cSw E/Y9Upkwd30PH87mXe6cSahPKPbAMROU+UlCAk2S0NBJIcyl07gOGAbP04j00XTIVAkFB5MRhvZVU ZOUlOBfTjsWCob7TJ5yQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYi8-00Drgb-6A; Thu, 07 Apr 2022 20:26:32 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhe-00DrVw-LS for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:05 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 8655420B9CEF; Thu, 7 Apr 2022 13:26:01 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 8655420B9CEF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363162; bh=nVQ+gSE19+X//O4Sk/QNhBvnuZzB5LkL5Rb13i7ZcgM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=OWvphc47b+i0m2hrrF/YK+J97hpiVwg11oPGOgHp+WyQ/kxYXSDdyvxPpRSF3QlLF Ha3DE7NTMcK093by8SoU3mv+gvHEDxUl4hmEGjYbJLaxGXxjxRrPLI0x/WWIYMThKF Ld6a1GspxHld3r2DbDCzm0GGq+b2AbEB/pyQxydM= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 2/9] objtool: Generate DWARF rules and place them in a special section Date: Thu, 7 Apr 2022 15:25:11 -0500 Message-Id: <20220407202518.19780-3-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132602_802328_1CCF8D4F X-CRM114-Status: GOOD ( 31.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Convert the DWARF Call Frame Information parsed by dwarf_parse() into compact DWARF rules that are usable by the kernel. Place the rules in a special section called .dwarf_rules. Also, place the PCs for the rules in a special section called .dwarf_pcs. In addition, define relocation entries for the PCs as they will change during linking. An entry in .dwarf_rules and its corresponding entry in .dwarf_pcs together describe a code range and DWARF rules for the code range. In the future, the kernel will use the rules to compute the frame pointer at a given instruction address. The unwinder can use the computed frame pointer to validate the actual frame pointer for a reliable stack trace. During rule generation, eliminate null offset rules and merge adjacent rules that are identical to minimize the number of rules. Also add an objtool option to dump the DWARF rules for debugging purposes. It is invoked as follows: objtool dwarf dump Signed-off-by: Madhavan T. Venkataraman --- include/linux/dwarf.h | 43 +++++ tools/include/linux/dwarf.h | 43 +++++ tools/objtool/builtin-dwarf.c | 22 ++- tools/objtool/dwarf_rules.c | 181 +++++++++++++++++++++- tools/objtool/include/objtool/dwarf_def.h | 12 ++ tools/objtool/include/objtool/objtool.h | 2 + tools/objtool/sync-check.sh | 6 + tools/objtool/weak.c | 11 ++ 8 files changed, 311 insertions(+), 9 deletions(-) create mode 100644 include/linux/dwarf.h create mode 100644 tools/include/linux/dwarf.h diff --git a/include/linux/dwarf.h b/include/linux/dwarf.h new file mode 100644 index 000000000000..16e9dd8c60c8 --- /dev/null +++ b/include/linux/dwarf.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * dwarf.h - DWARF data structures used by the unwinder. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (c) 2022 Microsoft Corporation + */ + +#ifndef _LINUX_DWARF_H +#define _LINUX_DWARF_H + +#include + +/* + * objtool generates two special sections that contain DWARF information that + * will be used by the reliable unwinder to validate the frame pointer in every + * frame: + * + * .dwarf_rules: + * This contains an array of struct dwarf_rule. Each rule contains the + * size of a code range. In addition, a rule contains the offsets that + * must be used to compute the frame pointer at any of the instructions + * within the code range. The computation is: + * + * CFA = %sp + sp_offset + * FP = CFA + fp_offset + * + * where %sp is the stack pointer at the instruction address and FP is + * the frame pointer. + * + * .dwarf_pcs: + * This contains an array of starting PCs, one for each rule. + */ +struct dwarf_rule { + unsigned int size:30; + unsigned int sp_saved:1; + unsigned int fp_saved:1; + short sp_offset; + short fp_offset; +}; + +#endif /* _LINUX_DWARF_H */ diff --git a/tools/include/linux/dwarf.h b/tools/include/linux/dwarf.h new file mode 100644 index 000000000000..16e9dd8c60c8 --- /dev/null +++ b/tools/include/linux/dwarf.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * dwarf.h - DWARF data structures used by the unwinder. + * + * Author: Madhavan T. Venkataraman (madvenka@linux.microsoft.com) + * + * Copyright (c) 2022 Microsoft Corporation + */ + +#ifndef _LINUX_DWARF_H +#define _LINUX_DWARF_H + +#include + +/* + * objtool generates two special sections that contain DWARF information that + * will be used by the reliable unwinder to validate the frame pointer in every + * frame: + * + * .dwarf_rules: + * This contains an array of struct dwarf_rule. Each rule contains the + * size of a code range. In addition, a rule contains the offsets that + * must be used to compute the frame pointer at any of the instructions + * within the code range. The computation is: + * + * CFA = %sp + sp_offset + * FP = CFA + fp_offset + * + * where %sp is the stack pointer at the instruction address and FP is + * the frame pointer. + * + * .dwarf_pcs: + * This contains an array of starting PCs, one for each rule. + */ +struct dwarf_rule { + unsigned int size:30; + unsigned int sp_saved:1; + unsigned int fp_saved:1; + short sp_offset; + short fp_offset; +}; + +#endif /* _LINUX_DWARF_H */ diff --git a/tools/objtool/builtin-dwarf.c b/tools/objtool/builtin-dwarf.c index f44b35eb3f55..1b451e830140 100644 --- a/tools/objtool/builtin-dwarf.c +++ b/tools/objtool/builtin-dwarf.c @@ -25,6 +25,10 @@ static const char * const dwarf_usage[] = { * information. */ "objtool dwarf generate file", + /* + * Dump DWARF rules for debugging purposes. + */ + "objtool dwarf dump file", NULL, }; @@ -37,6 +41,7 @@ int cmd_dwarf(int argc, const char **argv) { const char *object; struct objtool_file *file; + int ret; argc--; argv++; if (argc != 2) @@ -48,8 +53,21 @@ int cmd_dwarf(int argc, const char **argv) if (!file) return 1; - if (!strncmp(argv[0], "gen", 3)) - return dwarf_parse(file); + if (!strncmp(argv[0], "gen", 3)) { + ret = dwarf_parse(file); + if (!ret) + ret = dwarf_write(file); + if (!ret && file->elf->changed) + ret = elf_write(file->elf); + return ret; + } + + if (!strcmp(argv[0], "dump")) { + ret = dwarf_parse(file); + if (!ret) + dwarf_dump(); + return ret; + } usage_with_options(dwarf_usage, dwarf_options); diff --git a/tools/objtool/dwarf_rules.c b/tools/objtool/dwarf_rules.c index 9cf201de392a..a118b392aac8 100644 --- a/tools/objtool/dwarf_rules.c +++ b/tools/objtool/dwarf_rules.c @@ -13,25 +13,192 @@ #include #include -/* - * The following are stubs for now. Later, they will be filled to create - * DWARF rules that the kernel can use to compute the frame pointer at - * a given instruction address. - */ +struct section *dwarf_rules_sec; +struct section *dwarf_pcs_sec; + +static struct fde_entry *cur_entry; +static int nentries; + +static int dwarf_rule_insert(struct fde *fde, unsigned long addr, + struct rule *sp_rule, struct rule *fp_rule); + void dwarf_rule_start(struct fde *fde) { + fde->head = NULL; + fde->tail = NULL; + cur_entry = NULL; } int dwarf_rule_add(struct fde *fde, unsigned long addr, - struct rule *sp_rule, struct rule *fp_rule) + struct rule *sp_rule, struct rule *fp_rule) { - return 0; + if (cur_entry) { + struct rule *esp_rule = &cur_entry->sp_rule; + struct rule *efp_rule = &cur_entry->fp_rule; + + /* + * If the rules have not changed, there is nothing to do. + */ + if (esp_rule->offset == sp_rule->offset && + efp_rule->offset == fp_rule->offset && + esp_rule->saved == sp_rule->saved && + efp_rule->saved == fp_rule->saved) { + return 0; + } + /* Close out the current range. */ + cur_entry->size = addr - cur_entry->addr; + } + return dwarf_rule_insert(fde, addr, sp_rule, fp_rule); } void dwarf_rule_next(struct fde *fde, unsigned long addr) { + if (cur_entry) { + /* Close out the current range. */ + cur_entry->size = addr - cur_entry->addr; + cur_entry = NULL; + } } void dwarf_rule_reset(struct fde *fde) { + struct fde_entry *entry; + + while (fde->head) { + entry = fde->head; + fde->head = entry->next; + free(entry); + nentries--; + } + fde->tail = NULL; + cur_entry = NULL; +} + +static int dwarf_rule_insert(struct fde *fde, unsigned long addr, + struct rule *sp_rule, struct rule *fp_rule) +{ + struct fde_entry *entry; + + entry = dwarf_alloc(sizeof(*entry)); + if (!entry) + return -1; + + /* Add the entry to the FDE list. */ + if (fde->tail) + fde->tail->next = entry; + else + fde->head = entry; + fde->tail = entry; + entry->next = NULL; + + /* + * Record the starting address of the code range here. The size of + * the range will be known only when the next rule comes in. At that + * time, we will close out this range. + */ + entry->addr = addr; + + /* Copy the rules. */ + entry->sp_rule = *sp_rule; + entry->fp_rule = *fp_rule; + + cur_entry = entry; + nentries++; + return 0; +} + +static int dwarf_rule_write(struct elf *elf, struct fde *fde, + struct fde_entry *entry, unsigned int index) +{ + struct dwarf_rule rule, *drule; + + /* + * Encode the SP and FP rules from the entry into a single dwarf_rule + * for the kernel's benefit. Copy it into .dwarf_rules. + */ + rule.size = entry->size; + rule.sp_saved = entry->sp_rule.saved; + rule.fp_saved = entry->fp_rule.saved; + rule.sp_offset = entry->sp_rule.offset; + rule.fp_offset = entry->fp_rule.offset; + + drule = (struct dwarf_rule *) dwarf_rules_sec->data->d_buf + index; + memcpy(drule, &rule, sizeof(rule)); + + /* Add relocation information for the code range. */ + if (elf_add_reloc_to_insn(elf, dwarf_pcs_sec, + index * sizeof(unsigned long), + R_AARCH64_ABS64, + fde->section, entry->addr)) { + return -1; + } + return 0; +} + +int dwarf_write(struct objtool_file *file) +{ + struct elf *elf = file->elf; + struct fde *fde; + struct fde_entry *entry; + int index; + + /* + * Check if .dwarf_rules already exists. If it doesn't, we will + * assume that .dwarf_pcs doesn't exist either. + */ + if (find_section_by_name(elf, ".dwarf_rules")) { + WARN("file already has .dwarf_rules section"); + return -1; + } + + /* Create .dwarf_rules. */ + dwarf_rules_sec = elf_create_section(elf, ".dwarf_rules", 0, + sizeof(struct dwarf_rule), + nentries); + if (!dwarf_rules_sec) { + WARN("Unable to create .dwarf_rules"); + return -1; + } + + /* Create .dwarf_pcs. */ + dwarf_pcs_sec = elf_create_section(elf, ".dwarf_pcs", 0, + sizeof(unsigned long), nentries); + if (!dwarf_pcs_sec) { + WARN("Unable to create .dwarf_pcs"); + return -1; + } + + /* Write DWARF rules to sections. */ + index = 0; + for (fde = fdes; fde != NULL; fde = fde->next) { + for (entry = fde->head; entry != NULL; entry = entry->next) { + if (dwarf_rule_write(elf, fde, entry, index)) + return -1; + index++; + } + } + + return 0; +} + +void dwarf_dump(void) +{ + struct fde *fde; + struct fde_entry *entry; + struct rule *sp_rule, *fp_rule; + int index = 0; + + for (fde = fdes; fde != NULL; fde = fde->next) { + for (entry = fde->head; entry != NULL; entry = entry->next) { + sp_rule = &entry->sp_rule; + fp_rule = &entry->fp_rule; + + printf("addr=%lx size=%lx:", + entry->addr, entry->size); + printf("\tsp=%ld sp_saved=%d fp=%ld fp_saved=%d\n", + sp_rule->offset, sp_rule->saved, + fp_rule->offset, fp_rule->saved); + index++; + } + } } diff --git a/tools/objtool/include/objtool/dwarf_def.h b/tools/objtool/include/objtool/dwarf_def.h index 7a0a18480d2b..af56ccb52fff 100644 --- a/tools/objtool/include/objtool/dwarf_def.h +++ b/tools/objtool/include/objtool/dwarf_def.h @@ -10,6 +10,8 @@ #ifndef _OBJTOOL_DWARF_DEF_H #define _OBJTOOL_DWARF_DEF_H +#include + /* * The DWARF Call Frame Information (CFI) is encoded in a self-contained * section called .debug_frame. @@ -228,6 +230,14 @@ struct cie { bool unusable; }; +struct fde_entry { + struct fde_entry *next; + unsigned long addr; + size_t size; + struct rule sp_rule; + struct rule fp_rule; +}; + /* * Frame Description Entry (FDE): * @@ -290,6 +300,8 @@ struct fde { struct section *section; unsigned long offset; unsigned long sp_offset; + struct fde_entry *head; + struct fde_entry *tail; }; /* diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/include/objtool/objtool.h index 0344e89a10e8..93e62639ab01 100644 --- a/tools/objtool/include/objtool/objtool.h +++ b/tools/objtool/include/objtool/objtool.h @@ -42,5 +42,7 @@ int check(struct objtool_file *file); int orc_dump(const char *objname); int orc_create(struct objtool_file *file); int dwarf_parse(struct objtool_file *file); +void dwarf_dump(void); +int dwarf_write(struct objtool_file *file); #endif /* _OBJTOOL_H */ diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh index 105a291ff8e7..345c259a115c 100755 --- a/tools/objtool/sync-check.sh +++ b/tools/objtool/sync-check.sh @@ -27,6 +27,12 @@ arch/x86/lib/insn.c ' fi +if [ "$SRCARCH" = "arm64" ]; then +FILES="$FILES +include/linux/dwarf.h +" +fi + check_2 () { file1=$1 file2=$2 diff --git a/tools/objtool/weak.c b/tools/objtool/weak.c index 67b5016a8327..9d89d4fad8a1 100644 --- a/tools/objtool/weak.c +++ b/tools/objtool/weak.c @@ -38,6 +38,17 @@ int __weak dwarf_parse(struct objtool_file *file) return -EOPNOTSUPP; } +int __weak dwarf_write(struct objtool_file *file) +{ + fprintf(stderr, "error: objtool: %s not implemented\n", __func__); + return -1; +} + +void __weak dwarf_dump(void) +{ + fprintf(stderr, "error: objtool: %s not implemented\n", __func__); +} + int __weak arch_dwarf_fde_reloc(struct fde *fde) { fprintf(stderr, "error: objtool: %s not implemented\n", __func__); From patchwork Thu Apr 7 20:25:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31F70C433F5 for ; Thu, 7 Apr 2022 20:27:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wr+YD7+SQ3gUTOOHV5v1liq6YXTeagt01Qml4oS1nyI=; b=xkjrSLadAdV1lH 6qEuY/wfamGgamd2wA38DEpDGVo7LeMKXowjI0Y+CZbB5BKulPaapDjec/TGs+K2TeuNXO7KvGsVp aAP4Qw4y67eYz2G99WTvdcqYfVopINlZ9aS2gXQViT8XipdW1Agk0rtyqMmHyH3BVC4rXwYYpXPFb ZbbiZjXysG3Py2N0T0yFzRZQwOhwZLYsDr0bTWmQcljcQjZ+kg9pBmlemAjOrbKiTf0jKlhVXB3Sa 6AbqoDtwyq9SAIWa9NouAsBBP7FgMvsBomEU9NaGOVRLsCi4j3po3j3xDKt9dOMnprgxvITp0+zTN j4dClQKlfu3mDx8MABqw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhz-00Drfa-OY; Thu, 07 Apr 2022 20:26:23 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhf-00DrWg-Jq for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:05 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 798F420B9CF0; Thu, 7 Apr 2022 13:26:02 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 798F420B9CF0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363163; bh=wbp3KQRzbL8wBuPZAF+JGlo4GHESFi/Po72T2KLLghc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=R25TWtc3BkaKmcHdSg95pqX7kI1wwu5ptGUfUsniDru1WvlWHXVlclaqU2JEnbfzf w61K70ILGt2p/F+/E9xjBa8ZTbPtw8HapvDAXXqegLfOuyVTrYDlcDTboQqXn4DvF9 W6BBgdS0l6JCAooirPH5dmPDvzUsF1HitaGFsgd0= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 3/9] dwarf: Build the kernel with DWARF information Date: Thu, 7 Apr 2022 15:25:12 -0500 Message-Id: <20220407202518.19780-4-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132603_780933_241F5F69 X-CRM114-Status: GOOD ( 19.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Define CONFIG_DWARF_FP - to include DWARF based FP validation code. Define CONFIG_STACK_VALIDATION - to enable DWARF based FP validation. When these configs are enabled, invoke objtool on relocatable files during the kernel build with the following command: objtool dwarf generate Objtool creates the following sections in each object file: .dwarf_rules Array of DWARF rules .dwarf_pcs Array of PCs, one-to-one with rules In the future, the kernel can use these sections to find the rules for a given instruction address. The unwinder can then compute the FP at an instruction address and validate the actual FP with that. NOTE: CONFIG_STACK_VALIDATION needs to be turned on here. Otherwise, objtool will not be invoked during the kernel build process. The actual stack validation code will be added separately. This is harmless. Signed-off-by: Madhavan T. Venkataraman --- arch/Kconfig | 4 +++- arch/arm64/Kconfig | 2 ++ arch/arm64/Kconfig.debug | 5 +++++ arch/arm64/configs/defconfig | 1 + arch/arm64/kernel/vmlinux.lds.S | 22 ++++++++++++++++++++++ scripts/Makefile.build | 4 ++++ scripts/link-vmlinux.sh | 6 ++++++ 7 files changed, 43 insertions(+), 1 deletion(-) diff --git a/arch/Kconfig b/arch/Kconfig index d3c4ab249e9c..3b0d0db322b9 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1016,7 +1016,9 @@ config HAVE_STACK_VALIDATION bool help Architecture supports the 'objtool check' host tool command, which - performs compile-time stack metadata validation. + performs compile-time stack metadata validation. Or, on architectures + that use DWARF validated frame pointers, it supports the + 'objtool dwarf generate' host tool command. config HAVE_RELIABLE_STACKTRACE bool diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index c4207cf9bb17..c82a3a93297f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -220,6 +220,8 @@ config ARM64 select SWIOTLB select SYSCTL_EXCEPTION_TRACE select THREAD_INFO_IN_TASK + select HAVE_STACK_VALIDATION if DWARF_FP + select STACK_VALIDATION if HAVE_STACK_VALIDATION select HAVE_ARCH_USERFAULTFD_MINOR if USERFAULTFD select TRACE_IRQFLAGS_SUPPORT help diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug index 265c4461031f..585967062a1c 100644 --- a/arch/arm64/Kconfig.debug +++ b/arch/arm64/Kconfig.debug @@ -20,4 +20,9 @@ config ARM64_RELOC_TEST depends on m tristate "Relocation testing module" +config DWARF_FP + def_bool y + depends on FRAME_POINTER + depends on DEBUG_INFO_DWARF4 + source "drivers/hwtracing/coresight/Kconfig" diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig index f2e2b9bdd702..a59c448f442a 100644 --- a/arch/arm64/configs/defconfig +++ b/arch/arm64/configs/defconfig @@ -1233,3 +1233,4 @@ CONFIG_DEBUG_KERNEL=y # CONFIG_DEBUG_PREEMPT is not set # CONFIG_FTRACE is not set CONFIG_MEMTEST=y +CONFIG_DEBUG_INFO_DWARF4=y diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 50bab186c49b..fb3b9970453b 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -122,6 +122,25 @@ jiffies = jiffies_64; #define TRAMP_TEXT #endif +#ifdef CONFIG_DWARF_FP +#define DWARF_RULES \ + . = ALIGN(8); \ + .dwarf_rules : { \ + __dwarf_rules_start = .; \ + KEEP(*(.dwarf_rules)) \ + __dwarf_rules_end = .; \ + } + +#define DWARF_PCS \ + . = ALIGN(8); \ + __dwarf_pcs_start = .; \ + KEEP(*(.dwarf_pcs)) \ + __dwarf_pcs_end = .; +#else +#define DWARF_RULES +#define DWARF_PCS +#endif + /* * The size of the PE/COFF section that covers the kernel image, which * runs from _stext to _edata, must be a round multiple of the PE/COFF @@ -239,6 +258,7 @@ SECTIONS CON_INITCALL INIT_RAM_FS *(.init.altinstructions .init.bss) /* from the EFI stub */ + DWARF_PCS } .exit.data : { EXIT_DATA @@ -291,6 +311,8 @@ SECTIONS __mmuoff_data_end = .; } + DWARF_RULES + PECOFF_EDATA_PADDING __pecoff_data_rawsize = ABSOLUTE(. - __initdata_begin); _edata = .; diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 78656b527fe5..5e8d89c64572 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -227,6 +227,9 @@ ifdef CONFIG_STACK_VALIDATION objtool := $(objtree)/tools/objtool/objtool +ifdef CONFIG_DWARF_FP +objtool_args = dwarf generate +else objtool_args = \ $(if $(CONFIG_UNWINDER_ORC),orc generate,check) \ $(if $(part-of-module), --module) \ @@ -235,6 +238,7 @@ objtool_args = \ $(if $(CONFIG_RETPOLINE), --retpoline) \ $(if $(CONFIG_X86_SMAP), --uaccess) \ $(if $(CONFIG_FTRACE_MCOUNT_USE_OBJTOOL), --mcount) +endif cmd_objtool = $(if $(objtool-enabled), ; $(objtool) $(objtool_args) $@) cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(objtool))' ; } >> $(dot-target).cmd) diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 5cdd9bc5c385..433e395f977b 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -104,6 +104,12 @@ objtool_link() local objtoolcmd; local objtoolopt; + if [ "${CONFIG_LTO_CLANG} ${CONFIG_DWARF_FP}" = "y y" ] + then + tools/objtool/objtool dwarf generate ${1} + return + fi + if [ "${CONFIG_LTO_CLANG} ${CONFIG_STACK_VALIDATION}" = "y y" ]; then # Don't perform vmlinux validation unless explicitly requested, # but run objtool on vmlinux.o now that we have an object file. From patchwork Thu Apr 7 20:25:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805680 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D99EC433F5 for ; Thu, 7 Apr 2022 20:28:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=P06WEgqMoe2refC2kNZUpbHH2mMrFFEIQVGALhI75f0=; b=PrcZw05jWtyyso PIQdhifXh1lBq4GNduFDe+3t/nJSEAwuflABWNVSdSd3osWqPGUYA1hWy2Hmqx1abnwxUn9ops6Fc SfBH3qVWHfpAbsZWnX/pYmwjo9zG/dhEkZU81f37e68y1XVGMQ4tt0YMmNki1rXjJBCNUAnetd9mh pNfpYkLO36EOF1lD2THt5OBJeufpuh6zy26POBU+iEurNojrymR8vtMvUUsv43M5ob23FW2uaeCVB zK3hptT+v4lwCiFBvMDggwvgPlbzFEfNBVBGs8SWvIZU+kcbmhvG6ye9I7Xy9kqnwy+T3mv3CXRkn Xp0VNH0e9BZHcEo9EoKg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYiQ-00Drlg-2E; Thu, 07 Apr 2022 20:26:50 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhg-00DrYH-I3 for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:06 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 6BEC820BA5AC; Thu, 7 Apr 2022 13:26:03 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 6BEC820BA5AC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363164; bh=BSKbGYlk+dm3FX69ux9O3ehGfhjDAqvE7NkPhunNR+w=; h=From:To:Subject:Date:In-Reply-To:References:From; b=YTIymf1N9NBeKyBvzU6MdJLvXPltxW9AAuBmMxKb1Rm07zVgOrblLET1kxYHj8gUl DbsBO+fb2Gxzb/MODrfVhS0UBMXrb7wwdnJtvYP7IhKmFudPCi6k2LkTWDVf5vhQC8 lSBhPw5k1X5QbhKWb3GW1NJYIpwciPdusiXbaZ1A= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 4/9] dwarf: Implement DWARF rule processing in the kernel Date: Thu, 7 Apr 2022 15:25:13 -0500 Message-Id: <20220407202518.19780-5-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132604_702343_7EED6737 X-CRM114-Status: GOOD ( 28.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Define a struct dwarf_info to store all of the DWARF information needed to lookup the DWARF rules for an instruction address. There is one dwarf_info for vmlinux and one for every module. Implement a lookup function dwarf_lookup(). Given an instruction address, the function looks up the corresponding DWARF rules. The unwinder will use the lookup function in the future. Sort the rules based on instruction address. This allows a binary search. Divide the text range into fixed sized blocks and map the rules to their respective blocks. Given an instruction address, first locate the block for the address. Then, perform a binary search within the rules in the block. This minimizes the number of rules to consider in the binary search. dwarf_info contains an array of PCs to search. In order to save space, store the PCs array as an array of offsets from the base PC of the text range. This way, we only need 32 bits to store the PC. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/sections.h | 4 + include/linux/dwarf.h | 21 +++ kernel/Makefile | 1 + kernel/dwarf_fp.c | 244 ++++++++++++++++++++++++++++++ tools/include/linux/dwarf.h | 21 +++ 5 files changed, 291 insertions(+) create mode 100644 kernel/dwarf_fp.c diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 152cb35bf9df..d9095a9094b7 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -22,5 +22,9 @@ extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +#ifdef CONFIG_DWARF_FP +extern char __dwarf_rules_start[], __dwarf_rules_end[]; +extern char __dwarf_pcs_start[], __dwarf_pcs_end[]; +#endif #endif /* __ASM_SECTIONS_H */ diff --git a/include/linux/dwarf.h b/include/linux/dwarf.h index 16e9dd8c60c8..3df15e79003c 100644 --- a/include/linux/dwarf.h +++ b/include/linux/dwarf.h @@ -40,4 +40,25 @@ struct dwarf_rule { short fp_offset; }; +/* + * The whole text area is divided into fixed sized blocks. Rules are mapped + * to their respective blocks. To find a block for an instruction address, + * the block of the address is located. Then, a binary search is performed + * on just the rules in the block. This minimizes the number of rules to + * be considered for the search. + */ +struct dwarf_block { + int first_rule; + int last_rule; +}; + +#ifdef CONFIG_DWARF_FP +extern struct dwarf_rule *dwarf_lookup(unsigned long pc); +#else +static inline struct dwarf_rule *dwarf_lookup(unsigned long pc) +{ + return NULL; +} +#endif + #endif /* _LINUX_DWARF_H */ diff --git a/kernel/Makefile b/kernel/Makefile index 186c49582f45..7582a6323446 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -130,6 +130,7 @@ obj-$(CONFIG_WATCH_QUEUE) += watch_queue.o obj-$(CONFIG_RESOURCE_KUNIT_TEST) += resource_kunit.o obj-$(CONFIG_SYSCTL_KUNIT_TEST) += sysctl-test.o +obj-$(CONFIG_DWARF_FP) += dwarf_fp.o CFLAGS_stackleak.o += $(DISABLE_STACKLEAK_PLUGIN) obj-$(CONFIG_GCC_PLUGIN_STACKLEAK) += stackleak.o diff --git a/kernel/dwarf_fp.c b/kernel/dwarf_fp.c new file mode 100644 index 000000000000..bb14fbe3f3e1 --- /dev/null +++ b/kernel/dwarf_fp.c @@ -0,0 +1,244 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * dwarf_fp.c - Allocate DWARF info. There will be one info for vmlinux + * and one for every module. Implement a lookup function that + * can locate the rule for a given instruction address. + * + * Copyright (C) 2021 Microsoft, Inc. + * Author: Madhavan T. Venkataraman + */ +#include +#include +#include +#include +#include +#include + +#define OFFSET_BLOCK_SHIFT 12 +#define OFFSET_BLOCK(pc) ((pc) >> OFFSET_BLOCK_SHIFT) + +/* + * There is one struct dwarf_info for vmlinux and one for each module. + */ +struct dwarf_info { + struct dwarf_rule *rules; + int nrules; + unsigned int *offsets; + + struct dwarf_block *blocks; + int nblocks; + + unsigned long *pcs; + unsigned long base_pc; + unsigned long end_pc; +}; + +static DEFINE_MUTEX(dwarf_mutex); + +static struct dwarf_info *vmlinux_dwarf_info; +static struct dwarf_info *cur_info; + +static int dwarf_compare(const void *arg1, const void *arg2) +{ + const unsigned long *pc1 = arg1; + const unsigned long *pc2 = arg2; + + if (*pc1 > *pc2) + return 1; + if (*pc1 < *pc2) + return -1; + return 0; +} + +static void dwarf_swap(void *arg1, void *arg2, int size) +{ + struct dwarf_rule *rules = cur_info->rules; + unsigned long *pc1 = arg1; + unsigned long *pc2 = arg2; + int i = (int) (pc1 - cur_info->pcs); + int j = (int) (pc2 - cur_info->pcs); + unsigned long tmp_pc; + struct dwarf_rule tmp_rule; + + tmp_pc = *pc1; + *pc1 = *pc2; + *pc2 = tmp_pc; + + tmp_rule = rules[i]; + rules[i] = rules[j]; + rules[j] = tmp_rule; +} + +/* + * Sort DWARF Records based on instruction addresses. + */ +static void dwarf_sort(struct dwarf_info *info) +{ + mutex_lock(&dwarf_mutex); + + /* + * cur_info is a global that allows us to sort both arrays in one go. + */ + cur_info = info; + sort(info->pcs, info->nrules, sizeof(*info->pcs), + dwarf_compare, dwarf_swap); + + mutex_unlock(&dwarf_mutex); +} + +#define INVALID_RULE -1 + +static struct dwarf_info *dwarf_alloc(struct dwarf_rule *rules, int nrules, + unsigned long *pcs) +{ + struct dwarf_info *info; + unsigned int *offsets, last_offset; + struct dwarf_block *blocks; + int r, b, nblocks; + + info = kmalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return NULL; + + info->rules = rules; + info->nrules = nrules; + info->pcs = pcs; + + /* Sort pcs[] and rules[] in the increasing order of PC. */ + dwarf_sort(info); + + /* Compute the boundaries for the rules. */ + info->base_pc = pcs[0]; + info->end_pc = pcs[nrules - 1] + rules[nrules - 1].size; + + offsets = kmalloc_array(nrules, sizeof(*offsets), GFP_KERNEL); + if (!offsets) + goto free_info; + + /* Store the PCs as offsets from the base PC. This is to save memory. */ + for (r = 0; r < nrules; r++) + offsets[r] = pcs[r] - info->base_pc; + + /* Compute the number of blocks. */ + last_offset = offsets[nrules - 1]; + nblocks = OFFSET_BLOCK(last_offset) + 1; + + blocks = kmalloc_array(nblocks, sizeof(*blocks), GFP_KERNEL); + if (!blocks) + goto free_offsets; + + /* Initialize blocks. */ + for (b = 0; b < nblocks; b++) { + blocks[b].first_rule = INVALID_RULE; + blocks[b].last_rule = INVALID_RULE; + } + + /* Map rules to blocks. */ + for (r = 0; r < nrules; r++) { + b = OFFSET_BLOCK(offsets[r]); + if (blocks[b].first_rule == INVALID_RULE) + blocks[b].first_rule = r; + blocks[b].last_rule = r; + } + + /* Initialize empty blocks. */ + for (b = 0; b < nblocks; b++) { + if (blocks[b].first_rule == INVALID_RULE) { + blocks[b].first_rule = blocks[b - 1].last_rule; + blocks[b].last_rule = blocks[b - 1].last_rule; + } + } + + info->blocks = blocks; + info->nblocks = nblocks; + info->offsets = offsets; + + /* PCs for vmlinux is in init data. It will discarded. */ + info->pcs = NULL; + + return info; +free_offsets: + kfree(offsets); +free_info: + kfree(info); + return NULL; +} + +static struct dwarf_rule *dwarf_lookup_rule(struct dwarf_info *info, + unsigned long pc) +{ + struct dwarf_block *blocks = info->blocks; + unsigned int *offsets = info->offsets, off; + struct dwarf_rule *rule; + int start, mid, end, n, b; + + if (pc < info->base_pc || pc >= info->end_pc) + return NULL; + + /* Make PC relative to the base for binary search. */ + off = pc - info->base_pc; + + /* + * Locate the block for the offset. Do a binary search between the + * start and end rules in the block. + */ + b = OFFSET_BLOCK(off); + start = blocks[b].first_rule; + end = blocks[b].last_rule + 1; + + if (off < offsets[start]) + start--; + + /* + * Binary search. For cache performance, we search in offsets[] + * first and locate a candidate rule. Then, we perform a range check + * for the candidate rule at the end. This is so that rules[] + * is only accessed at the end of the search. + */ + for (n = end - start; n > 1; n = end - start) { + mid = start + (n >> 1); + + if (off >= offsets[mid]) + start = mid; + else + end = mid; + } + + /* Do a final range check. */ + rule = &info->rules[start]; + if (off >= offsets[start] && off < (offsets[start] + rule->size)) + return rule; + + return NULL; +} + +struct dwarf_rule *dwarf_lookup(unsigned long pc) +{ + /* + * Currently, only looks up vmlinux. Support for modules will be + * added later. + */ + return dwarf_lookup_rule(vmlinux_dwarf_info, pc); +} + +static int __init dwarf_init_feature(void) +{ + struct dwarf_rule *rules; + unsigned long *pcs; + int nrules, npcs; + + rules = (struct dwarf_rule *) __dwarf_rules_start; + nrules = (__dwarf_rules_end - __dwarf_rules_start) / sizeof(*rules); + if (!nrules) + return -EINVAL; + + pcs = (unsigned long *) __dwarf_pcs_start; + npcs = (__dwarf_pcs_end - __dwarf_pcs_start) / sizeof(*pcs); + if (npcs != nrules) + return -EINVAL; + + vmlinux_dwarf_info = dwarf_alloc(rules, nrules, pcs); + + return vmlinux_dwarf_info ? 0 : -EINVAL; +} +early_initcall(dwarf_init_feature); diff --git a/tools/include/linux/dwarf.h b/tools/include/linux/dwarf.h index 16e9dd8c60c8..3df15e79003c 100644 --- a/tools/include/linux/dwarf.h +++ b/tools/include/linux/dwarf.h @@ -40,4 +40,25 @@ struct dwarf_rule { short fp_offset; }; +/* + * The whole text area is divided into fixed sized blocks. Rules are mapped + * to their respective blocks. To find a block for an instruction address, + * the block of the address is located. Then, a binary search is performed + * on just the rules in the block. This minimizes the number of rules to + * be considered for the search. + */ +struct dwarf_block { + int first_rule; + int last_rule; +}; + +#ifdef CONFIG_DWARF_FP +extern struct dwarf_rule *dwarf_lookup(unsigned long pc); +#else +static inline struct dwarf_rule *dwarf_lookup(unsigned long pc) +{ + return NULL; +} +#endif + #endif /* _LINUX_DWARF_H */ From patchwork Thu Apr 7 20:25:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 245E6C433F5 for ; Thu, 7 Apr 2022 20:28:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JcW9pGbTVc0FZXXsiIbbGEKE5P+u16AgGI7zmJlyu9E=; b=mQxhNp6ty/6PJ2 ocgLAybMjRt47NpqsQWQ4Twx6PQ3NAU4TFsrmyRqVJKFTC//Nt4wiBfxkfk5f/GwAbV/CWjjvDkCm dcjXSlzrAaM7h3NcCOENAJEV+PU7EtxqBD4AJxK731CsgSd3VYb59bY16IC0Rp5bauXOrTBKlgHSx 2+o2SCrcOjdMSoLyIMvCjl/CArOVSWUxRYxfVRGbz/++wYB/f+xmkpJ//bIogStbSWc9+UCpo/ox7 EJfLl5zWiM2a5J6ZUNFzXRkPzK5ZMHYOjEM5oEVeFJuM15SchSWQUrDEuC5/1uUU38QldtupIvhjp PBkrCspkSSWTuFe2TFPg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYic-00Drr2-GU; Thu, 07 Apr 2022 20:27:03 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhh-00DrZg-Uz for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:08 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 5E4C720B36F7; Thu, 7 Apr 2022 13:26:04 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5E4C720B36F7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363165; bh=JukkwBZel5zBVGcAdYt8iwRbZEdTiwAJnMVwnLhJEtE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=oXz+FrSrlVvwkKE55Ydy2W8QVsrhMLJlxkM8NZroNmyHKwGszPgWGXfrvCxjiLeGZ cJqtSdg/FAvDDhCwhMJQFxCJE5eg+3WHx/qfU6GEO22MXh7a+8AoYu3ahI5ygtzD7H ek8IPT20NhahH6otb9Rokfbt9KUC2aFV/FtBf328= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 5/9] dwarf: Implement DWARF support for modules Date: Thu, 7 Apr 2022 15:25:14 -0500 Message-Id: <20220407202518.19780-6-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132606_108933_7E800955 X-CRM114-Status: GOOD ( 19.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" When a module is loaded, allocate and initialize its struct dwarf_info. When a module is unloaded, free the same. Add code in dwarf_lookup() to look up a given address in modules, if vmlinux does not contain the address. Signed-off-by: Madhavan T. Venkataraman --- include/linux/dwarf.h | 18 ++++++++++ include/linux/module.h | 3 ++ kernel/dwarf_fp.c | 71 ++++++++++++++++++++++++++++++++++--- kernel/module.c | 31 ++++++++++++++++ tools/include/linux/dwarf.h | 18 ++++++++++ 5 files changed, 136 insertions(+), 5 deletions(-) diff --git a/include/linux/dwarf.h b/include/linux/dwarf.h index 3df15e79003c..aa44a414b0b6 100644 --- a/include/linux/dwarf.h +++ b/include/linux/dwarf.h @@ -11,6 +11,7 @@ #define _LINUX_DWARF_H #include +#include /* * objtool generates two special sections that contain DWARF information that @@ -54,11 +55,28 @@ struct dwarf_block { #ifdef CONFIG_DWARF_FP extern struct dwarf_rule *dwarf_lookup(unsigned long pc); +#ifdef CONFIG_MODULES +extern void dwarf_module_alloc(struct module *mod, + struct dwarf_rule *rules, size_t rules_size, + unsigned long *pcs, size_t pcs_size); +extern void dwarf_module_free(struct module *mod); +#endif #else static inline struct dwarf_rule *dwarf_lookup(unsigned long pc) { return NULL; } +#ifdef CONFIG_MODULES +static inline void dwarf_module_alloc(struct module *mod, + struct dwarf_rule *rules, + size_t rules_size, + unsigned long *pcs, size_t pcs_size) +{ +} +static inline void dwarf_module_free(struct module *mod) +{ +} +#endif #endif #endif /* _LINUX_DWARF_H */ diff --git a/include/linux/module.h b/include/linux/module.h index c9f1200b2312..bd7c69b82808 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -538,6 +538,9 @@ struct module { struct error_injection_entry *ei_funcs; unsigned int num_ei_funcs; #endif +#ifdef CONFIG_DWARF_FP + void *dwarf_info; +#endif } ____cacheline_aligned __randomize_layout; #ifndef MODULE_ARCH_INIT #define MODULE_ARCH_INIT {} diff --git a/kernel/dwarf_fp.c b/kernel/dwarf_fp.c index bb14fbe3f3e1..07d647e828cd 100644 --- a/kernel/dwarf_fp.c +++ b/kernel/dwarf_fp.c @@ -164,6 +164,44 @@ static struct dwarf_info *dwarf_alloc(struct dwarf_rule *rules, int nrules, return NULL; } +#ifdef CONFIG_MODULES + +/* + * Errors encountered in this function should not be fatal. All it will mean + * is that stack traces through the module would be considered unreliable. + */ +void dwarf_module_alloc(struct module *mod, + struct dwarf_rule *rules, size_t rules_size, + unsigned long *pcs, size_t pcs_size) +{ + int nrules, npcs; + + mod->dwarf_info = NULL; + + nrules = rules_size / sizeof(*rules); + npcs = pcs_size / sizeof(*pcs); + if (!nrules || npcs != nrules) + return; + + mod->dwarf_info = dwarf_alloc(rules, nrules, pcs); +} + +void dwarf_module_free(struct module *mod) +{ + struct dwarf_info *info; + + info = mod->dwarf_info; + mod->dwarf_info = NULL; + + if (info) { + kfree(info->blocks); + kfree(info->offsets); + kfree(info); + } +} + +#endif + static struct dwarf_rule *dwarf_lookup_rule(struct dwarf_info *info, unsigned long pc) { @@ -212,13 +250,36 @@ static struct dwarf_rule *dwarf_lookup_rule(struct dwarf_info *info, return NULL; } +#ifdef CONFIG_MODULES + +static struct dwarf_rule *dwarf_module_lookup_rule(unsigned long pc) +{ + struct module *mod; + + mod = __module_address(pc); + if (!mod || !mod->dwarf_info) + return NULL; + + return dwarf_lookup_rule(mod->dwarf_info, pc); +} + +#else + +static struct dwarf_rule *dwarf_module_lookup_rule(unsigned long pc) +{ + return NULL; +} + +#endif + struct dwarf_rule *dwarf_lookup(unsigned long pc) { - /* - * Currently, only looks up vmlinux. Support for modules will be - * added later. - */ - return dwarf_lookup_rule(vmlinux_dwarf_info, pc); + struct dwarf_rule *rule; + + rule = dwarf_lookup_rule(vmlinux_dwarf_info, pc); + if (!rule) + rule = dwarf_module_lookup_rule(pc); + return rule; } static int __init dwarf_init_feature(void) diff --git a/kernel/module.c b/kernel/module.c index 84a9141a5e15..d9b73995b70a 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -59,6 +59,7 @@ #include #include #include "module-internal.h" +#include #define CREATE_TRACE_POINTS #include @@ -2153,6 +2154,7 @@ void __weak module_arch_freeing_init(struct module *mod) } static void cfi_cleanup(struct module *mod); +static void module_dwarf_free(struct module *mod); /* Free a module, remove from lists, etc. */ static void free_module(struct module *mod) @@ -2175,6 +2177,9 @@ static void free_module(struct module *mod) /* Arch-specific cleanup. */ module_arch_cleanup(mod); + /* Dwarf cleanup. */ + module_dwarf_free(mod); + /* Module unload stuff */ module_unload_free(mod); @@ -3946,6 +3951,7 @@ static int unknown_module_param_cb(char *param, char *val, const char *modname, } static void cfi_init(struct module *mod); +static void module_dwarf_init(struct module *mod, struct load_info *info); /* * Allocate and load the module: note that size of section 0 is always @@ -4074,6 +4080,8 @@ static int load_module(struct load_info *info, const char __user *uargs, if (err < 0) goto free_modinfo; + module_dwarf_init(mod, info); + flush_module_icache(mod); /* Setup CFI for the module. */ @@ -4154,6 +4162,7 @@ static int load_module(struct load_info *info, const char __user *uargs, kfree(mod->args); free_arch_cleanup: cfi_cleanup(mod); + module_dwarf_free(mod); module_arch_cleanup(mod); free_modinfo: free_modinfo(mod); @@ -4542,6 +4551,28 @@ static void cfi_cleanup(struct module *mod) #endif } +static void module_dwarf_init(struct module *mod, struct load_info *info) +{ + Elf_Shdr *dwarf_rules, *dwarf_pcs; + + dwarf_rules = &info->sechdrs[find_sec(info, ".dwarf_rules")]; + dwarf_pcs = &info->sechdrs[find_sec(info, ".dwarf_pcs")]; + + if (!dwarf_rules || !dwarf_pcs) + return; + + dwarf_module_alloc(mod, + (void *) dwarf_rules->sh_addr, + dwarf_rules->sh_size, + (void *) dwarf_pcs->sh_addr, + dwarf_pcs->sh_size); +} + +static void module_dwarf_free(struct module *mod) +{ + dwarf_module_free(mod); +} + /* Maximum number of characters written by module_flags() */ #define MODULE_FLAGS_BUF_SIZE (TAINT_FLAGS_COUNT + 4) diff --git a/tools/include/linux/dwarf.h b/tools/include/linux/dwarf.h index 3df15e79003c..aa44a414b0b6 100644 --- a/tools/include/linux/dwarf.h +++ b/tools/include/linux/dwarf.h @@ -11,6 +11,7 @@ #define _LINUX_DWARF_H #include +#include /* * objtool generates two special sections that contain DWARF information that @@ -54,11 +55,28 @@ struct dwarf_block { #ifdef CONFIG_DWARF_FP extern struct dwarf_rule *dwarf_lookup(unsigned long pc); +#ifdef CONFIG_MODULES +extern void dwarf_module_alloc(struct module *mod, + struct dwarf_rule *rules, size_t rules_size, + unsigned long *pcs, size_t pcs_size); +extern void dwarf_module_free(struct module *mod); +#endif #else static inline struct dwarf_rule *dwarf_lookup(unsigned long pc) { return NULL; } +#ifdef CONFIG_MODULES +static inline void dwarf_module_alloc(struct module *mod, + struct dwarf_rule *rules, + size_t rules_size, + unsigned long *pcs, size_t pcs_size) +{ +} +static inline void dwarf_module_free(struct module *mod) +{ +} +#endif #endif #endif /* _LINUX_DWARF_H */ From patchwork Thu Apr 7 20:25:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9147FC433F5 for ; Thu, 7 Apr 2022 20:28:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9C4etr6wrPvDnLfkmEKv2N4lcolALYMu7LHy7ew2ob0=; b=OfgbPAW3p1Un6X +1VAjahtv6rvCEdMpsW141VEVC/eF0fyT6xB9iPqkYN6DIyMA3+Mwpc3srGWFi5PRVjs+C9Lfz18x ZWcs+lSiwBPf/Fc9rTP87BH7kHyolZdZvV+bymSoGehKfpsI/vpwF5T4nm39b9ixtXBSqTxD3VgJq 6EeSGQTGIH2vp5/u6CVRACSkfTxrOMKPcJrLCE5wVuBPUYImnj9wn77LDLL1hBfyubsVyLnVFuOVL ZY7pi5ncxJCwUMgLB0KIn3GCtDq1B6MFmGnV9YHsQvhsBauFY25311sGBPowDK1KUoGiFcnK5k/cJ csGQjuyzVjNxHC/MEaRg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYiq-00Drwo-4J; Thu, 07 Apr 2022 20:27:16 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhi-00Dra3-F4 for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:08 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 5396D201CBCE; Thu, 7 Apr 2022 13:26:05 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5396D201CBCE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363166; bh=IxH5f/rG0qHAdKJ19wqUTznmank9iR+DfeF+leQMuho=; h=From:To:Subject:Date:In-Reply-To:References:From; b=OpiX1ntGZX7hhslS/KHl/yvU0PMaM3l/8k8ZBwmn42T/pvpIKsU6CMkMeVKdK5PVg BHokwtZ/S90gwxJH9SGOimfby3wNdOnnrcDZOchbV/B0sMGriMv0aTYjA9KjKdCcts 4c3M1J1ZKiJHAq7Y9nF+h7uKx3EsyZ3CAkq412IM= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 6/9] arm64: unwinder: Add a reliability check in the unwinder based on DWARF CFI Date: Thu, 7 Apr 2022 15:25:15 -0500 Message-Id: <20220407202518.19780-7-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132606_613569_C104CC79 X-CRM114-Status: GOOD ( 20.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Introduce a reliability flag in struct stackframe. This will be set to false if the PC does not have valid DWARF rules or if the frame pointer computed from the DWARF rules does not match the actual frame pointer. Now that the unwinder can validate the frame pointer, introduce arch_stack_walk_reliable(). Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 6 +++ arch/arm64/kernel/stacktrace.c | 69 +++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 6564a01cc085..93adee4219ed 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -5,6 +5,7 @@ #ifndef __ASM_STACKTRACE_H #define __ASM_STACKTRACE_H +#include #include #include #include @@ -35,6 +36,7 @@ struct stack_info { * A snapshot of a frame record or fp/lr register values, along with some * accounting information necessary for robust unwinding. * + * @sp: The sp value (CFA) at the call site of the current function. * @fp: The fp value in the frame record (or the real fp) * @pc: The lr value in the frame record (or the real lr) * @@ -47,8 +49,11 @@ struct stack_info { * @prev_type: The type of stack this frame record was on, or a synthetic * value of STACK_TYPE_UNKNOWN. This is used to detect a * transition from one stack to another. + * + * @reliable Stack trace is reliable. */ struct stackframe { + unsigned long sp; unsigned long fp; unsigned long pc; DECLARE_BITMAP(stacks_done, __NR_STACK_TYPES); @@ -57,6 +62,7 @@ struct stackframe { #ifdef CONFIG_KRETPROBES struct llist_node *kr_cur; #endif + bool reliable; }; extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame); diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 94f83cd44e50..f9ef7a3e7296 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -5,6 +5,7 @@ * Copyright (C) 2012 ARM Ltd. */ #include +#include #include #include #include @@ -36,8 +37,22 @@ void start_backtrace(struct stackframe *frame, unsigned long fp, unsigned long pc) { + struct dwarf_rule *rule; + + frame->reliable = true; frame->fp = fp; frame->pc = pc; + frame->sp = 0; + /* + * Lookup the dwarf rule for PC. If it exists, initialize the SP + * based on the frame pointer passed in. + */ + rule = dwarf_lookup(pc); + if (rule) + frame->sp = fp - rule->fp_offset; + else + frame->reliable = false; + #ifdef CONFIG_KRETPROBES frame->kr_cur = NULL; #endif @@ -67,6 +82,8 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) { unsigned long fp = frame->fp; struct stack_info info; + struct dwarf_rule *rule; + unsigned long lookup_pc; if (!tsk) tsk = current; @@ -137,6 +154,32 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur); #endif + /* + * If it is the last frame, no need to check dwarf. + */ + if (frame->fp == (unsigned long)task_pt_regs(tsk)->stackframe) + return 0; + + if (!frame->reliable) { + /* + * The sp value cannot be reliably computed anymore because a + * previous frame was unreliable. + */ + return 0; + } + lookup_pc = frame->pc; + + rule = dwarf_lookup(lookup_pc); + if (!rule) { + frame->reliable = false; + return 0; + } + + frame->sp += rule->sp_offset; + if (frame->fp != (frame->sp + rule->fp_offset)) { + frame->reliable = false; + return 0; + } return 0; } NOKPROBE_SYMBOL(unwind_frame); @@ -242,4 +285,30 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, walk_stackframe(task, &frame, consume_entry, cookie); } +noinline int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry, + void *cookie, struct task_struct *task) +{ + struct stackframe frame; + int ret = 0; + + if (task == current) { + start_backtrace(&frame, + (unsigned long)__builtin_frame_address(1), + (unsigned long)__builtin_return_address(0)); + } else { + start_backtrace(&frame, thread_saved_fp(task), + thread_saved_pc(task)); + } + + while (!ret) { + if (!frame.reliable) + return -EINVAL; + if (!consume_entry(cookie, frame.pc)) + return -EINVAL; + ret = unwind_frame(task, &frame); + } + + return ret == -ENOENT ? 0 : -EINVAL; +} + #endif From patchwork Thu Apr 7 20:25:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59F60C433F5 for ; Thu, 7 Apr 2022 20:30:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZXjAb3RWJfnHETluWFqF+tRbh4jMBnv/ZyrTpkNpYsA=; b=2Ri7fDz1VJ8+Fz fAeUvR4PsVkSeU8lTjvrXmd1ggTYScQE3D6n2eB9Xfk6IP9gcdcKo8CJAz6w1pjtKm0GpKwh7NNsf cd7dbP88b5tWqKlm9sv91iURCMVlrXOIHOoOjXH0RtyZGvhZR1hFHjrf73e9paPbLqzNdkqHpTgCy QViTw5Lr6tHjD8H0NMxYYhgvCF+WgcASFO7USAqmLRd/Nu37uanX9HxYTyJ4JqatvVz3rBPSVp+XY MCDEJZWl/ErRW9HpJsXpMGZLNn4wEdWABE7KqYD7m81nescnWkCq3XQuWl8TH5qahXIY31Ji/VYov 6cBfLCMHVgg6RusJaA8A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYkC-00DsqX-Ey; Thu, 07 Apr 2022 20:28:41 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhj-00DraT-8U for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:10 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 4BD3C201CBCF; Thu, 7 Apr 2022 13:26:06 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 4BD3C201CBCF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363167; bh=hYLMxT9RiPmkiYACbYAX5mWJRkIz7CIC+wOf5vZyu3Q=; h=From:To:Subject:Date:In-Reply-To:References:From; b=mBjCIDHJ96jxJKPOUf8zcQKMng/Hgf9OifpJdSyZA8VFqGFQJldl0S6+iDHW5YT8L sMkGhraHJ6cLNP6TATgVUol+0TNpuzpPxXqXg4acGS+9qdxu57n1yps409E7ij6DSL NCidy/LUpAqTU1kMkM592RTyhSDS3BeM2+npWN6U= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 7/9] arm64: dwarf: Implement unwind hints Date: Thu, 7 Apr 2022 15:25:16 -0500 Message-Id: <20220407202518.19780-8-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132607_433874_055EAA16 X-CRM114-Status: GOOD ( 36.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" DWARF information is not generated for assembly functions. However, we would like to unwind through some of these functions as they may occur frequently in a stacktrace. Implement unwind hints for this purpose. An unwind hint placed after a desired call instruction creates a DWARF rule for that instruction address in a special section called ".discard.unwind_hints". objtool merges these DWARF rules along with the ones generated by the compiler for C functions. Add unwind hints for the following cases: Exception handlers ================== When an exception is taken in the kernel, an exception frame gets pushed on the stack. To be able to unwind through the special frame, place an unwind hint in the exception handler macro. Interrupt handlers ================== When an interrupt happens, the exception handler switches to an IRQ stack and calls the interrupt handler. Place an unwind hint after the call to the interrupt handler so the unwinder can unwind through the switched stacks. FTrace stub =========== ftrace_common() calls ftrace_stub(). The stub is patched with tracer functions when tracing is enabled. Place an unwind hint right after the call to the stub function so that stack traces taken from within a tracer function can correctly unwind to the caller. FTrace Graph Caller =================== ftrace_graph_caller() calls a function, prepare_ftrace_return(), to prepare for graph tracing. Place an unwind hint after the call so a stack trace taken from within the prepare function can correctly unwind to the caller. FTrace callsite =============== ftrace_regs_entry() sets up two stackframes - one for the callsite and one for the ftrace entry code. Unwind hints have been placed for the ftrace entry code above. We need an unwind hint for the callsite. Callsites are numerous. But the unwind hint required for all the callsites is the same. Define a dummy function with the callsite unwind hint like this: SYM_CODE_START(ftrace_callsite) unwind_hint 4, 16, -16 // for the callsite ret SYM_CODE_END(ftrace_callsite) When the unwinder comes across an ftrace entry, it will change the PC that it needs to lookup to ftrace_callsite() to obtain the unwind hint for the callsite like this: #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS if (is_ftrace_entry(frame->prev_pc)) lookup_pc = (unsigned long)&ftrace_callsite; #endif This way, the unwinder can unwind through an ftrace callsite. Kretprobe Trampoline ==================== This trampoline sets up pt_regs on the stack and sets up a synthetic frame in the pt_regs. Place an unwind hint where trampoline_probe_handler() returns to __kretprobe_trampoline to unwind through the synthetic frame. is_ftrace_entry() ================= The code for this function has been borrowed from Suraj Jitindar Singh. Signed-off-by: Madhavan T. Venkataraman Signed-off-by: Suraj Jitindar Singh --- arch/arm64/include/asm/stacktrace.h | 3 + arch/arm64/include/asm/unwind_hints.h | 28 ++++++++ arch/arm64/kernel/entry-ftrace.S | 23 ++++++ arch/arm64/kernel/entry.S | 3 + arch/arm64/kernel/ftrace.c | 16 +++++ arch/arm64/kernel/probes/kprobes_trampoline.S | 2 + arch/arm64/kernel/stacktrace.c | 70 +++++++++++++++++-- include/linux/dwarf.h | 10 ++- include/linux/ftrace.h | 4 ++ tools/include/linux/dwarf.h | 10 ++- tools/objtool/dwarf_parse.c | 59 +++++++++++++++- tools/objtool/dwarf_rules.c | 65 ++++++++++++++++- tools/objtool/include/objtool/dwarf_def.h | 10 +++ 13 files changed, 294 insertions(+), 9 deletions(-) create mode 100644 arch/arm64/include/asm/unwind_hints.h diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 93adee4219ed..90392097a768 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -46,6 +46,7 @@ struct stack_info { * @prev_fp: The fp that pointed to this frame record, or a synthetic value * of 0. This is used to ensure that within a stack, each * subsequent frame record is at an increasing address. + * @prev_pc: The pc in the previous frame. * @prev_type: The type of stack this frame record was on, or a synthetic * value of STACK_TYPE_UNKNOWN. This is used to detect a * transition from one stack to another. @@ -58,11 +59,13 @@ struct stackframe { unsigned long pc; DECLARE_BITMAP(stacks_done, __NR_STACK_TYPES); unsigned long prev_fp; + unsigned long prev_pc; enum stack_type prev_type; #ifdef CONFIG_KRETPROBES struct llist_node *kr_cur; #endif bool reliable; + bool synthetic_frame; }; extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame); diff --git a/arch/arm64/include/asm/unwind_hints.h b/arch/arm64/include/asm/unwind_hints.h new file mode 100644 index 000000000000..b2312bfaf201 --- /dev/null +++ b/arch/arm64/include/asm/unwind_hints.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_ARM64_DWARF_UNWIND_HINTS_H +#define _ASM_ARM64_DWARF_UNWIND_HINTS_H + +#ifdef __ASSEMBLY__ + +#ifdef CONFIG_DWARF_FP + +.macro dwarf_unwind_hint, size, sp_offset, fp_offset +.Ldwarf_hint_pc_\@ : .pushsection .discard.unwind_hints + /* struct dwarf_unwind_hint */ + .long 0, (.Ldwarf_hint_pc_\@ - .) + .int \size + .short \sp_offset + .short \fp_offset + .popsection +.endm + +#else + +.macro dwarf_unwind_hint, size, sp_offset, fp_offset +.endm + +#endif + +#endif /* __ASSEMBLY__ */ + +#endif /* _ASM_ARM64_DWARF_UNWIND_HINTS_H */ diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 8cf970d219f5..f82dad9260ec 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -11,6 +11,7 @@ #include #include #include +#include #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS /* @@ -99,7 +100,14 @@ SYM_CODE_START(ftrace_common) mov x3, sp // regs SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) + /* + * Tracer functions are patched at ftrace_stub. Stack traces + * taken from tracer functions will end up here. Place an + * unwind hint based on the stackframe setup in ftrace_regs_entry. + */ bl ftrace_stub +SYM_INNER_LABEL(ftrace_call_entry, SYM_L_GLOBAL) + dwarf_unwind_hint 4, PT_REGS_SIZE, (S_STACKFRAME - PT_REGS_SIZE) #ifdef CONFIG_FUNCTION_GRAPH_TRACER SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) // ftrace_graph_caller(); @@ -138,10 +146,25 @@ SYM_CODE_START(ftrace_graph_caller) add x1, sp, #S_LR // parent_ip (callsite's LR) ldr x2, [sp, #PT_REGS_SIZE] // parent fp (callsite's FP) bl prepare_ftrace_return +SYM_INNER_LABEL(ftrace_graph_caller_entry, SYM_L_GLOBAL) + dwarf_unwind_hint 4, PT_REGS_SIZE, (S_STACKFRAME - PT_REGS_SIZE) b ftrace_common_return SYM_CODE_END(ftrace_graph_caller) #endif +/* + * ftrace_regs_entry() sets up two stackframes - one for the callsite and + * one for the ftrace entry code. Unwind hints have been placed for the + * ftrace entry code above. We need an unwind hint for the callsite. Callsites + * are numerous. But the unwind hint required for all the callsites is the + * same. Define a dummy function here with the callsite unwind hint for the + * benefit of the unwinder. + */ +SYM_CODE_START(ftrace_callsite) + dwarf_unwind_hint 4, 16, -16 // for the callsite + ret +SYM_CODE_END(ftrace_callsite) + #else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ /* diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 2f69ae43941d..a188e3a3068d 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -28,6 +28,7 @@ #include #include #include +#include .macro clear_gp_regs .irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29 @@ -551,6 +552,7 @@ SYM_CODE_START_LOCAL(el\el\ht\()_\regsize\()_\label) .if \el == 0 b ret_to_user .else + dwarf_unwind_hint 4, PT_REGS_SIZE, (S_STACKFRAME - PT_REGS_SIZE) b ret_to_kernel .endif SYM_CODE_END(el\el\ht\()_\regsize\()_\label) @@ -783,6 +785,7 @@ SYM_FUNC_START(call_on_irq_stack) /* Move to the new stack and call the function there */ mov sp, x16 blr x1 + dwarf_unwind_hint 4, 0, -16 /* * Restore the SP from the FP, and restore the FP and LR from the frame diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index 4506c4a90ac1..ec9a00d714e5 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -299,3 +299,19 @@ int ftrace_disable_ftrace_graph_caller(void) } #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + +bool is_ftrace_entry(unsigned long pc) +{ + if (pc == (unsigned long)&ftrace_call_entry) + return true; + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + if (pc == (unsigned long)&ftrace_graph_caller_entry) + return true; +#endif + return false; +} + +#endif diff --git a/arch/arm64/kernel/probes/kprobes_trampoline.S b/arch/arm64/kernel/probes/kprobes_trampoline.S index 9a6499bed58b..325932b49857 100644 --- a/arch/arm64/kernel/probes/kprobes_trampoline.S +++ b/arch/arm64/kernel/probes/kprobes_trampoline.S @@ -6,6 +6,7 @@ #include #include #include +#include .text @@ -71,6 +72,7 @@ SYM_CODE_START(__kretprobe_trampoline) mov x0, sp bl trampoline_probe_handler + dwarf_unwind_hint 4, PT_REGS_SIZE, (S_FP - PT_REGS_SIZE) /* * Replace trampoline address in lr with actual orig_ret_addr return * address. diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index f9ef7a3e7296..e1a9b695b6ae 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -19,6 +19,16 @@ #include #include +static inline bool is_synthetic_frame(struct dwarf_rule *rule) +{ + short regs_size = sizeof(struct pt_regs); + short frame_offset = offsetof(struct pt_regs, stackframe); + + return rule->hint && + rule->sp_offset == regs_size && + rule->fp_offset == (frame_offset - regs_size); +} + /* * AArch64 PCS assigns the frame pointer to x29. * @@ -40,6 +50,7 @@ void start_backtrace(struct stackframe *frame, unsigned long fp, struct dwarf_rule *rule; frame->reliable = true; + frame->synthetic_frame = false; frame->fp = fp; frame->pc = pc; frame->sp = 0; @@ -48,10 +59,12 @@ void start_backtrace(struct stackframe *frame, unsigned long fp, * based on the frame pointer passed in. */ rule = dwarf_lookup(pc); - if (rule) + if (rule) { frame->sp = fp - rule->fp_offset; - else + frame->synthetic_frame = is_synthetic_frame(rule); + } else { frame->reliable = false; + } #ifdef CONFIG_KRETPROBES frame->kr_cur = NULL; @@ -125,6 +138,7 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) * Record this frame record's values and location. The prev_fp and * prev_type are only meaningful to the next unwind_frame() invocation. */ + frame->prev_pc = frame->pc; frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp)); frame->pc = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8)); frame->prev_fp = fp; @@ -168,11 +182,59 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) return 0; } lookup_pc = frame->pc; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + if (is_ftrace_entry(frame->prev_pc)) + lookup_pc = (unsigned long)&ftrace_callsite; +#endif rule = dwarf_lookup(lookup_pc); if (!rule) { - frame->reliable = false; - return 0; + if (!frame->synthetic_frame) { + /* + * If the last instruction in a function happens to be + * a call instruction, the return address would fall + * outside of the function. This could be that case. + * This can happen, for instance, if the called function + * is a "noreturn" function. The compiler can optimize + * away the instructions after the call. So, adjust the + * PC so it falls inside the function and retry. + * + * However, if the previous frame was a synthetic frame + * (e.g., interrupt/exception), the return PC in the + * synthetic frame may not be the location after a call + * instruction at all. In such cases, we don't want to + * adjust the PC and retry. + * + * If this succeeds, adjust the frame pc below. + */ + lookup_pc -= 4; + rule = dwarf_lookup(lookup_pc); + } + if (!rule) { + frame->reliable = false; + return 0; + } + frame->pc = lookup_pc; + } + frame->synthetic_frame = false; + + if (rule->hint) { + if (!rule->sp_offset && !rule->fp_offset) { + frame->reliable = false; + return 0; + } + frame->synthetic_frame = is_synthetic_frame(rule); + if (!rule->sp_offset) { + /* + * This is the unwind hint in call_on_irq_stack(). The + * SP at this point is in the IRQ stack. The CFA and + * the FP are on the normal stack. To compute the CFA, + * rely on the unwind hint, assume that the FP is good + * and just compute the CFA from it. + */ + frame->sp = frame->fp - rule->fp_offset; + return 0; + } } frame->sp += rule->sp_offset; diff --git a/include/linux/dwarf.h b/include/linux/dwarf.h index aa44a414b0b6..fea42feb48a4 100644 --- a/include/linux/dwarf.h +++ b/include/linux/dwarf.h @@ -34,9 +34,17 @@ * This contains an array of starting PCs, one for each rule. */ struct dwarf_rule { - unsigned int size:30; + unsigned int size:29; unsigned int sp_saved:1; unsigned int fp_saved:1; + unsigned int hint:1; + short sp_offset; + short fp_offset; +}; + +struct dwarf_unwind_hint { + unsigned long pc; + unsigned int size; short sp_offset; short fp_offset; }; diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 9999e29187de..dbcd95053425 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -604,6 +604,10 @@ extern int ftrace_update_ftrace_func(ftrace_func_t func); extern void ftrace_caller(void); extern void ftrace_regs_caller(void); extern void ftrace_call(void); +extern void ftrace_call_entry(void); +extern void ftrace_graph_caller_entry(void); +extern void ftrace_callsite(void); +extern bool is_ftrace_entry(unsigned long pc); extern void ftrace_regs_call(void); extern void mcount_call(void); diff --git a/tools/include/linux/dwarf.h b/tools/include/linux/dwarf.h index aa44a414b0b6..fea42feb48a4 100644 --- a/tools/include/linux/dwarf.h +++ b/tools/include/linux/dwarf.h @@ -34,9 +34,17 @@ * This contains an array of starting PCs, one for each rule. */ struct dwarf_rule { - unsigned int size:30; + unsigned int size:29; unsigned int sp_saved:1; unsigned int fp_saved:1; + unsigned int hint:1; + short sp_offset; + short fp_offset; +}; + +struct dwarf_unwind_hint { + unsigned long pc; + unsigned int size; short sp_offset; short fp_offset; }; diff --git a/tools/objtool/dwarf_parse.c b/tools/objtool/dwarf_parse.c index d5ac5630fbba..7a5af7d5c2be 100644 --- a/tools/objtool/dwarf_parse.c +++ b/tools/objtool/dwarf_parse.c @@ -18,6 +18,10 @@ struct objtool_file *dwarf_file; struct section *debug_frame; +struct section *unwind_hints; +struct section *unwind_hints_reloc; +static int nhints; + struct cie *cies, *cur_cie; struct fde *fdes, *cur_fde; @@ -31,6 +35,8 @@ static unsigned int offset_size; static u64 entry_length; static unsigned char *saved_start; +static int dwarf_add_hints(void); + /* * Parse and create a new CIE. */ @@ -270,7 +276,7 @@ int dwarf_parse(struct objtool_file *file) */ debug_frame = find_section_by_name(file->elf, ".debug_frame"); if (!debug_frame) - return 0; + goto hints; dwarf_alloc_init(); @@ -290,5 +296,56 @@ int dwarf_parse(struct objtool_file *file) * Run all the DWARF instructions in the CIEs and FDEs. */ dwarf_parse_instructions(); +hints: + unwind_hints = find_section_by_name(file->elf, ".discard.unwind_hints"); + if (!unwind_hints) + return 0; + + unwind_hints_reloc = unwind_hints->reloc; + + if (unwind_hints->sh.sh_size % sizeof(struct dwarf_unwind_hint)) { + WARN("struct dwarf_unwind_hint size mismatch"); + return -1; + } + nhints = unwind_hints->sh.sh_size / sizeof(struct dwarf_unwind_hint); + + return dwarf_add_hints(); +} + +/* + * Errors in this function are non-fatal. The worst case is that the + * unwind hints will not be part of the dwarf rules. That is all. + */ +static int dwarf_add_hints(void) +{ + struct dwarf_unwind_hint *hints, *hint; + struct reloc *reloc; + int i; + struct section *sec; + + if (!nhints) + return 0; + + hints = (struct dwarf_unwind_hint *) unwind_hints->data->d_buf; + for (i = 0; i < nhints; i++) { + hint = &hints[i]; + if (unwind_hints_reloc) { + reloc = find_reloc_by_dest_range(dwarf_file->elf, + unwind_hints, + i * sizeof(*hint), + sizeof(*hint)); + if (!reloc) { + WARN("can't find reloc for hint %d", i); + return 0; + } + hint->pc = reloc->addend; + sec = reloc->sym->sec; + } else { + sec = find_section_by_name(dwarf_file->elf, ".text"); + if (!sec) + return 0; + } + dwarf_hint_add(sec, hint); + } return 0; } diff --git a/tools/objtool/dwarf_rules.c b/tools/objtool/dwarf_rules.c index a118b392aac8..632776463c23 100644 --- a/tools/objtool/dwarf_rules.c +++ b/tools/objtool/dwarf_rules.c @@ -19,6 +19,9 @@ struct section *dwarf_pcs_sec; static struct fde_entry *cur_entry; static int nentries; +static struct hint_entry *hint_list; +static int nhints; + static int dwarf_rule_insert(struct fde *fde, unsigned long addr, struct rule *sp_rule, struct rule *fp_rule); @@ -51,6 +54,25 @@ int dwarf_rule_add(struct fde *fde, unsigned long addr, return dwarf_rule_insert(fde, addr, sp_rule, fp_rule); } +int dwarf_hint_add(struct section *section, struct dwarf_unwind_hint *hint) +{ + struct hint_entry *entry; + + entry = dwarf_alloc(sizeof(*entry)); + if (!entry) + return -1; + + /* Add the entry to the hints list. */ + entry->next = hint_list; + hint_list = entry; + + entry->section = section; + entry->hint = hint; + nhints++; + + return 0; +} + void dwarf_rule_next(struct fde *fde, unsigned long addr) { if (cur_entry) { @@ -119,6 +141,7 @@ static int dwarf_rule_write(struct elf *elf, struct fde *fde, rule.size = entry->size; rule.sp_saved = entry->sp_rule.saved; rule.fp_saved = entry->fp_rule.saved; + rule.hint = 0; rule.sp_offset = entry->sp_rule.offset; rule.fp_offset = entry->fp_rule.offset; @@ -135,11 +158,42 @@ static int dwarf_rule_write(struct elf *elf, struct fde *fde, return 0; } +static int dwarf_hint_write(struct elf *elf, struct hint_entry *hentry, + unsigned int index) +{ + struct dwarf_rule rule, *drule; + struct dwarf_unwind_hint *hint = hentry->hint; + + /* + * Encode the SP and FP rules from the entry into a single dwarf_rule + * for the kernel's benefit. Copy it into .dwarf_rules. + */ + rule.size = hint->size; + rule.sp_saved = 0; + rule.fp_saved = 1; + rule.hint = 1; + rule.sp_offset = hint->sp_offset; + rule.fp_offset = hint->fp_offset; + + drule = (struct dwarf_rule *) dwarf_rules_sec->data->d_buf + index; + memcpy(drule, &rule, sizeof(rule)); + + /* Add relocation information for the code range. */ + if (elf_add_reloc_to_insn(elf, dwarf_pcs_sec, + index * sizeof(unsigned long), + R_AARCH64_ABS64, + hentry->section, hint->pc)) { + return -1; + } + return 0; +} + int dwarf_write(struct objtool_file *file) { struct elf *elf = file->elf; struct fde *fde; struct fde_entry *entry; + struct hint_entry *hentry; int index; /* @@ -154,7 +208,7 @@ int dwarf_write(struct objtool_file *file) /* Create .dwarf_rules. */ dwarf_rules_sec = elf_create_section(elf, ".dwarf_rules", 0, sizeof(struct dwarf_rule), - nentries); + nentries + nhints); if (!dwarf_rules_sec) { WARN("Unable to create .dwarf_rules"); return -1; @@ -162,7 +216,8 @@ int dwarf_write(struct objtool_file *file) /* Create .dwarf_pcs. */ dwarf_pcs_sec = elf_create_section(elf, ".dwarf_pcs", 0, - sizeof(unsigned long), nentries); + sizeof(unsigned long), + nentries + nhints); if (!dwarf_pcs_sec) { WARN("Unable to create .dwarf_pcs"); return -1; @@ -178,6 +233,12 @@ int dwarf_write(struct objtool_file *file) } } + for (hentry = hint_list; hentry; hentry = hentry->next) { + if (dwarf_hint_write(elf, hentry, index)) + return -1; + index++; + } + return 0; } diff --git a/tools/objtool/include/objtool/dwarf_def.h b/tools/objtool/include/objtool/dwarf_def.h index af56ccb52fff..afa6db6c3828 100644 --- a/tools/objtool/include/objtool/dwarf_def.h +++ b/tools/objtool/include/objtool/dwarf_def.h @@ -304,6 +304,15 @@ struct fde { struct fde_entry *tail; }; +/* + * Entry for an unwind hint. + */ +struct hint_entry { + struct hint_entry *next; + struct section *section; + struct dwarf_unwind_hint *hint; +}; + /* * These are identifiers for 32-bit and 64-bit CIE entries respectively. * These identifiers distinguish a CIE from an FDE in .debug_frame. @@ -429,6 +438,7 @@ extern u64 (*get_value)(unsigned char *field, unsigned int size); void dwarf_rule_start(struct fde *fde); int dwarf_rule_add(struct fde *fde, unsigned long addr, struct rule *sp_rule, struct rule *fp_rule); +int dwarf_hint_add(struct section *section, struct dwarf_unwind_hint *hint); void dwarf_rule_next(struct fde *fde, unsigned long addr); void dwarf_rule_reset(struct fde *fde); int arch_dwarf_fde_reloc(struct fde *fde); From patchwork Thu Apr 7 20:25:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17E23C433FE for ; Thu, 7 Apr 2022 20:28:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4i/tPsF+04dAHGfcviFUeQR9hbl/Q2+45bpKFDRWNPY=; b=NJehf2EQnp3Nkw 6IPlzNqls4K4UT6rZ5zloYjInPKfLH7eCOV/GpGnHTOrx8Pe+GjYkl9ouV6lEuboSFRddXAFY0g2Z H7OeDaoAMi1P+22/6gxSP3wXOzcrvIEfwISVJStOIL1eX0wfWwDUxlMtoUHHz9wayTvbnLidROBjh HRO8POV9sn0qyMec1rwR5adiDCJ/n17K5Sr5L0kCZohtgG8qx5nXoa1kIX5PQ8h3Y7ddPfnilEHy4 3kfB1xrO8vG+qnB0HMyNGiI4H2Zo6mBgNakyRemKwez+CX+G2XRJEAzJk+UKLMvVWwfw6qwZjTdPE C4efK/2z2YK72Pd+3QJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYjM-00DsHp-E6; Thu, 07 Apr 2022 20:27:48 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhk-00Drai-6y for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:09 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 3F0EA20BE4A2; Thu, 7 Apr 2022 13:26:07 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 3F0EA20BE4A2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363168; bh=1f354olQC82jt6SKYXp7qOC11JdNZtSHsPWQ+YANWz4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Mr+4sZOPJJ4ze0MzCAM8sWBqpc1ChwRlWwhuQWpM+3LQ2oMGgwPO07iENgnKGLMhH qVn40PS2Fv/tMaFsh36C01v6AxLahKYjfuyLEKJTxtkT2V9epGseQlnqGfPCxVfSB/ PzlBnPjsZbY7+tm4o7VN9PhMqHN957uSd42016nE= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 8/9] dwarf: Miscellaneous changes required for enabling livepatch Date: Thu, 7 Apr 2022 15:25:17 -0500 Message-Id: <20220407202518.19780-9-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132608_389778_71CF0FDA X-CRM114-Status: GOOD ( 17.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" - Create arch/arm64/include/asm/livepatch.h and define klp_arch_set_pc() and klp_get_ftrace_location() which are required for livepatch. - Define TIF_PATCH_PENDING in arch/arm64/include/asm/thread_info.h for livepatch. - Check TIF_PATCH_PENDING in do_notify_resume() to patch the current task for livepatch. Signed-off-by: Suraj Jitindar Singh --- arch/arm64/include/asm/livepatch.h | 42 ++++++++++++++++++++++++++++ arch/arm64/include/asm/thread_info.h | 4 ++- arch/arm64/kernel/signal.c | 4 +++ 3 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/livepatch.h diff --git a/arch/arm64/include/asm/livepatch.h b/arch/arm64/include/asm/livepatch.h new file mode 100644 index 000000000000..72d7cd86f158 --- /dev/null +++ b/arch/arm64/include/asm/livepatch.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * livepatch.h - arm64-specific Kernel Live Patching Core + */ +#ifndef _ASM_ARM64_LIVEPATCH_H +#define _ASM_ARM64_LIVEPATCH_H + +#include + +static inline void klp_arch_set_pc(struct ftrace_regs *fregs, unsigned long ip) +{ + struct pt_regs *regs = ftrace_get_regs(fregs); + + regs->pc = ip; +} + +/* + * klp_get_ftrace_location is expected to return the address of the BL to the + * relevant ftrace handler in the callsite. The location of this can vary based + * on several compilation options. + * CONFIG_DYNAMIC_FTRACE_WITH_REGS + * - Inserts 2 nops on function entry the second of which is the BL + * referenced above. (See ftrace_init_nop() for the callsite sequence) + * (this is required by livepatch and must be selected) + * CONFIG_ARM64_BTI_KERNEL: + * - Inserts a hint #0x22 on function entry if the function is called + * indirectly (to satisfy BTI requirements), which is inserted before + * the two nops from above. + */ +#define klp_get_ftrace_location klp_get_ftrace_location +static inline unsigned long klp_get_ftrace_location(unsigned long faddr) +{ + unsigned long addr = faddr + AARCH64_INSN_SIZE; + +#if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) + addr = ftrace_location_range(addr, addr + AARCH64_INSN_SIZE); +#endif + + return addr; +} + +#endif /* _ASM_ARM64_LIVEPATCH_H */ diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index e1317b7c4525..a1d8999dbdcc 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -68,6 +68,7 @@ int arch_dup_task_struct(struct task_struct *dst, #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_MTE_ASYNC_FAULT 5 /* MTE Asynchronous Tag Check Fault */ #define TIF_NOTIFY_SIGNAL 6 /* signal notifications exist */ +#define TIF_PATCH_PENDING 7 /* pending live patching update */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -98,11 +99,12 @@ int arch_dup_task_struct(struct task_struct *dst, #define _TIF_SVE (1 << TIF_SVE) #define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) +#define _TIF_PATCH_PENDING (1 << TIF_PATCH_PENDING) #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ _TIF_UPROBE | _TIF_MTE_ASYNC_FAULT | \ - _TIF_NOTIFY_SIGNAL) + _TIF_NOTIFY_SIGNAL | _TIF_PATCH_PENDING) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 8f6372b44b65..b42dffc71fb0 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -937,6 +938,9 @@ void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) (void __user *)NULL, current); } + if (thread_flags & _TIF_PATCH_PENDING) + klp_update_patch_state(current); + if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) do_signal(regs); From patchwork Thu Apr 7 20:25:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12805685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E86BCC433EF for ; Thu, 7 Apr 2022 20:29:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OzltYz8aX4LFHMEDTSUuXV7f2GI/fz1b1EcZhdH8/AU=; b=SABoUapsSgPFDP UcL5EpUEhTFexfTldXQtZKWM2ftYnSx7Lhl8EZFunBuzGRqE94S+wHsCw9IHhG80VwD7FeuaXbA23 /ACqQUZ4GM/TEWgM14iUo+mAcepG8z6oXEziKWvxhPZQzRxzzVXin25WgZH5wYzNEbboQB5JlhkhI SiPzP9+DfKa0hvskqnckghBrx7UpbRmlbFAZMYvNUXe3GItPsNbhayDKQAEmcjCBzE6GCikFPzap2 sPKieS4ShJmHo4gEqZZ/aLx1gLe+GBS/m6j7j4iV4gn2NKqo//IK+d9HaBdA22i9ErUfuw61hMfo3 bnv6VIdhikhhfqPYik0w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYjg-00DsW4-D1; Thu, 07 Apr 2022 20:28:08 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncYhl-00DrZg-45 for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 20:26:10 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 324E6201CBCC; Thu, 7 Apr 2022 13:26:08 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 324E6201CBCC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1649363168; bh=pBJjoP6HBM9BMHnhHHaiEURpWVqcRsKUpUuk8SoRtTQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=a6NRBZTILpb2I0lPawXCowwNYiD5FEHMK9pnYTYL3zeo3UtYuyZDY/GujINlEz7wv qFozDl4KHfDQ8vp5BnZmx++lCPlxttAFcDLT01GS2qmbo8vbFLwFaYM821aRGiAxuV JSfKebUPQR2MinG3+tfTzHuG2JDatLISytOmUJ74= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v1 9/9] dwarf: Enable livepatch for ARM64 Date: Thu, 7 Apr 2022 15:25:18 -0500 Message-Id: <20220407202518.19780-10-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220407202518.19780-1-madvenka@linux.microsoft.com> References: <95691cae4f4504f33d0fc9075541b1e7deefe96f> <20220407202518.19780-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_132609_206998_91D80349 X-CRM114-Status: UNSURE ( 7.08 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Enable livepatch in arch/arm64/Kconfig. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/Kconfig | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index c82a3a93297f..6cb00b3770cf 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -222,6 +222,9 @@ config ARM64 select THREAD_INFO_IN_TASK select HAVE_STACK_VALIDATION if DWARF_FP select STACK_VALIDATION if HAVE_STACK_VALIDATION + select HAVE_RELIABLE_STACKTRACE if STACK_VALIDATION + select HAVE_LIVEPATCH if HAVE_DYNAMIC_FTRACE_WITH_REGS && HAVE_RELIABLE_STACKTRACE + select HAVE_ARCH_USERFAULTFD_MINOR if USERFAULTFD select TRACE_IRQFLAGS_SUPPORT help @@ -2066,3 +2069,5 @@ source "arch/arm64/kvm/Kconfig" if CRYPTO source "arch/arm64/crypto/Kconfig" endif + +source "kernel/livepatch/Kconfig"