From patchwork Mon Oct 1 14:16:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Torsten Duwe X-Patchwork-Id: 10622247 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 810B11515 for ; Mon, 1 Oct 2018 14:33:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6302128DB1 for ; Mon, 1 Oct 2018 14:33:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 56A9928E23; Mon, 1 Oct 2018 14:33:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 014DF28DB1 for ; Mon, 1 Oct 2018 14:33:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:From:Date: Message-Id:References:In-Reply-To:Subject:To:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=HZvNJ8X+ED6int1ddcbsS0nyIzUeghcrnJ1SQfQJhDY=; b=XsNyqxZgvhT8xuRboN9dZNlNY2 71rYykKw/iWnuD8z5SeRM6ycD6Kx2qVO9sVurbfGQ6ItTqZgqfB8Dnz+UjiEk8pJ+H/nQWOiazAmZ szKTFQZA1nKd5PKLAhpqwPgOd9Qk4jtKTbGhELI+KwcIAVkKCUmTXEHgvoUg/b+gc72V+kwFio0Yw bbT3pFrziJiduqbvu4knlvzFI73so4OoGQF05+skj0s8i5AhQirhgKiWmLBXI/xMOENpmeR54oRXn gNGNv0x2RuSivGAzjKF3sT4cXY6AzjENG4ZSrjZeONOiNfTJsCRuRqk9wZ5Yu6cvBj5YblSAnD/+u 5APzcD/g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zGY-0007JZ-G6; Mon, 01 Oct 2018 14:33:42 +0000 Received: from casper.infradead.org ([85.118.1.10]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zEH-0005uL-Qw for linux-arm-kernel@bombadil.infradead.org; Mon, 01 Oct 2018 14:31:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=From:Date:Message-Id:References: In-Reply-To:Subject:Cc:To:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=FmMBnTb+JXJDbTA6rQ0ZWsJqHwmjUAelt4A2mqEM9xQ=; b=RxPvc3Ow9k16noQK9AsBMmzv2 cOqY6fnIpB+LZ1BzSDUO7XzmN96ahiGY2KCX/ZYSLNsjNgYTIeHPyGVZeun18GdOKHj5EkN5darqE r/O8DG7cdAbfcZCdxB0uSD5BWKBNRWXAycC9XNMkUkC++gnLqVq8kj6tMrTPG90lUJKHz4/W746Jd 0949EXbDislAkNhzMgNHta+46W8r/T3+br4ZCqHlXWv/qsCVQiwZV5rK2vGmG/JvLoufUxayIRa68 WpYeoZopAAcS3XU3UluYKMiPknr3ZYTepdqL0sPa9wGXNsc0bGEj+I+J4as4WzpoAb587DabLwONZ beQTTaREw==; Received: from verein.lst.de ([213.95.11.211] helo=newverein.lst.de) by casper.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6z0J-0004fn-7m for linux-arm-kernel@lists.infradead.org; Mon, 01 Oct 2018 14:16:56 +0000 Received: by newverein.lst.de (Postfix, from userid 2005) id 331EE68BC7; Mon, 1 Oct 2018 16:16:43 +0200 (CEST) To: Masami Hiramatsu , Will Deacon , Catalin Marinas , Julien Thierry , Steven Rostedt , Josh Poimboeuf , Ingo Molnar , Ard Biesheuvel , Arnd Bergmann , AKASHI Takahiro Subject: [PATCH v3 1/4] DYNAMIC_FTRACE configurable with and without REGS In-Reply-To: <20181001140910.086E768BC7@newverein.lst.de> References: <20181001140910.086E768BC7@newverein.lst.de> Message-Id: <20181001141643.331EE68BC7@newverein.lst.de> Date: Mon, 1 Oct 2018 16:16:43 +0200 (CEST) From: duwe@lst.de (Torsten Duwe) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181001_151655_344751_AFBB55FF X-CRM114-Status: GOOD ( 11.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP In commit 06aeaaeabf69da4, many ftrace-related config options are consolidated. By accident, I guess, the choice about DYNAMIC_FTRACE and DYNAMIC_FTRACE_WITH_REGS is no longer available explicitly but determined by the sole availability on the architecture. This makes it hard to introduce DYNAMIC_FTRACE_WITH_REGS if it depends on new compiler features or other new properties of the toolchain without breaking existing configurations. This patch turns the def_bool into an actual choice. Should the toolchain not meet the requirements for _WITH_REGS it can be turned off. Signed-off-by: Torsten Duwe --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -508,9 +508,15 @@ config DYNAMIC_FTRACE otherwise has native performance as long as no tracing is active. config DYNAMIC_FTRACE_WITH_REGS - def_bool y + bool "Include register content tracking in dynamic ftrace facility" + default y depends on DYNAMIC_FTRACE depends on HAVE_DYNAMIC_FTRACE_WITH_REGS + help + This architecture supports the inspection of register contents, + as passed between functions, at the dynamic ftrace points. + This is also a prerequisite for Kernel Live Patching (KLP). + When in doubt, say Y. config FUNCTION_PROFILER bool "Kernel function profiler" From patchwork Mon Oct 1 14:16:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Torsten Duwe X-Patchwork-Id: 10622243 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88A9C1515 for ; Mon, 1 Oct 2018 14:33:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65E22286F7 for ; Mon, 1 Oct 2018 14:33:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5992028709; Mon, 1 Oct 2018 14:33:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7110C286F7 for ; Mon, 1 Oct 2018 14:33:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:From:Date: Message-Id:References:In-Reply-To:Subject:To:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2RoXnZ+udIoVO+ATUz6mDCiPJ5PIXRdkuiyl4GzH9iI=; b=B9bxsTaB/kSS1qsUF4AeD9IQeB WPhFAMn1cssHpyKS/vAyDIlG4OXodBxDZnK5j2vjYNPQ25zw1inmOhJj0fC4WVjOHnlak7fGgU8eU XjBRK4F7mIQzxKwPUmtanPgc3ipiyHzZxmlxXaqxrtkW5I2xkAKLOVy+A38z9z0oPfGKxcOA/r31R 4hpDxfpVrlS1WGJV2ugbSk/UG96yyNrJCjCP26YRj9MmpiOHwXhQejqKHAFnvqh/OoF62cKWVRPKu tzOpXLjHCyyNbtFnNzmBmm58CVQ+aQQjcR2xN79DTWp/TkX+KtGmUJTb3abx627DXhvVeKbETAVN+ A+R1u9CA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zFt-000703-JX; Mon, 01 Oct 2018 14:33:01 +0000 Received: from casper.infradead.org ([85.118.1.10]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zEG-0005x0-Qs for linux-arm-kernel@bombadil.infradead.org; Mon, 01 Oct 2018 14:31:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=From:Date:Message-Id:References: In-Reply-To:Subject:Cc:To:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=wYRvcQZRDfcWWh/j6pYvSx4qUXhdBfncS/COCsxORYc=; b=wXVRoP3WKA2p1XpOoqT2T37Yn ieevBb5VP04qVEYovKWWhbNIG1grKl0YhBo/WvVBsuwcBiW+yDnwseczb82twOLtMieDP2MlFpW6C dX3XfnxjFdTf9yx7fugKycJIJ3FhWU6DhQgJdDURR3Nhk2LBVjVHlfqN1x0OUPcic0kxvPuQtVFfR ujK7hy4qvubsyWW9r6x6bxsa1aNBQwU4I2w2NZA5hazlZ2VEqhD6vHy8kRFNBMICQJ654gh7BYKMf hhHynfCqxx9X+wavggvgaIRQlx/aIKkGbyEoSvzHWCPw4QJJU/I6EXH2q0bdAlc69DPSTseU10Sm4 rsge1RWmw==; Received: from verein.lst.de ([213.95.11.211] helo=newverein.lst.de) by casper.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6z0J-0004g4-7p for linux-arm-kernel@lists.infradead.org; Mon, 01 Oct 2018 14:16:59 +0000 Received: by newverein.lst.de (Postfix, from userid 2005) id 1DBED68BDF; Mon, 1 Oct 2018 16:16:48 +0200 (CEST) To: Will Deacon , Catalin Marinas , Julien Thierry , Steven Rostedt , Josh Poimboeuf , Ingo Molnar , Ard Biesheuvel , Arnd Bergmann , AKASHI Takahiro Subject: [PATCH v3 2/4] arm64: implement ftrace with regs In-Reply-To: <20181001140910.086E768BC7@newverein.lst.de> References: <20181001140910.086E768BC7@newverein.lst.de> Message-Id: <20181001141648.1DBED68BDF@newverein.lst.de> Date: Mon, 1 Oct 2018 16:16:48 +0200 (CEST) From: duwe@lst.de (Torsten Duwe) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181001_151655_352222_5C46D7E7 X-CRM114-Status: GOOD ( 32.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Check for compiler support of -fpatchable-function-entry and use it to intercept functions immediately on entry, saving the LR in x9. patchable-function-entry in GCC disables IPA-RA, which means ABI register calling conventions are obeyed *and* scratch registers are available. Disable ftracing in efi/libstub, because this triggers cross-section linker errors now (-pg is disabled already for those files). Add an ftrace_caller which can handle LR in x9, as well as an ftrace_regs_caller that additionally writes out a set of pt_regs for inspection. Introduce and handle an ftrace_regs_trampoline for module PLTs. Signed-off-by: Torsten Duwe --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -110,6 +110,7 @@ config ARM64 select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE + select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_FTRACE_MCOUNT_RECORD select HAVE_FUNCTION_TRACER --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -78,6 +78,15 @@ ifeq ($(CONFIG_ARM64_MODULE_PLTS),y) KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/arm64/kernel/module.lds endif +ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_REGS),y) + CC_FLAGS_FTRACE := -fpatchable-function-entry=2 + KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY + ifeq ($(call cc-option,-fpatchable-function-entry=2),) + $(error Cannot use CONFIG_DYNAMIC_FTRACE_WITH_REGS: \ + -fpatchable-function-entry not supported by compiler) + endif +endif + # Default value head-y := arch/arm64/kernel/head.o --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -16,6 +16,17 @@ #define MCOUNT_ADDR ((unsigned long)_mcount) #define MCOUNT_INSN_SIZE AARCH64_INSN_SIZE +/* DYNAMIC_FTRACE_WITH_REGS is implemented by adding 2 NOPs at the beginning + of each function, with the second NOP actually calling ftrace. In contrary + to a classic _mcount call, the call instruction to be modified is thus + the second one, and not the only one. */ +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +#define ARCH_SUPPORTS_FTRACE_OPS 1 +#define REC_IP_BRANCH_OFFSET AARCH64_INSN_SIZE +#else +#define REC_IP_BRANCH_OFFSET 0 +#endif + #ifndef __ASSEMBLY__ #include --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -7,9 +7,9 @@ CPPFLAGS_vmlinux.lds := -DTEXT_OFFSET=$( AFLAGS_head.o := -DTEXT_OFFSET=$(TEXT_OFFSET) CFLAGS_armv8_deprecated.o := -I$(src) -CFLAGS_REMOVE_ftrace.o = -pg -CFLAGS_REMOVE_insn.o = -pg -CFLAGS_REMOVE_return_address.o = -pg +CFLAGS_REMOVE_ftrace.o = -pg $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_insn.o = -pg $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_return_address.o = -pg $(CC_FLAGS_FTRACE) # Object file lists. arm64-obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -11,7 +11,8 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__K -fPIC -fno-strict-aliasing -mno-red-zone \ -mno-mmx -mno-sse -fshort-wchar -cflags-$(CONFIG_ARM64) := $(subst -pg,,$(KBUILD_CFLAGS)) -fpie +cflags-$(CONFIG_ARM64) := $(filter-out -pg $(CC_FLAGS_FTRACE)\ + ,$(KBUILD_CFLAGS)) -fpie cflags-$(CONFIG_ARM) := $(subst -pg,,$(KBUILD_CFLAGS)) \ -fno-builtin -fpic -mno-single-pic-base --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -13,6 +13,8 @@ #include #include #include +#include +#include /* * Gcc with -pg will put the following code in the beginning of each function: @@ -123,6 +125,7 @@ skip_ftrace_call: // } ENDPROC(_mcount) #else /* CONFIG_DYNAMIC_FTRACE */ +#ifndef CC_USING_PATCHABLE_FUNCTION_ENTRY /* * _mcount() is used to build the kernel with -pg option, but all the branch * instructions to _mcount() are replaced to NOP initially at kernel start up, @@ -162,6 +165,92 @@ ftrace_graph_call: // ftrace_graph_cal mcount_exit ENDPROC(ftrace_caller) +#else /* CC_USING_PATCHABLE_FUNCTION_ENTRY */ + +/* Since no -pg or similar compiler flag is used, there should really be + no reference to _mcount; so do not define one. Only a function address + is needed in order to refer to it. */ +ENTRY(_mcount) + ret /* just in case, prevent any fall through. */ +ENDPROC(_mcount) + +ENTRY(ftrace_regs_caller) + sub sp, sp, #S_FRAME_SIZE + stp x29, x9, [sp, #-16] /* FP/LR link */ + + stp x10, x11, [sp, #S_X10] + stp x12, x13, [sp, #S_X12] + stp x14, x15, [sp, #112] + stp x16, x17, [sp, #128] + stp x18, x19, [sp, #144] + stp x20, x21, [sp, #160] + stp x22, x23, [sp, #176] + stp x24, x25, [sp, #192] + stp x26, x27, [sp, #208] + + b ftrace_common +ENDPROC(ftrace_regs_caller) + +ENTRY(ftrace_caller) + sub sp, sp, #S_FRAME_SIZE + stp x29, x9, [sp, #-16] /* FP/LR link */ + +ftrace_common: + stp x28, x29, [sp, #224] /* FP in pt_regs + "our" x28 */ + + /* save function arguments */ + stp x0, x1, [sp] + stp x2, x3, [sp, #16] + stp x4, x5, [sp, #32] + stp x6, x7, [sp, #48] + stp x8, x9, [sp, #64] + + /* The link Register at callee entry */ + str x9, [sp, #S_LR] /* to pt_regs.r[30] */ + /* The program counter just after the ftrace call site */ + str lr, [sp, #S_PC] + /* The stack pointer as it was on ftrace_caller entry... */ + add x29, sp, #S_FRAME_SIZE + str x29, [sp, #S_SP] + + ldr_l x2, function_trace_op, x0 + mov x1, x9 /* saved LR == parent IP */ + sub x0, lr, #8 /* function entry == IP */ + mov x3, sp /* pt_regs are @sp */ + sub sp, sp, #16 /* skip over FP/LR link */ + + .global ftrace_call +ftrace_call: + + bl ftrace_stub + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + .global ftrace_graph_call +ftrace_graph_call: // ftrace_graph_caller(); + nop // If enabled, this will be replaced + // "b ftrace_graph_caller" +#endif + +ftrace_common_return: + add sp, sp, #16 /* advance to pt_regs for restore */ + + ldp x0, x1, [sp] + ldp x2, x3, [sp, #16] + ldp x4, x5, [sp, #32] + ldp x6, x7, [sp, #48] + ldp x8, x9, [sp, #64] + + ldp x28, x29, [sp, #224] + + ldr x9, [sp, #S_PC] + ldr lr, [sp, #S_LR] + add sp, sp, #S_FRAME_SIZE + + ret x9 + +ENDPROC(ftrace_caller) + +#endif /* CC_USING_PATCHABLE_FUNCTION_ENTRY */ #endif /* CONFIG_DYNAMIC_FTRACE */ ENTRY(ftrace_stub) @@ -197,12 +286,20 @@ ENDPROC(ftrace_stub) * and run return_to_handler() later on its exit. */ ENTRY(ftrace_graph_caller) +#ifndef CC_USING_PATCHABLE_FUNCTION_ENTRY mcount_get_lr_addr x0 // pointer to function's saved lr mcount_get_pc x1 // function's pc mcount_get_parent_fp x2 // parent's fp bl prepare_ftrace_return // prepare_ftrace_return(&lr, pc, fp) mcount_exit +#else + add x0, sp, #S_LR+16 /* address of (LR pointing into caller) */ + ldr x1, [sp, #S_PC+16] + ldr x2, [sp, #S_X28 + 8+16] /* caller's frame pointer */ + bl prepare_ftrace_return + b ftrace_common_return +#endif ENDPROC(ftrace_graph_caller) /* --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -65,18 +65,66 @@ int ftrace_update_ftrace_func(ftrace_fun return ftrace_modify_code(pc, 0, new, false); } +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +/* Have the assembler generate a known "mov x9,x30" at compile time. */ +static void notrace noinline __attribute__((used)) mov_x9_x30(void) +{ + asm(" .global insn_mov_x9_x30\n" + "insn_mov_x9_x30: mov x9,x30\n" : : : "x9"); +} +#endif + +#ifdef CONFIG_ARM64_MODULE_PLTS +int use_ftrace_trampoline(struct module *mod, unsigned long *addr) +{ + struct plt_entry trampoline; + trampoline = get_plt_entry(*addr); + if (*addr == FTRACE_ADDR) { + if (!plt_entries_equal(mod->arch.ftrace_trampoline, + &trampoline)) { + + /* point the trampoline to our ftrace entry point */ + module_disable_ro(mod); + *mod->arch.ftrace_trampoline = trampoline; + module_enable_ro(mod, true); + + /* update trampoline before patching in the branch */ + smp_wmb(); + } + *addr = (unsigned long)(void *)mod->arch.ftrace_trampoline; + } + else if (*addr == FTRACE_REGS_ADDR) { + if (!plt_entries_equal(mod->arch.ftrace_regs_trampoline, + &trampoline)) { + + /* point the trampoline to our ftrace entry point */ + module_disable_ro(mod); + *mod->arch.ftrace_regs_trampoline = trampoline; + module_enable_ro(mod, true); + + /* update trampoline before patching in the branch */ + smp_wmb(); + } + *addr = (unsigned long)(void *)mod->arch.ftrace_regs_trampoline; + } + else + return -EINVAL; + return 0; +} +#endif + /* * Turn on the call to ftrace_caller() in instrumented function */ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) { - unsigned long pc = rec->ip; + unsigned long pc = rec->ip + REC_IP_BRANCH_OFFSET; + int ret; u32 old, new; long offset = (long)pc - (long)addr; if (offset < -SZ_128M || offset >= SZ_128M) { #ifdef CONFIG_ARM64_MODULE_PLTS - struct plt_entry trampoline; struct module *mod; /* @@ -96,54 +144,67 @@ int ftrace_make_call(struct dyn_ftrace * if (WARN_ON(!mod)) return -EINVAL; - /* - * There is only one ftrace trampoline per module. For now, - * this is not a problem since on arm64, all dynamic ftrace - * invocations are routed via ftrace_caller(). This will need - * to be revisited if support for multiple ftrace entry points - * is added in the future, but for now, the pr_err() below - * deals with a theoretical issue only. - */ - trampoline = get_plt_entry(addr); - if (!plt_entries_equal(mod->arch.ftrace_trampoline, - &trampoline)) { - if (!plt_entries_equal(mod->arch.ftrace_trampoline, - &(struct plt_entry){})) { - pr_err("ftrace: far branches to multiple entry points unsupported inside a single module\n"); - return -EINVAL; - } - - /* point the trampoline to our ftrace entry point */ - module_disable_ro(mod); - *mod->arch.ftrace_trampoline = trampoline; - module_enable_ro(mod, true); - - /* update trampoline before patching in the branch */ - smp_wmb(); + /* Check against our well-known list of ftrace entry points */ + if (addr == FTRACE_ADDR || addr == FTRACE_REGS_ADDR) { + ret = use_ftrace_trampoline(mod, &addr); + if (ret < 0) + return ret; } - addr = (unsigned long)(void *)mod->arch.ftrace_trampoline; + else + return -EINVAL; + #else /* CONFIG_ARM64_MODULE_PLTS */ return -EINVAL; #endif /* CONFIG_ARM64_MODULE_PLTS */ } old = aarch64_insn_gen_nop(); + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS)) { + new = *(u32*)&mov_x9_x30; + ret = ftrace_modify_code(pc - REC_IP_BRANCH_OFFSET, + old, new, true); + if (ret) + return ret; + smp_wmb(); /* ensure LR saver is in place before ftrace call */ + } new = aarch64_insn_gen_branch_imm(pc, addr, AARCH64_INSN_BRANCH_LINK); return ftrace_modify_code(pc, old, new, true); } +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, + unsigned long addr) +{ + unsigned long pc = rec->ip + REC_IP_BRANCH_OFFSET; + u32 old, new; + + old = aarch64_insn_gen_branch_imm(pc, old_addr, true); + new = aarch64_insn_gen_branch_imm(pc, addr, true); + + return ftrace_modify_code(pc, old, new, true); +} + + /* * Turn off the call to ftrace_caller() in instrumented function */ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr) { - unsigned long pc = rec->ip; + unsigned long pc = rec->ip + REC_IP_BRANCH_OFFSET; bool validate = true; + int ret; u32 old = 0, new; long offset = (long)pc - (long)addr; +#ifdef CC_USING_PATCHABLE_FUNCTION_ENTRY + /* -fpatchable-function-entry= does not generate a profiling call + * initially; the NOPs are already there. + */ + if (addr == MCOUNT_ADDR) + return 0; +#endif + if (offset < -SZ_128M || offset >= SZ_128M) { #ifdef CONFIG_ARM64_MODULE_PLTS u32 replaced; @@ -188,7 +249,16 @@ int ftrace_make_nop(struct module *mod, new = aarch64_insn_gen_nop(); - return ftrace_modify_code(pc, old, new, validate); + ret = ftrace_modify_code(pc, old, new, validate); + if (ret) + return ret; + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS)) { + smp_wmb(); /* ftrace call must not remain without LR saver. */ + old = *(u32*)&mov_x9_x30; + ret = ftrace_modify_code(pc - REC_IP_BRANCH_OFFSET, + old, new, true); + } + return ret; } void arch_ftrace_update_code(int command) --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -115,6 +115,7 @@ #define MCOUNT_REC() . = ALIGN(8); \ __start_mcount_loc = .; \ KEEP(*(__mcount_loc)) \ + KEEP(*(__patchable_function_entries)) \ __stop_mcount_loc = .; #else #define MCOUNT_REC() --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -61,8 +61,12 @@ extern void __chk_io_ptr(const volatile #if defined(CC_USING_HOTPATCH) && !defined(__CHECKER__) #define notrace __attribute__((hotpatch(0,0))) #else +#ifdef CC_USING_PATCHABLE_FUNCTION_ENTRY +#define notrace __attribute__((patchable_function_entry(0))) +#else #define notrace __attribute__((no_instrument_function)) #endif +#endif /* Intel compiler defines __GNUC__. So we will overwrite implementations * coming from above header files here --- a/arch/arm64/include/asm/module.h +++ b/arch/arm64/include/asm/module.h @@ -33,6 +33,7 @@ struct mod_arch_specific { /* for CONFIG_DYNAMIC_FTRACE */ struct plt_entry *ftrace_trampoline; + struct plt_entry *ftrace_regs_trampoline; }; #endif --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -454,6 +454,10 @@ int module_finalize(const Elf_Ehdr *hdr, if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE) && !strcmp(".text.ftrace_trampoline", secstrs + s->sh_name)) me->arch.ftrace_trampoline = (void *)s->sh_addr; + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && + !strcmp(".text.ftrace_regs_trampoline", + secstrs + s->sh_name)) + me->arch.ftrace_regs_trampoline = (void *)s->sh_addr; #endif } From patchwork Mon Oct 1 14:16:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Torsten Duwe X-Patchwork-Id: 10622251 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD26D15A7 for ; Mon, 1 Oct 2018 14:38:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F67029048 for ; Mon, 1 Oct 2018 14:38:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8146D29070; Mon, 1 Oct 2018 14:38:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 09FBB29048 for ; Mon, 1 Oct 2018 14:38:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:From:Date: Message-Id:References:In-Reply-To:Subject:To:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=SvUabHn4ScSbf53rFxpp9WFVHvAoUFvosxnFQqU0Llc=; b=FTvkEDc8wPE2Zv6NM3KnYV7zkL YpwU746IkeSbH4AlhyLS96wm3sAVgzARFUfHHKOZPqkpQdRbmtQNc+usIvpmrEg6rnme0N6kbN+KC xYUottCEks8yaz1G7WuDCsa9LZqtEvMrZipuA8+8T7rsyaz/vyHEAYzd+xLscKDrNg+8ZYHD3wa1N xXq75dinfuozXhulqy7KEOxSEJuAfhoybAuxd+kp5z5H91fQlfR4/eseZX6BzBwrXgUjWK7CAge5f md1KAyUW3inh5Oou9rfpCIDJBvTxEFJjNoACzz+5jq+K6TJs35rSAqSUtxUiqtCfMPBCqEtRN8kH5 hf1Pe7tQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zKt-0001At-Jp; Mon, 01 Oct 2018 14:38:11 +0000 Received: from casper.infradead.org ([85.118.1.10]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zEH-0005to-MY for linux-arm-kernel@bombadil.infradead.org; Mon, 01 Oct 2018 14:31:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=From:Date:Message-Id:References: In-Reply-To:Subject:Cc:To:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=s9quUeT6CrLx4smqPpSSOuXkfl7cXXqpJCB65Los0i8=; b=Nj2+bR8UB6gMKq3h4t6erCQxZ nAXKr/QuKB6cv1FS0He+HueYwgrObuSAFoldroGVwXUUjVM3JYuCMIs4uQar0GOkrI+PnGTO+SLQi YIE00sVrKCuVqsTsyPTkZ8/fQ4jLzEBfMVv99Kc7CzA02ypFg8H62jTfensOpHjrQcBq+8hhVjOQO NP0zGJEffhvJbRx5CZE+a9yvExTyTkILOWoBg1/rgs7hdM4pR0UEH5zKgWTAM0zwnkzUxNv2+NyVs 4q2S+fVjazg5F5NRSZZc4djwx+AZhNoDWOmWm3/YAHu8wt+I5jMLx/fe29jd6MPFCCFGEUQkO5g3F ojIYKG6jQ==; Received: from verein.lst.de ([213.95.11.211] helo=newverein.lst.de) by casper.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6z0I-0004gZ-Ux for linux-arm-kernel@lists.infradead.org; Mon, 01 Oct 2018 14:16:57 +0000 Received: by newverein.lst.de (Postfix, from userid 2005) id 5478C68BE1; Mon, 1 Oct 2018 16:16:52 +0200 (CEST) To: Will Deacon , Catalin Marinas , Julien Thierry , Steven Rostedt , Josh Poimboeuf , Ingo Molnar , Ard Biesheuvel , Arnd Bergmann , AKASHI Takahiro Subject: [PATCH v3 3/4] arm64: implement live patching In-Reply-To: <20181001140910.086E768BC7@newverein.lst.de> References: <20181001140910.086E768BC7@newverein.lst.de> Message-Id: <20181001141652.5478C68BE1@newverein.lst.de> Date: Mon, 1 Oct 2018 16:16:52 +0200 (CEST) From: duwe@lst.de (Torsten Duwe) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181001_151655_150942_2B14BC70 X-CRM114-Status: GOOD ( 17.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Based on ftrace with regs, do the usual thing. Also allocate a task flag for whatever consistency handling will be used. Watch out for interactions with the graph tracer. Signed-off-by: Torsten Duwe --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -119,6 +119,7 @@ config ARM64 select HAVE_GENERIC_DMA_COHERENT select HAVE_HW_BREAKPOINT if PERF_EVENTS select HAVE_IRQ_TIME_ACCOUNTING + select HAVE_LIVEPATCH select HAVE_MEMBLOCK select HAVE_MEMBLOCK_NODE_MAP if NUMA select HAVE_NMI @@ -1349,4 +1350,6 @@ if CRYPTO source "arch/arm64/crypto/Kconfig" endif +source "kernel/livepatch/Kconfig" + source "lib/Kconfig" --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -76,6 +76,7 @@ void arch_release_task_struct(struct tas #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ +#define TIF_PATCH_PENDING 6 #define TIF_NOHZ 7 #define TIF_SYSCALL_TRACE 8 #define TIF_SYSCALL_AUDIT 9 @@ -94,6 +95,7 @@ void arch_release_task_struct(struct tas #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_FOREIGN_FPSTATE (1 << TIF_FOREIGN_FPSTATE) +#define _TIF_PATCH_PENDING (1 << TIF_PATCH_PENDING) #define _TIF_NOHZ (1 << TIF_NOHZ) #define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) #define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) @@ -106,7 +108,8 @@ void arch_release_task_struct(struct tas #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK) + _TIF_UPROBE | _TIF_FSCHECK | \ + _TIF_PATCH_PENDING) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ --- /dev/null +++ b/arch/arm64/include/asm/livepatch.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * livepatch.h - arm64-specific Kernel Live Patching Core + * + * Copyright (C) 2016,2018 SUSE + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version 2 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see . + */ +#ifndef _ASM_ARM64_LIVEPATCH_H +#define _ASM_ARM64_LIVEPATCH_H + +#include +#include + +static inline int klp_check_compiler_support(void) +{ + return 0; +} + +static inline void klp_arch_set_pc(struct pt_regs *regs, unsigned long ip) +{ + regs->pc = ip; +} + +#endif /* _ASM_ARM64_LIVEPATCH_H */ --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -209,6 +209,9 @@ ftrace_common: str x9, [sp, #S_LR] /* to pt_regs.r[30] */ /* The program counter just after the ftrace call site */ str lr, [sp, #S_PC] +#if defined(CONFIG_LIVEPATCH) && defined(CONFIG_FUNCTION_GRAPH_TRACER) + mov x28,lr /* remember old return address */ +#endif /* The stack pointer as it was on ftrace_caller entry... */ add x29, sp, #S_FRAME_SIZE str x29, [sp, #S_SP] @@ -224,6 +227,16 @@ ftrace_call: bl ftrace_stub +#if defined(CONFIG_LIVEPATCH) && defined(CONFIG_FUNCTION_GRAPH_TRACER) + /* Is the trace function a live patcher an has messed with + * the return address? + */ + ldr x9, [sp, #S_PC+16] + cmp x9, x28 /* compare with the value we remembered */ + /* to not call graph tracer's "call" mechanism twice! */ + b.ne ftrace_common_return +#endif + #ifdef CONFIG_FUNCTION_GRAPH_TRACER .global ftrace_graph_call ftrace_graph_call: // ftrace_graph_caller(); --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -934,6 +935,9 @@ asmlinkage void do_notify_resume(struct if (thread_flags & _TIF_UPROBE) uprobe_notify_resume(regs); + if (thread_flags & _TIF_PATCH_PENDING) + klp_update_patch_state(current); + if (thread_flags & _TIF_SIGPENDING) do_signal(regs); From patchwork Mon Oct 1 14:16:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Torsten Duwe X-Patchwork-Id: 10622249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A2B61515 for ; Mon, 1 Oct 2018 14:38:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B04AD28F5D for ; Mon, 1 Oct 2018 14:38:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A43B028F8C; Mon, 1 Oct 2018 14:38:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3AC7028F5D for ; Mon, 1 Oct 2018 14:38:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:From:Date: Message-Id:References:In-Reply-To:Subject:To:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=uweiZeJD7WlldpQnFCS8HPys2lIolADgMg5p/61ftyo=; b=FuzOM5KaM68u+lYYlu699wEap6 A2+7nYeTSAZRDWXOrBj/lV8uZN0vhAD27pSlfloInQTNvdvOrR+lexIIl4fWobz7pIjpAw16v3sS1 aIG07BvAvtY89o7EwHZ5PQHDKStyNc5hoSTmXEfZ3Kf3qSltzNhMxQRtAAqbOSsJyhtg3k6H+zEW3 2gSiFxiPDpngiLSE6VAbVSck0mJ+n3jlehfRsfVr0faltbEiDQLlu9O4M80fokIZ6OSI717wCdYNF HbnzT1I4zQEbezaSbEE1AhsP1k6aDbpp911oAhbH52cBp57VNDhfxFPCYSs8qvTKk861QB+BRgzJ9 CQUCc8Qg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zKm-00019W-Rr; Mon, 01 Oct 2018 14:38:04 +0000 Received: from casper.infradead.org ([85.118.1.10]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6zEF-0005to-FN for linux-arm-kernel@bombadil.infradead.org; Mon, 01 Oct 2018 14:31:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=From:Date:Message-Id:References: In-Reply-To:Subject:Cc:To:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=FbXGPB7jFkMLuQ2gifkWmni08AdrCWwRJG2BX8Hm+xI=; b=gxGWH97QwV0YTJwUN0jvY4xXi HDYIGaa/73dmsNSuzGnUL857cMosAk5IUefsM6XQ7yuG9G8kF+J4BrjowmY9vFVsWQGP7LKVdp/me b9UFDSxUpzZk8gUCZp+PJ2N+wc14PQi4j9hIgDHrsotExoG2xgZkt8geW+0mHTcF7dN1zcrRJizSG 5zWHQNoePnvo34dTssQjMYA5XRFQyZgxZgMO6g02utHyG5JhQWpsDGZCbc+nsCCCeHK8JTM59M1Ar FIJYYCOz1nyUYaDzbegqLyrqJugMKH/ihYprsOSg96+r7q2nZZXHtmQ5DEqS4cl1bWZ66b4An597r hofZk+YpA==; Received: from verein.lst.de ([213.95.11.211] helo=newverein.lst.de) by casper.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g6z0X-0004i8-H4 for linux-arm-kernel@lists.infradead.org; Mon, 01 Oct 2018 14:17:12 +0000 Received: by newverein.lst.de (Postfix, from userid 2005) id A856D68BC7; Mon, 1 Oct 2018 16:16:58 +0200 (CEST) To: Will Deacon , Catalin Marinas , Julien Thierry , Steven Rostedt , Josh Poimboeuf , Ingo Molnar , Ard Biesheuvel , Arnd Bergmann , AKASHI Takahiro Subject: [PATCH v3 4/4] arm64: reliable stacktraces In-Reply-To: <20181001140910.086E768BC7@newverein.lst.de> References: <20181001140910.086E768BC7@newverein.lst.de> Message-Id: <20181001141658.A856D68BC7@newverein.lst.de> Date: Mon, 1 Oct 2018 16:16:58 +0200 (CEST) From: duwe@lst.de (Torsten Duwe) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181001_151709_653972_247A50E7 X-CRM114-Status: GOOD ( 15.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Make unwind_frame() report whether it had to stop normally or due to an error condition; walk_stackframe() will pass that info. __save_stack_trace() is used to check the validity of a frame; save_stack_trace_tsk_reliable() can now trivially be implemented. Modify arch/arm64/kernel/time.c for the new semantics. Signed-off-by: Torsten Duwe --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -127,8 +127,9 @@ config ARM64 select HAVE_PERF_EVENTS select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP - select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RCU_TABLE_FREE + select HAVE_REGS_AND_STACK_ACCESS_API + select HAVE_RELIABLE_STACKTRACE select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -33,7 +33,7 @@ struct stackframe { }; extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame); -extern void walk_stackframe(struct task_struct *tsk, struct stackframe *frame, +extern int walk_stackframe(struct task_struct *tsk, struct stackframe *frame, int (*fn)(struct stackframe *, void *), void *data); extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk); --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -40,6 +40,13 @@ * ldp x29, x30, [sp] * add sp, sp, #0x10 */ + +/* + * unwind_frame -- unwind a single stack frame. + * Returns 0 when there are more frames to go. + * 1 means reached end of stack; negative (error) + * means stopped because information is not reliable. + */ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) { unsigned long fp = frame->fp; @@ -81,23 +88,27 @@ int notrace unwind_frame(struct task_str * both are NULL. */ if (!frame->fp && !frame->pc) - return -EINVAL; + return 1; return 0; } -void notrace walk_stackframe(struct task_struct *tsk, struct stackframe *frame, +int notrace walk_stackframe(struct task_struct *tsk, struct stackframe *frame, int (*fn)(struct stackframe *, void *), void *data) { while (1) { int ret; - if (fn(frame, data)) - break; + ret = fn(frame, data); + if (ret) + return ret; ret = unwind_frame(tsk, frame); if (ret < 0) + return ret; + if (ret > 0) break; } + return 0; } #ifdef CONFIG_STACKTRACE @@ -145,14 +156,15 @@ void save_stack_trace_regs(struct pt_reg trace->entries[trace->nr_entries++] = ULONG_MAX; } -static noinline void __save_stack_trace(struct task_struct *tsk, +static noinline int __save_stack_trace(struct task_struct *tsk, struct stack_trace *trace, unsigned int nosched) { struct stack_trace_data data; struct stackframe frame; + int ret; if (!try_get_task_stack(tsk)) - return; + return -EBUSY; data.trace = trace; data.skip = trace->skip; @@ -171,11 +183,12 @@ static noinline void __save_stack_trace( frame.graph = tsk->curr_ret_stack; #endif - walk_stackframe(tsk, &frame, save_trace, &data); + ret = walk_stackframe(tsk, &frame, save_trace, &data); if (trace->nr_entries < trace->max_entries) trace->entries[trace->nr_entries++] = ULONG_MAX; put_task_stack(tsk); + return ret; } EXPORT_SYMBOL_GPL(save_stack_trace_tsk); @@ -190,4 +203,12 @@ void save_stack_trace(struct stack_trace } EXPORT_SYMBOL_GPL(save_stack_trace); + +int save_stack_trace_tsk_reliable(struct task_struct *tsk, + struct stack_trace *trace) +{ + return __save_stack_trace(tsk, trace, 1); +} +EXPORT_SYMBOL_GPL(save_stack_trace_tsk_reliable); + #endif --- a/arch/arm64/kernel/time.c +++ b/arch/arm64/kernel/time.c @@ -56,7 +56,7 @@ unsigned long profile_pc(struct pt_regs #endif do { int ret = unwind_frame(NULL, &frame); - if (ret < 0) + if (ret) return 0; } while (in_lock_functions(frame.pc));