From patchwork Tue May 24 00:16:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12859589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5C97C433F5 for ; Tue, 24 May 2022 00:24:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7if9dM3+9/8LzpwL/YAil/E0UK3UzyzI2wvSO3bcozc=; b=VgvuLz+hjAfwF3 GSrt8rg/pQpwvP0kX66a3mTCQx1HxpdkOG1c4MofMMGCcVfsOahRA/MvEW79spAOk3eS31QtwMRjs ytdaza3tmV0IfyIVn5cYPnSqJLc+Z40eUEus8NkPva0CspLiai8ESYFPGvBWL6k1AlVYwyLNFR9uB Ag7liaR6BxpnYfrk7GrCQsniolKyIzCXZuks0fzflrgmxvWetZDiGrQddMSMGCkSADiUJU5g85fGq BJXLghfI5h+qzpP1NdvubcBez5ilaYTF76luoLbVvzUpxQI2uJ+DzmIqDzZyxHyuVbWvziNWNDdpg E0Uqu5vG1YzBtXeNk1aw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ntIKZ-006MPu-UX; Tue, 24 May 2022 00:23:24 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ntIET-006JQ2-Km for linux-arm-kernel@lists.infradead.org; Tue, 24 May 2022 00:17:12 +0000 Received: from x64host.home (unknown [47.189.24.195]) by linux.microsoft.com (Postfix) with ESMTPSA id 8509520B5B4E; Mon, 23 May 2022 17:17:04 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 8509520B5B4E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1653351425; bh=F+06QZixvxLg5mYB/7rdwOhFtinfWWMMUPEPRY4OtFI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=fNUshMRvBVgso8WBJO4GkUQQK0Q8bi4QV+zVHOgHgrHmj0j+0jo9zP6py6SVb8xip fjz8QwDwctqwlJYT+ENWJT3lZZrgUpArvBvHx2po1pDhGjUVvmrLR3bfnkF2xE097s cq2Ds76QQ4c4IGWT4Sqi2kkTj0xKg50unsT0I/0Q= From: madvenka@linux.microsoft.com To: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, mark.rutland@arm.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v2 18/20] arm64: unwinder: Add a reliability check in the unwinder based on ORC Date: Mon, 23 May 2022 19:16:35 -0500 Message-Id: <20220524001637.1707472-19-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220524001637.1707472-1-madvenka@linux.microsoft.com> References: <20220524001637.1707472-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220523_171705_771061_6418422A X-CRM114-Status: GOOD ( 30.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Introduce a reliability flag in struct stackframe. This will be set to false if the PC does not have a valid ORC or if the frame pointer computed from the ORC does not match the actual frame pointer. Now that the unwinder can validate the frame pointer, introduce arch_stack_walk_reliable(). Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 9 ++ arch/arm64/kernel/ftrace.c | 16 +++ arch/arm64/kernel/stacktrace.c | 153 ++++++++++++++++++++++++++++ include/linux/ftrace.h | 4 + 4 files changed, 182 insertions(+) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index e77cdef9ca29..f5d9bbed53e6 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -44,6 +44,7 @@ struct stack_info { * @prev_fp: The fp that pointed to this frame record, or a synthetic value * of 0. This is used to ensure that within a stack, each * subsequent frame record is at an increasing address. + * @prev_pc: The pc in the previous frame. * @prev_type: The type of stack this frame record was on, or a synthetic * value of STACK_TYPE_UNKNOWN. This is used to detect a * transition from one stack to another. @@ -51,16 +52,24 @@ struct stack_info { * @kr_cur: When KRETPROBES is selected, holds the kretprobe instance * associated with the most recently encountered replacement lr * value. + * + * @cfa: The sp value at the call site of the current function. + * @unwind_type The previous frame's unwind type. + * @reliable: Stack trace is reliable. */ struct stackframe { unsigned long fp; unsigned long pc; DECLARE_BITMAP(stacks_done, __NR_STACK_TYPES); unsigned long prev_fp; + unsigned long prev_pc; enum stack_type prev_type; #ifdef CONFIG_KRETPROBES struct llist_node *kr_cur; #endif + unsigned long cfa; + int unwind_type; + bool reliable; }; extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index 4506c4a90ac1..ec9a00d714e5 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -299,3 +299,19 @@ int ftrace_disable_ftrace_graph_caller(void) } #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + +bool is_ftrace_entry(unsigned long pc) +{ + if (pc == (unsigned long)&ftrace_call_entry) + return true; + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + if (pc == (unsigned long)&ftrace_graph_caller_entry) + return true; +#endif + return false; +} + +#endif diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index e4103e085681..31184d64edb6 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -5,6 +5,8 @@ * Copyright (C) 2012 ARM Ltd. */ #include +#include +#include #include #include #include @@ -18,6 +20,120 @@ #include #include +#ifdef CONFIG_FRAME_POINTER_VALIDATION + +static void unwind_check_frame(struct stackframe *frame) +{ + unsigned long pc, fp; + struct orc_entry *orc; + bool adjust_pc = false; + + /* + * If a previous frame was unreliable, the CFA cannot be reliably + * computed anymore. + */ + if (!frame->reliable) + return; + + pc = frame->pc; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + if (is_ftrace_entry(frame->prev_pc)) + pc = (unsigned long)&ftrace_callsite; +#endif + + /* Don't let modules unload while we're reading their ORC data. */ + preempt_disable(); + + orc = orc_find(pc); + if (!orc || (!orc->fp_offset && orc->type == UNWIND_HINT_TYPE_CALL)) { + /* + * If the final instruction in a function happens to be a call + * instruction, the return address would fall outside of the + * function. That could be the case here. This can happen, for + * instance, if the called function is a "noreturn" function. + * The compiler can optimize away the instructions after the + * call. So, adjust the PC so it falls inside the function and + * retry. + * + * We only do this if the current and the previous frames + * are call frames and not hint frames. + */ + if (frame->unwind_type == UNWIND_HINT_TYPE_CALL) { + pc -= 4; + adjust_pc = true; + orc = orc_find(pc); + } + } + if (!orc) { + frame->reliable = false; + goto out; + } + frame->unwind_type = orc->type; + + if (!frame->cfa) { + /* Set up the initial CFA and return. */ + frame->cfa = frame->fp - orc->fp_offset; + goto out; + } + + /* Compute the next CFA and FP. */ + switch (orc->type) { + case UNWIND_HINT_TYPE_CALL: + /* Normal call */ + frame->cfa += orc->sp_offset; + fp = frame->cfa + orc->fp_offset; + break; + + case UNWIND_HINT_TYPE_REGS: + /* + * pt_regs hint: The frame pointer points to either the + * synthetic frame within pt_regs or to the place where + * x29 and x30 are saved in the register save area in + * pt_regs. + */ + frame->cfa += orc->sp_offset; + fp = frame->cfa + offsetof(struct pt_regs, stackframe) - + sizeof(struct pt_regs); + if (frame->fp != fp) { + fp = frame->cfa + offsetof(struct pt_regs, regs[29]) - + sizeof(struct pt_regs); + } + break; + + case UNWIND_HINT_TYPE_FTRACE: + /* ftrace callsite hint */ + frame->cfa += orc->sp_offset; + fp = frame->cfa - orc->sp_offset; + break; + + case UNWIND_HINT_TYPE_IRQ_STACK: + /* Hint to unwind from the IRQ stack to the task stack. */ + frame->cfa = frame->fp + orc->sp_offset; + fp = frame->fp; + break; + + default: + fp = 0; + break; + } + + /* Validate the actual FP with the computed one. */ + if (frame->fp != fp) + frame->reliable = false; +out: + if (frame->reliable && adjust_pc) + frame->pc = pc; + preempt_enable(); +} + +#else /* !CONFIG_FRAME_POINTER_VALIDATION */ + +static void unwind_check_frame(struct stackframe *frame) +{ +} + +#endif /* CONFIG_FRAME_POINTER_VALIDATION */ + /* * AArch64 PCS assigns the frame pointer to x29. * @@ -53,7 +169,13 @@ static notrace void start_backtrace(struct stackframe *frame, unsigned long fp, */ bitmap_zero(frame->stacks_done, __NR_STACK_TYPES); frame->prev_fp = 0; + frame->prev_pc = 0; frame->prev_type = STACK_TYPE_UNKNOWN; + + frame->reliable = true; + frame->cfa = 0; + frame->unwind_type = UNWIND_HINT_TYPE_CALL; + unwind_check_frame(frame); } NOKPROBE_SYMBOL(start_backtrace); @@ -110,6 +232,7 @@ static int notrace unwind_frame(struct task_struct *tsk, * Record this frame record's values and location. The prev_fp and * prev_type are only meaningful to the next unwind_frame() invocation. */ + frame->prev_pc = frame->pc; frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp)); frame->pc = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8)); frame->prev_fp = fp; @@ -139,6 +262,10 @@ static int notrace unwind_frame(struct task_struct *tsk, frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur); #endif + /* If it is the final frame, no need to check reliability. */ + if (frame->fp != (unsigned long)task_pt_regs(tsk)->stackframe) + unwind_check_frame(frame); + return 0; } NOKPROBE_SYMBOL(unwind_frame); @@ -210,3 +337,29 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, walk_stackframe(task, &frame, consume_entry, cookie); } + +noinline int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry, + void *cookie, struct task_struct *task) +{ + struct stackframe frame; + int ret = 0; + + if (task == current) { + start_backtrace(&frame, + (unsigned long)__builtin_frame_address(1), + (unsigned long)__builtin_return_address(0)); + } else { + start_backtrace(&frame, thread_saved_fp(task), + thread_saved_pc(task)); + } + + while (!ret) { + if (!frame.reliable) + return -EINVAL; + if (!consume_entry(cookie, frame.pc)) + return -EINVAL; + ret = unwind_frame(task, &frame); + } + + return ret == -ENOENT ? 0 : -EINVAL; +} diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 4816b7e11047..2324f0b25674 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -616,6 +616,10 @@ extern int ftrace_update_ftrace_func(ftrace_func_t func); extern void ftrace_caller(void); extern void ftrace_regs_caller(void); extern void ftrace_call(void); +extern void ftrace_call_entry(void); +extern void ftrace_graph_caller_entry(void); +extern void ftrace_callsite(void); +extern bool is_ftrace_entry(unsigned long pc); extern void ftrace_regs_call(void); extern void mcount_call(void);