From patchwork Mon May 3 17:36:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12236625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C7CEC433ED for ; Mon, 3 May 2021 17:38:25 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC5E8610E9 for ; Mon, 3 May 2021 17:38:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC5E8610E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=WPjpVc6UzCjrI5qtPm7GUhYlWB4SJLyQpS2Jmv4d8QY=; b=VLAoSDOkVqk7QhjtpPif1QgaG HSbvufzBabk3D8cq/de6sRNI9oTOSIU/RhRNCA8GA2VCYSupCiMkZnlp5EZZAsaD3Yg6RCyy0JbjP isByDFVLDA/YzBQ6bgJbW3/8Js0nXcVOdHLJUthDWjApqeMgoMFSpm62Vhxr+zVRLz2XjbyInSphs Uv3IvMpihuhI0Pny4ZNvrYXlJwCYcxBkAj3y0XCrcy6XU/2OUtQHQhaTSzYPHsav3RSVO9QeCgt0c ukDKspuhMxLrIPHX6LZPxdHcV8+l702Fg965K1e837cXmojecpS83IrsQJTuXLsor0oxgohjJiJQh eYX97HYgA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUk-00EWSQ-6n; Mon, 03 May 2021 17:36:34 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUg-00EWRy-11 for linux-arm-kernel@desiato.infradead.org; Mon, 03 May 2021 17:36:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=27H+BvFer3s1Y37vngYKoBI/y9wscqHm39MmyAoxw0U=; b=zHu+rOAeqpcYjvXC9k3YL72bnw x0c+MWwiUuWpWVWEN3W/oklzKUVRN9pROEjbUcOmk68rKwmFgwVsxBkNxrBxnuheuZhVv6gmWR+/d 5Eisa6aIcvuKREfZeVOXQE/3JiBX3Y8sEPEP68diauoU8KALLZefgMrjQOSnCPHvHo7gxqbITscG7 8jLRM/V4CIenUA0F4AQP0644pISnR3PnNKLHQy+dmE3430FBKq8PfP+t7Ekf6sSUJaK0tirj2Ecde sAltuGhQMiF7E0JL1hodDzm+LM13tdLGoJzHywJXz33vJir1adhBicAazTsBf1D9XyAerXjDnt1HQ DHL97YLg==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUd-003MIX-2C for linux-arm-kernel@lists.infradead.org; Mon, 03 May 2021 17:36:28 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id F400B20B8006; Mon, 3 May 2021 10:36:23 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com F400B20B8006 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1620063384; bh=27H+BvFer3s1Y37vngYKoBI/y9wscqHm39MmyAoxw0U=; h=From:To:Subject:Date:In-Reply-To:References:From; b=dl6V/Uswvh85+qjvxPyvO12cWqKGTHC7p2cooPOwG26LmWMxATKi/fYBN53tScEXR 1mhyM3FpBIYEpnthksh7zyag6o50NOTInqXXOS+BFbhMwN40GRdJA/denkOBcnikNf WGxKCP2a75rmSDeUb6UPt6EdYGgAM7nKWerpV8pc= From: madvenka@linux.microsoft.com To: broonie@kernel.org, jpoimboe@redhat.com, mark.rutland@arm.com, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 1/4] arm64: Introduce stack trace reliability checks in the unwinder Date: Mon, 3 May 2021 12:36:12 -0500 Message-Id: <20210503173615.21576-2-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210503173615.21576-1-madvenka@linux.microsoft.com> References: <65cf4dfbc439b010b50a0c46ec500432acde86d6> <20210503173615.21576-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210503_103627_177729_4EDFC29A X-CRM114-Status: GOOD ( 16.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" The unwinder should check for the presence of various features and conditions that can render the stack trace unreliable and mark the the stack trace as unreliable for the benefit of the caller. Introduce the first reliability check - If a return PC encountered in a stack trace is not a valid kernel text address, the stack trace is considered unreliable. It could be some generated code. Mark the stack trace unreliable. Other reliability checks will be added in the future. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 4 ++++ arch/arm64/kernel/stacktrace.c | 19 ++++++++++++++++--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index eb29b1fe8255..f1eab6b029f7 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -49,6 +49,8 @@ struct stack_info { * * @graph: When FUNCTION_GRAPH_TRACER is selected, holds the index of a * replacement lr value in the ftrace graph stack. + * + * @reliable: Is this stack frame reliable? */ struct stackframe { unsigned long fp; @@ -59,6 +61,7 @@ struct stackframe { #ifdef CONFIG_FUNCTION_GRAPH_TRACER int graph; #endif + bool reliable; }; extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame); @@ -169,6 +172,7 @@ static inline void start_backtrace(struct stackframe *frame, bitmap_zero(frame->stacks_done, __NR_STACK_TYPES); frame->prev_fp = 0; frame->prev_type = STACK_TYPE_UNKNOWN; + frame->reliable = true; } #endif /* __ASM_STACKTRACE_H */ diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index d55bdfb7789c..c21a1bca28f3 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -44,6 +44,8 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) unsigned long fp = frame->fp; struct stack_info info; + frame->reliable = true; + /* Terminal record; nothing to unwind */ if (!fp) return -ENOENT; @@ -86,12 +88,24 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) */ frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp)); frame->pc = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8)); + frame->pc = ptrauth_strip_insn_pac(frame->pc); frame->prev_fp = fp; frame->prev_type = info.type; + /* + * First, make sure that the return address is a proper kernel text + * address. A NULL or invalid return address probably means there's + * some generated code which __kernel_text_address() doesn't know + * about. Mark the stack trace as not reliable. + */ + if (!__kernel_text_address(frame->pc)) { + frame->reliable = false; + return 0; + } + #ifdef CONFIG_FUNCTION_GRAPH_TRACER if (tsk->ret_stack && - (ptrauth_strip_insn_pac(frame->pc) == (unsigned long)return_to_handler)) { + frame->pc == (unsigned long)return_to_handler) { struct ftrace_ret_stack *ret_stack; /* * This is a case where function graph tracer has @@ -103,11 +117,10 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) if (WARN_ON_ONCE(!ret_stack)) return -EINVAL; frame->pc = ret_stack->ret; + frame->pc = ptrauth_strip_insn_pac(frame->pc); } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ - frame->pc = ptrauth_strip_insn_pac(frame->pc); - return 0; } NOKPROBE_SYMBOL(unwind_frame); From patchwork Mon May 3 17:36:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12236623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C3BFC43461 for ; Mon, 3 May 2021 17:38:26 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D53C7610A6 for ; Mon, 3 May 2021 17:38:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D53C7610A6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XgbO8f/ZZk6DQoIov9czYV8Ix2m4DDFrACq04bvkJ6o=; b=EwVafWKG+ZOWPZXhXx1yydYXu 2hiyN4BXoBsMPcygesJusWXKnlodls4uuhFd6exzI9+LKsHzPl031GOKth0doZsMFniAm++VvFe2B +Pace1jQTtpBVhatBNU3DIBWnBf86pqaj5lwwqs5O7N0hir2J6D420IbAfIvC3YtebJ5FpeTKGGHX 0Zx8wOqW4JNlQfXDLnOK4vENffsFxbWmRIl+3Ssfh9XIqZjn1VRywTMfl7F94KZeATpgntMgUGjU6 GYYGBz5hU4Cwh8IG+l3zVY/eeKVVA4HbvthxutDbpOEYzUYsNchJ43tqsjwZAFqI+uVswtNLiX0TL vs3NPG3PQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcV8-00EWUe-Vz; Mon, 03 May 2021 17:36:59 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUh-00EWS0-2b for linux-arm-kernel@desiato.infradead.org; Mon, 03 May 2021 17:36:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=EYht7bnnZALqCAngJGQY0oHKDEKVB3mk09HnhdDDWvk=; b=2AhHoQj6UIO/tTtJy0UiLK3H1U 0acbr6lvspYf2VmudI3Ve2b3TpOEWS/OP96CWVJZHOATlpX0hIs0nOmxh/namrTsAHDudPjkejHKI 6kLkBKvvPI/FFpjHia7oE1weH8a7Y1qItjyLIk+u+blw/q2/RP97gTIh7ThdqrlGEdyBRngjPGkc7 NNt/DaAg/dJhwRJVIw7XN3/oTHaRWCobnFh5X5oLc6WvaoVc9MR5j1bccUgLavXQtk/yX1Hen+u5G 4a6Sgbe9oRCuyYAqhFND6VsSk+9DR7sHxmQkFkdnvdLWZxIkZBlnfJje+VBmhC06GZq1znq85udIY Dwt71iSA==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUd-003MIY-2s for linux-arm-kernel@lists.infradead.org; Mon, 03 May 2021 17:36:29 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id 02C9920B8008; Mon, 3 May 2021 10:36:24 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 02C9920B8008 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1620063385; bh=EYht7bnnZALqCAngJGQY0oHKDEKVB3mk09HnhdDDWvk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=k223RR53gCBeR0IYEEx2yTde/21IkoXVl7restM7bssa2WtPe/+q/LDATZTRexhGT apAZ38tojvUlpG1qXWDwE0WPk3Tt8Ylk2028pLx/ruhcc1CeTEQ7DeXnPcHE9dbg5k GYdcoU6v5P/kCeNomEshBtzuJRFn0RU3ONJILvsU= From: madvenka@linux.microsoft.com To: broonie@kernel.org, jpoimboe@redhat.com, mark.rutland@arm.com, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 2/4] arm64: Check the return PC against unreliable code sections Date: Mon, 3 May 2021 12:36:13 -0500 Message-Id: <20210503173615.21576-3-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210503173615.21576-1-madvenka@linux.microsoft.com> References: <65cf4dfbc439b010b50a0c46ec500432acde86d6> <20210503173615.21576-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210503_103627_224700_1C021C42 X-CRM114-Status: GOOD ( 19.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Create a sym_code_ranges[] array to cover the following text sections that contain functions defined as SYM_CODE_*(). These functions are low-level functions (and do not have a proper frame pointer prolog and epilog). So, they are inherently unreliable from a stack unwinding perspective. .entry.text .idmap.text .hyp.idmap.text .hyp.text .hibernate_exit.text .entry.tramp.text If a return PC falls in any of these, mark the stack trace unreliable. The only exception to this is - if the unwinder has reached the last frame already, it will not mark the stack trace unreliable since there is no more unwinding to do. E.g., - ret_from_fork() occurs at the end of the stack trace of kernel tasks. - el0_*() functions occur at the end of EL0 exception stack traces. This covers all user task entries into the kernel. NOTE: - EL1 exception handlers are in .entry.text. So, stack traces that contain those functions will be marked not reliable. This covers interrupts, exceptions and breakpoints encountered while executing in the kernel. - At the end of an interrupt, the kernel can preempt the current task if required. So, the stack traces of all preempted tasks will show the interrupt frame and will be considered unreliable. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/kernel/stacktrace.c | 54 ++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index c21a1bca28f3..1ff14615a55a 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -15,9 +15,48 @@ #include #include +#include #include #include +struct code_range { + unsigned long start; + unsigned long end; +}; + +struct code_range sym_code_ranges[] = +{ + /* non-unwindable ranges */ + { (unsigned long)__entry_text_start, + (unsigned long)__entry_text_end }, + { (unsigned long)__idmap_text_start, + (unsigned long)__idmap_text_end }, + { (unsigned long)__hyp_idmap_text_start, + (unsigned long)__hyp_idmap_text_end }, + { (unsigned long)__hyp_text_start, + (unsigned long)__hyp_text_end }, +#ifdef CONFIG_HIBERNATION + { (unsigned long)__hibernate_exit_text_start, + (unsigned long)__hibernate_exit_text_end }, +#endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { (unsigned long)__entry_tramp_text_start, + (unsigned long)__entry_tramp_text_end }, +#endif + { /* sentinel */ } +}; + +static struct code_range *lookup_range(unsigned long pc) +{ + struct code_range *range; + + for (range = sym_code_ranges; range->start; range++) { + if (pc >= range->start && pc < range->end) + return range; + } + return range; +} + /* * AArch64 PCS assigns the frame pointer to x29. * @@ -43,6 +82,7 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) { unsigned long fp = frame->fp; struct stack_info info; + struct code_range *range; frame->reliable = true; @@ -103,6 +143,8 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) return 0; } + range = lookup_range(frame->pc); + #ifdef CONFIG_FUNCTION_GRAPH_TRACER if (tsk->ret_stack && frame->pc == (unsigned long)return_to_handler) { @@ -118,9 +160,21 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) return -EINVAL; frame->pc = ret_stack->ret; frame->pc = ptrauth_strip_insn_pac(frame->pc); + return 0; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ + if (!range->start) + return 0; + + /* + * The return PC falls in an unreliable function. If the final frame + * has been reached, no more unwinding is needed. Otherwise, mark the + * stack trace not reliable. + */ + if (frame->fp) + frame->reliable = false; + return 0; } NOKPROBE_SYMBOL(unwind_frame); From patchwork Mon May 3 17:36:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12236631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C145FC433ED for ; Mon, 3 May 2021 17:38:45 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33126610A6 for ; Mon, 3 May 2021 17:38:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33126610A6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=FV3kqHu4NeH/ZFUqzfkS8GDsVYTo+AOK37khgFzyhm4=; b=AvxxUkz/cVb5C7Mg4vUI+cPZe X1/Z8Qcu5GsnCLmuaU9RKgQ/QrjHX1ccipiSs0xRkLjshqQMQ7D/0Ockedkl0l6EhkbUCK4RD3ZDW HqeJDlxN2u+gjFQ2q4alv7W5F/kYmUYlJIaAEMf62MZa9fetnHEr87rM9vIpiXNFLnuohrc5gdKHE dZdTEhP3PT4eelcnaQbjK+9hc6vNsMyDeOzcGtIaHxMQwSQoV2d42L1kRpDE75TjfZbG3HfAo+G/k gej5QII30jyYghkQiXnqLLBpg3fp3cVD4HWBo9f7TfgE1LhiSDzkGIQnaDnrPo0cVhnPGW+iqGIP+ 4GmEAHlYg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcVU-00EWXY-Cn; Mon, 03 May 2021 17:37:20 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUh-00EWS2-Ck for linux-arm-kernel@desiato.infradead.org; Mon, 03 May 2021 17:36:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=v2cyhZUNZcYe1H/VOKKxtcwZCxtIV7TELkPZ3X/Arwk=; b=raVo/q3M4eM1TzqWoj/enYwx66 tXPsBPJ/C7Ij51H6W7YnvnW+pyFC+gXNphzz4EoE7JhlOxbDGp/osACqyQJ5vLjHBIAOlzM8KyLac NbYnWFGjKj5BRGW8xCt4K5YA0kbHvIgjxBH6nOWr/6YsllQ0YCLSx+/mqeM1ct6P5HfnpV5ACjm3X libQAFpu3uqAzB+1GJD5r/BVIgkFkC9zyUs1ELEjMSrO2+RSyXcq+6Mt5bLo+txqYTmGohc6HtqQf tJD2xipXR4+GT8U6RliAwCq+QnS8ZMxJKHzOS+CnIYs7NFZnAeed7CoiMfsf6I0FZYLtfsuCHirFF 63o/iM0Q==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUd-003MIx-9E for linux-arm-kernel@lists.infradead.org; Mon, 03 May 2021 17:36:30 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id 06AE520B800D; Mon, 3 May 2021 10:36:25 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 06AE520B800D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1620063386; bh=v2cyhZUNZcYe1H/VOKKxtcwZCxtIV7TELkPZ3X/Arwk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=OoXki5imJ+8pNO4oOmUdP8FN9UipgdR+/oDvH64HF0NGvxuZJ06Ecd2HOx0bnt8sa gWjOTOJbur2XEx9clwqch3PG0bZ6F4msEG+lWIYBTPVCwSY3H5kYQYHsd7rkH0ZMeT 7Ns09s+vfBByCEms+XXf88eARNbEk7iXuPp6GBxQ= From: madvenka@linux.microsoft.com To: broonie@kernel.org, jpoimboe@redhat.com, mark.rutland@arm.com, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 3/4] arm64: Handle miscellaneous functions in .text and .init.text Date: Mon, 3 May 2021 12:36:14 -0500 Message-Id: <20210503173615.21576-4-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210503173615.21576-1-madvenka@linux.microsoft.com> References: <65cf4dfbc439b010b50a0c46ec500432acde86d6> <20210503173615.21576-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210503_103627_399839_07C42153 X-CRM114-Status: GOOD ( 18.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" There are some SYM_CODE functions that are currently in ".text" or ".init.text" sections. Some of these are functions that the unwinder does not care about as they are not "interesting" to livepatch. These will remain in their current sections. The rest I have moved into a new section called ".code.text". Include .code.text in sym_code_ranges[] so the unwinder can check it. I have listed the names of the functions along with the name of their existing section. Don't care functions ==================== efi-entry.S: efi_enter_kernel .init.text relocate_kernel.S: arm64_relocate_new_kernel .text sigreturn.S: __kernel_rt_sigreturn .text arch/arm64/kvm/hyp/hyp-entry.S: el2t_sync_invalid .text el2t_irq_invalid .text el2t_fiq_invalid .text el2t_error_invalid .text el2h_irq_invalid .text el2h_fiq_invalid .text el1_fiq_invalid .text __kvm_hyp_vector .text __bp_harden_hyp_vecs .text arch/arm64/kvm/hyp/nvhe/host.S: __kvm_hyp_host_vector .text __kvm_hyp_host_forward_smc .text Rest of the functions (moved to .code.text) ===================== entry.S: __swpan_entry_el1 .text __swpan_exit_el1 .text __swpan_entry_el0 .text __swpan_exit_el0 .text ret_from_fork .text __sdei_asm_handler .text head.S: primary_entry .init.text preserve_boot_args .init.text entry-ftrace.S: ftrace_regs_caller .text ftrace_caller .text ftrace_common .text ftrace_graph_caller .text return_to_handler .text kprobes_trampoline.S: kretprobe_trampoline .text Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/entry-ftrace.S | 5 +++++ arch/arm64/kernel/entry.S | 6 ++++++ arch/arm64/kernel/head.S | 3 ++- arch/arm64/kernel/probes/kprobes_trampoline.S | 2 ++ arch/arm64/kernel/stacktrace.c | 2 ++ arch/arm64/kernel/vmlinux.lds.S | 7 +++++++ 7 files changed, 25 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 2f36b16a5b5d..bceda68aaa79 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -20,5 +20,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __code_text_start[], __code_text_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index b3e4f9a088b1..c0831a49c290 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -12,7 +12,9 @@ #include #include + .text #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + .pushsection ".code.text", "ax" /* * Due to -fpatchable-function-entry=2, the compiler has placed two NOPs before * the regular function prologue. For an enabled callsite, ftrace_init_nop() and @@ -135,6 +137,7 @@ SYM_CODE_START(ftrace_graph_caller) b ftrace_common_return SYM_CODE_END(ftrace_graph_caller) #endif + .popsection #else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ @@ -315,6 +318,7 @@ SYM_FUNC_START(ftrace_stub) SYM_FUNC_END(ftrace_stub) #ifdef CONFIG_FUNCTION_GRAPH_TRACER + .pushsection ".code.text", "ax" /* * void return_to_handler(void) * @@ -342,4 +346,5 @@ SYM_CODE_START(return_to_handler) ret SYM_CODE_END(return_to_handler) + .popsection #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 6acfc5e6b5e0..3f9f7f80cd65 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -402,6 +402,7 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 .endm #ifdef CONFIG_ARM64_SW_TTBR0_PAN + .pushsection ".code.text", "ax" /* * Set the TTBR0 PAN bit in SPSR. When the exception is taken from * EL0, there is no need to check the state of TTBR0_EL1 since @@ -442,6 +443,7 @@ SYM_CODE_START_LOCAL(__swpan_exit_el0) */ b post_ttbr_update_workaround SYM_CODE_END(__swpan_exit_el0) + .popsection #endif .macro irq_stack_entry @@ -950,6 +952,7 @@ SYM_FUNC_START(cpu_switch_to) SYM_FUNC_END(cpu_switch_to) NOKPROBE(cpu_switch_to) + .pushsection ".code.text", "ax" /* * This is how we return from a fork. */ @@ -962,6 +965,7 @@ SYM_CODE_START(ret_from_fork) b ret_to_user SYM_CODE_END(ret_from_fork) NOKPROBE(ret_from_fork) + .popsection #ifdef CONFIG_ARM_SDE_INTERFACE @@ -1040,6 +1044,7 @@ SYM_DATA_END(__sdei_asm_trampoline_next_handler) #endif /* CONFIG_RANDOMIZE_BASE */ #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + .pushsection ".code.text", "ax" /* * Software Delegated Exception entry point. * @@ -1150,4 +1155,5 @@ alternative_else_nop_endif #endif SYM_CODE_END(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) + .popsection #endif /* CONFIG_ARM_SDE_INTERFACE */ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 840bda1869e9..4ce96dfac2b8 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -75,7 +75,7 @@ __EFI_PE_HEADER __INIT - + .pushsection ".code.text", "ax" /* * The following callee saved general purpose registers are used on the * primary lowlevel boot path: @@ -120,6 +120,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) mov x1, #0x20 // 4 x 8 bytes b __inval_dcache_area // tail call SYM_CODE_END(preserve_boot_args) + .popsection /* * Macro to create a table entry to the next page. diff --git a/arch/arm64/kernel/probes/kprobes_trampoline.S b/arch/arm64/kernel/probes/kprobes_trampoline.S index 288a84e253cc..9244e119af3e 100644 --- a/arch/arm64/kernel/probes/kprobes_trampoline.S +++ b/arch/arm64/kernel/probes/kprobes_trampoline.S @@ -8,6 +8,7 @@ #include .text + .pushsection ".code.text", "ax" .macro save_all_base_regs stp x0, x1, [sp, #S_X0] @@ -80,3 +81,4 @@ SYM_CODE_START(kretprobe_trampoline) ret SYM_CODE_END(kretprobe_trampoline) + .popsection diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 1ff14615a55a..33e174160f9b 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -43,6 +43,8 @@ struct code_range sym_code_ranges[] = { (unsigned long)__entry_tramp_text_start, (unsigned long)__entry_tramp_text_end }, #endif + { (unsigned long)__code_text_start, + (unsigned long)__code_text_end }, { /* sentinel */ } }; diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7eea7888bb02..c00b3232e6dc 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -103,6 +103,12 @@ jiffies = jiffies_64; #define TRAMP_TEXT #endif +#define CODE_TEXT \ + . = ALIGN(SZ_4K); \ + __code_text_start = .; \ + *(.code.text) \ + __code_text_end = .; + /* * The size of the PE/COFF section that covers the kernel image, which * runs from _stext to _edata, must be a round multiple of the PE/COFF @@ -145,6 +151,7 @@ SECTIONS SOFTIRQENTRY_TEXT ENTRY_TEXT TEXT_TEXT + CODE_TEXT SCHED_TEXT CPUIDLE_TEXT LOCK_TEXT From patchwork Mon May 3 17:36:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12236629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FEB5C43461 for ; Mon, 3 May 2021 17:38:30 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EAEEA610E9 for ; Mon, 3 May 2021 17:38:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EAEEA610E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Lqq3vytQ1zk5/JYurglAWhuMKx/m+G4sJ/mEWy6UeEM=; b=dEKd7A+Ka7qfVY+uJHxWWt4ZL MbgY8bLSLd5mlAXeYknYsvWo0CbAIpvoDV9smfI8IswGTUAPLPxI+aGI2MfxevlE5lzIqcrf44PQy hi8h7e7UiHZkFALjCxpJgf/ibg4O6W/gD/sgJ1aXnKd/CwD6VG+PfQR9ii1lGOoWIMiEVipYCjcxk RYW781os+G1AVsLX7jSdo7g08VbvAOAYA7N6SZJifMJYZTPluB14Cen3exSbSGDd0hcR9dIRXX9AN 37jI1TBrzp2Cq5QoHzYS1TjXIdXO5bfp034yTzV9TmjXpnvVJQehToOnJv97L/Y5doMEiykluE1Ol 3U/tx07ag==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcVJ-00EWVQ-Mq; Mon, 03 May 2021 17:37:09 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUh-00EWS1-AB for linux-arm-kernel@desiato.infradead.org; Mon, 03 May 2021 17:36:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=bYCJMeYNqN5Rl7cucEEXyAKwSqecSENtHSd9RFOfCiw=; b=4tQos6ounRW6fUvvGc4zE5Zqjt L+O8CLvoWUQkh6y0SvIxhb9WQQHzpD7oXQvusYy0X4spwTyW8gUMWJ1ywZfO9ZSSI6V0C/z4eQdk6 wVy5me+SUehwS2cHMZ7MMOKgitvPraySWoOjml0gIL2ZiWSuRIuNRaohIRog/HXqfYDfT0GBw4wBn OCQoSX4IO6e3orX0uWRIr3k3gBrkcwqvp/HyvOOkQiuDhdKu9g5caSOLhmjHDMYOkgQNglEsAn87v qthGY+XocHRqkUuWDSxiGDF6UXBMctj/82uz+zqF5+i09GaHv2K8tASnjq8uABQ2lW1cy8z1Q7lRG BncS+s9Q==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldcUe-003MJj-6t for linux-arm-kernel@lists.infradead.org; Mon, 03 May 2021 17:36:30 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id 0CEFF20B8016; Mon, 3 May 2021 10:36:26 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0CEFF20B8016 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1620063387; bh=bYCJMeYNqN5Rl7cucEEXyAKwSqecSENtHSd9RFOfCiw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=n6j/xBMhadoUBM8BC5SAejnw2WorVnwcXyM8I9FLKnihE+3sp1AzlpoKaNnOspqUJ StwncmZF/PXyvbXi0ZHShyESiDMjPTSHsmW7r9aD5340M4yFz6AiYf6DQPegeBU9a5 g/w6xnSg0NPRwqikcnZoeokiAmcNA1n44Dn6vup0= From: madvenka@linux.microsoft.com To: broonie@kernel.org, jpoimboe@redhat.com, mark.rutland@arm.com, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v3 4/4] arm64: Handle funtion graph tracer better in the unwinder Date: Mon, 3 May 2021 12:36:15 -0500 Message-Id: <20210503173615.21576-5-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210503173615.21576-1-madvenka@linux.microsoft.com> References: <65cf4dfbc439b010b50a0c46ec500432acde86d6> <20210503173615.21576-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210503_103628_330977_B7C46F9A X-CRM114-Status: GOOD ( 25.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" The Function Graph Tracer modifies the return address of a traced function to a return trampoline (return_to_handler()) to gather tracing data on function return. When the unwinder encounters return_to_handler(), it calls ftrace_graph_get_ret_stack() to lookup the original return address in the return address stack. This lookup will succeed as long as the unwinder is invoked when the traced function is executing. However, when the traced function returns and control goes to return_to_handler(), this lookup will not succeed because: - the return address on the stack would not be return_to_handler. It would be return_to_handler+someoffset. To solve this, get the address range for return_to_handler() by looking up its symbol table entry and check if frame->pc falls in the range. This is also required for the unwinder to maintain the index into the return address stack correctly as it unwinds through Function Graph trace return trampolines. - the original return address will be popped off the return address stack at some point. From this point till the end of return_to_handler(), the lookup will not succeed. The stack trace is unreliable in that window. On arm64, each return address stack entry also stores the FP of the caller of the traced function. Compare the FP in the current frame with the entry that is looked up. If the FP matches, then, all is well. Else, it is in the window. mark the stack trace unreliable. Although it is possible to close the window mentioned above, it is not worth it. It is a tiny window. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 3 ++ arch/arm64/kernel/stacktrace.c | 60 ++++++++++++++++++++++++----- 2 files changed, 53 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index f1eab6b029f7..e70a2a6451db 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -69,6 +69,7 @@ extern void walk_stackframe(struct task_struct *tsk, struct stackframe *frame, bool (*fn)(void *, unsigned long), void *data); extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, const char *loglvl); +extern void init_ranges(void); DECLARE_PER_CPU(unsigned long *, irq_stack_ptr); @@ -154,6 +155,8 @@ static inline bool on_accessible_stack(const struct task_struct *tsk, static inline void start_backtrace(struct stackframe *frame, unsigned long fp, unsigned long pc) { + init_ranges(); + frame->fp = fp; frame->pc = pc; #ifdef CONFIG_FUNCTION_GRAPH_TRACER diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 33e174160f9b..7504aec79faa 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -26,6 +26,9 @@ struct code_range { struct code_range sym_code_ranges[] = { + /* unwindable ranges */ + { (unsigned long)return_to_handler, 0 }, + /* non-unwindable ranges */ { (unsigned long)__entry_text_start, (unsigned long)__entry_text_end }, @@ -48,6 +51,33 @@ struct code_range sym_code_ranges[] = { /* sentinel */ } }; +void init_ranges(void) +{ + static char sym[KSYM_NAME_LEN]; + static bool inited = false; + struct code_range *range; + unsigned long pc, size, offset; + + if (inited) + return; + + for (range = sym_code_ranges; range->start; range++) { + if (range->end) + continue; + + pc = (unsigned long)range->start; + if (kallsyms_lookup(pc, &size, &offset, NULL, sym)) { + range->start = pc - offset; + range->end = range->start + size; + } else { + /* Range will only include one instruction */ + range->start = pc; + range->end = pc + 4; + } + } + inited = true; +} + static struct code_range *lookup_range(unsigned long pc) { struct code_range *range; @@ -149,19 +179,29 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) #ifdef CONFIG_FUNCTION_GRAPH_TRACER if (tsk->ret_stack && - frame->pc == (unsigned long)return_to_handler) { + range->start == (unsigned long)return_to_handler) { struct ftrace_ret_stack *ret_stack; /* - * This is a case where function graph tracer has - * modified a return address (LR) in a stack frame - * to hook a function return. - * So replace it to an original value. + * Either the function graph tracer has modified a return + * address (LR) in a stack frame to the return trampoline. + * Or, the return trampoline itself is executing upon the + * return of a traced function. Lookup the original return + * address and replace frame->pc with it. + * + * However, the return trampoline pops the original return + * address off the return address stack at some point. So, + * there is a small window towards the end of the return + * trampoline where the lookup will fail. In that case, + * mark the stack trace as unreliable and proceed. */ - ret_stack = ftrace_graph_get_ret_stack(tsk, frame->graph++); - if (WARN_ON_ONCE(!ret_stack)) - return -EINVAL; - frame->pc = ret_stack->ret; - frame->pc = ptrauth_strip_insn_pac(frame->pc); + ret_stack = ftrace_graph_get_ret_stack(tsk, frame->graph); + if (!ret_stack || frame->fp != ret_stack->fp) { + frame->reliable = false; + } else { + frame->pc = ret_stack->ret; + frame->pc = ptrauth_strip_insn_pac(frame->pc); + frame->graph++; + } return 0; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */