From patchwork Tue Nov 23 19:37:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12693496 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83A5FC433EF for ; Tue, 23 Nov 2021 19:39:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IpngilX6Hk29vywWdJxQ4UIIiiQdI3TQMCywxELaIoE=; b=qRO19WzZ19fIiy vc8mdNqIgpvQsSWgdMGNcM7yHe7NK3+VSG0pXP4S3M1jCz5Qf5V9Cyi6bZ1kAtqYvcmMoRgFEMugF t8oiDWg5IbOw6xh0WOJq6LX/v40IBFz+MY/ah7yQ44hadw+weeAT/aZnUAbiTXzQYPMF8cpnYslz2 Zzej+uxFvC5B3YcS4hmMlfZ8WRvK6JsRzqDl0HSXCqeSsGohMQZy4+iLesfQ8khoU9oJCOIgy39QY 8gPCYeeEr00Gia+y4/Ovc5WzOymsX2bJCtUDeFODBU7QeUjwaNDvGS3iqwJFX8wH/7WqX3OMtg/4M VxUVupTV/rE109FAmv4g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbbo-003KQs-IS; Tue, 23 Nov 2021 19:37:40 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbbk-003KOz-CL for linux-arm-kernel@lists.infradead.org; Tue, 23 Nov 2021 19:37:38 +0000 Received: from x64host.home (unknown [47.187.212.181]) by linux.microsoft.com (Postfix) with ESMTPSA id C5AF920D4D2F; Tue, 23 Nov 2021 11:37:33 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com C5AF920D4D2F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1637696254; bh=aBWkrJAFXMJSS+1WUdBJpRlaPcIVMzXm3IBpam3uJ9Y=; h=From:To:Subject:Date:In-Reply-To:References:From; b=TJytDyRf37Fgz7JgNuv0WuLabLv1H8vkSUBnbEQgY7ZF6U4RwYZ4K+WJLUlurJcBf kiR9pf1NI5YpZWeq7VWjgM/YiVre44eCrIUi4PrE2mExPC+gLT16G8657dAPf70qad I+4Ze/ulCKeoNicrQuBaxlwLMUFlUGFD5t3VdTKk= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [PATCH v11 1/5] arm64: Call stack_backtrace() only from within walk_stackframe() Date: Tue, 23 Nov 2021 13:37:19 -0600 Message-Id: <20211123193723.12112-2-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123193723.12112-1-madvenka@linux.microsoft.com> References: <8b861784d85a21a9bf08598938c11aff1b1249b9> <20211123193723.12112-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211123_113736_514466_EAE9892F X-CRM114-Status: GOOD ( 13.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Currently, arch_stack_walk() calls start_backtrace() and walk_stackframe() separately. There is no need to do that. Instead, call start_backtrace() from within walk_stackframe(). In other words, walk_stackframe() is the only unwind function a consumer needs to call. Currently, the only consumer is arch_stack_walk(). In the future, arch_stack_walk_reliable() will be another consumer. Currently, there is a check for a NULL task in unwind_frame(). It is not needed since all current consumers pass a non-NULL task. Use struct stackframe only within the unwind functions. Signed-off-by: Madhavan T. Venkataraman Reviewed-by: Mark Brown --- arch/arm64/kernel/stacktrace.c | 41 ++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 0fb58fed54cb..7217c4f63ef7 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -69,9 +69,6 @@ static int notrace unwind_frame(struct task_struct *tsk, unsigned long fp = frame->fp; struct stack_info info; - if (!tsk) - tsk = current; - /* Final frame; nothing to unwind */ if (fp == (unsigned long)task_pt_regs(tsk)->stackframe) return -ENOENT; @@ -143,15 +140,19 @@ static int notrace unwind_frame(struct task_struct *tsk, NOKPROBE_SYMBOL(unwind_frame); static void notrace walk_stackframe(struct task_struct *tsk, - struct stackframe *frame, + unsigned long fp, unsigned long pc, bool (*fn)(void *, unsigned long), void *data) { + struct stackframe frame; + + start_backtrace(&frame, fp, pc); + while (1) { int ret; - if (!fn(data, frame->pc)) + if (!fn(data, frame.pc)) break; - ret = unwind_frame(tsk, frame); + ret = unwind_frame(tsk, &frame); if (ret < 0) break; } @@ -195,17 +196,19 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, struct task_struct *task, struct pt_regs *regs) { - struct stackframe frame; - - if (regs) - start_backtrace(&frame, regs->regs[29], regs->pc); - else if (task == current) - start_backtrace(&frame, - (unsigned long)__builtin_frame_address(1), - (unsigned long)__builtin_return_address(0)); - else - start_backtrace(&frame, thread_saved_fp(task), - thread_saved_pc(task)); - - walk_stackframe(task, &frame, consume_entry, cookie); + unsigned long fp, pc; + + if (regs) { + fp = regs->regs[29]; + pc = regs->pc; + } else if (task == current) { + /* Skip arch_stack_walk() in the stack trace. */ + fp = (unsigned long)__builtin_frame_address(1); + pc = (unsigned long)__builtin_return_address(0); + } else { + /* Caller guarantees that the task is not running. */ + fp = thread_saved_fp(task); + pc = thread_saved_pc(task); + } + walk_stackframe(task, fp, pc, consume_entry, cookie); } From patchwork Tue Nov 23 19:37:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12693495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B30FEC433F5 for ; Tue, 23 Nov 2021 19:39:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=clIOCiebH5unIcPqhkKIjpZXc8U5YudrifH6Ww1MdOg=; b=eG5ywU2aIDGod9 JAb6fF0fVS4j4nBs0cagXhxpnWLvVBthxfgMIZdBGexmNyBbFMfYprMBImpdK/60Ta3crFDsUcYgG SMn8KgILEgFPAzT2D2Qyqc6V5xCbouSeHw7pkwkB5mAheFU5O41FZZyM5rV2WWyJMETFjHiA8s4IV 5mNCun35YpMpcZCWSqW8HE4fNMm0ao53zNT8atyV8kh/gkNyKjS9M3wL117xtY2ewW3oZO30LU3fF ur9qSRL3qxcaYdQ9g6VFX4IUJoFQbTqQsgNWnmPAxSG1TFb4fQKnEIi8O2mA9SNz/TdxMwPvzGPoX RD8U+0NUZiNzNoebLtmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbby-003KSo-VR; Tue, 23 Nov 2021 19:37:51 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbbk-003KP4-Js for linux-arm-kernel@lists.infradead.org; Tue, 23 Nov 2021 19:37:38 +0000 Received: from x64host.home (unknown [47.187.212.181]) by linux.microsoft.com (Postfix) with ESMTPSA id D82DA20D4D35; Tue, 23 Nov 2021 11:37:34 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com D82DA20D4D35 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1637696255; bh=H/EAM8XTL11m8uJh5cKvuLNCQnwF39y51M9DbDTiLvs=; h=From:To:Subject:Date:In-Reply-To:References:From; b=PPD3YXWkzU90QtJeg8CPKxmJitFE6ph3inHyeoaCG7+MRCWBnfa/I9sn74ilvyqtl oNTiQu35Xd9OIoEfjX+XvS32aXKNt2oPv1Tu7cqNDvy1poXAbOFTGDK/SahyseZ/In S5MhBZKPUEEelvDLrbFy9BDSxuD/Qwj1RPuDibxs= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [PATCH v11 2/5] arm64: Rename unwinder functions Date: Tue, 23 Nov 2021 13:37:20 -0600 Message-Id: <20211123193723.12112-3-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123193723.12112-1-madvenka@linux.microsoft.com> References: <8b861784d85a21a9bf08598938c11aff1b1249b9> <20211123193723.12112-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211123_113736_742227_7D0087C2 X-CRM114-Status: GOOD ( 14.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Rename unwinder functions for consistency and better naming. - Rename start_backtrace() to unwind_start(). - Rename unwind_frame() to unwind_next(). - Rename walk_stackframe() to unwind(). Signed-off-by: Madhavan T. Venkataraman Reviewed-by: Mark Brown Acked-by: Mark Rutland --- arch/arm64/kernel/stacktrace.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 7217c4f63ef7..918852cd2681 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -33,8 +33,8 @@ */ -static void start_backtrace(struct stackframe *frame, unsigned long fp, - unsigned long pc) +static void unwind_start(struct stackframe *frame, unsigned long fp, + unsigned long pc) { frame->fp = fp; frame->pc = pc; @@ -45,7 +45,7 @@ static void start_backtrace(struct stackframe *frame, unsigned long fp, /* * Prime the first unwind. * - * In unwind_frame() we'll check that the FP points to a valid stack, + * In unwind_next() we'll check that the FP points to a valid stack, * which can't be STACK_TYPE_UNKNOWN, and the first unwind will be * treated as a transition to whichever stack that happens to be. The * prev_fp value won't be used, but we set it to 0 such that it is @@ -63,8 +63,8 @@ static void start_backtrace(struct stackframe *frame, unsigned long fp, * records (e.g. a cycle), determined based on the location and fp value of A * and the location (but not the fp value) of B. */ -static int notrace unwind_frame(struct task_struct *tsk, - struct stackframe *frame) +static int notrace unwind_next(struct task_struct *tsk, + struct stackframe *frame) { unsigned long fp = frame->fp; struct stack_info info; @@ -104,7 +104,7 @@ static int notrace unwind_frame(struct task_struct *tsk, /* * Record this frame record's values and location. The prev_fp and - * prev_type are only meaningful to the next unwind_frame() invocation. + * prev_type are only meaningful to the next unwind_next() invocation. */ frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp)); frame->pc = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8)); @@ -137,27 +137,27 @@ static int notrace unwind_frame(struct task_struct *tsk, return 0; } -NOKPROBE_SYMBOL(unwind_frame); +NOKPROBE_SYMBOL(unwind_next); -static void notrace walk_stackframe(struct task_struct *tsk, - unsigned long fp, unsigned long pc, - bool (*fn)(void *, unsigned long), void *data) +static void notrace unwind(struct task_struct *tsk, + unsigned long fp, unsigned long pc, + bool (*fn)(void *, unsigned long), void *data) { struct stackframe frame; - start_backtrace(&frame, fp, pc); + unwind_start(&frame, fp, pc); while (1) { int ret; if (!fn(data, frame.pc)) break; - ret = unwind_frame(tsk, &frame); + ret = unwind_next(tsk, &frame); if (ret < 0) break; } } -NOKPROBE_SYMBOL(walk_stackframe); +NOKPROBE_SYMBOL(unwind); static bool dump_backtrace_entry(void *arg, unsigned long where) { @@ -210,5 +210,5 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, fp = thread_saved_fp(task); pc = thread_saved_pc(task); } - walk_stackframe(task, fp, pc, consume_entry, cookie); + unwind(task, fp, pc, consume_entry, cookie); } From patchwork Tue Nov 23 19:37:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12693497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9ED59C433F5 for ; Tue, 23 Nov 2021 19:39:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=q/FNc1mf9xIdtrZiJILml8vc9kirYzxAxjO/nNyedYw=; b=KeYK8XOGHKs/Fg m6AdzWFecvNLUP/8+C1dMmh8O/OsCJUQzUl0RXnCQIgZDqb9fIc8p4AZQzghe3fe7eqP7PxF+gwNW La/8S+GazAlXc/8t4rwnjUTDMVstb95qFLd5UR9Q9kPFKsEjbP4OqJekNnK17GCZgZ6Z2HvAG0s66 J9dZQYpoh6EUPhsXqR7zgGrPwxm/X7pEKTdwJ4vI2tO51yqHWueGf2jfs2ut3h28CPdUHsxZhQ6Mt q9kLAfju93+bLhB9DCWkBHc+1fBp3vCUAqg5vfnXrEKaKbyQA+XRzkxNTYT+Wd5BdPgiK56mN8S0T TbGdY5oHhlBiusQmR1Ng==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbc9-003KVd-JD; Tue, 23 Nov 2021 19:38:01 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbbl-003KPl-5o for linux-arm-kernel@lists.infradead.org; Tue, 23 Nov 2021 19:37:38 +0000 Received: from x64host.home (unknown [47.187.212.181]) by linux.microsoft.com (Postfix) with ESMTPSA id E8CAD20D4D39; Tue, 23 Nov 2021 11:37:35 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com E8CAD20D4D39 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1637696256; bh=4186IGzt9P0jgk6sDPjuuzrx/GJRpOlohBw5fEU6ZnY=; h=From:To:Subject:Date:In-Reply-To:References:From; b=NH1mpd/pqVIHkZwa0kb2q2g1Eu3HnL+HMfUmRnjAxgTMIakYJdyLKv2ou5oesH8Rq 6TJfVLVB+W9m5yYguic9FbIGxz2CUfmlw9VDulannLVk4axoBFi9r6vaegNSLk4oVt DI9JiII0OOI8WkWuv4tglMKWPh7uY5A+7ny5yvdw= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [PATCH v11 3/5] arm64: Make the unwind loop in unwind() similar to other architectures Date: Tue, 23 Nov 2021 13:37:21 -0600 Message-Id: <20211123193723.12112-4-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123193723.12112-1-madvenka@linux.microsoft.com> References: <8b861784d85a21a9bf08598938c11aff1b1249b9> <20211123193723.12112-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211123_113737_321417_F39F3179 X-CRM114-Status: GOOD ( 20.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Change the loop in unwind() =========================== Change the unwind loop in unwind() to: unwind_start(&frame, fp, pc); while (unwind_continue(tsk, &frame, fn, data)) unwind_next(tsk, &frame); New function unwind_continue() ============================== Define a new function unwind_continue() that is used in the unwind loop to check for conditions that terminate a stack trace. The conditions checked are: - If the bottom of the stack has been reached, terminate. - If the consume_entry() function returns false, the caller of unwind has asked to terminate the stack trace. So, terminate. - If unwind_next() failed for some reason (like stack corruption), terminate. Do not return an error value from unwind_next() =============================================== We want to check for terminating conditions only in unwind_continue() from the unwinder loop. So, do not return an error value from unwind_next(). Simply set a flag in the stackframe and check the flag in unwind_continue(). Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 3 ++ arch/arm64/kernel/stacktrace.c | 75 ++++++++++++++++++----------- 2 files changed, 50 insertions(+), 28 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 3a15d376ab36..d838586adef9 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -51,6 +51,8 @@ struct stack_info { * @kr_cur: When KRETPOLINES is selected, holds the kretprobe instance * associated with the most recently encountered replacement lr * value. + * + * @failed: Unwind failed. */ struct stackframe { unsigned long fp; @@ -61,6 +63,7 @@ struct stackframe { #ifdef CONFIG_KRETPROBES struct llist_node *kr_cur; #endif + bool failed; }; extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 918852cd2681..3b670ab1f0e9 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -54,6 +54,7 @@ static void unwind_start(struct stackframe *frame, unsigned long fp, bitmap_zero(frame->stacks_done, __NR_STACK_TYPES); frame->prev_fp = 0; frame->prev_type = STACK_TYPE_UNKNOWN; + frame->failed = false; } /* @@ -63,24 +64,26 @@ static void unwind_start(struct stackframe *frame, unsigned long fp, * records (e.g. a cycle), determined based on the location and fp value of A * and the location (but not the fp value) of B. */ -static int notrace unwind_next(struct task_struct *tsk, - struct stackframe *frame) +static void notrace unwind_next(struct task_struct *tsk, + struct stackframe *frame) { unsigned long fp = frame->fp; struct stack_info info; - /* Final frame; nothing to unwind */ - if (fp == (unsigned long)task_pt_regs(tsk)->stackframe) - return -ENOENT; - - if (fp & 0x7) - return -EINVAL; + if (fp & 0x7) { + frame->failed = true; + return; + } - if (!on_accessible_stack(tsk, fp, 16, &info)) - return -EINVAL; + if (!on_accessible_stack(tsk, fp, 16, &info)) { + frame->failed = true; + return; + } - if (test_bit(info.type, frame->stacks_done)) - return -EINVAL; + if (test_bit(info.type, frame->stacks_done)) { + frame->failed = true; + return; + } /* * As stacks grow downward, any valid record on the same stack must be @@ -96,8 +99,10 @@ static int notrace unwind_next(struct task_struct *tsk, * stack. */ if (info.type == frame->prev_type) { - if (fp <= frame->prev_fp) - return -EINVAL; + if (fp <= frame->prev_fp) { + frame->failed = true; + return; + } } else { set_bit(frame->prev_type, frame->stacks_done); } @@ -125,8 +130,10 @@ static int notrace unwind_next(struct task_struct *tsk, */ orig_pc = ftrace_graph_ret_addr(tsk, NULL, frame->pc, (void *)frame->fp); - if (WARN_ON_ONCE(frame->pc == orig_pc)) - return -EINVAL; + if (WARN_ON_ONCE(frame->pc == orig_pc)) { + frame->failed = true; + return; + } frame->pc = orig_pc; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ @@ -134,11 +141,31 @@ static int notrace unwind_next(struct task_struct *tsk, if (is_kretprobe_trampoline(frame->pc)) frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur); #endif - - return 0; } NOKPROBE_SYMBOL(unwind_next); +static bool unwind_continue(struct task_struct *task, + struct stackframe *frame, + stack_trace_consume_fn consume_entry, void *cookie) +{ + if (frame->failed) { + /* PC is suspect. Cannot consume it. */ + return false; + } + + if (!consume_entry(cookie, frame->pc)) { + /* Caller terminated the unwind. */ + frame->failed = true; + return false; + } + + if (frame->fp == (unsigned long)task_pt_regs(task)->stackframe) { + /* Final frame; nothing to unwind */ + return false; + } + return true; +} + static void notrace unwind(struct task_struct *tsk, unsigned long fp, unsigned long pc, bool (*fn)(void *, unsigned long), void *data) @@ -146,16 +173,8 @@ static void notrace unwind(struct task_struct *tsk, struct stackframe frame; unwind_start(&frame, fp, pc); - - while (1) { - int ret; - - if (!fn(data, frame.pc)) - break; - ret = unwind_next(tsk, &frame); - if (ret < 0) - break; - } + while (unwind_continue(tsk, &frame, fn, data)) + unwind_next(tsk, &frame); } NOKPROBE_SYMBOL(unwind); From patchwork Tue Nov 23 19:37:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12693499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1FC70C433F5 for ; Tue, 23 Nov 2021 19:39:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ohwqRDp8cLUqykiWVoNU3C90qSHSFAjWmNGF6+aNwII=; b=TZUUMeJAE9VB37 XlgnTitmNjslhiJqk3hFsxUESKSY/Gz5OWj7c6scl31pCgzeQgLrRzma5axO+jSpGFauOIOqKcOBL b5gCjzfQif4o+PLMtFuwbWacLr8MHxfsLXTBQgtnwYbRIPm72nZ/ujNXDQ176Bai6GPZUc/dH5iJV /RjM3I+pz6Oatx2jDQY5AAP0V9Wqsba3HszZKntU64+22jVPMt6OZ3KuvDVsXRFPmsEm8RYoXz2CZ SbN3S+A4R4YJq/T4L96ei3eLO8HseRfw3Pmh4SwHz2fZ2MgAFUeDvlJa0D/d1fEnQvuF6cPrszBvn zA3kZ0xfnNBDtk/2/39w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbcY-003Kc6-BS; Tue, 23 Nov 2021 19:38:26 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbbm-003KQ6-BY for linux-arm-kernel@lists.infradead.org; Tue, 23 Nov 2021 19:37:39 +0000 Received: from x64host.home (unknown [47.187.212.181]) by linux.microsoft.com (Postfix) with ESMTPSA id 062DB20D4D3D; Tue, 23 Nov 2021 11:37:36 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 062DB20D4D3D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1637696257; bh=ANijem9qrqDRCyqcDOusbbhCMatgyg1SMa0VQG+ycgk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=FDK5nwZ6mp0rwag3tiw+83PEtTTQG2hsym0HD4/QKmyTZH6ca4rRngn+vdO5ZzQTs hJgZMffOgqx1+WS2EgyScKVsK7BEeUOuvmURYFtcyjS6+rRgyAXnQQ9+IUFXpsNQb1 FOh07RjB5AJVx13n2fluXl7haaRuQjzzaJ5EDsoc= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [PATCH v11 4/5] arm64: Introduce stack trace reliability checks in the unwinder Date: Tue, 23 Nov 2021 13:37:22 -0600 Message-Id: <20211123193723.12112-5-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123193723.12112-1-madvenka@linux.microsoft.com> References: <8b861784d85a21a9bf08598938c11aff1b1249b9> <20211123193723.12112-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211123_113738_471632_B0A62E6D X-CRM114-Status: GOOD ( 23.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" There are some kernel features and conditions that make a stack trace unreliable. Callers may require the unwinder to detect these cases. E.g., livepatch. Introduce a new function called unwind_check_reliability() that will detect these cases and set a flag in the stack frame. Call unwind_check_reliability() for every frame, that is, in unwind_start() and unwind_next(). Introduce the first reliability check in unwind_check_reliability() - If a return PC is not a valid kernel text address, consider the stack trace unreliable. It could be some generated code. Other reliability checks will be added in the future. Let unwind() return a boolean to indicate if the stack trace is reliable. Introduce arch_stack_walk_reliable() for ARM64. This works like arch_stack_walk() except that it returns -EINVAL if the stack trace is not reliable. Until all the reliability checks are in place, arch_stack_walk_reliable() may not be used by livepatch. But it may be used by debug and test code. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 3 ++ arch/arm64/kernel/stacktrace.c | 59 +++++++++++++++++++++++++++-- 2 files changed, 58 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index d838586adef9..7143e80c3d96 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -53,6 +53,8 @@ struct stack_info { * value. * * @failed: Unwind failed. + * + * @reliable: Stack trace is reliable. */ struct stackframe { unsigned long fp; @@ -64,6 +66,7 @@ struct stackframe { struct llist_node *kr_cur; #endif bool failed; + bool reliable; }; extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 3b670ab1f0e9..77eb00e45558 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -18,6 +18,26 @@ #include #include +/* + * Check the stack frame for conditions that make further unwinding unreliable. + */ +static void unwind_check_reliability(struct task_struct *task, + struct stackframe *frame) +{ + if (frame->fp == (unsigned long)task_pt_regs(task)->stackframe) { + /* Final frame; no more unwind, no need to check reliability */ + return; + } + + /* + * If the PC is not a known kernel text address, then we cannot + * be sure that a subsequent unwind will be reliable, as we + * don't know that the code follows our unwind requirements. + */ + if (!__kernel_text_address(frame->pc)) + frame->reliable = false; +} + /* * AArch64 PCS assigns the frame pointer to x29. * @@ -33,8 +53,9 @@ */ -static void unwind_start(struct stackframe *frame, unsigned long fp, - unsigned long pc) +static void unwind_start(struct task_struct *task, + struct stackframe *frame, + unsigned long fp, unsigned long pc) { frame->fp = fp; frame->pc = pc; @@ -55,6 +76,8 @@ static void unwind_start(struct stackframe *frame, unsigned long fp, frame->prev_fp = 0; frame->prev_type = STACK_TYPE_UNKNOWN; frame->failed = false; + frame->reliable = true; + unwind_check_reliability(task, frame); } /* @@ -141,6 +164,7 @@ static void notrace unwind_next(struct task_struct *tsk, if (is_kretprobe_trampoline(frame->pc)) frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur); #endif + unwind_check_reliability(tsk, frame); } NOKPROBE_SYMBOL(unwind_next); @@ -166,15 +190,16 @@ static bool unwind_continue(struct task_struct *task, return true; } -static void notrace unwind(struct task_struct *tsk, +static bool notrace unwind(struct task_struct *tsk, unsigned long fp, unsigned long pc, bool (*fn)(void *, unsigned long), void *data) { struct stackframe frame; - unwind_start(&frame, fp, pc); + unwind_start(tsk, &frame, fp, pc); while (unwind_continue(tsk, &frame, fn, data)) unwind_next(tsk, &frame); + return !frame.failed && frame.reliable; } NOKPROBE_SYMBOL(unwind); @@ -231,3 +256,29 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, } unwind(task, fp, pc, consume_entry, cookie); } + +/* + * arch_stack_walk_reliable() may not be used for livepatch until all of + * the reliability checks are in place in unwind_consume(). However, + * debug and test code can choose to use it even if all the checks are not + * in place. + */ +noinline int notrace arch_stack_walk_reliable(stack_trace_consume_fn consume_fn, + void *cookie, + struct task_struct *task) +{ + unsigned long fp, pc; + + if (task == current) { + /* Skip arch_stack_walk_reliable() in the stack trace. */ + fp = (unsigned long)__builtin_frame_address(1); + pc = (unsigned long)__builtin_return_address(0); + } else { + /* Caller guarantees that the task is not running. */ + fp = thread_saved_fp(task); + pc = thread_saved_pc(task); + } + if (unwind(task, fp, pc, consume_fn, cookie)) + return 0; + return -EINVAL; +} From patchwork Tue Nov 23 19:37:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12693500 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64571C433F5 for ; Tue, 23 Nov 2021 19:40:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p9iw20ydcPuJt1asq2FpUs3Q1QwFSPGjEE8v5rubacw=; b=AkebfMV7P0rZyl ZjOzkgGGCSniHSlKfj2SsZ3gSkchjmPVQKtHTz3GuH3iE6LfXrMp9fZCsoODAeYhzGr6w8aRCHVZZ FF+CObvRtfUEtuflbLUZEoXGFbfIWfZR5pBbDAwl2kC3IabQ/x5+A9alzuFio74uf5lJ/ZhuBf7ie B1CYNOFPNys4N97gPryuXb14Eko0cfghsD8tuHgLzI1B5ft5yYyoNDz9lFkYX11Ju3u5MzeijLUJD VXYdpld0peDeJlmiM1BHAbRajiz2JWr2qs5V5qJwvVKrw/GERncUnHWqEwwnBUpH9EMyEAoE5gnnS TzWvrTIOiZ7tVFWALw0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbcn-003Kj0-VD; Tue, 23 Nov 2021 19:38:42 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpbbn-003KQe-7p for linux-arm-kernel@lists.infradead.org; Tue, 23 Nov 2021 19:37:40 +0000 Received: from x64host.home (unknown [47.187.212.181]) by linux.microsoft.com (Postfix) with ESMTPSA id 16CF920D4D3A; Tue, 23 Nov 2021 11:37:38 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 16CF920D4D3A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1637696258; bh=OQhzBj7Wen/J06yxGIiml0JSLgVuUwO6DOPVaGxCJ8I=; h=From:To:Subject:Date:In-Reply-To:References:From; b=q3ZqBQyyj4vXUauF/3jjrSnxqBpki3Mp5EpUGFvk0xo5keTmx+XEcKmEwssmidcc+ 2a3SQOpNcC54SkbhFe1CoeeS/etKkPBDo8F7v9Iq12deNyIkxsy5KhItvzU0KS7TPE 9RFBUwoxv5CvSNmg3NXoKdl7k8Si0oSAQLiqyOmc= From: madvenka@linux.microsoft.com To: mark.rutland@arm.com, broonie@kernel.org, jpoimboe@redhat.com, ardb@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [PATCH v11 5/5] arm64: Create a list of SYM_CODE functions, check return PC against list Date: Tue, 23 Nov 2021 13:37:23 -0600 Message-Id: <20211123193723.12112-6-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123193723.12112-1-madvenka@linux.microsoft.com> References: <8b861784d85a21a9bf08598938c11aff1b1249b9> <20211123193723.12112-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211123_113739_366479_E294C88F X-CRM114-Status: GOOD ( 20.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" SYM_CODE functions don't follow the usual calling conventions. Check if the return PC in a stack frame falls in any of these. If it does, consider the stack trace unreliable. Define a special section for unreliable functions ================================================= Define a SYM_CODE_END() macro for arm64 that adds the function address range to a new section called "sym_code_functions". Linker file =========== Include the "sym_code_functions" section under read-only data in vmlinux.lds.S. Initialization ============== Define an early_initcall() to create a sym_code_functions[] array from the linker data. Unwinder check ============== Add a reliability check in unwind_is_reliable() that compares a return PC with sym_code_functions[]. If there is a match, then return failure. Signed-off-by: Madhavan T. Venkataraman Reviewed-by: Mark Brown --- arch/arm64/include/asm/linkage.h | 12 +++++++ arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/stacktrace.c | 55 +++++++++++++++++++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 10 ++++++ 4 files changed, 78 insertions(+) diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h index 9906541a6861..616bad74e297 100644 --- a/arch/arm64/include/asm/linkage.h +++ b/arch/arm64/include/asm/linkage.h @@ -68,4 +68,16 @@ SYM_FUNC_END_ALIAS(x); \ SYM_FUNC_END_ALIAS(__pi_##x) +/* + * Record the address range of each SYM_CODE function in a struct code_range + * in a special section. + */ +#define SYM_CODE_END(name) \ + SYM_END(name, SYM_T_NONE) ;\ + 99: ;\ + .pushsection "sym_code_functions", "aw" ;\ + .quad name ;\ + .quad 99b ;\ + .popsection + #endif diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 152cb35bf9df..ac01189668c5 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -22,5 +22,6 @@ extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +extern char __sym_code_functions_start[], __sym_code_functions_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 77eb00e45558..e47f852ad12a 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -18,12 +18,41 @@ #include #include +struct code_range { + unsigned long start; + unsigned long end; +}; + +static struct code_range *sym_code_functions; +static int num_sym_code_functions; + +int __init init_sym_code_functions(void) +{ + size_t size = (unsigned long)__sym_code_functions_end - + (unsigned long)__sym_code_functions_start; + + sym_code_functions = (struct code_range *)__sym_code_functions_start; + /* + * Order it so that sym_code_functions is not visible before + * num_sym_code_functions. + */ + smp_mb(); + num_sym_code_functions = size / sizeof(struct code_range); + + return 0; +} +early_initcall(init_sym_code_functions); + /* * Check the stack frame for conditions that make further unwinding unreliable. */ static void unwind_check_reliability(struct task_struct *task, struct stackframe *frame) { + const struct code_range *range; + unsigned long pc; + int i; + if (frame->fp == (unsigned long)task_pt_regs(task)->stackframe) { /* Final frame; no more unwind, no need to check reliability */ return; @@ -36,6 +65,32 @@ static void unwind_check_reliability(struct task_struct *task, */ if (!__kernel_text_address(frame->pc)) frame->reliable = false; + + /* + * Check the return PC against sym_code_functions[]. If there is a + * match, then the consider the stack frame unreliable. + * + * As SYM_CODE functions don't follow the usual calling conventions, + * we assume by default that any SYM_CODE function cannot be unwound + * reliably. + * + * Note that this includes: + * + * - Exception handlers and entry assembly + * - Trampoline assembly (e.g., ftrace, kprobes) + * - Hypervisor-related assembly + * - Hibernation-related assembly + * - CPU start-stop, suspend-resume assembly + * - Kernel relocation assembly + */ + pc = frame->pc; + for (i = 0; i < num_sym_code_functions; i++) { + range = &sym_code_functions[i]; + if (pc >= range->start && pc < range->end) { + frame->reliable = false; + return; + } + } } /* diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 50bab186c49b..6381e75e566e 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -122,6 +122,14 @@ jiffies = jiffies_64; #define TRAMP_TEXT #endif +#define SYM_CODE_FUNCTIONS \ + . = ALIGN(16); \ + .symcode : AT(ADDR(.symcode) - LOAD_OFFSET) { \ + __sym_code_functions_start = .; \ + KEEP(*(sym_code_functions)) \ + __sym_code_functions_end = .; \ + } + /* * The size of the PE/COFF section that covers the kernel image, which * runs from _stext to _edata, must be a round multiple of the PE/COFF @@ -209,6 +217,8 @@ SECTIONS swapper_pg_dir = .; . += PAGE_SIZE; + SYM_CODE_FUNCTIONS + . = ALIGN(SEGMENT_ALIGN); __init_begin = .; __inittext_begin = .;