From patchwork Wed Mar 23 15:32:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12789844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB4B9C433EF for ; Wed, 23 Mar 2022 15:33:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A51EF6B0078; Wed, 23 Mar 2022 11:33:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D7D66B007B; Wed, 23 Mar 2022 11:33:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78E9E6B007D; Wed, 23 Mar 2022 11:33:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0046.hostedemail.com [216.40.44.46]) by kanga.kvack.org (Postfix) with ESMTP id 5E9E26B0078 for ; Wed, 23 Mar 2022 11:33:06 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 272C11828DEBB for ; Wed, 23 Mar 2022 15:33:06 +0000 (UTC) X-FDA: 79276044372.20.B72D0D0 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by imf07.hostedemail.com (Postfix) with ESMTP id 5C7AC4003F for ; Wed, 23 Mar 2022 15:33:05 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1648049584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MP8VVStpf2u5pFq9LJYVTfW5maMxRrDNwrS4C/iOhFE=; b=CHABMq/eZMH3CKmnepGiem7lAUyvtGITme8Qll4LQCROHN+V6C06OVGLwZuteRZ+PbizzX B/d0tsbkAj03Q2KNrb7PC4vE3rYeOl8qckRDCHswffvIEUY+QTm6icfg6nxgM6fpgtv52V PXKw6cD+WU2EotoizAcwfEiGoH3Y8cA= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Catalin Marinas , Will Deacon , Andrew Morton Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Mark Rutland , Vincenzo Frascino , Sami Tolvanen , Peter Collingbourne , Evgenii Stepanov , Florian Mayer , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 3/4] arm64: implement stack_trace_save_shadow Date: Wed, 23 Mar 2022 16:32:54 +0100 Message-Id: <0bb72ea8fa88ef9ae3508c23d993952a0ae6f0f9.1648049113.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-Stat-Signature: rjwyzh8yq8f1ebrdh7izupc9dsr97kq6 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="CHABMq/e"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf07.hostedemail.com: domain of andrey.konovalov@linux.dev designates 94.23.1.103 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 5C7AC4003F X-HE-Tag: 1648049585-483019 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Implement the stack_trace_save_shadow() interface that collects stack traces based on the Shadow Call Stack (SCS) for arm64. The implementation walks through available SCS pointers (the per-task one and the per-interrupt-type ones) and copies the frames. Note that the frame of the interrupted function is not included into the stack trace, as it is not yet saved on the SCS when an interrupt happens. Signed-off-by: Andrey Konovalov Reported-by: kernel test robot --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/stacktrace.c | 83 ++++++++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a659e238f196..d89cecf6c923 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -201,6 +201,7 @@ config ARM64 select MMU_GATHER_RCU_TABLE_FREE select HAVE_RSEQ select HAVE_RUST + select HAVE_SHADOW_STACKTRACE select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index e4103e085681..89daa710d91b 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -12,9 +12,11 @@ #include #include #include +#include #include #include +#include #include #include @@ -210,3 +212,84 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, walk_stackframe(task, &frame, consume_entry, cookie); } + +static const struct { + unsigned long ** __percpu saved; + unsigned long ** __percpu base; +} scs_parts[] = { +#ifdef CONFIG_ARM_SDE_INTERFACE + { + .saved = &sdei_shadow_call_stack_critical_saved_ptr, + .base = &sdei_shadow_call_stack_critical_ptr, + }, + { + .saved = &sdei_shadow_call_stack_normal_saved_ptr, + .base = &sdei_shadow_call_stack_normal_ptr, + }, +#endif /* CONFIG_ARM_SDE_INTERFACE */ + { + .saved = &irq_shadow_call_stack_saved_ptr, + .base = &irq_shadow_call_stack_ptr, + }, +}; + +static inline bool walk_shadow_stack_part( + unsigned long *scs_top, unsigned long *scs_base, + unsigned long *store, unsigned int size, + unsigned int *skipnr, unsigned int *len) +{ + unsigned long *frame; + + for (frame = scs_top; frame >= scs_base; frame--) { + if (*skipnr > 0) { + (*skipnr)--; + continue; + } + /* + * Do not leak PTR_AUTH tags in stack traces. + * Use READ_ONCE_NOCHECK as SCS is poisoned with Generic KASAN. + */ + store[(*len)++] = + ptrauth_strip_insn_pac(READ_ONCE_NOCHECK(*frame)); + if (*len >= size) + return true; + } + + return false; +} + +noinline notrace int arch_stack_walk_shadow(unsigned long *store, + unsigned int size, + unsigned int skipnr) +{ + unsigned long *scs_top, *scs_base, *scs_next; + unsigned int len = 0, part; + + preempt_disable(); + + /* Get the SCS pointer. */ + asm volatile("mov %0, x18" : "=&r" (scs_top)); + + /* The top SCS slot is empty. */ + scs_top -= 1; + + /* Handle SDEI and hardirq frames. */ + for (part = 0; part < ARRAY_SIZE(scs_parts); part++) { + scs_next = *this_cpu_ptr(scs_parts[part].saved); + if (scs_next) { + scs_base = *this_cpu_ptr(scs_parts[part].base); + if (walk_shadow_stack_part(scs_top, scs_base, store, + size, &skipnr, &len)) + goto out; + scs_top = scs_next; + } + } + + /* Handle task and softirq frames. */ + scs_base = task_scs(current); + walk_shadow_stack_part(scs_top, scs_base, store, size, &skipnr, &len); + +out: + preempt_enable(); + return len; +}