From patchwork Wed May 8 21:26:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13659168 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB09612BF23; Wed, 8 May 2024 21:26:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203575; cv=none; b=OgR/6ZHgtBBgLB+vGnP+bMvPHuR2Y0rZ1vmQmrCJdnQR5XnAtpnlNR47QFnx8DL0kz/KJAU6Ato6/ESky1sPJ61Xa9/bvurnl3ouyN32C8vK8n/j1VAhpFrEuehBDlXWvssvyv8OUIgPLcoyhdci0lsOaX5fsvg9+wnjzwS3T6c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203575; c=relaxed/simple; bh=PdIMsdaUCiSs4TuXIKskUpSJzvRucEq5dFDzB/EcSIQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tanMLuMTEcp6zibjXv46vuv3lgT6ciqlXm2TYgoXgRu0BMnf8eUxgzPOFwh4+VYS4mudx8XtGAtmNKgx6WjH6b7Vj1jpnqR6JviWtWgz7uEO0mpbBnN1ON4zF6vPmBDgY8DgGn/zXwvjkU6F2vN/F1b1KzzGRX/5XgBxioNpHz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Lu/stqCp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Lu/stqCp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3CF85C32781; Wed, 8 May 2024 21:26:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715203574; bh=PdIMsdaUCiSs4TuXIKskUpSJzvRucEq5dFDzB/EcSIQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Lu/stqCpGqXbJCbwBFZy/OhMMRDhP17FPPje8d87tprBaEoaifFfQ2HqPLhAl7V9W 6VB0IpbY1wVitIp2XrhAay3JJJId2Kpb7QrTb0mCnqaVsn76s8vqBjpW15DVJMrhrd a14WBvvjkDdYLnmRJ4aILUj/szfYXudWBplpb+q9Mtv2Q+JqUnRMb6YJ9cn9OfGDs6 jnHTHijCRNGTpXYG3uCnuTphVw679PZzD4jwaoBftl37PGipPEip2vsJ5eJBtdWLe7 gR0+6y7dU0NKpC6sBJA6EKC3y1rOat+EBVH+FP3H7E8plRE6lxfwEsxFZg4ITYeBd7 wZtSkwUruTuZg== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko Subject: [PATCH 1/4] uprobes: rename get_trampoline_vaddr() and make it global Date: Wed, 8 May 2024 14:26:02 -0700 Message-ID: <20240508212605.4012172-2-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240508212605.4012172-1-andrii@kernel.org> References: <20240508212605.4012172-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This helper is needed in another file, so make it a bit more uniquely named and expose it internally. Signed-off-by: Andrii Nakryiko --- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index f46e0ca0169c..0c57eec85339 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -138,6 +138,7 @@ extern bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check c extern bool arch_uprobe_ignore(struct arch_uprobe *aup, struct pt_regs *regs); extern void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, void *src, unsigned long len); +extern unsigned long uprobe_get_trampoline_vaddr(void); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 8ae0eefc3a34..d60d24f0f2f4 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1827,7 +1827,7 @@ void uprobe_copy_process(struct task_struct *t, unsigned long flags) * * Returns -1 in case the xol_area is not allocated. */ -static unsigned long get_trampoline_vaddr(void) +unsigned long uprobe_get_trampoline_vaddr(void) { struct xol_area *area; unsigned long trampoline_vaddr = -1; @@ -1878,7 +1878,7 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) if (!ri) return; - trampoline_vaddr = get_trampoline_vaddr(); + trampoline_vaddr = uprobe_get_trampoline_vaddr(); orig_ret_vaddr = arch_uretprobe_hijack_return_addr(trampoline_vaddr, regs); if (orig_ret_vaddr == -1) goto fail; @@ -2187,7 +2187,7 @@ static void handle_swbp(struct pt_regs *regs) int is_swbp; bp_vaddr = uprobe_get_swbp_addr(regs); - if (bp_vaddr == get_trampoline_vaddr()) + if (bp_vaddr == uprobe_get_trampoline_vaddr()) return handle_trampoline(regs); uprobe = find_active_uprobe(bp_vaddr, &is_swbp); From patchwork Wed May 8 21:26:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13659169 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 230C313248E; Wed, 8 May 2024 21:26:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203578; cv=none; b=KMSfhf0YQuNStjoQK2JLiSrVxWfeuYpHfJIvOWXTwYbf6hVNLMXufqSNmN3u56t/sk5HvylE5d5XDxKCi4+GLMZpruMgak9SrjnbvsGCxWtrmv5rYf4Dx5NRTkKsN0j95jZH0uC6cIz7wKVnTRh8MA3cYXBqHQq2elizfQlTZr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203578; c=relaxed/simple; bh=pRmppip20JV7W3kP/rLg2NLJC295oDyg0/sigGT7o9A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tOmBALKYGgnf4WqtWOOIXu1ko2VeiWYG9MtGdXdF6THsbBvWeRaqnM1MhbRVo0x3DEsBhor8En87YuXOJdA2Rh939niALtyxatDyCMxPOXPhyoK4YIMSPTbCSYjrUXUQn/EGA7FS7WlYEyD68Wh68aI09GqWYZCrAh40pggGFyg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZjzVVlwa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZjzVVlwa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81E1AC113CC; Wed, 8 May 2024 21:26:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715203577; bh=pRmppip20JV7W3kP/rLg2NLJC295oDyg0/sigGT7o9A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZjzVVlwaPSpEzfCSRZDP8dQ4ctyv5p56PJ3ad+qZkF619T7Iq7rU1aiqv+nfItbRR djMyS9hvZEXcmV5mxggvSNFKWnhffRXUGQF2n/Kgdfub8jTRu8+ZsNe3cNJ1pTP5iq 7i9yVYMhRZ7Q99fD6/kOrtvj7ndj6WFJ53IsZHvyTmdnvVltKFtYT9OnoZCl0l1RE5 4KSW8ODG4mRnop8NR9SVD4Do7Z3zIVGbkrrDf3P3D8xDVCPPoMWx3VOJhhV3GBfx5K VcHNcgudelNr62c6vRVisINadlJmwzneJ1axUNDD8GbxAn2fVYAkZuVB5wIZjfDVyF KL52qiBCGcejQ== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko , Riham Selim Subject: [PATCH 2/4] perf,uprobes: fix user stack traces in the presence of pending uretprobes Date: Wed, 8 May 2024 14:26:03 -0700 Message-ID: <20240508212605.4012172-3-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240508212605.4012172-1-andrii@kernel.org> References: <20240508212605.4012172-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When kernel has pending uretprobes installed, it hijacks original user function return address on the stack with a uretprobe trampoline address. There could be multiple such pending uretprobes (either on different user functions or on the same recursive one) at any given time within the same task. This approach interferes with the user stack trace capture logic, which would report suprising addresses (like 0x7fffffffe000) that correspond to a special "[uprobes]" section that kernel installs in the target process address space for uretprobe trampoline code, while logically it should be an address somewhere within the calling function of another traced user function. This is easy to correct for, though. Uprobes subsystem keeps track of pending uretprobes and records original return addresses. This patch is using this to do a post-processing step and restore each trampoline address entries with correct original return address. This is done only if there are pending uretprobes for current task. Reported-by: Riham Selim Signed-off-by: Andrii Nakryiko --- kernel/events/callchain.c | 42 ++++++++++++++++++++++++++++++++++++++- kernel/events/uprobes.c | 9 +++++++++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c index 1273be84392c..2f7ceca7ae3f 100644 --- a/kernel/events/callchain.c +++ b/kernel/events/callchain.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "internal.h" @@ -176,13 +177,50 @@ put_callchain_entry(int rctx) put_recursion_context(this_cpu_ptr(callchain_recursion), rctx); } +static void fixup_uretprobe_trampoline_entries(struct perf_callchain_entry *entry, + int start_entry_idx) +{ +#ifdef CONFIG_UPROBES + struct uprobe_task *utask = current->utask; + struct return_instance *ri; + __u64 *cur_ip, *last_ip, tramp_addr; + + if (likely(!utask || !utask->return_instances)) + return; + + cur_ip = &entry->ip[start_entry_idx]; + last_ip = &entry->ip[entry->nr - 1]; + ri = utask->return_instances; + tramp_addr = uprobe_get_trampoline_vaddr(); + + /* If there are pending uretprobes for current thread, they are + * recorded in a list inside utask->return_instances; each such + * pending uretprobe replaces traced user function's return address on + * the stack, so when stack trace is captured, instead of seeing + * actual function's return address, we'll have one or many uretprobe + * trampoline addresses in the stack trace, which are not helpful and + * misleading to users. + * So here we go over the pending list of uretprobes, and each + * encountered trampoline address is replaced with actual return + * address. + */ + while (ri && cur_ip <= last_ip) { + if (*cur_ip == tramp_addr) { + *cur_ip = ri->orig_ret_vaddr; + ri = ri->next; + } + cur_ip++; + } +#endif +} + struct perf_callchain_entry * get_perf_callchain(struct pt_regs *regs, u32 init_nr, bool kernel, bool user, u32 max_stack, bool crosstask, bool add_mark) { struct perf_callchain_entry *entry; struct perf_callchain_entry_ctx ctx; - int rctx; + int rctx, start_entry_idx; entry = get_callchain_entry(&rctx); if (!entry) @@ -215,7 +253,9 @@ get_perf_callchain(struct pt_regs *regs, u32 init_nr, bool kernel, bool user, if (add_mark) perf_callchain_store_context(&ctx, PERF_CONTEXT_USER); + start_entry_idx = entry->nr; perf_callchain_user(&ctx, regs); + fixup_uretprobe_trampoline_entries(entry, start_entry_idx); } } diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index d60d24f0f2f4..1c99380dc89d 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2149,6 +2149,15 @@ static void handle_trampoline(struct pt_regs *regs) instruction_pointer_set(regs, ri->orig_ret_vaddr); do { + /* pop current instance from the stack of pending return instances, + * as it's not pending anymore: we just fixed up original + * instruction pointer in regs and are about to call handlers; + * this allows fixup_uretprobe_trampoline_entries() to properly fix up + * captured stack traces from uretprobe handlers, in which pending + * trampoline addresses on the stack are replaced with correct + * original return addresses + */ + utask->return_instances = ri->next; if (valid) handle_uretprobe_chain(ri, regs); ri = free_ret_instance(ri); From patchwork Wed May 8 21:26:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13659170 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5849953381; Wed, 8 May 2024 21:26:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203581; cv=none; b=MqVSaG6Z/bJfw1RSyJ6a25HdJchgu8gYaVnhjISu4G4Y9xbYFdvJkM7dPln5By7ZDjAJogGz5FFgD9v6QYlUdbhTzwTlV6xG+eBlS069bSCPGKccmNt7bgfNcgI4ZryOSzXMZ+ZA3hKmL+88mq2ZEfGGdMaIUbTV4c57uQeJgTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203581; c=relaxed/simple; bh=IcCH6HvV+fA+Q/OECgPQYvioKi/QPVoGcj2BlQkwI7Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Id5lakDuorL/Xy3l5Lir9CO/EMR99PbJpMI2DK2vSZsZxU45RFVrbu3fW4KJP3OJgILg5zAR/cV8nyCU/8OxvxvGijODL5el0ZMFuNiDzmlQgQkli8ygvkRV9h6diRrfq6LrUgZFYgX+2tmKdrL1DVXAb734UdPJZWqIPcJpTXU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sw8hzS/a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sw8hzS/a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B82DDC113CC; Wed, 8 May 2024 21:26:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715203580; bh=IcCH6HvV+fA+Q/OECgPQYvioKi/QPVoGcj2BlQkwI7Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sw8hzS/adU8d5M49o6DylDQlNAy9IE+DtQDm5jWqYvG7O4mMwdyZz4EKdKi9E93zR Lw7owf95gWNLBHPXw/zf57BpK1qRNiUuke0Ce/mtm8TdbtJ8c1OTD420XGyiW0QgQS +g+/MH2MuIjvoec2TAnxliVuiuZoroh+lDTf+DxNdvU6wC4MmHTEXhBLYjJkFIX4F1 Ez6B6tGcijvmG/c8nJwCzrDYE452FU42AoHB8RNfhwOOgxPgx6TKy7O1rbDPhz8c3M t2VQYosBxFJGJzV836P1KVzPLFE0jZm3tPS7JEE+EnEFixk4czjCrj4XSs1YreuAd8 MNiD3rZsRalsA== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko Subject: [PATCH 3/4] perf,x86: avoid missing caller address in stack traces captured in uprobe Date: Wed, 8 May 2024 14:26:04 -0700 Message-ID: <20240508212605.4012172-4-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240508212605.4012172-1-andrii@kernel.org> References: <20240508212605.4012172-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When tracing user functions with uprobe functionality, it's common to install the probe (e.g., a BPF program) at the first instruction of the function. This is often going to be `push %rbp` instruction in function preamble, which means that within that function frame pointer hasn't been established yet. This leads to consistently missing an actual caller of the traced function, because perf_callchain_user() only records current IP (capturing traced function) and then following frame pointer chain (which would be caller's frame, containing the address of caller's caller). So when we have target_1 -> target_2 -> target_3 call chain and we are tracing an entry to target_3, captured stack trace will report target_1 -> target_3 call chain, which is wrong and confusing. This patch proposes a x86-64-specific heuristic to detect `push %rbp` instruction being traced. If that's the case, with the assumption that applicatoin is compiled with frame pointers, this instruction would be a strong indicator that this is the entry to the function. In that case, return address is still pointed to by %rsp, so we fetch it and add to stack trace before proceeding to unwind the rest using frame pointer-based logic. Signed-off-by: Andrii Nakryiko --- arch/x86/events/core.c | 20 ++++++++++++++++++++ include/linux/uprobes.h | 2 ++ kernel/events/uprobes.c | 2 ++ 3 files changed, 24 insertions(+) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 5b0dd07b1ef1..82d5570b58ff 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2884,6 +2884,26 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs return; pagefault_disable(); + +#ifdef CONFIG_UPROBES + /* + * If we are called from uprobe handler, and we are indeed at the very + * entry to user function (which is normally a `push %rbp` instruction, + * under assumption of application being compiled with frame pointers), + * we should read return address from *regs->sp before proceeding + * to follow frame pointers, otherwise we'll skip immediate caller + * as %rbp is not yet setup. + */ + if (current->utask) { + struct arch_uprobe *auprobe = current->utask->auprobe; + u64 ret_addr; + + if (auprobe && auprobe->insn[0] == 0x55 /* push %rbp */ && + !__get_user(ret_addr, (const u64 __user *)regs->sp)) + perf_callchain_store(entry, ret_addr); + } +#endif + while (entry->nr < entry->max_stack) { if (!valid_user_frame(fp, sizeof(frame))) break; diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 0c57eec85339..7b785cd30d86 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -76,6 +76,8 @@ struct uprobe_task { struct uprobe *active_uprobe; unsigned long xol_vaddr; + struct arch_uprobe *auprobe; + struct return_instance *return_instances; unsigned int depth; }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 1c99380dc89d..504693845187 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2072,6 +2072,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) bool need_prep = false; /* prepare return uprobe, when needed */ down_read(&uprobe->register_rwsem); + current->utask->auprobe = &uprobe->arch; for (uc = uprobe->consumers; uc; uc = uc->next) { int rc = 0; @@ -2086,6 +2087,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) remove &= rc; } + current->utask->auprobe = NULL; if (need_prep && !remove) prepare_uretprobe(uprobe, regs); /* put bp at return */ From patchwork Wed May 8 21:26:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13659171 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 992FB130AF6; Wed, 8 May 2024 21:26:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203585; cv=none; b=Uux7JpscFQJo7EkMxhyepmS7VKwS0KV3E/nMaQxKp2xGDk7lRHGuGWnTPxv3nOO2ygaKoySYWRAvmQL8rDHRQ/ItUYbBBzdWkMi7Zjrm1D7yOriLWem6Vfdeg80N6Ua9QmK80L+1sKqCmm78l4WRHBuxbKOUGh9Nz/0kEd5Cx0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715203585; c=relaxed/simple; bh=ki1+VYdxFSWKI78FFe8w0VMSWz/AqYegQbCtjt/2njc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ir5qfqYnBLj5f1BjoJP2Mas6O3wtvx+m6I03dfNAbRGrJ8bAHOGnH0KQt/wPoBlTDaX58L0MgzccDitRi7IfyLo9308iR/F2BQSsXqIbJcgx1PveQZNiXKo/NaEAcNXPAUCAPB7BTHD2ZqTV2USLdtonJk3Sny5QsNsebe7ms/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J1nXPEYo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J1nXPEYo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC832C113CC; Wed, 8 May 2024 21:26:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715203584; bh=ki1+VYdxFSWKI78FFe8w0VMSWz/AqYegQbCtjt/2njc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J1nXPEYo6SPyyUYQ3w3OfVaSsl1DgT1YBOzH/wEaTT3F6QEVGnKf+/SWi3SmLubTW nNIupLW+GaufeeeLq4owgE3bKg0hvoYGgt25rRU7t0O2hE5WcYn3ulKv/Z3i2vYr2d ttfHhwlW83T48m9oqpEqFjOYcq3utH7DmFckKMAcEG2YyWDvJvlPotX+nm7SjirPta bHX+PlD/vB/tKe6pAA1YzZ0e3ZMikyAlSW7Lzk98u7XE5PAuaeb5vygXddRRppY8Af +jdQGTEevv0hJFGqzG6cwGjrYsCgpUIL+fH4sjae3aOL5CearHy0FkG9yzRZ1i9mhI D2ymSdAgYZyCg== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko Subject: [PATCH 4/4] selftests/bpf: add test validating uprobe/uretprobe stack traces Date: Wed, 8 May 2024 14:26:05 -0700 Message-ID: <20240508212605.4012172-5-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240508212605.4012172-1-andrii@kernel.org> References: <20240508212605.4012172-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add a set of tests to validate that stack traces captured from or in the presence of active uprobes and uretprobes are valid and complete. For this we use BPF program that are installed either on entry or exit of user function, plus deep-nested USDT. One of target funtions (target_1) is recursive to generate two different entries in the stack trace for the same uprobe/uretprobe, testing potential edge conditions. Without fixes in this patch set, we get something like this for one of the scenarios: caller: 0x758fff - 0x7595ab target_1: 0x758fd5 - 0x758fff target_2: 0x758fca - 0x758fd5 target_3: 0x758fbf - 0x758fca target_4: 0x758fb3 - 0x758fbf ENTRY #0: 0x758fb3 (in target_4) ENTRY #1: 0x758fd3 (in target_2) ENTRY #2: 0x758ffd (in target_1) ENTRY #3: 0x7fffffffe000 ENTRY #4: 0x7fffffffe000 ENTRY #5: 0x6f8f39 ENTRY #6: 0x6fa6f0 ENTRY #7: 0x7f403f229590 Entry #3 and #4 (0x7fffffffe000) are uretprobe trampoline addresses which obscure actual target_1 and another target_1 invocations. Also note that between entry #0 and entry #1 we are missing an entry for target_3, which is fixed in patch #2. With all the fixes, we get desired full stack traces: caller: 0x758fff - 0x7595ab target_1: 0x758fd5 - 0x758fff target_2: 0x758fca - 0x758fd5 target_3: 0x758fbf - 0x758fca target_4: 0x758fb3 - 0x758fbf ENTRY #0: 0x758fb7 (in target_4) ENTRY #1: 0x758fc8 (in target_3) ENTRY #2: 0x758fd3 (in target_2) ENTRY #3: 0x758ffd (in target_1) ENTRY #4: 0x758ff3 (in target_1) ENTRY #5: 0x75922c (in caller) ENTRY #6: 0x6f8f39 ENTRY #7: 0x6fa6f0 ENTRY #8: 0x7f986adc4cd0 Now there is a logical and complete sequence of function calls. Signed-off-by: Andrii Nakryiko --- .../bpf/prog_tests/uretprobe_stack.c | 185 ++++++++++++++++++ .../selftests/bpf/progs/uretprobe_stack.c | 96 +++++++++ 2 files changed, 281 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c create mode 100644 tools/testing/selftests/bpf/progs/uretprobe_stack.c diff --git a/tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c b/tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c new file mode 100644 index 000000000000..2974553da6e5 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include +#include "uretprobe_stack.skel.h" +#include "../sdt.h" + +/* We set up target_1() -> target_2() -> target_3() -> target_4() -> USDT() + * call chain, each being traced by our BPF program. On entry or return from + * each target_*() we are capturing user stack trace and recording it in + * global variable, so that user space part of the test can validate it. + * + * Note, we put each target function into a custom section to get those + * __start_XXX/__stop_XXX symbols, generated by linker for us, which allow us + * to know address range of those functions + */ +__attribute__((section("uprobe__target_4"))) +__weak int target_4(void) +{ + STAP_PROBE1(uretprobe_stack, target, 42); + return 42; +} + +extern const void *__start_uprobe__target_4; +extern const void *__stop_uprobe__target_4; + +__attribute__((section("uprobe__target_3"))) +__weak int target_3(void) +{ + return target_4(); +} + +extern const void *__start_uprobe__target_3; +extern const void *__stop_uprobe__target_3; + +__attribute__((section("uprobe__target_2"))) +__weak int target_2(void) +{ + return target_3(); +} + +extern const void *__start_uprobe__target_2; +extern const void *__stop_uprobe__target_2; + +__attribute__((section("uprobe__target_1"))) +__weak int target_1(int depth) +{ + if (depth < 1) + return 1 + target_1(depth + 1); + else + return target_2(); +} + +extern const void *__start_uprobe__target_1; +extern const void *__stop_uprobe__target_1; + +extern const void *__start_uretprobe_stack_sec; +extern const void *__stop_uretprobe_stack_sec; + +struct range { + long start; + long stop; +}; + +static struct range targets[] = { + {}, /* we want target_1 to map to target[1], so need 1-based indexing */ + { (long)&__start_uprobe__target_1, (long)&__stop_uprobe__target_1 }, + { (long)&__start_uprobe__target_2, (long)&__stop_uprobe__target_2 }, + { (long)&__start_uprobe__target_3, (long)&__stop_uprobe__target_3 }, + { (long)&__start_uprobe__target_4, (long)&__stop_uprobe__target_4 }, +}; + +static struct range caller = { + (long)&__start_uretprobe_stack_sec, + (long)&__stop_uretprobe_stack_sec, +}; + +static void validate_stack(__u64 *ips, int stack_len, int cnt, ...) +{ + int i, j; + va_list args; + + if (!ASSERT_GT(stack_len, 0, "stack_len")) + return; + + stack_len /= 8; + + /* check if we have enough entries to satisfy test expectations */ + if (!ASSERT_GE(stack_len, cnt, "stack_len2")) + return; + + if (env.verbosity >= VERBOSE_NORMAL) { + printf("caller: %#lx - %#lx\n", caller.start, caller.stop); + for (i = 1; i < ARRAY_SIZE(targets); i++) + printf("target_%d: %#lx - %#lx\n", i, targets[i].start, targets[i].stop); + for (i = 0; i < stack_len; i++) { + for (j = 1; j < ARRAY_SIZE(targets); j++) { + if (ips[i] >= targets[j].start && ips[i] < targets[j].stop) + break; + } + if (j < ARRAY_SIZE(targets)) { /* found target match */ + printf("ENTRY #%d: %#lx (in target_%d)\n", i, (long)ips[i], j); + } else if (ips[i] >= caller.start && ips[i] < caller.stop) { + printf("ENTRY #%d: %#lx (in caller)\n", i, (long)ips[i]); + } else { + printf("ENTRY #%d: %#lx\n", i, (long)ips[i]); + } + } + } + + va_start(args, cnt); + + for (i = cnt - 1; i >= 0; i--) { + /* most recent entry is the deepest target function */ + const struct range *t = va_arg(args, const struct range *); + + ASSERT_GE(ips[i], t->start, "addr_start"); + ASSERT_LT(ips[i], t->stop, "addr_stop"); + } + + va_end(args); +} + +__attribute__((section("uretprobe_stack_sec"))) +void test_uretprobe_stack(void) +{ + LIBBPF_OPTS(bpf_uprobe_opts, uprobe_opts); + struct uretprobe_stack *skel; + int err; + + skel = uretprobe_stack__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + err = uretprobe_stack__attach(skel); + if (!ASSERT_OK(err, "skel_attach")) + goto cleanup; + + /* trigger */ + ASSERT_EQ(target_1(0), 42 + 1, "trigger_return"); + + /* + * Stacks captured on ENTRY uprobes + */ + + /* (uprobe 1) target_1 in stack trace*/ + validate_stack(skel->bss->entry_stack1, skel->bss->entry1_len, + 2, &caller, &targets[1]); + /* (uprobe 1, recursed) */ + validate_stack(skel->bss->entry_stack1_recur, skel->bss->entry1_recur_len, + 3, &caller, &targets[1], &targets[1]); + /* (uprobe 2) caller -> target_1 -> target_1 -> target_2 */ + validate_stack(skel->bss->entry_stack2, skel->bss->entry2_len, + 4, &caller, &targets[1], &targets[1], &targets[2]); + /* (uprobe 3) */ + validate_stack(skel->bss->entry_stack3, skel->bss->entry3_len, + 5, &caller, &targets[1], &targets[1], &targets[2], &targets[3]); + /* (uprobe 4) caller -> target_1 -> target_1 -> target_2 -> target_3 -> target_4 */ + validate_stack(skel->bss->entry_stack4, skel->bss->entry4_len, + 6, &caller, &targets[1], &targets[1], &targets[2], &targets[3], &targets[4]); + + /* (USDT): full caller -> target_1 -> target_1 -> target_2 (uretprobed) + * -> target_3 -> target_4 (uretprobes) chain + */ + validate_stack(skel->bss->usdt_stack, skel->bss->usdt_len, + 6, &caller, &targets[1], &targets[1], &targets[2], &targets[3], &targets[4]); + + /* + * Now stacks captured on the way out in EXIT uprobes + */ + + /* (uretprobe 4) everything up to target_4, but excluding it */ + validate_stack(skel->bss->exit_stack4, skel->bss->exit4_len, + 5, &caller, &targets[1], &targets[1], &targets[2], &targets[3]); + /* we didn't install uretprobes on target_2 and target_3 */ + /* (uretprobe 1, recur) first target_1 call only */ + validate_stack(skel->bss->exit_stack1_recur, skel->bss->exit1_recur_len, + 2, &caller, &targets[1]); + /* (uretprobe 1) just a caller in the stack trace */ + validate_stack(skel->bss->exit_stack1, skel->bss->exit1_len, + 1, &caller); + +cleanup: + uretprobe_stack__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/uretprobe_stack.c b/tools/testing/selftests/bpf/progs/uretprobe_stack.c new file mode 100644 index 000000000000..9fdcf396b8f4 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uretprobe_stack.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__u64 entry_stack1[32], exit_stack1[32]; +__u64 entry_stack1_recur[32], exit_stack1_recur[32]; +__u64 entry_stack2[32]; +__u64 entry_stack3[32]; +__u64 entry_stack4[32], exit_stack4[32]; +__u64 usdt_stack[32]; + +int entry1_len, exit1_len; +int entry1_recur_len, exit1_recur_len; +int entry2_len, exit2_len; +int entry3_len, exit3_len; +int entry4_len, exit4_len; +int usdt_len; + +#define SZ sizeof(usdt_stack) + +SEC("uprobe//proc/self/exe:target_1") +int BPF_UPROBE(uprobe_1) +{ + /* target_1 is recursive wit depth of 2, so we capture two separate + * stack traces, depending on which occurence it is + */ + static bool recur = false; + + if (!recur) + entry1_len = bpf_get_stack(ctx, &entry_stack1, SZ, BPF_F_USER_STACK); + else + entry1_recur_len = bpf_get_stack(ctx, &entry_stack1_recur, SZ, BPF_F_USER_STACK); + + recur = true; + return 0; +} + +SEC("uretprobe//proc/self/exe:target_1") +int BPF_URETPROBE(uretprobe_1) +{ + /* see above, target_1 is recursive */ + static bool recur = false; + + /* NOTE: order of returns is reversed to order of entries */ + if (!recur) + exit1_recur_len = bpf_get_stack(ctx, &exit_stack1_recur, SZ, BPF_F_USER_STACK); + else + exit1_len = bpf_get_stack(ctx, &exit_stack1, SZ, BPF_F_USER_STACK); + + recur = true; + return 0; +} + +SEC("uprobe//proc/self/exe:target_2") +int BPF_UPROBE(uprobe_2) +{ + entry2_len = bpf_get_stack(ctx, &entry_stack2, SZ, BPF_F_USER_STACK); + return 0; +} + +/* no uretprobe for target_2 */ + +SEC("uprobe//proc/self/exe:target_3") +int BPF_UPROBE(uprobe_3) +{ + entry3_len = bpf_get_stack(ctx, &entry_stack3, SZ, BPF_F_USER_STACK); + return 0; +} + +/* no uretprobe for target_3 */ + +SEC("uprobe//proc/self/exe:target_4") +int BPF_UPROBE(uprobe_4) +{ + entry4_len = bpf_get_stack(ctx, &entry_stack4, SZ, BPF_F_USER_STACK); + return 0; +} + +SEC("uretprobe//proc/self/exe:target_4") +int BPF_URETPROBE(uretprobe_4) +{ + exit4_len = bpf_get_stack(ctx, &exit_stack4, SZ, BPF_F_USER_STACK); + return 0; +} + +SEC("usdt//proc/self/exe:uretprobe_stack:target") +int BPF_USDT(usdt_probe) +{ + usdt_len = bpf_get_stack(ctx, &usdt_stack, SZ, BPF_F_USER_STACK); + return 0; +}