From patchwork Wed May 22 01:38:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13670139 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94BAA4C84; Wed, 22 May 2024 01:38:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341931; cv=none; b=VD5J4tIM5xz/sydcMPJEXEDEOkKCwNQaIKJ3gmQ9lqo9EHOSgTqedQ8CkxTxZDkf1j2+yNdqtAkyK43ExTsgiVk+6GzVSbqbRkem/sjYo14YtOjR/RVtQ/xQwK3Z53nmKzZT7rLuYhiUsDx+0dxjcxvugUR3VeCrAmo85/n+FbU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341931; c=relaxed/simple; bh=PdIMsdaUCiSs4TuXIKskUpSJzvRucEq5dFDzB/EcSIQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WZU4YtCXYKQPsejzsXs/MrMBDcdAnOcpN1CJxjiDVlWrIssC6eLP2gu1sBoaTEhZO9a0qErmO+aHWr7Cxcv6Op2dTMPTbGSvufZzD/8KTqz6/s2xk1tqLex+idgr1VILOFaqbbC0g8Y7t/kG7xcxf/AvSi2RLh+svcoYfGJZA6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bm5e8pAC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bm5e8pAC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3FE63C2BD11; Wed, 22 May 2024 01:38:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716341931; bh=PdIMsdaUCiSs4TuXIKskUpSJzvRucEq5dFDzB/EcSIQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bm5e8pACVky0OUTCldk0owYBNxh2s02U2g5RkVWWMOSxYt/df/bIrv0dhZA/nDOmh uNEiGT0fdbUHeGjWi4y7pw6wXCmtCwDQqF83c5dgeJu+gwKmuie4pRDYk1b/kkalHI Q8FBXG+yrEjIcvzVasX7V6Mf57h6FY9zhBOCG7VcQXmucRTOJbz/LklynDIe2oJund UT9OmlwsVyVKfacAbI3+WNEonueCqgmBl5U8Irvedr2ziZ9cKyc5fZ+hxlWItzsyNT I6Ag7FOoTaVqp9K5+W1ctz8DjANED9Zvej1nJFVOWfD3NRxqb09PJDbwfS4fQT3D20 Pj/jkAFPTrMNg== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko Subject: [PATCH v2 1/4] uprobes: rename get_trampoline_vaddr() and make it global Date: Tue, 21 May 2024 18:38:42 -0700 Message-ID: <20240522013845.1631305-2-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240522013845.1631305-1-andrii@kernel.org> References: <20240522013845.1631305-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This helper is needed in another file, so make it a bit more uniquely named and expose it internally. Signed-off-by: Andrii Nakryiko Acked-by: Masami Hiramatsu (Google) --- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index f46e0ca0169c..0c57eec85339 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -138,6 +138,7 @@ extern bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check c extern bool arch_uprobe_ignore(struct arch_uprobe *aup, struct pt_regs *regs); extern void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, void *src, unsigned long len); +extern unsigned long uprobe_get_trampoline_vaddr(void); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 8ae0eefc3a34..d60d24f0f2f4 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1827,7 +1827,7 @@ void uprobe_copy_process(struct task_struct *t, unsigned long flags) * * Returns -1 in case the xol_area is not allocated. */ -static unsigned long get_trampoline_vaddr(void) +unsigned long uprobe_get_trampoline_vaddr(void) { struct xol_area *area; unsigned long trampoline_vaddr = -1; @@ -1878,7 +1878,7 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) if (!ri) return; - trampoline_vaddr = get_trampoline_vaddr(); + trampoline_vaddr = uprobe_get_trampoline_vaddr(); orig_ret_vaddr = arch_uretprobe_hijack_return_addr(trampoline_vaddr, regs); if (orig_ret_vaddr == -1) goto fail; @@ -2187,7 +2187,7 @@ static void handle_swbp(struct pt_regs *regs) int is_swbp; bp_vaddr = uprobe_get_swbp_addr(regs); - if (bp_vaddr == get_trampoline_vaddr()) + if (bp_vaddr == uprobe_get_trampoline_vaddr()) return handle_trampoline(regs); uprobe = find_active_uprobe(bp_vaddr, &is_swbp); From patchwork Wed May 22 01:38:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13670140 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DDB579C4; Wed, 22 May 2024 01:38:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341935; cv=none; b=ieDJaSuJ5hCCUJ26L0bU0EjQm/L5lQZW19Rs8gqirLisXX1KOTRsXYiUc7nIrbqca2nBov6nRLjF4z01QgAe9CCPOgTWzXL9YIyv23p+7OytSEhHeI41XSiKduWLh4UD7ODSeQ88snpXToHh0X4ZSoyeuK8N1TOCLBqZuXmDrdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341935; c=relaxed/simple; bh=VrP1X4Gz4MX8XOFsrUPiW3NaGtzmRwltDxE5E1Rzj6k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hcT5VEUwBQ4DrzrXRg2Bl7jw5G3ZCDVJqpx4Aq1znNANmp7Owp9FIhcUJJZtrjQ+A4JjOTE7I52rZ1CJqVBuLowkvax6YUjOBfh9esm+BzNse9udO7CUjtzn0Tu9WZO6Qdw1fIKarthTzzO+hUgd3lQV/GADEsKA9XOI7FA6cFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DQ0UPcnK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DQ0UPcnK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 85F1BC2BD11; Wed, 22 May 2024 01:38:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716341934; bh=VrP1X4Gz4MX8XOFsrUPiW3NaGtzmRwltDxE5E1Rzj6k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DQ0UPcnKifjz85E/M1a+NpC7LhZpHBqX8L6+vSwlTBge217zXkMPmkVVjcAVmOy6q wTl6+V7MkIDMlMgMwnYSjHVj58iNg+DysNrUDopazwnajbuQaEoVCllQeDCbca/n2S 5o2K/LsaNk/MeyOHhdC2CdIR6RJ+k8MXgyLT8OE8cl/ky/OFNV9hSDq44YX6fzS77O ZyE32SjLKWmQDAE8QT6oA+W6bDuWpx06BsQImafLIUpsU4fPiY5EPllDTby7E0zehc CtvwTCZ6abuw4+hI3bU+jJTSnMOUlfv5l1AbHjAy5gPP6Cs4x2HIOYK+Xv7OSDoRoM ohxc5y45RuBNw== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko , Riham Selim Subject: [PATCH v2 2/4] perf,uprobes: fix user stack traces in the presence of pending uretprobes Date: Tue, 21 May 2024 18:38:43 -0700 Message-ID: <20240522013845.1631305-3-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240522013845.1631305-1-andrii@kernel.org> References: <20240522013845.1631305-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When kernel has pending uretprobes installed, it hijacks original user function return address on the stack with a uretprobe trampoline address. There could be multiple such pending uretprobes (either on different user functions or on the same recursive one) at any given time within the same task. This approach interferes with the user stack trace capture logic, which would report suprising addresses (like 0x7fffffffe000) that correspond to a special "[uprobes]" section that kernel installs in the target process address space for uretprobe trampoline code, while logically it should be an address somewhere within the calling function of another traced user function. This is easy to correct for, though. Uprobes subsystem keeps track of pending uretprobes and records original return addresses. This patch is using this to do a post-processing step and restore each trampoline address entries with correct original return address. This is done only if there are pending uretprobes for current task. This is a similar approach to what fprobe/kretprobe infrastructure is doing when capturing kernel stack traces in the presence of pending return probes. Reported-by: Riham Selim Signed-off-by: Andrii Nakryiko Reviewed-by: Masami Hiramatsu (Google) --- kernel/events/callchain.c | 43 ++++++++++++++++++++++++++++++++++++++- kernel/events/uprobes.c | 9 ++++++++ 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c index 1273be84392c..b17e3323f7f6 100644 --- a/kernel/events/callchain.c +++ b/kernel/events/callchain.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "internal.h" @@ -176,13 +177,51 @@ put_callchain_entry(int rctx) put_recursion_context(this_cpu_ptr(callchain_recursion), rctx); } +static void fixup_uretprobe_trampoline_entries(struct perf_callchain_entry *entry, + int start_entry_idx) +{ +#ifdef CONFIG_UPROBES + struct uprobe_task *utask = current->utask; + struct return_instance *ri; + __u64 *cur_ip, *last_ip, tramp_addr; + + if (likely(!utask || !utask->return_instances)) + return; + + cur_ip = &entry->ip[start_entry_idx]; + last_ip = &entry->ip[entry->nr - 1]; + ri = utask->return_instances; + tramp_addr = uprobe_get_trampoline_vaddr(); + + /* + * If there are pending uretprobes for the current thread, they are + * recorded in a list inside utask->return_instances; each such + * pending uretprobe replaces traced user function's return address on + * the stack, so when stack trace is captured, instead of seeing + * actual function's return address, we'll have one or many uretprobe + * trampoline addresses in the stack trace, which are not helpful and + * misleading to users. + * So here we go over the pending list of uretprobes, and each + * encountered trampoline address is replaced with actual return + * address. + */ + while (ri && cur_ip <= last_ip) { + if (*cur_ip == tramp_addr) { + *cur_ip = ri->orig_ret_vaddr; + ri = ri->next; + } + cur_ip++; + } +#endif +} + struct perf_callchain_entry * get_perf_callchain(struct pt_regs *regs, u32 init_nr, bool kernel, bool user, u32 max_stack, bool crosstask, bool add_mark) { struct perf_callchain_entry *entry; struct perf_callchain_entry_ctx ctx; - int rctx; + int rctx, start_entry_idx; entry = get_callchain_entry(&rctx); if (!entry) @@ -215,7 +254,9 @@ get_perf_callchain(struct pt_regs *regs, u32 init_nr, bool kernel, bool user, if (add_mark) perf_callchain_store_context(&ctx, PERF_CONTEXT_USER); + start_entry_idx = entry->nr; perf_callchain_user(&ctx, regs); + fixup_uretprobe_trampoline_entries(entry, start_entry_idx); } } diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index d60d24f0f2f4..1c99380dc89d 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2149,6 +2149,15 @@ static void handle_trampoline(struct pt_regs *regs) instruction_pointer_set(regs, ri->orig_ret_vaddr); do { + /* pop current instance from the stack of pending return instances, + * as it's not pending anymore: we just fixed up original + * instruction pointer in regs and are about to call handlers; + * this allows fixup_uretprobe_trampoline_entries() to properly fix up + * captured stack traces from uretprobe handlers, in which pending + * trampoline addresses on the stack are replaced with correct + * original return addresses + */ + utask->return_instances = ri->next; if (valid) handle_uretprobe_chain(ri, regs); ri = free_ret_instance(ri); From patchwork Wed May 22 01:38:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13670141 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C278BA4D; Wed, 22 May 2024 01:38:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341938; cv=none; b=EYG1EmnajJhy7/Kc9DKWxhOU4scpipw5+ql1OnECBd5cFapihtZ3oFR0MZCxP/Wa0dXKOsaNdryDovpiHkt4uyQrj94boHm+78Dc9Ziw8UxKLfEn41K5H0LCnAaHsrKar9vOI1kFvvis0Qij8HyzrwCf9QKoDWOC2kWmMlBNjmg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341938; c=relaxed/simple; bh=IcCH6HvV+fA+Q/OECgPQYvioKi/QPVoGcj2BlQkwI7Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aexvW7Sjm3nynFRS2puI/mwr+u2RGUICdMZ40WU9ApyVLJPkKed8rjiXmHIBmWu3ZH7Xi5mDIlNbxlInfcun95DReyTqMxl2a5PxX7WRJwGeLI2sLqcyme8ui0SvEx4z49cKrZbbRIVpEgpYZmtP8vFN7eYYTcqBCgHuG4hskww= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ofRodtza; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ofRodtza" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BFCECC2BD11; Wed, 22 May 2024 01:38:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716341938; bh=IcCH6HvV+fA+Q/OECgPQYvioKi/QPVoGcj2BlQkwI7Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ofRodtza/FgNXE4xmXfyT4qa1AIfvDq5rkLcxsb9CLEGMEZeFAGHyMnZJSVcW7CBt kV3HYqpoaRNKCppfCGuQIMu3w3ZGKQJQKLhlXROkxNMGdSq7u7MqHgI7oMNSbOotDX IllNutuhJ6T9tPNGWWAMQ+sqL6WYKgqYGlUsCbIpAqxTrmsL1xBQN0UVjjO9LihbfL VPADmJTeiO9kUiL+jiQ/RZgZ0FNnGz+GFu/x1VIMxV/+2KUV1VvQI7ZW4UAueuJPwe tJTSAk5etDMA8kLNtXByBB06FfE6qGrdM2DsgeakG+fyr89Z4c8/C9UtN/SdNJYeYr PI1XH/4MNO/pQ== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko Subject: [PATCH v2 3/4] perf,x86: avoid missing caller address in stack traces captured in uprobe Date: Tue, 21 May 2024 18:38:44 -0700 Message-ID: <20240522013845.1631305-4-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240522013845.1631305-1-andrii@kernel.org> References: <20240522013845.1631305-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When tracing user functions with uprobe functionality, it's common to install the probe (e.g., a BPF program) at the first instruction of the function. This is often going to be `push %rbp` instruction in function preamble, which means that within that function frame pointer hasn't been established yet. This leads to consistently missing an actual caller of the traced function, because perf_callchain_user() only records current IP (capturing traced function) and then following frame pointer chain (which would be caller's frame, containing the address of caller's caller). So when we have target_1 -> target_2 -> target_3 call chain and we are tracing an entry to target_3, captured stack trace will report target_1 -> target_3 call chain, which is wrong and confusing. This patch proposes a x86-64-specific heuristic to detect `push %rbp` instruction being traced. If that's the case, with the assumption that applicatoin is compiled with frame pointers, this instruction would be a strong indicator that this is the entry to the function. In that case, return address is still pointed to by %rsp, so we fetch it and add to stack trace before proceeding to unwind the rest using frame pointer-based logic. Signed-off-by: Andrii Nakryiko --- arch/x86/events/core.c | 20 ++++++++++++++++++++ include/linux/uprobes.h | 2 ++ kernel/events/uprobes.c | 2 ++ 3 files changed, 24 insertions(+) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 5b0dd07b1ef1..82d5570b58ff 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2884,6 +2884,26 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs return; pagefault_disable(); + +#ifdef CONFIG_UPROBES + /* + * If we are called from uprobe handler, and we are indeed at the very + * entry to user function (which is normally a `push %rbp` instruction, + * under assumption of application being compiled with frame pointers), + * we should read return address from *regs->sp before proceeding + * to follow frame pointers, otherwise we'll skip immediate caller + * as %rbp is not yet setup. + */ + if (current->utask) { + struct arch_uprobe *auprobe = current->utask->auprobe; + u64 ret_addr; + + if (auprobe && auprobe->insn[0] == 0x55 /* push %rbp */ && + !__get_user(ret_addr, (const u64 __user *)regs->sp)) + perf_callchain_store(entry, ret_addr); + } +#endif + while (entry->nr < entry->max_stack) { if (!valid_user_frame(fp, sizeof(frame))) break; diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 0c57eec85339..7b785cd30d86 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -76,6 +76,8 @@ struct uprobe_task { struct uprobe *active_uprobe; unsigned long xol_vaddr; + struct arch_uprobe *auprobe; + struct return_instance *return_instances; unsigned int depth; }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 1c99380dc89d..504693845187 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2072,6 +2072,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) bool need_prep = false; /* prepare return uprobe, when needed */ down_read(&uprobe->register_rwsem); + current->utask->auprobe = &uprobe->arch; for (uc = uprobe->consumers; uc; uc = uc->next) { int rc = 0; @@ -2086,6 +2087,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) remove &= rc; } + current->utask->auprobe = NULL; if (need_prep && !remove) prepare_uretprobe(uprobe, regs); /* put bp at return */ From patchwork Wed May 22 01:38:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13670142 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B68B7BA4D; Wed, 22 May 2024 01:39:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341941; cv=none; b=L0GtQl0O7r9KPn2WR8nQTQXxZHqj7iQG9iw6ofreFO2u7bh8gsqzHm50C0lrzCnye+92ZLtxIWuafj/ocGCoZ2qM72XpRfDl+VdnFYHf+w17B/zadQF8HosVgj33JQsU7fqCR5IfcMUsGsWbcmyfAamRiNBds+NIRtvlw2fsLJo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716341941; c=relaxed/simple; bh=AdIHiAuVg2hn61op8w/bLkkUCBP555S8lXKMcjxkn9g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oW5NI5JxJFlD4SCRvGi698ohYRaOfVek9cZ1KoHFMjja2cSIlVMzIp+THt4WvcQWbQXsw1tOX5NjzAZUntc54ApBwR2hR+Z2czyVDDwhi23+Ujz7j2fv1u8+XmsggoeBdrOGQ2q5sxIiR2ltWbKHPtRIYHPwhMUHzdvjQK6JDs8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hGX8tKYp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hGX8tKYp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E684C2BD11; Wed, 22 May 2024 01:39:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716341941; bh=AdIHiAuVg2hn61op8w/bLkkUCBP555S8lXKMcjxkn9g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hGX8tKYpxlmqA9NcrWOOoBSPwM2XovlJzzsRYz3VOZvaPQI9MLfJ9Dx8s7ih3b7SU CDBRhNniVp8e00RcXrVXZKyTHMVHRxZjZTTuhD9WySYYZ3030PtcRdh/gbuPBfQArV kHwg+y4ssLnZA4Ek4A8JOKjOMERTjQkGScXZj0U40fc4X5bPTBKpFgTU2PpxIkLXZ7 QX2gdHCFyFLE6LQuNmWUVWf4r25rnYsgshnrw7MlL/cs/TI/FrtivZgaFhWcij+77M 4f0Gd2X/TbINmJmOK8BrMDHSFJMTBZmb5E/u/HsnIhUX+1oc/vOXMIXxYg5HMG7NbG NvmaC/zlS534g== From: Andrii Nakryiko To: linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: x86@kernel.org, peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de, bpf@vger.kernel.org, rihams@fb.com, linux-perf-users@vger.kernel.org, Andrii Nakryiko Subject: [PATCH v2 4/4] selftests/bpf: add test validating uprobe/uretprobe stack traces Date: Tue, 21 May 2024 18:38:45 -0700 Message-ID: <20240522013845.1631305-5-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240522013845.1631305-1-andrii@kernel.org> References: <20240522013845.1631305-1-andrii@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add a set of tests to validate that stack traces captured from or in the presence of active uprobes and uretprobes are valid and complete. For this we use BPF program that are installed either on entry or exit of user function, plus deep-nested USDT. One of target funtions (target_1) is recursive to generate two different entries in the stack trace for the same uprobe/uretprobe, testing potential edge conditions. Without fixes in this patch set, we get something like this for one of the scenarios: caller: 0x758fff - 0x7595ab target_1: 0x758fd5 - 0x758fff target_2: 0x758fca - 0x758fd5 target_3: 0x758fbf - 0x758fca target_4: 0x758fb3 - 0x758fbf ENTRY #0: 0x758fb3 (in target_4) ENTRY #1: 0x758fd3 (in target_2) ENTRY #2: 0x758ffd (in target_1) ENTRY #3: 0x7fffffffe000 ENTRY #4: 0x7fffffffe000 ENTRY #5: 0x6f8f39 ENTRY #6: 0x6fa6f0 ENTRY #7: 0x7f403f229590 Entry #3 and #4 (0x7fffffffe000) are uretprobe trampoline addresses which obscure actual target_1 and another target_1 invocations. Also note that between entry #0 and entry #1 we are missing an entry for target_3, which is fixed in patch #2. With all the fixes, we get desired full stack traces: caller: 0x758fff - 0x7595ab target_1: 0x758fd5 - 0x758fff target_2: 0x758fca - 0x758fd5 target_3: 0x758fbf - 0x758fca target_4: 0x758fb3 - 0x758fbf ENTRY #0: 0x758fb7 (in target_4) ENTRY #1: 0x758fc8 (in target_3) ENTRY #2: 0x758fd3 (in target_2) ENTRY #3: 0x758ffd (in target_1) ENTRY #4: 0x758ff3 (in target_1) ENTRY #5: 0x75922c (in caller) ENTRY #6: 0x6f8f39 ENTRY #7: 0x6fa6f0 ENTRY #8: 0x7f986adc4cd0 Now there is a logical and complete sequence of function calls. Signed-off-by: Andrii Nakryiko Acked-by: Jiri Olsa --- .../bpf/prog_tests/uretprobe_stack.c | 186 ++++++++++++++++++ .../selftests/bpf/progs/uretprobe_stack.c | 96 +++++++++ 2 files changed, 282 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c create mode 100644 tools/testing/selftests/bpf/progs/uretprobe_stack.c diff --git a/tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c b/tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c new file mode 100644 index 000000000000..6deb8d560ddd --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/uretprobe_stack.c @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include +#include "uretprobe_stack.skel.h" +#include "../sdt.h" + +/* We set up target_1() -> target_2() -> target_3() -> target_4() -> USDT() + * call chain, each being traced by our BPF program. On entry or return from + * each target_*() we are capturing user stack trace and recording it in + * global variable, so that user space part of the test can validate it. + * + * Note, we put each target function into a custom section to get those + * __start_XXX/__stop_XXX symbols, generated by linker for us, which allow us + * to know address range of those functions + */ +__attribute__((section("uprobe__target_4"))) +__weak int target_4(void) +{ + STAP_PROBE1(uretprobe_stack, target, 42); + return 42; +} + +extern const void *__start_uprobe__target_4; +extern const void *__stop_uprobe__target_4; + +__attribute__((section("uprobe__target_3"))) +__weak int target_3(void) +{ + return target_4(); +} + +extern const void *__start_uprobe__target_3; +extern const void *__stop_uprobe__target_3; + +__attribute__((section("uprobe__target_2"))) +__weak int target_2(void) +{ + return target_3(); +} + +extern const void *__start_uprobe__target_2; +extern const void *__stop_uprobe__target_2; + +__attribute__((section("uprobe__target_1"))) +__weak int target_1(int depth) +{ + if (depth < 1) + return 1 + target_1(depth + 1); + else + return target_2(); +} + +extern const void *__start_uprobe__target_1; +extern const void *__stop_uprobe__target_1; + +extern const void *__start_uretprobe_stack_sec; +extern const void *__stop_uretprobe_stack_sec; + +struct range { + long start; + long stop; +}; + +static struct range targets[] = { + {}, /* we want target_1 to map to target[1], so need 1-based indexing */ + { (long)&__start_uprobe__target_1, (long)&__stop_uprobe__target_1 }, + { (long)&__start_uprobe__target_2, (long)&__stop_uprobe__target_2 }, + { (long)&__start_uprobe__target_3, (long)&__stop_uprobe__target_3 }, + { (long)&__start_uprobe__target_4, (long)&__stop_uprobe__target_4 }, +}; + +static struct range caller = { + (long)&__start_uretprobe_stack_sec, + (long)&__stop_uretprobe_stack_sec, +}; + +static void validate_stack(__u64 *ips, int stack_len, int cnt, ...) +{ + int i, j; + va_list args; + + if (!ASSERT_GT(stack_len, 0, "stack_len")) + return; + + stack_len /= 8; + + /* check if we have enough entries to satisfy test expectations */ + if (!ASSERT_GE(stack_len, cnt, "stack_len2")) + return; + + if (env.verbosity >= VERBOSE_NORMAL) { + printf("caller: %#lx - %#lx\n", caller.start, caller.stop); + for (i = 1; i < ARRAY_SIZE(targets); i++) + printf("target_%d: %#lx - %#lx\n", i, targets[i].start, targets[i].stop); + for (i = 0; i < stack_len; i++) { + for (j = 1; j < ARRAY_SIZE(targets); j++) { + if (ips[i] >= targets[j].start && ips[i] < targets[j].stop) + break; + } + if (j < ARRAY_SIZE(targets)) { /* found target match */ + printf("ENTRY #%d: %#lx (in target_%d)\n", i, (long)ips[i], j); + } else if (ips[i] >= caller.start && ips[i] < caller.stop) { + printf("ENTRY #%d: %#lx (in caller)\n", i, (long)ips[i]); + } else { + printf("ENTRY #%d: %#lx\n", i, (long)ips[i]); + } + } + } + + va_start(args, cnt); + + for (i = cnt - 1; i >= 0; i--) { + /* most recent entry is the deepest target function */ + const struct range *t = va_arg(args, const struct range *); + + ASSERT_GE(ips[i], t->start, "addr_start"); + ASSERT_LT(ips[i], t->stop, "addr_stop"); + } + + va_end(args); +} + +/* __weak prevents inlining */ +__attribute__((section("uretprobe_stack_sec"))) +__weak void test_uretprobe_stack(void) +{ + LIBBPF_OPTS(bpf_uprobe_opts, uprobe_opts); + struct uretprobe_stack *skel; + int err; + + skel = uretprobe_stack__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + err = uretprobe_stack__attach(skel); + if (!ASSERT_OK(err, "skel_attach")) + goto cleanup; + + /* trigger */ + ASSERT_EQ(target_1(0), 42 + 1, "trigger_return"); + + /* + * Stacks captured on ENTRY uprobes + */ + + /* (uprobe 1) target_1 in stack trace*/ + validate_stack(skel->bss->entry_stack1, skel->bss->entry1_len, + 2, &caller, &targets[1]); + /* (uprobe 1, recursed) */ + validate_stack(skel->bss->entry_stack1_recur, skel->bss->entry1_recur_len, + 3, &caller, &targets[1], &targets[1]); + /* (uprobe 2) caller -> target_1 -> target_1 -> target_2 */ + validate_stack(skel->bss->entry_stack2, skel->bss->entry2_len, + 4, &caller, &targets[1], &targets[1], &targets[2]); + /* (uprobe 3) */ + validate_stack(skel->bss->entry_stack3, skel->bss->entry3_len, + 5, &caller, &targets[1], &targets[1], &targets[2], &targets[3]); + /* (uprobe 4) caller -> target_1 -> target_1 -> target_2 -> target_3 -> target_4 */ + validate_stack(skel->bss->entry_stack4, skel->bss->entry4_len, + 6, &caller, &targets[1], &targets[1], &targets[2], &targets[3], &targets[4]); + + /* (USDT): full caller -> target_1 -> target_1 -> target_2 (uretprobed) + * -> target_3 -> target_4 (uretprobes) chain + */ + validate_stack(skel->bss->usdt_stack, skel->bss->usdt_len, + 6, &caller, &targets[1], &targets[1], &targets[2], &targets[3], &targets[4]); + + /* + * Now stacks captured on the way out in EXIT uprobes + */ + + /* (uretprobe 4) everything up to target_4, but excluding it */ + validate_stack(skel->bss->exit_stack4, skel->bss->exit4_len, + 5, &caller, &targets[1], &targets[1], &targets[2], &targets[3]); + /* we didn't install uretprobes on target_2 and target_3 */ + /* (uretprobe 1, recur) first target_1 call only */ + validate_stack(skel->bss->exit_stack1_recur, skel->bss->exit1_recur_len, + 2, &caller, &targets[1]); + /* (uretprobe 1) just a caller in the stack trace */ + validate_stack(skel->bss->exit_stack1, skel->bss->exit1_len, + 1, &caller); + +cleanup: + uretprobe_stack__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/uretprobe_stack.c b/tools/testing/selftests/bpf/progs/uretprobe_stack.c new file mode 100644 index 000000000000..9fdcf396b8f4 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uretprobe_stack.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__u64 entry_stack1[32], exit_stack1[32]; +__u64 entry_stack1_recur[32], exit_stack1_recur[32]; +__u64 entry_stack2[32]; +__u64 entry_stack3[32]; +__u64 entry_stack4[32], exit_stack4[32]; +__u64 usdt_stack[32]; + +int entry1_len, exit1_len; +int entry1_recur_len, exit1_recur_len; +int entry2_len, exit2_len; +int entry3_len, exit3_len; +int entry4_len, exit4_len; +int usdt_len; + +#define SZ sizeof(usdt_stack) + +SEC("uprobe//proc/self/exe:target_1") +int BPF_UPROBE(uprobe_1) +{ + /* target_1 is recursive wit depth of 2, so we capture two separate + * stack traces, depending on which occurence it is + */ + static bool recur = false; + + if (!recur) + entry1_len = bpf_get_stack(ctx, &entry_stack1, SZ, BPF_F_USER_STACK); + else + entry1_recur_len = bpf_get_stack(ctx, &entry_stack1_recur, SZ, BPF_F_USER_STACK); + + recur = true; + return 0; +} + +SEC("uretprobe//proc/self/exe:target_1") +int BPF_URETPROBE(uretprobe_1) +{ + /* see above, target_1 is recursive */ + static bool recur = false; + + /* NOTE: order of returns is reversed to order of entries */ + if (!recur) + exit1_recur_len = bpf_get_stack(ctx, &exit_stack1_recur, SZ, BPF_F_USER_STACK); + else + exit1_len = bpf_get_stack(ctx, &exit_stack1, SZ, BPF_F_USER_STACK); + + recur = true; + return 0; +} + +SEC("uprobe//proc/self/exe:target_2") +int BPF_UPROBE(uprobe_2) +{ + entry2_len = bpf_get_stack(ctx, &entry_stack2, SZ, BPF_F_USER_STACK); + return 0; +} + +/* no uretprobe for target_2 */ + +SEC("uprobe//proc/self/exe:target_3") +int BPF_UPROBE(uprobe_3) +{ + entry3_len = bpf_get_stack(ctx, &entry_stack3, SZ, BPF_F_USER_STACK); + return 0; +} + +/* no uretprobe for target_3 */ + +SEC("uprobe//proc/self/exe:target_4") +int BPF_UPROBE(uprobe_4) +{ + entry4_len = bpf_get_stack(ctx, &entry_stack4, SZ, BPF_F_USER_STACK); + return 0; +} + +SEC("uretprobe//proc/self/exe:target_4") +int BPF_URETPROBE(uretprobe_4) +{ + exit4_len = bpf_get_stack(ctx, &exit_stack4, SZ, BPF_F_USER_STACK); + return 0; +} + +SEC("usdt//proc/self/exe:uretprobe_stack:target") +int BPF_USDT(usdt_probe) +{ + usdt_len = bpf_get_stack(ctx, &usdt_stack, SZ, BPF_F_USER_STACK); + return 0; +}