From patchwork Thu Oct 10 20:09:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831028 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B08241CF2A8; Thu, 10 Oct 2024 20:10:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591017; cv=none; b=CFmW/U/ENlTzlAdy/y+xUbBJr2ohuO1Mw3y+qp/oMJv3gfC3lH8MUgPpRceHqB3rgDH1fQmHnoTaUomd8q4Jfq/gjticQeDoy2E/oZijkRrIWJfSCN3DuzSvWbYXLUJXwQeYCYo2RVdtYDkllDPJ2rPDAkdhcv0LJDnr5rt41sI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591017; c=relaxed/simple; bh=lyivNG7QkcWF//27U5OwYwaUvTLVf7GexR1n3yjWShI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TiK1lw3hIEG2zlnRp+qorXoWS2Y3hDUv1Ji/m6/dXcC5ufrHcJNjOgSLp2BYBfZ2ErbJIV0RptAM8JT5TKgAYHp4FuBXcOmEct8YSncXvbMuqNgGR4re7EPU2jp3MLFaWzbHeBCA1/uJQvRGgysvbjAdmaTFwjn9JRxa1OWKwNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=K/QqtC7J; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="K/QqtC7J" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BFD8C4CEC5; Thu, 10 Oct 2024 20:10:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591017; bh=lyivNG7QkcWF//27U5OwYwaUvTLVf7GexR1n3yjWShI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K/QqtC7Jo0JuyAmgfy2kv4VPsb9kariht0APfShst0S5gGC3lf8xDj8bWM2ntpvUW VIDxyHDHjo1Ki899ETpwyNMHHNuOod/FDUsYb93Pr1THGhj10N8ibkIesULnEh7e+C 2Ym4I4H7iJMI0F1FaNb2WalNZxrkV2X7hNbOSmc4U1NVADV1EwdAEyidZknSCVvfSL C+wllFf+onU9BeIrIzqq1TP0DJ8BjttVSpu+qfWFXIILaY5+c6s5HUTuJkKfIN9p5q FsYKTMIEpcQTe6iMDjCudwWX8yIqhSWP1vhMJjWQh8gvSBH3/5FxSiDp3tdTCatiu2 4yToljIBtrZoQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 perf/core 01/16] uprobe: Add data pointer to consumer handlers Date: Thu, 10 Oct 2024 22:09:42 +0200 Message-ID: <20241010200957.2750179-2-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding data pointer to both entry and exit consumer handlers and all its users. The functionality itself is coming in following change. Acked-by: Oleg Nesterov Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- include/linux/uprobes.h | 4 ++-- kernel/events/uprobes.c | 4 ++-- kernel/trace/bpf_trace.c | 6 ++++-- kernel/trace/trace_uprobe.c | 12 ++++++++---- .../testing/selftests/bpf/bpf_testmod/bpf_testmod.c | 2 +- 5 files changed, 17 insertions(+), 11 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 2b294bf1881f..bb265a632b91 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -37,10 +37,10 @@ struct uprobe_consumer { * for the current process. If filter() is omitted or returns true, * UPROBE_HANDLER_REMOVE is effectively ignored. */ - int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs); + int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs, __u64 *data); int (*ret_handler)(struct uprobe_consumer *self, unsigned long func, - struct pt_regs *regs); + struct pt_regs *regs, __u64 *data); bool (*filter)(struct uprobe_consumer *self, struct mm_struct *mm); struct list_head cons_node; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 2a0059464383..6b44c386a5df 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2090,7 +2090,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) int rc = 0; if (uc->handler) { - rc = uc->handler(uc, regs); + rc = uc->handler(uc, regs, NULL); WARN(rc & ~UPROBE_HANDLER_MASK, "bad rc=0x%x from %ps()\n", rc, uc->handler); } @@ -2128,7 +2128,7 @@ handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs) rcu_read_lock_trace(); list_for_each_entry_rcu(uc, &uprobe->consumers, cons_node, rcu_read_lock_trace_held()) { if (uc->ret_handler) - uc->ret_handler(uc, ri->func, regs); + uc->ret_handler(uc, ri->func, regs, NULL); } rcu_read_unlock_trace(); } diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index a582cd25ca87..fdab7ecd8dfa 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3244,7 +3244,8 @@ uprobe_multi_link_filter(struct uprobe_consumer *con, struct mm_struct *mm) } static int -uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs) +uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, + __u64 *data) { struct bpf_uprobe *uprobe; @@ -3253,7 +3254,8 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs) } static int -uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs) +uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs, + __u64 *data) { struct bpf_uprobe *uprobe; diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c index c40531d2cbad..5895eabe3581 100644 --- a/kernel/trace/trace_uprobe.c +++ b/kernel/trace/trace_uprobe.c @@ -89,9 +89,11 @@ static struct trace_uprobe *to_trace_uprobe(struct dyn_event *ev) static int register_uprobe_event(struct trace_uprobe *tu); static int unregister_uprobe_event(struct trace_uprobe *tu); -static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs); +static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs, + __u64 *data); static int uretprobe_dispatcher(struct uprobe_consumer *con, - unsigned long func, struct pt_regs *regs); + unsigned long func, struct pt_regs *regs, + __u64 *data); #ifdef CONFIG_STACK_GROWSUP static unsigned long adjust_stack_addr(unsigned long addr, unsigned int n) @@ -1517,7 +1519,8 @@ trace_uprobe_register(struct trace_event_call *event, enum trace_reg type, } } -static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs) +static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs, + __u64 *data) { struct trace_uprobe *tu; struct uprobe_dispatch_data udd; @@ -1548,7 +1551,8 @@ static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs) } static int uretprobe_dispatcher(struct uprobe_consumer *con, - unsigned long func, struct pt_regs *regs) + unsigned long func, struct pt_regs *regs, + __u64 *data) { struct trace_uprobe *tu; struct uprobe_dispatch_data udd; diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 8835761d9a12..12005e3dc3e4 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -461,7 +461,7 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = { static int uprobe_ret_handler(struct uprobe_consumer *self, unsigned long func, - struct pt_regs *regs) + struct pt_regs *regs, __u64 *data) { regs->ax = 0x12345678deadbeef; From patchwork Thu Oct 10 20:09:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831029 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EE1F1CF5C3; Thu, 10 Oct 2024 20:10:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591030; cv=none; b=VC3Jpsjj//zPD50pkYWL0SIrGyLgkBfySiZpSkO97BjnKob3unaUTpmx48dlHFFRyYrQEh5/v5Lai9Rau0jLHBnVLGdCULFO8Nyn4/CRqJAp+UKSpkivwJnc7kS0sepd27UkVZJCsggmdHedd09X0gu98FLOvtGeHok8bBQ6Tdc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591030; c=relaxed/simple; bh=6GdmK0JYu6jU1SnJ2p38uq1xov5i1UWI7bFqv0uXoFg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=L7a4DrrxB01NYtDFRar2WnJZvKfaXzsS06LlqtDF+kSF0coaltDoo6hNHOFboHgP7Cy9wYbzXgAOBlyG+cRbWqUUZUamtLR32lLv0uIqv4NQ7Juhc54ZPNmef7r0KzM1QUL0TKHpWPtBXKNoD2icuekOGR4vdL6vsaWgOZGCRXs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dcFsleX+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dcFsleX+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 68683C4CEC5; Thu, 10 Oct 2024 20:10:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591030; bh=6GdmK0JYu6jU1SnJ2p38uq1xov5i1UWI7bFqv0uXoFg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dcFsleX+fjc4Az8jBzwEe+iCrbctfKaD00JRt+uXF9ozjWrgwAAJ/5TYuCt1HdHOS aaU4Gy2ZALwo5t2oDkAY0OcuzPziUCxcAZtAP5Bbvn53eV+6PsGHIsqgjnRdHjYqq1 /Y85oHF2VdUFE+xMpl4wbeXnWoLvnjzSG0XnMgMmbEHSsrUozYH2dFkDEsvCKV05fQ k9/iT7fbK9AfbzfpCCyXEol+02VfQNqeoO3CGN4MeneKbenEFkJOhTiWDkhtqr4NSP MavIGP8xQrmy5c3W6Yl09zCRliGn0w6lXBP88cLrsXjyzSXPhc5S6AmZNBRYl5f7OF HB8AsuSe0Jn5Q== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 perf/core 02/16] uprobe: Add support for session consumer Date: Thu, 10 Oct 2024 22:09:43 +0200 Message-ID: <20241010200957.2750179-3-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This change allows the uprobe consumer to behave as session which means that 'handler' and 'ret_handler' callbacks are connected in a way that allows to: - control execution of 'ret_handler' from 'handler' callback - share data between 'handler' and 'ret_handler' callbacks The session concept fits to our common use case where we do filtering on entry uprobe and based on the result we decide to run the return uprobe (or not). It's also convenient to share the data between session callbacks. To achive this we are adding new return value the uprobe consumer can return from 'handler' callback: UPROBE_HANDLER_IGNORE - Ignore 'ret_handler' callback for this consumer. And store cookie and pass it to 'ret_handler' when consumer has both 'handler' and 'ret_handler' callbacks defined. We store shared data in the return_consumer object array as part of the return_instance object. This way the handle_uretprobe_chain can find related return_consumer and its shared data. We also store entry handler return value, for cases when there are multiple consumers on single uprobe and some of them are ignored and some of them not, in which case the return probe gets installed and we need to have a way to find out which consumer needs to be ignored. The tricky part is when consumer is registered 'after' the uprobe entry handler is hit. In such case this consumer's 'ret_handler' gets executed as well, but it won't have the proper data pointer set, so we can filter it out. Acked-by: Andrii Nakryiko Suggested-by: Oleg Nesterov Signed-off-by: Jiri Olsa Reviewed-by: Oleg Nesterov --- include/linux/uprobes.h | 21 +++++- kernel/events/uprobes.c | 148 ++++++++++++++++++++++++++++++++-------- 2 files changed, 139 insertions(+), 30 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index bb265a632b91..dbaf04189548 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -23,8 +23,17 @@ struct inode; struct notifier_block; struct page; +/* + * Allowed return values from uprobe consumer's handler callback + * with following meaning: + * + * UPROBE_HANDLER_REMOVE + * - Remove the uprobe breakpoint from current->mm. + * UPROBE_HANDLER_IGNORE + * - Ignore ret_handler callback for this consumer. + */ #define UPROBE_HANDLER_REMOVE 1 -#define UPROBE_HANDLER_MASK 1 +#define UPROBE_HANDLER_IGNORE 2 #define MAX_URETPROBE_DEPTH 64 @@ -44,6 +53,8 @@ struct uprobe_consumer { bool (*filter)(struct uprobe_consumer *self, struct mm_struct *mm); struct list_head cons_node; + + __u64 id; /* set when uprobe_consumer is registered */ }; #ifdef CONFIG_UPROBES @@ -83,14 +94,22 @@ struct uprobe_task { unsigned int depth; }; +struct return_consumer { + __u64 cookie; + __u64 id; +}; + struct return_instance { struct uprobe *uprobe; unsigned long func; unsigned long stack; /* stack pointer */ unsigned long orig_ret_vaddr; /* original return address */ bool chained; /* true, if instance is nested */ + int consumers_cnt; struct return_instance *next; /* keep as stack */ + + struct return_consumer consumers[] __counted_by(consumers_cnt); }; enum rp_check { diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 6b44c386a5df..4ef4b51776eb 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -64,7 +64,7 @@ struct uprobe { struct rcu_head rcu; loff_t offset; loff_t ref_ctr_offset; - unsigned long flags; + unsigned long flags; /* "unsigned long" so bitops work */ /* * The generic code assumes that it has two members of unknown type @@ -823,8 +823,11 @@ static struct uprobe *alloc_uprobe(struct inode *inode, loff_t offset, static void consumer_add(struct uprobe *uprobe, struct uprobe_consumer *uc) { + static atomic64_t id; + down_write(&uprobe->consumer_rwsem); list_add_rcu(&uc->cons_node, &uprobe->consumers); + uc->id = (__u64) atomic64_inc_return(&id); up_write(&uprobe->consumer_rwsem); } @@ -1761,6 +1764,34 @@ static struct uprobe_task *get_utask(void) return current->utask; } +static size_t ri_size(int consumers_cnt) +{ + struct return_instance *ri; + + return sizeof(*ri) + sizeof(ri->consumers[0]) * consumers_cnt; +} + +#define DEF_CNT 4 + +static struct return_instance *alloc_return_instance(void) +{ + struct return_instance *ri; + + ri = kzalloc(ri_size(DEF_CNT), GFP_KERNEL); + if (!ri) + return ZERO_SIZE_PTR; + + ri->consumers_cnt = DEF_CNT; + return ri; +} + +static struct return_instance *dup_return_instance(struct return_instance *old) +{ + size_t size = ri_size(old->consumers_cnt); + + return kmemdup(old, size, GFP_KERNEL); +} + static int dup_utask(struct task_struct *t, struct uprobe_task *o_utask) { struct uprobe_task *n_utask; @@ -1773,11 +1804,10 @@ static int dup_utask(struct task_struct *t, struct uprobe_task *o_utask) p = &n_utask->return_instances; for (o = o_utask->return_instances; o; o = o->next) { - n = kmalloc(sizeof(struct return_instance), GFP_KERNEL); + n = dup_return_instance(o); if (!n) return -ENOMEM; - *n = *o; /* * uprobe's refcnt has to be positive at this point, kept by * utask->return_instances items; return_instances can't be @@ -1870,35 +1900,31 @@ static void cleanup_return_instances(struct uprobe_task *utask, bool chained, utask->return_instances = ri; } -static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) +static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs, + struct return_instance *ri) { struct uprobe_task *utask = current->utask; unsigned long orig_ret_vaddr, trampoline_vaddr; - struct return_instance *ri; bool chained; if (!get_xol_area()) - return; + goto free; if (utask->depth >= MAX_URETPROBE_DEPTH) { printk_ratelimited(KERN_INFO "uprobe: omit uretprobe due to" " nestedness limit pid/tgid=%d/%d\n", current->pid, current->tgid); - return; + goto free; } /* we need to bump refcount to store uprobe in utask */ if (!try_get_uprobe(uprobe)) - return; - - ri = kmalloc(sizeof(struct return_instance), GFP_KERNEL); - if (!ri) - goto fail; + goto free; trampoline_vaddr = uprobe_get_trampoline_vaddr(); orig_ret_vaddr = arch_uretprobe_hijack_return_addr(trampoline_vaddr, regs); if (orig_ret_vaddr == -1) - goto fail; + goto put; /* drop the entries invalidated by longjmp() */ chained = (orig_ret_vaddr == trampoline_vaddr); @@ -1916,7 +1942,7 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) * attack from user-space. */ uprobe_warn(current, "handle tail call"); - goto fail; + goto put; } orig_ret_vaddr = utask->return_instances->orig_ret_vaddr; } @@ -1931,9 +1957,10 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) utask->return_instances = ri; return; -fail: - kfree(ri); +put: put_uprobe(uprobe); +free: + kfree(ri); } /* Prepare to single-step probed instruction out of line. */ @@ -2077,34 +2104,90 @@ static struct uprobe *find_active_uprobe_rcu(unsigned long bp_vaddr, int *is_swb return uprobe; } +static struct return_instance* +push_consumer(struct return_instance *ri, int idx, __u64 id, __u64 cookie) +{ + if (unlikely(ri == ZERO_SIZE_PTR)) + return ri; + + if (unlikely(idx >= ri->consumers_cnt)) { + struct return_instance *old_ri = ri; + + ri->consumers_cnt += DEF_CNT; + ri = krealloc(old_ri, ri_size(old_ri->consumers_cnt), GFP_KERNEL); + if (!ri) { + kfree(old_ri); + return ZERO_SIZE_PTR; + } + } + + ri->consumers[idx].id = id; + ri->consumers[idx].cookie = cookie; + return ri; +} + +static struct return_consumer * +return_consumer_find(struct return_instance *ri, int *iter, int id) +{ + struct return_consumer *ric; + int idx = *iter; + + for (ric = &ri->consumers[idx]; idx < ri->consumers_cnt; idx++, ric++) { + if (ric->id == id) { + *iter = idx + 1; + return ric; + } + } + return NULL; +} + +static bool ignore_ret_handler(int rc) +{ + return rc == UPROBE_HANDLER_REMOVE || rc == UPROBE_HANDLER_IGNORE; +} + static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) { struct uprobe_consumer *uc; - int remove = UPROBE_HANDLER_REMOVE; - bool need_prep = false; /* prepare return uprobe, when needed */ - bool has_consumers = false; + bool has_consumers = false, remove = true; + struct return_instance *ri = NULL; + int push_idx = 0; current->utask->auprobe = &uprobe->arch; list_for_each_entry_rcu(uc, &uprobe->consumers, cons_node, rcu_read_lock_trace_held()) { + bool session = uc->handler && uc->ret_handler; + __u64 cookie = 0; int rc = 0; if (uc->handler) { - rc = uc->handler(uc, regs, NULL); - WARN(rc & ~UPROBE_HANDLER_MASK, + rc = uc->handler(uc, regs, &cookie); + WARN(rc < 0 || rc > 2, "bad rc=0x%x from %ps()\n", rc, uc->handler); } - if (uc->ret_handler) - need_prep = true; - - remove &= rc; + remove &= rc == UPROBE_HANDLER_REMOVE; has_consumers = true; + + if (!uc->ret_handler || ignore_ret_handler(rc)) + continue; + + if (!ri) + ri = alloc_return_instance(); + + if (session) + ri = push_consumer(ri, push_idx++, uc->id, cookie); } current->utask->auprobe = NULL; - if (need_prep && !remove) - prepare_uretprobe(uprobe, regs); /* put bp at return */ + if (!ZERO_OR_NULL_PTR(ri)) { + /* + * The push_idx value has the final number of return consumers, + * and ri->consumers_cnt has number of allocated consumers. + */ + ri->consumers_cnt = push_idx; + prepare_uretprobe(uprobe, regs, ri); + } if (remove && has_consumers) { down_read(&uprobe->register_rwsem); @@ -2123,12 +2206,19 @@ static void handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs) { struct uprobe *uprobe = ri->uprobe; + struct return_consumer *ric; struct uprobe_consumer *uc; + int ric_idx = 0; rcu_read_lock_trace(); list_for_each_entry_rcu(uc, &uprobe->consumers, cons_node, rcu_read_lock_trace_held()) { - if (uc->ret_handler) - uc->ret_handler(uc, ri->func, regs, NULL); + bool session = uc->handler && uc->ret_handler; + + if (uc->ret_handler) { + ric = return_consumer_find(ri, &ric_idx, uc->id); + if (!session || ric) + uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL); + } } rcu_read_unlock_trace(); } From patchwork Thu Oct 10 20:09:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831030 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 440921CF5C3; Thu, 10 Oct 2024 20:10:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591042; cv=none; b=ngYGTY4IWHTRS5iy5PNJHddN/TFyKA/AWvRezjNwFERab4TLi+nPtF93fAsSfCn/Q4zLuUbDhjRY74MkSImbHBymkZf7Z6e1spEIOSarCwvnED62UE+DTV6yJBFfzZSl9SQqpQX7sPkJR7uZrd/CT3qXIg0i2iXXMJW1Ix7joyg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591042; c=relaxed/simple; bh=IpnexFqrLOrJyRd8LUIKHYxqoB4ni1PE1azc1ziZwCM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tm55sT5BDQUCujVbcqWkcUnxSAIOTQMfbs1UfHxs1L1b3B8JF/5nuqDr1XmoRH5s/iTbxwZOGVU1nyLIlw33Cabdr4tJYzeiOttVGzhkuzjTG5PuAG+mH7xH5SzC3OqmgoESLpTRI/GPOVqhbgc/v13uaIJM9PDw2DtBFCdvkJI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g4puKX0A; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g4puKX0A" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0A13AC4CEC5; Thu, 10 Oct 2024 20:10:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591042; bh=IpnexFqrLOrJyRd8LUIKHYxqoB4ni1PE1azc1ziZwCM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g4puKX0AuQrDAhsKkXwg3pFC7KccmN3IewO1Sq2xPUILWWbdmo9NCMnjSOB1Qdiv5 6H/csdFr93PqqRa3HbpWZRVNOvqtRFfvKW3vgSZarXxYaFtn24jeC/h/4nJbOvhwND YlQT3MTXASjgemUiSgY6zNB0zyZ67RaUzXrtdQpWAE+FUKLEN0FwSRq43gSWTmYGSb u0tgipWmPs4DaGA3+m4bPvFqzTZGf+Qh4TjbO1eDmey2F+Wnd3cg/5Z1IRb7R1+Q6d ty8ehpCu1DS8/9yWWT127nEhtP7FaXfY1BBOz66NDDgYMQlx81CyJHkBP4bvf+lgNt LXFIaSASO7ymw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 03/16] bpf: Allow return values 0 and 1 for kprobe session Date: Thu, 10 Oct 2024 22:09:44 +0200 Message-ID: <20241010200957.2750179-4-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The kprobe session program can return only 0 or 1, instruct verifier to check for that. Fixes: 535a3692ba72 ("bpf: Add support for kprobe session attach") Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- kernel/bpf/verifier.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7d9b38ffd220..787008872a14 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15910,6 +15910,15 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char return -ENOTSUPP; } break; + case BPF_PROG_TYPE_KPROBE: + switch (env->prog->expected_attach_type) { + case BPF_TRACE_KPROBE_SESSION: + range = retval_range(0, 1); + break; + default: + return 0; + } + break; case BPF_PROG_TYPE_SK_LOOKUP: range = retval_range(SK_DROP, SK_PASS); break; From patchwork Thu Oct 10 20:09:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831031 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 186A61CF28E; Thu, 10 Oct 2024 20:10:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591054; cv=none; b=Ll2km591HfT7F4raBQBLwjX1a/7ADCOEebvT7BXR9+TsxpKNyT+NaI3hbZ8udmKQg2ZszLLAO+AsEBS43tdsDHQnuKkXOpNGdrQXPPjeDAfS/rSGjdNK+c0OE8/Zjaqp9r2gSZDxJ7rJLcjaOEiIByCYuWGPLQq+ZbxdrlICG9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591054; c=relaxed/simple; bh=Nsb9smOufrjGnfYSOh3dlU2s4wT9SoXkoVS55B1Cq+Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KMn6a2FTqdpLPhPJeZ/yNVBquSapN7QOxLWPicrYckeSMK6k7Mgk09JS+OQSXO7mjF5tXwWs9LHtDVbXAAMDhoGcRS+5BROhbqS4lobmi6g+xfugKUxgEIqRDrGEiLplszr6OYW3U8BksH5V5GsL+oMIPFWumPjLINO1BlijN8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DcbR/dW3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DcbR/dW3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A910DC4CEC5; Thu, 10 Oct 2024 20:10:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591053; bh=Nsb9smOufrjGnfYSOh3dlU2s4wT9SoXkoVS55B1Cq+Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DcbR/dW39hN4vnaaBggNvTwwcIL3h73Na9N9bqkIWUWUfnTRiiBrIyyEnEW3GtbTB lszlWPHN2QrVCwTG6qXZGVE/TIgPmF05TFbSyzpB6sq4O5H8WMuHEwWQi4P7Ke/W5j Q/qeXBUJH0LvnmjQ5tYBTNiUxoXIHEtPsJ4KPTh7oAm0h4DCFQEYsdplQpRSWgo6Us laK14pS+aR7K1GaUvsZtCCmc0bbpuECccSO1b4oTiIIkX6VljF2/dP0YvoNcPyKRTn YwSjloj9KPB+4yX75yUsTYyV2J4qbaAzIs9y7+V5HLYEcKuwycejzpnHTCZe3OjKG2 Lm+UNVRoLojkg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 04/16] bpf: Force uprobe bpf program to always return 0 Date: Thu, 10 Oct 2024 22:09:45 +0200 Message-ID: <20241010200957.2750179-5-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net As suggested by Andrii make uprobe multi bpf programs to always return 0, so they can't force uprobe removal. Keeping the int return type for uprobe_prog_run, because it will be used in following session changes. Fixes: 89ae89f53d20 ("bpf: Add multi uprobe link") Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- kernel/trace/bpf_trace.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index fdab7ecd8dfa..3c1e5a561df4 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3209,7 +3209,6 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, struct bpf_prog *prog = link->link.prog; bool sleepable = prog->sleepable; struct bpf_run_ctx *old_run_ctx; - int err = 0; if (link->task && !same_thread_group(current, link->task)) return 0; @@ -3222,7 +3221,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, migrate_disable(); old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); - err = bpf_prog_run(link->link.prog, regs); + bpf_prog_run(link->link.prog, regs); bpf_reset_run_ctx(old_run_ctx); migrate_enable(); @@ -3231,7 +3230,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, rcu_read_unlock_trace(); else rcu_read_unlock(); - return err; + return 0; } static bool From patchwork Thu Oct 10 20:09:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831032 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2118919C553; Thu, 10 Oct 2024 20:11:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591066; cv=none; b=NZvLC5Mc6+eC6PPQNYCqszV1CiSIHosCG4eKynDk2bQy3JzcXSTCh+QN/oXiIBd6nfoo/hQphHk//1JFGs1sVKg6j6TbXi94bRczYS5n3UX4fmoKf3F85dfypQUjP6oaTNyhgtlyO4YGmOPlp/Un0ZJwkDFfHMKaB+PYWXSxDxs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591066; c=relaxed/simple; bh=Egd9yHpt7DStjWnY/MiUguu8Zo1DJ/1r6pyBYjgxp8M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rkTgG45hDvM9ginMWLDjmuoVqqV9/2etWSdO8SJJFJ0ZenfyiE5nx25P6dN8bJOxpVZDLx5eIwxlwfU4FUJd/1RURj5BrvbPsiBoML5ftLFxrFemc8PAzfqi9Q20/67my0W2XXiXOIUzWCQzAraUC0HEWmGk4hItAfxU/AESspg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=X6QCLt/k; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="X6QCLt/k" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A4473C4CEC5; Thu, 10 Oct 2024 20:11:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591065; bh=Egd9yHpt7DStjWnY/MiUguu8Zo1DJ/1r6pyBYjgxp8M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X6QCLt/kIhFfUGcBIi8lxt8yQoGGKlCFZDR50Wa6h4+0G9JiVSKXZVTnM0FL3SC3E x3Z/ugVzhTeLNxsg5hEcNurqu0LL6CRqHo9fUJT/Mje8zt41UlZEz3j1hxn/cI6YWu hHOvW40xirOiB8kjFNQgikBEB4pNL1aydlhVc0uny+fx4brLdy5Z/sfN5Ftpl7C+Da sFCxihXEfQ8FZsjctHkBxd+ycfkk0yjqchgNlHIclwiMzvQFGvYNYHcguzV3zCJ3mR vmzndMq+i+sW97gfO8BXk32QOhwASalAwDJulPISNbnE7fLpDe486SZy2hulhXj8yo b/Trhvv6sPfrA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 05/16] bpf: Add support for uprobe multi session attach Date: Thu, 10 Oct 2024 22:09:46 +0200 Message-ID: <20241010200957.2750179-6-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support to attach BPF program for entry and return probe of the same function. This is common use case which at the moment requires to create two uprobe multi links. Adding new BPF_TRACE_UPROBE_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe. It's possible to control execution of the BPF program on return probe simply by returning zero or non zero from the entry BPF program execution to execute or not the BPF program on return probe respectively. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 9 +++++++-- kernel/bpf/verifier.c | 1 + kernel/trace/bpf_trace.c | 36 +++++++++++++++++++++++++--------- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/libbpf.c | 1 + 6 files changed, 38 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 8ab4d8184b9d..77d0bc5fa986 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1116,6 +1116,7 @@ enum bpf_attach_type { BPF_NETKIT_PRIMARY, BPF_NETKIT_PEER, BPF_TRACE_KPROBE_SESSION, + BPF_TRACE_UPROBE_SESSION, __MAX_BPF_ATTACH_TYPE }; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index a8f1808a1ca5..0cf7617e6cb6 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3983,10 +3983,14 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI && attach_type != BPF_TRACE_UPROBE_MULTI) return -EINVAL; + if (prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION && + attach_type != BPF_TRACE_UPROBE_SESSION) + return -EINVAL; if (attach_type != BPF_PERF_EVENT && attach_type != BPF_TRACE_KPROBE_MULTI && attach_type != BPF_TRACE_KPROBE_SESSION && - attach_type != BPF_TRACE_UPROBE_MULTI) + attach_type != BPF_TRACE_UPROBE_MULTI && + attach_type != BPF_TRACE_UPROBE_SESSION) return -EINVAL; return 0; case BPF_PROG_TYPE_SCHED_CLS: @@ -5239,7 +5243,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI || attr->link_create.attach_type == BPF_TRACE_KPROBE_SESSION) ret = bpf_kprobe_multi_link_attach(attr, prog); - else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI) + else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI || + attr->link_create.attach_type == BPF_TRACE_UPROBE_SESSION) ret = bpf_uprobe_multi_link_attach(attr, prog); break; default: diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 787008872a14..c4d7b7369259 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15913,6 +15913,7 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char case BPF_PROG_TYPE_KPROBE: switch (env->prog->expected_attach_type) { case BPF_TRACE_KPROBE_SESSION: + case BPF_TRACE_UPROBE_SESSION: range = retval_range(0, 1); break; default: diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 3c1e5a561df4..5f10994376d0 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1557,6 +1557,17 @@ static inline bool is_kprobe_session(const struct bpf_prog *prog) return prog->expected_attach_type == BPF_TRACE_KPROBE_SESSION; } +static inline bool is_uprobe_multi(const struct bpf_prog *prog) +{ + return prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI || + prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION; +} + +static inline bool is_uprobe_session(const struct bpf_prog *prog) +{ + return prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION; +} + static const struct bpf_func_proto * kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { @@ -1574,13 +1585,13 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_get_func_ip: if (is_kprobe_multi(prog)) return &bpf_get_func_ip_proto_kprobe_multi; - if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + if (is_uprobe_multi(prog)) return &bpf_get_func_ip_proto_uprobe_multi; return &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: if (is_kprobe_multi(prog)) return &bpf_get_attach_cookie_proto_kmulti; - if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + if (is_uprobe_multi(prog)) return &bpf_get_attach_cookie_proto_umulti; return &bpf_get_attach_cookie_proto_trace; default: @@ -3074,6 +3085,7 @@ struct bpf_uprobe { u64 cookie; struct uprobe *uprobe; struct uprobe_consumer consumer; + bool session; }; struct bpf_uprobe_multi_link { @@ -3247,9 +3259,13 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, __u64 *data) { struct bpf_uprobe *uprobe; + int ret; uprobe = container_of(con, struct bpf_uprobe, consumer); - return uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + if (uprobe->session) + return ret ? UPROBE_HANDLER_IGNORE : 0; + return 0; } static int @@ -3259,7 +3275,8 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s struct bpf_uprobe *uprobe; uprobe = container_of(con, struct bpf_uprobe, consumer); - return uprobe_prog_run(uprobe, func, regs); + uprobe_prog_run(uprobe, func, regs); + return 0; } static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) @@ -3298,7 +3315,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr if (sizeof(u64) != sizeof(void *)) return -EOPNOTSUPP; - if (prog->expected_attach_type != BPF_TRACE_UPROBE_MULTI) + if (!is_uprobe_multi(prog)) return -EINVAL; flags = attr->link_create.uprobe_multi.flags; @@ -3374,11 +3391,12 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr uprobes[i].link = link; - if (flags & BPF_F_UPROBE_MULTI_RETURN) - uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; - else + if (!(flags & BPF_F_UPROBE_MULTI_RETURN)) uprobes[i].consumer.handler = uprobe_multi_link_handler; - + if (flags & BPF_F_UPROBE_MULTI_RETURN || is_uprobe_session(prog)) + uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; + if (is_uprobe_session(prog)) + uprobes[i].session = true; if (pid) uprobes[i].consumer.filter = uprobe_multi_link_filter; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 7610883c8191..09bdb1867d4a 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1116,6 +1116,7 @@ enum bpf_attach_type { BPF_NETKIT_PRIMARY, BPF_NETKIT_PEER, BPF_TRACE_KPROBE_SESSION, + BPF_TRACE_UPROBE_SESSION, __MAX_BPF_ATTACH_TYPE }; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 05ad264ff09b..1caa9d20de5d 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -133,6 +133,7 @@ static const char * const attach_type_name[] = { [BPF_NETKIT_PRIMARY] = "netkit_primary", [BPF_NETKIT_PEER] = "netkit_peer", [BPF_TRACE_KPROBE_SESSION] = "trace_kprobe_session", + [BPF_TRACE_UPROBE_SESSION] = "trace_uprobe_session", }; static const char * const link_type_name[] = { From patchwork Thu Oct 10 20:09:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831033 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E97F1CFEC9; Thu, 10 Oct 2024 20:11:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591077; cv=none; b=FWqzfHIvHOuB3Wt01F8h6YwpgEze6wZQiJwYa+pRcNl2dqNlE3cABgHZbTXo4eomQFPnBDREOBpc1rBt6/13ZjPR/6yyPHVC3KiFbF7RFwQrCHaTkd0rMEK1YGObUHH3GfawJxwCzOx+VkacT+evvaMFxuY1MoSAvnUhejbUkUM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591077; c=relaxed/simple; bh=EZ76JiF8nHrXB5Od2azxqfLxETX83Z2xeo/fMdQY5Ac=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p6idl4gxxxsUhk2STB9aoNgOs6QBeRCbguS+yMjeQDDAr5kD4x29tuFcoOfS3TsWNIEs9/p7emL4n9ns1bKbudAvCstyU4cJ4xO6mnLLD8n8r4OA/Cpds9oeay5jdffh18zQ8Qhx1kSqNmxc6AZ3vfSMPpF3sdFbtoeK2pP7nTw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cF2ErTXG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cF2ErTXG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2246CC4CEC5; Thu, 10 Oct 2024 20:11:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591077; bh=EZ76JiF8nHrXB5Od2azxqfLxETX83Z2xeo/fMdQY5Ac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cF2ErTXGVbFDjXDANDHwP0gBUVrnmRBbDVo6HY/7zOha/O3Y6bIkkOgPpH9in9LFp r4mh65sxyficJylPHDwQIOyKJpOtN9mZkczejeouF/FM2ip1oTTE6T+4JaE1F/Zsy5 Nn3uHtG4X2kqt2+YmF4SkUVMtG4BFCt/RAEXUNiLkSu07pJ4rpNdOI/g565di9Y4aO npKjrqbC2PthmQBqUGnFTTyc+nN00HNajDuqb9Z5HhDGIjPsHiZ4LsHI031E5LC2ji OH1z9HPQhw8f6+R0PxsIu5e82GKsvO3AFQwnjmREjrZu3XOV0MtAuR08V7hnpggt8g UoYvOLOMMkbIQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 06/16] bpf: Add support for uprobe multi session context Date: Thu, 10 Oct 2024 22:09:47 +0200 Message-ID: <20241010200957.2750179-7-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_uprobe_multi_run_ctx, so the session data can be retrieved from uprobe_multi link. Plus granting session kfuncs access to uprobe session programs. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- kernel/trace/bpf_trace.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 5f10994376d0..01868039d7bc 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3098,7 +3098,7 @@ struct bpf_uprobe_multi_link { }; struct bpf_uprobe_multi_run_ctx { - struct bpf_run_ctx run_ctx; + struct bpf_session_run_ctx session_ctx; unsigned long entry_ip; struct bpf_uprobe *uprobe; }; @@ -3211,16 +3211,22 @@ static const struct bpf_link_ops bpf_uprobe_multi_link_lops = { static int uprobe_prog_run(struct bpf_uprobe *uprobe, unsigned long entry_ip, - struct pt_regs *regs) + struct pt_regs *regs, + bool is_return, void *data) { struct bpf_uprobe_multi_link *link = uprobe->link; struct bpf_uprobe_multi_run_ctx run_ctx = { + .session_ctx = { + .is_return = is_return, + .data = data, + }, .entry_ip = entry_ip, .uprobe = uprobe, }; struct bpf_prog *prog = link->link.prog; bool sleepable = prog->sleepable; struct bpf_run_ctx *old_run_ctx; + int err; if (link->task && !same_thread_group(current, link->task)) return 0; @@ -3232,8 +3238,8 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, migrate_disable(); - old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); - bpf_prog_run(link->link.prog, regs); + old_run_ctx = bpf_set_run_ctx(&run_ctx.session_ctx.run_ctx); + err = bpf_prog_run(link->link.prog, regs); bpf_reset_run_ctx(old_run_ctx); migrate_enable(); @@ -3242,7 +3248,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, rcu_read_unlock_trace(); else rcu_read_unlock(); - return 0; + return err; } static bool @@ -3262,7 +3268,7 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, int ret; uprobe = container_of(con, struct bpf_uprobe, consumer); - ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs, false, data); if (uprobe->session) return ret ? UPROBE_HANDLER_IGNORE : 0; return 0; @@ -3275,7 +3281,7 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s struct bpf_uprobe *uprobe; uprobe = container_of(con, struct bpf_uprobe, consumer); - uprobe_prog_run(uprobe, func, regs); + uprobe_prog_run(uprobe, func, regs, true, data); return 0; } @@ -3283,7 +3289,8 @@ static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; - run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, + session_ctx.run_ctx); return run_ctx->entry_ip; } @@ -3291,7 +3298,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; - run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, + session_ctx.run_ctx); return run_ctx->uprobe->cookie; } @@ -3485,7 +3493,7 @@ static int bpf_kprobe_multi_filter(const struct bpf_prog *prog, u32 kfunc_id) if (!btf_id_set8_contains(&kprobe_multi_kfunc_set_ids, kfunc_id)) return 0; - if (!is_kprobe_session(prog)) + if (!is_kprobe_session(prog) && !is_uprobe_session(prog)) return -EACCES; return 0; From patchwork Thu Oct 10 20:09:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831034 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 657CD1D0481; Thu, 10 Oct 2024 20:11:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591089; cv=none; b=JhCrLi4F2nx0S3Cvk3h6x5bCcEHcS9gcxmQPmVse/HuhWgwsmv3to382LzdDeHIuPJ2m7nRSTrzEoBL5bsQ/++61JXmaCcoDzzFiO+3+DsiYUoddYYhCabeq7mYgDlK4HusItSiRtQQ9LMX482hYl59wNr+BJh3gbiMTAIZUlFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591089; c=relaxed/simple; bh=4azW8qaKhxfctdFyV9Az0HnyCyaJEks76fy52saE/FE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aZmmcXexLSwcOlDR8/8hPWCRfCFVkOHSV2XGkgrYxlSmCdGXAlQiwpvzE/Tm2+5fF0mN7aIh5KHy/+tIi+VQMKomw8iEAbPIER1oZyzjq0NdlF9QzGcxzYyodormzLqS9fhdNeRYRrHh5SnEmQ8F2zBfpVz//Y2F1+ZFnSFQla0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=X0ZoDToZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="X0ZoDToZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9557DC4CEC5; Thu, 10 Oct 2024 20:11:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591089; bh=4azW8qaKhxfctdFyV9Az0HnyCyaJEks76fy52saE/FE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X0ZoDToZRCHEYpfX6cPlSkC7EiiTCqDEHBA1V7/obCDtODh1z1AIwNzahS0JnTUQu Z9I69WXZ1u+7Taqqnzn3UQN3zPitpfbO6fy4VCUA0bvRL/UOKacObapaz1cbKGLyEU 4YFv6/pkZ0PLonO4FuXxrYoSbxm4Tdu5+b+PSXL/BKa6Du34reIuobp7V46HnmTANv RnTa/d/xqT2nJQ53OzA0pcvBvDLQxLxl223SLOONO+ridAEZUJYdFf5pv72qX38KSM VN6lo2wIoEr7qohmBcf0Sv9PetFcIiZp0mAAkPYF/fO5nUKODEEQjWWp1iNACGRYRd djG/cUgunROWA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 07/16] libbpf: Add support for uprobe multi session attach Date: Thu, 10 Oct 2024 22:09:48 +0200 Message-ID: <20241010200957.2750179-8-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support to attach program in uprobe session mode with bpf_program__attach_uprobe_multi function. Adding session bool to bpf_uprobe_multi_opts struct that allows to load and attach the bpf program via uprobe session. the attachment to create uprobe multi session. Also adding new program loader section that allows: SEC("uprobe.session/bpf_fentry_test*") and loads/attaches uprobe program as uprobe session. Adding sleepable hook (uprobe.session.s) as well. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 18 ++++++++++++++++-- tools/lib/bpf/libbpf.h | 4 +++- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2a4c71501a17..becdfa701c75 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -776,6 +776,7 @@ int bpf_link_create(int prog_fd, int target_fd, return libbpf_err(-EINVAL); break; case BPF_TRACE_UPROBE_MULTI: + case BPF_TRACE_UPROBE_SESSION: attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0); attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0); attr.link_create.uprobe_multi.path = ptr_to_u64(OPTS_GET(opts, uprobe_multi.path, 0)); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 1caa9d20de5d..cbaa8b8a824f 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -9410,8 +9410,10 @@ static const struct bpf_sec_def section_defs[] = { SEC_DEF("kprobe.session+", KPROBE, BPF_TRACE_KPROBE_SESSION, SEC_NONE, attach_kprobe_session), SEC_DEF("uprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), SEC_DEF("uretprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), + SEC_DEF("uprobe.session+", KPROBE, BPF_TRACE_UPROBE_SESSION, SEC_NONE, attach_uprobe_multi), SEC_DEF("uprobe.multi.s+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi), SEC_DEF("uretprobe.multi.s+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi), + SEC_DEF("uprobe.session.s+", KPROBE, BPF_TRACE_UPROBE_SESSION, SEC_SLEEPABLE, attach_uprobe_multi), SEC_DEF("ksyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("kretsyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_USDT, attach_usdt), @@ -11733,7 +11735,9 @@ static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, stru ret = 0; break; case 3: + opts.session = str_has_pfx(probe_type, "uprobe.session"); opts.retprobe = str_has_pfx(probe_type, "uretprobe.multi"); + *link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts); ret = libbpf_get_error(*link); break; @@ -11982,10 +11986,12 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL; LIBBPF_OPTS(bpf_link_create_opts, lopts); unsigned long *resolved_offsets = NULL; + enum bpf_attach_type attach_type; int err = 0, link_fd, prog_fd; struct bpf_link *link = NULL; char errmsg[STRERR_BUFSIZE]; char full_path[PATH_MAX]; + bool retprobe, session; const __u64 *cookies; const char **syms; size_t cnt; @@ -12056,12 +12062,20 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, offsets = resolved_offsets; } + retprobe = OPTS_GET(opts, retprobe, false); + session = OPTS_GET(opts, session, false); + + if (retprobe && session) + return libbpf_err_ptr(-EINVAL); + + attach_type = session ? BPF_TRACE_UPROBE_SESSION : BPF_TRACE_UPROBE_MULTI; + lopts.uprobe_multi.path = path; lopts.uprobe_multi.offsets = offsets; lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; lopts.uprobe_multi.cookies = cookies; lopts.uprobe_multi.cnt = cnt; - lopts.uprobe_multi.flags = OPTS_GET(opts, retprobe, false) ? BPF_F_UPROBE_MULTI_RETURN : 0; + lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0; if (pid == 0) pid = getpid(); @@ -12075,7 +12089,7 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, } link->detach = &bpf_link__detach_fd; - link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts); + link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts); if (link_fd < 0) { err = -errno; pr_warn("prog '%s': failed to attach multi-uprobe: %s\n", diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 91484303849c..b2ce3a72b11d 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -577,10 +577,12 @@ struct bpf_uprobe_multi_opts { size_t cnt; /* create return uprobes */ bool retprobe; + /* create session kprobes */ + bool session; size_t :0; }; -#define bpf_uprobe_multi_opts__last_field retprobe +#define bpf_uprobe_multi_opts__last_field session /** * @brief **bpf_program__attach_uprobe_multi()** attaches a BPF program From patchwork Thu Oct 10 20:09:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831035 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B3721E32D4; Thu, 10 Oct 2024 20:11:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591101; cv=none; b=hlXW/Kn2Qatk36f2Q56kTkXfCBsV15yx/3WHP+0atXE5CSlDs/UAR6r7Ieb3/3F2ZoeXwxd5x6aqyQuSc3uk/UQjPM0IbiS5dJ7eRjqsGEy91/S+I777ELoNCz5KrfdRbCbxr9zIBmF0k5K/Srp5XDR8ObByfe+rKvpCY2pG/LE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591101; c=relaxed/simple; bh=GYGtF9yDfymiuEvjDMpm+NelQffajB36h2rBIDSO+Sw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gpceNNCgO4zk3ts3ePeK+b4E6gTTcgORK6uXGOgh6Oo/faybhfU3hr8lIcHemzlLl4+6iFKHL0OmUxuQhqpFKKJPECWpU7GEghZfuhDuM4lSZa1pnQgE+2DWVxX4740cW0EWpwTQw5UEXqmdGXrSyunKW2kDkIpYo146L0h+NSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Z5lzm4Jg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Z5lzm4Jg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51C41C4CECC; Thu, 10 Oct 2024 20:11:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591100; bh=GYGtF9yDfymiuEvjDMpm+NelQffajB36h2rBIDSO+Sw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z5lzm4JguS2XbAp/fWMjp72NZgC+85khA5GEVxcwEm7j39TS750tfUVF4Zf3tJ7mP IjOGOpPTYs3yqxNy0Ig+ONxOqNpqJzqELbhMa80FuFw13GBQR+FIZtIJi5thnmv9EQ 6NPAhCODiC2FqDwwGRMfJnlfgIM7mKlZUcWdqHWJgAIYFbpcaxmzU2bqyezhVEhU7H A9ni12M64c8W52EZNjxcC4bEyBwMA4mSHJwXshw8eXquj0xjP0r7FvyOe3ysgXXGqY +Q0JRhW/VA2HPO54s4jZ7qQl/VM1gMcFhVLWrOBK3+FUQ1bNjXntz0XG48kFFwIDhZ S0rD5TjvZ2YOQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 08/16] selftests/bpf: Add uprobe session test Date: Thu, 10 Oct 2024 22:09:49 +0200 Message-ID: <20241010200957.2750179-9-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test and testing that the entry program return value controls execution of the return probe program. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/uprobe_multi_test.c | 47 ++++++++++++ .../bpf/progs/uprobe_multi_session.c | 71 +++++++++++++++++++ 2 files changed, 118 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 2c39902b8a09..b10d2dadb462 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -8,6 +8,7 @@ #include "uprobe_multi_usdt.skel.h" #include "uprobe_multi_consumers.skel.h" #include "uprobe_multi_pid_filter.skel.h" +#include "uprobe_multi_session.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -1015,6 +1016,50 @@ static void test_pid_filter_process(bool clone_vm) uprobe_multi_pid_filter__destroy(skel); } +static void test_session_skel_api(void) +{ + struct uprobe_multi_session *skel = NULL; + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + struct bpf_link *link = NULL; + int err; + + skel = uprobe_multi_session__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + skel->bss->user_ptr = test_data; + + err = uprobe_multi_session__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session__attach")) + goto cleanup; + + /* trigger all probes */ + skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3; + + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + /* + * We expect 2 for uprobe_multi_func_2 because it runs both entry/return probe, + * uprobe_multi_func_[13] run just the entry probe. All expected numbers are + * doubled, because we run extra test for sleepable session. + */ + ASSERT_EQ(skel->bss->uprobe_session_result[0], 2, "uprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uprobe_session_result[1], 4, "uprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uprobe_session_result[2], 2, "uprobe_multi_func_3_result"); + + /* We expect increase in 3 entry and 1 return session calls -> 4 */ + ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 4, "uprobe_multi_sleep_result"); + +cleanup: + bpf_link__destroy(link); + uprobe_multi_session__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -1111,4 +1156,6 @@ void test_uprobe_multi_test(void) test_pid_filter_process(false); if (test__start_subtest("filter_clone_vm")) test_pid_filter_process(true); + if (test__start_subtest("session")) + test_session_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session.c new file mode 100644 index 000000000000..30bff90b68dc --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__u64 uprobe_multi_func_1_addr = 0; +__u64 uprobe_multi_func_2_addr = 0; +__u64 uprobe_multi_func_3_addr = 0; + +__u64 uprobe_session_result[3] = {}; +__u64 uprobe_multi_sleep_result = 0; + +void *user_ptr = 0; +int pid = 0; + +static int uprobe_multi_check(void *ctx, bool is_return) +{ + const __u64 funcs[] = { + uprobe_multi_func_1_addr, + uprobe_multi_func_2_addr, + uprobe_multi_func_3_addr, + }; + unsigned int i; + __u64 addr; + + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + addr = bpf_get_func_ip(ctx); + + for (i = 0; i < ARRAY_SIZE(funcs); i++) { + if (funcs[i] == addr) { + uprobe_session_result[i]++; + break; + } + } + + /* only uprobe_multi_func_2 executes return probe */ + if ((addr == uprobe_multi_func_1_addr) || + (addr == uprobe_multi_func_3_addr)) + return 1; + + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_*") +int uprobe(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, bpf_session_is_return()); +} + +static __always_inline bool verify_sleepable_user_copy(void) +{ + char data[9]; + + bpf_copy_from_user(data, sizeof(data), user_ptr); + return bpf_strncmp(data, sizeof(data), "test_data") == 0; +} + +SEC("uprobe.session.s//proc/self/exe:uprobe_multi_func_*") +int uprobe_sleepable(struct pt_regs *ctx) +{ + if (verify_sleepable_user_copy()) + uprobe_multi_sleep_result++; + return uprobe_multi_check(ctx, bpf_session_is_return()); +} From patchwork Thu Oct 10 20:09:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831036 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AD5A188A08; Thu, 10 Oct 2024 20:11:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591112; cv=none; b=a/6SXmm/qm7isxIF4PzC4wLFhloNYjdC06Amf7YmCM/QGBbYe/hiiFccQm9DZ5j+qBfTMLL+7x5x43SN7F8dL85Hvf2AGrQ5pYyl8WzVWKDwfpgcbVZ/6Sb51Pef1HET9Uvhd3g6XdBsqPT9euoHMDsDOsQaYSAs9KV7Z7m+fsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591112; c=relaxed/simple; bh=tiFWq4stA78Nh3pJt3/jzsTBQ26T8kB6IeqRBp9iaoQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eKb6ig88HhNcqhMLWLbeHii54kTTV6VFNsouwdfQx+UQI7KR9kdS2hkJuvA0aftM8EFmJifYziqvvEgRdCW99xi3R5xEIaXuTEyYTXTdYbkavu82H5VYha6h8O9KrIPzNDj4W3ds1F61PcTyIqF0dOptJzg18XSPwUMPsmceCyM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TTVCtqLU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TTVCtqLU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9A8EC4CEC5; Thu, 10 Oct 2024 20:11:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591112; bh=tiFWq4stA78Nh3pJt3/jzsTBQ26T8kB6IeqRBp9iaoQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TTVCtqLUpTkTKjTxHBaEZ+2zGtm9EKin56+oNdrXhqL6ai2Fw4NOus6mfxKzsuqjs t2SLu4r1+2db/TVY6b+6I8lM10myZddmAgPaBRP7KLZBVwYxIsVPw2nc7gtWMC7fEd IoX/T4UzC781t+ypY8ke64dx3qTf9I9USs9l6fqnv6mxEgao1JuC1q/wSLJTjlVVQ/ cPca9I36/6MUAVv+etn23Ia1yLhBS5o83Ji7/XNZuShin074062esCUK65e/7hTZwy eiSOUTl5FhYxhs8gjPVmATxP6tJMkykSFDxf5Gj9uQJLs7lVs4K3j6MDcYdwipUSwL VLCqJ9mi9lJhg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 09/16] selftests/bpf: Add uprobe session cookie test Date: Thu, 10 Oct 2024 22:09:50 +0200 Message-ID: <20241010200957.2750179-10-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test that verifies the cookie value get properly propagated from entry to return program. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 31 ++++++++++++ .../bpf/progs/uprobe_multi_session_cookie.c | 48 +++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index b10d2dadb462..cc9030e86821 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -9,6 +9,7 @@ #include "uprobe_multi_consumers.skel.h" #include "uprobe_multi_pid_filter.skel.h" #include "uprobe_multi_session.skel.h" +#include "uprobe_multi_session_cookie.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -1060,6 +1061,34 @@ static void test_session_skel_api(void) uprobe_multi_session__destroy(skel); } +static void test_session_cookie_skel_api(void) +{ + struct uprobe_multi_session_cookie *skel = NULL; + int err; + + skel = uprobe_multi_session_cookie__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_cookie__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_cookie__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_cookie__attach")) + goto cleanup; + + /* trigger all probes */ + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + ASSERT_EQ(skel->bss->test_uprobe_1_result, 1, "test_uprobe_1_result"); + ASSERT_EQ(skel->bss->test_uprobe_2_result, 2, "test_uprobe_2_result"); + ASSERT_EQ(skel->bss->test_uprobe_3_result, 3, "test_uprobe_3_result"); + +cleanup: + uprobe_multi_session_cookie__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -1158,4 +1187,6 @@ void test_uprobe_multi_test(void) test_pid_filter_process(true); if (test__start_subtest("session")) test_session_skel_api(); + if (test__start_subtest("session_cookie")) + test_session_cookie_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c new file mode 100644 index 000000000000..5befdf944dc6 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" + +char _license[] SEC("license") = "GPL"; + +int pid = 0; + +__u64 test_uprobe_1_result = 0; +__u64 test_uprobe_2_result = 0; +__u64 test_uprobe_3_result = 0; + +static int check_cookie(__u64 val, __u64 *result) +{ + __u64 *cookie; + + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + cookie = bpf_session_cookie(); + + if (bpf_session_is_return()) + *result = *cookie == val ? val : 0; + else + *cookie = val; + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_1(struct pt_regs *ctx) +{ + return check_cookie(1, &test_uprobe_1_result); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_2") +int uprobe_2(struct pt_regs *ctx) +{ + return check_cookie(2, &test_uprobe_2_result); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_3") +int uprobe_3(struct pt_regs *ctx) +{ + return check_cookie(3, &test_uprobe_3_result); +} From patchwork Thu Oct 10 20:09:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831037 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83EB61CFECC; Thu, 10 Oct 2024 20:12:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591123; cv=none; b=RPRWqGNLqpsQOkuYJZUqCSlHNXEwy9dqPxRgqReiwzHR7qLKnwXyuG3swige97NroJiULQ4EjrZlgnIimD0OXgcaXtHlkBb/Lb+Hix4kZsWuj/zOghusQIjTkinvGnrpy8smYdNZJLG68+Ogdm8x8WKcCHil/n1ZLlQBcf9vLuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591123; c=relaxed/simple; bh=e/JgbYiCi/7UBhwN/I0I8AkHVXsJpTb8JQnSwgJINjw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VhcXOZ2apfUZ0AQGhc5Q8BtE8kKoYtCCGc0eVLnegrsu9z7ZYoxRzE9UCc2JgLl5VWoWwhTMYrOec/il5Yqzc2UHYK2EXICkI9h3HeiFuaYFwZnn1eMaMo6frjK/1P0Uhm0JnhbMQPBZ3vUsyMXkxCB9Gzbwcc0Fya0yclPquok= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZRp0S40x; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZRp0S40x" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E8DDC4CEC5; Thu, 10 Oct 2024 20:11:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591123; bh=e/JgbYiCi/7UBhwN/I0I8AkHVXsJpTb8JQnSwgJINjw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZRp0S40xqNZg4jd57AO5bUhsl49zeNyptCQfzVk13Euv5RP2VtvPweli/qxGktfKx DbZ1brhvGcjuURcDRNVJb9saA3gGRa2MeMu4J0Jw2YKr50VMamvPr3426CPQDy0zBK W5MCnDO8QP+MQ/d7mA01xMYqUVYSY746Cg9BWV3aqui+2vIUVMWYdpgRsYIntBZPrI KEk+WjC13PJABXSKNMbyfXlNyv+Dq7IDXqjPCITBVB2b5lLeHd13MkvL9BuWavGSiy YQPdf0vQePh0A8kB14XRAFOcmSYJpyWbn8Yx1YAzS4k/jK91jVZOoEaE3blx77H+Qy H3YbbvTWGlncA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 10/16] selftests/bpf: Add uprobe session recursive test Date: Thu, 10 Oct 2024 22:09:51 +0200 Message-ID: <20241010200957.2750179-11-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test that verifies the cookie value is stored properly when single uprobe-ed function is executed recursively. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 57 +++++++++++++++++++ .../progs/uprobe_multi_session_recursive.c | 44 ++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index cc9030e86821..284cd7fce576 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -10,6 +10,7 @@ #include "uprobe_multi_pid_filter.skel.h" #include "uprobe_multi_session.skel.h" #include "uprobe_multi_session_cookie.skel.h" +#include "uprobe_multi_session_recursive.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -36,6 +37,12 @@ noinline void usdt_trigger(void) STAP_PROBE(test, pid_filter_usdt); } +noinline void uprobe_session_recursive(int i) +{ + if (i) + uprobe_session_recursive(i - 1); +} + struct child { int go[2]; int c2p[2]; /* child -> parent channel */ @@ -1089,6 +1096,54 @@ static void test_session_cookie_skel_api(void) uprobe_multi_session_cookie__destroy(skel); } +static void test_session_recursive_skel_api(void) +{ + struct uprobe_multi_session_recursive *skel = NULL; + int i, err; + + skel = uprobe_multi_session_recursive__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_recursive__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_recursive__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_recursive__attach")) + goto cleanup; + + for (i = 0; i < ARRAY_SIZE(skel->bss->test_uprobe_cookie_entry); i++) + skel->bss->test_uprobe_cookie_entry[i] = i + 1; + + uprobe_session_recursive(5); + + /* + * entry uprobe: + * uprobe_session_recursive(5) { *cookie = 1, return 0 + * uprobe_session_recursive(4) { *cookie = 2, return 1 + * uprobe_session_recursive(3) { *cookie = 3, return 0 + * uprobe_session_recursive(2) { *cookie = 4, return 1 + * uprobe_session_recursive(1) { *cookie = 5, return 0 + * uprobe_session_recursive(0) { *cookie = 6, return 1 + * return uprobe: + * } i = 0 not executed + * } i = 1 test_uprobe_cookie_return[0] = 5 + * } i = 2 not executed + * } i = 3 test_uprobe_cookie_return[1] = 3 + * } i = 4 not executed + * } i = 5 test_uprobe_cookie_return[2] = 1 + */ + + ASSERT_EQ(skel->bss->idx_entry, 6, "idx_entry"); + ASSERT_EQ(skel->bss->idx_return, 3, "idx_return"); + + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[0], 5, "test_uprobe_cookie_return[0]"); + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[1], 3, "test_uprobe_cookie_return[1]"); + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[2], 1, "test_uprobe_cookie_return[2]"); + +cleanup: + uprobe_multi_session_recursive__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -1189,4 +1244,6 @@ void test_uprobe_multi_test(void) test_session_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); + if (test__start_subtest("session_cookie_recursive")) + test_session_recursive_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c new file mode 100644 index 000000000000..8fbcd69fae22 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +int pid = 0; + +int idx_entry = 0; +int idx_return = 0; + +__u64 test_uprobe_cookie_entry[6]; +__u64 test_uprobe_cookie_return[3]; + +static int check_cookie(void) +{ + __u64 *cookie = bpf_session_cookie(); + + if (bpf_session_is_return()) { + if (idx_return >= ARRAY_SIZE(test_uprobe_cookie_return)) + return 1; + test_uprobe_cookie_return[idx_return++] = *cookie; + return 0; + } + + if (idx_entry >= ARRAY_SIZE(test_uprobe_cookie_entry)) + return 1; + *cookie = test_uprobe_cookie_entry[idx_entry]; + return idx_entry++ % 2; +} + + +SEC("uprobe.session//proc/self/exe:uprobe_session_recursive") +int uprobe_recursive(struct pt_regs *ctx) +{ + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + return check_cookie(); +} From patchwork Thu Oct 10 20:09:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831038 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDA471CF28E; Thu, 10 Oct 2024 20:12:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591135; cv=none; b=ZMEtFbqrBTUlopSXNkZFb2EjOld0XrCwH0doPj/S6vZaGcLqBGT+lf2ODcFRPIWxlKiqnbXMGjPbD5wzyGx01UAynrCam1cOaG/zefmYGk7wX04WH3n683G/Wgn/AjNv0i0T8mC+uFguKl9kx41OqaMZih+BNJFmY2oGUMbetb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591135; c=relaxed/simple; bh=2ZRqx+7SKkGoE3vDYuntBskDrfcteLP+fK9qAu1N+7w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rC01sEY0nW2PoVYQ8C9iQSGgC8M0q/L6Q6N0DUmbQOF9vEpqsZ35p/ySW+2HeHoUp5pOlLWGbZc1KF0fEN9VWMLQNjVxB3TEZirg2c1UCykZMJBO01DLVOyi/Qnp1NkHhlzN8FJkQJ8KfJM7IdiJ4syjrGR0eIfTQo3g1UeW8l0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IXF+l5Kg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IXF+l5Kg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88241C4CEC5; Thu, 10 Oct 2024 20:12:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591134; bh=2ZRqx+7SKkGoE3vDYuntBskDrfcteLP+fK9qAu1N+7w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IXF+l5KgHIrWC88gjz0MOsAVYL07tDV/cbMfk7Ii/V1dmwY8+YSeZVqlorA50OTk/ ewuxf8F0JZnR6yhf4E/JTTKajsTAztSR6+FDfXaNZGohYy6kDHtOMdZIORx5JIHjGQ d6avEIl63Z05h3l6iGAbTbm38GebKNKcYgbYfSK92FFZ0TQ/84vERu2pWsLPcY3xiT qEFpJGO+x6e3MKVug4xjibBiKE/lHhV+V2EBjppYiBxHN5D6iID6R88MFzIzBFmcSI YbsfKH6CIC3gqe8X8zMGh1y2nCuWy3aU/UwzorH0qSZVkISks4PdxLOY1ArA0a9wrb VJQXWklGZnZmA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 11/16] selftests/bpf: Add uprobe session verifier test for return value Date: Thu, 10 Oct 2024 22:09:52 +0200 Message-ID: <20241010200957.2750179-12-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Making sure uprobe.session program can return only [0,1] values. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/uprobe_multi_test.c | 2 ++ .../bpf/progs/uprobe_multi_verifier.c | 31 +++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 284cd7fce576..e693eeb1a5a5 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -11,6 +11,7 @@ #include "uprobe_multi_session.skel.h" #include "uprobe_multi_session_cookie.skel.h" #include "uprobe_multi_session_recursive.skel.h" +#include "uprobe_multi_verifier.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -1246,4 +1247,5 @@ void test_uprobe_multi_test(void) test_session_cookie_skel_api(); if (test__start_subtest("session_cookie_recursive")) test_session_recursive_skel_api(); + RUN_TESTS(uprobe_multi_verifier); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c b/tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c new file mode 100644 index 000000000000..fe49f2cb5360 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + + +SEC("uprobe.session") +__success +int uprobe_sesison_return_0(struct pt_regs *ctx) +{ + return 0; +} + +SEC("uprobe.session") +__success +int uprobe_sesison_return_1(struct pt_regs *ctx) +{ + return 1; +} + +SEC("uprobe.session") +__failure +__msg("At program exit the register R0 has smin=2 smax=2 should have been in [0, 1]") +int uprobe_sesison_return_2(struct pt_regs *ctx) +{ + return 2; +} From patchwork Thu Oct 10 20:09:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831039 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87B6C15B0F2; Thu, 10 Oct 2024 20:12:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591146; cv=none; b=cC/7MHPOvZnUaxffOTYld3Ya1yV2Bi9tdjcEy/QQdl/6NWmSpLVWr9UjLwzoGbQx2+wA3ez9tCX582NxUu7whGPVKdScgtCBdK+24pY2fgHyl7qb23KBb65Sbq9iG1E+bOFhLxTUPXsHDb4/U/D84Nckl0PK5MLTF9Od1vaHBd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591146; c=relaxed/simple; bh=uSy+gDXv0e/W1F1L/aoCy9/PnJuM+20kIhQfaP/dM28=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NAFPlODblrd2DMFeOalpOeXuxk9pRM8AFcU+VQr9w6xiG+kok3G6o2waTmEX3dpEbPAuR/BzLdhKJeY7KNOvAsbsYuNbopoRLPk0wwwi9RViv/lZ9oQrBueOeY/pPQV4CMaZdYEKZhm3xtXfTtk9kQf01/VPNx6wgV4Kopr+TBY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LdCho8uA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LdCho8uA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23738C4CEC5; Thu, 10 Oct 2024 20:12:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591146; bh=uSy+gDXv0e/W1F1L/aoCy9/PnJuM+20kIhQfaP/dM28=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LdCho8uAmcGy7zbbFuPaDwPil3QAbB/vPEpz77CckmA+FOZT7KHHkkP6pOHDmNBiS WIFwczjPvvcNbMU9zIrUH+9nfXxX7KOexYTKVk5I7Gl3h6ALG9PPQFx4hd/8cI8Yh9 51SXbDakdTA2METgJiwH3WLFHXDi1tRC/dg0plzAmNCaMSjmuB+H9G/UH270q9yxxR I49M7vNFEirfsiJR8iWp2pw/rT0PP+RK+nHW9vz1K+c8ydjjPyRq1ZhXv/GXYr54Uy 4oxDQAoofWrc0qtqqvUey7AOLfXc89sa5qlzuuktP9gC6nyV2N9Zz/H8p+isCLQPxB TOeXkbb9TcZkA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 12/16] selftests/bpf: Add kprobe session verifier test for return value Date: Thu, 10 Oct 2024 22:09:53 +0200 Message-ID: <20241010200957.2750179-13-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Making sure kprobe.session program can return only [0,1] values. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/kprobe_multi_test.c | 2 ++ .../bpf/progs/kprobe_multi_verifier.c | 31 +++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c index 960c9323d1e0..66ab1cae923e 100644 --- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c @@ -6,6 +6,7 @@ #include "kprobe_multi_override.skel.h" #include "kprobe_multi_session.skel.h" #include "kprobe_multi_session_cookie.skel.h" +#include "kprobe_multi_verifier.skel.h" #include "bpf/libbpf_internal.h" #include "bpf/hashmap.h" @@ -764,4 +765,5 @@ void test_kprobe_multi_test(void) test_session_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); + RUN_TESTS(kprobe_multi_verifier); } diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c b/tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c new file mode 100644 index 000000000000..288577e81deb --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + + +SEC("kprobe.session") +__success +int kprobe_session_return_0(struct pt_regs *ctx) +{ + return 0; +} + +SEC("kprobe.session") +__success +int kprobe_session_return_1(struct pt_regs *ctx) +{ + return 1; +} + +SEC("kprobe.session") +__failure +__msg("At program exit the register R0 has smin=2 smax=2 should have been in [0, 1]") +int kprobe_session_return_2(struct pt_regs *ctx) +{ + return 2; +} From patchwork Thu Oct 10 20:09:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831040 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FC631D31B2; Thu, 10 Oct 2024 20:12:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591158; cv=none; b=hq5mfOUp5KwgIxG/zoJBEtxjlNz+IjSOWeFiqnHEW6VHOjekRnoVvKnJj2jN4Eekf1+plD9nTaZAlOckpwAOp+Zz5Ac0AzC4gPZ0oeEJTRIlgBG7qhUZTBobONixgBYiJf2wqlxSvACkRqfIo0Rkp2oZ13faquW/yvztCEMSa5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591158; c=relaxed/simple; bh=M1A+CmFYkP0eRr/nM49rcC/LF5rRjCJFgMXb1JJxegA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IRgLIjYWR/3zuxjgaeQOcn3bcxFfINNt7W2NTTQEZh+Xyu3t5sI6KY1wwluflcuole2/Ww77bLQvaBHwnqySS/lP7bEGh6B8ryCRKna99TLl6NnuuCNs06gMsQjMxgWEX7u6tn1n2+qX2+QNly54Fl1wRQ/dd49h4SG58hGdCZ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jtnjfCWi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jtnjfCWi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5B83C4CEC5; Thu, 10 Oct 2024 20:12:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591158; bh=M1A+CmFYkP0eRr/nM49rcC/LF5rRjCJFgMXb1JJxegA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jtnjfCWiAsfefr1EJA2QKM0PAxsytA7buoyDdGiT1whtKMnnDks6W5xE62TEJKQuv hGNRZ5+aDRWjhtSkvC6g5nko9rV05dc5GXm1iVOs6k2zsI/szzT28Beqovnd+CmZKV C9sXZIHHcPoADHi6mjdPAHabqEM3zk61iFD8y7ze3RuHxZPcAf4f6kueg0qMN6SqD0 +TZJaGH+zJU8d01xjK7AIetlLCaAbopUbp4BvV93vpkXXwdgivzyUC+h2jmmd2kysL kYANaNE+847O2RgStBk/Aj48qSmxvgl4NdHMJlGXe01S4GTdV+CnWiiupEkliBqu3+ QS8PVydaYZzTQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 13/16] selftests/bpf: Add uprobe session single consumer test Date: Thu, 10 Oct 2024 22:09:54 +0200 Message-ID: <20241010200957.2750179-14-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Testing that the session ret_handler bypass works on single uprobe with multiple consumers, each with different session ignore return value. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/uprobe_multi_test.c | 33 ++++++++++++++ .../bpf/progs/uprobe_multi_session_single.c | 44 +++++++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index e693eeb1a5a5..7e0228f8fcfc 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -9,6 +9,7 @@ #include "uprobe_multi_consumers.skel.h" #include "uprobe_multi_pid_filter.skel.h" #include "uprobe_multi_session.skel.h" +#include "uprobe_multi_session_single.skel.h" #include "uprobe_multi_session_cookie.skel.h" #include "uprobe_multi_session_recursive.skel.h" #include "uprobe_multi_verifier.skel.h" @@ -1069,6 +1070,36 @@ static void test_session_skel_api(void) uprobe_multi_session__destroy(skel); } +static void test_session_single_skel_api(void) +{ + struct uprobe_multi_session_single *skel = NULL; + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + int err; + + skel = uprobe_multi_session_single__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_single__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_single__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_single__attach")) + goto cleanup; + + uprobe_multi_func_1(); + + /* + * We expect consumer 0 and 2 to trigger just entry handler (value 1) + * and consumer 1 to hit both (value 2). + */ + ASSERT_EQ(skel->bss->uprobe_session_result[0], 1, "uprobe_session_result_0"); + ASSERT_EQ(skel->bss->uprobe_session_result[1], 2, "uprobe_session_result_1"); + ASSERT_EQ(skel->bss->uprobe_session_result[2], 1, "uprobe_session_result_2"); + +cleanup: + uprobe_multi_session_single__destroy(skel); +} + static void test_session_cookie_skel_api(void) { struct uprobe_multi_session_cookie *skel = NULL; @@ -1243,6 +1274,8 @@ void test_uprobe_multi_test(void) test_pid_filter_process(true); if (test__start_subtest("session")) test_session_skel_api(); + if (test__start_subtest("session_single")) + test_session_single_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); if (test__start_subtest("session_cookie_recursive")) diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c new file mode 100644 index 000000000000..1fa53d3785f6 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__u64 uprobe_session_result[3] = {}; +int pid = 0; + +static int uprobe_multi_check(void *ctx, bool is_return, int idx) +{ + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + uprobe_session_result[idx]++; + + /* only consumer 1 executes return probe */ + if (idx == 0 || idx == 2) + return 1; + + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_0(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, bpf_session_is_return(), 0); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_1(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, bpf_session_is_return(), 1); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_2(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, bpf_session_is_return(), 2); +} From patchwork Thu Oct 10 20:09:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831041 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 543181D31B2; Thu, 10 Oct 2024 20:12:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591170; cv=none; b=V+SQKTT0efWPnWySyLZLl0+gPspzLkuPVSgv6dJvI0nG8XSICcGDpwsi5tsvt9k4Q7OQpZWlUZ6pRHJnzl7Dw5lTS+Ij4X5FySD/hydaLoqKixb1ON784m+428elGDENqqfqL0pYVVEW47C2kw+SAKdKKHFRmFkcW57J5KZ/sl8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591170; c=relaxed/simple; bh=U/bLZDppAfaNNv5D7VdyCPV08xWP80i4ouw+8O8MYJ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IVe1D6jNmFiGO479sFwelHqTHB8XiV4DKrdzXrDbDdHjV5LGCs0E9LIS6JZ4hlOO+ocXCH5VFuZ+ENxl2PtmISz5XunEelCkN35qcNZ2thS7+Lxe+zruJ4eT9B/w1XoSbnkMtMQTSkapv66T9KF2OtD139kMVmSzejmAZ7INm/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=p0pScCIF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="p0pScCIF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95AEDC4CEC5; Thu, 10 Oct 2024 20:12:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591170; bh=U/bLZDppAfaNNv5D7VdyCPV08xWP80i4ouw+8O8MYJ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p0pScCIF7wGzfula9l/0/yy2HGyyFOYxiqajdAgEiejc4GGt2SYTp+XnvY8UxN315 yyz8pinFn7s/gMdF9xeSS2iqhezKPlSq7ozkDYC9Y9UYAGU9VUQKkKJ8gNGCUmDrOB F+k/FVManD61OVleXofiHlCfg+suYOrVsQGXxF20IQgjzZsddDVycFMCRdQsm+hWE2 8tTePKv8y/KcA3wRURwhS4w8jpotfFukqhEtfSjYUCDtwMYptE9PAam6TPkdcsia4o ofgC5uNz4o2JCB1HZDGSo4KjtMUmBPIg43IOBnmcr1WpBloSk/5GrRop2zpEELgFHz WWrc25ke247+A== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 14/16] selftests/bpf: Scale down uprobe multi consumer test Date: Thu, 10 Oct 2024 22:09:55 +0200 Message-ID: <20241010200957.2750179-15-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net We have currently 2 uprobes and 2 uretprobes and we are about to add sessions uprobes in following change, which makes the test time unsuitable for CI even with threads. It's enough for the test to have just 1 uprobe and 1 uretprobe. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 57 ++++++++----------- .../bpf/progs/uprobe_multi_consumers.c | 16 +----- 2 files changed, 25 insertions(+), 48 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 7e0228f8fcfc..2effe4d693b4 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -761,10 +761,6 @@ get_program(struct uprobe_multi_consumers *skel, int prog) return skel->progs.uprobe_0; case 1: return skel->progs.uprobe_1; - case 2: - return skel->progs.uprobe_2; - case 3: - return skel->progs.uprobe_3; default: ASSERT_FAIL("get_program"); return NULL; @@ -779,10 +775,6 @@ get_link(struct uprobe_multi_consumers *skel, int link) return &skel->links.uprobe_0; case 1: return &skel->links.uprobe_1; - case 2: - return &skel->links.uprobe_2; - case 3: - return &skel->links.uprobe_3; default: ASSERT_FAIL("get_link"); return NULL; @@ -799,10 +791,10 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) return -1; /* - * bit/prog: 0,1 uprobe entry - * bit/prog: 2,3 uprobe return + * bit/prog: 0 uprobe entry + * bit/prog: 1 uprobe return */ - opts.retprobe = idx == 2 || idx == 3; + opts.retprobe = idx == 1; *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", "uprobe_consumer_test", @@ -832,7 +824,7 @@ uprobe_consumer_test(struct uprobe_multi_consumers *skel, int idx; /* detach uprobe for each unset programs in 'before' state ... */ - for (idx = 0; idx < 4; idx++) { + for (idx = 0; idx < 1; idx++) { if (test_bit(idx, before) && !test_bit(idx, after)) uprobe_detach(skel, idx); } @@ -855,7 +847,7 @@ static int consumer_test(struct uprobe_multi_consumers *skel, printf("consumer_test before %lu after %lu\n", before, after); /* 'before' is each, we attach uprobe for every set idx */ - for (idx = 0; idx < 4; idx++) { + for (idx = 0; idx < 1; idx++) { if (test_bit(idx, before)) { if (!ASSERT_OK(uprobe_attach(skel, idx), "uprobe_attach_before")) goto cleanup; @@ -866,18 +858,18 @@ static int consumer_test(struct uprobe_multi_consumers *skel, if (!ASSERT_EQ(err, 0, "uprobe_consumer_test")) goto cleanup; - for (idx = 0; idx < 4; idx++) { + for (idx = 0; idx < 1; idx++) { const char *fmt = "BUG"; __u64 val = 0; - if (idx < 2) { + if (idx == 0) { /* * uprobe entry * +1 if define in 'before' */ if (test_bit(idx, before)) val++; - fmt = "prog 0/1: uprobe"; + fmt = "prog 0: uprobe"; } else { /* * to trigger uretprobe consumer, the uretprobe needs to be installed, @@ -885,11 +877,11 @@ static int consumer_test(struct uprobe_multi_consumers *skel, * * idxs: 2/3 uprobe return in 'installed' mask */ - unsigned long had_uretprobes = before & 0b1100; /* is uretprobe installed */ + unsigned long had_uretprobes = before & 0b10; /* is uretprobe installed */ if (had_uretprobes && test_bit(idx, after)) val++; - fmt = "idx 2/3: uretprobe"; + fmt = "prog 1: uretprobe"; } if (!ASSERT_EQ(skel->bss->uprobe_result[idx], val, fmt)) @@ -900,7 +892,7 @@ static int consumer_test(struct uprobe_multi_consumers *skel, ret = 0; cleanup: - for (idx = 0; idx < 4; idx++) + for (idx = 0; idx < 1; idx++) uprobe_detach(skel, idx); return ret; } @@ -918,42 +910,41 @@ static void test_consumers(void) * The idea of this test is to try all possible combinations of * uprobes consumers attached on single function. * - * - 2 uprobe entry consumer - * - 2 uprobe exit consumers + * - 1 uprobe entry consumer + * - 1 uprobe exit consumer * - * The test uses 4 uprobes attached on single function, but that - * translates into single uprobe with 4 consumers in kernel. + * The test uses 2 uprobes attached on single function, but that + * translates into single uprobe with 2 consumers in kernel. * * The before/after values present the state of attached consumers * before and after the probed function: * - * bit/prog 0,1 : uprobe entry - * bit/prog 2,3 : uprobe return + * bit/prog 0 : uprobe entry + * bit/prog 1 : uprobe return * * For example for: * - * before = 0b0101 - * after = 0b0110 + * before = 0b01 + * after = 0b10 * * it means that before we call 'uprobe_consumer_test' we attach * uprobes defined in 'before' value: * - * - bit/prog 0: uprobe entry - * - bit/prog 2: uprobe return + * - bit/prog 1: uprobe entry * * uprobe_consumer_test is called and inside it we attach and detach * uprobes based on 'after' value: * - * - bit/prog 0: stays untouched - * - bit/prog 2: uprobe return is detached + * - bit/prog 0: is detached + * - bit/prog 1: is attached * * uprobe_consumer_test returns and we check counters values increased * by bpf programs on each uprobe to match the expected count based on * before/after bits. */ - for (before = 0; before < 16; before++) { - for (after = 0; after < 16; after++) + for (before = 0; before < 4; before++) { + for (after = 0; after < 4; after++) if (consumer_test(skel, before, after)) goto out; } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c b/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c index 7e0fdcbbd242..1c64d7263911 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c @@ -8,7 +8,7 @@ char _license[] SEC("license") = "GPL"; -__u64 uprobe_result[4]; +__u64 uprobe_result[2]; SEC("uprobe.multi") int uprobe_0(struct pt_regs *ctx) @@ -23,17 +23,3 @@ int uprobe_1(struct pt_regs *ctx) uprobe_result[1]++; return 0; } - -SEC("uprobe.multi") -int uprobe_2(struct pt_regs *ctx) -{ - uprobe_result[2]++; - return 0; -} - -SEC("uprobe.multi") -int uprobe_3(struct pt_regs *ctx) -{ - uprobe_result[3]++; - return 0; -} From patchwork Thu Oct 10 20:09:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831042 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 396D31D31B2; Thu, 10 Oct 2024 20:13:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591182; cv=none; b=Fwg1sCCgyeaqu1kEtKgLxy2cDETmlgm6yOeafTl00iXGfQBdf2IoqyNFE/e3ytwHsLm+6x+NKzs+v7Aw5SywgNEE7cPrASHchwH4EHVxR4XQQwsOHa0sh42jvCbQ5pJMR3n2zb7yFZSZ7rJDBHLpCqP59Z9XVxHanlOTTYObIMQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591182; c=relaxed/simple; bh=JGc9i062+P//gbMEIPHizp07D5rtK3D6ZFWovEKATw0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gwI1i7pxqMUebe75aAaPNGaz0OEw4dTj6zwaddcMejSIcIyrquIf/Ry/UTRhrhv2X//XysmbPuJh2zgQpfFqUWBzn4DvI4RIuKu8uEE4sAXk11pCoRUrHTM08Q32Td0mO1F/Ti5POX8bsQLR7wJexHPDPM8Qzu6xuOABTFx8aW8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=A9xq3G4a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="A9xq3G4a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90DA6C4CEC5; Thu, 10 Oct 2024 20:12:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591181; bh=JGc9i062+P//gbMEIPHizp07D5rtK3D6ZFWovEKATw0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A9xq3G4ajGFnJ/xOnKI7s8+OWKGqLUrSGPBZuZlFZXqfhz5Y6DqpEwel3Tw/50Yzg 2NkNsbMBWI4CF34SzQBIVi2BzVtSJvbsy2vKaimyVy6q0yiumrZvpsaPdCWagudUJP gahhyJbmOaaBx7hr7FfszBqGWY0Sk3qZ3AoRq4vXE/RH1uYRzYbpskuAjYya1NvV4G Kziry/YV8hny2lf/Q3cGlx9kGqtmLL5V9A1VLEEBg+FDUVdMr6JL8hx7n/kbXjlK92 pXgUxP0Sh052+n44GtzK1AS4Yg4/lzPXtYqR5esskfWHswouxIb7eoCjFlFepE+f/1 hEfmUn08pRGvw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 15/16] selftests/bpf: Add uprobe sessions to consumer test Date: Thu, 10 Oct 2024 22:09:56 +0200 Message-ID: <20241010200957.2750179-16-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session consumers to the consumer test, so we get the session into the test mix. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 63 +++++++++++++++---- .../bpf/progs/uprobe_multi_consumers.c | 16 ++++- 2 files changed, 66 insertions(+), 13 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 2effe4d693b4..df9314309bc3 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -761,6 +761,10 @@ get_program(struct uprobe_multi_consumers *skel, int prog) return skel->progs.uprobe_0; case 1: return skel->progs.uprobe_1; + case 2: + return skel->progs.uprobe_2; + case 3: + return skel->progs.uprobe_3; default: ASSERT_FAIL("get_program"); return NULL; @@ -775,6 +779,10 @@ get_link(struct uprobe_multi_consumers *skel, int link) return &skel->links.uprobe_0; case 1: return &skel->links.uprobe_1; + case 2: + return &skel->links.uprobe_2; + case 3: + return &skel->links.uprobe_3; default: ASSERT_FAIL("get_link"); return NULL; @@ -793,8 +801,11 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) /* * bit/prog: 0 uprobe entry * bit/prog: 1 uprobe return + * bit/prog: 2 uprobe session without return + * bit/prog: 3 uprobe session with return */ opts.retprobe = idx == 1; + opts.session = idx == 2 || idx == 3; *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", "uprobe_consumer_test", @@ -824,7 +835,7 @@ uprobe_consumer_test(struct uprobe_multi_consumers *skel, int idx; /* detach uprobe for each unset programs in 'before' state ... */ - for (idx = 0; idx < 1; idx++) { + for (idx = 0; idx < 4; idx++) { if (test_bit(idx, before) && !test_bit(idx, after)) uprobe_detach(skel, idx); } @@ -847,7 +858,7 @@ static int consumer_test(struct uprobe_multi_consumers *skel, printf("consumer_test before %lu after %lu\n", before, after); /* 'before' is each, we attach uprobe for every set idx */ - for (idx = 0; idx < 1; idx++) { + for (idx = 0; idx < 4; idx++) { if (test_bit(idx, before)) { if (!ASSERT_OK(uprobe_attach(skel, idx), "uprobe_attach_before")) goto cleanup; @@ -858,11 +869,13 @@ static int consumer_test(struct uprobe_multi_consumers *skel, if (!ASSERT_EQ(err, 0, "uprobe_consumer_test")) goto cleanup; - for (idx = 0; idx < 1; idx++) { + for (idx = 0; idx < 4; idx++) { + unsigned long had_uretprobes; const char *fmt = "BUG"; __u64 val = 0; - if (idx == 0) { + switch (idx) { + case 0: /* * uprobe entry * +1 if define in 'before' @@ -870,18 +883,42 @@ static int consumer_test(struct uprobe_multi_consumers *skel, if (test_bit(idx, before)) val++; fmt = "prog 0: uprobe"; - } else { + break; + case 1: /* * to trigger uretprobe consumer, the uretprobe needs to be installed, * which means one of the 'return' uprobes was alive when probe was hit: * - * idxs: 2/3 uprobe return in 'installed' mask + * idxs: 1/2 uprobe return in 'installed' mask */ - unsigned long had_uretprobes = before & 0b10; /* is uretprobe installed */ + had_uretprobes = before & 0b0110; /* is uretprobe installed */ if (had_uretprobes && test_bit(idx, after)) val++; fmt = "prog 1: uretprobe"; + break; + case 2: + /* + * session with return + * +1 if defined in 'before' + * +1 if defined in 'after' + */ + if (test_bit(idx, before)) { + val++; + if (test_bit(idx, after)) + val++; + } + fmt = "prog 2 : session with return"; + break; + case 3: + /* + * session without return + * +1 if defined in 'before' + */ + if (test_bit(idx, before)) + val++; + fmt = "prog 3 : session with NO return"; + break; } if (!ASSERT_EQ(skel->bss->uprobe_result[idx], val, fmt)) @@ -892,7 +929,7 @@ static int consumer_test(struct uprobe_multi_consumers *skel, ret = 0; cleanup: - for (idx = 0; idx < 1; idx++) + for (idx = 0; idx < 4; idx++) uprobe_detach(skel, idx); return ret; } @@ -912,9 +949,11 @@ static void test_consumers(void) * * - 1 uprobe entry consumer * - 1 uprobe exit consumer + * - 1 uprobe session with return + * - 1 uprobe session without return * - * The test uses 2 uprobes attached on single function, but that - * translates into single uprobe with 2 consumers in kernel. + * The test uses 4 uprobes attached on single function, but that + * translates into single uprobe with 4 consumers in kernel. * * The before/after values present the state of attached consumers * before and after the probed function: @@ -943,8 +982,8 @@ static void test_consumers(void) * before/after bits. */ - for (before = 0; before < 4; before++) { - for (after = 0; after < 4; after++) + for (before = 0; before < 16; before++) { + for (after = 0; after < 16; after++) if (consumer_test(skel, before, after)) goto out; } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c b/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c index 1c64d7263911..93752bb5690b 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c @@ -8,7 +8,7 @@ char _license[] SEC("license") = "GPL"; -__u64 uprobe_result[2]; +__u64 uprobe_result[4]; SEC("uprobe.multi") int uprobe_0(struct pt_regs *ctx) @@ -23,3 +23,17 @@ int uprobe_1(struct pt_regs *ctx) uprobe_result[1]++; return 0; } + +SEC("uprobe.session") +int uprobe_2(struct pt_regs *ctx) +{ + uprobe_result[2]++; + return 0; +} + +SEC("uprobe.session") +int uprobe_3(struct pt_regs *ctx) +{ + uprobe_result[3]++; + return 1; +} From patchwork Thu Oct 10 20:09:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13831043 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D8DC1DFDBD; Thu, 10 Oct 2024 20:13:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591193; cv=none; b=TVoxxELjvBULg18AYQDxpkT3qCqjRPWj4BTpmZyf/xSHyWf5pWe0lYgTB4snORwirc/F5ni6G1PgkOwz/cC629is7CgME1UY5a/Yb4TwRjCHdVeWJyBEm02Y8j8yoJWkUjtBactH7EwpnIWxbzr96ZKGn+fyu/2Zlke/wvKRoGw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728591193; c=relaxed/simple; bh=CSsw5tOBYxTiS5xp4wjYUVZlyYXNLRecRGlqBFAZMwg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HW3FtAn441tgwwic4Oyo5ugmx7/AQTOaFsEitzYamJexxzMpuN4TB6l65KNx4dX/88h1wsM7+wf6Pq3oDA8OH0+yyEuRlABxgLPSivb/H2aCOhC2/ugBMLT4TkjyYdrh8DCdKsErSOgGROHfg5rWme+0Qm9I4ddyEMn5uAwIdXM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=K41/dm/V; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="K41/dm/V" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 09B9DC4CEC5; Thu, 10 Oct 2024 20:13:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728591193; bh=CSsw5tOBYxTiS5xp4wjYUVZlyYXNLRecRGlqBFAZMwg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K41/dm/VMXRxrUUIUkcHz4NAPFJG3HYLZg7iigs3QQZUaWB0OgwHbc14lPMSEa9R8 uWdWA/bkWTThOdKm7uKZfjs9XjQVbCBGgO6X5sxIhljJbn1Q/LDXvs0bhirPKPD03j CaLOHzABz/qXTuFBNrqL+lgnu6Jn5ZL8Xp69rJgySKaJJf9IAOMIPCgGueXQotWs+X CThUkVf/vY/Oh1j5HoP83wenVmAYH+5NdcEjMJHrck306BjBREGyykCoHwqrPsnsrq fBLCa4z0CDrQFwnAaVG1Zb5ktNIjrAoODKpDMQ8TMy32vBsSUWvT3pT+4t0H1zYeI7 w/JZVQ/hcBFcQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv6 bpf-next 16/16] selftests/bpf: Add threads to consumer test Date: Thu, 10 Oct 2024 22:09:57 +0200 Message-ID: <20241010200957.2750179-17-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241010200957.2750179-1-jolsa@kernel.org> References: <20241010200957.2750179-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net With recent uprobe fix [1] the sync time after unregistering uprobe is much longer and prolongs the consumer test which creates and destroys hundreds of uprobes. This change adds 16 threads (which fits the test logic) and speeds up the test. Before the change: # perf stat --null ./test_progs -t uprobe_multi_test/consumers #421/9 uprobe_multi_test/consumers:OK #421 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -t uprobe_multi_test/consumers': 28.818778973 seconds time elapsed 0.745518000 seconds user 0.919186000 seconds sys After the change: # perf stat --null ./test_progs -t uprobe_multi_test/consumers 2>&1 #421/9 uprobe_multi_test/consumers:OK #421 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -t uprobe_multi_test/consumers': 3.504790814 seconds time elapsed 0.012141000 seconds user 0.751760000 seconds sys [1] commit 87195a1ee332 ("uprobes: switch to RCU Tasks Trace flavor for better performance") Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/uprobe_multi_test.c | 98 +++++++++++++++---- 1 file changed, 80 insertions(+), 18 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index df9314309bc3..c1ef7355dce3 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -789,7 +789,7 @@ get_link(struct uprobe_multi_consumers *skel, int link) } } -static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) +static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx, unsigned long offset) { struct bpf_program *prog = get_program(skel, idx); struct bpf_link **link = get_link(skel, idx); @@ -798,6 +798,9 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) if (!prog || !link) return -1; + opts.offsets = &offset; + opts.cnt = 1; + /* * bit/prog: 0 uprobe entry * bit/prog: 1 uprobe return @@ -807,9 +810,7 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) opts.retprobe = idx == 1; opts.session = idx == 2 || idx == 3; - *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", - "uprobe_consumer_test", - &opts); + *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", NULL, &opts); if (!ASSERT_OK_PTR(*link, "bpf_program__attach_uprobe_multi")) return -1; return 0; @@ -830,7 +831,8 @@ static bool test_bit(int bit, unsigned long val) noinline int uprobe_consumer_test(struct uprobe_multi_consumers *skel, - unsigned long before, unsigned long after) + unsigned long before, unsigned long after, + unsigned long offset) { int idx; @@ -843,15 +845,43 @@ uprobe_consumer_test(struct uprobe_multi_consumers *skel, /* ... and attach all new programs in 'after' state */ for (idx = 0; idx < 4; idx++) { if (!test_bit(idx, before) && test_bit(idx, after)) { - if (!ASSERT_OK(uprobe_attach(skel, idx), "uprobe_attach_after")) + if (!ASSERT_OK(uprobe_attach(skel, idx, offset), "uprobe_attach_after")) return -1; } } return 0; } +/* + * We generate 16 consumer_testX functions that will have uprobe installed on + * and will be called in separate threads. All function pointer are stored in + * "consumers" section and each thread will pick one function based on index. + */ + +extern const void *__start_consumers; + +#define __CONSUMER_TEST(func) \ +noinline int func(struct uprobe_multi_consumers *skel, unsigned long before, \ + unsigned long after, unsigned long offset) \ +{ \ + return uprobe_consumer_test(skel, before, after, offset); \ +} \ +static void *__ ## func __used __attribute__((section("consumers"))) = (void *) func; + +#define CONSUMER_TEST(func) __CONSUMER_TEST(func) + +#define C1 CONSUMER_TEST(__PASTE(consumer_test, __COUNTER__)) +#define C4 C1 C1 C1 C1 +#define C16 C4 C4 C4 C4 + +C16 + +typedef int (*test_t)(struct uprobe_multi_consumers *, unsigned long, + unsigned long, unsigned long); + static int consumer_test(struct uprobe_multi_consumers *skel, - unsigned long before, unsigned long after) + unsigned long before, unsigned long after, + test_t test, unsigned long offset) { int err, idx, ret = -1; @@ -860,12 +890,12 @@ static int consumer_test(struct uprobe_multi_consumers *skel, /* 'before' is each, we attach uprobe for every set idx */ for (idx = 0; idx < 4; idx++) { if (test_bit(idx, before)) { - if (!ASSERT_OK(uprobe_attach(skel, idx), "uprobe_attach_before")) + if (!ASSERT_OK(uprobe_attach(skel, idx, offset), "uprobe_attach_before")) goto cleanup; } } - err = uprobe_consumer_test(skel, before, after); + err = test(skel, before, after, offset); if (!ASSERT_EQ(err, 0, "uprobe_consumer_test")) goto cleanup; @@ -934,14 +964,46 @@ static int consumer_test(struct uprobe_multi_consumers *skel, return ret; } -static void test_consumers(void) +#define CONSUMER_MAX 16 + +/* + * Each thread runs 1/16 of the load by running test for single + * 'before' number (based on thread index) and full scale of + * 'after' numbers. + */ +static void *consumer_thread(void *arg) { + unsigned long idx = (unsigned long) arg; struct uprobe_multi_consumers *skel; - int before, after; + unsigned long offset; + const void *func; + int after; skel = uprobe_multi_consumers__open_and_load(); if (!ASSERT_OK_PTR(skel, "uprobe_multi_consumers__open_and_load")) - return; + return NULL; + + func = *((&__start_consumers) + idx); + + offset = get_uprobe_offset(func); + if (!ASSERT_GE(offset, 0, "uprobe_offset")) + goto out; + + for (after = 0; after < CONSUMER_MAX; after++) + if (consumer_test(skel, idx, after, func, offset)) + goto out; + +out: + uprobe_multi_consumers__destroy(skel); + return NULL; +} + + +static void test_consumers(void) +{ + pthread_t pt[CONSUMER_MAX]; + unsigned long idx; + int err; /* * The idea of this test is to try all possible combinations of @@ -982,14 +1044,14 @@ static void test_consumers(void) * before/after bits. */ - for (before = 0; before < 16; before++) { - for (after = 0; after < 16; after++) - if (consumer_test(skel, before, after)) - goto out; + for (idx = 0; idx < CONSUMER_MAX; idx++) { + err = pthread_create(&pt[idx], NULL, consumer_thread, (void *) idx); + if (!ASSERT_OK(err, "pthread_create")) + break; } -out: - uprobe_multi_consumers__destroy(skel); + while (idx) + pthread_join(pt[--idx], NULL); } static struct bpf_program *uprobe_multi_program(struct uprobe_multi_pid_filter *skel, int idx)