From patchwork Mon Sep 9 07:45:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796221 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE6EA1B140D; Mon, 9 Sep 2024 07:46:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725867972; cv=none; b=nH37quH+EASS32Qs8V6N0AjorEh9B2lMP0lXAae2wGoCCDWR5NUyRFGgk3PPQ9nKldQxMFU+ajUf+IVHR0cPfbYXNZ7msve3OK0/TER4SIgwSmL5c7iZJ/yucGuDJncJ2aFLtwZyWHNmAT3NHbbtgZDRzZT+EsSeAQkc8JeW5AQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725867972; c=relaxed/simple; bh=jC2UKcAqriOc1B/CfeqJAZ75rwjnIUIWAZWKK2ISpck=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HLzqv8lWNjHS/fyzDbWE9ICk9GyIY32JfXT1UhYdJa0LP0RiPSR4Cd8l04SxbCRpHIvcn/h8m1L2n/lcrzRqR/wQT2DlHr3qIIKyX2sjBj1SUKkebwhGfQq/FiQ558CARbdM7Bb+rMXm79zLK5Exb2JF17raiGLaFVRSZBgDmzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gr/TFYwI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gr/TFYwI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B12AC4CEC8; Mon, 9 Sep 2024 07:46:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725867972; bh=jC2UKcAqriOc1B/CfeqJAZ75rwjnIUIWAZWKK2ISpck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gr/TFYwI2hnOL3k1ZKf41vVtIWh0Hj1NgMfVzCVNXF3BBjLSSuKARmScVQDSlL29Z 3URDvUhgyNuXC7ROv99wP3DlTFfnIKZfa9liujO9i/das+d0sSZKjqHHUIHPdLjIva iIKrPfWap/bUl3MgzazEQBcjDK5oJYZXNQGDAs+RfSOeiaJR27bsp2wRqDVFzvCWNW /U04Hc7Nk2IB8+ML/0zHbnuuXSIOPbatEzTpWfc4LgluKvAL+i23PFwnUc3dGj4QqO RatvE6TSTliClwbLukEKDq94utglqDkgnX/quCaVE92khmXKmx5duEK/rd6k9scAD/ Se7Fs+T5wPyGA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 1/7] uprobe: Add support for session consumer Date: Mon, 9 Sep 2024 09:45:48 +0200 Message-ID: <20240909074554.2339984-2-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support for uprobe consumer to be defined as session and have new behaviour for consumer's 'handler' and 'ret_handler' callbacks. The session means that 'handler' and 'ret_handler' callbacks are connected in a way that allows to: - control execution of 'ret_handler' from 'handler' callback - share data between 'handler' and 'ret_handler' callbacks The session is enabled by setting new 'session' bool field to true in uprobe_consumer object. We use return_consumer object to keep track of consumers with 'ret_handler'. This object also carries the shared data between 'handler' and and 'ret_handler' callbacks. The control of 'ret_handler' callback execution is done via return value of the 'handler' callback. This patch adds new 'ret_handler' return value (2) which means to ignore ret_handler callback. Actions on 'handler' callback return values are now: 0 - execute ret_handler (if it's defined) 1 - remove uprobe 2 - do nothing (ignore ret_handler) The session concept fits to our common use case where we do filtering on entry uprobe and based on the result we decide to run the return uprobe (or not). It's also convenient to share the data between session callbacks. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko Acked-by: Masami Hiramatsu (Google) --- include/linux/uprobes.h | 17 ++- kernel/events/uprobes.c | 132 ++++++++++++++---- kernel/trace/bpf_trace.c | 6 +- kernel/trace/trace_uprobe.c | 12 +- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 2 +- 5 files changed, 133 insertions(+), 36 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 2b294bf1881f..557901e04624 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -24,7 +24,7 @@ struct notifier_block; struct page; #define UPROBE_HANDLER_REMOVE 1 -#define UPROBE_HANDLER_MASK 1 +#define UPROBE_HANDLER_IGNORE 2 #define MAX_URETPROBE_DEPTH 64 @@ -37,13 +37,16 @@ struct uprobe_consumer { * for the current process. If filter() is omitted or returns true, * UPROBE_HANDLER_REMOVE is effectively ignored. */ - int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs); + int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs, __u64 *data); int (*ret_handler)(struct uprobe_consumer *self, unsigned long func, - struct pt_regs *regs); + struct pt_regs *regs, __u64 *data); bool (*filter)(struct uprobe_consumer *self, struct mm_struct *mm); struct list_head cons_node; + bool session; + + __u64 id; /* set when uprobe_consumer is registered */ }; #ifdef CONFIG_UPROBES @@ -83,14 +86,22 @@ struct uprobe_task { unsigned int depth; }; +struct return_consumer { + __u64 cookie; + __u64 id; +}; + struct return_instance { struct uprobe *uprobe; unsigned long func; unsigned long stack; /* stack pointer */ unsigned long orig_ret_vaddr; /* original return address */ bool chained; /* true, if instance is nested */ + int consumers_cnt; struct return_instance *next; /* keep as stack */ + + struct return_consumer consumers[] __counted_by(consumers_cnt); }; enum rp_check { diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4b7e590dc428..9e971f86afdf 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -67,6 +67,8 @@ struct uprobe { loff_t ref_ctr_offset; unsigned long flags; + unsigned int consumers_cnt; + /* * The generic code assumes that it has two members of unknown type * owned by the arch-specific code: @@ -826,8 +828,12 @@ static struct uprobe *alloc_uprobe(struct inode *inode, loff_t offset, static void consumer_add(struct uprobe *uprobe, struct uprobe_consumer *uc) { + static atomic64_t id; + down_write(&uprobe->consumer_rwsem); list_add_rcu(&uc->cons_node, &uprobe->consumers); + uc->id = (__u64) atomic64_inc_return(&id); + uprobe->consumers_cnt++; up_write(&uprobe->consumer_rwsem); } @@ -839,6 +845,7 @@ static void consumer_del(struct uprobe *uprobe, struct uprobe_consumer *uc) { down_write(&uprobe->consumer_rwsem); list_del_rcu(&uc->cons_node); + uprobe->consumers_cnt--; up_write(&uprobe->consumer_rwsem); } @@ -1786,6 +1793,30 @@ static struct uprobe_task *get_utask(void) return current->utask; } +static size_t ri_size(int consumers_cnt) +{ + struct return_instance *ri; + + return sizeof(*ri) + sizeof(ri->consumers[0]) * consumers_cnt; +} + +static struct return_instance *alloc_return_instance(int consumers_cnt) +{ + struct return_instance *ri; + + ri = kzalloc(ri_size(consumers_cnt), GFP_KERNEL); + if (ri) + ri->consumers_cnt = consumers_cnt; + return ri; +} + +static struct return_instance *dup_return_instance(struct return_instance *old) +{ + size_t size = ri_size(old->consumers_cnt); + + return kmemdup(old, size, GFP_KERNEL); +} + static int dup_utask(struct task_struct *t, struct uprobe_task *o_utask) { struct uprobe_task *n_utask; @@ -1798,11 +1829,10 @@ static int dup_utask(struct task_struct *t, struct uprobe_task *o_utask) p = &n_utask->return_instances; for (o = o_utask->return_instances; o; o = o->next) { - n = kmalloc(sizeof(struct return_instance), GFP_KERNEL); + n = dup_return_instance(o); if (!n) return -ENOMEM; - *n = *o; /* * uprobe's refcnt has to be positive at this point, kept by * utask->return_instances items; return_instances can't be @@ -1895,39 +1925,35 @@ static void cleanup_return_instances(struct uprobe_task *utask, bool chained, utask->return_instances = ri; } -static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) +static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs, + struct return_instance *ri) { - struct return_instance *ri; struct uprobe_task *utask; unsigned long orig_ret_vaddr, trampoline_vaddr; bool chained; if (!get_xol_area()) - return; + goto free; utask = get_utask(); if (!utask) - return; + goto free; if (utask->depth >= MAX_URETPROBE_DEPTH) { printk_ratelimited(KERN_INFO "uprobe: omit uretprobe due to" " nestedness limit pid/tgid=%d/%d\n", current->pid, current->tgid); - return; + goto free; } /* we need to bump refcount to store uprobe in utask */ if (!try_get_uprobe(uprobe)) - return; - - ri = kmalloc(sizeof(struct return_instance), GFP_KERNEL); - if (!ri) - goto fail; + goto free; trampoline_vaddr = uprobe_get_trampoline_vaddr(); orig_ret_vaddr = arch_uretprobe_hijack_return_addr(trampoline_vaddr, regs); if (orig_ret_vaddr == -1) - goto fail; + goto put; /* drop the entries invalidated by longjmp() */ chained = (orig_ret_vaddr == trampoline_vaddr); @@ -1945,7 +1971,7 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) * attack from user-space. */ uprobe_warn(current, "handle tail call"); - goto fail; + goto put; } orig_ret_vaddr = utask->return_instances->orig_ret_vaddr; } @@ -1960,9 +1986,10 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) utask->return_instances = ri; return; -fail: - kfree(ri); +put: put_uprobe(uprobe); +free: + kfree(ri); } /* Prepare to single-step probed instruction out of line. */ @@ -2114,35 +2141,78 @@ static struct uprobe *find_active_uprobe_rcu(unsigned long bp_vaddr, int *is_swb return uprobe; } +static struct return_consumer * +return_consumer_next(struct return_instance *ri, struct return_consumer *ric) +{ + return ric ? ric + 1 : &ri->consumers[0]; +} + +static struct return_consumer * +return_consumer_find(struct return_instance *ri, int *iter, int id) +{ + struct return_consumer *ric; + int idx = *iter; + + for (ric = &ri->consumers[idx]; idx < ri->consumers_cnt; idx++, ric++) { + if (ric->id == id) { + *iter = idx + 1; + return ric; + } + } + return NULL; +} + static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) { struct uprobe_consumer *uc; int remove = UPROBE_HANDLER_REMOVE; - bool need_prep = false; /* prepare return uprobe, when needed */ + struct return_consumer *ric = NULL; + struct return_instance *ri = NULL; bool has_consumers = false; current->utask->auprobe = &uprobe->arch; list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node, srcu_read_lock_held(&uprobes_srcu)) { + __u64 cookie = 0; int rc = 0; if (uc->handler) { - rc = uc->handler(uc, regs); - WARN(rc & ~UPROBE_HANDLER_MASK, + rc = uc->handler(uc, regs, &cookie); + WARN(rc < 0 || rc > 2, "bad rc=0x%x from %ps()\n", rc, uc->handler); } - if (uc->ret_handler) - need_prep = true; - + /* + * The handler can return following values: + * 0 - execute ret_handler (if it's defined) + * 1 - remove uprobe + * 2 - do nothing (ignore ret_handler) + */ remove &= rc; has_consumers = true; + + if (rc == 0 && uc->ret_handler) { + /* + * Preallocate return_instance object optimistically with + * all possible consumers, so we allocate just once. + */ + if (!ri) { + ri = alloc_return_instance(uprobe->consumers_cnt); + if (!ri) + return; + } + ric = return_consumer_next(ri, ric); + ric->cookie = cookie; + ric->id = uc->id; + } } current->utask->auprobe = NULL; - if (need_prep && !remove) - prepare_uretprobe(uprobe, regs); /* put bp at return */ + if (ri && !remove) + prepare_uretprobe(uprobe, regs, ri); /* put bp at return */ + else + kfree(ri); if (remove && has_consumers) { down_read(&uprobe->register_rwsem); @@ -2160,15 +2230,25 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) static void handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs) { + struct return_consumer *ric = NULL; struct uprobe *uprobe = ri->uprobe; struct uprobe_consumer *uc; - int srcu_idx; + int srcu_idx, iter = 0; srcu_idx = srcu_read_lock(&uprobes_srcu); list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node, srcu_read_lock_held(&uprobes_srcu)) { + /* + * If we don't find return consumer, it means uprobe consumer + * was added after we hit uprobe and return consumer did not + * get registered in which case we call the ret_handler only + * if it's not session consumer. + */ + ric = return_consumer_find(ri, &iter, uc->id); + if (!ric && uc->session) + continue; if (uc->ret_handler) - uc->ret_handler(uc, ri->func, regs); + uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL); } srcu_read_unlock(&uprobes_srcu, srcu_idx); } diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index ac0a01cc8634..de241503c8fb 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3332,7 +3332,8 @@ uprobe_multi_link_filter(struct uprobe_consumer *con, struct mm_struct *mm) } static int -uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs) +uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, + __u64 *data) { struct bpf_uprobe *uprobe; @@ -3341,7 +3342,8 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs) } static int -uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs) +uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs, + __u64 *data) { struct bpf_uprobe *uprobe; diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c index f7443e996b1b..11103dde897b 100644 --- a/kernel/trace/trace_uprobe.c +++ b/kernel/trace/trace_uprobe.c @@ -88,9 +88,11 @@ static struct trace_uprobe *to_trace_uprobe(struct dyn_event *ev) static int register_uprobe_event(struct trace_uprobe *tu); static int unregister_uprobe_event(struct trace_uprobe *tu); -static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs); +static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs, + __u64 *data); static int uretprobe_dispatcher(struct uprobe_consumer *con, - unsigned long func, struct pt_regs *regs); + unsigned long func, struct pt_regs *regs, + __u64 *data); #ifdef CONFIG_STACK_GROWSUP static unsigned long adjust_stack_addr(unsigned long addr, unsigned int n) @@ -1500,7 +1502,8 @@ trace_uprobe_register(struct trace_event_call *event, enum trace_reg type, } } -static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs) +static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs, + __u64 *data) { struct trace_uprobe *tu; struct uprobe_dispatch_data udd; @@ -1530,7 +1533,8 @@ static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs) } static int uretprobe_dispatcher(struct uprobe_consumer *con, - unsigned long func, struct pt_regs *regs) + unsigned long func, struct pt_regs *regs, + __u64 *data) { struct trace_uprobe *tu; struct uprobe_dispatch_data udd; diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 1fc16657cf42..e91ff5b285f0 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -421,7 +421,7 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = { static int uprobe_ret_handler(struct uprobe_consumer *self, unsigned long func, - struct pt_regs *regs) + struct pt_regs *regs, __u64 *data) { regs->ax = 0x12345678deadbeef; From patchwork Mon Sep 9 07:45:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796222 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 050F01AF4EB; Mon, 9 Sep 2024 07:46:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725867987; cv=none; b=ecCQ9jlb/DbmvhFbo0R2RN6uQNIdwEOlnUYZolBJsvnFyeZc8IYIe5mS06iT1l5rT2rmCCEHrohrlMosgT+/5FrlQKaAkY1H8WAWEGBZCqJ/CMsaXCkb3Z7VTNgaNsi04x3LFiJBXqt0SXAzK3t+P08k52oHNqfr7seD753/duE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725867987; c=relaxed/simple; bh=xAFY2t2/AZlQClCjMks65BvDZWK7PpFqi2ElzOn5PTk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i8e/caEvZHLL7IDAx1ojxTmpiDHfNI/hqNjoMn3O7SOMuoIreywqYXEp5BY0nr71FsBvnAChx/P7aoXDiLECAmy1WnW/vpe2e+yibrriRRnKju83MHAQsghvBkyuny+YSW0RyQmq5IFqLZ83ChFSlThDY5Sawt1CVUGPJ03EnnI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aXqmtOcp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aXqmtOcp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DBA75C4CEC8; Mon, 9 Sep 2024 07:46:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725867984; bh=xAFY2t2/AZlQClCjMks65BvDZWK7PpFqi2ElzOn5PTk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aXqmtOcp8YpYB+2gYlaJpfaxf6bCH365qqsO8TKDxjgQcfctK+hI1DG0TcB2zG/zC NCr/rlDFyhDozZnUE9OofPURFNb6Rili2BRvoVJp4VydR+OmhnsGXqGkQV9HEeytKs Br++ls07BgrmZpeboEqgOGOVfoh7wQv3CtVxEvnkK9Cf3UJ4Fhvbeu/dnAfg5NnLSG zYYZ23OsJEB0t7R4QfbnPluQUYGwDN+E7GWAy9nCUguQHpBv0Uoa9Dv2cXAKz+bzqV Yd5ejqsVLUVkgCUsoAXhsmeh6x+3hkBtiQMpuuEnt1mVEpSdFEXDsDniC6pdm4fE1q jWRHiBS0WxsKg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 2/7] bpf: Add support for uprobe multi session attach Date: Mon, 9 Sep 2024 09:45:49 +0200 Message-ID: <20240909074554.2339984-3-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support to attach bpf program for entry and return probe of the same function. This is common use case which at the moment requires to create two uprobe multi links. Adding new BPF_TRACE_UPROBE_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe. It's possible to control execution of the bpf program on return probe simply by returning zero or non zero from the entry bpf program execution to execute or not the bpf program on return probe respectively. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 9 +++++++-- kernel/trace/bpf_trace.c | 32 ++++++++++++++++++++++++-------- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/libbpf.c | 1 + 5 files changed, 34 insertions(+), 10 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 35bcf52dbc65..1d93cb014884 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1116,6 +1116,7 @@ enum bpf_attach_type { BPF_NETKIT_PRIMARY, BPF_NETKIT_PEER, BPF_TRACE_KPROBE_SESSION, + BPF_TRACE_UPROBE_SESSION, __MAX_BPF_ATTACH_TYPE }; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index bf6c5f685ea2..1347f3000bd0 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -4049,10 +4049,14 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI && attach_type != BPF_TRACE_UPROBE_MULTI) return -EINVAL; + if (prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION && + attach_type != BPF_TRACE_UPROBE_SESSION) + return -EINVAL; if (attach_type != BPF_PERF_EVENT && attach_type != BPF_TRACE_KPROBE_MULTI && attach_type != BPF_TRACE_KPROBE_SESSION && - attach_type != BPF_TRACE_UPROBE_MULTI) + attach_type != BPF_TRACE_UPROBE_MULTI && + attach_type != BPF_TRACE_UPROBE_SESSION) return -EINVAL; return 0; case BPF_PROG_TYPE_SCHED_CLS: @@ -5315,7 +5319,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI || attr->link_create.attach_type == BPF_TRACE_KPROBE_SESSION) ret = bpf_kprobe_multi_link_attach(attr, prog); - else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI) + else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI || + attr->link_create.attach_type == BPF_TRACE_UPROBE_SESSION) ret = bpf_uprobe_multi_link_attach(attr, prog); break; default: diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index de241503c8fb..a433e80771d2 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1645,6 +1645,17 @@ static inline bool is_kprobe_session(const struct bpf_prog *prog) return prog->expected_attach_type == BPF_TRACE_KPROBE_SESSION; } +static inline bool is_uprobe_multi(const struct bpf_prog *prog) +{ + return prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI || + prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION; +} + +static inline bool is_uprobe_session(const struct bpf_prog *prog) +{ + return prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION; +} + static const struct bpf_func_proto * kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { @@ -1662,13 +1673,13 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_get_func_ip: if (is_kprobe_multi(prog)) return &bpf_get_func_ip_proto_kprobe_multi; - if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + if (is_uprobe_multi(prog)) return &bpf_get_func_ip_proto_uprobe_multi; return &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: if (is_kprobe_multi(prog)) return &bpf_get_attach_cookie_proto_kmulti; - if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + if (is_uprobe_multi(prog)) return &bpf_get_attach_cookie_proto_umulti; return &bpf_get_attach_cookie_proto_trace; default: @@ -3336,9 +3347,13 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, __u64 *data) { struct bpf_uprobe *uprobe; + int ret; uprobe = container_of(con, struct bpf_uprobe, consumer); - return uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + if (uprobe->consumer.session) + return ret ? UPROBE_HANDLER_IGNORE : 0; + return ret; } static int @@ -3387,7 +3402,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr if (sizeof(u64) != sizeof(void *)) return -EOPNOTSUPP; - if (prog->expected_attach_type != BPF_TRACE_UPROBE_MULTI) + if (!is_uprobe_multi(prog)) return -EINVAL; flags = attr->link_create.uprobe_multi.flags; @@ -3463,11 +3478,12 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr uprobes[i].link = link; - if (flags & BPF_F_UPROBE_MULTI_RETURN) - uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; - else + if (!(flags & BPF_F_UPROBE_MULTI_RETURN)) uprobes[i].consumer.handler = uprobe_multi_link_handler; - + if (flags & BPF_F_UPROBE_MULTI_RETURN || is_uprobe_session(prog)) + uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; + if (is_uprobe_session(prog)) + uprobes[i].consumer.session = true; if (pid) uprobes[i].consumer.filter = uprobe_multi_link_filter; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 35bcf52dbc65..1d93cb014884 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1116,6 +1116,7 @@ enum bpf_attach_type { BPF_NETKIT_PRIMARY, BPF_NETKIT_PEER, BPF_TRACE_KPROBE_SESSION, + BPF_TRACE_UPROBE_SESSION, __MAX_BPF_ATTACH_TYPE }; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index a3be6f8fac09..274441674f92 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -133,6 +133,7 @@ static const char * const attach_type_name[] = { [BPF_NETKIT_PRIMARY] = "netkit_primary", [BPF_NETKIT_PEER] = "netkit_peer", [BPF_TRACE_KPROBE_SESSION] = "trace_kprobe_session", + [BPF_TRACE_UPROBE_SESSION] = "trace_uprobe_session", }; static const char * const link_type_name[] = { From patchwork Mon Sep 9 07:45:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796223 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C272718CBE7; Mon, 9 Sep 2024 07:46:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725867996; cv=none; b=oDpJJmh9pRYau9NK77BdeYWzgLF14KR+7A8VN89q+OcP0hhy7Q3u/Q37ubRxVxBhLDv6twmyx5xpDPDDbVrg0VaCxXHG971zRNPuqBBkmOJSyacVFVSEU3jdqBsTyd+FByNccM5awfAcGu79ngICCQ1DQPI8rfTazrJA6izuzbU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725867996; c=relaxed/simple; bh=UFGhLT5bf7NtJ04vSfF/t2JQZ9Q5WTVSQlVzP1e17kM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iC6RxIJ+/70vDQQaGZUTv2905VV5Z9sqHIrXD/k9fvm2LZyuNj/0Lj0XLSU2ZnpBa5/VCs0hT7HNHGM29G/6BnH99VMq5tB4E1RoFpD1IeC2OSYuREgWVOON0eN4Rfwz2W9DBZQSwKlR1EyBpuvPKBowOzCfAQmgPz8lH/q7nGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J8SyLfbq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J8SyLfbq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 317ECC4CECA; Mon, 9 Sep 2024 07:46:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725867996; bh=UFGhLT5bf7NtJ04vSfF/t2JQZ9Q5WTVSQlVzP1e17kM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J8SyLfbqhVI/psVgAh4J/HFprtFVGju4UQdPhz9CDWoWY2P0QXd8Ei6yhmoR66HLX kh5ugG1anI/D3dmFXkFkXdHSQVrzGq+YQdCPW61aGf7wlc/bU4Br8olFVRDsnglPTQ yiVVQfY16AVVSE+iM/ktcnDOsH9ntto6xMASBHWJJGOn0wKGpgbztoUGYdyDNXtZh6 7khwxZzbw2J9mTP+BCNYM+VOSEzt8I4jsZ4AJamARcLnRQSnhZtOZYsrLZSmbNMryB tZgrIizrlSEexVOEtZm1f4AXVaQenYqHMMpE323yInEkzz3js7cHj6jpqrW3qZQWlZ KeK2ORGbfZPeQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 3/7] bpf: Add support for uprobe multi session context Date: Mon, 9 Sep 2024 09:45:50 +0200 Message-ID: <20240909074554.2339984-4-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_uprobe_multi_run_ctx, so the session data can be retrieved from uprobe_multi link. Plus granting session kfuncs access to uprobe session programs. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- kernel/trace/bpf_trace.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index a433e80771d2..c8bd0ac98592 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3185,7 +3185,7 @@ struct bpf_uprobe_multi_link { }; struct bpf_uprobe_multi_run_ctx { - struct bpf_run_ctx run_ctx; + struct bpf_session_run_ctx session_ctx; unsigned long entry_ip; struct bpf_uprobe *uprobe; }; @@ -3298,10 +3298,15 @@ static const struct bpf_link_ops bpf_uprobe_multi_link_lops = { static int uprobe_prog_run(struct bpf_uprobe *uprobe, unsigned long entry_ip, - struct pt_regs *regs) + struct pt_regs *regs, + bool is_return, void *data) { struct bpf_uprobe_multi_link *link = uprobe->link; struct bpf_uprobe_multi_run_ctx run_ctx = { + .session_ctx = { + .is_return = is_return, + .data = data, + }, .entry_ip = entry_ip, .uprobe = uprobe, }; @@ -3320,7 +3325,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, migrate_disable(); - old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); + old_run_ctx = bpf_set_run_ctx(&run_ctx.session_ctx.run_ctx); err = bpf_prog_run(link->link.prog, regs); bpf_reset_run_ctx(old_run_ctx); @@ -3350,7 +3355,7 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, int ret; uprobe = container_of(con, struct bpf_uprobe, consumer); - ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs, false, data); if (uprobe->consumer.session) return ret ? UPROBE_HANDLER_IGNORE : 0; return ret; @@ -3363,14 +3368,15 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s struct bpf_uprobe *uprobe; uprobe = container_of(con, struct bpf_uprobe, consumer); - return uprobe_prog_run(uprobe, func, regs); + return uprobe_prog_run(uprobe, func, regs, true, data); } static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; - run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, + session_ctx.run_ctx); return run_ctx->entry_ip; } @@ -3378,7 +3384,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; - run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, + session_ctx.run_ctx); return run_ctx->uprobe->cookie; } @@ -3572,7 +3579,7 @@ static int bpf_kprobe_multi_filter(const struct bpf_prog *prog, u32 kfunc_id) if (!btf_id_set8_contains(&kprobe_multi_kfunc_set_ids, kfunc_id)) return 0; - if (!is_kprobe_session(prog)) + if (!is_kprobe_session(prog) && !is_uprobe_session(prog)) return -EACCES; return 0; From patchwork Mon Sep 9 07:45:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796224 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD65A1AD5C1; Mon, 9 Sep 2024 07:46:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868007; cv=none; b=t31P4V++ix4GYeA565bnCPDEjt3Y8YkyYtIt3bZwNl+B1gtMRO1zivvXYYXc1VlX/iAODZELUoyZVDncTLRrYqT7RhujRK1sU6tq/3IEMviAjMoS2YNTfTwFqSeuIuPwfWVRE0RuFmxm6f/lpSs6yUTLrB0RuUJ31iITeZ/H5OA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868007; c=relaxed/simple; bh=ObTqM6334eLsYB+Krs4fbcM5unxTfDttQTIQdBshKKI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fd16NZy2ACoPpdLM4VDtGSRC4E50JOVwToGp8vUlOgSVoA3YXM1i1/SL/VlxK6kEfmQQv34K6oZzCAidq6q65LSVS2GdVYplggPYMXwtt/HQvFZXqiDFC7xuCXQVc2TpSksFk96bzp5I471rNOkvzFiNOxROLBHnGZOh40xi6SU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HmDgSU0S; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HmDgSU0S" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5DF9C4CEC5; Mon, 9 Sep 2024 07:46:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725868007; bh=ObTqM6334eLsYB+Krs4fbcM5unxTfDttQTIQdBshKKI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HmDgSU0SGrENUi3/u+fMKiO/+krvucO7mSLZeBb9XZLadyMioCfsfGRaojqf8xdSX 3yQoGm8YQ6dbwI2ndHuzqAOyqLtotpyGTG/syOdp5P/0CCdu42VxoXuFofaNYnQi1F Gz8qQJ53Gj7TlZIuWezDJNBj1aQXqHDrmxfRQhrtvgqM9PqcbGs7BTmc0+ZJcEgI7N y+pmScz5mhC3FJbAF1E5W+YSJvn0pKo6d81lNGQPG++ceCnJpIdbNRGsYNhH+ubGu3 yCq915gth5/4dhqjw1KpbK4OtK0wsLBZmnCa4RHHwP92aomBumJBIDTZHpTVvhpt1k qudUKqg02bgpg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 4/7] libbpf: Add support for uprobe multi session attach Date: Mon, 9 Sep 2024 09:45:51 +0200 Message-ID: <20240909074554.2339984-5-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support to attach program in uprobe session mode with bpf_program__attach_uprobe_multi function. Adding session bool to bpf_uprobe_multi_opts struct that allows to load and attach the bpf program via uprobe session. the attachment to create uprobe multi session. Also adding new program loader section that allows: SEC("uprobe.session/bpf_fentry_test*") and loads/attaches uprobe program as uprobe session. Adding sleepable hook (uprobe.session.s) as well. Signed-off-by: Jiri Olsa --- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 50 ++++++++++++++++++++++++++++++++++++++++-- tools/lib/bpf/libbpf.h | 4 +++- 3 files changed, 52 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2a4c71501a17..becdfa701c75 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -776,6 +776,7 @@ int bpf_link_create(int prog_fd, int target_fd, return libbpf_err(-EINVAL); break; case BPF_TRACE_UPROBE_MULTI: + case BPF_TRACE_UPROBE_SESSION: attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0); attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0); attr.link_create.uprobe_multi.path = ptr_to_u64(OPTS_GET(opts, uprobe_multi.path, 0)); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 274441674f92..7b72a29b09a7 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -9345,6 +9345,7 @@ static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_kprobe_session(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link); +static int attach_uprobe_session(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link); @@ -9363,8 +9364,10 @@ static const struct bpf_sec_def section_defs[] = { SEC_DEF("kprobe.session+", KPROBE, BPF_TRACE_KPROBE_SESSION, SEC_NONE, attach_kprobe_session), SEC_DEF("uprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), SEC_DEF("uretprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), + SEC_DEF("uprobe.session+", KPROBE, BPF_TRACE_UPROBE_SESSION, SEC_NONE, attach_uprobe_session), SEC_DEF("uprobe.multi.s+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi), SEC_DEF("uretprobe.multi.s+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi), + SEC_DEF("uprobe.session.s+", KPROBE, BPF_TRACE_UPROBE_SESSION, SEC_SLEEPABLE, attach_uprobe_session), SEC_DEF("ksyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("kretsyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_USDT, attach_usdt), @@ -11699,6 +11702,39 @@ static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, stru return ret; } +static int attach_uprobe_session(const struct bpf_program *prog, long cookie, struct bpf_link **link) +{ + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts, .session = true); + char *binary_path = NULL, *func_name = NULL; + int n, ret = -EINVAL; + const char *spec; + + *link = NULL; + + spec = prog->sec_name + sizeof("uprobe.session/") - 1; + if (cookie & SEC_SLEEPABLE) + spec += 2; /* extra '.s' */ + n = sscanf(spec, "%m[^:]:%m[^\n]", &binary_path, &func_name); + + switch (n) { + case 1: + /* but auto-attach is impossible. */ + ret = 0; + break; + case 2: + *link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts); + ret = *link ? 0 : -errno; + break; + default: + pr_warn("prog '%s': invalid format of section definition '%s'\n", prog->name, + prog->sec_name); + break; + } + free(binary_path); + free(func_name); + return ret; +} + static void gen_uprobe_legacy_event_name(char *buf, size_t buf_sz, const char *binary_path, uint64_t offset) { @@ -11933,10 +11969,12 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL; LIBBPF_OPTS(bpf_link_create_opts, lopts); unsigned long *resolved_offsets = NULL; + enum bpf_attach_type attach_type; int err = 0, link_fd, prog_fd; struct bpf_link *link = NULL; char errmsg[STRERR_BUFSIZE]; char full_path[PATH_MAX]; + bool retprobe, session; const __u64 *cookies; const char **syms; size_t cnt; @@ -12007,12 +12045,20 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, offsets = resolved_offsets; } + retprobe = OPTS_GET(opts, retprobe, false); + session = OPTS_GET(opts, session, false); + + if (retprobe && session) + return libbpf_err_ptr(-EINVAL); + + attach_type = session ? BPF_TRACE_UPROBE_SESSION : BPF_TRACE_UPROBE_MULTI; + lopts.uprobe_multi.path = path; lopts.uprobe_multi.offsets = offsets; lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; lopts.uprobe_multi.cookies = cookies; lopts.uprobe_multi.cnt = cnt; - lopts.uprobe_multi.flags = OPTS_GET(opts, retprobe, false) ? BPF_F_UPROBE_MULTI_RETURN : 0; + lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0; if (pid == 0) pid = getpid(); @@ -12026,7 +12072,7 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, } link->detach = &bpf_link__detach_fd; - link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts); + link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts); if (link_fd < 0) { err = -errno; pr_warn("prog '%s': failed to attach multi-uprobe: %s\n", diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 64a6a3d323e3..f6a7835dc519 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -569,10 +569,12 @@ struct bpf_uprobe_multi_opts { size_t cnt; /* create return uprobes */ bool retprobe; + /* create session kprobes */ + bool session; size_t :0; }; -#define bpf_uprobe_multi_opts__last_field retprobe +#define bpf_uprobe_multi_opts__last_field session /** * @brief **bpf_program__attach_uprobe_multi()** attaches a BPF program From patchwork Mon Sep 9 07:45:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796225 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B63A717557C; Mon, 9 Sep 2024 07:46:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868019; cv=none; b=idSbegtJ3PksWeh/20OK944G5n80Exhi2ee0FZHjJ6EABIhEe9ovV+57SQBCTFCwslvMfRZctfHJrakQADD4aghj49ewrpUQAwOz8N9tSQaAXkhF8srKEOHTNRQwblqz27+3mV+wG/F/QBQgrL6LXG0E9bgtAQtVqu4Ty9+erR0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868019; c=relaxed/simple; bh=QcQ3tkDZArjXJxjIDoxxoCerB8p+FJ17FuuJb8viNQ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tS+OWTfS3qnOUVdbLdGiUHxByQlXEHSeT+/agGPPaFmOudeROvajCc6kl2+LphcDblgBmD84xlQ8MqYjh0CKI8eYeOxgUkG2dR/+iQq+o97TrHdaEyvecRQFH26MNoQoDUPYlbfHL246uM/La0Axgbyb0HNjXF7duJXppPOA4wA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nzXho6Ro; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nzXho6Ro" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A79DC4CEC5; Mon, 9 Sep 2024 07:46:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725868019; bh=QcQ3tkDZArjXJxjIDoxxoCerB8p+FJ17FuuJb8viNQ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nzXho6RomaZfCKV/SgPWwiPuXU/mRLlVdiIzqb58hD4XolklxNAGHT6pqObQLUfX7 BI73maYTab2TYCuMMm+4EZZudOZ/4pjq7Rv5sr7fVFlhAMhjpcYPz3R2rRqVSt9Kqo 1tmEJnluVF8R/cYCM4fTBLgKULGkQ5+3doDcRPJHJCOPscn30rFOJTyJZsrn5n0D0z Kva2H5hwrxPmuZpP2WE/HYX/f9ydVAb+rO1x1o6JPRxoIgjQl2C3Mwco0JlVZYcpMr +posI2XBiU/PGP+mMRcB59GYyAn0E0N1+xJhyyUP6nm14oXkvNMAYMSoTGWzKWXXBp D70eYgTukeuaw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 5/7] selftests/bpf: Add uprobe session test Date: Mon, 9 Sep 2024 09:45:52 +0200 Message-ID: <20240909074554.2339984-6-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test and testing that the entry program return value controls execution of the return probe program. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/uprobe_multi_test.c | 47 ++++++++++++ .../bpf/progs/uprobe_multi_session.c | 71 +++++++++++++++++++ 2 files changed, 118 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index bf6ca8e3eb13..cc32288bfe26 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -6,6 +6,7 @@ #include "uprobe_multi.skel.h" #include "uprobe_multi_bench.skel.h" #include "uprobe_multi_usdt.skel.h" +#include "uprobe_multi_session.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -615,6 +616,50 @@ static void test_link_api(void) __test_link_api(child); } +static void test_session_skel_api(void) +{ + struct uprobe_multi_session *skel = NULL; + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + struct bpf_link *link = NULL; + int err; + + skel = uprobe_multi_session__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + skel->bss->user_ptr = test_data; + + err = uprobe_multi_session__attach(skel); + if (!ASSERT_OK(err, " uprobe_multi_session__attach")) + goto cleanup; + + /* trigger all probes */ + skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3; + + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + /* + * We expect 2 for uprobe_multi_func_2 because it runs both entry/return probe, + * uprobe_multi_func_[13] run just the entry probe. All expected numbers are + * doubled, because we run extra test for sleepable session. + */ + ASSERT_EQ(skel->bss->uprobe_session_result[0], 2, "uprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uprobe_session_result[1], 4, "uprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uprobe_session_result[2], 2, "uprobe_multi_func_3_result"); + + /* We expect increase in 3 entry and 1 return session calls -> 4 */ + ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 4, "uprobe_multi_sleep_result"); + +cleanup: + bpf_link__destroy(link); + uprobe_multi_session__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -703,4 +748,6 @@ void test_uprobe_multi_test(void) test_bench_attach_usdt(); if (test__start_subtest("attach_api_fails")) test_attach_api_fails(); + if (test__start_subtest("session")) + test_session_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session.c new file mode 100644 index 000000000000..30bff90b68dc --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__u64 uprobe_multi_func_1_addr = 0; +__u64 uprobe_multi_func_2_addr = 0; +__u64 uprobe_multi_func_3_addr = 0; + +__u64 uprobe_session_result[3] = {}; +__u64 uprobe_multi_sleep_result = 0; + +void *user_ptr = 0; +int pid = 0; + +static int uprobe_multi_check(void *ctx, bool is_return) +{ + const __u64 funcs[] = { + uprobe_multi_func_1_addr, + uprobe_multi_func_2_addr, + uprobe_multi_func_3_addr, + }; + unsigned int i; + __u64 addr; + + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + addr = bpf_get_func_ip(ctx); + + for (i = 0; i < ARRAY_SIZE(funcs); i++) { + if (funcs[i] == addr) { + uprobe_session_result[i]++; + break; + } + } + + /* only uprobe_multi_func_2 executes return probe */ + if ((addr == uprobe_multi_func_1_addr) || + (addr == uprobe_multi_func_3_addr)) + return 1; + + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_*") +int uprobe(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, bpf_session_is_return()); +} + +static __always_inline bool verify_sleepable_user_copy(void) +{ + char data[9]; + + bpf_copy_from_user(data, sizeof(data), user_ptr); + return bpf_strncmp(data, sizeof(data), "test_data") == 0; +} + +SEC("uprobe.session.s//proc/self/exe:uprobe_multi_func_*") +int uprobe_sleepable(struct pt_regs *ctx) +{ + if (verify_sleepable_user_copy()) + uprobe_multi_sleep_result++; + return uprobe_multi_check(ctx, bpf_session_is_return()); +} From patchwork Mon Sep 9 07:45:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796226 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76A241B143D; Mon, 9 Sep 2024 07:47:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868031; cv=none; b=MN5cpjlSGvOKNfjsJAVXyhYzBSknSX5fEZp0622T9O8bOb9p36781mAaUkqLDYMonV4XPTRd5sfV49SmTOfTNQNaAUefrlqTaAPvgmNnnaTBmwZnzRC2Gj1qc2FaM177Mk/F3iYEziSL+HExSI6ThnxVw7ZdiGCJFcSOsLcKWMM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868031; c=relaxed/simple; bh=RU1guL+oeamaHcos0dxdcHWw1bg8Cmx/4IIKvNEPfcM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pk2D2His+qvzQgFmJt5/NPkPqUJd8f98heY/HIBlTeE5JH6UVIBuxCeQzwkbrnEmEgt6/7pA5Brl7gcjGlM5AoQHtKYYdNaLrej8BaCJGOYLlnskffrRcqOQIFfUO30ESVq719VDwEdkniPV4zkJauSxLUadQnExRIavoQx+CAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=upgC1/tM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="upgC1/tM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0418AC4CEC5; Mon, 9 Sep 2024 07:47:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725868031; bh=RU1guL+oeamaHcos0dxdcHWw1bg8Cmx/4IIKvNEPfcM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=upgC1/tM4mfKa9ONxXNoEkLId4JVdQBgYAJkjL0vb9v5IJoEIpT/jCD4T8c2rdcbu vglugM+Feiv+25YaC2YUdTFNMc+TXQm71eMYVp77fgU+4WOEktCarEZ4YrHzK45U2n fE2An136niRsX7jnQ6wFNmfh0aCqDsUyQbG38CyCX5RVPhaas3wiu2eH1pyE7yNR/F HM+DUIn8IiZjrfYA2X65CZVJysYhKS36t62R/EnPRMTjsAMIzMGhq2FEg+qTkopz2x ugJhXzojZH0AbC5iUs7/u3MJNasKXWTooLPeDSixcv034RB4taCCWbfDPsVVuLg2sW QR28iSsY9WUrA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 6/7] selftests/bpf: Add uprobe session cookie test Date: Mon, 9 Sep 2024 09:45:53 +0200 Message-ID: <20240909074554.2339984-7-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test that verifies the cookie value get properly propagated from entry to return program. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 31 ++++++++++++ .../bpf/progs/uprobe_multi_session_cookie.c | 48 +++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index cc32288bfe26..8f56066a0195 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -7,6 +7,7 @@ #include "uprobe_multi_bench.skel.h" #include "uprobe_multi_usdt.skel.h" #include "uprobe_multi_session.skel.h" +#include "uprobe_multi_session_cookie.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -660,6 +661,34 @@ static void test_session_skel_api(void) uprobe_multi_session__destroy(skel); } +static void test_session_cookie_skel_api(void) +{ + struct uprobe_multi_session_cookie *skel = NULL; + int err; + + skel = uprobe_multi_session_cookie__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_cookie__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_cookie__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_cookie__attach")) + goto cleanup; + + /* trigger all probes */ + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + ASSERT_EQ(skel->bss->test_uprobe_1_result, 1, "test_uprobe_1_result"); + ASSERT_EQ(skel->bss->test_uprobe_2_result, 2, "test_uprobe_2_result"); + ASSERT_EQ(skel->bss->test_uprobe_3_result, 3, "test_uprobe_3_result"); + +cleanup: + uprobe_multi_session_cookie__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -750,4 +779,6 @@ void test_uprobe_multi_test(void) test_attach_api_fails(); if (test__start_subtest("session")) test_session_skel_api(); + if (test__start_subtest("session_cookie")) + test_session_cookie_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c new file mode 100644 index 000000000000..5befdf944dc6 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" + +char _license[] SEC("license") = "GPL"; + +int pid = 0; + +__u64 test_uprobe_1_result = 0; +__u64 test_uprobe_2_result = 0; +__u64 test_uprobe_3_result = 0; + +static int check_cookie(__u64 val, __u64 *result) +{ + __u64 *cookie; + + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + cookie = bpf_session_cookie(); + + if (bpf_session_is_return()) + *result = *cookie == val ? val : 0; + else + *cookie = val; + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_1(struct pt_regs *ctx) +{ + return check_cookie(1, &test_uprobe_1_result); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_2") +int uprobe_2(struct pt_regs *ctx) +{ + return check_cookie(2, &test_uprobe_2_result); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_3") +int uprobe_3(struct pt_regs *ctx) +{ + return check_cookie(3, &test_uprobe_3_result); +} From patchwork Mon Sep 9 07:45:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13796227 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DC0C1B14F8; Mon, 9 Sep 2024 07:47:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868042; cv=none; b=qNBbtSN1ZAiBu4EfwMCeIxFO1L2TGtbSpOknubE4id2GZH04k1CPz0GgdydKbDTiala39+pspK/OBIzekUYNFiLF1XvLH4MdnpUewH0dkAzbt/gAEEPmMaxmMhyILs/AiEudAttF5oZhGao8FqFxPQVSZILz4FjZd/3+tS2Y2B8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725868042; c=relaxed/simple; bh=PEISwKvon8efj26pn461xWp2eb5EhQQ9wpFEP/2ywWc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DH4Tb/h44C6Uq43HlQjQq+j8zpnwg6DeIefB+uJEXxtMGlyASVor/yZWAGzaiIDsWJfCGL2piaKqaovg93wwVaTEeumtHrwmlLHkQJ4+7OvU2hfbdFAZUxoY5FT7BalUG+FtfXR9JZ7eyZ2/ZbJWI+pNbwHYslR8UpVYWvzH4fc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZXpYvmdB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZXpYvmdB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58560C4CEC8; Mon, 9 Sep 2024 07:47:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725868042; bh=PEISwKvon8efj26pn461xWp2eb5EhQQ9wpFEP/2ywWc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZXpYvmdBEIxQnyKn576dsBfBJPQCdgsodtlXtHUtUSOOdWcmr2sitEWf3jHJHlG67 WMQawbC5kSX1epBLCGLq6F6MuibW9RdK4zQ3FysK+H8JmvEwKldoZfBkQo9PJuxup+ wr+5Rb22+8ONdwN0ldwBsuxI9H6irkHc3pXEnf0swI/SkIRqU23VgeB8BA/Iy1yHwz An0gVZ7rdlHGWGXWXzZaC6I4em6xeqPns8jP4B4K6wJwnI79Rk3SALbDtU6jbuZWjn rhSOpAeMPTqJR70GfvEhyyRGQ0ZBF6R97wRNOLZhK0Dy/UiPwN2KB1u4UbG2oeTsF+ Vdf0lJFFNu0Sw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv3 7/7] selftests/bpf: Add uprobe session recursive test Date: Mon, 9 Sep 2024 09:45:54 +0200 Message-ID: <20240909074554.2339984-8-jolsa@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240909074554.2339984-1-jolsa@kernel.org> References: <20240909074554.2339984-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test that verifies the cookie value is stored properly when single uprobe-ed function is executed recursively. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 57 +++++++++++++++++++ .../progs/uprobe_multi_session_recursive.c | 44 ++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 8f56066a0195..71d1fc00b2f4 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -8,6 +8,7 @@ #include "uprobe_multi_usdt.skel.h" #include "uprobe_multi_session.skel.h" #include "uprobe_multi_session_cookie.skel.h" +#include "uprobe_multi_session_recursive.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -34,6 +35,12 @@ noinline void usdt_trigger(void) STAP_PROBE(test, pid_filter_usdt); } +noinline void uprobe_session_recursive(int i) +{ + if (i) + uprobe_session_recursive(i - 1); +} + struct child { int go[2]; int c2p[2]; /* child -> parent channel */ @@ -689,6 +696,54 @@ static void test_session_cookie_skel_api(void) uprobe_multi_session_cookie__destroy(skel); } +static void test_session_recursive_skel_api(void) +{ + struct uprobe_multi_session_recursive *skel = NULL; + int i, err; + + skel = uprobe_multi_session_recursive__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_recursive__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_recursive__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_recursive__attach")) + goto cleanup; + + for (i = 0; i < ARRAY_SIZE(skel->bss->test_uprobe_cookie_entry); i++) + skel->bss->test_uprobe_cookie_entry[i] = i + 1; + + uprobe_session_recursive(5); + + /* + * entry uprobe: + * uprobe_session_recursive(5) { *cookie = 1, return 0 + * uprobe_session_recursive(4) { *cookie = 2, return 1 + * uprobe_session_recursive(3) { *cookie = 3, return 0 + * uprobe_session_recursive(2) { *cookie = 4, return 1 + * uprobe_session_recursive(1) { *cookie = 5, return 0 + * uprobe_session_recursive(0) { *cookie = 6, return 1 + * return uprobe: + * } i = 0 not executed + * } i = 1 test_uprobe_cookie_return[0] = 5 + * } i = 2 not executed + * } i = 3 test_uprobe_cookie_return[1] = 3 + * } i = 4 not executed + * } i = 5 test_uprobe_cookie_return[2] = 1 + */ + + ASSERT_EQ(skel->bss->idx_entry, 6, "idx_entry"); + ASSERT_EQ(skel->bss->idx_return, 3, "idx_return"); + + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[0], 5, "test_uprobe_cookie_return[0]"); + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[1], 3, "test_uprobe_cookie_return[1]"); + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[2], 1, "test_uprobe_cookie_return[2]"); + +cleanup: + uprobe_multi_session_recursive__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -781,4 +836,6 @@ void test_uprobe_multi_test(void) test_session_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); + if (test__start_subtest("session_cookie_recursive")) + test_session_recursive_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c new file mode 100644 index 000000000000..8fbcd69fae22 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +int pid = 0; + +int idx_entry = 0; +int idx_return = 0; + +__u64 test_uprobe_cookie_entry[6]; +__u64 test_uprobe_cookie_return[3]; + +static int check_cookie(void) +{ + __u64 *cookie = bpf_session_cookie(); + + if (bpf_session_is_return()) { + if (idx_return >= ARRAY_SIZE(test_uprobe_cookie_return)) + return 1; + test_uprobe_cookie_return[idx_return++] = *cookie; + return 0; + } + + if (idx_entry >= ARRAY_SIZE(test_uprobe_cookie_entry)) + return 1; + *cookie = test_uprobe_cookie_entry[idx_entry]; + return idx_entry++ % 2; +} + + +SEC("uprobe.session//proc/self/exe:uprobe_session_recursive") +int uprobe_recursive(struct pt_regs *ctx) +{ + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + return check_cookie(); +}