From patchwork Fri Oct 18 20:40:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842351 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FC9918C35F; Fri, 18 Oct 2024 20:41:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284090; cv=none; b=lXBwC6TuOP1YPVATooj0zdeXTOp6xBdUlAkzTWuFd0OOKJ3ijyZ6gZUce/JNXfUBJTRd1Ux4xcOr6dHLd4tD3gm24InGeGwgtHbcL5VTIhYMfKdnFncl1W+3uxt7yxwF9vUoRNxSpQwp3TX0S9cK6ZOFXrJ2Ni73UOUr/RUOM4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284090; c=relaxed/simple; bh=duof+l519DJRmJt0vEL9iIZDscSPcBGOX2moFFwv8XE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p8zx/Q/7mwy0PzD1acafxlDyuzbxfhHbUl+saWSRdBVg6vItdn1mM3SR28diFPi4UOpN6pofKq2vQVA3KMm/FUCBE7W1iaFhSTyzNmClHdhyiWHZ9nwKIjp/dL/+/EljFT1VlDwKaULn0ZIrvdGQiyqpbloa1bAzmyg7+l3eRq4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LRL7Xu+/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LRL7Xu+/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE3D4C4CEC3; Fri, 18 Oct 2024 20:41:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284089; bh=duof+l519DJRmJt0vEL9iIZDscSPcBGOX2moFFwv8XE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LRL7Xu+//QSDd0Lq7RLnZ6Zue0p/vtdWjX/UShbVCT9ZUNOqD1+D77CMlmEkDat9x PSeaJ5r1ZL1cOHq/whz0yIRFEeJzuwIB36/vpGzTEEQW4W96NR0x8bdyihEVdfg4Ul ik7JYOYQAJNCvYawPL/i11zP9bsjZUoS7Q6PjVPTO93fW3/uizyP2NNG3DY1R7i7PD wg2diRlmwFJHj/1R9w1asikKvNWfx2kyqU+dnFDql8oO5e6ppevRLt/foThCNNGkeY i2IsIufMIKeBttD24F3WC1bybt2O1cMjnuKwdgR/O5a2NzkrS8mAFVXT3K74iWBCIq 5AGkzpd/Q3rdw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 01/13] bpf: Allow return values 0 and 1 for kprobe session Date: Fri, 18 Oct 2024 22:40:57 +0200 Message-ID: <20241018204109.713820-2-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The kprobe session program can return only 0 or 1, instruct verifier to check for that. Fixes: 535a3692ba72 ("bpf: Add support for kprobe session attach") Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- kernel/bpf/verifier.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f514247ba8ba..5c941fd1b141 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15915,6 +15915,15 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char return -ENOTSUPP; } break; + case BPF_PROG_TYPE_KPROBE: + switch (env->prog->expected_attach_type) { + case BPF_TRACE_KPROBE_SESSION: + range = retval_range(0, 1); + break; + default: + return 0; + } + break; case BPF_PROG_TYPE_SK_LOOKUP: range = retval_range(SK_DROP, SK_PASS); break; From patchwork Fri Oct 18 20:40:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842352 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73A4618C35F; Fri, 18 Oct 2024 20:41:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284103; cv=none; b=QzfIh9LyX2Ih4E5p0yxDFqlV7wMpmLBpxntb+FZ+7vRP0ajZnfIcbQg6p1lcRVEo+2xIYyJXoRQm4+vbKCWER4Lzu3DCg0o3r2f62+viMF1U/afBdYUBIFvmnJgJJUd1SYUGXVWSj46ObLMbkU4kM+8iSNfoSeqefHwfq/iem8A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284103; c=relaxed/simple; bh=1HQNG9iWWO3wV8At9GP/jjQuJFQ7TICtEW1ksq9n0g0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q9GxqxfeKS5lQ80kmdd6oSwLRubyonjZNW7jOIsqecc/jIzoHcIqVoi82HHv7RQ0ogeSWL+YBza8dxEgg4aXaufRQ8dVDAwp9nrkRiVQT+E6Fj7ySfac+kKvXWWfABxCDaWKrOIyudx2h+c2mClv/scqjeeOlgyHeyXU9KjMCWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KlcSv6rB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KlcSv6rB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D7DAC4CEC3; Fri, 18 Oct 2024 20:41:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284103; bh=1HQNG9iWWO3wV8At9GP/jjQuJFQ7TICtEW1ksq9n0g0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KlcSv6rByKSeGI5t1xyThHi3wv/uhdTSddZUpO7unvqsFzXz2q+kYzcfbIV+uhfdG kLznR+hqvwpppSsZpKK3mhdWYq2tbjhIhY75Kr7oPREjrJJQSJFZHcqSa0I+xEhDgZ Q3fewB10CVt72DGeqlZLmqni8FKX1k9+yEu4sFpIdO26Egl4vmntV8P7AJLlZHwOi2 11cpxxwQlCwfhMbcu25mGmjKcZk1JGhjUW42JerCSXQ8OZ6b6XJ6KkeafPmh4eJB4Z APFM+U5N06mgn1VDUAvldoyysibASurPcZ+I8TsHSWEOXKyGfm0Ocjsw8uQm46Reo4 29AStF+WFTFnA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 02/13] bpf: Force uprobe bpf program to always return 0 Date: Fri, 18 Oct 2024 22:40:58 +0200 Message-ID: <20241018204109.713820-3-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net As suggested by Andrii make uprobe multi bpf programs to always return 0, so they can't force uprobe removal. Keeping the int return type for uprobe_prog_run, because it will be used in following session changes. Fixes: 89ae89f53d20 ("bpf: Add multi uprobe link") Suggested-by: Andrii Nakryiko Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- kernel/trace/bpf_trace.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index fdab7ecd8dfa..3c1e5a561df4 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3209,7 +3209,6 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, struct bpf_prog *prog = link->link.prog; bool sleepable = prog->sleepable; struct bpf_run_ctx *old_run_ctx; - int err = 0; if (link->task && !same_thread_group(current, link->task)) return 0; @@ -3222,7 +3221,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, migrate_disable(); old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); - err = bpf_prog_run(link->link.prog, regs); + bpf_prog_run(link->link.prog, regs); bpf_reset_run_ctx(old_run_ctx); migrate_enable(); @@ -3231,7 +3230,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, rcu_read_unlock_trace(); else rcu_read_unlock(); - return err; + return 0; } static bool From patchwork Fri Oct 18 20:40:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842353 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CD82206962; Fri, 18 Oct 2024 20:41:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284115; cv=none; b=hlyJ4Fuht2Bjfu/bzrqNTPE4r25bNmUOJ0dwX+gkZNG1JRlFlF1Syy2io9q208Bj3OXJsJFpDcjII+UpTIwrJgpq1ZkGu7lI9swMCYIv1Y1Y/QRezYaArslLCoobpsODvrYtIHy/H5bdXDHAiXtY9JVeMcjIMVVuDHO3UyJ+gqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284115; c=relaxed/simple; bh=loAUBML9cd5SCu/fBjpNadgXnQfN1v031ENvKILIlB8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F05Tc69dUqJ/uUuGnNL4NS4tjQwSat5Pfk+YrXuWfRHRyKm256wBzA62jQyeA7DTYR1+tvtVl4D4IR442exDYA8rU0niRGEa0g03BReh/naSM0NS6Iuh3f5WPcRo71CUCb2ocFHB8tDsIzVl+vhHbEWv7RSC7SuL0cuV9LjBuoI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=i20b9iSC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="i20b9iSC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58D15C4CEC3; Fri, 18 Oct 2024 20:41:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284114; bh=loAUBML9cd5SCu/fBjpNadgXnQfN1v031ENvKILIlB8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i20b9iSCwlcNoqwjQoCgMUQcgX/eBtHCyYJtpA0uS5ZOnzMx2EnCxomFhlDaEhkmP kEOTLMeKG2fO7kUJ+80N6Kvzk0CQ/NFf6YVz4t2blvAlBIZURVMyIhUsX3/aOX4+kW jCFzFm2lSSgWz45c8jjyF8Gz30yba/+2mxqxgHTSJUQbvEhh1hvmGHGY7Z1+Yudw5B 0Vu+2la9YnAYxsvVgy9M5FygbH/xAbIm7NaqpkpnkGiw0iaUUPYWo1W88uo0C59W1b 9nh0fyPxRWr6mmKwVsFeDt1hLK4i3JMcJA+J6EIjroIGT+gNiLFiy+f+eswwjiErfG GR+yBLUJZ0SSQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 03/13] bpf: Add support for uprobe multi session attach Date: Fri, 18 Oct 2024 22:40:59 +0200 Message-ID: <20241018204109.713820-4-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support to attach BPF program for entry and return probe of the same function. This is common use case which at the moment requires to create two uprobe multi links. Adding new BPF_TRACE_UPROBE_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe. It's possible to control execution of the BPF program on return probe simply by returning zero or non zero from the entry BPF program execution to execute or not the BPF program on return probe respectively. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 9 +++++++-- kernel/bpf/verifier.c | 1 + kernel/trace/bpf_trace.c | 36 +++++++++++++++++++++++++--------- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/libbpf.c | 1 + 6 files changed, 38 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 627c4195f04f..9b1ae0ccd427 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1116,6 +1116,7 @@ enum bpf_attach_type { BPF_NETKIT_PRIMARY, BPF_NETKIT_PEER, BPF_TRACE_KPROBE_SESSION, + BPF_TRACE_UPROBE_SESSION, __MAX_BPF_ATTACH_TYPE }; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 4d04d4d9c1f3..a4bbdb924093 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3984,10 +3984,14 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI && attach_type != BPF_TRACE_UPROBE_MULTI) return -EINVAL; + if (prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION && + attach_type != BPF_TRACE_UPROBE_SESSION) + return -EINVAL; if (attach_type != BPF_PERF_EVENT && attach_type != BPF_TRACE_KPROBE_MULTI && attach_type != BPF_TRACE_KPROBE_SESSION && - attach_type != BPF_TRACE_UPROBE_MULTI) + attach_type != BPF_TRACE_UPROBE_MULTI && + attach_type != BPF_TRACE_UPROBE_SESSION) return -EINVAL; return 0; case BPF_PROG_TYPE_SCHED_CLS: @@ -5240,7 +5244,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI || attr->link_create.attach_type == BPF_TRACE_KPROBE_SESSION) ret = bpf_kprobe_multi_link_attach(attr, prog); - else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI) + else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI || + attr->link_create.attach_type == BPF_TRACE_UPROBE_SESSION) ret = bpf_uprobe_multi_link_attach(attr, prog); break; default: diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5c941fd1b141..89bf0a58edbd 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15918,6 +15918,7 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char case BPF_PROG_TYPE_KPROBE: switch (env->prog->expected_attach_type) { case BPF_TRACE_KPROBE_SESSION: + case BPF_TRACE_UPROBE_SESSION: range = retval_range(0, 1); break; default: diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 3c1e5a561df4..5f10994376d0 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1557,6 +1557,17 @@ static inline bool is_kprobe_session(const struct bpf_prog *prog) return prog->expected_attach_type == BPF_TRACE_KPROBE_SESSION; } +static inline bool is_uprobe_multi(const struct bpf_prog *prog) +{ + return prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI || + prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION; +} + +static inline bool is_uprobe_session(const struct bpf_prog *prog) +{ + return prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION; +} + static const struct bpf_func_proto * kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { @@ -1574,13 +1585,13 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_get_func_ip: if (is_kprobe_multi(prog)) return &bpf_get_func_ip_proto_kprobe_multi; - if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + if (is_uprobe_multi(prog)) return &bpf_get_func_ip_proto_uprobe_multi; return &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: if (is_kprobe_multi(prog)) return &bpf_get_attach_cookie_proto_kmulti; - if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + if (is_uprobe_multi(prog)) return &bpf_get_attach_cookie_proto_umulti; return &bpf_get_attach_cookie_proto_trace; default: @@ -3074,6 +3085,7 @@ struct bpf_uprobe { u64 cookie; struct uprobe *uprobe; struct uprobe_consumer consumer; + bool session; }; struct bpf_uprobe_multi_link { @@ -3247,9 +3259,13 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, __u64 *data) { struct bpf_uprobe *uprobe; + int ret; uprobe = container_of(con, struct bpf_uprobe, consumer); - return uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + if (uprobe->session) + return ret ? UPROBE_HANDLER_IGNORE : 0; + return 0; } static int @@ -3259,7 +3275,8 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s struct bpf_uprobe *uprobe; uprobe = container_of(con, struct bpf_uprobe, consumer); - return uprobe_prog_run(uprobe, func, regs); + uprobe_prog_run(uprobe, func, regs); + return 0; } static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) @@ -3298,7 +3315,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr if (sizeof(u64) != sizeof(void *)) return -EOPNOTSUPP; - if (prog->expected_attach_type != BPF_TRACE_UPROBE_MULTI) + if (!is_uprobe_multi(prog)) return -EINVAL; flags = attr->link_create.uprobe_multi.flags; @@ -3374,11 +3391,12 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr uprobes[i].link = link; - if (flags & BPF_F_UPROBE_MULTI_RETURN) - uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; - else + if (!(flags & BPF_F_UPROBE_MULTI_RETURN)) uprobes[i].consumer.handler = uprobe_multi_link_handler; - + if (flags & BPF_F_UPROBE_MULTI_RETURN || is_uprobe_session(prog)) + uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; + if (is_uprobe_session(prog)) + uprobes[i].session = true; if (pid) uprobes[i].consumer.filter = uprobe_multi_link_filter; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 0e49ce2981a0..cbbcd206f221 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1116,6 +1116,7 @@ enum bpf_attach_type { BPF_NETKIT_PRIMARY, BPF_NETKIT_PEER, BPF_TRACE_KPROBE_SESSION, + BPF_TRACE_UPROBE_SESSION, __MAX_BPF_ATTACH_TYPE }; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 7c40286c3948..e650809e7170 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -133,6 +133,7 @@ static const char * const attach_type_name[] = { [BPF_NETKIT_PRIMARY] = "netkit_primary", [BPF_NETKIT_PEER] = "netkit_peer", [BPF_TRACE_KPROBE_SESSION] = "trace_kprobe_session", + [BPF_TRACE_UPROBE_SESSION] = "trace_uprobe_session", }; static const char * const link_type_name[] = { From patchwork Fri Oct 18 20:41:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842354 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92F142071FD; Fri, 18 Oct 2024 20:42:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284127; cv=none; b=WuIRU4DhF0gwUdbKp3CX8j9mwplsMEFrL8Ix2MH8eYtfpH5FYFlRLBZG49zrABEMVeqg1sXJvSU8T8gSUwendqaRVUuNqmKM9M4FWd9x3XmIO7/CxhubAABGPTfZqvv1UA1Q27ozdTYb7qsqF8XbJpKUGzyr+JiXLcU5j1HAGUQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284127; c=relaxed/simple; bh=EZ76JiF8nHrXB5Od2azxqfLxETX83Z2xeo/fMdQY5Ac=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ULJ+B6JjwGwpp8Qx3TkFG4hFbeSXlXlvF7MIJLvRD18TlsYTmCeKg2v542ai6MRWq56z5KaVkUEwhyyJwOF8hRQ1zumLmtLb7TksPogeyT8rO2HKbvghAEEGYTHIe/6dlta8a+GW4Rh3FesqoLbbOwYckJGeA0vgf+ePM8WhqJY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LelGUkOj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LelGUkOj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 54D95C4CEC3; Fri, 18 Oct 2024 20:42:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284127; bh=EZ76JiF8nHrXB5Od2azxqfLxETX83Z2xeo/fMdQY5Ac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LelGUkOjfY2tsxPqcPId8ML40Hd7u38ZT/JS+0f+w3wmXclrgNXwv7cv/dqCqWpqw 18ZxTzJ2FisKFUCEAtAAYuBp6TQJyqrs1oNIHc1rl9bkufG+X50CwNKgeRG6W2b0ma Pf2zgujf67qUJsmqSlzdzm8huHDKtqj1mM11PDFKpJiE5I4GWpMd9FXjeceJsxmOeh pn8c+TIsUnCUlcZX+BZfrTUTVSecFziGUiUTrBIdX7pr8R14Et2ALe1u17sLNGw3uI TawdhFZdIqYrUYOxbnOOr/93B7Kpg8EwTbagLIcBAAqJuRgzQFTEPOi4R3D7fWefbW wcXqiYKNX/qAw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 04/13] bpf: Add support for uprobe multi session context Date: Fri, 18 Oct 2024 22:41:00 +0200 Message-ID: <20241018204109.713820-5-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_uprobe_multi_run_ctx, so the session data can be retrieved from uprobe_multi link. Plus granting session kfuncs access to uprobe session programs. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- kernel/trace/bpf_trace.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 5f10994376d0..01868039d7bc 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -3098,7 +3098,7 @@ struct bpf_uprobe_multi_link { }; struct bpf_uprobe_multi_run_ctx { - struct bpf_run_ctx run_ctx; + struct bpf_session_run_ctx session_ctx; unsigned long entry_ip; struct bpf_uprobe *uprobe; }; @@ -3211,16 +3211,22 @@ static const struct bpf_link_ops bpf_uprobe_multi_link_lops = { static int uprobe_prog_run(struct bpf_uprobe *uprobe, unsigned long entry_ip, - struct pt_regs *regs) + struct pt_regs *regs, + bool is_return, void *data) { struct bpf_uprobe_multi_link *link = uprobe->link; struct bpf_uprobe_multi_run_ctx run_ctx = { + .session_ctx = { + .is_return = is_return, + .data = data, + }, .entry_ip = entry_ip, .uprobe = uprobe, }; struct bpf_prog *prog = link->link.prog; bool sleepable = prog->sleepable; struct bpf_run_ctx *old_run_ctx; + int err; if (link->task && !same_thread_group(current, link->task)) return 0; @@ -3232,8 +3238,8 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, migrate_disable(); - old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); - bpf_prog_run(link->link.prog, regs); + old_run_ctx = bpf_set_run_ctx(&run_ctx.session_ctx.run_ctx); + err = bpf_prog_run(link->link.prog, regs); bpf_reset_run_ctx(old_run_ctx); migrate_enable(); @@ -3242,7 +3248,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, rcu_read_unlock_trace(); else rcu_read_unlock(); - return 0; + return err; } static bool @@ -3262,7 +3268,7 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs, int ret; uprobe = container_of(con, struct bpf_uprobe, consumer); - ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs); + ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs, false, data); if (uprobe->session) return ret ? UPROBE_HANDLER_IGNORE : 0; return 0; @@ -3275,7 +3281,7 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s struct bpf_uprobe *uprobe; uprobe = container_of(con, struct bpf_uprobe, consumer); - uprobe_prog_run(uprobe, func, regs); + uprobe_prog_run(uprobe, func, regs, true, data); return 0; } @@ -3283,7 +3289,8 @@ static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; - run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, + session_ctx.run_ctx); return run_ctx->entry_ip; } @@ -3291,7 +3298,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; - run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, + session_ctx.run_ctx); return run_ctx->uprobe->cookie; } @@ -3485,7 +3493,7 @@ static int bpf_kprobe_multi_filter(const struct bpf_prog *prog, u32 kfunc_id) if (!btf_id_set8_contains(&kprobe_multi_kfunc_set_ids, kfunc_id)) return 0; - if (!is_kprobe_session(prog)) + if (!is_kprobe_session(prog) && !is_uprobe_session(prog)) return -EACCES; return 0; From patchwork Fri Oct 18 20:41:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842355 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D677204953; Fri, 18 Oct 2024 20:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284140; cv=none; b=IsojjcO3tj43Z2CZixR+dbCvWx8WHuocfL5wrzKbWzIEtz2TMA1n5OzNXfCzXwMWt2VPgat0mFC1fu5GPoNXN09FbASa6WF9ilRYD1no700DXKXqnxM3yEa6Lhg/O5kNUxxaeVlP1q5fU0nArH6xJGs5PqE+Pcj95mx9UXJYDxY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284140; c=relaxed/simple; bh=DxW/KBqi6/DBmyzf1midTZ0gP5HMdtwGGGkMPIckHwc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WLhvjQSmj04Ut40RFPkofSkGWu+oGHXn16SAKL0Le3rY/Mvdzo59UppeXnA6l8d/QKalJlE9S3JmJZ9LCJvaDl3pVa+f8jfTNkYz31N5E7neXJTwgg/zAAr/m+ifnR5xDL/Pv+TANCa/ComFjxOZTwRXCNdOELYDdS7eFNsrXaY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oJrZhr9c; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oJrZhr9c" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D0B2C4CEC3; Fri, 18 Oct 2024 20:42:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284140; bh=DxW/KBqi6/DBmyzf1midTZ0gP5HMdtwGGGkMPIckHwc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oJrZhr9cvJm/A1K1Zeo9JBrH5QX48nJugD2JeJEvgnQG/EgetVSpyiiw9ZCsQW4UC fiVPkISDVFB25vQmeDPv2JKI8/ktd8XlKRZzKqStNvfd8IXzjz1clfvNqK/QLS30ZU 53SEY1RTplEEFh2G7/2cW3+H1IK6U2fK8kyLQXXRH7Pj3MDM5KzxcLSck+sFHEtNgD Cix/jI3PRCbvP1LoDG7QobfxvlxCOD4zavT4ZSmgBShr3ki2+VlQ+WXipy4pJwxAs6 Up4z2kVjUUmELi0a6/NtPeE1a0hqJyltvqD2eNpnaE501ekd3gcklOr4iTF2f6o7zk MLcqLBUWqVr6w== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 05/13] libbpf: Add support for uprobe multi session attach Date: Fri, 18 Oct 2024 22:41:01 +0200 Message-ID: <20241018204109.713820-6-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support to attach program in uprobe session mode with bpf_program__attach_uprobe_multi function. Adding session bool to bpf_uprobe_multi_opts struct that allows to load and attach the bpf program via uprobe session. the attachment to create uprobe multi session. Also adding new program loader section that allows: SEC("uprobe.session/bpf_fentry_test*") and loads/attaches uprobe program as uprobe session. Adding sleepable hook (uprobe.session.s) as well. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 18 ++++++++++++++++-- tools/lib/bpf/libbpf.h | 4 +++- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2a4c71501a17..becdfa701c75 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -776,6 +776,7 @@ int bpf_link_create(int prog_fd, int target_fd, return libbpf_err(-EINVAL); break; case BPF_TRACE_UPROBE_MULTI: + case BPF_TRACE_UPROBE_SESSION: attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0); attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0); attr.link_create.uprobe_multi.path = ptr_to_u64(OPTS_GET(opts, uprobe_multi.path, 0)); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index e650809e7170..c45cd48840ae 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -9410,8 +9410,10 @@ static const struct bpf_sec_def section_defs[] = { SEC_DEF("kprobe.session+", KPROBE, BPF_TRACE_KPROBE_SESSION, SEC_NONE, attach_kprobe_session), SEC_DEF("uprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), SEC_DEF("uretprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), + SEC_DEF("uprobe.session+", KPROBE, BPF_TRACE_UPROBE_SESSION, SEC_NONE, attach_uprobe_multi), SEC_DEF("uprobe.multi.s+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi), SEC_DEF("uretprobe.multi.s+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi), + SEC_DEF("uprobe.session.s+", KPROBE, BPF_TRACE_UPROBE_SESSION, SEC_SLEEPABLE, attach_uprobe_multi), SEC_DEF("ksyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("kretsyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_USDT, attach_usdt), @@ -11733,7 +11735,9 @@ static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, stru ret = 0; break; case 3: + opts.session = str_has_pfx(probe_type, "uprobe.session"); opts.retprobe = str_has_pfx(probe_type, "uretprobe.multi"); + *link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts); ret = libbpf_get_error(*link); break; @@ -11982,10 +11986,12 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL; LIBBPF_OPTS(bpf_link_create_opts, lopts); unsigned long *resolved_offsets = NULL; + enum bpf_attach_type attach_type; int err = 0, link_fd, prog_fd; struct bpf_link *link = NULL; char errmsg[STRERR_BUFSIZE]; char full_path[PATH_MAX]; + bool retprobe, session; const __u64 *cookies; const char **syms; size_t cnt; @@ -12056,12 +12062,20 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, offsets = resolved_offsets; } + retprobe = OPTS_GET(opts, retprobe, false); + session = OPTS_GET(opts, session, false); + + if (retprobe && session) + return libbpf_err_ptr(-EINVAL); + + attach_type = session ? BPF_TRACE_UPROBE_SESSION : BPF_TRACE_UPROBE_MULTI; + lopts.uprobe_multi.path = path; lopts.uprobe_multi.offsets = offsets; lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; lopts.uprobe_multi.cookies = cookies; lopts.uprobe_multi.cnt = cnt; - lopts.uprobe_multi.flags = OPTS_GET(opts, retprobe, false) ? BPF_F_UPROBE_MULTI_RETURN : 0; + lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0; if (pid == 0) pid = getpid(); @@ -12075,7 +12089,7 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog, } link->detach = &bpf_link__detach_fd; - link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts); + link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts); if (link_fd < 0) { err = -errno; pr_warn("prog '%s': failed to attach multi-uprobe: %s\n", diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 91484303849c..b2ce3a72b11d 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -577,10 +577,12 @@ struct bpf_uprobe_multi_opts { size_t cnt; /* create return uprobes */ bool retprobe; + /* create session kprobes */ + bool session; size_t :0; }; -#define bpf_uprobe_multi_opts__last_field retprobe +#define bpf_uprobe_multi_opts__last_field session /** * @brief **bpf_program__attach_uprobe_multi()** attaches a BPF program From patchwork Fri Oct 18 20:41:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842368 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9CC0207204; Fri, 18 Oct 2024 20:42:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284153; cv=none; b=kWCDISjWBg88ZQy5zyhW8SuSVNiW0kfscm7t4IAGW3o0zBil2QxWRcN3HyBLnN3BF/cIJpyGPoD32YT53Z5O8YCurB7Qvn8mSEdbNIWH4DDaZMok9Nmgq7ywtwR1mXN8po4bvj6FZZOLonSMCnoqF0thnwhodBIIIikdiDXdr1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284153; c=relaxed/simple; bh=j1dFqoJ7759xgKk8JhBaUr84BXCHk2ygiMi3yhbs9o8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ub6+eeorkhTT4CU/ceLuvoD8RIn0c41qgQ+PoGhGfc8dzBsc0WAq7jwIxdr7l5CZ15u1Ueka39zWV/qRrV05FsrW5rsvTPq7FWLk7tBTCybXVgtxLDr+I91IOjASnLpgGACtJQu7CDHZAh6bLqgGJ/23109KfPfw8ZWa+D17Irk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KedzNGKs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KedzNGKs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4C71C4CEC3; Fri, 18 Oct 2024 20:42:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284152; bh=j1dFqoJ7759xgKk8JhBaUr84BXCHk2ygiMi3yhbs9o8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KedzNGKshoMKlwD74gMvN5R/6AFSseQXvJHn3ekuRsoIqCcepdjlPtCrAg14/H2Bw a6TGUTsCfoIp7v7OmAW6NkCUmELnok6btLbLTu/hNfnigeeRAz+tKzaRpyRvQNAr8a saDhKILHfBxdvRNueWSdAEOEAEkKHIUGBHk41cGCyqhuePi8grydmHny/msqdVoXTm ksdX1tJTJVLvJ9byjQTzRUS78CKN2clbYCrQrBbfRr4pOHUGv7kRSl/YJD5lTVtlAs PNQQHRJTzmTIg1lyAUjASo/q98W/16DRbQlnYIL+J25J/rHVxr7gSf7crb6Keybm5e R1J/JUke3wrQg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 06/13] selftests/bpf: Add uprobe session test Date: Fri, 18 Oct 2024 22:41:02 +0200 Message-ID: <20241018204109.713820-7-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test and testing that the entry program return value controls execution of the return probe program. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 47 ++++++++++++ .../bpf/progs/uprobe_multi_session.c | 71 +++++++++++++++++++ 2 files changed, 118 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 2c39902b8a09..b10d2dadb462 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -8,6 +8,7 @@ #include "uprobe_multi_usdt.skel.h" #include "uprobe_multi_consumers.skel.h" #include "uprobe_multi_pid_filter.skel.h" +#include "uprobe_multi_session.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -1015,6 +1016,50 @@ static void test_pid_filter_process(bool clone_vm) uprobe_multi_pid_filter__destroy(skel); } +static void test_session_skel_api(void) +{ + struct uprobe_multi_session *skel = NULL; + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + struct bpf_link *link = NULL; + int err; + + skel = uprobe_multi_session__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + skel->bss->user_ptr = test_data; + + err = uprobe_multi_session__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session__attach")) + goto cleanup; + + /* trigger all probes */ + skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3; + + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + /* + * We expect 2 for uprobe_multi_func_2 because it runs both entry/return probe, + * uprobe_multi_func_[13] run just the entry probe. All expected numbers are + * doubled, because we run extra test for sleepable session. + */ + ASSERT_EQ(skel->bss->uprobe_session_result[0], 2, "uprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uprobe_session_result[1], 4, "uprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uprobe_session_result[2], 2, "uprobe_multi_func_3_result"); + + /* We expect increase in 3 entry and 1 return session calls -> 4 */ + ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 4, "uprobe_multi_sleep_result"); + +cleanup: + bpf_link__destroy(link); + uprobe_multi_session__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -1111,4 +1156,6 @@ void test_uprobe_multi_test(void) test_pid_filter_process(false); if (test__start_subtest("filter_clone_vm")) test_pid_filter_process(true); + if (test__start_subtest("session")) + test_session_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session.c new file mode 100644 index 000000000000..30bff90b68dc --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__u64 uprobe_multi_func_1_addr = 0; +__u64 uprobe_multi_func_2_addr = 0; +__u64 uprobe_multi_func_3_addr = 0; + +__u64 uprobe_session_result[3] = {}; +__u64 uprobe_multi_sleep_result = 0; + +void *user_ptr = 0; +int pid = 0; + +static int uprobe_multi_check(void *ctx, bool is_return) +{ + const __u64 funcs[] = { + uprobe_multi_func_1_addr, + uprobe_multi_func_2_addr, + uprobe_multi_func_3_addr, + }; + unsigned int i; + __u64 addr; + + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + addr = bpf_get_func_ip(ctx); + + for (i = 0; i < ARRAY_SIZE(funcs); i++) { + if (funcs[i] == addr) { + uprobe_session_result[i]++; + break; + } + } + + /* only uprobe_multi_func_2 executes return probe */ + if ((addr == uprobe_multi_func_1_addr) || + (addr == uprobe_multi_func_3_addr)) + return 1; + + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_*") +int uprobe(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, bpf_session_is_return()); +} + +static __always_inline bool verify_sleepable_user_copy(void) +{ + char data[9]; + + bpf_copy_from_user(data, sizeof(data), user_ptr); + return bpf_strncmp(data, sizeof(data), "test_data") == 0; +} + +SEC("uprobe.session.s//proc/self/exe:uprobe_multi_func_*") +int uprobe_sleepable(struct pt_regs *ctx) +{ + if (verify_sleepable_user_copy()) + uprobe_multi_sleep_result++; + return uprobe_multi_check(ctx, bpf_session_is_return()); +} From patchwork Fri Oct 18 20:41:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842369 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A715208988; Fri, 18 Oct 2024 20:42:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284165; cv=none; b=X545pFjOwODiRZ0PvLTxpbP/nByoOKgNIYLuafZhZbK3scwoNTlIoqRutZDN7CnzX6t8dIoCGQMBfxxW/MCDBBzC+jPRBXlCE68oIc/tnBCgG6Eh86MbrzLj0kt+htEQl/2i7sbLFwz7Y/CA6Tmd+M/NwHchiDQkVMLGA+fsjVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284165; c=relaxed/simple; bh=tiFWq4stA78Nh3pJt3/jzsTBQ26T8kB6IeqRBp9iaoQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IPmXFcM5p8wBPmHqBXjN6iFPGAupirci+GpylS9sYp1YZJP3z1OrcsrgrZkoCC88SJ16uDom0jbMVmm5H5selY0om97h9XjN5/CRS4IH8crIPMK06WgcFZVhiIq0KTzK7BrAVzlVYXVK3Iy23b8jo77uKDsj3IlrLcwsqKvPlYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cNuZm5+x; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cNuZm5+x" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 113AEC4CEC3; Fri, 18 Oct 2024 20:42:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284164; bh=tiFWq4stA78Nh3pJt3/jzsTBQ26T8kB6IeqRBp9iaoQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cNuZm5+x0S4tVeFLg0yfff5CbLMQ/NNcC2fWJS4Vq9EbP14zH5aCVGiYyxYntg6T5 xIek2CRQxRu9u5aA/uCSyv+zGGiAcBodKLg+cPU9X/SQChYCNGJ45x1/iL2Ji4wJ0c 5fX9k6EOu3u37A0TFblz1m/LmTJ6CP4U8UT/BH1XnFcyUE43CzkVxhldwXs/DoePOv 4gIDVl3S+hzWC0kWHfGMQKsdWteDPL4NTbx7RJ9mwG60SHzSG/IDR/W79oUaBQh0xk ZPzfwv7KeJafaH8fL+YDB2mQifyF35ZZgF/KNC6Rkm6n3T8sQP4wHmeSu0UCkxNgdv TCkbQiWxJpT9g== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 07/13] selftests/bpf: Add uprobe session cookie test Date: Fri, 18 Oct 2024 22:41:03 +0200 Message-ID: <20241018204109.713820-8-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test that verifies the cookie value get properly propagated from entry to return program. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 31 ++++++++++++ .../bpf/progs/uprobe_multi_session_cookie.c | 48 +++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index b10d2dadb462..cc9030e86821 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -9,6 +9,7 @@ #include "uprobe_multi_consumers.skel.h" #include "uprobe_multi_pid_filter.skel.h" #include "uprobe_multi_session.skel.h" +#include "uprobe_multi_session_cookie.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -1060,6 +1061,34 @@ static void test_session_skel_api(void) uprobe_multi_session__destroy(skel); } +static void test_session_cookie_skel_api(void) +{ + struct uprobe_multi_session_cookie *skel = NULL; + int err; + + skel = uprobe_multi_session_cookie__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_cookie__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_cookie__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_cookie__attach")) + goto cleanup; + + /* trigger all probes */ + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + ASSERT_EQ(skel->bss->test_uprobe_1_result, 1, "test_uprobe_1_result"); + ASSERT_EQ(skel->bss->test_uprobe_2_result, 2, "test_uprobe_2_result"); + ASSERT_EQ(skel->bss->test_uprobe_3_result, 3, "test_uprobe_3_result"); + +cleanup: + uprobe_multi_session_cookie__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -1158,4 +1187,6 @@ void test_uprobe_multi_test(void) test_pid_filter_process(true); if (test__start_subtest("session")) test_session_skel_api(); + if (test__start_subtest("session_cookie")) + test_session_cookie_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c new file mode 100644 index 000000000000..5befdf944dc6 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" + +char _license[] SEC("license") = "GPL"; + +int pid = 0; + +__u64 test_uprobe_1_result = 0; +__u64 test_uprobe_2_result = 0; +__u64 test_uprobe_3_result = 0; + +static int check_cookie(__u64 val, __u64 *result) +{ + __u64 *cookie; + + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + cookie = bpf_session_cookie(); + + if (bpf_session_is_return()) + *result = *cookie == val ? val : 0; + else + *cookie = val; + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_1(struct pt_regs *ctx) +{ + return check_cookie(1, &test_uprobe_1_result); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_2") +int uprobe_2(struct pt_regs *ctx) +{ + return check_cookie(2, &test_uprobe_2_result); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_3") +int uprobe_3(struct pt_regs *ctx) +{ + return check_cookie(3, &test_uprobe_3_result); +} From patchwork Fri Oct 18 20:41:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842370 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78CC4208984; Fri, 18 Oct 2024 20:42:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284178; cv=none; b=Vpht0jdIjGY6q60IveGw1JJ0+pJontNILdag2mT0nMU2Uyn0f6w5NxoFy3CNtBh5IX2rBVcbk6IrXYPZ6hW4aPNwhtApB3O0WbVqfXZXqhpPGqhoIYG+mdjl3JpOmvWGLn9xZkMesi/d1uIoRDPEhoohcvJ9PO3I2Zyff6eSXRA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284178; c=relaxed/simple; bh=e/JgbYiCi/7UBhwN/I0I8AkHVXsJpTb8JQnSwgJINjw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Mk2qiCHpmv0YI/YdBOnys2WvpdA4ZgGu7kMSbg6WcgLXzH5moEoEjzaSjFKpYB+KLnvjAs6fLx75gw5Hb5YBxvssVnIgum63vEJLyG5lLyG/tG8wmDHGrhBWWkgJG6Jq76TOBn0hJ6gT33QPLDSU71p9zvf7tVF2FSlJ8uzDqX0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g42RI5FU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g42RI5FU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EAB86C4CEC3; Fri, 18 Oct 2024 20:42:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284178; bh=e/JgbYiCi/7UBhwN/I0I8AkHVXsJpTb8JQnSwgJINjw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g42RI5FUqYvpCqgjSVAOjT/xJv1vRrxE50tNZteyEUIGWkxwGeIqbd8ykqDxRcXZp XS/khEBGp0uMTchlOSo992Liesp9njgo6Fsu4gM8vzpbT+zt7McIR13YuPjhxQFuPs MZ1pgWHEe3cbwwBySNzwh3nBNsrEWtq4UXw8BEDGiDjK4pgU13i/Io7E/1qT7dVg4+ 0To4nWTntpiRSOakrMYD3+tcLc4wUVgZrd02siZlLpGNXlC0YzKz1CeFaGfJO3DjDZ fPvcFHKX1C4nFSUb7w3v5mw2p/0B9Rjp446thD5//7kc0EuOjW/x4F/hzntUzXDkwA Uq+GRSzgupzwQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 08/13] selftests/bpf: Add uprobe session recursive test Date: Fri, 18 Oct 2024 22:41:04 +0200 Message-ID: <20241018204109.713820-9-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session test that verifies the cookie value is stored properly when single uprobe-ed function is executed recursively. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 57 +++++++++++++++++++ .../progs/uprobe_multi_session_recursive.c | 44 ++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index cc9030e86821..284cd7fce576 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -10,6 +10,7 @@ #include "uprobe_multi_pid_filter.skel.h" #include "uprobe_multi_session.skel.h" #include "uprobe_multi_session_cookie.skel.h" +#include "uprobe_multi_session_recursive.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -36,6 +37,12 @@ noinline void usdt_trigger(void) STAP_PROBE(test, pid_filter_usdt); } +noinline void uprobe_session_recursive(int i) +{ + if (i) + uprobe_session_recursive(i - 1); +} + struct child { int go[2]; int c2p[2]; /* child -> parent channel */ @@ -1089,6 +1096,54 @@ static void test_session_cookie_skel_api(void) uprobe_multi_session_cookie__destroy(skel); } +static void test_session_recursive_skel_api(void) +{ + struct uprobe_multi_session_recursive *skel = NULL; + int i, err; + + skel = uprobe_multi_session_recursive__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_recursive__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_recursive__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_recursive__attach")) + goto cleanup; + + for (i = 0; i < ARRAY_SIZE(skel->bss->test_uprobe_cookie_entry); i++) + skel->bss->test_uprobe_cookie_entry[i] = i + 1; + + uprobe_session_recursive(5); + + /* + * entry uprobe: + * uprobe_session_recursive(5) { *cookie = 1, return 0 + * uprobe_session_recursive(4) { *cookie = 2, return 1 + * uprobe_session_recursive(3) { *cookie = 3, return 0 + * uprobe_session_recursive(2) { *cookie = 4, return 1 + * uprobe_session_recursive(1) { *cookie = 5, return 0 + * uprobe_session_recursive(0) { *cookie = 6, return 1 + * return uprobe: + * } i = 0 not executed + * } i = 1 test_uprobe_cookie_return[0] = 5 + * } i = 2 not executed + * } i = 3 test_uprobe_cookie_return[1] = 3 + * } i = 4 not executed + * } i = 5 test_uprobe_cookie_return[2] = 1 + */ + + ASSERT_EQ(skel->bss->idx_entry, 6, "idx_entry"); + ASSERT_EQ(skel->bss->idx_return, 3, "idx_return"); + + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[0], 5, "test_uprobe_cookie_return[0]"); + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[1], 3, "test_uprobe_cookie_return[1]"); + ASSERT_EQ(skel->bss->test_uprobe_cookie_return[2], 1, "test_uprobe_cookie_return[2]"); + +cleanup: + uprobe_multi_session_recursive__destroy(skel); +} + static void test_bench_attach_uprobe(void) { long attach_start_ns = 0, attach_end_ns = 0; @@ -1189,4 +1244,6 @@ void test_uprobe_multi_test(void) test_session_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); + if (test__start_subtest("session_cookie_recursive")) + test_session_recursive_skel_api(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c new file mode 100644 index 000000000000..8fbcd69fae22 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +int pid = 0; + +int idx_entry = 0; +int idx_return = 0; + +__u64 test_uprobe_cookie_entry[6]; +__u64 test_uprobe_cookie_return[3]; + +static int check_cookie(void) +{ + __u64 *cookie = bpf_session_cookie(); + + if (bpf_session_is_return()) { + if (idx_return >= ARRAY_SIZE(test_uprobe_cookie_return)) + return 1; + test_uprobe_cookie_return[idx_return++] = *cookie; + return 0; + } + + if (idx_entry >= ARRAY_SIZE(test_uprobe_cookie_entry)) + return 1; + *cookie = test_uprobe_cookie_entry[idx_entry]; + return idx_entry++ % 2; +} + + +SEC("uprobe.session//proc/self/exe:uprobe_session_recursive") +int uprobe_recursive(struct pt_regs *ctx) +{ + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + return check_cookie(); +} From patchwork Fri Oct 18 20:41:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842371 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 598E220CCCF; Fri, 18 Oct 2024 20:43:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284190; cv=none; b=Shg3fM1CLI6cncwf9ZIr5esMr8B+I/zBqx9CZgbEpuhN20c4aJY2PAzvM94k6MNC5LCJKidM2+HOramI1iP/d+ps02d/OzWgrb2vT3Y9vxqnBypNIc9d93Da1r62bOFBF45OAy4apU8u02S8/U8aBiQD9Qdig2Rvg8vsZwjQ6MM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284190; c=relaxed/simple; bh=vWoH7hbVui+QQQvKzVGIAPGWJG2xt/YhgttiE1WizNY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qOOif1VS27bWn0w5jxQumpd4HS0Dt92yKoBKcZwzvAsCZLtyOw1gnP8hGh1RUjfS3OI1WVOsQwIpptmkq2QbW+cYkUfU+a8R75mPKdP6WWiJtU2F0aqonM6MDYaF3sVL/PbOtLjsUzWKlcgke5QLpay3SSymA1qz0SdQXjMbVuU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OwMYJZfh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OwMYJZfh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8092AC4CEC3; Fri, 18 Oct 2024 20:43:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284190; bh=vWoH7hbVui+QQQvKzVGIAPGWJG2xt/YhgttiE1WizNY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OwMYJZfhC3ujrslVfOGOLK0XJ/5NJx7DbBQuD0emP34DfrUdxR9CryGnD7i9+rZe5 pdh+9RN9riOkSBseaTlDCmrnbOcSgRGoQJ9lN/VG7k7SDTk/kw88b0UB4mb5564NOU g5+8VWIm1GvSgsrZ3OzssOJkrZzkYMH5yxCsLGHKNJvwajugM4r6fxXTXz3tcG4Nu/ vOoqiRP3vMO9Vl9ircH0fN5t1/+xoWfoBhgYMraRn/8KT2XqUimcz6ffw0UFg9cT9M SiAym6L4DWQy2AJH/L+9CPtTZvNMQDlZzC6j4kn8IKviqKtZ9y3ds2qAtAJ77p9/QF RWrsbuiAoyYnA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 09/13] selftests/bpf: Add uprobe session verifier test for return value Date: Fri, 18 Oct 2024 22:41:05 +0200 Message-ID: <20241018204109.713820-10-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Making sure uprobe.session program can return only [0,1] values. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 2 ++ .../bpf/progs/uprobe_multi_verifier.c | 31 +++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 284cd7fce576..e693eeb1a5a5 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -11,6 +11,7 @@ #include "uprobe_multi_session.skel.h" #include "uprobe_multi_session_cookie.skel.h" #include "uprobe_multi_session_recursive.skel.h" +#include "uprobe_multi_verifier.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" #include "../sdt.h" @@ -1246,4 +1247,5 @@ void test_uprobe_multi_test(void) test_session_cookie_skel_api(); if (test__start_subtest("session_cookie_recursive")) test_session_recursive_skel_api(); + RUN_TESTS(uprobe_multi_verifier); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c b/tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c new file mode 100644 index 000000000000..fe49f2cb5360 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + + +SEC("uprobe.session") +__success +int uprobe_sesison_return_0(struct pt_regs *ctx) +{ + return 0; +} + +SEC("uprobe.session") +__success +int uprobe_sesison_return_1(struct pt_regs *ctx) +{ + return 1; +} + +SEC("uprobe.session") +__failure +__msg("At program exit the register R0 has smin=2 smax=2 should have been in [0, 1]") +int uprobe_sesison_return_2(struct pt_regs *ctx) +{ + return 2; +} From patchwork Fri Oct 18 20:41:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842372 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93E13209F5E; Fri, 18 Oct 2024 20:43:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284202; cv=none; b=ee2s6X1xUtr7rkIZ8r0wqZ0tO4vrns5EnBjBCt7KBOUc0A4eQMyehHGt+PP4f2mpM3lrLyZiIfntqwu4Jcb2dUyDyvqI5qNrh6aODZQSDTQqUNaIx4RH4Oyt2e7MXZJ2qoHowkFHlip/RtjaLvPYVY9qUtZhUQU4AH+nldamQWU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284202; c=relaxed/simple; bh=5RL/n5Qek/9MkjgkUP42e/4QpifIj1MSiLJuLX162Xc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mPSCq3s2Srt6JG+dE3lIy80qAre+IQfL2i9cn9KkxfOFjiWOv175jvMYdOddiJhzPwSy7Z/TKjuvQVU8L8VqoMrvAvVOcrOpuvpoejnCsK3jlEJf9HFCrbwBz9TGz+NZcNT+ZUREQ8Fx4rVDs9MgpqJHw7WjNa+o4WSimQB6jPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JbBXvjZi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JbBXvjZi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E11B8C4CEC3; Fri, 18 Oct 2024 20:43:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284202; bh=5RL/n5Qek/9MkjgkUP42e/4QpifIj1MSiLJuLX162Xc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JbBXvjZiqEKHf1zOwBLPwX2mBff88s31rMIXoIGEp4SzmrrVfon33aDFn2GdrsOyu U78STdqHavl3Bdvmi0d/sG9WiuU/ihXsmPvKlPfEB5h8iYR/YYs9+Bqxr/fOGrK+96 SizFU3Au2ipXvlGGnvjDupZTr/i0/GQ6ydUI65L3ZKn581sIET6A8phZ9YHFtLndRS 9M8wDIhwDs84IHz4lT6v5LwudbBxQvnAmRtQstFwfLFDQqfLR+4R+/ZPLFP+TrTrXe 7np7bmvDBlr07mr28jIzFV+1ZItbMV6+qZz79ZKzQoffRczi/LSJI128HzmqvfUMjF u39LNSTOBW3tQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 10/13] selftests/bpf: Add kprobe session verifier test for return value Date: Fri, 18 Oct 2024 22:41:06 +0200 Message-ID: <20241018204109.713820-11-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Making sure kprobe.session program can return only [0,1] values. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/kprobe_multi_test.c | 2 ++ .../bpf/progs/kprobe_multi_verifier.c | 31 +++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c index 960c9323d1e0..66ab1cae923e 100644 --- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c @@ -6,6 +6,7 @@ #include "kprobe_multi_override.skel.h" #include "kprobe_multi_session.skel.h" #include "kprobe_multi_session_cookie.skel.h" +#include "kprobe_multi_verifier.skel.h" #include "bpf/libbpf_internal.h" #include "bpf/hashmap.h" @@ -764,4 +765,5 @@ void test_kprobe_multi_test(void) test_session_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); + RUN_TESTS(kprobe_multi_verifier); } diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c b/tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c new file mode 100644 index 000000000000..288577e81deb --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + + +SEC("kprobe.session") +__success +int kprobe_session_return_0(struct pt_regs *ctx) +{ + return 0; +} + +SEC("kprobe.session") +__success +int kprobe_session_return_1(struct pt_regs *ctx) +{ + return 1; +} + +SEC("kprobe.session") +__failure +__msg("At program exit the register R0 has smin=2 smax=2 should have been in [0, 1]") +int kprobe_session_return_2(struct pt_regs *ctx) +{ + return 2; +} From patchwork Fri Oct 18 20:41:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842373 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC20E209F5A; Fri, 18 Oct 2024 20:43:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284214; cv=none; b=PjV2TVOH5Qcv5CryauLowCVHYGE9dMuCAws15s4bPPrrB8g63llIFGP0tT08Gz9wdDEqs+KcklPwfhINW+pdpUvfRCRXvROLEcAWDetmHU+nCqIAhWelckhx5nrAhL9Y+vGzf5kAalj56I/oPbzmv4hZdBM9QvcCua5VJme3xHw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284214; c=relaxed/simple; bh=/2llFpdGYPtAAfCPyuTx0mCmbSBP7WmIfDSVF0KVTJo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p4wzg275xw82CZCAYJfkFOgzautEyKweYhlYPqZFTfczXQ7i5LK2cwOz6WKpB2POnMqxL3CFK0mCd0IWtzAeVnW3faiFGfhTzSb1N3garzK1hxzyHSICEHOx61Ds0CiJwfb8Lg1phRwyqcFRVjJcG+hCP36MC53nvG2VujcF3qs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Wo0HTjTI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Wo0HTjTI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DBA1BC4CEC3; Fri, 18 Oct 2024 20:43:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284214; bh=/2llFpdGYPtAAfCPyuTx0mCmbSBP7WmIfDSVF0KVTJo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wo0HTjTIhvAt6DWRCDEY6TvBmAb1JI6iOVas0rBJ1/lnX0D+0rCvlr84HgSXmoRwm K8rAcHuEyTHt5Nnn4zu1oqF+9mB5K0TXoQn18oY2iNWQ9yRAJ5C8ESscF7h4DzHyjR HbPAGnofcdbW/71SBDkdVMWgouxA4ox1Y79tcWyvaIRoeaLKR2irMN4886Ps7CQ72p Zn5K4MF8OHD1tjStZuXsaGy6c1lh0oqBFDd91SgvElcD0Wv3KMV2f9Klg8CLZAS360 fZWYj0lxOCfUdmmANhlAQ6oSsNmpPKIhDVAWCZKHXAzRY6R7jHpYiM3KUohpRrtgg2 J+VvyzpXeIuaQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 11/13] selftests/bpf: Add uprobe session single consumer test Date: Fri, 18 Oct 2024 22:41:07 +0200 Message-ID: <20241018204109.713820-12-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Testing that the session ret_handler bypass works on single uprobe with multiple consumers, each with different session ignore return value. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 33 ++++++++++++++ .../bpf/progs/uprobe_multi_session_single.c | 44 +++++++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index e693eeb1a5a5..7e0228f8fcfc 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -9,6 +9,7 @@ #include "uprobe_multi_consumers.skel.h" #include "uprobe_multi_pid_filter.skel.h" #include "uprobe_multi_session.skel.h" +#include "uprobe_multi_session_single.skel.h" #include "uprobe_multi_session_cookie.skel.h" #include "uprobe_multi_session_recursive.skel.h" #include "uprobe_multi_verifier.skel.h" @@ -1069,6 +1070,36 @@ static void test_session_skel_api(void) uprobe_multi_session__destroy(skel); } +static void test_session_single_skel_api(void) +{ + struct uprobe_multi_session_single *skel = NULL; + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + int err; + + skel = uprobe_multi_session_single__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_single__open_and_load")) + goto cleanup; + + skel->bss->pid = getpid(); + + err = uprobe_multi_session_single__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi_session_single__attach")) + goto cleanup; + + uprobe_multi_func_1(); + + /* + * We expect consumer 0 and 2 to trigger just entry handler (value 1) + * and consumer 1 to hit both (value 2). + */ + ASSERT_EQ(skel->bss->uprobe_session_result[0], 1, "uprobe_session_result_0"); + ASSERT_EQ(skel->bss->uprobe_session_result[1], 2, "uprobe_session_result_1"); + ASSERT_EQ(skel->bss->uprobe_session_result[2], 1, "uprobe_session_result_2"); + +cleanup: + uprobe_multi_session_single__destroy(skel); +} + static void test_session_cookie_skel_api(void) { struct uprobe_multi_session_cookie *skel = NULL; @@ -1243,6 +1274,8 @@ void test_uprobe_multi_test(void) test_pid_filter_process(true); if (test__start_subtest("session")) test_session_skel_api(); + if (test__start_subtest("session_single")) + test_session_single_skel_api(); if (test__start_subtest("session_cookie")) test_session_cookie_skel_api(); if (test__start_subtest("session_cookie_recursive")) diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c b/tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c new file mode 100644 index 000000000000..7c960376ae97 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_kfuncs.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__u64 uprobe_session_result[3] = {}; +int pid = 0; + +static int uprobe_multi_check(void *ctx, int idx) +{ + if (bpf_get_current_pid_tgid() >> 32 != pid) + return 1; + + uprobe_session_result[idx]++; + + /* only consumer 1 executes return probe */ + if (idx == 0 || idx == 2) + return 1; + + return 0; +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_0(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, 0); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_1(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, 1); +} + +SEC("uprobe.session//proc/self/exe:uprobe_multi_func_1") +int uprobe_2(struct pt_regs *ctx) +{ + return uprobe_multi_check(ctx, 2); +} From patchwork Fri Oct 18 20:41:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842374 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72AEA2071FD; Fri, 18 Oct 2024 20:43:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284227; cv=none; b=kbucgszDdPMIQ43VcMwsAEgBZztO+J8xEn/aH+fIEOyYLYhHDVuFTDcDAxAYQwQZDaLFPErppgsNHnFrqWHOiaG5ZcmDCZyO5ZN93ikNN1BJ2K6ANRYxD4madHjlyniGjUAGD8Amumde+Twds4BmkhbJhNGSpjpPCKXz5IsdKts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284227; c=relaxed/simple; bh=SRJ7iIqpFdc57svkJVfVWiyC4I6Wo1VBhjnyzeaFXpg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uJWs0tewPg5dh555sNvVJ4pOEN7gCEbzSiZRi2z3kpHkkb/qxjhanVJFDV1Nw/mKtq4cg7L28ZvzxwQc4uVeVEJRqSL1G/HcyuL3sLNVn9t22uujXkpj+33ynMQNiNhkb6LqLKbTKAIiUKre/eQER+OYp7FStvLj5pdPeziy0DI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CqFxPx2D; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CqFxPx2D" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E3E3C4CEC5; Fri, 18 Oct 2024 20:43:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284227; bh=SRJ7iIqpFdc57svkJVfVWiyC4I6Wo1VBhjnyzeaFXpg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CqFxPx2D/DHSYVsSN9Pu1y5oUu4HCBCmqWdOq2JnOTompR1Tg76p/T2rxIBuB3JLS hFZAgsRIyVjZY6dUp97kNQ/+0G575G0HUQBY0aOA5FwBpRUGmPEIh4N7d0AsaiK8Jr y+CCdBf+cRkb2a+LxiS7o06mV/ZbRNRzXdibC7B/3Hbwbuey9vF9zpEOJLByafTnjC u9ZEkmHUyZ/InvDkOi4QdxgnLv6q2EeCnOFr1faTLdkUMzHzHS6oaY5Ziir13zHg9l 8q0STCIiau8ePa3sVGMoJ3NrKPMGW9GGYnCwttn8qp+SBZ3oAc4/vHtq2BO8T1fZOC Y/173ZRz2e5Fw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 12/13] selftests/bpf: Add uprobe sessions to consumer test Date: Fri, 18 Oct 2024 22:41:08 +0200 Message-ID: <20241018204109.713820-13-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding uprobe session consumers to the consumer test, so we get the session into the test mix. In addition scaling down the test to have just 1 uprobe and 1 uretprobe, otherwise the test time grows and is unsuitable for CI even with threads. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 68 +++++++++++++------ .../bpf/progs/uprobe_multi_consumers.c | 6 +- 2 files changed, 52 insertions(+), 22 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 7e0228f8fcfc..e96b153a0f5d 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -799,10 +799,13 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) return -1; /* - * bit/prog: 0,1 uprobe entry - * bit/prog: 2,3 uprobe return + * bit/prog: 0 uprobe entry + * bit/prog: 1 uprobe return + * bit/prog: 2 uprobe session without return + * bit/prog: 3 uprobe session with return */ - opts.retprobe = idx == 2 || idx == 3; + opts.retprobe = idx == 1; + opts.session = idx == 2 || idx == 3; *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", "uprobe_consumer_test", @@ -867,29 +870,55 @@ static int consumer_test(struct uprobe_multi_consumers *skel, goto cleanup; for (idx = 0; idx < 4; idx++) { + unsigned long had_uretprobes; const char *fmt = "BUG"; __u64 val = 0; - if (idx < 2) { + switch (idx) { + case 0: /* * uprobe entry * +1 if define in 'before' */ if (test_bit(idx, before)) val++; - fmt = "prog 0/1: uprobe"; - } else { + fmt = "prog 0: uprobe"; + break; + case 1: /* * to trigger uretprobe consumer, the uretprobe needs to be installed, * which means one of the 'return' uprobes was alive when probe was hit: * - * idxs: 2/3 uprobe return in 'installed' mask + * idxs: 1/2 uprobe return in 'installed' mask */ - unsigned long had_uretprobes = before & 0b1100; /* is uretprobe installed */ + had_uretprobes = before & 0b0110; /* is uretprobe installed */ if (had_uretprobes && test_bit(idx, after)) val++; - fmt = "idx 2/3: uretprobe"; + fmt = "prog 1: uretprobe"; + break; + case 2: + /* + * session with return + * +1 if defined in 'before' + * +1 if defined in 'after' + */ + if (test_bit(idx, before)) { + val++; + if (test_bit(idx, after)) + val++; + } + fmt = "prog 2: session with return"; + break; + case 3: + /* + * session without return + * +1 if defined in 'before' + */ + if (test_bit(idx, before)) + val++; + fmt = "prog 3: session with NO return"; + break; } if (!ASSERT_EQ(skel->bss->uprobe_result[idx], val, fmt)) @@ -918,8 +947,10 @@ static void test_consumers(void) * The idea of this test is to try all possible combinations of * uprobes consumers attached on single function. * - * - 2 uprobe entry consumer - * - 2 uprobe exit consumers + * - 1 uprobe entry consumer + * - 1 uprobe exit consumer + * - 1 uprobe session with return + * - 1 uprobe session without return * * The test uses 4 uprobes attached on single function, but that * translates into single uprobe with 4 consumers in kernel. @@ -927,25 +958,24 @@ static void test_consumers(void) * The before/after values present the state of attached consumers * before and after the probed function: * - * bit/prog 0,1 : uprobe entry - * bit/prog 2,3 : uprobe return + * bit/prog 0 : uprobe entry + * bit/prog 1 : uprobe return * * For example for: * - * before = 0b0101 - * after = 0b0110 + * before = 0b01 + * after = 0b10 * * it means that before we call 'uprobe_consumer_test' we attach * uprobes defined in 'before' value: * - * - bit/prog 0: uprobe entry - * - bit/prog 2: uprobe return + * - bit/prog 1: uprobe entry * * uprobe_consumer_test is called and inside it we attach and detach * uprobes based on 'after' value: * - * - bit/prog 0: stays untouched - * - bit/prog 2: uprobe return is detached + * - bit/prog 0: is detached + * - bit/prog 1: is attached * * uprobe_consumer_test returns and we check counters values increased * by bpf programs on each uprobe to match the expected count based on diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c b/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c index 7e0fdcbbd242..93752bb5690b 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_consumers.c @@ -24,16 +24,16 @@ int uprobe_1(struct pt_regs *ctx) return 0; } -SEC("uprobe.multi") +SEC("uprobe.session") int uprobe_2(struct pt_regs *ctx) { uprobe_result[2]++; return 0; } -SEC("uprobe.multi") +SEC("uprobe.session") int uprobe_3(struct pt_regs *ctx) { uprobe_result[3]++; - return 0; + return 1; } From patchwork Fri Oct 18 20:41:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13842375 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86D9E204953; Fri, 18 Oct 2024 20:43:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284239; cv=none; b=BFr9k5ddRnk3hMuqC8MwpTY9iCmc5e3KrdZw5kbh3YMYnt7/TXp9fChbzgceHKaxM6Oj81PYPMoS1lfK20CGsrMRL68vk+jl+j6lYMWUIK2fG/liV4yXjUd7Y/ZZbZgCN4wze+QG1Qo3WkCBjOgUczDRGTI2rODhnYlwTxJOr8Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729284239; c=relaxed/simple; bh=pTG0eoCi2cwlEEnPjFZATQKK+Rr2HsrTjR6DuJXFZ58=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HqwxX5pUcjlR7i+3PQOdMaQJQ6i0lpix/+S+Uxr0U/92wQs3B6P5LOVUbtg6eOQH/KzHYHzDhfbGdFpZPIPo9+SqyeBkudCUk+sBRS+YEZiAn/Zajsq61kJnt1SSstU+XkCS3S4j19C8n+1eUdGP+4ig3u8mueTh5Kn8VyFtJn8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Wm/epFNd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Wm/epFNd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62444C4CEC3; Fri, 18 Oct 2024 20:43:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729284239; bh=pTG0eoCi2cwlEEnPjFZATQKK+Rr2HsrTjR6DuJXFZ58=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wm/epFNdp8/hI7SQO+v6P5DXmbo0qJDGb/Ia5OhuIeIthknWkIprwpBObOa5aaKHy 1cqgbAIHmgI4P5cCnFeoIRLEMwLotY0tEx5dV744VJF6nlg1RGDauAbK/0Mci2nwzA 6IVya/y4AfTa9JQQ7Iqx4CQ9Bgq0WkcDZwoSai7bGDBSXOa8wr6sNsr6oYa/5ZVO7W ymIEz9xpNGr2j5ZVhD0jTbaLKfUigYfyF9kCeSiKXX072hDiKmHbCDpw1yHvqNdEuZ X/Kvyj4G3RTypLqVYTQFrCu0qY8jIDjSp3pnn4tlw8RdqJ9ozjyonYAI7dIo3Im+ao AuquiSRlnWXwg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCHv8 bpf-next 13/13] selftests/bpf: Add threads to consumer test Date: Fri, 18 Oct 2024 22:41:09 +0200 Message-ID: <20241018204109.713820-14-jolsa@kernel.org> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241018204109.713820-1-jolsa@kernel.org> References: <20241018204109.713820-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net With recent uprobe fix [1] the sync time after unregistering uprobe is much longer and prolongs the consumer test which creates and destroys hundreds of uprobes. This change adds 16 threads (which fits the test logic) and speeds up the test. Before the change: # perf stat --null ./test_progs -t uprobe_multi_test/consumers #421/9 uprobe_multi_test/consumers:OK #421 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -t uprobe_multi_test/consumers': 28.818778973 seconds time elapsed 0.745518000 seconds user 0.919186000 seconds sys After the change: # perf stat --null ./test_progs -t uprobe_multi_test/consumers 2>&1 #421/9 uprobe_multi_test/consumers:OK #421 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -t uprobe_multi_test/consumers': 3.504790814 seconds time elapsed 0.012141000 seconds user 0.751760000 seconds sys [1] commit 87195a1ee332 ("uprobes: switch to RCU Tasks Trace flavor for better performance") Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 98 +++++++++++++++---- 1 file changed, 80 insertions(+), 18 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index e96b153a0f5d..cdb986871835 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -789,7 +789,7 @@ get_link(struct uprobe_multi_consumers *skel, int link) } } -static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) +static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx, unsigned long offset) { struct bpf_program *prog = get_program(skel, idx); struct bpf_link **link = get_link(skel, idx); @@ -798,6 +798,9 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) if (!prog || !link) return -1; + opts.offsets = &offset; + opts.cnt = 1; + /* * bit/prog: 0 uprobe entry * bit/prog: 1 uprobe return @@ -807,9 +810,7 @@ static int uprobe_attach(struct uprobe_multi_consumers *skel, int idx) opts.retprobe = idx == 1; opts.session = idx == 2 || idx == 3; - *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", - "uprobe_consumer_test", - &opts); + *link = bpf_program__attach_uprobe_multi(prog, 0, "/proc/self/exe", NULL, &opts); if (!ASSERT_OK_PTR(*link, "bpf_program__attach_uprobe_multi")) return -1; return 0; @@ -830,7 +831,8 @@ static bool test_bit(int bit, unsigned long val) noinline int uprobe_consumer_test(struct uprobe_multi_consumers *skel, - unsigned long before, unsigned long after) + unsigned long before, unsigned long after, + unsigned long offset) { int idx; @@ -843,15 +845,43 @@ uprobe_consumer_test(struct uprobe_multi_consumers *skel, /* ... and attach all new programs in 'after' state */ for (idx = 0; idx < 4; idx++) { if (!test_bit(idx, before) && test_bit(idx, after)) { - if (!ASSERT_OK(uprobe_attach(skel, idx), "uprobe_attach_after")) + if (!ASSERT_OK(uprobe_attach(skel, idx, offset), "uprobe_attach_after")) return -1; } } return 0; } +/* + * We generate 16 consumer_testX functions that will have uprobe installed on + * and will be called in separate threads. All function pointer are stored in + * "consumers" section and each thread will pick one function based on index. + */ + +extern const void *__start_consumers; + +#define __CONSUMER_TEST(func) \ +noinline int func(struct uprobe_multi_consumers *skel, unsigned long before, \ + unsigned long after, unsigned long offset) \ +{ \ + return uprobe_consumer_test(skel, before, after, offset); \ +} \ +static void *__ ## func __used __attribute__((section("consumers"))) = (void *) func; + +#define CONSUMER_TEST(func) __CONSUMER_TEST(func) + +#define C1 CONSUMER_TEST(__PASTE(consumer_test, __COUNTER__)) +#define C4 C1 C1 C1 C1 +#define C16 C4 C4 C4 C4 + +C16 + +typedef int (*test_t)(struct uprobe_multi_consumers *, unsigned long, + unsigned long, unsigned long); + static int consumer_test(struct uprobe_multi_consumers *skel, - unsigned long before, unsigned long after) + unsigned long before, unsigned long after, + test_t test, unsigned long offset) { int err, idx, ret = -1; @@ -860,12 +890,12 @@ static int consumer_test(struct uprobe_multi_consumers *skel, /* 'before' is each, we attach uprobe for every set idx */ for (idx = 0; idx < 4; idx++) { if (test_bit(idx, before)) { - if (!ASSERT_OK(uprobe_attach(skel, idx), "uprobe_attach_before")) + if (!ASSERT_OK(uprobe_attach(skel, idx, offset), "uprobe_attach_before")) goto cleanup; } } - err = uprobe_consumer_test(skel, before, after); + err = test(skel, before, after, offset); if (!ASSERT_EQ(err, 0, "uprobe_consumer_test")) goto cleanup; @@ -934,14 +964,46 @@ static int consumer_test(struct uprobe_multi_consumers *skel, return ret; } -static void test_consumers(void) +#define CONSUMER_MAX 16 + +/* + * Each thread runs 1/16 of the load by running test for single + * 'before' number (based on thread index) and full scale of + * 'after' numbers. + */ +static void *consumer_thread(void *arg) { + unsigned long idx = (unsigned long) arg; struct uprobe_multi_consumers *skel; - int before, after; + unsigned long offset; + const void *func; + int after; skel = uprobe_multi_consumers__open_and_load(); if (!ASSERT_OK_PTR(skel, "uprobe_multi_consumers__open_and_load")) - return; + return NULL; + + func = *((&__start_consumers) + idx); + + offset = get_uprobe_offset(func); + if (!ASSERT_GE(offset, 0, "uprobe_offset")) + goto out; + + for (after = 0; after < CONSUMER_MAX; after++) + if (consumer_test(skel, idx, after, func, offset)) + goto out; + +out: + uprobe_multi_consumers__destroy(skel); + return NULL; +} + + +static void test_consumers(void) +{ + pthread_t pt[CONSUMER_MAX]; + unsigned long idx; + int err; /* * The idea of this test is to try all possible combinations of @@ -982,14 +1044,14 @@ static void test_consumers(void) * before/after bits. */ - for (before = 0; before < 16; before++) { - for (after = 0; after < 16; after++) - if (consumer_test(skel, before, after)) - goto out; + for (idx = 0; idx < CONSUMER_MAX; idx++) { + err = pthread_create(&pt[idx], NULL, consumer_thread, (void *) idx); + if (!ASSERT_OK(err, "pthread_create")) + break; } -out: - uprobe_multi_consumers__destroy(skel); + while (idx) + pthread_join(pt[--idx], NULL); } static struct bpf_program *uprobe_multi_program(struct uprobe_multi_pid_filter *skel, int idx)