From patchwork Mon Aug 7 08:59:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13343192 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CA64138E for ; Mon, 7 Aug 2023 09:00:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BFADC433C8; Mon, 7 Aug 2023 09:00:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691398812; bh=L6dfMfoTIHQiLyfj0h5CqxGr2nlb7MgYloTqHS1ZYyc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fOi7rwezDGZxKn38AdRwkosIf10mDMn3e6piJw4ofJbH06y1ARuK486dEKrRs79bZ W96EPGExI8oe13894l2q+mwMiSQLfPtt2QNkykZX3Sv2CPw9Oc+IZ25WBPcaZS6mSl KrYfoRZdDSjq6PbquG8eQj9Tr0JyZ0D1S855ey5SkAIoiNBM9x/cYmAhlw6R4dxyjF CY32+QoTjxkU4HnR69iHU/Xnetv4k4fRMm/cz+nm0r2LpcfaHlR9IrpqMkvFUG8OBP m5mL+gQs6ZEi115TmcoumAypulf5oFHWDeGqpiZlRjFRWkK6+nCNIw0Y+0AignqFke rMl8bRxw0Iy9A== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Alan Maguire , bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" , Steven Rostedt Subject: [PATCHv3 bpf-next 1/3] bpf: Add support for bpf_get_func_ip helper for uprobe program Date: Mon, 7 Aug 2023 10:59:54 +0200 Message-ID: <20230807085956.2344866-2-jolsa@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230807085956.2344866-1-jolsa@kernel.org> References: <20230807085956.2344866-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding support for bpf_get_func_ip helper for uprobe program to return probed address for both uprobe and return uprobe. We discussed this in [1] and agreed that uprobe can have special use of bpf_get_func_ip helper that differs from kprobe. The kprobe bpf_get_func_ip returns: - address of the function if probe is attach on function entry for both kprobe and return kprobe - 0 if the probe is not attach on function entry The uprobe bpf_get_func_ip returns: - address of the probe for both uprobe and return uprobe The reason for this semantic change is that kernel can't really tell if the probe user space address is function entry. The uprobe program is actually kprobe type program attached as uprobe. One of the consequences of this design is that uprobes do not have its own set of helpers, but share them with kprobes. As we need different functionality for bpf_get_func_ip helper for uprobe, I'm adding the bool value to the bpf_trace_run_ctx, so the helper can detect that it's executed in uprobe context and call specific code. The is_uprobe bool is set as true in bpf_prog_run_array_sleepable, which is currently used only for executing bpf programs in uprobe. Renaming bpf_prog_run_array_sleepable to bpf_prog_run_array_uprobe to address that it's only used for uprobes and that it sets the run_ctx.is_uprobe as suggested by Yafang Shao. Suggested-by: Andrii Nakryiko Tested-by: Alan Maguire [1] https://lore.kernel.org/bpf/CAEf4BzZ=xLVkG5eurEuvLU79wAMtwho7ReR+XJAgwhFF4M-7Cg@mail.gmail.com/ Signed-off-by: Jiri Olsa Tested-by: Viktor Malik Acked-by: Yonghong Song --- include/linux/bpf.h | 9 +++++++-- include/uapi/linux/bpf.h | 7 ++++++- kernel/trace/bpf_trace.c | 11 ++++++++++- kernel/trace/trace_probe.h | 5 +++++ kernel/trace/trace_uprobe.c | 7 +------ tools/include/uapi/linux/bpf.h | 7 ++++++- 6 files changed, 35 insertions(+), 11 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index abe75063630b..db3fe5a61b05 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1819,6 +1819,7 @@ struct bpf_cg_run_ctx { struct bpf_trace_run_ctx { struct bpf_run_ctx run_ctx; u64 bpf_cookie; + bool is_uprobe; }; struct bpf_tramp_run_ctx { @@ -1867,6 +1868,8 @@ bpf_prog_run_array(const struct bpf_prog_array *array, if (unlikely(!array)) return ret; + run_ctx.is_uprobe = false; + migrate_disable(); old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); item = &array->items[0]; @@ -1891,8 +1894,8 @@ bpf_prog_run_array(const struct bpf_prog_array *array, * rcu-protected dynamically sized maps. */ static __always_inline u32 -bpf_prog_run_array_sleepable(const struct bpf_prog_array __rcu *array_rcu, - const void *ctx, bpf_prog_run_fn run_prog) +bpf_prog_run_array_uprobe(const struct bpf_prog_array __rcu *array_rcu, + const void *ctx, bpf_prog_run_fn run_prog) { const struct bpf_prog_array_item *item; const struct bpf_prog *prog; @@ -1906,6 +1909,8 @@ bpf_prog_run_array_sleepable(const struct bpf_prog_array __rcu *array_rcu, rcu_read_lock_trace(); migrate_disable(); + run_ctx.is_uprobe = true; + array = rcu_dereference_check(array_rcu, rcu_read_lock_trace_held()); if (unlikely(!array)) goto out; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 70da85200695..d21deb46f49f 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -5086,9 +5086,14 @@ union bpf_attr { * u64 bpf_get_func_ip(void *ctx) * Description * Get address of the traced function (for tracing and kprobe programs). + * + * When called for kprobe program attached as uprobe it returns + * probe address for both entry and return uprobe. + * * Return - * Address of the traced function. + * Address of the traced function for kprobe. * 0 for kprobes placed within the function (not at the entry). + * Address of the probe for uprobe and return uprobe. * * u64 bpf_get_attach_cookie(void *ctx) * Description diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index d6296d51a826..792445e1f3f0 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1055,7 +1055,16 @@ static unsigned long get_entry_ip(unsigned long fentry_ip) BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { - struct kprobe *kp = kprobe_running(); + struct bpf_trace_run_ctx *run_ctx __maybe_unused; + struct kprobe *kp; + +#ifdef CONFIG_UPROBES + run_ctx = container_of(current->bpf_ctx, struct bpf_trace_run_ctx, run_ctx); + if (run_ctx->is_uprobe) + return ((struct uprobe_dispatch_data *)current->utask->vaddr)->bp_addr; +#endif + + kp = kprobe_running(); if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY)) return 0; diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index 01ea148723de..7dde806be91e 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -519,3 +519,8 @@ void __trace_probe_log_err(int offset, int err); #define trace_probe_log_err(offs, err) \ __trace_probe_log_err(offs, TP_ERR_##err) + +struct uprobe_dispatch_data { + struct trace_uprobe *tu; + unsigned long bp_addr; +}; diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c index 555c223c3232..576b3bcb8ebd 100644 --- a/kernel/trace/trace_uprobe.c +++ b/kernel/trace/trace_uprobe.c @@ -88,11 +88,6 @@ static struct trace_uprobe *to_trace_uprobe(struct dyn_event *ev) static int register_uprobe_event(struct trace_uprobe *tu); static int unregister_uprobe_event(struct trace_uprobe *tu); -struct uprobe_dispatch_data { - struct trace_uprobe *tu; - unsigned long bp_addr; -}; - static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs); static int uretprobe_dispatcher(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs); @@ -1352,7 +1347,7 @@ static void __uprobe_perf_func(struct trace_uprobe *tu, if (bpf_prog_array_valid(call)) { u32 ret; - ret = bpf_prog_run_array_sleepable(call->prog_array, regs, bpf_prog_run); + ret = bpf_prog_run_array_uprobe(call->prog_array, regs, bpf_prog_run); if (!ret) return; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 70da85200695..d21deb46f49f 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -5086,9 +5086,14 @@ union bpf_attr { * u64 bpf_get_func_ip(void *ctx) * Description * Get address of the traced function (for tracing and kprobe programs). + * + * When called for kprobe program attached as uprobe it returns + * probe address for both entry and return uprobe. + * * Return - * Address of the traced function. + * Address of the traced function for kprobe. * 0 for kprobes placed within the function (not at the entry). + * Address of the probe for uprobe and return uprobe. * * u64 bpf_get_attach_cookie(void *ctx) * Description From patchwork Mon Aug 7 08:59:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13343193 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AB60C2DF for ; Mon, 7 Aug 2023 09:00:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81596C433C9; Mon, 7 Aug 2023 09:00:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691398822; bh=+DDY0D4Y6bFjiG8v3EZ/JqpUklUKAy8LVEiwQiD7ld4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fqZ20IofpK+W55w5suAx24vke+smiyNhCpep/pOCK+kG/ScyEVz2NNu7deAlPr5NN 36R9tLEMculZmIZjK1OWXzRXNMjeF2qg+EfsQi2GZfSAPvi2LFHBnZf+Gqb/EPaG0e DttBfI+RRtFYQh7ni0rGA5xPSZs9XuJi+3C0p2qV2bedgzwFFIotL61ykglNDfhcrd jOZUUTy2eE+p5By6h0N2Gu6KNdEE8SgegFszfqru69Xi9pxQv4tVbSQbQF3FLcAyoy JltysXOUfd1pNBiQXdytoPSrLnkzU9XL6FcvQAxk7W/ko2/L0PN2h3AG0SXDNIi1ex cY+AnjGY6dgwQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Alan Maguire , bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" , Steven Rostedt Subject: [PATCHv3 bpf-next 2/3] selftests/bpf: Add bpf_get_func_ip tests for uprobe on function entry Date: Mon, 7 Aug 2023 10:59:55 +0200 Message-ID: <20230807085956.2344866-3-jolsa@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230807085956.2344866-1-jolsa@kernel.org> References: <20230807085956.2344866-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding get_func_ip tests for uprobe on function entry that validates that bpf_get_func_ip returns proper values from both uprobe and return uprobe. Tested-by: Alan Maguire Signed-off-by: Jiri Olsa Acked-by: Yonghong Song --- .../bpf/prog_tests/get_func_ip_test.c | 11 ++++++++ .../selftests/bpf/progs/get_func_ip_test.c | 25 +++++++++++++++++-- 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c index fede8ef58b5b..114cdbc04caf 100644 --- a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c @@ -1,6 +1,11 @@ // SPDX-License-Identifier: GPL-2.0 #include #include "get_func_ip_test.skel.h" +#include "get_func_ip_uprobe_test.skel.h" + +static noinline void uprobe_trigger(void) +{ +} static void test_function_entry(void) { @@ -20,6 +25,8 @@ static void test_function_entry(void) if (!ASSERT_OK(err, "get_func_ip_test__attach")) goto cleanup; + skel->bss->uprobe_trigger = (unsigned long) uprobe_trigger; + prog_fd = bpf_program__fd(skel->progs.test1); err = bpf_prog_test_run_opts(prog_fd, &topts); ASSERT_OK(err, "test_run"); @@ -30,11 +37,15 @@ static void test_function_entry(void) ASSERT_OK(err, "test_run"); + uprobe_trigger(); + ASSERT_EQ(skel->bss->test1_result, 1, "test1_result"); ASSERT_EQ(skel->bss->test2_result, 1, "test2_result"); ASSERT_EQ(skel->bss->test3_result, 1, "test3_result"); ASSERT_EQ(skel->bss->test4_result, 1, "test4_result"); ASSERT_EQ(skel->bss->test5_result, 1, "test5_result"); + ASSERT_EQ(skel->bss->test7_result, 1, "test7_result"); + ASSERT_EQ(skel->bss->test8_result, 1, "test8_result"); cleanup: get_func_ip_test__destroy(skel); diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c index 8559e698b40d..8956eb78a226 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -1,8 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 -#include +#include "vmlinux.h" #include #include -#include char _license[] SEC("license") = "GPL"; @@ -83,3 +82,25 @@ int test6(struct pt_regs *ctx) test6_result = (const void *) addr == 0; return 0; } + +unsigned long uprobe_trigger; + +__u64 test7_result = 0; +SEC("uprobe//proc/self/exe:uprobe_trigger") +int BPF_UPROBE(test7) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test7_result = (const void *) addr == (const void *) uprobe_trigger; + return 0; +} + +__u64 test8_result = 0; +SEC("uretprobe//proc/self/exe:uprobe_trigger") +int BPF_URETPROBE(test8, int ret) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test8_result = (const void *) addr == (const void *) uprobe_trigger; + return 0; +} From patchwork Mon Aug 7 08:59:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13343194 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4282BC2DF for ; Mon, 7 Aug 2023 09:00:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02640C433C7; Mon, 7 Aug 2023 09:00:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691398833; bh=rd4PiPHvQFmoacLWFoTv/0Eyp1Igc9qd6VNck/4UW1o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SfSOGw3Zvn3kFtle4ykBUr7dLRVvzpq6tL9HuqR9eIC77adrJ+gtPBZRAXeM5KynP CYHsHwtigdW8WB1lLHH+kOXfVORlO/G8yVuFtQokEEkMqQaoVWmuvy1X1CoojaYtvB etlN4S4hODrQTFbCkNLCwHp2/LyQWXDcCm3CyAkM99aEjPOAAF9LxTZ64go9KwIBrU bDV3ycrM7sPhGqKP+XRTM/KzWNgq7qWeeM4qU8WKVrU01CEdFNZhu3zZXwEUN02aDh 5cR69vRRCT7TnNZuE6kXPrD/eIQqXmb+q6qRp8wJL2EsLCVlGRs3m9AruJ4SExj1m+ naW7eQLRl1FUg== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Alan Maguire , bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" , Steven Rostedt Subject: [PATCHv3 bpf-next 3/3] selftests/bpf: Add bpf_get_func_ip test for uprobe inside function Date: Mon, 7 Aug 2023 10:59:56 +0200 Message-ID: <20230807085956.2344866-4-jolsa@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230807085956.2344866-1-jolsa@kernel.org> References: <20230807085956.2344866-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Adding get_func_ip test for uprobe inside function that validates the get_func_ip helper returns correct probe address value. Tested-by: Alan Maguire Signed-off-by: Jiri Olsa Acked-by: Yonghong Song --- .../bpf/prog_tests/get_func_ip_test.c | 46 +++++++++++++++++-- .../bpf/progs/get_func_ip_uprobe_test.c | 18 ++++++++ 2 files changed, 60 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/get_func_ip_uprobe_test.c diff --git a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c index 114cdbc04caf..c40242dfa8fb 100644 --- a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c @@ -51,11 +51,17 @@ static void test_function_entry(void) get_func_ip_test__destroy(skel); } -/* test6 is x86_64 specific because of the instruction - * offset, disabling it for all other archs - */ #ifdef __x86_64__ -static void test_function_body(void) +extern void uprobe_trigger_body(void); +asm( +".globl uprobe_trigger_body\n" +".type uprobe_trigger_body, @function\n" +"uprobe_trigger_body:\n" +" nop\n" +" ret\n" +); + +static void test_function_body_kprobe(void) { struct get_func_ip_test *skel = NULL; LIBBPF_OPTS(bpf_test_run_opts, topts); @@ -67,6 +73,9 @@ static void test_function_body(void) if (!ASSERT_OK_PTR(skel, "get_func_ip_test__open")) return; + /* test6 is x86_64 specific and is disabled by default, + * enable it for body test. + */ bpf_program__set_autoload(skel->progs.test6, true); err = get_func_ip_test__load(skel); @@ -90,6 +99,35 @@ static void test_function_body(void) bpf_link__destroy(link6); get_func_ip_test__destroy(skel); } + +static void test_function_body_uprobe(void) +{ + struct get_func_ip_uprobe_test *skel = NULL; + int err; + + skel = get_func_ip_uprobe_test__open_and_load(); + if (!ASSERT_OK_PTR(skel, "get_func_ip_uprobe_test__open_and_load")) + return; + + err = get_func_ip_uprobe_test__attach(skel); + if (!ASSERT_OK(err, "get_func_ip_test__attach")) + goto cleanup; + + skel->bss->uprobe_trigger_body = (unsigned long) uprobe_trigger_body; + + uprobe_trigger_body(); + + ASSERT_EQ(skel->bss->test1_result, 1, "test1_result"); + +cleanup: + get_func_ip_uprobe_test__destroy(skel); +} + +static void test_function_body(void) +{ + test_function_body_kprobe(); + test_function_body_uprobe(); +} #else #define test_function_body() #endif diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_uprobe_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_uprobe_test.c new file mode 100644 index 000000000000..052f8a4345a8 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/get_func_ip_uprobe_test.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +unsigned long uprobe_trigger_body; + +__u64 test1_result = 0; +SEC("uprobe//proc/self/exe:uprobe_trigger_body+1") +int BPF_UPROBE(test1) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test1_result = (const void *) addr == (const void *) uprobe_trigger_body + 1; + return 0; +}