From patchwork Mon Apr 24 16:04:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222345 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 795B4C7618E for ; Mon, 24 Apr 2023 16:05:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231425AbjDXQFG (ORCPT ); Mon, 24 Apr 2023 12:05:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230434AbjDXQFF (ORCPT ); Mon, 24 Apr 2023 12:05:05 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88F7C5FE4 for ; Mon, 24 Apr 2023 09:05:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 08423619FC for ; Mon, 24 Apr 2023 16:05:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AD8CDC433D2; Mon, 24 Apr 2023 16:04:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352302; bh=Wpl2V0t7OSZNPoVz9hs1eafvBjE4FFr5lf6tX8CPPrQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NpOSgmQkOOI8wgQQeejPrym/cwaNDv/RwwydusJIDrnQ/jQQoyUpw8Tld9zxiOXZ+ T8fKul7EqRwnqLPXrFrh1mDFvWW3eAT0fAu9wmfmkdkp9EHTkhveg3UVFoyUt8IfJR w6s/G8Uc02Izpycmw3hzwa+l1HKW6s3XSS1dTFQrH67VvhjuIi92tv23rlqeNu14CO 00iy5MrztV6UR9TrREUGEqta3LE1YUwKyh2N02JLM7663ASoX7XIovxekBnDOHBISO g4v5oun91u7w9AvTP7roKKcPcdu5c9TiKeIH6CSCyVqEqdPtVNxF1/Y1iX1EOG/WXU vu8PGg7oR+j+w== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 01/20] bpf: Add multi uprobe link Date: Mon, 24 Apr 2023 18:04:28 +0200 Message-Id: <20230424160447.2005755-2-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding new multi uprobe link that allows to attach bpf program to multiple uprobes. Uprobes to attach are specified via new link_create uprobe_multi union: struct { __u32 flags; __u32 cnt; __aligned_u64 paths; __aligned_u64 offsets; __aligned_u64 ref_ctr_offsets; } uprobe_multi; Uprobes are defined in paths/offsets/ref_ctr_offsets arrays with the same 'cnt' length. Each uprobe is defined with a single index in all three arrays: paths[idx], offsets[idx] and/or ref_ctr_offsets[idx] The 'flags' supports single bit for now that marks the uprobe as return probe. Signed-off-by: Jiri Olsa --- include/linux/trace_events.h | 6 + include/uapi/linux/bpf.h | 14 +++ kernel/bpf/syscall.c | 16 ++- kernel/trace/bpf_trace.c | 231 +++++++++++++++++++++++++++++++++++ 4 files changed, 265 insertions(+), 2 deletions(-) diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index 0e373222a6df..b0db245fc0f5 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, u32 *fd_type, const char **buf, u64 *probe_offset, u64 *probe_addr); int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog); +int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog); #else static inline unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx) { @@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { return -EOPNOTSUPP; } +static inline int +bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + return -EOPNOTSUPP; +} #endif enum { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 1bb11a6ee667..debc041c6ca5 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1035,6 +1035,7 @@ enum bpf_attach_type { BPF_TRACE_KPROBE_MULTI, BPF_LSM_CGROUP, BPF_STRUCT_OPS, + BPF_TRACE_UPROBE_MULTI, __MAX_BPF_ATTACH_TYPE }; @@ -1052,6 +1053,7 @@ enum bpf_link_type { BPF_LINK_TYPE_KPROBE_MULTI = 8, BPF_LINK_TYPE_STRUCT_OPS = 9, BPF_LINK_TYPE_NETFILTER = 10, + BPF_LINK_TYPE_UPROBE_MULTI = 11, MAX_BPF_LINK_TYPE, }; @@ -1169,6 +1171,11 @@ enum bpf_link_type { */ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.uprobe_multi.flags used in LINK_CREATE command for + * BPF_TRACE_UPROBE_MULTI attach type to create return probe. + */ +#define BPF_F_UPROBE_MULTI_RETURN (1U << 0) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1568,6 +1575,13 @@ union bpf_attr { __s32 priority; __u32 flags; } netfilter; + struct { + __u32 flags; + __u32 cnt; + __aligned_u64 paths; + __aligned_u64 offsets; + __aligned_u64 ref_ctr_offsets; + } uprobe_multi; }; } link_create; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 14f39c1e573e..0b789a33317b 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -4601,7 +4601,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) break; case BPF_PROG_TYPE_KPROBE: if (attr->link_create.attach_type != BPF_PERF_EVENT && - attr->link_create.attach_type != BPF_TRACE_KPROBE_MULTI) { + attr->link_create.attach_type != BPF_TRACE_KPROBE_MULTI && + attr->link_create.attach_type != BPF_TRACE_UPROBE_MULTI) { ret = -EINVAL; goto out; } @@ -4666,10 +4667,21 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) ret = bpf_perf_link_attach(attr, prog); break; case BPF_PROG_TYPE_KPROBE: + /* Ensure that program with eBPF_TRACE_UPROBE_MULTI attach type can + * attach only to uprobe_multi link. It has its own runtime context + * which is specific for get_func_ip/get_attach_cookie helpers. + */ + if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI && + attr->link_create.attach_type != BPF_TRACE_UPROBE_MULTI) { + ret = -EINVAL; + goto out; + } if (attr->link_create.attach_type == BPF_PERF_EVENT) ret = bpf_perf_link_attach(attr, prog); - else + else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI) ret = bpf_kprobe_multi_link_attach(attr, prog); + else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI) + ret = bpf_uprobe_multi_link_attach(attr, prog); break; default: ret = -EINVAL; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index bcf91bc7bf71..b84a7d01abf4 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -23,6 +23,7 @@ #include #include #include +#include #include @@ -2901,3 +2902,233 @@ static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) return 0; } #endif + +#ifdef CONFIG_UPROBES +struct bpf_uprobe_multi_link; + +struct bpf_uprobe { + struct bpf_uprobe_multi_link *link; + struct inode *inode; + loff_t offset; + loff_t ref_ctr_offset; + struct uprobe_consumer consumer; +}; + +struct bpf_uprobe_multi_link { + struct bpf_link link; + u32 cnt; + struct bpf_uprobe *uprobes; +}; + +struct bpf_uprobe_multi_run_ctx { + struct bpf_run_ctx run_ctx; + unsigned long entry_ip; +}; + +static void bpf_uprobe_unregister(struct bpf_uprobe *uprobes, u32 cnt) +{ + u32 i; + + for (i = 0; i < cnt; i++) { + uprobe_unregister(uprobes[i].inode, uprobes[i].offset, + &uprobes[i].consumer); + } +} + +static void bpf_uprobe_multi_link_release(struct bpf_link *link) +{ + struct bpf_uprobe_multi_link *umulti_link; + + umulti_link = container_of(link, struct bpf_uprobe_multi_link, link); + bpf_uprobe_unregister(umulti_link->uprobes, umulti_link->cnt); +} + +static void bpf_uprobe_multi_link_dealloc(struct bpf_link *link) +{ + struct bpf_uprobe_multi_link *umulti_link; + + umulti_link = container_of(link, struct bpf_uprobe_multi_link, link); + kvfree(umulti_link->uprobes); + kfree(umulti_link); +} + +static const struct bpf_link_ops bpf_uprobe_multi_link_lops = { + .release = bpf_uprobe_multi_link_release, + .dealloc = bpf_uprobe_multi_link_dealloc, +}; + +static int uprobe_prog_run(struct bpf_uprobe *uprobe, + unsigned long entry_ip, + struct pt_regs *regs) +{ + struct bpf_uprobe_multi_link *link = uprobe->link; + struct bpf_uprobe_multi_run_ctx run_ctx = { + .entry_ip = entry_ip, + }; + struct bpf_run_ctx *old_run_ctx; + int err; + + preempt_disable(); + + if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) { + err = 0; + goto out; + } + + rcu_read_lock(); + old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); + err = bpf_prog_run(link->link.prog, regs); + bpf_reset_run_ctx(old_run_ctx); + rcu_read_unlock(); + + out: + __this_cpu_dec(bpf_prog_active); + preempt_enable(); + return err; +} + +static int +uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs) +{ + struct bpf_uprobe *uprobe; + + uprobe = container_of(con, struct bpf_uprobe, consumer); + return uprobe_prog_run(uprobe, instruction_pointer(regs), regs); +} + +static int +uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs) +{ + struct bpf_uprobe *uprobe; + + uprobe = container_of(con, struct bpf_uprobe, consumer); + return uprobe_prog_run(uprobe, func, regs); +} + +int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + void __user **upaths, __user *upath, __user *old_upath = NULL; + unsigned long __user *uref_ctr_offsets, ref_ctr_offset = 0; + struct bpf_uprobe_multi_link *link = NULL; + unsigned long __user *uoffsets, offset; + struct bpf_link_primer link_primer; + struct bpf_uprobe *uprobes = NULL; + struct inode *inode, *old_inode; + u32 flags, cnt, i; + int err; + + /* no support for 32bit archs yet */ + if (sizeof(u64) != sizeof(void *)) + return -EOPNOTSUPP; + + if (prog->expected_attach_type != BPF_TRACE_UPROBE_MULTI) + return -EINVAL; + + flags = attr->link_create.uprobe_multi.flags; + if (flags & ~BPF_F_UPROBE_MULTI_RETURN) + return -EINVAL; + + upaths = u64_to_user_ptr(attr->link_create.uprobe_multi.paths); + uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets); + if (!!upaths != !!uoffsets) + return -EINVAL; + + uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets); + + cnt = attr->link_create.uprobe_multi.cnt; + if (!cnt) + return -EINVAL; + + uprobes = kvcalloc(cnt, sizeof(*uprobes), GFP_KERNEL); + if (!uprobes) + return -ENOMEM; + + link = kzalloc(sizeof(*link), GFP_KERNEL); + if (!link) { + err = -ENOMEM; + goto error; + } + + for (i = 0; i < cnt; i++) { + if (uref_ctr_offsets && __get_user(ref_ctr_offset, uref_ctr_offsets + i)) { + err = -EFAULT; + goto error; + } + if (__get_user(offset, uoffsets + i)) { + err = -EFAULT; + goto error; + } + if (__get_user(upath, upaths + i)) { + err = -EFAULT; + goto error; + } + if (i && old_upath == upath) { + inode = old_inode; + } else { + struct path path; + char *name; + + name = strndup_user(upath, PATH_MAX); + if (IS_ERR(name)) { + err = PTR_ERR(name); + goto error; + } + err = kern_path(name, LOOKUP_FOLLOW, &path); + kfree(name); + if (err) + goto error; + if (!d_is_reg(path.dentry)) { + err = -EINVAL; + path_put(&path); + goto error; + } + inode = d_real_inode(path.dentry); + path_put(&path); + } + old_upath = upath; + + uprobes[i].inode = inode; + uprobes[i].offset = offset; + uprobes[i].ref_ctr_offset = ref_ctr_offset; + uprobes[i].link = link; + + if (flags & BPF_F_UPROBE_MULTI_RETURN) + uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; + else + uprobes[i].consumer.handler = uprobe_multi_link_handler; + } + + link->cnt = cnt; + link->uprobes = uprobes; + + bpf_link_init(&link->link, BPF_LINK_TYPE_UPROBE_MULTI, + &bpf_uprobe_multi_link_lops, prog); + + err = bpf_link_prime(&link->link, &link_primer); + if (err) + goto error; + + for (i = 0; i < cnt; i++) { + err = uprobe_register_refctr(uprobes[i].inode, uprobes[i].offset, + uprobes[i].ref_ctr_offset, + &uprobes[i].consumer); + if (err) { + bpf_uprobe_unregister(uprobes, i); + bpf_link_cleanup(&link_primer); + return err; + } + } + + return bpf_link_settle(&link_primer); + +error: + kfree(uprobes); + kfree(link); + return err; +} +#else /* !CONFIG_UPROBES */ +int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + return -EOPNOTSUPP; +} +#endif /* CONFIG_UPROBES */ From patchwork Mon Apr 24 16:04:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222346 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57D0DC7618E for ; Mon, 24 Apr 2023 16:05:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231600AbjDXQFV (ORCPT ); Mon, 24 Apr 2023 12:05:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231881AbjDXQFQ (ORCPT ); Mon, 24 Apr 2023 12:05:16 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5350B6A6B for ; Mon, 24 Apr 2023 09:05:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E0F4061B3B for ; Mon, 24 Apr 2023 16:05:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8ED12C433EF; Mon, 24 Apr 2023 16:05:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352312; bh=EhY7xWSqakRJM6mhVenwnioj+7nt3qIzNwRcwlYuGik=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FfCmYwSsNZblMKPbXq6E6QkSv4D4H1NZLdZVxQHIKiBJ9Z8Efe1yAcbUl8MYO+jll 6git68lIRhbJAEiqnlk6v6dEbkS3xCO1C2gqdTtiDXnvIlj9P4yMHtwpReXgyDV80b puIaH/uK8Ap0duhOJyCzkG3+eBJsqzLVUlByrzXA/n1pQtLXISShG2XUtVYE+lHpJV a5vdgAqNQetB0UY78vOkdaEoiYiE26y8GoGoj18Zd79gNTzm2sqJ9f8VDhYxDWMZfV 3lW73echsdBX6Yjskpgz/8KCoq3BkZgUukdIvwrKLkHb8VCDFOAsOi2PzG53kn8J14 TIMjjDktHxZYw== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 02/20] bpf: Add cookies support for uprobe_multi link Date: Mon, 24 Apr 2023 18:04:29 +0200 Message-Id: <20230424160447.2005755-3-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding support to specify cookies array for uprobe_multi link. The cookies array share indexes and length with other uprobe_multi arrays (paths/offsets/ref_ctr_offsets). The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 46 +++++++++++++++++++++++++++++++++++++--- 3 files changed, 45 insertions(+), 4 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index debc041c6ca5..77ce2159478d 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1581,6 +1581,7 @@ union bpf_attr { __aligned_u64 paths; __aligned_u64 offsets; __aligned_u64 ref_ctr_offsets; + __aligned_u64 cookies; } uprobe_multi; }; } link_create; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 0b789a33317b..5b2dc7ae8616 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -4566,7 +4566,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr, return err; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies static int link_create(union bpf_attr *attr, bpfptr_t uattr) { enum bpf_prog_type ptype; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index b84a7d01abf4..f795cfc00e5f 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -87,6 +87,8 @@ static int bpf_btf_printf_prepare(struct btf_ptr *ptr, u32 btf_ptr_size, static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx); static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx); +static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx); + /** * trace_call_bpf - invoke BPF program * @call: tracepoint event @@ -1089,6 +1091,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs) +{ + return bpf_uprobe_multi_cookie(current->bpf_ctx); +} + +static const struct bpf_func_proto bpf_get_attach_cookie_proto_umulti = { + .func = bpf_get_attach_cookie_uprobe_multi, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx) { struct bpf_trace_run_ctx *run_ctx; @@ -1535,9 +1549,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) &bpf_get_func_ip_proto_kprobe_multi : &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: - return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ? - &bpf_get_attach_cookie_proto_kmulti : - &bpf_get_attach_cookie_proto_trace; + if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI) + return &bpf_get_attach_cookie_proto_kmulti; + if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + return &bpf_get_attach_cookie_proto_umulti; + return &bpf_get_attach_cookie_proto_trace; default: return bpf_tracing_func_proto(func_id, prog); } @@ -2911,6 +2927,7 @@ struct bpf_uprobe { struct inode *inode; loff_t offset; loff_t ref_ctr_offset; + u64 cookie; struct uprobe_consumer consumer; }; @@ -2923,6 +2940,7 @@ struct bpf_uprobe_multi_link { struct bpf_uprobe_multi_run_ctx { struct bpf_run_ctx run_ctx; unsigned long entry_ip; + struct bpf_uprobe *uprobe; }; static void bpf_uprobe_unregister(struct bpf_uprobe *uprobes, u32 cnt) @@ -2964,6 +2982,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, struct bpf_uprobe_multi_link *link = uprobe->link; struct bpf_uprobe_multi_run_ctx run_ctx = { .entry_ip = entry_ip, + .uprobe = uprobe, }; struct bpf_run_ctx *old_run_ctx; int err; @@ -3005,6 +3024,16 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s return uprobe_prog_run(uprobe, func, regs); } +static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) +{ + struct bpf_uprobe_multi_run_ctx *run_ctx; + + if (WARN_ON_ONCE(!ctx)) + return 0; + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + return run_ctx->uprobe->cookie; +} + int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { void __user **upaths, __user *upath, __user *old_upath = NULL; @@ -3014,6 +3043,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr struct bpf_link_primer link_primer; struct bpf_uprobe *uprobes = NULL; struct inode *inode, *old_inode; + u64 __user *ucookies, cookie = 0; u32 flags, cnt, i; int err; @@ -3034,6 +3064,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr return -EINVAL; uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets); + ucookies = u64_to_user_ptr(attr->link_create.uprobe_multi.cookies); cnt = attr->link_create.uprobe_multi.cnt; if (!cnt) @@ -3050,6 +3081,10 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr } for (i = 0; i < cnt; i++) { + if (ucookies && __get_user(cookie, ucookies + i)) { + err = -EFAULT; + goto error; + } if (uref_ctr_offsets && __get_user(ref_ctr_offset, uref_ctr_offsets + i)) { err = -EFAULT; goto error; @@ -3090,6 +3125,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr uprobes[i].inode = inode; uprobes[i].offset = offset; uprobes[i].ref_ctr_offset = ref_ctr_offset; + uprobes[i].cookie = cookie; uprobes[i].link = link; if (flags & BPF_F_UPROBE_MULTI_RETURN) @@ -3131,4 +3167,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr { return -EOPNOTSUPP; } +static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) +{ + return 0; +} #endif /* CONFIG_UPROBES */ From patchwork Mon Apr 24 16:04:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222347 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5AA1C7618E for ; Mon, 24 Apr 2023 16:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231713AbjDXQF0 (ORCPT ); Mon, 24 Apr 2023 12:05:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230434AbjDXQFZ (ORCPT ); Mon, 24 Apr 2023 12:05:25 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E4335FE4 for ; Mon, 24 Apr 2023 09:05:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 16EB861B3B for ; Mon, 24 Apr 2023 16:05:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B0F5FC433D2; Mon, 24 Apr 2023 16:05:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352322; bh=6Xz5NBC/2mzygyBPXXz1IntEiDLJO7+qmSTPRMLiIuA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R6tgQ1ReLsnI0vk2/f+fzuNgm41FyiSs5rwBtBWRtkwROeUTabmaBWtL2xNhTwvmP xQm+B1p34sq2zkKU1dexvUue707wJvDL+MN7Nz/EaKLXktevqxKocvfWM+VSzNn3wQ Ylci/+biAu3HqLiaIylDlLyXp/ybw7PArCfT0vRrRqV+iEv1KBtMrF/ZXAhMdANT/i X5RC6aaOkwkLr6Jk4TQlg/6lSUQE0r9XtzBEEpsP7hy7ZRA4DhJJa/CYunyOHQUlVg HY1X3kmBsY+bUkUcz5sLhrtwcwYzmpOCFOAgGo/h49rBhZ+rhCkj2w8b+q2/q5kIFN kS1PT3FVGGdFA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 03/20] bpf: Add bpf_get_func_ip helper support for uprobe link Date: Mon, 24 Apr 2023 18:04:30 +0200 Message-Id: <20230424160447.2005755-4-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link. It returns the ip of the uprobe. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index f795cfc00e5f..b9a4a8ff51d7 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx); static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx); static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx); +static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx); /** * trace_call_bpf - invoke BPF program @@ -1091,6 +1092,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_func_ip_uprobe_multi, struct pt_regs *, regs) +{ + return bpf_uprobe_multi_entry_ip(current->bpf_ctx); +} + +static const struct bpf_func_proto bpf_get_func_ip_proto_uprobe_multi = { + .func = bpf_get_func_ip_uprobe_multi, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs) { return bpf_uprobe_multi_cookie(current->bpf_ctx); @@ -1545,9 +1558,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_override_return_proto; #endif case BPF_FUNC_get_func_ip: - return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ? - &bpf_get_func_ip_proto_kprobe_multi : - &bpf_get_func_ip_proto_kprobe; + if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI) + return &bpf_get_func_ip_proto_kprobe_multi; + if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI) + return &bpf_get_func_ip_proto_uprobe_multi; + return &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI) return &bpf_get_attach_cookie_proto_kmulti; @@ -3024,6 +3039,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s return uprobe_prog_run(uprobe, func, regs); } +static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) +{ + struct bpf_uprobe_multi_run_ctx *run_ctx; + + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); + return run_ctx->entry_ip; +} + static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) { struct bpf_uprobe_multi_run_ctx *run_ctx; @@ -3171,4 +3194,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) { return 0; } +static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) +{ + return 0; +} #endif /* CONFIG_UPROBES */ From patchwork Mon Apr 24 16:04:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222348 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D18BC77B61 for ; Mon, 24 Apr 2023 16:05:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230434AbjDXQFh (ORCPT ); Mon, 24 Apr 2023 12:05:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231625AbjDXQFg (ORCPT ); Mon, 24 Apr 2023 12:05:36 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BB196EA1 for ; Mon, 24 Apr 2023 09:05:33 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3A45861B3C for ; Mon, 24 Apr 2023 16:05:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5CC7C433D2; Mon, 24 Apr 2023 16:05:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352332; bh=LO+VFQYdH3MN8xOtSlZeI39Xncak2pqlFCQR+qwnkJI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bNKcp4edyJGBGLN7tYxBvemjEccZ2QWo0rag/MSR+l1niBUMRMeH6nlolufAPzrj0 UnyHgzcDFzlPKCk071NhzxNJ7vUT4nB4BwGM4LcnN8kAaQnrJtU+wj1sU0jqtco+ze 7dKWo0dZvMGYvEVFewDoMo9ff35o8Lcg7OP88W/6M4A+t+aYkC168XoPsnUcU4pWu8 F8wVBQxrXFxqk8xaUumlcaguaI9uCQ2IeNOslQ1JDn4rQEpI869365g5haAlM2KfbT afs5T79aBTEPMqR3wjb4TTLNRbsjBoXTS2oNdlzSkDgzzeHAo+aFgoHL8omqAy//1n f8y8Au88NjHyw== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 04/20] libbpf: Update uapi bpf.h tools header Date: Mon, 24 Apr 2023 18:04:31 +0200 Message-Id: <20230424160447.2005755-5-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Updating uapi bpf.h tools header with new uprobe_multi link interface. Signed-off-by: Jiri Olsa --- tools/include/uapi/linux/bpf.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 1bb11a6ee667..77ce2159478d 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1035,6 +1035,7 @@ enum bpf_attach_type { BPF_TRACE_KPROBE_MULTI, BPF_LSM_CGROUP, BPF_STRUCT_OPS, + BPF_TRACE_UPROBE_MULTI, __MAX_BPF_ATTACH_TYPE }; @@ -1052,6 +1053,7 @@ enum bpf_link_type { BPF_LINK_TYPE_KPROBE_MULTI = 8, BPF_LINK_TYPE_STRUCT_OPS = 9, BPF_LINK_TYPE_NETFILTER = 10, + BPF_LINK_TYPE_UPROBE_MULTI = 11, MAX_BPF_LINK_TYPE, }; @@ -1169,6 +1171,11 @@ enum bpf_link_type { */ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.uprobe_multi.flags used in LINK_CREATE command for + * BPF_TRACE_UPROBE_MULTI attach type to create return probe. + */ +#define BPF_F_UPROBE_MULTI_RETURN (1U << 0) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1568,6 +1575,14 @@ union bpf_attr { __s32 priority; __u32 flags; } netfilter; + struct { + __u32 flags; + __u32 cnt; + __aligned_u64 paths; + __aligned_u64 offsets; + __aligned_u64 ref_ctr_offsets; + __aligned_u64 cookies; + } uprobe_multi; }; } link_create; From patchwork Mon Apr 24 16:04:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222349 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71B7EC77B61 for ; Mon, 24 Apr 2023 16:05:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231828AbjDXQFp (ORCPT ); Mon, 24 Apr 2023 12:05:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231625AbjDXQFo (ORCPT ); Mon, 24 Apr 2023 12:05:44 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD1DB6A63 for ; Mon, 24 Apr 2023 09:05:43 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5734F61B47 for ; Mon, 24 Apr 2023 16:05:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 03D8AC433EF; Mon, 24 Apr 2023 16:05:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352342; bh=wq2SNcUk2IUQg/YAu2RwBStrExCr+1rRnd25SvVfb44=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G8quFtRPupD1KFIjgAtUrBUhXOzx0gQX1CuZEpIUhrLuxx/1FvXHZ1YJmerLZDsUU GyxeK0xhKEJVISk0OX8hx5tGYAQ/R3AZyaOQ2ONud7CNNCKoof4N3mAs6YJvRnju0D HMnX8LNbj20VDW72QXki6qDpSV0a6f8RQN69S/KWUbAd6bUkW4TTBYM3hsg7F0VUXf RrpNA2m5wLgnJZNVQ/KljqL7j6/ASWBF3+mr1vZumeGkuT+F0pihj0PCZwGBhN6imx 3FOLE4YN0mzDRr40GQbSShHjgvlb9ivmI+HuvnxncbJe+ShuJUVBs2UqEl7Do6FpvA cKIrzHTb5+BvQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 05/20] libbpf: Add uprobe_multi attach type and link names Date: Mon, 24 Apr 2023 18:04:32 +0200 Message-Id: <20230424160447.2005755-6-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding new uprobe_multi attach type and link names, so the functions can resolve the new values. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- tools/lib/bpf/libbpf.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 1cbacf9e71f3..b5bde1f19831 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -117,6 +117,7 @@ static const char * const attach_type_name[] = { [BPF_PERF_EVENT] = "perf_event", [BPF_TRACE_KPROBE_MULTI] = "trace_kprobe_multi", [BPF_STRUCT_OPS] = "struct_ops", + [BPF_TRACE_UPROBE_MULTI] = "trace_uprobe_multi", }; static const char * const link_type_name[] = { @@ -131,6 +132,7 @@ static const char * const link_type_name[] = { [BPF_LINK_TYPE_KPROBE_MULTI] = "kprobe_multi", [BPF_LINK_TYPE_STRUCT_OPS] = "struct_ops", [BPF_LINK_TYPE_NETFILTER] = "netfilter", + [BPF_LINK_TYPE_UPROBE_MULTI] = "uprobe_multi", }; static const char * const map_type_name[] = { From patchwork Mon Apr 24 16:04:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222350 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D44D1C77B61 for ; Mon, 24 Apr 2023 16:05:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231625AbjDXQF4 (ORCPT ); Mon, 24 Apr 2023 12:05:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231881AbjDXQFz (ORCPT ); Mon, 24 Apr 2023 12:05:55 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D74AF6A63 for ; Mon, 24 Apr 2023 09:05:53 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7227861B66 for ; Mon, 24 Apr 2023 16:05:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C743C433D2; Mon, 24 Apr 2023 16:05:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352352; bh=CoLqMMo4N6vU/BeEX3vwOSYaCzuCwcndOB/EdHcIEXg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bbrqGcl/q243LMmDgUfyWIZwJvKqk9zRvobdBh6GHHQ9pUCFBMHhnr83mmhygk9Ek KmthMRR82Ibl50M030p7ZVJQ7DoHwUpNexrEC22N3R4hT/s+U+JMGFwVvzsRfVVI3G 7nOVUyCwPxp0JxVLL5VMOKbNKYhsGCXYNyvVRd5ovD46SyG2GDdG1wIPC+NatN1XDl FtkeKxf6wCl6px3dmZedkw4LrRCO+0aJw8xBz710OJiKnk2uPs97XFeAGnoTctGEbb H1yMGg06e/I6pjEBudA1uTWfa2V1cXfrymyNvwVpqA5Gx1IFnDd2WQhfgYvk0d7zxE 6oSNNwPpx3hDA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 06/20] libbpf: Factor elf_for_each_symbol function Date: Mon, 24 Apr 2023 18:04:33 +0200 Message-Id: <20230424160447.2005755-7-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Currently we have elf_find_func_offset function that looks up symbol in the binary and returns its offset to be used for uprobe attachment. For attaching multiple uprobes we will need interface that allows us to get offsets for multiple symbols specified either by name or regular expression. Factoring out elf_for_each_symbol helper function that iterates all symbols in binary and calls following callbacks: fn_match - on each symbol if it returns error < 0, we bail out with that error fn_done - when we finish iterating symbol section, if it returns true, we don't iterate next section It will be used in following changes to lookup multiple symbols and their offsets. Changing elf_find_func_offset to use elf_for_each_symbol with single_match callback that's looking to match single function. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 185 +++++++++++++++++++++++++---------------- 1 file changed, 114 insertions(+), 71 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index b5bde1f19831..92c92ed2101f 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -10707,30 +10707,87 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) return NULL; } -/* Find offset of function name in the provided ELF object. "binary_path" is - * the path to the ELF binary represented by "elf", and only used for error - * reporting matters. "name" matches symbol name or name@@LIB for library - * functions. - */ -static long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) -{ - int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB }; - bool is_shared_lib, is_name_qualified; - long ret = -ENOENT; +struct elf_func_offset { + const char *name; + unsigned long offset; + int last_bind; size_t name_len; - GElf_Ehdr ehdr; + bool is_name_qualified; +}; - if (!gelf_getehdr(elf, &ehdr)) { - pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1)); - ret = -LIBBPF_ERRNO__FORMAT; - goto out; +static int single_done(void *_data) +{ + struct elf_func_offset *data = _data; + + return data->offset > 0; +} + +static int single_match(Elf *elf, const char *binary_path, const char *sname, + GElf_Sym *sym, void *_data) +{ + struct elf_func_offset *data = _data; + size_t name_len = data->name_len; + const char *name = data->name; + Elf_Scn *sym_scn; + GElf_Shdr sym_sh; + int curr_bind; + + curr_bind = GELF_ST_BIND(sym->st_info); + + /* User can specify func, func@@LIB or func@@LIB_VERSION. */ + if (strncmp(sname, name, name_len) != 0) + return 0; + /* ...but we don't want a search for "foo" to match 'foo2" also, so any + * additional characters in sname should be of the form "@@LIB". + */ + if (!data->is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@') + return 0; + + if (data->offset > 0) { + /* handle multiple matches */ + if (data->last_bind != STB_WEAK && curr_bind != STB_WEAK) { + /* Only accept one non-weak bind. */ + pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n", + sname, name, binary_path); + return -LIBBPF_ERRNO__FORMAT; + } else if (curr_bind == STB_WEAK) { + /* already have a non-weak bind, and + * this is a weak bind, so ignore. + */ + return 0; + } } - /* for shared lib case, we do not need to calculate relative offset */ - is_shared_lib = ehdr.e_type == ET_DYN; - name_len = strlen(name); - /* Does name specify "@@LIB"? */ - is_name_qualified = strstr(name, "@@") != NULL; + /* Transform symbol's virtual address (absolute for + * binaries and relative for shared libs) into file + * offset, which is what kernel is expecting for + * uprobe/uretprobe attachment. + * See Documentation/trace/uprobetracer.rst for more + * details. + * This is done by looking up symbol's containing + * section's header and using it's virtual address + * (sh_addr) and corresponding file offset (sh_offset) + * to transform sym.st_value (virtual address) into + * desired final file offset. + */ + sym_scn = elf_getscn(elf, sym->st_shndx); + if (!sym_scn) + return 0; + if (!gelf_getshdr(sym_scn, &sym_sh)) + return 0; + + data->offset = sym->st_value - sym_sh.sh_addr + sym_sh.sh_offset; + data->last_bind = curr_bind; + return 0; +} + +static int elf_for_each_symbol(Elf *elf, const char *binary_path, + int (*fn_match)(Elf *, const char *, const char *, GElf_Sym *, void *), + int (*fn_done)(void *), + void *data) +{ + int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB }; + int ret = -ENOENT; /* Search SHT_DYNSYM, SHT_SYMTAB for symbol. This search order is used because if * a binary is stripped, it may only have SHT_DYNSYM, and a fully-statically @@ -10741,7 +10798,6 @@ static long elf_find_func_offset(Elf *elf, const char *binary_path, const char * size_t nr_syms, strtabidx, idx; Elf_Data *symbols = NULL; Elf_Scn *scn = NULL; - int last_bind = -1; const char *sname; GElf_Shdr sh; @@ -10764,10 +10820,7 @@ static long elf_find_func_offset(Elf *elf, const char *binary_path, const char * nr_syms = symbols->d_size / sh.sh_entsize; for (idx = 0; idx < nr_syms; idx++) { - int curr_bind; GElf_Sym sym; - Elf_Scn *sym_scn; - GElf_Shdr sym_sh; if (!gelf_getsym(symbols, idx, &sym)) continue; @@ -10779,58 +10832,48 @@ static long elf_find_func_offset(Elf *elf, const char *binary_path, const char * if (!sname) continue; - curr_bind = GELF_ST_BIND(sym.st_info); - - /* User can specify func, func@@LIB or func@@LIB_VERSION. */ - if (strncmp(sname, name, name_len) != 0) - continue; - /* ...but we don't want a search for "foo" to match 'foo2" also, so any - * additional characters in sname should be of the form "@@LIB". - */ - if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@') - continue; + ret = fn_match(elf, binary_path, sname, &sym, data); + if (ret < 0) + goto out; + } + if (fn_done(data)) + break; + } - if (ret >= 0) { - /* handle multiple matches */ - if (last_bind != STB_WEAK && curr_bind != STB_WEAK) { - /* Only accept one non-weak bind. */ - pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n", - sname, name, binary_path); - ret = -LIBBPF_ERRNO__FORMAT; - goto out; - } else if (curr_bind == STB_WEAK) { - /* already have a non-weak bind, and - * this is a weak bind, so ignore. - */ - continue; - } - } +out: + return ret; +} - /* Transform symbol's virtual address (absolute for - * binaries and relative for shared libs) into file - * offset, which is what kernel is expecting for - * uprobe/uretprobe attachment. - * See Documentation/trace/uprobetracer.rst for more - * details. - * This is done by looking up symbol's containing - * section's header and using it's virtual address - * (sh_addr) and corresponding file offset (sh_offset) - * to transform sym.st_value (virtual address) into - * desired final file offset. - */ - sym_scn = elf_getscn(elf, sym.st_shndx); - if (!sym_scn) - continue; - if (!gelf_getshdr(sym_scn, &sym_sh)) - continue; +/* Find offset of function name in the provided ELF object. "binary_path" is + * the path to the ELF binary represented by "elf", and only used for error + * reporting matters. "name" matches symbol name or name@@LIB for library + * functions. + */ +static long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) +{ + struct elf_func_offset data = { + .name = name, + .last_bind = -1, + .name_len = strlen(name), + /* Does name specify "@@LIB"? */ + .is_name_qualified = strstr(name, "@@") != NULL, + }; + bool is_shared_lib; + GElf_Ehdr ehdr; + long ret; + int err; - ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset; - last_bind = curr_bind; - } - if (ret > 0) - break; + if (!gelf_getehdr(elf, &ehdr)) { + pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1)); + ret = -LIBBPF_ERRNO__FORMAT; + goto out; } + /* for shared lib case, we do not need to calculate relative offset */ + is_shared_lib = ehdr.e_type == ET_DYN; + + err = elf_for_each_symbol(elf, binary_path, single_match, single_done, &data); + ret = err < 0 ? (long) err : data.offset; if (ret > 0) { pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", name, binary_path, ret); From patchwork Mon Apr 24 16:04:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222351 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 358C0C77B73 for ; Mon, 24 Apr 2023 16:06:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231974AbjDXQGG (ORCPT ); Mon, 24 Apr 2023 12:06:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231945AbjDXQGF (ORCPT ); Mon, 24 Apr 2023 12:06:05 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C4B272BD for ; Mon, 24 Apr 2023 09:06:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9F8A861B47 for ; Mon, 24 Apr 2023 16:06:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51B64C433D2; Mon, 24 Apr 2023 16:06:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352363; bh=OTQYlcnuzdMEc76T8nbnYmY0E6KYAqPxBfN0HeJ4HL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SVuKnvUr5vJ+wXTx5cwKzvQF9/SCAAmDOnMwfkTSbvl8Mh74SpBh/NR0XahdsCJ77 uWD6tviJLAP52V3V92CEdf2eRsuQpZo7e9zSr+9SOKJUlZoy2yDYsSMYfFgOBUkSpg MtlJBSmy3e9A0GD7voIG7dRDtlIecoLnkjUi48BTwyRpor5ZjCFVdFcxEsB7i13fo9 aE2lszOjUVobOn81+jiv+lPjLTNwXES7OQFhkvMqmXem+7s1xNb1SWuEyJMSuIbOBL jTUkQqYbrhB7nm+ouaXOZRAW8+0Hkr3DD6PacH0zCdLCQC2iniF+yuFOHdcG9clvcL n1X6q8VoR9Gnw== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 07/20] libbpf: Add elf_find_multi_func_offset function Date: Mon, 24 Apr 2023 18:04:34 +0200 Message-Id: <20230424160447.2005755-8-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding elf_find_multi_func_offset function that looks up offsets for symbols specified in syms array argument. Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 161 +++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 6 ++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 168 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 92c92ed2101f..0b15609d4573 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -10713,6 +10713,7 @@ struct elf_func_offset { int last_bind; size_t name_len; bool is_name_qualified; + int idx; }; static int single_done(void *_data) @@ -10891,6 +10892,166 @@ static long elf_find_func_offset(Elf *elf, const char *binary_path, const char * return ret; } +struct match_multi_data { + int cnt; + int cnt_done; + struct elf_func_offset *func_offs; +}; + +static int cmp_func_offset(const void *_a, const void *_b) +{ + const struct elf_func_offset *a = _a; + const struct elf_func_offset *b = _b; + + return strcmp(a->name, b->name); +} + +static int multi_done(void *_data) +{ + struct match_multi_data *data = _data; + + return data->cnt == data->cnt_done; +} + +static int multi_match(Elf *elf, const char *binary_path, const char *sname, + GElf_Sym *sym, void *_data) +{ + struct match_multi_data *data = _data; + struct elf_func_offset *fo, func_offs = { + .name = sname, + }; + Elf_Scn *sym_scn; + GElf_Shdr sym_sh; + int curr_bind; + + fo = bsearch(&func_offs, data->func_offs, data->cnt, sizeof(*data->func_offs), + cmp_func_offset); + if (!fo) + return 0; + + curr_bind = GELF_ST_BIND(sym->st_info); + + if (fo->offset > 0) { + /* handle multiple matches */ + if (fo->last_bind != STB_WEAK && curr_bind != STB_WEAK) { + /* Only accept one non-weak bind. */ + pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n", + sname, fo->name, binary_path); + return -LIBBPF_ERRNO__FORMAT; + } else if (curr_bind == STB_WEAK) { + /* already have a non-weak bind, and + * this is a weak bind, so ignore. + */ + return 0; + } + } + + /* Transform symbol's virtual address (absolute for + * binaries and relative for shared libs) into file + * offset, which is what kernel is expecting for + * uprobe/uretprobe attachment. + * See Documentation/trace/uprobetracer.rst for more + * details. + * This is done by looking up symbol's containing + * section's header and using it's virtual address + * (sh_addr) and corresponding file offset (sh_offset) + * to transform sym.st_value (virtual address) into + * desired final file offset. + */ + sym_scn = elf_getscn(elf, sym->st_shndx); + if (!sym_scn) + return 0; + if (!gelf_getshdr(sym_scn, &sym_sh)) + return 0; + + if (!fo->offset) + data->cnt_done++; + + fo->offset = sym->st_value - sym_sh.sh_addr + sym_sh.sh_offset; + fo->last_bind = curr_bind; + return 0; +} + +static int +__elf_find_multi_func_offset(Elf *elf, const char *binary_path, int cnt, + const char **syms, unsigned long **poffsets) +{ + struct match_multi_data data = { + .cnt = cnt, + }; + struct elf_func_offset *func_offs; + unsigned long *offsets = NULL; + int err, i, idx; + + data.func_offs = func_offs = calloc(cnt, sizeof(*func_offs)); + if (!func_offs) + return -ENOMEM; + + for (i = 0; i < cnt; i++) { + func_offs[i].name = syms[i]; + func_offs[i].idx = i; + } + + qsort(func_offs, cnt, sizeof(*func_offs), cmp_func_offset); + + err = elf_for_each_symbol(elf, binary_path, multi_match, multi_done, &data); + if (err) + goto out; + + if (cnt != data.cnt_done) { + err = -ENOENT; + goto out; + } + + offsets = calloc(cnt, sizeof(*offsets)); + if (!offsets) { + err = -ENOMEM; + goto out; + } + + for (i = 0; i < cnt; i++) { + idx = func_offs[i].idx; + offsets[idx] = func_offs[i].offset; + } + +out: + *poffsets = offsets; + free(func_offs); + return err; +} + +LIBBPF_API int +elf_find_multi_func_offset(const char *binary_path, int cnt, + const char **syms, unsigned long **poffsets) +{ + char errmsg[STRERR_BUFSIZE]; + long ret = -ENOENT; + Elf *elf; + int fd; + + fd = open(binary_path, O_RDONLY | O_CLOEXEC); + if (fd < 0) { + ret = -errno; + pr_warn("failed to open %s: %s\n", binary_path, + libbpf_strerror_r(ret, errmsg, sizeof(errmsg))); + return ret; + } + if (elf_version(EV_CURRENT) == EV_NONE) { + pr_warn("failed to init libelf for %s\n", binary_path); + return -LIBBPF_ERRNO__LIBELF; + } + elf = elf_begin(fd, ELF_C_READ_MMAP, NULL); + if (!elf) { + pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1)); + close(fd); + return -LIBBPF_ERRNO__FORMAT; + } + ret = __elf_find_multi_func_offset(elf, binary_path, cnt, syms, poffsets); + elf_end(elf); + close(fd); + return ret; +} + /* Find offset of function name in ELF object specified by path. "name" matches * symbol name or name@@LIB for library functions. */ diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 0b7362397ea3..b1b159263d47 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -1649,6 +1649,12 @@ LIBBPF_API int libbpf_register_prog_handler(const char *sec, */ LIBBPF_API int libbpf_unregister_prog_handler(int handler_id); +/** + * @brief *elf_find_multi_func_offset()* return offsets for given *syms* + */ +LIBBPF_API int elf_find_multi_func_offset(const char *binary_path, int cnt, + const char **syms, unsigned long **poffsets); + #ifdef __cplusplus } /* extern "C" */ #endif diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index a5aa3a383d69..7b1bf3f9ce4f 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -390,4 +390,5 @@ LIBBPF_1.2.0 { bpf_link_get_info_by_fd; bpf_map_get_info_by_fd; bpf_prog_get_info_by_fd; + elf_find_multi_func_offset; } LIBBPF_1.1.0; From patchwork Mon Apr 24 16:04:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222352 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC8A7C77B61 for ; Mon, 24 Apr 2023 16:06:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231916AbjDXQGP (ORCPT ); Mon, 24 Apr 2023 12:06:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231823AbjDXQGP (ORCPT ); Mon, 24 Apr 2023 12:06:15 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BC986A69 for ; Mon, 24 Apr 2023 09:06:14 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9ECCC60C7A for ; Mon, 24 Apr 2023 16:06:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 56C52C433D2; Mon, 24 Apr 2023 16:06:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352373; bh=seAKlsjYskREnG4tQ1+NlssuXIJTxUZaDQir7NEaDCs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DKwfwatHunBMHHlj+vbn1e1sZq7PLbkIxgNKmaqo7V3naDf9ME4as4u+eRrVrBHKd XGmp9sPJWTzxn3YPX5Kh0wEE19bHGPLrgNDweOoTAh6dlMyIaoJ8Yrc2ciubWDwU5W Ihtj53OoMCy2pejsY922vB8HIPG9myZAGZSSEUcTbanblBO5nCWOyaMgitOi41V6Ax LtcwX+GjFysyPF//ym8OYkzS+9TO/e5b9riuqNv4q/nXlmDOW7VtLPUEKoQ5fuenCi 6WyBmUTiwUm5/KIsdSsTglt8vkBp7q2tsqZ+ARn+pzkO+TLnhPlMVb967QboHGQYUt CiAjskLAA1GUA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 08/20] libbpf: Add elf_find_patern_func_offset function Date: Mon, 24 Apr 2023 18:04:35 +0200 Message-Id: <20230424160447.2005755-9-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding elf_find_patern_func_offset function that looks up offsets for symbols specified by pattern argument. The 'pattern' argument allows wildcards (*?' supported). Offsets are returned in allocated array together with its size and needs to be released by the caller. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 121 +++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 7 +++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 129 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 0b15609d4573..7eb7035f7b73 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -11052,6 +11052,127 @@ elf_find_multi_func_offset(const char *binary_path, int cnt, return ret; } +struct match_pattern_data { + const char *pattern; + struct elf_func_offset *func_offs; + size_t func_offs_cnt; + size_t func_offs_cap; +}; + +static int pattern_done(void *_data) +{ + struct match_pattern_data *data = _data; + + // If we found anything in the first symbol section, do not search others + // to avoid duplicates. + return data->func_offs_cnt; +} + +static int pattern_match(Elf *elf, const char *binary_path, const char *sname, + GElf_Sym *sym, void *_data) +{ + struct match_pattern_data *data = _data; + unsigned long offset; + Elf_Scn *sym_scn; + GElf_Shdr sym_sh; + int err; + + if (!glob_match(sname, data->pattern)) + return 0; + sym_scn = elf_getscn(elf, sym->st_shndx); + if (!sym_scn) + return 0; + if (!gelf_getshdr(sym_scn, &sym_sh)) + return 0; + + err = libbpf_ensure_mem((void **) &data->func_offs, &data->func_offs_cap, + sizeof(*data->func_offs), data->func_offs_cnt + 1); + if (err) + return err; + + offset = sym->st_value - sym_sh.sh_addr + sym_sh.sh_offset; + data->func_offs[data->func_offs_cnt].offset = offset; + data->func_offs[data->func_offs_cnt].name = strdup(sname); + data->func_offs_cnt++; + return 0; +} + +static int +__elf_find_patern_func_offset(Elf *elf, const char *binary_path, const char *pattern, + const char ***pnames, unsigned long **poffsets, size_t *pcnt) +{ + struct match_pattern_data data = { + .pattern = pattern, + }; + unsigned long *offsets = NULL; + const char **names = NULL; + size_t cnt = 0; + int err, i; + + err = elf_for_each_symbol(elf, binary_path, pattern_match, pattern_done, &data); + if (err) + goto out; + + cnt = data.func_offs_cnt; + if (!cnt) { + err = -ENOENT; + goto out; + } + + offsets = calloc(cnt, sizeof(*offsets)); + names = calloc(cnt, sizeof(*names)); + if (!offsets || !names) { + free(offsets); + free(names); + err = -ENOMEM; + goto out; + } + + for (i = 0; i < cnt; i++) { + offsets[i] = data.func_offs[i].offset; + names[i] = data.func_offs[i].name; + } + +out: + *pnames = names; + *poffsets = offsets; + *pcnt = cnt; + free(data.func_offs); + return err; +} + +LIBBPF_API int +elf_find_patern_func_offset(const char *binary_path, const char *pattern, + const char ***pnames, unsigned long **poffsets, size_t *pcnt) +{ + char errmsg[STRERR_BUFSIZE]; + long ret = -ENOENT; + Elf *elf; + int fd; + + fd = open(binary_path, O_RDONLY | O_CLOEXEC); + if (fd < 0) { + ret = -errno; + pr_warn("failed to open %s: %s\n", binary_path, + libbpf_strerror_r(ret, errmsg, sizeof(errmsg))); + return ret; + } + if (elf_version(EV_CURRENT) == EV_NONE) { + pr_warn("failed to init libelf for %s\n", binary_path); + return -LIBBPF_ERRNO__LIBELF; + } + elf = elf_begin(fd, ELF_C_READ_MMAP, NULL); + if (!elf) { + pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1)); + close(fd); + return -LIBBPF_ERRNO__FORMAT; + } + ret = __elf_find_patern_func_offset(elf, binary_path, pattern, pnames, poffsets, pcnt); + elf_end(elf); + close(fd); + return ret; +} + /* Find offset of function name in ELF object specified by path. "name" matches * symbol name or name@@LIB for library functions. */ diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index b1b159263d47..96ed109ae011 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -1655,6 +1655,13 @@ LIBBPF_API int libbpf_unregister_prog_handler(int handler_id); LIBBPF_API int elf_find_multi_func_offset(const char *binary_path, int cnt, const char **syms, unsigned long **poffsets); +/** + * @brief *elf_find_patern_func_offset()* return symbols and offsets for given *pattern* + */ +LIBBPF_API int +elf_find_patern_func_offset(const char *binary_path, const char *pattern, + const char ***pnames, unsigned long **poffsets, size_t *pcnt); + #ifdef __cplusplus } /* extern "C" */ #endif diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 7b1bf3f9ce4f..2f091a0f093b 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -391,4 +391,5 @@ LIBBPF_1.2.0 { bpf_map_get_info_by_fd; bpf_prog_get_info_by_fd; elf_find_multi_func_offset; + elf_find_patern_func_offset; } LIBBPF_1.1.0; From patchwork Mon Apr 24 16:04:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222353 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7871C7618E for ; Mon, 24 Apr 2023 16:06:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231823AbjDXQGZ (ORCPT ); Mon, 24 Apr 2023 12:06:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230340AbjDXQGY (ORCPT ); Mon, 24 Apr 2023 12:06:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2DCB5FE4 for ; Mon, 24 Apr 2023 09:06:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7F0BB619FC for ; Mon, 24 Apr 2023 16:06:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32DD6C433EF; Mon, 24 Apr 2023 16:06:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352382; bh=AqRLMtmRhgwygSKYtfSyTQDKQgYEwziG5bQpBxvD168=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XqhOrlPEwKPOk/kpXh+NpXSTlT/bD3IZEnERg8AWGKH6dPcRNOmGg4IZFclZVNTnu BVSPq21aWZCwTamvEOLuGU8SPN5o+IsoFYFXcq00lySjiwOwEWy+n2QvtiGgzBhC7h PkMGcXuHspLvI7/Xpctq+h3dA/QfFA5G4otysBBn3xBiyYe8xoXC/5jW/yhUOwY5bN RlcMP6yBmZ8gsTMrpMnUFpuH1u2USHZstf2NRrUQKmRrvJnsWi60rvtLO5ISxhZVtb VsSvbDBC4pa0HAC0RaS1i1zjkD2edi7TzCyV5nEBYUH1tEILDEA49xFNbSr9WoJPI6 JvuiLlNYlbnhQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 09/20] libbpf: Add bpf_link_create support for multi uprobes Date: Mon, 24 Apr 2023 18:04:36 +0200 Message-Id: <20230424160447.2005755-10-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding new uprobe_multi struct to bpf_link_create_opts object to pass multiple uprobe data to link_create attr uapi. Signed-off-by: Jiri Olsa --- tools/lib/bpf/bpf.c | 10 ++++++++++ tools/lib/bpf/bpf.h | 10 +++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 128ac723c4ea..de227846fa3b 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -722,6 +722,16 @@ int bpf_link_create(int prog_fd, int target_fd, if (!OPTS_ZEROED(opts, kprobe_multi)) return libbpf_err(-EINVAL); break; + case BPF_TRACE_UPROBE_MULTI: + attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0); + attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0); + attr.link_create.uprobe_multi.paths = ptr_to_u64(OPTS_GET(opts, uprobe_multi.paths, 0)); + attr.link_create.uprobe_multi.offsets = ptr_to_u64(OPTS_GET(opts, uprobe_multi.offsets, 0)); + attr.link_create.uprobe_multi.ref_ctr_offsets = ptr_to_u64(OPTS_GET(opts, uprobe_multi.ref_ctr_offsets, 0)); + attr.link_create.uprobe_multi.cookies = ptr_to_u64(OPTS_GET(opts, uprobe_multi.cookies, 0)); + if (!OPTS_ZEROED(opts, uprobe_multi)) + return libbpf_err(-EINVAL); + break; case BPF_TRACE_FENTRY: case BPF_TRACE_FEXIT: case BPF_MODIFY_RETURN: diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index a2c091389b18..9404096b2cf0 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -332,13 +332,21 @@ struct bpf_link_create_opts { const unsigned long *addrs; const __u64 *cookies; } kprobe_multi; + struct { + __u32 flags; + __u32 cnt; + const char **paths; + const unsigned long *offsets; + const unsigned long *ref_ctr_offsets; + const __u64 *cookies; + } uprobe_multi; struct { __u64 cookie; } tracing; }; size_t :0; }; -#define bpf_link_create_opts__last_field kprobe_multi.cookies +#define bpf_link_create_opts__last_field uprobe_multi.cookies LIBBPF_API int bpf_link_create(int prog_fd, int target_fd, enum bpf_attach_type attach_type, From patchwork Mon Apr 24 16:04:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222354 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B17AC77B61 for ; Mon, 24 Apr 2023 16:06:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230340AbjDXQGg (ORCPT ); Mon, 24 Apr 2023 12:06:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231842AbjDXQGf (ORCPT ); Mon, 24 Apr 2023 12:06:35 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E556C6EB2 for ; Mon, 24 Apr 2023 09:06:33 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8138661B47 for ; Mon, 24 Apr 2023 16:06:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BAC5C433EF; Mon, 24 Apr 2023 16:06:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352392; bh=/9ywEhBkIC/m5wfYdpkWTwLs5LVLRAYXfjnzpncDMLU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K04ZpHuB6QPPKKg356JBG0eVDcDfxeNaqSuN1DQWezVwXUpxa8MrTFEiIF6qrmTMH 9JHU6LbEr7X3CCM60fJ0R9oEOXpWhySFRY9AZjkgodU8Spzs8y21Ipxza+yxRTjVmM tXgrjUPL6/IiVQCtyoy5u9/cdjplRlQ5q02y+huXtyAl2ind3ztDDxk9KAzvXJKliW P3zZ8Ett5KkGdB4HQkasWWP+PEyN6p4qLea9y2lxZu0bZ/HeDLPGA/+R7wGtuqbAsC MYAEdow5QZoOBZplWen3DVaAxsn0tq6UOJsRDhNpb9NEq+fxgFcojxAbvxX4N7zVfF w7Xge4olb9SEA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 10/20] libbpf: Add bpf_program__attach_uprobe_multi_opts function Date: Mon, 24 Apr 2023 18:04:37 +0200 Message-Id: <20230424160447.2005755-11-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding bpf_program__attach_uprobe_multi_opts function that allows to attach multiple uprobes with uprobe_multi link. The user can specify uprobes with direct arguments: binary_path/func_pattern or with struct bpf_uprobe_multi_opts opts argument fields: const char **paths; const char **syms; const unsigned long *offsets; const unsigned long *ref_ctr_offsets; User can specify 3 mutually exclusive set of incputs: 1) use only binary_path/func_pattern aruments 2) use only opts argument with allowed combinations of: paths/offsets/ref_ctr_offsets/cookies/cnt 3) use binary_path with allowed combinations of: syms/offsets/ref_ctr_offsets/cookies/cnt Any other usage results in error. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 137 +++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 28 ++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 166 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 7eb7035f7b73..c786bc142791 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -11345,6 +11345,143 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz) return -ENOENT; } +struct bpf_link * +bpf_program__attach_uprobe_multi_opts(const struct bpf_program *prog, + const char *binary_path, + const char *func_pattern, + const struct bpf_uprobe_multi_opts *opts) +{ + const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL; + LIBBPF_OPTS(bpf_link_create_opts, lopts); + unsigned long *resolved_offsets = NULL; + const char **resolved_symbols = NULL; + const char **resolved_paths = NULL; + int i, err = 0, link_fd, prog_fd; + struct bpf_link *link = NULL; + char errmsg[STRERR_BUFSIZE]; + const char **paths, **syms; + char full_path[PATH_MAX]; + const __u64 *cookies; + bool has_pattern; + bool retprobe; + size_t cnt; + + if (!OPTS_VALID(opts, bpf_uprobe_multi_opts)) + return libbpf_err_ptr(-EINVAL); + + paths = OPTS_GET(opts, paths, NULL); + syms = OPTS_GET(opts, syms, NULL); + offsets = OPTS_GET(opts, offsets, NULL); + ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL); + cookies = OPTS_GET(opts, cookies, NULL); + cnt = OPTS_GET(opts, cnt, 0); + + /* + * User can specify 3 mutually exclusive set of incputs: + * + * 1) use only binary_path/func_pattern aruments + * + * 2) use only opts argument with allowed combinations of: + * paths/offsets/ref_ctr_offsets/cookies/cnt + * + * 3) use binary_path with allowed combinations of: + * syms/offsets/ref_ctr_offsets/cookies/cnt + * + * Any other usage results in error. + */ + + if (!binary_path && !func_pattern && !cnt) + return libbpf_err_ptr(-EINVAL); + if (func_pattern && !binary_path) + return libbpf_err_ptr(-EINVAL); + + has_pattern = binary_path && func_pattern; + + if (has_pattern) { + if (paths || syms || offsets || ref_ctr_offsets || cookies || cnt) + return libbpf_err_ptr(-EINVAL); + } else { + if (!cnt) + return libbpf_err_ptr(-EINVAL); + if (!!paths == !!binary_path) + return libbpf_err_ptr(-EINVAL); + if (!!syms == !!offsets) + return libbpf_err_ptr(-EINVAL); + if (paths && syms) + return libbpf_err_ptr(-EINVAL); + } + + if (has_pattern) { + if (!strchr(binary_path, '/')) { + err = resolve_full_path(binary_path, full_path, sizeof(full_path)); + if (err) { + pr_warn("prog '%s': failed to resolve full path for '%s': %d\n", + prog->name, binary_path, err); + return libbpf_err_ptr(err); + } + binary_path = full_path; + } + + err = elf_find_patern_func_offset(binary_path, func_pattern, + &resolved_symbols, &resolved_offsets, + &cnt); + if (err < 0) + return libbpf_err_ptr(err); + offsets = resolved_offsets; + } else if (syms) { + err = elf_find_multi_func_offset(binary_path, cnt, syms, &resolved_offsets); + if (err < 0) + return libbpf_err_ptr(err); + offsets = resolved_offsets; + } + + if (binary_path) { + resolved_paths = calloc(cnt, sizeof(*paths)); + if (!resolved_paths) + goto error; + for (i = 0; i < cnt; i++) + resolved_paths[i] = binary_path; + paths = resolved_paths; + } + + retprobe = OPTS_GET(opts, retprobe, false); + + lopts.uprobe_multi.paths = paths; + lopts.uprobe_multi.offsets = offsets; + lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; + lopts.uprobe_multi.cookies = cookies; + lopts.uprobe_multi.cnt = cnt; + lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0; + + link = calloc(1, sizeof(*link)); + if (!link) { + err = -ENOMEM; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd = bpf_program__fd(prog); + link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts); + if (link_fd < 0) { + err = -errno; + pr_warn("prog '%s': failed to attach: %s\n", + prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg))); + goto error; + } + link->fd = link_fd; + free(resolved_offsets); + free(resolved_symbols); + free(resolved_paths); + return link; + +error: + free(resolved_offsets); + free(resolved_symbols); + free(resolved_paths); + free(link); + return libbpf_err_ptr(err); +} + LIBBPF_API struct bpf_link * bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, const char *binary_path, size_t func_offset, diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 96ed109ae011..921ab2a94cec 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -529,6 +529,34 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, const char *pattern, const struct bpf_kprobe_multi_opts *opts); +struct bpf_uprobe_multi_opts { + /* size of this struct, for forward/backward compatibility */ + size_t sz; + /* array of paths to attach */ + const char **paths; + /* array of function symbols to attach */ + const char **syms; + /* array of function addresses to attach */ + const unsigned long *offsets; + /* array of refctr offsets to attach */ + const unsigned long *ref_ctr_offsets; + /* array of user-provided values fetchable through bpf_get_attach_cookie */ + const __u64 *cookies; + /* number of elements in syms/addrs/cookies arrays */ + size_t cnt; + /* create return uprobes */ + bool retprobe; + size_t :0; +}; + +#define bpf_uprobe_multi_opts__last_field retprobe + +LIBBPF_API struct bpf_link * +bpf_program__attach_uprobe_multi_opts(const struct bpf_program *prog, + const char *binary_path, + const char *func_pattern, + const struct bpf_uprobe_multi_opts *opts); + struct bpf_ksyscall_opts { /* size of this struct, for forward/backward compatibility */ size_t sz; diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 2f091a0f093b..7aae41ff181d 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -392,4 +392,5 @@ LIBBPF_1.2.0 { bpf_prog_get_info_by_fd; elf_find_multi_func_offset; elf_find_patern_func_offset; + bpf_program__attach_uprobe_multi_opts; } LIBBPF_1.1.0; From patchwork Mon Apr 24 16:04:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222355 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D151DC77B61 for ; Mon, 24 Apr 2023 16:06:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231893AbjDXQGp (ORCPT ); Mon, 24 Apr 2023 12:06:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231842AbjDXQGo (ORCPT ); Mon, 24 Apr 2023 12:06:44 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D18516A69 for ; Mon, 24 Apr 2023 09:06:43 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6AE3261BD1 for ; Mon, 24 Apr 2023 16:06:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1791EC433D2; Mon, 24 Apr 2023 16:06:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352402; bh=V2v5G7vW0QNg1w/Ta3CnwwlfLUBck1eIJQKeJAWZ254=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xr6nyjTKelj/UgnIeBPNugZ8HSbWT8+mFsLoI7l+VRxOvA2TI0cfkD3uBgk5q1Lis HzmqQlsZJ0b/1PWmIzMZoUnTF92+5Nza4md/4O9iBDE92lsmr+X5182QGX839LZRKb 1XaM/Q17PIi5rxwcBOo9QgGcQ+/8O7Xe0BjRs+ZBoN9jnIevhyphmW6T2sO4ISBMZE bZ14SmktfBHakgl5+3oHNLnaouxULMfQuYj3a+ecNGElzSRqdsWE93770y9SDqZlBw 6BVwmX+1cA+Siyyp3dKbvVqoUngzsiM/LCxOc3aI2EJ5PTHjTPyNjmevZ8Vd/0uvqj RWBtpC72wUSgA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 11/20] libbpf: Add support for uprobe.multi/uprobe.multi program sections Date: Mon, 24 Apr 2023 18:04:38 +0200 Message-Id: <20230424160447.2005755-12-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding support for uprobe.multi/uprobe.multi program sections to allow auto attach of multi_uprobe programs. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index c786bc142791..70353aaac86e 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -8628,6 +8628,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link); +static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link); static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link); @@ -8643,6 +8644,8 @@ static const struct bpf_sec_def section_defs[] = { SEC_DEF("uretprobe.s+", KPROBE, 0, SEC_SLEEPABLE, attach_uprobe), SEC_DEF("kprobe.multi+", KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi), SEC_DEF("kretprobe.multi+", KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi), + SEC_DEF("uprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), + SEC_DEF("uretprobe.multi+", KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi), SEC_DEF("ksyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("kretsyscall+", KPROBE, 0, SEC_NONE, attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_NONE, attach_usdt), @@ -10611,6 +10614,41 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru return libbpf_get_error(*link); } +static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link) +{ + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts); + char *probe_type = NULL, *binary_path = NULL, *func_name = NULL; + int n, ret = -EINVAL; + + *link = NULL; + + n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.*?]+", + &probe_type, &binary_path, &func_name); + switch (n) { + case 1: + /* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */ + ret = 0; + break; + case 2: + pr_warn("prog '%s': section '%s' missing ':function[+offset]' specification\n", + prog->name, prog->sec_name); + break; + case 3: + opts.retprobe = strcmp(probe_type, "uretprobe.multi"); + *link = bpf_program__attach_uprobe_multi_opts(prog, binary_path, func_name, &opts); + ret = libbpf_get_error(*link); + break; + default: + pr_warn("prog '%s': invalid format of section definition '%s'\n", prog->name, + prog->sec_name); + break; + } + free(probe_type); + free(binary_path); + free(func_name); + return ret; +} + static void gen_uprobe_legacy_event_name(char *buf, size_t buf_sz, const char *binary_path, uint64_t offset) { From patchwork Mon Apr 24 16:04:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222356 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B238C77B61 for ; Mon, 24 Apr 2023 16:07:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231715AbjDXQG7 (ORCPT ); Mon, 24 Apr 2023 12:06:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232060AbjDXQGz (ORCPT ); Mon, 24 Apr 2023 12:06:55 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C063E7EFB for ; Mon, 24 Apr 2023 09:06:53 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4A1D561B3B for ; Mon, 24 Apr 2023 16:06:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F306FC433D2; Mon, 24 Apr 2023 16:06:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352412; bh=pNRjBUNyKBNcAnDVy1rTVmvLfFVckIGQV5XaTb0Zfb0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FYYxnDOYVhM0aIo/brcLuIbYted4gGrHIPdR8jQlQxglDFTEfPuynIHb/Ui9KFidD Od1JKWk6mm9Wt4bS2wzU70TXf/9Z3/YVlGEdwVwcqdsNCfrrYNz1GMwtTuZO9vgo5R S2Yek1ePq2Ok1WpxUS8yeoSkfuecWfSVtEjMW6VPHnUnb8RLjOgskNE0enPFXUPDo5 UWcgoyycl3wr8T/zPDboWqlN4FQqcPoGeoAXyr0s9OYgG8xxqW29oc0u7ZR+n5VBeB 86cudWc5bd8/wiFFObjXxvMzqfSoJ8PB+s5nLFL8WqRYr7+EpofhT5gic0uyTCY0G0 +JJPEA4LDe6xQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 12/20] libbpf: Add uprobe multi link support to bpf_program__attach_usdt Date: Mon, 24 Apr 2023 18:04:39 +0200 Message-Id: <20230424160447.2005755-13-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding uprobe_multi bool to struct bpf_usdt_opts. If it's true the usdt_manager_attach_usdt will use uprobe_multi link to attach to usdt probes. The bpf program for usdt probe needs to have BPF_TRACE_UPROBE_MULTI set as expected_attach_type. Because current uprobe is implemented through perf event interface, it allows the pid filter for uprobes. This is not the case for uprobe_multi link, so the pid filter is not allowed for that. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 9 ++- tools/lib/bpf/libbpf.h | 2 + tools/lib/bpf/libbpf_internal.h | 2 +- tools/lib/bpf/usdt.c | 127 ++++++++++++++++++++++++-------- 4 files changed, 105 insertions(+), 35 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 70353aaac86e..25d32aa605e8 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6817,7 +6817,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog if (!insns || !insns_cnt) return -EINVAL; - load_attr.expected_attach_type = prog->expected_attach_type; if (kernel_supports(obj, FEAT_PROG_NAME)) prog_name = prog->name; load_attr.attach_prog_fd = prog->attach_prog_fd; @@ -6853,6 +6852,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog insns_cnt = prog->insns_cnt; } + /* allow prog_prepare_load_fn to change expected_attach_type */ + load_attr.expected_attach_type = prog->expected_attach_type; + if (obj->gen_loader) { bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name, license, insns, insns_cnt, &load_attr, @@ -11730,6 +11732,7 @@ struct bpf_link *bpf_program__attach_usdt(const struct bpf_program *prog, struct bpf_object *obj = prog->obj; struct bpf_link *link; __u64 usdt_cookie; + bool uprobe_multi; int err; if (!OPTS_VALID(opts, bpf_uprobe_opts)) @@ -11766,8 +11769,10 @@ struct bpf_link *bpf_program__attach_usdt(const struct bpf_program *prog, } usdt_cookie = OPTS_GET(opts, usdt_cookie, 0); + uprobe_multi = OPTS_GET(opts, uprobe_multi, 0); link = usdt_manager_attach_usdt(obj->usdt_man, prog, pid, binary_path, - usdt_provider, usdt_name, usdt_cookie); + usdt_provider, usdt_name, usdt_cookie, + uprobe_multi); err = libbpf_get_error(link); if (err) return libbpf_err_ptr(err); diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 921ab2a94cec..025feb21c2ec 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -673,6 +673,8 @@ struct bpf_usdt_opts { size_t sz; /* custom user-provided value accessible through usdt_cookie() */ __u64 usdt_cookie; + /* use uprobe_multi link */ + bool uprobe_multi; size_t :0; }; #define bpf_usdt_opts__last_field usdt_cookie diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index e4d05662a96c..5d5f61d0bcfb 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -567,7 +567,7 @@ struct bpf_link * usdt_manager_attach_usdt(struct usdt_manager *man, const struct bpf_program *prog, pid_t pid, const char *path, const char *usdt_provider, const char *usdt_name, - __u64 usdt_cookie); + __u64 usdt_cookie, bool uprobe_multi); static inline bool is_pow_of_2(size_t x) { diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c index b8402e3f9eb2..f55dbd47d29e 100644 --- a/tools/lib/bpf/usdt.c +++ b/tools/lib/bpf/usdt.c @@ -803,11 +803,20 @@ struct bpf_link_usdt { size_t spec_cnt; int *spec_ids; + bool has_uprobe_multi; + size_t uprobe_cnt; struct { long abs_ip; struct bpf_link *link; } *uprobes; + struct { + char **paths; + unsigned long *offsets; + unsigned long *ref_ctr_offsets; + __u64 *cookies; + struct bpf_link *link; + } uprobe_multi; }; static int bpf_link_usdt_detach(struct bpf_link *link) @@ -816,19 +825,23 @@ static int bpf_link_usdt_detach(struct bpf_link *link) struct usdt_manager *man = usdt_link->usdt_man; int i; - for (i = 0; i < usdt_link->uprobe_cnt; i++) { - /* detach underlying uprobe link */ - bpf_link__destroy(usdt_link->uprobes[i].link); - /* there is no need to update specs map because it will be - * unconditionally overwritten on subsequent USDT attaches, - * but if BPF cookies are not used we need to remove entry - * from ip_to_spec_id map, otherwise we'll run into false - * conflicting IP errors - */ - if (!man->has_bpf_cookie) { - /* not much we can do about errors here */ - (void)bpf_map_delete_elem(bpf_map__fd(man->ip_to_spec_id_map), - &usdt_link->uprobes[i].abs_ip); + if (usdt_link->has_uprobe_multi) { + bpf_link__destroy(usdt_link->uprobe_multi.link); + } else { + for (i = 0; i < usdt_link->uprobe_cnt; i++) { + /* detach underlying uprobe link */ + bpf_link__destroy(usdt_link->uprobes[i].link); + /* there is no need to update specs map because it will be + * unconditionally overwritten on subsequent USDT attaches, + * but if BPF cookies are not used we need to remove entry + * from ip_to_spec_id map, otherwise we'll run into false + * conflicting IP errors + */ + if (!man->has_bpf_cookie) { + /* not much we can do about errors here */ + (void)bpf_map_delete_elem(bpf_map__fd(man->ip_to_spec_id_map), + &usdt_link->uprobes[i].abs_ip); + } } } @@ -868,9 +881,16 @@ static void bpf_link_usdt_dealloc(struct bpf_link *link) { struct bpf_link_usdt *usdt_link = container_of(link, struct bpf_link_usdt, link); - free(usdt_link->spec_ids); - free(usdt_link->uprobes); - free(usdt_link); + if (usdt_link->has_uprobe_multi) { + free(usdt_link->uprobe_multi.paths); + free(usdt_link->uprobe_multi.offsets); + free(usdt_link->uprobe_multi.ref_ctr_offsets); + free(usdt_link->uprobe_multi.cookies); + } else { + free(usdt_link->spec_ids); + free(usdt_link->uprobes); + free(usdt_link); + } } static size_t specs_hash_fn(long key, void *ctx) @@ -941,16 +961,22 @@ static int allocate_spec_id(struct usdt_manager *man, struct hashmap *specs_hash struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct bpf_program *prog, pid_t pid, const char *path, const char *usdt_provider, const char *usdt_name, - __u64 usdt_cookie) + __u64 usdt_cookie, bool uprobe_multi) { + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts_multi); int i, fd, err, spec_map_fd, ip_map_fd; LIBBPF_OPTS(bpf_uprobe_opts, opts); struct hashmap *specs_hash = NULL; struct bpf_link_usdt *link = NULL; struct usdt_target *targets = NULL; + struct bpf_link *uprobe_link; size_t target_cnt; Elf *elf; + /* The uprobe_multi link does not have pid filter. */ + if (uprobe_multi && pid >= 0) + return libbpf_err_ptr(-EINVAL); + spec_map_fd = bpf_map__fd(man->specs_map); ip_map_fd = bpf_map__fd(man->ip_to_spec_id_map); @@ -1001,19 +1027,32 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct goto err_out; } + link->has_uprobe_multi = uprobe_multi; link->usdt_man = man; link->link.detach = &bpf_link_usdt_detach; link->link.dealloc = &bpf_link_usdt_dealloc; - link->uprobes = calloc(target_cnt, sizeof(*link->uprobes)); - if (!link->uprobes) { - err = -ENOMEM; - goto err_out; + if (uprobe_multi) { + link->uprobe_multi.paths = calloc(target_cnt, sizeof(*link->uprobe_multi.paths)); + link->uprobe_multi.offsets = calloc(target_cnt, sizeof(*link->uprobe_multi.offsets)); + link->uprobe_multi.ref_ctr_offsets = calloc(target_cnt, sizeof(*link->uprobe_multi.ref_ctr_offsets)); + link->uprobe_multi.cookies = calloc(target_cnt, sizeof(*link->uprobe_multi.cookies)); + + if (!link->uprobe_multi.paths || !link->uprobe_multi.offsets || + !link->uprobe_multi.ref_ctr_offsets || !link->uprobe_multi.cookies) { + err = -ENOMEM; + goto err_out; + } + } else { + link->uprobes = calloc(target_cnt, sizeof(*link->uprobes)); + if (!link->uprobes) { + err = -ENOMEM; + goto err_out; + } } for (i = 0; i < target_cnt; i++) { struct usdt_target *target = &targets[i]; - struct bpf_link *uprobe_link; bool is_new; int spec_id; @@ -1049,20 +1088,44 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct goto err_out; } - opts.ref_ctr_offset = target->sema_off; - opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0; - uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path, - target->rel_ip, &opts); + if (uprobe_multi) { + link->uprobe_multi.paths[i] = (char *) path; + link->uprobe_multi.offsets[i] = target->rel_ip; + link->uprobe_multi.ref_ctr_offsets[i] = target->sema_off; + link->uprobe_multi.cookies[i] = spec_id; + } else { + opts.ref_ctr_offset = target->sema_off; + opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0; + uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path, + target->rel_ip, &opts); + err = libbpf_get_error(uprobe_link); + if (err) { + pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n", + i, usdt_provider, usdt_name, path, err); + goto err_out; + } + + link->uprobes[i].link = uprobe_link; + link->uprobes[i].abs_ip = target->abs_ip; + link->uprobe_cnt++; + } + } + + if (uprobe_multi) { + opts_multi.cnt = target_cnt; + opts_multi.paths = (const char **) link->uprobe_multi.paths; + opts_multi.offsets = link->uprobe_multi.offsets; + opts_multi.ref_ctr_offsets = link->uprobe_multi.ref_ctr_offsets; + opts_multi.cookies = link->uprobe_multi.cookies; + + uprobe_link = bpf_program__attach_uprobe_multi_opts(prog, NULL, NULL, &opts_multi); err = libbpf_get_error(uprobe_link); if (err) { - pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n", - i, usdt_provider, usdt_name, path, err); + pr_warn("usdt: failed to attach uprobe multi for '%s:%s' in '%s': %d\n", + usdt_provider, usdt_name, path, err); goto err_out; } - - link->uprobes[i].link = uprobe_link; - link->uprobes[i].abs_ip = target->abs_ip; - link->uprobe_cnt++; + link->uprobe_multi.link = uprobe_link; } free(targets); From patchwork Mon Apr 24 16:04:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222357 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F534C7618E for ; Mon, 24 Apr 2023 16:07:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231992AbjDXQHN (ORCPT ); Mon, 24 Apr 2023 12:07:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232025AbjDXQHK (ORCPT ); Mon, 24 Apr 2023 12:07:10 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF34783D5 for ; Mon, 24 Apr 2023 09:07:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 24032619FC for ; Mon, 24 Apr 2023 16:07:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D523FC433EF; Mon, 24 Apr 2023 16:06:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352422; bh=HYt5JDtZlLnwAK+JXhGXzLxuJvMWlEihTTaEk87/VPg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PBcB//a/AHIeSCPIuFN1ahrNO+Dd37iJd1uhUoLI7NigTp0IX7IMN1hZ+rdT8BfbZ TgoTeaGO4tAqRuS6wdWV1OE9dDBrC87rFjN6vvxzAQ31+KJj7edn4ZqjhIKX3GdZ6L QcUTtKp0CK2D8+Fd6NpipzLpCp0xdSO1zdK/v0sBAJvnIz+fxiq5EV15e0pRW+7KD1 vnc080iOiTQVxDrWna0TpmE1eUGu2WarlMfuPGyRwUZfKyJNoFL7ikC9slbgiVjarx yf/6TFZv08f/LpwOWojb3eb2nGQ1knKMdA48VgREwoejta9LBhkFWzA4eDO6RrkOm9 Jcr+pxrGTjkwg== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 13/20] selftests/bpf: Add uprobe_multi skel test Date: Mon, 24 Apr 2023 18:04:40 +0200 Message-Id: <20230424160447.2005755-14-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding uprobe_multi test for skeleton load/attach functions, to test skeleton auto attach for uprobe_multi link. Test that bpf_get_func_ip works properly for uprobe_multi attachment. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 66 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 63 ++++++++++++++++++ 2 files changed, 129 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c new file mode 100644 index 000000000000..f68cda122ac2 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "uprobe_multi.skel.h" + +noinline void uprobe_multi_func_1(void) +{ + asm volatile (""); +} + +noinline void uprobe_multi_func_2(void) +{ + asm volatile (""); +} + +noinline void uprobe_multi_func_3(void) +{ + asm volatile (""); +} + +static void uprobe_multi_test_run(struct uprobe_multi *skel) +{ + skel->bss->uprobe_multi_func_1_addr = (u64) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (u64) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (u64) uprobe_multi_func_3; + + skel->bss->pid = getpid(); + + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + ASSERT_EQ(skel->bss->uprobe_multi_func_1_result, 1, "uprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uprobe_multi_func_2_result, 1, "uprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uprobe_multi_func_3_result, 1, "uprobe_multi_func_3_result"); + + ASSERT_EQ(skel->bss->uretprobe_multi_func_1_result, 1, "uretprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uretprobe_multi_func_2_result, 1, "uretprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 1, "uretprobe_multi_func_3_result"); +} + +static void test_skel_api(void) +{ + struct uprobe_multi *skel = NULL; + int err; + + skel = uprobe_multi__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load")) + goto cleanup; + + err = uprobe_multi__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi__attach")) + goto cleanup; + + uprobe_multi_test_run(skel); + +cleanup: + uprobe_multi__destroy(skel); +} + +void test_uprobe_multi_test(void) +{ + if (test__start_subtest("skel_api")) + test_skel_api(); +} diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c new file mode 100644 index 000000000000..a5dc00938217 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +__u64 uprobe_multi_func_1_addr = 0; +__u64 uprobe_multi_func_2_addr = 0; +__u64 uprobe_multi_func_3_addr = 0; + +__u64 uprobe_multi_func_1_result = 0; +__u64 uprobe_multi_func_2_result = 0; +__u64 uprobe_multi_func_3_result = 0; + +__u64 uretprobe_multi_func_1_result = 0; +__u64 uretprobe_multi_func_2_result = 0; +__u64 uretprobe_multi_func_3_result = 0; + +int pid = 0; +bool test_cookie = false; + +static void uprobe_multi_check(void *ctx, bool is_return) +{ + if (bpf_get_current_pid_tgid() >> 32 != pid) + return; + + __u64 cookie = test_cookie ? bpf_get_attach_cookie(ctx) : 0; + __u64 addr = bpf_get_func_ip(ctx); + +#define SET(__var, __addr, __cookie) ({ \ + if (addr == __addr && \ + (!test_cookie || (cookie == __cookie))) \ + __var = 1; \ +}) + + if (is_return) { + SET(uretprobe_multi_func_1_result, uprobe_multi_func_1_addr, 2); + SET(uretprobe_multi_func_2_result, uprobe_multi_func_2_addr, 3); + SET(uretprobe_multi_func_3_result, uprobe_multi_func_3_addr, 1); + } else { + SET(uprobe_multi_func_1_result, uprobe_multi_func_1_addr, 3); + SET(uprobe_multi_func_2_result, uprobe_multi_func_2_addr, 1); + SET(uprobe_multi_func_3_result, uprobe_multi_func_3_addr, 2); + } + +#undef SET +} + +SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*") +int test_uprobe(struct pt_regs *ctx) +{ + uprobe_multi_check(ctx, false); + return 0; +} + +SEC("uretprobe.multi//proc/self/exe:uprobe_multi_func_*") +int test_uretprobe(struct pt_regs *ctx) +{ + uprobe_multi_check(ctx, true); + return 0; +} From patchwork Mon Apr 24 16:04:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222358 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 994F0C77B61 for ; Mon, 24 Apr 2023 16:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232032AbjDXQHT (ORCPT ); Mon, 24 Apr 2023 12:07:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232100AbjDXQHS (ORCPT ); Mon, 24 Apr 2023 12:07:18 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 778EC4C2E for ; Mon, 24 Apr 2023 09:07:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1321061C12 for ; Mon, 24 Apr 2023 16:07:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBD64C4339E; Mon, 24 Apr 2023 16:07:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352432; bh=Yua+mSx3jhG2H/ndNz/mXoRUf/3XrDIomHGmvzz0K44=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nV1oU4aeCBaifn7uNKNTvXgQgWSHzfMyl51eBBWt9A960DBKV+sulc8CrAZMyjDkG 2M9NGrqCI9kdLYMAjfLTyHGWHi4BHmBP3HZ+6VAoWJf+Jc4xuDsVzqByhHrny1v2ug Mn9C/h01Dw+5MLz+OCnrcvcdptbtD5SV8uvpzAYpiSeBdXh5uhRrF4mTavH9noOKbE 7kNQ/qTAxDJZD+1gYud4SGivN6a8pefwOhmltQ6/sEe2aKcE0rBqwekMMnZ5qj+wRH Yw8jOHqKvlgyowF7F+/WKgNg3pZUV8Cu6uAR3H6PqfFdJ5NtsQ64IuBErGRlXOVL3I reoQWezGiAb0A== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 14/20] selftests/bpf: Add uprobe_multi api test Date: Mon, 24 Apr 2023 18:04:41 +0200 Message-Id: <20230424160447.2005755-15-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding uprobe_multi test for bpf_program__attach_uprobe_multi_opts attach function. Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index f68cda122ac2..179b78a4b711 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -59,8 +59,63 @@ static void test_skel_api(void) uprobe_multi__destroy(skel); } +static void +test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts) +{ + struct bpf_link *link1 = NULL, *link2 = NULL; + struct uprobe_multi *skel = NULL; + + skel = uprobe_multi__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi")) + goto cleanup; + + link1 = bpf_program__attach_uprobe_multi_opts(skel->progs.test_uprobe, + binary, pattern, opts); + if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi_opts")) + goto cleanup; + + opts->retprobe = true; + link2 = bpf_program__attach_uprobe_multi_opts(skel->progs.test_uretprobe, + binary, pattern, opts); + if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_opts_retprobe")) + goto cleanup; + + uprobe_multi_test_run(skel); + +cleanup: + bpf_link__destroy(link2); + bpf_link__destroy(link1); + uprobe_multi__destroy(skel); +} + +static void test_attach_api_pattern(void) +{ + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts); + + test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts); + test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts); +} + +static void test_attach_api_syms(void) +{ + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts); + const char *syms[3] = { + "uprobe_multi_func_1", + "uprobe_multi_func_2", + "uprobe_multi_func_3", + }; + + opts.syms = syms; + opts.cnt = ARRAY_SIZE(syms); + test_attach_api("/proc/self/exe", NULL, &opts); +} + void test_uprobe_multi_test(void) { if (test__start_subtest("skel_api")) test_skel_api(); + if (test__start_subtest("attach_api_pattern")) + test_attach_api_pattern(); + if (test__start_subtest("attach_api_syms")) + test_attach_api_syms(); } From patchwork Mon Apr 24 16:04:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222359 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E95DC77B61 for ; Mon, 24 Apr 2023 16:07:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232065AbjDXQH0 (ORCPT ); Mon, 24 Apr 2023 12:07:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232100AbjDXQHY (ORCPT ); Mon, 24 Apr 2023 12:07:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4ED0C6E8A for ; Mon, 24 Apr 2023 09:07:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DDDF261C12 for ; Mon, 24 Apr 2023 16:07:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99994C433EF; Mon, 24 Apr 2023 16:07:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352442; bh=bLHKTOni5LYedtoiKJFxrtlK2CwKBVTN1otv5L69Bi0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JnpPV0aXuY4buFBBKoUvHIk2WkyOgdpYTb2Fe0xEqA6+UGAUQEkto9hdygu2xdceE yBYIDcaSg1xJjWS8be6yGhpunCW1hfsqc8WAWX2nF35dZ37a9RC4XAosh9bAFho8/T EI8DrzQaFxc5N9QqOCJCDjHh+5nIXD5tzs061edCMqf6czWcmX0y0cbsHo0AmvUWIg /OnmAITygiVjiuwbLBSTGt8Yd3gY5U1+zPjRWUAFQC08zLTKE5lr9WHFgT59XpGGHl aYjrWh0AHDl0U/CrCdGBXg3rrR8njoRlGlLXuKrJsPGF70QY7jlNQn77K4ZNKebxLT QoEFSL7N9J0ew== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 15/20] selftests/bpf: Add uprobe_multi link test Date: Mon, 24 Apr 2023 18:04:42 +0200 Message-Id: <20230424160447.2005755-16-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding uprobe_multi test for bpf_link_create attach function. Testing attachment using the struct bpf_link_create_opts. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 179b78a4b711..abe620d844e7 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -110,6 +110,59 @@ static void test_attach_api_syms(void) test_attach_api("/proc/self/exe", NULL, &opts); } +void test_link_api(void) +{ + int prog_fd, link1_fd = -1, link2_fd = -1; + LIBBPF_OPTS(bpf_link_create_opts, opts); + struct uprobe_multi *skel = NULL; + unsigned long *offsets = NULL; + const char *syms[3] = { + "uprobe_multi_func_1", + "uprobe_multi_func_2", + "uprobe_multi_func_3", + }; + const char *paths[3] = { + "/proc/self/exe", + "/proc/self/exe", + "/proc/self/exe", + }; + int err; + + err = elf_find_multi_func_offset(paths[0], 3, syms, (unsigned long **) &offsets); + if (!ASSERT_OK(err, "elf_find_multi_func_offset")) + return; + + opts.uprobe_multi.paths = paths; + opts.uprobe_multi.offsets = offsets; + opts.uprobe_multi.cnt = ARRAY_SIZE(syms); + + skel = uprobe_multi__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test_uprobe); + link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts); + if (!ASSERT_GE(link1_fd, 0, "link_fd")) + goto cleanup; + + opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN; + prog_fd = bpf_program__fd(skel->progs.test_uretprobe); + link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts); + if (!ASSERT_GE(link2_fd, 0, "link_fd")) + goto cleanup; + + uprobe_multi_test_run(skel); + +cleanup: + if (link1_fd != -1) + close(link1_fd); + if (link2_fd != -1) + close(link2_fd); + + uprobe_multi__destroy(skel); + free(offsets); +} + void test_uprobe_multi_test(void) { if (test__start_subtest("skel_api")) @@ -118,4 +171,6 @@ void test_uprobe_multi_test(void) test_attach_api_pattern(); if (test__start_subtest("attach_api_syms")) test_attach_api_syms(); + if (test__start_subtest("link_api")) + test_link_api(); } From patchwork Mon Apr 24 16:04:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222360 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B7FDC7618E for ; Mon, 24 Apr 2023 16:07:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232025AbjDXQHf (ORCPT ); Mon, 24 Apr 2023 12:07:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232009AbjDXQHe (ORCPT ); Mon, 24 Apr 2023 12:07:34 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 730106587 for ; Mon, 24 Apr 2023 09:07:33 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0F4E061B3C for ; Mon, 24 Apr 2023 16:07:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AAA04C433EF; Mon, 24 Apr 2023 16:07:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352452; bh=EeLuFGG2jFrSF+HdeDudYfEksqqtxO1sVe6LUcFytho=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aoBbDi6LQWiVo3oJCsLCIe08s9odxj3Mk0ZlM3g1EA5fUiz0lFXFrXc0T9hGQvxAB K6Ynuw7EOOwa8tWe1ap1Wr0lYnucFciC+o+dTUJghkCMTMb72DcP5LErz/nvdp0pzr gMnoMOQyVNBoY2AD97ST0HsZFAEfGE6lf1GPRbCfUiPdrzQ6lVh5YydIFaJtHB3NgF 1q5I1fyi4GmnkpYsFeFmuaAs4ggiNQv2Gf4FcdET28N9XHkJXurqmRhlA/EwKqJfIf T1TzdXJsrEM5Pe3UUBPpSI8pHJsfMswgVecJZcO7cyQnSGVi72Kmw/gyuyjzUhReeG bfIZtzXM73E2w== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 16/20] selftests/bpf: Add uprobe_multi test program Date: Mon, 24 Apr 2023 18:04:43 +0200 Message-Id: <20230424160447.2005755-17-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/Makefile | 5 ++ tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index c49e5403ad0e..506d92d5505a 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/liburandom_read.so \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ + $(OUTPUT)/uprobe_multi \ ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \ @@ -671,6 +672,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ +$(OUTPUT)/uprobe_multi: uprobe_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \ diff --git a/tools/testing/selftests/bpf/uprobe_multi.c b/tools/testing/selftests/bpf/uprobe_multi.c new file mode 100644 index 000000000000..115a7f6cebfa --- /dev/null +++ b/tools/testing/selftests/bpf/uprobe_multi.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +#define __PASTE(a, b) a##b +#define PASTE(a, b) __PASTE(a, b) + +#define NAME(name, idx) PASTE(name, idx) + +#define DEF(name, idx) int NAME(name, idx)(void) { return 0; } +#define CALL(name, idx) NAME(name, idx)(); + +#define F(body, name, idx) body(name, idx) + +#define F10(body, name, idx) \ + F(body, PASTE(name, idx), 0) F(body, PASTE(name, idx), 1) F(body, PASTE(name, idx), 2) \ + F(body, PASTE(name, idx), 3) F(body, PASTE(name, idx), 4) F(body, PASTE(name, idx), 5) \ + F(body, PASTE(name, idx), 6) F(body, PASTE(name, idx), 7) F(body, PASTE(name, idx), 8) \ + F(body, PASTE(name, idx), 9) + +#define F100(body, name, idx) \ + F10(body, PASTE(name, idx), 0) F10(body, PASTE(name, idx), 1) F10(body, PASTE(name, idx), 2) \ + F10(body, PASTE(name, idx), 3) F10(body, PASTE(name, idx), 4) F10(body, PASTE(name, idx), 5) \ + F10(body, PASTE(name, idx), 6) F10(body, PASTE(name, idx), 7) F10(body, PASTE(name, idx), 8) \ + F10(body, PASTE(name, idx), 9) + +#define F1000(body, name, idx) \ + F100(body, PASTE(name, idx), 0) F100(body, PASTE(name, idx), 1) F100(body, PASTE(name, idx), 2) \ + F100(body, PASTE(name, idx), 3) F100(body, PASTE(name, idx), 4) F100(body, PASTE(name, idx), 5) \ + F100(body, PASTE(name, idx), 6) F100(body, PASTE(name, idx), 7) F100(body, PASTE(name, idx), 8) \ + F100(body, PASTE(name, idx), 9) + +#define F10000(body, name, idx) \ + F1000(body, PASTE(name, idx), 0) F1000(body, PASTE(name, idx), 1) F1000(body, PASTE(name, idx), 2) \ + F1000(body, PASTE(name, idx), 3) F1000(body, PASTE(name, idx), 4) F1000(body, PASTE(name, idx), 5) \ + F1000(body, PASTE(name, idx), 6) F1000(body, PASTE(name, idx), 7) F1000(body, PASTE(name, idx), 8) \ + F1000(body, PASTE(name, idx), 9) + +F10000(DEF, uprobe_multi_func_, 0) +F10000(DEF, uprobe_multi_func_, 1) +F10000(DEF, uprobe_multi_func_, 2) +F10000(DEF, uprobe_multi_func_, 3) +F10000(DEF, uprobe_multi_func_, 4) + +int main(void) +{ + F10000(CALL, uprobe_multi_func_, 0) + F10000(CALL, uprobe_multi_func_, 1) + F10000(CALL, uprobe_multi_func_, 2) + F10000(CALL, uprobe_multi_func_, 3) + F10000(CALL, uprobe_multi_func_, 4) + return 0; +} From patchwork Mon Apr 24 16:04:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222361 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DD3CC77B61 for ; Mon, 24 Apr 2023 16:07:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232060AbjDXQHp (ORCPT ); Mon, 24 Apr 2023 12:07:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232009AbjDXQHo (ORCPT ); Mon, 24 Apr 2023 12:07:44 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60B276E8A for ; Mon, 24 Apr 2023 09:07:43 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E962361BD1 for ; Mon, 24 Apr 2023 16:07:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9C5A1C433D2; Mon, 24 Apr 2023 16:07:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352462; bh=cVv/wzPlAy0jrt61MxpQKs4w2jYP0P8rHhJTbXwVcBE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b80RBupCThnNFnjQT34Y5Ih2BQA8cMKpqqPx0oLPSMvbjY1S/hsmlYho5AaG46BmW rZXN1XAMnWS3jUs6rfNz/2Z/2dfBqsheAmZ9EgXGx1N8YG74Ep/twtOHdv9pMxTK5f RmaNgml9e3XxxiyEQ+9iJ8b/Ea/NLwdEWudm9gRqFH97DBgb04sdCYZPfYa744lZQK +11nKSbEHPX/HOOHavpdfTMymkbXQZ1utvh23jeWK8clL2zxBZBVrmvi7h+apILFV/ yeMoAgABLBnY/fts9HveuleVHBtRwyLPWVpgtYG2EczUoafBlGanm6VF4PCflWD38o U2sgLaFb3g6Yg== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 17/20] selftests/bpf: Add uprobe_multi bench test Date: Mon, 24 Apr 2023 18:04:44 +0200 Message-Id: <20230424160447.2005755-18-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding test that attaches 50k uprobes in uprobe_multi binary. After the attach is done we run the binary and make sure we get proper amount of hits. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 56 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 9 +++ 2 files changed, 65 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index abe620d844e7..9a89c61359a1 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -163,6 +163,60 @@ void test_link_api(void) free(offsets); } +static inline __u64 get_time_ns(void) +{ + struct timespec t; + + clock_gettime(CLOCK_MONOTONIC, &t); + return (__u64) t.tv_sec * 1000000000 + t.tv_nsec; +} + +void test_bench_attach_uprobe(void) +{ + long attach_start_ns, attach_end_ns; + long detach_start_ns, detach_end_ns; + double attach_delta, detach_delta; + struct uprobe_multi *skel = NULL; + struct bpf_program *prog; + int err; + + skel = uprobe_multi__open(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) + goto cleanup; + + bpf_object__for_each_program(prog, skel->obj) + bpf_program__set_autoload(prog, false); + + bpf_program__set_autoload(skel->progs.test_uprobe_bench, true); + + err = uprobe_multi__load(skel); + if (!ASSERT_EQ(err, 0, "strncmp_test load")) + goto cleanup; + + attach_start_ns = get_time_ns(); + + err = uprobe_multi__attach(skel); + if (!ASSERT_OK(err, "uprobe_multi__attach")) + goto cleanup; + + attach_end_ns = get_time_ns(); + + system("./uprobe_multi"); + + ASSERT_EQ(skel->bss->count, 50000, "uprobes_count"); + +cleanup: + detach_start_ns = get_time_ns(); + uprobe_multi__destroy(skel); + detach_end_ns = get_time_ns(); + + attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; + detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0; + + printf("%s: attached in %7.3lfs\n", __func__, attach_delta); + printf("%s: detached in %7.3lfs\n", __func__, detach_delta); +} + void test_uprobe_multi_test(void) { if (test__start_subtest("skel_api")) @@ -173,4 +227,6 @@ void test_uprobe_multi_test(void) test_attach_api_syms(); if (test__start_subtest("link_api")) test_link_api(); + if (test__start_subtest("bench_uprobe")) + test_bench_attach_uprobe(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c index a5dc00938217..126abd5aef89 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_multi.c +++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c @@ -61,3 +61,12 @@ int test_uretprobe(struct pt_regs *ctx) uprobe_multi_check(ctx, true); return 0; } + +int count; + +SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*") +int test_uprobe_bench(struct pt_regs *ctx) +{ + count++; + return 0; +} From patchwork Mon Apr 24 16:04:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222362 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A328C7618E for ; Mon, 24 Apr 2023 16:07:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232005AbjDXQHz (ORCPT ); Mon, 24 Apr 2023 12:07:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232009AbjDXQHy (ORCPT ); Mon, 24 Apr 2023 12:07:54 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 695E86587 for ; Mon, 24 Apr 2023 09:07:53 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 04B0560C7A for ; Mon, 24 Apr 2023 16:07:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFA6BC433D2; Mon, 24 Apr 2023 16:07:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352472; bh=KqYyrDf/DavyQZwTLj2tISjJdWnbVRwGS0L6EVvEp3Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G/pcvEudlwNQQpnkwtrt1y0TXPJEafWeRaQ8zKezzBWiaVXxXuRKrC1zzdezTxWsx 93wlnNYcabPEh34vNCGP62P+nVNFjCAuicEyL3V9tQEARPCqYSkPCyJXepJdX2iw7d I2mtq7S3KFecQvC/FnvghqwsLcnDgy7cFzFdZ7Lkt2Fq0u1766lLigYrMOSrY5CG2T CNvqzkCJ35G9Guz79xyTuAjx73bSo2FNMdFCZMGL3qUEdPKP2mvvCQ7Bkco/laMTmK du4Ezb/1k2X1SlcnIZdkurly3OBHckzIU6K+23TZdxinGIiF+42V51tx8QWFvxgElp JnZqCHEPvxQDw== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 18/20] selftests/bpf: Add usdt_multi test program Date: Mon, 24 Apr 2023 18:04:45 +0200 Message-Id: <20230424160447.2005755-19-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding usdt_multi test program that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/Makefile | 5 +++++ tools/testing/selftests/bpf/usdt_multi.c | 23 +++++++++++++++++++++++ 2 files changed, 28 insertions(+) create mode 100644 tools/testing/selftests/bpf/usdt_multi.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 506d92d5505a..33278c21ed9d 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -569,6 +569,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ $(OUTPUT)/uprobe_multi \ + $(OUTPUT)/usdt_multi \ ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \ @@ -676,6 +677,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ +$(OUTPUT)/usdt_multi: usdt_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \ diff --git a/tools/testing/selftests/bpf/usdt_multi.c b/tools/testing/selftests/bpf/usdt_multi.c new file mode 100644 index 000000000000..3216d15dcd76 --- /dev/null +++ b/tools/testing/selftests/bpf/usdt_multi.c @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +#define PROBE STAP_PROBE(test, usdt); + +#define PROBE10 PROBE PROBE PROBE PROBE PROBE PROBE PROBE PROBE PROBE PROBE +#define PROBE100 PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \ + PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 +#define PROBE1000 PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \ + PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 +#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \ + PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 + +int main(void) +{ + PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 + return 0; +} From patchwork Mon Apr 24 16:04:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222363 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97DDDC77B61 for ; Mon, 24 Apr 2023 16:08:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232090AbjDXQIG (ORCPT ); Mon, 24 Apr 2023 12:08:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232009AbjDXQIE (ORCPT ); Mon, 24 Apr 2023 12:08:04 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E90E6A6A for ; Mon, 24 Apr 2023 09:08:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F06C861B3C for ; Mon, 24 Apr 2023 16:08:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9C3C3C433EF; Mon, 24 Apr 2023 16:07:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352482; bh=jeRBYAPrZoVhm4oe2dQRln7UT1N8OBRKJkmPMI4mX/o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PHsU8zOZg3IiiFYPz07xsxiGthg6AqqV0UW8cOra6oO6iGM9pH8AhNMvEZ6mq94QC oo0rKBkRpOuHhJhm9hOFhGomxyNEvHnbYBTOxrnMVz71AiI4TrSrzKcBwabrxEF1zO KNdGTUpvZmml2lP1k7lDJQrAz/d2zgWQRcyQsi+RlJehHLoWCBQe6oMtB1MJzOA0C2 YJUEit5ttfi6Lu56z+ZeUArnwnuVcqJSh950FcVJuzBXV3QuobEKkXdMr0lVNlwZHk J1vgeq8tvLkD78QtR6T+DBAmEr+TAVpEqCLN3USGA/iOGzciQsiOv25JlclXe83ms4 MrBj5d1AO6nvQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 19/20] selftests/bpf: Add usdt_multi bench test Date: Mon, 24 Apr 2023 18:04:46 +0200 Message-Id: <20230424160447.2005755-20-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding test that attaches 50k usdt probes in usdt_multi binary. After the attach is done we run the binary and make sure we get proper amount of hits. With current uprobes: # perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -n 254/6': 1353.659680562 seconds time elapsed With uprobe_multi link: # perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Performance counter stats for './test_progs -n 254/6': 0.322046364 seconds time elapsed Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/uprobe_multi_test.c | 54 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++++ 2 files changed, 70 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 9a89c61359a1..ff08d4b8dbfe 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -3,6 +3,7 @@ #include #include #include "uprobe_multi.skel.h" +#include "uprobe_multi_usdt.skel.h" noinline void uprobe_multi_func_1(void) { @@ -217,6 +218,57 @@ void test_bench_attach_uprobe(void) printf("%s: detached in %7.3lfs\n", __func__, detach_delta); } +void test_bench_attach_usdt(void) +{ + struct uprobe_multi_usdt *skel = NULL; + long attach_start_ns, attach_end_ns; + long detach_start_ns, detach_end_ns; + double attach_delta, detach_delta; + LIBBPF_OPTS(bpf_usdt_opts, opts, + .uprobe_multi = true, + ); + struct bpf_program *prog; + int err; + + skel = uprobe_multi_usdt__open(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) + goto cleanup; + + bpf_object__for_each_program(prog, skel->obj) + bpf_program__set_autoload(prog, false); + + bpf_program__set_autoload(skel->progs.usdt0, true); + bpf_program__set_expected_attach_type(skel->progs.usdt0, BPF_TRACE_UPROBE_MULTI); + + err = uprobe_multi_usdt__load(skel); + if (!ASSERT_EQ(err, 0, "strncmp_test load")) + goto cleanup; + + attach_start_ns = get_time_ns(); + + skel->links.usdt0 = bpf_program__attach_usdt(skel->progs.usdt0, -1, "./usdt_multi", + "test", "usdt", &opts); + if (!ASSERT_OK_PTR(skel->links.usdt0, "usdt_link")) + goto cleanup; + + attach_end_ns = get_time_ns(); + + system("./usdt_multi"); + + ASSERT_EQ(skel->bss->count, 50000, "usdt_count"); + +cleanup: + detach_start_ns = get_time_ns(); + uprobe_multi_usdt__destroy(skel); + detach_end_ns = get_time_ns(); + + attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; + detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0; + + printf("%s: attached in %7.3lfs\n", __func__, attach_delta); + printf("%s: detached in %7.3lfs\n", __func__, detach_delta); +} + void test_uprobe_multi_test(void) { if (test__start_subtest("skel_api")) @@ -229,4 +281,6 @@ void test_uprobe_multi_test(void) test_link_api(); if (test__start_subtest("bench_uprobe")) test_bench_attach_uprobe(); + if (test__start_subtest("bench_usdt")) + test_bench_attach_usdt(); } diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c b/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c new file mode 100644 index 000000000000..9e1c33d0bd2f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +int count; + +SEC("usdt") +int usdt0(struct pt_regs *ctx) +{ + count++; + return 0; +} From patchwork Mon Apr 24 16:04:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 13222364 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60727C77B73 for ; Mon, 24 Apr 2023 16:08:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232102AbjDXQIO (ORCPT ); Mon, 24 Apr 2023 12:08:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232100AbjDXQIO (ORCPT ); Mon, 24 Apr 2023 12:08:14 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5214A6A6B for ; Mon, 24 Apr 2023 09:08:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E38E660C7A for ; Mon, 24 Apr 2023 16:08:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 899CBC4339E; Mon, 24 Apr 2023 16:08:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1682352492; bh=DOqitYxEBJmzYF5oGx5UAJL4LhJhfOlSHLkMOOk4GB4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VMPUPF7lEYpEpf2gULsB+Rc8LUiC+s6DgOcwe6N7C+rLgYu6zRKPOpJ93DLkUU1h3 QJGEzSMZKFWKqSwfV2ISm7oqUgIxWSYbfGVcsG7X/52IYgVLaFhyCUKhgkQmsE7vRK 2OotoGapHUyeglLsnt6wMfSJ5oV2LsJ0NZC4CMZbY6XkBkXIKvi6MH+pTtOe0oz2+6 SqzYd6p/rKilaUMdhn6j7YNh1R0iT80TKsHKJMcV3W4S/Mk1DpxS2nTGpLzWlgTTL5 OU087Fip6z+3Kf31DhvRegJFErZnFHfUkXnzdYOJjoQVN66j8UM5pdvRW8FnSZWYSp t+rP2DH2sgeTg== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 20/20] selftests/bpf: Add uprobe_multi cookie test Date: Mon, 24 Apr 2023 18:04:47 +0200 Message-Id: <20230424160447.2005755-21-jolsa@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230424160447.2005755-1-jolsa@kernel.org> References: <20230424160447.2005755-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Adding test for cookies setup/retrieval in uprobe_link uprobes and making sure bpf_get_attach_cookie works properly. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/bpf_cookie.c | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c index 26b2d1bffdfd..0e2d9ebe9258 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c @@ -11,6 +11,7 @@ #include #include "test_bpf_cookie.skel.h" #include "kprobe_multi.skel.h" +#include "uprobe_multi.skel.h" /* uprobe attach point */ static noinline void trigger_func(void) @@ -239,6 +240,81 @@ static void kprobe_multi_attach_api_subtest(void) bpf_link__destroy(link1); kprobe_multi__destroy(skel); } + +/* defined in prog_tests/uprobe_multi_test.c */ +void uprobe_multi_func_1(void); +void uprobe_multi_func_2(void); +void uprobe_multi_func_3(void); + +static void uprobe_multi_test_run(struct uprobe_multi *skel) +{ + skel->bss->uprobe_multi_func_1_addr = (u64) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (u64) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (u64) uprobe_multi_func_3; + + skel->bss->pid = getpid(); + skel->bss->test_cookie = true; + + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + + ASSERT_EQ(skel->bss->uprobe_multi_func_1_result, 1, "uprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uprobe_multi_func_2_result, 1, "uprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uprobe_multi_func_3_result, 1, "uprobe_multi_func_3_result"); + + ASSERT_EQ(skel->bss->uretprobe_multi_func_1_result, 1, "uretprobe_multi_func_1_result"); + ASSERT_EQ(skel->bss->uretprobe_multi_func_2_result, 1, "uretprobe_multi_func_2_result"); + ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 1, "uretprobe_multi_func_3_result"); +} + +static void uprobe_multi_attach_api_subtest(void) +{ + struct bpf_link *link1 = NULL, *link2 = NULL; + struct uprobe_multi *skel = NULL; + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts); + const char *syms[3] = { + "uprobe_multi_func_1", + "uprobe_multi_func_2", + "uprobe_multi_func_3", + }; + __u64 cookies[3]; + + cookies[0] = 3; /* uprobe_multi_func_1 */ + cookies[1] = 1; /* uprobe_multi_func_2 */ + cookies[2] = 2; /* uprobe_multi_func_3 */ + + opts.syms = syms; + opts.cnt = ARRAY_SIZE(syms); + opts.cookies = &cookies[0]; + + skel = uprobe_multi__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_multi")) + goto cleanup; + + link1 = bpf_program__attach_uprobe_multi_opts(skel->progs.test_uprobe, + "/proc/self/exe", NULL, &opts); + if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi_opts")) + goto cleanup; + + cookies[0] = 2; /* uprobe_multi_func_1 */ + cookies[1] = 3; /* uprobe_multi_func_2 */ + cookies[2] = 1; /* uprobe_multi_func_3 */ + + opts.retprobe = true; + link2 = bpf_program__attach_uprobe_multi_opts(skel->progs.test_uretprobe, + "/proc/self/exe", NULL, &opts); + if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_opts_retprobe")) + goto cleanup; + + uprobe_multi_test_run(skel); + +cleanup: + bpf_link__destroy(link2); + bpf_link__destroy(link1); + uprobe_multi__destroy(skel); +} + static void uprobe_subtest(struct test_bpf_cookie *skel) { DECLARE_LIBBPF_OPTS(bpf_uprobe_opts, opts); @@ -515,6 +591,8 @@ void test_bpf_cookie(void) kprobe_multi_attach_api_subtest(); if (test__start_subtest("uprobe")) uprobe_subtest(skel); + if (test__start_subtest("multi_uprobe_attach_api")) + uprobe_multi_attach_api_subtest(); if (test__start_subtest("tracepoint")) tp_subtest(skel); if (test__start_subtest("perf_event"))