From patchwork Mon Apr 18 12:48:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12816579 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91671C433F5 for ; Mon, 18 Apr 2022 13:09:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241310AbiDRNLy (ORCPT ); Mon, 18 Apr 2022 09:11:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242700AbiDRNJU (ORCPT ); Mon, 18 Apr 2022 09:09:20 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 683202CC84; Mon, 18 Apr 2022 05:48:52 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 61FBB61257; Mon, 18 Apr 2022 12:48:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3DF1BC385A7; Mon, 18 Apr 2022 12:48:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650286128; bh=OETyOZf1v9T/pfPCssjXE8D9IRePA6pQEc/6z6FIKLI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UJeUqePvZsT4JyX4gmvU+xloiRRxrwfEpWSeSrwznw7ga/B8nNxaCVfSLmdNFnUpM 8JyvGJVYkzl6SKsHBgFS61roqedQarqWuNFovWUxUYtxGba9MbXAymGU+g28Smpm1b W0iql5q6poRioRSg5Rv8Mjvp9TuSTxJtSiOF4zZbJ/JLfG7vEN9nV0JVyQ/kGiYSxC H9bBsQlCqckJ8YzUK45oOHGtC0opGm2/rbtI9SBWvnBGWn4csATKrKyvhUk6I1e1xb ZnJ+64JW0r+WC4R0gaTKfxoKXdQGkxxTtwyDDnfm6Q3DUKvPAFwm79sSxcSN29ymzP 6zTNH7tNHtpPQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh Subject: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function Date: Mon, 18 Apr 2022 14:48:31 +0200 Message-Id: <20220418124834.829064-2-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220418124834.829064-1-jolsa@kernel.org> References: <20220418124834.829064-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding kallsyms_lookup_names function that resolves array of symbols with single pass over kallsyms. The user provides array of string pointers with count and pointer to allocated array for resolved values. int kallsyms_lookup_names(const char **syms, size_t cnt, unsigned long *addrs) It iterates all kalsyms symbols and tries to loop up each in provided symbols array with bsearch. The symbols array needs to be sorted by name for this reason. We also check each symbol to pass ftrace_location, because this API will be used for fprobe symbols resolving. This can be optional in future if there's a need. We need kallsyms_on_each_symbol function, so enabling it and also the new function for CONFIG_FPROBE option. Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- include/linux/kallsyms.h | 6 ++++ kernel/kallsyms.c | 70 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index ce1bd2fbf23e..7c82fa7445d4 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -72,6 +72,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, #ifdef CONFIG_KALLSYMS /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name(const char *name); +int kallsyms_lookup_names(const char **syms, size_t cnt, unsigned long *addrs); extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize, @@ -103,6 +104,11 @@ static inline unsigned long kallsyms_lookup_name(const char *name) return 0; } +static inline int kallsyms_lookup_names(const char **syms, size_t cnt, unsigned long *addrs) +{ + return -ERANGE; +} + static inline int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize, unsigned long *offset) diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 79f2eb617a62..ef940b25f3fc 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -29,6 +29,7 @@ #include #include #include +#include /* * These will be re-linked against their real values @@ -228,7 +229,7 @@ unsigned long kallsyms_lookup_name(const char *name) return module_kallsyms_lookup_name(name); } -#ifdef CONFIG_LIVEPATCH +#if defined(CONFIG_LIVEPATCH) || defined(CONFIG_FPROBE) /* * Iterate over all symbols in vmlinux. For symbols from modules use * module_kallsyms_on_each_symbol instead. @@ -572,6 +573,73 @@ int sprint_backtrace_build_id(char *buffer, unsigned long address) return __sprint_symbol(buffer, address, -1, 1, 1); } +#ifdef CONFIG_FPROBE +static int symbols_cmp(const void *a, const void *b) +{ + const char **str_a = (const char **) a; + const char **str_b = (const char **) b; + + return strcmp(*str_a, *str_b); +} + +struct kallsyms_data { + unsigned long *addrs; + const char **syms; + size_t cnt; + size_t found; +}; + +static int kallsyms_callback(void *data, const char *name, + struct module *mod, unsigned long addr) +{ + struct kallsyms_data *args = data; + + if (!bsearch(&name, args->syms, args->cnt, sizeof(*args->syms), symbols_cmp)) + return 0; + + addr = ftrace_location(addr); + if (!addr) + return 0; + + args->addrs[args->found++] = addr; + return args->found == args->cnt ? 1 : 0; +} + +/** + * kallsyms_lookup_names - Lookup addresses for array of symbols + * + * @syms: array of symbols pointers symbols to resolve, must be + * alphabetically sorted + * @cnt: number of symbols/addresses in @syms/@addrs arrays + * @addrs: array for storing resulting addresses + * + * This function looks up addresses for array of symbols provided in + * @syms array (must be alphabetically sorted) and stores them in + * @addrs array, which needs to be big enough to store at least @cnt + * addresses. + * + * This function returns 0 if all provided symbols are found, + * -ESRCH otherwise. + */ +int kallsyms_lookup_names(const char **syms, size_t cnt, unsigned long *addrs) +{ + struct kallsyms_data args; + + args.addrs = addrs; + args.syms = syms; + args.cnt = cnt; + args.found = 0; + kallsyms_on_each_symbol(kallsyms_callback, &args); + + return args.found == args.cnt ? 0 : -ESRCH; +} +#else +int kallsyms_lookup_names(const char **syms, size_t cnt, unsigned long *addrs) +{ + return -ERANGE; +} +#endif /* CONFIG_FPROBE */ + /* To avoid using get_symbol_offset for every symbol, we carry prefix along. */ struct kallsym_iter { loff_t pos; From patchwork Mon Apr 18 12:48:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12816578 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93B13C433F5 for ; Mon, 18 Apr 2022 13:09:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241320AbiDRNLx (ORCPT ); Mon, 18 Apr 2022 09:11:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242816AbiDRNJ3 (ORCPT ); Mon, 18 Apr 2022 09:09:29 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9512369C6; Mon, 18 Apr 2022 05:49:01 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C2E87B80EDB; Mon, 18 Apr 2022 12:48:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 005B7C385A1; Mon, 18 Apr 2022 12:48:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650286138; bh=l5dooTjbgCNig025z/CpTFdhYCb1guxzIXi8auXSJ1I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hGZlpXTtzMxkqwRnpBxWwuPLvnGxRVMlX30XmdfK4s5DDoy4MeBWTXsxtuBxaoEWH mJSpRi1BCInVFtuCO9qofR4+zv4imTIvfHyi0yYgKojUwiPYXB9GD7Di/zCJBDJRyK vN3oMA1gHMjtL0zuyhh3n2NGJcSWc0TQ98zBUaLSE4cr63h/rMPBYXd5JrYbhNCO5C GE9LcDw5GUXzaJPtmvlTk664jj7fO4+X3gVKajpRXZTUAIzhI4w5OPR+87Q96NwZuK DpKvjLvmQFiGdc+YtZRrz+gv5LzHcZY6QPa2bF0E7rboV0zTxmcmpKtbOzQzjEntjn HW8Y2h9Kng27A== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh Subject: [PATCHv2 bpf-next 2/4] fprobe: Resolve symbols with kallsyms_lookup_names Date: Mon, 18 Apr 2022 14:48:32 +0200 Message-Id: <20220418124834.829064-3-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220418124834.829064-1-jolsa@kernel.org> References: <20220418124834.829064-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Using kallsyms_lookup_names to speed up symbols lookup in register_fprobe_syms API. This requires syms array to be alphabetically sorted. Signed-off-by: Jiri Olsa --- kernel/trace/fprobe.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index 89d9f994ebb0..6419501a0036 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -85,39 +85,31 @@ static void fprobe_exit_handler(struct rethook_node *rh, void *data, } NOKPROBE_SYMBOL(fprobe_exit_handler); +static int symbols_cmp(const void *a, const void *b) +{ + const char **str_a = (const char **) a; + const char **str_b = (const char **) b; + + return strcmp(*str_a, *str_b); +} + /* Convert ftrace location address from symbols */ static unsigned long *get_ftrace_locations(const char **syms, int num) { - unsigned long addr, size; unsigned long *addrs; - int i; /* Convert symbols to symbol address */ addrs = kcalloc(num, sizeof(*addrs), GFP_KERNEL); if (!addrs) return ERR_PTR(-ENOMEM); - for (i = 0; i < num; i++) { - addr = kallsyms_lookup_name(syms[i]); - if (!addr) /* Maybe wrong symbol */ - goto error; - - /* Convert symbol address to ftrace location. */ - if (!kallsyms_lookup_size_offset(addr, &size, NULL) || !size) - goto error; + /* kallsyms_lookup_names expects sorted symbols */ + sort(syms, num, sizeof(*syms), symbols_cmp, NULL); - addr = ftrace_location_range(addr, addr + size - 1); - if (!addr) /* No dynamic ftrace there. */ - goto error; + if (!kallsyms_lookup_names(syms, num, addrs)) + return addrs; - addrs[i] = addr; - } - - return addrs; - -error: kfree(addrs); - return ERR_PTR(-ENOENT); } From patchwork Mon Apr 18 12:48:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12816580 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF28CC433EF for ; Mon, 18 Apr 2022 13:09:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232643AbiDRNMC (ORCPT ); Mon, 18 Apr 2022 09:12:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243231AbiDRNJ6 (ORCPT ); Mon, 18 Apr 2022 09:09:58 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2770637A07; Mon, 18 Apr 2022 05:49:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 91DA2B80EE7; Mon, 18 Apr 2022 12:49:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C345BC385A1; Mon, 18 Apr 2022 12:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650286148; bh=wT4/WeIwjYBzff+W5ZOYqpym7Y597fnbp/CAYefVOIE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sgjn8gsRszB6q4ey1OP/i5SzwAo5b60Dtcb7gpMpCZZV82QYAPyEcXKZZjZK9rlgV K+fUMDnTzqiaFrYyAQujbrgYQEbGEL5zcUuPsVq8jILIGkxMThZ7smJUyZk0nRWBbP S/8NhCSCWiDX3AEzy/gNF0Sssb4JXfkNT3QxVcNuSXp0rJ00pl9+6bO1qreOrUlvtu AO5dCuXrj+yCkWKa6wPqmi0shzqXCycS76wM/mni9vqxhF/NgjuFFOTlLtw3vRLa3r z8fngrW8+fpq823RjavNKC0GzI2nNryfYc5zP+3DeAg0GixTPx3UOou6Av3SN6G6AN f+fU2Ns2C7FVA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh Subject: [PATCHv2 bpf-next 3/4] bpf: Resolve symbols with kallsyms_lookup_names for kprobe multi link Date: Mon, 18 Apr 2022 14:48:33 +0200 Message-Id: <20220418124834.829064-4-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220418124834.829064-1-jolsa@kernel.org> References: <20220418124834.829064-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Using kallsyms_lookup_names function to speed up symbols lookup in kprobe multi link attachment and replacing with it the current kprobe_multi_resolve_syms function. This speeds up bpftrace kprobe attachment: # perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% ) After: # perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% ) Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- kernel/trace/bpf_trace.c | 113 +++++++++++++++++++++++---------------- 1 file changed, 67 insertions(+), 46 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index b26f3da943de..f49cdc46a21f 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2226,6 +2226,60 @@ struct bpf_kprobe_multi_run_ctx { unsigned long entry_ip; }; +struct user_syms { + const char **syms; + char *buf; +}; + +static int copy_user_syms(struct user_syms *us, unsigned long __user *usyms, u32 cnt) +{ + unsigned long __user usymbol; + const char **syms = NULL; + char *buf = NULL, *p; + int err = -EFAULT; + unsigned int i; + + err = -ENOMEM; + syms = kvmalloc(cnt * sizeof(*syms), GFP_KERNEL); + if (!syms) + goto error; + + buf = kvmalloc(cnt * KSYM_NAME_LEN, GFP_KERNEL); + if (!buf) + goto error; + + for (p = buf, i = 0; i < cnt; i++) { + if (__get_user(usymbol, usyms + i)) { + err = -EFAULT; + goto error; + } + err = strncpy_from_user(p, (const char __user *) usymbol, KSYM_NAME_LEN); + if (err == KSYM_NAME_LEN) + err = -E2BIG; + if (err < 0) + goto error; + syms[i] = p; + p += err + 1; + } + + err = 0; + us->syms = syms; + us->buf = buf; + +error: + if (err) { + kvfree(syms); + kvfree(buf); + } + return err; +} + +static void free_user_syms(struct user_syms *us) +{ + kvfree(us->syms); + kvfree(us->buf); +} + static void bpf_kprobe_multi_link_release(struct bpf_link *link) { struct bpf_kprobe_multi_link *kmulti_link; @@ -2346,53 +2400,12 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long entry_ip, kprobe_multi_link_prog_run(link, entry_ip, regs); } -static int -kprobe_multi_resolve_syms(const void __user *usyms, u32 cnt, - unsigned long *addrs) +static int symbols_cmp(const void *a, const void *b) { - unsigned long addr, size; - const char __user **syms; - int err = -ENOMEM; - unsigned int i; - char *func; - - size = cnt * sizeof(*syms); - syms = kvzalloc(size, GFP_KERNEL); - if (!syms) - return -ENOMEM; - - func = kmalloc(KSYM_NAME_LEN, GFP_KERNEL); - if (!func) - goto error; - - if (copy_from_user(syms, usyms, size)) { - err = -EFAULT; - goto error; - } - - for (i = 0; i < cnt; i++) { - err = strncpy_from_user(func, syms[i], KSYM_NAME_LEN); - if (err == KSYM_NAME_LEN) - err = -E2BIG; - if (err < 0) - goto error; - err = -EINVAL; - addr = kallsyms_lookup_name(func); - if (!addr) - goto error; - if (!kallsyms_lookup_size_offset(addr, &size, NULL)) - goto error; - addr = ftrace_location_range(addr, addr + size - 1); - if (!addr) - goto error; - addrs[i] = addr; - } + const char **str_a = (const char **) a; + const char **str_b = (const char **) b; - err = 0; -error: - kvfree(syms); - kfree(func); - return err; + return strcmp(*str_a, *str_b); } int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) @@ -2438,7 +2451,15 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr goto error; } } else { - err = kprobe_multi_resolve_syms(usyms, cnt, addrs); + struct user_syms us; + + err = copy_user_syms(&us, usyms, cnt); + if (err) + goto error; + + sort(us.syms, cnt, sizeof(*us.syms), symbols_cmp, NULL); + err = kallsyms_lookup_names(us.syms, cnt, addrs); + free_user_syms(&us); if (err) goto error; } From patchwork Mon Apr 18 12:48:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12816581 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E791C4332F for ; Mon, 18 Apr 2022 13:12:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240464AbiDRNPa (ORCPT ); Mon, 18 Apr 2022 09:15:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243140AbiDRNJv (ORCPT ); Mon, 18 Apr 2022 09:09:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23F65377C8; Mon, 18 Apr 2022 05:49:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 01BEB6124A; Mon, 18 Apr 2022 12:49:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BAE41C385A8; Mon, 18 Apr 2022 12:49:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650286158; bh=lBl8uds7Jfl9k1iKeNXaVIGGGJfONnxxms9Sd0YLPoo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NF63zzSyYP9XW4uApY83MraFba3LZD9+2F5T+tpML1DyFTcjL9dIcO8PB0wjD7p5d JmKifUGp7XxNpv9eMA5sq3ecPlpGySyQDNiKfSL5viOSTAqVBT2F71dA2TARQ2Vux+ yGD+20a4Ggd0V7HkOG98Z+yFGf6CwiO8ys5jrsWiMqK7tH5w8ML3dGBpsUhX9shBVZ doXnbLhiZqhoAGgM8sYTqY3bLvK5xYllo4bLj32+6SgOmuxva+7ZE6w4FjmFCfwwYr TIxHtVP0gO2hlfGqM453QCfA+B8lqC3e/OKhM7Z7tjlgBCRfjHx94Muj8GoNKZhu4e AD35I4qdvek9g== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh Subject: [PATCHv2 bpf-next 4/4] selftests/bpf: Add attach bench test Date: Mon, 18 Apr 2022 14:48:34 +0200 Message-Id: <20220418124834.829064-5-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220418124834.829064-1-jolsa@kernel.org> References: <20220418124834.829064-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding test that reads all functions from ftrace available_filter_functions file and attach them all through kprobe_multi API. It also prints stats info with -v option, like on my setup: test_bench_attach: found 48712 functions test_bench_attach: attached in 1.069s test_bench_attach: detached in 0.373s Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/kprobe_multi_test.c | 136 ++++++++++++++++++ .../selftests/bpf/progs/kprobe_multi_empty.c | 12 ++ 2 files changed, 148 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_empty.c diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c index b9876b55fc0c..05f0fab8af89 100644 --- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c @@ -2,6 +2,9 @@ #include #include "kprobe_multi.skel.h" #include "trace_helpers.h" +#include "kprobe_multi_empty.skel.h" +#include "bpf/libbpf_internal.h" +#include "bpf/hashmap.h" static void kprobe_multi_test_run(struct kprobe_multi *skel, bool test_return) { @@ -301,6 +304,137 @@ static void test_attach_api_fails(void) kprobe_multi__destroy(skel); } +static inline __u64 get_time_ns(void) +{ + struct timespec t; + + clock_gettime(CLOCK_MONOTONIC, &t); + return (__u64) t.tv_sec * 1000000000 + t.tv_nsec; +} + +static size_t symbol_hash(const void *key, void *ctx __maybe_unused) +{ + return str_hash((const char *) key); +} + +static bool symbol_equal(const void *key1, const void *key2, void *ctx __maybe_unused) +{ + return strcmp((const char *) key1, (const char *) key2) == 0; +} + +#define DEBUGFS "/sys/kernel/debug/tracing/" + +static int get_syms(char ***symsp, size_t *cntp) +{ + size_t cap = 0, cnt = 0, i; + char *name, **syms = NULL; + struct hashmap *map; + char buf[256]; + FILE *f; + int err; + + /* + * The available_filter_functions contains many duplicates, + * but other than that all symbols are usable in kprobe multi + * interface. + * Filtering out duplicates by using hashmap__add, which won't + * add existing entry. + */ + f = fopen(DEBUGFS "available_filter_functions", "r"); + if (!f) + return -EINVAL; + + map = hashmap__new(symbol_hash, symbol_equal, NULL); + err = libbpf_get_error(map); + if (err) + goto error; + + while (fgets(buf, sizeof(buf), f)) { + /* skip modules */ + if (strchr(buf, '[')) + continue; + if (sscanf(buf, "%ms$*[^\n]\n", &name) != 1) + continue; + err = hashmap__add(map, name, NULL); + if (err) { + free(name); + if (err == -EEXIST) + continue; + goto error; + } + err = libbpf_ensure_mem((void **) &syms, &cap, + sizeof(*syms), cnt + 1); + if (err) { + free(name); + goto error; + } + syms[cnt] = name; + cnt++; + } + + *symsp = syms; + *cntp = cnt; + +error: + fclose(f); + hashmap__free(map); + if (err) { + for (i = 0; i < cnt; i++) + free(syms[cnt]); + free(syms); + } + return err; +} + +static void test_bench_attach(void) +{ + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + struct kprobe_multi_empty *skel = NULL; + long attach_start_ns, attach_end_ns; + long detach_start_ns, detach_end_ns; + double attach_delta, detach_delta; + struct bpf_link *link = NULL; + char **syms = NULL; + size_t cnt, i; + + if (!ASSERT_OK(get_syms(&syms, &cnt), "get_syms")) + return; + + skel = kprobe_multi_empty__open_and_load(); + if (!ASSERT_OK_PTR(skel, "kprobe_multi_empty__open_and_load")) + goto cleanup; + + opts.syms = (const char **) syms; + opts.cnt = cnt; + + attach_start_ns = get_time_ns(); + link = bpf_program__attach_kprobe_multi_opts(skel->progs.test_kprobe_empty, + NULL, &opts); + attach_end_ns = get_time_ns(); + + if (!ASSERT_OK_PTR(link, "bpf_program__attach_kprobe_multi_opts")) + goto cleanup; + + detach_start_ns = get_time_ns(); + bpf_link__destroy(link); + detach_end_ns = get_time_ns(); + + attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; + detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0; + + fprintf(stderr, "%s: found %lu functions\n", __func__, cnt); + fprintf(stderr, "%s: attached in %7.3lfs\n", __func__, attach_delta); + fprintf(stderr, "%s: detached in %7.3lfs\n", __func__, detach_delta); + +cleanup: + kprobe_multi_empty__destroy(skel); + if (syms) { + for (i = 0; i < cnt; i++) + free(syms[i]); + free(syms); + } +} + void test_kprobe_multi_test(void) { if (!ASSERT_OK(load_kallsyms(), "load_kallsyms")) @@ -320,4 +454,6 @@ void test_kprobe_multi_test(void) test_attach_api_syms(); if (test__start_subtest("attach_api_fails")) test_attach_api_fails(); + if (test__start_subtest("bench_attach")) + test_bench_attach(); } diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi_empty.c b/tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file mode 100644 index 000000000000..be9e3d891d46 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kprobe_multi_empty.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +SEC("kprobe.multi/*") +int test_kprobe_empty(struct pt_regs *ctx) +{ + return 0; +}