From patchwork Fri Apr 22 10:00:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12823168 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71E02C433F5 for ; Fri, 22 Apr 2022 10:00:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1446340AbiDVKDi (ORCPT ); Fri, 22 Apr 2022 06:03:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348202AbiDVKDg (ORCPT ); Fri, 22 Apr 2022 06:03:36 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38C5253B4E; Fri, 22 Apr 2022 03:00:44 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C687A61E0D; Fri, 22 Apr 2022 10:00:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D75BBC385A0; Fri, 22 Apr 2022 10:00:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650621643; bh=eUbVKBGrvVUxVF2TsSLPPb4Ds3rqV3aCUYt+UJyghLs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hxBz3l2kxMglAVT+iU0pp5Y2nD3XzeuClYC534Ajn4Tgchx/dczxyZdXHPtBUeIFM f9FYWVuoOP9k87y7qR6ijHRmisrB0ZC7UzdesQ4LNuLLmNfZezR0wze2bBtccDBVfc PoKXaONvj0OTiH2O1L4UF6tsUyvB8yxYFUP4kIjXIBz3BdcnzAhdK2FELPSbwW+QIW LPPkEZQgLGk769n2CAqWXB726G9tGpTo7g18no1zrl3EPyEGS1tZqwOPT5LEA7SXlw g+QeWU6Okq8fMz2JB72/1w6iHC4+wkcVFdKzfky2nP6KKzR63lCa0Bn22n5Rz+XRIp Soyu4RhrjtOfQ== From: Jiri Olsa To: Arnaldo Carvalho de Melo , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: linux-perf-users@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Ingo Molnar , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , Ian Rogers Subject: [PATCH perf/core 1/5] libbpf: Add bpf_program__set_insns function Date: Fri, 22 Apr 2022 12:00:21 +0200 Message-Id: <20220422100025.1469207-2-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220422100025.1469207-1-jolsa@kernel.org> References: <20220422100025.1469207-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding bpf_program__set_insns that allows to set new instructions for program. Also moving bpf_program__attach_kprobe_multi_opts on the proper name sorted place in map file. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 8 ++++++++ tools/lib/bpf/libbpf.h | 12 ++++++++++++ tools/lib/bpf/libbpf.map | 3 ++- 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 809fe209cdcc..284790d81c1b 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -8457,6 +8457,14 @@ size_t bpf_program__insn_cnt(const struct bpf_program *prog) return prog->insns_cnt; } +void bpf_program__set_insns(struct bpf_program *prog, + struct bpf_insn *insns, size_t insns_cnt) +{ + free(prog->insns); + prog->insns = insns; + prog->insns_cnt = insns_cnt; +} + int bpf_program__set_prep(struct bpf_program *prog, int nr_instances, bpf_program_prep_t prep) { diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 05dde85e19a6..b31ad58d335f 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -323,6 +323,18 @@ struct bpf_insn; * different. */ LIBBPF_API const struct bpf_insn *bpf_program__insns(const struct bpf_program *prog); + +/** + * @brief **bpf_program__set_insns()** can set BPF program's underlying + * BPF instructions. + * @param prog BPF program for which to return instructions + * @param insn a pointer to an array of BPF instructions + * @param insns_cnt number of `struct bpf_insn`'s that form + * specified BPF program + */ +LIBBPF_API void bpf_program__set_insns(struct bpf_program *prog, + struct bpf_insn *insns, size_t insns_cnt); + /** * @brief **bpf_program__insn_cnt()** returns number of `struct bpf_insn`'s * that form specified BPF program. diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index dd35ee58bfaa..afa10d24ab41 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -444,7 +444,8 @@ LIBBPF_0.8.0 { global: bpf_object__destroy_subskeleton; bpf_object__open_subskeleton; + bpf_program__attach_kprobe_multi_opts; + bpf_program__set_insns; libbpf_register_prog_handler; libbpf_unregister_prog_handler; - bpf_program__attach_kprobe_multi_opts; } LIBBPF_0.7.0; From patchwork Fri Apr 22 10:00:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12823169 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E3BEC433EF for ; Fri, 22 Apr 2022 10:00:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1446347AbiDVKDt (ORCPT ); Fri, 22 Apr 2022 06:03:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348202AbiDVKDr (ORCPT ); Fri, 22 Apr 2022 06:03:47 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A15223CFFD; Fri, 22 Apr 2022 03:00:55 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3E6AB61E0D; Fri, 22 Apr 2022 10:00:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B97CBC385A4; Fri, 22 Apr 2022 10:00:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650621654; bh=l3QQdpXdux2M4AyDc26VJwNvtk8Vaco1LZHB5M52YYE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ru6TTEKUp6T28F5HUuIo6smBeiiu1WspTfaSHfTaXJAF6wExVAde87EJH58jNlOvl o95d1M/wY0AbXrumgzvmzSgFT9MHtqbHMMY6yT+syjMBAauz5NIk/eEyQI4GhsldAs //PZRcbLM4LpZh+KKpT7uLlbI9fOkO9Ti+fjgTDaXUw3qlxJJJJqSxasiO8wiwynUN sJtc4SHugo+KicvHM1IqWfDprNK7pC0JNK+lKM/n48WbPugDmQtBe1jz5H6LMXoteI qrMu32Mg5DgEcnLlAvd7evPh0nWIgeMFAFLDLVwJKOp1CC09PDqUM62E31NWcSNLg9 55UpmRflEDj2g== From: Jiri Olsa To: Arnaldo Carvalho de Melo , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: linux-perf-users@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Ingo Molnar , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , Ian Rogers Subject: [PATCH perf/core 2/5] libbpf: Load prog's instructions after prog_prepare_load_fn callback Date: Fri, 22 Apr 2022 12:00:22 +0200 Message-Id: <20220422100025.1469207-3-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220422100025.1469207-1-jolsa@kernel.org> References: <20220422100025.1469207-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net We can have changed instructions after calling prog_prepare_load_fn callback, reload them. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 284790d81c1b..c8df74e5f658 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6695,6 +6695,8 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_prog prog->name, err); return err; } + insns = prog->insns; + insns_cnt = prog->insns_cnt; } if (obj->gen_loader) { From patchwork Fri Apr 22 10:00:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12823170 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5CC6C433EF for ; Fri, 22 Apr 2022 10:01:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1446165AbiDVKED (ORCPT ); Fri, 22 Apr 2022 06:04:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348202AbiDVKEB (ORCPT ); Fri, 22 Apr 2022 06:04:01 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A292D53B4E; Fri, 22 Apr 2022 03:01:08 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5EA7EB82B7E; Fri, 22 Apr 2022 10:01:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26809C385A4; Fri, 22 Apr 2022 10:01:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650621666; bh=qjGK3fbxdhgDQNS87gNp4USM55RS7Z4iOxgE1kHd7lM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P15evWdThpim+dKsPlve4yeyjZcu7/2zKARb3kEi/e5zit9Ks0eroXcHtUz2mijqe fOwMRxNae2lCmGnixBiQE9wBPbwhWOga9hSD53c0vwOMkCoJ4gkkxw0VXyTkY2Kv2V J9oOaiASh6iQ606kzHGyv06aDoSb1ZNO95F/RBvcz8eSLDRS9Okg0UxKHEimI8pjvE 5jSoPhtyVeBU+baN6AWwLE9IuvKL8F7naLLLiuwrf/14x94RgjMXmniIVM2/nZde42 4eIG6T95uM82AtHdr7QqHb2Ir5Z+d9PQDskfyn+i5dWuGyopzcmBrjDHBg9KeAhEku 1NTyat87QCwPg== From: Jiri Olsa To: Arnaldo Carvalho de Melo , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: linux-perf-users@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Ingo Molnar , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , Ian Rogers Subject: [PATCH perf/core 3/5] perf tools: Move libbpf init in libbpf_init function Date: Fri, 22 Apr 2022 12:00:23 +0200 Message-Id: <20220422100025.1469207-4-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220422100025.1469207-1-jolsa@kernel.org> References: <20220422100025.1469207-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Moving the libbpf init code into single function, so we have single place doing that. Signed-off-by: Jiri Olsa --- tools/perf/util/bpf-loader.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c index b72cef1ae959..f8ad581ea247 100644 --- a/tools/perf/util/bpf-loader.c +++ b/tools/perf/util/bpf-loader.c @@ -99,16 +99,26 @@ static int bpf_perf_object__add(struct bpf_object *obj) return perf_obj ? 0 : -ENOMEM; } +static int libbpf_init(void) +{ + if (libbpf_initialized) + return 0; + + libbpf_set_print(libbpf_perf_print); + libbpf_initialized = true; + return 0; +} + struct bpf_object * bpf__prepare_load_buffer(void *obj_buf, size_t obj_buf_sz, const char *name) { LIBBPF_OPTS(bpf_object_open_opts, opts, .object_name = name); struct bpf_object *obj; + int err; - if (!libbpf_initialized) { - libbpf_set_print(libbpf_perf_print); - libbpf_initialized = true; - } + err = libbpf_init(); + if (err) + return ERR_PTR(err); obj = bpf_object__open_mem(obj_buf, obj_buf_sz, &opts); if (IS_ERR_OR_NULL(obj)) { @@ -135,14 +145,13 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source) { LIBBPF_OPTS(bpf_object_open_opts, opts, .object_name = filename); struct bpf_object *obj; + int err; - if (!libbpf_initialized) { - libbpf_set_print(libbpf_perf_print); - libbpf_initialized = true; - } + err = libbpf_init(); + if (err) + return ERR_PTR(err); if (source) { - int err; void *obj_buf; size_t obj_buf_sz; From patchwork Fri Apr 22 10:00:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12823171 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A84EC433EF for ; Fri, 22 Apr 2022 10:01:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1446365AbiDVKEO (ORCPT ); Fri, 22 Apr 2022 06:04:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1446354AbiDVKEM (ORCPT ); Fri, 22 Apr 2022 06:04:12 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36EE253B7C; Fri, 22 Apr 2022 03:01:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C7C8961E25; Fri, 22 Apr 2022 10:01:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02857C385AA; Fri, 22 Apr 2022 10:01:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650621678; bh=SoXivtN+WDMyu86XjuSaxFCb9sQo39F8l+g0EQFcewY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lJolsis8aO6YR5ZczGJaUsva2pb/UDje6HqW1zQxTgfzGnnkXJ1BTW3HWhXhU7q/k KCng61TLm2CCRtbaPtKnn5oCKVmcNhr7y12IzJtjcbhADwRn+HktpQJ385f8yx6srf +KD65U4HVkut0pg+g2If5zeNFJJVXdRrHvlo6eAieAwZtyXKece+6M+ionkq+YF9fE 7T7eYZeEsAEV5YVGGu0ugEMJupWlgC8lJBhs4fz408f9kj18gQJZvIZ92ncW5Cf+2S JkwzBiIWRk71Cg0N6e8FxK8wnzmqNfxRZfW82aSh1EYYA7VzytDEL/A7cZ4C3+zi9e vSY/AMJrd1dkw== From: Jiri Olsa To: Arnaldo Carvalho de Melo , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: linux-perf-users@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Ingo Molnar , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , Ian Rogers Subject: [PATCH perf/core 4/5] perf tools: Register perfkprobe libbpf section handler Date: Fri, 22 Apr 2022 12:00:24 +0200 Message-Id: <20220422100025.1469207-5-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220422100025.1469207-1-jolsa@kernel.org> References: <20220422100025.1469207-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Perf is using section name to declare special kprobe arguments, which no longer works with current libbpf, that either requires certain form of the section name or allows to register custom handler. Adding support for 'perfkprobe/' section name handler to take care of perf kprobe programs. The handler servers two purposes: - allows perf programs to have special arguments in section name - allows perf to use pre-load callback where we can attach init code (zeroing all argument registers) to each perf program The second is essential part of new prologue generation code, that's coming in following patch. Signed-off-by: Jiri Olsa --- tools/perf/util/bpf-loader.c | 50 ++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c index f8ad581ea247..92dd8cc18edb 100644 --- a/tools/perf/util/bpf-loader.c +++ b/tools/perf/util/bpf-loader.c @@ -86,6 +86,7 @@ bpf_perf_object__next(struct bpf_perf_object *prev) (perf_obj) = (tmp), (tmp) = bpf_perf_object__next(tmp)) static bool libbpf_initialized; +static int libbpf_sec_handler; static int bpf_perf_object__add(struct bpf_object *obj) { @@ -99,12 +100,61 @@ static int bpf_perf_object__add(struct bpf_object *obj) return perf_obj ? 0 : -ENOMEM; } +static struct bpf_insn prologue_init_insn[] = { + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_MOV64_IMM(BPF_REG_1, 0), + BPF_MOV64_IMM(BPF_REG_2, 0), + BPF_MOV64_IMM(BPF_REG_3, 0), + BPF_MOV64_IMM(BPF_REG_4, 0), + BPF_MOV64_IMM(BPF_REG_5, 0), +}; + +#define LIBBPF_SEC_PREFIX "perfkprobe/" + +static int libbpf_prog_prepare_load_fn(struct bpf_program *prog, + struct bpf_prog_load_opts *opts __maybe_unused, + long cookie __maybe_unused) +{ + size_t init_size_cnt = ARRAY_SIZE(prologue_init_insn); + size_t orig_insn_cnt, insn_cnt, init_size, orig_size; + const struct bpf_insn *orig_insn; + struct bpf_insn *insn; + + /* prepend initialization code to program instructions */ + orig_insn = bpf_program__insns(prog); + orig_insn_cnt = bpf_program__insn_cnt(prog); + init_size = init_size_cnt * sizeof(*insn); + orig_size = orig_insn_cnt * sizeof(*insn); + + insn_cnt = orig_insn_cnt + init_size_cnt; + insn = malloc(insn_cnt * sizeof(*insn)); + if (!insn) + return -ENOMEM; + + memcpy(insn, prologue_init_insn, init_size); + memcpy((char *) insn + init_size, orig_insn, orig_size); + bpf_program__set_insns(prog, insn, insn_cnt); + return 0; +} + static int libbpf_init(void) { + LIBBPF_OPTS(libbpf_prog_handler_opts, handler_opts, + .prog_prepare_load_fn = libbpf_prog_prepare_load_fn, + ); + if (libbpf_initialized) return 0; libbpf_set_print(libbpf_perf_print); + libbpf_sec_handler = libbpf_register_prog_handler(LIBBPF_SEC_PREFIX, + BPF_PROG_TYPE_KPROBE, + 0, &handler_opts); + if (libbpf_sec_handler < 0) { + pr_debug("bpf: failed to register libbpf section handler: %d\n", + libbpf_sec_handler); + return -BPF_LOADER_ERRNO__INTERNAL; + } libbpf_initialized = true; return 0; } From patchwork Fri Apr 22 10:00:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12823172 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4602C433F5 for ; Fri, 22 Apr 2022 10:01:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1446353AbiDVKEc (ORCPT ); Fri, 22 Apr 2022 06:04:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1446367AbiDVKEY (ORCPT ); Fri, 22 Apr 2022 06:04:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30FE7541A6; Fri, 22 Apr 2022 03:01:31 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C26A861E25; Fri, 22 Apr 2022 10:01:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CDE84C385A4; Fri, 22 Apr 2022 10:01:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650621690; bh=NUItUtsCwaH8t+rgiyagJdkrcn3JwgnJlI9AIhlyX/w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E5vy0r/5ruek8frwintwh371kfseVC78eJCayBE46YwgSBrztGM6UrenjSyocGDV2 sT99gi5e72Qa1fqGX3B7x7hY1wWHFTRuMv8pUpccDFzMoIbTEifgY476PA0PqV/2G7 CiJSCyOpo6GPNssBwEoopDPYvJXKGy/imYTrAekjyg/VA95qAb9cq7BFNa8LZi/oxP R20XyZD5/sx9jiFY83YThYRbuu9x7mNCnRQMmhYD/e4J+VNIP/CdarSRszr3lS4SuW eKfRjR0pZHrh0rP/f69KtrfykqZ3h2IkagyOFDhlvWVK/VxHGGYdfeA1c1A75xFOLF sP2F18ns+z5VQ== From: Jiri Olsa To: Arnaldo Carvalho de Melo , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: linux-perf-users@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Ingo Molnar , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , Ian Rogers Subject: [PATCH perf/core 5/5] perf tools: Rework prologue generation code Date: Fri, 22 Apr 2022 12:00:25 +0200 Message-Id: <20220422100025.1469207-6-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220422100025.1469207-1-jolsa@kernel.org> References: <20220422100025.1469207-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Some functions we use for bpf prologue generation are going to be deprecated. This change reworks current code not to use them. We need to replace following functions/struct: bpf_program__set_prep bpf_program__nth_fd struct bpf_prog_prep_result We use bpf_program__set_prep to hook perf callback before program is loaded and provide new instructions with the prologue. We workaround this by taking instructions for specific program, attaching prologue to them and load such new ebpf programs with prologue using separate bpf_prog_load calls (outside libbpf load machinery). Before we can take program instructions, we need libbpf to actually load it. This way we get the final shape of its instructions (with all relocations and verifier adjustments). There's one glitch though.. perf kprobe program already assumes generated prologue code with proper values in argument registers, so loading such program directly will fail in the verifier. That's where 'perfkprobe/' pre-load handler fits in and prepends the initialization code to the program. Once such program is loaded we take its instructions, cut off the initialization code and prepend the prologue. I know.. sorry ;-) Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- tools/perf/include/bpf/bpf.h | 2 +- tools/perf/tests/bpf-script-example.c | 2 +- tools/perf/tests/bpf-script-test-prologue.c | 2 +- tools/perf/util/bpf-loader.c | 136 +++++++++++++++++--- 4 files changed, 120 insertions(+), 22 deletions(-) diff --git a/tools/perf/include/bpf/bpf.h b/tools/perf/include/bpf/bpf.h index b422aeef5339..91869f6fb672 100644 --- a/tools/perf/include/bpf/bpf.h +++ b/tools/perf/include/bpf/bpf.h @@ -50,7 +50,7 @@ static void (*bpf_tail_call)(void *ctx, void *map, int index) = (void *)BPF_FUNC #define SEC(NAME) __attribute__((section(NAME), used)) #define probe(function, vars) \ - SEC(#function "=" #function " " #vars) function + SEC("perfkprobe/" #function "=" #function " " #vars) function #define syscall_enter(name) \ SEC("syscalls:sys_enter_" #name) syscall_enter_ ## name diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c index ab4b98b3165d..56673fa1f30d 100644 --- a/tools/perf/tests/bpf-script-example.c +++ b/tools/perf/tests/bpf-script-example.c @@ -32,7 +32,7 @@ struct bpf_map_def SEC("maps") flip_table = { .max_entries = 1, }; -SEC("func=do_epoll_wait") +SEC("perfkprobe/func=do_epoll_wait") int bpf_func__SyS_epoll_pwait(void *ctx) { int ind =0; diff --git a/tools/perf/tests/bpf-script-test-prologue.c b/tools/perf/tests/bpf-script-test-prologue.c index bd83d364cf30..00dac5a23938 100644 --- a/tools/perf/tests/bpf-script-test-prologue.c +++ b/tools/perf/tests/bpf-script-test-prologue.c @@ -26,7 +26,7 @@ static void (*bpf_trace_printk)(const char *fmt, int fmt_size, ...) = (void *) 6; -SEC("func=null_lseek file->f_mode offset orig") +SEC("perfkprobe/func=null_lseek file->f_mode offset orig") int bpf_func__null_lseek(void *ctx, int err, unsigned long _f_mode, unsigned long offset, unsigned long orig) { diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c index 92dd8cc18edb..10151da862c8 100644 --- a/tools/perf/util/bpf-loader.c +++ b/tools/perf/util/bpf-loader.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -49,6 +50,7 @@ struct bpf_prog_priv { struct bpf_insn *insns_buf; int nr_types; int *type_mapping; + int *proglogue_fds; }; struct bpf_perf_object { @@ -56,6 +58,11 @@ struct bpf_perf_object { struct bpf_object *obj; }; +struct bpf_preproc_result { + struct bpf_insn *new_insn_ptr; + int new_insn_cnt; +}; + static LIST_HEAD(bpf_objects_list); static struct hashmap *bpf_program_hash; static struct hashmap *bpf_map_hash; @@ -238,14 +245,31 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source) return obj; } +static void close_prologue_programs(struct bpf_prog_priv *priv) +{ + struct perf_probe_event *pev; + int i, fd; + + if (!priv->need_prologue) + return; + pev = &priv->pev; + for (i = 0; i < pev->ntevs; i++) { + fd = priv->proglogue_fds[i]; + if (fd != -1) + close(fd); + } +} + static void clear_prog_priv(const struct bpf_program *prog __maybe_unused, void *_priv) { struct bpf_prog_priv *priv = _priv; + close_prologue_programs(priv); cleanup_perf_probe_events(&priv->pev, 1); zfree(&priv->insns_buf); + zfree(&priv->proglogue_fds); zfree(&priv->type_mapping); zfree(&priv->sys_name); zfree(&priv->evt_name); @@ -480,9 +504,15 @@ static int parse_prog_config(const char *config_str, const char **p_main_str, bool *is_tp, struct perf_probe_event *pev) { + const char *main_str, *parse_str; int err; - const char *main_str = parse_prog_config_kvpair(config_str, pev); + /* Make sure it's our section with 'perfkprobe/' prefix check. */ + if (!strstarts(config_str, LIBBPF_SEC_PREFIX)) + return -EINVAL; + + parse_str = config_str + sizeof(LIBBPF_SEC_PREFIX) - 1; + main_str = parse_prog_config_kvpair(parse_str, pev); if (IS_ERR(main_str)) return PTR_ERR(main_str); @@ -608,8 +638,8 @@ static int bpf__prepare_probe(void) static int preproc_gen_prologue(struct bpf_program *prog, int n, - struct bpf_insn *orig_insns, int orig_insns_cnt, - struct bpf_prog_prep_result *res) + const struct bpf_insn *orig_insns, int orig_insns_cnt, + struct bpf_preproc_result *res) { struct bpf_prog_priv *priv = program_priv(prog); struct probe_trace_event *tev; @@ -657,7 +687,6 @@ preproc_gen_prologue(struct bpf_program *prog, int n, res->new_insn_ptr = buf; res->new_insn_cnt = prologue_cnt + orig_insns_cnt; - res->pfd = NULL; return 0; errout: @@ -765,7 +794,7 @@ static int hook_load_preprocessor(struct bpf_program *prog) struct bpf_prog_priv *priv = program_priv(prog); struct perf_probe_event *pev; bool need_prologue = false; - int err, i; + int i; if (IS_ERR_OR_NULL(priv)) { pr_debug("Internal error when hook preprocessor\n"); @@ -803,6 +832,13 @@ static int hook_load_preprocessor(struct bpf_program *prog) return -ENOMEM; } + priv->proglogue_fds = malloc(sizeof(int) * pev->ntevs); + if (!priv->proglogue_fds) { + pr_debug("Not enough memory: alloc prologue fds failed\n"); + return -ENOMEM; + } + memset(priv->proglogue_fds, -1, sizeof(int) * pev->ntevs); + priv->type_mapping = malloc(sizeof(int) * pev->ntevs); if (!priv->type_mapping) { pr_debug("Not enough memory: alloc type_mapping failed\n"); @@ -811,13 +847,7 @@ static int hook_load_preprocessor(struct bpf_program *prog) memset(priv->type_mapping, -1, sizeof(int) * pev->ntevs); - err = map_prologue(pev, priv->type_mapping, &priv->nr_types); - if (err) - return err; - - err = bpf_program__set_prep(prog, priv->nr_types, - preproc_gen_prologue); - return err; + return map_prologue(pev, priv->type_mapping, &priv->nr_types); } int bpf__probe(struct bpf_object *obj) @@ -924,6 +954,77 @@ int bpf__unprobe(struct bpf_object *obj) return ret; } +static int bpf_object__load_prologue(struct bpf_object *obj) +{ + int init_cnt = ARRAY_SIZE(prologue_init_insn); + const struct bpf_insn *orig_insns; + struct bpf_preproc_result res; + struct perf_probe_event *pev; + struct bpf_program *prog; + int orig_insns_cnt; + + bpf_object__for_each_program(prog, obj) { + struct bpf_prog_priv *priv = program_priv(prog); + int err, i, fd; + + if (IS_ERR_OR_NULL(priv)) { + pr_debug("bpf: failed to get private field\n"); + return -BPF_LOADER_ERRNO__INTERNAL; + } + + if (!priv->need_prologue) + continue; + + /* + * For each program that needs prologue we do following: + * + * - take its current instructions and use them + * to generate the new code with prologue + * - load new instructions with bpf_prog_load + * and keep the fd in proglogue_fds + * - new fd will be used in bpf__foreach_event + * to connect this program with perf evsel + */ + orig_insns = bpf_program__insns(prog); + orig_insns_cnt = bpf_program__insn_cnt(prog); + + pev = &priv->pev; + for (i = 0; i < pev->ntevs; i++) { + /* + * Skipping artificall prologue_init_insn instructions + * (init_cnt), so the prologue can be generated instead + * of them. + */ + err = preproc_gen_prologue(prog, i, + orig_insns + init_cnt, + orig_insns_cnt - init_cnt, + &res); + if (err) + return err; + + fd = bpf_prog_load(bpf_program__get_type(prog), + bpf_program__name(prog), "GPL", + res.new_insn_ptr, + res.new_insn_cnt, NULL); + if (fd < 0) { + char bf[128]; + + libbpf_strerror(-errno, bf, sizeof(bf)); + pr_debug("bpf: load objects with prologue failed: err=%d: (%s)\n", + -errno, bf); + return -errno; + } + priv->proglogue_fds[i] = fd; + } + /* + * We no longer need the original program, + * we can unload it. + */ + bpf_program__unload(prog); + } + return 0; +} + int bpf__load(struct bpf_object *obj) { int err; @@ -935,7 +1036,7 @@ int bpf__load(struct bpf_object *obj) pr_debug("bpf: load objects failed: err=%d: (%s)\n", err, bf); return err; } - return 0; + return bpf_object__load_prologue(obj); } int bpf__foreach_event(struct bpf_object *obj, @@ -970,13 +1071,10 @@ int bpf__foreach_event(struct bpf_object *obj, for (i = 0; i < pev->ntevs; i++) { tev = &pev->tevs[i]; - if (priv->need_prologue) { - int type = priv->type_mapping[i]; - - fd = bpf_program__nth_fd(prog, type); - } else { + if (priv->need_prologue) + fd = priv->proglogue_fds[i]; + else fd = bpf_program__fd(prog); - } if (fd < 0) { pr_debug("bpf: failed to get file descriptor\n");