From patchwork Wed Feb 9 22:26:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12740984 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C58DC433EF for ; Wed, 9 Feb 2022 22:28:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235660AbiBIW21 (ORCPT ); Wed, 9 Feb 2022 17:28:27 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:57924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235843AbiBIW2Y (ORCPT ); Wed, 9 Feb 2022 17:28:24 -0500 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0EC7E01A21B for ; Wed, 9 Feb 2022 14:27:33 -0800 (PST) Received: by mail-qk1-x72e.google.com with SMTP id m25so3060036qka.9 for ; Wed, 09 Feb 2022 14:27:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Hk75HQixjBgYteD3wu68c7ddwNDtfzUNbwLB6J1iel0=; b=U+lMrrs6wiMIg9TVKFE4wLa9hsB2uxILlCRogdU0htIn+2cfTaLccTDWHIqZlf7Sme wGxtirb9z0vBPivO2JK6mwWlIjXC+tFSzoPyQU4bSf/B/m7XdEBEiMyvzEuDbJskx9zz 1uLVqkJ5wMrGixp3UHmBOxO8pRfsz3UjFBKHk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Hk75HQixjBgYteD3wu68c7ddwNDtfzUNbwLB6J1iel0=; b=MeBqDD96IcTo/qCQrJ+fHR1nzRm8xD8rru+T8sEy42EkgilGOp6tvzQVzhsxP72sIX SESld/9ZZoa1AlVz6N3uzkzUiBRsPeOM1y6iz7qlLtjBecfT32zVAZ+XjkA/CqwWmVzG ed2wKBfA2KGWKGhIDFcN85GuUKaPMz5t3PPtquXVCF4nj1p/0BcvQbPkfA2zwzQTfjiz TpJy/eAuH+Qifw0FLIDHUCMJobBXskLvtlkn85gdpAbC8dL2Iqtt4KegC4XyprVW/NGM qoztx2KzP7KOFOc73eZRSRdVwjWtVreK9IVhsQbr1b/WuT0nLxEb9CQMC6xiHLfiOgmo j5Gg== X-Gm-Message-State: AOAM531lENSBsQbOqPdLlMGdly/gdOvG66fuGJFCnenHLp0IHPleSbaK O8P0Y8R9MrmSUmFd4T/eiQ7NRs3/cyhuzfvTlty8jBR/OMOuGKUQpRVcAMMbVRHPW7bNd6ctCPl 50qaF8IKmpxXQqJHO86EWLfIfprek5mB8CUE6zjiDgliZI1MYY2CAeFruyi7H+J9GB4I5mA== X-Google-Smtp-Source: ABdhPJxNh49jI4X/Ahn4cjn6FTCHUNR3QDNPqkhHBeQf1/vlA5U53gF2FBst0eptuX7lYgALzrUgMA== X-Received: by 2002:a05:620a:2a10:: with SMTP id o16mr2368701qkp.762.1644445650743; Wed, 09 Feb 2022 14:27:30 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:30 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 1/7] libbpf: split bpf_core_apply_relo() Date: Wed, 9 Feb 2022 17:26:40 -0500 Message-Id: <20220209222646.348365-2-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net BTFGen needs to run the core relocation logic in order to understand what are the types involved in a given relocation. Currently bpf_core_apply_relo() calculates and **applies** a relocation to an instruction. Having both operations in the same function makes it difficult to only calculate the relocation without patching the instruction. This commit splits that logic in two different phases: (1) calculate the relocation and (2) patch the instruction. For the first phase bpf_core_apply_relo() is renamed to bpf_core_calc_relo_insn() who is now only on charge of calculating the relocation, the second phase uses the already existing bpf_core_patch_insn(). bpf_object__relocate_core() uses both of them and the BTFGen will use only bpf_core_calc_relo_insn(). Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato Acked-by: Andrii Nakryiko --- kernel/bpf/btf.c | 13 +++++-- tools/lib/bpf/libbpf.c | 71 ++++++++++++++++++++--------------- tools/lib/bpf/relo_core.c | 79 ++++++++++++--------------------------- tools/lib/bpf/relo_core.h | 42 ++++++++++++++++++--- 4 files changed, 109 insertions(+), 96 deletions(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 11740b300de9..f1d3d2a2f5f6 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -7225,6 +7225,7 @@ int bpf_core_apply(struct bpf_core_ctx *ctx, const struct bpf_core_relo *relo, { bool need_cands = relo->kind != BPF_CORE_TYPE_ID_LOCAL; struct bpf_core_cand_list cands = {}; + struct bpf_core_relo_res targ_res; struct bpf_core_spec *specs; int err; @@ -7264,13 +7265,19 @@ int bpf_core_apply(struct bpf_core_ctx *ctx, const struct bpf_core_relo *relo, cands.len = cc->cnt; /* cand_cache_mutex needs to span the cache lookup and * copy of btf pointer into bpf_core_cand_list, - * since module can be unloaded while bpf_core_apply_relo_insn + * since module can be unloaded while bpf_core_calc_relo_insn * is working with module's btf. */ } - err = bpf_core_apply_relo_insn((void *)ctx->log, insn, relo->insn_off / 8, - relo, relo_idx, ctx->btf, &cands, specs); + err = bpf_core_calc_relo_insn((void *)ctx->log, relo, relo_idx, ctx->btf, &cands, specs, + &targ_res); + if (err) + goto out; + + err = bpf_core_patch_insn((void *)ctx->log, insn, relo->insn_off / 8, relo, relo_idx, + &targ_res); + out: kfree(specs); if (need_cands) { diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 2262bcdfee92..d3c457fb045e 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -5530,11 +5530,12 @@ static int record_relo_core(struct bpf_program *prog, return 0; } -static int bpf_core_apply_relo(struct bpf_program *prog, - const struct bpf_core_relo *relo, - int relo_idx, - const struct btf *local_btf, - struct hashmap *cand_cache) +static int bpf_core_resolve_relo(struct bpf_program *prog, + const struct bpf_core_relo *relo, + int relo_idx, + const struct btf *local_btf, + struct hashmap *cand_cache, + struct bpf_core_relo_res *targ_res) { struct bpf_core_spec specs_scratch[3] = {}; const void *type_key = u32_as_hash_key(relo->type_id); @@ -5543,20 +5544,7 @@ static int bpf_core_apply_relo(struct bpf_program *prog, const struct btf_type *local_type; const char *local_name; __u32 local_id = relo->type_id; - struct bpf_insn *insn; - int insn_idx, err; - - if (relo->insn_off % BPF_INSN_SZ) - return -EINVAL; - insn_idx = relo->insn_off / BPF_INSN_SZ; - /* adjust insn_idx from section frame of reference to the local - * program's frame of reference; (sub-)program code is not yet - * relocated, so it's enough to just subtract in-section offset - */ - insn_idx = insn_idx - prog->sec_insn_off; - if (insn_idx >= prog->insns_cnt) - return -EINVAL; - insn = &prog->insns[insn_idx]; + int err; local_type = btf__type_by_id(local_btf, local_id); if (!local_type) @@ -5566,15 +5554,6 @@ static int bpf_core_apply_relo(struct bpf_program *prog, if (!local_name) return -EINVAL; - if (prog->obj->gen_loader) { - const char *spec_str = btf__name_by_offset(local_btf, relo->access_str_off); - - pr_debug("record_relo_core: prog %td insn[%d] %s %s %s final insn_idx %d\n", - prog - prog->obj->programs, relo->insn_off / 8, - btf_kind_str(local_type), local_name, spec_str, insn_idx); - return record_relo_core(prog, relo, insn_idx); - } - if (relo->kind != BPF_CORE_TYPE_ID_LOCAL && !hashmap__find(cand_cache, type_key, (void **)&cands)) { cands = bpf_core_find_cands(prog->obj, local_btf, local_id); @@ -5591,19 +5570,21 @@ static int bpf_core_apply_relo(struct bpf_program *prog, } } - return bpf_core_apply_relo_insn(prog_name, insn, insn_idx, relo, - relo_idx, local_btf, cands, specs_scratch); + return bpf_core_calc_relo_insn(prog_name, relo, relo_idx, local_btf, cands, specs_scratch, + targ_res); } static int bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path) { const struct btf_ext_info_sec *sec; + struct bpf_core_relo_res targ_res; const struct bpf_core_relo *rec; const struct btf_ext_info *seg; struct hashmap_entry *entry; struct hashmap *cand_cache = NULL; struct bpf_program *prog; + struct bpf_insn *insn; const char *sec_name; int i, err = 0, insn_idx, sec_idx; @@ -5654,6 +5635,8 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path) sec_name, sec->num_info); for_each_btf_ext_rec(seg, sec, i, rec) { + if (rec->insn_off % BPF_INSN_SZ) + return -EINVAL; insn_idx = rec->insn_off / BPF_INSN_SZ; prog = find_prog_by_sec_insn(obj, sec_idx, insn_idx); if (!prog) { @@ -5668,12 +5651,38 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path) if (!prog->load) continue; - err = bpf_core_apply_relo(prog, rec, i, obj->btf, cand_cache); + /* adjust insn_idx from section frame of reference to the local + * program's frame of reference; (sub-)program code is not yet + * relocated, so it's enough to just subtract in-section offset + */ + insn_idx = insn_idx - prog->sec_insn_off; + if (insn_idx >= prog->insns_cnt) + return -EINVAL; + insn = &prog->insns[insn_idx]; + + if (prog->obj->gen_loader) { + err = record_relo_core(prog, rec, insn_idx); + if (err) { + pr_warn("prog '%s': relo #%d: failed to record relocation: %d\n", + prog->name, i, err); + goto out; + } + continue; + } + + err = bpf_core_resolve_relo(prog, rec, i, obj->btf, cand_cache, &targ_res); if (err) { pr_warn("prog '%s': relo #%d: failed to relocate: %d\n", prog->name, i, err); goto out; } + + err = bpf_core_patch_insn(prog->name, insn, insn_idx, rec, i, &targ_res); + if (err) { + pr_warn("prog '%s': relo #%d: failed to patch insn #%u: %d\n", + prog->name, i, insn_idx, err); + goto out; + } } } diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c index 910865e29edc..f946f23eab20 100644 --- a/tools/lib/bpf/relo_core.c +++ b/tools/lib/bpf/relo_core.c @@ -775,31 +775,6 @@ static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo, return 0; } -struct bpf_core_relo_res -{ - /* expected value in the instruction, unless validate == false */ - __u32 orig_val; - /* new value that needs to be patched up to */ - __u32 new_val; - /* relocation unsuccessful, poison instruction, but don't fail load */ - bool poison; - /* some relocations can't be validated against orig_val */ - bool validate; - /* for field byte offset relocations or the forms: - * *(T *)(rX + ) = rY - * rX = *(T *)(rY + ), - * we remember original and resolved field size to adjust direct - * memory loads of pointers and integers; this is necessary for 32-bit - * host kernel architectures, but also allows to automatically - * relocate fields that were resized from, e.g., u32 to u64, etc. - */ - bool fail_memsz_adjust; - __u32 orig_sz; - __u32 orig_type_id; - __u32 new_sz; - __u32 new_type_id; -}; - /* Calculate original and target relocation values, given local and target * specs and relocation kind. These values are calculated for each candidate. * If there are multiple candidates, resulting values should all be consistent @@ -951,9 +926,9 @@ static int insn_bytes_to_bpf_size(__u32 sz) * 5. *(T *)(rX + ) = rY, where T is one of {u8, u16, u32, u64}; * 6. *(T *)(rX + ) = , where T is one of {u8, u16, u32, u64}. */ -static int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn, - int insn_idx, const struct bpf_core_relo *relo, - int relo_idx, const struct bpf_core_relo_res *res) +int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn, + int insn_idx, const struct bpf_core_relo *relo, + int relo_idx, const struct bpf_core_relo_res *res) { __u32 orig_val, new_val; __u8 class; @@ -1128,7 +1103,7 @@ static void bpf_core_dump_spec(const char *prog_name, int level, const struct bp } /* - * CO-RE relocate single instruction. + * Calculate CO-RE relocation target result. * * The outline and important points of the algorithm: * 1. For given local type, find corresponding candidate target types. @@ -1177,18 +1152,18 @@ static void bpf_core_dump_spec(const char *prog_name, int level, const struct bp * between multiple relocations for the same type ID and is updated as some * of the candidates are pruned due to structural incompatibility. */ -int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, - int insn_idx, - const struct bpf_core_relo *relo, - int relo_idx, - const struct btf *local_btf, - struct bpf_core_cand_list *cands, - struct bpf_core_spec *specs_scratch) +int bpf_core_calc_relo_insn(const char *prog_name, + const struct bpf_core_relo *relo, + int relo_idx, + const struct btf *local_btf, + struct bpf_core_cand_list *cands, + struct bpf_core_spec *specs_scratch, + struct bpf_core_relo_res *targ_res) { struct bpf_core_spec *local_spec = &specs_scratch[0]; struct bpf_core_spec *cand_spec = &specs_scratch[1]; struct bpf_core_spec *targ_spec = &specs_scratch[2]; - struct bpf_core_relo_res cand_res, targ_res; + struct bpf_core_relo_res cand_res; const struct btf_type *local_type; const char *local_name; __u32 local_id; @@ -1223,12 +1198,12 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, /* TYPE_ID_LOCAL relo is special and doesn't need candidate search */ if (relo->kind == BPF_CORE_TYPE_ID_LOCAL) { /* bpf_insn's imm value could get out of sync during linking */ - memset(&targ_res, 0, sizeof(targ_res)); - targ_res.validate = false; - targ_res.poison = false; - targ_res.orig_val = local_spec->root_type_id; - targ_res.new_val = local_spec->root_type_id; - goto patch_insn; + memset(targ_res, 0, sizeof(*targ_res)); + targ_res->validate = false; + targ_res->poison = false; + targ_res->orig_val = local_spec->root_type_id; + targ_res->new_val = local_spec->root_type_id; + return 0; } /* libbpf doesn't support candidate search for anonymous types */ @@ -1262,7 +1237,7 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, return err; if (j == 0) { - targ_res = cand_res; + *targ_res = cand_res; *targ_spec = *cand_spec; } else if (cand_spec->bit_offset != targ_spec->bit_offset) { /* if there are many field relo candidates, they @@ -1272,7 +1247,8 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, prog_name, relo_idx, cand_spec->bit_offset, targ_spec->bit_offset); return -EINVAL; - } else if (cand_res.poison != targ_res.poison || cand_res.new_val != targ_res.new_val) { + } else if (cand_res.poison != targ_res->poison || + cand_res.new_val != targ_res->new_val) { /* all candidates should result in the same relocation * decision and value, otherwise it's dangerous to * proceed due to ambiguity @@ -1280,7 +1256,7 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, pr_warn("prog '%s': relo #%d: relocation decision ambiguity: %s %u != %s %u\n", prog_name, relo_idx, cand_res.poison ? "failure" : "success", cand_res.new_val, - targ_res.poison ? "failure" : "success", targ_res.new_val); + targ_res->poison ? "failure" : "success", targ_res->new_val); return -EINVAL; } @@ -1314,19 +1290,10 @@ int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, prog_name, relo_idx); /* calculate single target relo result explicitly */ - err = bpf_core_calc_relo(prog_name, relo, relo_idx, local_spec, NULL, &targ_res); + err = bpf_core_calc_relo(prog_name, relo, relo_idx, local_spec, NULL, targ_res); if (err) return err; } -patch_insn: - /* bpf_core_patch_insn() should know how to handle missing targ_spec */ - err = bpf_core_patch_insn(prog_name, insn, insn_idx, relo, relo_idx, &targ_res); - if (err) { - pr_warn("prog '%s': relo #%d: failed to patch insn #%u: %d\n", - prog_name, relo_idx, relo->insn_off / 8, err); - return -EINVAL; - } - return 0; } diff --git a/tools/lib/bpf/relo_core.h b/tools/lib/bpf/relo_core.h index 17799819ad7c..a28bf3711ce2 100644 --- a/tools/lib/bpf/relo_core.h +++ b/tools/lib/bpf/relo_core.h @@ -44,14 +44,44 @@ struct bpf_core_spec { __u32 bit_offset; }; -int bpf_core_apply_relo_insn(const char *prog_name, - struct bpf_insn *insn, int insn_idx, - const struct bpf_core_relo *relo, int relo_idx, - const struct btf *local_btf, - struct bpf_core_cand_list *cands, - struct bpf_core_spec *specs_scratch); +struct bpf_core_relo_res { + /* expected value in the instruction, unless validate == false */ + __u32 orig_val; + /* new value that needs to be patched up to */ + __u32 new_val; + /* relocation unsuccessful, poison instruction, but don't fail load */ + bool poison; + /* some relocations can't be validated against orig_val */ + bool validate; + /* for field byte offset relocations or the forms: + * *(T *)(rX + ) = rY + * rX = *(T *)(rY + ), + * we remember original and resolved field size to adjust direct + * memory loads of pointers and integers; this is necessary for 32-bit + * host kernel architectures, but also allows to automatically + * relocate fields that were resized from, e.g., u32 to u64, etc. + */ + bool fail_memsz_adjust; + __u32 orig_sz; + __u32 orig_type_id; + __u32 new_sz; + __u32 new_type_id; +}; + int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id, const struct btf *targ_btf, __u32 targ_id); size_t bpf_core_essential_name_len(const char *name); + +int bpf_core_calc_relo_insn(const char *prog_name, + const struct bpf_core_relo *relo, int relo_idx, + const struct btf *local_btf, + struct bpf_core_cand_list *cands, + struct bpf_core_spec *specs_scratch, + struct bpf_core_relo_res *targ_res); + +int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn, + int insn_idx, const struct bpf_core_relo *relo, + int relo_idx, const struct bpf_core_relo_res *res); + #endif From patchwork Wed Feb 9 22:26:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12740989 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EB18C43217 for ; Wed, 9 Feb 2022 22:28:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235694AbiBIW22 (ORCPT ); Wed, 9 Feb 2022 17:28:28 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:59310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235529AbiBIW2Z (ORCPT ); Wed, 9 Feb 2022 17:28:25 -0500 Received: from mail-qv1-xf2b.google.com (mail-qv1-xf2b.google.com [IPv6:2607:f8b0:4864:20::f2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 569C7E015677 for ; Wed, 9 Feb 2022 14:27:35 -0800 (PST) Received: by mail-qv1-xf2b.google.com with SMTP id o5so3190969qvm.3 for ; Wed, 09 Feb 2022 14:27:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3x9p3pCpwlKSG1m+jcqKD4VH4T7Zh9Qtt1uRAHJFIEc=; b=AFNmuBRrdA0Rn2i128mSB1ono01t0Yn218qLhKYuwr0lL8XAkNMhfj19G2BOo3/RjF DsbSnWzb01ilKfbBJz7A12kb5Onr9zENB579KQHMo8EhDU/EU5Pi98QX7UxMMXqYfwMb kmVQ0iz0cY1ZV2waA3VlDe2oZYiCVbgXHh++M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3x9p3pCpwlKSG1m+jcqKD4VH4T7Zh9Qtt1uRAHJFIEc=; b=0xZGDesVb0zz7UqBz9fmeh3aX7akTSZbiPDhk18av1I9gZ9P/V/dyGtQfMgXUPhvLx JyV0f/Gru7lm/jRADB/k6zAW/bNLu7FRlntCnDyy0KAsCnzQc0TCX+GnDFZBnomuUqNL QLQk7sI5RU9ssjS74JFLBUPqwM3l669NqTZUOGNBd6c8jHiNUJc7BK+usqqKUHxJFp7n EtfJ5zDDTzHkHV+yOuCRX6llmQ1VC27ZVCIyP16pM6fvN1IFVzVbT0D+grFQAlCEXLO8 SqpfYI44cX1Vq0huAS4yHHhGXZRzsnlFHa8Qxs18QIJJFnb2TdUC4wfbcC8AE2QRbDdD H/yA== X-Gm-Message-State: AOAM5312XmK1Rm0SA7IITLUNwxU7Y3eiEqimwJ4ukP9e+KxE6MJNnnIK fadfEDTwIR5tkBW85QDSE5I/4g== X-Google-Smtp-Source: ABdhPJzMJAM+eeL8iG6gA7ZnPCvzmqPpJ5+yz3svE3zUcjnhuqNEw12iRLuS8DMn58OZkLHMPWr1lQ== X-Received: by 2002:ad4:5f8a:: with SMTP id jp10mr3137887qvb.50.1644445653353; Wed, 09 Feb 2022 14:27:33 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:32 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 2/7] libbpf: Expose bpf_core_{add,free}_cands() to bpftool Date: Wed, 9 Feb 2022 17:26:41 -0500 Message-Id: <20220209222646.348365-3-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Expose bpf_core_add_cands() and bpf_core_free_cands() to handle candidates list. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato Acked-by: Andrii Nakryiko --- tools/lib/bpf/libbpf.c | 17 ++++++++++------- tools/lib/bpf/libbpf_internal.h | 9 +++++++++ 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index d3c457fb045e..ad43b6ce825e 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -5192,18 +5192,21 @@ size_t bpf_core_essential_name_len(const char *name) return n; } -static void bpf_core_free_cands(struct bpf_core_cand_list *cands) +void bpf_core_free_cands(struct bpf_core_cand_list *cands) { + if (!cands) + return; + free(cands->cands); free(cands); } -static int bpf_core_add_cands(struct bpf_core_cand *local_cand, - size_t local_essent_len, - const struct btf *targ_btf, - const char *targ_btf_name, - int targ_start_id, - struct bpf_core_cand_list *cands) +int bpf_core_add_cands(struct bpf_core_cand *local_cand, + size_t local_essent_len, + const struct btf *targ_btf, + const char *targ_btf_name, + int targ_start_id, + struct bpf_core_cand_list *cands) { struct bpf_core_cand *new_cands, *cand; const struct btf_type *t, *local_t; diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index bc86b82e90d1..4fda8bdf0a0d 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -529,4 +529,13 @@ static inline int ensure_good_fd(int fd) return fd; } +/* The following two functions are exposed to bpftool */ +int bpf_core_add_cands(struct bpf_core_cand *local_cand, + size_t local_essent_len, + const struct btf *targ_btf, + const char *targ_btf_name, + int targ_start_id, + struct bpf_core_cand_list *cands); +void bpf_core_free_cands(struct bpf_core_cand_list *cands); + #endif /* __LIBBPF_LIBBPF_INTERNAL_H */ From patchwork Wed Feb 9 22:26:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12740985 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E73DC433F5 for ; Wed, 9 Feb 2022 22:28:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235594AbiBIW22 (ORCPT ); Wed, 9 Feb 2022 17:28:28 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:59316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235622AbiBIW2Z (ORCPT ); Wed, 9 Feb 2022 17:28:25 -0500 Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A0E1E019D0D for ; Wed, 9 Feb 2022 14:27:38 -0800 (PST) Received: by mail-qv1-xf31.google.com with SMTP id k4so3181597qvt.6 for ; Wed, 09 Feb 2022 14:27:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AoeSGIFKDFsOKzYaQ8dlCFXs0nV+HPcZ36Zoh0gvrgQ=; b=E0eaEQdwhB1Vtot4JvTCkatoDq+T/f1GBEEfFesuwakvLwHrThe8f2XcPck+GbLoI5 oSEKuWCH/N4b2I6NxAyfYn1XYgIKKVaLZIds8p7R5/CaOL3HT/APZ+hbCJmbLZua2wYD WO4i8NcDitTLn+DDVF3HtUd732VY4qaxcyVDY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AoeSGIFKDFsOKzYaQ8dlCFXs0nV+HPcZ36Zoh0gvrgQ=; b=kUXVCIHlbL4AgJ3PWEMMcunsW9vn6i9nFzAuHAqUrWRTiEO7gQDz/5Iz6Ix1BYwSwA q12JzuGN6tzDcEiYtKBz96D6RcfBQHenP6ewXzENXrqMI0CwOocdDdgsrGoqJmIZNqHZ +RcFv/BeXz6z2+gV+XQMu6k2Vo+gVXuQJEiExi3wB7GDrFPiE+Zn3vwpM+QJ8w/RXroj XoOtgHHY1cyEWLjgNYwHDZAgvUZU/Lap20HD47lub0cfWBaWZOSENJKfwXL0R+GNNu2M TXQfOOQ98FQaQEET49yPWQkNKTccJxoVfBrHb94+/5vZGZ3XqZG1yFzZe4+j/kqssI31 1GVA== X-Gm-Message-State: AOAM533unMqPByDzIqdlzR2SIU5HmTOKUNR1sTSlxkNEXxOhcl4oHs1c GzET2KelzsJLudsn3RenuCo+W2PyV6v/3dn2MpcoDqfGq5RkK211URIAn1/xBFpBIYNg2ZXW8l7 78cWf9o2Wt1AzEN9FaUsjI+/qh/EAHIFTAduUnjTDYbKIvPfPVUdL1Orb/tSKUphzWs7kEg== X-Google-Smtp-Source: ABdhPJw8SBdHOJkyNJaDUzMw/zo5rlaJrPdABrCrS/DcEaHac8sQbuVu+jHHPrLQUVkB9Tx1wNFH5w== X-Received: by 2002:a05:6214:2a85:: with SMTP id jr5mr2932956qvb.3.1644445656529; Wed, 09 Feb 2022 14:27:36 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:36 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 3/7] bpftool: Add gen min_core_btf command Date: Wed, 9 Feb 2022 17:26:42 -0500 Message-Id: <20220209222646.348365-4-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This command is implemented under the "gen" command in bpftool and the syntax is the following: $ bpftool gen min_core_btf INPUT OUTPUT OBJECT [OBJECT...] INPUT is the file that contains all the BTF types for a kernel and OUTPUT is the path of the minimize BTF file that will be created with only the types needed by the objects. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato Acked-by: Andrii Nakryiko --- tools/bpf/bpftool/bash-completion/bpftool | 6 +++- tools/bpf/bpftool/gen.c | 42 +++++++++++++++++++++-- 2 files changed, 44 insertions(+), 4 deletions(-) diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index 493753a4962e..958e1fd71b5c 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -1003,9 +1003,13 @@ _bpftool() ;; esac ;; + min_core_btf) + _filedir + return 0 + ;; *) [[ $prev == $object ]] && \ - COMPREPLY=( $( compgen -W 'object skeleton help' -- "$cur" ) ) + COMPREPLY=( $( compgen -W 'object skeleton help min_core_btf' -- "$cur" ) ) ;; esac ;; diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index eacfc6a2060d..582c20602639 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -1087,6 +1087,7 @@ static int do_help(int argc, char **argv) fprintf(stderr, "Usage: %1$s %2$s object OUTPUT_FILE INPUT_FILE [INPUT_FILE...]\n" " %1$s %2$s skeleton FILE [name OBJECT_NAME]\n" + " %1$s %2$s min_core_btf INPUT OUTPUT OBJECT [OBJECT...]\n" " %1$s %2$s help\n" "\n" " " HELP_SPEC_OPTIONS " |\n" @@ -1097,10 +1098,45 @@ static int do_help(int argc, char **argv) return 0; } +/* Create minimized BTF file for a set of BPF objects */ +static int minimize_btf(const char *src_btf, const char *dst_btf, const char *objspaths[]) +{ + return -EOPNOTSUPP; +} + +static int do_min_core_btf(int argc, char **argv) +{ + const char *input, *output, **objs; + int i, err; + + if (!REQ_ARGS(3)) { + usage(); + return -1; + } + + input = GET_ARG(); + output = GET_ARG(); + + objs = (const char **) calloc(argc + 1, sizeof(*objs)); + if (!objs) { + p_err("failed to allocate array for object names"); + return -ENOMEM; + } + + i = 0; + while (argc) + objs[i++] = GET_ARG(); + + err = minimize_btf(input, output, objs); + free(objs); + return err; +} + static const struct cmd cmds[] = { - { "object", do_object }, - { "skeleton", do_skeleton }, - { "help", do_help }, + { "object", do_object }, + { "skeleton", do_skeleton }, + { "min_core_btf", do_min_core_btf}, + { "help", do_help }, { 0 } }; From patchwork Wed Feb 9 22:26:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12740990 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36224C43219 for ; Wed, 9 Feb 2022 22:28:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235529AbiBIW23 (ORCPT ); Wed, 9 Feb 2022 17:28:29 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:58404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235565AbiBIW2Z (ORCPT ); Wed, 9 Feb 2022 17:28:25 -0500 Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2048BE01A2CA for ; Wed, 9 Feb 2022 14:27:42 -0800 (PST) Received: by mail-qt1-x835.google.com with SMTP id g4so2693034qto.1 for ; Wed, 09 Feb 2022 14:27:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OEb+73Bs6DjNMKvBc173Y+LLF0SX/J9C9ef6SZv7LKQ=; b=erSbulTv2zjtNfGH/YuDolJrQotc9kCxo2KwujX5s6aM9xAfeB1ErPRTv+Kn0G2n43 dy7THUoVPczgWdRzXZMCRiC4CgL2gmcb0PJdQ9mwUmCn8DuFMH1nRwnJ2UPT1jdq9i7v FYfS0dQrYOyC8RHWCz2gFprbllTjhDDnly624= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OEb+73Bs6DjNMKvBc173Y+LLF0SX/J9C9ef6SZv7LKQ=; b=7kboTW5IgouOHjRfQ8ZLk3e8VHOZXMAqA6Y0armCqiH+vM9XveEu02i1KDMD43/EgV EXpblDgWfNuOMv6KfAjyxfCruBK5lhepSbBKFJ6r/RGyxk4NpTVzkAvXDXH3mbGqXqgU h4sShk2JXThGBm2P69qIXO7GKAKRNQWIsQh8t9WGuvkbGTS1cwz7p4MGzpOCMCXG8sqD TuW+kymykM9lZcqvr5HHl9I2SpbQMOOWmJ1+7u4ERw+i8I7U7KMSU0v88GAmxGXvHF2z liepqXLAcoXQNWF7ibLkKRmpnDM4FScZFTSJuexs0YV9PPz7UWQGbb/H4EyQE9lRjQ2G S0yA== X-Gm-Message-State: AOAM533+mo6v8fgGKtJAoDP15tMRXOaBZ25huv1DfTIyPiDDDFlfiaNH WChuehok68PxOm25Y9lLvk11tw== X-Google-Smtp-Source: ABdhPJzPnfe4p6aUzPkEJIhe7weCCLDVpQitMFxylqlpeS6gqAMENjUNF/LkmVVULWl0ZAOYHcBk7A== X-Received: by 2002:ac8:7e8f:: with SMTP id w15mr2950124qtj.314.1644445659062; Wed, 09 Feb 2022 14:27:39 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:38 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 4/7] bpftool: Implement minimize_btf() and relocations recording for BTFGen Date: Wed, 9 Feb 2022 17:26:43 -0500 Message-Id: <20220209222646.348365-5-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net minimize_btf() receives the path of a source and destination BTF files and a list of BPF objects. This function records the relocations for all objects and then generates the BTF file by calling btfgen_get_btf() (implemented in the following commit). btfgen_record_obj() loads the BTF and BTF.ext sections of the BPF objects and loops through all CO-RE relocations. It uses bpf_core_calc_relo_insn() from libbpf and passes the target spec to btfgen_record_reloc(), that calls one of the following functions depending on the relocation kind. btfgen_record_field_relo() uses the target specification to mark all the types that are involved in a field-based CO-RE relocation. In this case types resolved and marked recursively using btfgen_mark_type(). Only the struct and union members (and their types) involved in the relocation are marked to optimize the size of the generated BTF file. btfgen_record_type_relo() marks the types involved in a type-based CO-RE relocation. In this case no members for the struct and union types are marked as libbpf doesn't use them while performing this kind of relocation. Pointed types are marked as they are used by libbpf in this case. btfgen_record_enumval_relo() marks the whole enum type for enum-based relocations. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato --- tools/bpf/bpftool/Makefile | 8 +- tools/bpf/bpftool/gen.c | 452 ++++++++++++++++++++++++++++++++++++- 2 files changed, 454 insertions(+), 6 deletions(-) diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile index 83369f55df61..97d447135536 100644 --- a/tools/bpf/bpftool/Makefile +++ b/tools/bpf/bpftool/Makefile @@ -34,10 +34,10 @@ LIBBPF_BOOTSTRAP_INCLUDE := $(LIBBPF_BOOTSTRAP_DESTDIR)/include LIBBPF_BOOTSTRAP_HDRS_DIR := $(LIBBPF_BOOTSTRAP_INCLUDE)/bpf LIBBPF_BOOTSTRAP := $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a -# We need to copy hashmap.h and nlattr.h which is not otherwise exported by -# libbpf, but still required by bpftool. -LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h) -LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h) +# We need to copy hashmap.h, nlattr.h, relo_core.h and libbpf_internal.h +# which are not otherwise exported by libbpf, but still required by bpftool. +LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h relo_core.h libbpf_internal.h) +LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h relo_core.h libbpf_internal.h) ifeq ($(BPFTOOL_VERSION),) BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index 582c20602639..c3e34db2ec8a 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -1098,10 +1099,457 @@ static int do_help(int argc, char **argv) return 0; } -/* Create minimized BTF file for a set of BPF objects */ +static int btf_save_raw(const struct btf *btf, const char *path) +{ + const void *data; + FILE *f = NULL; + __u32 data_sz; + int err = 0; + + data = btf__raw_data(btf, &data_sz); + if (!data) + return -ENOMEM; + + f = fopen(path, "wb"); + if (!f) + return -errno; + + if (fwrite(data, 1, data_sz, f) != data_sz) + err = -errno; + + fclose(f); + return err; +} + +struct btfgen_info { + struct btf *src_btf; + struct btf *marked_btf; // btf structure used to mark used types +}; + +static size_t btfgen_hash_fn(const void *key, void *ctx) +{ + return (size_t)key; +} + +static bool btfgen_equal_fn(const void *k1, const void *k2, void *ctx) +{ + return k1 == k2; +} + +static void *uint_as_hash_key(int x) +{ + return (void *)(uintptr_t)x; +} + +static void *u32_as_hash_key(__u32 x) +{ + return (void *)(uintptr_t)x; +} + +static void btfgen_free_info(struct btfgen_info *info) +{ + if (!info) + return; + + btf__free(info->src_btf); + btf__free(info->marked_btf); + + free(info); +} + +static struct btfgen_info * +btfgen_new_info(const char *targ_btf_path) +{ + struct btfgen_info *info; + int err; + + info = calloc(1, sizeof(*info)); + if (!info) + return NULL; + + info->src_btf = btf__parse(targ_btf_path, NULL); + if (!info->src_btf) { + p_err("failed parsing '%s' BTF file: %s", targ_btf_path, strerror(errno)); + err = -errno; + goto err_out; + } + + info->marked_btf = btf__parse(targ_btf_path, NULL); + if (!info->marked_btf) { + p_err("failed parsing '%s' BTF file: %s", targ_btf_path, strerror(errno)); + err = -errno; + goto err_out; + } + + return info; + +err_out: + btfgen_free_info(info); + errno = -err; + return NULL; +} + +#define MARKED UINT32_MAX + +static void btfgen_mark_member(struct btfgen_info *info, int type_id, int idx) +{ + const struct btf_type *t = btf__type_by_id(info->marked_btf, type_id); + struct btf_member *m = btf_members(t) + idx; + + m->name_off = MARKED; +} + +static int +btfgen_mark_type(struct btfgen_info *info, unsigned int id, bool follow_pointers) +{ + const struct btf_type *btf_type = btf__type_by_id(info->src_btf, id); + struct btf_type *cloned_type; + struct btf_param *param; + struct btf_array *array; + int err, i; + + if (id == 0) + return 0; + + /* mark type on cloned BTF as used */ + cloned_type = (struct btf_type *) btf__type_by_id(info->marked_btf, id); + cloned_type->name_off = MARKED; + + /* recursively mark other types needed by it */ + switch (btf_kind(btf_type)) { + case BTF_KIND_UNKN: + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + break; + case BTF_KIND_PTR: + if (follow_pointers) { + err = btfgen_mark_type(info, btf_type->type, follow_pointers); + if (err) + return err; + } + break; + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_TYPEDEF: + err = btfgen_mark_type(info, btf_type->type, follow_pointers); + if (err) + return err; + break; + case BTF_KIND_ARRAY: + array = btf_array(btf_type); + + /* mark array type */ + err = btfgen_mark_type(info, array->type, follow_pointers); + /* mark array's index type */ + err = err ? : btfgen_mark_type(info, array->index_type, follow_pointers); + if (err) + return err; + break; + case BTF_KIND_FUNC_PROTO: + /* mark ret type */ + err = btfgen_mark_type(info, btf_type->type, follow_pointers); + if (err) + return err; + + /* mark parameters types */ + param = btf_params(btf_type); + for (i = 0; i < btf_vlen(btf_type); i++) { + err = btfgen_mark_type(info, param->type, follow_pointers); + if (err) + return err; + param++; + } + break; + /* tells if some other type needs to be handled */ + default: + p_err("unsupported kind: %s (%d)", btf_kind_str(btf_type), id); + return -EINVAL; + } + + return 0; +} + +static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec) +{ + struct btf *btf = (struct btf *) info->src_btf; + const struct btf_type *btf_type; + struct btf_member *btf_member; + struct btf_array *array; + unsigned int id = targ_spec->root_type_id; + int idx, err; + + /* mark root type */ + btf_type = btf__type_by_id(btf, id); + err = btfgen_mark_type(info, id, false); + if (err) + return err; + + /* mark types for complex types (arrays, unions, structures) */ + for (int i = 1; i < targ_spec->raw_len; i++) { + /* skip typedefs and mods */ + while (btf_is_mod(btf_type) || btf_is_typedef(btf_type)) { + id = btf_type->type; + btf_type = btf__type_by_id(btf, id); + } + + switch (btf_kind(btf_type)) { + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + idx = targ_spec->raw_spec[i]; + btf_member = btf_members(btf_type) + idx; + + /* mark member */ + btfgen_mark_member(info, id, idx); + + /* mark member's type */ + id = btf_member->type; + btf_type = btf__type_by_id(btf, id); + err = btfgen_mark_type(info, id, false); + if (err) + return err; + break; + case BTF_KIND_ARRAY: + array = btf_array(btf_type); + id = array->type; + btf_type = btf__type_by_id(btf, id); + break; + default: + p_err("unsupported kind: %s (%d)", + btf_kind_str(btf_type), btf_type->type); + return -EINVAL; + } + } + + return 0; +} + +static int btfgen_record_type_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec) +{ + return btfgen_mark_type(info, targ_spec->root_type_id, true); +} + +static int btfgen_record_enumval_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec) +{ + return btfgen_mark_type(info, targ_spec->root_type_id, false); +} + +static int btfgen_record_reloc(struct btfgen_info *info, struct bpf_core_spec *res) +{ + switch (res->relo_kind) { + case BPF_CORE_FIELD_BYTE_OFFSET: + case BPF_CORE_FIELD_BYTE_SIZE: + case BPF_CORE_FIELD_EXISTS: + case BPF_CORE_FIELD_SIGNED: + case BPF_CORE_FIELD_LSHIFT_U64: + case BPF_CORE_FIELD_RSHIFT_U64: + return btfgen_record_field_relo(info, res); + case BPF_CORE_TYPE_ID_LOCAL: /* BPF_CORE_TYPE_ID_LOCAL doesn't require kernel BTF */ + return 0; + case BPF_CORE_TYPE_ID_TARGET: + case BPF_CORE_TYPE_EXISTS: + case BPF_CORE_TYPE_SIZE: + return btfgen_record_type_relo(info, res); + case BPF_CORE_ENUMVAL_EXISTS: + case BPF_CORE_ENUMVAL_VALUE: + return btfgen_record_enumval_relo(info, res); + default: + return -EINVAL; + } +} + +static struct bpf_core_cand_list * +btfgen_find_cands(const struct btf *local_btf, const struct btf *targ_btf, __u32 local_id) +{ + const struct btf_type *local_type; + struct bpf_core_cand_list *cands = NULL; + struct bpf_core_cand local_cand = {}; + size_t local_essent_len; + const char *local_name; + int err; + + local_cand.btf = local_btf; + local_cand.id = local_id; + + local_type = btf__type_by_id(local_btf, local_id); + if (!local_type) { + err = -EINVAL; + goto err_out; + } + + local_name = btf__name_by_offset(local_btf, local_type->name_off); + if (!local_name) { + err = -EINVAL; + goto err_out; + } + local_essent_len = bpf_core_essential_name_len(local_name); + + cands = calloc(1, sizeof(*cands)); + if (!cands) + return NULL; + + err = bpf_core_add_cands(&local_cand, local_essent_len, targ_btf, "vmlinux", 1, cands); + if (err) + goto err_out; + + return cands; + +err_out: + bpf_core_free_cands(cands); + errno = -err; + return NULL; +} + +/* Record relocation information for a single BPF object*/ +static int btfgen_record_obj(struct btfgen_info *info, const char *obj_path) +{ + const struct btf_ext_info_sec *sec; + const struct bpf_core_relo *relo; + const struct btf_ext_info *seg; + struct hashmap_entry *entry; + struct hashmap *cand_cache = NULL; + struct btf_ext *btf_ext = NULL; + unsigned int relo_idx; + struct btf *btf = NULL; + size_t i; + int err; + + btf = btf__parse(obj_path, &btf_ext); + if (!btf) { + p_err("failed to parse BPF object '%s': %s", obj_path, strerror(errno)); + return -errno; + } + + if (btf_ext->core_relo_info.len == 0) { + err = 0; + goto out; + } + + cand_cache = hashmap__new(btfgen_hash_fn, btfgen_equal_fn, NULL); + if (IS_ERR(cand_cache)) { + err = PTR_ERR(cand_cache); + goto out; + } + + seg = &btf_ext->core_relo_info; + for_each_btf_ext_sec(seg, sec) { + for_each_btf_ext_rec(seg, sec, relo_idx, relo) { + struct bpf_core_spec specs_scratch[3] = {}; + struct bpf_core_relo_res targ_res = {}; + struct bpf_core_cand_list *cands = NULL; + const void *type_key = u32_as_hash_key(relo->type_id); + const char *sec_name = btf__name_by_offset(btf, sec->sec_name_off); + + if (relo->kind != BPF_CORE_TYPE_ID_LOCAL && + !hashmap__find(cand_cache, type_key, (void **)&cands)) { + cands = btfgen_find_cands(btf, info->src_btf, relo->type_id); + if (!cands) { + err = -errno; + goto out; + } + + err = hashmap__set(cand_cache, type_key, cands, NULL, NULL); + if (err) + goto out; + } + + err = bpf_core_calc_relo_insn(sec_name, relo, relo_idx, btf, cands, + specs_scratch, &targ_res); + if (err) + goto out; + + /* specs_scratch[2] is the target spec */ + err = btfgen_record_reloc(info, &specs_scratch[2]); + if (err) + goto out; + } + } + +out: + btf__free(btf); + btf_ext__free(btf_ext); + + if (!IS_ERR_OR_NULL(cand_cache)) { + hashmap__for_each_entry(cand_cache, entry, i) { + bpf_core_free_cands(entry->value); + } + hashmap__free(cand_cache); + } + + return err; +} + +/* Generate BTF from relocation information previously recorded */ +static struct btf *btfgen_get_btf(struct btfgen_info *info) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +/* Create minimized BTF file for a set of BPF objects. + * + * The BTFGen algorithm is divided in two main parts: (1) collect the + * BTF types that are involved in relocations and (2) generate the BTF + * object using the collected types. + * + * In order to collect the types involved in the relocations, we parse + * the BTF and BTF.ext sections of the BPF objects and use + * bpf_core_calc_relo_insn() to get the target specification, this + * indicates how the types and fields are used in a relocation. + * + * Types are recorded in different ways according to the kind of the + * relocation. For field-based relocations only the members that are + * actually used are saved in order to reduce the size of the generated + * BTF file. For type-based relocations empty struct / unions are + * generated and for enum-based relocations the whole type is saved. + * + * The second part of the algorithm generates the BTF object. It creates + * an empty BTF object and fills it with the types recorded in the + * previous step. This function takes care of only adding the structure + * and union members that were marked as used and it also fixes up the + * type IDs on the generated BTF object. + */ static int minimize_btf(const char *src_btf, const char *dst_btf, const char *objspaths[]) { - return -EOPNOTSUPP; + struct btfgen_info *info; + struct btf *btf_new = NULL; + int err, i; + + info = btfgen_new_info(src_btf); + if (!info) { + p_err("failed to allocate info structure: %s", strerror(errno)); + err = -errno; + goto out; + } + + for (i = 0; objspaths[i] != NULL; i++) { + err = btfgen_record_obj(info, objspaths[i]); + if (err) { + p_err("error recording relocations for %s: %s", objspaths[i], + strerror(errno)); + goto out; + } + } + + btf_new = btfgen_get_btf(info); + if (!btf_new) { + err = -errno; + p_err("error generating BTF: %s", strerror(errno)); + goto out; + } + + err = btf_save_raw(btf_new, dst_btf); + if (err) { + p_err("error saving btf file: %s", strerror(errno)); + goto out; + } + +out: + btf__free(btf_new); + btfgen_free_info(info); + + return err; } static int do_min_core_btf(int argc, char **argv) From patchwork Wed Feb 9 22:26:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12740988 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41FF3C433F5 for ; Wed, 9 Feb 2022 22:28:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235673AbiBIW21 (ORCPT ); Wed, 9 Feb 2022 17:28:27 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:57960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235491AbiBIW2Z (ORCPT ); Wed, 9 Feb 2022 17:28:25 -0500 Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1320AE01A237 for ; Wed, 9 Feb 2022 14:27:43 -0800 (PST) Received: by mail-qt1-x834.google.com with SMTP id s1so3321043qtw.9 for ; Wed, 09 Feb 2022 14:27:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xcPa28Cy25WoANEeaXLZrreqoOMkyJYWDgWdVWp2OFg=; b=KRYe4me4LxBtrqLehSrA1W8X+pX/NYEM2eLB9UMXoDlMnAWiHJR5cukTm9PvwH3IDs 4jHnDxD0xvkIfX+kpGxKS6MHudso0USaN63XSB1acRYnpDiw+Ac714BUhWlGdDP6ix+O FE4uYcSb1tEksiobfBXnhhIVm2tgh+qp+4eUA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xcPa28Cy25WoANEeaXLZrreqoOMkyJYWDgWdVWp2OFg=; b=3jG6XHQQwWI4ZqVd4JB8rKORyfDejUv57i4Cwe+7f88PMBrC3GkzAgcrmoAw1+b/CK rHZIpxmWG2avBurhuWKkb3Vwaq94rIRE8DqwAPrdlRGw4l147bzzpFP88Lq6gxQS6RX8 XmjBkf5UTNg7lBj5F7+Fx2uPVRQ7ia6N2dsQ/8zx22cEASMIhEiHR/eBA4nJk91VPi1H yrKAOxORzYDsBZYQbtjhtF8xzN4yMVlTbrrgqFUAHdNh59SNfaQn08XVI6vc2xx11H9h 0dBOFXUk2bb9DHuUh6VwbrnI/4066mPqAkNVL+LJWxBzQqIgI2dbslE/1EyDx4SpVwau 3nVA== X-Gm-Message-State: AOAM532J56HvXuAkqunIRh5an4WIS6vh5nxK4FqvghmICt26ZayVWliQ EkR7kEVIpXVu/Ds3FX8rw7PS+g== X-Google-Smtp-Source: ABdhPJz1rAv6qaa1pM9fKMBVr5EVj2gP9AjXVC36XcmNwn56/9SBP7cyzQe4Sj0KTrFe7c4KGxjeWw== X-Received: by 2002:ac8:41cf:: with SMTP id o15mr2938178qtm.254.1644445661960; Wed, 09 Feb 2022 14:27:41 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:41 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 5/7] bpftool: Implement btfgen_get_btf() Date: Wed, 9 Feb 2022 17:26:44 -0500 Message-Id: <20220209222646.348365-6-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The last part of the BTFGen algorithm is to create a new BTF object with all the types that were recorded in the previous steps. This function performs two different steps: 1. Add the types to the new BTF object by using btf__add_type(). Some special logic around struct and unions is implemented to only add the members that are really used in the field-based relocations. The type ID on the new and old BTF objects is stored on a map. 2. Fix all the type IDs on the new BTF object by using the IDs saved in the previous step. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato --- tools/bpf/bpftool/gen.c | 136 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 135 insertions(+), 1 deletion(-) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index c3e34db2ec8a..1efc7f3c64b2 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -1481,10 +1481,144 @@ static int btfgen_record_obj(struct btfgen_info *info, const char *obj_path) return err; } +static unsigned int btfgen_get_id(struct hashmap *ids, unsigned int old) +{ + uintptr_t new; + + if (!hashmap__find(ids, uint_as_hash_key(old), (void **)&new)) + /* return id for BTF_KIND_VOID as it's possible that the + * ID we're looking for is the type of a pointer that + * we're not adding. + */ + return 0; + + return (unsigned int)(uintptr_t)new; +} + +static int btfgen_add_id(struct hashmap *ids, unsigned int old, unsigned int new) +{ + return hashmap__add(ids, uint_as_hash_key(old), uint_as_hash_key(new)); +} + +static int btfgen_remap_id(__u32 *type_id, void *ctx) +{ + struct hashmap *ids = ctx; + + *type_id = btfgen_get_id(ids, *type_id); + + return 0; +} + /* Generate BTF from relocation information previously recorded */ static struct btf *btfgen_get_btf(struct btfgen_info *info) { - return ERR_PTR(-EOPNOTSUPP); + struct btf *btf_new = NULL; + struct hashmap *ids = NULL; + unsigned int i; + int err = 0; + + btf_new = btf__new_empty(); + if (!btf_new) { + err = -errno; + goto err_out; + } + + ids = hashmap__new(btfgen_hash_fn, btfgen_equal_fn, NULL); + if (IS_ERR(ids)) { + err = PTR_ERR(ids); + goto err_out; + } + + /* first pass: add all marked types to btf_new and add their new ids to the ids map */ + for (i = 1; i < btf__type_cnt(info->marked_btf); i++) { + const struct btf_type *cloned_type, *btf_type; + int new_id; + + cloned_type = btf__type_by_id(info->marked_btf, i); + + if (cloned_type->name_off != MARKED) + continue; + + btf_type = btf__type_by_id(info->src_btf, i); + + /* add members for struct and union */ + if (btf_is_struct(btf_type) || btf_is_union(btf_type)) { + struct btf_type *btf_type_cpy; + int nmembers = 0, idx_dst, idx_src; + size_t new_size; + + /* calculate nmembers */ + for (idx_src = 0; idx_src < btf_vlen(cloned_type); idx_src++) { + struct btf_member *cloned_m = btf_members(cloned_type) + idx_src; + + if (cloned_m->name_off == MARKED) + nmembers++; + } + + new_size = sizeof(struct btf_type) + nmembers * sizeof(struct btf_member); + + btf_type_cpy = malloc(new_size); + if (!btf_type_cpy) + goto err_out; + + /* copy btf type */ + *btf_type_cpy = *btf_type; + + idx_dst = 0; + for (idx_src = 0; idx_src < btf_vlen(cloned_type); idx_src++) { + struct btf_member *btf_member_src, *btf_member_dst; + struct btf_member *cloned_m = btf_members(cloned_type) + idx_src; + + /* copy only members that are marked as used */ + if (cloned_m->name_off != MARKED) + continue; + + btf_member_src = btf_members(btf_type) + idx_src; + btf_member_dst = btf_members(btf_type_cpy) + idx_dst; + + *btf_member_dst = *btf_member_src; + + idx_dst++; + } + + /* set new vlen */ + btf_type_cpy->info = btf_type_info(btf_kind(btf_type_cpy), nmembers, + btf_kflag(btf_type_cpy)); + + err = btf__add_type(btf_new, info->src_btf, btf_type_cpy); + free(btf_type_cpy); + } else { + err = btf__add_type(btf_new, info->src_btf, btf_type); + } + + if (err < 0) + goto err_out; + + new_id = err; + + /* add ID mapping */ + err = btfgen_add_id(ids, i, new_id); + if (err) + goto err_out; + } + + /* second pass: fix up type ids */ + for (i = 1; i < btf__type_cnt(btf_new); i++) { + struct btf_type *btf_type = (struct btf_type *) btf__type_by_id(btf_new, i); + + err = btf_type_visit_type_ids(btf_type, btfgen_remap_id, ids); + if (err) + goto err_out; + } + + hashmap__free(ids); + return btf_new; + +err_out: + btf__free(btf_new); + hashmap__free(ids); + errno = -err; + return NULL; } /* Create minimized BTF file for a set of BPF objects. From patchwork Wed Feb 9 22:26:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12741018 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65616C433EF for ; Wed, 9 Feb 2022 22:28:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235738AbiBIW2n (ORCPT ); Wed, 9 Feb 2022 17:28:43 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:58812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235627AbiBIW2Z (ORCPT ); Wed, 9 Feb 2022 17:28:25 -0500 Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8443DE00F7E0 for ; Wed, 9 Feb 2022 14:27:47 -0800 (PST) Received: by mail-qt1-x82e.google.com with SMTP id p14so3391519qtx.0 for ; Wed, 09 Feb 2022 14:27:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+q//gu5C97YlNqxm+yauLZBIzY4Rq1ySJmMuhxJxuLQ=; b=ZhTHc4OvqnTHOkc1wWQ7H6pL7ziN0Fmalr2jn/AfiZfzoAG1cJyesWUj3PLgQlAJBj RYR9DTO1bo0CwXZO2vTDf9kQLQAPLL4pYwU9iR4LRmT8/2SLXNxZDnsKhbuXst6+WeHw C2dHPX0VpAf+HfK1CEXsKyiTz+q2jNKc+OfKM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+q//gu5C97YlNqxm+yauLZBIzY4Rq1ySJmMuhxJxuLQ=; b=BE/THZJpBOSoxxn6Ss9uZl9lYHnpJl4lbpvZmd3doj1AuhWzDE1So9qR7qR/JFH10t FLTCrWiUMnBq+RaEXXHNiJ5Uk5RgWxzcA3RFMeDPJJv/O8yuEp3Q4vI+nJ5mdWq4UMAq JkR0d0gH09ClS5DEsi9dp319ECDGFj/yLPI/Al6W1xyawjfLSwsx18YWLq9CS48/GOMg VwWAr+OfkDmIcbdslfqztPaMdbbJFnnXsDg+L0rxDWK8ju6TlhJe5+G6UWqwpLojrzyN dR1zYWSwmYhbgNj0AHzg0Fu+azmtqHNovsiYnuYwtnhiPIn8b0NHKupCoUTL8vPdXCd3 GGdw== X-Gm-Message-State: AOAM532XAFSNB8ZWHv6awmxBqaI9xzAL74UQOL97Q4AexHq6lnBCeMH0 fO77JeEfuZPOAFCwMPd13bWFrMxV5E6d4GoFn0YdnseuDVsg0rcs5m/nXhNWMiQ3/s1hiFD4OuT sB7q1QpjDLBPQzghNuWQcCA4gFxlAmjPD8zB6bmeyOtDOVDJ/DAAWYZ2LlRtzbNfG2Vh8MQ== X-Google-Smtp-Source: ABdhPJwuXVg7QqZBlNWf3V3ppilboCOrV8vZrYOM7cEw/+UR9trVNB6OKEswgJp10Rv7JeqQFavk3A== X-Received: by 2002:ac8:7d4d:: with SMTP id h13mr2996911qtb.489.1644445663991; Wed, 09 Feb 2022 14:27:43 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:43 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 6/7] bpftool: gen min_core_btf explanation and examples Date: Wed, 9 Feb 2022 17:26:45 -0500 Message-Id: <20220209222646.348365-7-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Rafael David Tinoco Add "min_core_btf" feature explanation and one example of how to use it to bpftool-gen man page. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato --- .../bpf/bpftool/Documentation/bpftool-gen.rst | 93 +++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/tools/bpf/bpftool/Documentation/bpftool-gen.rst b/tools/bpf/bpftool/Documentation/bpftool-gen.rst index bc276388f432..4e654b16dfd0 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-gen.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-gen.rst @@ -25,6 +25,7 @@ GEN COMMANDS | **bpftool** **gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...] | **bpftool** **gen skeleton** *FILE* [**name** *OBJECT_NAME*] +| **bpftool** **gen min_core_btf** *INPUT* *OUTPUT* *OBJECT* [*OBJECT*...] | **bpftool** **gen help** DESCRIPTION @@ -149,6 +150,26 @@ DESCRIPTION (non-read-only) data from userspace, with same simplicity as for BPF side. + **bpftool** **gen min_core_btf** *INPUT* *OUTPUT* *OBJECT* [*OBJECT*...] + Generate a minimum BTF file as *OUTPUT*, derived from a given + *INPUT* BTF file, containing all needed BTF types so one, or + more, given eBPF objects CO-RE relocations may be satisfied. + + When kernels aren't compiled with CONFIG_DEBUG_INFO_BTF, + libbpf, when loading an eBPF object, has to rely in external + BTF files to be able to calculate CO-RE relocations. + + Usually, an external BTF file is built from existing kernel + DWARF data using pahole. It contains all the types used by + its respective kernel image and, because of that, is big. + + The min_core_btf feature builds smaller BTF files, customized + to one or multiple eBPF objects, so they can be distributed + together with an eBPF CO-RE based application, turning the + application portable to different kernel versions. + + Check examples bellow for more information how to use it. + **bpftool gen help** Print short help message. @@ -215,7 +236,9 @@ This is example BPF application with two BPF programs and a mix of BPF maps and global variables. Source code is split across two source code files. **$ clang -target bpf -g example1.bpf.c -o example1.bpf.o** + **$ clang -target bpf -g example2.bpf.c -o example2.bpf.o** + **$ bpftool gen object example.bpf.o example1.bpf.o example2.bpf.o** This set of commands compiles *example1.bpf.c* and *example2.bpf.c* @@ -329,3 +352,73 @@ BPF ELF object file *example.bpf.o*. my_static_var: 7 This is a stripped-out version of skeleton generated for above example code. + +min_core_btf +------------ + +**$ bpftool btf dump file ./5.4.0-example.btf format raw** + +:: + + [1] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none) + [2] CONST '(anon)' type_id=1 + [3] VOLATILE '(anon)' type_id=1 + [4] ARRAY '(anon)' type_id=1 index_type_id=21 nr_elems=2 + [5] PTR '(anon)' type_id=8 + [6] CONST '(anon)' type_id=5 + [7] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=(none) + [8] CONST '(anon)' type_id=7 + [9] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none) + + +**$ bpftool btf dump file ./one.bpf.o format raw** + +:: + + [1] PTR '(anon)' type_id=2 + [2] STRUCT 'trace_event_raw_sys_enter' size=64 vlen=4 + 'ent' type_id=3 bits_offset=0 + 'id' type_id=7 bits_offset=64 + 'args' type_id=9 bits_offset=128 + '__data' type_id=12 bits_offset=512 + [3] STRUCT 'trace_entry' size=8 vlen=4 + 'type' type_id=4 bits_offset=0 + 'flags' type_id=5 bits_offset=16 + 'preempt_count' type_id=5 bits_offset=24 + + +**$ bpftool gen min_core_btf ./5.4.0-example.btf ./5.4.0-smaller.btf ./one.bpf.o** + +**$ bpftool btf dump file ./5.4.0-smaller.btf format raw** + +:: + + [1] TYPEDEF 'pid_t' type_id=6 + [2] STRUCT 'trace_event_raw_sys_enter' size=64 vlen=1 + 'args' type_id=4 bits_offset=128 + [3] STRUCT 'task_struct' size=9216 vlen=2 + 'pid' type_id=1 bits_offset=17920 + 'real_parent' type_id=7 bits_offset=18048 + [4] ARRAY '(anon)' type_id=5 index_type_id=8 nr_elems=6 + [5] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none) + [6] TYPEDEF '__kernel_pid_t' type_id=8 + [7] PTR '(anon)' type_id=3 + [8] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED + + +Now, the "5.4.0-smaller.btf" file may be used by libbpf as an external BTF file +when loading the "one.bpf.o" object into the "5.4.0-example" kernel. Note that +the generated BTF file won't allow other eBPF objects to be loaded, just the +ones given to min_core_btf. + +:: + + struct bpf_object *obj = NULL; + struct bpf_object_open_opts openopts = {}; + + openopts.sz = sizeof(struct bpf_object_open_opts); + openopts.btf_custom_path = "./5.4.0-smaller.btf"; + + obj = bpf_object__open_file("./one.bpf.o", &openopts); + + ... From patchwork Wed Feb 9 22:26:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mauricio_V=C3=A1squez?= X-Patchwork-Id: 12740991 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FFD4C4332F for ; Wed, 9 Feb 2022 22:28:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235721AbiBIW2e (ORCPT ); Wed, 9 Feb 2022 17:28:34 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:58630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235601AbiBIW2Z (ORCPT ); Wed, 9 Feb 2022 17:28:25 -0500 Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 670F7E01645B for ; Wed, 9 Feb 2022 14:27:48 -0800 (PST) Received: by mail-qt1-x833.google.com with SMTP id j12so3371510qtr.2 for ; Wed, 09 Feb 2022 14:27:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kTBJ1Pql1z9VTh/Shv6+u258Fz4OzaJBfE1p/mIR/xk=; b=Oa8IeBWnsdLRbrHtaHc1jMJxjRgFqEo1MKfKh9LagsiF3U5gHAy60XoONhXFvgWsQI 90X4XGvbl8g2QrW876tEqXxJ4n0zmnlhL2QLwOdj8Z1aJyeGlO6Hg66uXglovcU2XrBq sJ/7HdVRDgM1oHl2bmKfabRJBCskqzcg3vuTQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kTBJ1Pql1z9VTh/Shv6+u258Fz4OzaJBfE1p/mIR/xk=; b=aWlnqVRLIWqVPKUyi+DK5H+KpicRytbtHrOaHsun6ZxK58jnXLRAM5pYoXw2VMEoVs ccQL8vTyrl2hmVy+YUDGJxPuOVcN0kSuf3jHrTjFLZxausr9jrlD85YKGRbch04tE1Mw /Le8XyF1eX/mQfGu78Tml680sgrHsAtt1jl++V7E5NEqn3Oae8B2qCWRkw6CS3Ix6Vdw PFXiaEcvCUtGFDJv0AGdERF/f/oZJLHS376V0n9VXK3pj6IgGg8B1Jw0KUwj7itaeTMK IHkbSApNoVLnGIj330BxRPz86NvXBusUsFEe0OetySA+QIvWnW30zYWrM8/tLLTRLmVH qwXw== X-Gm-Message-State: AOAM531mefRx6Av6kglhnlW6WqYYE233Ejes8WLrVZt7K6bM0lENG+L8 Rhw9MY2aNN3ia4w+qjdj8JtUAw== X-Google-Smtp-Source: ABdhPJz48N3gIue1a+5AB2HgSeqz3q1RcamQcQmnJaNRN3bI8739fyCxPhGVmU8xRfpsIxd/yTUGBA== X-Received: by 2002:ac8:5bc5:: with SMTP id b5mr2990286qtb.572.1644445666831; Wed, 09 Feb 2022 14:27:46 -0800 (PST) Received: from localhost.localdomain ([181.136.110.101]) by smtp.gmail.com with ESMTPSA id h6sm9706287qtx.65.2022.02.09.14.27.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 14:27:46 -0800 (PST) From: =?utf-8?q?Mauricio_V=C3=A1squez?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Quentin Monnet , Rafael David Tinoco , Lorenzo Fontana , Leonardo Di Donato Subject: [PATCH bpf-next v6 7/7] selftests/bpf: Test "bpftool gen min_core_btf" Date: Wed, 9 Feb 2022 17:26:46 -0500 Message-Id: <20220209222646.348365-8-mauricio@kinvolk.io> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220209222646.348365-1-mauricio@kinvolk.io> References: <20220209222646.348365-1-mauricio@kinvolk.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit reuses the core_reloc test to check if the BTF files generated with "bpftool gen min_core_btf" are correct. This introduces test_core_btfgen() that runs all the core_reloc tests, but this time the source BTF files are generated by using "bpftool gen min_core_btf". The goal of this test is to check that the generated files are usable, and not to check if the algorithm is creating an optimized BTF file. Signed-off-by: Mauricio Vásquez Signed-off-by: Rafael David Tinoco Signed-off-by: Lorenzo Fontana Signed-off-by: Leonardo Di Donato --- .../selftests/bpf/prog_tests/core_reloc.c | 46 ++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/core_reloc.c b/tools/testing/selftests/bpf/prog_tests/core_reloc.c index b8bdd1c3efca..10a1d5fb788e 100644 --- a/tools/testing/selftests/bpf/prog_tests/core_reloc.c +++ b/tools/testing/selftests/bpf/prog_tests/core_reloc.c @@ -2,6 +2,7 @@ #include #include "progs/core_reloc_types.h" #include "bpf_testmod/bpf_testmod.h" +#include #include #include #include @@ -354,6 +355,8 @@ static int duration = 0; .fails = true, \ } +#define BTFGEN_BTF_PATH "/tmp/btfgen.btf" + struct core_reloc_test_case; typedef int (*setup_test_fn)(struct core_reloc_test_case *test); @@ -836,7 +839,21 @@ static size_t roundup_page(size_t sz) return (sz + page_size - 1) / page_size * page_size; } -void test_core_reloc(void) +static int run_btfgen(const char *src_btf, const char *dst_btf, const char *objpath) +{ + char command[4096]; + int n; + + n = snprintf(command, sizeof(command), + "./tools/build/bpftool/bpftool gen min_core_btf %s %s %s", + src_btf, dst_btf, objpath); + if (n < 0 || n >= sizeof(command)) + return -1; + + return system(command); +} + +static void _test_core_reloc(bool btfgen) { const size_t mmap_sz = roundup_page(sizeof(struct data)); DECLARE_LIBBPF_OPTS(bpf_object_open_opts, open_opts); @@ -863,6 +880,22 @@ void test_core_reloc(void) continue; } + /* generate a "minimal" BTF file and use it as source */ + if (btfgen) { + if (!test_case->btf_src_file || test_case->fails) { + test__skip(); + continue; + } + + unlink(BTFGEN_BTF_PATH); + err = run_btfgen(test_case->btf_src_file, BTFGEN_BTF_PATH, + test_case->bpf_obj_file); + if (!ASSERT_OK(err, "run_btfgen")) + goto cleanup; + + test_case->btf_src_file = BTFGEN_BTF_PATH; + } + if (test_case->setup) { err = test_case->setup(test_case); if (CHECK(err, "test_setup", "test #%d setup failed: %d\n", i, err)) @@ -954,8 +987,19 @@ void test_core_reloc(void) CHECK_FAIL(munmap(mmap_data, mmap_sz)); mmap_data = NULL; } + unlink(BTFGEN_BTF_PATH); bpf_link__destroy(link); link = NULL; bpf_object__close(obj); } } + +void test_core_reloc(void) +{ + _test_core_reloc(false); +} + +void test_core_btfgen(void) +{ + _test_core_reloc(true); +}