From patchwork Mon Sep 20 14:15:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505457 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C0DAC433EF for ; Mon, 20 Sep 2021 14:15:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E888560F0F for ; Mon, 20 Sep 2021 14:15:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235575AbhITORD (ORCPT ); Mon, 20 Sep 2021 10:17:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235377AbhITORC (ORCPT ); Mon, 20 Sep 2021 10:17:02 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79ABAC061760; Mon, 20 Sep 2021 07:15:35 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id f3-20020a17090a638300b00199097ddf1aso15357932pjj.0; Mon, 20 Sep 2021 07:15:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H4+Sywiu78/ZWqvRnRpYlhOmecgwW6KLOtEzX8JbMVc=; b=lwJCpqkCgMcU1j4kgHVp1eWi9ZBkwTfID6D617hajRQqj06khjJc+HvtOgEthwW1LR 01inzEkS1iIAqPmb6sba+b8OS17QJUAXNHlUo23YF20/MY6V9FHcuK2e2Ac9CEmxPdpn DAfrIOlXSSfc0A1gAiowR2mBIyz7IouTJz620I8qUpsq8MMhUxxoQYTG0Xe6JtNV6J75 14tF/bq9jxx6CtC/933ZhcrBO0ItHagjV1CKHzyGv5LPfpOX+omni+rTfaiR22W7zn7+ hC97s1/QuC//mXS9N2MB7YNJIRfczE/LJqHasQDU8C2oM7esOsStFacFT5WYTVVe+leb mr1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H4+Sywiu78/ZWqvRnRpYlhOmecgwW6KLOtEzX8JbMVc=; b=FuR6VLzAglZP3ps8QH3cv2F5iZJxFp6vs+xcUT+lcvd08LrabS1I2wPyxFtqqu9U6t JrPgW9ueku6FD07kqh0935+Rx4jcx8rDkTrsktwZHr+5M3MqrWC0UuWHhJrGAlyrEfbe uUuv2a20u4pD3ZSo4xgb4rEv/4AOntRTdVe63lpFmCFEzzjbBsmoKfvAgLu7IdEZp9dW UPY4lVt268hdey6AMIxiM2RSiBGm0Qp7f0LHTFeZYZWZ7Vfw3dvNzGta5/+dwS2GNpwL A6raZ3e04hv23Y6cz90VV1BXFaX6V+0ELQTAI10monkVZbzW/7K1j24OZCZU0Rg36GAG 9NuQ== X-Gm-Message-State: AOAM530ToIWnep9EnkD2U70lsbTZ8UAdXnYlJcU8CWdPouaGB4+E7o16 DSz5hsxRR6TrH7L3rdpnZSt/QPXUWswkGA== X-Google-Smtp-Source: ABdhPJzcPvxtxzRKUKCR9fPrNocWGViBi6f4yVHPO62+qwysS0VpAFDQRo1YqJ4PR4Gpg4VRcgpxHg== X-Received: by 2002:a17:90a:ba0e:: with SMTP id s14mr2165985pjr.213.1632147334710; Mon, 20 Sep 2021 07:15:34 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id a11sm15884232pgj.75.2021.09.20.07.15.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:34 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 01/11] bpf: Introduce BPF support for kernel module function calls Date: Mon, 20 Sep 2021 19:45:16 +0530 Message-Id: <20210920141526.3940002-2-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17724; h=from:subject; bh=YYhLh2NEYoz0aH/JN9qE6gZqWRRerwMmCkU0qqEfTYY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaDGbRTKZUgb7mLYTksiRlSamXwDsTamk/y5c5y InLiJrSJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWgwAKCRBM4MiGSL8RyrAwD/ 9yCH1+KjmqjSM7UUhEMfdKboO1IITJ0Vq3UmOf3yWpePSyzPfy+YQusrLnJY6+qoA+A7SMsFinpldv X3rjYYqJ8+DC9Ll6HXKl3ZLh4wnmSyNRELsMcMgYBIGMkXVNVjP9wvZ0+1fifXUdFRO71/LAskQAUc EcyN94CVOW+QICK9jZc8QBYWJ6d32D3Y1PZXxFBKftpSrgBd2Wf9t2cCWkYwV2GZ6iUuG9gleEgjEw yQAfnHdrMja36FCKGDKPYAY/TiUer64YxC9preWOJM1rARMv34G8V25+whlO2jwZTKJ/Ma+DOdNrwP j+g5qhpq9rd7Z3QdOztdbKfPVmrmCfiA8lvXN3kON+D9Qdxr95bXBdt5jcAl1veML8G3DkrHzUghFU MXkoaiBFo+x6tPEwvPgUWXLZXw+Tws3AaCmn/OgsqZyXiUf0dtjG1/FPZkZhSHb+fTMrTaBP5S0vZi uXC6+jKrF9PAosPbr+cPdZTcF8iw0m7fipAhB8gSPG6b4Z5l0dsqeGMpwHxmn27vdu0eSl4JBl3zmC lC0nHf5Kz5737sRQMfUqmfQuXCVJsllhkZlxY93o7MTpPe1T3k5KRJTlJU77kUX1NzPjz+gmJjlgZ/ uDo8t94Thsl/0M6WcjthmmsAO4f8hRWYeG9ypGwLprymuBoChYIeMJS5gyfA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This change adds support on the kernel side to allow for BPF programs to call kernel module functions. Userspace will prepare an array of module BTF fds that is passed in during BPF_PROG_LOAD using fd_array parameter. In the kernel, the module BTFs are placed in the auxilliary struct for bpf_prog, and loaded as needed. The verifier then uses insn->off to index into the fd_array. insn->off 0 is reserved for vmlinux BTF (for backwards compat), so userspace must use an fd_array index > 0 for module kfunc support. kfunc_btf_tab is sorted based on offset in an array, and each offset corresponds to one descriptor, with a max limit up to 256 such module BTFs. Another change is to check_kfunc_call callback, which now include a struct module * pointer, this is to be used in later patch such that the kfunc_id and module pointer are matched for dynamically registered BTF sets from loadable modules, so that same kfunc_id in two modules doesn't lead to check_kfunc_call succeeding. For the duration of the check_kfunc_call, the reference to struct module exists, as it returns the pointer stored in kfunc_btf_tab. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 8 +- include/linux/bpf_verifier.h | 2 + kernel/bpf/core.c | 4 + kernel/bpf/verifier.c | 202 ++++++++++++++++++++++++++++++----- net/bpf/test_run.c | 2 +- net/ipv4/bpf_tcp_ca.c | 2 +- 6 files changed, 188 insertions(+), 32 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index b6c45a6cbbba..81b1b3d0d546 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -511,7 +511,7 @@ struct bpf_verifier_ops { const struct btf_type *t, int off, int size, enum bpf_access_type atype, u32 *next_btf_id); - bool (*check_kfunc_call)(u32 kfunc_btf_id); + bool (*check_kfunc_call)(u32 kfunc_btf_id, struct module *owner); }; struct bpf_prog_offload_ops { @@ -874,6 +874,7 @@ struct bpf_prog_aux { void *jit_data; /* JIT specific data. arch dependent */ struct bpf_jit_poke_descriptor *poke_tab; struct bpf_kfunc_desc_tab *kfunc_tab; + struct bpf_kfunc_btf_tab *kfunc_btf_tab; u32 size_poke_tab; struct bpf_ksym ksym; const struct bpf_prog_ops *ops; @@ -1636,7 +1637,7 @@ int bpf_prog_test_run_raw_tp(struct bpf_prog *prog, int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id); +bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner); bool btf_ctx_access(int off, int size, enum bpf_access_type type, const struct bpf_prog *prog, struct bpf_insn_access_aux *info); @@ -1857,7 +1858,8 @@ static inline int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog, return -ENOTSUPP; } -static inline bool bpf_prog_test_check_kfunc_call(u32 kfunc_id) +static inline bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, + struct module *owner) { return false; } diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 5424124dbe36..c8a78e830fca 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -527,5 +527,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, const struct bpf_prog *tgt_prog, u32 btf_id, struct bpf_attach_target_info *tgt_info); +void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab); + #endif /* _LINUX_BPF_VERIFIER_H */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 6fddc13fe67f..084df01625cd 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -2255,6 +2256,9 @@ static void bpf_prog_free_deferred(struct work_struct *work) int i; aux = container_of(work, struct bpf_prog_aux, work); +#ifdef CONFIG_BPF_SYSCALL + bpf_free_kfunc_btf_tab(aux->kfunc_btf_tab); +#endif bpf_free_used_maps(aux); bpf_free_used_btfs(aux); if (bpf_prog_is_dev_bound(aux)) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e76b55917905..5b465d67cf65 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1626,52 +1626,173 @@ static int add_subprog(struct bpf_verifier_env *env, int off) return env->subprog_cnt - 1; } +#define MAX_KFUNC_DESCS 256 +#define MAX_KFUNC_BTFS 256 + struct bpf_kfunc_desc { struct btf_func_model func_model; u32 func_id; s32 imm; + u16 offset; +}; + +struct bpf_kfunc_btf { + struct btf *btf; + struct module *module; + u16 offset; }; -#define MAX_KFUNC_DESCS 256 struct bpf_kfunc_desc_tab { struct bpf_kfunc_desc descs[MAX_KFUNC_DESCS]; u32 nr_descs; }; -static int kfunc_desc_cmp_by_id(const void *a, const void *b) +struct bpf_kfunc_btf_tab { + struct bpf_kfunc_btf descs[MAX_KFUNC_BTFS]; + u32 nr_descs; +}; + +static int kfunc_desc_cmp_by_id_off(const void *a, const void *b) { const struct bpf_kfunc_desc *d0 = a; const struct bpf_kfunc_desc *d1 = b; /* func_id is not greater than BTF_MAX_TYPE */ - return d0->func_id - d1->func_id; + return d0->func_id - d1->func_id ?: d0->offset - d1->offset; +} + +static int kfunc_btf_cmp_by_off(const void *a, const void *b) +{ + const struct bpf_kfunc_btf *d0 = a; + const struct bpf_kfunc_btf *d1 = b; + + return d0->offset - d1->offset; } static const struct bpf_kfunc_desc * -find_kfunc_desc(const struct bpf_prog *prog, u32 func_id) +find_kfunc_desc(const struct bpf_prog *prog, u32 func_id, u16 offset) { struct bpf_kfunc_desc desc = { .func_id = func_id, + .offset = offset, }; struct bpf_kfunc_desc_tab *tab; tab = prog->aux->kfunc_tab; return bsearch(&desc, tab->descs, tab->nr_descs, - sizeof(tab->descs[0]), kfunc_desc_cmp_by_id); + sizeof(tab->descs[0]), kfunc_desc_cmp_by_id_off); +} + +static struct btf *__find_kfunc_desc_btf(struct bpf_verifier_env *env, + s16 offset, struct module **btf_modp) +{ + struct bpf_kfunc_btf kf_btf = { .offset = offset }; + struct bpf_kfunc_btf_tab *tab; + struct bpf_kfunc_btf *b; + struct module *mod; + struct btf *btf; + int btf_fd; + + tab = env->prog->aux->kfunc_btf_tab; + b = bsearch(&kf_btf, tab->descs, tab->nr_descs, + sizeof(tab->descs[0]), kfunc_btf_cmp_by_off); + if (!b) { + if (tab->nr_descs == MAX_KFUNC_BTFS) { + verbose(env, "too many different module BTFs\n"); + return ERR_PTR(-E2BIG); + } + + if (bpfptr_is_null(env->fd_array)) { + verbose(env, "kfunc offset > 0 without fd_array is invalid\n"); + return ERR_PTR(-EPROTO); + } + + if (copy_from_bpfptr_offset(&btf_fd, env->fd_array, + offset * sizeof(btf_fd), + sizeof(btf_fd))) + return ERR_PTR(-EFAULT); + + btf = btf_get_by_fd(btf_fd); + if (IS_ERR(btf)) + return btf; + + if (!btf_is_module(btf)) { + verbose(env, "BTF fd for kfunc is not a module BTF\n"); + btf_put(btf); + return ERR_PTR(-EINVAL); + } + + mod = btf_try_get_module(btf); + if (!mod) { + btf_put(btf); + return ERR_PTR(-ENXIO); + } + + b = &tab->descs[tab->nr_descs++]; + b->btf = btf; + b->module = mod; + b->offset = offset; + + sort(tab->descs, tab->nr_descs, sizeof(tab->descs[0]), + kfunc_btf_cmp_by_off, NULL); + } + if (btf_modp) + *btf_modp = b->module; + return b->btf; } -static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) +void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab) +{ + if (!tab) + return; + + while (tab->nr_descs--) { + module_put(tab->descs[tab->nr_descs].module); + btf_put(tab->descs[tab->nr_descs].btf); + } + kfree(tab); +} + +static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, + u32 func_id, s16 offset, + struct module **btf_modp) +{ + struct btf *kfunc_btf; + + if (offset) { + if (offset < 0) { + /* In the future, this can be allowed to increase limit + * of fd index into fd_array, interpreted as u16. + */ + verbose(env, "negative offset disallowed for kernel module function call\n"); + return ERR_PTR(-EINVAL); + } + + kfunc_btf = __find_kfunc_desc_btf(env, offset, btf_modp); + if (IS_ERR_OR_NULL(kfunc_btf)) { + verbose(env, "cannot find module BTF for func_id %u\n", func_id); + return kfunc_btf ?: ERR_PTR(-ENOENT); + } + return kfunc_btf; + } + return btf_vmlinux ?: ERR_PTR(-ENOENT); +} + +static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) { const struct btf_type *func, *func_proto; + struct bpf_kfunc_btf_tab *btf_tab; struct bpf_kfunc_desc_tab *tab; struct bpf_prog_aux *prog_aux; struct bpf_kfunc_desc *desc; const char *func_name; + struct btf *desc_btf; unsigned long addr; int err; prog_aux = env->prog->aux; tab = prog_aux->kfunc_tab; + btf_tab = prog_aux->kfunc_btf_tab; if (!tab) { if (!btf_vmlinux) { verbose(env, "calling kernel function is not supported without CONFIG_DEBUG_INFO_BTF\n"); @@ -1699,7 +1820,20 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) prog_aux->kfunc_tab = tab; } - if (find_kfunc_desc(env->prog, func_id)) + if (!btf_tab && offset) { + btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL); + if (!btf_tab) + return -ENOMEM; + prog_aux->kfunc_btf_tab = btf_tab; + } + + desc_btf = find_kfunc_desc_btf(env, func_id, offset, NULL); + if (IS_ERR(desc_btf)) { + verbose(env, "failed to find BTF for kernel function\n"); + return PTR_ERR(desc_btf); + } + + if (find_kfunc_desc(env->prog, func_id, offset)) return 0; if (tab->nr_descs == MAX_KFUNC_DESCS) { @@ -1707,20 +1841,20 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) return -E2BIG; } - func = btf_type_by_id(btf_vmlinux, func_id); + func = btf_type_by_id(desc_btf, func_id); if (!func || !btf_type_is_func(func)) { verbose(env, "kernel btf_id %u is not a function\n", func_id); return -EINVAL; } - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func_proto = btf_type_by_id(desc_btf, func->type); if (!func_proto || !btf_type_is_func_proto(func_proto)) { verbose(env, "kernel function btf_id %u does not have a valid func_proto\n", func_id); return -EINVAL; } - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); + func_name = btf_name_by_offset(desc_btf, func->name_off); addr = kallsyms_lookup_name(func_name); if (!addr) { verbose(env, "cannot find address for kernel function %s\n", @@ -1731,12 +1865,13 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) desc = &tab->descs[tab->nr_descs++]; desc->func_id = func_id; desc->imm = BPF_CAST_CALL(addr) - __bpf_call_base; - err = btf_distill_func_proto(&env->log, btf_vmlinux, + desc->offset = offset; + err = btf_distill_func_proto(&env->log, desc_btf, func_proto, func_name, &desc->func_model); if (!err) sort(tab->descs, tab->nr_descs, sizeof(tab->descs[0]), - kfunc_desc_cmp_by_id, NULL); + kfunc_desc_cmp_by_id_off, NULL); return err; } @@ -1815,7 +1950,7 @@ static int add_subprog_and_kfunc(struct bpf_verifier_env *env) } else if (bpf_pseudo_call(insn)) { ret = add_subprog(env, i + insn->imm + 1); } else { - ret = add_kfunc_call(env, insn->imm); + ret = add_kfunc_call(env, insn->imm, insn->off); } if (ret < 0) @@ -2152,12 +2287,17 @@ static int get_prev_insn_idx(struct bpf_verifier_state *st, int i, static const char *disasm_kfunc_name(void *data, const struct bpf_insn *insn) { const struct btf_type *func; + struct btf *desc_btf; if (insn->src_reg != BPF_PSEUDO_KFUNC_CALL) return NULL; - func = btf_type_by_id(btf_vmlinux, insn->imm); - return btf_name_by_offset(btf_vmlinux, func->name_off); + desc_btf = find_kfunc_desc_btf(data, insn->imm, insn->off, NULL); + if (IS_ERR(desc_btf)) + return ""; + + func = btf_type_by_id(desc_btf, insn->imm); + return btf_name_by_offset(desc_btf, func->name_off); } /* For given verifier state backtrack_insn() is called from the last insn to @@ -6485,23 +6625,29 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) struct bpf_reg_state *regs = cur_regs(env); const char *func_name, *ptr_type_name; u32 i, nargs, func_id, ptr_type_id; + struct module *btf_mod = NULL; const struct btf_param *args; + struct btf *desc_btf; int err; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off, &btf_mod); + if (IS_ERR(desc_btf)) + return PTR_ERR(desc_btf); + func_id = insn->imm; - func = btf_type_by_id(btf_vmlinux, func_id); - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func = btf_type_by_id(desc_btf, func_id); + func_name = btf_name_by_offset(desc_btf, func->name_off); + func_proto = btf_type_by_id(desc_btf, func->type); if (!env->ops->check_kfunc_call || - !env->ops->check_kfunc_call(func_id)) { + !env->ops->check_kfunc_call(func_id, btf_mod)) { verbose(env, "calling kernel function %s is not allowed\n", func_name); return -EACCES; } /* Check the arguments */ - err = btf_check_kfunc_arg_match(env, btf_vmlinux, func_id, regs); + err = btf_check_kfunc_arg_match(env, desc_btf, func_id, regs); if (err) return err; @@ -6509,15 +6655,15 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) mark_reg_not_init(env, regs, caller_saved[i]); /* Check return type */ - t = btf_type_skip_modifiers(btf_vmlinux, func_proto->type, NULL); + t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); mark_btf_func_reg_size(env, BPF_REG_0, t->size); } else if (btf_type_is_ptr(t)) { - ptr_type = btf_type_skip_modifiers(btf_vmlinux, t->type, + ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); if (!btf_type_is_struct(ptr_type)) { - ptr_type_name = btf_name_by_offset(btf_vmlinux, + ptr_type_name = btf_name_by_offset(desc_btf, ptr_type->name_off); verbose(env, "kernel function %s returns pointer type %s %s is not supported\n", func_name, btf_type_str(ptr_type), @@ -6525,7 +6671,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) return -EINVAL; } mark_reg_known_zero(env, regs, BPF_REG_0); - regs[BPF_REG_0].btf = btf_vmlinux; + regs[BPF_REG_0].btf = desc_btf; regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; mark_btf_func_reg_size(env, BPF_REG_0, sizeof(void *)); @@ -6536,7 +6682,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) for (i = 0; i < nargs; i++) { u32 regno = i + 1; - t = btf_type_skip_modifiers(btf_vmlinux, args[i].type, NULL); + t = btf_type_skip_modifiers(desc_btf, args[i].type, NULL); if (btf_type_is_ptr(t)) mark_btf_func_reg_size(env, regno, sizeof(void *)); else @@ -11076,7 +11222,8 @@ static int do_check(struct bpf_verifier_env *env) env->jmps_processed++; if (opcode == BPF_CALL) { if (BPF_SRC(insn->code) != BPF_K || - insn->off != 0 || + (insn->src_reg != BPF_PSEUDO_KFUNC_CALL + && insn->off != 0) || (insn->src_reg != BPF_REG_0 && insn->src_reg != BPF_PSEUDO_CALL && insn->src_reg != BPF_PSEUDO_KFUNC_CALL) || @@ -12432,6 +12579,7 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->stack_depth = env->subprog_info[i].stack_depth; func[i]->jit_requested = 1; func[i]->aux->kfunc_tab = prog->aux->kfunc_tab; + func[i]->aux->kfunc_btf_tab = prog->aux->kfunc_btf_tab; func[i]->aux->linfo = prog->aux->linfo; func[i]->aux->nr_linfo = prog->aux->nr_linfo; func[i]->aux->jited_linfo = prog->aux->jited_linfo; @@ -12621,7 +12769,7 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, /* insn->imm has the btf func_id. Replace it with * an address (relative to __bpf_base_call). */ - desc = find_kfunc_desc(env->prog, insn->imm); + desc = find_kfunc_desc(env->prog, insn->imm, insn->off); if (!desc) { verbose(env, "verifier internal error: kernel function descriptor not found for func_id %u\n", insn->imm); diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index fcb2f493f710..fe5c34f414a2 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -241,7 +241,7 @@ BTF_ID(func, bpf_kfunc_call_test2) BTF_ID(func, bpf_kfunc_call_test3) BTF_SET_END(test_sk_kfunc_ids) -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id) +bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner) { return btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id); } diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 0dcee9df1326..b3afd3361f34 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -255,7 +255,7 @@ BTF_ID(func, bbr_set_state) #endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) -static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id) +static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) { return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); } From patchwork Mon Sep 20 14:15:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505459 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D371C433EF for ; Mon, 20 Sep 2021 14:15:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 89FE260F0F for ; Mon, 20 Sep 2021 14:15:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235844AbhITORG (ORCPT ); Mon, 20 Sep 2021 10:17:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235714AbhITORF (ORCPT ); Mon, 20 Sep 2021 10:17:05 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D62FC061574; Mon, 20 Sep 2021 07:15:38 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id 203so8517977pfy.13; Mon, 20 Sep 2021 07:15:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YC/5g23SPpiZ7A5VfOJx+Ff2nYKoj2UfUfHHpa2IJDY=; b=eaNr/HPJ4eandUEJWIhsl4IOLgaT6YvdDpDWsyGDUU+/+oHxEXfIpqM3DH+07neDIf uVI/p+Qh9k+sE2NrU5Jkwl040jh0YZHfe5Ob9titrCz6WIipCQDRDkJS8cjxVlPpeGqx IrpK2e6MwSBUisU+FpBWNdzqWLsfsG6JMIc1FMYGiGl2sLTZ6wZxu1dBg9aLYnP45mAl 2+167W8zWqnWrBKxv37F9A0c04s6sIi+3Hz19IlI7eUfw864tFydJWlq38f4Y5cJVyfj kYGuoANocoUxZDZg7w7zWS5b5pOzjx4pdZGvMGVL/fBQ+H0O9jDTPcIeCVAiKbIz9N+Y UGnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YC/5g23SPpiZ7A5VfOJx+Ff2nYKoj2UfUfHHpa2IJDY=; b=6aUWn0sucRyMJTcuGMq2tZfMEQExtmaQaFHpDxzfiS8SW3J/LAzRFl8yaHp5o7t6hi eXXPfoinx4QUAe1is6LUoQNgW0hwZhuABfuitoKtCHnEiTeVWWqlRLlF06bRv970E5iL BghQLLuiY+YsFJ6WfNkppkj67IJPXR8hEGGoHsQiqAubR4P6OmTfv4Vn8B10ge3ohQp0 D2uJ0H90D7g5qX25iteD5qh1v21o5wknXPIdTN1QuoTlhLPSAYeebdK/yV7fpK2CIi5t hlGqj0Ut0iyuPdp+zjaSjqO6br5XSRDM8OIP4JIyjqDwSgsoz4hxDMT2HnnhfA6fwAgj STZg== X-Gm-Message-State: AOAM5329OzoDIj3++sJeRwlB+zIrGHqTDDmlieLfmzcHKuQuxqC2S04O uU6NV3RbLCsfd4nydVG+afah7dnsVFIwUg== X-Google-Smtp-Source: ABdhPJz7y59Bho1YPw+kRTpPQoB66EGTzsySXGLQyLYvimG1ktBXhhtYErND7p9c9vuvvQPY7Kp63g== X-Received: by 2002:a63:515f:: with SMTP id r31mr23484743pgl.41.1632147337780; Mon, 20 Sep 2021 07:15:37 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id e13sm10360574pfn.212.2021.09.20.07.15.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:37 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 02/11] bpf: Be conservative while processing invalid kfunc calls Date: Mon, 20 Sep 2021 19:45:17 +0530 Message-Id: <20210920141526.3940002-3-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2214; h=from:subject; bh=FKRyNZlxBXZubpWKuL4SYjb1veTjvmV2mok2OrudHXU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaDvRD+YlTis4ZpxqZMjImP98Z2FoCqLKhYqPfs a52iZuSJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWgwAKCRBM4MiGSL8Rymj6D/ 9lL7f3voRclVpm4wPEixV6+/oIwsq5tfzBkKkQZBJWZywiiRdmLoY0D+drfbvt9GjO2xcIkpi4+kno EjyM6jEhXEvRzMY2Dz5qkxSCYopLliC/e3ORc2gVBqWGyi1XUE7d5RsCKDHA+x1o+hyAcJfIA8gpRj bdYxBw8EdCHAOdwoFa/XcMLveTUKuj0N/dzv66euCkEc9qXxS+TyJhdlHlXXdZ0fcwNRf20qS34P7Q zi0JmctCyC3Ih+GkZhVwpNJfvNaV4INguAPv1xdrJnAQwToFH4eqicPcnmsTUOV0m8etZp6Yf84ldS +DsW05rPhpxwdB4UpUwwSFsxAb0daJe/zE/G8ViT2kvzo9FflFkrbQUrEyctfYBz0l+zB6yJvcmCBh Hpv3kRSK0eaL/kE/u2vmYCLm+NFT3gYehSoQrIjlkFwwehj1k9267o3Xsd4bmArc6R2+Nd+FJjFhLh nm8yvqvuwRhGFdHvCZL19DFWH0DhdgoAjI0mNuHMebqkSJHuem2l94MPfiBM94Yrz5+R8TRcqjhlc+ IlCxfWHlp4xcPToJqmzqb/jzsShTlKkkG7bAeAhyW0MRqXB8bdfOT6h+SGX9Y64Avm7M3FkNjIpdlh xFOB0l/Pn5SMNFgBBWraYSGdvyp1NIrlQJBuGn5nJdZykKucXh1p8pMSNidg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch also modifies the BPF verifier to only return error for invalid kfunc calls specially marked by userspace (with insn->imm == 0, insn->off == 0) after the verifier has eliminated dead instructions. This can be handled in the fixup stage, and skip processing during add and check stages. If such an invalid call is dropped, the fixup stage will not encounter insn->imm as 0, otherwise it bails out and returns an error. This will be exposed as weak ksym support in libbpf in later patches. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/verifier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5b465d67cf65..58c86670dc55 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1820,6 +1820,15 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) prog_aux->kfunc_tab = tab; } + /* func_id == 0 is always invalid, but instead of returning an error, be + * conservative and wait until the code elimination pass before returning + * error, so that invalid calls that get pruned out can be in BPF programs + * loaded from userspace. It is also required that offset be untouched + * for such calls. + */ + if (!func_id && !offset) + return 0; + if (!btf_tab && offset) { btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL); if (!btf_tab) @@ -6630,6 +6639,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) struct btf *desc_btf; int err; + /* skip for now, but return error when we find this in fixup_kfunc_call */ + if (!insn->imm) + return 0; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off, &btf_mod); if (IS_ERR(desc_btf)) return PTR_ERR(desc_btf); @@ -12766,6 +12779,11 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, { const struct bpf_kfunc_desc *desc; + if (!insn->imm) { + verbose(env, "invalid kernel function call not eliminated in verifier pass\n"); + return -EINVAL; + } + /* insn->imm has the btf func_id. Replace it with * an address (relative to __bpf_base_call). */ From patchwork Mon Sep 20 14:15:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505461 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC10AC433FE for ; Mon, 20 Sep 2021 14:15:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 94C4260F0F for ; Mon, 20 Sep 2021 14:15:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235714AbhITORK (ORCPT ); Mon, 20 Sep 2021 10:17:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236770AbhITORI (ORCPT ); Mon, 20 Sep 2021 10:17:08 -0400 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A94EDC061762; Mon, 20 Sep 2021 07:15:41 -0700 (PDT) Received: by mail-pf1-x443.google.com with SMTP id m26so16410412pff.3; Mon, 20 Sep 2021 07:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JQzJbmsNcTyJiM7wxqK/N5d0ieXGaf7vJrciuGA2fl4=; b=QCA+QVyrDpBByGTJltTbMn9R+X3N7Uq/Yo44UsMvGfbptJbdK1GSFreCIcqbekvN0n vuiHnPwUUp5k+atiV5ct4Z0QwZLLdHDSkihDEmG7IWvOv7pNnaV4V5veBJfk5XItHrbQ uWyXPk5TFrJuM4lwSXhNHN+/ntUknktHHan7b/W5Zv0klz+Snyp9qAqPw8xibWR0SNeO bUO3Wck3pmi+SzkuO4SsRxQWBis2ZJUc6z94fw2h4y2y7ls3lktXEQNcYs84XXgzOsFD rgLQcFOelGPgfbMg5mK9Rt149FwKOj9m2fMOkfy08zhkQWr2g4cqH/6HCIoaDt6983Rz dngw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JQzJbmsNcTyJiM7wxqK/N5d0ieXGaf7vJrciuGA2fl4=; b=Fy71rL3KukLSLixnCNfUoweseU+fawH4o/2NglcklISd8+OsK1oizRodmkFA3c3GDL MWGdLIunL5aWe0lBYyKS9cALTLroqRUU2C3g/xeDZ2c3M5NunK2qoMgc+APHeEY64lKG fLIFdSADhXiafq54OSN6p8oUCYQpaXVFK6Go1dGxtyHO/3wVIx/XRdWfiNtcQFlbqokF X6HJLOXNCW8Zyh0jQFR0YeawNlTYVY+vdWAaF2Piknc/2+OCLI+n68qgGumFjJAwWB5d URVSPlnrVNaUtfN7gGYR8Fe5hJOV6PRGK+1XQrEjJ6gmULaPUzaQ/66mVvF5w08AszGH Q4qg== X-Gm-Message-State: AOAM530V4ISc0njRee2e36rGzy3UxnFnkwA1DrkAxc5xUSqOKV3Da4/B BW4HXQnn1RxzZY6LAIVwf92LHHK9P5BuJg== X-Google-Smtp-Source: ABdhPJyi9GGX+6pvRGJFtnDHGljIQcWkjfUveZfEOZ9Dq7TDkT78nj9Iet7jq5Ufj7Q9spsrbon7tQ== X-Received: by 2002:a63:6f82:: with SMTP id k124mr19176982pgc.218.1632147341090; Mon, 20 Sep 2021 07:15:41 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id y80sm9946310pfb.196.2021.09.20.07.15.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:40 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 03/11] bpf: btf: Introduce helpers for dynamic BTF set registration Date: Mon, 20 Sep 2021 19:45:18 +0530 Message-Id: <20210920141526.3940002-4-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4928; h=from:subject; bh=vv7OxTBJcY0m6TE2e5nTlXV2ed0AWIV5lzxY9L0vytY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaDMM+KYhyuQojuYIX8o5xbBorPLN8l61XoA6Fp vU0g7q6JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWgwAKCRBM4MiGSL8RymAxD/ kBFsQBrAoxKBiOKIBmETDTOz6YbOC+KOaq8ReKuSdz882gbIMmedbZRunxbH4KArKcV+AZXTzOEJfa s66hC58mOhx1Ii4Hykg/nM6v/h0lPWFM6sAr9ax+dNpWXMFLlMa6qLA5JQ8ciaPBadv71/1N/wqi++ pCi3k4WvWvMI+MGm8zAcGo+IWrsk8mQ/ZG3tcK6+HUjWaGBL8gEF+KzDT3ubonolFZcA3ED7kH/Xfk VsmU0QZQjbLZl06F7vNM5JixNs2KyWyO2Odgp84HkLaAvdRrNoV5OMA9VyWZPxpO520KqXAWpyeIer XSew5U7gKdLwhkTzG+7s9NZOAwNdmtazodpQWnyC4LzmRevNAc7mea88UADtSPIDXrXew4YhTxwGsp RQcR8abtbp8w2UkEFrpyCFJk/K2MCzcikif40p527brK4U13x7er2VjZBILCqt4Sr8Ni6PcHKYx6hF 7HZJqWKJ0EUi1WAFthgo2fBjcKR5g4k4Ntzf8U2nZ/06iWfulKkvRp3RENrS/e+H3IEWpmbnM59qQI /vUNIMFUm5FmRPdOxmDkj7tprKEArwa1f6uGceIDczxJmS75pPazayQ0ok5B/HenxvlHUv5K8pDdU1 sM3oQhU0z1LP7OB3ANaKc2GF6q1f2Lx2YCSNiNZA15zmhniaDf/j6WBUFVjw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This adds helpers for registering btf_id_set from modules and the check_kfunc_call callback that can be used to look them up. With in kernel sets, the way this is supposed to work is, in kernel callback looks up within the in-kernel kfunc whitelist, and then defers to the dynamic BTF set lookup if it doesn't find the BTF id. If there is no in-kernel BTF id set, this callback can be used directly. Also fix includes for btf.h and bpfptr.h so that they can included in isolation. This is in preparation for their usage in tcp_bbr, tcp_cubic and tcp_dctcp modules in the next patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpfptr.h | 1 + include/linux/btf.h | 31 +++++++++++++++++++++++++ kernel/bpf/btf.c | 51 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+) diff --git a/include/linux/bpfptr.h b/include/linux/bpfptr.h index 546e27fc6d46..46e1757d06a3 100644 --- a/include/linux/bpfptr.h +++ b/include/linux/bpfptr.h @@ -3,6 +3,7 @@ #ifndef _LINUX_BPFPTR_H #define _LINUX_BPFPTR_H +#include #include typedef sockptr_t bpfptr_t; diff --git a/include/linux/btf.h b/include/linux/btf.h index 214fde93214b..7884983857d9 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -5,6 +5,7 @@ #define _LINUX_BTF_H 1 #include +#include #include #include @@ -238,4 +239,34 @@ static inline const char *btf_name_by_offset(const struct btf *btf, } #endif +struct kfunc_btf_id_set { + struct list_head list; + struct btf_id_set *set; + struct module *owner; +}; + +struct kfunc_btf_id_list; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES +void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s); +void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s); +#else +static inline void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ +} +static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ +} +#endif + +#define DECLARE_CHECK_KFUNC_CALLBACK(type) \ + bool __bpf_check_##type##_kfunc_call(u32 kfunc_id, struct module *owner) +#define DEFINE_KFUNC_BTF_ID_SET(set, name) \ + struct kfunc_btf_id_set name = { LIST_HEAD_INIT(name.list), (set), \ + THIS_MODULE } + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index c3d605b22473..d17f45b163f5 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6343,3 +6343,54 @@ const struct bpf_func_proto bpf_btf_find_by_name_kind_proto = { }; BTF_ID_LIST_GLOBAL_SINGLE(btf_task_struct_ids, struct, task_struct) + +struct kfunc_btf_id_list { + struct list_head list; + struct mutex mutex; +}; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + +void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ + mutex_lock(&l->mutex); + list_add(&s->list, &l->list); + mutex_unlock(&l->mutex); +} +EXPORT_SYMBOL_GPL(register_kfunc_btf_id_set); + +void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ + mutex_lock(&l->mutex); + list_del_init(&s->list); + mutex_unlock(&l->mutex); +} +EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); + +#endif + +#define DEFINE_KFUNC_BTF_ID_LIST(name) \ + struct kfunc_btf_id_list name = { LIST_HEAD_INIT(name.list), \ + __MUTEX_INITIALIZER(name.mutex) }; \ + EXPORT_SYMBOL_GPL(name) + +#define DEFINE_CHECK_KFUNC_CALLBACK(type, list_name) \ + bool __bpf_check_##type##_kfunc_call(u32 kfunc_id, \ + struct module *owner) \ + { \ + struct kfunc_btf_id_set *s; \ + if (!owner || !IS_ENABLED(CONFIG_DEBUG_INFO_BTF_MODULES)) \ + return false; \ + mutex_lock(&list_name.mutex); \ + list_for_each_entry(s, &list_name.list, list) { \ + if (s->owner == owner && \ + btf_id_set_contains(s->set, kfunc_id)) { \ + mutex_unlock(&list_name.mutex); \ + return true; \ + } \ + } \ + mutex_unlock(&list_name.mutex); \ + return false; \ + } From patchwork Mon Sep 20 14:15:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505463 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13855C433EF for ; Mon, 20 Sep 2021 14:15:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F0CB0610CE for ; Mon, 20 Sep 2021 14:15:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236694AbhITORQ (ORCPT ); Mon, 20 Sep 2021 10:17:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237353AbhITORL (ORCPT ); Mon, 20 Sep 2021 10:17:11 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCC15C061760; Mon, 20 Sep 2021 07:15:44 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id j14so4035805plx.4; Mon, 20 Sep 2021 07:15:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yU4+xx6iAwgsGxgz5Ht4AFD9WjITvbQsNdCqZ6ULtK4=; b=mBiyEuqmZ1PuLj+ohMREoKbQOdJNazYDg+Wwlus/vZKwldJEY1WOtlKOjTweVhb/sN dnd8ZHc0AiQ/E4PUa5a/2mTpHI/5a4pnzBTQ83hm8f8YMo3mhIcwNR5nAh5JXNWhB+6o uYmwYgy9jR5r9lW7q5/VVi7kiUIBLF+C+lwm2BjI1VgkgrDeiUlWenufy/EzatAqWsnl 7hzDVicHNnhLjaWSuGrgwReJDMAIMDb/sQQl5gN52PBWhFSj41bONpuE9QtOyKjyZsKQ Cgg/k/2qvG55UT7SLQQMiJz2+rIfcX8qdRar0SnthlrNBih1LARrIrqpKOpZWef2HAcj aJ0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yU4+xx6iAwgsGxgz5Ht4AFD9WjITvbQsNdCqZ6ULtK4=; b=GWoFJ4WKtDCHG+uj0j2u/HUXhfYkN5DbxmSOQK3up9IkTtKs9DDGjzmLdavCBK81ji VtoEvcOn+8yZx3SW9Nx2zkqMCpWVn6elSzU+NXHQZ3uaay1B81TQ1mNINIPsEwt+TpBS Tk6FrGxxEwZpgao0c8RwniPpavrseN6F83cnYISlVmYNh6cy8kPGhuhVlWYUQymFywZf bybRUmjhtxbyDFbG3L8uN68d+VSoafA+Ni0BeK140JBNRlRfVQmW+ObXa3ev0ZNjKvyF Nm3fB3lfiF2f50v7/HEBfCw+69jduNcYHABEM0j26awf6dhZKZx/WLaRgiTRg/hRtB+G FsCA== X-Gm-Message-State: AOAM531lxyNOglWVk89EKl5TGCCf7AjVPr5xpTJyvj5COqLgkVl9yPda kTWvWlEkGsGrlSu2kfMHX6WvaW3TrgoDuQ== X-Google-Smtp-Source: ABdhPJy9JsUM1dybe+LpVnHynrp2ADztC/kCv3XoON7avqS3AZCCjSztYSdvdFvE9NQfzMR2e1nraA== X-Received: by 2002:a17:902:9a06:b0:13c:86d8:ce0b with SMTP id v6-20020a1709029a0600b0013c86d8ce0bmr22942577plp.51.1632147344268; Mon, 20 Sep 2021 07:15:44 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id i7sm2648407pgd.54.2021.09.20.07.15.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:43 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 04/11] tools: Allow specifying base BTF file in resolve_btfids Date: Mon, 20 Sep 2021 19:45:19 +0530 Message-Id: <20210920141526.3940002-5-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2098; h=from:subject; bh=kZhRM1Uo6vUje4gN/WsUjSsDNLJ5u0WuItuVyu5D9Zk=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaEDhwehFEbRic2pkL2wCQ10r1eCg0KKIKoWVSG lKYtwOmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8RyrcoD/ sEGqCUVCMkiIBpO2zDXStqilTry+NxWaawcwZ1ciAsYkq2RIUrCvWiQsSb8vXJ/8UPpzwmXNGIyqO4 XB5eTDWWlXOJBFwhQ0HS4StdJoFdKPMjUAOG2ALYgvTl+LE3VEBVTcyj2CGPFDZGiaul+KnxC6GZr2 EdbqiDV13ew+yDpb+zs+klVcn3Z3dq48VqyDTvsvbQfIZLWMzNOh2CmpbiH/fXaSgM7tfD1Fnq8fbw yFhTc3f6xUW5BW0wPggQmOQbW9KfAb0rcpqWx+sP0XKJANyVJOc/ImzAx82iUIptIxHhEiQxhE6n1U aCnNtJ2Ul8PYaqSSC+enIYEDbFpjurJS4rEHnalndEWAf4xclUKRUCoxt/kXp3NVQOiV3B3KlMYg0t DhOx2/+WSgMY4b91LBF2y2PEw1I2NnJRwDT4nZBT2cksx6byFlMh0CbkSRPwlOZOSO/kpDVaZQjFsv iP28qWHn/x2AEAKawp8DB/1rNmisC2AU3tF5hLYy6WjipnMtJ84YcATcw7V6oLRscwtJKIueFy1K3m pOtlFmC7vWaKTBLsxPIQSBx88LMO97ZWIYllDjvBCDeCtGwt0IYHKnyUVfis7BghCn24mb7obXDkPf 5QDEgW6Ib1FaXDYCpDRu4fxsDS1Gp7Gq/DCUI8qvV5XJxVTFTpkdM2Tk9UjQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commits allows specifying the base BTF for resolving btf id lists/sets during link time in the resolve_btfids tool. The base BTF is set to NULL if no path is passed. This allows resolving BTF ids for module kernel objects. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/bpf/resolve_btfids/main.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index de6365b53c9c..206e1120082f 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -89,6 +89,7 @@ struct btf_id { struct object { const char *path; const char *btf; + const char *base_btf_path; struct { int fd; @@ -477,16 +478,27 @@ static int symbols_resolve(struct object *obj) int nr_structs = obj->nr_structs; int nr_unions = obj->nr_unions; int nr_funcs = obj->nr_funcs; + struct btf *base_btf = NULL; int err, type_id; struct btf *btf; __u32 nr_types; - btf = btf__parse(obj->btf ?: obj->path, NULL); + if (obj->base_btf_path) { + base_btf = btf__parse(obj->base_btf_path, NULL); + err = libbpf_get_error(base_btf); + if (err) { + pr_err("FAILED: load base BTF from %s: %s\n", + obj->base_btf_path, strerror(-err)); + return -1; + } + } + + btf = btf__parse_split(obj->btf ?: obj->path, base_btf); err = libbpf_get_error(btf); if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); - return -1; + goto out; } err = -1; @@ -545,6 +557,7 @@ static int symbols_resolve(struct object *obj) err = 0; out: + btf__free(base_btf); btf__free(btf); return err; } @@ -697,6 +710,8 @@ int main(int argc, const char **argv) "BTF data"), OPT_BOOLEAN(0, "no-fail", &no_fail, "do not fail if " BTF_IDS_SECTION " section is not found"), + OPT_STRING('s', "base-btf", &obj.base_btf_path, "file", + "path of file providing base BTF data"), OPT_END() }; int err = -1; From patchwork Mon Sep 20 14:15:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505465 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AFCBC433EF for ; Mon, 20 Sep 2021 14:15:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33379610CE for ; Mon, 20 Sep 2021 14:15:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235734AbhITORV (ORCPT ); Mon, 20 Sep 2021 10:17:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236890AbhITORT (ORCPT ); Mon, 20 Sep 2021 10:17:19 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDE1FC0613D9; Mon, 20 Sep 2021 07:15:47 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id g184so17466251pgc.6; Mon, 20 Sep 2021 07:15:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vyez4xTVBfXMJKcmfLApVkyHLDOlGbGNwFCTxvU2m5Y=; b=m/vL6T8OMoQ7rYbHHQg31wXvH+ngk3j+EWR4GKFp/LzMrTBkffb7iGH4tHJx0I/tlz 4frDpBXGjcxVA3ezQQknp+PDr8rFTXYB9EZktCbHRGDVCHtzTZOhxIUkvKcHcQKGXODs IK3Ubx72kD1G6etsHlZ0ABwwt1uFgpovQIzrcsieNZdHJ2p3IlS68jR50y+FCnG5Pfff cTIn3y2xox8aXXAttcCilwgS7w3CUH1WplUmCtSb8fZBZS3dGrAjMSpeLm9jaoU++WCx wRGrr+rXnmpu4FhHdiXzCuKUwHkolb2R3EKL7ZLwVv0TgvyynKqZYhS6fIO3mmNI2lIp Nufw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vyez4xTVBfXMJKcmfLApVkyHLDOlGbGNwFCTxvU2m5Y=; b=zoEtqji33Zi4YKLX7eb61ODbKl9GXVcljvA5CfKtkxEB/GC5nUwldPi9g63jzBk/RD 0YWGy1K+vGq/XTucRJ8cqhhVQ1rCheETQyxWIFZF9FhZfIy9vrLV6J905XRNKB9B1kGf 15Lr4IBUU5HE8mAaRZSzWepyQXYgveNFdf0AfhwfRBPZJYPv1m/Jp6rdnKpITLRwcCA3 npNRwl/Y3Zx5UUrt4NQyvk+v09hWDzppqj3WHoBw5nl013uLUSNFVaV3TpoVgY12AlR4 asdZ7FvA48ushJUUdemAMdp9wu/C3OJNI8TZCCa31MCiL++Xm916r+Dm8Z/M0glQaQF4 RhGQ== X-Gm-Message-State: AOAM5324XFsRGPJ7mHk5lwPKVKEWqfGlPzHyG/RObxDANgEMEk4S3MTP OAgEc/5/uGqy0fMdA8OByObU9WvggS2cUQ== X-Google-Smtp-Source: ABdhPJxlBh4bvrdBD0L/VVv54qNpYxETip2YZLNkOOA7J1UG5SBswdc7u+yHF2eM4bgrWhbO6Awb7Q== X-Received: by 2002:a63:4b4c:: with SMTP id k12mr23396441pgl.172.1632147347185; Mon, 20 Sep 2021 07:15:47 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id x17sm14305800pfn.10.2021.09.20.07.15.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:46 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 05/11] bpf: Enable TCP congestion control kfunc from modules Date: Mon, 20 Sep 2021 19:45:20 +0530 Message-Id: <20210920141526.3940002-6-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9235; h=from:subject; bh=TlZ9FFFmnJVTmvWz2hWMu74nVhb5HS3B2twgLBzNnnk=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaEAsA+AKG1bZjDD9oxEmzlDtIa/PUACfoUF2sn kPenQeCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8RyhHCD/ 9HBkfvsrwCgE5+qnpNKNBed3kJ8B75IucYWQcVwB19AzmlihqbyAtaPjg6hpoy5CvywtiG9MMy8PXg Tho5FOeygvJ860ieQgH+/B4WQTn5URiR/gKNByhhzeGG/ZMuehJiatSPEdDfUtEi+q7SvrsKuQn6CF DSAoUCWnSr0ZDz6qkAzbLvlFyL/0YIwjsAfyMZAQlOOsN6y1L9LvWdhWzF4wBExF//zlWTx57Pkjr6 4Oiq9FrOEtkY9cGDzmkne0pLzvUCtym323vk+MK4NyRTuOnDKc/vbuA6NImP70qHHTGrXLAeDXc61e ws0QBgSiVtTVH/VG8j5lxj9ROqH3SccOznQ7C7mR6gYK08Tfv4yBXZJeypbmZDMqqzXfzlAr1Mhv+0 EJ3XF+2MVWn0QOXhkOAznN100Ixn5xj3wLXufj9Jfh0bAnoYROsFdeMcP7s27pk+sPTqvxQZJvE/L6 K9fSUwpk0j9OnDohzhkeVXfXFcV3VAMpuOwqcwyA9/ICbx5jpNi/bHCpggmZMKmhXK+NYSGj6J6hXz ceb2KIXK07Tyeo+dAgISXau8DuwEbRQaHBDL0Vi8a0zkuS5Qvi/J/eePGd2tjaTYUsLprRJPMEw1Uj /CXHuOxNpDnkWYrcMXyb2inr0DDnnUTD2CyVddgj3z4N6xoUNTufvAV6rT7w== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit moves BTF ID lookup into the newly added registration helper, in a way that the bbr, cubic, and dctcp implementation set up their sets in the bpf_tcp_ca kfunc_btf_set list, while the ones not dependent on modules are looked up from the wrapper function. This lifts the restriction for them to be compiled as built in objects, and can be loaded as modules if required. Also modify Makefile.modfinal to resolve_btfids in TCP congestion control modules if the config option is set, using the base BTF support added in the previous commit. See following commits for background on use of: CONFIG_X86 ifdef: 569c484f9995 (bpf: Limit static tcp-cc functions in the .BTF_ids list to x86) CONFIG_DYNAMIC_FTRACE ifdef: 7aae231ac93b (bpf: tcp: Limit calling some tcp cc functions to CONFIG_DYNAMIC_FTRACE) [ resolve_btfids uses --no-fail because some crypto kernel modules under arch/x86/crypto generated from ASM do not have the .BTF sections ] Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 4 ++++ kernel/bpf/btf.c | 3 +++ net/ipv4/bpf_tcp_ca.c | 34 +++------------------------------- net/ipv4/tcp_bbr.c | 28 +++++++++++++++++++++++++++- net/ipv4/tcp_cubic.c | 26 +++++++++++++++++++++++++- net/ipv4/tcp_dctcp.c | 26 +++++++++++++++++++++++++- scripts/Makefile.modfinal | 1 + 7 files changed, 88 insertions(+), 34 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index 7884983857d9..f5ae81e225be 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -269,4 +269,8 @@ static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, struct kfunc_btf_id_set name = { LIST_HEAD_INIT(name.list), (set), \ THIS_MODULE } +extern struct kfunc_btf_id_list bpf_tcp_ca_kfunc_list; + +DECLARE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca); + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index d17f45b163f5..671b4f713a51 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6394,3 +6394,6 @@ EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); mutex_unlock(&list_name.mutex); \ return false; \ } + +DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list); +DEFINE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca, bpf_tcp_ca_kfunc_list); diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index b3afd3361f34..c9f1d2dcf67b 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -223,41 +223,13 @@ BTF_ID(func, tcp_reno_cong_avoid) BTF_ID(func, tcp_reno_undo_cwnd) BTF_ID(func, tcp_slow_start) BTF_ID(func, tcp_cong_avoid_ai) -#ifdef CONFIG_X86 -#ifdef CONFIG_DYNAMIC_FTRACE -#if IS_BUILTIN(CONFIG_TCP_CONG_CUBIC) -BTF_ID(func, cubictcp_init) -BTF_ID(func, cubictcp_recalc_ssthresh) -BTF_ID(func, cubictcp_cong_avoid) -BTF_ID(func, cubictcp_state) -BTF_ID(func, cubictcp_cwnd_event) -BTF_ID(func, cubictcp_acked) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_DCTCP) -BTF_ID(func, dctcp_init) -BTF_ID(func, dctcp_update_alpha) -BTF_ID(func, dctcp_cwnd_event) -BTF_ID(func, dctcp_ssthresh) -BTF_ID(func, dctcp_cwnd_undo) -BTF_ID(func, dctcp_state) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_BBR) -BTF_ID(func, bbr_init) -BTF_ID(func, bbr_main) -BTF_ID(func, bbr_sndbuf_expand) -BTF_ID(func, bbr_undo_cwnd) -BTF_ID(func, bbr_cwnd_event) -BTF_ID(func, bbr_ssthresh) -BTF_ID(func, bbr_min_tso_segs) -BTF_ID(func, bbr_set_state) -#endif -#endif /* CONFIG_DYNAMIC_FTRACE */ -#endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) { - return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); + if (btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id)) + return true; + return __bpf_check_bpf_tcp_ca_kfunc_call(kfunc_btf_id, owner); } static const struct bpf_verifier_ops bpf_tcp_ca_verifier_ops = { diff --git a/net/ipv4/tcp_bbr.c b/net/ipv4/tcp_bbr.c index 6274462b86b4..ec5550089b4d 100644 --- a/net/ipv4/tcp_bbr.c +++ b/net/ipv4/tcp_bbr.c @@ -56,6 +56,8 @@ * otherwise TCP stack falls back to an internal pacing using one high * resolution timer per TCP socket and may use more resources. */ +#include +#include #include #include #include @@ -1152,14 +1154,38 @@ static struct tcp_congestion_ops tcp_bbr_cong_ops __read_mostly = { .set_state = bbr_set_state, }; +BTF_SET_START(tcp_bbr_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, bbr_init) +BTF_ID(func, bbr_main) +BTF_ID(func, bbr_sndbuf_expand) +BTF_ID(func, bbr_undo_cwnd) +BTF_ID(func, bbr_cwnd_event) +BTF_ID(func, bbr_ssthresh) +BTF_ID(func, bbr_min_tso_segs) +BTF_ID(func, bbr_set_state) +#endif +#endif +BTF_SET_END(tcp_bbr_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_bbr_kfunc_ids, tcp_bbr_kfunc_btf_set); + static int __init bbr_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bbr) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&tcp_bbr_cong_ops); + ret = tcp_register_congestion_control(&tcp_bbr_cong_ops); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); + return 0; } static void __exit bbr_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); tcp_unregister_congestion_control(&tcp_bbr_cong_ops); } diff --git a/net/ipv4/tcp_cubic.c b/net/ipv4/tcp_cubic.c index 4a30deaa9a37..5e9d9c51164c 100644 --- a/net/ipv4/tcp_cubic.c +++ b/net/ipv4/tcp_cubic.c @@ -25,6 +25,8 @@ */ #include +#include +#include #include #include #include @@ -482,8 +484,25 @@ static struct tcp_congestion_ops cubictcp __read_mostly = { .name = "cubic", }; +BTF_SET_START(tcp_cubic_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, cubictcp_init) +BTF_ID(func, cubictcp_recalc_ssthresh) +BTF_ID(func, cubictcp_cong_avoid) +BTF_ID(func, cubictcp_state) +BTF_ID(func, cubictcp_cwnd_event) +BTF_ID(func, cubictcp_acked) +#endif +#endif +BTF_SET_END(tcp_cubic_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_cubic_kfunc_ids, tcp_cubic_kfunc_btf_set); + static int __init cubictcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bictcp) > ICSK_CA_PRIV_SIZE); /* Precompute a bunch of the scaling factors that are used per-packet @@ -514,11 +533,16 @@ static int __init cubictcp_register(void) /* divide by bic_scale and by constant Srtt (100ms) */ do_div(cube_factor, bic_scale * 10); - return tcp_register_congestion_control(&cubictcp); + ret = tcp_register_congestion_control(&cubictcp); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); + return 0; } static void __exit cubictcp_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); tcp_unregister_congestion_control(&cubictcp); } diff --git a/net/ipv4/tcp_dctcp.c b/net/ipv4/tcp_dctcp.c index 79f705450c16..0d7ab3cc7b61 100644 --- a/net/ipv4/tcp_dctcp.c +++ b/net/ipv4/tcp_dctcp.c @@ -36,6 +36,8 @@ * Glenn Judd */ +#include +#include #include #include #include @@ -236,14 +238,36 @@ static struct tcp_congestion_ops dctcp_reno __read_mostly = { .name = "dctcp-reno", }; +BTF_SET_START(tcp_dctcp_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, dctcp_init) +BTF_ID(func, dctcp_update_alpha) +BTF_ID(func, dctcp_cwnd_event) +BTF_ID(func, dctcp_ssthresh) +BTF_ID(func, dctcp_cwnd_undo) +BTF_ID(func, dctcp_state) +#endif +#endif +BTF_SET_END(tcp_dctcp_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_dctcp_kfunc_ids, tcp_dctcp_kfunc_btf_set); + static int __init dctcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct dctcp) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&dctcp); + ret = tcp_register_congestion_control(&dctcp); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); + return 0; } static void __exit dctcp_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); tcp_unregister_congestion_control(&dctcp); } diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index ff805777431c..b4f83533eda6 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -41,6 +41,7 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ -f vmlinux ]; then \ LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) --no-fail -s vmlinux $@; \ else \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ fi; From patchwork Mon Sep 20 14:15:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505467 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9811C433FE for ; Mon, 20 Sep 2021 14:15:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B36C460F0F for ; Mon, 20 Sep 2021 14:15:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240115AbhITORX (ORCPT ); Mon, 20 Sep 2021 10:17:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240109AbhITORU (ORCPT ); Mon, 20 Sep 2021 10:17:20 -0400 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DC72C061760; Mon, 20 Sep 2021 07:15:51 -0700 (PDT) Received: by mail-pf1-x442.google.com with SMTP id 203so8518474pfy.13; Mon, 20 Sep 2021 07:15:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dRGphm3b+XT9QQT+YO+BQZpkTXn1fo620/l+VWTepmA=; b=aANnd5bu86kFqZyct/nKh3nZLFuVgp3QdXLZ3+0NiSekm5aLqG09/3Vg8PaQvdoLnj DCCBwF4wKvnnKx7p2SGn9N1qotSec4Mig+mBFh7NIYPXmGj90CWmjSyuSW4k4Irrt8sy y8O5JLwpAjz2vzE4pl30B0zDVvv76nG7rLlFQLbNkfbwnr6ClPl0VI2wYFkoOcXEG2JI fj03xgfOwEz9KDee0XeueoE2EO1opGu1kHVrVtSSqT/740TFXXFPEwqzWRTEM13hxH3T SQff7fY7x8q2k2NLi4upjFgIr+pgPK73K1pch3iko/FyruzyigtCpJJZb0iw5bHX32jd sDnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dRGphm3b+XT9QQT+YO+BQZpkTXn1fo620/l+VWTepmA=; b=7no1/+Kr/hgohVQrZETGhkVbtyY8E1t0vgRVyrG9LJ7cbir478OfDky6kxOgn6iZRB UuOtBI8i2tKRvk0gD1N0wfTz0wa8faWzJa4toEqUF0SwPROeHckMK7PxOikDAXyrFNGS 07nzJgNdSo8f0IRRlb+NcCRkeXyqzYPF4oY3+9aLEzWmtvQxzF5IxNDOdvcZiY2cP4Mx S1dvoBqhnG8oPmTfDM63iLfp2VzDBXq2xAA6jJhTCTICAlbcjyRWpVoxxAZKZG5bxBuF LBvGWQ92MShCtJBge8w/3TnPPaIWj+qPSqRWGIZhmEmP6HTb2bcViOtH6yBFyTkDOEah UGAQ== X-Gm-Message-State: AOAM531cUnFbUAorR/zJ25zcrFP1GhiqM1pceHCB12tbjPXmlKrOPblv yjJVn4gk6y9L2HtKuF1gkQnq1pdoE3j2+Q== X-Google-Smtp-Source: ABdhPJy7ARAbSva0OXWwMgurIYfCzz+mX9PFRBF+qIQK8K+eYl5LAxDCmohMQoi+fU9EtB+MTpcXww== X-Received: by 2002:a63:2b4b:: with SMTP id r72mr2579896pgr.57.1632147350453; Mon, 20 Sep 2021 07:15:50 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id h15sm2392715pjg.34.2021.09.20.07.15.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:50 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 06/11] libbpf: Support kernel module function calls Date: Mon, 20 Sep 2021 19:45:21 +0530 Message-Id: <20210920141526.3940002-7-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4883; h=from:subject; bh=NzeLk3EBOD7kX+YzOxCmipgaRdkG/mH0QJLLStwF1Y0=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaEVy6AuHGJjWhyXdSSiK9J799QWgbPiW0OO2Yr ykXEGWmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8Rym0kD/ 9BEsAen1zi7NqZgAnfqTZQCWPoMZk6ZR2W2wTbBafhB/AbIlKvyWXlrVnBtBLoRS282yBs9zJgZnmF M6JZWr+to5h9NStXRwUK58bMC3gUc/Qcq5ni9HQS0DL0+viFA2+ZcX4LmPRCT6vqueL3NFZ26frMQX nXjRXEuhe4NLVl3V1Yla7KUvQSzmD64Wg6A8n3ypmPAoDTzgT/BYtgYlUXhaZUggoMjMLLS6DMUUuH PHPG4MIoDvjhCOLOF9wNud63P0PX/MlH2rKqg61UtFIbCDd1FbNV1j2Die9SQ4RDgh5qcYPD/2ecav oChhG42azgp20W6YMRsq47/B0TrDi7xReVzt0VWEEaQWSeXGW+xjj0CUzqsc9w3S3Hjyyinw+yT5Ip Vh4EmaqksDdQnOWHaX25A7Tuif59ZEclKVLRM0CG/d/0w5VT0eA7J3uOjqGV8qGo88UoFXbNzX8hSN hYgOXXwW9qioPpS0N5RGnKo95F03nVVMipoO0pSJWFICAHhEGbEuagutRfvlwR3qSMbTRYj3TILytf MrwTm/52cy8ignPtj4LrbiUSou2QA8FAwU+SBZXAu6v99p++C6t1D2F/NKF/JKMWRfW/Z9bWhBumWi lfM5G9GIMfd1BxbfqZKmU7AZX4WJ7XvfOOUDITN7A1OBuOpkBQ/QNEH+tUSw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch adds libbpf support for kernel module function call support. The fd_array parameter is used during BPF program load is used to pass module BTFs referenced by the program. insn->off is set to index into this array, but starts from 1, because insn->off as 0 is reserved for btf_vmlinux. We try to use existing insn->off for a module, since the kernel limits the maximum distinct module BTFs for kfuncs to 256, and also because index must never exceed the maximum allowed value that can fit in insn->off (INT16_MAX). In the future, if kernel interprets signed offset as unsigned for kfunc calls, this limit can be increased to UINT16_MAX. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 58 +++++++++++++++++++++++++++++++-- tools/lib/bpf/libbpf_internal.h | 1 + 3 files changed, 57 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2401fad090c5..7d1741ceaa32 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -264,6 +264,7 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr) attr.line_info_rec_size = load_attr->line_info_rec_size; attr.line_info_cnt = load_attr->line_info_cnt; attr.line_info = ptr_to_u64(load_attr->line_info); + attr.fd_array = ptr_to_u64(load_attr->fd_array); if (load_attr->name) memcpy(attr.prog_name, load_attr->name, diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index da65a1666a5e..3049dfc6088e 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -420,6 +420,12 @@ struct extern_desc { /* local btf_id of the ksym extern's type. */ __u32 type_id; + /* offset to be patched in for insn->off, + * this is 0 for btf_vmlinux, and index + 1 + * for module BTF, where index is BTF index in + * obj->fd_array + */ + __s16 offset; } ksym; }; }; @@ -516,6 +522,10 @@ struct bpf_object { void *priv; bpf_object_clear_priv_t clear_priv; + int *fd_array; + size_t fd_cap_cnt; + int nr_fds; + char path[]; }; #define obj_elf_valid(o) ((o)->efile.elf) @@ -5357,6 +5367,7 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6151,6 +6162,7 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt, } load_attr.log_level = prog->log_level; load_attr.prog_flags = prog->prog_flags; + load_attr.fd_array = prog->obj->fd_array; if (prog->obj->gen_loader) { bpf_gen__prog_load(prog->obj->gen_loader, &load_attr, @@ -6763,9 +6775,46 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, } if (kern_btf != obj->btf_vmlinux) { - pr_warn("extern (func ksym) '%s': function in kernel module is not supported\n", - ext->name); - return -ENOTSUP; + int index = -1; + + if (!obj->fd_array) { + obj->fd_array = calloc(8, sizeof(*obj->fd_array)); + if (!obj->fd_array) + return -ENOMEM; + obj->fd_cap_cnt = 8; + /* index = 0 is for vmlinux BTF, so skip it */ + obj->nr_fds = 1; + } + + for (int i = 0; i < obj->nr_fds; i++) { + if (obj->fd_array[i] == kern_btf_fd) { + index = i; + break; + } + } + + if (index == -1) { + if (obj->nr_fds == obj->fd_cap_cnt) { + ret = libbpf_ensure_mem((void **)&obj->fd_array, + &obj->fd_cap_cnt, sizeof(int), + obj->fd_cap_cnt + 1); + if (ret) + return ret; + } + + index = obj->nr_fds; + obj->fd_array[obj->nr_fds++] = kern_btf_fd; + } + + if (index > INT16_MAX) { + /* insn->off is s16 */ + pr_warn("extern (func ksym) '%s': module btf fd index too big\n", + ext->name); + return -E2BIG; + } + ext->ksym.offset = index; + } else { + ext->ksym.offset = 0; } kern_func = btf__type_by_id(kern_btf, kfunc_id); @@ -6941,6 +6990,9 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr) err = bpf_gen__finish(obj->gen_loader); } + /* clean up fd_array */ + zfree(&obj->fd_array); + /* clean up module BTFs */ for (i = 0; i < obj->btf_module_cnt; i++) { close(obj->btf_modules[i].fd); diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index ceb0c98979bc..44b8f381b035 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -291,6 +291,7 @@ struct bpf_prog_load_params { __u32 log_level; char *log_buf; size_t log_buf_sz; + int *fd_array; }; int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr); From patchwork Mon Sep 20 14:15:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505469 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DD9EC4332F for ; Mon, 20 Sep 2021 14:15:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 368C060F0F for ; Mon, 20 Sep 2021 14:15:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237342AbhITORY (ORCPT ); Mon, 20 Sep 2021 10:17:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231276AbhITORU (ORCPT ); Mon, 20 Sep 2021 10:17:20 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C865C061764; Mon, 20 Sep 2021 07:15:54 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id g184so17466543pgc.6; Mon, 20 Sep 2021 07:15:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=te9g4sIFu/T32nk42OfmNuAINw4PTbgEMbJ1MLNAZRY=; b=PEujym+HhOfZ6b/S6Ai3WOVRqS6uh6YyEq5OdFo+6kX1qn1dmBH5p2NjAU5MymteQ2 eDlbUfblawUklzBO/JvOVBxCsU4qeFa1EYrQhsHGzCPH3nlmVR1MNjkxTLXqUCHmdKYK lspA7IwN/1J0jxFkxzqUHdqPjegcssj+sROaz5bGNXslb/1sDdUmVicpaObYDg6ArzIW G+LShCKyiaVTrUM7ym+6XsningCAdhrK1MoAEuij9/8dBPTgtl7wYh+kdXKKrwsb7cLc NJX1EdGRW+Z2nBSc4z9M/FUguTbwjZuasQ4m5vdx7sX1emcSTGnQsxspi5KV5Xjz2Ndy qlfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=te9g4sIFu/T32nk42OfmNuAINw4PTbgEMbJ1MLNAZRY=; b=hdGzNDO5myloIrOkjeSelJFy6JFFDVu1Jrud3HDHdUzQPND/h4Axm1xzAxtFyAOWtE O83k+lN/vMtNlVSFx3U0wpTsxQvXsGFUmrkt5T1POCvWXaZDz/7e1x+pALQOH8BqukqD 8/dt2fk8S3yYfTNY4sXxYvUSatsQPRq8JBqJ5Qd1p8QUAo6uQh+M1wVwDWCj7IMhOUou 5NK4KzLAH1jNL8VJ9425+FnN1m8zrkqzCIQolRu7k8nLmr5ueDuJHnWY/Ag38lv8tdhY zhNlNupElGOuPOH/HVs3DimjijH1jc9x3XCrHOW/I7Klsp5ehppvT8iHFa289CuetwJR LqHw== X-Gm-Message-State: AOAM5302lZ+3yzT1+tngETO9kwbMg8xiCjlc95G8bB0iRlkP8iINlmZK jo/DoLym2oEpMwmQToE9fvE+NwgN8vTzHA== X-Google-Smtp-Source: ABdhPJwxWKG7W0EPGaX4wdLYUy25V7c9ZBwRQDCOV4lMepVVbvUpf/l2h2Z2UrcYbj4xSurqd4qEhg== X-Received: by 2002:a65:51c7:: with SMTP id i7mr23617938pgq.300.1632147353456; Mon, 20 Sep 2021 07:15:53 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id e7sm5149504pfc.114.2021.09.20.07.15.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:53 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 07/11] libbpf: Resolve invalid weak kfunc calls with imm = 0, off = 0 Date: Mon, 20 Sep 2021 19:45:22 +0530 Message-Id: <20210920141526.3940002-8-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2121; h=from:subject; bh=DIKZk02eVbFz8E418BE9H99i/d/b1w8QfPx3AHedmCw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaE6bY7tiJUVYdXMNqPlgqVEb1XuP3C5mvGecT0 TVm4drGJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8RyusxD/ 9mcB4O+uz9l2RhaLJsLWuGjve1XsV//MBEyGg8Xt+3wBUE1OIE1T0xPU1TMeC9A2TznHA4zYCjNwZq aIW2hqJA4WLtGRguBeADXbpq8WK3Je7oWWoVsZX6qKMIqs3b7y5NfWd8+G7lD05KFEHWGWd+DWvJi+ zzBAq/+FFmP0MuQeybmdEVIMJxhp/PNwBdKDIwOQmpBhIc4mjQZCNqAgISqHlSrF6kNXme6p1WhzwP MqeSNaVOc/B8p2THOaR+/3kzueyVSD1+wqTxmmNzRs+PfaFAEDRpB9Ov8WzlBb+MtdW6YFdoeTgYf1 4XGb44bOrLLUA0OZasGwIsWVzDFP6ca6fERtp5rEn1yOsJ2bhRf93IBjY9uWG+bomXilFopQkfgf5v wqHXgQB4BviioxRo3HjDMG5VDO1594n8e2VRtiZ7FlCgUHCIGK/6b7ekzjc8BNMkH+T2tpGt+icn7R ubnZhqrNnXQxZqBAxfu/URagAUjfyhya1hDXL68Gu5HKpX2qNMwtFIAWgi1s/jkJSeaAblBZ3JcDxq bXl4+MLdvy7BvYUguw0QPmYRGCD70yx+HZQq1nk7qvHTwj5JffIaauQuDRhvfxB8hFUM0OU4ZxH9kG zJQEJf6r0LC0wsOSMsDVybtR3v127RjjFZ7NNHApzFqEQ3oJayWx0LH/00vQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Preserve these calls as it allows verifier to succeed in loading the program if they are determined to be unreachable after dead code elimination during program load. If not, the verifier will fail at runtime. This is done for ext->is_weak symbols similar to the case for variable ksyms. Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Andrii Nakryiko --- tools/lib/bpf/libbpf.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 3049dfc6088e..3c195eaadf56 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -3413,11 +3413,6 @@ static int bpf_object__collect_externs(struct bpf_object *obj) return -ENOTSUP; } } else if (strcmp(sec_name, KSYMS_SEC) == 0) { - if (btf_is_func(t) && ext->is_weak) { - pr_warn("extern weak function %s is unsupported\n", - ext->name); - return -ENOTSUP; - } ksym_sec = sec; ext->type = EXT_KSYM; skip_mods_and_typedefs(obj->btf, t->type, @@ -5366,8 +5361,12 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) case RELO_EXTERN_FUNC: ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; - insn[0].imm = ext->ksym.kernel_btf_id; - insn[0].off = ext->ksym.offset; + if (ext->is_set) { + insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; + } else { /* unresolved weak kfunc */ + insn[0].imm = insn[0].off = 0; + } break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6768,8 +6767,10 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, kfunc_id = find_ksym_btf_id(obj, ext->name, BTF_KIND_FUNC, &kern_btf, &kern_btf_fd); - if (kfunc_id < 0) { - pr_warn("extern (func ksym) '%s': not found in kernel BTF\n", + if (kfunc_id == -ESRCH && ext->is_weak) { + return 0; + } else if (kfunc_id < 0) { + pr_warn("extern (func ksym) '%s': not found in kernel or module BTFs\n", ext->name); return kfunc_id; } From patchwork Mon Sep 20 14:15:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505471 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69580C433F5 for ; Mon, 20 Sep 2021 14:16:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4A603610CE for ; Mon, 20 Sep 2021 14:16:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240075AbhITORa (ORCPT ); Mon, 20 Sep 2021 10:17:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237173AbhITORY (ORCPT ); Mon, 20 Sep 2021 10:17:24 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C9F5C061574; Mon, 20 Sep 2021 07:15:57 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id on12-20020a17090b1d0c00b001997c60aa29so34236pjb.1; Mon, 20 Sep 2021 07:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9SO3kL5K2fSLEp8aEfH2nqz/JNKO6SY8nNAA+9KbF94=; b=CFeXiRwZFkha2+6K5c24ivK4mldUgcZtsl10wijX9jmWIF6CUHyqRqKEFZSr2JRaU2 nun07B0bROEV03CqQrhh0g8LpGO+tF8BUcrOsKnvvQ3HSlMdvLJgbgQP/NqH2+ADMADn Rk2byIiLHYFgUwD0dOmGdgQpL9/a0w8QGTh1zPuZUAamscSb6SiKMasMmAHqUoNe5gPh 0ihmy59g6gNsKJIWwN4j9tZPaY0XZZVZVCV400zTt/23+m6xQ6d4B1ublbV6chbw+BvW mUueEv8GlZIAP+/3njyoo064IVe1aqpiueFDY3hPqIO49Zu6EtLclokkGP/lJeAN3BZ2 YH5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9SO3kL5K2fSLEp8aEfH2nqz/JNKO6SY8nNAA+9KbF94=; b=c6nkTRNgvE97k5v4TEF56VvgP0ug/McWfsTttILtR1zeduhXwG9YBMwTC4RM+lxC7V OusaaePN4jNl/E2IUfzfKBh3qXiUs1MuNLGl1DxBwPk96/uEuXVoQ9BikwzesHtV4c10 a7UpeUXGnlqgq+MjJVnzl1T2pi0IiEIg3zd3dRf5Tu8i5EYKjsYRRqwAoA9bElAn454J NuXpKdmcr9RU1+eHknE234lJVNkdXVQYKW5xp3r5VxTk3i4nvWZvnTebODurfDEBWuUD BQH7DIMNaM+rOKOS9xQrUvTfoWbhLYxjNd3s8BH/kPYS2yv7xcQGSZLBGYGdVG0c1qYr d7Bg== X-Gm-Message-State: AOAM531joYExV6Ralg+eHBr8wdQeTbanyGne5e7YVwkUqzVyjBQ08z3c Y3qhtrQal3HLgp0vzPkVqfTbp0fxmeInxA== X-Google-Smtp-Source: ABdhPJyfbrcR4e4w3gNlo0vjDEfGK1J8JvQFzSOANC+5miaK+T3kNBIPUgn5R31ubcdCjiYWUNJK2Q== X-Received: by 2002:a17:902:d202:b0:13a:709b:dfb0 with SMTP id t2-20020a170902d20200b0013a709bdfb0mr23057456ply.34.1632147356549; Mon, 20 Sep 2021 07:15:56 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id n15sm12850973pjj.36.2021.09.20.07.15.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:56 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 08/11] libbpf: Update gen_loader to emit BTF_KIND_FUNC relocations Date: Mon, 20 Sep 2021 19:45:23 +0530 Message-Id: <20210920141526.3940002-9-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=20790; h=from:subject; bh=9CJC2xQ8mHQae/Vr5ETzWMQvkQtSSCIGi9QHwLsnR3s=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaE59gJH3MXM0Yklle2XZJRrdcX7KlPXwZ6H4pK Da3ewOiJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8Ryln7D/ kBsJ3zzlwBE3U9VnUlxD51pQL64PomtIsVXUpR/OcZMO6zXviPHacvT7P1lQNJynZLmGmJY1emtQOC U9cTeXI/Mqx37tGbJNDDyn3xlUibSFKnVg/X2VvUHM4tDzOcfkiflWMbEwfyeBDW5tCpK5GTw+ErhI 1p8LpvT3Lcs1dWz1po5x++NK00rDvI2DHBZr1xjNNH9UWjDkB6LRDQjugRUfHHAx+h3UyNLFgX/YZ9 MOewp70tEmFaw9nQZc1J6gtj8u2QARnW+LzQYRD9G6vxPoVo0Jt+1NRBDYNYSJox/fR1tMgdpevalL xzy/bN9eWKc6G1T+sz9CSnvYzsL8QEptBRRd4ceQ2BpRCcAyzrGaPCYVus8IE/bAOPep/KVvUwfiO+ pGuJKeWcILm4cAu2G7Jhh7hlrWhhg0wGMhPLxxzZpBAXPxzzJwpQNaIkthF+VBpO+3VY8uX9Rg9Bmm 84xPMAckJlTQ9ulVtnABp6sqlIT6eQRqnoctQCqRISO+NCSzerkpkeDAmlBKYUEFvKl8c80RK6rxiW kjhLPtGEcsUGZH1Aoy2KCwFClylM+Y2DgYyIPYjhW/uV/w3KnFilK0DRy/fE4EEO7HdON2eIBIcv+N QszZ3YrLJLgjxF/VnaoTXvysrGd5OLgLYQSOneqvGISls423Nr5NFHrL7mhg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This change updates the BPF syscall loader to relocate BTF_KIND_FUNC relocations, with support for weak kfunc relocations. The next commit adds bpftool supports to set up the fd_array_sz parameter for light skeleton. A second map for keeping fds is used instead of adding fds to existing loader.map because of following reasons: If reserving an area for map and BTF fds, we would waste the remaining of (MAX_USED_MAPS + MAX_KFUNC_DESCS) * sizeof(int), which in most cases will be unused by the program. Also, we must place some limit on the amount of map and BTF fds a program can possibly open. If setting gen->fd_array to first map_fd offset, and then just finding the offset relative to this (for later BTF fds), such that they can be packed without wasting space, we run the risk of unnecessarily running out of valid offset for emit_relo stage (for kfuncs), because gen map creation and relocation stages are separated by other steps that can add lots of data (including bpf_object__populate_internal_map). It is also prone to break silently if features are added between map and BTF fd emits that possibly add more data (just ~128KB to break BTF fd, since insn->off allows for INT16_MAX (32767) * 4 bytes). Both of these issues are compounded by the fact that data map is shared by all programs, so it is easy to end up with invalid offset for BTF fd. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf_gen_internal.h | 16 ++- tools/lib/bpf/gen_loader.c | 222 ++++++++++++++++++++++++++----- tools/lib/bpf/libbpf.c | 8 +- tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/skel_internal.h | 27 +++- 5 files changed, 230 insertions(+), 44 deletions(-) diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_internal.h index 615400391e57..c4aa86865b65 100644 --- a/tools/lib/bpf/bpf_gen_internal.h +++ b/tools/lib/bpf/bpf_gen_internal.h @@ -7,6 +7,16 @@ struct ksym_relo_desc { const char *name; int kind; int insn_idx; + bool is_weak; +}; + +/* maximum distinct calls */ +#define MAX_KFUNC_DESCS 256 + +struct kfunc_desc { + const char *name; + int ref; + int off; }; struct bpf_gen { @@ -23,7 +33,10 @@ struct bpf_gen { struct ksym_relo_desc *relos; int relo_cnt; char attach_target[128]; + struct kfunc_desc kdescs[MAX_KFUNC_DESCS]; int attach_kind; + __u32 nr_kfuncs; + int fd_array_sz; }; void bpf_gen__init(struct bpf_gen *gen, int log_level); @@ -36,6 +49,7 @@ void bpf_gen__prog_load(struct bpf_gen *gen, struct bpf_prog_load_params *load_a void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *value, __u32 value_size); void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx); void bpf_gen__record_attach_target(struct bpf_gen *gen, const char *name, enum bpf_attach_type type); -void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, int insn_idx); +void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak, int kind, + int insn_idx); #endif diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 8df718a6b142..c471e0844b22 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "btf.h" #include "bpf.h" #include "libbpf.h" @@ -13,7 +14,6 @@ #include "bpf_gen_internal.h" #include "skel_internal.h" -#define MAX_USED_MAPS 64 #define MAX_USED_PROGS 32 /* The following structure describes the stack layout of the loader program. @@ -26,10 +26,10 @@ * stack - bpf program stack * blob - bpf_attr-s, strings, insns, map data. * All the bytes that loader prog will use for read/write. + * fd_blob - map fds, kfunc module btf fds */ struct loader_stack { __u32 btf_fd; - __u32 map_fd[MAX_USED_MAPS]; __u32 prog_fd[MAX_USED_PROGS]; __u32 inner_map_fd; }; @@ -145,6 +145,16 @@ static int add_data(struct bpf_gen *gen, const void *data, __u32 size) return prev - gen->data_start; } +/* return offset in fd_blob */ +static int add_fd(struct bpf_gen *gen) +{ + int prev; + + prev = gen->fd_array_sz; + gen->fd_array_sz += sizeof(int); + return prev; +} + static int insn_bytes_to_bpf_size(__u32 sz) { switch (sz) { @@ -166,16 +176,34 @@ static void emit_rel_store(struct bpf_gen *gen, int off, int data) emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0)); } -/* *(u64 *)(blob + off) = (u64)(void *)(%sp + stack_off) */ -static void emit_rel_store_sp(struct bpf_gen *gen, int off, int stack_off) +/* *(u64 *)(blob + off) = (u64)(void *)(fd_blob + data) */ +static void emit_rel_store_fd(struct bpf_gen *gen, int off, int data) { - emit(gen, BPF_MOV64_REG(BPF_REG_0, BPF_REG_10)); - emit(gen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, stack_off)); + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 1, data)); emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0, 0, 0, off)); emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0)); } +static void move_fd_blob2blob(struct bpf_gen *gen, int off, int size, int blob_off) +{ + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, off)); + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_7, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 1, blob_off)); + emit(gen, BPF_LDX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_0, BPF_REG_7, 0)); + emit(gen, BPF_STX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_1, BPF_REG_0, 0)); +} + +static void move_fd_blob2ctx(struct bpf_gen *gen, int off, int size, int blob_off) +{ + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 1, blob_off)); + emit(gen, BPF_LDX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_0, BPF_REG_1, 0)); + emit(gen, BPF_STX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_6, BPF_REG_0, off)); +} + static void move_ctx2blob(struct bpf_gen *gen, int off, int size, int ctx_off, bool check_non_zero) { @@ -300,14 +328,24 @@ static void emit_sys_close_stack(struct bpf_gen *gen, int stack_off) __emit_sys_close(gen); } -static void emit_sys_close_blob(struct bpf_gen *gen, int blob_off) +static void __emit_sys_close_blob(struct bpf_gen *gen, int blob_idx, int blob_off) { emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, - 0, 0, 0, blob_off)); + 0, 0, blob_idx, blob_off)); emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0)); __emit_sys_close(gen); } +static void emit_sys_close_blob(struct bpf_gen *gen, int blob_off) +{ + __emit_sys_close_blob(gen, 0, blob_off); +} + +static void emit_sys_close_fd_blob(struct bpf_gen *gen, int blob_off) +{ + __emit_sys_close_blob(gen, 1, blob_off); +} + int bpf_gen__finish(struct bpf_gen *gen) { int i; @@ -321,11 +359,11 @@ int bpf_gen__finish(struct bpf_gen *gen) offsetof(struct bpf_prog_desc, prog_fd), 4, stack_off(prog_fd[i])); for (i = 0; i < gen->nr_maps; i++) - move_stack2ctx(gen, - sizeof(struct bpf_loader_ctx) + - sizeof(struct bpf_map_desc) * i + - offsetof(struct bpf_map_desc, map_fd), 4, - stack_off(map_fd[i])); + move_fd_blob2ctx(gen, + sizeof(struct bpf_loader_ctx) + + sizeof(struct bpf_map_desc) * i + + offsetof(struct bpf_map_desc, map_fd), 4, + sizeof(int) * i); emit(gen, BPF_MOV64_IMM(BPF_REG_0, 0)); emit(gen, BPF_EXIT_INSN()); pr_debug("gen: finish %d\n", gen->error); @@ -336,6 +374,7 @@ int bpf_gen__finish(struct bpf_gen *gen) opts->insns_sz = gen->insn_cur - gen->insn_start; opts->data = gen->data_start; opts->data_sz = gen->data_cur - gen->data_start; + opts->fd_array_sz = gen->fd_array_sz; } return gen->error; } @@ -385,7 +424,7 @@ void bpf_gen__map_create(struct bpf_gen *gen, { int attr_size = offsetofend(union bpf_attr, btf_vmlinux_value_type_id); bool close_inner_map_fd = false; - int map_create_attr; + int map_create_attr, off; union bpf_attr attr; memset(&attr, 0, attr_size); @@ -462,8 +501,10 @@ void bpf_gen__map_create(struct bpf_gen *gen, gen->error = -EDOM; /* internal bug */ return; } else { - emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_7, - stack_off(map_fd[map_idx]))); + off = add_fd(gen); + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 1, off)); + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_7, 0)); gen->nr_maps++; } if (close_inner_map_fd) @@ -506,8 +547,8 @@ static void emit_find_attach_target(struct bpf_gen *gen) */ } -void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, - int insn_idx) +void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak, + int kind, int insn_idx) { struct ksym_relo_desc *relo; @@ -519,11 +560,119 @@ void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, gen->relos = relo; relo += gen->relo_cnt; relo->name = name; + relo->is_weak = is_weak; relo->kind = kind; relo->insn_idx = insn_idx; gen->relo_cnt++; } +/* returns existing kfunc_desc with ref incremented, or inserts a new one */ +static struct kfunc_desc *get_kfunc_desc(struct bpf_gen *gen, const char *name) +{ + struct kfunc_desc *kdesc; + + for (int i = 0; i < gen->nr_kfuncs; i++) { + if (!strcmp(gen->kdescs[i].name, name)) { + gen->kdescs[i].ref++; + return &gen->kdescs[i]; + } + } + if (gen->nr_kfuncs == ARRAY_SIZE(gen->kdescs)) + return NULL; + kdesc = &gen->kdescs[gen->nr_kfuncs++]; + kdesc->name = name; + kdesc->ref = 1; + kdesc->off = 0; + return kdesc; +} + +/* Expects: + * BPF_REG_0 - pointer to instruction + * BPF_REG_7 - return value of bpf_btf_find_by_name_kind + * + * We need to reuse BTF fd for same symbol otherwise each relocation takes a new + * index, while kernel limits total kfunc BTFs to 256. For duplicate symbols, + * this would mean a new BTF fd index for each entry. By pairing symbol name + * with index, we get the insn->imm, insn->off pairing that kernel uses for + * kfunc_tab, which becomes the effective limit even though all of them may + * share same index in fd_array (such that kfunc_btf_tab has 1 element). + */ +static void emit_relo_kfunc_btf(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insn) +{ + struct kfunc_desc *kdesc, def_kdesc; + int off, btf_fd; + + kdesc = get_kfunc_desc(gen, relo->name); + if (!kdesc) { + /* Fallback to storing fd in fd_array, and let kernel handle too many BTFs/kfuncs */ + pr_warn("Out of slots for kfunc %s, disabled BTF fd dedup for relocation\n", + relo->name); + def_kdesc.name = relo->name; + def_kdesc.ref = 1; + def_kdesc.off = 0; + kdesc = &def_kdesc; + } + + btf_fd = kdesc->ref > 1 ? kdesc->off : add_fd(gen); + /* load slot pointer */ + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_8, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 1, btf_fd)); + /* Try to map one insn->off to one insn->imm */ + if (kdesc->ref > 1) { + emit(gen, BPF_MOV64_REG(BPF_REG_9, BPF_REG_7)); + goto skip_btf_fd; + } else { + /* cannot use index == 0 */ + if (!btf_fd) { + btf_fd = add_fd(gen); + /* shift to next slot */ + emit(gen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_8, sizeof(int))); + } + kdesc->off = btf_fd; + } + + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_8, 0, 0)); + emit(gen, BPF_MOV64_REG(BPF_REG_9, BPF_REG_7)); + /* store BTF fd if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 3)); + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32)); + /* store BTF fd in slot */ + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_8, BPF_REG_7, 0)); + emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_9)); +skip_btf_fd: + /* remember BTF fd to skip insn->off store for vmlinux case */ + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_9, 32)); + /* store index into insn[insn_idx].off */ + off = btf_fd / sizeof(int); + off = off > INT16_MAX ? 0 : off; + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_0, offsetof(struct bpf_insn, off), 0)); + /* skip insn->off store if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 2)); + /* skip if vmlinux BTF */ + emit(gen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_9, 0, 1)); + /* store insn->off as the index of BTF fd */ + emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_0, offsetof(struct bpf_insn, off), off)); + /* close fd that we skipped storing in fd_array map */ + if (kdesc->ref > 1) { + emit(gen, BPF_MOV64_REG(BPF_REG_1, BPF_REG_9)); + __emit_sys_close(gen); + } + if (gen->log_level) { + /* reload register 0, overwritten by sys_close */ + if (kdesc->ref > 1) + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, insn)); + emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, + offsetof(struct bpf_insn, imm))); + emit(gen, BPF_LDX_MEM(BPF_H, BPF_REG_8, BPF_REG_0, + offsetof(struct bpf_insn, off))); + debug_regs(gen, BPF_REG_7, BPF_REG_8, "sym (%s:%d): imm: %%d, off: %%d", + relo->name, kdesc->ref); + } +} + static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insns) { int name, insn, len = strlen(relo->name) + 1; @@ -539,18 +688,24 @@ static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insn emit(gen, BPF_EMIT_CALL(BPF_FUNC_btf_find_by_name_kind)); emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_0)); debug_ret(gen, "find_by_name_kind(%s,%d)", relo->name, relo->kind); - emit_check_err(gen); + /* if not weak kfunc, emit err check */ + if (relo->kind != BTF_KIND_FUNC || !relo->is_weak) + emit_check_err(gen); + insn = insns + sizeof(struct bpf_insn) * relo->insn_idx; + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, 0, 0, 0, insn)); + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_0, offsetof(struct bpf_insn, imm), 0)); + /* skip success case store if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 1)); /* store btf_id into insn[insn_idx].imm */ - insn = insns + sizeof(struct bpf_insn) * relo->insn_idx + - offsetof(struct bpf_insn, imm); - emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, - 0, 0, 0, insn)); - emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, 0)); + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, offsetof(struct bpf_insn, imm))); if (relo->kind == BTF_KIND_VAR) { /* store btf_obj_fd into insn[insn_idx + 1].imm */ emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32)); emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, - sizeof(struct bpf_insn))); + sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm))); + } else if (relo->kind == BTF_KIND_FUNC) { + emit_relo_kfunc_btf(gen, relo, insn); } } @@ -558,6 +713,8 @@ static void emit_relos(struct bpf_gen *gen, int insns) { int i; + /* clear kfunc_desc table */ + gen->nr_kfuncs = 0; for (i = 0; i < gen->relo_cnt; i++) emit_relo(gen, gen->relos + i, insns); } @@ -566,6 +723,8 @@ static void cleanup_relos(struct bpf_gen *gen, int insns) { int i, insn; + for (i = 0; i < gen->nr_kfuncs; i++) + emit_sys_close_fd_blob(gen, gen->kdescs[i].off); for (i = 0; i < gen->relo_cnt; i++) { if (gen->relos[i].kind != BTF_KIND_VAR) continue; @@ -632,9 +791,8 @@ void bpf_gen__prog_load(struct bpf_gen *gen, /* populate union bpf_attr with a pointer to line_info */ emit_rel_store(gen, attr_field(prog_load_attr, line_info), line_info); - /* populate union bpf_attr fd_array with a pointer to stack where map_fds are saved */ - emit_rel_store_sp(gen, attr_field(prog_load_attr, fd_array), - stack_off(map_fd[0])); + /* populate union bpf_attr fd_array with a pointer to data where map_fds are saved */ + emit_rel_store_fd(gen, attr_field(prog_load_attr, fd_array), 0); /* populate union bpf_attr with user provided log details */ move_ctx2blob(gen, attr_field(prog_load_attr, log_level), 4, @@ -701,8 +859,8 @@ void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *pvalue, emit(gen, BPF_EMIT_CALL(BPF_FUNC_copy_from_user)); map_update_attr = add_data(gen, &attr, attr_size); - move_stack2blob(gen, attr_field(map_update_attr, map_fd), 4, - stack_off(map_fd[map_idx])); + move_fd_blob2blob(gen, attr_field(map_update_attr, map_fd), 4, + sizeof(int) * map_idx); emit_rel_store(gen, attr_field(map_update_attr, key), key); emit_rel_store(gen, attr_field(map_update_attr, value), value); /* emit MAP_UPDATE_ELEM command */ @@ -720,8 +878,8 @@ void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx) memset(&attr, 0, attr_size); pr_debug("gen: map_freeze: idx %d\n", map_idx); map_freeze_attr = add_data(gen, &attr, attr_size); - move_stack2blob(gen, attr_field(map_freeze_attr, map_fd), 4, - stack_off(map_fd[map_idx])); + move_fd_blob2blob(gen, attr_field(map_freeze_attr, map_fd), 4, + sizeof(int) * map_idx); /* emit MAP_FREEZE command */ emit_sys_bpf(gen, BPF_MAP_FREEZE, map_freeze_attr, attr_size); debug_ret(gen, "map_freeze"); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 3c195eaadf56..77dffb66c1fd 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6264,12 +6264,12 @@ static int bpf_program__record_externs(struct bpf_program *prog) ext->name); return -ENOTSUP; } - bpf_gen__record_extern(obj->gen_loader, ext->name, BTF_KIND_VAR, - relo->insn_idx); + bpf_gen__record_extern(obj->gen_loader, ext->name, ext->is_weak, + BTF_KIND_VAR, relo->insn_idx); break; case RELO_EXTERN_FUNC: - bpf_gen__record_extern(obj->gen_loader, ext->name, BTF_KIND_FUNC, - relo->insn_idx); + bpf_gen__record_extern(obj->gen_loader, ext->name, ext->is_weak, + BTF_KIND_FUNC, relo->insn_idx); break; default: continue; diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index c90e3d79e72c..fdcdcfab5bce 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -885,6 +885,7 @@ struct gen_loader_opts { const char *insns; __u32 data_sz; __u32 insns_sz; + __u32 fd_array_sz; }; #define gen_loader_opts__last_field insns_sz diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index b22b50c1b173..6c0f0adfd42f 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -44,6 +44,7 @@ struct bpf_load_and_run_opts { const void *insns; __u32 data_sz; __u32 insns_sz; + __u32 fd_array_sz; const char *errstr; }; @@ -62,31 +63,41 @@ static inline int skel_closenz(int fd) static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) { - int map_fd = -1, prog_fd = -1, key = 0, err; + int map_fd[2] = {-1, -1}, prog_fd = -1, key = 0, err; union bpf_attr attr; - map_fd = bpf_create_map_name(BPF_MAP_TYPE_ARRAY, "__loader.map", 4, + map_fd[0] = bpf_create_map_name(BPF_MAP_TYPE_ARRAY, "__loader.map", 4, opts->data_sz, 1, 0); - if (map_fd < 0) { + if (map_fd[0] < 0) { opts->errstr = "failed to create loader map"; err = -errno; goto out; } - err = bpf_map_update_elem(map_fd, &key, opts->data, 0); + err = bpf_map_update_elem(map_fd[0], &key, opts->data, 0); if (err < 0) { opts->errstr = "failed to update loader map"; err = -errno; goto out; } + if (opts->fd_array_sz) { + map_fd[1] = bpf_create_map_name(BPF_MAP_TYPE_ARRAY, "__loader.fd.map", 4, + opts->fd_array_sz, 1, 0); + if (map_fd[1] < 0) { + opts->errstr = "failed to create loader fd map"; + err = -errno; + goto out; + } + } + memset(&attr, 0, sizeof(attr)); attr.prog_type = BPF_PROG_TYPE_SYSCALL; attr.insns = (long) opts->insns; attr.insn_cnt = opts->insns_sz / sizeof(struct bpf_insn); attr.license = (long) "Dual BSD/GPL"; memcpy(attr.prog_name, "__loader.prog", sizeof("__loader.prog")); - attr.fd_array = (long) &map_fd; + attr.fd_array = (long) map_fd; attr.log_level = opts->ctx->log_level; attr.log_size = opts->ctx->log_size; attr.log_buf = opts->ctx->log_buf; @@ -113,8 +124,10 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) } err = 0; out: - if (map_fd >= 0) - close(map_fd); + if (map_fd[0] >= 0) + close(map_fd[0]); + if (map_fd[1] >= 0) + close(map_fd[1]); if (prog_fd >= 0) close(prog_fd); return err; From patchwork Mon Sep 20 14:15:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505473 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D473C433F5 for ; Mon, 20 Sep 2021 14:16:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6675C60F0F for ; Mon, 20 Sep 2021 14:16:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240110AbhITORo (ORCPT ); Mon, 20 Sep 2021 10:17:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240074AbhITOR3 (ORCPT ); Mon, 20 Sep 2021 10:17:29 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0436C061767; Mon, 20 Sep 2021 07:16:00 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id w19so16307262pfn.12; Mon, 20 Sep 2021 07:16:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JEoGrDkWbNbm7f7hgUxBaYzaSXmo3n5r7w/q8QLAfZA=; b=lWrt821KvYmdQ2e7DLU6hwhCF9d7LlQOSQjgyIP9t5bfZ7gn9wLISpZMtvtqgcRyym tcnbYEf8J9Bo6vgwqtidJpCBKyHmuKp+uUQW7y+lpuADI/dpXtLHaxkiR8u1WkF6ydUS 2lYCJv1if+EqEHHtfc251qRCCaPAzkrZnUOrk9I2CACPaJoS7U8XLAEvdDdZuvzoYF8/ i41aBz28ftN4Fwlg3mYfbvVvF4s8GmZRNOHf6R3Nx3IgPEdjauzkNqHxXy5TYe05GAq5 PTKIJrWIQPgNwvhY7tpUjv4jROjUuOu5TRct7bhJO3QloWOddE2f2SxkZ70sg3Brkrf/ hUQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JEoGrDkWbNbm7f7hgUxBaYzaSXmo3n5r7w/q8QLAfZA=; b=ven1qX1hPkZYlRnE+sBnYrNfeXUG43qlL2dxA5ROSwSpTvxtJ2EDrFn4p6fBtQOqtB 3ZqbGWhU/lUatg/DoM75eDw7pbBGZ5Mz/Q8xyBgqzQcyzEVpI1fb6L5atyvYN7STRt6t 6cDProlHuD/OragdWAQe02KUV8GryHRnb/zVruAzWkFrcQmpvJ2TdxZMsMdBISbbkWrl eFE1VdWcMZDlBPawN6hNnRzR2o+ke2luqOV0KRakzTUfcK0f6PjLj+ElKTsG2Oym62xx ryixvQPE759aqnRx+B2BVlVFDX0cDohRpgn+uon1f5ldv+GonU/ziaw5mdmTo1NHNhEq 9hCA== X-Gm-Message-State: AOAM533+FapMU2xWdSHpqqsxClNsROO4lWVhzPfxyrB/jK7h7blpeEnk f52IyQVsEKhvuHwMW00VQ1shis7D1MAp+w== X-Google-Smtp-Source: ABdhPJxT+Sh2kVFIxZSwAVAv2O7CepLT2VCcOBuaM1xlPY9Eu5rSEk5UELXe9RoDQJiOejTVbvo/og== X-Received: by 2002:a63:2d07:: with SMTP id t7mr23886374pgt.101.1632147359841; Mon, 20 Sep 2021 07:15:59 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id b129sm12963642pfg.157.2021.09.20.07.15.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:15:59 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 09/11] tools: bpftool: Add separate fd_array map support for light skeleton Date: Mon, 20 Sep 2021 19:45:24 +0530 Message-Id: <20210920141526.3940002-10-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1310; h=from:subject; bh=2Sp/0Ej9RChgmna89RwHMC6AwnAZ35ulV5rtInB+ISw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaEFPw5EfsYGk37EJe1dUaupisaiklHdr1Iuyf9 TN6c3f+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8Ryg3QEA CNNc4LHNrz99hisn3dVA+xSbblcqRI4D9ydytO/8LBPwQp+y/Su6rrCAe2pRW975aRzKjlgKdQfEyu QY8v8xSmWON41L9g+/dqV7Fgn4yY/7qBn0j+Br6x0Q0h3W0tWgGHvjyKXNutBVDSlC2nibIjcsSoMy iU+g3kACYBtvBszUwJ1hZnOtRslDsWsylE1Gzb2LfvtspP7KUiwt3c638IMLAZFhlGmFjqC4fWq15V rP9d7JDLAli9b7A0oLwHTLlJsc4fsptqOmpaUb3ZFj8Tk1A1rbP+lndYOpk3GTagpL3dkHTT0s9q6a RosV+cH0bO85Ij2QwpEsWgcRgvc57hSETfJo+GHtUZR5RKy6KV4DLF8fiNdUtM4ATjukqBfsjfHK7T jVWgNIZd36UMycsaZLre4+LJo6G2otcrchXX9ca9bls9J7hWbDfMW1/A2P6nTHVdUFuKfgYKR5GCyo LZvmBVhKGOvUEoQjAlnfOavwL9ETxoWEWnCQcrNolq0PKJzavCFxVyI2+wquEoau4AjCxys6bDKW9Z IIJmGqd8bA50c4SX49H+Vq3ic4z0j0UVI1fBC16jqXA47aIgqweOHJIEzWJAbgmRnp7C9/qGFYfH+i HnXdlRE79J0vCkFfKDtn/PMzv2Ni4Bdcc/y0sKt/nKmP/vtCatDoDHnQ+zKQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This prepares bpftool to pass fd_array_sz parameter, so that the map can be created by bpf_load_and_run function to hold map and BTF fds. --- tools/bpf/bpftool/gen.c | 3 ++- tools/bpf/bpftool/prog.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index e3ec47a6a612..5bd3650956a4 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -532,10 +532,11 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h codegen("\ \n\ + opts.fd_array_sz = %d; \n\ opts.insns_sz = %d; \n\ opts.insns = (void *)\"\\ \n\ ", - opts.insns_sz); + opts.fd_array_sz, opts.insns_sz); print_hex(opts.insns, opts.insns_sz); codegen("\ \n\ diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c index 9c3e343b7d87..1d6286d727f4 100644 --- a/tools/bpf/bpftool/prog.c +++ b/tools/bpf/bpftool/prog.c @@ -1693,6 +1693,7 @@ static int try_loader(struct gen_loader_opts *gen) opts.data_sz = gen->data_sz; opts.insns = gen->insns; opts.insns_sz = gen->insns_sz; + opts.fd_array_sz = gen->fd_array_sz; fds_before = count_open_fds(); err = bpf_load_and_run(&opts); fd_delta = count_open_fds() - fds_before; From patchwork Mon Sep 20 14:15:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505475 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E43DDC433EF for ; Mon, 20 Sep 2021 14:16:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE17D610F9 for ; Mon, 20 Sep 2021 14:16:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240186AbhITORp (ORCPT ); Mon, 20 Sep 2021 10:17:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240226AbhITORa (ORCPT ); Mon, 20 Sep 2021 10:17:30 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1A88C061574; Mon, 20 Sep 2021 07:16:03 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id p12-20020a17090adf8c00b0019c959bc795so65679pjv.1; Mon, 20 Sep 2021 07:16:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K7x3S/edhEU5r9uoGNmxaTm92F+PqX19nR/7ykkGglg=; b=fuBp9jv+JURwurvOUDLwA6o0GxFgYDIsN/etNRgmgpT3o7Q/SirN7VhfVFhNChY21T CI1QpLOG768JVmGiCeXo6eSmxqrAT0EWzOjPLVWwBSBb8vAv8KUbQaHV69rb3yy0MOTF it9nezDRNz40Pku7F8n2ScwWwvdvTaTZH/wfsl4Al1gWW8Nl2HI8cekQHnIUP/vh4yOB Z0Hja+TSZsptGElJrTYwI0fuYIPirnkAHqdOSORMMOUmbCuSV826daOYrqR65cRd7VLX JBI6CxI7YI/7OekDhy7Ml9b9dOfXeQ9z3VDm1kWcducZSpD1L2S7YxNj1pFXVZp3Vzw6 spsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K7x3S/edhEU5r9uoGNmxaTm92F+PqX19nR/7ykkGglg=; b=CDPz32DIRG/IaTFZhIc6eXQPUvIPjt5aATgNFK43xYSE0zA7o8SW3B0RUwgh4J9a9C NTW1RGCnTHfUGONhBgq13TskGt+foTrt3Oj52U2ojfYbuQfqBvUFCpbT8ACBMOZ/zv5d iNDL5UMRereM1rqK/3OXWi2UfVXXic6oGKc+yLqwrGTeyfdTizlFsaqTc6x41Ok3WDLx TAtiZv2UrlaqF+AYfxJeW9FK7lObMh6jPC/Mbg0ymhDmSw33ECHD9D2hTS4Hs2f/X5fp WiFrD8bRSMg3dznMz4d0PpqgnKOQOdQQvTqCtDX9RSxmuVpY15baZCDOnzho9SWF37Bc IQLQ== X-Gm-Message-State: AOAM531I22Avtp/Sml9xoyy2l1PAhz1F+X03iD5xG0KyYmG+XClIPKtb uiOA/YtxybEEPGaXC7fnfjqnB1zsej/GSw== X-Google-Smtp-Source: ABdhPJyM2hs5Hzmn5hvwanP1i7pK+bbF9qROsfAGsMji35mKpoHc9L+KupD4ADwnbz0Kca63DVpaCA== X-Received: by 2002:a17:902:bb17:b0:13b:963f:1461 with SMTP id im23-20020a170902bb1700b0013b963f1461mr22767827plb.1.1632147363111; Mon, 20 Sep 2021 07:16:03 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id x10sm18586622pjv.57.2021.09.20.07.16.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:16:02 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 10/11] libbpf: Fix skel_internal.h to set errno on loader retval < 0 Date: Mon, 20 Sep 2021 19:45:25 +0530 Message-Id: <20210920141526.3940002-11-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1647; h=from:subject; bh=XTsRc9MfA1L0L3BTBN2MpE1xShjQT7waFGJZvT5D87k=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaEu3/m7R2K7Iat77LPkHL7ZMvQAdSGhwZduHkF eIDwXkaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8RyhPxEA DBbNmCBBeAgV+ye+66dbRjP9TUTIbEKhZ6wPUpon4zQF3+yNjHahfG0z4j1ai+FSx1XHZ9mbHphmth 3lX2kZE48WHdbAlEtMiHo0G3tkaOTBZUToY/d92OMZdvIkOECLAQ8wPmHt16UjYMcuC+wLmGWqp4bU /qGZDl1BcCZafxIxwVbF7560EYZfVpY7r7kEbOG29Gmwn4VIrTBN78xIj8Sk/5HGgSd55lXlQReEzZ HmT5nSm1AhqPmOe65KBKWPl2JWAroXQHDW+f1cNGHlFDTNUtXPAIDrXNpn3u/LU3u+MMx4F6m2wr6I DkTIAjfFUiPCQMi/t1si4cSS9qiSl6OEAYbFKnVJeg9MmltGIrl2Wx3gmm3uO6ZS8zlLACNevf4t38 Ljsa40ADBmw67qryKZMZWnUkGQtm6x+hhsu2Trx2VEWM9irydPgS4JAWKmqEbXF83CisBVNnebdESl PtIkLWXE0Hj5AOTkGD6UWbwSmtHnF/CUmXYiYbQWFc+oV3x2lM7J5fGXice4sv5EMYOaE9cMZjKms/ q79CDZWiew58jzwHOg2/a7ZWEe7pgJBFd36W0OzX3oTupUPl8LL1rXNwNPPkA/x4NvfSkKQ8Mkuoep aeqJsbGoC28mnpaAhqhtznzYLfI3sTHk1H+3I1Mf9z6NdzlZ72eIoKnIAHXA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net When the loader indicates an internal error (result of a checked bpf system call), it returns the result in attr.test.retval. However, tests that rely on ASSERT_OK_PTR on NULL (returned from light skeleton) may miss that NULL denotes an error if errno is set to 0. This would result in skel pointer being NULL, while ASSERT_OK_PTR returning 1, leading to a SEGV on dereference of skel, because libbpf_get_error relies on the assumption that errno is always set in case of error for ptr == NULL. In particular, this was observed for the ksyms_module test. When executed using `./test_progs -t ksyms`, prior tests manipulated errno and the test didn't crash when it failed at ksyms_module load, while using `./test_progs -t ksyms_module` crashed due to errno being untouched. Fixes: 67234743736a (libbpf: Generate loader program out of BPF ELF file.) Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/skel_internal.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index 6c0f0adfd42f..c7eb88040d6b 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -116,10 +116,12 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) err = skel_sys_bpf(BPF_PROG_RUN, &attr, sizeof(attr)); if (err < 0 || (int)attr.test.retval < 0) { opts->errstr = "failed to execute loader prog"; - if (err < 0) + if (err < 0) { err = -errno; - else + } else { err = (int)attr.test.retval; + errno = -err; + } goto out; } err = 0; From patchwork Mon Sep 20 14:15:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12505477 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 746F5C433F5 for ; Mon, 20 Sep 2021 14:16:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5AA5860F0F for ; Mon, 20 Sep 2021 14:16:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240280AbhITORx (ORCPT ); Mon, 20 Sep 2021 10:17:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237100AbhITORo (ORCPT ); Mon, 20 Sep 2021 10:17:44 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8B97C0613D8; Mon, 20 Sep 2021 07:16:07 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id h3-20020a17090a580300b0019ce70f8243so816765pji.4; Mon, 20 Sep 2021 07:16:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UEq+Y5CQwsqGuSVeljYsMjr1dnOEynCwvC9AfsACIYY=; b=ba3RgOFkiQReW3dM8fZ1ZgrAgteOB2vud20BTNaB8vpPtESNZANJigpCql4vxSotJL MIi5tP4XsOU6hL8nosx1pLQGrgZN1so/xXVnsDPBQ7gBJxDbQtSwcbq1GWVhil2Cf1FB wYcik69XuQtozqxYhiW2ARi4WUC6PqSa6Tn1ynminolnILDYovFxqdFYHQ4POus3E58J XU5ARq/FGWeigJ1j16YYaawPbU5GTjJBnMR+0ItELrDuTTm6mswHJRpA86sYwrOWthpX ISotTW81SG57n178uCJaqEkABqjpKO1KpIWyZHAXDgi2uePK+Jy6pOE1XwYwlGlWm8uf 5EQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UEq+Y5CQwsqGuSVeljYsMjr1dnOEynCwvC9AfsACIYY=; b=neDU0Ln/VWkeJrD1yEtki+bOKXl1oIk6mxI2geTlAtEF32RYUSUa3evgjGkosmjgoW AkDg9xCZ4CyTfHhHf/y5eJwakaWf9O+YBTw6A8MIR4WFp7N71UGzFgPvX9eT47T4/rQX 27YZU1w7Ij59pckJVX4Nhow3j26YFFbEcY1/viA7eV1/v55wJJjBVuxLfB3sIneR+OAc YuiCu15YX3lTuYeOVuXf5Ael6w2PrpeRp7YvkZARg5bLe2DPDMrXhtP48mX4LGlzfiiF GTl0uilAiLls5R3OqXcyOuhdNOH62NJUlYi+RJvB7j5gWJsFZBpHBajjsQId3JDVVL7d 2FDA== X-Gm-Message-State: AOAM533rmS3JyRH/EeHRiwvSvjH5U/WtexhT65NrSd9zydMYP0k4xoXf WyD5Z4LpQ1aqmUV1tsUSJzAs1Pixz76OcQ== X-Google-Smtp-Source: ABdhPJzMcAojg1MqxuMzZnWKiEolDQaYbAnXC4ohP5Mw5SPvGsC7N0rutGZu5JLMD3xKKPQFfwVkjQ== X-Received: by 2002:a17:902:760b:b0:13b:122:5ff0 with SMTP id k11-20020a170902760b00b0013b01225ff0mr22758132pll.22.1632147366744; Mon, 20 Sep 2021 07:16:06 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id g22sm8989124pfj.15.2021.09.20.07.16.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 07:16:05 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v4 11/11] bpf: selftests: Add selftests for module kfunc support Date: Mon, 20 Sep 2021 19:45:26 +0530 Message-Id: <20210920141526.3940002-12-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920141526.3940002-1-memxor@gmail.com> References: <20210920141526.3940002-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=18843; h=from:subject; bh=9oaki4DdqaEzmp0XdWmwbTkYK1LrGPYRZn0nGYiaB3Y=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhSJaEQflotWbuNhBxaQoGjvEnS0oRBHjMrmwLtVhd nZEdONGJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUiWhAAKCRBM4MiGSL8RykVKEA CIr9Pb6xeV61MusfvnVkB7n0JF7OX+G2ITvi7sogQIllOYjlSsL4HxPHMSKqVQsHiETUNwJy413AAx ll4ONwNtcg4futPy0RQltEIb2Wo0XHpUVyqglBymS8ZsVTzWkfl1Lm3lqBTFCLTceM3u8tvCqUXrI6 xkKNpr99YrCt0P381B+bC16R5MMNp54ZgJFmHmIaf+MXLmuBYsMXpgh+05XY8bON+9sJYut+q4Pp4R qHYx3uQmWEujWuN/kjwCLH2z19EuNYksCRdH3Ft1HfdT8TK+FvtLH+pQmTESVdrBByzcZqwZjgQDO5 n0fGUck1P9Ys3IZaAGrobKZldg7nCM9smcSNVmT5rCeEmJDaiCL58Gk3LiA2QKIvcLFiaUIypbWDA2 XB82PfVhlQ+VMBFHNdU50wMyE6iOHneeIDfR0cvOlvRcEGHbA85IambufzbLG0WTAkDBkZObWr3PME RKnzGe6ScquhQEk1k6TewpG3ujAiisy1wQ+k6c1nz4IoBhwTE7rHRTxixUEJvRcQByK5m99RL3wnkP wJHFiZFf9nJoZOjBxh0AgFFmuabRF9XfgVdHYZ5BB2Efg+SlnVCDVHrs0dGPFHmTm6l3cTMvWVwQ4A YEF6yUsT5NnBhYU6cgTS3mHhdZQWlo7SXDLx9680jZb8P4sz9pJY0MzpM5AA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This adds selftests that tests the success and failure path for modules kfuncs (in presence of invalid kfunc calls) for both libbpf and gen_loader. It also adds a prog_test kfunc_btf_id_list so that we can add module BTF ID set from bpf_testmod. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 2 + kernel/bpf/btf.c | 2 + net/bpf/test_run.c | 5 +- tools/testing/selftests/bpf/Makefile | 5 +- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 26 ++++++- .../selftests/bpf/prog_tests/ksyms_module.c | 52 ++++++++++---- .../bpf/prog_tests/ksyms_module_libbpf.c | 44 ++++++++++++ .../selftests/bpf/progs/test_ksyms_module.c | 41 ++++++++--- .../bpf/progs/test_ksyms_module_fail.c | 29 ++++++++ .../progs/test_ksyms_module_fail_toomany.c | 19 +++++ .../bpf/progs/test_ksyms_module_libbpf.c | 71 +++++++++++++++++++ .../bpf/progs/test_ksyms_module_util.h | 48 +++++++++++++ 12 files changed, 317 insertions(+), 27 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c create mode 100644 tools/testing/selftests/bpf/progs/test_ksyms_module_fail.c create mode 100644 tools/testing/selftests/bpf/progs/test_ksyms_module_fail_toomany.c create mode 100644 tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c create mode 100644 tools/testing/selftests/bpf/progs/test_ksyms_module_util.h diff --git a/include/linux/btf.h b/include/linux/btf.h index f5ae81e225be..cab6aadf59de 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -270,7 +270,9 @@ static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, THIS_MODULE } extern struct kfunc_btf_id_list bpf_tcp_ca_kfunc_list; +extern struct kfunc_btf_id_list prog_test_kfunc_list; DECLARE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca); +DECLARE_CHECK_KFUNC_CALLBACK(prog_test); #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 671b4f713a51..998aca7f42d0 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6397,3 +6397,5 @@ EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list); DEFINE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca, bpf_tcp_ca_kfunc_list); +DEFINE_KFUNC_BTF_ID_LIST(prog_test_kfunc_list); +DEFINE_CHECK_KFUNC_CALLBACK(prog_test, prog_test_kfunc_list); diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index fe5c34f414a2..caa6831c1849 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -2,6 +2,7 @@ /* Copyright (c) 2017 Facebook */ #include +#include #include #include #include @@ -243,7 +244,9 @@ BTF_SET_END(test_sk_kfunc_ids) bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner) { - return btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id); + if (btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id)) + return true; + return __bpf_check_prog_test_kfunc_call(kfunc_id, owner); } static void *bpf_test_init(const union bpf_attr *kattr, u32 size, diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 326ea75ce99e..d20ff0563120 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -174,6 +174,7 @@ $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes $(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation $(Q)$(MAKE) $(submake_extras) -C bpf_testmod $(Q)cp bpf_testmod/bpf_testmod.ko $@ + $(Q)$(RESOLVE_BTFIDS) -s ../../../../vmlinux bpf_testmod.ko $(OUTPUT)/test_stub.o: test_stub.c $(BPFOBJ) $(call msg,CC,,$@) @@ -315,8 +316,8 @@ LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h \ linked_vars.skel.h linked_maps.skel.h LSKELS := kfunc_call_test.c fentry_test.c fexit_test.c fexit_sleep.c \ - test_ksyms_module.c test_ringbuf.c atomics.c trace_printk.c \ - trace_vprintk.c + test_ksyms_module.c test_ksyms_module_fail.c test_ksyms_module_fail_toomany.c \ + test_ringbuf.c atomics.c trace_printk.c trace_vprintk.c SKEL_BLACKLIST += $$(LSKELS) test_static_linked.skel.h-deps := test_static_linked1.o test_static_linked2.o diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 50fc5561110a..4157d8497963 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ +#include +#include #include #include #include @@ -10,9 +12,17 @@ #define CREATE_TRACE_POINTS #include "bpf_testmod-events.h" +#include "../progs/test_ksyms_module_util.h" +KFUNC_DEFINE_VALID_DISTINCT_256; DEFINE_PER_CPU(int, bpf_testmod_ksym_percpu) = 123; +noinline void +bpf_testmod_test_mod_kfunc(int i) +{ + *(int *)this_cpu_ptr(&bpf_testmod_ksym_percpu) = i; +} + noinline int bpf_testmod_loop_test(int n) { int i, sum = 0; @@ -71,13 +81,27 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = { .write = bpf_testmod_test_write, }; +BTF_SET_START(bpf_testmod_kfunc_ids) +BTF_ID(func, bpf_testmod_test_mod_kfunc) +KFUNC_BTF_ID_VALID_DISTINCT_256 +BTF_SET_END(bpf_testmod_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&bpf_testmod_kfunc_ids, bpf_testmod_kfunc_btf_set); + static int bpf_testmod_init(void) { - return sysfs_create_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); + int ret; + + ret = sysfs_create_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); + if (ret) + return ret; + register_kfunc_btf_id_set(&prog_test_kfunc_list, &bpf_testmod_kfunc_btf_set); + return 0; } static void bpf_testmod_exit(void) { + unregister_kfunc_btf_id_set(&prog_test_kfunc_list, &bpf_testmod_kfunc_btf_set); return sysfs_remove_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); } diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_module.c b/tools/testing/selftests/bpf/prog_tests/ksyms_module.c index 2cd5cded543f..a0dd60f00c57 100644 --- a/tools/testing/selftests/bpf/prog_tests/ksyms_module.c +++ b/tools/testing/selftests/bpf/prog_tests/ksyms_module.c @@ -2,30 +2,54 @@ /* Copyright (c) 2021 Facebook */ #include -#include -#include +#include #include "test_ksyms_module.lskel.h" +#include "test_ksyms_module_fail.lskel.h" +#include "test_ksyms_module_fail_toomany.lskel.h" -static int duration; - -void test_ksyms_module(void) +void test_ksyms_module_main(void) { - struct test_ksyms_module* skel; + struct test_ksyms_module *skel; + int retval; int err; + if (!env.has_testmod) { + test__skip(); + return; + } + skel = test_ksyms_module__open_and_load(); - if (CHECK(!skel, "skel_open", "failed to open skeleton\n")) + if (!ASSERT_OK_PTR(skel, "test_ksyms_module__open_and_load")) return; - err = test_ksyms_module__attach(skel); - if (CHECK(err, "skel_attach", "skeleton attach failed: %d\n", err)) + err = bpf_prog_test_run(skel->progs.handler.prog_fd, 1, &pkt_v4, sizeof(pkt_v4), + NULL, NULL, (__u32 *)&retval, NULL); + if (!ASSERT_OK(err, "bpf_prog_test_run")) goto cleanup; + ASSERT_EQ(retval, 0, "retval"); + ASSERT_EQ(skel->bss->out_bpf_testmod_ksym, 42, "bpf_testmod_ksym"); +cleanup: + test_ksyms_module__destroy(skel); +} - usleep(1); +void test_ksyms_module_fail(void) +{ + struct test_ksyms_module_fail_toomany *skel2; + struct test_ksyms_module_fail *skel1; - ASSERT_EQ(skel->bss->triggered, true, "triggered"); - ASSERT_EQ(skel->bss->out_mod_ksym_global, 123, "global_ksym_val"); + skel1 = test_ksyms_module_fail__open_and_load(); + if (!ASSERT_EQ(skel1, NULL, "test_ksyms_module_fail__open_and_load")) + test_ksyms_module_fail__destroy(skel1); -cleanup: - test_ksyms_module__destroy(skel); + skel2 = test_ksyms_module_fail_toomany__open_and_load(); + if (!ASSERT_EQ(skel2, NULL, "test_ksyms_module_fail_toomany__open_and_load")) + test_ksyms_module_fail_toomany__destroy(skel2); +} + +void test_ksyms_module(void) +{ + if (test__start_subtest("main")) + test_ksyms_module_main(); + if (test__start_subtest("fail")) + test_ksyms_module_fail(); } diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c b/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c new file mode 100644 index 000000000000..d83297724ce8 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "test_ksyms_module_libbpf.skel.h" + +void test_ksyms_module_libbpf(void) +{ + struct test_ksyms_module_libbpf *skel; + int retval, err; + + if (!env.has_testmod) { + test__skip(); + return; + } + + skel = test_ksyms_module_libbpf__open(); + if (!ASSERT_OK_PTR(skel, "test_ksyms_module_libbpf__open")) + return; + err = bpf_program__set_autoload(skel->progs.load_fail1, false); + if (!ASSERT_OK(err, "bpf_program__set_autoload false load_fail1")) + goto cleanup; + err = bpf_program__set_autoload(skel->progs.load_fail2, false); + if (!ASSERT_OK(err, "bpf_program__set_autoload false load_fail2")) + goto cleanup; + err = test_ksyms_module_libbpf__load(skel); + if (!ASSERT_OK(err, "test_ksyms_module_libbpf__load")) + goto cleanup; + err = bpf_prog_test_run(bpf_program__fd(skel->progs.handler), 1, &pkt_v4, + sizeof(pkt_v4), NULL, NULL, (__u32 *)&retval, NULL); + if (!ASSERT_OK(err, "bpf_prog_test_run")) + goto cleanup; + ASSERT_EQ(retval, 0, "retval"); + ASSERT_EQ(skel->bss->out_bpf_testmod_ksym, 42, "bpf_testmod_ksym"); + + err = bpf_program__load(skel->progs.load_fail1, "GPL", 0); + if (!ASSERT_NEQ(err, 0, "bpf_program__load load_fail1")) + goto cleanup; + err = bpf_program__load(skel->progs.load_fail2, "GPL", 0); + if (!ASSERT_NEQ(err, 0, "bpf_program__load load_fail2")) + goto cleanup; +cleanup: + test_ksyms_module_libbpf__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module.c b/tools/testing/selftests/bpf/progs/test_ksyms_module.c index d6a0b3086b90..6b460c69a25a 100644 --- a/tools/testing/selftests/bpf/progs/test_ksyms_module.c +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module.c @@ -4,22 +4,45 @@ #include "vmlinux.h" #include +#include "test_ksyms_module_util.h" +KFUNC_KSYM_DECLARE_VALID_DISTINCT_256; extern const int bpf_testmod_ksym_percpu __ksym; +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; +extern void bpf_testmod_invalid_mod_kfunc(void) __ksym __weak; -int out_mod_ksym_global = 0; -bool triggered = false; +int out_bpf_testmod_ksym = 0; +const volatile int x = 0; -SEC("raw_tp/sys_enter") -int handler(const void *ctx) +SEC("classifier") +int handler(struct __sk_buff *skb) { - int *val; - __u32 cpu; + /* This should be preserved by clang, but DCE'd by verifier, and still + * allow loading the classifier prog + */ + if (x) { + bpf_testmod_invalid_mod_kfunc(); + return -1; + } + bpf_testmod_test_mod_kfunc(42); + out_bpf_testmod_ksym = *(int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); + return 0; +} - val = (int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); - out_mod_ksym_global = *val; - triggered = true; +SEC("classifier") +int load_256(struct __sk_buff *skb) +{ + /* this will fail if kfunc doesn't reuse its own btf fd index */ + KFUNC_VALID_SAME_256; + KFUNC_VALID_SAME_ONE; + return 0; +} +SEC("classifier") +int load_distinct256(struct __sk_buff *skb) +{ + /* kfuncs with distinct insn->imm, insn->off */ + KFUNC_VALID_DISTINCT_256; return 0; } diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module_fail.c b/tools/testing/selftests/bpf/progs/test_ksyms_module_fail.c new file mode 100644 index 000000000000..bcf98e814a7a --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module_fail.c @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" + +#include + +extern const int bpf_testmod_ksym_percpu __ksym; +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; +extern void bpf_testmod_invalid_mod_kfunc(void) __ksym __weak; + +int out_bpf_testmod_ksym = 0; +const volatile int x = 0; + +SEC("classifier") +int load(struct __sk_buff *skb) +{ + /* This should be preserved by clang, but not DCE'd by verifier, + * hence fail loading + */ + if (!x) { + bpf_testmod_invalid_mod_kfunc(); + return -1; + } + bpf_testmod_test_mod_kfunc(42); + out_bpf_testmod_ksym = *(int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); + return 0; +} + +char LICENSE[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module_fail_toomany.c b/tools/testing/selftests/bpf/progs/test_ksyms_module_fail_toomany.c new file mode 100644 index 000000000000..633a743a67c7 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module_fail_toomany.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" + +#include +#include "test_ksyms_module_util.h" + +KFUNC_KSYM_DECLARE_VALID_DISTINCT_256; +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; + +SEC("classifier") +int load(struct __sk_buff *skb) +{ + KFUNC_VALID_DISTINCT_256; + KFUNC_VALID_SAME_ONE; + return 0; +} + +char LICENSE[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c b/tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c new file mode 100644 index 000000000000..079a039ccda5 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" + +#include +#include "test_ksyms_module_util.h" + +KFUNC_KSYM_DECLARE_VALID_DISTINCT_256; +extern const int bpf_testmod_ksym_percpu __ksym; +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; +extern void bpf_testmod_invalid_mod_kfunc(void) __ksym __weak; + +int out_bpf_testmod_ksym = 0; +const volatile int x = 0; + +SEC("classifier") +int handler(struct __sk_buff *skb) +{ + /* This should be preserved by clang, but DCE'd by verifier, and still + * allow loading the classifier prog + */ + if (x) { + bpf_testmod_invalid_mod_kfunc(); + return -1; + } + bpf_testmod_test_mod_kfunc(42); + out_bpf_testmod_ksym = *(int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); + return 0; +} + +SEC("classifier") +int load_fail1(struct __sk_buff *skb) +{ + /* This should be preserved by clang, but not DCE'd by verifier, + * hence fail loading + */ + if (!x) { + bpf_testmod_invalid_mod_kfunc(); + return -1; + } + bpf_testmod_test_mod_kfunc(42); + out_bpf_testmod_ksym = *(int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); + return 0; +} + +SEC("classifier") +int load_fail2(struct __sk_buff *skb) +{ + KFUNC_VALID_DISTINCT_256; + KFUNC_VALID_SAME_ONE; + return 0; +} + +SEC("classifier") +int load_256(struct __sk_buff *skb) +{ + /* this will fail if kfunc doesn't reuse its own btf fd index */ + KFUNC_VALID_SAME_256; + KFUNC_VALID_SAME_ONE; + return 0; +} + +SEC("classifier") +int load_distinct256(struct __sk_buff *skb) +{ + /* kfuncs with distinct insn->imm, insn->off */ + KFUNC_VALID_DISTINCT_256; + return 0; +} + +char LICENSE[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module_util.h b/tools/testing/selftests/bpf/progs/test_ksyms_module_util.h new file mode 100644 index 000000000000..3afa74841ae0 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module_util.h @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef __KSYMS_MODULE_UTIL_H__ +#define __KSYMS_MODULE_UTIL_H__ + +#define __KFUNC_NR_EXP(Y) \ +Y(0) Y(1) Y(2) Y(3) Y(4) Y(5) Y(6) Y(7) Y(8) Y(9) Y(10) Y(11) Y(12) \ +Y(13) Y(14) Y(15) Y(16) Y(17) Y(18) Y(19) Y(20) Y(21) Y(22) Y(23) \ +Y(24) Y(25) Y(26) Y(27) Y(28) Y(29) Y(30) Y(31) Y(32) Y(33) Y(34) \ +Y(35) Y(36) Y(37) Y(38) Y(39) Y(40) Y(41) Y(42) Y(43) Y(44) Y(45) \ +Y(46) Y(47) Y(48) Y(49) Y(50) Y(51) Y(52) Y(53) Y(54) Y(55) Y(56) \ +Y(57) Y(58) Y(59) Y(60) Y(61) Y(62) Y(63) Y(64) Y(65) Y(66) Y(67) \ +Y(68) Y(69) Y(70) Y(71) Y(72) Y(73) Y(74) Y(75) Y(76) Y(77) Y(78) \ +Y(79) Y(80) Y(81) Y(82) Y(83) Y(84) Y(85) Y(86) Y(87) Y(88) Y(89) \ +Y(90) Y(91) Y(92) Y(93) Y(94) Y(95) Y(96) Y(97) Y(98) Y(99) Y(100) \ +Y(101) Y(102) Y(103) Y(104) Y(105) Y(106) Y(107) Y(108) Y(109) Y(110) \ +Y(111) Y(112) Y(113) Y(114) Y(115) Y(116) Y(117) Y(118) Y(119) Y(120) \ +Y(121) Y(122) Y(123) Y(124) Y(125) Y(126) Y(127) Y(128) Y(129) Y(130) \ +Y(131) Y(132) Y(133) Y(134) Y(135) Y(136) Y(137) Y(138) Y(139) Y(140) \ +Y(141) Y(142) Y(143) Y(144) Y(145) Y(146) Y(147) Y(148) Y(149) Y(150) \ +Y(151) Y(152) Y(153) Y(154) Y(155) Y(156) Y(157) Y(158) Y(159) Y(160) \ +Y(161) Y(162) Y(163) Y(164) Y(165) Y(166) Y(167) Y(168) Y(169) Y(170) \ +Y(171) Y(172) Y(173) Y(174) Y(175) Y(176) Y(177) Y(178) Y(179) Y(180) \ +Y(181) Y(182) Y(183) Y(184) Y(185) Y(186) Y(187) Y(188) Y(189) Y(190) \ +Y(191) Y(192) Y(193) Y(194) Y(195) Y(196) Y(197) Y(198) Y(199) Y(200) \ +Y(201) Y(202) Y(203) Y(204) Y(205) Y(206) Y(207) Y(208) Y(209) Y(210) \ +Y(211) Y(212) Y(213) Y(214) Y(215) Y(216) Y(217) Y(218) Y(219) Y(220) \ +Y(221) Y(222) Y(223) Y(224) Y(225) Y(226) Y(227) Y(228) Y(229) Y(230) \ +Y(231) Y(232) Y(233) Y(234) Y(235) Y(236) Y(237) Y(238) Y(239) Y(240) \ +Y(241) Y(242) Y(243) Y(244) Y(245) Y(246) Y(247) Y(248) Y(249) Y(250) \ +Y(251) Y(252) Y(253) Y(254) Y(255) + +#define __KFUNC_A(nr) bpf_testmod_test_mod_kfunc_##nr(); +#define KFUNC_VALID_DISTINCT_256 __KFUNC_NR_EXP(__KFUNC_A) + +#define __KFUNC_B(nr) extern void bpf_testmod_test_mod_kfunc_##nr(void) __ksym; +#define KFUNC_KSYM_DECLARE_VALID_DISTINCT_256 __KFUNC_NR_EXP(__KFUNC_B) + +#define __KFUNC_C(nr) noinline void bpf_testmod_test_mod_kfunc_##nr(void) {}; +#define KFUNC_DEFINE_VALID_DISTINCT_256 __KFUNC_NR_EXP(__KFUNC_C) + +#define __KFUNC_D(nr) BTF_ID(func, bpf_testmod_test_mod_kfunc_##nr) +#define KFUNC_BTF_ID_VALID_DISTINCT_256 __KFUNC_NR_EXP(__KFUNC_D) + +#define __KFUNC_E(nr) bpf_testmod_test_mod_kfunc(nr); +#define KFUNC_VALID_SAME_ONE __KFUNC_E(0) +#define KFUNC_VALID_SAME_256 __KFUNC_NR_EXP(__KFUNC_E) + +#endif