From patchwork Tue Sep 14 12:37:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493047 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A17A5C433EF for ; Tue, 14 Sep 2021 12:38:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F99060F92 for ; Tue, 14 Sep 2021 12:38:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232884AbhINMjU (ORCPT ); Tue, 14 Sep 2021 08:39:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232796AbhINMjS (ORCPT ); Tue, 14 Sep 2021 08:39:18 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A9F9C061574; Tue, 14 Sep 2021 05:38:01 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id n13-20020a17090a4e0d00b0017946980d8dso2000935pjh.5; Tue, 14 Sep 2021 05:38:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Myd4YXzv22aL42IZTVuAouvq2pjYEs71DALuk1TgQ4s=; b=BdsNYN6AVMt1fwB5Mrwg6BTIcv2FSHin8tPvxCKVdfp+mgHooyfoCZZdYg6/4RjWRP ZQduzDzEanzko7pxf+uh+QWLZLOM56X3+i6X9IfE+LhijbgTnlIvyvogj9jXMpjN7RZd k0iKLyQT6PMek8HEVZR+oKfRiTmLLMLRsI2EMx24IVc5417WOm6WXYB3P7dWJRr0RlJf QKnMglgVCwZMUg6ttVGAksgCocrP2F9nHvbYPN3awtaNibyK3812sYeVmxhdhdCH2upO jW8jqIrtwMO7aofoxsJ0+Z3z4mLCzqEatWnm7elqJzmXZhQs9DevQClJE5VNkfSXXcdY PuxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Myd4YXzv22aL42IZTVuAouvq2pjYEs71DALuk1TgQ4s=; b=38FoCSZ9BEBWauBlef3aDGfY4RAU82wLaAMBUXaqrJOo9biERdboWE8mx5qtWk6oG5 lUE+90ZWxTCB0Vj1BfUwJzRDkXmnaMzNlfBPXY0vELszymMTylaPMS3maQSLbu4bmktT j5rERd61RZIoLRN7cj0puKn0wYdr2oPZ6UPWbIMQjoKyPTvUgxjQY9G3+p3xSFMr8+A3 S3ABr2F9GzbOqMq0OI8J/sQtj5grPnQWNwW56CxmwA0bMDSqhPadFq9N3NhnWDGRZFE4 VaN3HP+1WJx1X6rO9ZlSBpStRQB+jwXVb3p85r/iTcZdwnyQj9cjVZkDEkD+r1rr6Lh1 RS3w== X-Gm-Message-State: AOAM530gRI5bspS+ZkFortURO8i4z/JgvKjl886eZUkDURoxLUW5C2Tp f9KWE4kG0YbdE7dttz7Fk5+g887lhwXcOw== X-Google-Smtp-Source: ABdhPJwPHWN3F9MPLDPlASDPa5ReAe3izHP+iAVX3nbvOceyCBYmLbwfFxXxq25pgv5QfT1Z5fTZng== X-Received: by 2002:a17:902:760b:b0:13b:122:5ff0 with SMTP id k11-20020a170902760b00b0013b01225ff0mr14732068pll.22.1631623080250; Tue, 14 Sep 2021 05:38:00 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id y25sm8311656pfe.28.2021.09.14.05.37.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:00 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 01/10] bpf: Introduce BPF support for kernel module function calls Date: Tue, 14 Sep 2021 18:07:40 +0530 Message-Id: <20210914123750.460750-2-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=16566; h=from:subject; bh=doZCWeYxbS+gEiVXiHHZlHeZ/xFk1Q9odBs67zUbcxk=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdVg89EIwmclx9K7P8aZc1kThrkyK6oQ9ubVSCE BrstwJyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVQAKCRBM4MiGSL8RygdKD/ 0bUDwbZJQxF0sBbd6NgkwgXQKJ+b7HLR/QbcUZdHh7i+QSuLyBVuvwY4DJyg+IfsAvE7cMMFb3kGf4 TgpJB4FE3gublAQpQURsUhtnbnrGczT989tP8Q7MZq095hsNHWwluf8BrKUpJLSlcOIH+MjrDCSZPR A0SQoNjVzPHq+vKnrTev0AA2ZzMMddVfkg7dun9gFRcw1GzU/tTArX7EjeiXcgWEGuaKYVSXwJCsW+ rX/E9pA1p17rU02Pk9BgF3jZL35VQU6b095ry/YjMSwZYoQdqiOxgjzq7D7X4l/Zcb1vhWR1yIj6YL Q5nNIyJGq7ABXwNkUOx+Fi6AF8bZtV+NS8i8zbUCrCo9xqLMaVc10HAJCPRYcSfpMinufOzwF5h3av EG95nkExjUa0uhJbDROW1bL2pjPpWa8l5Iankw6pcdZ6E85WLbSfLjMVWLbwaTZEmdrRsr6jf521GL TQVxSAHd2Ekh/4NguuwXZeZf5Rr2E1USSrdwqmLL6x5Mmlwkq8WhmpFrLLx8WGFJq0cP/m999Kg3Lr /99jXMK8Z5sTuIJqTrOU3hJdV3P4W/QWYVl3hzEvUTjcL0zPvv06UkfVeNuZoMOmrHekRauEuJl1TA 7q9fTIJeCpUDeOxITocusomlgTT/+0g0+2eCU8qEFR7Eoaqj022jTZLxM3BA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This change adds support on the kernel side to allow for BPF programs to call kernel module functions. Userspace will prepare an array of module BTF fds that is passed in during BPF_PROG_LOAD using fd_array parameter. In the kernel, the module BTFs are placed in the auxilliary struct for bpf_prog, and loaded as needed. The verifier then uses insn->off to index into the fd_array. insn->off is used by subtracting one from it, as userspace has to set the index of array in insn->off incremented by 1. This lets us denote vmlinux btf by insn->off == 0, and the module kfunc using insn->off > 0. They are sorted based on offset in an array, and each offset corresponds to one descriptor, with a max limit up to 256 such module BTFs. Another change is to check_kfunc_call callback, which now include a struct module * pointer, this is to be used in later patch such that the kfunc_id and module pointer are matched for dynamically registered BTF sets from loadable modules, so that same kfunc_id in two modules doesn't lead to check_kfunc_call succeeding. For the duration of the check_kfunc_call, the reference to struct module exists, as it returns the pointer stored in kfunc_btf_tab. Signed-off-by: Kumar Kartikeya Dwivedi Reported-by: kernel test robot Reported-by: kernel test robot --- include/linux/bpf.h | 8 +- include/linux/bpf_verifier.h | 2 + kernel/bpf/core.c | 2 + kernel/bpf/verifier.c | 182 +++++++++++++++++++++++++++++++---- net/bpf/test_run.c | 2 +- net/ipv4/bpf_tcp_ca.c | 2 +- 6 files changed, 173 insertions(+), 25 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f4c16f19f83e..148bd899411e 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -511,7 +511,7 @@ struct bpf_verifier_ops { const struct btf_type *t, int off, int size, enum bpf_access_type atype, u32 *next_btf_id); - bool (*check_kfunc_call)(u32 kfunc_btf_id); + bool (*check_kfunc_call)(u32 kfunc_btf_id, struct module *owner); }; struct bpf_prog_offload_ops { @@ -874,6 +874,7 @@ struct bpf_prog_aux { void *jit_data; /* JIT specific data. arch dependent */ struct bpf_jit_poke_descriptor *poke_tab; struct bpf_kfunc_desc_tab *kfunc_tab; + struct bpf_kfunc_btf_tab *kfunc_btf_tab; u32 size_poke_tab; struct bpf_ksym ksym; const struct bpf_prog_ops *ops; @@ -1635,7 +1636,7 @@ int bpf_prog_test_run_raw_tp(struct bpf_prog *prog, int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id); +bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner); bool btf_ctx_access(int off, int size, enum bpf_access_type type, const struct bpf_prog *prog, struct bpf_insn_access_aux *info); @@ -1856,7 +1857,8 @@ static inline int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog, return -ENOTSUPP; } -static inline bool bpf_prog_test_check_kfunc_call(u32 kfunc_id) +static inline bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, + struct module *owner) { return false; } diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 5424124dbe36..c8a78e830fca 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -527,5 +527,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, const struct bpf_prog *tgt_prog, u32 btf_id, struct bpf_attach_target_info *tgt_info); +void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab); + #endif /* _LINUX_BPF_VERIFIER_H */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 9f4636d021b1..d01eb043b0b8 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -2255,6 +2256,7 @@ static void bpf_prog_free_deferred(struct work_struct *work) int i; aux = container_of(work, struct bpf_prog_aux, work); + bpf_free_kfunc_btf_tab(aux->kfunc_btf_tab); bpf_free_used_maps(aux); bpf_free_used_btfs(aux); if (bpf_prog_is_dev_bound(aux)) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 047ac4b4703b..6bbbb98f4ee2 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1626,18 +1626,31 @@ static int add_subprog(struct bpf_verifier_env *env, int off) return env->subprog_cnt - 1; } +#define MAX_KFUNC_DESCS 256 +#define MAX_KFUNC_BTFS 256 + struct bpf_kfunc_desc { struct btf_func_model func_model; u32 func_id; s32 imm; }; -#define MAX_KFUNC_DESCS 256 +struct bpf_kfunc_btf { + struct btf *btf; + struct module *module; + u16 offset; +}; + struct bpf_kfunc_desc_tab { struct bpf_kfunc_desc descs[MAX_KFUNC_DESCS]; u32 nr_descs; }; +struct bpf_kfunc_btf_tab { + struct bpf_kfunc_btf descs[MAX_KFUNC_BTFS]; + u32 nr_descs; +}; + static int kfunc_desc_cmp_by_id(const void *a, const void *b) { const struct bpf_kfunc_desc *d0 = a; @@ -1647,6 +1660,14 @@ static int kfunc_desc_cmp_by_id(const void *a, const void *b) return d0->func_id - d1->func_id; } +static int kfunc_btf_cmp_by_off(const void *a, const void *b) +{ + const struct bpf_kfunc_btf *d0 = a; + const struct bpf_kfunc_btf *d1 = b; + + return d0->offset - d1->offset; +} + static const struct bpf_kfunc_desc * find_kfunc_desc(const struct bpf_prog *prog, u32 func_id) { @@ -1660,18 +1681,113 @@ find_kfunc_desc(const struct bpf_prog *prog, u32 func_id) sizeof(tab->descs[0]), kfunc_desc_cmp_by_id); } -static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) +/* Expects offset always greater than 0 */ +static struct btf *__find_kfunc_desc_btf(struct bpf_verifier_env *env, + s16 offset, struct module **btf_modp) +{ + struct bpf_kfunc_btf kf_btf = { .offset = offset - 1 }; + struct bpf_kfunc_btf_tab *tab; + struct module *mod = NULL; + struct bpf_kfunc_btf *b; + struct btf *btf; + int btf_fd; + + tab = env->prog->aux->kfunc_btf_tab; + b = bsearch(&kf_btf, tab->descs, tab->nr_descs, + sizeof(tab->descs[0]), kfunc_btf_cmp_by_off); + if (!b) { + if (tab->nr_descs == MAX_KFUNC_BTFS) { + verbose(env, "too many different module BTFs\n"); + return ERR_PTR(-E2BIG); + } + + offset -= 1; + if (copy_from_bpfptr_offset(&btf_fd, env->fd_array, + offset * sizeof(btf_fd), + sizeof(btf_fd))) + return ERR_PTR(-EFAULT); + + btf = btf_get_by_fd(btf_fd); + if (IS_ERR(btf)) + return btf; + + if (!btf_is_module(btf)) { + verbose(env, "BTF fd for kfunc is not a module BTF\n"); + btf_put(btf); + return ERR_PTR(-EINVAL); + } + + mod = btf_try_get_module(btf); + if (!mod) { + btf_put(btf); + return ERR_PTR(-ENXIO); + } + + b = &tab->descs[tab->nr_descs++]; + b->btf = btf; + b->module = mod; + b->offset = offset; + + sort(tab->descs, tab->nr_descs, sizeof(tab->descs[0]), + kfunc_btf_cmp_by_off, NULL); + } + if (btf_modp) + *btf_modp = b->module; + return b->btf; +} + +void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab) +{ + if (!tab) + return; + + while (tab->nr_descs--) { + module_put(tab->descs[tab->nr_descs].module); + btf_put(tab->descs[tab->nr_descs].btf); + } + kfree(tab); +} + +static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, + u32 func_id, s16 offset, + struct module **btf_modp) +{ + struct btf *kfunc_btf; + + if (offset) { + if (offset < 0) { + /* In the future, this can be allowed to increase limit + * of fd index into fd_array, interpreted as unsigned u16. + */ + verbose(env, "negative offset disallowed for kernel module function call\n"); + return ERR_PTR(-EINVAL); + } + + kfunc_btf = __find_kfunc_desc_btf(env, offset, btf_modp); + if (IS_ERR_OR_NULL(kfunc_btf)) { + verbose(env, "cannot find module BTF for func_id %u\n", func_id); + return kfunc_btf ?: ERR_PTR(-ENOENT); + } + return kfunc_btf; + } + return btf_vmlinux ?: ERR_PTR(-ENOENT); +} + +static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) { const struct btf_type *func, *func_proto; + struct bpf_kfunc_btf_tab *btf_tab; struct bpf_kfunc_desc_tab *tab; struct bpf_prog_aux *prog_aux; struct bpf_kfunc_desc *desc; const char *func_name; + struct btf *desc_btf; unsigned long addr; int err; prog_aux = env->prog->aux; tab = prog_aux->kfunc_tab; + btf_tab = prog_aux->kfunc_btf_tab; if (!tab) { if (!btf_vmlinux) { verbose(env, "calling kernel function is not supported without CONFIG_DEBUG_INFO_BTF\n"); @@ -1699,6 +1815,19 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) prog_aux->kfunc_tab = tab; } + if (!btf_tab && offset) { + btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL); + if (!btf_tab) + return -ENOMEM; + prog_aux->kfunc_btf_tab = btf_tab; + } + + desc_btf = find_kfunc_desc_btf(env, func_id, offset, NULL); + if (IS_ERR(desc_btf)) { + verbose(env, "failed to find BTF for kernel function\n"); + return PTR_ERR(desc_btf); + } + if (find_kfunc_desc(env->prog, func_id)) return 0; @@ -1707,20 +1836,20 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) return -E2BIG; } - func = btf_type_by_id(btf_vmlinux, func_id); + func = btf_type_by_id(desc_btf, func_id); if (!func || !btf_type_is_func(func)) { verbose(env, "kernel btf_id %u is not a function\n", func_id); return -EINVAL; } - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func_proto = btf_type_by_id(desc_btf, func->type); if (!func_proto || !btf_type_is_func_proto(func_proto)) { verbose(env, "kernel function btf_id %u does not have a valid func_proto\n", func_id); return -EINVAL; } - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); + func_name = btf_name_by_offset(desc_btf, func->name_off); addr = kallsyms_lookup_name(func_name); if (!addr) { verbose(env, "cannot find address for kernel function %s\n", @@ -1731,7 +1860,7 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) desc = &tab->descs[tab->nr_descs++]; desc->func_id = func_id; desc->imm = BPF_CAST_CALL(addr) - __bpf_call_base; - err = btf_distill_func_proto(&env->log, btf_vmlinux, + err = btf_distill_func_proto(&env->log, desc_btf, func_proto, func_name, &desc->func_model); if (!err) @@ -1815,7 +1944,7 @@ static int add_subprog_and_kfunc(struct bpf_verifier_env *env) } else if (bpf_pseudo_call(insn)) { ret = add_subprog(env, i + insn->imm + 1); } else { - ret = add_kfunc_call(env, insn->imm); + ret = add_kfunc_call(env, insn->imm, insn->off); } if (ret < 0) @@ -2152,12 +2281,17 @@ static int get_prev_insn_idx(struct bpf_verifier_state *st, int i, static const char *disasm_kfunc_name(void *data, const struct bpf_insn *insn) { const struct btf_type *func; + struct btf *desc_btf; if (insn->src_reg != BPF_PSEUDO_KFUNC_CALL) return NULL; - func = btf_type_by_id(btf_vmlinux, insn->imm); - return btf_name_by_offset(btf_vmlinux, func->name_off); + desc_btf = find_kfunc_desc_btf(data, insn->imm, insn->off, NULL); + if (IS_ERR(desc_btf)) + return ""; + + func = btf_type_by_id(desc_btf, insn->imm); + return btf_name_by_offset(desc_btf, func->name_off); } /* For given verifier state backtrack_insn() is called from the last insn to @@ -6485,23 +6619,29 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) struct bpf_reg_state *regs = cur_regs(env); const char *func_name, *ptr_type_name; u32 i, nargs, func_id, ptr_type_id; + struct module *btf_mod = NULL; const struct btf_param *args; + struct btf *desc_btf; int err; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off, &btf_mod); + if (IS_ERR(desc_btf)) + return PTR_ERR(desc_btf); + func_id = insn->imm; - func = btf_type_by_id(btf_vmlinux, func_id); - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func = btf_type_by_id(desc_btf, func_id); + func_name = btf_name_by_offset(desc_btf, func->name_off); + func_proto = btf_type_by_id(desc_btf, func->type); if (!env->ops->check_kfunc_call || - !env->ops->check_kfunc_call(func_id)) { + !env->ops->check_kfunc_call(func_id, btf_mod)) { verbose(env, "calling kernel function %s is not allowed\n", func_name); return -EACCES; } /* Check the arguments */ - err = btf_check_kfunc_arg_match(env, btf_vmlinux, func_id, regs); + err = btf_check_kfunc_arg_match(env, desc_btf, func_id, regs); if (err) return err; @@ -6509,15 +6649,15 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) mark_reg_not_init(env, regs, caller_saved[i]); /* Check return type */ - t = btf_type_skip_modifiers(btf_vmlinux, func_proto->type, NULL); + t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); mark_btf_func_reg_size(env, BPF_REG_0, t->size); } else if (btf_type_is_ptr(t)) { - ptr_type = btf_type_skip_modifiers(btf_vmlinux, t->type, + ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); if (!btf_type_is_struct(ptr_type)) { - ptr_type_name = btf_name_by_offset(btf_vmlinux, + ptr_type_name = btf_name_by_offset(desc_btf, ptr_type->name_off); verbose(env, "kernel function %s returns pointer type %s %s is not supported\n", func_name, btf_type_str(ptr_type), @@ -6525,7 +6665,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) return -EINVAL; } mark_reg_known_zero(env, regs, BPF_REG_0); - regs[BPF_REG_0].btf = btf_vmlinux; + regs[BPF_REG_0].btf = desc_btf; regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; mark_btf_func_reg_size(env, BPF_REG_0, sizeof(void *)); @@ -6536,7 +6676,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) for (i = 0; i < nargs; i++) { u32 regno = i + 1; - t = btf_type_skip_modifiers(btf_vmlinux, args[i].type, NULL); + t = btf_type_skip_modifiers(desc_btf, args[i].type, NULL); if (btf_type_is_ptr(t)) mark_btf_func_reg_size(env, regno, sizeof(void *)); else @@ -11074,7 +11214,8 @@ static int do_check(struct bpf_verifier_env *env) env->jmps_processed++; if (opcode == BPF_CALL) { if (BPF_SRC(insn->code) != BPF_K || - insn->off != 0 || + (insn->src_reg != BPF_PSEUDO_KFUNC_CALL + && insn->off != 0) || (insn->src_reg != BPF_REG_0 && insn->src_reg != BPF_PSEUDO_CALL && insn->src_reg != BPF_PSEUDO_KFUNC_CALL) || @@ -12430,6 +12571,7 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->stack_depth = env->subprog_info[i].stack_depth; func[i]->jit_requested = 1; func[i]->aux->kfunc_tab = prog->aux->kfunc_tab; + func[i]->aux->kfunc_btf_tab = prog->aux->kfunc_btf_tab; func[i]->aux->linfo = prog->aux->linfo; func[i]->aux->nr_linfo = prog->aux->nr_linfo; func[i]->aux->jited_linfo = prog->aux->jited_linfo; diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index fcb2f493f710..fe5c34f414a2 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -241,7 +241,7 @@ BTF_ID(func, bpf_kfunc_call_test2) BTF_ID(func, bpf_kfunc_call_test3) BTF_SET_END(test_sk_kfunc_ids) -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id) +bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner) { return btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id); } diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 0dcee9df1326..b3afd3361f34 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -255,7 +255,7 @@ BTF_ID(func, bbr_set_state) #endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) -static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id) +static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) { return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); } From patchwork Tue Sep 14 12:37:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493049 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71529C433FE for ; Tue, 14 Sep 2021 12:38:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B6AF60FED for ; Tue, 14 Sep 2021 12:38:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232796AbhINMjV (ORCPT ); Tue, 14 Sep 2021 08:39:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232898AbhINMjV (ORCPT ); Tue, 14 Sep 2021 08:39:21 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9127C061574; Tue, 14 Sep 2021 05:38:03 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id 5so8090646plo.5; Tue, 14 Sep 2021 05:38:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8H2t4nrL6OKI1S9DG0MWY1o7DOImDjh2AQh5r73KfwE=; b=HSSkXn9E/xjh7qvVIHKvBqT9uHQd/tWBsDzMUuyKyO3jtlr8qVf2MZzZXJ5JKHY7Vt Uy4tKOpvsr0ZRqnv0HpQh7KaH/dnlXQIeEsqNUmQHDMx20yaYXlUBiPV5QO6qMFWIYeV Re+7WggvdBKONFLMuk+MF7iEZ9CbpDEpG6TuahgqYRw/BWszvfuLrQ1LafqHhWU/Ee+k 3o20Eln8VKH+/3jkHPQdbKb94tGhv69cI70tF70jzzn1taf04e0dV8qcV9BkqpBQ5C0u 8MCVPMFGkFT/K3/uIFZycAF/EJjZ7vTwpjEcJvE7lkRDn1IgT4Y8uJ9Braj3F2RLpIMY Ni1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8H2t4nrL6OKI1S9DG0MWY1o7DOImDjh2AQh5r73KfwE=; b=eNkG3PVq0IyaJyCVNAcJzdehvhLY6GVm8N6uNNFCBY1Gbm0SS63Rt0deiMSXccaoR5 10gXokIjuUOWsLkcMc9UjfeshfPOV1+YrosqgHZDZ0RYoZ08wujGc0Jkxv9uum3uk5aH VuMU65IUWVDRGX/oXIslt5UQgnxytiUYZkYeYReD1wqK7TWFRg+qOozagsySZTi5glnU e5VJ+TgVfhmqYicVLon7bARpzPvMR1JcKBOM1CzLHRtTWy9r5Bl5w03EwYlk8ET3ib+C ZtHw5xQr76Ib5Es/ZbRPy/JUS70CDM+0AfVKtiT0wMbII2C8aWuUpomY7oHhjw2IxToq JmqQ== X-Gm-Message-State: AOAM5307Pn6gpUGk86MV9M2yFYdzMpnr28FAwf/dQq8LdeL6HoWCK0xB cDd7ghfm6sKSSFw2mjlHu8Lwnm7+MpY3HQ== X-Google-Smtp-Source: ABdhPJw9XmePtM+qL5wus8UU1NXXk34eNWX701/q+n8eJowfw55o4wMlGJlmYS4hWcAA3bB1T9ZQpg== X-Received: by 2002:a17:90b:46cd:: with SMTP id jx13mr1870661pjb.122.1631623083263; Tue, 14 Sep 2021 05:38:03 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id cm5sm1738453pjb.24.2021.09.14.05.38.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:03 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 02/10] bpf: Be conservative while processing invalid kfunc calls Date: Tue, 14 Sep 2021 18:07:41 +0530 Message-Id: <20210914123750.460750-3-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2253; h=from:subject; bh=KwrtasYlogJBoIL6nFroYhICQdUm8Kr8EgPEOr5ZFWY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdWIlzQb2Ml0Ubt+UAOvxa6xmTwJ5H8mn5f8ZNS wCNP8PyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVgAKCRBM4MiGSL8RyoDeD/ 9fkForr9OuFOwjsaguAQOmYPqm1GiSyGvBWc99FqQoCDLL4WNyFF/7TFkQZOK+8p8nnZSJhaWSu+Ga E2BwwQbCEytEck+XVoyrB48NlyGe+xm2O3NgcruIPcVO0ZKzMYPcW5LzMUy4spQ6tImVd2lme+ilW5 bzMU0J3Gmsg6Csutkmb70bB/Em9Do7ML0hUw4FKmHcgor71qwqVCKywkIp2PkR4FuDmJf8p4V8MN9P ZhDIFM9CWSfm5V6MW0mIDoqqW8J6wHvMMFZl5Uy0QnMPrjrK3lS0bUAVhb8ZeR+rl+bfUd8BuRSU4V L2O2mPX+AuccMUBwaJopAuaWN6jlmGczTgeBjRg5jdcXNf7HWSK8MFExTGYtznj83yLerdPIQvvqUk RH6IOsCZ/7zaJmm8Uu5Aj9JFmKnwYJaOwv0o0Hd5O+vuBSK4fQKR+7y2xrq0Y8KCl0pQo2WEY3ho6Z nOoPGYyhg9Ub9fAtezXZ5sOexOvopKsa/h9vof5VFtw30uC9fZ+z6wxZPvLhTrYyjdGRH6N7oXjJD5 CLGy5Ll5brlY4s/2N9jjzObcudrZHPD0CFFaHvfx3NSb9dH7nJUCvH9j4GOcZSoP/fv5EW9R/i/hgD 2NMfXQmL0tkJLg4QLHeyfPb4Tmr+2nEYGtzI1fGm2HpJ4oSuPKLunbIBhWzw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch also modifies the BPF verifier to only return error for invalid kfunc calls specially marked by userspace (with insn->imm == 0, insn->off == 0) after the verifier has eliminated dead instructions. This can be handled in the fixup stage, and skip processing during add and check stages. If such an invalid call is dropped, the fixup stage will not encounter insn->imm as 0, otherwise it bails out and returns an error. This will be exposed as weak ksym support in libbpf in subsequent patch. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/verifier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 6bbbb98f4ee2..7dd2a632ea6f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1815,6 +1815,15 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) prog_aux->kfunc_tab = tab; } + /* btf idr allocates IDs from 1, so func_id == 0 is always invalid, but + * instead of returning an error, be conservative and wait until the + * code elimination pass before returning error, so that invalid calls + * that get pruned out can be in BPF programs loaded from userspace. + * It is also required that offset be untouched (0) for such calls. + */ + if (!func_id && !offset) + return 0; + if (!btf_tab && offset) { btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL); if (!btf_tab) @@ -6624,6 +6633,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) struct btf *desc_btf; int err; + /* skip for now, but return error when we find this in fixup_kfunc_call */ + if (!insn->imm) + return 0; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off, &btf_mod); if (IS_ERR(desc_btf)) return PTR_ERR(desc_btf); @@ -12758,6 +12771,11 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, { const struct bpf_kfunc_desc *desc; + if (!insn->imm) { + verbose(env, "invalid kernel function call not eliminated in verifier pass\n"); + return -EINVAL; + } + /* insn->imm has the btf func_id. Replace it with * an address (relative to __bpf_base_call). */ From patchwork Tue Sep 14 12:37:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493051 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2FD0C433F5 for ; Tue, 14 Sep 2021 12:38:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C878B60FED for ; Tue, 14 Sep 2021 12:38:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232990AbhINMjb (ORCPT ); Tue, 14 Sep 2021 08:39:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232909AbhINMjY (ORCPT ); Tue, 14 Sep 2021 08:39:24 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0475AC061574; Tue, 14 Sep 2021 05:38:07 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id f3-20020a17090a638300b00199097ddf1aso2058626pjj.0; Tue, 14 Sep 2021 05:38:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o2uir4PaWOGPcMyQFzYq+u0pT9SxwzyBbttgTlyAw3k=; b=GRhH4wC3fTltWPJzNjdpktt0sG4vnb1Jv6mK4JsXL+Yt9kf9pvBBXssimCBiFEV37K SYisBm/rAkOfljKq15tVtcCAbGxoI3ZHo41rEEJki9DYBy2fguiFbgorPLx6D8cXy66j 7DJCcceIEPBifVHNbvL8A00AeRaL8kET0mwuI+A2d8YS78kXG+zYuK1saEi8Tm3iQv10 rCKf6pGeOAKzCBsn5ecuiaMT1OC2VeqTbGR7LIgPdk7zyWJnfOmgGD7JcWyAGRx3swCV VptiQIgV72d87KIRdIZ4sOSImMMlI5hNnh5jTmVbvslWVmgS8OGIrFFEstQ+FZ+naiHW n3eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o2uir4PaWOGPcMyQFzYq+u0pT9SxwzyBbttgTlyAw3k=; b=v3ym2TwlSpbnTBeFkacCWTtdvBne75s3qSEg6dbffmdwSCldREIjayJWmhKOj8JqGH i6TAAwbit0RK0WrrIw+cI0VKMBM3L1niemgsJN9Ayj2f6FLBl9l+TSNfmq+TLDgRwnpb XfJIntP/zN3Ud2iWMkpTGgw9y2sxj6YuIRSLQc8wwazTod9VBX1y8FfU68zc5EulDUWc S4VLAQaXo18CeDdyAkzotk14FXxkbUjcfKhhfUObEak3/YEOXREX1I1WRJj3ix+fRqJA m62OJ27YlZmGwGE4ASG0VO9LF4mrBPxWsimHgr7PcNnHv2RJUzF1Y7fNiilYAspLeZTG o4hg== X-Gm-Message-State: AOAM532T2sUEoZZNHAV7HULXzSKfur+bD5ViiIJa9JtIrcUqG2iGFALB 4i0ksVpf5lxNaMfGgyhfc66QM7+r5wesNg== X-Google-Smtp-Source: ABdhPJxYxBXDFKCZPaTAHe9SzfUfhHbx3LheQrnFY4AIiwCHvt/TL3+LDMOU4P1nZHhs4p9A9dCJeg== X-Received: by 2002:a17:90b:4a44:: with SMTP id lb4mr1890054pjb.140.1631623086344; Tue, 14 Sep 2021 05:38:06 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id x26sm10413697pfm.77.2021.09.14.05.38.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:06 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 03/10] bpf: btf: Introduce helpers for dynamic BTF set registration Date: Tue, 14 Sep 2021 18:07:42 +0530 Message-Id: <20210914123750.460750-4-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5061; h=from:subject; bh=xIk+3p0yR6nDontP2RM+Qd15PO5zXVr+rOfltuzXi+A=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdWQTDX6mqAQpBBQhjssU+jDKZWQWpRO+akKeCp w/G8cAqJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVgAKCRBM4MiGSL8RyjOuD/ 9CVzLyyx/fzakF923YE3zGD0O+H01tFG7xg4lwPHGc3s1sMzODtomuDHBHcR3bKR2MA3NBLSqR0q8/ 1NgJGokExfM4INmng1po58QHPDMrsKDX7PWtdn/FINJxzp/Bv4QBcbaKVPUq9DtkyEMKfXu8Vs4TYp gLZqGE8/swk2O1Q3EZxbDbrBiYJiJAZI+zYCPDn2WnjrkIDcGEdZm36+mMvi3/mXCKGuOPQXBBRldl VgcoOspzl7BFLzyHZDTidZnkP67KLzvvVdL8c/8ZMo8rL4/jq2I2NwFB4H3zeEwcR1w6/HRjlKkR1D TJkgfXXcn+2a8Z9Usm2Mp9pe7jx7hB89sKLeOpFYElVqol3RKL0DbRo0xZ595CBqgKVA6oALtP7bQB GUEKi+RLs2Ze0GHYGxKnO75X48VtJIpitqeUb7v7Xe+Fm6lm9hznEbJ5m1dWuyBtUnezSrncM74ELY NkLweevfFRi/g0FEOO5lHhvtND+oJ9YBXUv9ymw34JQtw634Dx6x2RcUl3u4ntWelff/we/AShwsPg n+pRIKECoeIO2HyoAVkQ3BJi0vzKkLb4sG8SuNPua62lL1uUUxY1mMZTruM4pJBREdMT12LnnVo3Wd g/EoI465vgDLufoe6sepNB0vJNCnhDXlSlL4+rm0F6UlUpq2gtEIw0qIgsDA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This adds helpers for registering btf_id_set from modules and the check_kfunc_call callback that can be used to look them up. With in kernel sets, the way this is supposed to work is, in kernel callback looks up within the in-kernel kfunc whitelist, and then defers to the dynamic BTF set lookup if it doesn't find the BTF id. If there is no in-kernel BTF id set, this callback can be used directly. Also fix includes for btf.h and bpfptr.h so that they can included in isolation. This is in preparation for their usage in tcp_bbr, tcp_cubic and tcp_dctcp modules in the next patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpfptr.h | 1 + include/linux/btf.h | 32 ++++++++++++++++++++++++++ kernel/bpf/btf.c | 51 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 84 insertions(+) diff --git a/include/linux/bpfptr.h b/include/linux/bpfptr.h index 546e27fc6d46..46e1757d06a3 100644 --- a/include/linux/bpfptr.h +++ b/include/linux/bpfptr.h @@ -3,6 +3,7 @@ #ifndef _LINUX_BPFPTR_H #define _LINUX_BPFPTR_H +#include #include typedef sockptr_t bpfptr_t; diff --git a/include/linux/btf.h b/include/linux/btf.h index 214fde93214b..e29a486d09d4 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -5,8 +5,10 @@ #define _LINUX_BTF_H 1 #include +#include #include #include +#include #define BTF_TYPE_EMIT(type) ((void)(type *)0) #define BTF_TYPE_EMIT_ENUM(enum_val) ((void)enum_val) @@ -238,4 +240,34 @@ static inline const char *btf_name_by_offset(const struct btf *btf, } #endif +struct kfunc_btf_id_set { + struct list_head list; + struct btf_id_set *set; + struct module *owner; +}; + +struct kfunc_btf_id_list; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES +void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s); +void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s); +#else +static inline void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ +} +static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ +} +#endif + +#define DECLARE_CHECK_KFUNC_CALLBACK(type) \ + bool __bpf_check_##type##_kfunc_call(u32 kfunc_id, struct module *owner) +#define DEFINE_KFUNC_BTF_ID_SET(set, name) \ + struct kfunc_btf_id_set name = { LIST_HEAD_INIT(name.list), (set), \ + THIS_MODULE } + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index dfe61df4f974..eecafed56300 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6215,3 +6215,54 @@ const struct bpf_func_proto bpf_btf_find_by_name_kind_proto = { }; BTF_ID_LIST_GLOBAL_SINGLE(btf_task_struct_ids, struct, task_struct) + +struct kfunc_btf_id_list { + struct list_head list; + struct mutex mutex; +}; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + +void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ + mutex_lock(&l->mutex); + list_add(&s->list, &l->list); + mutex_unlock(&l->mutex); +} +EXPORT_SYMBOL_GPL(register_kfunc_btf_id_set); + +void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ + mutex_lock(&l->mutex); + list_del_init(&s->list); + mutex_unlock(&l->mutex); +} +EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); + +#endif + +#define DEFINE_KFUNC_BTF_ID_LIST(name) \ + struct kfunc_btf_id_list name = { LIST_HEAD_INIT(name.list), \ + __MUTEX_INITIALIZER(name.mutex) }; \ + EXPORT_SYMBOL_GPL(name) + +#define DEFINE_CHECK_KFUNC_CALLBACK(type, list_name) \ + bool __bpf_check_##type##_kfunc_call(u32 kfunc_id, \ + struct module *owner) \ + { \ + struct kfunc_btf_id_set *s; \ + if (!owner || !IS_ENABLED(CONFIG_DEBUG_INFO_BTF_MODULES)) \ + return false; \ + mutex_lock(&list_name.mutex); \ + list_for_each_entry(s, &list_name.list, list) { \ + if (s->owner == owner && \ + btf_id_set_contains(s->set, kfunc_id)) { \ + mutex_unlock(&list_name.mutex); \ + return true; \ + } \ + } \ + mutex_unlock(&list_name.mutex); \ + return false; \ + } From patchwork Tue Sep 14 12:37:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493053 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C95A7C4332F for ; Tue, 14 Sep 2021 12:38:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B151F60FED for ; Tue, 14 Sep 2021 12:38:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232909AbhINMjc (ORCPT ); Tue, 14 Sep 2021 08:39:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232902AbhINMja (ORCPT ); Tue, 14 Sep 2021 08:39:30 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05B74C0613C1; Tue, 14 Sep 2021 05:38:10 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id 17so12590296pgp.4; Tue, 14 Sep 2021 05:38:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yU4+xx6iAwgsGxgz5Ht4AFD9WjITvbQsNdCqZ6ULtK4=; b=C8+gBjG8gvVycJFtB7zCILzp77d7RShKQD2jaimyINWujoTBuZ2CTnoFdV6asYKcUZ e/4ZDEXnQ4oyewRWjBiX4sULtUhgVZ7VxQmoqlq2Qd6wauIbZeaGG+bjQBMoDMJyqwue Ysy9jyEUumjsisI2NaBRSdocijhsXP5AwgboHXWPzZQfEq2HGlPrLPY09nK/Jy8Eud6v Qh40jBxlRmz/SKLI1OUiEU+rIXfG8IkcMzS2XtpwC7+TFztxXyz5G7oJ74lv8JyXBPM7 r9w2/UbaZLQEhhfvXIG9ACy3+vM0qU1I4E/xs1WYDWGxT9yXTD/TBHVHXV9PLlHl5TIo FwMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yU4+xx6iAwgsGxgz5Ht4AFD9WjITvbQsNdCqZ6ULtK4=; b=eIuDYAItGyhEKJTHMaHjJ8K2UwXWw9VYRL/xd+oehoLsDtgjs2/8B2v6XzaB82TFFx ymYfHpUrrkJtgVezUlNHvP3HPqa+j4uMrdeRlHN7TfdlmCa5d/Gv2wsVONdC1LKyYij3 hj8sstF3+D1SmU3yW6X7O69AV5UIbStZMZ1OtFiYCGMferZhpjhlV0GQVduehX/6lbOt S+7f1DRlbgmXUTwKazAMshRWA5kLSeA3HORrLddfi7yI4X+G8bGjPLk8QHnIf9vidoai BhLx0jsvMA7XnlfT96pyLPjYJhFOzWcb8qlcSb7TnuEZwTG1OP4gzgpuQZ2bJCmkHjrO mnxA== X-Gm-Message-State: AOAM533+GyNNBu37e2DNE68SIx821kj2sCACqD+QhU79jEkk/MRlGPur d/fhAee0w+APhTt38YeTuenpoO4OOgnnfA== X-Google-Smtp-Source: ABdhPJwXEgJUgD8L7WqLCR+jmsAzNCoZX5BPHOTUKm6XgG5EVVS1sK8T9hfbtq96byx/YrhHRC9r+w== X-Received: by 2002:a62:6d07:0:b0:40a:33cd:d3ea with SMTP id i7-20020a626d07000000b0040a33cdd3eamr4441713pfc.61.1631623089404; Tue, 14 Sep 2021 05:38:09 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id z8sm10040931pfa.113.2021.09.14.05.38.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:09 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 04/10] tools: Allow specifying base BTF file in resolve_btfids Date: Tue, 14 Sep 2021 18:07:43 +0530 Message-Id: <20210914123750.460750-5-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2098; h=from:subject; bh=kZhRM1Uo6vUje4gN/WsUjSsDNLJ5u0WuItuVyu5D9Zk=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdWDhwehFEbRic2pkL2wCQ10r1eCg0KKIKoWVSG lKYtwOmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVgAKCRBM4MiGSL8RymBXD/ 49y4gvzxmoWUtayiH1hrKzBuVcNgBJogUNJYKJAoZIeUGsAvseQhwIB+UN2ojkb5ONSX4gPRd8IJ5A jc2tOe3f0+KmWcyiGHKfEecS35Ef4gHoTSZhBrjuH20lbZ7qt3qCg4laqNZ9t1XnHmApAD0Ovh48xf fvrQh1gvtUXn0jDuFdZoyeVWAoeDasoL+sdxcM013eK/58Va923mLwKBXCtZvmyBJG8JkR39XuLq2W 9Plm2k1nd3WsjTZBrGyqPkl6UgDHFTSuj4mvKiKHx/LHJUFNpe27H61owKQgBkrcxf6ISPbOQER8Vw sf7tkmsi8bxfq4O2iLtqK4AsSM7JuS5WOsilIOxSf4RNpJGvJtRLkB1kzG4RPJW9Qvo33MLgWhcGIi VitHZe7ds0E19aaLTLAfsMVhgRKwlCuVDIRFa3EFaKreNkeGdO1fbAiUXbHHI1Rjx9OecGbE1RkmU7 HgW7RbVfSdOb1/MozuG1TftIYKBovBR16ITuvi8wfNyvbEZWX4cNHt31BcnrQ7BF5TN4qwKGSMItcy 8kvJHI0WWVJk0miZ0BXr7/Y6TmdEG1EV1Ks4IfWxbu5D5QI6gVWaX1XBs/7PtygUv9JhqeLBZh0TeC WCbHj14V03iHHZf3HEMXo7S69dEiXf+2tc7RZvbq/gu3TnVuNtLd1S9m7kmw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commits allows specifying the base BTF for resolving btf id lists/sets during link time in the resolve_btfids tool. The base BTF is set to NULL if no path is passed. This allows resolving BTF ids for module kernel objects. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/bpf/resolve_btfids/main.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index de6365b53c9c..206e1120082f 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -89,6 +89,7 @@ struct btf_id { struct object { const char *path; const char *btf; + const char *base_btf_path; struct { int fd; @@ -477,16 +478,27 @@ static int symbols_resolve(struct object *obj) int nr_structs = obj->nr_structs; int nr_unions = obj->nr_unions; int nr_funcs = obj->nr_funcs; + struct btf *base_btf = NULL; int err, type_id; struct btf *btf; __u32 nr_types; - btf = btf__parse(obj->btf ?: obj->path, NULL); + if (obj->base_btf_path) { + base_btf = btf__parse(obj->base_btf_path, NULL); + err = libbpf_get_error(base_btf); + if (err) { + pr_err("FAILED: load base BTF from %s: %s\n", + obj->base_btf_path, strerror(-err)); + return -1; + } + } + + btf = btf__parse_split(obj->btf ?: obj->path, base_btf); err = libbpf_get_error(btf); if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); - return -1; + goto out; } err = -1; @@ -545,6 +557,7 @@ static int symbols_resolve(struct object *obj) err = 0; out: + btf__free(base_btf); btf__free(btf); return err; } @@ -697,6 +710,8 @@ int main(int argc, const char **argv) "BTF data"), OPT_BOOLEAN(0, "no-fail", &no_fail, "do not fail if " BTF_IDS_SECTION " section is not found"), + OPT_STRING('s', "base-btf", &obj.base_btf_path, "file", + "path of file providing base BTF data"), OPT_END() }; int err = -1; From patchwork Tue Sep 14 12:37:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493057 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D99DC433EF for ; Tue, 14 Sep 2021 12:38:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 562006108B for ; Tue, 14 Sep 2021 12:38:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232952AbhINMjd (ORCPT ); Tue, 14 Sep 2021 08:39:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232983AbhINMjb (ORCPT ); Tue, 14 Sep 2021 08:39:31 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28A50C0613D8; Tue, 14 Sep 2021 05:38:13 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id f65so12058062pfb.10; Tue, 14 Sep 2021 05:38:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mqoITvHXLse1CtgZzTC8zsmR2oDKQ32w61pGdrlO134=; b=lJ/sga4HKlMyCBXCYuA5kbsp4SYbtcR+SY2VkTI7Sz55h0xfax3oHFMEG7m3e30Os7 CwqUzabjIvNUi+r12XD6fGp4LkKlQncPAN8eXQcngpOlt+bfzP9sG1VcSP/gU0MZN7H/ +dHKCFTXhgef19sEjM+7QzPstsLqYMj0NRRHuwESqXnQ0jZgb8xfiGQrKLStwTG3tdan vISeloNS4+evglIIjqgooMCtUHBEJlmuqTef2PkyTK9C0mkBw9ajIVveVrjTX5eB1Yz1 Jh4XZnmpOf9LVOppKBbXpOFkFgZyKT4KYrhI5vFXb3PEj/drO9GsrWgWZs+2cUFuVRD0 XC6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mqoITvHXLse1CtgZzTC8zsmR2oDKQ32w61pGdrlO134=; b=foMfLR0ahL4Ax7cFKIg7RFHjgDOX5jqkcDZIfvZO/pFybJhxyNtAzcUrPGp3cePwgE J3Gg6KVvsBqtcHUMBgqGcRU9p0eemqbgyXNo21MUayIKRLdcdV39mBUcuOwRyDKSxrHT F29h3Ny7Pb9WL5IHBCt20mqy2Y6BsYBdtCH4J2CT8yWtiiWUWHlvfkvesgpAA3/1ZhbP UNjpQdI9JmmYvrJhVm141W9ga/i0GOOom06+0fmXKV3RtbShU1dQ3Y+2Sm/8dDAqv+aH RknIsVr7/OVQHPCI3hwMn2sbYkEWE6ABpUZBM3ybC5zcTKl7q5T4lJ7j5C9/81RcJRb3 nGSg== X-Gm-Message-State: AOAM533BUGHDyrDJfpL/OV8fFjBZ/+YD0keCFdsbNTaZNItcBwDBnAIU 6DBSZeWxxCy+7Y1ZLV5W2Ab3P5sjFKzplw== X-Google-Smtp-Source: ABdhPJxV7JnLNfkztxhRoKJYAbyGK3ELCIib71XYQ+IkgDUdOV88aWxza3m9xZJEQzrBgAyq1Vhlew== X-Received: by 2002:a63:4c1f:: with SMTP id z31mr15479907pga.50.1631623092349; Tue, 14 Sep 2021 05:38:12 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id j128sm10708893pfd.38.2021.09.14.05.38.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:12 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 05/10] bpf: Enable TCP congestion control kfunc from modules Date: Tue, 14 Sep 2021 18:07:44 +0530 Message-Id: <20210914123750.460750-6-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9235; h=from:subject; bh=m9TZ+KdIbneN6+JbSrnpTeto3oY8+468yP/Y99kWUeA=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdW+lyJs/2rzqQm9zAFAR9Ff5iaBUkPwjsAUNGP CiQrob+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVgAKCRBM4MiGSL8RykwZD/ 4rjozNckLlXgbTbcn1m/+Ujnk4/P7pdGulpMr4tMVpKxOjv4ZLImlqsJpAMRUlwLOCyNilqe04dPvl aZ6L4tJqTDmGkO5PRwX5WZw2LFWtivTwOpDU/wPSM5A1M9wnscvyTPj5kwiRiy9C6mUyndB8zRX9ck 2OC+H1iJ8U8fBZHoXfzR+7trw+oFc0aG4qQ2f1kuWdXxbA+ZuhL7ImKr05zpwWz3kS7a+4Q7nMuok+ 7GZT7URRhUZfnZv+yjX6VYrXC9ik+vrUiXtaIp2+XWpokmyWNDvxT5dZvepXTNinTd0Ka5aqmgug7u Od50CPzmwtivFRW5HPYyhRuusykrnTENK2Cz0+WjUgJ9yk0jF/irJ41Jt/vANvLDnvEr6gDeznibcx bFSw5EuouVx/u0HMIjeOpUq3iAcdU3PBbN1s1bQ00HYalP3Zz+IDRHT+4h1zmBKLjdiVHXJHXPKOPr aq0g/fsWLifjEPdHyUiKzn8F1n3/dNLRXeevyDokio+cwuO4FMOEW43sIsMfgTIsXLo7AjwABPC7rI 1ZDfXgT1RY/fAaMkvf8w8E1IH4Ao8tmiY6tY49CPWRkKL6IjDWHiUjxC9eQL7RU3Lo22ZRseD6iakt w4t3IVBPn8dPRTiHl1poDeAhB8yVxlCxXyq6AIIoVR7XSpvumHgMM8u7Ti1g== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit moves BTF ID lookup into the newly added registration helper, in a way that the bbr, cubic, and dctcp implementation set up their sets in the bpf_tcp_ca kfunc_btf_set list, while the ones not dependent on modules are looked up from the wrapper function. This lifts the restriction for them to be compiled as built in objects, and can be loaded as modules if required. Also modify Makefile.modfinal to resolve_btfids in TCP congestion control modules if the config option is set, using the base BTF support added in the previous commit. See following commits for background on use of: CONFIG_X86 ifdef: 569c484f9995 (bpf: Limit static tcp-cc functions in the .BTF_ids list to x86) CONFIG_DYNAMIC_FTRACE ifdef: 7aae231ac93b (bpf: tcp: Limit calling some tcp cc functions to CONFIG_DYNAMIC_FTRACE) [ resolve_btfids uses --no-fail because some crypto kernel modules under arch/x86/crypto generated from ASM do not have the .BTF sections ] Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 4 ++++ kernel/bpf/btf.c | 3 +++ net/ipv4/bpf_tcp_ca.c | 34 +++------------------------------- net/ipv4/tcp_bbr.c | 28 +++++++++++++++++++++++++++- net/ipv4/tcp_cubic.c | 26 +++++++++++++++++++++++++- net/ipv4/tcp_dctcp.c | 26 +++++++++++++++++++++++++- scripts/Makefile.modfinal | 1 + 7 files changed, 88 insertions(+), 34 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index e29a486d09d4..c7b6382123e1 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -270,4 +270,8 @@ static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, struct kfunc_btf_id_set name = { LIST_HEAD_INIT(name.list), (set), \ THIS_MODULE } +extern struct kfunc_btf_id_list bpf_tcp_ca_kfunc_list; + +DECLARE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca); + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index eecafed56300..8240478b7398 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6266,3 +6266,6 @@ EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); mutex_unlock(&list_name.mutex); \ return false; \ } + +DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list); +DEFINE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca, bpf_tcp_ca_kfunc_list); diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index b3afd3361f34..c9f1d2dcf67b 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -223,41 +223,13 @@ BTF_ID(func, tcp_reno_cong_avoid) BTF_ID(func, tcp_reno_undo_cwnd) BTF_ID(func, tcp_slow_start) BTF_ID(func, tcp_cong_avoid_ai) -#ifdef CONFIG_X86 -#ifdef CONFIG_DYNAMIC_FTRACE -#if IS_BUILTIN(CONFIG_TCP_CONG_CUBIC) -BTF_ID(func, cubictcp_init) -BTF_ID(func, cubictcp_recalc_ssthresh) -BTF_ID(func, cubictcp_cong_avoid) -BTF_ID(func, cubictcp_state) -BTF_ID(func, cubictcp_cwnd_event) -BTF_ID(func, cubictcp_acked) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_DCTCP) -BTF_ID(func, dctcp_init) -BTF_ID(func, dctcp_update_alpha) -BTF_ID(func, dctcp_cwnd_event) -BTF_ID(func, dctcp_ssthresh) -BTF_ID(func, dctcp_cwnd_undo) -BTF_ID(func, dctcp_state) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_BBR) -BTF_ID(func, bbr_init) -BTF_ID(func, bbr_main) -BTF_ID(func, bbr_sndbuf_expand) -BTF_ID(func, bbr_undo_cwnd) -BTF_ID(func, bbr_cwnd_event) -BTF_ID(func, bbr_ssthresh) -BTF_ID(func, bbr_min_tso_segs) -BTF_ID(func, bbr_set_state) -#endif -#endif /* CONFIG_DYNAMIC_FTRACE */ -#endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) { - return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); + if (btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id)) + return true; + return __bpf_check_bpf_tcp_ca_kfunc_call(kfunc_btf_id, owner); } static const struct bpf_verifier_ops bpf_tcp_ca_verifier_ops = { diff --git a/net/ipv4/tcp_bbr.c b/net/ipv4/tcp_bbr.c index 6274462b86b4..ec5550089b4d 100644 --- a/net/ipv4/tcp_bbr.c +++ b/net/ipv4/tcp_bbr.c @@ -56,6 +56,8 @@ * otherwise TCP stack falls back to an internal pacing using one high * resolution timer per TCP socket and may use more resources. */ +#include +#include #include #include #include @@ -1152,14 +1154,38 @@ static struct tcp_congestion_ops tcp_bbr_cong_ops __read_mostly = { .set_state = bbr_set_state, }; +BTF_SET_START(tcp_bbr_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, bbr_init) +BTF_ID(func, bbr_main) +BTF_ID(func, bbr_sndbuf_expand) +BTF_ID(func, bbr_undo_cwnd) +BTF_ID(func, bbr_cwnd_event) +BTF_ID(func, bbr_ssthresh) +BTF_ID(func, bbr_min_tso_segs) +BTF_ID(func, bbr_set_state) +#endif +#endif +BTF_SET_END(tcp_bbr_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_bbr_kfunc_ids, tcp_bbr_kfunc_btf_set); + static int __init bbr_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bbr) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&tcp_bbr_cong_ops); + ret = tcp_register_congestion_control(&tcp_bbr_cong_ops); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); + return 0; } static void __exit bbr_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); tcp_unregister_congestion_control(&tcp_bbr_cong_ops); } diff --git a/net/ipv4/tcp_cubic.c b/net/ipv4/tcp_cubic.c index 4a30deaa9a37..5e9d9c51164c 100644 --- a/net/ipv4/tcp_cubic.c +++ b/net/ipv4/tcp_cubic.c @@ -25,6 +25,8 @@ */ #include +#include +#include #include #include #include @@ -482,8 +484,25 @@ static struct tcp_congestion_ops cubictcp __read_mostly = { .name = "cubic", }; +BTF_SET_START(tcp_cubic_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, cubictcp_init) +BTF_ID(func, cubictcp_recalc_ssthresh) +BTF_ID(func, cubictcp_cong_avoid) +BTF_ID(func, cubictcp_state) +BTF_ID(func, cubictcp_cwnd_event) +BTF_ID(func, cubictcp_acked) +#endif +#endif +BTF_SET_END(tcp_cubic_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_cubic_kfunc_ids, tcp_cubic_kfunc_btf_set); + static int __init cubictcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bictcp) > ICSK_CA_PRIV_SIZE); /* Precompute a bunch of the scaling factors that are used per-packet @@ -514,11 +533,16 @@ static int __init cubictcp_register(void) /* divide by bic_scale and by constant Srtt (100ms) */ do_div(cube_factor, bic_scale * 10); - return tcp_register_congestion_control(&cubictcp); + ret = tcp_register_congestion_control(&cubictcp); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); + return 0; } static void __exit cubictcp_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); tcp_unregister_congestion_control(&cubictcp); } diff --git a/net/ipv4/tcp_dctcp.c b/net/ipv4/tcp_dctcp.c index 79f705450c16..0d7ab3cc7b61 100644 --- a/net/ipv4/tcp_dctcp.c +++ b/net/ipv4/tcp_dctcp.c @@ -36,6 +36,8 @@ * Glenn Judd */ +#include +#include #include #include #include @@ -236,14 +238,36 @@ static struct tcp_congestion_ops dctcp_reno __read_mostly = { .name = "dctcp-reno", }; +BTF_SET_START(tcp_dctcp_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, dctcp_init) +BTF_ID(func, dctcp_update_alpha) +BTF_ID(func, dctcp_cwnd_event) +BTF_ID(func, dctcp_ssthresh) +BTF_ID(func, dctcp_cwnd_undo) +BTF_ID(func, dctcp_state) +#endif +#endif +BTF_SET_END(tcp_dctcp_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_dctcp_kfunc_ids, tcp_dctcp_kfunc_btf_set); + static int __init dctcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct dctcp) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&dctcp); + ret = tcp_register_congestion_control(&dctcp); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); + return 0; } static void __exit dctcp_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); tcp_unregister_congestion_control(&dctcp); } diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index ff805777431c..b4f83533eda6 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -41,6 +41,7 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ -f vmlinux ]; then \ LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) --no-fail -s vmlinux $@; \ else \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ fi; From patchwork Tue Sep 14 12:37:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493055 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BABE3C433FE for ; Tue, 14 Sep 2021 12:38:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4029610A6 for ; Tue, 14 Sep 2021 12:38:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232690AbhINMje (ORCPT ); Tue, 14 Sep 2021 08:39:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232937AbhINMjd (ORCPT ); Tue, 14 Sep 2021 08:39:33 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7C06C061574; Tue, 14 Sep 2021 05:38:15 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id v1so8082155plo.10; Tue, 14 Sep 2021 05:38:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hBFUZiw6zxf4pUKWQiQT11iEjQyVaR1toou8fKVqzJw=; b=YeAfWMtchwQfhWC7kL+ghw7be+SBhtzT0PSa1lKdsXAwUAgb2n6HPiaOeZ0iWrLOtW 7wfsncn2D5XMIL+wGGxwxLm9GveA/t5Ak/fZC/e5vIqVJe8rS8m82XFTAvWGOVYGidHe vtjOJjFNJxYu0tbC2UO1gFK9b4+OuYUmavSe1+cXw1OS6vLuxr3M9m3EGSaxY0csdBGi RPurR2NzXngjyzE/KMm9XD+saRAZcvLm/Uge+YmNEsabZ6zwb7G0IZjpEalOJHI+1pHC Hck63BfE716ayTDjePqQhQmNIsOl3jhoIooILuhu507NRYLbPiGRNtHWE5EzGX6d1S/n w0Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hBFUZiw6zxf4pUKWQiQT11iEjQyVaR1toou8fKVqzJw=; b=XVGjJudUpvcCGb20sbQn9bSZe9DwztU0uHm/gPr68JohAP9PFRTqea7yaqKhKY9Vd5 iMUuFmXHK9nie7FRkzPCpHqNSuxXS85o8Mi4V/mAbIAPbmupls3i5Gw4q5ucojD6hyv9 arweuniHC+/E4ZrZSDZ+yJdF4Pvp1l63IfcDvKQGwjSN+8ivyE0VBBngl0iBzXFindNo P9ByedrsEEy5rghbRSsdzhm7E7Xirls/ikP52aYLB2qY29Hrao+gIGc+kodkSgdcAvha Ov2R8UPFMWtboqnyzy/2rfqWw9ID9tzQjTgUvWmho1IV9sdk6TKBp0jW5XvxTAsGTMgs 6jxA== X-Gm-Message-State: AOAM531Qab1Z/kShB+7VK8zh+E0MiWhbsE//n9B7h7Ll5FXTtdrGggXk irjMeMKP8qIAcIsZe+c+Ak4aNqekkmEgBA== X-Google-Smtp-Source: ABdhPJzX72bxMhcDvNcWPTP2qi2JplW7H194IQ06bcsBBd+nQOGAksif2c8MMb2kKzWHXo2aMpWlfg== X-Received: by 2002:a17:90a:4d8d:: with SMTP id m13mr1883197pjh.190.1631623095243; Tue, 14 Sep 2021 05:38:15 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id mi18sm1893661pjb.15.2021.09.14.05.38.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:15 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 06/10] bpf: Bump MAX_BPF_STACK size to 768 bytes Date: Tue, 14 Sep 2021 18:07:45 +0530 Message-Id: <20210914123750.460750-7-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=912; h=from:subject; bh=B19ILwR6K3rnkHTsjQYwJPYCeDgJJfCXMOYYCnC9Rck=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdW3LKRLbzT7lxBhQPvK4fFFNC2ILXCx90krbkD YYg+LBWJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVgAKCRBM4MiGSL8RymO/D/ 96CbCCvvSNET6IxUjrLbI/OHXI2yo5UeZ9puFOxSgPOyhxx005s/C8bVWtAEllZvdh/htmO7rc+7uq oGEOGHG9DU0XhjcgaMIvbKwQHQ/PDX9DXcigUGbDqND6Ysf98qRMwVKExMaF9GQ795L2nG0JMoGv0j zRjdrl/iB+dh/Bp6eOmBDlgpNb+JyqHRjQqhp5HbCeMr2DKR+yMvxwVlXbWz3sUKlTqXHnwisMqXes Bn16rdhSA9CzBKEAcj1efIy/6S7tqBAW/55lFw4WP4ZLvgH7QtpIBNRQvdFOkHfL7YDXsbQXA1YiZT rNEm6ftxqkjZKEB39U6mAauKLQo0S7Wmmm1JqwBUlr7J7WjfLipxFu/rpbBzsKeqhQ+EPn2Tvxr4cK KvqJC69uVy3+bKQqjcB99oCPsdXZOU0/QINqjODE6qVJv0z5TrsNWcYBlWoDp6OUPNEUMznhsjQRcR Dnti5q2dRrKu1nNzErCgDG0a1cb4MGeCdYJFzumLGXLRiT6MZQaC8OoDPl8hLZeLSZ8Nd8YLA2si+y 878myrCUx7l73qoEXI0uDoQHJaC1UIGDLB6qyEZF4XTZF6PhUH8NorzbydEV0h2OJ4+Zxm4Mid26GG 9Hdmumq5x5u5eb87OZ8n9VV4Qexd2U7aMb+2CC08LiMON2YHxkp7YDYsxk7g== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Increase the maximum stack size accessible to BPF program to 768 bytes. This is done so that gen_loader can use 94 additional fds for kfunc BTFs that it passes in to fd_array from the remaining space available for the loader_stack struct to expand. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/filter.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index 4a93c12543ee..b214189ece62 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -82,8 +82,8 @@ struct ctl_table_header; */ #define BPF_SYM_ELF_TYPE 't' -/* BPF program can access up to 512 bytes of stack space. */ -#define MAX_BPF_STACK 512 +/* BPF program can access up to 768 bytes of stack space. */ +#define MAX_BPF_STACK 768 /* Helper macros for filter block array initializers. */ From patchwork Tue Sep 14 12:37:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493059 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 515D0C433EF for ; Tue, 14 Sep 2021 12:38:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3874A6108B for ; Tue, 14 Sep 2021 12:38:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233027AbhINMjl (ORCPT ); Tue, 14 Sep 2021 08:39:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232957AbhINMjg (ORCPT ); Tue, 14 Sep 2021 08:39:36 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 146DCC061574; Tue, 14 Sep 2021 05:38:19 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id 5so8091113plo.5; Tue, 14 Sep 2021 05:38:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FJfnGeMTrgzS5XEpdbIRzvfzOUWBznUfpOiCHuYJ/KY=; b=qf/VIJ8kfstOIS+6IQ5VCbUaijb4UOJFSXbZw9BYnDAh17hIDtKz4eZ463Vz0nGBpa Gfcv0FtRGR1QuHriFZ4a/fbIH8kDNJ7+cRv3hTE65wEnnxMdVfmvpo/ZoRwlKlWWWBO4 JKOgTXtzhjzYtlhje1UuSxco4ePY0R0lp21z5T8F4Q2PWW25usSK+djSo3q3zeZeOB8b kSM/tetLsEC0LOO4LMyJsISdVE/+QO3JT8KogCW7V2BADCinmNOlyTlCMQPxwVuJqMWg hNQefEnpdQY6VpsFhV/MUidglC2A3PiMKlbXvGLwrhV4yTtAo1+pCAEsns/Kedm4uzPj fgfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FJfnGeMTrgzS5XEpdbIRzvfzOUWBznUfpOiCHuYJ/KY=; b=MgqXmUPEX0WfxPorLAk3EoKhHYcCCs/g1SUPQ50N8TgHznk5ToVQT7LNr1ElK7z/05 NkmAlLrLm4HhwE2Ua9sGYXSJ9WSFCr3Y/xG1kKG9RklXvCzIcRgnflFYhFw+3qpKxgik 4GR4ubFYwWJwrnaquAAyO2+Rh3wYBAXC9vGPP/mmAWg2h4fAiJybUjuaINHsP1XEerEd z1AxdjnXzFbFll6CZwrJAzMdOKEXg083+Yqz0gST/87+3ab+DTd73zvga89Kc8U6hWWj uZtcBX/ADnHBr5TH0bQ8wDhQhvxm37mqMfQsfOCQANJBwS1i3T4XWC1wxp8IDa/IGIlL PDHA== X-Gm-Message-State: AOAM530Mqsgc6/RZGljugjJoIEeADM7FQf8d7012vsjQFvUcIBoKdCE3 VcOugJ463GueyFa18r/TNUFC6NMJAlv5fw== X-Google-Smtp-Source: ABdhPJxAQ752bpmvWdnqro9et2IqWmRyrg+zxzRfXOsdKNwhCTsEyk0aJd7L8ujfMVcuX3oXklbZRQ== X-Received: by 2002:a17:90a:a791:: with SMTP id f17mr1834714pjq.225.1631623098138; Tue, 14 Sep 2021 05:38:18 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id gn11sm1616246pjb.39.2021.09.14.05.38.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:17 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 07/10] libbpf: Support kernel module function calls Date: Tue, 14 Sep 2021 18:07:46 +0530 Message-Id: <20210914123750.460750-8-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4871; h=from:subject; bh=/QTomyMTxP3ElJBV9UogAOv6leVi7U3issFhKkwHUR4=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdW/clEWXeJqcK/WMVktGAjIwiMWAZqbaqmetId 8mYU4kGJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVgAKCRBM4MiGSL8RyvpDEA CG5za7L8COw6VTi3JnFkspQoUWy9G1KWH6FEadoWwcixNdlKk9aX6pNBqngQVLL5tvUAF6fberk8Lh JyrTMXkj8UUHVSjwpSIYVskE8SVK8SYWPxnPZWhNI7+AMG34AHpQb27GTeqU5cvz2NDL76GvZwvEel KAJoyJ6oj6Ebkl4iXzZZs/hOYXZl+cAM/sRJHmwhBEp6yelfaa+4G8j830WaUxIGtm8SGCAvrTCFYw A+pr1XX+etCcTgmQk2RsvHgRlZO9eW3u+3NP4bzCUtQJuEVd9zPhC0RG3aKeGjxZsbsaaJ1VyxzVUZ 6HI7KgfmBBfWGSzJykxk/I5qvLQpz5MH6Eb+q6+L6h21c4ADjmUNXtreK5FX8kIeP3lPKpYsb1pnoC qpzxSlmCWZVYrI/JWbXr/b+8J/hlrCm2xpTeJ1UPtfnM7Iz6leBemxhSgsOAlfPj2AGH5H+AQXUW7p 6lHQEC780Gtz6gUoGiL/lTX61aKbpAvHlVWPyMd1wf1MoWw0HBtOdb3svhZCehOGbAtZnx6RLswx8o pf90UUWlJPG2f60yD5FXapZfMC3OTAiFmPSKJmBX6bOGokrwaTm+ZLEwUNpvj2Ns04ZN0J3wc11JIV vUvwe9PpywArlUDKZ5qKu3XI/McPKDOAcirqZeVlckcyemSs6KGI1qdLgebw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch adds libbpf support for kernel module function call support. The fd_array parameter is used during BPF program load is used to pass module BTFs referenced by the program. insn->off is set to index into this array + 1, because insn->off as 0 is reserved for btf_vmlinux. The kernel subtracts 1 from insn->off when indexing into env->fd_array. We try to use existing insn->off for a module, since the kernel limits the maximum distinct module BTFs for kfuncs to 256, and also because index must never exceed the maximum allowed value that can fit in insn->off (INT16_MAX). In the future, if kernel interprets signed offset as unsigned for kfunc calls, this limit can be increased to UINT16_MAX. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 56 +++++++++++++++++++++++++++++++-- tools/lib/bpf/libbpf_internal.h | 1 + 3 files changed, 55 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2401fad090c5..7d1741ceaa32 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -264,6 +264,7 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr) attr.line_info_rec_size = load_attr->line_info_rec_size; attr.line_info_cnt = load_attr->line_info_cnt; attr.line_info = ptr_to_u64(load_attr->line_info); + attr.fd_array = ptr_to_u64(load_attr->fd_array); if (load_attr->name) memcpy(attr.prog_name, load_attr->name, diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 6ecfdc1fa7ba..e78ae57e4379 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -419,6 +419,12 @@ struct extern_desc { /* local btf_id of the ksym extern's type. */ __u32 type_id; + /* offset to be patched in for insn->off, + * this is 0 for btf_vmlinux, and index + 1 + * for module BTF, where index is BTF index in + * obj->fd_array + */ + __s16 offset; } ksym; }; }; @@ -515,6 +521,10 @@ struct bpf_object { void *priv; bpf_object_clear_priv_t clear_priv; + int *fd_array; + size_t fd_cap_cnt; + int nr_fds; + char path[]; }; #define obj_elf_valid(o) ((o)->efile.elf) @@ -5333,6 +5343,7 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6127,6 +6138,7 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt, } load_attr.log_level = prog->log_level; load_attr.prog_flags = prog->prog_flags; + load_attr.fd_array = prog->obj->fd_array; if (prog->obj->gen_loader) { bpf_gen__prog_load(prog->obj->gen_loader, &load_attr, @@ -6729,9 +6741,44 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, } if (kern_btf != obj->btf_vmlinux) { - pr_warn("extern (func ksym) '%s': function in kernel module is not supported\n", - ext->name); - return -ENOTSUP; + int index = -1; + + if (!obj->fd_array) { + obj->fd_array = calloc(8, sizeof(*obj->fd_array)); + if (!obj->fd_array) + return -ENOMEM; + obj->fd_cap_cnt = 8; + } + + for (int i = 0; i < obj->nr_fds; i++) { + if (obj->fd_array[i] == kern_btf_fd) { + index = i; + break; + } + } + + if (index == -1) { + if (obj->nr_fds == obj->fd_cap_cnt) { + ret = libbpf_ensure_mem((void **)&obj->fd_array, + &obj->fd_cap_cnt, sizeof(int), + obj->fd_cap_cnt + 1); + if (ret) + return ret; + } + + index = obj->nr_fds; + obj->fd_array[obj->nr_fds++] = kern_btf_fd; + } + + if (index >= INT16_MAX) { + /* insn->off is s16 */ + pr_warn("extern (func ksym) '%s': module btf fd index too big\n", + ext->name); + return -E2BIG; + } + ext->ksym.offset = index + 1; + } else { + ext->ksym.offset = 0; } kern_func = btf__type_by_id(kern_btf, kfunc_id); @@ -6907,6 +6954,9 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr) err = bpf_gen__finish(obj->gen_loader); } + /* clean up fd_array */ + zfree(&obj->fd_array); + /* clean up module BTFs */ for (i = 0; i < obj->btf_module_cnt; i++) { close(obj->btf_modules[i].fd); diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index 4f6ff5c23695..4a948318aa89 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -289,6 +289,7 @@ struct bpf_prog_load_params { __u32 log_level; char *log_buf; size_t log_buf_sz; + int *fd_array; }; int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr); From patchwork Tue Sep 14 12:37:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493061 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2031C433FE for ; Tue, 14 Sep 2021 12:38:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C90A560F92 for ; Tue, 14 Sep 2021 12:38:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233050AbhINMjl (ORCPT ); Tue, 14 Sep 2021 08:39:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232975AbhINMji (ORCPT ); Tue, 14 Sep 2021 08:39:38 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD385C061760; Tue, 14 Sep 2021 05:38:21 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id j6so11762104pfa.4; Tue, 14 Sep 2021 05:38:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/JqEGDgo9QAxQWCfqMkHbW50F2FbnzIMeTf5fXEX65o=; b=cU4F/xWic8PF946vqRjKPyxM6TbXGuPC5c0bsKmABGoNTfulMsAxQwKBSVXn6FPOUP b+5FCHtZ+8AtZvHLrX7BxFPG9XbzpUQDwVf6h5o978wOUE7onKP743+7FpyPRe55Bzfc FiT+zvyuy5UuHiL/X3nv86BwFWCEI5bYjq8cqAbDXzmb9SHoPZbHaVDsj4hiJqJReNLS p/AaQYUFnHM5f5FGgsisk7h8tzxkpQe2Af1yZLI3mHe0iafLl7JVVs+V1cJbUfSIU8g3 NH6emvIHG2uggzETLaD4XwnHzkbxYb/fK1mT/Pw1xY+6vq8nhyOw847kJ7ykZqEKhy4r 0s+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/JqEGDgo9QAxQWCfqMkHbW50F2FbnzIMeTf5fXEX65o=; b=TLaRET8p5ms0U/tzcQ8eVutUrV92ZejKQM2hCbVeTRTHA6DrjuoPNThFwOqeC3oqh3 d8z1mulE4oRSxHbGO/PC0M1dnAdSfJ685Fx4jmcZTmv2NOfJv3O+xIpvWdw1NHeFwNMj 3xZT/EAUfEVqgkqihmEe2oPKynEU21vylFmt8Lxsi3KdmVLYVyRVTyGeZLC4MUlilfvg YbIBVFvQC6GLc/zn59s8C9CwyGIMcmaGfQ5WNYMiUgQH4Z8daalBBA8clumwsuM2+cMy 1rh6cYZINVAxEdsEnU44JZfXQkhyL+7y8N6ymoEH2qCVIimThe4HrH3m5YJ9KMEhGOf/ xvVQ== X-Gm-Message-State: AOAM532vS//KeIQB3Q5VgJBbGTNrCXvj5txvPJiSBqvSior3qH9Ml8Lw Megnd66pGySV2w2tgtG/56AabNb8V3TbUw== X-Google-Smtp-Source: ABdhPJw0JAfegqXd6ciYVqKN5WCSQ3ZhvICuxuhAL8GtdR1B6Y5sRNkW32wDOmu5NddRkKqR3+KzBA== X-Received: by 2002:a65:6107:: with SMTP id z7mr15312305pgu.43.1631623101065; Tue, 14 Sep 2021 05:38:21 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id i5sm1564973pjk.47.2021.09.14.05.38.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:20 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 08/10] libbpf: Resolve invalid weak kfunc calls with imm = 0, off = 0 Date: Tue, 14 Sep 2021 18:07:47 +0530 Message-Id: <20210914123750.460750-9-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2121; h=from:subject; bh=rZU9k+Cv2J6k/VY3eXGkvA4+HCz67lZYgXlspqSObDQ=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdXdG1+CGM1VJ/ZpXQLhLzk3deoejsZNIRvDpvc sDVlYD6JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVwAKCRBM4MiGSL8Ryt/jD/ 0bvzwt+9hV+MGlLUqQbZJ+/1FR4JWodIXgRfTE8Go/KBh7YNd7grTUqnMd/7FmzArKpJMJOYMmrsLx 7p6f+VVZdYV4pnir1PPB4Qdz3nTAehgtZ5mpPcuhee7EtxuvV3lGmlcabEx+oUr7B2I8VRM8nFLmwR 3mOgXcjkUU0pvXfhkywS+arCKYO7ybMoaE8wPNr3m/DTRzVWcpJfXfR4EIgfuAPLOqlWE8toWgXnCn FwwPi0DQjipa/mNnYOMsF7TImTPOMv6ZfDP4YygDYYU65dsbJOsRqGUwo9r2/1E/HhvGMxVvdNLhKO y5DCZrP9pESwVJh0YO1b1UqPfcm+KNVqOPEo933MwHTSw+lFZHcZ3yfhY2na5L+H727b2pBdCbBqhp VvZMIQf8FgGpUkdJIeJWh1ZXmg0/P1bSLoFpJcuQydqr+HfqEO1B2tsxmS444psfuqcLVfRwTNnWc8 OgQd+zQOzXNi4l8/V7qUy32qg0sp9ydfHl55tTmcEti8D2w36meGOgGBKNgC3MXWNw3Ln6Em0+tApB bqFLHj4oza5ODIrTXY/Ah3XBOCr7RmruSAYy9mLzSKRoGzGcp0ypFai9ouueFaxs30X/bUE07uB1AE 92Cwu15rBtHjVzYQE772OE6enruCH2zC8HvDpuW46Ctgjd1gcfke9UCZP4IA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Preserve these calls as it allows verifier to succeed in loading the program if they are determined to be unreachable after dead code elimination during program load. If not, the verifier will fail at runtime. This is done for ext->is_weak symbols similar to the case for variable ksyms. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/libbpf.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index e78ae57e4379..9c6c1aa73e35 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -3409,11 +3409,6 @@ static int bpf_object__collect_externs(struct bpf_object *obj) return -ENOTSUP; } } else if (strcmp(sec_name, KSYMS_SEC) == 0) { - if (btf_is_func(t) && ext->is_weak) { - pr_warn("extern weak function %s is unsupported\n", - ext->name); - return -ENOTSUP; - } ksym_sec = sec; ext->type = EXT_KSYM; skip_mods_and_typedefs(obj->btf, t->type, @@ -5342,8 +5337,12 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) case RELO_EXTERN_FUNC: ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; - insn[0].imm = ext->ksym.kernel_btf_id; - insn[0].off = ext->ksym.offset; + if (ext->is_set) { + insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; + } else { /* unresolved weak kfunc */ + insn[0].imm = insn[0].off = 0; + } break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6734,8 +6733,10 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, kfunc_id = find_ksym_btf_id(obj, ext->name, BTF_KIND_FUNC, &kern_btf, &kern_btf_fd); - if (kfunc_id < 0) { - pr_warn("extern (func ksym) '%s': not found in kernel BTF\n", + if (kfunc_id == -ESRCH && ext->is_weak) { + return 0; + } else if (kfunc_id < 0) { + pr_warn("extern (func ksym) '%s': not found in kernel or module BTFs\n", ext->name); return kfunc_id; } From patchwork Tue Sep 14 12:37:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493063 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4CB3C43217 for ; Tue, 14 Sep 2021 12:38:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CFDCD60FED for ; Tue, 14 Sep 2021 12:38:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233009AbhINMjn (ORCPT ); Tue, 14 Sep 2021 08:39:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233076AbhINMjm (ORCPT ); Tue, 14 Sep 2021 08:39:42 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C22BFC061764; Tue, 14 Sep 2021 05:38:24 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id d18so8079665pll.11; Tue, 14 Sep 2021 05:38:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tS94kaQlG2vX4CT1TvfjfE1uI4gq2voa3SQwROtG5wc=; b=b5G9hIirCbdJ/3Zpq7fldi8IQEFjpKdtzYOqyGd1681B/uUzJ4SxuxosuJbxb610Iw XMyQDSQV/Rin+jMbcY6Vg7873V6rqorory1LDA5eSs0v3GMxrHR23WBjyNhdeRngmGXD b1GZs4T/ncoQ+XGZCU4oT65y2k81/LuNIYtYYQZEuTtR6kDZB4DOSSUQtagq123gXAEh fFJwX7n1DV9ZgqVlncWbuMj3DC4eLD87fBY8b8Dv4aJeI1UCdFWRy21JRRqE3sxp7H0a 1FfYQi3YzeV7Pis0zztKfOENO7l85wnKl1uQC5qYDaSNC2TK4+Dwj7s+y0TWPw8EpCsw zfCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tS94kaQlG2vX4CT1TvfjfE1uI4gq2voa3SQwROtG5wc=; b=0UT8oITjuG3u1diiyDLWyYNmQYFrYSn2A3NdBo9/X6YU6xWy138vFKogVdLLGSCxMS mJkf2TlH7kxUXEZ9hEkqMa/fQfjpxDQGWI8DUOCKdGylrBThRpbFPY9FRJnehBlrVBsB ZRD4jOJDxcKdbU9kZSJa64wUT5mPpwjCkna00pXPPQ6Qp6+yBoMJhvzswH7RMrvknbW3 xZ8+TKu3AB+438cmOcxbXgwwX5T6e7dEdCCVVRwvqgq1O0G6oedXWSVloSzgctlqzrI5 OlDUSR2kyNUUM/JAN2u9iuFXZrIkZb1WnPOlBdhLVLe2jDOwchdJxhqgax4HMgNp6Mca jDXQ== X-Gm-Message-State: AOAM531XgnvUr8NeCn3RYaKn/PJ8/thVzwEaD+lPSVCTtsdMtBrhx6JC 6Se4Zdq4Izow/j0pTvh7g1EQcKawJUVI/w== X-Google-Smtp-Source: ABdhPJxKYLisnvSRl6u5hpUvWHqXyxbBghOJym7Mid+DLuD8BDg9mTlme4CUaZJ1QL231Lo/QjAGfg== X-Received: by 2002:a17:902:cec8:b0:13b:9ce1:b3ef with SMTP id d8-20020a170902cec800b0013b9ce1b3efmr6281355plg.4.1631623104042; Tue, 14 Sep 2021 05:38:24 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id d13sm10306514pfn.114.2021.09.14.05.38.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:23 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 09/10] libbpf: Update gen_loader to emit BTF_KIND_FUNC relocations Date: Tue, 14 Sep 2021 18:07:48 +0530 Message-Id: <20210914123750.460750-10-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=8801; h=from:subject; bh=DPIbJoKg4frzr51LEaLuHgculBnpynYjSGmMs82hyAI=; b=owEBbAKT/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdXfipYss2uGEW+VK8l2VJWU0BAyvrpo1hpu3bK yLqvJQiJAjIEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVwAKCRBM4MiGSL8RyjT9D/ imFI5HeSOYi2Wf0l5TXvwG1TWloofB9zuTbZyDeoFNk1kluMGiZ1/CK1Mh11yzxkYjalCO+SB5NiII EpufBvkHTo9+FnviN7qAchxXYHLpEUzRl6NxVaQBA6cnR1I0z3pTircCE18TcXTRqQgZtdQLLrhIML uq3X1ji6q4wrGgf7tC63w2+a/I15I6MYZ0zv1LllpKRl/nhfxMs7yGVxqpDNdfpvB7u5lJe1atHjUC rVChKlv5pH78gi7HXywy+4h6Q70vkkdSJa6CDPn9lAC34gEhHMqFfWtiUrxNq6EXwpxOGaAnTc6LAN wXr8So4qX/4pX7oqe7ltRPxHe5RxIFtmDtrMjvZWC6gXN43es39/GHFBoIUkyfzMueIAAGY96woslp KP1hZTNVY4EUlzAlHbH/4fUdB7cLWMUNaFdcSM7JPkOmetyZSf1OIQuxfzS8MtO1cQtFmUIv9Co4pt 3U05+w0uYWbBy2s9y/lRUN2YdZUZw3IlpJZx1lgLG9NpnVY2Wvz0TycPu7bt+HHZBkSp7Rmx2fdViQ jq+hxlb4endsSzpj2jqS9IleTJZwaouHhgwXV0f043E5OxgIFekFwiu4fGu/WymeZYloqIn3Rae2+O LBmHF/RZkO/R99ohm5tMxW2PCJnePlGm/o/M1CFajUptEDHGV7OTYu9sAx X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This change updates the BPF syscall loader to relocate BTF_KIND_FUNC relocations, with support for weak kfunc relocations. One of the disadvantages of gen_loader is that due to stack size limitation, BTF fd array size is clamped to a smaller limit than what the kernel allows. Also, finding an existing BTF fd's slot is not trivial, because that would require to open all module BTFs and match on the open fds (like we do for libbpf), so we do the next best thing: deduplicate slots for the same symbol. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf_gen_internal.h | 12 ++++- tools/lib/bpf/gen_loader.c | 93 ++++++++++++++++++++++++++++---- tools/lib/bpf/libbpf.c | 8 +-- 3 files changed, 99 insertions(+), 14 deletions(-) diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_internal.h index 615400391e57..4826adf18d7b 100644 --- a/tools/lib/bpf/bpf_gen_internal.h +++ b/tools/lib/bpf/bpf_gen_internal.h @@ -7,8 +7,15 @@ struct ksym_relo_desc { const char *name; int kind; int insn_idx; + bool is_weak; }; +struct kfunc_desc { + const char *name; + int index; +}; + +#define MAX_KFUNC_DESCS 94 struct bpf_gen { struct gen_loader_opts *opts; void *data_start; @@ -24,6 +31,8 @@ struct bpf_gen { int relo_cnt; char attach_target[128]; int attach_kind; + struct kfunc_desc kfunc_descs[MAX_KFUNC_DESCS]; + __u32 nr_kfuncs; }; void bpf_gen__init(struct bpf_gen *gen, int log_level); @@ -36,6 +45,7 @@ void bpf_gen__prog_load(struct bpf_gen *gen, struct bpf_prog_load_params *load_a void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *value, __u32 value_size); void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx); void bpf_gen__record_attach_target(struct bpf_gen *gen, const char *name, enum bpf_attach_type type); -void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, int insn_idx); +void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak, int kind, + int insn_idx); #endif diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 8df718a6b142..5e8c15e36c46 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "btf.h" #include "bpf.h" #include "libbpf.h" @@ -13,6 +14,7 @@ #include "bpf_gen_internal.h" #include "skel_internal.h" +/* MAX_BPF_STACK is 768 bytes, so (64 + 32 + 94 (MAX_KFUNC_DESCS) + 2) * 4 */ #define MAX_USED_MAPS 64 #define MAX_USED_PROGS 32 @@ -31,6 +33,8 @@ struct loader_stack { __u32 btf_fd; __u32 map_fd[MAX_USED_MAPS]; __u32 prog_fd[MAX_USED_PROGS]; + /* Update insn->off store when reordering kfunc_btf_fd */ + __u32 kfunc_btf_fd[MAX_KFUNC_DESCS]; __u32 inner_map_fd; }; @@ -506,8 +510,8 @@ static void emit_find_attach_target(struct bpf_gen *gen) */ } -void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, - int insn_idx) +void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak, + int kind, int insn_idx) { struct ksym_relo_desc *relo; @@ -519,14 +523,39 @@ void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, gen->relos = relo; relo += gen->relo_cnt; relo->name = name; + relo->is_weak = is_weak; relo->kind = kind; relo->insn_idx = insn_idx; gen->relo_cnt++; } +static struct kfunc_desc *find_kfunc_desc(struct bpf_gen *gen, const char *name) +{ + /* Try to reuse BTF fd index for repeating symbol */ + for (int i = 0; i < gen->nr_kfuncs; i++) { + if (!strcmp(gen->kfunc_descs[i].name, name)) + return &gen->kfunc_descs[i]; + } + return NULL; +} + +static struct kfunc_desc *add_kfunc_desc(struct bpf_gen *gen, const char *name) +{ + struct kfunc_desc *kdesc; + + if (gen->nr_kfuncs == ARRAY_SIZE(gen->kfunc_descs)) + return NULL; + kdesc = &gen->kfunc_descs[gen->nr_kfuncs]; + kdesc->name = name; + kdesc->index = gen->nr_kfuncs++; + return kdesc; +} + static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insns) { int name, insn, len = strlen(relo->name) + 1; + int off = MAX_USED_MAPS + MAX_USED_PROGS; + struct kfunc_desc *kdesc; pr_debug("gen: emit_relo: %s at %d\n", relo->name, relo->insn_idx); name = add_data(gen, relo->name, len); @@ -539,18 +568,64 @@ static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insn emit(gen, BPF_EMIT_CALL(BPF_FUNC_btf_find_by_name_kind)); emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_0)); debug_ret(gen, "find_by_name_kind(%s,%d)", relo->name, relo->kind); - emit_check_err(gen); + /* if not weak kfunc, emit err check */ + if (relo->kind != BTF_KIND_FUNC || !relo->is_weak) + emit_check_err(gen); + insn = insns + sizeof(struct bpf_insn) * relo->insn_idx; + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, 0, 0, 0, insn)); + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_0, offsetof(struct bpf_insn, imm), 0)); + /* skip success case store if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 1)); /* store btf_id into insn[insn_idx].imm */ - insn = insns + sizeof(struct bpf_insn) * relo->insn_idx + - offsetof(struct bpf_insn, imm); - emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, - 0, 0, 0, insn)); - emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, 0)); + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, offsetof(struct bpf_insn, imm))); if (relo->kind == BTF_KIND_VAR) { /* store btf_obj_fd into insn[insn_idx + 1].imm */ emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32)); emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, - sizeof(struct bpf_insn))); + sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm))); + } else if (relo->kind == BTF_KIND_FUNC) { + kdesc = find_kfunc_desc(gen, relo->name); + if (!kdesc) + kdesc = add_kfunc_desc(gen, relo->name); + if (kdesc) { + /* store btf_obj_fd in index in kfunc_btf_fd array + * but skip storing fd if ret < 0 + */ + emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_10, + stack_off(kfunc_btf_fd[kdesc->index]), 0)); + emit(gen, BPF_MOV64_REG(BPF_REG_8, BPF_REG_7)); + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 4)); + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32)); + /* if vmlinux BTF, skip store */ + emit(gen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0, 1)); + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_7, + stack_off(kfunc_btf_fd[kdesc->index]))); + emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_8)); + /* remember BTF obj fd */ + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_8, 32)); + } else { + pr_warn("Out of BTF fd slots (total: %u), skipping for %s\n", + gen->nr_kfuncs, relo->name); + emit(gen, BPF_MOV64_REG(BPF_REG_1, BPF_REG_7)); + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_1, 32)); + __emit_sys_close(gen); + } + /* store index + 1 into insn[insn_idx].off */ + off = kdesc ? off + kdesc->index + 1 : 0; + off = off > INT16_MAX ? 0 : off; + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_0, offsetof(struct bpf_insn, off), 0)); + /* skip success case store if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 2)); + /* skip if vmlinux BTF */ + emit(gen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_8, 0, 1)); + /* store offset */ + emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_0, offsetof(struct bpf_insn, off), off)); + /* log relocation */ + emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, offsetof(struct bpf_insn, imm))); + emit(gen, BPF_LDX_MEM(BPF_H, BPF_REG_8, BPF_REG_0, offsetof(struct bpf_insn, off))); + debug_regs(gen, BPF_REG_7, BPF_REG_8, "sym (%s): imm: %%d, off: %%d", relo->name); } } diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 9c6c1aa73e35..6a100c2e7d1c 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6240,12 +6240,12 @@ static int bpf_program__record_externs(struct bpf_program *prog) ext->name); return -ENOTSUP; } - bpf_gen__record_extern(obj->gen_loader, ext->name, BTF_KIND_VAR, - relo->insn_idx); + bpf_gen__record_extern(obj->gen_loader, ext->name, ext->is_weak, + BTF_KIND_VAR, relo->insn_idx); break; case RELO_EXTERN_FUNC: - bpf_gen__record_extern(obj->gen_loader, ext->name, BTF_KIND_FUNC, - relo->insn_idx); + bpf_gen__record_extern(obj->gen_loader, ext->name, ext->is_weak, + BTF_KIND_FUNC, relo->insn_idx); break; default: continue; From patchwork Tue Sep 14 12:37:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12493065 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 130AAC433EF for ; Tue, 14 Sep 2021 12:38:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA8666108B for ; Tue, 14 Sep 2021 12:38:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232937AbhINMjp (ORCPT ); Tue, 14 Sep 2021 08:39:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233056AbhINMjp (ORCPT ); Tue, 14 Sep 2021 08:39:45 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0756C061760; Tue, 14 Sep 2021 05:38:27 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id j6so11762394pfa.4; Tue, 14 Sep 2021 05:38:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=48Nh6eT11E5btG6zkXtV4a+WFM1wP5G2R8aeIs0Yjjs=; b=ldLvGhRvda89SCmdEUT5mSHjOGH0VAZRFEqd59ZyZ6vYrz8Ky47NSI96LQQe7X4dxq XEgd6xZo1Oe+hFPGZdW6DqB3V/2uQI2oZvdhk+5DUb3DljuJSK4wGSWBbJEZSx1ui1YD uENqq5JbSTmPcntHPvRG5TwbuTbzdS/NkEZf6p9dAISh8uVYUvjmIZ5Jj8+aFfPtRbK6 g4JX7x3l6kcwDU2bOTHoZP9oyrS92ZK64pCQBzYhaAAO1WcvnlncBhLN6vqnzQ7nYkIw v/q5IoozUGAXhC55pByFMBDrSNYn0JMLsCFOXiv+5sHpuMM0hfdI5AzP2dKyu4mbC1zj bmnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=48Nh6eT11E5btG6zkXtV4a+WFM1wP5G2R8aeIs0Yjjs=; b=gNraCi7myqA8NYGgAPQKI6sT46MjHWjTadG+CDIk84E0j5x9AzzX9OjsuTg1UyE4qS F8Kxa2s5B1knvnlZVxSQ/uA3QsOtOL7LiTfBuuTQIipa9Q0vfvEwniuGUuFQgIEXMRCj EuBsJtQzTCw5BkRfb5yEYQvdjuu5+JB6K/H1dFy/cXnnf1cYhN9VwnZHTeAHDLrIsS// jB0f+UtPyiTH3mpyFRL3kVEaJYIJBPWGaDFl3Axp8CWv1lBwHnuyMEN8XK30cx9IVbPt n3zxyy/SvEggmj1qjtzyfn6QiU14eiPclXMgeRuEEaPGhi2OcsgHe9tB7HxtHOE6MFLs vRug== X-Gm-Message-State: AOAM532rMPxljALSokFsyMiXftFjcjvvFI95Gpk5nJ4qbg3ENGosVQJT VKP3qVLsKIViwQVzlYkxsDu6+gUzzzKukg== X-Google-Smtp-Source: ABdhPJwEsBFRRJ8p62mLdI6wSuI2cOCpiarUNpR4xyWMV6NKCwiSb7y3Y9iVqNVXjT7Mo2zzbsurQw== X-Received: by 2002:a63:9752:: with SMTP id d18mr15199853pgo.320.1631623107051; Tue, 14 Sep 2021 05:38:27 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id b4sm11785925pga.69.2021.09.14.05.38.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 05:38:26 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v2 10/10] bpf, selftests: Add basic test for module kfunc call Date: Tue, 14 Sep 2021 18:07:49 +0530 Message-Id: <20210914123750.460750-11-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210914123750.460750-1-memxor@gmail.com> References: <20210914123750.460750-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=14147; h=from:subject; bh=foLr6v1Cv+mwvK44wwwM0rB5kpCMmwWdQ0x5W3e5oxU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhQJdXcLl9WqVT7a7vu5Wl9Ld5Q2TGGAps3IQc6DOL flbJgjOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYUCXVwAKCRBM4MiGSL8Ryl8HD/ 9O5kyf+nskJqSGE8yH1mT5N2G4j/LODI/F6VjDcO0HfxOZww7pBRFHsyJvi8qM1l05NNynYwalIbku DnCCXe7/VXpTJv/jKZdL+cBiLAYUutizU7+L4/2yAQpS3MJ2QpFQXWse90Nzi81KUDYryXiOoXssLy A/hziB6OMF7KoiuZiPWE/LGMRcboM/jE1zvJVNm2VnMofTTnjHiBbpf0rKXDshJEJK2OSKkt7vbzWs y9QtXltklWsYlpWEjhqf1o2PgIOc5ZZvJGJOA6EoRqDw8VPjjQ98t9yqNPbYs/DE7us2vQWWbdYss4 tr8lnZD3F06N59H4284rhxzjB56hjPTZQ3dae60T50ihWhAAkemFMhuzoFyBM5Ya3qFY20mz7a7lxc hgRm6sMjICr0Muv4Pbcaso92vdddrTx4ewZhruiv2+3b6SvVjiABVQ1kV9U491iSYyemrIv+TsmloB /UnyL2lB5mjCJbLgB0APvFk4J5ao01eXv4iRTRT52sQKMQMRUIT9tWxCiB1fMkwn03/UgTx3r/Cou8 fIUese3fkoLj8Afu1Dz+AgjDM8txHI+NTY/mh2awfNgkFm90tSHUEDJvlpsScgE5Tu0TxsZCbv4kdI uNeE3KBjh71G+dwJlWs6nsMbZwgSStyZVvNczZLajBuURBhH1vAmnyTI2eig== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This also tests support for invalid kfunc calls we added in prior changes, such that verifier handles invalid call as long as it is removed by code elimination pass (before fixup_kfunc_call). A separate test for libbpf is added, which tests failure in loading. Also adjust verifier selftests which assume 512 byte stack to now assume 768 byte stack. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 2 ++ kernel/bpf/btf.c | 2 ++ kernel/trace/bpf_trace.c | 1 + tools/testing/selftests/bpf/Makefile | 1 + .../selftests/bpf/bpf_testmod/bpf_testmod.c | 23 +++++++++++- .../selftests/bpf/prog_tests/ksyms_module.c | 13 ++++--- .../bpf/prog_tests/ksyms_module_libbpf.c | 18 ++++++++++ .../selftests/bpf/progs/test_ksyms_module.c | 9 +++++ .../bpf/progs/test_ksyms_module_libbpf.c | 35 +++++++++++++++++++ tools/testing/selftests/bpf/verifier/calls.c | 22 ++++++------ .../selftests/bpf/verifier/raw_stack.c | 4 +-- .../selftests/bpf/verifier/stack_ptr.c | 6 ++-- .../testing/selftests/bpf/verifier/var_off.c | 4 +-- 13 files changed, 116 insertions(+), 24 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c create mode 100644 tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c diff --git a/include/linux/btf.h b/include/linux/btf.h index c7b6382123e1..585c66aa0529 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -271,7 +271,9 @@ static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, THIS_MODULE } extern struct kfunc_btf_id_list bpf_tcp_ca_kfunc_list; +extern struct kfunc_btf_id_list raw_tp_kfunc_list; DECLARE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca); +DECLARE_CHECK_KFUNC_CALLBACK(raw_tp); #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 8240478b7398..06b21c4c50e5 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6269,3 +6269,5 @@ EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list); DEFINE_CHECK_KFUNC_CALLBACK(bpf_tcp_ca, bpf_tcp_ca_kfunc_list); +DEFINE_KFUNC_BTF_ID_LIST(raw_tp_kfunc_list); +DEFINE_CHECK_KFUNC_CALLBACK(raw_tp, raw_tp_kfunc_list); diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 067e88c3d2ee..54cba3391f35 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1629,6 +1629,7 @@ int __weak bpf_prog_test_run_tracing(struct bpf_prog *prog, const struct bpf_verifier_ops raw_tracepoint_verifier_ops = { .get_func_proto = raw_tp_prog_func_proto, .is_valid_access = raw_tp_prog_is_valid_access, + .check_kfunc_call = __bpf_check_raw_tp_kfunc_call, }; const struct bpf_prog_ops raw_tracepoint_prog_ops = { diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 1a4d30ff3275..064eef69e96a 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -174,6 +174,7 @@ $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes $(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation $(Q)$(MAKE) $(submake_extras) -C bpf_testmod $(Q)cp bpf_testmod/bpf_testmod.ko $@ + $(Q)$(RESOLVE_BTFIDS) -s ../../../../vmlinux bpf_testmod.ko $(OUTPUT)/test_stub.o: test_stub.c $(BPFOBJ) $(call msg,CC,,$@) diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 50fc5561110a..5b365a7b3f93 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -1,6 +1,8 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ #include +#include +#include #include #include #include @@ -13,6 +15,12 @@ DEFINE_PER_CPU(int, bpf_testmod_ksym_percpu) = 123; +noinline void +bpf_testmod_test_mod_kfunc(int i) +{ + pr_info("mod kfunc i=%d\n", i); +} + noinline int bpf_testmod_loop_test(int n) { int i, sum = 0; @@ -71,13 +79,26 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = { .write = bpf_testmod_test_write, }; +BTF_SET_START(bpf_testmod_kfunc_ids) +BTF_ID(func, bpf_testmod_test_mod_kfunc) +BTF_SET_END(bpf_testmod_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&bpf_testmod_kfunc_ids, bpf_testmod_kfunc_btf_set); + static int bpf_testmod_init(void) { - return sysfs_create_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); + int ret; + + ret = sysfs_create_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); + if (ret) + return ret; + register_kfunc_btf_id_set(&raw_tp_kfunc_list, &bpf_testmod_kfunc_btf_set); + return 0; } static void bpf_testmod_exit(void) { + unregister_kfunc_btf_id_set(&raw_tp_kfunc_list, &bpf_testmod_kfunc_btf_set); return sysfs_remove_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); } diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_module.c b/tools/testing/selftests/bpf/prog_tests/ksyms_module.c index 2cd5cded543f..7643141ec67b 100644 --- a/tools/testing/selftests/bpf/prog_tests/ksyms_module.c +++ b/tools/testing/selftests/bpf/prog_tests/ksyms_module.c @@ -6,19 +6,22 @@ #include #include "test_ksyms_module.lskel.h" -static int duration; - void test_ksyms_module(void) { - struct test_ksyms_module* skel; + struct test_ksyms_module *skel; int err; + if (!env.has_testmod) { + test__skip(); + return; + } + skel = test_ksyms_module__open_and_load(); - if (CHECK(!skel, "skel_open", "failed to open skeleton\n")) + if (!ASSERT_OK_PTR(skel, "test_ksyms_module__open_and_load")) return; err = test_ksyms_module__attach(skel); - if (CHECK(err, "skel_attach", "skeleton attach failed: %d\n", err)) + if (!ASSERT_OK(err, "test_ksyms_module__attach")) goto cleanup; usleep(1); diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c b/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c new file mode 100644 index 000000000000..61fa2a0e156e --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include "test_ksyms_module_libbpf.skel.h" + +void test_ksyms_module_libbpf(void) +{ + struct test_ksyms_module_libbpf *skel; + + if (!env.has_testmod) { + test__skip(); + return; + } + + skel = test_ksyms_module_libbpf__open_and_load(); + if (!ASSERT_EQ(skel, NULL, "test_ksyms_module__open_and_load")) + test_ksyms_module_libbpf__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module.c b/tools/testing/selftests/bpf/progs/test_ksyms_module.c index d6a0b3086b90..d3fff47791fc 100644 --- a/tools/testing/selftests/bpf/progs/test_ksyms_module.c +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module.c @@ -6,8 +6,11 @@ #include extern const int bpf_testmod_ksym_percpu __ksym; +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; +extern void bpf_testmod_invalid_mod_kfunc(void) __ksym __weak; int out_mod_ksym_global = 0; +const volatile int x = 0; bool triggered = false; SEC("raw_tp/sys_enter") @@ -16,6 +19,12 @@ int handler(const void *ctx) int *val; __u32 cpu; + /* This should be preserved by clang, but DCE'd by verifier, and still + * allow loading the raw_tp prog + */ + if (x) + bpf_testmod_invalid_mod_kfunc(); + bpf_testmod_test_mod_kfunc(42); val = (int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); out_mod_ksym_global = *val; triggered = true; diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c b/tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c new file mode 100644 index 000000000000..52162858d25d --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module_libbpf.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" + +#include + +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; +extern void bpf_testmod_invalid_mod_kfunc(void) __ksym __weak; + +const volatile int x = 0; + +SEC("raw_tp/sys_enter") +int handler_pass(const void *ctx) +{ + /* This should be preserved by clang, but DCE'd by verifier, and still + * allow loading the raw_tp prog + */ + if (x) + bpf_testmod_invalid_mod_kfunc(); + bpf_testmod_test_mod_kfunc(42); + return 0; +} + +SEC("raw_tp/sys_enter") +int handler_fail(const void *ctx) +{ + /* This call should be preserved by clang, but fail verification. + */ + if (!x) + bpf_testmod_invalid_mod_kfunc(); + bpf_testmod_test_mod_kfunc(42); + return 0; +} + +char LICENSE[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/verifier/calls.c b/tools/testing/selftests/bpf/verifier/calls.c index 336a749673d1..03467053996c 100644 --- a/tools/testing/selftests/bpf/verifier/calls.c +++ b/tools/testing/selftests/bpf/verifier/calls.c @@ -750,12 +750,12 @@ "calls: stack overflow using two frames (pre-call access)", .insns = { /* prog 1 */ - BPF_ST_MEM(BPF_B, BPF_REG_10, -300, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -400, 0), BPF_RAW_INSN(BPF_JMP|BPF_CALL, 0, 1, 0, 1), BPF_EXIT_INSN(), /* prog 2 */ - BPF_ST_MEM(BPF_B, BPF_REG_10, -300, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -400, 0), BPF_MOV64_IMM(BPF_REG_0, 0), BPF_EXIT_INSN(), }, @@ -768,11 +768,11 @@ .insns = { /* prog 1 */ BPF_RAW_INSN(BPF_JMP|BPF_CALL, 0, 1, 0, 2), - BPF_ST_MEM(BPF_B, BPF_REG_10, -300, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -400, 0), BPF_EXIT_INSN(), /* prog 2 */ - BPF_ST_MEM(BPF_B, BPF_REG_10, -300, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -400, 0), BPF_MOV64_IMM(BPF_REG_0, 0), BPF_EXIT_INSN(), }, @@ -846,12 +846,12 @@ /* B */ BPF_JMP_IMM(BPF_JGT, BPF_REG_1, 2, 1), BPF_RAW_INSN(BPF_JMP|BPF_CALL, 0, 1, 0, -6), /* call A */ - BPF_ST_MEM(BPF_B, BPF_REG_10, -256, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -512, 0), BPF_EXIT_INSN(), }, .prog_type = BPF_PROG_TYPE_XDP, - /* stack_main=64, stack_A=224, stack_B=256 - * and max(main+A, main+A+B) > 512 + /* stack_main=64, stack_A=224, stack_B=512 + * and max(main+A, main+A+B) > 768 */ .errstr = "combined stack", .result = REJECT, @@ -865,14 +865,14 @@ * } * void func1(int alloc_or_recurse) { * if (alloc_or_recurse) { - * frame_pointer[-300] = 1; + * frame_pointer[-400] = 1; * } else { * func2(alloc_or_recurse); * } * } * void func2(int alloc_or_recurse) { * if (alloc_or_recurse) { - * frame_pointer[-300] = 1; + * frame_pointer[-400] = 1; * } * } */ @@ -888,13 +888,13 @@ BPF_EXIT_INSN(), /* A */ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 2), - BPF_ST_MEM(BPF_B, BPF_REG_10, -300, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -400, 0), BPF_EXIT_INSN(), BPF_RAW_INSN(BPF_JMP|BPF_CALL, 0, 1, 0, 1), /* call B */ BPF_EXIT_INSN(), /* B */ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 1), - BPF_ST_MEM(BPF_B, BPF_REG_10, -300, 0), + BPF_ST_MEM(BPF_B, BPF_REG_10, -400, 0), BPF_EXIT_INSN(), }, .prog_type = BPF_PROG_TYPE_XDP, diff --git a/tools/testing/selftests/bpf/verifier/raw_stack.c b/tools/testing/selftests/bpf/verifier/raw_stack.c index cc8e8c3cdc03..238dedb3aa47 100644 --- a/tools/testing/selftests/bpf/verifier/raw_stack.c +++ b/tools/testing/selftests/bpf/verifier/raw_stack.c @@ -197,7 +197,7 @@ .insns = { BPF_MOV64_IMM(BPF_REG_2, 4), BPF_ALU64_REG(BPF_MOV, BPF_REG_6, BPF_REG_10), - BPF_ALU64_IMM(BPF_ADD, BPF_REG_6, -513), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_6, -769), BPF_MOV64_REG(BPF_REG_3, BPF_REG_6), BPF_MOV64_IMM(BPF_REG_4, 8), BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_skb_load_bytes), @@ -205,7 +205,7 @@ BPF_EXIT_INSN(), }, .result = REJECT, - .errstr = "invalid indirect access to stack R3 off=-513 size=8", + .errstr = "invalid indirect access to stack R3 off=-769 size=8", .prog_type = BPF_PROG_TYPE_SCHED_CLS, }, { diff --git a/tools/testing/selftests/bpf/verifier/stack_ptr.c b/tools/testing/selftests/bpf/verifier/stack_ptr.c index 8ab94d65f3d5..566d79299ccd 100644 --- a/tools/testing/selftests/bpf/verifier/stack_ptr.c +++ b/tools/testing/selftests/bpf/verifier/stack_ptr.c @@ -165,7 +165,7 @@ "PTR_TO_STACK check low 2", .insns = { BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), - BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -513), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -769), BPF_ST_MEM(BPF_B, BPF_REG_1, 1, 42), BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, 1), BPF_EXIT_INSN(), @@ -179,13 +179,13 @@ "PTR_TO_STACK check low 3", .insns = { BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), - BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -513), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -769), BPF_ST_MEM(BPF_B, BPF_REG_1, 0, 42), BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, 0), BPF_EXIT_INSN(), }, .errstr_unpriv = "R1 stack pointer arithmetic goes out of range", - .errstr = "invalid write to stack R1 off=-513 size=1", + .errstr = "invalid write to stack R1 off=-769 size=1", .result = REJECT, }, { diff --git a/tools/testing/selftests/bpf/verifier/var_off.c b/tools/testing/selftests/bpf/verifier/var_off.c index eab1f7f56e2f..407bdee522a6 100644 --- a/tools/testing/selftests/bpf/verifier/var_off.c +++ b/tools/testing/selftests/bpf/verifier/var_off.c @@ -196,8 +196,8 @@ BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0), /* Make it small and 4-byte aligned */ BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 4), - BPF_ALU64_IMM(BPF_SUB, BPF_REG_2, 516), - /* add it to fp. We now have either fp-516 or fp-512, but + BPF_ALU64_IMM(BPF_SUB, BPF_REG_2, 772), + /* add it to fp. We now have either fp-772 or fp-768, but * we don't know which */ BPF_ALU64_REG(BPF_ADD, BPF_REG_2, BPF_REG_10),