| Message ID | 20230317201920.62030-1-alexei.starovoitov@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | bpf: Add detection of kfuncs. | expand |
Hello: This series was applied to bpf/bpf-next.git (master) by Andrii Nakryiko <andrii@kernel.org>: On Fri, 17 Mar 2023 13:19:16 -0700 you wrote: > From: Alexei Starovoitov <ast@kernel.org> > > Allow BPF programs detect at load time whether particular kfunc exists. > > Patch 1: Allow ld_imm64 to point to kfunc in the kernel. > Patch 2: Fix relocation of kfunc in ld_imm64 insn when kfunc is in kernel module. > Patch 3: Introduce bpf_ksym_exists() macro. > Patch 4: selftest. > > [...] Here is the summary with links: - [v2,bpf-next,1/4] bpf: Allow ld_imm64 instruction to point to kfunc. https://git.kernel.org/bpf/bpf-next/c/58aa2afbb1e6 - [v2,bpf-next,2/4] libbpf: Fix relocation of kfunc ksym in ld_imm64 insn. https://git.kernel.org/bpf/bpf-next/c/5fc13ad59b60 - [v2,bpf-next,3/4] libbpf: Introduce bpf_ksym_exists() macro. https://git.kernel.org/bpf/bpf-next/c/5cbd3fe3a91d - [v2,bpf-next,4/4] selftests/bpf: Add test for bpf_ksym_exists(). https://git.kernel.org/bpf/bpf-next/c/95fdf6e313a9 You are awesome, thank you!
Hey Alexei/Andrii, On Fri, Mar 17, 2023 at 01:19:16PM -0700, Alexei Starovoitov wrote: > From: Alexei Starovoitov <ast@kernel.org> > > Allow BPF programs detect at load time whether particular kfunc exists. So, I'm running a GCC built 6.3.7 Linux kernel and I'm attempting to detect whether a specific kfunc i.e. bpf_rcu_read_lock/unlock() exists using the bpf_ksym_exists() macro. However, I'm running into several BPF verifier constraints that I'm not entirely sure how to work around on the aforementioned Linux kernel version, and hence why I'm reaching out for some guidance. The first BPF verifier constraint that I'm running into is that prior to commit 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to kfunc"), it seems that the ld_imm64 instruction with BPF_PSEUDO_BTF_ID can only hold a ksym address for the kind KIND_VAR. However, when attempting to use the kfuncs bpf_rcu_read_lock/unlock() from a BPF program, the kind associated with the BPF_PSEUDO_BTF_ID is actually KIND_FUNC, and therefore trips over this BPF verifier. The code within the example BPF program is along the lines of the following: ``` ... void bpf_rcu_read_lock(void) __ksym __weak; void bpf_rcu_read_unlock(void) __ksym __weak; ... if (bpf_ksym_exists(bpf_rcu_read_lock)) { bpf_rcu_read_lock(); } ... if (bpf_ksym_exists(bpf_rcu_read_unlock)) { bpf_rcu_read_unlock(); } ... ``` The BPF verifier error message that is generated on a 6.3.7 Linux kernel when attempting to load a BPF program that makes use of the above approach is as follows: * "pseudo btf_id {BTF_ID} in ldimm64 isn't KIND_VAR" The second BPF verifier constraint comes from attempting to work around the first BPF verifier constraint that I've mentioned above. This is trivially by dropping the conditionals that contain the bpf_ksym_exists() check and unconditionally calling the kfuncs bpf_rcu_read_lock/unlock(). The code within the example BPF program is along the lines of the following: ``` ... void bpf_rcu_read_lock(void) __ksym __weak; void bpf_rcu_read_unlock(void) __ksym __weak; ... bpf_rcu_read_lock(); ... bpf_rcu_read_unlock(); ... ``` However, in this case the BPF verifier error message that is generated on a 6.3.7 Linux kernel is as follows: * "no vmlinux btf rcu tag support for kfunc bpf_rcu_read_lock" This approach would be suboptimal anyway as the BPF program would fail to load on older Linux kernels complaining that the kfunc is referenced but couldn't be resolved. Having said this, what's the best way to resolve this on a 6.3.7 Linux kernel? The first BPF program I mentioned above making use of the bpf_ksym_exists() macro works on a 6.4 Linux kernel with commit 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to kfunc") applied. Also, the first BPF program I mentioned above works on a 6.1.* Linux kernel... > Patch 1: Allow ld_imm64 to point to kfunc in the kernel. > Patch 2: Fix relocation of kfunc in ld_imm64 insn when kfunc is in kernel module. > Patch 3: Introduce bpf_ksym_exists() macro. > Patch 4: selftest. > > NOTE: detection of kfuncs from light skeleton is not supported yet. > > Alexei Starovoitov (4): > bpf: Allow ld_imm64 instruction to point to kfunc. > libbpf: Fix relocation of kfunc ksym in ld_imm64 insn. > libbpf: Introduce bpf_ksym_exists() macro. > selftests/bpf: Add test for bpf_ksym_exists(). /M
On Tue, Jul 25, 2023 at 1:45 PM Matt Bobrowski <mattbobrowski@google.com> wrote: > > Hey Alexei/Andrii, > > On Fri, Mar 17, 2023 at 01:19:16PM -0700, Alexei Starovoitov wrote: > > From: Alexei Starovoitov <ast@kernel.org> > > > > Allow BPF programs detect at load time whether particular kfunc exists. > > So, I'm running a GCC built 6.3.7 Linux kernel and I'm attempting to > detect whether a specific kfunc i.e. bpf_rcu_read_lock/unlock() exists > using the bpf_ksym_exists() macro. However, I'm running into several > BPF verifier constraints that I'm not entirely sure how to work around > on the aforementioned Linux kernel version, and hence why I'm reaching > out for some guidance. > > The first BPF verifier constraint that I'm running into is that prior > to commit 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to > kfunc"), it seems that the ld_imm64 instruction with BPF_PSEUDO_BTF_ID > can only hold a ksym address for the kind KIND_VAR. However, when > attempting to use the kfuncs bpf_rcu_read_lock/unlock() from a BPF > program, the kind associated with the BPF_PSEUDO_BTF_ID is actually > KIND_FUNC, and therefore trips over this BPF verifier. > > The code within the example BPF program is along the lines of the > following: > ``` > ... > void bpf_rcu_read_lock(void) __ksym __weak; > void bpf_rcu_read_unlock(void) __ksym __weak; > ... > if (bpf_ksym_exists(bpf_rcu_read_lock)) { > bpf_rcu_read_lock(); > } > ... > if (bpf_ksym_exists(bpf_rcu_read_unlock)) { > bpf_rcu_read_unlock(); > } > ... > ``` > > The BPF verifier error message that is generated on a 6.3.7 Linux > kernel when attempting to load a BPF program that makes use of the > above approach is as follows: > * "pseudo btf_id {BTF_ID} in ldimm64 isn't KIND_VAR" > > The second BPF verifier constraint comes from attempting to work > around the first BPF verifier constraint that I've mentioned > above. This is trivially by dropping the conditionals that contain the > bpf_ksym_exists() check and unconditionally calling the kfuncs > bpf_rcu_read_lock/unlock(). > > The code within the example BPF program is along the lines of the > following: > ``` > ... > void bpf_rcu_read_lock(void) __ksym __weak; > void bpf_rcu_read_unlock(void) __ksym __weak; > ... > bpf_rcu_read_lock(); > ... > bpf_rcu_read_unlock(); > ... > ``` > > However, in this case the BPF verifier error message that is generated > on a 6.3.7 Linux kernel is as follows: > * "no vmlinux btf rcu tag support for kfunc bpf_rcu_read_lock" > > This approach would be suboptimal anyway as the BPF program would fail > to load on older Linux kernels complaining that the kfunc is > referenced but couldn't be resolved. > > Having said this, what's the best way to resolve this on a 6.3.7 Linux > kernel? The first BPF program I mentioned above making use of the > bpf_ksym_exists() macro works on a 6.4 Linux kernel with commit > 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to kfunc") > applied. Also, the first BPF program I mentioned above works on a > 6.1.* Linux kernel... Backport of that commit to 6.3.x is probably the only way. And I don't think bpf_ksym_exists() actually works on 6.1
On Tue, Jul 25, 2023 at 02:00:40PM -0700, Alexei Starovoitov wrote: > On Tue, Jul 25, 2023 at 1:45 PM Matt Bobrowski <mattbobrowski@google.com> wrote: > > > > Hey Alexei/Andrii, > > > > On Fri, Mar 17, 2023 at 01:19:16PM -0700, Alexei Starovoitov wrote: > > > From: Alexei Starovoitov <ast@kernel.org> > > > > > > Allow BPF programs detect at load time whether particular kfunc exists. > > > > So, I'm running a GCC built 6.3.7 Linux kernel and I'm attempting to > > detect whether a specific kfunc i.e. bpf_rcu_read_lock/unlock() exists > > using the bpf_ksym_exists() macro. However, I'm running into several > > BPF verifier constraints that I'm not entirely sure how to work around > > on the aforementioned Linux kernel version, and hence why I'm reaching > > out for some guidance. > > > > The first BPF verifier constraint that I'm running into is that prior > > to commit 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to > > kfunc"), it seems that the ld_imm64 instruction with BPF_PSEUDO_BTF_ID > > can only hold a ksym address for the kind KIND_VAR. However, when > > attempting to use the kfuncs bpf_rcu_read_lock/unlock() from a BPF > > program, the kind associated with the BPF_PSEUDO_BTF_ID is actually > > KIND_FUNC, and therefore trips over this BPF verifier. > > > > The code within the example BPF program is along the lines of the > > following: > > ``` > > ... > > void bpf_rcu_read_lock(void) __ksym __weak; > > void bpf_rcu_read_unlock(void) __ksym __weak; > > ... > > if (bpf_ksym_exists(bpf_rcu_read_lock)) { > > bpf_rcu_read_lock(); > > } > > ... > > if (bpf_ksym_exists(bpf_rcu_read_unlock)) { > > bpf_rcu_read_unlock(); > > } > > ... > > ``` > > > > The BPF verifier error message that is generated on a 6.3.7 Linux > > kernel when attempting to load a BPF program that makes use of the > > above approach is as follows: > > * "pseudo btf_id {BTF_ID} in ldimm64 isn't KIND_VAR" > > > > The second BPF verifier constraint comes from attempting to work > > around the first BPF verifier constraint that I've mentioned > > above. This is trivially by dropping the conditionals that contain the > > bpf_ksym_exists() check and unconditionally calling the kfuncs > > bpf_rcu_read_lock/unlock(). > > > > The code within the example BPF program is along the lines of the > > following: > > ``` > > ... > > void bpf_rcu_read_lock(void) __ksym __weak; > > void bpf_rcu_read_unlock(void) __ksym __weak; > > ... > > bpf_rcu_read_lock(); > > ... > > bpf_rcu_read_unlock(); > > ... > > ``` > > > > However, in this case the BPF verifier error message that is generated > > on a 6.3.7 Linux kernel is as follows: > > * "no vmlinux btf rcu tag support for kfunc bpf_rcu_read_lock" > > > > This approach would be suboptimal anyway as the BPF program would fail > > to load on older Linux kernels complaining that the kfunc is > > referenced but couldn't be resolved. > > > > Having said this, what's the best way to resolve this on a 6.3.7 Linux > > kernel? The first BPF program I mentioned above making use of the > > bpf_ksym_exists() macro works on a 6.4 Linux kernel with commit > > 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to kfunc") > > applied. Also, the first BPF program I mentioned above works on a > > 6.1.* Linux kernel... > > Backport of that commit to 6.3.x is probably the only way. Ah, that's very unfortunate. Should we consider sending this patch series to linux-stable so that it can be considered for 6.3.x? /M
From: Alexei Starovoitov <ast@kernel.org> Allow BPF programs detect at load time whether particular kfunc exists. Patch 1: Allow ld_imm64 to point to kfunc in the kernel. Patch 2: Fix relocation of kfunc in ld_imm64 insn when kfunc is in kernel module. Patch 3: Introduce bpf_ksym_exists() macro. Patch 4: selftest. NOTE: detection of kfuncs from light skeleton is not supported yet. Alexei Starovoitov (4): bpf: Allow ld_imm64 instruction to point to kfunc. libbpf: Fix relocation of kfunc ksym in ld_imm64 insn. libbpf: Introduce bpf_ksym_exists() macro. selftests/bpf: Add test for bpf_ksym_exists(). kernel/bpf/verifier.c | 17 ++++++++++------ tools/lib/bpf/bpf_helpers.h | 3 +++ tools/lib/bpf/libbpf.c | 6 ++++++ .../selftests/bpf/progs/task_kfunc_success.c | 20 ++++++++++++++++++- 4 files changed, 39 insertions(+), 7 deletions(-)