| Message ID | 20210930113306.14950-7-quentin@isovalent.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | BPF |
| Headers | show |
| Series | install libbpf headers when using the library | expand |
| Context | Check | Description |
|---|---|---|
| bpf/vmtest-bpf-next | fail | VM_Test |
| bpf/vmtest-bpf-next-PR | fail | PR summary |
| netdev/cover_letter | success | Link |
| netdev/fixes_present | success | Link |
| netdev/patch_count | success | Link |
| netdev/tree_selection | success | Clearly marked for bpf-next |
| netdev/subject_prefix | success | Link |
| netdev/cc_maintainers | warning | 5 maintainers not CCed: kpsingh@kernel.org kafai@fb.com john.fastabend@gmail.com yhs@fb.com songliubraving@fb.com |
| netdev/source_inline | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Link |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/build_32bit | success | Errors and warnings before: 0 this patch: 0 |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/verify_fixes | success | Link |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 33 lines checked |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 0 this patch: 0 |
| netdev/header_inline | success | Link |
2021-09-30 12:33 UTC+0100 ~ Quentin Monnet <quentin@isovalent.com> > API headers from libbpf should not be accessed directly from the > library's source directory. Instead, they should be exported with "make > install_headers". Let's make sure that bpf/preload/iterators/Makefile > installs the headers properly when building. CI complains when trying to build kernel/bpf/preload/iterators/iterators.o. I'll look more into this. Quentin
2021-09-30 13:17 UTC+0100 ~ Quentin Monnet <quentin@isovalent.com> > 2021-09-30 12:33 UTC+0100 ~ Quentin Monnet <quentin@isovalent.com> >> API headers from libbpf should not be accessed directly from the >> library's source directory. Instead, they should be exported with "make >> install_headers". Let's make sure that bpf/preload/iterators/Makefile >> installs the headers properly when building. > > CI complains when trying to build > kernel/bpf/preload/iterators/iterators.o. I'll look more into this. My error was in fact on the previous patch for kernel/preload/Makefile, where iterators.o is handled. The resulting Makefile in my v1 contained: bpf_preload_umd-objs := iterators/iterators.o bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz $(obj)/bpf_preload_umd: $(LIBBPF_A) This declares a dependency on $(LIBBPF_A) for building the final bpf_preload_umd target, when iterators/iterators.o is linked against the libraries. It does not declare the dependency for iterators/iterators.o itself. So when we attempt to build the object file, libbpf has not been compiled yet (not an issue per se), and the API headers from libbpf have not been installed and made available to iterators.o, causing the build to fail. Before this patch, there was no issue because the headers would be included directly from tools/lib/bpf, so they would always be present. I'll fix this by adding the relevant dependency, and send a v2. As a side note, I couldn't reproduce the issue locally or in the VM for the selftests, I'm not sure why. I struggled to get helpful logs from the kernel CI (kernel build in non-verbose mode), so I ended up copying the CI infra (running on kernel-patches/bpf on GitHub) to my own GitHub repository to add debug info and do other runs without re-posting every time to the mailing list. In case anyone else is interested, I figured I might share the steps: - Clone the linux repo on GitHub, push the bpf-next branch - Copy all files and directories from the kernel-patches/vmtest GitHub repo (including the .github directory) to the root of my linux repo, on my development branch. - Update the checks on "kernel-patches/bpf" repository name in .github/workflows/test.yaml, to avoid pulling new Linux sources and overwriting the files on my branch. - (Add as much build debug info as necessary.) - Push the branch to GitHub and open a PR against my own bpf-next branch. This should trigger the Action. Or was there a simpler way to test my set on the CI, that I ignore? Quentin
On Fri, Oct 1, 2021 at 4:06 AM Quentin Monnet <quentin@isovalent.com> wrote: > > 2021-09-30 13:17 UTC+0100 ~ Quentin Monnet <quentin@isovalent.com> > > 2021-09-30 12:33 UTC+0100 ~ Quentin Monnet <quentin@isovalent.com> > >> API headers from libbpf should not be accessed directly from the > >> library's source directory. Instead, they should be exported with "make > >> install_headers". Let's make sure that bpf/preload/iterators/Makefile > >> installs the headers properly when building. > > > > CI complains when trying to build > > kernel/bpf/preload/iterators/iterators.o. I'll look more into this. > > My error was in fact on the previous patch for kernel/preload/Makefile, > where iterators.o is handled. The resulting Makefile in my v1 contained: > > bpf_preload_umd-objs := iterators/iterators.o > bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz > > $(obj)/bpf_preload_umd: $(LIBBPF_A) > > This declares a dependency on $(LIBBPF_A) for building the final > bpf_preload_umd target, when iterators/iterators.o is linked against the > libraries. It does not declare the dependency for iterators/iterators.o > itself. So when we attempt to build the object file, libbpf has not been > compiled yet (not an issue per se), and the API headers from libbpf have > not been installed and made available to iterators.o, causing the build > to fail. > > Before this patch, there was no issue because the headers would be > included directly from tools/lib/bpf, so they would always be present. > I'll fix this by adding the relevant dependency, and send a v2. > > As a side note, I couldn't reproduce the issue locally or in the VM for > the selftests, I'm not sure why. I struggled to get helpful logs from > the kernel CI (kernel build in non-verbose mode), so I ended up copying > the CI infra (running on kernel-patches/bpf on GitHub) to my own GitHub > repository to add debug info and do other runs without re-posting every > time to the mailing list. In case anyone else is interested, I figured I > might share the steps: > > - Clone the linux repo on GitHub, push the bpf-next branch > - Copy all files and directories from the kernel-patches/vmtest GitHub > repo (including the .github directory) to the root of my linux repo, on > my development branch. > - Update the checks on "kernel-patches/bpf" repository name in > .github/workflows/test.yaml, to avoid pulling new Linux sources and > overwriting the files on my branch. > - (Add as much build debug info as necessary.) > - Push the branch to GitHub and open a PR against my own bpf-next > branch. This should trigger the Action. > > Or was there a simpler way to test my set on the CI, that I ignore? Don't know, I never tried :) But maybe Yucong (cc'ed) knows some tips and tricks? > > Quentin
diff --git a/kernel/bpf/preload/iterators/Makefile b/kernel/bpf/preload/iterators/Makefile index 28fa8c1440f4..cf549dab3e20 100644 --- a/kernel/bpf/preload/iterators/Makefile +++ b/kernel/bpf/preload/iterators/Makefile @@ -6,9 +6,11 @@ LLVM_STRIP ?= llvm-strip DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool BPFTOOL ?= $(DEFAULT_BPFTOOL) LIBBPF_SRC := $(abspath ../../../../tools/lib/bpf) -BPFOBJ := $(OUTPUT)/libbpf.a -BPF_INCLUDE := $(OUTPUT) -INCLUDES := -I$(OUTPUT) -I$(BPF_INCLUDE) -I$(abspath ../../../../tools/lib) \ +LIBBPF_OUTPUT := $(abspath $(OUTPUT))/libbpf +LIBBPF_DESTDIR := $(LIBBPF_OUTPUT) +LIBBPF_INCLUDE := $(LIBBPF_DESTDIR)/include +BPFOBJ := $(LIBBPF_OUTPUT)/libbpf.a +INCLUDES := -I$(OUTPUT) -I$(LIBBPF_INCLUDE) \ -I$(abspath ../../../../tools/include/uapi) CFLAGS := -g -Wall @@ -44,13 +46,15 @@ $(OUTPUT)/iterators.bpf.o: iterators.bpf.c $(BPFOBJ) | $(OUTPUT) -c $(filter %.c,$^) -o $@ && \ $(LLVM_STRIP) -g $@ -$(OUTPUT): +$(OUTPUT) $(LIBBPF_OUTPUT) $(LIBBPF_INCLUDE): $(call msg,MKDIR,$@) - $(Q)mkdir -p $(OUTPUT) + $(Q)mkdir -p $@ -$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT) +$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) \ + | $(LIBBPF_OUTPUT) $(LIBBPF_INCLUDE) $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) \ - OUTPUT=$(abspath $(dir $@))/ $(abspath $@) + OUTPUT=$(abspath $(dir $@))/ prefix= \ + DESTDIR=$(LIBBPF_DESTDIR) $(abspath $@) install_headers $(DEFAULT_BPFTOOL): $(Q)$(MAKE) $(submake_extras) -C ../../../../tools/bpf/bpftool \
API headers from libbpf should not be accessed directly from the library's source directory. Instead, they should be exported with "make install_headers". Let's make sure that bpf/preload/iterators/Makefile installs the headers properly when building. Signed-off-by: Quentin Monnet <quentin@isovalent.com> --- kernel/bpf/preload/iterators/Makefile | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)