diff mbox series

[bpf-next] selftests/bpf: Cross-compile bpftool

Message ID 20230210084326.1802597-1-bjorn@kernel.org (mailing list archive)
State Superseded
Delegated to: BPF
Headers show
Series [bpf-next] selftests/bpf: Cross-compile bpftool | expand

Checks

Context Check Description
netdev/tree_selection success Clearly marked for bpf-next
netdev/fixes_present success Fixes tag not required for -next series
netdev/subject_prefix success Link
netdev/cover_letter success Single patches do not need cover letters
netdev/patch_count success Link
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 0 this patch: 0
netdev/cc_maintainers warning 10 maintainers not CCed: john.fastabend@gmail.com sdf@google.com shuah@kernel.org jolsa@kernel.org song@kernel.org mykolal@fb.com martin.lau@linux.dev haoluo@google.com yhs@fb.com kpsingh@kernel.org
netdev/build_clang success Errors and warnings before: 0 this patch: 0
netdev/module_param success Was 0 now: 0
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 0 this patch: 0
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 56 lines checked
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-7 success Logs for llvm-toolchain
bpf/vmtest-bpf-next-VM_Test-8 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-6 success Logs for build for x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-2 success Logs for build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-3 success Logs for build for aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-5 success Logs for build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-4 success Logs for build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-37 success Logs for test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-9 success Logs for test_maps on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-10 success Logs for test_maps on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-12 success Logs for test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-13 success Logs for test_maps on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-14 success Logs for test_progs on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-15 success Logs for test_progs on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-17 success Logs for test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-18 success Logs for test_progs on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-19 success Logs for test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-20 success Logs for test_progs_no_alu32 on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-22 success Logs for test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-23 success Logs for test_progs_no_alu32 on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-24 success Logs for test_progs_no_alu32_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-25 success Logs for test_progs_no_alu32_parallel on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-27 success Logs for test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-28 success Logs for test_progs_no_alu32_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-29 success Logs for test_progs_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-30 success Logs for test_progs_parallel on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-32 success Logs for test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-33 success Logs for test_progs_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-34 success Logs for test_verifier on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-35 success Logs for test_verifier on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-38 success Logs for test_verifier on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-26 success Logs for test_progs_no_alu32_parallel on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-21 success Logs for test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-16 success Logs for test_progs on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-36 success Logs for test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-31 success Logs for test_progs_parallel on s390x with gcc
bpf/vmtest-bpf-next-PR success PR summary
bpf/vmtest-bpf-next-VM_Test-11 success Logs for test_maps on s390x with gcc

Commit Message

Björn Töpel Feb. 10, 2023, 8:43 a.m. UTC 
From: Björn Töpel <bjorn@rivosinc.com>

When the BPF selftests are cross-compiled, only the a host version of
bpftool is built. This version of bpftool is used to generate various
intermediates, e.g., skeletons.

The test runners are also using bpftool. The Makefile will symlink
bpftool from the selftest/bpf root, where the test runners will look
for the tool:

  | ...
  | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
  |    $(OUTPUT)/$(if $2,$2/)bpftool

There are two issues for cross-compilation builds:

 1. There is no native (cross-compilation target) build of bpftool
 2. The bootstrap variant of bpftool is never cross-compiled (by
    design)

Make sure that a native/cross-compiled version of bpftool is built,
and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
version.

Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
---
 tools/testing/selftests/bpf/Makefile | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)


base-commit: 06744f24696e1e7598412c3df61a538b57ebec22

Comments

Björn Töpel Feb. 13, 2023, 2:30 p.m. UTC | #1 
Björn Töpel <bjorn@kernel.org> writes:

> From: Björn Töpel <bjorn@rivosinc.com>
>
> When the BPF selftests are cross-compiled, only the a host version of
> bpftool is built. This version of bpftool is used to generate various
> intermediates, e.g., skeletons.
>
> The test runners are also using bpftool. The Makefile will symlink
> bpftool from the selftest/bpf root, where the test runners will look
> for the tool:
>
>   | ...
>   | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
>   |    $(OUTPUT)/$(if $2,$2/)bpftool
>
> There are two issues for cross-compilation builds:
>
>  1. There is no native (cross-compilation target) build of bpftool
>  2. The bootstrap variant of bpftool is never cross-compiled (by
>     design)
>
> Make sure that a native/cross-compiled version of bpftool is built,
> and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
> version.

...and the grand master plan is to add BPF CI support for riscv64, where
this patch a prerequisite to [1]. I would suspect that other platforms
might benefit from cross-compilation builds as well.

[1] https://github.com/kernel-patches/vmtest/pull/194
Quentin Monnet Feb. 13, 2023, 8:53 p.m. UTC | #2 
2023-02-10 09:43 UTC+0100 ~ Björn Töpel <bjorn@kernel.org>
> From: Björn Töpel <bjorn@rivosinc.com>
> 
> When the BPF selftests are cross-compiled, only the a host version of
> bpftool is built. This version of bpftool is used to generate various
> intermediates, e.g., skeletons.
> 
> The test runners are also using bpftool. The Makefile will symlink
> bpftool from the selftest/bpf root, where the test runners will look
> for the tool:
> 
>   | ...
>   | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
>   |    $(OUTPUT)/$(if $2,$2/)bpftool
> 
> There are two issues for cross-compilation builds:
> 
>  1. There is no native (cross-compilation target) build of bpftool
>  2. The bootstrap variant of bpftool is never cross-compiled (by
>     design)
> 
> Make sure that a native/cross-compiled version of bpftool is built,
> and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
> version.
> 
> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
> ---
>  tools/testing/selftests/bpf/Makefile | 28 +++++++++++++++++++++++++---
>  1 file changed, 25 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index b2eb3201b85a..b706750f71e2 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -157,8 +157,9 @@ $(notdir $(TEST_GEN_PROGS)						\
>  	 $(TEST_CUSTOM_PROGS)): %: $(OUTPUT)/% ;
>  
>  # sort removes libbpf duplicates when not cross-building
> -MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	       \
> -	       $(HOST_BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/resolve_btfids      \
> +MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	\
> +	       $(BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/bpftool		\
> +	       $(HOST_BUILD_DIR)/resolve_btfids				\
>  	       $(RUNQSLOWER_OUTPUT) $(INCLUDE_DIR))
>  $(MAKE_DIRS):
>  	$(call msg,MKDIR,,$@)
> @@ -208,6 +209,14 @@ $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes
>  	$(Q)cp bpf_testmod/bpf_testmod.ko $@
>  
>  DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
> +ifneq ($(CROSS_COMPILE),)
> +CROSS_BPFTOOL := $(SCRATCH_DIR)/sbin/bpftool
> +TRUNNER_BPFTOOL := $(CROSS_BPFTOOL)
> +USE_BOOTSTRAP := ""
> +else
> +TRUNNER_BPFTOOL := $(DEFAULT_BPFTOOL)
> +USE_BOOTSTRAP := "bootstrap"
> +endif
>  
>  $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
>  	$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower	       \
> @@ -255,6 +264,18 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
>  		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
>  		    prefix= DESTDIR=$(HOST_SCRATCH_DIR)/ install-bin
>  
> +ifneq ($(CROSS_COMPILE),)
> +$(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
> +		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
> +	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
> +		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
> +		    EXTRA_CFLAGS='-g -O0'					\
> +		    OUTPUT=$(BUILD_DIR)/bpftool/				\
> +		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
> +		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
> +		    prefix= DESTDIR=$(SCRATCH_DIR)/ install-bin
> +endif
> +
>  all: docs
>  
>  docs:
> @@ -518,11 +539,12 @@ endif
>  $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)			\
>  			     $(TRUNNER_EXTRA_OBJS) $$(BPFOBJ)		\
>  			     $(RESOLVE_BTFIDS)				\
> +			     $(TRUNNER_BPFTOOL)				\
>  			     | $(TRUNNER_BINARY)-extras
>  	$$(call msg,BINARY,,$$@)
>  	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@
>  	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
> -	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
> +	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)/bpftool \

Nit: You'll have a double slash in this path when USE_BOOSTRAP is empty
(.../tools/build/bpftool//bpftool), but it probably doesn't matter much.

>  		   $(OUTPUT)/$(if $2,$2/)bpftool
>  
>  endef
> 
> base-commit: 06744f24696e1e7598412c3df61a538b57ebec22

The changes look good to me, thanks!

Acked-by: Quentin Monnet <quentin@isovalent.com>

Jean-Philippe, I know you do some cross-compiling with bpftool, how does
this look from your side?
Zachary Leaf Feb. 14, 2023, 8:57 a.m. UTC | #3 
Hi Bjorn,

Thanks for the patch, I've tested it and it works for me.

I have a minor suggestion but otherwise happy to see this getting fixed.

On 10/02/2023 08:43, Björn Töpel wrote:
> From: Björn Töpel <bjorn@rivosinc.com>
> 
> When the BPF selftests are cross-compiled, only the a host version of
> bpftool is built. This version of bpftool is used to generate various
> intermediates, e.g., skeletons.
> 
> The test runners are also using bpftool. The Makefile will symlink
> bpftool from the selftest/bpf root, where the test runners will look
> for the tool:
> 
>   | ...
>   | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
>   |    $(OUTPUT)/$(if $2,$2/)bpftool
> 
> There are two issues for cross-compilation builds:
> 
>  1. There is no native (cross-compilation target) build of bpftool
>  2. The bootstrap variant of bpftool is never cross-compiled (by
>     design)
> 
> Make sure that a native/cross-compiled version of bpftool is built,
> and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
> version.
> 
> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
> ---
>  tools/testing/selftests/bpf/Makefile | 28 +++++++++++++++++++++++++---
>  1 file changed, 25 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index b2eb3201b85a..b706750f71e2 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -157,8 +157,9 @@ $(notdir $(TEST_GEN_PROGS)						\
>  	 $(TEST_CUSTOM_PROGS)): %: $(OUTPUT)/% ;
>  
>  # sort removes libbpf duplicates when not cross-building
> -MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	       \
> -	       $(HOST_BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/resolve_btfids      \
> +MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	\
> +	       $(BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/bpftool		\
> +	       $(HOST_BUILD_DIR)/resolve_btfids				\
>  	       $(RUNQSLOWER_OUTPUT) $(INCLUDE_DIR))
>  $(MAKE_DIRS):
>  	$(call msg,MKDIR,,$@)
> @@ -208,6 +209,14 @@ $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes
>  	$(Q)cp bpf_testmod/bpf_testmod.ko $@
>  
>  DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
> +ifneq ($(CROSS_COMPILE),)
> +CROSS_BPFTOOL := $(SCRATCH_DIR)/sbin/bpftool
> +TRUNNER_BPFTOOL := $(CROSS_BPFTOOL)
> +USE_BOOTSTRAP := ""
> +else
> +TRUNNER_BPFTOOL := $(DEFAULT_BPFTOOL)
> +USE_BOOTSTRAP := "bootstrap"
> +endif
>  
>  $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
>  	$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower	       \

...


-TEST_GEN_PROGS_EXTENDED += $(DEFAULT_BPFTOOL)
+TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)

Ensure the target arch bpftool is copied into the kselftest_bpf_install
dir by selftests/lib.mk instead of always the default/host version.

Thanks,
Zach

> @@ -255,6 +264,18 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
>  		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
>  		    prefix= DESTDIR=$(HOST_SCRATCH_DIR)/ install-bin
>  
> +ifneq ($(CROSS_COMPILE),)
> +$(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
> +		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
> +	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
> +		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
> +		    EXTRA_CFLAGS='-g -O0'					\
> +		    OUTPUT=$(BUILD_DIR)/bpftool/				\
> +		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
> +		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
> +		    prefix= DESTDIR=$(SCRATCH_DIR)/ install-bin
> +endif
> +
>  all: docs
>  
>  docs:
> @@ -518,11 +539,12 @@ endif
>  $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TESAT_OBJS)			\
>  			     $(TRUNNER_EXTRA_OBJS) $$(BPFOBJ)		\
>  			     $(RESOLVE_BTFIDS)		related to		\
> +			     $(TRUNNER_BPFTOOL)				\
>  			     | $(TRUNNER_BINARY)-extras
>  	$$(call msg,BINARY,,$$@)
>  	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@
>  	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
> -	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
> +	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)/bpftool \>  		   $(OUTPUT)/$(if $2,$2/)bpftool
>  
>  endef
> 
> base-commit: 06744f24696e1e7598412c3df61a538b57ebec22
Zachary Leaf Feb. 14, 2023, 9:16 a.m. UTC | #4 
On 13/02/2023 14:30, Björn Töpel wrote:
> Björn Töpel <bjorn@kernel.org> writes:
> 
>> From: Björn Töpel <bjorn@rivosinc.com>
>>
>> When the BPF selftests are cross-compiled, only the a host version of
>> bpftool is built. This version of bpftool is used to generate various
>> intermediates, e.g., skeletons.
>>
>> The test runners are also using bpftool. The Makefile will symlink
>> bpftool from the selftest/bpf root, where the test runners will look
>> for the tool:
>>
>>   | ...
>>   | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
>>   |    $(OUTPUT)/$(if $2,$2/)bpftool
>>
>> There are two issues for cross-compilation builds:
>>
>>  1. There is no native (cross-compilation target) build of bpftool
>>  2. The bootstrap variant of bpftool is never cross-compiled (by
>>     design)
>>
>> Make sure that a native/cross-compiled version of bpftool is built,
>> and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
>> version.
> 
> ...and the grand master plan is to add BPF CI support for riscv64, where
> this patch a prerequisite to [1]. I would suspect that other platforms
> might benefit from cross-compilation builds as well.

Similar use case. There also seems to be a lot of issues building these
tests out of tree.

I have some potential fixes up to 6.1 but linux-next seems to have
introduced a few more issues on top.

> 
> [1] https://github.com/kernel-patches/vmtest/pull/194
Björn Töpel Feb. 14, 2023, 9:41 a.m. UTC | #5 
Zachary Leaf <zachary.leaf@arm.com> writes:

> Hi Bjorn,
>
> Thanks for the patch, I've tested it and it works for me.

Good!

> I have a minor suggestion but otherwise happy to see this getting fixed.
>
> ...
>
>
> -TEST_GEN_PROGS_EXTENDED += $(DEFAULT_BPFTOOL)
> +TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)
>
> Ensure the target arch bpftool is copied into the kselftest_bpf_install
> dir by selftests/lib.mk instead of always the default/host version.

Good one. I'll spin a v2, and also fix Quentin's "double-slash".


Björn
Björn Töpel Feb. 14, 2023, 9:44 a.m. UTC | #6 
Zachary Leaf <zachary.leaf@arm.com> writes:

> On 13/02/2023 14:30, Björn Töpel wrote:
>> Björn Töpel <bjorn@kernel.org> writes:
>> 
>>> From: Björn Töpel <bjorn@rivosinc.com>
>>>
>>> When the BPF selftests are cross-compiled, only the a host version of
>>> bpftool is built. This version of bpftool is used to generate various
>>> intermediates, e.g., skeletons.
>>>
>>> The test runners are also using bpftool. The Makefile will symlink
>>> bpftool from the selftest/bpf root, where the test runners will look
>>> for the tool:
>>>
>>>   | ...
>>>   | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
>>>   |    $(OUTPUT)/$(if $2,$2/)bpftool
>>>
>>> There are two issues for cross-compilation builds:
>>>
>>>  1. There is no native (cross-compilation target) build of bpftool
>>>  2. The bootstrap variant of bpftool is never cross-compiled (by
>>>     design)
>>>
>>> Make sure that a native/cross-compiled version of bpftool is built,
>>> and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
>>> version.
>> 
>> ...and the grand master plan is to add BPF CI support for riscv64, where
>> this patch a prerequisite to [1]. I would suspect that other platforms
>> might benefit from cross-compilation builds as well.
>
> Similar use case. There also seems to be a lot of issues building these
> tests out of tree.
>
> I have some potential fixes up to 6.1 but linux-next seems to have
> introduced a few more issues on top.

Ah, yes. FWIW, the BPF CI builds the selftests *in-tree*, so with this
patch (and my PRs) the BPF CI is capable of cross-compiling.


Björn
Jean-Philippe Brucker Feb. 14, 2023, 9:46 a.m. UTC | #7 
Hi Quentin,

On Mon, Feb 13, 2023 at 08:53:31PM +0000, Quentin Monnet wrote:
> Jean-Philippe, I know you do some cross-compiling with bpftool, how does
> this look from your side?

I don't have enough time for BPF at the moment unfortunately. Zachary is
looking at cross-compiling the selftests for arm64

Thanks,
Jean
diff mbox series

Patch

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index b2eb3201b85a..b706750f71e2 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -157,8 +157,9 @@  $(notdir $(TEST_GEN_PROGS)						\
 	 $(TEST_CUSTOM_PROGS)): %: $(OUTPUT)/% ;
 
 # sort removes libbpf duplicates when not cross-building
-MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	       \
-	       $(HOST_BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/resolve_btfids      \
+MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	\
+	       $(BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/bpftool		\
+	       $(HOST_BUILD_DIR)/resolve_btfids				\
 	       $(RUNQSLOWER_OUTPUT) $(INCLUDE_DIR))
 $(MAKE_DIRS):
 	$(call msg,MKDIR,,$@)
@@ -208,6 +209,14 @@  $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes
 	$(Q)cp bpf_testmod/bpf_testmod.ko $@
 
 DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
+ifneq ($(CROSS_COMPILE),)
+CROSS_BPFTOOL := $(SCRATCH_DIR)/sbin/bpftool
+TRUNNER_BPFTOOL := $(CROSS_BPFTOOL)
+USE_BOOTSTRAP := ""
+else
+TRUNNER_BPFTOOL := $(DEFAULT_BPFTOOL)
+USE_BOOTSTRAP := "bootstrap"
+endif
 
 $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
 	$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower	       \
@@ -255,6 +264,18 @@  $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
 		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
 		    prefix= DESTDIR=$(HOST_SCRATCH_DIR)/ install-bin
 
+ifneq ($(CROSS_COMPILE),)
+$(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
+		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
+	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
+		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
+		    EXTRA_CFLAGS='-g -O0'					\
+		    OUTPUT=$(BUILD_DIR)/bpftool/				\
+		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
+		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
+		    prefix= DESTDIR=$(SCRATCH_DIR)/ install-bin
+endif
+
 all: docs
 
 docs:
@@ -518,11 +539,12 @@  endif
 $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)			\
 			     $(TRUNNER_EXTRA_OBJS) $$(BPFOBJ)		\
 			     $(RESOLVE_BTFIDS)				\
+			     $(TRUNNER_BPFTOOL)				\
 			     | $(TRUNNER_BINARY)-extras
 	$$(call msg,BINARY,,$$@)
 	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@
 	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
-	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
+	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)/bpftool \
 		   $(OUTPUT)/$(if $2,$2/)bpftool
 
 endef