| Message ID | 20211216163842.829836-2-jean-philippe@linaro.org (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | cebdb7374577ac6e14afb11311af8c2c44a259fa |
| Headers | show |
| Series | tools/bpf: Enable cross-building with clang | expand |
On Thu, Dec 16, 2021 at 8:50 AM Jean-Philippe Brucker <jean-philippe@linaro.org> wrote: > > Cross-compilation with clang uses the -target parameter rather than a > toolchain prefix. Just like the kernel Makefile, add that parameter to > CFLAGS when CROSS_COMPILE is set. > > Unlike the kernel Makefile, we use the --sysroot and --gcc-toolchain > options because unlike the kernel, tools require standard libraries. > Commit c91d4e47e10e ("Makefile: Remove '--gcc-toolchain' flag") provides > some background about --gcc-toolchain. Normally clang finds on its own > the additional utilities and libraries that it needs (for example GNU ld > or glibc). On some systems however, this autodetection doesn't work. > There, our only recourse is asking GCC directly, and pass the result to > --sysroot and --gcc-toolchain. Of course that only works when a cross > GCC is available. > > Autodetection worked fine on Debian, but to use the aarch64-linux-gnu > toolchain from Archlinux I needed both --sysroot (for crt1.o) and > --gcc-toolchain (for crtbegin.o, -lgcc). The --prefix parameter wasn't > needed there, but it might be useful on other distributions. > > Use the CLANG_CROSS_FLAGS variable instead of CLANG_FLAGS because it > allows tools such as bpftool, that need to build both host and target > binaries, to easily filter out the cross-build flags from CFLAGS. > > Acked-by: Quentin Monnet <quentin@isovalent.com> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > --- > Most tools I looked at needed additional changes to support cross-build > with clang. I've only done the work for bpf tools. > --- > tools/scripts/Makefile.include | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include > index 071312f5eb92..b0be5f40a3f1 100644 > --- a/tools/scripts/Makefile.include > +++ b/tools/scripts/Makefile.include > @@ -87,7 +87,18 @@ LLVM_STRIP ?= llvm-strip > > ifeq ($(CC_NO_CLANG), 1) > EXTRA_WARNINGS += -Wstrict-aliasing=3 > -endif > + > +else ifneq ($(CROSS_COMPILE),) > +CLANG_CROSS_FLAGS := --target=$(notdir $(CROSS_COMPILE:%-=%)) > +GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)gcc)) > +ifneq ($(GCC_TOOLCHAIN_DIR),) > +CLANG_CROSS_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) > +CLANG_CROSS_FLAGS += --sysroot=$(shell $(CROSS_COMPILE)gcc -print-sysroot) ^ this is a neat trick; it does rely on having $(CROSS_COMPILE)gcc installed though. Technically only the header include path is needed and the path to the cross libc shared object. I think we should take this patch&series as a starting point, but there might be future revisions here to support environments that don't have $(CROSS_COMPILE)gcc (such as Android's build env). Acked-by: Nick Desaulniers <ndesaulniers@google.com> Thanks for the series! > +CLANG_CROSS_FLAGS += --gcc-toolchain=$(realpath $(GCC_TOOLCHAIN_DIR)/..) > +endif # GCC_TOOLCHAIN_DIR > +CFLAGS += $(CLANG_CROSS_FLAGS) > +AFLAGS += $(CLANG_CROSS_FLAGS) > +endif # CROSS_COMPILE > > # Hack to avoid type-punned warnings on old systems such as RHEL5: > # We should be changing CFLAGS and checking gcc version, but this > -- > 2.34.1 >
Hi Jean-Philippe, On Thu, Dec 16, 2021 at 04:38:38PM +0000, Jean-Philippe Brucker wrote: > Cross-compilation with clang uses the -target parameter rather than a > toolchain prefix. Just like the kernel Makefile, add that parameter to > CFLAGS when CROSS_COMPILE is set. > > Unlike the kernel Makefile, we use the --sysroot and --gcc-toolchain > options because unlike the kernel, tools require standard libraries. > Commit c91d4e47e10e ("Makefile: Remove '--gcc-toolchain' flag") provides > some background about --gcc-toolchain. Normally clang finds on its own > the additional utilities and libraries that it needs (for example GNU ld > or glibc). On some systems however, this autodetection doesn't work. > There, our only recourse is asking GCC directly, and pass the result to > --sysroot and --gcc-toolchain. Of course that only works when a cross > GCC is available. > > Autodetection worked fine on Debian, but to use the aarch64-linux-gnu > toolchain from Archlinux I needed both --sysroot (for crt1.o) and > --gcc-toolchain (for crtbegin.o, -lgcc). The --prefix parameter wasn't > needed there, but it might be useful on other distributions. > > Use the CLANG_CROSS_FLAGS variable instead of CLANG_FLAGS because it > allows tools such as bpftool, that need to build both host and target > binaries, to easily filter out the cross-build flags from CFLAGS. > > Acked-by: Quentin Monnet <quentin@isovalent.com> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > --- > Most tools I looked at needed additional changes to support cross-build > with clang. I've only done the work for bpf tools. > --- > tools/scripts/Makefile.include | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include > index 071312f5eb92..b0be5f40a3f1 100644 > --- a/tools/scripts/Makefile.include > +++ b/tools/scripts/Makefile.include > @@ -87,7 +87,18 @@ LLVM_STRIP ?= llvm-strip > > ifeq ($(CC_NO_CLANG), 1) > EXTRA_WARNINGS += -Wstrict-aliasing=3 > -endif > + > +else ifneq ($(CROSS_COMPILE),) > +CLANG_CROSS_FLAGS := --target=$(notdir $(CROSS_COMPILE:%-=%)) > +GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)gcc)) Apologies for noticing this so late, I only ran into this recently. This line causes a warning when running 'make clean' when '$(CROSS_COMPILE)gcc' does not exist in PATH. For example: $ make -skj"$(nproc)" ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- LLVM=1 LLVM_IAS=0 clean which: no powerpc-linux-gnu-gcc in ($PATH) I only have powerpc-linux-gnu binutils in my PATH, not GCC, as I am only working with clang. This happens because of the 'resolve_btfids_clean target', which always runs when running the 'clean' target on an in-tree build (since $(objtree) = $(srctree)). I tried looking into the best way to fix this but I am not at all familiar with the tools/ build system; would you mind taking a look? I see some machinery at the top of tools/bpf/Makefile for avoiding running some commands under certain commands but I am unsure how to shuffle that around to make everything work. Cheers, Nathan > +ifneq ($(GCC_TOOLCHAIN_DIR),) > +CLANG_CROSS_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) > +CLANG_CROSS_FLAGS += --sysroot=$(shell $(CROSS_COMPILE)gcc -print-sysroot) > +CLANG_CROSS_FLAGS += --gcc-toolchain=$(realpath $(GCC_TOOLCHAIN_DIR)/..) > +endif # GCC_TOOLCHAIN_DIR > +CFLAGS += $(CLANG_CROSS_FLAGS) > +AFLAGS += $(CLANG_CROSS_FLAGS) > +endif # CROSS_COMPILE > > # Hack to avoid type-punned warnings on old systems such as RHEL5: > # We should be changing CFLAGS and checking gcc version, but this > -- > 2.34.1 > >
Hi Nathan, On Fri, Jan 28, 2022 at 06:10:24PM -0700, Nathan Chancellor wrote: > > diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include > > index 071312f5eb92..b0be5f40a3f1 100644 > > --- a/tools/scripts/Makefile.include > > +++ b/tools/scripts/Makefile.include > > @@ -87,7 +87,18 @@ LLVM_STRIP ?= llvm-strip > > > > ifeq ($(CC_NO_CLANG), 1) > > EXTRA_WARNINGS += -Wstrict-aliasing=3 > > -endif > > + > > +else ifneq ($(CROSS_COMPILE),) > > +CLANG_CROSS_FLAGS := --target=$(notdir $(CROSS_COMPILE:%-=%)) > > +GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)gcc)) > > Apologies for noticing this so late, I only ran into this recently. > > This line causes a warning when running 'make clean' when > '$(CROSS_COMPILE)gcc' does not exist in PATH. For example: > > $ make -skj"$(nproc)" ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- LLVM=1 LLVM_IAS=0 clean > which: no powerpc-linux-gnu-gcc in ($PATH) > > I only have powerpc-linux-gnu binutils in my PATH, not GCC, as I am only > working with clang. > > This happens because of the 'resolve_btfids_clean target', which always > runs when running the 'clean' target on an in-tree build (since > $(objtree) = $(srctree)). > > I tried looking into the best way to fix this but I am not at all > familiar with the tools/ build system; would you mind taking a look? > I see some machinery at the top of tools/bpf/Makefile for avoiding > running some commands under certain commands but I am unsure how to > shuffle that around to make everything work. I think it's simpler than that, we should just suppress the errors from 'which'. It's fine that $(CROSS_COMPILE)gcc doesn't exist and $(GCC_TOOLCHAIN_DIR) is empty, but 'which' should keep quiet about it. I did test this patch with cross-build and no gcc, but Debian's 'which' is quiet by default so I missed the error. I'll send a fix shortly. Thanks, Jean
On Mon, Jan 31, 2022 at 02:10:30PM +0000, Jean-Philippe Brucker wrote: > Hi Nathan, > > On Fri, Jan 28, 2022 at 06:10:24PM -0700, Nathan Chancellor wrote: > > > diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include > > > index 071312f5eb92..b0be5f40a3f1 100644 > > > --- a/tools/scripts/Makefile.include > > > +++ b/tools/scripts/Makefile.include > > > @@ -87,7 +87,18 @@ LLVM_STRIP ?= llvm-strip > > > > > > ifeq ($(CC_NO_CLANG), 1) > > > EXTRA_WARNINGS += -Wstrict-aliasing=3 > > > -endif > > > + > > > +else ifneq ($(CROSS_COMPILE),) > > > +CLANG_CROSS_FLAGS := --target=$(notdir $(CROSS_COMPILE:%-=%)) > > > +GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)gcc)) > > > > Apologies for noticing this so late, I only ran into this recently. > > > > This line causes a warning when running 'make clean' when > > '$(CROSS_COMPILE)gcc' does not exist in PATH. For example: > > > > $ make -skj"$(nproc)" ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- LLVM=1 LLVM_IAS=0 clean > > which: no powerpc-linux-gnu-gcc in ($PATH) > > > > I only have powerpc-linux-gnu binutils in my PATH, not GCC, as I am only > > working with clang. > > > > This happens because of the 'resolve_btfids_clean target', which always > > runs when running the 'clean' target on an in-tree build (since > > $(objtree) = $(srctree)). > > > > I tried looking into the best way to fix this but I am not at all > > familiar with the tools/ build system; would you mind taking a look? > > I see some machinery at the top of tools/bpf/Makefile for avoiding > > running some commands under certain commands but I am unsure how to > > shuffle that around to make everything work. > > I think it's simpler than that, we should just suppress the errors from > 'which'. It's fine that $(CROSS_COMPILE)gcc doesn't exist and > $(GCC_TOOLCHAIN_DIR) is empty, but 'which' should keep quiet about it. Ha, that's what I get for looking into something at the very end of a long work week because that is a great and simple solution I probably would not have seen :) > I did test this patch with cross-build and no gcc, but Debian's 'which' is > quiet by default so I missed the error. I'll send a fix shortly. Ah, that is an interesting observation. TIL that Debian rolls their own which [1], versus most other distros, which use GNU which [2]. I will keep an eye out for your patch so I can review it. [1]: https://salsa.debian.org/debian/debianutils/-/blob/de14223e5bffe15e374a441302c528ffc1cbed57/which [2]: https://savannah.gnu.org/projects/which/ Cheers, Nathan
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include index 071312f5eb92..b0be5f40a3f1 100644 --- a/tools/scripts/Makefile.include +++ b/tools/scripts/Makefile.include @@ -87,7 +87,18 @@ LLVM_STRIP ?= llvm-strip ifeq ($(CC_NO_CLANG), 1) EXTRA_WARNINGS += -Wstrict-aliasing=3 -endif + +else ifneq ($(CROSS_COMPILE),) +CLANG_CROSS_FLAGS := --target=$(notdir $(CROSS_COMPILE:%-=%)) +GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)gcc)) +ifneq ($(GCC_TOOLCHAIN_DIR),) +CLANG_CROSS_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) +CLANG_CROSS_FLAGS += --sysroot=$(shell $(CROSS_COMPILE)gcc -print-sysroot) +CLANG_CROSS_FLAGS += --gcc-toolchain=$(realpath $(GCC_TOOLCHAIN_DIR)/..) +endif # GCC_TOOLCHAIN_DIR +CFLAGS += $(CLANG_CROSS_FLAGS) +AFLAGS += $(CLANG_CROSS_FLAGS) +endif # CROSS_COMPILE # Hack to avoid type-punned warnings on old systems such as RHEL5: # We should be changing CFLAGS and checking gcc version, but this
Cross-compilation with clang uses the -target parameter rather than a toolchain prefix. Just like the kernel Makefile, add that parameter to CFLAGS when CROSS_COMPILE is set. Unlike the kernel Makefile, we use the --sysroot and --gcc-toolchain options because unlike the kernel, tools require standard libraries. Commit c91d4e47e10e ("Makefile: Remove '--gcc-toolchain' flag") provides some background about --gcc-toolchain. Normally clang finds on its own the additional utilities and libraries that it needs (for example GNU ld or glibc). On some systems however, this autodetection doesn't work. There, our only recourse is asking GCC directly, and pass the result to --sysroot and --gcc-toolchain. Of course that only works when a cross GCC is available. Autodetection worked fine on Debian, but to use the aarch64-linux-gnu toolchain from Archlinux I needed both --sysroot (for crt1.o) and --gcc-toolchain (for crtbegin.o, -lgcc). The --prefix parameter wasn't needed there, but it might be useful on other distributions. Use the CLANG_CROSS_FLAGS variable instead of CLANG_FLAGS because it allows tools such as bpftool, that need to build both host and target binaries, to easily filter out the cross-build flags from CFLAGS. Acked-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> --- Most tools I looked at needed additional changes to support cross-build with clang. I've only done the work for bpf tools. --- tools/scripts/Makefile.include | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)