| Message ID | 20210209050047.1958473-1-daniel.diaz@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | scripts: Fix linking extract-cert against libcrypto | expand |
Am Dienstag, 9. Februar 2021, 05:59:56 CET schrieb Daniel Díaz: > When compiling under OpenEmbedded, the following error is seen > as of recently: > > /srv/oe/build/tmp/hosttools/ld: cannot find /lib/libc.so.6 inside / > /srv/oe/build/tmp/hosttools/ld: cannot find /usr/lib/libc_nonshared.a > inside / /srv/oe/build/tmp/hosttools/ld: cannot find > /lib/ld-linux-x86-64.so.2 inside / collect2: error: ld returned 1 exit > status > make[2]: *** [scripts/Makefile.host:95: scripts/extract-cert] Error 1 [...] > As per `make`'s documentation: > > LDFLAGS > Extra flags to give to compilers when they are supposed to > invoke the linker, ‘ld’, such as -L. Libraries (-lfoo) > should be added to the LDLIBS variable instead. > > LDLIBS > Library flags or names given to compilers when they are > supposed to invoke the linker, ‘ld’. LOADLIBES is a > deprecated (but still supported) alternative to LDLIBS. > Non-library linker flags, such as -L, should go in the > LDFLAGS variable. Correct. And the patch I use for my local 4.19 build actually uses LDLIBS, so it must have gone wrong in some rebase for one of the intermediate versions. Acked-by: Rolf Eike Beer <eb@emlix.com>
On Tue, Feb 9, 2021 at 2:01 PM Daniel Díaz <daniel.diaz@linaro.org> wrote: > > When compiling under OpenEmbedded, the following error is seen > as of recently: > > /srv/oe/build/tmp/hosttools/ld: cannot find /lib/libc.so.6 inside / > /srv/oe/build/tmp/hosttools/ld: cannot find /usr/lib/libc_nonshared.a inside / > /srv/oe/build/tmp/hosttools/ld: cannot find /lib/ld-linux-x86-64.so.2 inside / > collect2: error: ld returned 1 exit status > make[2]: *** [scripts/Makefile.host:95: scripts/extract-cert] Error 1 > > This is because 2cea4a7a1885 ("scripts: use pkg-config to > locate libcrypto") now calls for `pkg-config --libs libcrypto` > and inserts that into the Makefile rules as LDLIBS when > building extract-cert.c. > > The problem is that --libs will include both -l and -L, which > will be out of order when compiling/linking. > > This (very ugly) command is what's produced with OpenEmbedded: > > gcc -Wp,-MMD,scripts/.extract-cert.d -Wall -Wmissing-prototypes -Wstrict-prototypes \ > -O2 -fomit-frame-pointer -std=gnu89 \ > -isystem/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include \ > -O2 -pipe -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ > -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ > -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ > -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ > -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ > -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ > -Wl,-O1 -I/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include \ > -I ./scripts -o scripts/extract-cert \ > /oe/build/tmp/work-shared/intel-corei7-64/kernel-source/scripts/extract-cert.c \ > -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot/usr//lib \ > -lcrypto > > As per `make`'s documentation: > > LDFLAGS > Extra flags to give to compilers when they are supposed to > invoke the linker, ‘ld’, such as -L. Libraries (-lfoo) > should be added to the LDLIBS variable instead. > > LDLIBS > Library flags or names given to compilers when they are > supposed to invoke the linker, ‘ld’. LOADLIBES is a > deprecated (but still supported) alternative to LDLIBS. > Non-library linker flags, such as -L, should go in the > LDFLAGS variable. > > Fixes: 2cea4a7a1885 ("scripts: use pkg-config to locate libcrypto") > Cc: stable@vger.kernel.org # 5.6.x > Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> > Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org> > --- > scripts/Makefile | 9 ++++++--- > 1 file changed, 6 insertions(+), 3 deletions(-) > > diff --git a/scripts/Makefile b/scripts/Makefile > index 9de3c03b94aa..4b4e938b4ba7 100644 > --- a/scripts/Makefile > +++ b/scripts/Makefile > @@ -3,7 +3,8 @@ > # scripts contains sources for various helper programs used throughout > # the kernel for the build process. > > -CRYPTO_LIBS = $(shell pkg-config --libs libcrypto 2> /dev/null || echo -lcrypto) > +CRYPTO_LDFLAGS = $(shell pkg-config --libs-only-L libcrypto 2> /dev/null) > +CRYPTO_LDLIBS = $(shell pkg-config --libs-only-l libcrypto 2> /dev/null || echo -lcrypto) > CRYPTO_CFLAGS = $(shell pkg-config --cflags libcrypto 2> /dev/null) > > hostprogs-always-$(CONFIG_BUILD_BIN2C) += bin2c > @@ -17,9 +18,11 @@ hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert > > HOSTCFLAGS_sorttable.o = -I$(srctree)/tools/include > HOSTCFLAGS_asn1_compiler.o = -I$(srctree)/include > -HOSTLDLIBS_sign-file = $(CRYPTO_LIBS) > +HOSTLDFLAGS_sign-file = $(CRYPTO_LDFLAGS) > +HOSTLDLIBS_sign-file = $(CRYPTO_LDLIBS) > HOSTCFLAGS_extract-cert.o = $(CRYPTO_CFLAGS) > -HOSTLDLIBS_extract-cert = $(CRYPTO_LIBS) > +HOSTLDFLAGS_extract-cert = $(CRYPTO_LDFLAGS) > +HOSTLDLIBS_extract-cert = $(CRYPTO_LDLIBS) > > ifdef CONFIG_UNWINDER_ORC > ifeq ($(ARCH),x86_64) > -- > 2.25.1 > I am wondering how "HOSTLDFLAGS_sign-file" and "HOSTLDFLAGS_extract-cert" worked for you. Kbuild supports HOSTLDLIBS_<target> syntax, but not HOSTLDFLAGS_<target>. I see $(HOSTLDLIBS_$(target-stem) in scripts/Makefile.host but failed to find $(HOSTLDFLAGS_$(target-stem)). So, presumably you will get the same result (OE build error will be fixed) even without HOSTLDFLAGS_sign-file or HOSTLDFLAGS_extract-cert. But, I am still wondering what the correct approach is. Basically, there are two ways to link libraries in non-standard paths. [1] Give linker flags via HOSTLDFLAGS This is documented in Documentation/kbuild/kbuild.rst HOSTLDFLAGS ----------- Additional flags to be passed when linking host programs. [2] Use pkg-config OE already adopted [1]. I think the following long lines came from HOSTLDFLAGS. -isystem/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include \ -O2 -pipe -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ -Wl,-O1 -I/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include \ -I ./scripts -o scripts/extract-cert \ But, some people are not satisfied with [1] (or do not notice it)? Then, 2cea4a7a1885 introduced the second one [2]. Mixing [1] and [2] perhaps confuses the build system somehow? So, 2cea4a7a1885 was a problem, but I do not think this patch is the right one either. (At least, HOSTLDFLAGS_* additions are pointless)
Am Dienstag, 9. Februar 2021, 09:44:33 CET schrieb Rolf Eike Beer: > Am Dienstag, 9. Februar 2021, 05:59:56 CET schrieb Daniel Díaz: > > When compiling under OpenEmbedded, the following error is seen > > > > as of recently: > > /srv/oe/build/tmp/hosttools/ld: cannot find /lib/libc.so.6 inside / > > /srv/oe/build/tmp/hosttools/ld: cannot find /usr/lib/libc_nonshared.a > > > > inside / /srv/oe/build/tmp/hosttools/ld: cannot find > > /lib/ld-linux-x86-64.so.2 inside / collect2: error: ld returned 1 exit > > status > > > > make[2]: *** [scripts/Makefile.host:95: scripts/extract-cert] Error 1 > > [...] > > > As per `make`'s documentation: > > LDFLAGS > > > > Extra flags to give to compilers when they are supposed to > > invoke the linker, ‘ld’, such as -L. Libraries (-lfoo) > > should be added to the LDLIBS variable instead. > > > > LDLIBS > > > > Library flags or names given to compilers when they are > > supposed to invoke the linker, ‘ld’. LOADLIBES is a > > deprecated (but still supported) alternative to LDLIBS. > > Non-library linker flags, such as -L, should go in the > > LDFLAGS variable. > > Correct. And the patch I use for my local 4.19 build actually uses LDLIBS, > so it must have gone wrong in some rebase for one of the intermediate > versions. > > Acked-by: Rolf Eike Beer <eb@emlix.com> Oh, scrap that. I misread your patch. I was actually using LDLIBS exclusively, no LDFLAGS at all. I'll have to get my test setup ready for this, can take a moment. Eike
Am Dienstag, 9. Februar 2021, 09:44:33 CET schrieb Rolf Eike Beer: > Am Dienstag, 9. Februar 2021, 05:59:56 CET schrieb Daniel Díaz: > > When compiling under OpenEmbedded, the following error is seen > > > > as of recently: > > /srv/oe/build/tmp/hosttools/ld: cannot find /lib/libc.so.6 inside / > > /srv/oe/build/tmp/hosttools/ld: cannot find /usr/lib/libc_nonshared.a > > > > inside / /srv/oe/build/tmp/hosttools/ld: cannot find > > /lib/ld-linux-x86-64.so.2 inside / collect2: error: ld returned 1 exit > > status > > > > make[2]: *** [scripts/Makefile.host:95: scripts/extract-cert] Error 1 > > [...] > > > As per `make`'s documentation: > > LDFLAGS > > > > Extra flags to give to compilers when they are supposed to > > invoke the linker, ‘ld’, such as -L. Libraries (-lfoo) > > should be added to the LDLIBS variable instead. > > > > LDLIBS > > > > Library flags or names given to compilers when they are > > supposed to invoke the linker, ‘ld’. LOADLIBES is a > > deprecated (but still supported) alternative to LDLIBS. > > Non-library linker flags, such as -L, should go in the > > LDFLAGS variable. > > Correct. And the patch I use for my local 4.19 build actually uses LDLIBS, > so it must have gone wrong in some rebase for one of the intermediate > versions. > > Acked-by: Rolf Eike Beer <eb@emlix.com> Ok, now actually with proper testing: no, your patch doesn't work. When changing LDLIBS to LDFLAGS things do not show up on the commandline at all. LDLIBS: gcc -Wp,-MMD,scripts/.sign-file.d -Wall -Wmissing-prototypes -Wstrict- prototypes -O2 -fomit-frame-pointer -std=gnu89 -I/opt/emlix/test/include -I ./scripts -o scripts/sign-file /tmp/e2/build/linux-kernel/scripts/sign- file.c -L/opt/emlix/test/lib -lcrypto -lz -ldl -pthread LDFLAGS: gcc -Wp,-MMD,scripts/.sign-file.d -Wall -Wmissing-prototypes -Wstrict- prototypes -O2 -fomit-frame-pointer -std=gnu89 -I/opt/emlix/test/include -I ./scripts -o scripts/sign-file /tmp/e2/build/linux-kernel/scripts/sign- file.c When looking closely you may notice that this is not entirely the same as current master would output: I missed the CFLAGS for sign-file in my patch. When testing your patch I accidentially had a .config that had module signatures disabled, so I have not tested it actually, that's why I didn't notice that it doesn't work. I'm just guessing, but your build error looks like you are also cross-building the tools, which is wrong. You want them to be host-tools. So don't export PKG_CONFIG_SYSROOT_DIR, it would then try to link target libraries into a host binary. Eike
Am Donnerstag, 11. Februar 2021, 11:29:33 CET schrieb Rolf Eike Beer: > I'm just guessing, but your build error looks like you are also > cross-building the tools, which is wrong. You want them to be host-tools. > So don't export PKG_CONFIG_SYSROOT_DIR, it would then try to link target > libraries into a host binary. I have looked again how I do it: # this is for additional _host_ .pc files export PKG_CONFIG_PATH=${prefix}/lib/pkgconfig Then have a target-pkg-config, so this code inside several kernel Makefiles will work: PKG_CONFIG ?= $(CROSS_COMPILE)pkg-config And then export your PKG_CONFIG_SYSROOT_DIR and the like inside that. I bet you have all of this already in place, so just remove the SYSROOT_DIR from your kernel build script and things should work. Eike
Hello! Apologies for the delay -- Currently experiencing power/connectivity issues. On Thu, 11 Feb 2021 at 01:12, Masahiro Yamada <masahiroy@kernel.org> wrote: > I am wondering how "HOSTLDFLAGS_sign-file" and > "HOSTLDFLAGS_extract-cert" worked for you. > Kbuild supports HOSTLDLIBS_<target> syntax, > but not HOSTLDFLAGS_<target>. Thanks for the insight! The pointer you have provided are very valuable to try to fix our problem. What effectively happened was that LDFLAGS was removed, and therefore the gcc line did not include one -L path that's proving difficult to use under OpenEmbedded cross-compilations. This discussion has provided much light in areas that are unknown to me, but so far it looks like the fix will need to happen in the OE recipe and not in the kernel itself. Thanks and greetings! Daniel Díaz daniel.diaz@linaro.org > I see $(HOSTLDLIBS_$(target-stem) in scripts/Makefile.host > but failed to find $(HOSTLDFLAGS_$(target-stem)). > > So, presumably you will get the same result > (OE build error will be fixed) > even without HOSTLDFLAGS_sign-file > or HOSTLDFLAGS_extract-cert. > > > > But, I am still wondering what the correct approach is. > > > > Basically, there are two ways to link libraries > in non-standard paths. > > > > [1] Give linker flags via HOSTLDFLAGS > > This is documented in Documentation/kbuild/kbuild.rst > > HOSTLDFLAGS > ----------- > Additional flags to be passed when linking host programs. > > > > > [2] Use pkg-config > > > > > > > OE already adopted [1]. > > I think the following long lines came from HOSTLDFLAGS. > > -isystem/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include > \ > -O2 -pipe -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib > \ > -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib > \ > -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib > \ > -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib > \ > -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib > \ > -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib > \ > -Wl,-O1 -I/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include > \ > -I ./scripts -o scripts/extract-cert \ > > > > > But, some people are not satisfied with [1] (or do not notice it)? > > > Then, 2cea4a7a1885 introduced the second one [2]. > > Mixing [1] and [2] perhaps confuses the build system somehow? > > > > > So, 2cea4a7a1885 was a problem, but > I do not think this patch is the right one either. > > (At least, HOSTLDFLAGS_* additions are pointless) > > > > > -- > Best Regards > Masahiro Yamada
Hello! On Fri, 12 Feb 2021 at 01:44, Rolf Eike Beer <eb@emlix.com> wrote: > > Am Donnerstag, 11. Februar 2021, 11:29:33 CET schrieb Rolf Eike Beer: > > > I'm just guessing, but your build error looks like you are also > > cross-building the tools, which is wrong. You want them to be host-tools. > > So don't export PKG_CONFIG_SYSROOT_DIR, it would then try to link target > > libraries into a host binary. > > I have looked again how I do it: > > # this is for additional _host_ .pc files > export PKG_CONFIG_PATH=${prefix}/lib/pkgconfig > > Then have a target-pkg-config, so this code inside several kernel Makefiles > will work: > > PKG_CONFIG ?= $(CROSS_COMPILE)pkg-config > > And then export your PKG_CONFIG_SYSROOT_DIR and the like inside that. I bet > you have all of this already in place, so just remove the SYSROOT_DIR from > your kernel build script and things should work. Thank you for your comments! I will try this in our environment in the upcoming days. Greetings! Daniel Díaz daniel.diaz@linaro.org
diff --git a/scripts/Makefile b/scripts/Makefile index 9de3c03b94aa..4b4e938b4ba7 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -3,7 +3,8 @@ # scripts contains sources for various helper programs used throughout # the kernel for the build process. -CRYPTO_LIBS = $(shell pkg-config --libs libcrypto 2> /dev/null || echo -lcrypto) +CRYPTO_LDFLAGS = $(shell pkg-config --libs-only-L libcrypto 2> /dev/null) +CRYPTO_LDLIBS = $(shell pkg-config --libs-only-l libcrypto 2> /dev/null || echo -lcrypto) CRYPTO_CFLAGS = $(shell pkg-config --cflags libcrypto 2> /dev/null) hostprogs-always-$(CONFIG_BUILD_BIN2C) += bin2c @@ -17,9 +18,11 @@ hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert HOSTCFLAGS_sorttable.o = -I$(srctree)/tools/include HOSTCFLAGS_asn1_compiler.o = -I$(srctree)/include -HOSTLDLIBS_sign-file = $(CRYPTO_LIBS) +HOSTLDFLAGS_sign-file = $(CRYPTO_LDFLAGS) +HOSTLDLIBS_sign-file = $(CRYPTO_LDLIBS) HOSTCFLAGS_extract-cert.o = $(CRYPTO_CFLAGS) -HOSTLDLIBS_extract-cert = $(CRYPTO_LIBS) +HOSTLDFLAGS_extract-cert = $(CRYPTO_LDFLAGS) +HOSTLDLIBS_extract-cert = $(CRYPTO_LDLIBS) ifdef CONFIG_UNWINDER_ORC ifeq ($(ARCH),x86_64)
When compiling under OpenEmbedded, the following error is seen as of recently: /srv/oe/build/tmp/hosttools/ld: cannot find /lib/libc.so.6 inside / /srv/oe/build/tmp/hosttools/ld: cannot find /usr/lib/libc_nonshared.a inside / /srv/oe/build/tmp/hosttools/ld: cannot find /lib/ld-linux-x86-64.so.2 inside / collect2: error: ld returned 1 exit status make[2]: *** [scripts/Makefile.host:95: scripts/extract-cert] Error 1 This is because 2cea4a7a1885 ("scripts: use pkg-config to locate libcrypto") now calls for `pkg-config --libs libcrypto` and inserts that into the Makefile rules as LDLIBS when building extract-cert.c. The problem is that --libs will include both -l and -L, which will be out of order when compiling/linking. This (very ugly) command is what's produced with OpenEmbedded: gcc -Wp,-MMD,scripts/.extract-cert.d -Wall -Wmissing-prototypes -Wstrict-prototypes \ -O2 -fomit-frame-pointer -std=gnu89 \ -isystem/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include \ -O2 -pipe -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ -Wl,-rpath-link,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/lib \ -Wl,-rpath,/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/lib \ -Wl,-O1 -I/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot-native/usr/include \ -I ./scripts -o scripts/extract-cert \ /oe/build/tmp/work-shared/intel-corei7-64/kernel-source/scripts/extract-cert.c \ -L/oe/build/tmp/work/MACHINE/linux/5.10+gitAUTOINC+b01f250d83-r0/recipe-sysroot/usr//lib \ -lcrypto As per `make`'s documentation: LDFLAGS Extra flags to give to compilers when they are supposed to invoke the linker, ‘ld’, such as -L. Libraries (-lfoo) should be added to the LDLIBS variable instead. LDLIBS Library flags or names given to compilers when they are supposed to invoke the linker, ‘ld’. LOADLIBES is a deprecated (but still supported) alternative to LDLIBS. Non-library linker flags, such as -L, should go in the LDFLAGS variable. Fixes: 2cea4a7a1885 ("scripts: use pkg-config to locate libcrypto") Cc: stable@vger.kernel.org # 5.6.x Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org> --- scripts/Makefile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)