| Message ID | 06d96bd81fe812a9718098a383678ad3beba98b1.1691215074.git.falcon@tinylab.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | selftests/nolibc: add misc improvments | expand |
On Sat, Aug 05, 2023 at 02:13:11PM +0800, Zhangjin Wu wrote: > To avoid pollute the source code tree and avoid mrproper for every > architecture switch, the O= argument must be supported. > > Both IMAGE and .config are from the building directory, let's use > objtree instead of srctree for them. > > If no O= option specified, means building kernel in source code tree, > objtree should be srctree in such case. > > To support relative path, as suggested by Thomas, $(COMMAND_O) is used > to pass the O=$(ABSOLUTE_O) to the $(MAKE) commands. Zhangjin, I cannot get this one to work, regardless of what I try: $ make -j8 O=$PWD/output-i386 nolibc-test XARCH=i386 CROSS_COMPILE=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux- CC=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux-gcc (...) CC nolibc-test In file included from sysroot/i386/include/stdio.h:14, from nolibc-test.c:13: sysroot/i386/include/errno.h:10:10: fatal error: asm/errno.h: No such file or directory 10 | #include <asm/errno.h> | ^~~~~~~~~~~~~ compilation terminated. I'll leave it aside for now as I've spent way longer than I hoped on these series. I could take the previous two patches however. Thanks, Willy
> On Sat, Aug 05, 2023 at 02:13:11PM +0800, Zhangjin Wu wrote: > > To avoid pollute the source code tree and avoid mrproper for every > > architecture switch, the O= argument must be supported. > > > > Both IMAGE and .config are from the building directory, let's use > > objtree instead of srctree for them. > > > > If no O= option specified, means building kernel in source code tree, > > objtree should be srctree in such case. > > > > To support relative path, as suggested by Thomas, $(COMMAND_O) is used > > to pass the O=$(ABSOLUTE_O) to the $(MAKE) commands. > > Zhangjin, I cannot get this one to work, regardless of what I try: > > $ make -j8 O=$PWD/output-i386 nolibc-test XARCH=i386 CROSS_COMPILE=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux- CC=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux-gcc > (...) > CC nolibc-test > In file included from sysroot/i386/include/stdio.h:14, > from nolibc-test.c:13: > sysroot/i386/include/errno.h:10:10: fatal error: asm/errno.h: No such file or directory > 10 | #include <asm/errno.h> > | ^~~~~~~~~~~~~ > compilation terminated. > Willy, I also just reproduced the issue, seems only i386 sysroot has no asm/errno.h, will check why it is no rightly installed later. $ find sysroot/ -name "errno.h" sysroot/arm/include/errno.h sysroot/arm/include/asm-generic/errno.h sysroot/arm/include/asm/errno.h sysroot/arm/include/linux/errno.h sysroot/riscv/include/errno.h sysroot/riscv/include/asm-generic/errno.h sysroot/riscv/include/asm/errno.h sysroot/riscv/include/linux/errno.h sysroot/s390/include/errno.h sysroot/s390/include/asm-generic/errno.h sysroot/s390/include/asm/errno.h sysroot/s390/include/linux/errno.h sysroot/arm64/include/errno.h sysroot/arm64/include/asm-generic/errno.h sysroot/arm64/include/asm/errno.h sysroot/arm64/include/linux/errno.h sysroot/mips/include/errno.h sysroot/mips/include/asm-generic/errno.h sysroot/mips/include/asm/errno.h sysroot/mips/include/linux/errno.h sysroot/x86_64/include/errno.h sysroot/x86_64/include/asm-generic/errno.h sysroot/x86_64/include/asm/errno.h sysroot/x86_64/include/linux/errno.h sysroot/i386/include/errno.h sysroot/i386/include/asm-generic/errno.h sysroot/i386/include/linux/errno.h sysroot/powerpc/include/errno.h sysroot/powerpc/include/asm-generic/errno.h sysroot/powerpc/include/asm/errno.h sysroot/powerpc/include/linux/errno.h > I'll leave it aside for now as I've spent way longer than I hoped on > these series. I could take the previous two patches however. > Ok, let's ignore this one, I will find why sysroot not install well for i386. Thanks, Zhangjin > Thanks, > Willy
Hi, Willy > > On Sat, Aug 05, 2023 at 02:13:11PM +0800, Zhangjin Wu wrote: > > > To avoid pollute the source code tree and avoid mrproper for every > > > architecture switch, the O= argument must be supported. > > > > > > Both IMAGE and .config are from the building directory, let's use > > > objtree instead of srctree for them. > > > > > > If no O= option specified, means building kernel in source code tree, > > > objtree should be srctree in such case. > > > > > > To support relative path, as suggested by Thomas, $(COMMAND_O) is used > > > to pass the O=$(ABSOLUTE_O) to the $(MAKE) commands. > > > > Zhangjin, I cannot get this one to work, regardless of what I try: > > > > $ make -j8 O=$PWD/output-i386 nolibc-test XARCH=i386 CROSS_COMPILE=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux- CC=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux-gcc > > (...) > > CC nolibc-test > > In file included from sysroot/i386/include/stdio.h:14, > > from nolibc-test.c:13: > > sysroot/i386/include/errno.h:10:10: fatal error: asm/errno.h: No such file or directory > > 10 | #include <asm/errno.h> > > | ^~~~~~~~~~~~~ > > compilation terminated. > > > > Willy, I also just reproduced the issue, seems only i386 sysroot has no > asm/errno.h, will check why it is no rightly installed later. > > $ find sysroot/ -name "errno.h" > sysroot/arm/include/errno.h > sysroot/arm/include/asm-generic/errno.h > sysroot/arm/include/asm/errno.h > sysroot/arm/include/linux/errno.h > sysroot/riscv/include/errno.h > sysroot/riscv/include/asm-generic/errno.h > sysroot/riscv/include/asm/errno.h > sysroot/riscv/include/linux/errno.h > sysroot/s390/include/errno.h > sysroot/s390/include/asm-generic/errno.h > sysroot/s390/include/asm/errno.h > sysroot/s390/include/linux/errno.h > sysroot/arm64/include/errno.h > sysroot/arm64/include/asm-generic/errno.h > sysroot/arm64/include/asm/errno.h > sysroot/arm64/include/linux/errno.h > sysroot/mips/include/errno.h > sysroot/mips/include/asm-generic/errno.h > sysroot/mips/include/asm/errno.h > sysroot/mips/include/linux/errno.h > sysroot/x86_64/include/errno.h > sysroot/x86_64/include/asm-generic/errno.h > sysroot/x86_64/include/asm/errno.h > sysroot/x86_64/include/linux/errno.h > sysroot/i386/include/errno.h > sysroot/i386/include/asm-generic/errno.h > sysroot/i386/include/linux/errno.h > sysroot/powerpc/include/errno.h > sysroot/powerpc/include/asm-generic/errno.h > sysroot/powerpc/include/asm/errno.h > sysroot/powerpc/include/linux/errno.h > > > I'll leave it aside for now as I've spent way longer than I hoped on > > these series. I could take the previous two patches however. > > > > Ok, let's ignore this one, I will find why sysroot not install well for i386. > Thanks to Yuan, he have done some testing and have found the root cause, that is mrproper on top-level source code tree is required before installing sysroot, otherwise, the 'generated' headers will not be installed (removed by scripts/Makefile.asm-generic). After mrproper (not with O=out, must on top-level source code tree), the asm/errno.h will be there: ubuntu@linux-lab:/labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc$ find sysroot/ -name "errno.h" sysroot/i386/include/errno.h sysroot/i386/include/asm-generic/errno.h sysroot/i386/include/asm/errno.h --> here it is sysroot/i386/include/linux/errno.h sysroot/x86/include/errno.h sysroot/x86/include/asm-generic/errno.h sysroot/x86/include/asm/errno.h --> here it is sysroot/x86/include/linux/errno.h That also means, to use O=out for run-user, we also need to use O=out for defconfig (and kernel ...) too, otherwise, the top-level source code tree will be polluated. Seems a manual mrproper on top-level source code tree is always required for a new iteration, so, it may be ok to pick this patch with a note on the potential error. Best Regards, Zhangjin > Thanks, > Zhangjin > > > Thanks, > > Willy
Hi Zhangjin and Willy, On 8/26/2023 12:30 PM, Zhangjin Wu wrote: > Hi, Willy > >>> On Sat, Aug 05, 2023 at 02:13:11PM +0800, Zhangjin Wu wrote: >>>> To avoid pollute the source code tree and avoid mrproper for every >>>> architecture switch, the O= argument must be supported. >>>> >>>> Both IMAGE and .config are from the building directory, let's use >>>> objtree instead of srctree for them. >>>> >>>> If no O= option specified, means building kernel in source code tree, >>>> objtree should be srctree in such case. >>>> >>>> To support relative path, as suggested by Thomas, $(COMMAND_O) is used >>>> to pass the O=$(ABSOLUTE_O) to the $(MAKE) commands. >>> Zhangjin, I cannot get this one to work, regardless of what I try: >>> >>> $ make -j8 O=$PWD/output-i386 nolibc-test XARCH=i386 CROSS_COMPILE=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux- CC=/f/tc/nolibc/gcc-12.3.0-nolibc/i386-linux/bin/i386-linux-gcc >>> (...) >>> CC nolibc-test >>> In file included from sysroot/i386/include/stdio.h:14, >>> from nolibc-test.c:13: >>> sysroot/i386/include/errno.h:10:10: fatal error: asm/errno.h: No such file or directory >>> 10 | #include <asm/errno.h> >>> | ^~~~~~~~~~~~~ >>> compilation terminated. >>> >> Willy, I also just reproduced the issue, seems only i386 sysroot has no >> asm/errno.h, will check why it is no rightly installed later. >> >> $ find sysroot/ -name "errno.h" >> sysroot/arm/include/errno.h >> sysroot/arm/include/asm-generic/errno.h >> sysroot/arm/include/asm/errno.h >> sysroot/arm/include/linux/errno.h >> sysroot/riscv/include/errno.h >> sysroot/riscv/include/asm-generic/errno.h >> sysroot/riscv/include/asm/errno.h >> sysroot/riscv/include/linux/errno.h >> sysroot/s390/include/errno.h >> sysroot/s390/include/asm-generic/errno.h >> sysroot/s390/include/asm/errno.h >> sysroot/s390/include/linux/errno.h >> sysroot/arm64/include/errno.h >> sysroot/arm64/include/asm-generic/errno.h >> sysroot/arm64/include/asm/errno.h >> sysroot/arm64/include/linux/errno.h >> sysroot/mips/include/errno.h >> sysroot/mips/include/asm-generic/errno.h >> sysroot/mips/include/asm/errno.h >> sysroot/mips/include/linux/errno.h >> sysroot/x86_64/include/errno.h >> sysroot/x86_64/include/asm-generic/errno.h >> sysroot/x86_64/include/asm/errno.h >> sysroot/x86_64/include/linux/errno.h >> sysroot/i386/include/errno.h >> sysroot/i386/include/asm-generic/errno.h >> sysroot/i386/include/linux/errno.h >> sysroot/powerpc/include/errno.h >> sysroot/powerpc/include/asm-generic/errno.h >> sysroot/powerpc/include/asm/errno.h >> sysroot/powerpc/include/linux/errno.h >> >>> I'll leave it aside for now as I've spent way longer than I hoped on >>> these series. I could take the previous two patches however. >>> >> Ok, let's ignore this one, I will find why sysroot not install well for i386. >> > Thanks to Yuan, he have done some testing and have found the root cause, that > is mrproper on top-level source code tree is required before installing > sysroot, otherwise, the 'generated' headers will not be installed (removed by > scripts/Makefile.asm-generic). More specifically, building kernel will generate 'linux/arch/x86/include/generated/uapi/asm/errno.h', which prevents nolibc-test with O= generate 'output-x86/arch/x86/include/generated/asm/errno.h'. > > After mrproper (not with O=out, must on top-level source code tree), the > asm/errno.h will be there: > > ubuntu@linux-lab:/labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc$ find sysroot/ -name "errno.h" > sysroot/i386/include/errno.h > sysroot/i386/include/asm-generic/errno.h > sysroot/i386/include/asm/errno.h --> here it is > sysroot/i386/include/linux/errno.h > sysroot/x86/include/errno.h > sysroot/x86/include/asm-generic/errno.h > sysroot/x86/include/asm/errno.h --> here it is > sysroot/x86/include/linux/errno.h > > That also means, to use O=out for run-user, we also need to use O=out for > defconfig (and kernel ...) too, otherwise, the top-level source code tree will > be polluated. > > Seems a manual mrproper on top-level source code tree is always required for a > new iteration, so, it may be ok to pick this patch with a note on the potential > error. There are two potential solutions for addressing this issue. The first option involves copying 'linux/arch/x86/include/generated/uapi/asm/' to 'sysroot/x86/include/' during the make of target headers_standalone. The second approach entails displaying an error message when the code tree is not clean and prompting users to manually execute 'make mrproper'. In this case, we can utilize the target 'outputmakefile' from the root makefile directly. Willy, which method do you prefer? Do you have any alternative suggestions? :) Besides. using x86_64-linux-gnu- won't reproduce this issue. It searched and found 'asm/errno.h' in '/usr/include/x86_64-linux-gnu', whereas 'x86_64-linux' lacks these libraries. > > Best Regards, > Zhangjin > >> Thanks, >> Zhangjin >> >>> Thanks, >>> Willy
Hi, Yuan > Hi Zhangjin and Willy, > > On 8/26/2023 12:30 PM, Zhangjin Wu wrote: [...] > > > > ubuntu@linux-lab:/labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc$ find sysroot/ -name "errno.h" > > sysroot/i386/include/errno.h > > sysroot/i386/include/asm-generic/errno.h > > sysroot/i386/include/asm/errno.h --> here it is > > sysroot/i386/include/linux/errno.h > > sysroot/x86/include/errno.h > > sysroot/x86/include/asm-generic/errno.h > > sysroot/x86/include/asm/errno.h --> here it is > > sysroot/x86/include/linux/errno.h > > > > That also means, to use O=out for run-user, we also need to use O=out for > > defconfig (and kernel ...) too, otherwise, the top-level source code tree will > > be polluated. > > > > Seems a manual mrproper on top-level source code tree is always required for a > > new iteration, so, it may be ok to pick this patch with a note on the potential > > error. > > There are two potential solutions for addressing this issue. > > The first option involves copying > 'linux/arch/x86/include/generated/uapi/asm/' to 'sysroot/x86/include/' > during the make of target headers_standalone. > Will this work when linux/arch/x86/include/generated/uapi/asm/ is stale? for example, when linux/arch/x86/include/generated/uapi/asm/ is there for v6.1, but we are installing v6.5 sysroot? > The second approach entails displaying an error message when the code > tree is not clean and prompting users to manually execute 'make > mrproper'. In this case, we can utilize the target 'outputmakefile' from > the root makefile directly. > A question is, without O= option, will this 'outputmakefile' always require to do mrproper (when we have already done defconfig, kernel or run without O=)? but currently, seems we don't require to do so. Currently, the asm/errno.h is installed whenever the source code tree is clean or not, the issue is asm/errno.h will not be installed only when O= specified. The root solution may be the generic-y (or generated-y?) in scripts/Makefile.asm-generic should be corrected to work with O= option? but this common code want us to be more careful. > Willy, which method do you prefer? Do you have any alternative > suggestions? :) > > > Besides. using x86_64-linux-gnu- won't reproduce this issue. It searched > and found 'asm/errno.h' in '/usr/include/x86_64-linux-gnu', whereas > 'x86_64-linux' lacks these libraries. > So, the -gnu- toolchain may hide some issues in scripts/Makefile.asm-generic for O= option. Thanks, Zhangjin
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 51fef5e6a152..20797ba5d393 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -9,6 +9,9 @@ ifeq ($(srctree),) srctree := $(patsubst %/tools/testing/selftests/,%,$(dir $(CURDIR))) endif +# add objtree for O= option, required by IMAGE and .config +objtree ?= $(srctree) + ifeq ($(ARCH),) include $(srctree)/scripts/subarch.include ARCH = $(SUBARCH) @@ -52,7 +55,7 @@ IMAGE_ppc64le = arch/powerpc/boot/zImage IMAGE_riscv = arch/riscv/boot/Image IMAGE_s390 = arch/s390/boot/bzImage IMAGE_loongarch = arch/loongarch/boot/vmlinuz.efi -IMAGE = $(IMAGE_$(XARCH)) +IMAGE = $(objtree)/$(IMAGE_$(XARCH)) IMAGE_NAME = $(notdir $(IMAGE)) # CROSS_COMPILE: cross toolchain prefix by architecture @@ -173,7 +176,7 @@ sysroot: sysroot/$(ARCH)/include sysroot/$(ARCH)/include: $(Q)rm -rf sysroot/$(ARCH) sysroot/sysroot $(QUIET_MKDIR)mkdir -p sysroot - $(Q)$(MAKE) -C ../../../include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone + $(Q)$(MAKE) -C ../../../include/nolibc $(COMMAND_O) ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone $(Q)mv sysroot/sysroot sysroot/$(ARCH) ifneq ($(NOLIBC_SYSROOT),0) @@ -210,19 +213,19 @@ initramfs: nolibc-test $(Q)cp nolibc-test initramfs/init defconfig: - $(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare + $(Q)$(MAKE) -C $(srctree) $(COMMAND_O) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare kernel: initramfs - $(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs + $(Q)$(MAKE) -C $(srctree) $(COMMAND_O) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs # run the tests after building the kernel run: kernel - $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out" + $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out" $(Q)$(REPORT) $(CURDIR)/run.out # re-run the tests from an existing kernel rerun: - $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out" + $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out" $(Q)$(REPORT) $(CURDIR)/run.out # report with existing test log
To avoid pollute the source code tree and avoid mrproper for every architecture switch, the O= argument must be supported. Both IMAGE and .config are from the building directory, let's use objtree instead of srctree for them. If no O= option specified, means building kernel in source code tree, objtree should be srctree in such case. To support relative path, as suggested by Thomas, $(COMMAND_O) is used to pass the O=$(ABSOLUTE_O) to the $(MAKE) commands. Suggested-by: Willy Tarreau <w@1wt.eu> Link: https://lore.kernel.org/lkml/ZK0AB1OXH1s2xYsh@1wt.eu/ Suggested-by: Thomas Weißschuh <linux@weissschuh.net> Link: https://lore.kernel.org/lkml/058a264d-45bd-4f1f-8af3-56ed337b3251@t-8ch.de/ Link: https://lore.kernel.org/lkml/500504f6-fed1-45a4-a518-4631a6f3e463@t-8ch.de/ Signed-off-by: Zhangjin Wu <falcon@tinylab.org> --- tools/testing/selftests/nolibc/Makefile | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)