| Message ID | 20200614144341.1077495-2-masahiroy@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/2] kbuild: improve cc-option to clean up all temporary files | expand |
On Sun, Jun 14, 2020 at 11:43:41PM +0900, Masahiro Yamada wrote: > cc-option and as-option are almost the same; both pass the flag to > $(CC). The main difference is the cc-option stops before the assemble > stage (-S option) whereas as-option stops after it (-c option). > > I chose -S because it is slightly faster, but $(cc-option,-gz=zlib) > returns a wrong result (https://lkml.org/lkml/2020/6/9/1529). > It has been fixed by a separate patch, but using -c is more robust. > > However, you cannot simply replace -S with -c because the following > code would break: > > depends on $(cc-option,-gsplit-dwarf) > > The combination of -c and -gsplit-dwarf does not accept /dev/null as > output. > > $ cat /dev/null | gcc -gsplit-dwarf -S -x c - -o /dev/null > $ echo $? > 0 > > $ cat /dev/null | gcc -gsplit-dwarf -c -x c - -o /dev/null > objcopy: Warning: '/dev/null' is not an ordinary file > $ echo $? > 1 > > $ cat /dev/null | gcc -gsplit-dwarf -c -x c - -o tmp.o > $ echo $? > 0 > > There is another flag that creates an separate file based on the > object file path: > > $ cat /dev/null | gcc -ftest-coverage -c -x c - -o /dev/null > <stdin>:1: error: cannot open /dev/null.gcno > > So, we cannot use /dev/null to sink the output. > > Align the cc-option implementation with scripts/Kbuild.include. > > With -c option used in cc-option, as-option is unneeded. > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > arch/arm64/Kconfig | 2 +- > lib/Kconfig.debug | 1 - > scripts/Kconfig.include | 8 +------- > 3 files changed, 2 insertions(+), 9 deletions(-) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 31380da53689..6eb18f45258e 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -1564,7 +1564,7 @@ config CC_HAS_SIGN_RETURN_ADDRESS > def_bool $(cc-option,-msign-return-address=all) > > config AS_HAS_PAC > - def_bool $(as-option,-Wa$(comma)-march=armv8.3-a) > + def_bool $(cc-option,-Wa$(comma)-march=armv8.3-a) For this arm64 part: Acked-by: Will Deacon <will@kernel.org> Will
Hi, So... I realized it's been a while: On Sun, 2020-06-14 at 23:43 +0900, Masahiro Yamada wrote: > cc-option and as-option are almost the same; both pass the flag to > $(CC). The main difference is the cc-option stops before the assemble > stage (-S option) whereas as-option stops after it (-c option). > But, I had noticed for a while now that M= build for an out-of-tree driver were causing some trouble. Not really completely "out-of-tree" but rather backported (https://backports.wiki.kernel.org/). And then I finally narrowed it down to this commit, specifically this: > # Return y if the compiler supports <flag>, n otherwise > -cc-option = $(success,$(CC) -Werror $(CLANG_FLAGS) $(1) -S -x c /dev/null -o /dev/null) > +cc-option = $(success,mkdir .tmp_$$$$; trap "rm -rf .tmp_$$$$" EXIT; $(CC) -Werror $(CLANG_FLAGS) $(1) -c -x c /dev/null -o .tmp_$$$$/tmp.o) What happens is that we're doing make -C /path/to/kernel M=/path/to/driver But /path/to/kernel may be the installed distro kernel headers, and thus not be writable to the user doing the driver compile. Obviously, the user may need to 'sudo' anyway to install the result, but if just test- compiling, or even as better practice to not run everything as root, this ".tmp_$$" dir cannot be created. IOW, this broke compiler option detection when KBUILD_EXTMOD=/M= is used. It seems this is still supported (documented in kbuild docs), so I'm kind of hoping it could be fixed? But OTOH, I really don't know how, perhaps just using "mktemp -d" here instead of the hardcoded temp dir? Thanks, johannes
On Wed, May 5, 2021 at 5:17 AM Johannes Berg <johannes@sipsolutions.net> wrote: > > Hi, > > So... I realized it's been a while: > > On Sun, 2020-06-14 at 23:43 +0900, Masahiro Yamada wrote: > > cc-option and as-option are almost the same; both pass the flag to > > $(CC). The main difference is the cc-option stops before the assemble > > stage (-S option) whereas as-option stops after it (-c option). > > > > But, I had noticed for a while now that M= build for an out-of-tree > driver were causing some trouble. Not really completely "out-of-tree" > but rather backported (https://backports.wiki.kernel.org/). > > And then I finally narrowed it down to this commit, specifically this: > > > # Return y if the compiler supports <flag>, n otherwise > > -cc-option = $(success,$(CC) -Werror $(CLANG_FLAGS) $(1) -S -x c /dev/null -o /dev/null) > > +cc-option = $(success,mkdir .tmp_$$$$; trap "rm -rf .tmp_$$$$" EXIT; $(CC) -Werror $(CLANG_FLAGS) $(1) -c -x c /dev/null -o .tmp_$$$$/tmp.o) > > What happens is that we're doing > > make -C /path/to/kernel M=/path/to/driver > > But /path/to/kernel may be the installed distro kernel headers, and thus > not be writable to the user doing the driver compile. Obviously, the > user may need to 'sudo' anyway to install the result, but if just test- > compiling, or even as better practice to not run everything as root, > this ".tmp_$$" dir cannot be created. > > IOW, this broke compiler option detection when KBUILD_EXTMOD=/M= is > used. It seems this is still supported (documented in kbuild docs), so > I'm kind of hoping it could be fixed? But OTOH, I really don't know how, > perhaps just using "mktemp -d" here instead of the hardcoded temp dir? > > Thanks, > johannes > - This commit touches scripts/Kconfig.include. - External module builds (M= builds) never invoke Kconfig Putting these two together, your claim is really odd. If external module builds invoke Kconfig, your kernel is already broken. If you claim this is an issue, please describe how to reproduce it in *upstream* kernel. I do not know (or care about) your backport kernel. -- Best Regards Masahiro Yamada
On Wed, 2021-05-05 at 05:46 +0900, Masahiro Yamada wrote: > > - This commit touches scripts/Kconfig.include. > - External module builds (M= builds) never invoke Kconfig > > Putting these two together, your claim is really odd. Hmm. > If external module builds invoke Kconfig, > your kernel is already broken. Well, it's not about the kernel, that's just the normal upstream (or perhaps distribution) kernel. Anyway, you're right, it's much simpler. The problem isn't invoking M= or something like that, that happens much later and sent me on the completely wrong track. The problem is simply doing make kernelversion to determine the version of a tree that's not writable to the user, e.g. make -C /lib/modules/$(uname -r)/build/ kernelversion Which basically also means that it's harmless, since the version is of course not affected by cc-option. johannes
On Wed, May 5, 2021 at 5:52 AM Johannes Berg <johannes@sipsolutions.net> wrote: > > On Wed, 2021-05-05 at 05:46 +0900, Masahiro Yamada wrote: > > > > - This commit touches scripts/Kconfig.include. > > - External module builds (M= builds) never invoke Kconfig > > > > Putting these two together, your claim is really odd. > > Hmm. > > > If external module builds invoke Kconfig, > > your kernel is already broken. > > Well, it's not about the kernel, that's just the normal upstream (or > perhaps distribution) kernel. > > Anyway, you're right, it's much simpler. The problem isn't invoking M= > or something like that, that happens much later and sent me on the > completely wrong track. > > The problem is simply doing > > make kernelversion > > to determine the version of a tree that's not writable to the user, e.g. > > make -C /lib/modules/$(uname -r)/build/ kernelversion > > Which basically also means that it's harmless, since the version is of > course not affected by cc-option. It was fixed. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=805b2e1d427aab4bb27fa7c51ebb9db7547551b1 If you want to make it work without that commit, make -C /lib/modules/$(uname -r)/build/ kernelversion M=/tmp will work. Pass a writable directory to M=. M= build never touches the kernel source tree. > johannes >
On Wed, 2021-05-05 at 06:05 +0900, Masahiro Yamada wrote: > > It was fixed. > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=805b2e1d427aab4bb27fa7c51ebb9db7547551b1 > Ah, ok, great. > > > If you want to make it work without that commit, > > make -C /lib/modules/$(uname -r)/build/ kernelversion M=/tmp > > will work. I guess that works. I was just going to ignore the errors now that I realized what it was :) Thanks! johannes
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 31380da53689..6eb18f45258e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1564,7 +1564,7 @@ config CC_HAS_SIGN_RETURN_ADDRESS def_bool $(cc-option,-msign-return-address=all) config AS_HAS_PAC - def_bool $(as-option,-Wa$(comma)-march=armv8.3-a) + def_bool $(cc-option,-Wa$(comma)-march=armv8.3-a) config AS_HAS_CFI_NEGATE_RA_STATE def_bool $(as-instr,.cfi_startproc\n.cfi_negate_ra_state\n.cfi_endproc\n) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 96999d4d2dda..9ad9210d70a1 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -229,7 +229,6 @@ config DEBUG_INFO_COMPRESSED bool "Compressed debugging information" depends on DEBUG_INFO depends on $(cc-option,-gz=zlib) - depends on $(as-option,-gz=zlib) depends on $(ld-option,--compress-debug-sections=zlib) help Compress the debug information using zlib. Requires GCC 5.0+ or Clang diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include index c264da2b9b30..a5fe72c504ff 100644 --- a/scripts/Kconfig.include +++ b/scripts/Kconfig.include @@ -25,18 +25,12 @@ failure = $(if-success,$(1),n,y) # $(cc-option,<flag>) # Return y if the compiler supports <flag>, n otherwise -cc-option = $(success,$(CC) -Werror $(CLANG_FLAGS) $(1) -S -x c /dev/null -o /dev/null) +cc-option = $(success,mkdir .tmp_$$$$; trap "rm -rf .tmp_$$$$" EXIT; $(CC) -Werror $(CLANG_FLAGS) $(1) -c -x c /dev/null -o .tmp_$$$$/tmp.o) # $(ld-option,<flag>) # Return y if the linker supports <flag>, n otherwise ld-option = $(success,$(LD) -v $(1)) -# $(as-option,<flag>) -# /dev/zero is used as output instead of /dev/null as some assembler cribs when -# both input and output are same. Also both of them have same write behaviour so -# can be easily substituted. -as-option = $(success, $(CC) $(CLANG_FLAGS) $(1) -c -x assembler /dev/null -o /dev/zero) - # $(as-instr,<instr>) # Return y if the assembler supports <instr>, n otherwise as-instr = $(success,printf "%b\n" "$(1)" | $(CC) $(CLANG_FLAGS) -c -x assembler -o /dev/null -)
cc-option and as-option are almost the same; both pass the flag to $(CC). The main difference is the cc-option stops before the assemble stage (-S option) whereas as-option stops after it (-c option). I chose -S because it is slightly faster, but $(cc-option,-gz=zlib) returns a wrong result (https://lkml.org/lkml/2020/6/9/1529). It has been fixed by a separate patch, but using -c is more robust. However, you cannot simply replace -S with -c because the following code would break: depends on $(cc-option,-gsplit-dwarf) The combination of -c and -gsplit-dwarf does not accept /dev/null as output. $ cat /dev/null | gcc -gsplit-dwarf -S -x c - -o /dev/null $ echo $? 0 $ cat /dev/null | gcc -gsplit-dwarf -c -x c - -o /dev/null objcopy: Warning: '/dev/null' is not an ordinary file $ echo $? 1 $ cat /dev/null | gcc -gsplit-dwarf -c -x c - -o tmp.o $ echo $? 0 There is another flag that creates an separate file based on the object file path: $ cat /dev/null | gcc -ftest-coverage -c -x c - -o /dev/null <stdin>:1: error: cannot open /dev/null.gcno So, we cannot use /dev/null to sink the output. Align the cc-option implementation with scripts/Kbuild.include. With -c option used in cc-option, as-option is unneeded. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- arch/arm64/Kconfig | 2 +- lib/Kconfig.debug | 1 - scripts/Kconfig.include | 8 +------- 3 files changed, 2 insertions(+), 9 deletions(-)