diff mbox series

[v4,2/3] tools build: Avoid circular .fixdep-in.o.cmd issues

Message ID 20240715203325.3832977-3-briannorris@chromium.org (mailing list archive)
State Not Applicable
Headers show
Series tools build: Incorrect fixdep dependencies | expand

Checks

Context Check Description
netdev/tree_selection success Not a local patch, async
bpf/vmtest-bpf-next-VM_Test-2 success Logs for Unittests
bpf/vmtest-bpf-next-VM_Test-3 success Logs for Validate matrix.py
bpf/vmtest-bpf-next-VM_Test-0 success Logs for Lint
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-5 success Logs for aarch64-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-10 success Logs for aarch64-gcc / veristat
bpf/vmtest-bpf-next-VM_Test-12 success Logs for s390x-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-4 success Logs for aarch64-gcc / build / build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-9 success Logs for aarch64-gcc / test (test_verifier, false, 360) / test_verifier on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-11 success Logs for s390x-gcc / build / build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-13 fail Logs for s390x-gcc / test (test_maps, false, 360) / test_maps on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-7 success Logs for aarch64-gcc / test (test_progs, false, 360) / test_progs on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-6 success Logs for aarch64-gcc / test (test_maps, false, 360) / test_maps on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-15 success Logs for s390x-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-18 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-16 success Logs for s390x-gcc / test (test_verifier, false, 360) / test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-8 success Logs for aarch64-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-20 success Logs for x86_64-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-28 success Logs for x86_64-llvm-17 / build / build for x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-17 success Logs for s390x-gcc / veristat
bpf/vmtest-bpf-next-VM_Test-29 success Logs for x86_64-llvm-17 / build-release / build for x86_64 with llvm-17-O2
bpf/vmtest-bpf-next-VM_Test-19 success Logs for x86_64-gcc / build / build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-34 success Logs for x86_64-llvm-17 / veristat
bpf/vmtest-bpf-next-VM_Test-35 success Logs for x86_64-llvm-18 / build / build for x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-36 success Logs for x86_64-llvm-18 / build-release / build for x86_64 with llvm-18-O2
bpf/vmtest-bpf-next-VM_Test-42 success Logs for x86_64-llvm-18 / veristat
bpf/vmtest-bpf-next-VM_Test-32 success Logs for x86_64-llvm-17 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-27 success Logs for x86_64-gcc / veristat / veristat on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-24 success Logs for x86_64-gcc / test (test_progs_no_alu32_parallel, true, 30) / test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-31 success Logs for x86_64-llvm-17 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-25 success Logs for x86_64-gcc / test (test_progs_parallel, true, 30) / test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-23 success Logs for x86_64-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-22 success Logs for x86_64-gcc / test (test_progs, false, 360) / test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-26 success Logs for x86_64-gcc / test (test_verifier, false, 360) / test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-30 success Logs for x86_64-llvm-17 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-21 success Logs for x86_64-gcc / test (test_maps, false, 360) / test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-33 success Logs for x86_64-llvm-17 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-37 success Logs for x86_64-llvm-18 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-39 success Logs for x86_64-llvm-18 / test (test_progs_cpuv4, false, 360) / test_progs_cpuv4 on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-38 success Logs for x86_64-llvm-18 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-40 success Logs for x86_64-llvm-18 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-41 success Logs for x86_64-llvm-18 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-14 success Logs for s390x-gcc / test (test_progs, false, 360) / test_progs on s390x with gcc
bpf/vmtest-bpf-next-PR fail PR summary

Commit Message

Brian Norris July 15, 2024, 8:32 p.m. UTC 
The 'fixdep' tool is used to post-process dependency files for various
reasons, and it runs after every object file generation command. This
even includes 'fixdep' itself.

In Kbuild, this isn't actually a problem, because it uses a single
command to generate fixdep (a compile-and-link command on fixdep.c), and
afterward runs the fixdep command on the accompanying .fixdep.cmd file.

In tools/ builds (which notably is maintained separately from Kbuild),
fixdep is generated in several phases:

 1. fixdep.c -> fixdep-in.o
 2. fixdep-in.o -> fixdep

Thus, fixdep is not available in the post-processing for step 1, and
instead, we generate .cmd files that look like:

  ## from tools/objtool/libsubcmd/.fixdep.o.cmd
  # cannot find fixdep (/path/to/linux/tools/objtool/libsubcmd//fixdep)
  [...]

These invalid .cmd files are benign in some respects, but cause problems
in others (such as the linked reports).

Because the tools/ build system is rather complicated in its own right
(and pointedly different than Kbuild), I choose to simply open-code the
rule for building fixdep, and avoid the recursive-make indirection that
produces the problem in the first place.

Link: https://lore.kernel.org/all/Zk-C5Eg84yt6_nml@google.com/
Signed-off-by: Brian Norris <briannorris@chromium.org>
---

(no changes since v3)

Changes in v3:
 - Drop unnecessary tools/build/Build

 tools/build/Build    |  3 ---
 tools/build/Makefile | 11 ++---------
 2 files changed, 2 insertions(+), 12 deletions(-)
 delete mode 100644 tools/build/Build

Comments

Jiri Olsa July 16, 2024, 7:55 a.m. UTC | #1 
On Mon, Jul 15, 2024 at 01:32:43PM -0700, Brian Norris wrote:
> The 'fixdep' tool is used to post-process dependency files for various
> reasons, and it runs after every object file generation command. This
> even includes 'fixdep' itself.
> 
> In Kbuild, this isn't actually a problem, because it uses a single
> command to generate fixdep (a compile-and-link command on fixdep.c), and
> afterward runs the fixdep command on the accompanying .fixdep.cmd file.
> 
> In tools/ builds (which notably is maintained separately from Kbuild),
> fixdep is generated in several phases:
> 
>  1. fixdep.c -> fixdep-in.o
>  2. fixdep-in.o -> fixdep
> 
> Thus, fixdep is not available in the post-processing for step 1, and
> instead, we generate .cmd files that look like:
> 
>   ## from tools/objtool/libsubcmd/.fixdep.o.cmd
>   # cannot find fixdep (/path/to/linux/tools/objtool/libsubcmd//fixdep)
>   [...]
> 
> These invalid .cmd files are benign in some respects, but cause problems
> in others (such as the linked reports).
> 
> Because the tools/ build system is rather complicated in its own right
> (and pointedly different than Kbuild), I choose to simply open-code the
> rule for building fixdep, and avoid the recursive-make indirection that
> produces the problem in the first place.
> 
> Link: https://lore.kernel.org/all/Zk-C5Eg84yt6_nml@google.com/
> Signed-off-by: Brian Norris <briannorris@chromium.org>
> ---
> 
> (no changes since v3)
> 
> Changes in v3:
>  - Drop unnecessary tools/build/Build

Acked-by: Jiri Olsa <jolsa@kernel.org>

so usually Arnaldo takes changes for tools/build, Arnaldo, could you please take a look?
but still there'are the tools/lib/bpf bits..

thanks,
jirka

> 
>  tools/build/Build    |  3 ---
>  tools/build/Makefile | 11 ++---------
>  2 files changed, 2 insertions(+), 12 deletions(-)
>  delete mode 100644 tools/build/Build
> 
> diff --git a/tools/build/Build b/tools/build/Build
> deleted file mode 100644
> index 76d1a4960973..000000000000
> --- a/tools/build/Build
> +++ /dev/null
> @@ -1,3 +0,0 @@
> -hostprogs := fixdep
> -
> -fixdep-y := fixdep.o
> diff --git a/tools/build/Makefile b/tools/build/Makefile
> index 17cdf01e29a0..fea3cf647f5b 100644
> --- a/tools/build/Makefile
> +++ b/tools/build/Makefile
> @@ -43,12 +43,5 @@ ifneq ($(wildcard $(TMP_O)),)
>  	$(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
>  endif
>  
> -$(OUTPUT)fixdep-in.o: FORCE
> -	$(Q)$(MAKE) $(build)=fixdep
> -
> -$(OUTPUT)fixdep: $(OUTPUT)fixdep-in.o
> -	$(QUIET_LINK)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
> -
> -FORCE:
> -
> -.PHONY: FORCE
> +$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
> +	$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
> -- 
> 2.45.2.993.g49e7a77208-goog
> 
>
Andrii Nakryiko July 19, 2024, 6:32 p.m. UTC | #2 
On Tue, Jul 16, 2024 at 12:55 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Mon, Jul 15, 2024 at 01:32:43PM -0700, Brian Norris wrote:
> > The 'fixdep' tool is used to post-process dependency files for various
> > reasons, and it runs after every object file generation command. This
> > even includes 'fixdep' itself.
> >
> > In Kbuild, this isn't actually a problem, because it uses a single
> > command to generate fixdep (a compile-and-link command on fixdep.c), and
> > afterward runs the fixdep command on the accompanying .fixdep.cmd file.
> >
> > In tools/ builds (which notably is maintained separately from Kbuild),
> > fixdep is generated in several phases:
> >
> >  1. fixdep.c -> fixdep-in.o
> >  2. fixdep-in.o -> fixdep
> >
> > Thus, fixdep is not available in the post-processing for step 1, and
> > instead, we generate .cmd files that look like:
> >
> >   ## from tools/objtool/libsubcmd/.fixdep.o.cmd
> >   # cannot find fixdep (/path/to/linux/tools/objtool/libsubcmd//fixdep)
> >   [...]
> >
> > These invalid .cmd files are benign in some respects, but cause problems
> > in others (such as the linked reports).
> >
> > Because the tools/ build system is rather complicated in its own right
> > (and pointedly different than Kbuild), I choose to simply open-code the
> > rule for building fixdep, and avoid the recursive-make indirection that
> > produces the problem in the first place.
> >
> > Link: https://lore.kernel.org/all/Zk-C5Eg84yt6_nml@google.com/
> > Signed-off-by: Brian Norris <briannorris@chromium.org>
> > ---
> >
> > (no changes since v3)
> >
> > Changes in v3:
> >  - Drop unnecessary tools/build/Build
>
> Acked-by: Jiri Olsa <jolsa@kernel.org>
>
> so usually Arnaldo takes changes for tools/build, Arnaldo, could you please take a look?
> but still there'are the tools/lib/bpf bits..

I think it should be fine for libbpf bits to go through Arnaldo's tree
and get back to bpf-next eventually. Unlikely that we'll have any
conflict in libbpf's Makefile specifically, we rarely change it.

>
> thanks,
> jirka
>
> >
> >  tools/build/Build    |  3 ---
> >  tools/build/Makefile | 11 ++---------
> >  2 files changed, 2 insertions(+), 12 deletions(-)
> >  delete mode 100644 tools/build/Build
> >
> > diff --git a/tools/build/Build b/tools/build/Build
> > deleted file mode 100644
> > index 76d1a4960973..000000000000
> > --- a/tools/build/Build
> > +++ /dev/null
> > @@ -1,3 +0,0 @@
> > -hostprogs := fixdep
> > -
> > -fixdep-y := fixdep.o
> > diff --git a/tools/build/Makefile b/tools/build/Makefile
> > index 17cdf01e29a0..fea3cf647f5b 100644
> > --- a/tools/build/Makefile
> > +++ b/tools/build/Makefile
> > @@ -43,12 +43,5 @@ ifneq ($(wildcard $(TMP_O)),)
> >       $(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
> >  endif
> >
> > -$(OUTPUT)fixdep-in.o: FORCE
> > -     $(Q)$(MAKE) $(build)=fixdep
> > -
> > -$(OUTPUT)fixdep: $(OUTPUT)fixdep-in.o
> > -     $(QUIET_LINK)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
> > -
> > -FORCE:
> > -
> > -.PHONY: FORCE
> > +$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
> > +     $(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
> > --
> > 2.45.2.993.g49e7a77208-goog
> >
> >
>
Brian Norris Aug. 2, 2024, 9:17 p.m. UTC | #3 
Hi Arnaldo,

On Tue, Jul 16, 2024 at 09:55:44AM +0200, Jiri Olsa wrote:
> so usually Arnaldo takes changes for tools/build, Arnaldo, could you please take a look?
> but still there'are the tools/lib/bpf bits..

Would you have some time to look at this series and whether it's ready
to be applied? Several folks have already acked one or more patches.

In case you've lost context on the series, here's a lore link for the
cover letter:

  [PATCH v4 0/3] tools build: Incorrect fixdep dependencies
  https://lore.kernel.org/all/20240715203325.3832977-1-briannorris@chromium.org/

Thanks,
Brian
Arnaldo Carvalho de Melo Aug. 5, 2024, 3:31 p.m. UTC | #4 
On Fri, Jul 19, 2024 at 11:32:08AM -0700, Andrii Nakryiko wrote:
> On Tue, Jul 16, 2024 at 12:55 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> >
> > On Mon, Jul 15, 2024 at 01:32:43PM -0700, Brian Norris wrote:
> > > The 'fixdep' tool is used to post-process dependency files for various
> > > reasons, and it runs after every object file generation command. This
> > > even includes 'fixdep' itself.
> > >
> > > In Kbuild, this isn't actually a problem, because it uses a single
> > > command to generate fixdep (a compile-and-link command on fixdep.c), and
> > > afterward runs the fixdep command on the accompanying .fixdep.cmd file.
> > >
> > > In tools/ builds (which notably is maintained separately from Kbuild),
> > > fixdep is generated in several phases:
> > >
> > >  1. fixdep.c -> fixdep-in.o
> > >  2. fixdep-in.o -> fixdep
> > >
> > > Thus, fixdep is not available in the post-processing for step 1, and
> > > instead, we generate .cmd files that look like:
> > >
> > >   ## from tools/objtool/libsubcmd/.fixdep.o.cmd
> > >   # cannot find fixdep (/path/to/linux/tools/objtool/libsubcmd//fixdep)
> > >   [...]
> > >
> > > These invalid .cmd files are benign in some respects, but cause problems
> > > in others (such as the linked reports).
> > >
> > > Because the tools/ build system is rather complicated in its own right
> > > (and pointedly different than Kbuild), I choose to simply open-code the
> > > rule for building fixdep, and avoid the recursive-make indirection that
> > > produces the problem in the first place.
> > >
> > > Link: https://lore.kernel.org/all/Zk-C5Eg84yt6_nml@google.com/
> > > Signed-off-by: Brian Norris <briannorris@chromium.org>
> > > ---
> > >
> > > (no changes since v3)
> > >
> > > Changes in v3:
> > >  - Drop unnecessary tools/build/Build
> >
> > Acked-by: Jiri Olsa <jolsa@kernel.org>
> >
> > so usually Arnaldo takes changes for tools/build, Arnaldo, could you please take a look?
> > but still there'are the tools/lib/bpf bits..
> 
> I think it should be fine for libbpf bits to go through Arnaldo's tree
> and get back to bpf-next eventually. Unlikely that we'll have any
> conflict in libbpf's Makefile specifically, we rarely change it.

I got this series now in perf-tools-next,

Thanks,

- Arnaldo
Thorsten Leemhuis Aug. 12, 2024, 6:32 a.m. UTC | #5 
On 15.07.24 22:32, Brian Norris wrote:
> The 'fixdep' tool is used to post-process dependency files for various
> reasons, and it runs after every object file generation command. This
> even includes 'fixdep' itself.

Lo! TWIMC, this change broke my daily arm64 and x86_64 Fedora vanilla RPM
builds on all Fedora releases when it hit -next a few days ago. Reverting
it fixes the problem.

The problem is related to the RPM magic somehow, as building worked fine
when when I omitted stuff like "-specs=/usr/lib/rpm/redhat/redhat-
hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" from the
make call. So the real problem might be that space somewhere.


This is how the build fails on x86_64:

+ /usr/bin/make -s 'HOSTCFLAGS=-O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection   ' 'HOSTLDFLAGS=-Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes ' ARCH=x86_64 'KCFLAGS= ' WITH_GCOV=0 -j2 bzImage
/usr/bin/ld: /tmp/ccMoR0Wr.o: relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:47: /builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.x86_64/tools/objtool/fixdep] Error 1
make[3]: *** [/builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.x86_64/tools/build/Makefile.include:15: fixdep] Error 2
make[2]: *** [Makefile:73: objtool] Error 2
make[1]: *** [/builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.x86_64/Makefile:1361: tools/objtool] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:226: __sub-make] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.ZQfBFY (%build)


This is how it fails on arm64:

+ /usr/bin/make -s 'HOSTCFLAGS=-O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -mbranch-protection=standard -fasynchronous-unwind-tables -fstack-clash-protection   ' 'HOSTLDFLAGS=-Wl,-z,relro -Wl,--as-needed   -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes ' ARCH=arm64 'KCFLAGS= ' WITH_GCOV=0 -j4 vmlinuz.efi
/usr/bin/ld: /tmp/ccRCv9ot.o: relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `stderr@@GLIBC_2.17' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /tmp/ccRCv9ot.o(.text+0x8): unresolvable R_AARCH64_ADR_PREL_PG_HI21 relocation against symbol `stderr@@GLIBC_2.17'
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:47: /builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.aarch64/tools/bpf/resolve_btfids/fixdep] Error 1
make[3]: *** [/builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.aarch64/tools/build/Makefile.include:15: fixdep] Error 2
make[2]: *** [Makefile:76: bpf/resolve_btfids] Error 2
make[1]: *** [/builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.aarch64/Makefile:1362: tools/bpf/resolve_btfids] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:226: __sub-make] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.60qjsr (%build)


Full logs:
https://download.copr.fedorainfracloud.org/results/@kernel-vanilla/next/fedora-40-x86_64/07900504-next-next-all/builder-live.log.gz
https://download.copr.fedorainfracloud.org/results/@kernel-vanilla/next/fedora-40-aarch64/07900504-next-next-all/builder-live.log.gz

My ppc64le builds worked fine.
 
Ciao, Thorsten

> In Kbuild, this isn't actually a problem, because it uses a single
> command to generate fixdep (a compile-and-link command on fixdep.c), and
> afterward runs the fixdep command on the accompanying .fixdep.cmd file.
> 
> In tools/ builds (which notably is maintained separately from Kbuild),
> fixdep is generated in several phases:
> 
>  1. fixdep.c -> fixdep-in.o
>  2. fixdep-in.o -> fixdep
> 
> Thus, fixdep is not available in the post-processing for step 1, and
> instead, we generate .cmd files that look like:
> 
>   ## from tools/objtool/libsubcmd/.fixdep.o.cmd
>   # cannot find fixdep (/path/to/linux/tools/objtool/libsubcmd//fixdep)
>   [...]
> 
> These invalid .cmd files are benign in some respects, but cause problems
> in others (such as the linked reports).
> 
> Because the tools/ build system is rather complicated in its own right
> (and pointedly different than Kbuild), I choose to simply open-code the
> rule for building fixdep, and avoid the recursive-make indirection that
> produces the problem in the first place.
> 
> Link: https://lore.kernel.org/all/Zk-C5Eg84yt6_nml@google.com/
> Signed-off-by: Brian Norris <briannorris@chromium.org>
> ---
> 
> (no changes since v3)
> 
> Changes in v3:
>  - Drop unnecessary tools/build/Build
> 
>  tools/build/Build    |  3 ---
>  tools/build/Makefile | 11 ++---------
>  2 files changed, 2 insertions(+), 12 deletions(-)
>  delete mode 100644 tools/build/Build
> 
> diff --git a/tools/build/Build b/tools/build/Build
> deleted file mode 100644
> index 76d1a4960973..000000000000
> --- a/tools/build/Build
> +++ /dev/null
> @@ -1,3 +0,0 @@
> -hostprogs := fixdep
> -
> -fixdep-y := fixdep.o
> diff --git a/tools/build/Makefile b/tools/build/Makefile
> index 17cdf01e29a0..fea3cf647f5b 100644
> --- a/tools/build/Makefile
> +++ b/tools/build/Makefile
> @@ -43,12 +43,5 @@ ifneq ($(wildcard $(TMP_O)),)
>  	$(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
>  endif
>  
> -$(OUTPUT)fixdep-in.o: FORCE
> -	$(Q)$(MAKE) $(build)=fixdep
> -
> -$(OUTPUT)fixdep: $(OUTPUT)fixdep-in.o
> -	$(QUIET_LINK)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
> -
> -FORCE:
> -
> -.PHONY: FORCE
> +$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
> +	$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
Brian Norris Aug. 13, 2024, 4:40 p.m. UTC | #6 
Hi Thorsten,

On Mon, Aug 12, 2024 at 08:32:29AM +0200, Thorsten Leemhuis wrote:
> Lo! TWIMC, this change broke my daily arm64 and x86_64 Fedora vanilla RPM
> builds on all Fedora releases when it hit -next a few days ago. Reverting
> it fixes the problem.
> 
> The problem is related to the RPM magic somehow, as building worked fine
> when when I omitted stuff like "-specs=/usr/lib/rpm/redhat/redhat-
> hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" from the
> make call. So the real problem might be that space somewhere.
> 
> 
> This is how the build fails on x86_64:
> 
> + /usr/bin/make -s 'HOSTCFLAGS=-O2  -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection   ' 'HOSTLDFLAGS=-Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes ' ARCH=x86_64 'KCFLAGS= ' WITH_GCOV=0 -j2 bzImage
> /usr/bin/ld: /tmp/ccMoR0Wr.o: relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
> /usr/bin/ld: failed to set dynamic section sizes: bad value
> collect2: error: ld returned 1 exit status
> make[4]: *** [Makefile:47: /builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.x86_64/tools/objtool/fixdep] Error 1
> make[3]: *** [/builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.x86_64/tools/build/Makefile.include:15: fixdep] Error 2
> make[2]: *** [Makefile:73: objtool] Error 2
> make[1]: *** [/builddir/build/BUILD/kernel-next-20240812/linux-6.11.0-0.0.next.20240812.329.vanilla.fc40.x86_64/Makefile:1361: tools/objtool] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:226: __sub-make] Error 2
> error: Bad exit status from /var/tmp/rpm-tmp.ZQfBFY (%build)

I don't have a Fedora installation on hand at the moment, and the logs
don't seem to include most of the actual kernel build logs
(stdout+stderr of a V=1 build might help), but I think what you've
provided so far has highlighted one possible problem -- that the new
one-shot compile+link is ignoring HOSTCFLAGS, which were previously
respected via tools/build/Build.include. Could you try the following
diff? I'll cook a proper patch and description later, but for now:

--- a/tools/build/Makefile
+++ b/tools/build/Makefile
@@ -44,4 +44,4 @@ ifneq ($(wildcard $(TMP_O)),)
 endif
 
 $(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
-	$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
+	$(QUIET_CC)$(HOSTCC) $(HOSTCFLAGS) $(KBUILD_HOSTLDFLAGS) -o $@ $<
Thorsten Leemhuis Aug. 13, 2024, 4:59 p.m. UTC | #7 
On 13.08.24 18:40, Brian Norris wrote:
> On Mon, Aug 12, 2024 at 08:32:29AM +0200, Thorsten Leemhuis wrote:
>> Lo! TWIMC, this change broke my daily arm64 and x86_64 Fedora vanilla RPM
>> builds on all Fedora releases when it hit -next a few days ago. Reverting
>> it fixes the problem.
>>
>> The problem is related to the RPM magic somehow, as building worked fine
>> when when I omitted stuff like "-specs=/usr/lib/rpm/redhat/redhat-
>> hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" from the
>> make call. So the real problem might be that space somewhere.
> [...]
> I don't have a Fedora installation on hand at the moment, and the logs
> don't seem to include most of the actual kernel build logs
> (stdout+stderr of a V=1 build might help), but I think what you've
> provided so far has highlighted one possible problem -- that the new
> one-shot compile+link is ignoring HOSTCFLAGS, which were previously
> respected via tools/build/Build.include. Could you try the following
> diff? I'll cook a proper patch and description later, but for now:
> 
> --- a/tools/build/Makefile
> +++ b/tools/build/Makefile
> @@ -44,4 +44,4 @@ ifneq ($(wildcard $(TMP_O)),)
>  endif
>  
>  $(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
> -	$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
> +	$(QUIET_CC)$(HOSTCC) $(HOSTCFLAGS) $(KBUILD_HOSTLDFLAGS) -o $@ $<

Many thx for looking into this. Seems that does resolve the problem (I
did not perform a full build, but without this the build fails after a
few seconds, and now it ran through).

I don't care much, but feel free to add a

Tested-by: Thorsten Leemhuis <linux@leemhuis.info>

Thx again!

Ciao, Thorsten
diff mbox series

Patch

diff --git a/tools/build/Build b/tools/build/Build
deleted file mode 100644
index 76d1a4960973..000000000000
--- a/tools/build/Build
+++ /dev/null
@@ -1,3 +0,0 @@ 
-hostprogs := fixdep
-
-fixdep-y := fixdep.o
diff --git a/tools/build/Makefile b/tools/build/Makefile
index 17cdf01e29a0..fea3cf647f5b 100644
--- a/tools/build/Makefile
+++ b/tools/build/Makefile
@@ -43,12 +43,5 @@  ifneq ($(wildcard $(TMP_O)),)
 	$(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
 endif
 
-$(OUTPUT)fixdep-in.o: FORCE
-	$(Q)$(MAKE) $(build)=fixdep
-
-$(OUTPUT)fixdep: $(OUTPUT)fixdep-in.o
-	$(QUIET_LINK)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
-
-FORCE:
-
-.PHONY: FORCE
+$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
+	$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<