| Message ID | 20240612044234.212156-1-npiggin@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [kvm-unit-tests] build: retain intermediate .aux.o targets | expand |
On Wed, Jun 12, 2024 at 02:42 PM +1000, Nicholas Piggin <npiggin@gmail.com> wrote: > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > in dependency chains that cause them to be made as intermediate files, > which get removed when make finishes. This results in unnecessary > partial rebuilds. If make is run again, this time the .aux.o targets > are not intermediate, possibly due to being made via different > dependencies. > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > the rebuild problem. > > s390x does not have the problem because .SECONDARY prevents dependancies > from being built as intermediate. However the same change is made for > s390x, for consistency. > > Suggested-by: Marc Hartmayer <mhartmay@linux.ibm.com> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- […snip…] > @@ -85,7 +85,7 @@ CFLAGS += -fno-delete-null-pointer-checks > LDFLAGS += -Wl,--build-id=none > > # We want to keep intermediate files > -.PRECIOUS: %.o %.lds > +.PRECIOUS: %.o %.aux.o %.lds > > asm-offsets = lib/$(ARCH)/asm-offsets.h > include $(SRCDIR)/scripts/asm-offsets.mak > -- > 2.45.1 > > Thanks for fixing this! Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
On Wed, Jun 12, 2024 at 02:42:32PM GMT, Nicholas Piggin wrote: > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > in dependency chains that cause them to be made as intermediate files, > which get removed when make finishes. This results in unnecessary > partial rebuilds. If make is run again, this time the .aux.o targets > are not intermediate, possibly due to being made via different > dependencies. > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > the rebuild problem. > > s390x does not have the problem because .SECONDARY prevents dependancies > from being built as intermediate. However the same change is made for > s390x, for consistency. > > Suggested-by: Marc Hartmayer <mhartmay@linux.ibm.com> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > arm/Makefile.common | 2 +- > powerpc/Makefile.common | 2 +- > riscv/Makefile | 2 +- > s390x/Makefile | 2 +- > 4 files changed, 4 insertions(+), 4 deletions(-) > Acked-by: Andrew Jones <andrew.jones@linux.dev>
On Wed, Jun 12, 2024 at 02:42:32PM +1000, Nicholas Piggin wrote: > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > in dependency chains that cause them to be made as intermediate files, > which get removed when make finishes. This results in unnecessary > partial rebuilds. If make is run again, this time the .aux.o targets > are not intermediate, possibly due to being made via different > dependencies. > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > the rebuild problem. > > s390x does not have the problem because .SECONDARY prevents dependancies > from being built as intermediate. However the same change is made for > s390x, for consistency. This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well? Segher
On Wed Jun 12, 2024 at 6:28 PM AEST, Segher Boessenkool wrote: > On Wed, Jun 12, 2024 at 02:42:32PM +1000, Nicholas Piggin wrote: > > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > > in dependency chains that cause them to be made as intermediate files, > > which get removed when make finishes. This results in unnecessary > > partial rebuilds. If make is run again, this time the .aux.o targets > > are not intermediate, possibly due to being made via different > > dependencies. > > > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > > the rebuild problem. > > > > s390x does not have the problem because .SECONDARY prevents dependancies > > from being built as intermediate. However the same change is made for > > s390x, for consistency. > > This is exactly what .SECONDARY is for, as its documentation says, > even. Wouldn't it be better to just add a .SECONDARY to the other > targets as well? Yeah we were debating that and agreed .PRECIOUS may not be the cleanest fix but since we already use that it's okay for a minimal fix. Thanks, Nick
On Fri, Jun 14, 2024 at 10:43:39AM +1000, Nicholas Piggin wrote: > On Wed Jun 12, 2024 at 6:28 PM AEST, Segher Boessenkool wrote: > > On Wed, Jun 12, 2024 at 02:42:32PM +1000, Nicholas Piggin wrote: > > > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > > > in dependency chains that cause them to be made as intermediate files, > > > which get removed when make finishes. This results in unnecessary > > > partial rebuilds. If make is run again, this time the .aux.o targets > > > are not intermediate, possibly due to being made via different > > > dependencies. > > > > > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > > > the rebuild problem. > > > > > > s390x does not have the problem because .SECONDARY prevents dependancies > > > from being built as intermediate. However the same change is made for > > > s390x, for consistency. > > > > This is exactly what .SECONDARY is for, as its documentation says, > > even. Wouldn't it be better to just add a .SECONDARY to the other > > targets as well? > > Yeah we were debating that and agreed .PRECIOUS may not be the > cleanest fix but since we already use that it's okay for a > minimal fix. But why add it to s390x then? It is not a fix there at all! Segher
On Fri Jun 14, 2024 at 11:08 AM AEST, Segher Boessenkool wrote: > On Fri, Jun 14, 2024 at 10:43:39AM +1000, Nicholas Piggin wrote: > > On Wed Jun 12, 2024 at 6:28 PM AEST, Segher Boessenkool wrote: > > > On Wed, Jun 12, 2024 at 02:42:32PM +1000, Nicholas Piggin wrote: > > > > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > > > > in dependency chains that cause them to be made as intermediate files, > > > > which get removed when make finishes. This results in unnecessary > > > > partial rebuilds. If make is run again, this time the .aux.o targets > > > > are not intermediate, possibly due to being made via different > > > > dependencies. > > > > > > > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > > > > the rebuild problem. > > > > > > > > s390x does not have the problem because .SECONDARY prevents dependancies > > > > from being built as intermediate. However the same change is made for > > > > s390x, for consistency. > > > > > > This is exactly what .SECONDARY is for, as its documentation says, > > > even. Wouldn't it be better to just add a .SECONDARY to the other > > > targets as well? > > > > Yeah we were debating that and agreed .PRECIOUS may not be the > > cleanest fix but since we already use that it's okay for a > > minimal fix. > > But why add it to s390x then? It is not a fix there at all! Eh, not a big deal. I mentioned that in the changelog it doesn't seem to pracicaly fix something. And I rather the makefiles converge as much as possible rather than diverge more. .SECONDARY was added independently and not to fix this problem in s390x. And s390x has .SECONDARY slightly wrong AFAIKS. It mentions .SECONDARY: twice in a way that looks like it was meant to depend on specific targets, it actually gives it no dependencies and the resulting semantics are that all intermediate files in the build are treated as secondary. So somethig there should be cleaned up. If the .SECONDARY was changed to only depend on the .gobj and .hdr.obj then suddenly that would break .aux.o if I don't make the change. So I'm meaning to work out what to do with all that, i.e., whether to add blanket .SECONDARY for all and trim or remove the .PRECIOUS files, or remove s390x's secondary, or make it more specific, or something else. But it takes a while for me to do makefile work. Thanks, Nick
On Fri Jun 14, 2024 at 6:38 PM AEST, Nicholas Piggin wrote: > On Fri Jun 14, 2024 at 11:08 AM AEST, Segher Boessenkool wrote: > > On Fri, Jun 14, 2024 at 10:43:39AM +1000, Nicholas Piggin wrote: > > > On Wed Jun 12, 2024 at 6:28 PM AEST, Segher Boessenkool wrote: > > > > On Wed, Jun 12, 2024 at 02:42:32PM +1000, Nicholas Piggin wrote: > > > > > arm, powerpc, riscv, build .aux.o targets with implicit pattern rules > > > > > in dependency chains that cause them to be made as intermediate files, > > > > > which get removed when make finishes. This results in unnecessary > > > > > partial rebuilds. If make is run again, this time the .aux.o targets > > > > > are not intermediate, possibly due to being made via different > > > > > dependencies. > > > > > > > > > > Adding .aux.o files to .PRECIOUS prevents them being removed and solves > > > > > the rebuild problem. > > > > > > > > > > s390x does not have the problem because .SECONDARY prevents dependancies > > > > > from being built as intermediate. However the same change is made for > > > > > s390x, for consistency. > > > > > > > > This is exactly what .SECONDARY is for, as its documentation says, > > > > even. Wouldn't it be better to just add a .SECONDARY to the other > > > > targets as well? > > > > > > Yeah we were debating that and agreed .PRECIOUS may not be the > > > cleanest fix but since we already use that it's okay for a > > > minimal fix. > > > > But why add it to s390x then? It is not a fix there at all! > > Eh, not a big deal. I mentioned that in the changelog it doesn't seem to > pracicaly fix something. And I rather the makefiles converge as much as > possible rather than diverge more. > > .SECONDARY was added independently and not to fix this problem in > s390x. And s390x has .SECONDARY slightly wrong AFAIKS. It mentions > .SECONDARY: twice in a way that looks like it was meant to depend on > specific targets, it actually gives it no dependencies and the > resulting semantics are that all intermediate files in the build are > treated as secondary. So somethig there should be cleaned up. If the > .SECONDARY was changed to only depend on the .gobj and .hdr.obj then > suddenly that would break .aux.o if I don't make the change. > > So I'm meaning to work out what to do with all that, i.e., whether to > add blanket .SECONDARY for all and trim or remove the .PRECIOUS files, > or remove s390x's secondary, or make it more specific, or something > else. But it takes a while for me to do makefile work. Hi Thomas, Ping on this patch? Thanks, Nick
On 26/07/2024 06.15, Nicholas Piggin wrote: > On Fri Jun 14, 2024 at 6:38 PM AEST, Nicholas Piggin wrote: >> On Fri Jun 14, 2024 at 11:08 AM AEST, Segher Boessenkool wrote: >>> On Fri, Jun 14, 2024 at 10:43:39AM +1000, Nicholas Piggin wrote: >>>> On Wed Jun 12, 2024 at 6:28 PM AEST, Segher Boessenkool wrote: >>>>> On Wed, Jun 12, 2024 at 02:42:32PM +1000, Nicholas Piggin wrote: >>>>>> arm, powerpc, riscv, build .aux.o targets with implicit pattern rules >>>>>> in dependency chains that cause them to be made as intermediate files, >>>>>> which get removed when make finishes. This results in unnecessary >>>>>> partial rebuilds. If make is run again, this time the .aux.o targets >>>>>> are not intermediate, possibly due to being made via different >>>>>> dependencies. >>>>>> >>>>>> Adding .aux.o files to .PRECIOUS prevents them being removed and solves >>>>>> the rebuild problem. >>>>>> >>>>>> s390x does not have the problem because .SECONDARY prevents dependancies >>>>>> from being built as intermediate. However the same change is made for >>>>>> s390x, for consistency. >>>>> >>>>> This is exactly what .SECONDARY is for, as its documentation says, >>>>> even. Wouldn't it be better to just add a .SECONDARY to the other >>>>> targets as well? >>>> >>>> Yeah we were debating that and agreed .PRECIOUS may not be the >>>> cleanest fix but since we already use that it's okay for a >>>> minimal fix. >>> >>> But why add it to s390x then? It is not a fix there at all! >> >> Eh, not a big deal. I mentioned that in the changelog it doesn't seem to >> pracicaly fix something. And I rather the makefiles converge as much as >> possible rather than diverge more. >> >> .SECONDARY was added independently and not to fix this problem in >> s390x. And s390x has .SECONDARY slightly wrong AFAIKS. It mentions >> .SECONDARY: twice in a way that looks like it was meant to depend on >> specific targets, it actually gives it no dependencies and the >> resulting semantics are that all intermediate files in the build are >> treated as secondary. So somethig there should be cleaned up. If the >> .SECONDARY was changed to only depend on the .gobj and .hdr.obj then >> suddenly that would break .aux.o if I don't make the change. >> >> So I'm meaning to work out what to do with all that, i.e., whether to >> add blanket .SECONDARY for all and trim or remove the .PRECIOUS files, >> or remove s390x's secondary, or make it more specific, or something >> else. But it takes a while for me to do makefile work. > > Hi Thomas, > > Ping on this patch? I assumed that Marc would chime in on the s390x part here? Thomas
diff --git a/arm/Makefile.common b/arm/Makefile.common index f828dbe01..0b26a92a6 100644 --- a/arm/Makefile.common +++ b/arm/Makefile.common @@ -31,7 +31,7 @@ CFLAGS += -O2 CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib # We want to keep intermediate files -.PRECIOUS: %.elf %.o +.PRECIOUS: %.elf %.o %.aux.o asm-offsets = lib/$(ARCH)/asm-offsets.h include $(SRCDIR)/scripts/asm-offsets.mak diff --git a/powerpc/Makefile.common b/powerpc/Makefile.common index b98f71c2f..16f14577e 100644 --- a/powerpc/Makefile.common +++ b/powerpc/Makefile.common @@ -30,7 +30,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib CFLAGS += -Wa,-mregnames # We want to keep intermediate files -.PRECIOUS: %.o +.PRECIOUS: %.o %.aux.o asm-offsets = lib/$(ARCH)/asm-offsets.h include $(SRCDIR)/scripts/asm-offsets.mak diff --git a/riscv/Makefile b/riscv/Makefile index 919a3ebb5..7207ff988 100644 --- a/riscv/Makefile +++ b/riscv/Makefile @@ -53,7 +53,7 @@ AUXFLAGS ?= 0x0 KEEP_FRAME_POINTER := y # We want to keep intermediate files -.PRECIOUS: %.elf %.o +.PRECIOUS: %.elf %.o %.aux.o define arch_elf_check = $(if $(shell ! $(READELF) -rW $(1) >&/dev/null && echo "nok"), diff --git a/s390x/Makefile b/s390x/Makefile index 23342bd64..d436c6e9a 100644 --- a/s390x/Makefile +++ b/s390x/Makefile @@ -85,7 +85,7 @@ CFLAGS += -fno-delete-null-pointer-checks LDFLAGS += -Wl,--build-id=none # We want to keep intermediate files -.PRECIOUS: %.o %.lds +.PRECIOUS: %.o %.aux.o %.lds asm-offsets = lib/$(ARCH)/asm-offsets.h include $(SRCDIR)/scripts/asm-offsets.mak
arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies. Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem. s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency. Suggested-by: Marc Hartmayer <mhartmay@linux.ibm.com> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arm/Makefile.common | 2 +- powerpc/Makefile.common | 2 +- riscv/Makefile | 2 +- s390x/Makefile | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)