[v4,00/23] Makefile: dependency fixes, make noop runtime ~1.4x faster

Message

Ævar Arnfjörð Bjarmason Nov. 17, 2021, 10:19 a.m. UTC

As noted in v2[1] this series fixes various dependency issues in the
Makfile. See [2] for before/after benchmark numbers.

This v4 addresses small issues Mike Hommey noted, and clarifies
various questions that came up with updated commit messages.

There's also a fix-for-the-fix here in removing the "hook.c"
dependency on hook-list.h in 19/23.

1. https://lore.kernel.org/git/[e-mail]/
2. https://lore.kernel.org/git/[e-mail]/

Ævar Arnfjörð Bjarmason (23):
  Makefile: don't invoke msgfmt with --statistics
  Makefile: don't set up "perl/build" rules under NO_PERL=Y
  Makefile: use "=" not ":=" for po/* and perl/*
  Makefile: clean perl/build/ even with NO_PERL=Y
  Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR
  Makefile: guard Perl-only variable assignments
  Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"
  Makefile: adjust Perl-related comments & whitespace
  Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph
  Makefile: create a GIT-PYTHON-DEFINES, like "PERL"
  Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts
  Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
  Makefile: move $(comma), $(empty) and $(space) to shared.mak
  Makefile: re-add and use the "shellquote" macros
  Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...}
  Makefile: add "$(QUIET)" boilerplate to shared.mak
  Makefile: use $(wspfx) for $(QUIET...) in shared.mak
  Makefiles: add and use wildcard "mkdir -p" template
  Makefile: correct the dependency graph of hook-list.h
  Makefile: use $(file) I/O instead of "FORCE" when possible
  Makefile: disable GNU make built-in wildcard rules
  Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
  Makefile: move ".SUFFIXES" rule to shared.mak

 .gitignore             |   2 +-
 Documentation/Makefile |  71 +---------
 Makefile               | 314 +++++++++++++++--------------------------
 config.mak.uname       |   1 -
 git-cvsserver.perl     |   6 +-
 git-send-email.perl    |   7 +-
 git-svn.perl           |   2 +-
 shared.mak             | 187 ++++++++++++++++++++++++
 t/Makefile             |  34 ++---
 templates/Makefile     |  19 +--
 10 files changed, 337 insertions(+), 306 deletions(-)
 create mode 100644 shared.mak

Range-diff against v3:
 1:  1621ca72c1d =  1:  1621ca72c1d Makefile: don't invoke msgfmt with --statistics
 2:  b7c36c9fea0 =  2:  b7c36c9fea0 Makefile: don't set up "perl/build" rules under NO_PERL=Y
 3:  510499d18ba !  3:  29b000eb0f1 Makefile: use "=" not ":=" for po/* and perl/*
    @@ Commit message
         think I copied some POC code), and in 2017 I used the 2011 commit for
         reference.
     
    +    This doesn't make much if any of a practical difference, doing this is
    +    cheap either way, but as simply-expanded variables in our Makefile
    +    generally indicate special behavior (e.g. making a copy now, and
    +    modifying the RHS later) let's change these to show that nothing odd
    +    is going on here).
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <[e-mail]>
     
      ## Makefile ##
 4:  37f3591bcca =  4:  daead5ec293 Makefile: clean perl/build/ even with NO_PERL=Y
 5:  e38c90ad0b6 !  5:  3c987590740 Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR
    @@ Commit message
         file that'll be used as a dependency for other files, as in this case
         for GIT-PERL-HEADER.
     
    +    We have had a hard dependency on .DELETE_ON_ERROR since
    +    7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR" flag,
    +    2021-06-29), so this is a pure cleanup as a follow-up to that
    +    commit. Support for the ".DELETE_ON_ERROR" target itself is much older
    +    than any GNU make version we support, it was added to GNU make in
    +    1994.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <[e-mail]>
     
      ## Makefile ##
 6:  98e14c7eba9 =  6:  b57f582ccd3 Makefile: guard Perl-only variable assignments
 7:  047a42b01cf =  7:  fcdee92f64c Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"
 8:  0c0a3de390e =  8:  1e25b532ca2 Makefile: adjust Perl-related comments & whitespace
 9:  1ece3160915 =  9:  77d9855bfcf Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph
10:  e9b61cd0ba5 = 10:  6004cdcd8d9 Makefile: create a GIT-PYTHON-DEFINES, like "PERL"
11:  b020f8e3257 = 11:  17b30e96057 Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts
12:  19539ce7d2d = 12:  30ddf7da2c8 Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
13:  6c9291c2c9f = 13:  f378a7dc35e Makefile: move $(comma), $(empty) and $(space) to shared.mak
14:  e811a907b08 = 14:  13cbb851d32 Makefile: re-add and use the "shellquote" macros
15:  fac30fe8b56 ! 15:  337953e4994 Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...}
    @@ shared.mak: shelldquote = '"$(call shdq,$(call shq,$(1)))"'
     +$(1): FORCE
     +	@FLAGS='$$($(2))'; \
     +	if ! test -f $(1) ; then \
    -+		echo $(wspfx_sq) "$(1) PARAMETERS (new)" $@; \
    ++		echo $(wspfx_sq) "$(1) PARAMETERS (new)"; \
     +		echo "$$$$FLAGS" >$(1); \
     +	elif test x"$$$$FLAGS" != x"`cat $(1) 2>/dev/null`" ; then \
    -+		echo $(wspfx_sq) "$(1) PARAMETERS (changed)" $@; \
    ++		echo $(wspfx_sq) "$(1) PARAMETERS (changed)"; \
     +		echo "$$$$FLAGS" >$(1); \
     +	fi
     +endef
16:  a3e3acea82d = 16:  5bb597c1993 Makefile: add "$(QUIET)" boilerplate to shared.mak
17:  22264f431c8 = 17:  3c4d0589667 Makefile: use $(wspfx) for $(QUIET...) in shared.mak
18:  d61e2b44f68 = 18:  be5882b2c99 Makefiles: add and use wildcard "mkdir -p" template
19:  234b4eb613c ! 19:  2710f8af6cd Makefile: correct the dependency graph of hook-list.h
    @@ Commit message
         inadvertently made to depend on hook-list.h, but it's used by
         builtin/bugreport.c.
     
    +    The hook.c also does not depend on hook-list.h. It did in an earlier
    +    version of the greater series cfe853e66be was extracted from, but not
    +    anymore. We might end up needing that line again, but let's remove it
    +    for now.
    +
         Reported-by: Mike Hommey <[e-mail]>
         Signed-off-by: Ævar Arnfjörð Bjarmason <[e-mail]>
     
      ## Makefile ##
     @@ Makefile: git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
    + 		$(filter %.o,$^) $(LIBS)
      
      help.sp help.s help.o: command-list.h
    - hook.sp hook.s hook.o: hook-list.h
    +-hook.sp hook.s hook.o: hook-list.h
     +builtin/bugreport.sp builtin/bugreport.s builtin/bugreport.o: hook-list.h
      
     -builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
20:  567ad5c3ebc = 20:  59f22a0269a Makefile: use $(file) I/O instead of "FORCE" when possible
21:  cb3ae5ce00b ! 21:  dd569a59c74 Makefile: disable GNU make built-in wildcard rules
    @@ Commit message
         benchmark command shows (under --show-output) that we went from ~7716
         syscalls to ~7519, mostly a reduction in [l]stat().
     
    +    We could also invoke make with "-r" by setting "MAKEFLAGS = -r" early,
    +    adding a "-r" variant to the above benchmark shows that it may be 1.01
    +    or so faster (but in my tests, always with a much bigger error
    +    bar). But doing so is a much bigger hammer, since it will disable all
    +    built-in rules, some (all?) of which can be seen with:
    +
    +        make -f/dev/null -p | grep -v -e ^# -e ^$
    +
    +    We may have something that relies on them, so let's go for the more
    +    isolated optimization here that gives us most or all of the wins.
    +
         1. https://lists.gnu.org/archive/html/help-make/2002-11/msg00063.html
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <[e-mail]>
22:  88cfc946b37 = 22:  4168a7e3b30 Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
23:  276e226f0a8 = 23:  48a3927d972 Makefile: move ".SUFFIXES" rule to shared.mak