| Message ID | cover.1729254070.git.ps@pks.im (mailing list archive) |
|---|---|
| Headers | show |
| Series | Modernize the build system | expand |
On 18/10/2024 13:23, Patrick Steinhardt wrote: > Hi, > > this is the third version of my patch series that aims to modernize our > build system infrastructure. It refactors our existing build infra to be > ready for out-of-tree builds and then wires up Meson. > > Changes compared to v2: > > - I have split out semi-related changes like the platform > compatibility fixes into separate series. More on that further down > below, where I explain the current base. > > - I have polished the CMake-related changes I've been doing as prereqs > while refactoring our build system. Some of the changes weren't > tested before, but now CMake works as expected. Just curious, does the cmake build system work on Linux now? (cmake is another system that I don't really know, except in a 'monkey-see monkey-do' kinda way) > > - Fixed up some dependencies in our Makefile, as I didn't amend them > to depend on the new generator scripts. > > - I have extracted some more scripts to massage our Perl library code, > shells cripts and gitweb.cgi such that they can be used by Make, > CMake and Meson. This also fixes generation of "gitweb.cgi" on > Meson as reported by Ramsay. I haven't had time to test this yet (I haven't even properly read the patches), but from a quick squint, this looks similar to my attempts to do the same thing the other night (minus the cmake stuff, of course). However, I didn't notice anything in the commit message about changing the format of the GITWEB-BUILD-OPTIONS file, which already exists before that commit. I did the same, btw, but it seems that file was only used to 'notice' when the configuration changed to enable the re-build of gitweb.cgi. In which case, the change in format (rather than content) was not an issue. > > - I have fixed the cyclic dependency in our Makefile that I have > introduced with v2. > > - I have addressed some feedback regarding "build-systems.txt". > > - I have made things work with OpenBSD 7.6. > > - Things now work on Cygwin and MinGW64 as expected, no more test > failures except for t9700, which also fails with our Makefile. This caused me some concern for a little while, since I don't recall that the t9700 test has ever failed for me on cygwin. I have the test output files for all versions of git (each rc and rel) from v2.12.0-rc0 (5th Feb 2017) to today (all 143 of them) and none of them show that test failing. (I remove files from that directory on occasion when it takes up too much disk space). However, you qualify that statement in the commit message to make clear that it works on cygwin, but fails on other windows 'targets'. phew! :) > - I have improved how we set up the required tooling on Windows such > that we detect Git for Windows-provided tools as we do with CMake. > Like this, VS Code can now trivially import the Meson project and > configure it. Furthermore, generation of the Visual Studio solution > is trivial as well. So overall, things work fine now on all the > important Windows environments that I'm aware of. > > - Several smaller fixes all over the place. > > I've included a range-diff, but it's quite crowded overall. > > In general I think that this is in a pretty good shape now. I have > tested things on NixOS, macOS 10.15, OpenBSD 7.6, FreeBSD 14.1, Windows > 10 with MSVC, VS Code, MinGW and Cygwin, and things are generally smooth > and just work. I'm sure there's still going to be weirdnesses on some > platforms regardless of that. I had a very quick attempt on cygwin the other night, and it behaved differently at the 'ninja test' stage. It printed something like '[0/1] all tests' and then seemed to do nothing, except 'python3.9' was hovering up all cycles (well over 75% anyway) with no indication of any progress. This was nothing like the behaviour on Linux (which also looked a bit strange, but at least showed progress on each test as it ran). I ^C-ed it, since I didn't want to wait 6 hours before it (hopefully) finished running. Thanks! ATB, Ramsay Jones
On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > This series is based on 15030f9556 (The second batch, 2024-10-15) and > has the following dependencies: > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > dependencies for generated clar headers, 2024-10-15). > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > generating "clar-decls.h", 2024-10-14). > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > FreeBSD, 2024-10-16). > > These deps are mostly just there to make tests pass on all platforms, > but I wouldn't want folks to go test things and discover breakage that > is not caused by Meson itself :) You can also find this version of the > patch series at [1], branch "pks-meson-v3". @Taylor, I'm fine with > keeping this out of "seen" if you prefer to wait for those dependencies > to land first. Of the three, only the last one is marked as "Will merge to 'next'?" in the next integration round[^1]. All three are in 'seen', so this should apply cleanly on top of those. What is the state of those first two topics? Are they ready to go, or are we expecting a new round for any of them? Thanks, Taylor [^1]: I think I will do another full integration cycle on Tuesday of next week, but I am not entirely sure yet.
On 10/18/24 12:08 PM, Ramsay Jones wrote: > I had a very quick attempt on cygwin the other night, and it behaved > differently at the 'ninja test' stage. It printed something like > '[0/1] all tests' and then seemed to do nothing, except 'python3.9' > was hovering up all cycles (well over 75% anyway) with no indication > of any progress. This was nothing like the behaviour on Linux (which > also looked a bit strange, but at least showed progress on each test > as it ran). > > I ^C-ed it, since I didn't want to wait 6 hours before it (hopefully) > finished running. Running "ninja test" can be thought of as a convenience target to make all commands available via the Makefile equivalent. What it actually does, though, is first depend on "all" + any explicit test dependencies, and then dispatch to the separate "meson test" tool with some tastefully chosen arguments such as "--no-rebuild" (to prevent rerunning ninja) as well as --print-errorlogs. Running "meson test" directly is still possible, and maybe required if you want to do things such as --interactive, or running a specific testcase rather than all of them, or --maxfail, or well, all kinds of fiddly and specific fine-tuning. :) But in particular, I am wondering if maybe your cygwin issue is somehow due to ninja deciding that it really really wanted to buffer all output and wait until it completes before showing anything. It's not supposed to do that, since we wire it up to the "console" pool: https://ninja-build.org/manual.html#_the_literal_console_literal_pool The other possibility is that the python installation has decided to buffer itself: https://docs.python.org/3/using/cmdline.html#cmdoption-u possibly because due to ninja's juggling of pseudo ttys, the python runtime for "meson test" thinks it is *not* running in a tty. I know that ninja messes around with this in counterintuitive ways in general, but I am not sure exactly how that interacts with the console pool. But regardless of the reason -- if it is happening at the python level, then exporting PYTHONUNBUFFERED=1 may help. It's something I've had to do before for programs other than meson, at least.
On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > This series is based on 15030f9556 (The second batch, 2024-10-15) and > has the following dependencies: > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > dependencies for generated clar headers, 2024-10-15). > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > generating "clar-decls.h", 2024-10-14). > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > FreeBSD, 2024-10-16). While this is still sitting on my review queue, I rebuilt this topic based on the above (extremely helpful) information, but it seems to produce some conflicts when I try to merge it in at the end of 'seen': --- 8< --- diff --cc Makefile index 2c1afd08cb,a93aeb3bc8..0000000000 --- a/Makefile +++ b/Makefile @@@ -3932,10 -3859,9 +3886,18 @@@ GIT-TEST-SUITES: FORC fi $(UNIT_TEST_DIR)/clar-decls.h: $(patsubst %,$(UNIT_TEST_DIR)/%.c,$(CLAR_TEST_SUITES)) GIT-TEST-SUITES ++<<<<<<< HEAD + $(QUIET_GEN)$(SHELL_PATH) $(UNIT_TEST_DIR)/generate-clar-decls.sh "$@" $(filter %.c,$^) ++||||||| merged common ancestors ++ $(QUIET_GEN)for suite in $(CLAR_TEST_SUITES); do \ ++ sed -ne "s/^\(void test_$${suite}__[a-zA-Z_0-9][a-zA-Z_0-9]*(void)$$\)/extern \1;/p" $(UNIT_TEST_DIR)/$$suite.c; \ ++ done >$@ ++======= + $(QUIET_GEN)$(SHELL_PATH) $(UNIT_TEST_DIR)/generate-clar-decls.sh "$@" $(patsubst %,$(UNIT_TEST_DIR)/%.c,$(CLAR_TEST_SUITES)) ++>>>>>>> ps/build $(UNIT_TEST_DIR)/clar.suite: $(UNIT_TEST_DIR)/clar-decls.h $(QUIET_GEN)awk -f $(UNIT_TEST_DIR)/clar-generate.awk $< >$(UNIT_TEST_DIR)/clar.suite +$(UNIT_TEST_DIR)/clar/clar.o: $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS): $(UNIT_TEST_DIR)/clar-decls.h $(CLAR_TEST_OBJS): EXTRA_CPPFLAGS = -I$(UNIT_TEST_DIR) $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS diff --cc t/unit-tests/clar/clar.c index d54e455367,64879cf2bd..0000000000 --- a/t/unit-tests/clar/clar.c +++ b/t/unit-tests/clar/clar.c @@@ -812,9 -801,8 +812,17 @@@ void clar__assert_equal } } } ++<<<<<<< HEAD +#endif /* CLAR_HAVE_WCHAR */ else if (!strcmp("%"PRIuMAX, fmt) || !strcmp("%"PRIxMAX, fmt)) { uintmax_t sz1 = va_arg(args, uintmax_t), sz2 = va_arg(args, uintmax_t); ++||||||| merged common ancestors ++ else if (!strcmp("%"PRIuZ, fmt) || !strcmp("%"PRIxZ, fmt)) { ++ size_t sz1 = va_arg(args, size_t), sz2 = va_arg(args, size_t); ++======= ++ else if (!strcmp("%"PRIuMAX, fmt) || !strcmp("%"PRIxMAX, fmt)) { ++ uintmax_t sz1 = va_arg(args, uintmax_t), sz2 = va_arg(args, uintmax_t); ++>>>>>>> ps/build is_equal = (sz1 == sz2); if (!is_equal) { int offset = p_snprintf(buf, sizeof(buf), fmt, sz1); diff --cc t/unit-tests/generate-clar-decls.sh index 688e0885f4,81da732917..0000000000 --- a/t/unit-tests/generate-clar-decls.sh +++ b/t/unit-tests/generate-clar-decls.sh @@@ -1,16 -1,16 +1,36 @@@ ++<<<<<<< HEAD +#!/bin/sh + +if test $# -lt 2 +then + echo "USAGE: $0 <OUTPUT> <SUITE>..." 2>&1 + exit 1 +fi + +OUTPUT="$1" +shift + +for suite in "$@" +do + sed -ne "s/^\(void test_$(basename "${suite%.c}")__[a-zA-Z_0-9][a-zA-Z_0-9]*(void)\)$/extern \1;/p" "$suite" || + exit 1 +done >"$OUTPUT" ++||||||| merged common ancestors ++======= + #!/bin/sh + + if test $# -lt 2 + then + echo "USAGE: $0 <OUTPUT> <SUITE>..." 2>&1 + exit 1 + fi + + OUTPUT="$1" + shift + + for suite in "$@" + do + sed -ne "s/^\(void test_$suite__[a-zA-Z_0-9][a-zA-Z_0-9]*(void)$\)/extern \1;/p" "$suite" || + exit 1 + done >"$OUTPUT" ++>>>>>>> ps/build --- >8 --- Thanks, Taylor
On 20/10/2024 09:51, Eli Schwartz wrote: > On 10/18/24 12:08 PM, Ramsay Jones wrote: [snip] > possibly because due to ninja's juggling of pseudo ttys, the python > runtime for "meson test" thinks it is *not* running in a tty. I know > that ninja messes around with this in counterintuitive ways in general, > but I am not sure exactly how that interacts with the console pool. But > regardless of the reason -- if it is happening at the python level, then > exporting PYTHONUNBUFFERED=1 may help. It's something I've had to do > before for programs other than meson, at least. Unfortunately, the 'PYTHONUNBUFFERED=1' idea didn't work. In fact nothing I have tried on cygwin has worked. For example, if I use meson to run just few tests, like so on Linux: $ meson test --no-rebuild --print-errorlogs 't000*' 1/9 t0000-basic OK 7.52s 2/9 t0001-init OK 1.88s 3/9 t0002-gitfile OK 0.37s 4/9 t0003-attributes OK 1.35s 5/9 t0004-unwritable OK 0.23s 6/9 t0005-signals OK 0.16s 7/9 t0006-date OK 0.73s 8/9 t0007-git-var OK 0.40s 9/9 t0008-ignores OK 3.09s Ok: 9 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 0 Timeout: 0 Full log written to /home/ramsay/git/build/meson-logs/testlog.txt $ On cygwin it looks like: $ meson test --no-rebuild --print-errorlogs 't000*' 1/9 t0002-gitfile OK 12.32s [2-4/9]
On Fri, Oct 18, 2024 at 05:08:41PM +0100, Ramsay Jones wrote: > On 18/10/2024 13:23, Patrick Steinhardt wrote: > > Hi, > > > > this is the third version of my patch series that aims to modernize our > > build system infrastructure. It refactors our existing build infra to be > > ready for out-of-tree builds and then wires up Meson. > > > > Changes compared to v2: > > > > - I have split out semi-related changes like the platform > > compatibility fixes into separate series. More on that further down > > below, where I explain the current base. > > > > - I have polished the CMake-related changes I've been doing as prereqs > > while refactoring our build system. Some of the changes weren't > > tested before, but now CMake works as expected. > > Just curious, does the cmake build system work on Linux now? > > (cmake is another system that I don't really know, except in a > 'monkey-see monkey-do' kinda way) To some extent. Some issues have been fixed and/or improved by this series, but we're not fully there for CMake: - It builds, but the build leaks outside of the build directory into the source tree. In theory we now have all parts together to fix that, as demonstrated by the Meson build support, but I didn't wire this up. - The tests do not run at all for me, might be I'm holding it wrong. But with the patch that allows us to override the GIT-BUILD-DIR we should be able to get out-of-tree tests running, as well. Again, I didn't wire this up. - Some things simply haven't been wired up at all, like building "gitweb.cgi". Any gitweb-tests would thus fail. But in any case, this patch series doesn't regress support for CMake anymore like previous versions did. > > - Fixed up some dependencies in our Makefile, as I didn't amend them > > to depend on the new generator scripts. > > > > - I have extracted some more scripts to massage our Perl library code, > > shells cripts and gitweb.cgi such that they can be used by Make, > > CMake and Meson. This also fixes generation of "gitweb.cgi" on > > Meson as reported by Ramsay. > > I haven't had time to test this yet (I haven't even properly read the > patches), but from a quick squint, this looks similar to my attempts > to do the same thing the other night (minus the cmake stuff, of course). > > However, I didn't notice anything in the commit message about changing > the format of the GITWEB-BUILD-OPTIONS file, which already exists before > that commit. I did the same, btw, but it seems that file was only used to > 'notice' when the configuration changed to enable the re-build of gitweb.cgi. > In which case, the change in format (rather than content) was not an issue. Yeah, it's been a bit weird before. It contained sed commands to execute, but also some other relevant build options. Made this whole thing a bit weird to use. In any case, it works the same way that GIT-BUILD-OPTIONS work now, which I think is a good thing. The plumbing around it is a bit verbose, but other than that I think it's straight-forward. > > > > - I have fixed the cyclic dependency in our Makefile that I have > > introduced with v2. > > > > - I have addressed some feedback regarding "build-systems.txt". > > > > - I have made things work with OpenBSD 7.6. > > > > - Things now work on Cygwin and MinGW64 as expected, no more test > > failures except for t9700, which also fails with our Makefile. > > This caused me some concern for a little while, since I don't recall > that the t9700 test has ever failed for me on cygwin. I have the test > output files for all versions of git (each rc and rel) from v2.12.0-rc0 > (5th Feb 2017) to today (all 143 of them) and none of them show that > test failing. (I remove files from that directory on occasion when it > takes up too much disk space). > > However, you qualify that statement in the commit message to make clear > that it works on cygwin, but fails on other windows 'targets'. phew! :) Oh, yeah, that one seems misleading indeed. > > - I have improved how we set up the required tooling on Windows such > > that we detect Git for Windows-provided tools as we do with CMake. > > Like this, VS Code can now trivially import the Meson project and > > configure it. Furthermore, generation of the Visual Studio solution > > is trivial as well. So overall, things work fine now on all the > > important Windows environments that I'm aware of. > > > > - Several smaller fixes all over the place. > > > > I've included a range-diff, but it's quite crowded overall. > > > > In general I think that this is in a pretty good shape now. I have > > tested things on NixOS, macOS 10.15, OpenBSD 7.6, FreeBSD 14.1, Windows > > 10 with MSVC, VS Code, MinGW and Cygwin, and things are generally smooth > > and just work. I'm sure there's still going to be weirdnesses on some > > platforms regardless of that. > > I had a very quick attempt on cygwin the other night, and it behaved > differently at the 'ninja test' stage. It printed something like > '[0/1] all tests' and then seemed to do nothing, except 'python3.9' > was hovering up all cycles (well over 75% anyway) with no indication > of any progress. This was nothing like the behaviour on Linux (which > also looked a bit strange, but at least showed progress on each test > as it ran). > > I ^C-ed it, since I didn't want to wait 6 hours before it (hopefully) > finished running. Interesting, that's something I didn't experience yet. I'll chime in further down in this thread. Patrick
On Fri, Oct 18, 2024 at 05:09:43PM -0400, Taylor Blau wrote: > On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > > This series is based on 15030f9556 (The second batch, 2024-10-15) and > > has the following dependencies: > > > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > > dependencies for generated clar headers, 2024-10-15). > > > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > > generating "clar-decls.h", 2024-10-14). > > > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > > FreeBSD, 2024-10-16). > > > > These deps are mostly just there to make tests pass on all platforms, > > but I wouldn't want folks to go test things and discover breakage that > > is not caused by Meson itself :) You can also find this version of the > > patch series at [1], branch "pks-meson-v3". @Taylor, I'm fine with > > keeping this out of "seen" if you prefer to wait for those dependencies > > to land first. > > Of the three, only the last one is marked as "Will merge to 'next'?" in > the next integration round[^1]. All three are in 'seen', so this should > apply cleanly on top of those. > > What is the state of those first two topics? Are they ready to go, or > are we expecting a new round for any of them? These have been merged into a single topic now and should be ready to go from my point of view. But they still require reviews. Patrick
On Mon, Oct 21, 2024 at 06:10:10PM -0400, Taylor Blau wrote: > On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > > This series is based on 15030f9556 (The second batch, 2024-10-15) and > > has the following dependencies: > > > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > > dependencies for generated clar headers, 2024-10-15). > > > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > > generating "clar-decls.h", 2024-10-14). > > > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > > FreeBSD, 2024-10-16). > > While this is still sitting on my review queue, I rebuilt this topic > based on the above (extremely helpful) information, but it seems to > produce some conflicts when I try to merge it in at the end of 'seen': Huh, interesting. The first two series had conflicts with each other, so I wouldn't be surprised if what you see comes from that. Those two topics have been merged into a single topic now. I'll send an updated version of this series at the end of this week that also aims to fix conflicts with 'seen'. Patrick
On Mon, Oct 21, 2024 at 11:56:42PM +0100, Ramsay Jones wrote: > On 20/10/2024 09:51, Eli Schwartz wrote: > > On 10/18/24 12:08 PM, Ramsay Jones wrote: > [snip] > > possibly because due to ninja's juggling of pseudo ttys, the python > > runtime for "meson test" thinks it is *not* running in a tty. I know > > that ninja messes around with this in counterintuitive ways in general, > > but I am not sure exactly how that interacts with the console pool. But > > regardless of the reason -- if it is happening at the python level, then > > exporting PYTHONUNBUFFERED=1 may help. It's something I've had to do > > before for programs other than meson, at least. > > Unfortunately, the 'PYTHONUNBUFFERED=1' idea didn't work. In fact nothing > I have tried on cygwin has worked. For example, if I use meson to run just > few tests, like so on Linux: > > $ meson test --no-rebuild --print-errorlogs 't000*' > 1/9 t0000-basic OK 7.52s > 2/9 t0001-init OK 1.88s > 3/9 t0002-gitfile OK 0.37s > 4/9 t0003-attributes OK 1.35s > 5/9 t0004-unwritable OK 0.23s > 6/9 t0005-signals OK 0.16s > 7/9 t0006-date OK 0.73s > 8/9 t0007-git-var OK 0.40s > 9/9 t0008-ignores OK 3.09s > > Ok: 9 > Expected Fail: 0 > Fail: 0 > Unexpected Pass: 0 > Skipped: 0 > Timeout: 0 > > Full log written to /home/ramsay/git/build/meson-logs/testlog.txt > $ > > On cygwin it looks like: > > $ meson test --no-rebuild --print-errorlogs 't000*' > 1/9 t0002-gitfile OK 12.32s > [2-4/9]
On 23/10/2024 13:04, Patrick Steinhardt wrote: > On Mon, Oct 21, 2024 at 11:56:42PM +0100, Ramsay Jones wrote: >> On 20/10/2024 09:51, Eli Schwartz wrote: >>> On 10/18/24 12:08 PM, Ramsay Jones wrote: >> [snip] >> Patrick, how do you run the tests on cygwin? > > I didn't have to do anything special here, so this is quite puzzling. In > a fully-updated Cygwin installation: > > $ which meson > /usr/bin/meson > $ meson --version > 1.3.2 > $ python --version > 3.9.16 > $ mkdir build > $ meson setup .. > ... autoconfiguration logs ... > $ meson test 't000*' Hmm, I could have sworn that this caused a complete rebuild for me (even when having just built), hence the '--no-rebuild --print-errorlogs' parameters to meson test. (Oh, maybe that was just 'meson test' - so how do you run *all* tests with meson - rather than 'ninja test'). > ninja: Entering directory `/home/Patrick Steinhardt/git/build' > [582/582] Linking target git-receive-pack.exe > 1/9 t0005-signals OK 4.17s > 2/9 t0004-unwritable OK 4.58s > 3/9 t0002-gitfile OK 6.95s > 4/9 t0007-git-var OK 8.15s > 5/9 t0006-date OK 15.42s > 6/9 t0003-attributes OK 26.84s > 7/9 t0001-init OK 29.09s > 8/9 t0008-ignores OK 57.17s > 9/9 t0000-basic OK 83.82s > > Ok: 9 > Expected Fail: 0 > Fail: 0 > Unexpected Pass: 0 > Skipped: 0 > Timeout: 0 > > Full log written to /home/Patrick Steinhardt/git/build/meson-logs/testlog.txt > > This is starting with a fresh repo, I executed `git clean -dfx` > beforehand. Hmm, I have far to many 'precious' files to use 'git clean'! (perhaps I should clone the repo to get a totally clean slate). > > Do any of the versions used maybe differ? Ah, yes, I should have noted the versions: $ uname -a CYGWIN_NT-10.0-19045 satellite 3.5.4-1.x86_64 2024-08-25 16:52 UTC x86_64 Cygwin $ $ which meson ninja python /usr/bin/meson /usr/bin/ninja /usr/bin/python $ $ meson --version 1.3.2 $ $ ninja --version 1.12.0 $ $ python --version Python 3.9.18 $ So, a slightly newer python - what versions of ninja and the cygwin '.dll' are you using? (the latter is more likely to cause an issue). BTW, I have been doing: $ meson setup .. -Dprefix=$HOME so that it matches the default prefix from the Makefile (not that I have attempted to actually install yet!;) ). Can the default be set in the meson.build file (with command-line override, of course)? ATB, Ramsay Jones
On 10/23/24 10:38 AM, Ramsay Jones wrote: > Hmm, I could have sworn that this caused a complete rebuild for me > (even when having just built), hence the '--no-rebuild --print-errorlogs' > parameters to meson test. (Oh, maybe that was just 'meson test' - so > how do you run *all* tests with meson - rather than 'ninja test'). "meson test" without --no-rebuild will first run "ninja" to verify that everything has been built, and build it if it hasn't been. When passing specific test names, it will filter for dependencies of that specific testcase, and only ensure *those* are up to date. Assuming those dependencies are accurate, of course. :D What do you mean by "complete rebuild"? Do you mean all binaries that were already built and up to date get marked stale and recreated? > BTW, I have been doing: > > $ meson setup .. -Dprefix=$HOME > > so that it matches the default prefix from the Makefile (not that I > have attempted to actually install yet!;) ). Can the default be set > in the meson.build file (with command-line override, of course)? project('git', 'c', meson_version: '>=1.3.0', # MSVC does not support GNU99, and C99 does not define __STDC_VERSION__ # on MSVC. So we instead fall back to C11 there. default_options: ['c_std=gnu99,c11'], You can pass any command-line options in as default_options, including prefix. Defaulting to $HOME is somewhat unconventional regardless of whether it's done in a Makefile or a configure.ac or a meson.build or a CMakeLists.txt, but given the Makefile does indeed do that I suppose it's not incredibly bizarre to do it in meson.build as well. :)
On Wed, Oct 23, 2024 at 02:04:21PM +0200, Patrick Steinhardt wrote: > On Mon, Oct 21, 2024 at 06:10:10PM -0400, Taylor Blau wrote: > > On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > > > This series is based on 15030f9556 (The second batch, 2024-10-15) and > > > has the following dependencies: > > > > > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > > > dependencies for generated clar headers, 2024-10-15). > > > > > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > > > generating "clar-decls.h", 2024-10-14). > > > > > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > > > FreeBSD, 2024-10-16). > > > > While this is still sitting on my review queue, I rebuilt this topic > > based on the above (extremely helpful) information, but it seems to > > produce some conflicts when I try to merge it in at the end of 'seen': > > Huh, interesting. The first two series had conflicts with each other, so > I wouldn't be surprised if what you see comes from that. Those two > topics have been merged into a single topic now. > > I'll send an updated version of this series at the end of this week that > also aims to fix conflicts with 'seen'. Great, thanks! Thanks, Taylor
On Wed, Oct 23, 2024 at 02:04:17PM +0200, Patrick Steinhardt wrote: > On Fri, Oct 18, 2024 at 05:09:43PM -0400, Taylor Blau wrote: > > On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > > > This series is based on 15030f9556 (The second batch, 2024-10-15) and > > > has the following dependencies: > > > > > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > > > dependencies for generated clar headers, 2024-10-15). > > > > > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > > > generating "clar-decls.h", 2024-10-14). > > > > > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > > > FreeBSD, 2024-10-16). > > > > > > These deps are mostly just there to make tests pass on all platforms, > > > but I wouldn't want folks to go test things and discover breakage that > > > is not caused by Meson itself :) You can also find this version of the > > > patch series at [1], branch "pks-meson-v3". @Taylor, I'm fine with > > > keeping this out of "seen" if you prefer to wait for those dependencies > > > to land first. > > > > Of the three, only the last one is marked as "Will merge to 'next'?" in > > the next integration round[^1]. All three are in 'seen', so this should > > apply cleanly on top of those. > > > > What is the state of those first two topics? Are they ready to go, or > > are we expecting a new round for any of them? > > These have been merged into a single topic now and should be ready to go > from my point of view. But they still require reviews. OK, so: ps/platform-compat-fixes is in 'next', and ps/cmake-clar has been folded into ps/upgrade-clar. Have all three have been combined into ps/build? If so, should this topic be rebuilt on ps/upgrade-clar so that we don't see the same patches twice, perhaps? Or am I still missing something here? Thanks, Taylor
On 23/10/2024 16:58, Eli Schwartz wrote: > On 10/23/24 10:38 AM, Ramsay Jones wrote: >> Hmm, I could have sworn that this caused a complete rebuild for me >> (even when having just built), hence the '--no-rebuild --print-errorlogs' >> parameters to meson test. (Oh, maybe that was just 'meson test' - so >> how do you run *all* tests with meson - rather than 'ninja test'). > > > "meson test" without --no-rebuild will first run "ninja" to verify that > everything has been built, and build it if it hasn't been. When passing > specific test names, it will filter for dependencies of that specific > testcase, and only ensure *those* are up to date. Assuming those > dependencies are accurate, of course. :D Heh, yes, I just tried it tonight, and it did just as you describe: $ meson test ninja: Entering directory `/home/ramsay/new-git/build' ninja: no work to do. $ > What do you mean by "complete rebuild"? Do you mean all binaries that > were already built and up to date get marked stale and recreated? My bad, I was a little loose in my language. Whatever it was that I typed (and I obviously can't remember what it was!), resulted in meson going through the configuration step again followed by compilation/linking of all binaries (or at least, that's what it looked like). >> BTW, I have been doing: >> >> $ meson setup .. -Dprefix=$HOME >> >> so that it matches the default prefix from the Makefile (not that I >> have attempted to actually install yet!;) ). Can the default be set >> in the meson.build file (with command-line override, of course)? > > > project('git', 'c', > meson_version: '>=1.3.0', > # MSVC does not support GNU99, and C99 does not define __STDC_VERSION__ > # on MSVC. So we instead fall back to C11 there. > default_options: ['c_std=gnu99,c11'], Ah yes, this seems to work: $ git diff diff --git a/meson.build b/meson.build index 7c7a59d7fb..23b6f0baa2 100644 --- a/meson.build +++ b/meson.build @@ -2,7 +2,7 @@ project('git', 'c', meson_version: '>=1.3.0', # MSVC does not support GNU99, and C99 does not define __STDC_VERSION__ # on MSVC. So we instead fall back to C11 there. - default_options: ['c_std=gnu99,c11'], + default_options: ['c_std=gnu99,c11', 'prefix=$HOME'], version: 'v2.47.GIT', ) $ $ meson setup .. --reconfigure ... User defined options prefix: /home/ramsay Found ninja-1.12.0 at /usr/bin/ninja Cleaning... 0 files. $ Using the command-line to override also works: $ meson setup .. --reconfigure -Dprefix=/usr/local ... User defined options prefix: /usr/local Found ninja-1.12.0 at /usr/bin/ninja Cleaning... 0 files. $ > You can pass any command-line options in as default_options, including > prefix. Defaulting to $HOME is somewhat unconventional regardless of > whether it's done in a Makefile or a configure.ac or a meson.build or a > CMakeLists.txt, but given the Makefile does indeed do that I suppose > it's not incredibly bizarre to do it in meson.build as well. :) I don't have any strong feelings either way, it's just that people are used to doing a 'make install' and having the git build installed in their home directory (including me). But it is not a huge effort to add the '-Dprefix=$HOME', it simply needs to be documented. I think I need to go away and RTFM at this point and not annoy you (and others) with such noob questions! :) [For me ATM, meson works fine on Linux, but it simply doesn't work at all on cygwin as far as the tests are concerned (and I haven't tried installing yet). Patrick is not seeing any problems, so I would like to understand why we have such different results.] Thanks! ATB, Ramsay Jones
On Thu, Oct 24, 2024 at 02:43:40AM +0100, Ramsay Jones wrote: > > You can pass any command-line options in as default_options, including > > prefix. Defaulting to $HOME is somewhat unconventional regardless of > > whether it's done in a Makefile or a configure.ac or a meson.build or a > > CMakeLists.txt, but given the Makefile does indeed do that I suppose > > it's not incredibly bizarre to do it in meson.build as well. :) > > I don't have any strong feelings either way, it's just that people are > used to doing a 'make install' and having the git build installed in > their home directory (including me). But it is not a huge effort to > add the '-Dprefix=$HOME', it simply needs to be documented. For now I'll leave it as-is, but if people feel strongly about it I don't mind changing the default value. > I think I need to go away and RTFM at this point and not annoy you (and > others) with such noob questions! :) I find your input to be highly valuable, so thanks for testing things! Patrick
On Wed, Oct 23, 2024 at 03:38:18PM +0100, Ramsay Jones wrote: > > Do any of the versions used maybe differ? > > Ah, yes, I should have noted the versions: > > $ uname -a > CYGWIN_NT-10.0-19045 satellite 3.5.4-1.x86_64 2024-08-25 16:52 UTC x86_64 Cygwin I'm on basically the same system: CYGWIN_NT-10.0-19045 DESKTOP-OLQRBOP 3.5.4-1.x86_64 2024-08-25 16:52 UTC x86_64 Cygwin Which makes this even weirder. > $ > > $ which meson ninja python > /usr/bin/meson > /usr/bin/ninja > /usr/bin/python > $ > > $ meson --version > 1.3.2 > $ > > $ ninja --version > 1.12.0 > $ > > $ python --version > Python 3.9.18 > $ Let me try to update Cygwin to the latest versions... huh. Why is your Python version newer than mine? I cannot seem to find this version of Python in any of the mirrors. Oh, wait! I can see that there's a "python3" 3.9.18 available as a testing version. I don't quite understand how this works though, because when I install it I still have Python 3.9.16. Seems like there are two Python packages: "python3", which has the 3.9.18 test version, and then there's "python39", which only has 3.9.16. The first one just seems to be a meta package, so it doesn't contain anything. So... where did you get Python 3.9.18 from? You mentioned that it also hogs all CPU, so could it be that this is basically a broken Python installation? Patrick
On Wed, Oct 23, 2024 at 04:15:12PM -0400, Taylor Blau wrote: > On Wed, Oct 23, 2024 at 02:04:17PM +0200, Patrick Steinhardt wrote: > > On Fri, Oct 18, 2024 at 05:09:43PM -0400, Taylor Blau wrote: > > > On Fri, Oct 18, 2024 at 02:23:47PM +0200, Patrick Steinhardt wrote: > > > > This series is based on 15030f9556 (The second batch, 2024-10-15) and > > > > has the following dependencies: > > > > > > > > - ps/cmake-clar at c6d3e52c91 (cmake: set up proper > > > > dependencies for generated clar headers, 2024-10-15). > > > > > > > > - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for > > > > generating "clar-decls.h", 2024-10-14). > > > > > > > > - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on > > > > FreeBSD, 2024-10-16). > > > > > > > > These deps are mostly just there to make tests pass on all platforms, > > > > but I wouldn't want folks to go test things and discover breakage that > > > > is not caused by Meson itself :) You can also find this version of the > > > > patch series at [1], branch "pks-meson-v3". @Taylor, I'm fine with > > > > keeping this out of "seen" if you prefer to wait for those dependencies > > > > to land first. > > > > > > Of the three, only the last one is marked as "Will merge to 'next'?" in > > > the next integration round[^1]. All three are in 'seen', so this should > > > apply cleanly on top of those. > > > > > > What is the state of those first two topics? Are they ready to go, or > > > are we expecting a new round for any of them? > > > > These have been merged into a single topic now and should be ready to go > > from my point of view. But they still require reviews. > > OK, so: ps/platform-compat-fixes is in 'next', and ps/cmake-clar has > been folded into ps/upgrade-clar. Have all three have been combined into > ps/build? > > If so, should this topic be rebuilt on ps/upgrade-clar so that we don't > see the same patches twice, perhaps? Or am I still missing something > here? No, you don't miss anything here, rebuilding this on top of ps/upgrade-clar plus ps/platform-compat-fixes is exactly the right thing to do. Patrick
Hi, this is the third version of my patch series that aims to modernize our build system infrastructure. It refactors our existing build infra to be ready for out-of-tree builds and then wires up Meson. Changes compared to v2: - I have split out semi-related changes like the platform compatibility fixes into separate series. More on that further down below, where I explain the current base. - I have polished the CMake-related changes I've been doing as prereqs while refactoring our build system. Some of the changes weren't tested before, but now CMake works as expected. - Fixed up some dependencies in our Makefile, as I didn't amend them to depend on the new generator scripts. - I have extracted some more scripts to massage our Perl library code, shells cripts and gitweb.cgi such that they can be used by Make, CMake and Meson. This also fixes generation of "gitweb.cgi" on Meson as reported by Ramsay. - I have fixed the cyclic dependency in our Makefile that I have introduced with v2. - I have addressed some feedback regarding "build-systems.txt". - I have made things work with OpenBSD 7.6. - Things now work on Cygwin and MinGW64 as expected, no more test failures except for t9700, which also fails with our Makefile. - I have improved how we set up the required tooling on Windows such that we detect Git for Windows-provided tools as we do with CMake. Like this, VS Code can now trivially import the Meson project and configure it. Furthermore, generation of the Visual Studio solution is trivial as well. So overall, things work fine now on all the important Windows environments that I'm aware of. - Several smaller fixes all over the place. I've included a range-diff, but it's quite crowded overall. In general I think that this is in a pretty good shape now. I have tested things on NixOS, macOS 10.15, OpenBSD 7.6, FreeBSD 14.1, Windows 10 with MSVC, VS Code, MinGW and Cygwin, and things are generally smooth and just work. I'm sure there's still going to be weirdnesses on some platforms regardless of that. This series is based on 15030f9556 (The second batch, 2024-10-15) and has the following dependencies: - ps/cmake-clar at c6d3e52c91 (cmake: set up proper dependencies for generated clar headers, 2024-10-15). - ps/upgrade-clar at f3501edb0e (Makefile: adjust sed command for generating "clar-decls.h", 2024-10-14). - ps/platform-compat-fixes at 80ebd91b83 (http: fix build error on FreeBSD, 2024-10-16). These deps are mostly just there to make tests pass on all platforms, but I wouldn't want folks to go test things and discover breakage that is not caused by Meson itself :) You can also find this version of the patch series at [1], branch "pks-meson-v3". @Taylor, I'm fine with keeping this out of "seen" if you prefer to wait for those dependencies to land first. Thanks! Patrick [1]: https://gitlab.com/gitlab-org/git.git Patrick Steinhardt (15): Makefile: use common template for GIT-BUILD-OPTIONS Makefile: consistently use @PLACEHOLDER@ to substitute Makefile: consistently use PERL_PATH Makefile: extract script to massage Perl scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Shell scripts Makefile: extract script to generate gitweb.cgi Makefile: refactor GIT-VERSION-GEN to be reusable Makefile: refactor generators to be PWD-independent Makefile: allow "bin-wrappers/" directory to exist Makefile: simplify building of templates t: better support for out-of-tree builds t: allow overriding build dir Documentation: add comparison of build systems Introduce support for the Meson build system .gitignore | 1 - Documentation/CodingGuidelines | 2 +- Documentation/Makefile | 1 + Documentation/technical/build-systems.txt | 224 +++ GIT-BUILD-OPTIONS.in | 47 + GIT-VERSION-GEN | 12 +- Makefile | 208 +-- bin-wrappers/.gitignore | 9 + bin-wrappers/meson.build | 28 + bin-wrappers/wrap-for-bin.sh | 37 + configure.ac | 2 +- contrib/buildsystems/CMakeLists.txt | 218 ++- contrib/completion/meson.build | 8 + contrib/meson.build | 1 + generate-cmdlist.sh | 42 +- generate-configlist.sh | 20 +- generate-hooklist.sh | 15 +- generate-perl.sh | 36 + generate-script.sh | 34 + git-cvsserver.perl | 2 +- git-instaweb.sh | 8 +- git-request-pull.sh | 2 +- git-send-email.perl | 2 +- git-sh-i18n.sh | 6 +- git-sh-setup.sh | 6 +- git-svn.perl | 2 +- gitk-git/po/vi.po | 2 +- gitweb/GITWEB-BUILD-OPTIONS.in | 25 + gitweb/Makefile | 58 +- gitweb/generate-gitweb.sh | 45 + gitweb/gitweb.perl | 44 +- gitweb/meson.build | 63 + meson.build | 1614 +++++++++++++++++ meson_options.txt | 70 + perl/FromCPAN/Mail/meson.build | 7 + perl/FromCPAN/meson.build | 9 + perl/Git/I18N.pm | 6 +- perl/Git/LoadCPAN.pm | 6 +- perl/Git/LoadCPAN/Mail/meson.build | 7 + perl/Git/LoadCPAN/meson.build | 9 + perl/Git/SVN/Memoize/meson.build | 7 + perl/Git/SVN/meson.build | 20 + perl/Git/meson.build | 18 + .../header_templates/fixed_prefix.template.pl | 2 +- .../runtime_prefix.template.pl | 8 +- perl/meson.build | 12 + po/meson.build | 28 + subprojects/.gitignore | 1 + subprojects/curl.wrap | 13 + subprojects/expat.wrap | 13 + subprojects/openssl.wrap | 15 + subprojects/pcre2.wrap | 16 + subprojects/zlib.wrap | 13 + t/helper/meson.build | 91 + t/lib-gettext.sh | 4 +- t/meson.build | 1103 +++++++++++ t/t7609-mergetool--lib.sh | 2 +- t/test-lib.sh | 22 +- templates/Makefile | 39 +- templates/branches-- | 1 - templates/{this--description => description} | 0 .../applypatch-msg.sample} | 0 .../commit-msg.sample} | 0 .../fsmonitor-watchman.sample} | 0 templates/hooks/meson.build | 24 + .../post-update.sample} | 0 .../pre-applypatch.sample} | 0 .../pre-commit.sample} | 0 .../pre-merge-commit.sample} | 0 .../pre-push.sample} | 0 .../pre-rebase.sample} | 0 .../pre-receive.sample} | 0 .../prepare-commit-msg.sample} | 0 .../push-to-checkout.sample} | 0 .../sendemail-validate.sample} | 0 .../update.sample} | 0 templates/{info--exclude => info/exclude} | 0 templates/info/meson.build | 5 + templates/meson.build | 13 + unimplemented.sh | 2 +- wrap-for-bin.sh | 36 - 81 files changed, 4040 insertions(+), 406 deletions(-) create mode 100644 Documentation/technical/build-systems.txt create mode 100644 GIT-BUILD-OPTIONS.in create mode 100644 bin-wrappers/.gitignore create mode 100644 bin-wrappers/meson.build create mode 100755 bin-wrappers/wrap-for-bin.sh create mode 100644 contrib/completion/meson.build create mode 100644 contrib/meson.build create mode 100755 generate-perl.sh create mode 100755 generate-script.sh create mode 100644 gitweb/GITWEB-BUILD-OPTIONS.in create mode 100755 gitweb/generate-gitweb.sh create mode 100644 gitweb/meson.build create mode 100644 meson.build create mode 100644 meson_options.txt create mode 100644 perl/FromCPAN/Mail/meson.build create mode 100644 perl/FromCPAN/meson.build create mode 100644 perl/Git/LoadCPAN/Mail/meson.build create mode 100644 perl/Git/LoadCPAN/meson.build create mode 100644 perl/Git/SVN/Memoize/meson.build create mode 100644 perl/Git/SVN/meson.build create mode 100644 perl/Git/meson.build create mode 100644 perl/meson.build create mode 100644 po/meson.build create mode 100644 subprojects/.gitignore create mode 100644 subprojects/curl.wrap create mode 100644 subprojects/expat.wrap create mode 100644 subprojects/openssl.wrap create mode 100644 subprojects/pcre2.wrap create mode 100644 subprojects/zlib.wrap create mode 100644 t/helper/meson.build create mode 100644 t/meson.build delete mode 100644 templates/branches-- rename templates/{this--description => description} (100%) rename templates/{hooks--applypatch-msg.sample => hooks/applypatch-msg.sample} (100%) rename templates/{hooks--commit-msg.sample => hooks/commit-msg.sample} (100%) rename templates/{hooks--fsmonitor-watchman.sample => hooks/fsmonitor-watchman.sample} (100%) create mode 100644 templates/hooks/meson.build rename templates/{hooks--post-update.sample => hooks/post-update.sample} (100%) rename templates/{hooks--pre-applypatch.sample => hooks/pre-applypatch.sample} (100%) rename templates/{hooks--pre-commit.sample => hooks/pre-commit.sample} (100%) rename templates/{hooks--pre-merge-commit.sample => hooks/pre-merge-commit.sample} (100%) rename templates/{hooks--pre-push.sample => hooks/pre-push.sample} (100%) rename templates/{hooks--pre-rebase.sample => hooks/pre-rebase.sample} (100%) rename templates/{hooks--pre-receive.sample => hooks/pre-receive.sample} (100%) rename templates/{hooks--prepare-commit-msg.sample => hooks/prepare-commit-msg.sample} (100%) rename templates/{hooks--push-to-checkout.sample => hooks/push-to-checkout.sample} (100%) rename templates/{hooks--sendemail-validate.sample => hooks/sendemail-validate.sample} (100%) rename templates/{hooks--update.sample => hooks/update.sample} (100%) rename templates/{info--exclude => info/exclude} (100%) create mode 100644 templates/info/meson.build create mode 100644 templates/meson.build delete mode 100644 wrap-for-bin.sh Range-diff against v2: 1: 6dd59db9b2b < -: ----------- t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE 2: 5ec91ac12da < -: ----------- t/test-lib: wire up NO_ICONV prerequisite 3: 94664e2cc06 < -: ----------- t/lib-gitweb: test against the build version of gitweb 4: ffb661dad71 < -: ----------- t/lib-gpg: fix setup of GNUPGHOME in MinGW 5: 958cf3a6b57 < -: ----------- t1401: make invocation of tar(1) work with Win32-provided one 6: bada3a82fdd < -: ----------- t3404: work around platform-specific behaviour on macOS 10.15 7: b4bdefe9e34 < -: ----------- t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin 8: 48ce19d876f < -: ----------- t7300: work around platform-specific behaviour with long paths on MinGW 9: 73b4362164a < -: ----------- t/unit-tests: update clar unit test framework 10: 161f6b063af < -: ----------- builtin/credential-cache: fix missing parameter for stub function 11: e4ef4bdf1c5 < -: ----------- http: fix build error on FreeBSD 12: 6ea97bc92f2 < -: ----------- Makefile: extract script to generate clar declarations 16: dbf9e39e3df = 1: 800fb080f45 Makefile: use common template for GIT-BUILD-OPTIONS 17: 97b9789dd68 ! 2: 1a4a23c66ca Makefile: consistently use @PLACEHOLDER@ to substitute @@ Makefile: GIT-SCRIPT-DEFINES: FORCE $@.sh >$@+ endef +@@ Makefile: $(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE + -e ' r GIT-PERL-HEADER' \ + -e ' G' \ + -e '}' \ +- -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \ ++ -e 's/@GIT_VERSION@/$(GIT_VERSION)/g' \ + $< >$@+ && \ + chmod +x $@+ && \ + mv $@+ $@ @@ Makefile: GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES Makefile INSTLIBDIR='$(perllibdir_SQ)' && \ INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ @@ contrib/buildsystems/CMakeLists.txt: set(wrapper_test_scripts #options for configuring test options - ## generate-perl.sh ## -@@ generate-perl.sh: sed -e '1{' \ - -e " r $PERL_HEADER" \ - -e ' G' \ - -e '}' \ -- -e "s/@@GIT_VERSION@@/$GIT_VERSION/g" \ -+ -e "s/@GIT_VERSION@/$GIT_VERSION/g" \ - "$PERL_SCRIPT" >"$OUT" - chmod a+x "$OUT" - ## git-cvsserver.perl ## @@ use File::Basename; 18: 1ee9c61fc33 = 3: 8261f108968 Makefile: consistently use PERL_PATH 13: 7f567d7ce23 ! 4: 4894179b948 Makefile: extract script to massage Perl scripts @@ Commit message ## Makefile ## @@ Makefile: endif + PERL_DEFINES += $(gitexecdir) $(perllibdir) $(localedir) - $(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE +-$(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE - $(QUIET_GEN) \ - sed -e '1{' \ - -e ' s|#!.*perl|#!$(PERL_PATH_SQ)|' \ - -e ' r GIT-PERL-HEADER' \ - -e ' G' \ - -e '}' \ -- -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \ +- -e 's/@GIT_VERSION@/$(GIT_VERSION)/g' \ - $< >$@+ && \ - chmod +x $@+ && \ -+ $(QUIET_GEN)$(SHELL_PATH) generate-perl.sh $(GIT_VERSION) GIT-PERL-HEADER "$(PERL_PATH_SQ)" "$<" "$@+" && \ ++$(SCRIPT_PERL_GEN): % : %.perl generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE ++ $(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS $(GIT_VERSION) GIT-PERL-HEADER "$<" "$@+" && \ mv $@+ $@ PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES)) + ## contrib/buildsystems/CMakeLists.txt ## +@@ contrib/buildsystems/CMakeLists.txt: foreach(script ${git_shell_scripts}) + endforeach() + + #perl scripts +-parse_makefile_for_scripts(git_perl_scripts "SCRIPT_PERL" ".perl") ++parse_makefile_for_scripts(git_perl_scripts "SCRIPT_PERL" "") + + #create perl header + file(STRINGS ${CMAKE_SOURCE_DIR}/perl/header_templates/fixed_prefix.template.pl perl_header ) + string(REPLACE "@PATHSEP@" ":" perl_header "${perl_header}") + string(REPLACE "@INSTLIBDIR@" "${INSTLIBDIR}" perl_header "${perl_header}") ++file(WRITE ${CMAKE_BINARY_DIR}/PERL-HEADER ${perl_header}) + + foreach(script ${git_perl_scripts}) +- file(STRINGS ${CMAKE_SOURCE_DIR}/${script}.perl content NEWLINE_CONSUME) +- string(REPLACE "#!/usr/bin/perl" "#!/usr/bin/perl\n${perl_header}\n" content "${content}") +- string(REPLACE "@GIT_VERSION@" "${PROJECT_VERSION}" content "${content}") +- file(WRITE ${CMAKE_BINARY_DIR}/${script} ${content}) ++ string(REPLACE ".perl" "" perl_gen_path "${script}") ++ ++ add_custom_command(OUTPUT ${CMAKE_BINARY_DIR}/${perl_gen_path} ++ COMMAND ${CMAKE_SOURCE_DIR}/generate-perl.sh ++ ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS ++ ${PROJECT_VERSION} ++ ${CMAKE_BINARY_DIR}/PERL-HEADER ++ ${CMAKE_SOURCE_DIR}/${script} ++ ${CMAKE_BINARY_DIR}/${perl_gen_path} ++ DEPENDS ${CMAKE_SOURCE_DIR}/generate-perl.sh ++ ${CMAKE_SOURCE_DIR}/${script}) ++ list(APPEND perl_gen ${CMAKE_BINARY_DIR}/${perl_gen_path}) + endforeach() ++add_custom_target(perl-gen ALL DEPENDS ${perl_gen}) + + #python script + file(STRINGS ${CMAKE_SOURCE_DIR}/git-p4.py content NEWLINE_CONSUME) + ## generate-perl.sh (new) ## @@ +#!/bin/sh + ++set -e ++ +if test $# -ne 5 +then -+ echo "USAGE: $0 <GIT_VERSION> <PERL_HEADER> <PERL_PATH> <PERL_SCRIPT> <OUT>" >&2 ++ echo "USAGE: $0 <GIT_BUILD_OPTIONS> <GIT_VERSION> <PERL_HEADER> <INPUT> <OUTPUT>" >&2 + exit 1 +fi + -+GIT_VERSION="$1" -+PERL_HEADER="$2" -+PERL_PATH="$3" -+PERL_SCRIPT="$4" -+OUT="$5" ++GIT_BUILD_OPTIONS="$1" ++GIT_VERSION="$2" ++PERL_HEADER="$3" ++INPUT="$4" ++OUTPUT="$5" ++ ++. "$GIT_BUILD_OPTIONS" + +sed -e '1{' \ + -e " s|#!.*perl|#!$PERL_PATH|" \ + -e " r $PERL_HEADER" \ + -e ' G' \ + -e '}' \ -+ -e "s/@@GIT_VERSION@@/$GIT_VERSION/g" \ -+ "$PERL_SCRIPT" >"$OUT" -+chmod a+x "$OUT" ++ -e "s/@GIT_VERSION@/$GIT_VERSION/g" \ ++ "$INPUT" >"$OUTPUT" ++chmod a+x "$OUTPUT" -: ----------- > 5: b40bc302291 Makefile: use "generate-perl.sh" to massage Perl library -: ----------- > 6: c13ce99be3a Makefile: extract script to massage Shell scripts -: ----------- > 7: ebffd855836 Makefile: extract script to generate gitweb.cgi 14: dfc941cd18d ! 8: d2cdfad10f5 Makefile: refactor GIT-VERSION-GEN to be reusable @@ Commit message easier to compute the same version as our Makefile would without having to write the "GIT-VERSION-FILE". - We should ideally refactor our CMake build instructions to stop writing - into the source directory. But this step is out of scope of the current - patch series. - Signed-off-by: Patrick Steinhardt <ps@pks.im> ## GIT-VERSION-GEN ## @@ GIT-VERSION-GEN #!/bin/sh -GVF=GIT-VERSION-FILE - DEF_VER=v2.47.0 + DEF_VER=v2.47.GIT LF=' @@ GIT-VERSION-GEN: fi @@ Makefile: include shared.mak # Set our default configuration. ## contrib/buildsystems/CMakeLists.txt ## -@@ contrib/buildsystems/CMakeLists.txt: endif() - if(NOT EXISTS ${CMAKE_SOURCE_DIR}/GIT-VERSION-FILE) - message("Generating GIT-VERSION-FILE") - execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/GIT-VERSION-GEN -- WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}) -+ WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} -+ OUTPUT_VARIABLE GIT_VERSION -+ OUTPUT_STRIP_TRAILING_WHITESPACE) -+ file(WRITE ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "GIT_VERSION = '${GIT_VERSION}'\n") +@@ contrib/buildsystems/CMakeLists.txt: if(NOT SH_EXE) + "On Windows, you can get it as part of 'Git for Windows' install at https://gitforwindows.org/") endif() +-#Create GIT-VERSION-FILE using GIT-VERSION-GEN +-if(NOT EXISTS ${CMAKE_SOURCE_DIR}/GIT-VERSION-FILE) +- message("Generating GIT-VERSION-FILE") +- execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/GIT-VERSION-GEN +- WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}) +-endif() ++execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/GIT-VERSION-GEN ++ WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} ++ OUTPUT_VARIABLE git_version ++ OUTPUT_STRIP_TRAILING_WHITESPACE) + #Parse GIT-VERSION-FILE to get the version +-file(STRINGS ${CMAKE_SOURCE_DIR}/GIT-VERSION-FILE git_version REGEX "GIT_VERSION = (.*)") +-string(REPLACE "GIT_VERSION = " "" git_version ${git_version}) + string(FIND ${git_version} "GIT" location) + if(location EQUAL -1) + string(REGEX MATCH "[0-9]*\\.[0-9]*\\.[0-9]*" git_version ${git_version}) 15: e468d3751f0 ! 9: 9be0719ce1b Makefile: refactor generators to be PWD-independent @@ contrib/buildsystems/CMakeLists.txt: set(EXCLUSION_PROGS_CACHE ${EXCLUSION_PROGS - execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-configlist.sh - WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} - OUTPUT_FILE ${CMAKE_BINARY_DIR}/config-list.h) -+ execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-configlist.sh ${CMAKE_SOURCE_DIR} ${CMAKE_BINARY_DIR}/config.list) ++ execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-configlist.sh ${CMAKE_SOURCE_DIR} ${CMAKE_BINARY_DIR}/config-list.h) endif() if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h) 19: 9fd1aeb1e96 ! 10: 93869e53178 Makefile: allow "bin-wrappers/" directory to exist @@ Makefile: test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_ -bin-wrappers/%: wrap-for-bin.sh - $(call mkdir_p_parent_template) -+bin-wrappers/%: bin-wrappers/wrap-for-bin.sh ++$(test_bindir_programs): bin-wrappers/%: bin-wrappers/wrap-for-bin.sh $(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \ -e 's|@BUILD_DIR@|$(shell pwd)|' \ -e 's|@PROG@|$(patsubst test-%,t/helper/test-%,$(@F))$(if $(filter-out $(BINDIR_PROGRAMS_NO_X),$(@F)),$(X),)|' < $< > $@ && \ 20: 4eda54cf0f5 ! 11: 1c37d6d1f19 Makefile: simplify building of templates @@ Commit message Signed-off-by: Patrick Steinhardt <ps@pks.im> ## contrib/buildsystems/CMakeLists.txt ## -@@ contrib/buildsystems/CMakeLists.txt: endforeach() +@@ contrib/buildsystems/CMakeLists.txt: project(git + #TODO Enable NLS on windows natively + #macros for parsing the Makefile for sources and scripts +-macro(parse_makefile_for_sources list_var regex) +- file(STRINGS ${CMAKE_SOURCE_DIR}/Makefile ${list_var} REGEX "^${regex} \\+=(.*)") ++macro(parse_makefile_for_sources list_var makefile regex) ++ file(STRINGS ${makefile} ${list_var} REGEX "^${regex} \\+=(.*)") + string(REPLACE "${regex} +=" "" ${list_var} ${${list_var}}) + string(REPLACE "$(COMPAT_OBJS)" "" ${list_var} ${${list_var}}) #remove "$(COMPAT_OBJS)" This is only for libgit. + string(STRIP ${${list_var}} ${list_var}) #remove trailing/leading whitespaces +@@ contrib/buildsystems/CMakeLists.txt: include_directories(${CMAKE_BINARY_DIR}) - #templates --file(GLOB templates "${CMAKE_SOURCE_DIR}/templates/*") -+file(GLOB templates "${CMAKE_SOURCE_DIR}/templates/**") - list(TRANSFORM templates REPLACE "${CMAKE_SOURCE_DIR}/templates/" "") - list(REMOVE_ITEM templates ".gitignore") - list(REMOVE_ITEM templates "Makefile") - list(REMOVE_ITEM templates "blt")# Prevents an error when reconfiguring for in source builds + #build + #libgit +-parse_makefile_for_sources(libgit_SOURCES "LIB_OBJS") ++parse_makefile_for_sources(libgit_SOURCES ${CMAKE_SOURCE_DIR}/Makefile "LIB_OBJS") + + list(TRANSFORM libgit_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/") + list(TRANSFORM compat_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/") + add_library(libgit ${libgit_SOURCES} ${compat_SOURCES}) + + #libxdiff +-parse_makefile_for_sources(libxdiff_SOURCES "XDIFF_OBJS") ++parse_makefile_for_sources(libxdiff_SOURCES ${CMAKE_SOURCE_DIR}/Makefile "XDIFF_OBJS") + + list(TRANSFORM libxdiff_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/") + add_library(xdiff STATIC ${libxdiff_SOURCES}) + + #reftable +-parse_makefile_for_sources(reftable_SOURCES "REFTABLE_OBJS") ++parse_makefile_for_sources(reftable_SOURCES ${CMAKE_SOURCE_DIR}/Makefile "REFTABLE_OBJS") + + list(TRANSFORM reftable_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/") + add_library(reftable STATIC ${reftable_SOURCES}) +@@ contrib/buildsystems/CMakeLists.txt: elseif(UNIX) + endif() + + #git +-parse_makefile_for_sources(git_SOURCES "BUILTIN_OBJS") ++parse_makefile_for_sources(git_SOURCES ${CMAKE_SOURCE_DIR}/Makefile "BUILTIN_OBJS") + list(TRANSFORM git_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/") + add_executable(git ${CMAKE_SOURCE_DIR}/git.c ${git_SOURCES}) +@@ contrib/buildsystems/CMakeLists.txt: file(STRINGS ${CMAKE_SOURCE_DIR}/git-p4.py content NEWLINE_CONSUME) + string(REPLACE "#!/usr/bin/env python" "#!/usr/bin/python" content "${content}") + file(WRITE ${CMAKE_BINARY_DIR}/git-p4 ${content}) + +-#templates +-file(GLOB templates "${CMAKE_SOURCE_DIR}/templates/*") +-list(TRANSFORM templates REPLACE "${CMAKE_SOURCE_DIR}/templates/" "") +-list(REMOVE_ITEM templates ".gitignore") +-list(REMOVE_ITEM templates "Makefile") +-list(REMOVE_ITEM templates "blt")# Prevents an error when reconfiguring for in source builds +- -list(REMOVE_ITEM templates "branches--") -file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/templates/blt/branches) #create branches - ++#${CMAKE_SOURCE_DIR}/Makefile templates ++parse_makefile_for_sources(templates ${CMAKE_SOURCE_DIR}/templates/Makefile "TEMPLATES") ++string(REPLACE " " ";" templates ${templates}) #templates have @.*@ replacement so use configure_file instead foreach(tm ${templates}) - string(REPLACE "--" "/" blt_tm ${tm}) - string(REPLACE "this" "" blt_tm ${blt_tm})# for this-- - configure_file(${CMAKE_SOURCE_DIR}/templates/${tm} ${CMAKE_BINARY_DIR}/templates/blt/${blt_tm} @ONLY) +- configure_file(${CMAKE_SOURCE_DIR}/templates/${tm} ${CMAKE_BINARY_DIR}/templates/blt/${blt_tm} @ONLY) ++ configure_file(${CMAKE_SOURCE_DIR}/templates/${tm} ${CMAKE_BINARY_DIR}/templates/blt/${tm} @ONLY) endforeach() - #translations if(MSGFMT_EXE) file(GLOB po_files "${CMAKE_SOURCE_DIR}/po/*.po") +@@ contrib/buildsystems/CMakeLists.txt: add_executable(test-fake-ssh ${CMAKE_SOURCE_DIR}/t/helper/test-fake-ssh.c) + target_link_libraries(test-fake-ssh common-main) + + #unit-tests +-parse_makefile_for_sources(unit-test_SOURCES "UNIT_TEST_OBJS") ++parse_makefile_for_sources(unit-test_SOURCES ${CMAKE_SOURCE_DIR}/Makefile "UNIT_TEST_OBJS") + list(TRANSFORM unit-test_SOURCES REPLACE "\\$\\(UNIT_TEST_DIR\\)/" "${CMAKE_SOURCE_DIR}/t/unit-tests/") + add_library(unit-test-lib STATIC ${unit-test_SOURCES}) + +@@ contrib/buildsystems/CMakeLists.txt: if(MSVC) + endif() + + #test-tool +-parse_makefile_for_sources(test-tool_SOURCES "TEST_BUILTINS_OBJS") ++parse_makefile_for_sources(test-tool_SOURCES ${CMAKE_SOURCE_DIR}/Makefile "TEST_BUILTINS_OBJS") + add_library(test-lib OBJECT ${CMAKE_SOURCE_DIR}/t/unit-tests/test-lib.c) + + list(TRANSFORM test-tool_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/t/helper/") ## templates/Makefile ## @@ templates/Makefile: all: boilerplates.made custom @@ templates/Makefile: all: boilerplates.made custom -boilerplates.made : $(bpsrc) - $(QUIET)umask 022 && ls *--* 2>/dev/null | \ - while read boilerplate; \ -+TEMPLATES = description ++TEMPLATES = ++TEMPLATES += description +TEMPLATES += hooks/applypatch-msg.sample +TEMPLATES += hooks/commit-msg.sample +TEMPLATES += hooks/fsmonitor-watchman.sample 21: 1172c440600 ! 12: c45d2df6b8d t: better support for out-of-tree builds @@ GIT-BUILD-OPTIONS.in: GIT_PERF_MAKE_COMMAND=@GIT_PERF_MAKE_COMMAND@ +GIT_TEST_GITPERLLIB=@GIT_TEST_GITPERLLIB@ +GIT_TEST_MERGE_TOOLS_DIR=@GIT_TEST_MERGE_TOOLS_DIR@ RUNTIME_PREFIX=@RUNTIME_PREFIX@ + GITWEBDIR=@GITWEBDIR@ + USE_GETTEXT_SCHEME=@USE_GETTEXT_SCHEME@ ## Makefile ## @@ Makefile: GIT-BUILD-OPTIONS: FORCE @@ Makefile: GIT-BUILD-OPTIONS: FORCE + -e "s|@GIT_TEST_GITPERLLIB@|\'$(shell pwd)/perl/build/lib\'|" \ + -e "s|@GIT_TEST_MERGE_TOOLS_DIR@|\'$(shell pwd)/mergetools\'|" \ -e "s|@RUNTIME_PREFIX@|\'$(RUNTIME_PREFIX)\'|" \ - GIT-BUILD-OPTIONS.in >$@+ - @if cmp $@+ $@ >/dev/null 2>&1; then $(RM) $@+; else mv $@+ $@; fi + -e "s|@GITWEBDIR@|\'$(gitwebdir_SQ)\'|" \ + -e "s|@USE_GETTEXT_SCHEME@|\'$(USE_GETTEXT_SCHEME)\'|" \ @@ Makefile: all:: $(TEST_PROGRAMS) $(test_bindir_programs) $(UNIT_TEST_PROGS) $(CLAR_TEST_PR - bin-wrappers/%: bin-wrappers/wrap-for-bin.sh + $(test_bindir_programs): bin-wrappers/%: bin-wrappers/wrap-for-bin.sh $(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \ -e 's|@BUILD_DIR@|$(shell pwd)|' \ + -e 's|@GIT_TEXTDOMAINDIR@|$(shell pwd)/po/build/locale|' \ @@ contrib/buildsystems/CMakeLists.txt: endforeach() string(REPLACE "@PROG@" "git-cvsserver" content "${content}") file(WRITE ${CMAKE_BINARY_DIR}/bin-wrappers/git-cvsserver ${content}) -@@ contrib/buildsystems/CMakeLists.txt: string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_optio +@@ contrib/buildsystems/CMakeLists.txt: string(REPLACE "@GIT_PERF_MAKE_COMMAND@" "" git_build_options "${git_build_optio + string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_options}") string(REPLACE "@GIT_TEST_INDEX_VERSION@" "" git_build_options "${git_build_options}") string(REPLACE "@GIT_TEST_PERL_FATAL_WARNINGS@" "" git_build_options "${git_build_options}") - string(REPLACE "@RUNTIME_PREFIX@" "${RUNTIME_PREFIX}" git_build_options "${git_build_options}") +string(REPLACE "@GIT_TEST_TEXTDOMAINDIR@" "${CMAKE_BINARY_DIR}/po/build/locale" git_build_options "${git_build_options}") +string(REPLACE "@GIT_TEST_POPATH@" "${CMAKE_BINARY_DIR}/po" git_build_options "${git_build_options}") +string(REPLACE "@GIT_TEST_TEMPLATE_DIR@" "${CMAKE_BINARY_DIR}/templates/blt" git_build_options "${git_build_options}") +string(REPLACE "@GIT_TEST_GITPERLLIB@" "${CMAKE_BINARY_DIR}/perl/build/lib" git_build_options "${git_build_options}") +string(REPLACE "@GIT_TEST_MERGE_TOOLS_DIR@" "${RUNTIME_PREFIX}" git_build_options "${git_build_options}") - if(USE_VCPKG) - string(APPEND git_build_options "PATH=\"$PATH:$TEST_DIRECTORY/../compat/vcbuild/vcpkg/installed/x64-windows/bin\"\n") - endif() + string(REPLACE "@RUNTIME_PREFIX@" "${RUNTIME_PREFIX}" git_build_options "${git_build_options}") + string(REPLACE "@GITWEBDIR@" "${GITWEBDIR}" git_build_options "${git_build_options}") + string(REPLACE "@USE_GETTEXT_SCHEME@" "" git_build_options "${git_build_options}") ## t/lib-gettext.sh ## @@ 22: b5cd5250b77 = 13: 542b01515e1 t: allow overriding build dir 23: afa2f79d46c ! 14: bd59e31bb55 Documentation: add comparison of build systems @@ Documentation/technical/build-systems.txt (new) +== Requirements + +The following subsections present a list of requirements that we have for any -+potential build system. Sections are sorted by decreasing priority, even though -+these priorities will naturally differ between users. ++potential build system. Sections are sorted by decreasing priority. + +=== Platform support + -+The build system must have support for all of our primary platforms as outlined -+by. These platforms are: ++The build system must have support for all of our platforms that we continually ++test against as outlined by our platform support policy. These platforms are: + + - Linux + - Windows + - macOS + -+Furthermore, the build system should have support for the following secondary -+platforms: ++Furthermore, the build system should have support for the following platforms ++that generally have somebody running test pipelines against regularly: + + - AIX + - FreeBSD + - NetBSD ++ - NonStop + - OpenBSD + +The platforms which must be supported by the tool should be aligned with our @@ Documentation/technical/build-systems.txt (new) + - Detection of available features. + - Discovery of relevant toolchains. + - Linking libraries and executables. ++ - Templating placeholders in scripts. + +=== Test integration + @@ Documentation/technical/build-systems.txt (new) + - Track build-time dependencies for respective tests. Unit tests have + different requirements than integration tests. + - Allow filtering of which tests to run. -+ - Allow interactive tests that drop the user into a shell with `test_pause` or -+ `debug`. ++ - Allow running tests such that utilities like `test_pause` or `debug` work. + +== Comparison + 24: 724232b0fad ! 15: ac327d98e9c Introduce support for the Meson build system @@ Commit message - Apple macOS 10.15. - - FreeBSD 14.1, except gitweb-based tests. + - FreeBSD 14.1. - NixOS 24.11. + - OpenBSD 7.6. + - Ubuntu 24.04. - - Windows 10 with Cygwin, using `-Dperl=disabled -Dpython=disabled`. + - Windows 10 with Cygwin. + + - Windows 10 with MinGW64, except for t9700, which is also broken with + our Makefile. + + - Windows 10 with Visual Studio 2022 toolchain, using the Native Tools + Command Prompt with `meson setup --vsenv`. Tests pass, except for + t9700. - - Windows 10 with MinGW64, using `-Dperl=disabled -Dpython=disabled`. + - Windows 10 with Visual Studio 2022 solution, using the Native Tools + Command Prompt with `meson setup --backend vs2022`. Tests pass, + except for t9700. - - Windows 10 with MSVC 2022, using `-Dperl=disabled -Dpython=disabled - -Dgettext=disabled -Diconv=disabled --wrap-mode=forcefallback`. + - Windows 10 with VS Code, using the Meson plug-in. It is expected that there will still be rough edges in the current version. If this patch lands the expectation is that it will coexist @@ Commit message dependencies preinstalled, and Meson will fetch and build them automatically. This is especially helpful on Windows. + Helped-by: Eli Schwartz <eschwartz@gentoo.org> Signed-off-by: Patrick Steinhardt <ps@pks.im> ## bin-wrappers/meson.build (new) ## @@ contrib/meson.build (new) ## gitweb/meson.build (new) ## @@ ++gitweb_config = configuration_data() ++gitweb_config.set_quoted('PERL_PATH', perl.full_path()) ++gitweb_config.set_quoted('CSSMIN', '') ++gitweb_config.set_quoted('JSMIN', '') ++gitweb_config.set_quoted('GIT_VERSION', git_version) ++gitweb_config.set_quoted('GIT_BINDIR', get_option('prefix') / get_option('bindir')) ++gitweb_config.set_quoted('GITWEB_CONFIG', get_option('gitweb_config')) ++gitweb_config.set_quoted('GITWEB_CONFIG_SYSTEM', get_option('gitweb_config_system')) ++gitweb_config.set_quoted('GITWEB_CONFIG_COMMON', get_option('gitweb_config_common')) ++gitweb_config.set_quoted('GITWEB_HOME_LINK_STR', get_option('gitweb_home_link_str')) ++gitweb_config.set_quoted('GITWEB_SITENAME', get_option('gitweb_sitename')) ++gitweb_config.set_quoted('GITWEB_PROJECTROOT', get_option('gitweb_projectroot')) ++gitweb_config.set_quoted('GITWEB_PROJECT_MAXDEPTH', get_option('gitweb_project_maxdepth')) ++gitweb_config.set_quoted('GITWEB_EXPORT_OK', get_option('gitweb_export_ok')) ++gitweb_config.set_quoted('GITWEB_STRICT_EXPORT', get_option('gitweb_strict_export')) ++gitweb_config.set_quoted('GITWEB_BASE_URL', get_option('gitweb_base_url')) ++gitweb_config.set_quoted('GITWEB_LIST', get_option('gitweb_list')) ++gitweb_config.set_quoted('GITWEB_HOMETEXT', get_option('gitweb_hometext')) ++gitweb_config.set_quoted('GITWEB_CSS', get_option('gitweb_css')) ++gitweb_config.set_quoted('GITWEB_LOGO', get_option('gitweb_logo')) ++gitweb_config.set_quoted('GITWEB_FAVICON', get_option('gitweb_favicon')) ++gitweb_config.set_quoted('GITWEB_JS', get_option('gitweb_js')) ++gitweb_config.set_quoted('GITWEB_SITE_HTML_HEAD_STRING', get_option('gitweb_site_html_head_string')) ++gitweb_config.set_quoted('GITWEB_SITE_HEADER', get_option('gitweb_site_header')) ++gitweb_config.set_quoted('GITWEB_SITE_FOOTER', get_option('gitweb_site_footer')) ++gitweb_config.set_quoted('HIGHLIGHT_BIN', get_option('highlight_bin')) ++ +configure_file( ++ input: 'GITWEB-BUILD-OPTIONS.in', ++ output: 'GITWEB-BUILD-OPTIONS', ++ configuration: gitweb_config, ++) ++ ++test_dependencies += custom_target(script, + input: 'gitweb.perl', + output: 'gitweb.cgi', -+ configuration: { -+ 'GIT_VERSION': meson.project_version(), -+ 'GIT_BINDIR': get_option('prefix') / get_option('bindir'), -+ 'GITWEB_CONFIG': get_option('gitweb_config'), -+ 'GITWEB_CONFIG_SYSTEM': get_option('gitweb_config_system'), -+ 'GITWEB_CONFIG_COMMON': get_option('gitweb_config_common'), -+ 'GITWEB_HOME_LINK_STR': get_option('gitweb_home_link_str'), -+ 'GITWEB_SITENAME': get_option('gitweb_sitename'), -+ 'GITWEB_PROJECTROOT': get_option('gitweb_projectroot'), -+ 'GITWEB_PROJECT_MAXDEPTH': get_option('gitweb_project_maxdepth'), -+ 'GITWEB_EXPORT_OK': get_option('gitweb_export_ok'), -+ 'GITWEB_STRICT_EXPORT': get_option('gitweb_strict_export'), -+ 'GITWEB_BASE_URL': get_option('gitweb_base_url'), -+ 'GITWEB_LIST': get_option('gitweb_list'), -+ 'GITWEB_HOMETEXT': get_option('gitweb_hometext'), -+ 'GITWEB_CSS': get_option('gitweb_css'), -+ 'GITWEB_LOGO': get_option('gitweb_logo'), -+ 'GITWEB_FAVICON': get_option('gitweb_favicon'), -+ 'GITWEB_JS': get_option('gitweb_js'), -+ 'GITWEB_SITE_HTML_HEAD_STRING': get_option('gitweb_site_html_head_string'), -+ 'GITWEB_SITE_HEADER': get_option('gitweb_site_header'), -+ 'GITWEB_SITE_FOOTER': get_option('gitweb_site_footer'), -+ 'HIGHLIGHT_BIN': get_option('highlight_bin'), -+ }, ++ command: [ ++ shell, ++ meson.current_source_dir() / 'generate-gitweb.sh', ++ meson.current_build_dir() / 'GITWEB-BUILD-OPTIONS', ++ '@INPUT@', ++ '@OUTPUT@', ++ ], + install: true, + install_dir: get_option('datadir') / 'gitweb', +) + -+subdir('static') - - ## gitweb/static/meson.build (new) ## -@@ +foreach asset : [ -+ 'git-favicon.png', -+ 'git-logo.png', -+ 'gitweb.css', ++ 'static/git-favicon.png', ++ 'static/git-logo.png', ++ 'static/gitweb.css', ++ 'static/js/adjust-timezone.js', ++ 'static/js/blame_incremental.js', ++ 'static/js/javascript-detection.js', ++ 'static/js/lib/common-lib.js', ++ 'static/js/lib/cookies.js', ++ 'static/js/lib/datetime.js', +] + fs.copyfile(asset, + install: true, -+ install_dir: get_option('datadir') / 'gitweb/static', ++ install_dir: get_option('datadir') / 'gitweb' / fs.parent(asset), + ) +endforeach -+ -+javascript_sources = [ -+ meson.current_source_dir() / 'js/adjust-timezone.js', -+ meson.current_source_dir() / 'js/blame_incremental.js', -+ meson.current_source_dir() / 'js/javascript-detection.js', -+ meson.current_source_dir() / 'js/lib/common-lib.js', -+ meson.current_source_dir() / 'js/lib/cookies.js', -+ meson.current_source_dir() / 'js/lib/datetime.js', -+] -+ -+custom_target( -+ input: javascript_sources, -+ output: 'gitweb.js', -+ capture: true, -+ command: ['cat'] + javascript_sources, -+ install: true, -+ install_dir: get_option('datadir') / 'gitweb/static', -+) ## meson.build (new) ## @@ @@ meson.build (new) + # MSVC does not support GNU99, and C99 does not define __STDC_VERSION__ + # on MSVC. So we instead fall back to C11 there. + default_options: ['c_std=gnu99,c11'], -+ version: run_command('GIT-VERSION-GEN', check: true).stdout().strip(), ++ version: 'v2.47.GIT', +) + +fs = import('fs') + -+compiler = meson.get_compiler('c') ++program_path = [] ++# Git for Windows provides all the tools we need to build Git. ++if host_machine.system() == 'windows' ++ program_path += [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ] ++endif + -+cygpath = find_program('cygpath', required: false) -+shell = find_program('sh') -+tar = find_program('tar') -+diff = find_program('diff') ++awk = find_program('awk', dirs: program_path) ++cygpath = find_program('cygpath', dirs: program_path, required: false) ++diff = find_program('diff', dirs: program_path) ++shell = find_program('sh', dirs: program_path) ++tar = find_program('tar', dirs: program_path) ++ ++script_environment = environment() ++foreach tool : ['cat', 'cut', 'git', 'grep', 'sed', 'sort', 'tr', 'uname'] ++ program = find_program(tool, dirs: program_path) ++ script_environment.prepend('PATH', fs.parent(program.full_path())) ++endforeach ++ ++git_version = run_command(shell, 'GIT-VERSION-GEN', check: false, env: script_environment).stdout().strip() ++if git_version == '' ++ git_version = meson.project_version() ++endif ++ ++compiler = meson.get_compiler('c') + +libgit_sources = [ + 'abspath.c', @@ meson.build (new) + 'reftable/block.c', + 'reftable/blocksource.c', + 'reftable/iter.c', -+ 'reftable/publicbasics.c', + 'reftable/merged.c', + 'reftable/pq.c', + 'reftable/reader.c', @@ meson.build (new) + 'builtin/write-tree.c', +] + -+script_environment = environment() -+foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname'] -+ program = find_program(tool) -+ script_environment.prepend('PATH', fs.parent(program.full_path())) -+endforeach -+ +libgit_sources += custom_target( + 'command-list.h', + input: 'command-list.txt', @@ meson.build (new) +build_options_config.set('GIT_TEST_UTF8_LOCALE', '') +build_options_config.set('SANITIZE_ADDRESS', '') +build_options_config.set('SANITIZE_LEAK', '') ++build_options_config.set('BROKEN_PATH_FIX', '') ++build_options_config.set_quoted('LOCALEDIR', fs.as_posix(get_option('prefix') / get_option('localedir'))) ++build_options_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb')) + +test_output_directory = get_option('test_output_directory') +if test_output_directory == '' @@ meson.build (new) + +# Note that we only set NO_PERL if the Perl features were disabled by the user. +# It may not be set when we have found Perl, but only use it to run tests. -+perl = find_program('perl', version: '>=5.8.1', required: perl_required) ++perl = find_program('perl', version: '>=5.8.1', dirs: program_path, required: perl_required) +perl_features_enabled = perl.found() and get_option('perl').allowed() +if perl_features_enabled + build_options_config.set('NO_PERL', '') ++ ++ if get_option('runtime_prefix') ++ build_options_config.set('PERL_LOCALEDIR', '') ++ else ++ build_options_config.set_quoted('PERL_LOCALEDIR', fs.as_posix(get_option('prefix') / get_option('localedir'))) ++ endif ++ ++ if get_option('perl_cpan_fallback') ++ build_options_config.set('NO_PERL_CPAN_FALLBACKS', '') ++ else ++ build_options_config.set_quoted('NO_PERL_CPAN_FALLBACKS', 'YesPlease') ++ endif +else + libgit_c_args += '-DNO_PERL' + build_options_config.set('NO_PERL', '1') ++ build_options_config.set('PERL_LOCALEDIR', '') ++ build_options_config.set('NO_PERL_CPAN_FALLBACKS', '') +endif + +zlib = dependency('zlib', default_options: ['default_library=static', 'tests=disabled']) -+if zlib.type_name() != 'internal' and not compiler.has_function('deflateBound', -+ prefix: '#include <zlib.h>', -+ dependencies: zlib, -+) ++if zlib.version().version_compare('<1.2.0') + libgit_c_args += '-DNO_DEFLATE_BOUND' +endif +libgit_dependencies += zlib @@ meson.build (new) +if intl.found() + libgit_dependencies += intl + build_options_config.set('NO_GETTEXT', '') ++ build_options_config.set('USE_GETTEXT_SCHEME', '') +else + libgit_c_args += '-DNO_GETTEXT' + build_options_config.set('NO_GETTEXT', '1') ++ build_options_config.set('USE_GETTEXT_SCHEME', 'fallthrough') +endif + +iconv = dependency('iconv', required: get_option('iconv')) @@ meson.build (new) +libgit_version_library = static_library('git-version', + sources: ['version.c'], + c_args: libgit_c_args + [ -+ '-DGIT_VERSION="' + meson.project_version() + '"', -+ '-DGIT_USER_AGENT="' + 'git/' + meson.project_version() + '"', ++ '-DGIT_VERSION="' + git_version + '"', ++ '-DGIT_USER_AGENT="' + 'git/' + git_version + '"', + '-DGIT_BUILT_FROM_COMMIT="' + run_command('git', 'rev-parse', '-q', '--verify', 'HEAD', check: false).stdout().strip() + '"', + ], + dependencies: libgit_dependencies, @@ meson.build (new) + '-DMINOR=1', + '-DMICRO=1', + '-DPATCHLEVEL=0', -+ '-DGIT_VERSION=\\\"' + meson.project_version() + '.GIT\\\"', ++ '-DGIT_VERSION=\\\"' + git_version + '.GIT\\\"', + ]) + if compiler.get_argument_syntax() == 'gcc' + common_main_link_args += [ @@ meson.build (new) + ) +endforeach + -+script_sh_config = configuration_data() -+script_sh_config.set('BROKEN_PATH_FIX', '') -+script_sh_config.set('DIFF', fs.as_posix(diff.full_path())) -+script_sh_config.set('PAGER_ENV', get_option('pager_environment')) -+script_sh_config.set('PERL_PATH', perl.found() ? fs.as_posix(perl.full_path()) : '') -+script_sh_config.set('SHELL_PATH', fs.as_posix(shell.full_path())) -+script_sh_config.set_quoted('LOCALEDIR', fs.as_posix(get_option('prefix') / get_option('localedir'))) -+if intl.found() -+ script_sh_config.set('USE_GETTEXT_SCHEME', '') -+else -+ script_sh_config.set('USE_GETTEXT_SCHEME', 'fallthrough') -+endif -+script_sh_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb')) -+ +scripts_sh = [ + 'git-difftool--helper.sh', + 'git-filter-branch.sh', + 'git-merge-octopus.sh', + 'git-merge-one-file.sh', + 'git-merge-resolve.sh', ++ 'git-mergetool--lib.sh', + 'git-mergetool.sh', + 'git-quiltimport.sh', + 'git-request-pull.sh', ++ 'git-sh-i18n.sh', ++ 'git-sh-setup.sh', + 'git-submodule.sh', + 'git-web--browse.sh', +] -+ +if perl_features_enabled + scripts_sh += 'git-instaweb.sh' +endif + +foreach script : scripts_sh -+ configure_file( ++ test_dependencies += custom_target(script, + input: script, + output: fs.stem(script), -+ configuration: script_sh_config, -+ install: true, -+ install_dir: get_option('libexecdir') / 'git-core', -+ ) -+endforeach -+ -+scripts_lib = [ -+ 'git-mergetool--lib.sh', -+ 'git-sh-i18n.sh', -+ 'git-sh-setup.sh', -+] -+ -+foreach script : scripts_lib -+ configure_file( -+ input: script, -+ output: fs.stem(script), -+ configuration: script_sh_config, ++ command: [ ++ shell, ++ meson.project_source_root() / 'generate-script.sh', ++ '@INPUT@', ++ '@OUTPUT@', ++ meson.project_build_root() / 'GIT-BUILD-OPTIONS', ++ ], + install: true, + install_dir: get_option('libexecdir') / 'git-core', + ) @@ meson.build (new) + }, + ) + ++ generate_perl_command = [ ++ shell, ++ meson.project_source_root() / 'generate-perl.sh', ++ meson.project_build_root() / 'GIT-BUILD-OPTIONS', ++ git_version, ++ perl_header, ++ '@INPUT@', ++ '@OUTPUT@', ++ ] ++ + foreach script : scripts_perl + generated_script = custom_target(script, + input: script, + output: fs.stem(script), -+ command: [ -+ shell, -+ meson.project_source_root() / 'generate-perl.sh', -+ meson.project_version(), -+ perl_header, -+ perl.full_path(), -+ '@INPUT@', -+ '@OUTPUT@', -+ ], ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5', + ) @@ meson_options.txt (new) + description: 'Build helpers used to access remotes with the HTTP transport.') +option('expat', type: 'feature', value: 'enabled', + description: 'Build helpers used to push to remotes with the HTTP transport.') -+option('gettext', type: 'feature', value: 'enabled', ++option('gettext', type: 'feature', value: 'auto', + description: 'Build translation files.') -+option('iconv', type: 'feature', value: 'enabled', ++option('iconv', type: 'feature', value: 'auto', + description: 'Support reencoding strings with different encodings.') +option('pcre2', type: 'feature', value: 'enabled', + description: 'Support Perl-compatible regular expressions in e.g. git-grep(1).') -+option('perl', type: 'feature', value: 'enabled', ++option('perl', type: 'feature', value: 'auto', + description: 'Build tools written in Perl.') +option('perl_cpan_fallback', type: 'boolean', value: true, + description: 'Install bundled copies of CPAN modules that serve as a fallback in case the modules are not available on the system.') -+option('python', type: 'feature', value: 'enabled', ++option('python', type: 'feature', value: 'auto', + description: 'Build tools written in Python.') + +option('openssl', type: 'feature', value: 'auto', @@ meson_options.txt (new) ## perl/FromCPAN/Mail/meson.build (new) ## @@ -+configure_file( ++test_dependencies += custom_target('Address.pm', + input: 'Address.pm', + output: 'Address.pm', -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/FromCPAN/Mail', +) ## perl/FromCPAN/meson.build (new) ## @@ -+configure_file( ++test_dependencies += custom_target('Error.pm', + input: 'Error.pm', + output: 'Error.pm', -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/FromCPAN', +) @@ perl/FromCPAN/meson.build (new) ## perl/Git/LoadCPAN/Mail/meson.build (new) ## @@ -+configure_file( ++test_dependencies += custom_target('Address.pm', + input: 'Address.pm', + output: 'Address.pm', -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/Git/LoadCPAN/Mail', +) ## perl/Git/LoadCPAN/meson.build (new) ## @@ -+configure_file( ++test_dependencies += custom_target('Error.pm', + input: 'Error.pm', + output: 'Error.pm', -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/Git/LoadCPAN', +) @@ perl/Git/LoadCPAN/meson.build (new) ## perl/Git/SVN/Memoize/meson.build (new) ## @@ -+configure_file( ++test_dependencies += custom_target('YAML.pm', + input: 'YAML.pm', + output: 'YAML.pm', -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/Git/SVN', +) @@ perl/Git/SVN/meson.build (new) + 'Ra.pm', + 'Utils.pm', +] -+ configure_file( ++ test_dependencies += custom_target(source, + input: source, + output: source, -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/Git/SVN', + ) @@ perl/Git/meson.build (new) + 'Packet.pm', + 'SVN.pm', +] -+ configure_file( ++ test_dependencies += custom_target(source, + input: source, + output: source, -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5/Git', + ) @@ perl/Git/meson.build (new) ## perl/meson.build (new) ## @@ -+perl_config = { -+ 'LOCALEDIR': get_option('prefix') / get_option('datadir') / 'locale', -+ 'NO_GETTEXT': iconv.found() ? '' : '1', -+ 'NO_PERL_CPAN_FALLBACKS': get_option('perl_cpan_fallback') ? '' : '1', -+} -+ -+configure_file( ++test_dependencies += custom_target('Git.pm', + input: 'Git.pm', + output: 'Git.pm', -+ configuration: perl_config, ++ command: generate_perl_command, + install: true, + install_dir: get_option('datadir') / 'perl5', +) @@ po/meson.build (new) +) +test_dependencies += translations[0] + ## subprojects/.gitignore (new) ## +@@ ++/*/ + ## subprojects/curl.wrap (new) ## @@ +[wrap-file] @@ t/helper/meson.build (new) ## t/meson.build (new) ## @@ -+awk = find_program('awk') -+ +clar_test_suites = [ + 'unit-tests/ctype.c', + 'unit-tests/strvec.c', @@ templates/hooks/meson.build (new) + configure_file( + input: hook, + output: hook, -+ configuration: script_sh_config, ++ configuration: template_config, + ) +endforeach @@ templates/info/meson.build (new) +configure_file( + input: 'exclude', + output: 'exclude', -+ configuration: script_sh_config, ++ configuration: template_config, +) ## templates/meson.build (new) ## @@ ++template_config = configuration_data() ++template_config.set('PERL_PATH', perl.found() ? fs.as_posix(perl.full_path()) : '') ++template_config.set('SHELL_PATH', fs.as_posix(shell.full_path())) ++template_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb')) ++ +configure_file( + input: 'description', + output: 'description', -+ configuration: script_sh_config, ++ configuration: template_config, +) + +subdir('hooks')