mbox series

[RFC,00/21] Modernize the build system

Message ID cover.1727881164.git.ps@pks.im (mailing list archive)
Headers show
Series Modernize the build system | expand

Message

Patrick Steinhardt Oct. 2, 2024, 3:15 p.m. UTC
Hi,

this patch series wires up support for the Meson build system. It is
intended as a request for comments and acts based on the feedback I've
got during the Git contributor's summit and the discussion in [1].

The aim of the patch series is to modernize our build system and set us
up such that we can eventually replace the autoconf- and Makefile-based
build system that we have right now with a more modern alternative. As a
result I'm hoping for the build system to become easier to hack on and
use, have better integration and be faster.

This patch series is rather large. It is structured as follows:

  - Patch 1 introduces a comparison of viable build systems. In its
    current form, the document is of course rather biased because I am
    its author. I am very happy to receive feedback on it to add factors
    that I didn't mention yet.

  - Patches 2 to 9 fix various different bugs I have found while working
    on many different platforms. They can likely go into the tree
    separately, and I'm happy to split these out into a separate patch
    series if we want to.

  - Patches 10 to 20 refactor parts of our preexisting build systems and
    testing infrastructure to provide better support for out-of-tree
    builds. While we already have CMake build instructions in our tree,
    they do not currently have "proper" out-of-tree builds as they still
    end up writing data into the source directory. So while these
    patches are written with Meson in mind, they should also help CMake.
    As a result of these patches, Meson builds and test runs are fully
    self-contained.

  - Patch 21 implements support for Meson. This is a big change, but
    more than half of the lines can be attributed to file listings. I
    have tested the build instructions on Linux, FreeBSD, macOS and
    Windows. Not all of the platforms pass all of the tests, and I did
    not yet get to integrating all of this with MSVC on Windows. Windows
    is quite foreign to me, and I spent the last two days ripping my
    hair out trying to get everything set up there, so this port is
    still a bit more on the rough side. In any case, if we decide that
    Meson might be a viable build system for us I will address those
    shortcomings, but in the RFC phase I didn't want perfect to be the
    enemy of good.

As I've said multiple times, I am strongly in favor of Meson over CMake,
and that's why I invested as much time as I did to wire it up this
extensively. But it goes without saying that this of course doesn't mean
that the project cannot come to a different conclusion than I did.

The patch series is built on top of e9356ba3ea (another batch after
2.47-rc0, 2024-09-30).

Thanks!

[1]: <GV1PR02MB848925A79A9DD733848182D58D662@GV1PR02MB8489.eurprd02.prod.outlook.com>

Patrick Steinhardt (21):
  Documentation: add comparison of build systems
  t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE
  t/lib-gitweb: test against the build version of gitweb
  t/lib-gpg: fix setup of GNUPGHOME in MinGW
  t3404: work around platform-specific behaviour on macOS 10.15
  t/unit-tests: update clar unit test framework
  t/clar: simplify how the clar derives `struct stat`
  builtin/credential-cache: fix missing parameter for stub function
  http: fix build error on FreeBSD
  Makefile: extract script to generate clar declarations
  Makefile: extract script to massage Perl scripts
  Makefile: refactor GIT-VERSION-GEN to be reusable
  Makefile: refactor generators to be PWD-independent
  Makefile: use common template for GIT-BUILD-OPTIONS
  Makefile: consistently use @PLACEHOLDER@ to substitute
  Makefile: consistently use PERL_PATH
  Makefile: allow "bin-wrappers/" directory to exist
  Makefile: simplify building of templates
  t: better support for out-of-tree builds
  t: allow overriding build dir
  Introduce support for the Meson build system

 .gitignore                                    |    1 -
 Documentation/CodingGuidelines                |    2 +-
 Documentation/Makefile                        |    1 +
 Documentation/technical/build-systems.txt     |  164 ++
 GIT-BUILD-OPTIONS.in                          |   40 +
 GIT-VERSION-GEN                               |   12 +-
 Makefile                                      |  186 +-
 bin-wrappers/.gitignore                       |    9 +
 bin-wrappers/meson.build                      |   28 +
 bin-wrappers/wrap-for-bin.sh                  |   37 +
 builtin/credential-cache.c                    |    3 +-
 configure.ac                                  |    2 +-
 contrib/buildsystems/CMakeLists.txt           |  129 +-
 contrib/completion/meson.build                |    8 +
 contrib/meson.build                           |    1 +
 generate-cmdlist.sh                           |   42 +-
 generate-configlist.sh                        |   20 +-
 generate-hooklist.sh                          |   15 +-
 generate-perl.sh                              |   22 +
 git-cvsserver.perl                            |    2 +-
 git-instaweb.sh                               |    8 +-
 git-request-pull.sh                           |    2 +-
 git-send-email.perl                           |    2 +-
 git-sh-i18n.sh                                |    6 +-
 git-sh-setup.sh                               |    6 +-
 git-svn.perl                                  |    2 +-
 gitk-git/po/vi.po                             |    2 +-
 gitweb/Makefile                               |   44 +-
 gitweb/gitweb.perl                            |   44 +-
 gitweb/meson.build                            |   32 +
 gitweb/static/meson.build                     |   28 +
 http.c                                        |   10 +-
 meson.build                                   | 1567 +++++++++++++++++
 meson_options.txt                             |   67 +
 perl/FromCPAN/Mail/meson.build                |    7 +
 perl/FromCPAN/meson.build                     |    9 +
 perl/Git/I18N.pm                              |    6 +-
 perl/Git/LoadCPAN.pm                          |    6 +-
 perl/Git/LoadCPAN/Mail/meson.build            |    7 +
 perl/Git/LoadCPAN/meson.build                 |    9 +
 perl/Git/SVN/Memoize/meson.build              |    7 +
 perl/Git/SVN/meson.build                      |   20 +
 perl/Git/meson.build                          |   18 +
 .../header_templates/fixed_prefix.template.pl |    2 +-
 .../runtime_prefix.template.pl                |    8 +-
 perl/meson.build                              |   18 +
 po/meson.build                                |   28 +
 t/helper/meson.build                          |   91 +
 t/lib-gettext.sh                              |    4 +-
 t/lib-gitweb.sh                               |    2 +-
 t/lib-gpg.sh                                  |    2 +-
 t/meson.build                                 | 1107 ++++++++++++
 t/t3404-rebase-interactive.sh                 |   38 +-
 t/t7609-mergetool--lib.sh                     |    2 +-
 t/test-lib.sh                                 |   24 +-
 t/unit-tests/clar/.editorconfig               |   13 +
 t/unit-tests/clar/.github/workflows/ci.yml    |   20 +-
 t/unit-tests/clar/.gitignore                  |    1 +
 t/unit-tests/clar/CMakeLists.txt              |   28 +
 t/unit-tests/clar/clar.c                      |  111 +-
 t/unit-tests/clar/clar/print.h                |   11 +-
 t/unit-tests/clar/clar/sandbox.h              |   17 +-
 t/unit-tests/clar/clar/summary.h              |   14 +-
 t/unit-tests/clar/test/.gitignore             |    4 -
 t/unit-tests/clar/test/CMakeLists.txt         |   41 +
 t/unit-tests/clar/test/Makefile               |   39 -
 t/unit-tests/generate-clar-decls.sh           |   16 +
 templates/Makefile                            |   38 +-
 templates/branches--                          |    1 -
 templates/{this--description => description}  |    0
 .../applypatch-msg.sample}                    |    0
 .../commit-msg.sample}                        |    0
 .../fsmonitor-watchman.sample}                |    0
 templates/hooks/meson.build                   |   24 +
 .../post-update.sample}                       |    0
 .../pre-applypatch.sample}                    |    0
 .../pre-commit.sample}                        |    0
 .../pre-merge-commit.sample}                  |    0
 .../pre-push.sample}                          |    0
 .../pre-rebase.sample}                        |    0
 .../pre-receive.sample}                       |    0
 .../prepare-commit-msg.sample}                |    0
 .../push-to-checkout.sample}                  |    0
 .../sendemail-validate.sample}                |    0
 .../update.sample}                            |    0
 templates/{info--exclude => info/exclude}     |    0
 templates/info/meson.build                    |    5 +
 templates/meson.build                         |    8 +
 unimplemented.sh                              |    2 +-
 wrap-for-bin.sh                               |   36 -
 90 files changed, 3901 insertions(+), 487 deletions(-)
 create mode 100644 Documentation/technical/build-systems.txt
 create mode 100644 GIT-BUILD-OPTIONS.in
 create mode 100644 bin-wrappers/.gitignore
 create mode 100644 bin-wrappers/meson.build
 create mode 100755 bin-wrappers/wrap-for-bin.sh
 create mode 100644 contrib/completion/meson.build
 create mode 100644 contrib/meson.build
 create mode 100755 generate-perl.sh
 create mode 100644 gitweb/meson.build
 create mode 100644 gitweb/static/meson.build
 create mode 100644 meson.build
 create mode 100644 meson_options.txt
 create mode 100644 perl/FromCPAN/Mail/meson.build
 create mode 100644 perl/FromCPAN/meson.build
 create mode 100644 perl/Git/LoadCPAN/Mail/meson.build
 create mode 100644 perl/Git/LoadCPAN/meson.build
 create mode 100644 perl/Git/SVN/Memoize/meson.build
 create mode 100644 perl/Git/SVN/meson.build
 create mode 100644 perl/Git/meson.build
 create mode 100644 perl/meson.build
 create mode 100644 po/meson.build
 create mode 100644 t/helper/meson.build
 create mode 100644 t/meson.build
 create mode 100644 t/unit-tests/clar/.editorconfig
 create mode 100644 t/unit-tests/clar/.gitignore
 create mode 100644 t/unit-tests/clar/CMakeLists.txt
 delete mode 100644 t/unit-tests/clar/test/.gitignore
 create mode 100644 t/unit-tests/clar/test/CMakeLists.txt
 delete mode 100644 t/unit-tests/clar/test/Makefile
 create mode 100755 t/unit-tests/generate-clar-decls.sh
 delete mode 100644 templates/branches--
 rename templates/{this--description => description} (100%)
 rename templates/{hooks--applypatch-msg.sample => hooks/applypatch-msg.sample} (100%)
 rename templates/{hooks--commit-msg.sample => hooks/commit-msg.sample} (100%)
 rename templates/{hooks--fsmonitor-watchman.sample => hooks/fsmonitor-watchman.sample} (100%)
 create mode 100644 templates/hooks/meson.build
 rename templates/{hooks--post-update.sample => hooks/post-update.sample} (100%)
 rename templates/{hooks--pre-applypatch.sample => hooks/pre-applypatch.sample} (100%)
 rename templates/{hooks--pre-commit.sample => hooks/pre-commit.sample} (100%)
 rename templates/{hooks--pre-merge-commit.sample => hooks/pre-merge-commit.sample} (100%)
 rename templates/{hooks--pre-push.sample => hooks/pre-push.sample} (100%)
 rename templates/{hooks--pre-rebase.sample => hooks/pre-rebase.sample} (100%)
 rename templates/{hooks--pre-receive.sample => hooks/pre-receive.sample} (100%)
 rename templates/{hooks--prepare-commit-msg.sample => hooks/prepare-commit-msg.sample} (100%)
 rename templates/{hooks--push-to-checkout.sample => hooks/push-to-checkout.sample} (100%)
 rename templates/{hooks--sendemail-validate.sample => hooks/sendemail-validate.sample} (100%)
 rename templates/{hooks--update.sample => hooks/update.sample} (100%)
 rename templates/{info--exclude => info/exclude} (100%)
 create mode 100644 templates/info/meson.build
 create mode 100644 templates/meson.build
 delete mode 100644 wrap-for-bin.sh

Comments

Phillip Wood Oct. 8, 2024, 2:11 p.m. UTC | #1
Hi Patrick

On 02/10/2024 16:15, Patrick Steinhardt wrote:
> 
>    - Patches 2 to 9 fix various different bugs I have found while working
>      on many different platforms. They can likely go into the tree
>      separately, and I'm happy to split these out into a separate patch
>      series if we want to.

These all look like useful improvements in their own right, splitting 
them out would make sense to me.

>    - Patches 10 to 20 refactor parts of our preexisting build systems and
>      testing infrastructure to provide better support for out-of-tree
>      builds. While we already have CMake build instructions in our tree,
>      they do not currently have "proper" out-of-tree builds as they still
>      end up writing data into the source directory. So while these
>      patches are written with Meson in mind, they should also help CMake.
>      As a result of these patches, Meson builds and test runs are fully
>      self-contained.

Patches 14-16 and 18 look useful in their own right even if we don't 
adopt Meson. The others all look sensible in the context of supporting 
multiple build systems and might be worth doing anyway especially if 
they simplify the existing CMakeLists.txt.

I've skipped patches 1 and 21 for now, I'll have a look at those later 
in the week.

Best Wishes

Phillip
Patrick Steinhardt Oct. 8, 2024, 2:23 p.m. UTC | #2
On Tue, Oct 08, 2024 at 03:11:35PM +0100, Phillip Wood wrote:
> Hi Patrick
> 
> On 02/10/2024 16:15, Patrick Steinhardt wrote:
> > 
> >    - Patches 2 to 9 fix various different bugs I have found while working
> >      on many different platforms. They can likely go into the tree
> >      separately, and I'm happy to split these out into a separate patch
> >      series if we want to.
> 
> These all look like useful improvements in their own right, splitting them
> out would make sense to me.

For now I'll reorder them such that they are all up front. Like that it
should be easy enough to just merge the first X commits while keeping
the overall series together. The reviewers don't have to go through all
commits.

If that's creating problems though I'll split this out into a standalone
series.

> >    - Patches 10 to 20 refactor parts of our preexisting build systems and
> >      testing infrastructure to provide better support for out-of-tree
> >      builds. While we already have CMake build instructions in our tree,
> >      they do not currently have "proper" out-of-tree builds as they still
> >      end up writing data into the source directory. So while these
> >      patches are written with Meson in mind, they should also help CMake.
> >      As a result of these patches, Meson builds and test runs are fully
> >      self-contained.
> 
> Patches 14-16 and 18 look useful in their own right even if we don't adopt
> Meson. The others all look sensible in the context of supporting multiple
> build systems and might be worth doing anyway especially if they simplify
> the existing CMakeLists.txt.
> 
> I've skipped patches 1 and 21 for now, I'll have a look at those later in
> the week.

I'll also send an updated version soonish that includes support for
MSVC, likely tomorrow. So you may want to hold off reviewing them for
now.

Thanks for your review!

Patrick