| Message ID | 20221013130513.52440-1-anthony.perard@citrix.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Toolstack build system improvement, toward non-recursive makefiles | expand |
On 13/10/2022 14:04, Anthony PERARD wrote: > Patch series available in this git branch: > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5 I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and it does resolve the Ocaml library linking issues. Thanks. So T-by. If an allowance for 4.17 appears, I can see about getting some of the more obviously-trivial areas reviewed and committed. ~Andrew
Hi Andrew, > -----Original Message----- > From: Andrew Cooper <Andrew.Cooper3@citrix.com> > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system > improvement, toward non-recursive makefiles > > On 13/10/2022 14:04, Anthony PERARD wrote: > > Patch series available in this git branch: > > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git > br.toolstack-build-system-v5 > > I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and > it does resolve the Ocaml library linking issues. Thanks. > > So T-by. If an allowance for 4.17 appears, I can see about getting some > of the more obviously-trivial areas reviewed and committed. Release-acked-by: Henry Wang <Henry.Wang@arm.com> Since this series is quite large and just out of curiosity, do we have a pipeline or CI jobs to test all the modules that affected by the makefile touched by this series? Would be good to have some test results to boost our confidence of committing the series. Kind regards, Henry > > ~Andrew
On Fri, Oct 14, 2022 at 06:26:09AM +0000, Henry Wang wrote: > > -----Original Message----- > > From: Andrew Cooper <Andrew.Cooper3@citrix.com> > > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system > > improvement, toward non-recursive makefiles > > > > On 13/10/2022 14:04, Anthony PERARD wrote: > > > Patch series available in this git branch: > > > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git > > br.toolstack-build-system-v5 > > > > I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and > > it does resolve the Ocaml library linking issues. Thanks. > > > > So T-by. If an allowance for 4.17 appears, I can see about getting some > > of the more obviously-trivial areas reviewed and committed. > > Release-acked-by: Henry Wang <Henry.Wang@arm.com> Thanks. > Since this series is quite large and just out of curiosity, do we have a > pipeline or CI jobs to test all the modules that affected by the makefile > touched by this series? Would be good to have some test results to > boost our confidence of committing the series. I did push my branch to Gitlab, and that the pipeline: https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/665987083 (private to xen-project members) Only failures are I think the same one as on staging, test failure on arm, but no build issues. I'm not sure that everything is tested, but most of it is. Cheers,
Hi Anthony, > -----Original Message----- > From: Anthony PERARD <anthony.perard@citrix.com> > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system > improvement, toward non-recursive makefiles > > On Fri, Oct 14, 2022 at 06:26:09AM +0000, Henry Wang wrote: > > > -----Original Message----- > > > From: Andrew Cooper <Andrew.Cooper3@citrix.com> > > > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system > > > improvement, toward non-recursive makefiles > > > > > > On 13/10/2022 14:04, Anthony PERARD wrote: > > > > Patch series available in this git branch: > > > > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git > > > br.toolstack-build-system-v5 > > > > > > I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and > > > it does resolve the Ocaml library linking issues.� Thanks. > > > > > > So T-by.� If an allowance for 4.17 appears, I can see about getting some > > > of the more obviously-trivial areas reviewed and committed. > > > > Release-acked-by: Henry Wang <Henry.Wang@arm.com> > > Thanks. No problem, my pleasure. > > > Since this series is quite large and just out of curiosity, do we have a > > pipeline or CI jobs to test all the modules that affected by the makefile > > touched by this series? Would be good to have some test results to > > boost our confidence of committing the series. > > I did push my branch to Gitlab, and that the pipeline: > https://gitlab.com/xen-project/people/anthonyper/xen/- > /pipelines/665987083 > (private to xen-project members) > Only failures are I think the same one as on staging, test failure on > arm, but no build issues. Sorry for the arm failure, I will try to respin the patch ASAP and hopefully the CI will be unblocked soon. Kind regards, Henry > > I'm not sure that everything is tested, but most of it is. > > Cheers, > > -- > Anthony PERARD
On 14/10/2022 11:14, Anthony PERARD wrote: > On Fri, Oct 14, 2022 at 06:26:09AM +0000, Henry Wang wrote: >>> -----Original Message----- >>> From: Andrew Cooper <Andrew.Cooper3@citrix.com> >>> Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system >>> improvement, toward non-recursive makefiles >>> >>> On 13/10/2022 14:04, Anthony PERARD wrote: >>>> Patch series available in this git branch: >>>> https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git >>> br.toolstack-build-system-v5 >>> >>> I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and >>> it does resolve the Ocaml library linking issues. Thanks. >>> >>> So T-by. If an allowance for 4.17 appears, I can see about getting some >>> of the more obviously-trivial areas reviewed and committed. >> Release-acked-by: Henry Wang <Henry.Wang@arm.com> > Thanks. > >> Since this series is quite large and just out of curiosity, do we have a >> pipeline or CI jobs to test all the modules that affected by the makefile >> touched by this series? Would be good to have some test results to >> boost our confidence of committing the series. > I did push my branch to Gitlab, and that the pipeline: > https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/665987083 > (private to xen-project members) > Only failures are I think the same one as on staging, test failure on > arm, but no build issues. > > I'm not sure that everything is tested, but most of it is. The XenServer specfile lists all build artefacts specifically, because we've had too many bugs where things have silently disappeared. My testing confirms that nothing has appeared or disappeared as a result of these changes, and the functional looks pretty good. (except for the latest shadow assertion found, which was a regression in XSA-410. Fix is in review elsewhere.) ~Andrew
Hi Andrew, > -----Original Message----- > From: Andrew Cooper <Andrew.Cooper3@citrix.com> > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system > improvement, toward non-recursive makefiles > The XenServer specfile lists all build artefacts specifically, because > we've had too many bugs where things have silently disappeared. > > My testing confirms that nothing has appeared or disappeared as a result > of these changes, and the functional looks pretty good. (except for the > latest shadow assertion found, which was a regression in XSA-410. Fix > is in review elsewhere.) This is good to know. Thanks for the confirmation. No problem from my side then. Kind regards, Henry > > ~Andrew
On 13 Oct 2022, at 14:04, Anthony PERARD <anthony.perard@citrix.com<mailto:anthony.perard@citrix.com>> wrote:
Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5
Changes in v5:
- rebased on staging
- added "tools: Rework linking options for ocaml binding libraries"
Changes in v4:
- several new patches
- some changes to other patches listed in their changelogs
Changes in v3:
- rebased
- several new patches, starting with 13/25 "tools/libs/util: cleanup Makefile"
- introducing macros to deal with linking with in-tree xen libraries
- Add -Werror to CFLAGS for all builds in tools/
Changes in v2:
- one new patch
- other changes described in patch notes
Acked-by: Christian Lindig <christian.lindig@citrix.com<mailto:christian.lindig@citrix.com>>
Patch series available in this git branch: https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5 Changes in v5: - rebased on staging - added "tools: Rework linking options for ocaml binding libraries" Changes in v4: - several new patches - some changes to other patches listed in their changelogs Changes in v3: - rebased - several new patches, starting with 13/25 "tools/libs/util: cleanup Makefile" - introducing macros to deal with linking with in-tree xen libraries - Add -Werror to CFLAGS for all builds in tools/ Changes in v2: - one new patch - other changes described in patch notes Hi everyone, I've been looking at reworking the build system we have for the "tools/", and transforming it to something that suit it better. There are a lot of dependencies between different sub-directories so it would be nice if GNU make could actually handle them. This is possible with "non-recursive makefiles". With non-recursive makefiles, make will have to load/include all the makefiles and thus will have complete overview of all the dependencies. This will allow make to build the necessary targets in other directory, and we won't need to build sub-directories one by one. To help with this transformation, I've chosen to go with a recent project called "subdirmk". It help to deal with the fact that all makefiles will share the same namespace, it is hooked into autoconf, we can easily run `make` from any subdirectory. Together "autoconf" and "subdirmk" will also help to get closer to be able to do out-of-tree build of the tools, but I'm mainly looking to have non-recursive makefile. Link to the project: https://www.chiark.greenend.org.uk/ucgi/~ian/git/subdirmk.git/ But before getting to the main course, I've got quite a few cleanup and some changes to the makefiles. I start the patch series with patches that remove old left over stuff, then start reworking makefiles. They are some common changes like removing the "build" targets in many places as "all" would be the more common way to spell it and "all" is the default target anyway. They are other changes related to the conversion to "subdirmk", I start to use the variable $(TARGETS) in several makefiles, this variable will have a special meaning in subdirmk which will build those target by default. As for the conversion to non-recursive makefile, with subdirmk, I have this WIP branch, it contains some changes that I'm trying out, some notes, and the conversion, one Makefile per commit. Cleanup are still needed, some makefile not converted yet, but it's otherwise mostly done. https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v1-wip-extra With that branch, you could tried something like: ./configure; cd tools/xl; make and `xl` should be built as well as all the xen library needed. Also, things like `make clean` or rebuild should be faster in the all tools/ directory. Cheers, Anthony PERARD (17): tools/debugger/gdbsx: Fix and cleanup makefiles tools/xentrace: rework Makefile tools: Introduce $(xenlibs-ldflags, ) macro tools: Add -Werror by default to all tools/ tools: Remove -Werror everywhere else tools/hotplug: Generate "hotplugpath.sh" with configure libs/light/gentypes.py: allow to generate headers in subdirectory git-checkout.sh: handle running git-checkout from a different directory libs: Avoid exposing -Wl,--version-script to other built library libs: Fix auto-generation of version-script for unstable libs tools/include: Rework Makefile libs/light: Rework acpi table build targets libs/light: Rework generation of include/_libxl_*.h libs/light: Rework targets prerequisites libs/light: Makefile cleanup tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go tools: Rework linking options for ocaml binding libraries tools/configure.ac | 2 + tools/console/client/Makefile | 1 - tools/console/daemon/Makefile | 1 - tools/debugger/gdbsx/Makefile | 20 +++---- tools/debugger/gdbsx/gx/Makefile | 15 +++--- tools/debugger/gdbsx/xg/Makefile | 25 +++------ tools/debugger/kdd/Makefile | 1 - tools/flask/utils/Makefile | 1 - tools/fuzz/cpu-policy/Makefile | 2 +- tools/golang/xenlight/Makefile | 8 +-- tools/hotplug/common/Makefile | 10 +--- tools/include/Makefile | 28 +++++----- tools/libs/call/Makefile | 1 + tools/libs/ctrl/Makefile | 3 -- tools/libs/devicemodel/Makefile | 1 + tools/libs/evtchn/Makefile | 1 + tools/libs/foreignmemory/Makefile | 1 + tools/libs/gnttab/Makefile | 1 + tools/libs/guest/Makefile | 3 -- tools/libs/hypfs/Makefile | 1 + tools/libs/light/Makefile | 72 +++++++++++++++----------- tools/libs/stat/Makefile | 2 +- tools/libs/store/Makefile | 1 + tools/libs/toolcore/Makefile | 1 + tools/libs/toollog/Makefile | 1 + tools/libs/util/Makefile | 3 -- tools/libs/vchan/Makefile | 3 -- tools/misc/Makefile | 1 - tools/ocaml/libs/eventchn/Makefile | 2 +- tools/ocaml/libs/xc/Makefile | 2 +- tools/ocaml/libs/xentoollog/Makefile | 2 +- tools/ocaml/libs/xl/Makefile | 2 +- tools/tests/cpu-policy/Makefile | 2 +- tools/tests/depriv/Makefile | 2 +- tools/tests/resource/Makefile | 1 - tools/tests/tsx/Makefile | 1 - tools/tests/xenstore/Makefile | 1 - tools/xcutils/Makefile | 2 - tools/xenmon/Makefile | 1 - tools/xenpaging/Makefile | 1 - tools/xenpmd/Makefile | 1 - tools/xentop/Makefile | 2 +- tools/xentrace/Makefile | 29 ++++------- tools/xl/Makefile | 2 +- tools/Rules.mk | 20 +++++++ tools/debugger/gdbsx/Rules.mk | 2 +- tools/firmware/Rules.mk | 2 - tools/libfsimage/common.mk | 2 +- tools/libs/libs.mk | 21 +++++--- tools/libs/light/libxl_x86_acpi.c | 2 +- tools/ocaml/common.make | 2 +- .gitignore | 6 --- config/Tools.mk.in | 1 + scripts/git-checkout.sh | 4 +- tools/configure | 29 ++++++++++- tools/golang/xenlight/gengotypes.py | 10 +++- tools/hotplug/common/hotplugpath.sh.in | 16 ++++++ tools/libs/light/gentypes.py | 9 ++-- tools/xenstore/Makefile.common | 1 - 59 files changed, 218 insertions(+), 172 deletions(-) create mode 100644 tools/hotplug/common/hotplugpath.sh.in