Message ID | 20211125134006.1076646-1-anthony.perard@citrix.com (mailing list archive) |
---|---|
Headers | show |
Series | xen: Build system improvements, now with out-of-tree build! | expand |
On 25.11.2021 14:39, Anthony PERARD wrote: > Patch series available in this git branch: > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v8 > > v8: > Mostly rework of v7. With many patch already applied. > Some detail changes that are spread through many patches: > - `make cloc` recipe should now work throughout the series, update of it is > done in 3 patches. > - new patch "build: fix enforce unique symbols for recent clang version" > to fix an issue with clang. > - introducing $(srctree) and $(objtree) earlier > - introducing $(srcdir) as shortcut for $(srctree)/$(src) > - introduce usage of -iquote instead of -I in some cases > More detail change log can be found in patches notes. > > Also this v8 present a work-in-progress of the ability to do out-of-tree > build without setting VPATH. This is presented as an alternative to force > use of out-of-tree build. As the last patch show, it allows to build the > xen-shim without the linkfarm and we don't need to make any other changes > to any thing that build xen (osstest, distribution packages, xen.git, ..., > and developers finger macros). The patches are only there as WIP / RFC as > they were some concern about the usefulness and extra changes needed. > We can decide whether those changes are good or if this is too much and we > should force out-of-tree build for the hypervisor. I'm afraid I'm of two minds here. I don't think we want to force people to do out-of-tree builds, but I also dislike the idea of mixing in-tree and out-of-tree builds. Yet reading the above I understand that the shim build would conflict with an in-tree build because certain files would be picked (the shim build being an out-of-tree one) from the (dirtied) source tree, rather than the shim's build tree. Perhaps the extra path prefixes that I commented upon in an individual patch are then indeed the least bad route to take. There's one compromise which comes to mind, but which may also not be liked: We could simply fail out-of-tree builds when the source tree is dirty. Then people wanting the shim built would need to use out-of-tree builds also for the "main" hypervisor, but people suppressing the shim build anyway (or doing it separately, e.g. using a non-default .config) could continue to do in-tree builds. The one puzzle piece I'm lacking so far (perhaps simply because of having overlooked where it is) is how, for a full-build-of-everything, one would control where the xen/ part of the build would go _outside_ of the source (sub-)tree. Jan
On 21.12.2021 16:26, Jan Beulich wrote: > On 25.11.2021 14:39, Anthony PERARD wrote: >> Patch series available in this git branch: >> https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v8 >> >> v8: >> Mostly rework of v7. With many patch already applied. >> Some detail changes that are spread through many patches: >> - `make cloc` recipe should now work throughout the series, update of it is >> done in 3 patches. >> - new patch "build: fix enforce unique symbols for recent clang version" >> to fix an issue with clang. >> - introducing $(srctree) and $(objtree) earlier >> - introducing $(srcdir) as shortcut for $(srctree)/$(src) >> - introduce usage of -iquote instead of -I in some cases >> More detail change log can be found in patches notes. >> >> Also this v8 present a work-in-progress of the ability to do out-of-tree >> build without setting VPATH. This is presented as an alternative to force >> use of out-of-tree build. As the last patch show, it allows to build the >> xen-shim without the linkfarm and we don't need to make any other changes >> to any thing that build xen (osstest, distribution packages, xen.git, ..., >> and developers finger macros). The patches are only there as WIP / RFC as >> they were some concern about the usefulness and extra changes needed. >> We can decide whether those changes are good or if this is too much and we >> should force out-of-tree build for the hypervisor. > > I'm afraid I'm of two minds here. I don't think we want to force people to > do out-of-tree builds, but I also dislike the idea of mixing in-tree and > out-of-tree builds. Yet reading the above I understand that the shim build > would conflict with an in-tree build because certain files would be picked > (the shim build being an out-of-tree one) from the (dirtied) source tree, > rather than the shim's build tree. Perhaps the extra path prefixes that I > commented upon in an individual patch are then indeed the least bad route > to take. Having looked at the RFC patches near the end of the series, I find the need to convert from '#include "..."' to '#include <...>' particularly problematic: Using "" for "local" files is common practice, and hence even if you take care of the existing cases I'm afraid we'll be at constant risk of re-introducing such an issue somewhere. And someone hitting this issue may have a hard time figuring that it's simply the wrong file which gets included somewhere. IOW I guess I'm now favoring the "require clean source tree for out-of-tree builds" model. Jan
On Tue, Dec 21, 2021 at 04:26:48PM +0100, Jan Beulich wrote: > On 25.11.2021 14:39, Anthony PERARD wrote: > > Also this v8 present a work-in-progress of the ability to do out-of-tree > > build without setting VPATH. This is presented as an alternative to force > > use of out-of-tree build. As the last patch show, it allows to build the > > xen-shim without the linkfarm and we don't need to make any other changes > > to any thing that build xen (osstest, distribution packages, xen.git, ..., > > and developers finger macros). The patches are only there as WIP / RFC as > > they were some concern about the usefulness and extra changes needed. > > We can decide whether those changes are good or if this is too much and we > > should force out-of-tree build for the hypervisor. > > I'm afraid I'm of two minds here. I don't think we want to force people to > do out-of-tree builds, but I also dislike the idea of mixing in-tree and > out-of-tree builds. Yet reading the above I understand that the shim build > would conflict with an in-tree build because certain files would be picked > (the shim build being an out-of-tree one) from the (dirtied) source tree, > rather than the shim's build tree. Perhaps the extra path prefixes that I > commented upon in an individual patch are then indeed the least bad route > to take. To me, the problem is with a few build artefact and configuration: the ".config" file that osstest write and edit, and the final location of the "xen*" artefacts. Having something or someone wanting to edit ".config" breaks out-of-tree build. Otherwise, we could have forced out-of-tree build like libvirt or qemu are doing. > There's one compromise which comes to mind, but which may also not be > liked: We could simply fail out-of-tree builds when the source tree is > dirty. Then people wanting the shim built would need to use out-of-tree This isn't a compromise, it is already implemented in "build: adding out-of-tree support to the xen build". I revert this in patch "RFC, no-VPATH: remove check for clean source tree for out-of-tree". > builds also for the "main" hypervisor, but people suppressing the shim > build anyway (or doing it separately, e.g. using a non-default .config) > could continue to do in-tree builds. The one puzzle piece I'm lacking so > far (perhaps simply because of having overlooked where it is) is how, for > a full-build-of-everything, one would control where the xen/ part of the > build would go _outside_ of the source (sub-)tree. I guess that's not really an issue. It would be easy to run `make -C xen-build -f ../xen/Makefile` (or `make -C xen O=../xen-build`, or even `make -C xen O=build` would be fine too). With maybe a make variable to let someone specify a different output tree. For a full-build-of-everything, we can easily do out-of-tree build. The full-install-of-everything can pick the build artefact from the new place. The issue is when someone want that and edit ".config" of the hypervisor, as adding the file will stop out-of-tree build, so pv shim will not be able to use out-of-tree build. But maybe going with mixing in-tree and out-of-tree build is going to be too complicated. The hard part is for the build system to differentiate between source file and generated source file, so that might break from time to time. So maybe avoiding mixing and thus breaking build of some users might be better and more reliable in the long term. Cheers,
On 25.11.2021 14:39, Anthony PERARD wrote: > Patch series available in this git branch: > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v8 > > v8: > Mostly rework of v7. With many patch already applied. > Some detail changes that are spread through many patches: > - `make cloc` recipe should now work throughout the series, update of it is > done in 3 patches. > - new patch "build: fix enforce unique symbols for recent clang version" > to fix an issue with clang. > - introducing $(srctree) and $(objtree) earlier > - introducing $(srcdir) as shortcut for $(srctree)/$(src) > - introduce usage of -iquote instead of -I in some cases One more question: With an out-of-tree build, where do I put the (build flavor specific) ./.config (not xen/.config)? I'm using this extensively for cross builds and to override tool chain components (to avoid having to remember to always specify the right combination for a certain flavor on the command line). Jan
On Fri, Jan 21, 2022 at 07:59:40AM +0100, Jan Beulich wrote: > One more question: With an out-of-tree build, where do I put the > (build flavor specific) ./.config (not xen/.config)? I'm using this > extensively for cross builds and to override tool chain components > (to avoid having to remember to always specify the right combination > for a certain flavor on the command line). You mean the optional ".config" that "Config.mk" includes? This file is still included as "$(XEN_ROOT)/.config", and XEN_ROOT is set to be the source tree. Maybe you could abuse a file named "xen-version" which is included near the top of "xen/Makefile" instead.
On 21.01.2022 11:34, Anthony PERARD wrote: > On Fri, Jan 21, 2022 at 07:59:40AM +0100, Jan Beulich wrote: >> One more question: With an out-of-tree build, where do I put the >> (build flavor specific) ./.config (not xen/.config)? I'm using this >> extensively for cross builds and to override tool chain components >> (to avoid having to remember to always specify the right combination >> for a certain flavor on the command line). > > You mean the optional ".config" that "Config.mk" includes? This file is > still included as "$(XEN_ROOT)/.config", and XEN_ROOT is set to be the > source tree. It being taken from the source tree is precisely the issue. > Maybe you could abuse a file named "xen-version" which is included near > the top of "xen/Makefile" instead. I don't think that would work. The settings from ./.config are needed ahead of include $(XEN_ROOT)/config/$(XEN_OS).mk include $(XEN_ROOT)/config/$(XEN_TARGET_ARCH).mk which come later in Config.mk, at least when e.g. config/StdGNU.mk properly used ?= . Plus obviously for the 2nd of the lines above to use the right file, XEN_TARGET_ARCH already needs to be set to its designated value (same would of course be true for XEN_OS, if one was to override that). Jan