diff mbox series

[XEN,1/4] build: xen/include: use if_changed

Message ID 20220601165909.46588-2-anthony.perard@citrix.com (mailing list archive)
State Superseded
Headers show
Series xen: rework compat headers generation | expand

Commit Message

Anthony PERARD June 1, 2022, 4:59 p.m. UTC
Use "define" for the headers*_chk commands as otherwise the "#"
is interpreted as a comment and make can't find the end of
$(foreach,).

Adding several .PRECIOUS as without them `make` deletes the
intermediate targets. This is an issue because the macro $(if_changed,)
check if the target exist in order to decide whether to recreate the
target.

Removing the call to `mkdir` from the commands. Those aren't needed
anymore because a rune in Rules.mk creates the directory for each
$(targets).

Remove "export PYTHON" as it is already exported.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/include/Makefile | 108 ++++++++++++++++++++++++++++++-------------
 1 file changed, 76 insertions(+), 32 deletions(-)

Comments

Jan Beulich June 2, 2022, 9:11 a.m. UTC | #1
On 01.06.2022 18:59, Anthony PERARD wrote:
> Use "define" for the headers*_chk commands as otherwise the "#"
> is interpreted as a comment and make can't find the end of
> $(foreach,).

In cmd_xlat_lst you use $(pound) - any reason this doesn't work in
these rules? Note that I don't mind the use of "define", just that I'm
puzzled by the justification.

Jan
Anthony PERARD June 6, 2022, 1:39 p.m. UTC | #2
On Thu, Jun 02, 2022 at 11:11:15AM +0200, Jan Beulich wrote:
> On 01.06.2022 18:59, Anthony PERARD wrote:
> > Use "define" for the headers*_chk commands as otherwise the "#"
> > is interpreted as a comment and make can't find the end of
> > $(foreach,).
> 
> In cmd_xlat_lst you use $(pound) - any reason this doesn't work in
> these rules? Note that I don't mind the use of "define", just that I'm
> puzzled by the justification.

I think I just forgot about $(pound) when I tried to debug why the
command didn't work in some environment (that is when not using define).

I think the second thing that make me not replacing '#' by "$(pound)" is
that reading "#include" in the Makefile is probably better that
reading "$(pound)include".

I guess we should add something to the justification like:
    That allow us to keep writing "#include" in the Makefile without
    having to replace that by "$(pound)include" which would be a bit
    less obvious about the command line purpose.

Thanks,
Jan Beulich June 7, 2022, 6:26 a.m. UTC | #3
On 06.06.2022 15:39, Anthony PERARD wrote:
> On Thu, Jun 02, 2022 at 11:11:15AM +0200, Jan Beulich wrote:
>> On 01.06.2022 18:59, Anthony PERARD wrote:
>>> Use "define" for the headers*_chk commands as otherwise the "#"
>>> is interpreted as a comment and make can't find the end of
>>> $(foreach,).
>>
>> In cmd_xlat_lst you use $(pound) - any reason this doesn't work in
>> these rules? Note that I don't mind the use of "define", just that I'm
>> puzzled by the justification.
> 
> I think I just forgot about $(pound) when I tried to debug why the
> command didn't work in some environment (that is when not using define).
> 
> I think the second thing that make me not replacing '#' by "$(pound)" is
> that reading "#include" in the Makefile is probably better that
> reading "$(pound)include".
> 
> I guess we should add something to the justification like:
>     That allow us to keep writing "#include" in the Makefile without
>     having to replace that by "$(pound)include" which would be a bit
>     less obvious about the command line purpose.

I'd be okay with this, and I'd also be okay with adding this while
committing. With it added
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan
Bertrand Marquis June 9, 2022, 10:16 a.m. UTC | #4
Hi Anthony,

> On 1 Jun 2022, at 17:59, Anthony PERARD <anthony.perard@citrix.com> wrote:
> 
> Use "define" for the headers*_chk commands as otherwise the "#"
> is interpreted as a comment and make can't find the end of
> $(foreach,).
> 
> Adding several .PRECIOUS as without them `make` deletes the
> intermediate targets. This is an issue because the macro $(if_changed,)
> check if the target exist in order to decide whether to recreate the
> target.
> 
> Removing the call to `mkdir` from the commands. Those aren't needed
> anymore because a rune in Rules.mk creates the directory for each
> $(targets).
> 
> Remove "export PYTHON" as it is already exported.

With this change, compiling for x86 is now ending up in:
CHK     include/headers99.chk
make[9]: execvp: /bin/sh: Argument list too long
make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127

Not quite sure yet why but I wanted to signal it early as other might be impacted.

Arm and arm64 builds are not impacted.

Cheers
Bertrand

> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> xen/include/Makefile | 108 ++++++++++++++++++++++++++++++-------------
> 1 file changed, 76 insertions(+), 32 deletions(-)
> 
> diff --git a/xen/include/Makefile b/xen/include/Makefile
> index 03baf10efb..6d9bcc19b0 100644
> --- a/xen/include/Makefile
> +++ b/xen/include/Makefile
> @@ -45,38 +45,65 @@ public-$(CONFIG_ARM) := $(wildcard $(srcdir)/public/arch-arm/*.h $(srcdir)/publi
> .PHONY: all
> all: $(addprefix $(obj)/,$(headers-y))
> 
> -$(obj)/compat/%.h: $(obj)/compat/%.i $(srcdir)/Makefile $(srctree)/tools/compat-build-header.py
> -	$(PYTHON) $(srctree)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
> -	mv -f $@.new $@
> -
> -$(obj)/compat/%.i: $(obj)/compat/%.c $(srcdir)/Makefile
> -	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
> -
> -$(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srcdir)/Makefile $(srctree)/tools/compat-build-source.py
> -	mkdir -p $(@D)
> -	$(PYTHON) $(srctree)/tools/compat-build-source.py $(srcdir)/xlat.lst <$< >$@.new
> -	mv -f $@.new $@
> -
> -$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh $(srcdir)/Makefile
> -	export PYTHON=$(PYTHON); \
> -	while read what name; do \
> -		$(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
> -	done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >$@.new
> -	mv -f $@.new $@
> +quiet_cmd_compat_h = GEN     $@
> +cmd_compat_h = \
> +    $(PYTHON) $(srctree)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
> +    mv -f $@.new $@
> +
> +quiet_cmd_compat_i = CPP     $@
> +cmd_compat_i = $(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
> +
> +quiet_cmd_compat_c = GEN     $@
> +cmd_compat_c = \
> +   $(PYTHON) $(srctree)/tools/compat-build-source.py $(srcdir)/xlat.lst <$< >$@.new; \
> +   mv -f $@.new $@
> +
> +quiet_cmd_xlat_headers = GEN     $@
> +cmd_xlat_headers = \
> +    while read what name; do \
> +        $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
> +    done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >$@.new; \
> +    mv -f $@.new $@
> +
> +targets += $(headers-y)
> +$(obj)/compat/%.h: $(obj)/compat/%.i $(srctree)/tools/compat-build-header.py FORCE
> +	$(call if_changed,compat_h)
> +
> +.PRECIOUS: $(obj)/compat/%.i
> +targets += $(patsubst %.h, %.i, $(headers-y))
> +$(obj)/compat/%.i: $(obj)/compat/%.c FORCE
> +	$(call if_changed,compat_i)
> +
> +.PRECIOUS: $(obj)/compat/%.c
> +targets += $(patsubst %.h, %.c, $(headers-y))
> +$(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srctree)/tools/compat-build-source.py FORCE
> +	$(call if_changed,compat_c)
> +
> +targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y))
> +$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh FORCE
> +	$(call if_changed,xlat_headers)
> +
> +quiet_cmd_xlat_lst = GEN     $@
> +cmd_xlat_lst = \
> +	grep -v '^[[:blank:]]*$(pound)' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new; \
> +	$(call move-if-changed,$@.new,$@)
> 
> .PRECIOUS: $(obj)/compat/.xlat/%.lst
> -$(obj)/compat/.xlat/%.lst: $(srcdir)/xlat.lst $(srcdir)/Makefile
> -	mkdir -p $(@D)
> -	grep -v '^[[:blank:]]*#' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new
> -	$(call move-if-changed,$@.new,$@)
> +targets += $(patsubst compat/%.h, compat/.xlat/%.lst, $(headers-y))
> +$(obj)/compat/.xlat/%.lst: $(srcdir)/xlat.lst FORCE
> +	$(call if_changed,xlat_lst)
> 
> xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(srcdir)/xlat.lst | uniq)
> xlat-y := $(filter $(patsubst compat/%,%,$(headers-y)),$(xlat-y))
> 
> -$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf $(srcdir)/Makefile
> -	cat $(filter %.h,$^) >$@.new
> +quiet_cmd_xlat_h = GEN     $@
> +cmd_xlat_h = \
> +	cat $(filter %.h,$^) >$@.new; \
> 	mv -f $@.new $@
> 
> +$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf FORCE
> +	$(call if_changed,xlat_h)
> +
> ifeq ($(XEN_TARGET_ARCH),$(XEN_COMPILE_ARCH))
> 
> all: $(obj)/headers.chk $(obj)/headers99.chk $(obj)/headers++.chk
> @@ -102,27 +129,31 @@ PUBLIC_C99_HEADERS := $(call public-filter-headers,public-c99-headers)
> $(src)/public/io/9pfs.h-prereq := string
> $(src)/public/io/pvcalls.h-prereq := string
> 
> -$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(srcdir)/Makefile
> +quiet_cmd_header_chk = CHK     $@
> +cmd_header_chk = \
> 	for i in $(filter %.h,$^); do \
> 	    $(CC) -x c -ansi -Wall -Werror -include stdint.h \
> 	          -S -o /dev/null $$i || exit 1; \
> 	    echo $$i; \
> -	done >$@.new
> +	done >$@.new; \
> 	mv $@.new $@
> 
> -$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(srcdir)/Makefile
> -	rm -f $@.new
> +quiet_cmd_headers99_chk = CHK     $@
> +define cmd_headers99_chk
> +	rm -f $@.new; \
> 	$(foreach i, $(filter %.h,$^),                                        \
> 	    echo "#include "\"$(i)\"                                          \
> 	    | $(CC) -x c -std=c99 -Wall -Werror                               \
> 	      -include stdint.h                                               \
> 	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include $(j).h) \
> 	      -S -o /dev/null -                                               \
> -	    || exit $$?; echo $(i) >> $@.new;)
> +	    || exit $$?; echo $(i) >> $@.new;) \
> 	mv $@.new $@
> +endef
> 
> -$(obj)/headers++.chk: $(PUBLIC_HEADERS) $(srcdir)/Makefile
> -	rm -f $@.new
> +quiet_cmd_headerscxx_chk = CHK     $@
> +define cmd_headerscxx_chk
> +	rm -f $@.new; \
> 	if ! $(CXX) -v >/dev/null 2>&1; then                                  \
> 	    touch $@.new;                                                     \
> 	    exit 0;                                                           \
> @@ -133,8 +164,21 @@ $(obj)/headers++.chk: $(PUBLIC_HEADERS) $(srcdir)/Makefile
> 	      -include stdint.h -include $(srcdir)/public/xen.h               \
> 	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \
> 	      -S -o /dev/null -                                               \
> -	    || exit $$?; echo $(i) >> $@.new;)
> +	    || exit $$?; echo $(i) >> $@.new;) \
> 	mv $@.new $@
> +endef
> +
> +targets += headers.chk
> +$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) FORCE
> +	$(call if_changed,header_chk)
> +
> +targets += headers99.chk
> +$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) FORCE
> +	$(call if_changed,headers99_chk)
> +
> +targets += headers++.chk
> +$(obj)/headers++.chk: $(PUBLIC_HEADERS) FORCE
> +	$(call if_changed,headerscxx_chk)
> 
> endif
> 
> -- 
> Anthony PERARD
> 
>
Jan Beulich June 9, 2022, 10:26 a.m. UTC | #5
On 09.06.2022 12:16, Bertrand Marquis wrote:
>> On 1 Jun 2022, at 17:59, Anthony PERARD <anthony.perard@citrix.com> wrote:
>>
>> Use "define" for the headers*_chk commands as otherwise the "#"
>> is interpreted as a comment and make can't find the end of
>> $(foreach,).
>>
>> Adding several .PRECIOUS as without them `make` deletes the
>> intermediate targets. This is an issue because the macro $(if_changed,)
>> check if the target exist in order to decide whether to recreate the
>> target.
>>
>> Removing the call to `mkdir` from the commands. Those aren't needed
>> anymore because a rune in Rules.mk creates the directory for each
>> $(targets).
>>
>> Remove "export PYTHON" as it is already exported.
> 
> With this change, compiling for x86 is now ending up in:
> CHK     include/headers99.chk
> make[9]: execvp: /bin/sh: Argument list too long
> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
> 
> Not quite sure yet why but I wanted to signal it early as other might be impacted.
> 
> Arm and arm64 builds are not impacted.

Hmm, that patch has passed the smoke push gate already, so there likely is
more to it than there being an unconditional issue. I did build-test this
before pushing, and I've just re-tested on a 2nd system without seeing an
issue.

Also please remember to trim your replies.

Jan
Bertrand Marquis June 9, 2022, 11:51 a.m. UTC | #6
Hi,

> On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 09.06.2022 12:16, Bertrand Marquis wrote:
>>> On 1 Jun 2022, at 17:59, Anthony PERARD <anthony.perard@citrix.com> wrote:
>>> 
>>> Use "define" for the headers*_chk commands as otherwise the "#"
>>> is interpreted as a comment and make can't find the end of
>>> $(foreach,).
>>> 
>>> Adding several .PRECIOUS as without them `make` deletes the
>>> intermediate targets. This is an issue because the macro $(if_changed,)
>>> check if the target exist in order to decide whether to recreate the
>>> target.
>>> 
>>> Removing the call to `mkdir` from the commands. Those aren't needed
>>> anymore because a rune in Rules.mk creates the directory for each
>>> $(targets).
>>> 
>>> Remove "export PYTHON" as it is already exported.
>> 
>> With this change, compiling for x86 is now ending up in:
>> CHK     include/headers99.chk
>> make[9]: execvp: /bin/sh: Argument list too long
>> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>> 
>> Not quite sure yet why but I wanted to signal it early as other might be impacted.
>> 
>> Arm and arm64 builds are not impacted.
> 
> Hmm, that patch has passed the smoke push gate already, so there likely is
> more to it than there being an unconditional issue. I did build-test this
> before pushing, and I've just re-tested on a 2nd system without seeing an
> issue.

I have the problem only when building using Yocto, I did a normal build and the
issue is not coming.

Doing a verbose compilation I have this (sorry for the long lines):

 for i in include/public/vcpu.h include/public/errno.h include/public/kexec.h include/public/argo.h include/public/xen.h include/public/nmi.h include/public/xencomm.h include/public/xenoprof.h include/public/device_tree_defs.h include/public/version.h include/public/memory.h include/public/features.h include/public/sched.h include/public/xen-compat.h include/public/callback.h include/public/vm_event.h include/public/grant_table.h include/public/physdev.h include/public/tmem.h include/public/hypfs.h include/public/platform.h include/public/pmu.h include/public/elfnote.h include/public/trace.h include/public/event_channel.h include/public/io/vscsiif.h include/public/io/kbdif.h include/public/io/protocols.h include/public/io/ring.h include/public/io/displif.h include/public/io/fsif.h include/public/io/blkif.h include/public/io/console.h include/public/io/sndif.h include/public/io/fbif.h include/public/io/libxenvchan.h include/public/io/netif.h include/public/io/usbif.h include/public/io/pciif.h include/public/io/tpmif.h include/public/io/xs_wire.h include/public/io/xenbus.h include/public/io/cameraif.h include/public/hvm/pvdrivers.h include/public/hvm/e820.h include/public/hvm/hvm_xs_strings.h include/public/hvm/dm_op.h include/public/hvm/ioreq.h include/public/hvm/hvm_info_table.h include/public/hvm/hvm_vcpu.h include/public/hvm/hvm_op.h include/public/hvm/params.h; do x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -ansi -Wall -Werror -include stdint.h -S -o /dev/null $i || exit 1; echo $i; done >include/headers.chk.new; mv include/headers.chk.new include/headers.chk
|       rm -f include/headers99.chk.new;  echo "#include "\"include/public/io/9pfs.h\" | x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o /dev/null - || exit $?; echo include/public/io/9pfs.h >> include/headers99.chk.new;  echo "#include "\"include/public/io/pvcalls.h\" | x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o /dev/null - || exit $?; echo include/public/io/pvcalls.h >> include/headers99.chk.new; mv include/headers99.chk.new include/headers99.chk
| make[9]: execvp: /bin/sh: Argument list too long
| make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
| make[9]: *** Waiting for unfinished jobs....

So the command passed to the sub shell by make is quite long.

No idea why this comes out only when building in Yocto but I will dig a bit.

> 
> Also please remember to trim your replies.
> 

Will do.

Bertrand
Jan Beulich June 9, 2022, 12:16 p.m. UTC | #7
On 09.06.2022 13:51, Bertrand Marquis wrote:
>> On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@suse.com> wrote:
>> On 09.06.2022 12:16, Bertrand Marquis wrote:
>>>> On 1 Jun 2022, at 17:59, Anthony PERARD <anthony.perard@citrix.com> wrote:
>>>>
>>>> Use "define" for the headers*_chk commands as otherwise the "#"
>>>> is interpreted as a comment and make can't find the end of
>>>> $(foreach,).
>>>>
>>>> Adding several .PRECIOUS as without them `make` deletes the
>>>> intermediate targets. This is an issue because the macro $(if_changed,)
>>>> check if the target exist in order to decide whether to recreate the
>>>> target.
>>>>
>>>> Removing the call to `mkdir` from the commands. Those aren't needed
>>>> anymore because a rune in Rules.mk creates the directory for each
>>>> $(targets).
>>>>
>>>> Remove "export PYTHON" as it is already exported.
>>>
>>> With this change, compiling for x86 is now ending up in:
>>> CHK     include/headers99.chk
>>> make[9]: execvp: /bin/sh: Argument list too long
>>> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>>>
>>> Not quite sure yet why but I wanted to signal it early as other might be impacted.
>>>
>>> Arm and arm64 builds are not impacted.
>>
>> Hmm, that patch has passed the smoke push gate already, so there likely is
>> more to it than there being an unconditional issue. I did build-test this
>> before pushing, and I've just re-tested on a 2nd system without seeing an
>> issue.
> 
> I have the problem only when building using Yocto, I did a normal build and the
> issue is not coming.
> 
> Doing a verbose compilation I have this (sorry for the long lines):
> 
>  for i in include/public/vcpu.h include/public/errno.h include/public/kexec.h include/public/argo.h include/public/xen.h include/public/nmi.h include/public/xencomm.h include/public/xenoprof.h include/public/device_tree_defs.h include/public/version.h include/public/memory.h include/public/features.h include/public/sched.h include/public/xen-compat.h include/public/callback.h include/public/vm_event.h include/public/grant_table.h include/public/physdev.h include/public/tmem.h include/public/hypfs.h include/public/platform.h include/public/pmu.h include/public/elfnote.h include/public/trace.h include/public/event_channel.h include/public/io/vscsiif.h include/public/io/kbdif.h include/public/io/protocols.h include/public/io/ring.h include/public/io/displif.h include/public/io/fsif.h include/public/io/blkif.h include/public/io/console.h include/public/io/sndif.h include/public/io/fbif.h include/public/io/libxenvchan.h include/public/io/netif.h include/public/io/usbif.h include/public/io/pciif.h include/public/io/tpmif.h include/public/io/xs_wire.h include/public/io/xenbus.h include/public/io/cameraif.h include/public/hvm/pvdrivers.h include/public/hvm/e820.h include/public/hvm/hvm_xs_strings.h include/public/hvm/dm_op.h include/public/hvm/ioreq.h include/public/hvm/hvm_info_table.h include/public/hvm/hvm_vcpu.h include/public/hvm/hvm_op.h include/public/hvm/params.h; do x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -ansi -Wall -Werror -include stdint.h -S -o /dev/null $i || exit 1; echo $i; done >include/headers.chk.new; mv include/headers.chk.new include/headers.chk
> |       rm -f include/headers99.chk.new;  echo "#include "\"include/public/io/9pfs.h\" | x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o /dev/null - || exit $?; echo include/public/io/9pfs.h >> include/headers99.chk.new;  echo "#include "\"include/public/io/pvcalls.h\" | x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o /dev/null - || exit $?; echo include/public/io/pvcalls.h >> include/headers99.chk.new; mv include/headers99.chk.new include/headers99.chk
> | make[9]: execvp: /bin/sh: Argument list too long
> | make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
> | make[9]: *** Waiting for unfinished jobs....
> 
> So the command passed to the sub shell by make is quite long.
> 
> No idea why this comes out only when building in Yocto but I will dig a bit.

Maybe Yocto has an unusually low limit on command arguments' total size?
The whole thing is just over 2500 chars, which doesn't look to be unusually
long for Unix-like environments.

Jan
Bertrand Marquis June 9, 2022, 12:48 p.m. UTC | #8
Hi,

> On 9 Jun 2022, at 13:16, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 09.06.2022 13:51, Bertrand Marquis wrote:
>>> On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@suse.com> wrote:
>>> On 09.06.2022 12:16, Bertrand Marquis wrote:
>>>>> On 1 Jun 2022, at 17:59, Anthony PERARD <anthony.perard@citrix.com> wrote:
>>>>> 
>>>>> Use "define" for the headers*_chk commands as otherwise the "#"
>>>>> is interpreted as a comment and make can't find the end of
>>>>> $(foreach,).
>>>>> 
>>>>> Adding several .PRECIOUS as without them `make` deletes the
>>>>> intermediate targets. This is an issue because the macro $(if_changed,)
>>>>> check if the target exist in order to decide whether to recreate the
>>>>> target.
>>>>> 
>>>>> Removing the call to `mkdir` from the commands. Those aren't needed
>>>>> anymore because a rune in Rules.mk creates the directory for each
>>>>> $(targets).
>>>>> 
>>>>> Remove "export PYTHON" as it is already exported.
>>>> 
>>>> With this change, compiling for x86 is now ending up in:
>>>> CHK     include/headers99.chk
>>>> make[9]: execvp: /bin/sh: Argument list too long
>>>> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>>>> 
>>>> Not quite sure yet why but I wanted to signal it early as other might be impacted.
>>>> 
>>>> Arm and arm64 builds are not impacted.
>>> 
>>> Hmm, that patch has passed the smoke push gate already, so there likely is
>>> more to it than there being an unconditional issue. I did build-test this
>>> before pushing, and I've just re-tested on a 2nd system without seeing an
>>> issue.
>> 
>> I have the problem only when building using Yocto, I did a normal build and the
>> issue is not coming.
>> 
>> Doing a verbose compilation I have this (sorry for the long lines):
>> 
>> for i in include/public/vcpu.h include/public/errno.h include/public/kexec.h include/public/argo.h include/public/xen.h include/public/nmi.h include/public/xencomm.h include/public/xenoprof.h include/public/device_tree_defs.h include/public/version.h include/public/memory.h include/public/features.h include/public/sched.h include/public/xen-compat.h include/public/callback.h include/public/vm_event.h include/public/grant_table.h include/public/physdev.h include/public/tmem.h include/public/hypfs.h include/public/platform.h include/public/pmu.h include/public/elfnote.h include/public/trace.h include/public/event_channel.h include/public/io/vscsiif.h include/public/io/kbdif.h include/public/io/protocols.h include/public/io/ring.h include/public/io/displif.h include/public/io/fsif.h include/public/io/blkif.h include/public/io/console.h include/public/io/sndif.h include/public/io/fbif.h include/public/io/libxenvchan.h include/public/io/netif.h include/public/io/usbif.h include/public/io/pciif.h include/public/io/tpmif.h include/public/io/xs_wire.h include/public/io/xenbus.h include/public/io/cameraif.h include/public/hvm/pvdrivers.h include/public/hvm/e820.h include/public/hvm/hvm_xs_strings.h include/public/hvm/dm_op.h include/public/hvm/ioreq.h include/public/hvm/hvm_info_table.h include/public/hvm/hvm_vcpu.h include/public/hvm/hvm_op.h include/public/hvm/params.h; do x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -ansi -Wall -Werror -include stdint.h -S -o /dev/null $i || exit 1; echo $i; done >include/headers.chk.new; mv include/headers.chk.new include/headers.chk
>> |       rm -f include/headers99.chk.new;  echo "#include "\"include/public/io/9pfs.h\" | x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o /dev/null - || exit $?; echo include/public/io/9pfs.h >> include/headers99.chk.new;  echo "#include "\"include/public/io/pvcalls.h\" | x86_64-poky-linux-gcc  --sysroot=/home/bermar01/Development/xen-dev/build/profile-qemu-x86_64.prj/tmp/work/core2-64-poky-linux/xen/4.17+git1-r0/recipe-sysroot  -x c -std=c99 -Wall -Werror -include stdint.h  -include string.h -S -o /dev/null - || exit $?; echo include/public/io/pvcalls.h >> include/headers99.chk.new; mv include/headers99.chk.new include/headers99.chk
>> | make[9]: execvp: /bin/sh: Argument list too long
>> | make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>> | make[9]: *** Waiting for unfinished jobs....
>> 
>> So the command passed to the sub shell by make is quite long.
>> 
>> No idea why this comes out only when building in Yocto but I will dig a bit.
> 
> Maybe Yocto has an unusually low limit on command arguments' total size?
> The whole thing is just over 2500 chars, which doesn't look to be unusually
> long for Unix-like environments.
> 

Actually the command to generate headers++.chk is 15294 characters when I build xen normally.
In Yocto it becomes a lot bigger as CC includes a sysroot parameter with a path.

Bertrand
Anthony PERARD June 9, 2022, 12:55 p.m. UTC | #9
On Thu, Jun 09, 2022 at 11:51:20AM +0000, Bertrand Marquis wrote:
> Hi,
> 
> > On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@suse.com> wrote:
> > 
> > On 09.06.2022 12:16, Bertrand Marquis wrote:
> >> With this change, compiling for x86 is now ending up in:
> >> CHK     include/headers99.chk
> >> make[9]: execvp: /bin/sh: Argument list too long
> >> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
> >> 
> >> Not quite sure yet why but I wanted to signal it early as other might be impacted.
> >> 
> >> Arm and arm64 builds are not impacted.
> > 
> > Hmm, that patch has passed the smoke push gate already, so there likely is
> > more to it than there being an unconditional issue. I did build-test this
> > before pushing, and I've just re-tested on a 2nd system without seeing an
> > issue.
> 
> I have the problem only when building using Yocto, I did a normal build and the
> issue is not coming.
> 

Will the following patch help?


From 0f32f749304b233c0d5574dc6b14f66e8709feba Mon Sep 17 00:00:00 2001
From: Anthony PERARD <anthony.perard@citrix.com>
Date: Thu, 9 Jun 2022 13:42:52 +0100
Subject: [XEN PATCH] build,include: rework shell script for headers++.chk

The command line generated for headers++.chk by make is quite long,
and in some environment it is too long. This issue have been seen in
Yocto build environment.

Error messages:
    make[9]: execvp: /bin/sh: Argument list too long
    make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127

Rework to that we do the foreach loop in shell rather that make to
reduce the command line size by a lot. We also need a way to get
headers prerequisite for some public headers so we use a switch "case"
in shell to be able to do some simple pattern matching. Variables
alone in POSIX shell don't allow to work with associative array or
variables with "/".

Reported-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
Fixes: 28e13c7f43 ("build: xen/include: use if_changed")
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/include/Makefile | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/xen/include/Makefile b/xen/include/Makefile
index 6d9bcc19b0..ca5e868f38 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -158,13 +158,22 @@ define cmd_headerscxx_chk
 	    touch $@.new;                                                     \
 	    exit 0;                                                           \
 	fi;                                                                   \
-	$(foreach i, $(filter %.h,$^),                                        \
-	    echo "#include "\"$(i)\"                                          \
+	get_prereq() {                                                        \
+	    case $$1 in                                                       \
+	    $(foreach i, $(filter %.h,$^),                                    \
+	    $(if $($(patsubst $(srctree)/%,%,$i)-prereq),                     \
+		$(patsubst $(srctree)/%,%,$i)$(close)                         \
+		echo "$(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq),   \
+			-include c$(j))";;))                                  \
+	    esac;                                                             \
+	};                                                                    \
+	for i in $(filter %.h,$^); do                                         \
+	    echo "#include "\"$$i\"                                           \
 	    | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__        \
 	      -include stdint.h -include $(srcdir)/public/xen.h               \
-	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \
+	      `get_prereq $$i`                                                \
 	      -S -o /dev/null -                                               \
-	    || exit $$?; echo $(i) >> $@.new;) \
+	    || exit $$?; echo $$i >> $@.new; done;                            \
 	mv $@.new $@
 endef
Bertrand Marquis June 9, 2022, 2:26 p.m. UTC | #10
Hi Antony,

> On 9 Jun 2022, at 13:55, Anthony PERARD <anthony.perard@citrix.com> wrote:
> 
> On Thu, Jun 09, 2022 at 11:51:20AM +0000, Bertrand Marquis wrote:
>> Hi,
>> 
>>> On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@suse.com> wrote:
>>> 
>>> On 09.06.2022 12:16, Bertrand Marquis wrote:
>>>> With this change, compiling for x86 is now ending up in:
>>>> CHK     include/headers99.chk
>>>> make[9]: execvp: /bin/sh: Argument list too long
>>>> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>>>> 
>>>> Not quite sure yet why but I wanted to signal it early as other might be impacted.
>>>> 
>>>> Arm and arm64 builds are not impacted.
>>> 
>>> Hmm, that patch has passed the smoke push gate already, so there likely is
>>> more to it than there being an unconditional issue. I did build-test this
>>> before pushing, and I've just re-tested on a 2nd system without seeing an
>>> issue.
>> 
>> I have the problem only when building using Yocto, I did a normal build and the
>> issue is not coming.
>> 
> 
> Will the following patch help?

Yes it does, thanks a lot.

You can add my:
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> 
> 
> From 0f32f749304b233c0d5574dc6b14f66e8709feba Mon Sep 17 00:00:00 2001
> From: Anthony PERARD <anthony.perard@citrix.com>
> Date: Thu, 9 Jun 2022 13:42:52 +0100
> Subject: [XEN PATCH] build,include: rework shell script for headers++.chk
> 
> The command line generated for headers++.chk by make is quite long,
> and in some environment it is too long. This issue have been seen in
> Yocto build environment.
> 
> Error messages:
>    make[9]: execvp: /bin/sh: Argument list too long
>    make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
> 
> Rework to that we do the foreach loop in shell rather that make to
> reduce the command line size by a lot. We also need a way to get
> headers prerequisite for some public headers so we use a switch "case"
> in shell to be able to do some simple pattern matching. Variables
> alone in POSIX shell don't allow to work with associative array or
> variables with "/".
> 
> Reported-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
> Fixes: 28e13c7f43 ("build: xen/include: use if_changed")
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> xen/include/Makefile | 17 +++++++++++++----
> 1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/include/Makefile b/xen/include/Makefile
> index 6d9bcc19b0..ca5e868f38 100644
> --- a/xen/include/Makefile
> +++ b/xen/include/Makefile
> @@ -158,13 +158,22 @@ define cmd_headerscxx_chk
> 	    touch $@.new;                                                     \
> 	    exit 0;                                                           \
> 	fi;                                                                   \
> -	$(foreach i, $(filter %.h,$^),                                        \
> -	    echo "#include "\"$(i)\"                                          \
> +	get_prereq() {                                                        \
> +	    case $$1 in                                                       \
> +	    $(foreach i, $(filter %.h,$^),                                    \
> +	    $(if $($(patsubst $(srctree)/%,%,$i)-prereq),                     \
> +		$(patsubst $(srctree)/%,%,$i)$(close)                         \
> +		echo "$(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq),   \
> +			-include c$(j))";;))                                  \
> +	    esac;                                                             \
> +	};                                                                    \
> +	for i in $(filter %.h,$^); do                                         \
> +	    echo "#include "\"$$i\"                                           \
> 	    | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__        \
> 	      -include stdint.h -include $(srcdir)/public/xen.h               \
> -	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \
> +	      `get_prereq $$i`                                                \
> 	      -S -o /dev/null -                                               \
> -	    || exit $$?; echo $(i) >> $@.new;) \
> +	    || exit $$?; echo $$i >> $@.new; done;                            \
> 	mv $@.new $@
> endef
> 
> 
> 
> 
> -- 
> Anthony PERARD
Bertrand Marquis June 13, 2022, 1:58 p.m. UTC | #11
Hi Anthony,

> On 9 Jun 2022, at 13:55, Anthony PERARD <anthony.perard@citrix.com> wrote:
> 
> On Thu, Jun 09, 2022 at 11:51:20AM +0000, Bertrand Marquis wrote:
>> Hi,
>> 
>>> On 9 Jun 2022, at 11:26, Jan Beulich <jbeulich@suse.com> wrote:
>>> 
>>> On 09.06.2022 12:16, Bertrand Marquis wrote:
>>>> With this change, compiling for x86 is now ending up in:
>>>> CHK     include/headers99.chk
>>>> make[9]: execvp: /bin/sh: Argument list too long
>>>> make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
>>>> 
>>>> Not quite sure yet why but I wanted to signal it early as other might be impacted.
>>>> 
>>>> Arm and arm64 builds are not impacted.
>>> 
>>> Hmm, that patch has passed the smoke push gate already, so there likely is
>>> more to it than there being an unconditional issue. I did build-test this
>>> before pushing, and I've just re-tested on a 2nd system without seeing an
>>> issue.
>> 
>> I have the problem only when building using Yocto, I did a normal build and the
>> issue is not coming.
>> 
> 
> Will the following patch help?
> 
> 
> From 0f32f749304b233c0d5574dc6b14f66e8709feba Mon Sep 17 00:00:00 2001
> From: Anthony PERARD <anthony.perard@citrix.com>
> Date: Thu, 9 Jun 2022 13:42:52 +0100
> Subject: [XEN PATCH] build,include: rework shell script for headers++.chk
> 
> The command line generated for headers++.chk by make is quite long,
> and in some environment it is too long. This issue have been seen in
> Yocto build environment.
> 
> Error messages:
>    make[9]: execvp: /bin/sh: Argument list too long
>    make[9]: *** [include/Makefile:181: include/headers++.chk] Error 127
> 
> Rework to that we do the foreach loop in shell rather that make to
> reduce the command line size by a lot. We also need a way to get
> headers prerequisite for some public headers so we use a switch "case"
> in shell to be able to do some simple pattern matching. Variables
> alone in POSIX shell don't allow to work with associative array or
> variables with "/".
> 
> Reported-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
> Fixes: 28e13c7f43 ("build: xen/include: use if_changed")
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> xen/include/Makefile | 17 +++++++++++++----
> 1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/include/Makefile b/xen/include/Makefile
> index 6d9bcc19b0..ca5e868f38 100644
> --- a/xen/include/Makefile
> +++ b/xen/include/Makefile
> @@ -158,13 +158,22 @@ define cmd_headerscxx_chk
> 	    touch $@.new;                                                     \
> 	    exit 0;                                                           \
> 	fi;                                                                   \
> -	$(foreach i, $(filter %.h,$^),                                        \
> -	    echo "#include "\"$(i)\"                                          \
> +	get_prereq() {                                                        \
> +	    case $$1 in                                                       \
> +	    $(foreach i, $(filter %.h,$^),                                    \
> +	    $(if $($(patsubst $(srctree)/%,%,$i)-prereq),                     \
> +		$(patsubst $(srctree)/%,%,$i)$(close)                         \
> +		echo "$(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq),   \
> +			-include c$(j))";;))                                  \
> +	    esac;                                                             \
> +	};                                                                    \
> +	for i in $(filter %.h,$^); do                                         \
> +	    echo "#include "\"$$i\"                                           \
> 	    | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__        \
> 	      -include stdint.h -include $(srcdir)/public/xen.h               \
> -	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \
> +	      `get_prereq $$i`                                                \
> 	      -S -o /dev/null -                                               \
> -	    || exit $$?; echo $(i) >> $@.new;) \
> +	    || exit $$?; echo $$i >> $@.new; done;                            \
> 	mv $@.new $@
> endef
> 

Just a small reminder that you need to push this patch officially :-)

Cheers
Bertrand
diff mbox series

Patch

diff --git a/xen/include/Makefile b/xen/include/Makefile
index 03baf10efb..6d9bcc19b0 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -45,38 +45,65 @@  public-$(CONFIG_ARM) := $(wildcard $(srcdir)/public/arch-arm/*.h $(srcdir)/publi
 .PHONY: all
 all: $(addprefix $(obj)/,$(headers-y))
 
-$(obj)/compat/%.h: $(obj)/compat/%.i $(srcdir)/Makefile $(srctree)/tools/compat-build-header.py
-	$(PYTHON) $(srctree)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
-	mv -f $@.new $@
-
-$(obj)/compat/%.i: $(obj)/compat/%.c $(srcdir)/Makefile
-	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
-
-$(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srcdir)/Makefile $(srctree)/tools/compat-build-source.py
-	mkdir -p $(@D)
-	$(PYTHON) $(srctree)/tools/compat-build-source.py $(srcdir)/xlat.lst <$< >$@.new
-	mv -f $@.new $@
-
-$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh $(srcdir)/Makefile
-	export PYTHON=$(PYTHON); \
-	while read what name; do \
-		$(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
-	done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >$@.new
-	mv -f $@.new $@
+quiet_cmd_compat_h = GEN     $@
+cmd_compat_h = \
+    $(PYTHON) $(srctree)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
+    mv -f $@.new $@
+
+quiet_cmd_compat_i = CPP     $@
+cmd_compat_i = $(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
+
+quiet_cmd_compat_c = GEN     $@
+cmd_compat_c = \
+   $(PYTHON) $(srctree)/tools/compat-build-source.py $(srcdir)/xlat.lst <$< >$@.new; \
+   mv -f $@.new $@
+
+quiet_cmd_xlat_headers = GEN     $@
+cmd_xlat_headers = \
+    while read what name; do \
+        $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
+    done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >$@.new; \
+    mv -f $@.new $@
+
+targets += $(headers-y)
+$(obj)/compat/%.h: $(obj)/compat/%.i $(srctree)/tools/compat-build-header.py FORCE
+	$(call if_changed,compat_h)
+
+.PRECIOUS: $(obj)/compat/%.i
+targets += $(patsubst %.h, %.i, $(headers-y))
+$(obj)/compat/%.i: $(obj)/compat/%.c FORCE
+	$(call if_changed,compat_i)
+
+.PRECIOUS: $(obj)/compat/%.c
+targets += $(patsubst %.h, %.c, $(headers-y))
+$(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srctree)/tools/compat-build-source.py FORCE
+	$(call if_changed,compat_c)
+
+targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y))
+$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh FORCE
+	$(call if_changed,xlat_headers)
+
+quiet_cmd_xlat_lst = GEN     $@
+cmd_xlat_lst = \
+	grep -v '^[[:blank:]]*$(pound)' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new; \
+	$(call move-if-changed,$@.new,$@)
 
 .PRECIOUS: $(obj)/compat/.xlat/%.lst
-$(obj)/compat/.xlat/%.lst: $(srcdir)/xlat.lst $(srcdir)/Makefile
-	mkdir -p $(@D)
-	grep -v '^[[:blank:]]*#' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new
-	$(call move-if-changed,$@.new,$@)
+targets += $(patsubst compat/%.h, compat/.xlat/%.lst, $(headers-y))
+$(obj)/compat/.xlat/%.lst: $(srcdir)/xlat.lst FORCE
+	$(call if_changed,xlat_lst)
 
 xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(srcdir)/xlat.lst | uniq)
 xlat-y := $(filter $(patsubst compat/%,%,$(headers-y)),$(xlat-y))
 
-$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf $(srcdir)/Makefile
-	cat $(filter %.h,$^) >$@.new
+quiet_cmd_xlat_h = GEN     $@
+cmd_xlat_h = \
+	cat $(filter %.h,$^) >$@.new; \
 	mv -f $@.new $@
 
+$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf FORCE
+	$(call if_changed,xlat_h)
+
 ifeq ($(XEN_TARGET_ARCH),$(XEN_COMPILE_ARCH))
 
 all: $(obj)/headers.chk $(obj)/headers99.chk $(obj)/headers++.chk
@@ -102,27 +129,31 @@  PUBLIC_C99_HEADERS := $(call public-filter-headers,public-c99-headers)
 $(src)/public/io/9pfs.h-prereq := string
 $(src)/public/io/pvcalls.h-prereq := string
 
-$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(srcdir)/Makefile
+quiet_cmd_header_chk = CHK     $@
+cmd_header_chk = \
 	for i in $(filter %.h,$^); do \
 	    $(CC) -x c -ansi -Wall -Werror -include stdint.h \
 	          -S -o /dev/null $$i || exit 1; \
 	    echo $$i; \
-	done >$@.new
+	done >$@.new; \
 	mv $@.new $@
 
-$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(srcdir)/Makefile
-	rm -f $@.new
+quiet_cmd_headers99_chk = CHK     $@
+define cmd_headers99_chk
+	rm -f $@.new; \
 	$(foreach i, $(filter %.h,$^),                                        \
 	    echo "#include "\"$(i)\"                                          \
 	    | $(CC) -x c -std=c99 -Wall -Werror                               \
 	      -include stdint.h                                               \
 	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include $(j).h) \
 	      -S -o /dev/null -                                               \
-	    || exit $$?; echo $(i) >> $@.new;)
+	    || exit $$?; echo $(i) >> $@.new;) \
 	mv $@.new $@
+endef
 
-$(obj)/headers++.chk: $(PUBLIC_HEADERS) $(srcdir)/Makefile
-	rm -f $@.new
+quiet_cmd_headerscxx_chk = CHK     $@
+define cmd_headerscxx_chk
+	rm -f $@.new; \
 	if ! $(CXX) -v >/dev/null 2>&1; then                                  \
 	    touch $@.new;                                                     \
 	    exit 0;                                                           \
@@ -133,8 +164,21 @@  $(obj)/headers++.chk: $(PUBLIC_HEADERS) $(srcdir)/Makefile
 	      -include stdint.h -include $(srcdir)/public/xen.h               \
 	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \
 	      -S -o /dev/null -                                               \
-	    || exit $$?; echo $(i) >> $@.new;)
+	    || exit $$?; echo $(i) >> $@.new;) \
 	mv $@.new $@
+endef
+
+targets += headers.chk
+$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) FORCE
+	$(call if_changed,header_chk)
+
+targets += headers99.chk
+$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) FORCE
+	$(call if_changed,headers99_chk)
+
+targets += headers++.chk
+$(obj)/headers++.chk: $(PUBLIC_HEADERS) FORCE
+	$(call if_changed,headerscxx_chk)
 
 endif