From patchwork Wed Feb 26 11:33:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony PERARD X-Patchwork-Id: 11406307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 398271871 for ; Wed, 26 Feb 2020 11:42:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0AF8124685 for ; Wed, 26 Feb 2020 11:42:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="iMdf0krr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0AF8124685 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j6v3k-0002rA-Es; Wed, 26 Feb 2020 11:41:00 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j6v3j-0002r5-5z for xen-devel@lists.xenproject.org; Wed, 26 Feb 2020 11:40:59 +0000 X-Inumbo-ID: da77f780-588c-11ea-93ff-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id da77f780-588c-11ea-93ff-12813bfff9fa; Wed, 26 Feb 2020 11:40:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1582717257; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=doQK3s5pI+qjw5dpz3WGVz5y0EayaounLL0Eu1jgXW0=; b=iMdf0krr3R9nQlKiQmNv5qRLdbFsAQW2oG9u0bMr5l/eL3ZtdTEwmr4F F9fdPoNCTkYezQNruifLgUrA88JHrKBGZ/uvVrpw2zVKNGqIpKeHc1Qq1 UbRc3SyJf0SOwVr4OQnEIkDwHMRcP+j4MqwxwfyVAhKcrTnRs0TL8QgfD s=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=anthony.perard@citrix.com; spf=Pass smtp.mailfrom=anthony.perard@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of anthony.perard@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of anthony.perard@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: YUy/HLyt8levPAhCrFEfEdi4VMCILD6ZLmISmjVR5S4LW2LFn1FjLp80aH4bxw81eHyixMLu6H yIsKYPxOF5otaozAefwi3Scxd8LJvVhdc3GVSmhx4+lW6CLTnexRDzMpxuE59bMFukV3vZsbt5 hIk7/m29pzmZuoNoOfxr5xLmrMfgdnjkw3fNBYYnc9IJM+Nq3CaWblYTnuI5OUS8pvhPk7NlHb fnjDW2jx+8ab7nukLgBf1s6pm/IpEoq5CW3PejG1EMkGiLUk25LzOl8u8+7toc2NatiDN+rn7Q /rQ= X-SBRS: 2.7 X-MesageID: 13385560 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,487,1574139600"; d="scan'208";a="13385560" From: Anthony PERARD To: Date: Wed, 26 Feb 2020 11:33:48 +0000 Message-ID: <20200226113355.2532224-17-anthony.perard@citrix.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200226113355.2532224-1-anthony.perard@citrix.com> References: <20200226113355.2532224-1-anthony.perard@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Jan Beulich , Anthony PERARD Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The if_changed macro from Linux can record the command used to build a target then compare it on rebuild. Thus if a command has changed, for example due to introducing new flags in CFLAGS or due to using a different compiler, the target will be rebuilt. if_changed_rule checks dependencies like if_changed, but execute rule_$(1) instead of cmd_$(1) when the command is different. A rule_ macro can call more than one cmd_ macro. One of the cmd_ macro in a rule need to be call using a macro that record the command line, so cmd_and_record is introduced. It is similar to cmd_and_fixup from Linux but without a call to fixdep which we don't have yet. (We will later replace cmd_and_record by cmd_and_fixup.) Example of a rule_ macro: define rule_cc_o_c $(call cmd_and_record,cc_o_o) $(call cmd,objcopy) endef This needs one of the call to use cmd_and_record, otherwise no .*.cmd file will be created, and the target will keep been rebuilt. In order for if_changed to works correctly, we need to load the .%.cmd files that the macro generates, this is done by adding targets in to the $(targets) variable. We use intermediate_targets to add %.init.o dependency %.o to target since there aren't in obj-y. We also add $(MAKECMDGOALS) to targets so that when running for example `make common/memory.i`, make will load the associated .%.cmd dependency file. Beside the if_changed*, we import the machinery used for a "beautify output". The important one is when running make with V=2 which help to debug the makefiles by printing why a target is been rebuilt, via the $(echo-why) macro. if_changed and if_changed_rule aren't used yet. Most of this code is copied from Linux v5.4. Signed-off-by: Anthony PERARD --- .gitignore | 1 + xen/Makefile | 53 +++++++++++++++++- xen/Rules.mk | 33 +++++++++++- xen/scripts/Kbuild.include | 107 +++++++++++++++++++++++++++++++++++++ 4 files changed, 192 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index 4ca679ddbc9a..c73f9f480780 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,7 @@ *.o *.d *.d2 +.*.cmd *.opic *.a *.so diff --git a/xen/Makefile b/xen/Makefile index da017dc29d36..fbd087e6f360 100644 --- a/xen/Makefile +++ b/xen/Makefile @@ -52,7 +52,57 @@ dist: install ifeq ($(root-make-done),) # section to run before calling Rules.mk, but only once. + +# Beautify output +# --------------------------------------------------------------------------- +# +# Normally, we echo the whole command before executing it. By making +# that echo $($(quiet)$(cmd)), we now have the possibility to set +# $(quiet) to choose other forms of output instead, e.g. +# +# quiet_cmd_cc_o_c = Compiling $(RELDIR)/$@ +# cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< +# +# If $(quiet) is empty, the whole command will be printed. +# If it is set to "quiet_", only the short version will be printed. +# If it is set to "silent_", nothing will be printed at all, since +# the variable $(silent_cmd_cc_o_c) doesn't exist. +# +# A simple variant is to prefix commands with $(Q) - that's useful +# for commands that shall be hidden in non-verbose mode. # +# $(Q)ln $@ :< +# +# If KBUILD_VERBOSE equals 0 then the above command will be hidden. +# If KBUILD_VERBOSE equals 1 then the above command is displayed. +# +# To put more focus on warnings, be less verbose as default +# Use 'make V=1' to see the full commands + +ifeq ("$(origin V)", "command line") + KBUILD_VERBOSE = $(V) +endif +ifndef KBUILD_VERBOSE + KBUILD_VERBOSE = 0 +endif + +ifeq ($(KBUILD_VERBOSE),1) + quiet = + Q = +else + quiet=quiet_ + Q = @ +endif + +# If the user is running make -s (silent mode), suppress echoing of +# commands + +ifneq ($(findstring s,$(filter-out --%,$(MAKEFLAGS))),) + quiet=silent_ +endif + +export quiet Q KBUILD_VERBOSE + # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to tools/kconfig/Makefile @@ -258,7 +308,8 @@ _clean: delete-unfresh-files $(MAKE) $(clean) arch/x86 $(MAKE) $(clean) test $(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) clean - find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" -o -name "*.gcno" \) -exec rm -f {} \; + find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" \ + -o -name "*.gcno" -o -name ".*.cmd" \) -exec rm -f {} \; rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core rm -f include/asm-*/asm-offsets.h rm -f .banner diff --git a/xen/Rules.mk b/xen/Rules.mk index f1311c45a372..8807a2e21c94 100644 --- a/xen/Rules.mk +++ b/xen/Rules.mk @@ -38,6 +38,7 @@ ALL_OBJS-y += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o ALL_OBJS-$(CONFIG_CRYPTO) += $(BASEDIR)/crypto/built_in.o # Initialise some variables +targets := CFLAGS-y := AFLAGS-y := @@ -65,6 +66,10 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend subdir-y := $(subdir-y) $(filter %/, $(obj-y)) obj-y := $(patsubst %/, %/built_in.o, $(obj-y)) +# $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to +# tell kbuild to descend +subdir-obj-y := $(filter %/built_in.o, $(obj-y)) + $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY ifeq ($(CONFIG_COVERAGE),y) @@ -120,6 +125,10 @@ else endif endif +targets += built_in.o +targets += $(filter-out $(subdir-obj-y), $(obj-y)) $(extra-y) +targets += $(MAKECMDGOALS) + built_in_bin.o: $(obj-bin-y) $(extra-y) ifeq ($(obj-bin-y),) $(CC) $(a_flags) -c -x assembler /dev/null -o $@ @@ -128,7 +137,7 @@ else endif # Force execution of pattern rules (for which PHONY cannot be directly used). -.PHONY: FORCE +PHONY += FORCE FORCE: %/built_in.o: FORCE @@ -176,4 +185,26 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile %.s: %.S Makefile $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@ +# Add intermediate targets: +# When building objects with specific suffix patterns, add intermediate +# targets that the final targets are derived from. +intermediate_targets = $(foreach sfx, $(2), \ + $(patsubst %$(strip $(1)),%$(sfx), \ + $(filter %$(strip $(1)), $(targets)))) +# %.init.o <- %.o +targets += $(call intermediate_targets, .init.o, .o) + -include $(DEPS_INCLUDE) + +# Read all saved command lines and dependencies for the $(targets) we +# may be building above, using $(if_changed{,_dep}). As an +# optimization, we don't need to read them if the target does not +# exist, we will rebuild anyway in that case. + +existing-targets := $(wildcard $(sort $(targets))) + +-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd) + +# Declare the contents of the PHONY variable as phony. We keep that +# information in a variable so we can use it in if_changed and friends. +.PHONY: $(PHONY) diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include index 14bd4e110b45..f24d664db5ff 100644 --- a/xen/scripts/Kbuild.include +++ b/xen/scripts/Kbuild.include @@ -2,11 +2,30 @@ #### # kbuild: Generic definitions +# Convenient variables +squote := ' +empty := +space := $(empty) $(empty) +space_escape := _-_SPACE_-_ +pound := \# + +### +# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o +dot-target = $(@D)/.$(@F) + ### # dependencies DEPS = .*.d DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS)))) +### +# real prerequisites without phony targets +real-prereqs = $(filter-out $(PHONY), $^) + +### +# Escape single quote for use in echo statements +escsq = $(subst $(squote),'\$(squote)',$1) + # as-insn: Check whether assembler supports an instruction. # Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no) as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \ @@ -32,3 +51,91 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e # Usage: # $(MAKE) $(clean) dir clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C + +# echo command. +# Short version is used, if $(quiet) equals `quiet_', otherwise full one. +echo-cmd = $(if $($(quiet)cmd_$(1)),\ + echo ' $(call escsq,$($(quiet)cmd_$(1)))$(echo-why)';) + +# printing commands +cmd = @set -e; $(echo-cmd) $(cmd_$(1)) + +### +# if_changed - execute command if any prerequisite is newer than +# target, or command line has changed +# if_changed_rule - as if_changed but execute rule instead + +ifneq ($(KBUILD_NOCMDDEP),1) +# Check if both commands are the same including their order. Result is empty +# string if equal. User may override this check using make KBUILD_NOCMDDEP=1 +cmd-check = $(filter-out $(subst $(space),$(space_escape),$(strip $(cmd_$@))), \ + $(subst $(space),$(space_escape),$(strip $(cmd_$1)))) +else +cmd-check = $(if $(strip $(cmd_$@)),,1) +endif + +# Replace >$< with >$$< to preserve $ when reloading the .cmd file +# (needed for make) +# Replace >#< with >$(pound)< to avoid starting a comment in the .cmd file +# (needed for make) +# Replace >'< with >'\''< to be able to enclose the whole string in '...' +# (needed for the shell) +make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1))))) + +# Find any prerequisites that is newer than target or that does not exist. +# PHONY targets skipped in both cases. +any-prereq = $(filter-out $(PHONY),$?)$(filter-out $(PHONY) $(wildcard $^),$^) + +# Execute command if command has changed or prerequisite(s) are updated. +if_changed = $(if $(any-prereq)$(cmd-check), \ + $(cmd); \ + printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:) + +# Usage: $(call if_changed_rule,foo) +# Will check if $(cmd_foo) or any of the prerequisites changed, +# and if so will execute $(rule_foo). +if_changed_rule = $(if $(any-prereq)$(cmd-check),$(rule_$(1)),@:) + +cmd_and_record = \ + $(cmd); \ + printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd + +### +# why - tell why a target got built +# enabled by make V=2 +# Output (listed in the order they are checked): +# (1) - due to target is PHONY +# (2) - due to target missing +# (3) - due to: file1.h file2.h +# (4) - due to command line change +# (5) - due to missing .cmd file +# (6) - due to target not in $(targets) +# (1) PHONY targets are always build +# (2) No target, so we better build it +# (3) Prerequisite is newer than target +# (4) The command line stored in the file named dir/.target.cmd +# differed from actual command line. This happens when compiler +# options changes +# (5) No dir/.target.cmd file (used to store command line) +# (6) No dir/.target.cmd file and target not listed in $(targets) +# This is a good hint that there is a bug in the kbuild file +ifeq ($(KBUILD_VERBOSE),2) +why = \ + $(if $(filter $@, $(PHONY)),- due to target is PHONY, \ + $(if $(wildcard $@), \ + $(if $(any-prereq),- due to: $(any-prereq), \ + $(if $(cmd-check), \ + $(if $(cmd_$@),- due to command line change, \ + $(if $(filter $@, $(targets)), \ + - due to missing .cmd file, \ + - due to $(notdir $@) not in $$(targets) \ + ) \ + ) \ + ) \ + ), \ + - due to target missing \ + ) \ + ) + +echo-why = $(call escsq, $(strip $(why))) +endif