diff mbox series

[XEN,v2,12/12] xen/build: have the root Makefile generates the CFLAGS

Message ID 20200117105358.607910-13-anthony.perard@citrix.com (mailing list archive)
State Superseded
Headers show
Series xen: Build system improvements | expand

Commit Message

Anthony PERARD Jan. 17, 2020, 10:53 a.m. UTC 
Instead of generating the CFLAGS in Rules.mk everytime we enter a new
subdirectory, we are going to generate most of them a single time, and
export the result in the environment so that Rules.mk can use it.  The
only flags left to generates are the one that depends on the targets,
but the variable $(c_flags) takes care of that.

Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
which is included by the root Makefile.

In order to allow some variable that are specific to one arch and
needs to be regenerated for each target, there's a new variable
$(arch_ccflags).

When running Rules.mk in the root directory (xen/), the variable
`root-make-done' is set, so `need-config' will remain undef and so the
root Makefile will not generate the cflags again.

There is on FIXME added about LTO build, but since LTO is marked as
BROKEN, this commit doesn't attempt to filter -flto flags out of the
CFLAGS.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Makefile               | 70 ++++++++++++++++++++++++++++
 xen/Rules.mk               | 70 +++++-----------------------
 xen/arch/arm/Makefile      | 10 ++--
 xen/arch/arm/Rules.mk      | 93 --------------------------------------
 xen/arch/arm/arch.mk       | 88 ++++++++++++++++++++++++++++++++++++
 xen/arch/x86/Makefile      | 16 +++----
 xen/arch/x86/Rules.mk      | 91 +++----------------------------------
 xen/arch/x86/arch.mk       | 87 +++++++++++++++++++++++++++++++++++
 xen/common/libelf/Makefile |  2 +-
 xen/common/libfdt/Makefile |  2 +-
 10 files changed, 278 insertions(+), 251 deletions(-)
 create mode 100644 xen/arch/arm/arch.mk
 create mode 100644 xen/arch/x86/arch.mk

Comments

Jan Beulich Jan. 30, 2020, 2:33 p.m. UTC | #1 
On 17.01.2020 11:53, Anthony PERARD wrote:
> Instead of generating the CFLAGS in Rules.mk everytime we enter a new
> subdirectory, we are going to generate most of them a single time, and
> export the result in the environment so that Rules.mk can use it.  The
> only flags left to generates are the one that depends on the targets,
> but the variable $(c_flags) takes care of that.
> 
> Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
> which is included by the root Makefile.
> 
> In order to allow some variable that are specific to one arch and
> needs to be regenerated for each target, there's a new variable
> $(arch_ccflags).

And simply adding to c_flags is considered bad? Or does not work?

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -113,6 +113,76 @@ $(KCONFIG_CONFIG):
>  %/auto.conf %/auto.conf.cmd: $(KCONFIG_CONFIG)
>  	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
>  
> +ifeq ($(CONFIG_DEBUG),y)
> +CFLAGS += -O1
> +else
> +CFLAGS += -O2
> +endif

Why does this start with +=, not := (or = )?

> +ifeq ($(CONFIG_FRAME_POINTER),y)
> +CFLAGS += -fno-omit-frame-pointer
> +else
> +CFLAGS += -fomit-frame-pointer
> +endif
> +
> +CFLAGS += -nostdinc -fno-builtin -fno-common
> +CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
> +$(call cc-option-add,CFLAGS,CC,-Wvla)
> +CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
> +CFLAGS-$(CONFIG_DEBUG_INFO) += -g
> +
> +ifneq ($(CONFIG_CC_IS_CLANG),y)
> +# Clang doesn't understand this command line argument, and doesn't appear to
> +# have an suitable alternative.  The resulting compiled binary does function,
> +# but has an excessively large symbol table.
> +CFLAGS += -Wa,--strip-local-absolute
> +endif
> +
> +AFLAGS-y                += -D__ASSEMBLY__

Why not just AFLAGS? I think in a overhaul like what you do,
anomalies like this one would better be eliminated. The -y
forms should be added into the base variables (like you do ...

> +CFLAGS += $(CFLAGS-y)

... here), but be added to only via CFLAGS-$(variable)
constructs. Or otherwise there should be only CFLAGS-y, and
no plain CFLAGS at all.

> +# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
> +CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
> +
> +# Most CFLAGS are safe for assembly files:
> +#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
> +#  -flto makes no sense and annoys clang
> +AFLAGS += $(AFLAGS-y) $(filter-out -std=gnu% -flto,$(CFLAGS))
> +
> +# LDFLAGS are only passed directly to $(LD)
> +LDFLAGS += $(LDFLAGS_DIRECT)
> +
> +LDFLAGS += $(LDFLAGS-y)

These two could be folded.

> +ifeq ($(CONFIG_COVERAGE),y)
> +ifeq ($(CONFIG_CC_IS_CLANG),y)
> +    COV_FLAGS := -fprofile-instr-generate -fcoverage-mapping
> +else
> +    COV_FLAGS := -fprofile-arcs -ftest-coverage
> +endif
> +else
> +COV_FLAGS :=
> +endif

COV_FLAGS gets propagated through the environment, despite being
invariant. Can't this stay in Rules.mk?

> --- a/xen/arch/x86/Rules.mk
> +++ b/xen/arch/x86/Rules.mk
> @@ -1,89 +1,10 @@
>  ########################################
>  # x86-specific definitions
>  
> -XEN_IMG_OFFSET := 0x200000
> -
> -CFLAGS += -I$(BASEDIR)/include
> -CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
> -CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
> -CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
> -CFLAGS += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
> -
> -# Prevent floating-point variables from creeping into Xen.
> -CFLAGS += -msoft-float
> -
> -ifeq ($(CONFIG_CC_IS_CLANG),y)
> -# Note: Any test which adds -no-integrated-as will cause subsequent tests to
> -# succeed, and not trigger further additions.
> -#
> -# The tests to select whether the integrated assembler is usable need to happen
> -# before testing any assembler features, or else the result of the tests would
> -# be stale if the integrated assembler is not used.
> -
> -# Older clang's built-in assembler doesn't understand .skip with labels:
> -# https://bugs.llvm.org/show_bug.cgi?id=27369
> -$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
> -                     -no-integrated-as)
> -
> -# Check whether clang asm()-s support .include.
> -$(call as-option-add,CFLAGS,CC,".include \"asm/indirect_thunk_asm.h\"",,\
> -                     -no-integrated-as)
> -
> -# Check whether clang keeps .macro-s between asm()-s:
> -# https://bugs.llvm.org/show_bug.cgi?id=36110
> -$(call as-option-add,CFLAGS,CC,\
> -                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
> -                     -no-integrated-as)
> -endif
> -
> -$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> -$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
> -$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
> -$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
> -$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
> -$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
> -$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
> -$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
> -$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
> -$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
> -$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1", \
> -                     -U__OBJECT_LABEL__ -DHAVE_AS_QUOTED_SYM \
> -                     '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$$@')
> -$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
> -
> -# GAS's idea of true is -1.  Clang's idea is 1
> -$(call as-option-add,CFLAGS,CC,\
> -    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
> -
> -# Check to see whether the assmbler supports the .nop directive.
> -$(call as-option-add,CFLAGS,CC,\
> -    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
> -
> -CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
> -
> -# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
> -# SSE setup for variadic function calls.
> -CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
> -
> -# Compile with thunk-extern, indirect-branch-register if avaiable.
> -ifeq ($(CONFIG_INDIRECT_THUNK),y)
> -CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
> -CFLAGS += -fno-jump-tables
> +ifdef HAVE_AS_QUOTED_SYM
> +arch_ccflags += -DHAVE_AS_QUOTED_SYM \
> +		'-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$@'
> +else
> +arch_ccflags += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
>  endif

Why does HAVE_AS_QUOTED_SYM need a make / environment variable to
propagate? Can't this be as-option-add against arch_ccflags (or
c_flags), in arch.mk or Rules.mk? Or can't arch.mk have

$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1",-DHAVE_AS_QUOTED_SYM)

and then here you simply check CFLAGS for this specific -D option?

> --- /dev/null
> +++ b/xen/arch/x86/arch.mk
> @@ -0,0 +1,87 @@
> +########################################
> +# x86-specific definitions
> +
> +export XEN_IMG_OFFSET := 0x200000
> +
> +CFLAGS += -I$(BASEDIR)/include
> +CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
> +CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
> +CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
> +
> +# Prevent floating-point variables from creeping into Xen.
> +CFLAGS += -msoft-float
> +
> +ifeq ($(CONFIG_CC_IS_CLANG),y)
> +# Note: Any test which adds -no-integrated-as will cause subsequent tests to
> +# succeed, and not trigger further additions.
> +#
> +# The tests to select whether the integrated assembler is usable need to happen
> +# before testing any assembler features, or else the result of the tests would
> +# be stale if the integrated assembler is not used.
> +
> +# Older clang's built-in assembler doesn't understand .skip with labels:
> +# https://bugs.llvm.org/show_bug.cgi?id=27369
> +$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
> +                     -no-integrated-as)
> +
> +# Check whether clang asm()-s support .include.
> +$(call as-option-add,CFLAGS,CC,".include \"asm/indirect_thunk_asm.h\"",,\
> +                     -no-integrated-as)
> +
> +# Check whether clang keeps .macro-s between asm()-s:
> +# https://bugs.llvm.org/show_bug.cgi?id=36110
> +$(call as-option-add,CFLAGS,CC,\
> +                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
> +                     -no-integrated-as)
> +endif
> +
> +$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> +$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
> +$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
> +$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
> +$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
> +$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
> +$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
> +$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
> +$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
> +$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
> +ifeq ($(call as-insn,$(CC) $(CFLAGS),".equ \"x\"$(comma)1",y),y)
> +  export HAVE_AS_QUOTED_SYM := y
> +endif
> +$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
> +
> +# GAS's idea of true is -1.  Clang's idea is 1
> +$(call as-option-add,CFLAGS,CC,\
> +    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
> +
> +# Check to see whether the assmbler supports the .nop directive.
> +$(call as-option-add,CFLAGS,CC,\
> +    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
> +
> +CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
> +
> +# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
> +# SSE setup for variadic function calls.
> +CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
> +
> +# Compile with thunk-extern, indirect-branch-register if avaiable.
> +ifeq ($(CONFIG_INDIRECT_THUNK),y)
> +CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
> +CFLAGS += -fno-jump-tables
> +endif

CFLAGS-$(CONFIG_INDIRECT_THUNK) += ... ?

> +# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
> +# this to be overridden elsewhere.
> +$(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
> +export CFLAGS-stack-boundary

I find such random export unfortunate, but I can see why - within
the targeted model - this is the least bad alternative.

Jan
Anthony PERARD Feb. 3, 2020, 1:57 p.m. UTC | #2 
On Thu, Jan 30, 2020 at 03:33:15PM +0100, Jan Beulich wrote:
> On 17.01.2020 11:53, Anthony PERARD wrote:
> > Instead of generating the CFLAGS in Rules.mk everytime we enter a new
> > subdirectory, we are going to generate most of them a single time, and
> > export the result in the environment so that Rules.mk can use it.  The
> > only flags left to generates are the one that depends on the targets,
> > but the variable $(c_flags) takes care of that.
> > 
> > Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
> > which is included by the root Makefile.
> > 
> > In order to allow some variable that are specific to one arch and
> > needs to be regenerated for each target, there's a new variable
> > $(arch_ccflags).
> 
> And simply adding to c_flags is considered bad? Or does not work?

I could try to add directly to c_flags, it might not be that bad.
arch_ccflags is my invention, Linux doesn't have something similar.

> > --- a/xen/Makefile
> > +++ b/xen/Makefile
> > @@ -113,6 +113,76 @@ $(KCONFIG_CONFIG):
> >  %/auto.conf %/auto.conf.cmd: $(KCONFIG_CONFIG)
> >  	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
> >  
> > +ifeq ($(CONFIG_DEBUG),y)
> > +CFLAGS += -O1
> > +else
> > +CFLAGS += -O2
> > +endif
> 
> Why does this start with +=, not := (or = )?

Makefile includes Config.mk before these lines, so having += simply add
to the CFLAGS already generated by Config.mk.

> > +ifeq ($(CONFIG_FRAME_POINTER),y)
> > +CFLAGS += -fno-omit-frame-pointer
> > +else
> > +CFLAGS += -fomit-frame-pointer
> > +endif
> > +
> > +CFLAGS += -nostdinc -fno-builtin -fno-common
> > +CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
> > +$(call cc-option-add,CFLAGS,CC,-Wvla)
> > +CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
> > +CFLAGS-$(CONFIG_DEBUG_INFO) += -g
> > +
> > +ifneq ($(CONFIG_CC_IS_CLANG),y)
> > +# Clang doesn't understand this command line argument, and doesn't appear to
> > +# have an suitable alternative.  The resulting compiled binary does function,
> > +# but has an excessively large symbol table.
> > +CFLAGS += -Wa,--strip-local-absolute
> > +endif
> > +
> > +AFLAGS-y                += -D__ASSEMBLY__
> 
> Why not just AFLAGS? I think in a overhaul like what you do,
> anomalies like this one would better be eliminated. The -y
> forms should be added into the base variables (like you do ...

I wanted to avoid too much modification, and mostly want to move the code
around. So it would be easier to check that the commit doesn't introduce
errors.  So, if your are fine with patch that move code and modify it,
I'll fix some of the oddities. (Or I can have another patch for it.)

> > +CFLAGS += $(CFLAGS-y)
> 
> ... here), but be added to only via CFLAGS-$(variable)
> constructs. Or otherwise there should be only CFLAGS-y, and
> no plain CFLAGS at all.
> 
> > +# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
> > +CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
> > +
> > +# Most CFLAGS are safe for assembly files:
> > +#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
> > +#  -flto makes no sense and annoys clang
> > +AFLAGS += $(AFLAGS-y) $(filter-out -std=gnu% -flto,$(CFLAGS))
> > +
> > +# LDFLAGS are only passed directly to $(LD)
> > +LDFLAGS += $(LDFLAGS_DIRECT)
> > +
> > +LDFLAGS += $(LDFLAGS-y)
> 
> These two could be folded.
> 
> > +ifeq ($(CONFIG_COVERAGE),y)
> > +ifeq ($(CONFIG_CC_IS_CLANG),y)
> > +    COV_FLAGS := -fprofile-instr-generate -fcoverage-mapping
> > +else
> > +    COV_FLAGS := -fprofile-arcs -ftest-coverage
> > +endif
> > +else
> > +COV_FLAGS :=
> > +endif
> 
> COV_FLAGS gets propagated through the environment, despite being
> invariant. Can't this stay in Rules.mk?

I guess that's fine.

> > --- a/xen/arch/x86/Rules.mk
> > +++ b/xen/arch/x86/Rules.mk
> > @@ -1,89 +1,10 @@
> >  ########################################
> >  # x86-specific definitions
> >  
> > -XEN_IMG_OFFSET := 0x200000
> > -
> > -CFLAGS += -I$(BASEDIR)/include
> > -CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
> > -CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
> > -CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
> > -CFLAGS += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
> > -
> > -# Prevent floating-point variables from creeping into Xen.
> > -CFLAGS += -msoft-float
> > -
> > -ifeq ($(CONFIG_CC_IS_CLANG),y)
> > -# Note: Any test which adds -no-integrated-as will cause subsequent tests to
> > -# succeed, and not trigger further additions.
> > -#
> > -# The tests to select whether the integrated assembler is usable need to happen
> > -# before testing any assembler features, or else the result of the tests would
> > -# be stale if the integrated assembler is not used.
> > -
> > -# Older clang's built-in assembler doesn't understand .skip with labels:
> > -# https://bugs.llvm.org/show_bug.cgi?id=27369
> > -$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
> > -                     -no-integrated-as)
> > -
> > -# Check whether clang asm()-s support .include.
> > -$(call as-option-add,CFLAGS,CC,".include \"asm/indirect_thunk_asm.h\"",,\
> > -                     -no-integrated-as)
> > -
> > -# Check whether clang keeps .macro-s between asm()-s:
> > -# https://bugs.llvm.org/show_bug.cgi?id=36110
> > -$(call as-option-add,CFLAGS,CC,\
> > -                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
> > -                     -no-integrated-as)
> > -endif
> > -
> > -$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> > -$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
> > -$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
> > -$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
> > -$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
> > -$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
> > -$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
> > -$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
> > -$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
> > -$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
> > -$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1", \
> > -                     -U__OBJECT_LABEL__ -DHAVE_AS_QUOTED_SYM \
> > -                     '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$$@')
> > -$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
> > -
> > -# GAS's idea of true is -1.  Clang's idea is 1
> > -$(call as-option-add,CFLAGS,CC,\
> > -    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
> > -
> > -# Check to see whether the assmbler supports the .nop directive.
> > -$(call as-option-add,CFLAGS,CC,\
> > -    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
> > -
> > -CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
> > -
> > -# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
> > -# SSE setup for variadic function calls.
> > -CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
> > -
> > -# Compile with thunk-extern, indirect-branch-register if avaiable.
> > -ifeq ($(CONFIG_INDIRECT_THUNK),y)
> > -CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
> > -CFLAGS += -fno-jump-tables
> > +ifdef HAVE_AS_QUOTED_SYM
> > +arch_ccflags += -DHAVE_AS_QUOTED_SYM \
> > +		'-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$@'
> > +else
> > +arch_ccflags += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
> >  endif
> 
> Why does HAVE_AS_QUOTED_SYM need a make / environment variable to
> propagate? Can't this be as-option-add against arch_ccflags (or
> c_flags), in arch.mk or Rules.mk? Or can't arch.mk have
> 
> $(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1",-DHAVE_AS_QUOTED_SYM)
> 
> and then here you simply check CFLAGS for this specific -D option?

Sounds good, I'll do that.

> > +ifeq ($(CONFIG_INDIRECT_THUNK),y)
> > +CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
> > +CFLAGS += -fno-jump-tables
> > +endif
> 
> CFLAGS-$(CONFIG_INDIRECT_THUNK) += ... ?

Will change.

Thanks,
Jan Beulich Feb. 3, 2020, 3:26 p.m. UTC | #3 
On 03.02.2020 14:57, Anthony PERARD wrote:
> On Thu, Jan 30, 2020 at 03:33:15PM +0100, Jan Beulich wrote:
>> On 17.01.2020 11:53, Anthony PERARD wrote:
>>> +ifneq ($(CONFIG_CC_IS_CLANG),y)
>>> +# Clang doesn't understand this command line argument, and doesn't appear to
>>> +# have an suitable alternative.  The resulting compiled binary does function,
>>> +# but has an excessively large symbol table.
>>> +CFLAGS += -Wa,--strip-local-absolute
>>> +endif
>>> +
>>> +AFLAGS-y                += -D__ASSEMBLY__
>>
>> Why not just AFLAGS? I think in a overhaul like what you do,
>> anomalies like this one would better be eliminated. The -y
>> forms should be added into the base variables (like you do ...
> 
> I wanted to avoid too much modification, and mostly want to move the code
> around. So it would be easier to check that the commit doesn't introduce
> errors.  So, if your are fine with patch that move code and modify it,
> I'll fix some of the oddities. (Or I can have another patch for it.)

With such extra adjustments at least briefly mentioned in the
description, folding in is fine as far as I'm concerned.

Jan
diff mbox series

Patch

diff --git a/xen/Makefile b/xen/Makefile
index 80679b4a6b17..22ca963994c8 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -113,6 +113,76 @@  $(KCONFIG_CONFIG):
 %/auto.conf %/auto.conf.cmd: $(KCONFIG_CONFIG)
 	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
 
+ifeq ($(CONFIG_DEBUG),y)
+CFLAGS += -O1
+else
+CFLAGS += -O2
+endif
+
+ifeq ($(CONFIG_FRAME_POINTER),y)
+CFLAGS += -fno-omit-frame-pointer
+else
+CFLAGS += -fomit-frame-pointer
+endif
+
+CFLAGS += -nostdinc -fno-builtin -fno-common
+CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
+$(call cc-option-add,CFLAGS,CC,-Wvla)
+CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
+CFLAGS-$(CONFIG_DEBUG_INFO) += -g
+
+ifneq ($(CONFIG_CC_IS_CLANG),y)
+# Clang doesn't understand this command line argument, and doesn't appear to
+# have an suitable alternative.  The resulting compiled binary does function,
+# but has an excessively large symbol table.
+CFLAGS += -Wa,--strip-local-absolute
+endif
+
+AFLAGS-y                += -D__ASSEMBLY__
+
+CFLAGS += $(CFLAGS-y)
+# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
+CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
+
+# Most CFLAGS are safe for assembly files:
+#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
+#  -flto makes no sense and annoys clang
+AFLAGS += $(AFLAGS-y) $(filter-out -std=gnu% -flto,$(CFLAGS))
+
+# LDFLAGS are only passed directly to $(LD)
+LDFLAGS += $(LDFLAGS_DIRECT)
+
+LDFLAGS += $(LDFLAGS-y)
+
+ifeq ($(CONFIG_COVERAGE),y)
+ifeq ($(CONFIG_CC_IS_CLANG),y)
+    COV_FLAGS := -fprofile-instr-generate -fcoverage-mapping
+else
+    COV_FLAGS := -fprofile-arcs -ftest-coverage
+endif
+else
+COV_FLAGS :=
+endif
+
+ifeq ($(CONFIG_UBSAN),y)
+CFLAGS_UBSAN := -fsanitize=undefined
+else
+CFLAGS_UBSAN :=
+endif
+
+ifeq ($(CONFIG_LTO),y)
+CFLAGS += -flto
+LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
+endif
+
+include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
+
+# define new variables to avoid the ones defines in Config.mk
+export XEN_CFLAGS := $(CFLAGS)
+export XEN_AFLAGS := $(AFLAGS)
+export XEN_LDFLAGS := $(LDFLAGS)
+export COV_FLAGS CFLAGS_UBSAN
+
 endif # need-config
 
 .PHONY: build install uninstall clean distclean MAP
diff --git a/xen/Rules.mk b/xen/Rules.mk
index f0111f2bc1b4..faf462a0178a 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -38,52 +38,12 @@  ALL_OBJS-y               += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o
 ALL_OBJS-$(CONFIG_CRYPTO)   += $(BASEDIR)/crypto/built_in.o
 
 # Initialise some variables
-CFLAGS_UBSAN :=
 ccflags-y :=
-
-ifeq ($(CONFIG_DEBUG),y)
-CFLAGS += -O1
-else
-CFLAGS += -O2
-endif
-
-ifeq ($(CONFIG_FRAME_POINTER),y)
-CFLAGS += -fno-omit-frame-pointer
-else
-CFLAGS += -fomit-frame-pointer
-endif
-
-CFLAGS += -nostdinc -fno-builtin -fno-common
-CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
-$(call cc-option-add,CFLAGS,CC,-Wvla)
-CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
-CFLAGS-$(CONFIG_DEBUG_INFO) += -g
-
-ifneq ($(CONFIG_CC_IS_CLANG),y)
-# Clang doesn't understand this command line argument, and doesn't appear to
-# have an suitable alternative.  The resulting compiled binary does function,
-# but has an excessively large symbol table.
-CFLAGS += -Wa,--strip-local-absolute
-endif
-
-AFLAGS-y                += -D__ASSEMBLY__
+# Allow arch specific cflags, to be calculated for each objects.
+arch_ccflags =
 
 ALL_OBJS := $(ALL_OBJS-y)
 
-CFLAGS += $(CFLAGS-y)
-# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
-CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
-
-# Most CFLAGS are safe for assembly files:
-#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
-#  -flto makes no sense and annoys clang
-AFLAGS += $(AFLAGS-y) $(filter-out -std=gnu% -flto,$(CFLAGS))
-
-# LDFLAGS are only passed directly to $(LD)
-LDFLAGS += $(LDFLAGS_DIRECT)
-
-LDFLAGS += $(LDFLAGS-y)
-
 include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
 include Makefile
@@ -103,27 +63,19 @@  __subdir-y	:= $(filter %/, $(obj-y))
 subdir-y	+= $(__subdir-y)
 obj-y		:= $(patsubst %/, %/built_in.o, $(obj-y))
 
-$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += -DINIT_SECTIONS_ONLY
+$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): ccflags-y += -DINIT_SECTIONS_ONLY
 
 ifeq ($(CONFIG_COVERAGE),y)
-ifeq ($(CONFIG_CC_IS_CLANG),y)
-    COV_FLAGS := -fprofile-instr-generate -fcoverage-mapping
-else
-    COV_FLAGS := -fprofile-arcs -ftest-coverage
-endif
-$(filter-out %.init.o $(nocov-y),$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += $(COV_FLAGS)
+$(filter-out %.init.o $(nocov-y),$(obj-y) $(obj-bin-y) $(extra-y)): ccflags-y += $(COV_FLAGS)
 endif
 
 ifeq ($(CONFIG_UBSAN),y)
-CFLAGS_UBSAN += -fsanitize=undefined
 # Any -fno-sanitize= options need to come after any -fsanitize= options
 $(filter-out %.init.o $(noubsan-y),$(obj-y) $(obj-bin-y) $(extra-y)): \
-CFLAGS += $(filter-out -fno-%,$(CFLAGS_UBSAN)) $(filter -fno-%,$(CFLAGS_UBSAN))
+ccflags-y += $(filter-out -fno-%,$(CFLAGS_UBSAN)) $(filter -fno-%,$(CFLAGS_UBSAN))
 endif
 
 ifeq ($(CONFIG_LTO),y)
-CFLAGS += -flto
-LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
 # Would like to handle all object files as bitcode, but objects made from
 # pure asm are in a different format and have to be collected separately.
 # Mirror the directory tree, collecting them as built_in_bin.o.
@@ -136,18 +88,20 @@  obj-bin-y :=
 endif
 
 # Always build obj-bin files as binary even if they come from C source. 
-$(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
+# FIXME LTO broken, but we would need a different way to filter -flto out
+# $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
 
 # target with its suffix stripped
 target-stem = $(basename $@)
 
 c_flags = -MMD -MF $(@D)/.$(@F).d \
-          $(CFLAGS) \
+          $(XEN_CFLAGS) \
+          $(arch_ccflags) \
           '-D__OBJECT_FILE__="$@"' \
           $(ccflags-y) $(CFLAGS_$(target-stem).o)
 
 a_flags = -MMD -MF $(@D)/.$(@F).d \
-          $(AFLAGS)
+          $(XEN_AFLAGS)
 
 built_in.o: $(obj-y) $(extra-y)
 ifeq ($(obj-y),)
@@ -156,7 +110,7 @@  else
 ifeq ($(CONFIG_LTO),y)
 	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
 else
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
 endif
 endif
 
@@ -164,7 +118,7 @@  built_in_bin.o: $(obj-bin-y) $(extra-y)
 ifeq ($(obj-bin-y),)
 	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
 else
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
 endif
 
 # Force execution of pattern rules (for which PHONY cannot be directly used).
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 7f1427630b96..1599e2ba4058 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -96,24 +96,24 @@  prelink_lto.o: $(ALL_OBJS)
 
 # Link it with all the binary objects
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
 prelink.o: $(ALL_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
index 022a3a6f82ba..e69de29bb2d1 100644
--- a/xen/arch/arm/Rules.mk
+++ b/xen/arch/arm/Rules.mk
@@ -1,93 +0,0 @@ 
-########################################
-# arm-specific definitions
-
-#
-# If you change any of these configuration options then you must
-# 'make clean' before rebuilding.
-#
-
-CFLAGS += -I$(BASEDIR)/include
-
-$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
-$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
-
-# Prevent floating-point variables from creeping into Xen.
-CFLAGS-$(CONFIG_ARM_32) += -msoft-float
-CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
-
-CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
-CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
-
-EARLY_PRINTK := n
-
-ifeq ($(CONFIG_DEBUG),y)
-
-# See docs/misc/arm/early-printk.txt for syntax
-
-EARLY_PRINTK_brcm           := 8250,0xF040AB00,2
-EARLY_PRINTK_dra7           := 8250,0x4806A000,2
-EARLY_PRINTK_fastmodel      := pl011,0x1c090000,115200
-EARLY_PRINTK_exynos5250     := exynos4210,0x12c20000
-EARLY_PRINTK_hikey960       := pl011,0xfff32000
-EARLY_PRINTK_juno           := pl011,0x7ff80000
-EARLY_PRINTK_lager          := scif,0xe6e60000
-EARLY_PRINTK_midway         := pl011,0xfff36000
-EARLY_PRINTK_mvebu          := mvebu,0xd0012000
-EARLY_PRINTK_omap5432       := 8250,0x48020000,2
-EARLY_PRINTK_rcar3          := scif,0xe6e88000
-EARLY_PRINTK_seattle        := pl011,0xe1010000
-EARLY_PRINTK_sun6i          := 8250,0x01c28000,2
-EARLY_PRINTK_sun7i          := 8250,0x01c28000,2
-EARLY_PRINTK_thunderx       := pl011,0x87e024000000
-EARLY_PRINTK_vexpress       := pl011,0x1c090000
-EARLY_PRINTK_xgene-mcdivitt := 8250,0x1c021000,2
-EARLY_PRINTK_xgene-storm    := 8250,0x1c020000,2
-EARLY_PRINTK_zynqmp         := cadence,0xff000000
-
-ifneq ($(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)),)
-EARLY_PRINTK_CFG := $(subst $(comma), ,$(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)))
-else
-EARLY_PRINTK_CFG := $(subst $(comma), ,$(CONFIG_EARLY_PRINTK))
-endif
-
-# Extract configuration from string
-EARLY_PRINTK_INC := $(word 1,$(EARLY_PRINTK_CFG))
-EARLY_UART_BASE_ADDRESS := $(word 2,$(EARLY_PRINTK_CFG))
-
-# UART specific options
-ifeq ($(EARLY_PRINTK_INC),8250)
-EARLY_UART_REG_SHIFT := $(word 3,$(EARLY_PRINTK_CFG))
-endif
-ifeq ($(EARLY_PRINTK_INC),pl011)
-ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-EARLY_PRINTK_INIT_UART := y
-EARLY_PRINTK_BAUD := $(word 3,$(EARLY_PRINTK_CFG))
-endif
-endif
-ifeq ($(EARLY_PRINTK_INC),scif)
-ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-CFLAGS-y += -DEARLY_PRINTK_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
-else
-CFLAGS-y += -DEARLY_PRINTK_VERSION_NONE
-endif
-endif
-
-ifneq ($(EARLY_PRINTK_INC),)
-EARLY_PRINTK := y
-endif
-
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK
-CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DEARLY_PRINTK_INIT_UART
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_BAUD=$(EARLY_PRINTK_BAUD)
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
-
-else # !CONFIG_DEBUG
-
-ifneq ($(CONFIG_EARLY_PRINTK),)
-# Early printk is dependant on a debug build.
-$(error CONFIG_EARLY_PRINTK enabled for non-debug build)
-endif
-
-endif
diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
new file mode 100644
index 000000000000..296d6c6cf526
--- /dev/null
+++ b/xen/arch/arm/arch.mk
@@ -0,0 +1,88 @@ 
+########################################
+# arm-specific definitions
+
+CFLAGS += -I$(BASEDIR)/include
+
+$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
+$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
+
+# Prevent floating-point variables from creeping into Xen.
+CFLAGS-$(CONFIG_ARM_32) += -msoft-float
+CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
+
+CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
+CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
+
+EARLY_PRINTK := n
+
+ifeq ($(CONFIG_DEBUG),y)
+
+# See docs/misc/arm/early-printk.txt for syntax
+
+EARLY_PRINTK_brcm           := 8250,0xF040AB00,2
+EARLY_PRINTK_dra7           := 8250,0x4806A000,2
+EARLY_PRINTK_fastmodel      := pl011,0x1c090000,115200
+EARLY_PRINTK_exynos5250     := exynos4210,0x12c20000
+EARLY_PRINTK_hikey960       := pl011,0xfff32000
+EARLY_PRINTK_juno           := pl011,0x7ff80000
+EARLY_PRINTK_lager          := scif,0xe6e60000
+EARLY_PRINTK_midway         := pl011,0xfff36000
+EARLY_PRINTK_mvebu          := mvebu,0xd0012000
+EARLY_PRINTK_omap5432       := 8250,0x48020000,2
+EARLY_PRINTK_rcar3          := scif,0xe6e88000
+EARLY_PRINTK_seattle        := pl011,0xe1010000
+EARLY_PRINTK_sun6i          := 8250,0x01c28000,2
+EARLY_PRINTK_sun7i          := 8250,0x01c28000,2
+EARLY_PRINTK_thunderx       := pl011,0x87e024000000
+EARLY_PRINTK_vexpress       := pl011,0x1c090000
+EARLY_PRINTK_xgene-mcdivitt := 8250,0x1c021000,2
+EARLY_PRINTK_xgene-storm    := 8250,0x1c020000,2
+EARLY_PRINTK_zynqmp         := cadence,0xff000000
+
+ifneq ($(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)),)
+EARLY_PRINTK_CFG := $(subst $(comma), ,$(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)))
+else
+EARLY_PRINTK_CFG := $(subst $(comma), ,$(CONFIG_EARLY_PRINTK))
+endif
+
+# Extract configuration from string
+EARLY_PRINTK_INC := $(word 1,$(EARLY_PRINTK_CFG))
+EARLY_UART_BASE_ADDRESS := $(word 2,$(EARLY_PRINTK_CFG))
+
+# UART specific options
+ifeq ($(EARLY_PRINTK_INC),8250)
+EARLY_UART_REG_SHIFT := $(word 3,$(EARLY_PRINTK_CFG))
+endif
+ifeq ($(EARLY_PRINTK_INC),pl011)
+ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
+EARLY_PRINTK_INIT_UART := y
+EARLY_PRINTK_BAUD := $(word 3,$(EARLY_PRINTK_CFG))
+endif
+endif
+ifeq ($(EARLY_PRINTK_INC),scif)
+ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
+CFLAGS-y += -DEARLY_PRINTK_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
+else
+CFLAGS-y += -DEARLY_PRINTK_VERSION_NONE
+endif
+endif
+
+ifneq ($(EARLY_PRINTK_INC),)
+EARLY_PRINTK := y
+endif
+
+CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK
+CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DEARLY_PRINTK_INIT_UART
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_BAUD=$(EARLY_PRINTK_BAUD)
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
+
+else # !CONFIG_DEBUG
+
+ifneq ($(CONFIG_EARLY_PRINTK),)
+# Early printk is dependant on a debug build.
+$(error CONFIG_EARLY_PRINTK enabled for non-debug build)
+endif
+
+endif
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index acf4c145c896..6724572eb84b 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -124,32 +124,32 @@  prelink-efi_lto.o: $(ALL_OBJS) efi/runtime.o efi/compat.o
 
 # Link it with all the binary objects
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 
 prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink-efi_lto.o efi/boot.init.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
 prelink.o: $(ALL_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 
 prelink-efi.o: $(ALL_OBJS) efi/boot.init.o efi/runtime.o efi/compat.o
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
@@ -162,7 +162,7 @@  note.o: $(TARGET)-syms
 		--rename-section=.data=.note.gnu.build-id -S $@.bin $@
 	rm -f $@.bin
 
-EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(LDFLAGS)) --subsystem=10
+EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(XEN_LDFLAGS)) --subsystem=10
 EFI_LDFLAGS += --image-base=$(1) --stack=0,0 --heap=0,0 --strip-debug
 EFI_LDFLAGS += --section-alignment=0x200000 --file-alignment=0x20
 EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index e69b8e697cc0..7867f9ccef83 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -1,89 +1,10 @@ 
 ########################################
 # x86-specific definitions
 
-XEN_IMG_OFFSET := 0x200000
-
-CFLAGS += -I$(BASEDIR)/include
-CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
-CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
-CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
-CFLAGS += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
-
-# Prevent floating-point variables from creeping into Xen.
-CFLAGS += -msoft-float
-
-ifeq ($(CONFIG_CC_IS_CLANG),y)
-# Note: Any test which adds -no-integrated-as will cause subsequent tests to
-# succeed, and not trigger further additions.
-#
-# The tests to select whether the integrated assembler is usable need to happen
-# before testing any assembler features, or else the result of the tests would
-# be stale if the integrated assembler is not used.
-
-# Older clang's built-in assembler doesn't understand .skip with labels:
-# https://bugs.llvm.org/show_bug.cgi?id=27369
-$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
-                     -no-integrated-as)
-
-# Check whether clang asm()-s support .include.
-$(call as-option-add,CFLAGS,CC,".include \"asm/indirect_thunk_asm.h\"",,\
-                     -no-integrated-as)
-
-# Check whether clang keeps .macro-s between asm()-s:
-# https://bugs.llvm.org/show_bug.cgi?id=36110
-$(call as-option-add,CFLAGS,CC,\
-                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
-                     -no-integrated-as)
-endif
-
-$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
-$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
-$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
-$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
-$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
-$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
-$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
-$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
-$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
-$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
-$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1", \
-                     -U__OBJECT_LABEL__ -DHAVE_AS_QUOTED_SYM \
-                     '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$$@')
-$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
-
-# GAS's idea of true is -1.  Clang's idea is 1
-$(call as-option-add,CFLAGS,CC,\
-    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
-
-# Check to see whether the assmbler supports the .nop directive.
-$(call as-option-add,CFLAGS,CC,\
-    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
-
-CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
-
-# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
-# SSE setup for variadic function calls.
-CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
-
-# Compile with thunk-extern, indirect-branch-register if avaiable.
-ifeq ($(CONFIG_INDIRECT_THUNK),y)
-CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
-CFLAGS += -fno-jump-tables
+ifdef HAVE_AS_QUOTED_SYM
+arch_ccflags += -DHAVE_AS_QUOTED_SYM \
+		'-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$@'
+else
+arch_ccflags += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
 endif
-
-# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
-# this to be overridden elsewhere.
-$(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
-CFLAGS += $(CFLAGS-stack-boundary)
-
-ifeq ($(CONFIG_UBSAN),y)
-# Don't enable alignment sanitisation.  x86 has efficient unaligned accesses,
-# and various things (ACPI tables, hypercall pages, stubs, etc) are wont-fix.
-# It also causes an as-yet-unidentified crash on native boot before the
-# console starts.
-$(call cc-option-add,CFLAGS_UBSAN,CC,-fno-sanitize=alignment)
-endif
-
-# Set up the assembler include path properly for older toolchains.
-CFLAGS += -Wa,-I$(BASEDIR)/include
-
+arch_ccflags += $(CFLAGS-stack-boundary)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
new file mode 100644
index 000000000000..1073f6cc99e9
--- /dev/null
+++ b/xen/arch/x86/arch.mk
@@ -0,0 +1,87 @@ 
+########################################
+# x86-specific definitions
+
+export XEN_IMG_OFFSET := 0x200000
+
+CFLAGS += -I$(BASEDIR)/include
+CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
+CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
+CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
+
+# Prevent floating-point variables from creeping into Xen.
+CFLAGS += -msoft-float
+
+ifeq ($(CONFIG_CC_IS_CLANG),y)
+# Note: Any test which adds -no-integrated-as will cause subsequent tests to
+# succeed, and not trigger further additions.
+#
+# The tests to select whether the integrated assembler is usable need to happen
+# before testing any assembler features, or else the result of the tests would
+# be stale if the integrated assembler is not used.
+
+# Older clang's built-in assembler doesn't understand .skip with labels:
+# https://bugs.llvm.org/show_bug.cgi?id=27369
+$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
+                     -no-integrated-as)
+
+# Check whether clang asm()-s support .include.
+$(call as-option-add,CFLAGS,CC,".include \"asm/indirect_thunk_asm.h\"",,\
+                     -no-integrated-as)
+
+# Check whether clang keeps .macro-s between asm()-s:
+# https://bugs.llvm.org/show_bug.cgi?id=36110
+$(call as-option-add,CFLAGS,CC,\
+                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
+                     -no-integrated-as)
+endif
+
+$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
+$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
+$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
+$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
+$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
+$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
+$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
+$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
+$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
+$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
+ifeq ($(call as-insn,$(CC) $(CFLAGS),".equ \"x\"$(comma)1",y),y)
+  export HAVE_AS_QUOTED_SYM := y
+endif
+$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
+
+# GAS's idea of true is -1.  Clang's idea is 1
+$(call as-option-add,CFLAGS,CC,\
+    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
+
+# Check to see whether the assmbler supports the .nop directive.
+$(call as-option-add,CFLAGS,CC,\
+    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
+
+CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
+
+# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
+# SSE setup for variadic function calls.
+CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
+
+# Compile with thunk-extern, indirect-branch-register if avaiable.
+ifeq ($(CONFIG_INDIRECT_THUNK),y)
+CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
+CFLAGS += -fno-jump-tables
+endif
+
+# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
+# this to be overridden elsewhere.
+$(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
+export CFLAGS-stack-boundary
+
+ifeq ($(CONFIG_UBSAN),y)
+# Don't enable alignment sanitisation.  x86 has efficient unaligned accesses,
+# and various things (ACPI tables, hypercall pages, stubs, etc) are wont-fix.
+# It also causes an as-yet-unidentified crash on native boot before the
+# console starts.
+$(call cc-option-add,CFLAGS_UBSAN,CC,-fno-sanitize=alignment)
+endif
+
+# Set up the assembler include path properly for older toolchains.
+CFLAGS += -Wa,-I$(BASEDIR)/include
diff --git a/xen/common/libelf/Makefile b/xen/common/libelf/Makefile
index 9a433f01fbd4..8f8508dd842e 100644
--- a/xen/common/libelf/Makefile
+++ b/xen/common/libelf/Makefile
@@ -9,4 +9,4 @@  libelf.o: libelf-temp.o Makefile
 	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 libelf-temp.o: libelf-tools.o libelf-loader.o libelf-dominfo.o #libelf-relocate.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index 9ea5c696d52a..42986e118ebf 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -11,4 +11,4 @@  libfdt.o: libfdt-temp.o Makefile
 	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 libfdt-temp.o: $(LIBFDT_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^