mbox series

[RFC,XEN,00/23] Kconfig update + WIP of using Kbuild

Message ID 20191023164837.2700240-1-anthony.perard@citrix.com (mailing list archive)
Headers show
Series Kconfig update + WIP of using Kbuild | expand

Message

Anthony PERARD Oct. 23, 2019, 4:48 p.m. UTC 
Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v1

Hi,

I have work toward building Xen (the hypervisor) with Linux's build system,
Kbuild.

The main reason for that is to be able to have out-of-tree build. It's annoying
when a build fail because of the pvshim. Other benefit is a much faster
rebuild, and `make clean` doesn't take ages.

The series start with a few cleanup patch, and an update to Kconfig that could
be committed. The other patches that import Kbuild aren't ready. Quite a
few functionality of the current build system are missing, like EFI build,
Arm build, or build with clang.

New features of Kconfig:
    - Can run shell commands!
        This could be use to generate the CFLAGS and cleanup the Kconfig files
        to not have to depends on some env var (like XEN_HAS_BUILD_ID).
    - Update to the graphical menuconfig, xconfig. It's now built with Qt4/Qt5.
    - Probably others that I forgot.

Kbuild:
    I'm not sure how to move forward, easier change everything all at once with
    one big fat patch, or we could try to improve the current build system
    while taking inspiration from Kbuild.

    There are some nice make functions like if_changed which can also rebuild a
    target when the command line as changed (CFLAGS changed for e.g.). Also,
    with the `fixdep' program, files can depend on change to a single CONFIG_*
    instead of having to rebuild everything when .config change.
    With Kbuild, the list of CFLAGS is only done once, in the root Makefile,
    instead of every time a recursive make is done.

Anyway, feel free to try that out. Some example (when in xen.git/xen dir):
    make V=2    -> show you why a target is rebuilt
    make V=1    -> show you the command line executed
    make O=builddir     -> out-of-tree build

(I haven't CCed anyone as I don't want to CC everyone from THE REST at this
stage)

Cheers,

Anthony PERARD (23):
  xen: Fix strange byte in common/Kconfig
  xen: Makefile: Remove outdated comment
  xen: Update Kconfig to Linux v5.3
  build: Import Kbuild from Linux v5.3
  xen: Kbuild: Remove extra -include from C flags
  xen,Kbuild: filter-out -Wa,% from CFLAGS for %.i:%.c rules
  xen,Kbuild: Handle obj-bin-y and %.init.o objects
  xen,Kbuild: Change filechk_offsets
  xen,Kbuild: Filter-out -Wa, in %.s:%.c rules
  xen,Kbuild: Handle __OBJECT_*__
  First conversion to kbuild makefiles
  Build guest_walk* in arch/x86/mm
  convert common/libelf/Makefile to kbuild makefile
  convert common/libfdt to kbuild makefile
  convert tools/Makefile to kbuild
  convert include/Makefile to Kbuild
  convert arch/x86/boot/Makefile to kbuild
  arch/x86: Start moving build targets out of Makefile into Kbuild.
  update *FLAGS for arch/x86/Makefile
  update arch/Makefile
  Import root Makefile from Linux v5.3
  Changes to root Makefile
  xen/tools/kconfig: Delete duplicate makefiles

 docs/misc/kconfig-language.rst                |  701 +++++
 docs/misc/kconfig-language.txt                |  395 ---
 docs/misc/kconfig-macro-language.rst          |  247 ++
 docs/misc/{kconfig.txt => kconfig.rst}        |  185 +-
 xen/.gitignore                                |    6 +
 xen/Kbuild                                    |   10 +
 xen/Kconfig                                   |   18 +-
 xen/Makefile                                  | 1270 ++++++--
 xen/arch/arm/Kconfig                          |    2 +-
 xen/arch/arm/Makefile                         |   14 +-
 xen/arch/arm/arm32/Makefile                   |    2 +-
 xen/arch/arm/arm64/Makefile                   |    2 +-
 xen/arch/arm/efi/Makefile                     |    2 +-
 xen/arch/x86/Kbuild                           |  153 +
 xen/arch/x86/Makefile                         |  262 +-
 xen/arch/x86/acpi/Makefile                    |    2 +-
 xen/arch/x86/boot/Makefile                    |   72 +-
 xen/arch/x86/boot/build32.mk                  |   40 -
 xen/arch/x86/cpu/Makefile                     |    4 +-
 xen/arch/x86/efi/Makefile                     |    2 +-
 xen/arch/x86/hvm/Makefile                     |    6 +-
 xen/arch/x86/mm/Makefile                      |   18 +-
 xen/arch/x86/mm/hap/Makefile                  |   14 +-
 xen/arch/x86/mm/shadow/Makefile               |   14 +-
 xen/arch/x86/x86_64/Makefile                  |    2 +-
 xen/common/Kconfig                            |   14 +-
 xen/common/Makefile                           |    8 +-
 xen/common/libelf/Makefile                    |   15 +-
 xen/common/libfdt/Makefile                    |   14 +-
 xen/drivers/Makefile                          |   14 +-
 xen/drivers/acpi/Makefile                     |    6 +-
 xen/drivers/passthrough/Makefile              |    8 +-
 xen/drivers/passthrough/vtd/Makefile          |    2 +-
 xen/include/Makefile                          |  125 +-
 xen/lib/Makefile                              |    2 +-
 xen/scripts/Kbuild.include                    |  343 +++
 xen/scripts/Makefile.asm-generic              |   58 +
 xen/scripts/Makefile.build                    |  552 ++++
 xen/scripts/Makefile.clean                    |   90 +
 xen/scripts/Makefile.gcc-plugins              |   59 +
 xen/scripts/Makefile.headersinst              |  102 +
 xen/{tools/kconfig => scripts}/Makefile.host  |   94 +-
 xen/scripts/Makefile.kasan                    |   51 +
 xen/scripts/Makefile.kcov                     |   10 +
 xen/scripts/Makefile.lib                      |  430 +++
 xen/scripts/Makefile.ubsan                    |   19 +
 xen/scripts/basic/.gitignore                  |    1 +
 xen/scripts/basic/Makefile                    |   16 +
 xen/scripts/basic/fixdep.c                    |  410 +++
 xen/scripts/mkmakefile                        |   17 +
 xen/tools/Makefile                            |   15 +-
 xen/tools/kconfig/.gitignore                  |    6 +-
 xen/tools/kconfig/Makefile                    |  268 +-
 xen/tools/kconfig/Makefile.kconfig            |   62 -
 xen/tools/kconfig/conf.c                      |  191 +-
 xen/tools/kconfig/confdata.c                  |  491 ++--
 xen/tools/kconfig/expr.c                      |  213 +-
 xen/tools/kconfig/expr.h                      |  108 +-
 xen/tools/kconfig/gconf-cfg.sh                |   30 +
 xen/tools/kconfig/gconf.c                     |   39 +-
 xen/tools/kconfig/images.c                    |   34 +-
 xen/tools/kconfig/images.h                    |   33 +
 xen/tools/kconfig/lexer.l                     |  471 +++
 xen/tools/kconfig/list.h                      |    1 +
 xen/tools/kconfig/lkc.h                       |   38 +-
 xen/tools/kconfig/lkc_proto.h                 |   21 +-
 xen/tools/kconfig/lxdialog/.gitignore         |    4 -
 xen/tools/kconfig/lxdialog/BIG.FAT.WARNING    |    2 +-
 xen/tools/kconfig/lxdialog/check-lxdialog.sh  |   91 -
 xen/tools/kconfig/lxdialog/checklist.c        |   15 +-
 xen/tools/kconfig/lxdialog/dialog.h           |   17 +-
 xen/tools/kconfig/lxdialog/inputbox.c         |   18 +-
 xen/tools/kconfig/lxdialog/menubox.c          |   15 +-
 xen/tools/kconfig/lxdialog/textbox.c          |   15 +-
 xen/tools/kconfig/lxdialog/util.c             |   15 +-
 xen/tools/kconfig/lxdialog/yesno.c            |   15 +-
 xen/tools/kconfig/mconf-cfg.sh                |   47 +
 xen/tools/kconfig/mconf.c                     |   27 +-
 xen/tools/kconfig/menu.c                      |  288 +-
 xen/tools/kconfig/merge_config.sh             |   77 +-
 xen/tools/kconfig/nconf-cfg.sh                |   47 +
 xen/tools/kconfig/nconf.c                     |   42 +-
 xen/tools/kconfig/nconf.gui.c                 |   30 +-
 xen/tools/kconfig/nconf.h                     |    9 +-
 xen/tools/kconfig/{zconf.y => parser.y}       |  409 ++-
 xen/tools/kconfig/preprocess.c                |  574 ++++
 xen/tools/kconfig/qconf-cfg.sh                |   32 +
 xen/tools/kconfig/qconf.cc                    |  750 +++--
 xen/tools/kconfig/qconf.h                     |  153 +-
 xen/tools/kconfig/streamline_config.pl        |   53 +-
 xen/tools/kconfig/symbol.c                    |  295 +-
 xen/tools/kconfig/tests/auto_submenu/Kconfig  |   52 +
 .../kconfig/tests/auto_submenu/__init__.py    |   13 +
 .../tests/auto_submenu/expected_stdout        |   10 +
 xen/tools/kconfig/tests/choice/Kconfig        |   56 +
 xen/tools/kconfig/tests/choice/__init__.py    |   41 +
 .../tests/choice/alldef_expected_config       |    5 +
 .../tests/choice/allmod_expected_config       |    9 +
 .../tests/choice/allno_expected_config        |    5 +
 .../tests/choice/allyes_expected_config       |    9 +
 .../tests/choice/oldask0_expected_stdout      |   10 +
 xen/tools/kconfig/tests/choice/oldask1_config |    2 +
 .../tests/choice/oldask1_expected_stdout      |   15 +
 .../tests/choice_value_with_m_dep/Kconfig     |   21 +
 .../tests/choice_value_with_m_dep/__init__.py |   16 +
 .../tests/choice_value_with_m_dep/config      |    2 +
 .../choice_value_with_m_dep/expected_config   |    3 +
 .../choice_value_with_m_dep/expected_stdout   |    4 +
 xen/tools/kconfig/tests/conftest.py           |  291 ++
 .../kconfig/tests/err_recursive_dep/Kconfig   |   63 +
 .../tests/err_recursive_dep/__init__.py       |   10 +
 .../tests/err_recursive_dep/expected_stderr   |   38 +
 .../kconfig/tests/err_recursive_inc/Kconfig   |    2 +
 .../tests/err_recursive_inc/Kconfig.inc1      |    5 +
 .../tests/err_recursive_inc/Kconfig.inc2      |    4 +
 .../tests/err_recursive_inc/Kconfig.inc3      |    2 +
 .../tests/err_recursive_inc/__init__.py       |   11 +
 .../tests/err_recursive_inc/expected_stderr   |    6 +
 xen/tools/kconfig/tests/inter_choice/Kconfig  |   25 +
 .../kconfig/tests/inter_choice/__init__.py    |   15 +
 .../kconfig/tests/inter_choice/defconfig      |    1 +
 .../tests/inter_choice/expected_config        |    4 +
 .../kconfig/tests/new_choice_with_dep/Kconfig |   39 +
 .../tests/new_choice_with_dep/__init__.py     |   15 +
 .../kconfig/tests/new_choice_with_dep/config  |    3 +
 .../tests/new_choice_with_dep/expected_stdout |   10 +
 .../tests/no_write_if_dep_unmet/Kconfig       |   16 +
 .../tests/no_write_if_dep_unmet/__init__.py   |   20 +
 .../tests/no_write_if_dep_unmet/config        |    1 +
 .../no_write_if_dep_unmet/expected_config     |    5 +
 .../tests/preprocess/builtin_func/Kconfig     |   27 +
 .../tests/preprocess/builtin_func/__init__.py |    9 +
 .../preprocess/builtin_func/expected_stderr   |    5 +
 .../preprocess/builtin_func/expected_stdout   |    1 +
 .../preprocess/circular_expansion/Kconfig     |    5 +
 .../preprocess/circular_expansion/__init__.py |   11 +
 .../circular_expansion/expected_stderr        |    1 +
 .../kconfig/tests/preprocess/escape/Kconfig   |   44 +
 .../tests/preprocess/escape/__init__.py       |    8 +
 .../tests/preprocess/escape/expected_stderr   |   10 +
 .../kconfig/tests/preprocess/variable/Kconfig |   53 +
 .../tests/preprocess/variable/__init__.py     |    8 +
 .../tests/preprocess/variable/expected_stderr |    9 +
 xen/tools/kconfig/tests/pytest.ini            |    7 +
 .../kconfig/tests/rand_nested_choice/Kconfig  |   35 +
 .../tests/rand_nested_choice/__init__.py      |   17 +
 .../tests/rand_nested_choice/expected_stdout0 |    2 +
 .../tests/rand_nested_choice/expected_stdout1 |    4 +
 .../tests/rand_nested_choice/expected_stdout2 |    5 +
 xen/tools/kconfig/util.c                      |   86 +-
 xen/tools/kconfig/zconf.gperf                 |   49 -
 xen/tools/kconfig/zconf.hash.c_shipped        |  293 --
 xen/tools/kconfig/zconf.l                     |  374 ---
 xen/tools/kconfig/zconf.lex.c_shipped         | 2473 ----------------
 xen/tools/kconfig/zconf.tab.c_shipped         | 2580 -----------------
 xen/xsm/Makefile                              |    2 +-
 xen/xsm/flask/Makefile                        |    4 +-
 xen/xsm/flask/ss/Makefile                     |    2 +-
 158 files changed, 9225 insertions(+), 8722 deletions(-)
 create mode 100644 docs/misc/kconfig-language.rst
 delete mode 100644 docs/misc/kconfig-language.txt
 create mode 100644 docs/misc/kconfig-macro-language.rst
 rename docs/misc/{kconfig.txt => kconfig.rst} (70%)
 create mode 100644 xen/.gitignore
 create mode 100644 xen/Kbuild
 create mode 100644 xen/arch/x86/Kbuild
 delete mode 100644 xen/arch/x86/boot/build32.mk
 create mode 100644 xen/scripts/Kbuild.include
 create mode 100644 xen/scripts/Makefile.asm-generic
 create mode 100644 xen/scripts/Makefile.build
 create mode 100644 xen/scripts/Makefile.clean
 create mode 100644 xen/scripts/Makefile.gcc-plugins
 create mode 100644 xen/scripts/Makefile.headersinst
 rename xen/{tools/kconfig => scripts}/Makefile.host (51%)
 create mode 100644 xen/scripts/Makefile.kasan
 create mode 100644 xen/scripts/Makefile.kcov
 create mode 100644 xen/scripts/Makefile.lib
 create mode 100644 xen/scripts/Makefile.ubsan
 create mode 100644 xen/scripts/basic/.gitignore
 create mode 100644 xen/scripts/basic/Makefile
 create mode 100644 xen/scripts/basic/fixdep.c
 create mode 100755 xen/scripts/mkmakefile
 delete mode 100644 xen/tools/kconfig/Makefile.kconfig
 create mode 100755 xen/tools/kconfig/gconf-cfg.sh
 create mode 100644 xen/tools/kconfig/images.h
 create mode 100644 xen/tools/kconfig/lexer.l
 delete mode 100644 xen/tools/kconfig/lxdialog/.gitignore
 delete mode 100755 xen/tools/kconfig/lxdialog/check-lxdialog.sh
 create mode 100755 xen/tools/kconfig/mconf-cfg.sh
 create mode 100755 xen/tools/kconfig/nconf-cfg.sh
 rename xen/tools/kconfig/{zconf.y => parser.y} (66%)
 create mode 100644 xen/tools/kconfig/preprocess.c
 create mode 100755 xen/tools/kconfig/qconf-cfg.sh
 create mode 100644 xen/tools/kconfig/tests/auto_submenu/Kconfig
 create mode 100644 xen/tools/kconfig/tests/auto_submenu/__init__.py
 create mode 100644 xen/tools/kconfig/tests/auto_submenu/expected_stdout
 create mode 100644 xen/tools/kconfig/tests/choice/Kconfig
 create mode 100644 xen/tools/kconfig/tests/choice/__init__.py
 create mode 100644 xen/tools/kconfig/tests/choice/alldef_expected_config
 create mode 100644 xen/tools/kconfig/tests/choice/allmod_expected_config
 create mode 100644 xen/tools/kconfig/tests/choice/allno_expected_config
 create mode 100644 xen/tools/kconfig/tests/choice/allyes_expected_config
 create mode 100644 xen/tools/kconfig/tests/choice/oldask0_expected_stdout
 create mode 100644 xen/tools/kconfig/tests/choice/oldask1_config
 create mode 100644 xen/tools/kconfig/tests/choice/oldask1_expected_stdout
 create mode 100644 xen/tools/kconfig/tests/choice_value_with_m_dep/Kconfig
 create mode 100644 xen/tools/kconfig/tests/choice_value_with_m_dep/__init__.py
 create mode 100644 xen/tools/kconfig/tests/choice_value_with_m_dep/config
 create mode 100644 xen/tools/kconfig/tests/choice_value_with_m_dep/expected_config
 create mode 100644 xen/tools/kconfig/tests/choice_value_with_m_dep/expected_stdout
 create mode 100644 xen/tools/kconfig/tests/conftest.py
 create mode 100644 xen/tools/kconfig/tests/err_recursive_dep/Kconfig
 create mode 100644 xen/tools/kconfig/tests/err_recursive_dep/__init__.py
 create mode 100644 xen/tools/kconfig/tests/err_recursive_dep/expected_stderr
 create mode 100644 xen/tools/kconfig/tests/err_recursive_inc/Kconfig
 create mode 100644 xen/tools/kconfig/tests/err_recursive_inc/Kconfig.inc1
 create mode 100644 xen/tools/kconfig/tests/err_recursive_inc/Kconfig.inc2
 create mode 100644 xen/tools/kconfig/tests/err_recursive_inc/Kconfig.inc3
 create mode 100644 xen/tools/kconfig/tests/err_recursive_inc/__init__.py
 create mode 100644 xen/tools/kconfig/tests/err_recursive_inc/expected_stderr
 create mode 100644 xen/tools/kconfig/tests/inter_choice/Kconfig
 create mode 100644 xen/tools/kconfig/tests/inter_choice/__init__.py
 create mode 100644 xen/tools/kconfig/tests/inter_choice/defconfig
 create mode 100644 xen/tools/kconfig/tests/inter_choice/expected_config
 create mode 100644 xen/tools/kconfig/tests/new_choice_with_dep/Kconfig
 create mode 100644 xen/tools/kconfig/tests/new_choice_with_dep/__init__.py
 create mode 100644 xen/tools/kconfig/tests/new_choice_with_dep/config
 create mode 100644 xen/tools/kconfig/tests/new_choice_with_dep/expected_stdout
 create mode 100644 xen/tools/kconfig/tests/no_write_if_dep_unmet/Kconfig
 create mode 100644 xen/tools/kconfig/tests/no_write_if_dep_unmet/__init__.py
 create mode 100644 xen/tools/kconfig/tests/no_write_if_dep_unmet/config
 create mode 100644 xen/tools/kconfig/tests/no_write_if_dep_unmet/expected_config
 create mode 100644 xen/tools/kconfig/tests/preprocess/builtin_func/Kconfig
 create mode 100644 xen/tools/kconfig/tests/preprocess/builtin_func/__init__.py
 create mode 100644 xen/tools/kconfig/tests/preprocess/builtin_func/expected_stderr
 create mode 100644 xen/tools/kconfig/tests/preprocess/builtin_func/expected_stdout
 create mode 100644 xen/tools/kconfig/tests/preprocess/circular_expansion/Kconfig
 create mode 100644 xen/tools/kconfig/tests/preprocess/circular_expansion/__init__.py
 create mode 100644 xen/tools/kconfig/tests/preprocess/circular_expansion/expected_stderr
 create mode 100644 xen/tools/kconfig/tests/preprocess/escape/Kconfig
 create mode 100644 xen/tools/kconfig/tests/preprocess/escape/__init__.py
 create mode 100644 xen/tools/kconfig/tests/preprocess/escape/expected_stderr
 create mode 100644 xen/tools/kconfig/tests/preprocess/variable/Kconfig
 create mode 100644 xen/tools/kconfig/tests/preprocess/variable/__init__.py
 create mode 100644 xen/tools/kconfig/tests/preprocess/variable/expected_stderr
 create mode 100644 xen/tools/kconfig/tests/pytest.ini
 create mode 100644 xen/tools/kconfig/tests/rand_nested_choice/Kconfig
 create mode 100644 xen/tools/kconfig/tests/rand_nested_choice/__init__.py
 create mode 100644 xen/tools/kconfig/tests/rand_nested_choice/expected_stdout0
 create mode 100644 xen/tools/kconfig/tests/rand_nested_choice/expected_stdout1
 create mode 100644 xen/tools/kconfig/tests/rand_nested_choice/expected_stdout2
 delete mode 100644 xen/tools/kconfig/zconf.gperf
 delete mode 100644 xen/tools/kconfig/zconf.hash.c_shipped
 delete mode 100644 xen/tools/kconfig/zconf.l
 delete mode 100644 xen/tools/kconfig/zconf.lex.c_shipped
 delete mode 100644 xen/tools/kconfig/zconf.tab.c_shipped

Comments

Andrew Cooper Oct. 24, 2019, 10:41 a.m. UTC | #1 
On 23/10/2019 17:48, Anthony PERARD wrote:
> Current description of the file by `file`:
>     common/Kconfig: Non-ISO extended-ASCII text
>
> Change that byte to an ascii quote so the file can become properly
> encoded, and all ASCII.
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

I see "about Xen~Rs execution" in menuconfig, and this is new content in
4.13 so wants fixing.

> ---
>  xen/common/Kconfig | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 7b5dd9d49596..5c0f8d30c709 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -386,7 +386,7 @@ config TRACEBUFFER
>  	default y
>  	---help---
>  	  Enable tracing infrastructure and pre-defined tracepoints within Xen.
> -	  This will allow live information about Xen�s execution and performance
> +	  This will allow live information about Xen's execution and performance
>  	  to be collected at run time for debugging or performance analysis.
>  	  Memory and execution overhead when not active is minimal.
>  
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
Jürgen Groß Oct. 24, 2019, 10:48 a.m. UTC | #2 
On 24.10.19 12:41, Andrew Cooper wrote:
> On 23/10/2019 17:48, Anthony PERARD wrote:
>> Current description of the file by `file`:
>>      common/Kconfig: Non-ISO extended-ASCII text
>>
>> Change that byte to an ascii quote so the file can become properly
>> encoded, and all ASCII.
>>
>> Signed-off-by: Anthony PERARD<anthony.perard@citrix.com>
> 
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> I see "about Xen~Rs execution" in menuconfig, and this is new content in 
> 4.13 so wants fixing.

Agreed.

Release-acked-by: Juergen Gross <jgross@suse.com>


Juergen