| Message ID | CAK7LNAQzEHv7Y69xFTPDEhyMP6JX4ghb96UMqMdxYyVyah6-VA@mail.gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL,1/4] Kbuild updates for v4.21 | expand |
The pull request you sent on Sat, 29 Dec 2018 00:58:38 +0900:
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git tags/kbuild-v4.21
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/668c35f69cc750aaf07bd5fe7710a47e2aed6e43
Thank you!
Masahiro,
this isn't really related to the your request, but to just another
pull request that caused much more of a rebuild than I expected.
Doing a "make --debug", I see that the cause was that it touched
include/uapi/linux/elf-em.h, and make says things like
Prerequisite 'include/uapi/linux/elf-em.h' is newer than target
'drivers/net/ethernet/sfc/falcon/efx.o'.
Must remake target 'drivers/net/ethernet/sfc/falcon/efx.o'.
which is obviously true, but was unexpected.
The chain seems to be
<linux/module.h> ->
<linux/elf.h> ->
<uapi/linux/elf.h> ->
<linux/elf-em.h>
or similar, and the reason <linux/module.h> does that is it wants some
of the elf types (ie code like this:
struct mod_kallsyms {
Elf_Sym *symtab;
unsigned int num_symtab;
char *strtab;
};
and for declarations of module_bug_finalize() etc.
Fine, fine. I can see why everybody includes <linux/module.h>, and I
can see why module.h in turn wants the elf information.
And I think I can see a way to avoid this chain, particularly with
lots of those things really being internal to kernel/module.h and
shouldn't be visible to random <linux/module.h> users.
But the reason for this email to you is simply to ask whether you
use/have any tools for seeing these kinds of deep include chains.
Getting rid of some of the deep chains of header includes would likely
speed up kernel builds even when a rebuild is forced, and when it's
something like "we really shouldn't have rebuilt that file at all" the
speedup is obviously even bigger than just a "the compiler saw much
too much unnecessary header contents".
Linus
Hi Linus. > But the reason for this email to you is simply to ask whether you > use/have any tools for seeing these kinds of deep include chains. Not exactly what you ask for - but we have make V=2 Example: $ touch include/uapi/linux/elf-em.h $ make V=2 CALL scripts/checksyscalls.sh - due to target missing DESCEND objtool CC init/main.o - due to: include/uapi/linux/elf-em.h CHK include/generated/compile.h CC init/version.o - due to: include/uapi/linux/elf-em.h CC init/do_mounts.o - due to: include/uapi/linux/elf-em.h CC init/do_mounts_initrd.o - due to: include/uapi/linux/elf-em.h CC init/do_mounts_md.o - due to: include/uapi/linux/elf-em.h ... With V=2 kbuild will try to tell you why a target is rebuild. This can sometimes be useful. Here kbuild will tell you: - due to: include/uapi/linux/elf-em.h Which may be a good clue. But you need to figure out yourself why a certain file depends on include/uapi/linux/elf-em.h One way is to look at the .cmd file for a target: $ cat init/.do_mounts_md.o.cmd ... include/linux/module.h \ $(wildcard include/config/modules/tree/lookup.h) \ $(wildcard include/config/module/sig.h) \ $(wildcard include/config/module/unload.h) \ $(wildcard include/config/constructors.h) \ $(wildcard include/config/function/error/injection.h) \ include/linux/kmod.h \ include/linux/umh.h \ include/linux/elf.h \ arch/x86/include/asm/elf.h \ arch/x86/include/asm/user.h \ arch/x86/include/asm/user_64.h \ arch/x86/include/asm/fsgsbase.h \ arch/x86/include/asm/msr-index.h \ arch/x86/include/asm/vdso.h \ $(wildcard include/config/x86/x32.h) \ include/uapi/linux/elf.h \ include/uapi/linux/elf-em.h \ ... In the above all the $(wildcard ...) is the CONFIG_ symbols in the files that trigger a possible dependency on a file representing the CONFIG_ symbol. We can see that the target depends on elf-em.h and if we follow it back we end up with module.h included from do_mounts_md.c Sam
On Sat, Jan 5, 2019 at 12:39 PM Sam Ravnborg <sam@ravnborg.org> wrote: > > Not exactly what you ask for - but we have make V=2 Yeah, that's certainly more convenient than "make --debug". That said, I was more thinking of not any particular "oh, it's recompiling everything" situation (by then it's too late, obviously), but if somebody has been looking at tools for finding and maybe breaking some of our deeper include chains. I know it has come up before (the x86 people did the <linux/schedule.h> split some years ago) and then it was all manual. But I was kind of hoping that maybe some of the kbuild people had looked at this? Linus
Hi Linus, On Sun, Jan 6, 2019 at 5:44 AM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > On Sat, Jan 5, 2019 at 12:39 PM Sam Ravnborg <sam@ravnborg.org> wrote: > > > > Not exactly what you ask for - but we have make V=2 > > Yeah, that's certainly more convenient than "make --debug". > > That said, I was more thinking of not any particular "oh, it's > recompiling everything" situation (by then it's too late, obviously), > but if somebody has been looking at tools for finding and maybe > breaking some of our deeper include chains. > > I know it has come up before (the x86 people did the > <linux/schedule.h> split some years ago) and then it was all manual. > But I was kind of hoping that maybe some of the kbuild people had > looked at this? Sorry, I do not know more information than Sam provided. I got rid of some unneeded header inclusions in the past, but my work was also manual.
Hi Linus, Please pull Kbuild updates for v4.21 You will see a conflict in the top Makefile. The correct fix is available in linux-next. The following changes since commit ccda4af0f4b92f7b4c308d3acc262f4a7e3affad: Linux 4.20-rc2 (2018-11-11 17:12:31 -0600) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git tags/kbuild-v4.21 for you to fetch changes up to dec28d8ea2f731b7ec68a2c9421e99a165d47b57: kbuild: remove unused cmd_gentimeconst (2018-12-25 00:10:30 +0900) ---------------------------------------------------------------- Kbuild updates for v4.21 Kbuild core: - remove unneeded $(call cc-option,...) switches - consolidate Clang compiler flags into CLANG_FLAGS - announce the deprecation of SUBDIRS - fix single target build for external module - simplify the dependencies of 'prepare' stage targets - allow fixdep to directly write to .*.cmd files - simplify dependency generation for CONFIG_TRIM_UNUSED_KSYMS - change if_changed_rule to accept multi-line recipe - move .SECONDARY special target to scripts/Kbuild.include - remove redundant 'set -e' - improve parallel execution for CONFIG_HEADERS_CHECK - misc cleanups Treewide fixes and cleanups - set Clang flags correctly for PowerPC boot images - fix UML build error with CONFIG_GCC_PLUGINS - remove unneeded patterns from .gitignore files - refactor firmware/Makefile - remove unneeded rules for *offsets.s - avoid unneeded regeneration of intermediate .s files - clean up ./Kbuild Modpost: - remove unused -M, -K options - fix false positive warnings about section mismatch - use simple devtable lookup instead of linker magic - misc cleanups Coccinelle: - relax boolinit.cocci checks for overall consistency - fix warning messages of boolinit.cocci Other tools: - improve -dirty check of scripts/setlocalversion - add a tool to generate compile_commands.json from .*.cmd files ---------------------------------------------------------------- Brian Norris (1): scripts/setlocalversion: Improve -dirty check with git-status --no-optional-locks Eugen Cazacu (1): user/Makefile: Fix typo and capitalization in comment section Joel Stanley (2): Makefile: Export clang toolchain variables powerpc/boot: Set target when cross-compiling for clang Julia Lawall (2): scripts: coccinelle: only suggest true/false in files that already use them scripts: coccinelle: Correct warning message Masahiro Yamada (41): kbuild: add -no-integrated-as Clang option unconditionally kbuild: consolidate Clang compiler flags kbuild: announce removal of SUBDIRS if used kbuild: fix single target build for external module modpost: file2alias: go back to simple devtable lookup modpost: file2alias: check prototype of handler modpost: refactor seen flag clearing in add_depends() modpost: merge module iterations modpost: move unresolved symbol checks to check_exports() kbuild: fix UML build error with CONFIG_GCC_PLUGINS kbuild: move modpost out of 'scripts' target kbuild: make 'archprepare' depend on 'scripts' kbuild: descend into scripts/gcc-plugins/ via scripts/Makefile kbuild: let fixdep directly write to .*.cmd files kbuild: remove redundant 'set -e' from filechk_offsets kbuild: remove redundant 'set -e' from sub_cmd_record_mcount kbuild: refactor modversions build rules kbuild: simplify dependency generation for CONFIG_TRIM_UNUSED_KSYMS kbuild: change if_changed_rule for multi-line recipe kbuild: remove trailing semicolon from cmd_* passed to if_changed_rule kbuild: refactor if_changed kbuild: remove redundant 'set -e' from cmd_* defines kbuild: move .SECONDARY special target to Kbuild.include kbuild: remove a special handling for *.agh in Makefile.headersinst kbuild: exploit parallel building for CONFIG_HEADERS_CHECK kbuild: refactor Makefile.asm-generic kbuild: remove redundant quiet_modtag for $(obj-m) kbuild: refactor quiet_modtag kbuild: refactor part-of-module kbuild: handle part-of-module correctly for *.ll and *.symtypes kbuild: add -fno-PIE flag unconditionally kbuild: add -Werror=strict-prototypes flag unconditionally kbuild: add -Werror=implicit-int flag unconditionally um: remove unused filechk_gen_header in Makefile scripts: remove unnecessary ihex2fw and check-lc_ctypes from .gitignore firmware: remove unnecessary patterns from .gitignore firmware: refactor firmware/Makefile treewide: remove explicit rules for *offsets.s treewide: add intermediate .s files to targets kbuild: remove $(obj)/ prefixes in ./Kbuild kbuild: remove unused cmd_gentimeconst Paul Walmsley (2): modpost: drop unused command line switches modpost: skip ELF local symbols during section mismatch check Sami Tolvanen (1): modpost: validate symbol names also in find_elf_symbol Tom Roeder (1): scripts: add a tool to produce a compile_commands.json file Documentation/kbuild/kbuild.txt | 7 +-- Kbuild | 21 ++------ Makefile | 111 +++++++++++++++++++---------------------- arch/arm/crypto/Makefile | 2 +- arch/arm/mach-at91/Makefile | 5 +- arch/arm/mach-omap2/Makefile | 5 +- arch/arm64/crypto/Makefile | 2 +- arch/ia64/kernel/Makefile | 7 +-- arch/powerpc/boot/Makefile | 5 ++ arch/um/Makefile | 26 +--------- arch/x86/um/Makefile | 5 +- drivers/memory/Makefile.asm-offsets | 5 +- drivers/mtd/maps/scx200_docflash.c | 7 --- drivers/watchdog/scx200_wdt.c | 7 --- firmware/.gitignore | 5 -- firmware/Makefile | 84 ++++++++++++------------------- include/asm-generic/export.h | 13 +++-- include/linux/export.h | 18 +++---- samples/bpf/Makefile | 4 +- samples/connector/Makefile | 2 +- scripts/.gitignore | 2 - scripts/Kbuild.include | 52 ++++--------------- scripts/Makefile | 4 +- scripts/Makefile.asm-generic | 37 +++++++------- scripts/Makefile.build | 128 ++++++++++++++++++----------------------------- scripts/Makefile.gcc-plugins | 8 --- scripts/Makefile.headersinst | 1 - scripts/Makefile.lib | 2 +- scripts/basic/fixdep.c | 31 ++---------- scripts/coccinelle/misc/boolinit.cocci | 43 ++++++++++------ scripts/gen_compile_commands.py | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ scripts/gen_ksymdeps.sh | 25 ++++++++++ scripts/mod/Makefile | 2 - scripts/mod/file2alias.c | 149 +++++++++++++++++++------------------------------------ scripts/mod/modpost.c | 114 +++++++++++++++++++++++------------------- scripts/package/Makefile | 1 - scripts/setlocalversion | 12 ++++- usr/Makefile | 4 +- 38 files changed, 543 insertions(+), 564 deletions(-) create mode 100755 scripts/gen_compile_commands.py create mode 100755 scripts/gen_ksymdeps.sh -- Best Regards Masahiro Yamada