| Message ID | 20200207180755.100561-4-qperret@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | kbuild: allow symbol whitelisting with TRIM_UNUSED_KSYM | expand |
On Fri, Feb 7, 2020 at 7:08 PM Quentin Perret <qperret@google.com> wrote: > > When doing a cold build, autoksyms.h starts empty, and is updated late > in the build process to have visibility over the symbols used by in-tree > drivers. But since the symbol whitelist is known upfront, it can be used > to pre-populate autoksyms.h and maximize the amount of code that can be > compiled to its final state in a single pass, hence reducing build time. > > Do this by using gen_autoksyms.sh to initialize autoksyms.h instead of > creating an empty file. > > Signed-off-by: Quentin Perret <qperret@google.com> > --- > Makefile | 2 +- > scripts/gen_autoksyms.sh | 3 ++- > 2 files changed, 3 insertions(+), 2 deletions(-) > > diff --git a/Makefile b/Makefile > index 6a01b073915e..e5c389d189f7 100644 > --- a/Makefile > +++ b/Makefile > @@ -1065,7 +1065,7 @@ autoksyms_h := $(if $(CONFIG_TRIM_UNUSED_KSYMS), include/generated/autoksyms.h) > > $(autoksyms_h): > $(Q)mkdir -p $(dir $@) > - $(Q)touch $@ > + $(Q)$(CONFIG_SHELL) $(srctree)/scripts/gen_autoksyms.sh $@ Now that this line is doing a non-trivial task, it might be a good idea to show a short log, like this: GEN include/generated/autoksyms.h You can do it like this: quiet_cmd_autoksyms_h = GEN $@ cmd_autoksyms_h = mkdir -p $(dir $@); $(BASH) $(srctree)/scripts/gen_autoksyms.sh $@ $(autoksyms_h): $(call cmd,autoksyms_h) > > ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink) > > diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh > index ce0919c3791a..ae033ab03a4a 100755 > --- a/scripts/gen_autoksyms.sh > +++ b/scripts/gen_autoksyms.sh > @@ -32,7 +32,8 @@ cat > "$output_file" << EOT > > EOT > > -sed 's/ko$/mod/' modules.order | > +[[ -f modules.order ]] && modlist=modules.order || modlist=/dev/null > +sed 's/ko$/mod/' $modlist | > xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | > cat - "$ksym_wl" | > sort -u | > -- > 2.25.0.341.g760bfbb309-goog > -- Best Regards Masahiro Yamada
Hi Quentin,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on kbuild/for-next]
[also build test ERROR on linux/master linus/master v5.6-rc1 next-20200210]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Quentin-Perret/kbuild-allow-symbol-whitelisting-with-TRIM_UNUSED_KSYM/20200211-020659
base: https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: i386-randconfig-c002-20200211 (attached as .config)
compiler: gcc-7 (Debian 7.5.0-4) 7.5.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
>> ERROR: "trace_event_raw_init" [lib/objagg.ko] undefined!
>> ERROR: "trace_event_reg" [lib/objagg.ko] undefined!
>> ERROR: "ida_destroy" [lib/objagg.ko] undefined!
>> ERROR: "rhashtable_destroy" [lib/objagg.ko] undefined!
>> ERROR: "kmalloc_order_trace" [lib/objagg.ko] undefined!
>> ERROR: "sort" [lib/objagg.ko] undefined!
>> ERROR: "__raw_spin_lock_init" [lib/objagg.ko] undefined!
>> ERROR: "rhashtable_init" [lib/objagg.ko] undefined!
>> ERROR: "rht_bucket_nested" [lib/objagg.ko] undefined!
>> ERROR: "__list_del_entry_valid" [lib/objagg.ko] undefined!
>> ERROR: "__rht_bucket_nested" [lib/objagg.ko] undefined!
>> ERROR: "kmem_cache_alloc_trace" [lib/objagg.ko] undefined!
>> ERROR: "kmalloc_caches" [lib/objagg.ko] undefined!
>> ERROR: "queue_work_on" [lib/objagg.ko] undefined!
>> ERROR: "system_wq" [lib/objagg.ko] undefined!
>> ERROR: "kfree" [lib/objagg.ko] undefined!
>> ERROR: "__list_add_valid" [lib/objagg.ko] undefined!
>> ERROR: "lockdep_rht_bucket_is_held" [lib/objagg.ko] undefined!
>> ERROR: "rhashtable_insert_slow" [lib/objagg.ko] undefined!
>> ERROR: "__local_bh_enable_ip" [lib/objagg.ko] undefined!
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
On Saturday 08 Feb 2020 at 06:09:06 (+0100), Masahiro Yamada wrote: > Now that this line is doing a non-trivial task, > it might be a good idea to show a short log, like this: > > GEN include/generated/autoksyms.h Sounds good. > You can do it like this: > > > quiet_cmd_autoksyms_h = GEN $@ > cmd_autoksyms_h = mkdir -p $(dir $@); $(BASH) > $(srctree)/scripts/gen_autoksyms.sh $@ > > $(autoksyms_h): > $(call cmd,autoksyms_h) Nice, thanks for sharing! Quentin
On Tuesday 11 Feb 2020 at 10:14:14 (+0800), kbuild test robot wrote: > Hi Quentin, > > Thank you for the patch! Yet something to improve: > > [auto build test ERROR on kbuild/for-next] > [also build test ERROR on linux/master linus/master v5.6-rc1 next-20200210] > [if your patch is applied to the wrong git tree, please drop us a note to help > improve the system. BTW, we also suggest to use '--base' option to specify the > base tree in git format-patch, please see https://stackoverflow.com/a/37406982] > > url: https://github.com/0day-ci/linux/commits/Quentin-Perret/kbuild-allow-symbol-whitelisting-with-TRIM_UNUSED_KSYM/20200211-020659 > base: https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next > config: i386-randconfig-c002-20200211 (attached as .config) > compiler: gcc-7 (Debian 7.5.0-4) 7.5.0 > reproduce: > # save the attached .config to linux build tree > make ARCH=i386 > > If you fix the issue, kindly add following tag > Reported-by: kbuild test robot <lkp@intel.com> > > All errors (new ones prefixed by >>): > > >> ERROR: "trace_event_raw_init" [lib/objagg.ko] undefined! > >> ERROR: "trace_event_reg" [lib/objagg.ko] undefined! > >> ERROR: "ida_destroy" [lib/objagg.ko] undefined! > >> ERROR: "rhashtable_destroy" [lib/objagg.ko] undefined! > >> ERROR: "kmalloc_order_trace" [lib/objagg.ko] undefined! > >> ERROR: "sort" [lib/objagg.ko] undefined! > >> ERROR: "__raw_spin_lock_init" [lib/objagg.ko] undefined! > >> ERROR: "rhashtable_init" [lib/objagg.ko] undefined! > >> ERROR: "rht_bucket_nested" [lib/objagg.ko] undefined! > >> ERROR: "__list_del_entry_valid" [lib/objagg.ko] undefined! > >> ERROR: "__rht_bucket_nested" [lib/objagg.ko] undefined! > >> ERROR: "kmem_cache_alloc_trace" [lib/objagg.ko] undefined! > >> ERROR: "kmalloc_caches" [lib/objagg.ko] undefined! > >> ERROR: "queue_work_on" [lib/objagg.ko] undefined! > >> ERROR: "system_wq" [lib/objagg.ko] undefined! > >> ERROR: "kfree" [lib/objagg.ko] undefined! > >> ERROR: "__list_add_valid" [lib/objagg.ko] undefined! > >> ERROR: "lockdep_rht_bucket_is_held" [lib/objagg.ko] undefined! > >> ERROR: "rhashtable_insert_slow" [lib/objagg.ko] undefined! > >> ERROR: "__local_bh_enable_ip" [lib/objagg.ko] undefined! I find myself unable to reproduce this error on my box. Could you please provide the full build log ? In the meantime I'll proceed to send a v4. Thanks, Quentin
On Thursday 13 Feb 2020 at 16:42:51 (+0800), kernel test robot wrote: > I attached the build log for your reference. Thanks! <snip> > scripts/kconfig/conf --syncconfig Kconfig > SYSTBL arch/x86/include/generated/asm/syscalls_32.h > SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h > UPD include/config/kernel.release > SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h > SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h > UPD include/generated/utsrelease.h > HOSTCC scripts/dtc/dtc.o > HOSTCC scripts/dtc/flattree.o > ./scripts/gen_autoksyms.sh: 24: ./scripts/gen_autoksyms.sh: [[: not found > ./scripts/gen_autoksyms.sh: 35: ./scripts/gen_autoksyms.sh: [[: not found > cat: /home/nfs/linux//dev/null: No such file or directory OK, so that's the issue, which Nicolas pointed out earlier. That should be fixed in v4. Thanks, Quentin
diff --git a/Makefile b/Makefile index 6a01b073915e..e5c389d189f7 100644 --- a/Makefile +++ b/Makefile @@ -1065,7 +1065,7 @@ autoksyms_h := $(if $(CONFIG_TRIM_UNUSED_KSYMS), include/generated/autoksyms.h) $(autoksyms_h): $(Q)mkdir -p $(dir $@) - $(Q)touch $@ + $(Q)$(CONFIG_SHELL) $(srctree)/scripts/gen_autoksyms.sh $@ ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink) diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh index ce0919c3791a..ae033ab03a4a 100755 --- a/scripts/gen_autoksyms.sh +++ b/scripts/gen_autoksyms.sh @@ -32,7 +32,8 @@ cat > "$output_file" << EOT EOT -sed 's/ko$/mod/' modules.order | +[[ -f modules.order ]] && modlist=modules.order || modlist=/dev/null +sed 's/ko$/mod/' $modlist | xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | cat - "$ksym_wl" | sort -u |
When doing a cold build, autoksyms.h starts empty, and is updated late in the build process to have visibility over the symbols used by in-tree drivers. But since the symbol whitelist is known upfront, it can be used to pre-populate autoksyms.h and maximize the amount of code that can be compiled to its final state in a single pass, hence reducing build time. Do this by using gen_autoksyms.sh to initialize autoksyms.h instead of creating an empty file. Signed-off-by: Quentin Perret <qperret@google.com> --- Makefile | 2 +- scripts/gen_autoksyms.sh | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-)