mbox series

[v4,00/14] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)

Message ID 20220508190631.2386038-1-masahiroy@kernel.org (mailing list archive)
Headers show
Series kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) | expand

Message

Masahiro Yamada May 8, 2022, 7:06 p.m. UTC
This is the third batch of cleanups in this development cycle.

Major changes in v4:
 - Move static EXPORT_SYMBOL check to a script
 - Some refactoring

Major changes in v3:

 - Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.

Major changes in v2:

 - V1 did not work with CONFIG_MODULE_REL_CRCS.
   I fixed this for v2.

 - Reflect some review comments in v1

 - Refactor the code more

 - Avoid too long argument error



Masahiro Yamada (14):
  modpost: remove left-over cross_compile declaration
  modpost: change the license of EXPORT_SYMBOL to bool type
  modpost: split the section mismatch checks into section-check.c
  modpost: add sym_find_with_module() helper
  modpost: extract symbol versions from *.cmd files
  kbuild: link symbol CRCs at final link, removing
    CONFIG_MODULE_REL_CRCS
  kbuild: stop merging *.symversions
  genksyms: adjust the output format to modpost
  kbuild: do not create *.prelink.o for Clang LTO or IBT
  kbuild: check static EXPORT_SYMBOL* by script instead of modpost
  kbuild: make built-in.a rule robust against too long argument error
  kbuild: make *.mod rule robust against too long argument error
  kbuild: add cmd_and_savecmd macro
  kbuild: rebuild multi-object modules when objtool is updated

 arch/powerpc/Kconfig            |    1 -
 arch/s390/Kconfig               |    1 -
 arch/um/Kconfig                 |    1 -
 include/asm-generic/export.h    |   22 +-
 include/linux/export-internal.h |   16 +
 include/linux/export.h          |   30 +-
 init/Kconfig                    |    4 -
 kernel/module.c                 |   10 +-
 scripts/Kbuild.include          |   10 +-
 scripts/Makefile.build          |  134 +--
 scripts/Makefile.lib            |    7 -
 scripts/Makefile.modfinal       |    5 +-
 scripts/Makefile.modpost        |    9 +-
 scripts/check-local-export      |   48 +
 scripts/genksyms/genksyms.c     |   18 +-
 scripts/link-vmlinux.sh         |   33 +-
 scripts/mod/Makefile            |    2 +-
 scripts/mod/modpost.c           | 1499 ++++---------------------------
 scripts/mod/modpost.h           |   35 +-
 scripts/mod/section-check.c     | 1222 +++++++++++++++++++++++++
 20 files changed, 1551 insertions(+), 1556 deletions(-)
 create mode 100644 include/linux/export-internal.h
 create mode 100755 scripts/check-local-export
 create mode 100644 scripts/mod/section-check.c

Comments

Masahiro Yamada May 9, 2022, 4:24 a.m. UTC | #1
On Mon, May 9, 2022 at 4:09 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> This is the third batch of cleanups in this development cycle.
>
> Major changes in v4:
>  - Move static EXPORT_SYMBOL check to a script
>  - Some refactoring
>
> Major changes in v3:
>
>  - Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.
>
> Major changes in v2:
>
>  - V1 did not work with CONFIG_MODULE_REL_CRCS.
>    I fixed this for v2.
>
>  - Reflect some review comments in v1
>
>  - Refactor the code more
>
>  - Avoid too long argument error

This series is available at
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
lto-cleanup-v4




>
>
> Masahiro Yamada (14):
>   modpost: remove left-over cross_compile declaration
>   modpost: change the license of EXPORT_SYMBOL to bool type
>   modpost: split the section mismatch checks into section-check.c
>   modpost: add sym_find_with_module() helper
>   modpost: extract symbol versions from *.cmd files
>   kbuild: link symbol CRCs at final link, removing
>     CONFIG_MODULE_REL_CRCS
>   kbuild: stop merging *.symversions
>   genksyms: adjust the output format to modpost
>   kbuild: do not create *.prelink.o for Clang LTO or IBT
>   kbuild: check static EXPORT_SYMBOL* by script instead of modpost
>   kbuild: make built-in.a rule robust against too long argument error
>   kbuild: make *.mod rule robust against too long argument error
>   kbuild: add cmd_and_savecmd macro
>   kbuild: rebuild multi-object modules when objtool is updated
>
>  arch/powerpc/Kconfig            |    1 -
>  arch/s390/Kconfig               |    1 -
>  arch/um/Kconfig                 |    1 -
>  include/asm-generic/export.h    |   22 +-
>  include/linux/export-internal.h |   16 +
>  include/linux/export.h          |   30 +-
>  init/Kconfig                    |    4 -
>  kernel/module.c                 |   10 +-
>  scripts/Kbuild.include          |   10 +-
>  scripts/Makefile.build          |  134 +--
>  scripts/Makefile.lib            |    7 -
>  scripts/Makefile.modfinal       |    5 +-
>  scripts/Makefile.modpost        |    9 +-
>  scripts/check-local-export      |   48 +
>  scripts/genksyms/genksyms.c     |   18 +-
>  scripts/link-vmlinux.sh         |   33 +-
>  scripts/mod/Makefile            |    2 +-
>  scripts/mod/modpost.c           | 1499 ++++---------------------------
>  scripts/mod/modpost.h           |   35 +-
>  scripts/mod/section-check.c     | 1222 +++++++++++++++++++++++++
>  20 files changed, 1551 insertions(+), 1556 deletions(-)
>  create mode 100644 include/linux/export-internal.h
>  create mode 100755 scripts/check-local-export
>  create mode 100644 scripts/mod/section-check.c
>
> --
> 2.32.0
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20220508190631.2386038-1-masahiroy%40kernel.org.
Nathan Chancellor May 9, 2022, 10:19 p.m. UTC | #2
[resending due to mailing list bounces from a large plain text attachment...]

On Mon, May 09, 2022 at 01:24:33PM +0900, Masahiro Yamada wrote:
> On Mon, May 9, 2022 at 4:09 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > This is the third batch of cleanups in this development cycle.
> >
> > Major changes in v4:
> >  - Move static EXPORT_SYMBOL check to a script
> >  - Some refactoring
> >
> > Major changes in v3:
> >
> >  - Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.
> >
> > Major changes in v2:
> >
> >  - V1 did not work with CONFIG_MODULE_REL_CRCS.
> >    I fixed this for v2.
> >
> >  - Reflect some review comments in v1
> >
> >  - Refactor the code more
> >
> >  - Avoid too long argument error
> 
> This series is available at
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> lto-cleanup-v4

Hi Masahiro,

I checked this out and went to run it through my QEMU tests but I see
two new errors.

Failure #1:

In file included from scripts/mod/section-check.c:3:
scripts/mod/modpost.h:15:10: fatal error: 'elfconfig.h' file not found
#include "elfconfig.h"
         ^~~~~~~~~~~~~
1 error generated.

I was able to get past that with

diff --git a/scripts/mod/Makefile b/scripts/mod/Makefile
index ca739c6c68a1..c33b83bfbcad 100644
--- a/scripts/mod/Makefile
+++ b/scripts/mod/Makefile
@@ -16,7 +16,7 @@ targets += $(devicetable-offsets-file) devicetable-offsets.s

 # dependencies on generated files need to be listed explicitly

-$(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o: $(obj)/elfconfig.h
+$(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o $(obj)/section-check.o: $(obj)/elfconfig.h
 $(obj)/file2alias.o: $(obj)/$(devicetable-offsets-file)

 quiet_cmd_elfconfig = MKELF   $@

Failure #2:

  GEN     .version
  CHK     include/generated/compile.h
  GEN     .tmp_initcalls.lds
  LTO     vmlinux.o
  OBJTOOL vmlinux.o
  MODPOST vmlinux.symvers
  MODINFO modules.builtin.modinfo
  GEN     modules.builtin
  LD      .tmp_vmlinux.btf
ld.lld: error: cannot open .vmlinux.export.o: No such file or directory
  BTF     .btf.vmlinux.bin.o
pahole: .tmp_vmlinux.btf: No such file or directory
  CC      .vmlinux.export.c
  LD      .tmp_vmlinux.kallsyms1
ld.lld: error: .btf.vmlinux.bin.o: unknown file type
make[1]: *** [Makefile:1159: vmlinux] Error 1

I was not really able to see what is going wrong here. Attached is the
configuration that I ran into this with. If you need any other
information, please let me know!

Cheers,
Nathan
Masahiro Yamada May 10, 2022, 6:55 a.m. UTC | #3
On Tue, May 10, 2022 at 7:13 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Mon, May 09, 2022 at 01:24:33PM +0900, Masahiro Yamada wrote:
> > On Mon, May 9, 2022 at 4:09 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > This is the third batch of cleanups in this development cycle.
> > >
> > > Major changes in v4:
> > >  - Move static EXPORT_SYMBOL check to a script
> > >  - Some refactoring
> > >
> > > Major changes in v3:
> > >
> > >  - Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.
> > >
> > > Major changes in v2:
> > >
> > >  - V1 did not work with CONFIG_MODULE_REL_CRCS.
> > >    I fixed this for v2.
> > >
> > >  - Reflect some review comments in v1
> > >
> > >  - Refactor the code more
> > >
> > >  - Avoid too long argument error
> >
> > This series is available at
> > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > lto-cleanup-v4
>
> Hi Masahiro,
>
> I checked this out and went to run it through my QEMU tests but I see
> two new errors.
>
> Failure #1:
>
> In file included from scripts/mod/section-check.c:3:
> scripts/mod/modpost.h:15:10: fatal error: 'elfconfig.h' file not found
> #include "elfconfig.h"
>          ^~~~~~~~~~~~~
> 1 error generated.
>
> I was able to get past that with
>
> diff --git a/scripts/mod/Makefile b/scripts/mod/Makefile
> index ca739c6c68a1..c33b83bfbcad 100644
> --- a/scripts/mod/Makefile
> +++ b/scripts/mod/Makefile
> @@ -16,7 +16,7 @@ targets += $(devicetable-offsets-file) devicetable-offsets.s
>
>  # dependencies on generated files need to be listed explicitly
>
> -$(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o: $(obj)/elfconfig.h
> +$(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o $(obj)/section-check.o: $(obj)/elfconfig.h
>  $(obj)/file2alias.o: $(obj)/$(devicetable-offsets-file)
>

Thanks for testing.


I will slightly refactor the code as follows.




@@ -16,7 +16,7 @@ targets += $(devicetable-offsets-file) devicetable-offsets.s

 # dependencies on generated files need to be listed explicitly

-$(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o: $(obj)/elfconfig.h
+$(addprefix $(obj)/, $(modpost-objs)): $(obj)/elfconfig.h

 $(obj)/file2alias.o: $(obj)/$(devicetable-offsets-file)

 quiet_cmd_elfconfig = MKELF   $@








>  quiet_cmd_elfconfig = MKELF   $@
>
> Failure #2:
>
>   GEN     .version
>   CHK     include/generated/compile.h
>   GEN     .tmp_initcalls.lds
>   LTO     vmlinux.o
>   OBJTOOL vmlinux.o
>   MODPOST vmlinux.symvers
>   MODINFO modules.builtin.modinfo
>   GEN     modules.builtin
>   LD      .tmp_vmlinux.btf
> ld.lld: error: cannot open .vmlinux.export.o: No such file or directory
>   BTF     .btf.vmlinux.bin.o
> pahole: .tmp_vmlinux.btf: No such file or directory
>   CC      .vmlinux.export.c
>   LD      .tmp_vmlinux.kallsyms1
> ld.lld: error: .btf.vmlinux.bin.o: unknown file type
> make[1]: *** [Makefile:1159: vmlinux] Error 1
>
> I was not really able to see what is going wrong here. Attached is the
> configuration that I ran into this with. If you need any other
> information, please let me know!

Ah, OK.
This is because .vmlinux.export.o is compiled after gen_btf.

I will swap the order in v5.





> Cheers,
> Nathan



--
Best Regards
Masahiro Yamada