Message ID | 20210226062548.3334081-1-masahiroy@kernel.org (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | [v2] kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO | expand |
On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > does not work as expected if the .config file has already specified > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling > CONFIG_LTO_CLANG. > > So, the user-supplied whitelist and LTO-specific white list must be > independent of each other. > > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO > handle whitelists in the same way. > > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > Changes in v2: > - Rebase on top of Arnd's patch: > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/ > > init/Kconfig | 1 - > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++--------- > scripts/lto-used-symbollist.txt | 6 ------ People who want to use their own "white-listed" (allow-listed) symbollist-file can do that via CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list". Correct? Why do you delete the default "scripts/lto-used-symbollist.txt" file and remove the default in the appropriate Kconfig for people who want to enable Clang-(Thin)LTO? These people should now use CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"? But again - the file was deleted with your patch. Do I miss something? - Sedat - > 3 files changed, 26 insertions(+), 16 deletions(-) > delete mode 100644 scripts/lto-used-symbollist.txt > > diff --git a/init/Kconfig b/init/Kconfig > index 719871f8727c..64c32300d1b4 100644 > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -2283,7 +2283,6 @@ config TRIM_UNUSED_KSYMS > config UNUSED_KSYMS_WHITELIST > string "Whitelist of symbols to keep in ksymtab" > depends on TRIM_UNUSED_KSYMS > - default "scripts/lto-used-symbollist.txt" if LTO_CLANG > help > By default, all unused exported symbols will be un-exported from the > build when TRIM_UNUSED_KSYMS is selected. > diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh > index d54dfba15bf2..da320151e7c3 100755 > --- a/scripts/gen_autoksyms.sh > +++ b/scripts/gen_autoksyms.sh > @@ -19,7 +19,26 @@ esac > # We need access to CONFIG_ symbols > . include/config/auto.conf > > -ksym_wl=/dev/null > +needed_symbols= > + > +# Special case for modversions (see modpost.c) > +if [ -n "$CONFIG_MODVERSIONS" ]; then > + needed_symbols="$needed_symbols module_layout" > +fi > + > +# With CONFIG_LTO_CLANG, LLVM bitcode has not yet been compiled into a binary > +# when the .mod files are generated, which means they don't yet contain > +# references to certain symbols that will be present in the final binaries. > +if [ -n "$CONFIG_LTO_CLANG" ]; then > + # intrinsic functions > + needed_symbols="$needed_symbols memcpy memmove memset" > + # ftrace > + needed_symbols="$needed_symbols _mcount" > + # stack protector symbols > + needed_symbols="$needed_symbols __stack_chk_fail __stack_chk_guard" > +fi > + > +ksym_wl= > if [ -n "$CONFIG_UNUSED_KSYMS_WHITELIST" ]; then > # Use 'eval' to expand the whitelist path and check if it is relative > eval ksym_wl="$CONFIG_UNUSED_KSYMS_WHITELIST" > @@ -40,16 +59,14 @@ cat > "$output_file" << EOT > EOT > > [ -f modules.order ] && modlist=modules.order || modlist=/dev/null > -sed 's/ko$/mod/' $modlist | > -xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | > -cat - "$ksym_wl" | > + > +{ > + sed 's/ko$/mod/' $modlist | xargs -n1 sed -n -e '2p' > + echo "$needed_symbols" > + [ -n "$ksym_wl" ] && cat "$ksym_wl" > +} | sed -e 's/ /\n/g' | sed -n -e '/^$/!p' | > # Remove the dot prefix for ppc64; symbol names with a dot (.) hold entry > # point addresses. > sed -e 's/^\.//' | > sort -u | > sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$output_file" > - > -# Special case for modversions (see modpost.c) > -if [ -n "$CONFIG_MODVERSIONS" ]; then > - echo "#define __KSYM_module_layout 1" >> "$output_file" > -fi > diff --git a/scripts/lto-used-symbollist.txt b/scripts/lto-used-symbollist.txt > deleted file mode 100644 > index 406ada65e926..000000000000 > --- a/scripts/lto-used-symbollist.txt > +++ /dev/null > @@ -1,6 +0,0 @@ > -memcpy > -memmove > -memset > -_mcount > -__stack_chk_fail > -__stack_chk_guard > -- > 2.27.0 > > -- > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210226062548.3334081-1-masahiroy%40kernel.org.
On Fri, Feb 26, 2021 at 6:26 PM Sedat Dilek <sedat.dilek@gmail.com> wrote: > > On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > > does not work as expected if the .config file has already specified > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling > > CONFIG_LTO_CLANG. > > > > So, the user-supplied whitelist and LTO-specific white list must be > > independent of each other. > > > > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO > > handle whitelists in the same way. > > > > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > --- > > > > Changes in v2: > > - Rebase on top of Arnd's patch: > > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/ > > > > init/Kconfig | 1 - > > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++--------- > > scripts/lto-used-symbollist.txt | 6 ------ > > People who want to use their own "white-listed" (allow-listed) > symbollist-file can do that via > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list". > Correct? > > Why do you delete the default "scripts/lto-used-symbollist.txt" file > and remove the default in the appropriate Kconfig for people who want > to enable Clang-(Thin)LTO? > These people should now use > CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"? > But again - the file was deleted with your patch. > Do I miss something? I think so. I moved those symbols to scripts/gen_autoksyms.sh
On Sat, Feb 27, 2021 at 7:55 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > On Fri, Feb 26, 2021 at 6:26 PM Sedat Dilek <sedat.dilek@gmail.com> wrote: > > > > On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > > > does not work as expected if the .config file has already specified > > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling > > > CONFIG_LTO_CLANG. > > > > > > So, the user-supplied whitelist and LTO-specific white list must be > > > independent of each other. > > > > > > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO > > > handle whitelists in the same way. > > > > > > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > > --- > > > > > > Changes in v2: > > > - Rebase on top of Arnd's patch: > > > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/ > > > > > > init/Kconfig | 1 - > > > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++--------- > > > scripts/lto-used-symbollist.txt | 6 ------ > > > > People who want to use their own "white-listed" (allow-listed) > > symbollist-file can do that via > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list". > > Correct? > > > > Why do you delete the default "scripts/lto-used-symbollist.txt" file > > and remove the default in the appropriate Kconfig for people who want > > to enable Clang-(Thin)LTO? > > These people should now use > > CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"? > > But again - the file was deleted with your patch. > > Do I miss something? > > I think so. > > I moved those symbols to scripts/gen_autoksyms.sh > OK, I have overseen hat. - Sedat -
On Sat, Feb 27, 2021 at 8:08 AM Sedat Dilek <sedat.dilek@gmail.com> wrote: > > On Sat, Feb 27, 2021 at 7:55 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > On Fri, Feb 26, 2021 at 6:26 PM Sedat Dilek <sedat.dilek@gmail.com> wrote: > > > > > > On Fri, Feb 26, 2021 at 7:26 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > > > Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > > > > does not work as expected if the .config file has already specified > > > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling > > > > CONFIG_LTO_CLANG. > > > > > > > > So, the user-supplied whitelist and LTO-specific white list must be > > > > independent of each other. > > > > > > > > I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO > > > > handle whitelists in the same way. > > > > > > > > Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols") > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> - Sedat - > > > > --- > > > > > > > > Changes in v2: > > > > - Rebase on top of Arnd's patch: > > > > https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/ > > > > > > > > init/Kconfig | 1 - > > > > scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++--------- > > > > scripts/lto-used-symbollist.txt | 6 ------ > > > > > > People who want to use their own "white-listed" (allow-listed) > > > symbollist-file can do that via > > > CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list". > > > Correct? > > > > > > Why do you delete the default "scripts/lto-used-symbollist.txt" file > > > and remove the default in the appropriate Kconfig for people who want > > > to enable Clang-(Thin)LTO? > > > These people should now use > > > CONFIG_UNUSED_KSYMS_WHITELIST="scripts/lto-used-symbollist.txt"? > > > But again - the file was deleted with your patch. > > > Do I miss something? > > > > I think so. > > > > I moved those symbols to scripts/gen_autoksyms.sh > > > > OK, I have overseen hat. > > - Sedat -
diff --git a/init/Kconfig b/init/Kconfig index 719871f8727c..64c32300d1b4 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2283,7 +2283,6 @@ config TRIM_UNUSED_KSYMS config UNUSED_KSYMS_WHITELIST string "Whitelist of symbols to keep in ksymtab" depends on TRIM_UNUSED_KSYMS - default "scripts/lto-used-symbollist.txt" if LTO_CLANG help By default, all unused exported symbols will be un-exported from the build when TRIM_UNUSED_KSYMS is selected. diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh index d54dfba15bf2..da320151e7c3 100755 --- a/scripts/gen_autoksyms.sh +++ b/scripts/gen_autoksyms.sh @@ -19,7 +19,26 @@ esac # We need access to CONFIG_ symbols . include/config/auto.conf -ksym_wl=/dev/null +needed_symbols= + +# Special case for modversions (see modpost.c) +if [ -n "$CONFIG_MODVERSIONS" ]; then + needed_symbols="$needed_symbols module_layout" +fi + +# With CONFIG_LTO_CLANG, LLVM bitcode has not yet been compiled into a binary +# when the .mod files are generated, which means they don't yet contain +# references to certain symbols that will be present in the final binaries. +if [ -n "$CONFIG_LTO_CLANG" ]; then + # intrinsic functions + needed_symbols="$needed_symbols memcpy memmove memset" + # ftrace + needed_symbols="$needed_symbols _mcount" + # stack protector symbols + needed_symbols="$needed_symbols __stack_chk_fail __stack_chk_guard" +fi + +ksym_wl= if [ -n "$CONFIG_UNUSED_KSYMS_WHITELIST" ]; then # Use 'eval' to expand the whitelist path and check if it is relative eval ksym_wl="$CONFIG_UNUSED_KSYMS_WHITELIST" @@ -40,16 +59,14 @@ cat > "$output_file" << EOT EOT [ -f modules.order ] && modlist=modules.order || modlist=/dev/null -sed 's/ko$/mod/' $modlist | -xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- | -cat - "$ksym_wl" | + +{ + sed 's/ko$/mod/' $modlist | xargs -n1 sed -n -e '2p' + echo "$needed_symbols" + [ -n "$ksym_wl" ] && cat "$ksym_wl" +} | sed -e 's/ /\n/g' | sed -n -e '/^$/!p' | # Remove the dot prefix for ppc64; symbol names with a dot (.) hold entry # point addresses. sed -e 's/^\.//' | sort -u | sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$output_file" - -# Special case for modversions (see modpost.c) -if [ -n "$CONFIG_MODVERSIONS" ]; then - echo "#define __KSYM_module_layout 1" >> "$output_file" -fi diff --git a/scripts/lto-used-symbollist.txt b/scripts/lto-used-symbollist.txt deleted file mode 100644 index 406ada65e926..000000000000 --- a/scripts/lto-used-symbollist.txt +++ /dev/null @@ -1,6 +0,0 @@ -memcpy -memmove -memset -_mcount -__stack_chk_fail -__stack_chk_guard
Commit fbe078d397b4 ("kbuild: lto: add a default list of used symbols") does not work as expected if the .config file has already specified CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling CONFIG_LTO_CLANG. So, the user-supplied whitelist and LTO-specific white list must be independent of each other. I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO handle whitelists in the same way. Fixes: fbe078d397b4 ("kbuild: lto: add a default list of used symbols") Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- Changes in v2: - Rebase on top of Arnd's patch: https://lore.kernel.org/lkml/20210225143456.3829513-1-arnd@kernel.org/ init/Kconfig | 1 - scripts/gen_autoksyms.sh | 35 ++++++++++++++++++++++++--------- scripts/lto-used-symbollist.txt | 6 ------ 3 files changed, 26 insertions(+), 16 deletions(-) delete mode 100644 scripts/lto-used-symbollist.txt