diff mbox series

[kbuild,v3,2/2] kbuild: add an elfnote with type BUILD_COMPILER_LTO_INFO

Message ID 20210401012417.1802681-1-yhs@fb.com (mailing list archive)
State New
Headers show
Series add an elfnote with type BUILD_COMPILER_LTO_INFO | expand

Commit Message

Yonghong Song April 1, 2021, 1:24 a.m. UTC
Currently, clang LTO built vmlinux won't work with pahole.
LTO introduced cross-cu dwarf tag references and broke
current pahole model which handles one cu as a time.
The solution is to merge all cu's as one pahole cu as in [1].
We would like to do this merging only if cross-cu dwarf
references happens. The LTO build mode is a pretty good
indication for that.

In earlier version of this patch ([2]), clang flag
-grecord-gcc-switches is proposed to add to compilation flags
so pahole could detect "-flto" and then merging cu's.
This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux
is built with LTO. Such a cheap way to get whether the vmlinux
is built with LTO or not helps pahole but is also useful
for tracing as LTO may inline/delete/demote global functions,
promote static functions, etc.

So this patch added an elfnote with type BUILD_COMPILER_LTO_INFO.
The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got
  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:
     description data: 01 00 00 00
which indicates the vmlinux is built with LTO.

 [1] https://lore.kernel.org/bpf/20210325065316.3121287-1-yhs@fb.com/
 [2] https://lore.kernel.org/bpf/20210331001623.2778934-1-yhs@fb.com/

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 include/linux/compiler.h | 8 ++++++++
 include/linux/elfnote.h  | 1 +
 init/version.c           | 2 ++
 scripts/mod/modpost.c    | 1 +
 4 files changed, 12 insertions(+)

Comments

Nick Desaulniers April 1, 2021, 6:28 p.m. UTC | #1
On Wed, Mar 31, 2021 at 6:24 PM Yonghong Song <yhs@fb.com> wrote:
>
> Currently, clang LTO built vmlinux won't work with pahole.
> LTO introduced cross-cu dwarf tag references and broke
> current pahole model which handles one cu as a time.
> The solution is to merge all cu's as one pahole cu as in [1].
> We would like to do this merging only if cross-cu dwarf
> references happens. The LTO build mode is a pretty good
> indication for that.
>
> In earlier version of this patch ([2]), clang flag
> -grecord-gcc-switches is proposed to add to compilation flags
> so pahole could detect "-flto" and then merging cu's.
> This will increate the binary size of 1% without LTO though.
>
> Arnaldo suggested to use a note to indicate the vmlinux
> is built with LTO. Such a cheap way to get whether the vmlinux
> is built with LTO or not helps pahole but is also useful
> for tracing as LTO may inline/delete/demote global functions,
> promote static functions, etc.
>
> So this patch added an elfnote with type BUILD_COMPILER_LTO_INFO.
> The owner of the note is "Linux".
>
> With gcc 8.4.1 and clang trunk, without LTO, I got
>   $ readelf -n vmlinux
>   Displaying notes found in: .notes
>     Owner                Data size        Description
>   ...
>     Linux                0x00000004       func
>      description data: 00 00 00 00
>   ...
> With "readelf -x ".notes" vmlinux", I can verify the above "func"
> with type code 0x101.
>
> With clang thin-LTO, I got the same as above except the following:
>      description data: 01 00 00 00
> which indicates the vmlinux is built with LTO.
>
>  [1] https://lore.kernel.org/bpf/20210325065316.3121287-1-yhs@fb.com/
>  [2] https://lore.kernel.org/bpf/20210331001623.2778934-1-yhs@fb.com/
>
> Signed-off-by: Yonghong Song <yhs@fb.com>
> ---
>  include/linux/compiler.h | 8 ++++++++
>  include/linux/elfnote.h  | 1 +
>  init/version.c           | 2 ++
>  scripts/mod/modpost.c    | 1 +
>  4 files changed, 12 insertions(+)
>
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index df5b405e6305..b92930877277 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -245,6 +245,14 @@ static inline void *offset_to_ptr(const int *off)
>   */
>  #define prevent_tail_call_optimization()       mb()
>
> +#include <linux/elfnote.h>
> +
> +#ifdef CONFIG_LTO
> +#define BUILD_COMPILER_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_BUILD_LTO, 1)
> +#else
> +#define BUILD_COMPILER_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_BUILD_LTO, 0)
> +#endif

With this approach BUILD_COMPILER_LTO_INFO won't be available `#ifdef
__ASSEMBLER__`; we don't need it today, and perhaps YAGNI, but I think
I prefer how include/linux/build-salt.h defines
LINUX_ELFNOTE_BUILD_SALT and keeps it isolated there.  Similarly, I
think it would be better to create a new header, say
include/linux/elfnote-lto.h that is basically a copy of
include/linux/build-salt.h, but with the relevant defines replaced
with the LTO identifiers you add above.  Then init/version.c and
scripts/mod/modpost.c can include include/linux/elfnote-lto.h and you
don't have to touch include/linux/build-salt.h and we can keep the
elfnote "types" isolated to their respective headers (otherwise this
approach reduces the usefulness of include/linux/build-salt.h even
existing, IMO. Feels like it should just be merged into
include/linux/elfnote.h entirely at that point).

But, this is a much nicer approach! I forgot that elf notes were a thing!

> +
>  #include <asm/rwonce.h>
>
>  #endif /* __LINUX_COMPILER_H */
> diff --git a/include/linux/elfnote.h b/include/linux/elfnote.h
> index 04af7ac40b1a..f5ec2b50ab7d 100644
> --- a/include/linux/elfnote.h
> +++ b/include/linux/elfnote.h
> @@ -100,5 +100,6 @@
>   * The types for "Linux" owned notes.
>   */
>  #define LINUX_ELFNOTE_BUILD_SALT       0x100
> +#define LINUX_ELFNOTE_BUILD_LTO                0x101
>
>  #endif /* _LINUX_ELFNOTE_H */
> diff --git a/init/version.c b/init/version.c
> index 92afc782b043..a4f74b06fe78 100644
> --- a/init/version.c
> +++ b/init/version.c
> @@ -9,6 +9,7 @@
>
>  #include <generated/compile.h>
>  #include <linux/build-salt.h>
> +#include <linux/compiler.h>
>  #include <linux/export.h>
>  #include <linux/uts.h>
>  #include <linux/utsname.h>
> @@ -45,3 +46,4 @@ const char linux_proc_banner[] =
>         " (" LINUX_COMPILER ") %s\n";
>
>  BUILD_SALT;
> +BUILD_COMPILER_LTO_INFO;
> diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
> index 24725e50c7b4..713c0d5d5525 100644
> --- a/scripts/mod/modpost.c
> +++ b/scripts/mod/modpost.c
> @@ -2195,6 +2195,7 @@ static void add_header(struct buffer *b, struct module *mod)
>         buf_printf(b, "#include <linux/compiler.h>\n");
>         buf_printf(b, "\n");
>         buf_printf(b, "BUILD_SALT;\n");
> +       buf_printf(b, "BUILD_COMPILER_LTO_INFO;\n");
>         buf_printf(b, "\n");
>         buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
>         buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
> --
> 2.30.2
>
Yonghong Song April 1, 2021, 8:50 p.m. UTC | #2
On 4/1/21 11:28 AM, Nick Desaulniers wrote:
> On Wed, Mar 31, 2021 at 6:24 PM Yonghong Song <yhs@fb.com> wrote:
>>
>> Currently, clang LTO built vmlinux won't work with pahole.
>> LTO introduced cross-cu dwarf tag references and broke
>> current pahole model which handles one cu as a time.
>> The solution is to merge all cu's as one pahole cu as in [1].
>> We would like to do this merging only if cross-cu dwarf
>> references happens. The LTO build mode is a pretty good
>> indication for that.
>>
>> In earlier version of this patch ([2]), clang flag
>> -grecord-gcc-switches is proposed to add to compilation flags
>> so pahole could detect "-flto" and then merging cu's.
>> This will increate the binary size of 1% without LTO though.
>>
>> Arnaldo suggested to use a note to indicate the vmlinux
>> is built with LTO. Such a cheap way to get whether the vmlinux
>> is built with LTO or not helps pahole but is also useful
>> for tracing as LTO may inline/delete/demote global functions,
>> promote static functions, etc.
>>
>> So this patch added an elfnote with type BUILD_COMPILER_LTO_INFO.
>> The owner of the note is "Linux".
>>
>> With gcc 8.4.1 and clang trunk, without LTO, I got
>>    $ readelf -n vmlinux
>>    Displaying notes found in: .notes
>>      Owner                Data size        Description
>>    ...
>>      Linux                0x00000004       func
>>       description data: 00 00 00 00
>>    ...
>> With "readelf -x ".notes" vmlinux", I can verify the above "func"
>> with type code 0x101.
>>
>> With clang thin-LTO, I got the same as above except the following:
>>       description data: 01 00 00 00
>> which indicates the vmlinux is built with LTO.
>>
>>   [1] https://lore.kernel.org/bpf/20210325065316.3121287-1-yhs@fb.com/
>>   [2] https://lore.kernel.org/bpf/20210331001623.2778934-1-yhs@fb.com/
>>
>> Signed-off-by: Yonghong Song <yhs@fb.com>
>> ---
>>   include/linux/compiler.h | 8 ++++++++
>>   include/linux/elfnote.h  | 1 +
>>   init/version.c           | 2 ++
>>   scripts/mod/modpost.c    | 1 +
>>   4 files changed, 12 insertions(+)
>>
>> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
>> index df5b405e6305..b92930877277 100644
>> --- a/include/linux/compiler.h
>> +++ b/include/linux/compiler.h
>> @@ -245,6 +245,14 @@ static inline void *offset_to_ptr(const int *off)
>>    */
>>   #define prevent_tail_call_optimization()       mb()
>>
>> +#include <linux/elfnote.h>
>> +
>> +#ifdef CONFIG_LTO
>> +#define BUILD_COMPILER_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_BUILD_LTO, 1)
>> +#else
>> +#define BUILD_COMPILER_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_BUILD_LTO, 0)
>> +#endif
> 
> With this approach BUILD_COMPILER_LTO_INFO won't be available `#ifdef
> __ASSEMBLER__`; we don't need it today, and perhaps YAGNI, but I think

That is true. I didn't add it since I don't feel it. BUILD_SALT also 
added to vdso binary which I feel we don't need it today.

> I prefer how include/linux/build-salt.h defines
> LINUX_ELFNOTE_BUILD_SALT and keeps it isolated there.  Similarly, I
> think it would be better to create a new header, say
> include/linux/elfnote-lto.h that is basically a copy of
> include/linux/build-salt.h, but with the relevant defines replaced

Having a separate header like elfnote-lto.h sounds okay. Originally
I am reluctant to add a new header file, but maybe a new header
file is much cleaner than otherwise.

> with the LTO identifiers you add above.  Then init/version.c and
> scripts/mod/modpost.c can include include/linux/elfnote-lto.h and you
> don't have to touch include/linux/build-salt.h and we can keep the
> elfnote "types" isolated to their respective headers (otherwise this
> approach reduces the usefulness of include/linux/build-salt.h even
> existing, IMO. Feels like it should just be merged into
> include/linux/elfnote.h entirely at that point).

The only "drawback" is the type values are scattered in different
files which I am not really comfortable with it. But with consistent
naming convention, all values can be easily searched so we may not
have issue at all.

> 
> But, this is a much nicer approach! I forgot that elf notes were a thing!
> 
>> +
>>   #include <asm/rwonce.h>
>>
>>   #endif /* __LINUX_COMPILER_H */
>> diff --git a/include/linux/elfnote.h b/include/linux/elfnote.h
>> index 04af7ac40b1a..f5ec2b50ab7d 100644
>> --- a/include/linux/elfnote.h
>> +++ b/include/linux/elfnote.h
>> @@ -100,5 +100,6 @@
>>    * The types for "Linux" owned notes.
>>    */
>>   #define LINUX_ELFNOTE_BUILD_SALT       0x100
>> +#define LINUX_ELFNOTE_BUILD_LTO                0x101
>>
>>   #endif /* _LINUX_ELFNOTE_H */
>> diff --git a/init/version.c b/init/version.c
>> index 92afc782b043..a4f74b06fe78 100644
>> --- a/init/version.c
>> +++ b/init/version.c
>> @@ -9,6 +9,7 @@
>>
>>   #include <generated/compile.h>
>>   #include <linux/build-salt.h>
>> +#include <linux/compiler.h>
>>   #include <linux/export.h>
>>   #include <linux/uts.h>
>>   #include <linux/utsname.h>
>> @@ -45,3 +46,4 @@ const char linux_proc_banner[] =
>>          " (" LINUX_COMPILER ") %s\n";
>>
>>   BUILD_SALT;
>> +BUILD_COMPILER_LTO_INFO;
>> diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
>> index 24725e50c7b4..713c0d5d5525 100644
>> --- a/scripts/mod/modpost.c
>> +++ b/scripts/mod/modpost.c
>> @@ -2195,6 +2195,7 @@ static void add_header(struct buffer *b, struct module *mod)
>>          buf_printf(b, "#include <linux/compiler.h>\n");
>>          buf_printf(b, "\n");
>>          buf_printf(b, "BUILD_SALT;\n");
>> +       buf_printf(b, "BUILD_COMPILER_LTO_INFO;\n");
>>          buf_printf(b, "\n");
>>          buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
>>          buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
>> --
>> 2.30.2
>>
> 
>
diff mbox series

Patch

diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index df5b405e6305..b92930877277 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -245,6 +245,14 @@  static inline void *offset_to_ptr(const int *off)
  */
 #define prevent_tail_call_optimization()	mb()
 
+#include <linux/elfnote.h>
+
+#ifdef CONFIG_LTO
+#define BUILD_COMPILER_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_BUILD_LTO, 1)
+#else
+#define BUILD_COMPILER_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_BUILD_LTO, 0)
+#endif
+
 #include <asm/rwonce.h>
 
 #endif /* __LINUX_COMPILER_H */
diff --git a/include/linux/elfnote.h b/include/linux/elfnote.h
index 04af7ac40b1a..f5ec2b50ab7d 100644
--- a/include/linux/elfnote.h
+++ b/include/linux/elfnote.h
@@ -100,5 +100,6 @@ 
  * The types for "Linux" owned notes.
  */
 #define LINUX_ELFNOTE_BUILD_SALT	0x100
+#define LINUX_ELFNOTE_BUILD_LTO		0x101
 
 #endif /* _LINUX_ELFNOTE_H */
diff --git a/init/version.c b/init/version.c
index 92afc782b043..a4f74b06fe78 100644
--- a/init/version.c
+++ b/init/version.c
@@ -9,6 +9,7 @@ 
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/compiler.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@  const char linux_proc_banner[] =
 	" (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_COMPILER_LTO_INFO;
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 24725e50c7b4..713c0d5d5525 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -2195,6 +2195,7 @@  static void add_header(struct buffer *b, struct module *mod)
 	buf_printf(b, "#include <linux/compiler.h>\n");
 	buf_printf(b, "\n");
 	buf_printf(b, "BUILD_SALT;\n");
+	buf_printf(b, "BUILD_COMPILER_LTO_INFO;\n");
 	buf_printf(b, "\n");
 	buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
 	buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");