| Message ID | 20200504031340.7103-1-nick.desaulniers@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Makefile: support compressed debug info | expand |
On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers <nick.desaulniers@gmail.com> wrote: > > As debug information gets larger and larger, it helps significantly save > the size of vmlinux images to compress the information in the debug > information sections. Note: this debug info is typically split off from > the final compressed kernel image, which is why vmlinux is what's used > in conjunction with GDB. Minimizing the debug info size should have no > impact on boot times, or final compressed kernel image size. > > All of the debug sections will have a `C` flag set. > $ readelf -S <object file> > > $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ > vmlinux.gcc75.uncompressed.dwarf4 > > FILE SIZE VM SIZE > -------------- -------------- > +0.0% +18 [ = ] 0 [Unmapped] > -73.3% -114Ki [ = ] 0 .debug_aranges > -76.2% -2.01Mi [ = ] 0 .debug_frame > -73.6% -2.89Mi [ = ] 0 .debug_str > -80.7% -4.66Mi [ = ] 0 .debug_abbrev > -82.9% -4.88Mi [ = ] 0 .debug_ranges > -70.5% -9.04Mi [ = ] 0 .debug_line > -79.3% -10.9Mi [ = ] 0 .debug_loc > -39.5% -88.6Mi [ = ] 0 .debug_info > -18.2% -123Mi [ = ] 0 TOTAL > > $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ > vmlinux.clang11.uncompressed.dwarf4 > > FILE SIZE VM SIZE > -------------- -------------- > +0.0% +23 [ = ] 0 [Unmapped] > -65.6% -871 [ = ] 0 .debug_aranges > -77.4% -1.84Mi [ = ] 0 .debug_frame > -82.9% -2.33Mi [ = ] 0 .debug_abbrev > -73.1% -2.43Mi [ = ] 0 .debug_str > -84.8% -3.07Mi [ = ] 0 .debug_ranges > -65.9% -8.62Mi [ = ] 0 .debug_line > -86.2% -40.0Mi [ = ] 0 .debug_loc > -42.0% -64.1Mi [ = ] 0 .debug_info > -22.1% -122Mi [ = ] 0 TOTAL > Hi Nick, thanks for the patch. I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?). Which linker did you use and has it an impact if you switch from ld.bfd to ld.lld? I tried a first normal run and in a 2nd one with CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10. My numbers (sizes in MiB): [ diffconfig ] $ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang /boot/config-5.7.0-rc4-2-amd64-clang BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" +DEBUG_INFO_COMPRESSED y [ compiler and linker ] $ clang-10 -v ClangBuiltLinux clang version 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/dileks/src/llvm-toolchain/install/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64 $ ld.lld-10 -v LLD 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU linkers) [ sizes vmlinux ] $ du -m 5.7.0-rc4-*/vmlinux* 409 5.7.0-rc4-1-amd64-clang/vmlinux 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed 404 5.7.0-rc4-1-amd64-clang/vmlinux.o 324 5.7.0-rc4-2-amd64-clang/vmlinux 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed 299 5.7.0-rc4-2-amd64-clang/vmlinux.o [ readelf (.debug_info as example) ] $ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 0000000006be1ee6 0000000000000000 0 0 1 $ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01749f18 0000000002ef04d2 0000000000000000 C 0 0 1 <--- XXX: "C (compressed)" Flag Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific) [ sizes linux-image debian packages ] $ du -m 5.7.0-rc4-*/linux-image*.deb 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb [ sizes linux-git dir (compilation finished ] 5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux 5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux [ xz compressed linux-image-dbg packages ] $ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz [ file-lists ] $ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | wc -l 4395 $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | wc -l 4395 [ file-lists vmlinux ] $ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 428588312 2020-05-04 06:15 ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 339341456 2020-05-04 12:24 ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux [ conclusion ] As you can see there is a size-reduction in case of vmlinux/vmlinux.o (debug) files... ...and my linux-git directory in total is smaller: 17963M vs. 14328M. But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in a fatter linux-image-dbg package. For a usage of vmlinux (debug) in a VM scenario this might be nice but seen from a debian repository perspective not. For the sake of completeness: I have just installed and booted the "normal" linux-image debian package - not the debug packages. Thanks. Regards, - Sedat - > Suggested-by: David Blaikie <blakie@google.com> > Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com> > --- > Makefile | 5 +++++ > lib/Kconfig.debug | 9 +++++++++ > 2 files changed, 14 insertions(+) > > diff --git a/Makefile b/Makefile > index 981eb902384b..313a054e5dc6 100644 > --- a/Makefile > +++ b/Makefile > @@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED > DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ > $(call cc-option,-fno-var-tracking) > endif > + > +ifdef CONFIG_DEBUG_INFO_COMPRESSED > +DEBUG_CFLAGS += -gz=zlib > +KBUILD_LDFLAGS += --compress-debug-sections=zlib > +endif > endif > > KBUILD_CFLAGS += $(DEBUG_CFLAGS) > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > index f6f9a039f736..1f4a47ba6c1b 100644 > --- a/lib/Kconfig.debug > +++ b/lib/Kconfig.debug > @@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED > DEBUG_INFO build and compile times are reduced too. > Only works with newer gcc versions. > > +config DEBUG_INFO_COMPRESSED > + bool "Compressed debugging information" > + depends on DEBUG_INFO > + depends on $(cc-option,-gz=zlib) > + depends on $(ld-option,--compress-debug-sections=zlib) > + help > + Compress the debug information using zlib. Requires GCC 5.0+ or Clang > + 5.0+. > + > config DEBUG_INFO_SPLIT > bool "Produce split debuginfo in .dwo files" > depends on DEBUG_INFO > -- > 2.17.1 > > -- > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200504031340.7103-1-nick.desaulniers%40gmail.com.
On 2020-05-04, Sedat Dilek wrote: >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers ><nick.desaulniers@gmail.com> wrote: >> >> As debug information gets larger and larger, it helps significantly save >> the size of vmlinux images to compress the information in the debug >> information sections. Note: this debug info is typically split off from >> the final compressed kernel image, which is why vmlinux is what's used >> in conjunction with GDB. Minimizing the debug info size should have no >> impact on boot times, or final compressed kernel image size. >> >> All of the debug sections will have a `C` flag set. >> $ readelf -S <object file> >> >> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ >> vmlinux.gcc75.uncompressed.dwarf4 >> >> FILE SIZE VM SIZE >> -------------- -------------- >> +0.0% +18 [ = ] 0 [Unmapped] >> -73.3% -114Ki [ = ] 0 .debug_aranges >> -76.2% -2.01Mi [ = ] 0 .debug_frame >> -73.6% -2.89Mi [ = ] 0 .debug_str >> -80.7% -4.66Mi [ = ] 0 .debug_abbrev >> -82.9% -4.88Mi [ = ] 0 .debug_ranges >> -70.5% -9.04Mi [ = ] 0 .debug_line >> -79.3% -10.9Mi [ = ] 0 .debug_loc >> -39.5% -88.6Mi [ = ] 0 .debug_info >> -18.2% -123Mi [ = ] 0 TOTAL >> >> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ >> vmlinux.clang11.uncompressed.dwarf4 >> >> FILE SIZE VM SIZE >> -------------- -------------- >> +0.0% +23 [ = ] 0 [Unmapped] >> -65.6% -871 [ = ] 0 .debug_aranges >> -77.4% -1.84Mi [ = ] 0 .debug_frame >> -82.9% -2.33Mi [ = ] 0 .debug_abbrev >> -73.1% -2.43Mi [ = ] 0 .debug_str >> -84.8% -3.07Mi [ = ] 0 .debug_ranges >> -65.9% -8.62Mi [ = ] 0 .debug_line >> -86.2% -40.0Mi [ = ] 0 .debug_loc >> -42.0% -64.1Mi [ = ] 0 .debug_info >> -22.1% -122Mi [ = ] 0 TOTAL >> > >Hi Nick, > >thanks for the patch. > >I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?). > >Which linker did you use and has it an impact if you switch from >ld.bfd to ld.lld? lld has supported the linker option --compress-debug-sections=zlib since about 5.0.0 (https://reviews.llvm.org/D31941) >I tried a first normal run and in a 2nd one with >CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10. > >My numbers (sizes in MiB): > >[ diffconfig ] > >$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang >/boot/config-5.7.0-rc4-2-amd64-clang > BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" >+DEBUG_INFO_COMPRESSED y > >[ compiler and linker ] > >$ clang-10 -v >ClangBuiltLinux clang version 10.0.1 >(https://github.com/llvm/llvm-project >92d5c1be9ee93850c0a8903f05f36a23ee835dc2) >Target: x86_64-unknown-linux-gnu >Thread model: posix >InstalledDir: /home/dileks/src/llvm-toolchain/install/bin >Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 >Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 >Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 >Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 >Candidate multilib: .;@m64 >Candidate multilib: 32;@m32 >Candidate multilib: x32;@mx32 >Selected multilib: .;@m64 > >$ ld.lld-10 -v >LLD 10.0.1 (https://github.com/llvm/llvm-project >92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU >linkers) > >[ sizes vmlinux ] > >$ du -m 5.7.0-rc4-*/vmlinux* >409 5.7.0-rc4-1-amd64-clang/vmlinux >7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed >404 5.7.0-rc4-1-amd64-clang/vmlinux.o >324 5.7.0-rc4-2-amd64-clang/vmlinux >7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed >299 5.7.0-rc4-2-amd64-clang/vmlinux.o > >[ readelf (.debug_info as example) ] > >$ readelf -S vmlinux.o > [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 > 0000000006be1ee6 0000000000000000 0 0 1 > >$ readelf -S vmlinux.o > [33] .debug_info PROGBITS 0000000000000000 01749f18 > 0000000002ef04d2 0000000000000000 C 0 0 1 <--- >XXX: "C (compressed)" Flag > >Key to Flags: > W (write), A (alloc), X (execute), M (merge), S (strings), I (info), > L (link order), O (extra OS processing required), G (group), T (TLS), > C (compressed), x (unknown), o (OS specific), E (exclude), > l (large), p (processor specific) > >[ sizes linux-image debian packages ] > >$ du -m 5.7.0-rc4-*/linux-image*.deb >47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb >424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb >47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb >771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > >[ sizes linux-git dir (compilation finished ] > >5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux >5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux > >[ xz compressed linux-image-dbg packages ] > >$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb >linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: >Debian binary package (format 2.0), with control.tar.xz, data >compression xz >$ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb >linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: >Debian binary package (format 2.0), with control.tar.xz, data >compression xz > >[ file-lists ] > >$ dpkg --contents >linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb >| wc -l >4395 >$ dpkg --contents >linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb >| wc -l >4395 > >[ file-lists vmlinux ] > >$ dpkg --contents >linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb >| grep vmlinux >-rwxr-xr-x root/root 428588312 2020-05-04 06:15 >./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux >lrwxrwxrwx root/root 0 2020-05-04 06:15 >./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> >../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux >lrwxrwxrwx root/root 0 2020-05-04 06:15 >./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> >lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > >$ dpkg --contents >linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb >| grep vmlinux >-rwxr-xr-x root/root 339341456 2020-05-04 12:24 >./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux >lrwxrwxrwx root/root 0 2020-05-04 12:24 >./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> >../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux >lrwxrwxrwx root/root 0 2020-05-04 12:24 >./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> >lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > >[ conclusion ] > >As you can see there is a size-reduction in case of vmlinux/vmlinux.o >(debug) files... >...and my linux-git directory in total is smaller: 17963M vs. 14328M. > >But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. >XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in >a fatter linux-image-dbg package. > >For a usage of vmlinux (debug) in a VM scenario this might be nice but >seen from a debian repository perspective not. > >For the sake of completeness: I have just installed and booted the >"normal" linux-image debian package - not the debug packages. > >Thanks. > >Regards, >- Sedat - > >> Suggested-by: David Blaikie <blakie@google.com> >> Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com> >> --- >> Makefile | 5 +++++ >> lib/Kconfig.debug | 9 +++++++++ >> 2 files changed, 14 insertions(+) >> >> diff --git a/Makefile b/Makefile >> index 981eb902384b..313a054e5dc6 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED >> DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ >> $(call cc-option,-fno-var-tracking) >> endif >> + >> +ifdef CONFIG_DEBUG_INFO_COMPRESSED >> +DEBUG_CFLAGS += -gz=zlib >> +KBUILD_LDFLAGS += --compress-debug-sections=zlib >> +endif >> endif >> >> KBUILD_CFLAGS += $(DEBUG_CFLAGS) >> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug >> index f6f9a039f736..1f4a47ba6c1b 100644 >> --- a/lib/Kconfig.debug >> +++ b/lib/Kconfig.debug >> @@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED >> DEBUG_INFO build and compile times are reduced too. >> Only works with newer gcc versions. >> >> +config DEBUG_INFO_COMPRESSED >> + bool "Compressed debugging information" >> + depends on DEBUG_INFO >> + depends on $(cc-option,-gz=zlib) >> + depends on $(ld-option,--compress-debug-sections=zlib) >> + help >> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang >> + 5.0+. >> + linker option --compress-debug-sections=zlib require binutils >= 2.26 (ld-option) Assembler option --compress-debug-sections=zlib also require binutils >= 2.26 (cc-option,-gz=zlib) >> config DEBUG_INFO_SPLIT >> bool "Produce split debuginfo in .dwo files" >> depends on DEBUG_INFO >> -- >> 2.17.1 >> >> -- >> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200504031340.7103-1-nick.desaulniers%40gmail.com. > >-- >You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. >To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. >To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CA%2BicZUUOaqeKeh6n4BJq2k6XQWAfNghUj57j42ZX5qyd3iOmLw%40mail.gmail.com.
Hi Sedat, On Tue, May 5, 2020 at 1:25 AM Sedat Dilek <sedat.dilek@gmail.com> wrote: > > On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > <nick.desaulniers@gmail.com> wrote: > > > > As debug information gets larger and larger, it helps significantly save > > the size of vmlinux images to compress the information in the debug > > information sections. Note: this debug info is typically split off from > > the final compressed kernel image, which is why vmlinux is what's used > > in conjunction with GDB. Minimizing the debug info size should have no > > impact on boot times, or final compressed kernel image size. > > > > All of the debug sections will have a `C` flag set. > > $ readelf -S <object file> > > > > $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ > > vmlinux.gcc75.uncompressed.dwarf4 > > > > FILE SIZE VM SIZE > > -------------- -------------- > > +0.0% +18 [ = ] 0 [Unmapped] > > -73.3% -114Ki [ = ] 0 .debug_aranges > > -76.2% -2.01Mi [ = ] 0 .debug_frame > > -73.6% -2.89Mi [ = ] 0 .debug_str > > -80.7% -4.66Mi [ = ] 0 .debug_abbrev > > -82.9% -4.88Mi [ = ] 0 .debug_ranges > > -70.5% -9.04Mi [ = ] 0 .debug_line > > -79.3% -10.9Mi [ = ] 0 .debug_loc > > -39.5% -88.6Mi [ = ] 0 .debug_info > > -18.2% -123Mi [ = ] 0 TOTAL > > > > $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ > > vmlinux.clang11.uncompressed.dwarf4 > > > > FILE SIZE VM SIZE > > -------------- -------------- > > +0.0% +23 [ = ] 0 [Unmapped] > > -65.6% -871 [ = ] 0 .debug_aranges > > -77.4% -1.84Mi [ = ] 0 .debug_frame > > -82.9% -2.33Mi [ = ] 0 .debug_abbrev > > -73.1% -2.43Mi [ = ] 0 .debug_str > > -84.8% -3.07Mi [ = ] 0 .debug_ranges > > -65.9% -8.62Mi [ = ] 0 .debug_line > > -86.2% -40.0Mi [ = ] 0 .debug_loc > > -42.0% -64.1Mi [ = ] 0 .debug_info > > -22.1% -122Mi [ = ] 0 TOTAL > > > > Hi Nick, > > thanks for the patch. > > I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?). > > Which linker did you use and has it an impact if you switch from > ld.bfd to ld.lld? > > I tried a first normal run and in a 2nd one with > CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10. > > My numbers (sizes in MiB): > > [ diffconfig ] > > $ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang > /boot/config-5.7.0-rc4-2-amd64-clang > BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" > +DEBUG_INFO_COMPRESSED y > > [ compiler and linker ] > > $ clang-10 -v > ClangBuiltLinux clang version 10.0.1 > (https://github.com/llvm/llvm-project > 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) > Target: x86_64-unknown-linux-gnu > Thread model: posix > InstalledDir: /home/dileks/src/llvm-toolchain/install/bin > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 > Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 > Candidate multilib: .;@m64 > Candidate multilib: 32;@m32 > Candidate multilib: x32;@mx32 > Selected multilib: .;@m64 > > $ ld.lld-10 -v > LLD 10.0.1 (https://github.com/llvm/llvm-project > 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU > linkers) > > [ sizes vmlinux ] > > $ du -m 5.7.0-rc4-*/vmlinux* > 409 5.7.0-rc4-1-amd64-clang/vmlinux > 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed > 404 5.7.0-rc4-1-amd64-clang/vmlinux.o > 324 5.7.0-rc4-2-amd64-clang/vmlinux > 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed > 299 5.7.0-rc4-2-amd64-clang/vmlinux.o > > [ readelf (.debug_info as example) ] > > $ readelf -S vmlinux.o > [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 > 0000000006be1ee6 0000000000000000 0 0 1 > > $ readelf -S vmlinux.o > [33] .debug_info PROGBITS 0000000000000000 01749f18 > 0000000002ef04d2 0000000000000000 C 0 0 1 <--- > XXX: "C (compressed)" Flag > > Key to Flags: > W (write), A (alloc), X (execute), M (merge), S (strings), I (info), > L (link order), O (extra OS processing required), G (group), T (TLS), > C (compressed), x (unknown), o (OS specific), E (exclude), > l (large), p (processor specific) > > [ sizes linux-image debian packages ] > > $ du -m 5.7.0-rc4-*/linux-image*.deb > 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > [ sizes linux-git dir (compilation finished ] > > 5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux > 5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux > > [ xz compressed linux-image-dbg packages ] > > $ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: > Debian binary package (format 2.0), with control.tar.xz, data > compression xz > $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: > Debian binary package (format 2.0), with control.tar.xz, data > compression xz > > [ file-lists ] > > $ dpkg --contents > linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > | wc -l > 4395 > $ dpkg --contents > linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > | wc -l > 4395 > > [ file-lists vmlinux ] > > $ dpkg --contents > linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > | grep vmlinux > -rwxr-xr-x root/root 428588312 2020-05-04 06:15 > ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > lrwxrwxrwx root/root 0 2020-05-04 06:15 > ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> > ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > lrwxrwxrwx root/root 0 2020-05-04 06:15 > ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> > lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > > $ dpkg --contents > linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > | grep vmlinux > -rwxr-xr-x root/root 339341456 2020-05-04 12:24 > ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > lrwxrwxrwx root/root 0 2020-05-04 12:24 > ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> > ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > lrwxrwxrwx root/root 0 2020-05-04 12:24 > ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> > lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > > [ conclusion ] > > As you can see there is a size-reduction in case of vmlinux/vmlinux.o > (debug) files... > ...and my linux-git directory in total is smaller: 17963M vs. 14328M. > > But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. > XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in > a fatter linux-image-dbg package. I also confirmed that, but this would not be a blocker of this patch. Users can disable CONFIG_DEBUG_INFO_COMPRESSED if they care about the debug package size.
On Tue, May 5, 2020 at 9:47 AM Fangrui Song <maskray@google.com> wrote: > > > On 2020-05-04, Sedat Dilek wrote: > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > ><nick.desaulniers@gmail.com> wrote: > >> > >> As debug information gets larger and larger, it helps significantly save > >> the size of vmlinux images to compress the information in the debug > >> information sections. Note: this debug info is typically split off from > >> the final compressed kernel image, which is why vmlinux is what's used > >> in conjunction with GDB. Minimizing the debug info size should have no > >> impact on boot times, or final compressed kernel image size. > >> > >> All of the debug sections will have a `C` flag set. > >> $ readelf -S <object file> > >> > >> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ > >> vmlinux.gcc75.uncompressed.dwarf4 > >> > >> FILE SIZE VM SIZE > >> -------------- -------------- > >> +0.0% +18 [ = ] 0 [Unmapped] > >> -73.3% -114Ki [ = ] 0 .debug_aranges > >> -76.2% -2.01Mi [ = ] 0 .debug_frame > >> -73.6% -2.89Mi [ = ] 0 .debug_str > >> -80.7% -4.66Mi [ = ] 0 .debug_abbrev > >> -82.9% -4.88Mi [ = ] 0 .debug_ranges > >> -70.5% -9.04Mi [ = ] 0 .debug_line > >> -79.3% -10.9Mi [ = ] 0 .debug_loc > >> -39.5% -88.6Mi [ = ] 0 .debug_info > >> -18.2% -123Mi [ = ] 0 TOTAL > >> > >> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ > >> vmlinux.clang11.uncompressed.dwarf4 > >> > >> FILE SIZE VM SIZE > >> -------------- -------------- > >> +0.0% +23 [ = ] 0 [Unmapped] > >> -65.6% -871 [ = ] 0 .debug_aranges > >> -77.4% -1.84Mi [ = ] 0 .debug_frame > >> -82.9% -2.33Mi [ = ] 0 .debug_abbrev > >> -73.1% -2.43Mi [ = ] 0 .debug_str > >> -84.8% -3.07Mi [ = ] 0 .debug_ranges > >> -65.9% -8.62Mi [ = ] 0 .debug_line > >> -86.2% -40.0Mi [ = ] 0 .debug_loc > >> -42.0% -64.1Mi [ = ] 0 .debug_info > >> -22.1% -122Mi [ = ] 0 TOTAL > >> > > > >Hi Nick, > > > >thanks for the patch. > > > >I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?). > > > >Which linker did you use and has it an impact if you switch from > >ld.bfd to ld.lld? > > lld has supported the linker option --compress-debug-sections=zlib since > about 5.0.0 (https://reviews.llvm.org/D31941) > > >I tried a first normal run and in a 2nd one with > >CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10. > > > >My numbers (sizes in MiB): > > > >[ diffconfig ] > > > >$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang > >/boot/config-5.7.0-rc4-2-amd64-clang > > BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" > >+DEBUG_INFO_COMPRESSED y > > > >[ compiler and linker ] > > > >$ clang-10 -v > >ClangBuiltLinux clang version 10.0.1 > >(https://github.com/llvm/llvm-project > >92d5c1be9ee93850c0a8903f05f36a23ee835dc2) > >Target: x86_64-unknown-linux-gnu > >Thread model: posix > >InstalledDir: /home/dileks/src/llvm-toolchain/install/bin > >Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 > >Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 > >Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 > >Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 > >Candidate multilib: .;@m64 > >Candidate multilib: 32;@m32 > >Candidate multilib: x32;@mx32 > >Selected multilib: .;@m64 > > > >$ ld.lld-10 -v > >LLD 10.0.1 (https://github.com/llvm/llvm-project > >92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU > >linkers) > > > >[ sizes vmlinux ] > > > >$ du -m 5.7.0-rc4-*/vmlinux* > >409 5.7.0-rc4-1-amd64-clang/vmlinux > >7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed > >404 5.7.0-rc4-1-amd64-clang/vmlinux.o > >324 5.7.0-rc4-2-amd64-clang/vmlinux > >7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed > >299 5.7.0-rc4-2-amd64-clang/vmlinux.o > > > >[ readelf (.debug_info as example) ] > > > >$ readelf -S vmlinux.o > > [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 > > 0000000006be1ee6 0000000000000000 0 0 1 > > > >$ readelf -S vmlinux.o > > [33] .debug_info PROGBITS 0000000000000000 01749f18 > > 0000000002ef04d2 0000000000000000 C 0 0 1 <--- > >XXX: "C (compressed)" Flag > > > >Key to Flags: > > W (write), A (alloc), X (execute), M (merge), S (strings), I (info), > > L (link order), O (extra OS processing required), G (group), T (TLS), > > C (compressed), x (unknown), o (OS specific), E (exclude), > > l (large), p (processor specific) > > > >[ sizes linux-image debian packages ] > > > >$ du -m 5.7.0-rc4-*/linux-image*.deb > >47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > >424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > >47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > >771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > > >[ sizes linux-git dir (compilation finished ] > > > >5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux > >5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux > > > >[ xz compressed linux-image-dbg packages ] > > > >$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > >linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: > >Debian binary package (format 2.0), with control.tar.xz, data > >compression xz > >$ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > >linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: > >Debian binary package (format 2.0), with control.tar.xz, data > >compression xz > > > >[ file-lists ] > > > >$ dpkg --contents > >linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > >| wc -l > >4395 > >$ dpkg --contents > >linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > >| wc -l > >4395 > > > >[ file-lists vmlinux ] > > > >$ dpkg --contents > >linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > >| grep vmlinux > >-rwxr-xr-x root/root 428588312 2020-05-04 06:15 > >./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > >lrwxrwxrwx root/root 0 2020-05-04 06:15 > >./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> > >../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > >lrwxrwxrwx root/root 0 2020-05-04 06:15 > >./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> > >lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > > > >$ dpkg --contents > >linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > >| grep vmlinux > >-rwxr-xr-x root/root 339341456 2020-05-04 12:24 > >./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > >lrwxrwxrwx root/root 0 2020-05-04 12:24 > >./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> > >../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > >lrwxrwxrwx root/root 0 2020-05-04 12:24 > >./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> > >lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > > > >[ conclusion ] > > > >As you can see there is a size-reduction in case of vmlinux/vmlinux.o > >(debug) files... > >...and my linux-git directory in total is smaller: 17963M vs. 14328M. > > > >But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. > >XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in > >a fatter linux-image-dbg package. > > > >For a usage of vmlinux (debug) in a VM scenario this might be nice but > >seen from a debian repository perspective not. > > > >For the sake of completeness: I have just installed and booted the > >"normal" linux-image debian package - not the debug packages. > > > >Thanks. > > > >Regards, > >- Sedat - > > > >> Suggested-by: David Blaikie <blakie@google.com> > >> Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com> > >> --- > >> Makefile | 5 +++++ > >> lib/Kconfig.debug | 9 +++++++++ > >> 2 files changed, 14 insertions(+) > >> > >> diff --git a/Makefile b/Makefile > >> index 981eb902384b..313a054e5dc6 100644 > >> --- a/Makefile > >> +++ b/Makefile > >> @@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED > >> DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ > >> $(call cc-option,-fno-var-tracking) > >> endif > >> + > >> +ifdef CONFIG_DEBUG_INFO_COMPRESSED > >> +DEBUG_CFLAGS += -gz=zlib > >> +KBUILD_LDFLAGS += --compress-debug-sections=zlib > >> +endif > >> endif > >> > >> KBUILD_CFLAGS += $(DEBUG_CFLAGS) > >> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > >> index f6f9a039f736..1f4a47ba6c1b 100644 > >> --- a/lib/Kconfig.debug > >> +++ b/lib/Kconfig.debug > >> @@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED > >> DEBUG_INFO build and compile times are reduced too. > >> Only works with newer gcc versions. > >> > >> +config DEBUG_INFO_COMPRESSED > >> + bool "Compressed debugging information" > >> + depends on DEBUG_INFO > >> + depends on $(cc-option,-gz=zlib) > >> + depends on $(ld-option,--compress-debug-sections=zlib) > >> + help > >> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang > >> + 5.0+. > >> + > > linker option --compress-debug-sections=zlib require binutils >= 2.26 (ld-option) > Assembler option --compress-debug-sections=zlib also require binutils >= 2.26 (cc-option,-gz=zlib) Nick, I am OK with this patch. Fangrui provided the minimal requirement for --compress-debug-sections=zlib Is it worth recording in the help text? Do you want to send v2?
On Tue, May 12, 2020 at 7:47 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > Hi Sedat, > > > On Tue, May 5, 2020 at 1:25 AM Sedat Dilek <sedat.dilek@gmail.com> wrote: > > > > On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > > <nick.desaulniers@gmail.com> wrote: > > > > > > As debug information gets larger and larger, it helps significantly save > > > the size of vmlinux images to compress the information in the debug > > > information sections. Note: this debug info is typically split off from > > > the final compressed kernel image, which is why vmlinux is what's used > > > in conjunction with GDB. Minimizing the debug info size should have no > > > impact on boot times, or final compressed kernel image size. > > > > > > All of the debug sections will have a `C` flag set. > > > $ readelf -S <object file> > > > > > > $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ > > > vmlinux.gcc75.uncompressed.dwarf4 > > > > > > FILE SIZE VM SIZE > > > -------------- -------------- > > > +0.0% +18 [ = ] 0 [Unmapped] > > > -73.3% -114Ki [ = ] 0 .debug_aranges > > > -76.2% -2.01Mi [ = ] 0 .debug_frame > > > -73.6% -2.89Mi [ = ] 0 .debug_str > > > -80.7% -4.66Mi [ = ] 0 .debug_abbrev > > > -82.9% -4.88Mi [ = ] 0 .debug_ranges > > > -70.5% -9.04Mi [ = ] 0 .debug_line > > > -79.3% -10.9Mi [ = ] 0 .debug_loc > > > -39.5% -88.6Mi [ = ] 0 .debug_info > > > -18.2% -123Mi [ = ] 0 TOTAL > > > > > > $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ > > > vmlinux.clang11.uncompressed.dwarf4 > > > > > > FILE SIZE VM SIZE > > > -------------- -------------- > > > +0.0% +23 [ = ] 0 [Unmapped] > > > -65.6% -871 [ = ] 0 .debug_aranges > > > -77.4% -1.84Mi [ = ] 0 .debug_frame > > > -82.9% -2.33Mi [ = ] 0 .debug_abbrev > > > -73.1% -2.43Mi [ = ] 0 .debug_str > > > -84.8% -3.07Mi [ = ] 0 .debug_ranges > > > -65.9% -8.62Mi [ = ] 0 .debug_line > > > -86.2% -40.0Mi [ = ] 0 .debug_loc > > > -42.0% -64.1Mi [ = ] 0 .debug_info > > > -22.1% -122Mi [ = ] 0 TOTAL > > > > > > > Hi Nick, > > > > thanks for the patch. > > > > I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?). > > > > Which linker did you use and has it an impact if you switch from > > ld.bfd to ld.lld? > > > > I tried a first normal run and in a 2nd one with > > CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10. > > > > My numbers (sizes in MiB): > > > > [ diffconfig ] > > > > $ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang > > /boot/config-5.7.0-rc4-2-amd64-clang > > BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" > > +DEBUG_INFO_COMPRESSED y > > > > [ compiler and linker ] > > > > $ clang-10 -v > > ClangBuiltLinux clang version 10.0.1 > > (https://github.com/llvm/llvm-project > > 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) > > Target: x86_64-unknown-linux-gnu > > Thread model: posix > > InstalledDir: /home/dileks/src/llvm-toolchain/install/bin > > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 > > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 > > Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 > > Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 > > Candidate multilib: .;@m64 > > Candidate multilib: 32;@m32 > > Candidate multilib: x32;@mx32 > > Selected multilib: .;@m64 > > > > $ ld.lld-10 -v > > LLD 10.0.1 (https://github.com/llvm/llvm-project > > 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU > > linkers) > > > > [ sizes vmlinux ] > > > > $ du -m 5.7.0-rc4-*/vmlinux* > > 409 5.7.0-rc4-1-amd64-clang/vmlinux > > 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed > > 404 5.7.0-rc4-1-amd64-clang/vmlinux.o > > 324 5.7.0-rc4-2-amd64-clang/vmlinux > > 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed > > 299 5.7.0-rc4-2-amd64-clang/vmlinux.o > > > > [ readelf (.debug_info as example) ] > > > > $ readelf -S vmlinux.o > > [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 > > 0000000006be1ee6 0000000000000000 0 0 1 > > > > $ readelf -S vmlinux.o > > [33] .debug_info PROGBITS 0000000000000000 01749f18 > > 0000000002ef04d2 0000000000000000 C 0 0 1 <--- > > XXX: "C (compressed)" Flag > > > > Key to Flags: > > W (write), A (alloc), X (execute), M (merge), S (strings), I (info), > > L (link order), O (extra OS processing required), G (group), T (TLS), > > C (compressed), x (unknown), o (OS specific), E (exclude), > > l (large), p (processor specific) > > > > [ sizes linux-image debian packages ] > > > > $ du -m 5.7.0-rc4-*/linux-image*.deb > > 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > > 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > > 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > > > [ sizes linux-git dir (compilation finished ] > > > > 5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux > > 5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux > > > > [ xz compressed linux-image-dbg packages ] > > > > $ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > > linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: > > Debian binary package (format 2.0), with control.tar.xz, data > > compression xz > > $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: > > Debian binary package (format 2.0), with control.tar.xz, data > > compression xz > > > > [ file-lists ] > > > > $ dpkg --contents > > linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > > | wc -l > > 4395 > > $ dpkg --contents > > linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > | wc -l > > 4395 > > > > [ file-lists vmlinux ] > > > > $ dpkg --contents > > linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb > > | grep vmlinux > > -rwxr-xr-x root/root 428588312 2020-05-04 06:15 > > ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > > lrwxrwxrwx root/root 0 2020-05-04 06:15 > > ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> > > ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > > lrwxrwxrwx root/root 0 2020-05-04 06:15 > > ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> > > lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux > > > > $ dpkg --contents > > linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb > > | grep vmlinux > > -rwxr-xr-x root/root 339341456 2020-05-04 12:24 > > ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > > lrwxrwxrwx root/root 0 2020-05-04 12:24 > > ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> > > ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > > lrwxrwxrwx root/root 0 2020-05-04 12:24 > > ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> > > lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux > > > > [ conclusion ] > > > > As you can see there is a size-reduction in case of vmlinux/vmlinux.o > > (debug) files... > > ...and my linux-git directory in total is smaller: 17963M vs. 14328M. > > > > But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. > > XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in > > a fatter linux-image-dbg package. > > > I also confirmed that, but this would not > be a blocker of this patch. > Hi Masahiro, No, it is not a blocker. I have this patch now in my Linux v5.7-rc5 series. I see a lot of more benefits concerning disc-usage - in my linux-git and a reduced vmlinux file when I wanted to test with QEMU. Feel free to add: Tested-by: Sedat Dilek <sedat.dilek@gmail.com> - Sedat - > Users can disable CONFIG_DEBUG_INFO_COMPRESSED > if they care about the debug package size. > > > > > > -- > Best Regards > Masahiro Yamada > > -- > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAK7LNAR%2Bpm-_nd5%3DB2OeLpimW42FXxm8TQUMru9DR_asT3qYnA%40mail.gmail.com.
On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > > ><nick.desaulniers@gmail.com> wrote: > > >> > > >> As debug information gets larger and larger, it helps significantly save > > >> the size of vmlinux images to compress the information in the debug > > >> information sections. Note: this debug info is typically split off from > > >> the final compressed kernel image, which is why vmlinux is what's used > > >> in conjunction with GDB. Minimizing the debug info size should have no > > >> impact on boot times, or final compressed kernel image size. > > >> > Nick, > > I am OK with this patch. > > Fangrui provided the minimal requirement for > --compress-debug-sections=zlib > > > Is it worth recording in the help text? > Do you want to send v2? Yes I'd like to record that information. I can also record Sedat's Tested-by tag. Thank you for testing Sedat. I don't know what "linux-image-dbg file" are, or why they would be bigger. The size of the debug info is the primary concern with this config. It sounds like however that file is created might be problematic. Fangrui, I wasn't able to easily find what version of binutils first added support. Can you please teach me how to fish? Another question I had for Fangrui is, if the linker can compress these sections, shouldn't we just have the linker do it, not the the compiler and assembler? IIUC the debug info can contain relocations, so the linker would have to decompress these, perform relocations, then recompress these? I guess having the compiler and assembler compress the debug info as well would minimize the size of the .o files on disk. Otherwise I should add this flag to the assembler invocation, too, in v2. Thoughts? I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
On 2020-05-12, Nick Desaulniers wrote: >On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote: >> >> > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers >> > ><nick.desaulniers@gmail.com> wrote: >> > >> >> > >> As debug information gets larger and larger, it helps significantly save >> > >> the size of vmlinux images to compress the information in the debug >> > >> information sections. Note: this debug info is typically split off from >> > >> the final compressed kernel image, which is why vmlinux is what's used >> > >> in conjunction with GDB. Minimizing the debug info size should have no >> > >> impact on boot times, or final compressed kernel image size. >> > >> >> Nick, >> >> I am OK with this patch. >> >> Fangrui provided the minimal requirement for >> --compress-debug-sections=zlib >> >> >> Is it worth recording in the help text? >> Do you want to send v2? > >Yes I'd like to record that information. I can also record Sedat's >Tested-by tag. Thank you for testing Sedat. > >I don't know what "linux-image-dbg file" are, or why they would be >bigger. The size of the debug info is the primary concern with this >config. It sounds like however that file is created might be >problematic. > >Fangrui, I wasn't able to easily find what version of binutils first >added support. Can you please teach me how to fish? I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and located the sources... I think an easier way is: % cd binutils-gdb % git show binutils-2_26:./gas/as.c | grep compress-debug-sections --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ ... GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in newer versions. Similarly, for GNU ld: % git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ (I have spent a lot of time investigating GNU ld's behavior :) >Another question I had for Fangrui is, if the linker can compress >these sections, shouldn't we just have the linker do it, not the the >compiler and assembler? IIUC the debug info can contain relocations, >so the linker would have to decompress these, perform relocations, >then recompress these? I guess having the compiler and assembler >compress the debug info as well would minimize the size of the .o >files on disk. The linker will decompress debug info unconditionally. Because input .debug_info sections need to be concatenated to form the output .debug_info . Whether the output .debug_info is compressed is controlled by the linker option --compress-debug-sections=zlib, which is not affected by the compression state of object files. Both GNU as and GNU ld name the option --compress-debug-sections=zlib. In a compiler driver context, an unfamiliar user may find -Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib confusing:/ >Otherwise I should add this flag to the assembler invocation, too, in >v2. Thoughts? Compressing object files along with the linked output should be fine. It can save disk space. (It'd be great if you paste the comparison with and w/o object files compressed) Feel free to add: Reviewed-by: Fangrui Song <maskray@google.com> >I have a patch series that enables dwarf5 support in the kernel that >I'm working up to. I wanted to send this first. Both roughly reduce >the debug info size by 20% each, though I haven't measured them >together, yet. Requires ToT binutils because there have been many >fixes from reports of mine recently. This will be awesome! I also heard that enabling DWARF v5 for our object files can easily make debug info size smaller by 20%. Glad that the kernel can benefit it as well:)
On Tue, May 12, 2020 at 9:23 PM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > > > ><nick.desaulniers@gmail.com> wrote: > > > >> > > > >> As debug information gets larger and larger, it helps significantly save > > > >> the size of vmlinux images to compress the information in the debug > > > >> information sections. Note: this debug info is typically split off from > > > >> the final compressed kernel image, which is why vmlinux is what's used > > > >> in conjunction with GDB. Minimizing the debug info size should have no > > > >> impact on boot times, or final compressed kernel image size. > > > >> > > Nick, > > > > I am OK with this patch. > > > > Fangrui provided the minimal requirement for > > --compress-debug-sections=zlib > > > > > > Is it worth recording in the help text? > > Do you want to send v2? > > Yes I'd like to record that information. I can also record Sedat's > Tested-by tag. Thank you for testing Sedat. > > I don't know what "linux-image-dbg file" are, or why they would be > bigger. The size of the debug info is the primary concern with this > config. It sounds like however that file is created might be > problematic. > Hi Nick, sorry, I try to explain the magic of "linux-image-dbg file". In my workflow, I use the "scripts/package/{builddeb,mkdebian}" shipped with the Linux-kernel as a base to build my Debian packages. With enabled debugging (CONFIG_DEBUG_INFO=y) a "linux-image-$KERNELRELEASE-dbg" Debian package is created. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/package/mkdebian#n203 As you can see below... [ CONFIG_DEBUG_INFO_COMPRESSED=n ] 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb [ CONFIG_DEBUG_INFO_COMPRESSED=y ] 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb ...there is minimal change in the size for the Debian package w/o debug-infos - approx. 47M. As said 424M vs. 771M for the dbg packages. There is another big benefit checking my recorded stats: $ grep 'cache size' stats/5.7.0-rc4-*/ccache-s.txt stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:cache size 4.7 GB stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:max cache size 10.0 GB stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:cache size 3.4 GB stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:max cache size 10.0 GB So the cache of ccache is reduced: 4.7 GB vs. 3.4 GB If you have any questions, "Don't ask to ask - just ask." :-). Thanks. Regards, - Sedat -
On Tue, May 12, 2020 at 10:01 PM Fangrui Song <maskray@google.com> wrote: > > On 2020-05-12, Nick Desaulniers wrote: ... > >I have a patch series that enables dwarf5 support in the kernel that > >I'm working up to. I wanted to send this first. Both roughly reduce > >the debug info size by 20% each, though I haven't measured them > >together, yet. Requires ToT binutils because there have been many > >fixes from reports of mine recently. > > This will be awesome! I also heard that enabling DWARF v5 for our object > files can easily make debug info size smaller by 20%. Glad that the > kernel can benefit it as well:) Please CC me on your pathcset - 20% smaller - I flip out. - Sedat -
Nick, On Wed, May 13, 2020 at 4:23 AM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > > > ><nick.desaulniers@gmail.com> wrote: > > > >> > > > >> As debug information gets larger and larger, it helps significantly save > > > >> the size of vmlinux images to compress the information in the debug > > > >> information sections. Note: this debug info is typically split off from > > > >> the final compressed kernel image, which is why vmlinux is what's used > > > >> in conjunction with GDB. Minimizing the debug info size should have no > > > >> impact on boot times, or final compressed kernel image size. > > > >> > > Nick, > > > > I am OK with this patch. > > > > Fangrui provided the minimal requirement for > > --compress-debug-sections=zlib > > > > > > Is it worth recording in the help text? > > Do you want to send v2? > > Yes I'd like to record that information. I can also record Sedat's > Tested-by tag. Thank you for testing Sedat. > > I don't know what "linux-image-dbg file" are, or why they would be > bigger. The size of the debug info is the primary concern with this > config. It sounds like however that file is created might be > problematic. As Sedat explained, deb package data is compressed by xz, which is default. You can use another compression method, or disable compression if you desire. "man dpkg-deb" says as follows: -Zcompress-type Specify which compression type to use when building a package. Allowed values are gzip, xz (since dpkg 1.15.6), and none (default is xz). Kbuild supports KDEB_COMPRESS variable to change the compression method. See line 46 of scripts/package/builddeb. If you are interested, try "make bindeb-pkg" with/without CONFIG_DEBUG_INFO_COMPRESSED, and compare the size of the generated debug package. As Sedat stated, (plain data) -> compress by gzip -> compress by xz is often less efficient than (plain data) -> compress by xz I hope this is clearer. > Fangrui, I wasn't able to easily find what version of binutils first > added support. Can you please teach me how to fish? > > Another question I had for Fangrui is, if the linker can compress > these sections, shouldn't we just have the linker do it, not the the > compiler and assembler? IIUC the debug info can contain relocations, > so the linker would have to decompress these, perform relocations, > then recompress these? I guess having the compiler and assembler > compress the debug info as well would minimize the size of the .o > files on disk. > > Otherwise I should add this flag to the assembler invocation, too, in > v2. Thoughts? > > I have a patch series that enables dwarf5 support in the kernel that > I'm working up to. I wanted to send this first. Both roughly reduce > the debug info size by 20% each, though I haven't measured them > together, yet. Requires ToT binutils because there have been many > fixes from reports of mine recently. > -- > Thanks, > ~Nick Desaulniers
On Wed, May 13, 2020 at 4:52 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > Nick, > > On Wed, May 13, 2020 at 4:23 AM Nick Desaulniers > <ndesaulniers@google.com> wrote: > > > > On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers > > > > ><nick.desaulniers@gmail.com> wrote: > > > > >> > > > > >> As debug information gets larger and larger, it helps significantly save > > > > >> the size of vmlinux images to compress the information in the debug > > > > >> information sections. Note: this debug info is typically split off from > > > > >> the final compressed kernel image, which is why vmlinux is what's used > > > > >> in conjunction with GDB. Minimizing the debug info size should have no > > > > >> impact on boot times, or final compressed kernel image size. > > > > >> > > > Nick, > > > > > > I am OK with this patch. > > > > > > Fangrui provided the minimal requirement for > > > --compress-debug-sections=zlib > > > > > > > > > Is it worth recording in the help text? > > > Do you want to send v2? > > > > Yes I'd like to record that information. I can also record Sedat's > > Tested-by tag. Thank you for testing Sedat. > > > > I don't know what "linux-image-dbg file" are, or why they would be > > bigger. The size of the debug info is the primary concern with this > > config. It sounds like however that file is created might be > > problematic. > > > > As Sedat explained, deb package data > is compressed by xz, which is default. > > You can use another compression method, > or disable compression if you desire. > > > > "man dpkg-deb" says as follows: > > -Zcompress-type > Specify which compression type to use when building a package. > Allowed values are gzip, xz (since dpkg 1.15.6), and none > (default is xz). > > > > Kbuild supports KDEB_COMPRESS variable > to change the compression method. > See line 46 of scripts/package/builddeb. > Hi Masahiro, thanks for the clarification and the informations. With next RC I will add KDEB_COMPRESS=gzip to my build-script and report. Regards, - Sedat - > > If you are interested, > try "make bindeb-pkg" with/without CONFIG_DEBUG_INFO_COMPRESSED, > and compare the size of the generated debug package. > > > > > As Sedat stated, > > (plain data) -> compress by gzip -> compress by xz > > is often less efficient than > > (plain data) -> compress by xz > > > > I hope this is clearer. > > > > > > > > > > Fangrui, I wasn't able to easily find what version of binutils first > > added support. Can you please teach me how to fish? > > > > Another question I had for Fangrui is, if the linker can compress > > these sections, shouldn't we just have the linker do it, not the the > > compiler and assembler? IIUC the debug info can contain relocations, > > so the linker would have to decompress these, perform relocations, > > then recompress these? I guess having the compiler and assembler > > compress the debug info as well would minimize the size of the .o > > files on disk. > > > > Otherwise I should add this flag to the assembler invocation, too, in > > v2. Thoughts? > > > > I have a patch series that enables dwarf5 support in the kernel that > > I'm working up to. I wanted to send this first. Both roughly reduce > > the debug info size by 20% each, though I haven't measured them > > together, yet. Requires ToT binutils because there have been many > > fixes from reports of mine recently. > > -- > > Thanks, > > ~Nick Desaulniers > > > > -- > Best Regards > Masahiro Yamada
On Tue, May 12, 2020 at 1:01 PM Fangrui Song <maskray@google.com> wrote: > > >Fangrui, I wasn't able to easily find what version of binutils first > >added support. Can you please teach me how to fish? > > I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and > located the sources... I think an easier way is: > > % cd binutils-gdb > % git show binutils-2_26:./gas/as.c | grep compress-debug-sections This assumes you knew to look at the binutils-2_26 tag, which is putting the cart before the horse. ;) I guess: $ git log gas/as.c /compress-debug-sections commit 19a7fe52ae3d ("Make default compression gABI compliant") looks related $ git describe --contains "19a7fe52ae3d" | sed 's/~.*//' users/hjl/linux/release/2.25.51.0.4 so it landed in 2.25.51.0.4. + Nick, H.J. I'm unfamiliar with the git tag conventions of binutils. Does a patch that landed in 2.25.51.0.4 mean it shipped in the official 2.25 release, or 2.26 release? Specifically, commit 19a7fe52ae3d. > --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ > ... > > GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in > newer versions. > > Similarly, for GNU ld: > > % git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections > --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ > > (I have spent a lot of time investigating GNU ld's behavior :) > > >Another question I had for Fangrui is, if the linker can compress > >these sections, shouldn't we just have the linker do it, not the the > >compiler and assembler? IIUC the debug info can contain relocations, > >so the linker would have to decompress these, perform relocations, > >then recompress these? I guess having the compiler and assembler > >compress the debug info as well would minimize the size of the .o > >files on disk. > > The linker will decompress debug info unconditionally. Because > input .debug_info sections need to be concatenated to form the output > .debug_info . Whether the output .debug_info is compressed is controlled > by the linker option --compress-debug-sections=zlib, which is not > affected by the compression state of object files. > > Both GNU as and GNU ld name the option --compress-debug-sections=zlib. > In a compiler driver context, an unfamiliar user may find > -Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib > confusing:/ The kernel uses the compiler as the driver for out of line assembly, as they are all preprocessed first. Most out of line assembly in the kernel uses the C preprocessor to #include headers that share #defines of common constants shared between C and asm. #ifdef __ASSEMBLY__ is used frequently in these headers. But for the linker, the linker itself is invoked as the driver, though there are a few inconsistencies we've cleaned up or still have to. > > >Otherwise I should add this flag to the assembler invocation, too, in > >v2. Thoughts? > > Compressing object files along with the linked output should be fine. It > can save disk space. (It'd be great if you paste the comparison > with and w/o object files compressed) > > Feel free to add: > > Reviewed-by: Fangrui Song <maskray@google.com> Thanks, will add that to v2. > > >I have a patch series that enables dwarf5 support in the kernel that > >I'm working up to. I wanted to send this first. Both roughly reduce > >the debug info size by 20% each, though I haven't measured them > >together, yet. Requires ToT binutils because there have been many > >fixes from reports of mine recently. > > This will be awesome! I also heard that enabling DWARF v5 for our object > files can easily make debug info size smaller by 20%. Glad that the > kernel can benefit it as well:)
Hi Nick, > + Nick, H.J. > I'm unfamiliar with the git tag conventions of binutils. Does a patch > that landed in 2.25.51.0.4 mean it shipped in the official 2.25 > release, or 2.26 release? Specifically, commit 19a7fe52ae3d. 2.26. The convention is that a released form of the binutils has a version number of X.XX or possible X.XX.N. The current mainline development sources have a version of X.XX.50 where X.XX is the latest release. (So the current mainline sources are version 2.34.50). When a release happens the XX value is incremented by one as part of the release process, and the .50 is dropped. (So the next binutils release will be 2.35). So 2.25.51.0.4 is a development version which will then have been released as binutils 2.26. Cheers Nick
diff --git a/Makefile b/Makefile index 981eb902384b..313a054e5dc6 100644 --- a/Makefile +++ b/Makefile @@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif + +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif endif KBUILD_CFLAGS += $(DEBUG_CFLAGS) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index f6f9a039f736..1f4a47ba6c1b 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size. All of the debug sections will have a `C` flag set. $ readelf -S <object file> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4 FILE SIZE VM SIZE -------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4 FILE SIZE VM SIZE -------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL Suggested-by: David Blaikie <blakie@google.com> Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com> --- Makefile | 5 +++++ lib/Kconfig.debug | 9 +++++++++ 2 files changed, 14 insertions(+)