| Message ID | 20200428221419.2530697-5-natechancellor@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Allow ld.lld to link the MIPS VDSO | expand |
On 2020-04-28, Nathan Chancellor wrote: >Currently, the VDSO is being linked through $(CC). This does not match >how the rest of the kernel links objects, which is through the $(LD) >variable. > >When clang is built in a default configuration, it first attempts to use >the target triple's default linker then the system's default linker, >unless told otherwise through -fuse-ld=... We do not use -fuse-ld= >because it can be brittle and we have support for invoking $(LD) >directly. See commit fe00e50b2db8c ("ARM: 8858/1: vdso: use $(LD) >instead of $(CC) to link VDSO") and commit 691efbedc60d2 ("arm64: vdso: >use $(LD) instead of $(CC) to link VDSO") for examples of doing this in >the VDSO. > >Do the same thing here. Replace the custom linking logic with $(cmd_ld) >and ldflags-y so that $(LD) is respected. We need to explicitly add two >flags to the linker that were implicitly passed by the compiler: >-G 0 (which comes from ccflags-vdso) and --eh-frame-hdr. > >Before this patch (generated by adding '-v' to VDSO_LDFLAGS): > ><gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/collect2 \ >-plugin <gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/liblto_plugin.so \ >-plugin-opt=<gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/lto-wrapper \ >-plugin-opt=-fresolution=/tmp/ccGEi5Ka.res \ >--eh-frame-hdr \ >-G 0 \ >-EB \ >-mips64r2 \ >-shared \ >-melf64btsmip \ >-o arch/mips/vdso/vdso.so.dbg.raw \ >-L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0/64 \ >-L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0 \ >-L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0/../../../../mips64-linux/lib \ >-Bsymbolic \ >--no-undefined \ >-soname=linux-vdso.so.1 \ >-EB \ >--hash-style=sysv \ >--build-id \ >-T arch/mips/vdso/vdso.lds \ >arch/mips/vdso/elf.o \ >arch/mips/vdso/vgettimeofday.o \ >arch/mips/vdso/sigreturn.o > >After this patch: > ><gcc_prefix>/bin/mips64-linux-ld \ >-m elf64btsmip \ >-Bsymbolic \ >--no-undefined \ >-soname=linux-vdso.so.1 \ >-EB \ >-nostdlib \ >-shared \ >-G 0 \ >--eh-frame-hdr \ >--hash-style=sysv \ >--build-id \ >-T arch/mips/vdso/vdso.lds \ >arch/mips/vdso/elf.o \ >arch/mips/vdso/vgettimeofday.o >arch/mips/vdso/sigreturn.o \ >-o arch/mips/vdso/vdso.so.dbg.raw > >Note that we leave behind -mips64r2. Turns out that ld ignores it (see >get_emulation in ld/ldmain.c). This is true of current trunk and 2.23, >which is the minimum supported version for the kernel: > >https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/ldmain.c;hb=aa4209e7b679afd74a3860ce25659e71cc4847d5#l593 >https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/ldmain.c;hb=a55e30b51bc6227d8d41f707654d0a5620978dcf#l641 > >Before this patch, LD=ld.lld did nothing: > >$ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' >String dump of section '.comment': >[ 0] ClangBuiltLinux clang version 11.0.0 > >After this patch, it does: > >$ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' >String dump of section '.comment': >[ 0] Linker: LLD 11.0.0 >[ 62] ClangBuiltLinux clang version 11.0.0 > >Link: https://github.com/ClangBuiltLinux/linux/issues/785 >Signed-off-by: Nathan Chancellor <natechancellor@gmail.com> >--- > >v3 -> v4: > >* Improve commit message to show that ld command is effectively the > same as the one generated by GCC. > >* Add '-G 0' and '--eh-frame-hdr' because they were added by GCC. My understanding is that we start to use more -fasynchronous-unwind-tables to eliminate .eh_frame in object files. Without .eh_frame, LD --eh-frame-hdr is really not useful. Sigh... -G 0. This is an option ignored by LLD. GCC devs probably should have used the long option --gpsize rather than take the short option -G. Even better, -z gpsize= or similar if this option is specific to ELF. >v2 -> v3: > >* New patch. > > arch/mips/vdso/Makefile | 13 ++++--------- > 1 file changed, 4 insertions(+), 9 deletions(-) > >diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile >index 92b53d1df42c3..2e64c7600eead 100644 >--- a/arch/mips/vdso/Makefile >+++ b/arch/mips/vdso/Makefile >@@ -60,10 +60,9 @@ ifdef CONFIG_MIPS_DISABLE_VDSO > endif > > # VDSO linker flags. >-VDSO_LDFLAGS := \ >- -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1 \ >- $(addprefix -Wl$(comma),$(filter -E%,$(KBUILD_CFLAGS))) \ >- -nostdlib -shared -Wl,--hash-style=sysv -Wl,--build-id >+ldflags-y := -Bsymbolic --no-undefined -soname=linux-vdso.so.1 \ >+ $(filter -E%,$(KBUILD_CFLAGS)) -nostdlib -shared \ >+ -G 0 --eh-frame-hdr --hash-style=sysv --build-id -T > > CFLAGS_REMOVE_vdso.o = -pg > >@@ -82,11 +81,7 @@ quiet_cmd_vdso_mips_check = VDSOCHK $@ > # > > quiet_cmd_vdsold_and_vdso_check = LD $@ >- cmd_vdsold_and_vdso_check = $(cmd_vdsold); $(cmd_vdso_check); $(cmd_vdso_mips_check) >- >-quiet_cmd_vdsold = VDSO $@ >- cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) \ >- -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@ >+ cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check); $(cmd_vdso_mips_check) > > quiet_cmd_vdsoas_o_S = AS $@ > cmd_vdsoas_o_S = $(CC) $(a_flags) -c -o $@ $< >-- >2.26.2 >
On Tue, Apr 28, 2020 at 03:54:01PM -0700, Fangrui Song wrote: > > On 2020-04-28, Nathan Chancellor wrote: > > Currently, the VDSO is being linked through $(CC). This does not match > > how the rest of the kernel links objects, which is through the $(LD) > > variable. > > > > When clang is built in a default configuration, it first attempts to use > > the target triple's default linker then the system's default linker, > > unless told otherwise through -fuse-ld=... We do not use -fuse-ld= > > because it can be brittle and we have support for invoking $(LD) > > directly. See commit fe00e50b2db8c ("ARM: 8858/1: vdso: use $(LD) > > instead of $(CC) to link VDSO") and commit 691efbedc60d2 ("arm64: vdso: > > use $(LD) instead of $(CC) to link VDSO") for examples of doing this in > > the VDSO. > > > > Do the same thing here. Replace the custom linking logic with $(cmd_ld) > > and ldflags-y so that $(LD) is respected. We need to explicitly add two > > flags to the linker that were implicitly passed by the compiler: > > -G 0 (which comes from ccflags-vdso) and --eh-frame-hdr. > > > > Before this patch (generated by adding '-v' to VDSO_LDFLAGS): > > > > <gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/collect2 \ > > -plugin <gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/liblto_plugin.so \ > > -plugin-opt=<gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/lto-wrapper \ > > -plugin-opt=-fresolution=/tmp/ccGEi5Ka.res \ > > --eh-frame-hdr \ > > -G 0 \ > > -EB \ > > -mips64r2 \ > > -shared \ > > -melf64btsmip \ > > -o arch/mips/vdso/vdso.so.dbg.raw \ > > -L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0/64 \ > > -L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0 \ > > -L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0/../../../../mips64-linux/lib \ > > -Bsymbolic \ > > --no-undefined \ > > -soname=linux-vdso.so.1 \ > > -EB \ > > --hash-style=sysv \ > > --build-id \ > > -T arch/mips/vdso/vdso.lds \ > > arch/mips/vdso/elf.o \ > > arch/mips/vdso/vgettimeofday.o \ > > arch/mips/vdso/sigreturn.o > > > > After this patch: > > > > <gcc_prefix>/bin/mips64-linux-ld \ > > -m elf64btsmip \ > > -Bsymbolic \ > > --no-undefined \ > > -soname=linux-vdso.so.1 \ > > -EB \ > > -nostdlib \ > > -shared \ > > -G 0 \ > > --eh-frame-hdr \ > > --hash-style=sysv \ > > --build-id \ > > -T arch/mips/vdso/vdso.lds \ > > arch/mips/vdso/elf.o \ > > arch/mips/vdso/vgettimeofday.o > > arch/mips/vdso/sigreturn.o \ > > -o arch/mips/vdso/vdso.so.dbg.raw > > > > Note that we leave behind -mips64r2. Turns out that ld ignores it (see > > get_emulation in ld/ldmain.c). This is true of current trunk and 2.23, > > which is the minimum supported version for the kernel: > > > > https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/ldmain.c;hb=aa4209e7b679afd74a3860ce25659e71cc4847d5#l593 > > https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/ldmain.c;hb=a55e30b51bc6227d8d41f707654d0a5620978dcf#l641 > > > > Before this patch, LD=ld.lld did nothing: > > > > $ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' > > String dump of section '.comment': > > [ 0] ClangBuiltLinux clang version 11.0.0 > > > > After this patch, it does: > > > > $ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' > > String dump of section '.comment': > > [ 0] Linker: LLD 11.0.0 > > [ 62] ClangBuiltLinux clang version 11.0.0 > > > > Link: https://github.com/ClangBuiltLinux/linux/issues/785 > > Signed-off-by: Nathan Chancellor <natechancellor@gmail.com> > > --- > > > > v3 -> v4: > > > > * Improve commit message to show that ld command is effectively the > > same as the one generated by GCC. > > > > * Add '-G 0' and '--eh-frame-hdr' because they were added by GCC. > > My understanding is that we start to use more -fasynchronous-unwind-tables to eliminate .eh_frame in object files. > Without .eh_frame, LD --eh-frame-hdr is really not useful. Ah, I was not paying attention; I figured that this was necessary because the x86 VDSO broke without it: cd01544a268ad ("x86/vdso: Pass --eh-frame-hdr to the linker") However, they explicitly add -fasynchronous-unwind-tables so it seems like this indeed can be removed. Kind of odd that GCC passes it along even with -fno-asynchronous-unwind-tables. I will do that in v5 once I get some feedback on whether or not anything else breaks. Cheers, Nathan > Sigh... -G 0. This is an option ignored by LLD. GCC devs probably should > have used the long option --gpsize rather than take the short option -G. > Even better, -z gpsize= or similar if this option is specific to ELF. > > v2 -> v3: > > > > * New patch. > > > > arch/mips/vdso/Makefile | 13 ++++--------- > > 1 file changed, 4 insertions(+), 9 deletions(-) > > > > diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile > > index 92b53d1df42c3..2e64c7600eead 100644 > > --- a/arch/mips/vdso/Makefile > > +++ b/arch/mips/vdso/Makefile > > @@ -60,10 +60,9 @@ ifdef CONFIG_MIPS_DISABLE_VDSO > > endif > > > > # VDSO linker flags. > > -VDSO_LDFLAGS := \ > > - -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1 \ > > - $(addprefix -Wl$(comma),$(filter -E%,$(KBUILD_CFLAGS))) \ > > - -nostdlib -shared -Wl,--hash-style=sysv -Wl,--build-id > > +ldflags-y := -Bsymbolic --no-undefined -soname=linux-vdso.so.1 \ > > + $(filter -E%,$(KBUILD_CFLAGS)) -nostdlib -shared \ > > + -G 0 --eh-frame-hdr --hash-style=sysv --build-id -T > > > > CFLAGS_REMOVE_vdso.o = -pg > > > > @@ -82,11 +81,7 @@ quiet_cmd_vdso_mips_check = VDSOCHK $@ > > # > > > > quiet_cmd_vdsold_and_vdso_check = LD $@ > > - cmd_vdsold_and_vdso_check = $(cmd_vdsold); $(cmd_vdso_check); $(cmd_vdso_mips_check) > > - > > -quiet_cmd_vdsold = VDSO $@ > > - cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) \ > > - -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@ > > + cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check); $(cmd_vdso_mips_check) > > > > quiet_cmd_vdsoas_o_S = AS $@ > > cmd_vdsoas_o_S = $(CC) $(a_flags) -c -o $@ $< > > -- > > 2.26.2 > >
On Tue, 28 Apr 2020, Fangrui Song wrote: > Sigh... -G 0. This is an option ignored by LLD. GCC devs probably should > have used the long option --gpsize rather than take the short option -G. > Even better, -z gpsize= or similar if this option is specific to ELF. Well, the `-G' option is some 30 years old and comes from RISC-OS where the vendor linker had it; it was already present with the initial MIPS port of GCC: commit fe3ec4f798ceea52e1b542b481670b83c12347fd Author: Michael Meissner <meissner@gcc.gnu.org> Date: Sun Dec 1 05:02:56 1991 +0000 Initial revision From-SVN: r88 specifically: +#define LINK_SPEC "%{G*} \ there, so I don't know of what GCC developers' choice you are talking about. Much of GCC legacy comes from various vendors' compilation systems; in this case it was the MIPS Computer Systems (aka MIPSCO) compiler. There may not have been a GNU linker port to RISC-OS at that point (or ever), and the assembler and linker invocation interfaces were kept compatible as ports were added to individual GNU development tools, for obvious reasons. I still remember using GCC with vendor's assembler and linker on DEC Ultrix/MIPS myself many years ago, to overcome some vendor compiler's limitations. And FTR this was still a few years before ELF was even invented; MIPS OSs used the COFF binary format back then. Sorry. I think it's LLVM/LLD that ignores compatibility, not the other way round. Maciej
On Tue, 28 Apr 2020, Nathan Chancellor wrote: > Before this patch, LD=ld.lld did nothing: > > $ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' > String dump of section '.comment': > [ 0] ClangBuiltLinux clang version 11.0.0 What does it mean "did nothing", is `arch/mips/vdso/vdso.so.dbg.raw' not produced? Where does `arch/mips/vdso/vdso.so.dbg' come from then? Maciej
On Sat, May 02, 2020 at 02:50:34PM +0100, Maciej W. Rozycki wrote: > On Tue, 28 Apr 2020, Nathan Chancellor wrote: > > > Before this patch, LD=ld.lld did nothing: > > > > $ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' > > String dump of section '.comment': > > [ 0] ClangBuiltLinux clang version 11.0.0 > > What does it mean "did nothing", is `arch/mips/vdso/vdso.so.dbg.raw' not > produced? Where does `arch/mips/vdso/vdso.so.dbg' come from then? > > Maciej A better wording might be "Before this patch, specifying a linker like ld.lld via the LD variable was not respected by the MIPS VDSO". I should also probably expand on the second paragraph, maybe something like: When clang is built in a default configuration, it first attempts to use the target triple's default linker then the system's default linker, which is almost always ld.bfd. To use ld.lld, '-fuse-ld=lld' must be passed to clang. However, we do not use -fuse-ld=lld because it can be brittle and we have support for invoking $(LD) directly because we have separate compilation and link steps. See commit fe00e50b2db8c ("ARM: 8858/1: vdso: use $(LD) instead of $(CC) to link VDSO") and commit 691efbedc60d2 ("arm64: vdso: use $(LD) instead of $(CC) to link VDSO") for examples of doing this in the VDSO.
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile index 92b53d1df42c3..2e64c7600eead 100644 --- a/arch/mips/vdso/Makefile +++ b/arch/mips/vdso/Makefile @@ -60,10 +60,9 @@ ifdef CONFIG_MIPS_DISABLE_VDSO endif # VDSO linker flags. -VDSO_LDFLAGS := \ - -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1 \ - $(addprefix -Wl$(comma),$(filter -E%,$(KBUILD_CFLAGS))) \ - -nostdlib -shared -Wl,--hash-style=sysv -Wl,--build-id +ldflags-y := -Bsymbolic --no-undefined -soname=linux-vdso.so.1 \ + $(filter -E%,$(KBUILD_CFLAGS)) -nostdlib -shared \ + -G 0 --eh-frame-hdr --hash-style=sysv --build-id -T CFLAGS_REMOVE_vdso.o = -pg @@ -82,11 +81,7 @@ quiet_cmd_vdso_mips_check = VDSOCHK $@ # quiet_cmd_vdsold_and_vdso_check = LD $@ - cmd_vdsold_and_vdso_check = $(cmd_vdsold); $(cmd_vdso_check); $(cmd_vdso_mips_check) - -quiet_cmd_vdsold = VDSO $@ - cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) \ - -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@ + cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check); $(cmd_vdso_mips_check) quiet_cmd_vdsoas_o_S = AS $@ cmd_vdsoas_o_S = $(CC) $(a_flags) -c -o $@ $<
Currently, the VDSO is being linked through $(CC). This does not match how the rest of the kernel links objects, which is through the $(LD) variable. When clang is built in a default configuration, it first attempts to use the target triple's default linker then the system's default linker, unless told otherwise through -fuse-ld=... We do not use -fuse-ld= because it can be brittle and we have support for invoking $(LD) directly. See commit fe00e50b2db8c ("ARM: 8858/1: vdso: use $(LD) instead of $(CC) to link VDSO") and commit 691efbedc60d2 ("arm64: vdso: use $(LD) instead of $(CC) to link VDSO") for examples of doing this in the VDSO. Do the same thing here. Replace the custom linking logic with $(cmd_ld) and ldflags-y so that $(LD) is respected. We need to explicitly add two flags to the linker that were implicitly passed by the compiler: -G 0 (which comes from ccflags-vdso) and --eh-frame-hdr. Before this patch (generated by adding '-v' to VDSO_LDFLAGS): <gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/collect2 \ -plugin <gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/liblto_plugin.so \ -plugin-opt=<gcc_prefix>/libexec/gcc/mips64-linux/9.3.0/lto-wrapper \ -plugin-opt=-fresolution=/tmp/ccGEi5Ka.res \ --eh-frame-hdr \ -G 0 \ -EB \ -mips64r2 \ -shared \ -melf64btsmip \ -o arch/mips/vdso/vdso.so.dbg.raw \ -L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0/64 \ -L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0 \ -L<gcc_prefix>/lib/gcc/mips64-linux/9.3.0/../../../../mips64-linux/lib \ -Bsymbolic \ --no-undefined \ -soname=linux-vdso.so.1 \ -EB \ --hash-style=sysv \ --build-id \ -T arch/mips/vdso/vdso.lds \ arch/mips/vdso/elf.o \ arch/mips/vdso/vgettimeofday.o \ arch/mips/vdso/sigreturn.o After this patch: <gcc_prefix>/bin/mips64-linux-ld \ -m elf64btsmip \ -Bsymbolic \ --no-undefined \ -soname=linux-vdso.so.1 \ -EB \ -nostdlib \ -shared \ -G 0 \ --eh-frame-hdr \ --hash-style=sysv \ --build-id \ -T arch/mips/vdso/vdso.lds \ arch/mips/vdso/elf.o \ arch/mips/vdso/vgettimeofday.o arch/mips/vdso/sigreturn.o \ -o arch/mips/vdso/vdso.so.dbg.raw Note that we leave behind -mips64r2. Turns out that ld ignores it (see get_emulation in ld/ldmain.c). This is true of current trunk and 2.23, which is the minimum supported version for the kernel: https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/ldmain.c;hb=aa4209e7b679afd74a3860ce25659e71cc4847d5#l593 https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/ldmain.c;hb=a55e30b51bc6227d8d41f707654d0a5620978dcf#l641 Before this patch, LD=ld.lld did nothing: $ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' String dump of section '.comment': [ 0] ClangBuiltLinux clang version 11.0.0 After this patch, it does: $ llvm-readelf -p.comment arch/mips/vdso/vdso.so.dbg | sed 's/(.*//' String dump of section '.comment': [ 0] Linker: LLD 11.0.0 [ 62] ClangBuiltLinux clang version 11.0.0 Link: https://github.com/ClangBuiltLinux/linux/issues/785 Signed-off-by: Nathan Chancellor <natechancellor@gmail.com> --- v3 -> v4: * Improve commit message to show that ld command is effectively the same as the one generated by GCC. * Add '-G 0' and '--eh-frame-hdr' because they were added by GCC. v2 -> v3: * New patch. arch/mips/vdso/Makefile | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-)