| Message ID | 20181201115324.20847-1-ard.biesheuvel@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [RFT] arm64: relocatable: build the kernel as a proper shared library | expand |
On Sat, Dec 1, 2018 at 3:54 AM Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > > readelf complains about the section layout of vmlinux when building > with CONFIG_RELOCATABLE=y (for KASLR): > > readelf: Warning: [21]: Link field (0) should index a symtab section. > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > Also, it seems that our use of '-pie -shared' is contradictory, and > thus ambiguous. In general, the way KASLR is wired up at the moment > is highly tailored to how ld.bfd happens to implement (and conflate) > PIE executables and shared libraries, so given the current effort to > support other toolchains, let's fix some of these issues as well. > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > the differences between them are unclear (except for the ELF type > of the produced image [0]) but lld chokes on seeing both at the > same time. This lines up with Peter's suggestion in https://bugs.llvm.org/show_bug.cgi?id=39810#c15 > > - Rename the .rela output section to .rela.dyn, as is customary for > shared libraries and PIE executables. > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > Instead, make sure that they are [mostly] empty by marking all > symbols as local, and emit them into the .init segment. > > These changes only affect the ELF image, and produce the same binary > image, with the exception of a couple of bytes of .init data for the > empty .dynsym and .dynstr sections. > > [0] b9dce7f1ba01 ("arm64: kernel: force ET_DYN ELF type for ...") > > Cc: Nick Desaulniers <ndesaulniers@google.com> > Cc: Peter Smith <peter.smith@linaro.org> > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Thanks for this patch Ard! Boot tested in QEMU w/ and w/o CONFIG_RELOCATABLE=y. Tested-by: Nick Desaulniers <ndesaulniers@google.com> > --- > arch/arm64/Makefile | 2 +- > arch/arm64/kernel/vmlinux.lds.S | 30 ++++++++++++++++++++++++------ > 2 files changed, 25 insertions(+), 7 deletions(-) > > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile > index 6cb9fc7e9382..7221494bcf60 100644 > --- a/arch/arm64/Makefile > +++ b/arch/arm64/Makefile > @@ -18,7 +18,7 @@ ifeq ($(CONFIG_RELOCATABLE), y) > # Pass --no-apply-dynamic-relocs to restore pre-binutils-2.27 behaviour > # for relative relocs, since this leads to better Image compression > # with the relocation offsets always being zero. > -LDFLAGS_vmlinux += -pie -shared -Bsymbolic \ > +LDFLAGS_vmlinux += -shared -Bsymbolic \ > $(call ld-option, --no-apply-dynamic-relocs) > endif > > diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S > index 03b00007553d..9ba4016090b1 100644 > --- a/arch/arm64/kernel/vmlinux.lds.S > +++ b/arch/arm64/kernel/vmlinux.lds.S > @@ -98,8 +98,6 @@ SECTIONS > EXIT_CALL > *(.discard) > *(.discard.*) > - *(.interp .dynamic) > - *(.dynsym .dynstr .hash) > } > > . = KIMAGE_VADDR + TEXT_OFFSET; > @@ -192,12 +190,25 @@ SECTIONS > > PERCPU_SECTION(L1_CACHE_BYTES) > > - .rela : ALIGN(8) { > +#ifdef CONFIG_RELOCATABLE > + .rela.dyn : ALIGN(8) { > *(.rela .rela*) > } > - > - __rela_offset = ABSOLUTE(ADDR(.rela) - KIMAGE_VADDR); > - __rela_size = SIZEOF(.rela); > + .dynamic : { > + *(.dynamic) > + } > + .dynsym : ALIGN(8) { > + *(.dynsym) > + } > + .dynstr : { > + *(.dynstr) > + } > + .hash : { > + *(.hash) > + } > + __rela_offset = ABSOLUTE(ADDR(.rela.dyn) - KIMAGE_VADDR); > + __rela_size = SIZEOF(.rela.dyn); > +#endif > > . = ALIGN(SEGMENT_ALIGN); > __initdata_end = .; > @@ -244,6 +255,13 @@ SECTIONS > HEAD_SYMBOLS > } > > +#ifdef CONFIG_RELOCATABLE > +VERSION { > + /* mark all symbols as local so they are not listed in .dynsym */ > + { local: *; }; > +} > +#endif > + > /* > * The HYP init code and ID map text can't be longer than a page each, > * and should not cross a page boundary. > -- > 2.19.2 >
Hi Ard, On Sat, Dec 01, 2018 at 12:53:24PM +0100, Ard Biesheuvel wrote: > readelf complains about the section layout of vmlinux when building > with CONFIG_RELOCATABLE=y (for KASLR): > > readelf: Warning: [21]: Link field (0) should index a symtab section. > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > Also, it seems that our use of '-pie -shared' is contradictory, and > thus ambiguous. In general, the way KASLR is wired up at the moment > is highly tailored to how ld.bfd happens to implement (and conflate) > PIE executables and shared libraries, so given the current effort to > support other toolchains, let's fix some of these issues as well. > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > the differences between them are unclear (except for the ELF type > of the produced image [0]) but lld chokes on seeing both at the > same time. > > - Rename the .rela output section to .rela.dyn, as is customary for > shared libraries and PIE executables. > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > Instead, make sure that they are [mostly] empty by marking all > symbols as local, and emit them into the .init segment. These second two changes seem fairly arbitrary to me: is lld unhappy without them? My concern is that we'll make some other change in the future and run into more arbitrary breakage. Is there any build bots hooked up with lld so help us catch regressions quickly? (i.e nightly builds of mainline and next) Will
On Mon, 3 Dec 2018 at 18:08, Will Deacon <will.deacon@arm.com> wrote: > > Hi Ard, > > On Sat, Dec 01, 2018 at 12:53:24PM +0100, Ard Biesheuvel wrote: > > readelf complains about the section layout of vmlinux when building > > with CONFIG_RELOCATABLE=y (for KASLR): > > > > readelf: Warning: [21]: Link field (0) should index a symtab section. > > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > > > Also, it seems that our use of '-pie -shared' is contradictory, and > > thus ambiguous. In general, the way KASLR is wired up at the moment > > is highly tailored to how ld.bfd happens to implement (and conflate) > > PIE executables and shared libraries, so given the current effort to > > support other toolchains, let's fix some of these issues as well. > > > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > > the differences between them are unclear (except for the ELF type > > of the produced image [0]) but lld chokes on seeing both at the > > same time. > > > > - Rename the .rela output section to .rela.dyn, as is customary for > > shared libraries and PIE executables. > > > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > > Instead, make sure that they are [mostly] empty by marking all > > symbols as local, and emit them into the .init segment. > > These second two changes seem fairly arbitrary to me: is lld unhappy without > them? My concern is that we'll make some other change in the future and run > into more arbitrary breakage. Is there any build bots hooked up with lld so > help us catch regressions quickly? (i.e nightly builds of mainline and next) > The .rela.dyn change is for readelf actually, not for lld. Discarding the .dyn* sections is indeed something lld chokes on. It seems reasonable to make the vmlinux ELF image look as much as an ordinary shared library as we can, to avoid issues with tooling in general, not just lld. This is why I separated this patch from the one that caters for lld in particular.
On Mon, 3 Dec 2018 at 17:08, Will Deacon <will.deacon@arm.com> wrote: > > Hi Ard, > > On Sat, Dec 01, 2018 at 12:53:24PM +0100, Ard Biesheuvel wrote: > > readelf complains about the section layout of vmlinux when building > > with CONFIG_RELOCATABLE=y (for KASLR): > > > > readelf: Warning: [21]: Link field (0) should index a symtab section. > > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > > > Also, it seems that our use of '-pie -shared' is contradictory, and > > thus ambiguous. In general, the way KASLR is wired up at the moment > > is highly tailored to how ld.bfd happens to implement (and conflate) > > PIE executables and shared libraries, so given the current effort to > > support other toolchains, let's fix some of these issues as well. > > > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > > the differences between them are unclear (except for the ELF type > > of the produced image [0]) but lld chokes on seeing both at the > > same time. > > > > - Rename the .rela output section to .rela.dyn, as is customary for > > shared libraries and PIE executables. > > > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > > Instead, make sure that they are [mostly] empty by marking all > > symbols as local, and emit them into the .init segment. > > These second two changes seem fairly arbitrary to me: is lld unhappy without > them? My concern is that we'll make some other change in the future and run > into more arbitrary breakage. Is there any build bots hooked up with lld so > help us catch regressions quickly? (i.e nightly builds of mainline and next) > > Will Hello Will, LLD currently doesn't permit the .dynamic, .dynstr and .hash sections from being removed. At the moment the Linaro kernel CI job doesn't have a LLD configuration. We are planning to rectify this shortly, hopefully next week and if not then straight after Christmas. If you do need support for removal of .dynamic etc. to happen it would be great if you could comment on the LLD PR https://bugs.llvm.org/show_bug.cgi?id=39810 . There is some pushback over whether LLD needs to do this if it has already been worked around. Peter p.s. apologies to the people that got this twice, the message bounced first time.
On Mon, Dec 3, 2018 at 9:08 AM Will Deacon <will.deacon@arm.com> wrote: > > Hi Ard, > > On Sat, Dec 01, 2018 at 12:53:24PM +0100, Ard Biesheuvel wrote: > > readelf complains about the section layout of vmlinux when building > > with CONFIG_RELOCATABLE=y (for KASLR): > > > > readelf: Warning: [21]: Link field (0) should index a symtab section. > > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > > > Also, it seems that our use of '-pie -shared' is contradictory, and > > thus ambiguous. In general, the way KASLR is wired up at the moment > > is highly tailored to how ld.bfd happens to implement (and conflate) > > PIE executables and shared libraries, so given the current effort to > > support other toolchains, let's fix some of these issues as well. > > > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > > the differences between them are unclear (except for the ELF type > > of the produced image [0]) but lld chokes on seeing both at the > > same time. > > > > - Rename the .rela output section to .rela.dyn, as is customary for > > shared libraries and PIE executables. > > > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > > Instead, make sure that they are [mostly] empty by marking all > > symbols as local, and emit them into the .init segment. > > These second two changes seem fairly arbitrary to me: is lld unhappy without > them? I don't know about .rela -> .rela.dyn, or .hash, but LLD developers just issued patches allowing the following sections to be discarded: .dynamic: https://reviews.llvm.org/D55211 .dynsym: https://reviews.llvm.org/D55218 .dynstr: https://reviews.llvm.org/D55215 > My concern is that we'll make some other change in the future and run > into more arbitrary breakage. Is there any build bots hooked up with lld so > help us catch regressions quickly? (i.e nightly builds of mainline and next) Yes. We're running continuous integration at: https://github.com/ClangBuiltLinux/continuous-integration We have nightly cron jobs building arm64 defconfigs with clang for mainline, -next, and some LTS branches of the stable tree. ex. https://travis-ci.com/ClangBuiltLinux/continuous-integration/builds/93312295 Some arm64 targets are additionally being linked with LLD. CONFIG_RANDOMIZE_BASE is not on in the defconfig (see: http://lists.infradead.org/pipermail/linux-arm-kernel/2018-December/616877.html) We're working with kernel CI to get Clang builds (and LLD) setup ASAP, as they cover more trees, bisect failures and notify developers automatically.
On Mon, Dec 03, 2018 at 06:11:47PM +0100, Ard Biesheuvel wrote: > On Mon, 3 Dec 2018 at 18:08, Will Deacon <will.deacon@arm.com> wrote: > > > > Hi Ard, > > > > On Sat, Dec 01, 2018 at 12:53:24PM +0100, Ard Biesheuvel wrote: > > > readelf complains about the section layout of vmlinux when building > > > with CONFIG_RELOCATABLE=y (for KASLR): > > > > > > readelf: Warning: [21]: Link field (0) should index a symtab section. > > > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > > > > > Also, it seems that our use of '-pie -shared' is contradictory, and > > > thus ambiguous. In general, the way KASLR is wired up at the moment > > > is highly tailored to how ld.bfd happens to implement (and conflate) > > > PIE executables and shared libraries, so given the current effort to > > > support other toolchains, let's fix some of these issues as well. > > > > > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > > > the differences between them are unclear (except for the ELF type > > > of the produced image [0]) but lld chokes on seeing both at the > > > same time. > > > > > > - Rename the .rela output section to .rela.dyn, as is customary for > > > shared libraries and PIE executables. > > > > > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > > > Instead, make sure that they are [mostly] empty by marking all > > > symbols as local, and emit them into the .init segment. > > > > These second two changes seem fairly arbitrary to me: is lld unhappy without > > them? My concern is that we'll make some other change in the future and run > > into more arbitrary breakage. Is there any build bots hooked up with lld so > > help us catch regressions quickly? (i.e nightly builds of mainline and next) > > > > The .rela.dyn change is for readelf actually, not for lld. Discarding > the .dyn* sections is indeed something lld chokes on. > > It seems reasonable to make the vmlinux ELF image look as much as an > ordinary shared library as we can, to avoid issues with tooling in > general, not just lld. This is why I separated this patch from the one > that caters for lld in particular. Given that the LLD developers are looking at addressing the section discards, can you repost without that part please? Will
On Mon, Dec 03, 2018 at 10:29:00AM -0800, Nick Desaulniers wrote: > On Mon, Dec 3, 2018 at 9:08 AM Will Deacon <will.deacon@arm.com> wrote: > > > > Hi Ard, > > > > On Sat, Dec 01, 2018 at 12:53:24PM +0100, Ard Biesheuvel wrote: > > > readelf complains about the section layout of vmlinux when building > > > with CONFIG_RELOCATABLE=y (for KASLR): > > > > > > readelf: Warning: [21]: Link field (0) should index a symtab section. > > > readelf: Warning: [21]: Info field (0) should index a relocatable section. > > > > > > Also, it seems that our use of '-pie -shared' is contradictory, and > > > thus ambiguous. In general, the way KASLR is wired up at the moment > > > is highly tailored to how ld.bfd happens to implement (and conflate) > > > PIE executables and shared libraries, so given the current effort to > > > support other toolchains, let's fix some of these issues as well. > > > > > > - Drop the -pie linker argument and just leave -shared. In ld.bfd, > > > the differences between them are unclear (except for the ELF type > > > of the produced image [0]) but lld chokes on seeing both at the > > > same time. > > > > > > - Rename the .rela output section to .rela.dyn, as is customary for > > > shared libraries and PIE executables. > > > > > > - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. > > > Instead, make sure that they are [mostly] empty by marking all > > > symbols as local, and emit them into the .init segment. > > > > These second two changes seem fairly arbitrary to me: is lld unhappy without > > them? > > I don't know about .rela -> .rela.dyn, or .hash, but LLD developers > just issued patches allowing the following sections to be discarded: > > .dynamic: > https://reviews.llvm.org/D55211 > .dynsym: > https://reviews.llvm.org/D55218 > .dynstr: > https://reviews.llvm.org/D55215 Thanks again. > > My concern is that we'll make some other change in the future and run > > into more arbitrary breakage. Is there any build bots hooked up with lld so > > help us catch regressions quickly? (i.e nightly builds of mainline and next) > > Yes. We're running continuous integration at: > https://github.com/ClangBuiltLinux/continuous-integration > > We have nightly cron jobs building arm64 defconfigs with clang for > mainline, -next, and some LTS branches of the stable tree. ex. > https://travis-ci.com/ClangBuiltLinux/continuous-integration/builds/93312295 Great, that looks very promising! Hopefully that means we can keep things building reliably once we get to that point. Will
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 6cb9fc7e9382..7221494bcf60 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -18,7 +18,7 @@ ifeq ($(CONFIG_RELOCATABLE), y) # Pass --no-apply-dynamic-relocs to restore pre-binutils-2.27 behaviour # for relative relocs, since this leads to better Image compression # with the relocation offsets always being zero. -LDFLAGS_vmlinux += -pie -shared -Bsymbolic \ +LDFLAGS_vmlinux += -shared -Bsymbolic \ $(call ld-option, --no-apply-dynamic-relocs) endif diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 03b00007553d..9ba4016090b1 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -98,8 +98,6 @@ SECTIONS EXIT_CALL *(.discard) *(.discard.*) - *(.interp .dynamic) - *(.dynsym .dynstr .hash) } . = KIMAGE_VADDR + TEXT_OFFSET; @@ -192,12 +190,25 @@ SECTIONS PERCPU_SECTION(L1_CACHE_BYTES) - .rela : ALIGN(8) { +#ifdef CONFIG_RELOCATABLE + .rela.dyn : ALIGN(8) { *(.rela .rela*) } - - __rela_offset = ABSOLUTE(ADDR(.rela) - KIMAGE_VADDR); - __rela_size = SIZEOF(.rela); + .dynamic : { + *(.dynamic) + } + .dynsym : ALIGN(8) { + *(.dynsym) + } + .dynstr : { + *(.dynstr) + } + .hash : { + *(.hash) + } + __rela_offset = ABSOLUTE(ADDR(.rela.dyn) - KIMAGE_VADDR); + __rela_size = SIZEOF(.rela.dyn); +#endif . = ALIGN(SEGMENT_ALIGN); __initdata_end = .; @@ -244,6 +255,13 @@ SECTIONS HEAD_SYMBOLS } +#ifdef CONFIG_RELOCATABLE +VERSION { + /* mark all symbols as local so they are not listed in .dynsym */ + { local: *; }; +} +#endif + /* * The HYP init code and ID map text can't be longer than a page each, * and should not cross a page boundary.
readelf complains about the section layout of vmlinux when building with CONFIG_RELOCATABLE=y (for KASLR): readelf: Warning: [21]: Link field (0) should index a symtab section. readelf: Warning: [21]: Info field (0) should index a relocatable section. Also, it seems that our use of '-pie -shared' is contradictory, and thus ambiguous. In general, the way KASLR is wired up at the moment is highly tailored to how ld.bfd happens to implement (and conflate) PIE executables and shared libraries, so given the current effort to support other toolchains, let's fix some of these issues as well. - Drop the -pie linker argument and just leave -shared. In ld.bfd, the differences between them are unclear (except for the ELF type of the produced image [0]) but lld chokes on seeing both at the same time. - Rename the .rela output section to .rela.dyn, as is customary for shared libraries and PIE executables. - Don't discard the .dynamic, .dynsym, .dynstr and .hash sections. Instead, make sure that they are [mostly] empty by marking all symbols as local, and emit them into the .init segment. These changes only affect the ELF image, and produce the same binary image, with the exception of a couple of bytes of .init data for the empty .dynsym and .dynstr sections. [0] b9dce7f1ba01 ("arm64: kernel: force ET_DYN ELF type for ...") Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Peter Smith <peter.smith@linaro.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/arm64/Makefile | 2 +- arch/arm64/kernel/vmlinux.lds.S | 30 ++++++++++++++++++++++++------ 2 files changed, 25 insertions(+), 7 deletions(-)