[1/5] kbuild: thin archives final link close --whole-archives option

Commit Message

Nicholas Piggin June 19, 2017, 3:52 p.m. UTC

On Mon, 19 Jun 2017 17:35:32 +0900
Masahiro Yamada <[e-mail]> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-19 17:27 GMT+09:00 Nicholas Piggin <[e-mail]>:
> > On Mon, 19 Jun 2017 17:14:22 +0900
> > Masahiro Yamada <[e-mail]> wrote:
> >  
> >> Hi Nicholas,
> >>
> >>
> >> 2017-06-19 15:51 GMT+09:00 Nicholas Piggin <[e-mail]>:  
> >> > On Mon, 19 Jun 2017 15:16:17 +0900
> >> > Masahiro Yamada <[e-mail]> wrote:
> >> >  
> >> >> Hi Nicholas,
> >> >>
> >> >>
> >> >> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <[e-mail]>:  
> >> >> > Close the --whole-archives option with --no-whole-archive. Some
> >> >> > architectures end up including additional .o and files multiple
> >> >> > times after this, and they get duplicate symbols when they are
> >> >> > brought under the --whole-archives option.  
> >> >>
> >> >> Which architectures have additional files after --no-whole-archive ?
> >> >>
> >> >> I see this case only for ARCH = "um" in vmlinux_link()
> >> >> where it adds some -l* options after --no-whole-archive.
> >> >>
> >> >> I think it is good to close the --whole-archives everywhere.
> >> >> So, the code looks OK to me, but I just wondered about the log.  
> >> >
> >> > Actually a number of archs seemed to get build errors without this,
> >> > and they seem to come from libgcc perhaps.
> >> >
> >> > binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
> >> > /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
> >> > /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
> >> > /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
> >> > /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
> >> >
> >> > /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
> >> >
> >> > etc  
> >>
> >>
> >> Oops, I did not test such architectures.
> >>
> >>  
> >> > m32r, parisc, xtensa seemed to be getting such errors. I wonder if
> >> > the linker implicitly adds some runtime libs at the end that get
> >> > caught up here. Either way I didn't look too far into it because this
> >> > fix seems obviously correct and solved the problem.
> >> >
> >> > Thanks,  
> >>
> >>
> >> Which toolchains do you use?  
> >
> > I just had them run through the kbuild 0day tests. Not sure exactly
> > what they use.
> >  
> >> I downloaded xtensa-linux- from this site:
> >> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/
> >>
> >>
> >> I see a different error for xtensa.
> >>
> >>
> >> I applied all of you 5 patches, but xtensa build still fails.
> >>
> >>   LD      vmlinux.o
> >> xtensa-linux-ld: internal error
> >> /home/tony/buildall/src/binutils/ld/ldlang.c 6178
> >>
> >>
> >> Could you check it?  
> >
> > Ahh, the old ld: internal error... what ld version is that?
> >  
> 
> 
> seems 2.24 according to the following:
> 
> 
> masahiro@pug:~$ xtensa-linux-ld -v
> GNU ld (GNU Binutils) 2.24

Ah, thank you. It must have been that the 0day build (cc'ed) did
not return any error to be from the ld internal error.

This has led me to the true cause of the error which is the way
external archives are handled. After implementing the fix, I think
the lib.a size regression for thin archives should be solved as
well.

This patch should be added between patches 2 and 3 of this series.


kbuild: handle libs-y archives separately from built-in.o archives

The thin archives build currently puts all lib.a and built-in.o
files together and links them with --whole-archive.

This works because thin archives can recursively refer to thin
archives. However some architectures include libgcc.a, which may
not be a thin archive, or it may not be constructed with the "P"
option, in which case its contents do not get linked correctly.

So don't pull .a libs into the root built-in.o archive. These
libs should already have symbol tables and indexes built, so they
can be direct linker inputs. Move them out of the --whole-archive
option, which restore the conditional linking behaviour of lib.a
to thin archives builds.

Signed-off-by: Nicholas Piggin <[e-mail]>
---
 Documentation/kbuild/kbuild.txt |  8 +++++--
 Makefile                        |  7 +++---
 scripts/link-vmlinux.sh         | 47 ++++++++++++++++++++++++++++++++---------
 3 files changed, 47 insertions(+), 15 deletions(-)

Comments

Josh Triplett June 19, 2017, 11:48 p.m. UTC | #1

On Tue, Jun 20, 2017 at 01:52:05AM +1000, Nicholas Piggin wrote:
> On Mon, 19 Jun 2017 17:35:32 +0900
> Masahiro Yamada <[e-mail]> wrote:
> 
> > Hi Nicholas,
> > 
> > 
> > 2017-06-19 17:27 GMT+09:00 Nicholas Piggin <[e-mail]>:
> > > On Mon, 19 Jun 2017 17:14:22 +0900
> > > Masahiro Yamada <[e-mail]> wrote:
> > >  
> > >> Hi Nicholas,
> > >>
> > >>
> > >> 2017-06-19 15:51 GMT+09:00 Nicholas Piggin <[e-mail]>:  
> > >> > On Mon, 19 Jun 2017 15:16:17 +0900
> > >> > Masahiro Yamada <[e-mail]> wrote:
> > >> >  
> > >> >> Hi Nicholas,
> > >> >>
> > >> >>
> > >> >> 2017-06-09 14:24 GMT+09:00 Nicholas Piggin <[e-mail]>:  
> > >> >> > Close the --whole-archives option with --no-whole-archive. Some
> > >> >> > architectures end up including additional .o and files multiple
> > >> >> > times after this, and they get duplicate symbols when they are
> > >> >> > brought under the --whole-archives option.  
> > >> >>
> > >> >> Which architectures have additional files after --no-whole-archive ?
> > >> >>
> > >> >> I see this case only for ARCH = "um" in vmlinux_link()
> > >> >> where it adds some -l* options after --no-whole-archive.
> > >> >>
> > >> >> I think it is good to close the --whole-archives everywhere.
> > >> >> So, the code looks OK to me, but I just wondered about the log.  
> > >> >
> > >> > Actually a number of archs seemed to get build errors without this,
> > >> > and they seem to come from libgcc perhaps.
> > >> >
> > >> > binfmt_elf.c:(.text+0x4851): undefined reference to `__kernel_syscall_via_epc'
> > >> > /c/gcc/libgcc/libgcc2.c:1318: multiple definition of `__ucmpdi2'
> > >> > /c/gcc/libgcc/libgcc2.c:405: multiple definition of `__lshrdi3'
> > >> > /c/gcc/libgcc/libgcc2.c:433: multiple definition of `__ashldi3'
> > >> > /c/gcc/libgcc/libgcc2.c:461: multiple definition of `__ashrdi3'
> > >> >
> > >> > /home/tony/buildall/src/gcc/libgcc/config/xtensa/lib2funcs.S:36: multiple definition of `__xtensa_libgcc_window_spill'
> > >> >
> > >> > etc  
> > >>
> > >>
> > >> Oops, I did not test such architectures.
> > >>
> > >>  
> > >> > m32r, parisc, xtensa seemed to be getting such errors. I wonder if
> > >> > the linker implicitly adds some runtime libs at the end that get
> > >> > caught up here. Either way I didn't look too far into it because this
> > >> > fix seems obviously correct and solved the problem.
> > >> >
> > >> > Thanks,  
> > >>
> > >>
> > >> Which toolchains do you use?  
> > >
> > > I just had them run through the kbuild 0day tests. Not sure exactly
> > > what they use.
> > >  
> > >> I downloaded xtensa-linux- from this site:
> > >> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/
> > >>
> > >>
> > >> I see a different error for xtensa.
> > >>
> > >>
> > >> I applied all of you 5 patches, but xtensa build still fails.
> > >>
> > >>   LD      vmlinux.o
> > >> xtensa-linux-ld: internal error
> > >> /home/tony/buildall/src/binutils/ld/ldlang.c 6178
> > >>
> > >>
> > >> Could you check it?  
> > >
> > > Ahh, the old ld: internal error... what ld version is that?
> > >  
> > 
> > 
> > seems 2.24 according to the following:
> > 
> > 
> > masahiro@pug:~$ xtensa-linux-ld -v
> > GNU ld (GNU Binutils) 2.24
> 
> Ah, thank you. It must have been that the 0day build (cc'ed) did
> not return any error to be from the ld internal error.
> 
> This has led me to the true cause of the error which is the way
> external archives are handled. After implementing the fix, I think
> the lib.a size regression for thin archives should be solved as
> well.

Thanks for addressing this!  Much appreciated.

> This patch should be added between patches 2 and 3 of this series.
> 
> 
> kbuild: handle libs-y archives separately from built-in.o archives
> 
> The thin archives build currently puts all lib.a and built-in.o
> files together and links them with --whole-archive.
> 
> This works because thin archives can recursively refer to thin
> archives. However some architectures include libgcc.a, which may
> not be a thin archive, or it may not be constructed with the "P"
> option, in which case its contents do not get linked correctly.
> 
> So don't pull .a libs into the root built-in.o archive. These
> libs should already have symbol tables and indexes built, so they
> can be direct linker inputs. Move them out of the --whole-archive
> option, which restore the conditional linking behaviour of lib.a
> to thin archives builds.
> 
> Signed-off-by: Nicholas Piggin <[e-mail]>
> ---
>  Documentation/kbuild/kbuild.txt |  8 +++++--
>  Makefile                        |  7 +++---
>  scripts/link-vmlinux.sh         | 47 ++++++++++++++++++++++++++++++++---------
>  3 files changed, 47 insertions(+), 15 deletions(-)
> 
> diff --git a/Documentation/kbuild/kbuild.txt b/Documentation/kbuild/kbuild.txt
> index 0ff6a466a05b..ac2363ea05c5 100644
> --- a/Documentation/kbuild/kbuild.txt
> +++ b/Documentation/kbuild/kbuild.txt
> @@ -236,5 +236,9 @@ Files specified with KBUILD_VMLINUX_INIT are linked first.
>  KBUILD_VMLINUX_MAIN
>  --------------------------------------------------
>  All object files for the main part of vmlinux.
> -KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN together specify
> -all the object files used to link vmlinux.
> +
> +KBUILD_VMLINUX_LIBS
> +--------------------------------------------------
> +All .a "lib" files for vmlinux.
> +KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, and KBUILD_VMLINUX_LIBS together
> +specify all the object files used to link vmlinux.
> diff --git a/Makefile b/Makefile
> index 470bd4d9513a..a145a537ad42 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -952,19 +952,20 @@ core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
>  drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
>  net-y		:= $(patsubst %/, %/built-in.o, $(net-y))
>  libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
> -libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
> +libs-y2		:= $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
>  libs-y		:= $(libs-y1) $(libs-y2)
>  virt-y		:= $(patsubst %/, %/built-in.o, $(virt-y))
>  
>  # Externally visible symbols (used by link-vmlinux.sh)
>  export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
> -export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(virt-y)
> +export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)
> +export KBUILD_VMLINUX_LIBS := $(libs-y1)
>  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
>  export LDFLAGS_vmlinux
>  # used by scripts/pacmage/Makefile
>  export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) arch Documentation include samples scripts tools)
>  
> -vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN)
> +vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)
>  
>  # Include targets which we want to execute sequentially if the rest of the
>  # kernel build went well. If CONFIG_TRIM_UNUSED_KSYMS is set, this might be
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 2a062ea130b5..e7b7eee31538 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -3,9 +3,12 @@
>  # link vmlinux
>  #
>  # vmlinux is linked from the objects selected by $(KBUILD_VMLINUX_INIT) and
> -# $(KBUILD_VMLINUX_MAIN). Most are built-in.o files from top-level directories
> -# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
> -# Ordering when linking is important, and $(KBUILD_VMLINUX_INIT) must be first.
> +# $(KBUILD_VMLINUX_MAIN) and $(KBUILD_VMLINUX_LIBS). Most are built-in.o files
> +# from top-level directories in the kernel tree, others are specified in
> +# arch/$(ARCH)/Makefile. Ordering when linking is important, and
> +# $(KBUILD_VMLINUX_INIT) must be first. $(KBUILD_VMLINUX_LIBS) are archives
> +# which are linked conditionally (not within --whole-archive), and do not
> +# require symbol indexes added.
>  #
>  # vmlinux
>  #   ^
> @@ -16,6 +19,9 @@
>  #   +--< $(KBUILD_VMLINUX_MAIN)
>  #   |    +--< drivers/built-in.o mm/built-in.o + more
>  #   |
> +#   +--< $(KBUILD_VMLINUX_LIBS)
> +#   |    +--< lib/lib.a + more
> +#   |
>  #   +-< ${kallsymso} (see description in KALLSYMS section)
>  #
>  # vmlinux version (uname -v) cannot be updated during normal
> @@ -37,9 +43,10 @@ info()
>  	fi
>  }
>  
> -# Thin archive build here makes a final archive with
> -# symbol table and indexes from vmlinux objects, which can be
> -# used as input to linker.
> +# Thin archive build here makes a final archive with symbol table and indexes
> +# from vmlinux objects INIT and MAIN, which can be used as input to linker.
> +# KBUILD_VMLINUX_LIBS archives should already have symbol table and indexes
> +# added.
>  #
>  # Traditional incremental style of link does not require this step
>  #
> @@ -50,7 +57,7 @@ archive_builtin()
>  	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
>  		info AR built-in.o
>  		rm -f built-in.o;
> -		${AR} rcsT${KBUILD_ARFLAGS} built-in.o			\
> +		${AR} rcsTP${KBUILD_ARFLAGS} built-in.o			\
>  					${KBUILD_VMLINUX_INIT}		\
>  					${KBUILD_VMLINUX_MAIN}
>  	fi
> @@ -63,11 +70,17 @@ modpost_link()
>  	local objects
>  
>  	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -		objects="--whole-archive built-in.o --no-whole-archive"
> +		objects="--whole-archive				\
> +			built-in.o					\
> +			--no-whole-archive				\
> +			--start-group					\
> +			${KBUILD_VMLINUX_LIBS}				\
> +			--end-group"
>  	else
>  		objects="${KBUILD_VMLINUX_INIT}				\
>  			--start-group					\
>  			${KBUILD_VMLINUX_MAIN}				\
> +			${KBUILD_VMLINUX_LIBS}				\
>  			--end-group"
>  	fi
>  	${LD} ${LDFLAGS} -r -o ${1} ${objects}
> @@ -83,11 +96,18 @@ vmlinux_link()
>  
>  	if [ "${SRCARCH}" != "um" ]; then
>  		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -			objects="--whole-archive built-in.o ${1} --no-whole-archive"
> +			objects="--whole-archive			\
> +				built-in.o				\
> +				--no-whole-archive			\
> +				--start-group				\
> +				${KBUILD_VMLINUX_LIBS}			\
> +				--end-group				\
> +				${1}"
>  		else
>  			objects="${KBUILD_VMLINUX_INIT}			\
>  				--start-group				\
>  				${KBUILD_VMLINUX_MAIN}			\
> +				${KBUILD_VMLINUX_LIBS}			\
>  				--end-group				\
>  				${1}"
>  		fi
> @@ -96,11 +116,18 @@ vmlinux_link()
>  			-T ${lds} ${objects}
>  	else
>  		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -			objects="-Wl,--whole-archive built-in.o ${1} -Wl,--no-whole-archive"
> +			objects="-Wl,--whole-archive			\
> +				built-in.o				\
> +				-Wl,--no-whole-archive			\
> +				-Wl,--start-group			\
> +				${KBUILD_VMLINUX_LIBS}			\
> +				-Wl,--end-group				\
> +				${1}"
>  		else
>  			objects="${KBUILD_VMLINUX_INIT}			\
>  				-Wl,--start-group			\
>  				${KBUILD_VMLINUX_MAIN}			\
> +				${KBUILD_VMLINUX_LIBS}			\
>  				-Wl,--end-group				\
>  				${1}"
>  		fi
> -- 
> 2.11.0
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Masahiro Yamada June 21, 2017, 1:17 a.m. UTC | #2

Hi Nicholas,

2017-06-20 0:52 GMT+09:00 Nicholas Piggin <[e-mail]>:
>>
>> masahiro@pug:~$ xtensa-linux-ld -v
>> GNU ld (GNU Binutils) 2.24
>
> Ah, thank you. It must have been that the 0day build (cc'ed) did
> not return any error to be from the ld internal error.
>
> This has led me to the true cause of the error which is the way
> external archives are handled. After implementing the fix, I think
> the lib.a size regression for thin archives should be solved as
> well.


Thanks for figuring this out!


> This patch should be added between patches 2 and 3 of this series.
>

Done.


I applied all 6 patches to linux-kbuild/thin-ar.

I fixed up some:

[1] fix a typo  "tihs"  -> "this"
[2] reword the document as "The build system has, as of 4.13, switched
to using..."
[3] Add "P" to link_vmlinux.sh as well


Could you double-check if I did them correctly?




> --- a/Makefile
> +++ b/Makefile
> @@ -952,19 +952,20 @@ core-y            := $(patsubst %/, %/built-in.o, $(core-y))
>  drivers-y      := $(patsubst %/, %/built-in.o, $(drivers-y))
>  net-y          := $(patsubst %/, %/built-in.o, $(net-y))
>  libs-y1                := $(patsubst %/, %/lib.a, $(libs-y))
> -libs-y2                := $(patsubst %/, %/built-in.o, $(libs-y))
> +libs-y2                := $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
>  libs-y         := $(libs-y1) $(libs-y2)


After I applied this patch, I noticed one more thing.

With this patch, I think "libs-y" will be unnecessary.

If you ack me to fix-up locally, I will remove:
   libs-y         := $(libs-y1) $(libs-y2)




> @@ -50,7 +57,7 @@ archive_builtin()
>         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
>                 info AR built-in.o
>                 rm -f built-in.o;
> -               ${AR} rcsT${KBUILD_ARFLAGS} built-in.o                  \
> +               ${AR} rcsTP${KBUILD_ARFLAGS} built-in.o                 \
>                                         ${KBUILD_VMLINUX_INIT}          \
>                                         ${KBUILD_VMLINUX_MAIN}
>         fi

I moved this hunk to  2/6 "thin archives use P option to ar".

Nicholas Piggin June 21, 2017, 2:47 a.m. UTC | #3

On Wed, 21 Jun 2017 10:17:11 +0900
Masahiro Yamada <[e-mail]> wrote:

> Hi Nicholas,
> 
> 2017-06-20 0:52 GMT+09:00 Nicholas Piggin <[e-mail]>:
> >>
> >> masahiro@pug:~$ xtensa-linux-ld -v
> >> GNU ld (GNU Binutils) 2.24  
> >
> > Ah, thank you. It must have been that the 0day build (cc'ed) did
> > not return any error to be from the ld internal error.
> >
> > This has led me to the true cause of the error which is the way
> > external archives are handled. After implementing the fix, I think
> > the lib.a size regression for thin archives should be solved as
> > well.  
> 
> 
> Thanks for figuring this out!
> 
> 
> > This patch should be added between patches 2 and 3 of this series.
> >  
> 
> Done.
> 
> 
> I applied all 6 patches to linux-kbuild/thin-ar.
> 
> I fixed up some:
> 
> [1] fix a typo  "tihs"  -> "this"
> [2] reword the document as "The build system has, as of 4.13, switched
> to using..."
> [3] Add "P" to link_vmlinux.sh as well
> 
> 
> Could you double-check if I did them correctly?

Yes these look good to me, thank you.

> > --- a/Makefile
> > +++ b/Makefile
> > @@ -952,19 +952,20 @@ core-y            := $(patsubst %/, %/built-in.o, $(core-y))
> >  drivers-y      := $(patsubst %/, %/built-in.o, $(drivers-y))
> >  net-y          := $(patsubst %/, %/built-in.o, $(net-y))
> >  libs-y1                := $(patsubst %/, %/lib.a, $(libs-y))
> > -libs-y2                := $(patsubst %/, %/built-in.o, $(libs-y))
> > +libs-y2                := $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
> >  libs-y         := $(libs-y1) $(libs-y2)  
> 
> 
> After I applied this patch, I noticed one more thing.
> 
> With this patch, I think "libs-y" will be unnecessary.
> 
> If you ack me to fix-up locally, I will remove:
>    libs-y         := $(libs-y1) $(libs-y2)

Yes I suppose that can be removed.

I wonder if the names could be a bit more descriptive?

libs-y-liba
libs-y-builtin

?

> > @@ -50,7 +57,7 @@ archive_builtin()
> >         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> >                 info AR built-in.o
> >                 rm -f built-in.o;
> > -               ${AR} rcsT${KBUILD_ARFLAGS} built-in.o                  \
> > +               ${AR} rcsTP${KBUILD_ARFLAGS} built-in.o                 \
> >                                         ${KBUILD_VMLINUX_INIT}          \
> >                                         ${KBUILD_VMLINUX_MAIN}
> >         fi  
> 
> I moved this hunk to  2/6 "thin archives use P option to ar".

Thanks,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Masahiro Yamada June 21, 2017, 3:29 a.m. UTC | #4

Hi Nicholas,


2017-06-21 11:47 GMT+09:00 Nicholas Piggin <[e-mail]>:
> I wonder if the names could be a bit more descriptive?
>
> libs-y-liba
> libs-y-builtin
>
> ?

I do not have a strong opinion.  Either way sounds OK to me.
If you like, I can fix it up.


I think you mentioned LD_DEAD_CODE_DATA_ELIMINATION is under way for
ARM and x86.
If we can switch all architectures, lib.a will turn into built-in.o.  Correct?
(I am not sure how difficult the global conversion is.)




BTW, I saw abuse of lib.a in
https://patchwork.kernel.org/patch/9768439/

I see it in linux-next.

commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
Author: Stephen Boyd <[e-mail]>
Date:   Fri Jun 2 15:30:06 2017 -0700

    clk: sunxi-ng: Move all clock types to a library




Now drivers/clk/sunxi-ng/lib.a
will go into thin archives.
The result might be different from what they expect...

Nicholas Piggin June 21, 2017, 4:04 a.m. UTC | #5

On Wed, 21 Jun 2017 12:29:33 +0900
Masahiro Yamada <[e-mail]> wrote:

> Hi Nicholas,
> 
> 
> 2017-06-21 11:47 GMT+09:00 Nicholas Piggin <[e-mail]>:
> > I wonder if the names could be a bit more descriptive?
> >
> > libs-y-liba
> > libs-y-builtin
> >
> > ?  
> 
> I do not have a strong opinion.  Either way sounds OK to me.
> If you like, I can fix it up.

I have nothing against it if you want to clean it up. You're the
maintainer so whatever you prefer :)

> I think you mentioned LD_DEAD_CODE_DATA_ELIMINATION is under way for
> ARM and x86.

Yes it seems to be quite successful. ARM and MIPS people are looking
at it, I have it working for powerpc and x86 quite easily. I will
send patches to arch maintainers if we get these thin archives changes
merged in the next window.

> If we can switch all architectures, lib.a will turn into built-in.o.  Correct?
> (I am not sure how difficult the global conversion is.)

Yes I think it should be no longer necessary.

The only problem with LD_DEAD_CODE_DATA_ELIMINATION is that it overloads
elf section names to pass data to the linker, so it's not 100% transparent.
In practice it's been very minor changes. On balance it will end up being
better than a whole lot of conditional compiling and linking workarounds
such as the one you mention below.

> BTW, I saw abuse of lib.a in
> https://patchwork.kernel.org/patch/9768439/
> 
> I see it in linux-next.
> 
> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> Author: Stephen Boyd <[e-mail]>
> Date:   Fri Jun 2 15:30:06 2017 -0700
> 
>     clk: sunxi-ng: Move all clock types to a library
> 
> 
> 
> 
> Now drivers/clk/sunxi-ng/lib.a
> will go into thin archives.
> The result might be different from what they expect...

Yes I see. With thin archives, that is just going to cause the
same behaviour as built-in.o (everything will be linked). So the
build should not break, but they won't get savings.

Does it even save space with incremental linking? If the lib.a gets
linked into drivers/built-in.o, I wonder what happens then?

Thanks,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 7:15 a.m. UTC | #6

On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 12:29:33 +0900
> Masahiro Yamada <[e-mail]> wrote:

>> BTW, I saw abuse of lib.a in
>> https://patchwork.kernel.org/patch/9768439/
>>
>> I see it in linux-next.
>>
>> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
>> Author: Stephen Boyd <[e-mail]>
>> Date:   Fri Jun 2 15:30:06 2017 -0700
>>
>>     clk: sunxi-ng: Move all clock types to a library
>>
>>
>>
>>
>> Now drivers/clk/sunxi-ng/lib.a
>> will go into thin archives.
>> The result might be different from what they expect...
>
> Yes I see. With thin archives, that is just going to cause the
> same behaviour as built-in.o (everything will be linked). So the
> build should not break, but they won't get savings.
>
> Does it even save space with incremental linking? If the lib.a gets
> linked into drivers/built-in.o, I wonder what happens then?

Ah, too bad. I thought we had found a way to use a library correctly
here, but I just verified that indeed all the just gets linked into built-in.o

I played around with it some more now, but without success: if I
build sunxi-ng as a loadable module (using a few modifications),
then the unneeded objects from lib.a are dropped as I had hoped,
but for built-in code we now always include everything.

I suppose that we can ignore this once we get
LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
until then, we have a code size regression. Any other ideas for
how this could be solved? We used to have a Kconfig symbol
for each file, but those would frequently get out of sync.

         Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Nicholas Piggin June 21, 2017, 9:16 a.m. UTC | #7

On Wed, 21 Jun 2017 09:15:09 +0200
Arnd Bergmann <[e-mail]> wrote:

> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <[e-mail]> wrote:
> > On Wed, 21 Jun 2017 12:29:33 +0900
> > Masahiro Yamada <[e-mail]> wrote:  
> 
> >> BTW, I saw abuse of lib.a in
> >> https://patchwork.kernel.org/patch/9768439/
> >>
> >> I see it in linux-next.
> >>
> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> Author: Stephen Boyd <[e-mail]>
> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >>
> >>     clk: sunxi-ng: Move all clock types to a library
> >>
> >>
> >>
> >>
> >> Now drivers/clk/sunxi-ng/lib.a
> >> will go into thin archives.
> >> The result might be different from what they expect...  
> >
> > Yes I see. With thin archives, that is just going to cause the
> > same behaviour as built-in.o (everything will be linked). So the
> > build should not break, but they won't get savings.
> >
> > Does it even save space with incremental linking? If the lib.a gets
> > linked into drivers/built-in.o, I wonder what happens then?  
> 
> Ah, too bad. I thought we had found a way to use a library correctly
> here, but I just verified that indeed all the just gets linked into built-in.o
> 
> I played around with it some more now, but without success: if I
> build sunxi-ng as a loadable module (using a few modifications),
> then the unneeded objects from lib.a are dropped as I had hoped,
> but for built-in code we now always include everything.
> 
> I suppose that we can ignore this once we get
> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> until then, we have a code size regression.

I didn't follow the thread there, is it a regression caused by
thin archives, or just by removing the Kconfig symbol from each
file?

> Any other ideas for
> how this could be solved? We used to have a Kconfig symbol
> for each file, but those would frequently get out of sync.

Not really sure. Possibly the way to do it would be to plumb
lib-y through drivers/ as well (like obj-y is) so they can be
linked on their own for the final vmlinux link.

But that may be quite a bit of churn, so it may be better to
hold such changes until we see how LD_DEAD_CODE_DATA_ELIMINATION
works out.

Thanks,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 10:21 a.m. UTC | #8

On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 09:15:09 +0200
> Arnd Bergmann <[e-mail]> wrote:
>
>> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <[e-mail]> wrote:
>> > On Wed, 21 Jun 2017 12:29:33 +0900
>> > Masahiro Yamada <[e-mail]> wrote:
>>
>> >> BTW, I saw abuse of lib.a in
>> >> https://patchwork.kernel.org/patch/9768439/
>> >>
>> >> I see it in linux-next.
>> >>
>> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
>> >> Author: Stephen Boyd <[e-mail]>
>> >> Date:   Fri Jun 2 15:30:06 2017 -0700
>> >>
>> >>     clk: sunxi-ng: Move all clock types to a library
>> >>
>> >>
>> >>
>> >>
>> >> Now drivers/clk/sunxi-ng/lib.a
>> >> will go into thin archives.
>> >> The result might be different from what they expect...
>> >
>> > Yes I see. With thin archives, that is just going to cause the
>> > same behaviour as built-in.o (everything will be linked). So the
>> > build should not break, but they won't get savings.
>> >
>> > Does it even save space with incremental linking? If the lib.a gets
>> > linked into drivers/built-in.o, I wonder what happens then?
>>
>> Ah, too bad. I thought we had found a way to use a library correctly
>> here, but I just verified that indeed all the just gets linked into built-in.o
>>
>> I played around with it some more now, but without success: if I
>> build sunxi-ng as a loadable module (using a few modifications),
>> then the unneeded objects from lib.a are dropped as I had hoped,
>> but for built-in code we now always include everything.
>>
>> I suppose that we can ignore this once we get
>> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
>> until then, we have a code size regression.
>
> I didn't follow the thread there, is it a regression caused by
> thin archives, or just by removing the Kconfig symbol from each
> file?

I thought it was the latter, but actually it only happens with thin
archives, so we are fine as long as we enable THIN_ARCHIVES
and LD_DEAD_CODE_DATA_ELIMINATION at the same time
on ARM.

        Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Nicholas Piggin June 21, 2017, 10:38 a.m. UTC | #9

On Wed, 21 Jun 2017 12:21:16 +0200
Arnd Bergmann <[e-mail]> wrote:

> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <[e-mail]> wrote:
> > On Wed, 21 Jun 2017 09:15:09 +0200
> > Arnd Bergmann <[e-mail]> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <[e-mail]> wrote:  
> >> > On Wed, 21 Jun 2017 12:29:33 +0900
> >> > Masahiro Yamada <[e-mail]> wrote:  
> >>  
> >> >> BTW, I saw abuse of lib.a in
> >> >> https://patchwork.kernel.org/patch/9768439/
> >> >>
> >> >> I see it in linux-next.
> >> >>
> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> >> Author: Stephen Boyd <[e-mail]>
> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >> >>
> >> >>     clk: sunxi-ng: Move all clock types to a library
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Now drivers/clk/sunxi-ng/lib.a
> >> >> will go into thin archives.
> >> >> The result might be different from what they expect...  
> >> >
> >> > Yes I see. With thin archives, that is just going to cause the
> >> > same behaviour as built-in.o (everything will be linked). So the
> >> > build should not break, but they won't get savings.
> >> >
> >> > Does it even save space with incremental linking? If the lib.a gets
> >> > linked into drivers/built-in.o, I wonder what happens then?  
> >>
> >> Ah, too bad. I thought we had found a way to use a library correctly
> >> here, but I just verified that indeed all the just gets linked into built-in.o
> >>
> >> I played around with it some more now, but without success: if I
> >> build sunxi-ng as a loadable module (using a few modifications),
> >> then the unneeded objects from lib.a are dropped as I had hoped,
> >> but for built-in code we now always include everything.
> >>
> >> I suppose that we can ignore this once we get
> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> >> until then, we have a code size regression.  
> >
> > I didn't follow the thread there, is it a regression caused by
> > thin archives, or just by removing the Kconfig symbol from each
> > file?  
> 
> I thought it was the latter, but actually it only happens with thin
> archives, 

Is this including these changes now in the kbuild tree?

I can take a look at ARM and try to get it at least to parity with
incremental link. Any particular config options required?

> so we are fine as long as we enable THIN_ARCHIVES
> and LD_DEAD_CODE_DATA_ELIMINATION at the same time
> on ARM.

Well the current proposal is to unconditionally enable it for all archs
for 4.13. After that I'll submit patches to x86 and powerpc arch
maintainers to allow LD_DEAD_CODE_DATA_ELIMINATION as an option. I guess
you will do ARM and there have been MIPS guys looking at it too.

That leaves a window of one release. ARM could unselect thin archives if
necessary but I think it would be much better to enable it and flush out
any toolchain and build issues before doing the LD_DCDE option. Disabling
should be a last resort if we can't fix something in time for release.

Thanks,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 10:49 a.m. UTC | #10

On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 12:21:16 +0200
> Arnd Bergmann <[e-mail]> wrote:
>
>> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <[e-mail]> wrote:
>> > On Wed, 21 Jun 2017 09:15:09 +0200
>> > Arnd Bergmann <[e-mail]> wrote:
>> >
>> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <[e-mail]> wrote:
>> >> > On Wed, 21 Jun 2017 12:29:33 +0900
>> >> > Masahiro Yamada <[e-mail]> wrote:
>> >>
>> >> >> BTW, I saw abuse of lib.a in
>> >> >> https://patchwork.kernel.org/patch/9768439/
>> >> >>
>> >> >> I see it in linux-next.
>> >> >>
>> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
>> >> >> Author: Stephen Boyd <[e-mail]>
>> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
>> >> >>
>> >> >>     clk: sunxi-ng: Move all clock types to a library
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> Now drivers/clk/sunxi-ng/lib.a
>> >> >> will go into thin archives.
>> >> >> The result might be different from what they expect...
>> >> >
>> >> > Yes I see. With thin archives, that is just going to cause the
>> >> > same behaviour as built-in.o (everything will be linked). So the
>> >> > build should not break, but they won't get savings.
>> >> >
>> >> > Does it even save space with incremental linking? If the lib.a gets
>> >> > linked into drivers/built-in.o, I wonder what happens then?
>> >>
>> >> Ah, too bad. I thought we had found a way to use a library correctly
>> >> here, but I just verified that indeed all the just gets linked into built-in.o
>> >>
>> >> I played around with it some more now, but without success: if I
>> >> build sunxi-ng as a loadable module (using a few modifications),
>> >> then the unneeded objects from lib.a are dropped as I had hoped,
>> >> but for built-in code we now always include everything.
>> >>
>> >> I suppose that we can ignore this once we get
>> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
>> >> until then, we have a code size regression.
>> >
>> > I didn't follow the thread there, is it a regression caused by
>> > thin archives, or just by removing the Kconfig symbol from each
>> > file?
>>
>> I thought it was the latter, but actually it only happens with thin
>> archives,
>
> Is this including these changes now in the kbuild tree?

I'm building on top of yesterday's linux-next at the moment,
with a number of my own patches applied

> I can take a look at ARM and try to get it at least to parity with
> incremental link. Any particular config options required?

This is the patch I am testing with:

https://pastebin.com/HQuhCEmK

I have not looked at that in a while, no idea if it works, or
if it has known problems.

I last posted the patch in March for discussion:

https://patchwork.kernel.org/patch/9626207/

>> so we are fine as long as we enable THIN_ARCHIVES
>> and LD_DEAD_CODE_DATA_ELIMINATION at the same time
>> on ARM.
>
> Well the current proposal is to unconditionally enable it for all archs
> for 4.13. After that I'll submit patches to x86 and powerpc arch
> maintainers to allow LD_DEAD_CODE_DATA_ELIMINATION as an option. I guess
> you will do ARM and there have been MIPS guys looking at it too.
>
> That leaves a window of one release. ARM could unselect thin archives if
> necessary but I think it would be much better to enable it and flush out
> any toolchain and build issues before doing the LD_DCDE option. Disabling
> should be a last resort if we can't fix something in time for release.

I see.

       Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 10:51 a.m. UTC | #11

On Wed, Jun 21, 2017 at 12:49 PM, Arnd Bergmann <[e-mail]> wrote:
> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
>> On Wed, 21 Jun 2017 12:21:16 +0200
>> Arnd Bergmann <[e-mail]> wrote:

>>> so we are fine as long as we enable THIN_ARCHIVES
>>> and LD_DEAD_CODE_DATA_ELIMINATION at the same time
>>> on ARM.
>>
>> Well the current proposal is to unconditionally enable it for all archs
>> for 4.13. After that I'll submit patches to x86 and powerpc arch
>> maintainers to allow LD_DEAD_CODE_DATA_ELIMINATION as an option. I guess
>> you will do ARM and there have been MIPS guys looking at it too.
>>
>> That leaves a window of one release. ARM could unselect thin archives if
>> necessary but I think it would be much better to enable it and flush out
>> any toolchain and build issues before doing the LD_DCDE option. Disabling
>> should be a last resort if we can't fix something in time for release.
>
> I see.

[hit 'send' too early]

We can always revert 06e226c7fb23 ("clk: sunxi-ng: Move all clock types to a
library"), it's also scheduled for 4.13 and has not been in a release yet.

       Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Nicholas Piggin June 21, 2017, 11:10 a.m. UTC | #12

On Wed, 21 Jun 2017 12:49:10 +0200
Arnd Bergmann <[e-mail]> wrote:

> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
> > On Wed, 21 Jun 2017 12:21:16 +0200
> > Arnd Bergmann <[e-mail]> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 11:16 AM, Nicholas Piggin <[e-mail]> wrote:  
> >> > On Wed, 21 Jun 2017 09:15:09 +0200
> >> > Arnd Bergmann <[e-mail]> wrote:
> >> >  
> >> >> On Wed, Jun 21, 2017 at 6:04 AM, Nicholas Piggin <[e-mail]> wrote:  
> >> >> > On Wed, 21 Jun 2017 12:29:33 +0900
> >> >> > Masahiro Yamada <[e-mail]> wrote:  
> >> >>  
> >> >> >> BTW, I saw abuse of lib.a in
> >> >> >> https://patchwork.kernel.org/patch/9768439/
> >> >> >>
> >> >> >> I see it in linux-next.
> >> >> >>
> >> >> >> commit 06e226c7fb233f676b01b144d0b321ebe510fdcd
> >> >> >> Author: Stephen Boyd <[e-mail]>
> >> >> >> Date:   Fri Jun 2 15:30:06 2017 -0700
> >> >> >>
> >> >> >>     clk: sunxi-ng: Move all clock types to a library
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Now drivers/clk/sunxi-ng/lib.a
> >> >> >> will go into thin archives.
> >> >> >> The result might be different from what they expect...  
> >> >> >
> >> >> > Yes I see. With thin archives, that is just going to cause the
> >> >> > same behaviour as built-in.o (everything will be linked). So the
> >> >> > build should not break, but they won't get savings.
> >> >> >
> >> >> > Does it even save space with incremental linking? If the lib.a gets
> >> >> > linked into drivers/built-in.o, I wonder what happens then?  
> >> >>
> >> >> Ah, too bad. I thought we had found a way to use a library correctly
> >> >> here, but I just verified that indeed all the just gets linked into built-in.o
> >> >>
> >> >> I played around with it some more now, but without success: if I
> >> >> build sunxi-ng as a loadable module (using a few modifications),
> >> >> then the unneeded objects from lib.a are dropped as I had hoped,
> >> >> but for built-in code we now always include everything.
> >> >>
> >> >> I suppose that we can ignore this once we get
> >> >> LD_DEAD_CODE_DATA_ELIMINATION enabled on ARM, but
> >> >> until then, we have a code size regression.  
> >> >
> >> > I didn't follow the thread there, is it a regression caused by
> >> > thin archives, or just by removing the Kconfig symbol from each
> >> > file?  
> >>
> >> I thought it was the latter, but actually it only happens with thin
> >> archives,  
> >
> > Is this including these changes now in the kbuild tree?  
> 
> I'm building on top of yesterday's linux-next at the moment,
> with a number of my own patches applied
> 
> > I can take a look at ARM and try to get it at least to parity with
> > incremental link. Any particular config options required?  
> 
> This is the patch I am testing with:
> 
> https://pastebin.com/HQuhCEmK
> I have not looked at that in a while, no idea if it works, or
> if it has known problems.
> 
> I last posted the patch in March for discussion:
> 
> https://patchwork.kernel.org/patch/9626207/

Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
Just want to try getting thin archives up to a point where there are
no serious regressions first.

Thanks,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 11:32 a.m. UTC | #13

On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 12:49:10 +0200
> Arnd Bergmann <[e-mail]> wrote:
>
>> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
>> > On Wed, 21 Jun 2017 12:21:16 +0200
>> > Arnd Bergmann <[e-mail]> wrote:
>> >
>>
>> I'm building on top of yesterday's linux-next at the moment,
>> with a number of my own patches applied
>>
>> > I can take a look at ARM and try to get it at least to parity with
>> > incremental link. Any particular config options required?
>>
>> This is the patch I am testing with:
>>
>> https://pastebin.com/HQuhCEmK
>> I have not looked at that in a while, no idea if it works, or
>> if it has known problems.
>>
>> I last posted the patch in March for discussion:
>>
>> https://patchwork.kernel.org/patch/9626207/
>
> Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> Just want to try getting thin archives up to a point where there are
> no serious regressions first.

For my build testing, I have now reverted most of my patch and only
left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.

         Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Nicholas Piggin June 21, 2017, 12:02 p.m. UTC | #14

On Wed, 21 Jun 2017 13:32:17 +0200
Arnd Bergmann <[e-mail]> wrote:

> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:
> > On Wed, 21 Jun 2017 12:49:10 +0200
> > Arnd Bergmann <[e-mail]> wrote:
> >  
> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:  
> >> > On Wed, 21 Jun 2017 12:21:16 +0200
> >> > Arnd Bergmann <[e-mail]> wrote:
> >> >  
> >>
> >> I'm building on top of yesterday's linux-next at the moment,
> >> with a number of my own patches applied
> >>  
> >> > I can take a look at ARM and try to get it at least to parity with
> >> > incremental link. Any particular config options required?  
> >>
> >> This is the patch I am testing with:
> >>
> >> https://pastebin.com/HQuhCEmK
> >> I have not looked at that in a while, no idea if it works, or
> >> if it has known problems.
> >>
> >> I last posted the patch in March for discussion:
> >>
> >> https://patchwork.kernel.org/patch/9626207/  
> >
> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> > Just want to try getting thin archives up to a point where there are
> > no serious regressions first.  
> 
> For my build testing, I have now reverted most of my patch and only
> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.

Okay. Currently with the patches in the kbuild tree, thin archives
actually does better than incremental linking for ARM (defconfig).

11651574	6090952	 418616	18161142	1151df6	vmlinux.thinarc
11656118	6095208	 418952	18170278	11541a6	vmlinux.inc

In mainline, thin archives is sub-optimal for lib-y libraries (just
pulls them all in). But now that I test it, even that is better than
incremental linking for ARM:

11653926	6090888	 418620	18163434	11526ea	vmlinux.thinarc

I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
patch yet to check. What tree is it in? I wonder if I could convince you
to drop that for now and bring it up on linux-kernel/linux-kbuild? I
think it's quite reasonable to reduce Kconfig hell of very fine grained
conditional compilation/linking if we can make the toolchain do it for
us. So we should consider how to support it nicely rather than having
such hacks IMO.

Thanks,
Nick

--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 12:21 p.m. UTC | #15

On Wed, Jun 21, 2017 at 2:02 PM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 13:32:17 +0200
> Arnd Bergmann <[e-mail]> wrote:
>
>> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:
>> > On Wed, 21 Jun 2017 12:49:10 +0200
>> > Arnd Bergmann <[e-mail]> wrote:
>> >
>> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
>> >> > On Wed, 21 Jun 2017 12:21:16 +0200
>> >> > Arnd Bergmann <[e-mail]> wrote:
>> >> >
>> >>
>> >> I'm building on top of yesterday's linux-next at the moment,
>> >> with a number of my own patches applied
>> >>
>> >> > I can take a look at ARM and try to get it at least to parity with
>> >> > incremental link. Any particular config options required?
>> >>
>> >> This is the patch I am testing with:
>> >>
>> >> https://pastebin.com/HQuhCEmK
>> >> I have not looked at that in a while, no idea if it works, or
>> >> if it has known problems.
>> >>
>> >> I last posted the patch in March for discussion:
>> >>
>> >> https://patchwork.kernel.org/patch/9626207/
>> >
>> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
>> > Just want to try getting thin archives up to a point where there are
>> > no serious regressions first.
>>
>> For my build testing, I have now reverted most of my patch and only
>> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.
>
> Okay. Currently with the patches in the kbuild tree, thin archives
> actually does better than incremental linking for ARM (defconfig).
>
> 11651574        6090952  418616 18161142        1151df6 vmlinux.thinarc
> 11656118        6095208  418952 18170278        11541a6 vmlinux.inc
>
> In mainline, thin archives is sub-optimal for lib-y libraries (just
> pulls them all in). But now that I test it, even that is better than
> incremental linking for ARM:
>
> 11653926        6090888  418620 18163434        11526ea vmlinux.thinarc
>
> I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
> patch yet to check. What tree is it in? I wonder if I could convince you
> to drop that for now and bring it up on linux-kernel/linux-kbuild? I
> think it's quite reasonable to reduce Kconfig hell of very fine grained
> conditional compilation/linking if we can make the toolchain do it for
> us. So we should consider how to support it nicely rather than having
> such hacks IMO.

[adding Maxime and Stephen to cc]

Stephen,

The hack has come to bite us after all, with Nick's proposed change to
CONFIG_THIN_ARCHIVES for all architectures, the entire lib.a file will
end up being linked into each kernel that enables the directory.

I'm out of other ideas for fixing it, so it seems better to revert back to
individual Kconfig options. Once we enable
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (which is the
next step after CONFIG_THIN_ARCHIVES), we can simply list
all files, and the linker will drop all unused functions by itself.

       Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Stephen Boyd June 21, 2017, 4:19 p.m. UTC | #16

On 06/21, Arnd Bergmann wrote:
> On Wed, Jun 21, 2017 at 2:02 PM, Nicholas Piggin <[e-mail]> wrote:
> > On Wed, 21 Jun 2017 13:32:17 +0200
> > Arnd Bergmann <[e-mail]> wrote:
> >
> >> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:
> >> > On Wed, 21 Jun 2017 12:49:10 +0200
> >> > Arnd Bergmann <[e-mail]> wrote:
> >> >
> >> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
> >> >> > On Wed, 21 Jun 2017 12:21:16 +0200
> >> >> > Arnd Bergmann <[e-mail]> wrote:
> >> >> >
> >> >>
> >> >> I'm building on top of yesterday's linux-next at the moment,
> >> >> with a number of my own patches applied
> >> >>
> >> >> > I can take a look at ARM and try to get it at least to parity with
> >> >> > incremental link. Any particular config options required?
> >> >>
> >> >> This is the patch I am testing with:
> >> >>
> >> >> https://pastebin.com/HQuhCEmK
> >> >> I have not looked at that in a while, no idea if it works, or
> >> >> if it has known problems.
> >> >>
> >> >> I last posted the patch in March for discussion:
> >> >>
> >> >> https://patchwork.kernel.org/patch/9626207/
> >> >
> >> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> >> > Just want to try getting thin archives up to a point where there are
> >> > no serious regressions first.
> >>
> >> For my build testing, I have now reverted most of my patch and only
> >> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.
> >
> > Okay. Currently with the patches in the kbuild tree, thin archives
> > actually does better than incremental linking for ARM (defconfig).
> >
> > 11651574        6090952  418616 18161142        1151df6 vmlinux.thinarc
> > 11656118        6095208  418952 18170278        11541a6 vmlinux.inc
> >
> > In mainline, thin archives is sub-optimal for lib-y libraries (just
> > pulls them all in). But now that I test it, even that is better than
> > incremental linking for ARM:
> >
> > 11653926        6090888  418620 18163434        11526ea vmlinux.thinarc
> >
> > I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
> > patch yet to check. What tree is it in? I wonder if I could convince you
> > to drop that for now and bring it up on linux-kernel/linux-kbuild? I
> > think it's quite reasonable to reduce Kconfig hell of very fine grained
> > conditional compilation/linking if we can make the toolchain do it for
> > us. So we should consider how to support it nicely rather than having
> > such hacks IMO.
> 
> [adding Maxime and Stephen to cc]
> 
> Stephen,
> 
> The hack has come to bite us after all, with Nick's proposed change to
> CONFIG_THIN_ARCHIVES for all architectures, the entire lib.a file will
> end up being linked into each kernel that enables the directory.
> 
> I'm out of other ideas for fixing it, so it seems better to revert back to
> individual Kconfig options. Once we enable
> CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (which is the
> next step after CONFIG_THIN_ARCHIVES), we can simply list
> all files, and the linker will drop all unused functions by itself.
> 

Ok. Can you send a revert patch to the list with some information
on why we can't do the hack anymore and also Cc lkml/kbuild
lists? The commit is in linux-next now as commit 06e226c7fb23
(clk: sunxi-ng: Move all clock types to a library, 2017-06-02).

Nicholas Piggin June 21, 2017, 6:08 p.m. UTC | #17

On Wed, 21 Jun 2017 09:19:13 -0700
Stephen Boyd <[e-mail]> wrote:

> On 06/21, Arnd Bergmann wrote:
> > On Wed, Jun 21, 2017 at 2:02 PM, Nicholas Piggin <[e-mail]> wrote:  
> > > On Wed, 21 Jun 2017 13:32:17 +0200
> > > Arnd Bergmann <[e-mail]> wrote:
> > >  
> > >> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:  
> > >> > On Wed, 21 Jun 2017 12:49:10 +0200
> > >> > Arnd Bergmann <[e-mail]> wrote:
> > >> >  
> > >> >> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:  
> > >> >> > On Wed, 21 Jun 2017 12:21:16 +0200
> > >> >> > Arnd Bergmann <[e-mail]> wrote:
> > >> >> >  
> > >> >>
> > >> >> I'm building on top of yesterday's linux-next at the moment,
> > >> >> with a number of my own patches applied
> > >> >>  
> > >> >> > I can take a look at ARM and try to get it at least to parity with
> > >> >> > incremental link. Any particular config options required?  
> > >> >>
> > >> >> This is the patch I am testing with:
> > >> >>
> > >> >> https://pastebin.com/HQuhCEmK
> > >> >> I have not looked at that in a while, no idea if it works, or
> > >> >> if it has known problems.
> > >> >>
> > >> >> I last posted the patch in March for discussion:
> > >> >>
> > >> >> https://patchwork.kernel.org/patch/9626207/  
> > >> >
> > >> > Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> > >> > Just want to try getting thin archives up to a point where there are
> > >> > no serious regressions first.  
> > >>
> > >> For my build testing, I have now reverted most of my patch and only
> > >> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.  
> > >
> > > Okay. Currently with the patches in the kbuild tree, thin archives
> > > actually does better than incremental linking for ARM (defconfig).
> > >
> > > 11651574        6090952  418616 18161142        1151df6 vmlinux.thinarc
> > > 11656118        6095208  418952 18170278        11541a6 vmlinux.inc
> > >
> > > In mainline, thin archives is sub-optimal for lib-y libraries (just
> > > pulls them all in). But now that I test it, even that is better than
> > > incremental linking for ARM:
> > >
> > > 11653926        6090888  418620 18163434        11526ea vmlinux.thinarc
> > >
> > > I haven't grabbed that "clk: sunxi-ng: Move all clock types to a library"
> > > patch yet to check. What tree is it in? I wonder if I could convince you
> > > to drop that for now and bring it up on linux-kernel/linux-kbuild? I
> > > think it's quite reasonable to reduce Kconfig hell of very fine grained
> > > conditional compilation/linking if we can make the toolchain do it for
> > > us. So we should consider how to support it nicely rather than having
> > > such hacks IMO.  
> > 
> > [adding Maxime and Stephen to cc]
> > 
> > Stephen,
> > 
> > The hack has come to bite us after all, with Nick's proposed change to
> > CONFIG_THIN_ARCHIVES for all architectures, the entire lib.a file will
> > end up being linked into each kernel that enables the directory.
> > 
> > I'm out of other ideas for fixing it, so it seems better to revert back to
> > individual Kconfig options. Once we enable
> > CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (which is the
> > next step after CONFIG_THIN_ARCHIVES), we can simply list
> > all files, and the linker will drop all unused functions by itself.
> >   
> 
> Ok. Can you send a revert patch to the list with some information
> on why we can't do the hack anymore and also Cc lkml/kbuild
> lists? The commit is in linux-next now as commit 06e226c7fb23
> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> 

I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
with thin archives enabled and disabled with arm defconfig which ends up
setting CONFIG_SUNXI_CCU=y.

The patch makes no difference to vmlinux size whether using traditional
incremental link, or thin archives (there is a small difference between
inclink and thinarc but that's unrelated).

So this thin archives change does not break your patch, it's just that
it doesn't really do the right thing. I like the general idea, but we
need to work out how to make it properly supported by the build system.

With the patch applied, this is what the build system currently does:

arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o

This incremental link pulls in all your lib.a objects and links them
into built-in.o. They can no longer be selectively linked.

Thanks,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 8:52 p.m. UTC | #18

On Wed, Jun 21, 2017 at 1:32 PM, Arnd Bergmann <[e-mail]> wrote:
> On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:
>> On Wed, 21 Jun 2017 12:49:10 +0200
>> Arnd Bergmann <[e-mail]> wrote:
>>
>>> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:
>>> > On Wed, 21 Jun 2017 12:21:16 +0200
>>> > Arnd Bergmann <[e-mail]> wrote:
>>> >
>>>
>>> I'm building on top of yesterday's linux-next at the moment,
>>> with a number of my own patches applied
>>>
>>> > I can take a look at ARM and try to get it at least to parity with
>>> > incremental link. Any particular config options required?
>>>
>>> This is the patch I am testing with:
>>>
>>> https://pastebin.com/HQuhCEmK
>>> I have not looked at that in a while, no idea if it works, or
>>> if it has known problems.
>>>
>>> I last posted the patch in March for discussion:
>>>
>>> https://patchwork.kernel.org/patch/9626207/
>>
>> Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
>> Just want to try getting thin archives up to a point where there are
>> no serious regressions first.
>
> For my build testing, I have now reverted most of my patch and only
> left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.

I just got one build failure on a randconfig build with THIN_ARCHIVES
but without LD_DCDE:

/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .data VMA
[0000000000808000,000000000096cc7f] overlaps section .text VMA
[0000000000080080,00000000008a8427]
/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .rodata VMA
[00000000008a9000,0000000000ea216f] overlaps section .data VMA
[0000000000808000,000000000096cc7f]
/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .bss VMA
[000000000096cd00,000000000157896f] overlaps section .rodata VMA
[00000000008a9000,0000000000ea216f]
/home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section __param VMA
[0000000000ea2170,0000000000ea5257] overlaps section .bss VMA
[000000000096cd00,000000000157896f]

haven't spent any time analyzing it further, maybe you can see right
away what caused this. If not, you could try the .config file at
https://pastebin.com/raw/YTZKH9Xe

        Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 8:55 p.m. UTC | #19

On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 09:19:13 -0700
> Stephen Boyd <[e-mail]> wrote:
>> On 06/21, Arnd Bergmann wrote:
>>
>> Ok. Can you send a revert patch to the list with some information
>> on why we can't do the hack anymore and also Cc lkml/kbuild
>> lists? The commit is in linux-next now as commit 06e226c7fb23
>> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
>>
>
> I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> with thin archives enabled and disabled with arm defconfig which ends up
> setting CONFIG_SUNXI_CCU=y.
>
> The patch makes no difference to vmlinux size whether using traditional
> incremental link, or thin archives (there is a small difference between
> inclink and thinarc but that's unrelated).
>
> So this thin archives change does not break your patch, it's just that
> it doesn't really do the right thing. I like the general idea, but we
> need to work out how to make it properly supported by the build system.
>
> With the patch applied, this is what the build system currently does:
>
> arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
>
> This incremental link pulls in all your lib.a objects and links them
> into built-in.o. They can no longer be selectively linked.

I think the ARM defconfig actually needs all those objects because it
enables all the high-level drivers. You could try disabling e.g. all except
CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
unused.

       Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Nicholas Piggin June 21, 2017, 9:30 p.m. UTC | #20

On Wed, 21 Jun 2017 22:52:25 +0200
Arnd Bergmann <[e-mail]> wrote:

> On Wed, Jun 21, 2017 at 1:32 PM, Arnd Bergmann <[e-mail]> wrote:
> > On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:  
> >> On Wed, 21 Jun 2017 12:49:10 +0200
> >> Arnd Bergmann <[e-mail]> wrote:
> >>  
> >>> On Wed, Jun 21, 2017 at 12:38 PM, Nicholas Piggin <[e-mail]> wrote:  
> >>> > On Wed, 21 Jun 2017 12:21:16 +0200
> >>> > Arnd Bergmann <[e-mail]> wrote:
> >>> >  
> >>>
> >>> I'm building on top of yesterday's linux-next at the moment,
> >>> with a number of my own patches applied
> >>>  
> >>> > I can take a look at ARM and try to get it at least to parity with
> >>> > incremental link. Any particular config options required?  
> >>>
> >>> This is the patch I am testing with:
> >>>
> >>> https://pastebin.com/HQuhCEmK
> >>> I have not looked at that in a while, no idea if it works, or
> >>> if it has known problems.
> >>>
> >>> I last posted the patch in March for discussion:
> >>>
> >>> https://patchwork.kernel.org/patch/9626207/  
> >>
> >> Well I just mean the stuff now in kbuild/thin-ac, not the LD_DCDE.
> >> Just want to try getting thin archives up to a point where there are
> >> no serious regressions first.  
> >
> > For my build testing, I have now reverted most of my patch and only
> > left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.  
> 
> I just got one build failure on a randconfig build with THIN_ARCHIVES
> but without LD_DCDE:
> 
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .data VMA
> [0000000000808000,000000000096cc7f] overlaps section .text VMA
> [0000000000080080,00000000008a8427]
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .rodata VMA
> [00000000008a9000,0000000000ea216f] overlaps section .data VMA
> [0000000000808000,000000000096cc7f]
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .bss VMA
> [000000000096cd00,000000000157896f] overlaps section .rodata VMA
> [00000000008a9000,0000000000ea216f]
> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section __param VMA
> [0000000000ea2170,0000000000ea5257] overlaps section .bss VMA
> [000000000096cd00,000000000157896f]
> 
> haven't spent any time analyzing it further, maybe you can see right
> away what caused this. If not, you could try the .config file at
> https://pastebin.com/raw/YTZKH9Xe

It's due to vmlinux-xip.lds.S explicitly putting sections inside each
other

        . = PAGE_OFFSET + TEXT_OFFSET;

Same config fails without thin archives for me.

If XIP can't place these dynamically, at least you should use an
ASSERT() linker script function if sizes overlap. E.g.,

    ASSERT(. <= PAGE_OFFSET + TEXT_OFFSET, "kernel too big");
    . = PAGE_OFFSET + TEXT_OFFSET;

--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arnd Bergmann June 21, 2017, 9:44 p.m. UTC | #21

On Wed, Jun 21, 2017 at 11:30 PM, Nicholas Piggin <[e-mail]> wrote:
> On Wed, 21 Jun 2017 22:52:25 +0200
> Arnd Bergmann <[e-mail]> wrote:
>
>> On Wed, Jun 21, 2017 at 1:32 PM, Arnd Bergmann <[e-mail]> wrote:
>> > On Wed, Jun 21, 2017 at 1:10 PM, Nicholas Piggin <[e-mail]> wrote:
>> >> On Wed, 21 Jun 2017 12:49:10 +0200
>> > For my build testing, I have now reverted most of my patch and only
>> > left the 'select THIN_ARCHIVES'. I'll let you know if I run into problems.
>>
>> I just got one build failure on a randconfig build with THIN_ARCHIVES
>> but without LD_DCDE:
>>
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .data VMA
>> [0000000000808000,000000000096cc7f] overlaps section .text VMA
>> [0000000000080080,00000000008a8427]
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .rodata VMA
>> [00000000008a9000,0000000000ea216f] overlaps section .data VMA
>> [0000000000808000,000000000096cc7f]
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section .bss VMA
>> [000000000096cd00,000000000157896f] overlaps section .rodata VMA
>> [00000000008a9000,0000000000ea216f]
>> /home/arnd/cross-gcc/bin/arm-linux-gnueabi-ld: section __param VMA
>> [0000000000ea2170,0000000000ea5257] overlaps section .bss VMA
>> [000000000096cd00,000000000157896f]
>>
>> haven't spent any time analyzing it further, maybe you can see right
>> away what caused this. If not, you could try the .config file at
>> https://pastebin.com/raw/YTZKH9Xe
>
> It's due to vmlinux-xip.lds.S explicitly putting sections inside each
> other
>
>         . = PAGE_OFFSET + TEXT_OFFSET;
>
> Same config fails without thin archives for me.
>
> If XIP can't place these dynamically, at least you should use an
> ASSERT() linker script function if sizes overlap. E.g.,
>
>     ASSERT(. <= PAGE_OFFSET + TEXT_OFFSET, "kernel too big");
>     . = PAGE_OFFSET + TEXT_OFFSET;
>

Right, makes sense. The linker error actually shows this quite clearly,
though I wonder why I never saw that with my LD_DCDE patch.
It's possible that the sections were always small enough in that
case, but there is probably something else going on as well.

I'd like to end up with a version that lets me build randconfig kernels
without warnings or errors (probably keeping LD_DCDE enabled
is sufficient). I have to think about that some more, but for now
this doesn't seem like a reason against doing THIN_ARCHIVES
in mainline for ARM.

       Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Maxime Ripard June 22, 2017, 6:18 a.m. UTC | #22

On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
> On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <[e-mail]> wrote:
> > On Wed, 21 Jun 2017 09:19:13 -0700
> > Stephen Boyd <[e-mail]> wrote:
> >> On 06/21, Arnd Bergmann wrote:
> >>
> >> Ok. Can you send a revert patch to the list with some information
> >> on why we can't do the hack anymore and also Cc lkml/kbuild
> >> lists? The commit is in linux-next now as commit 06e226c7fb23
> >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> >>
> >
> > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> > with thin archives enabled and disabled with arm defconfig which ends up
> > setting CONFIG_SUNXI_CCU=y.
> >
> > The patch makes no difference to vmlinux size whether using traditional
> > incremental link, or thin archives (there is a small difference between
> > inclink and thinarc but that's unrelated).
> >
> > So this thin archives change does not break your patch, it's just that
> > it doesn't really do the right thing. I like the general idea, but we
> > need to work out how to make it properly supported by the build system.
> >
> > With the patch applied, this is what the build system currently does:
> >
> > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
> >
> > This incremental link pulls in all your lib.a objects and links them
> > into built-in.o. They can no longer be selectively linked.
> 
> I think the ARM defconfig actually needs all those objects because it
> enables all the high-level drivers. You could try disabling e.g. all except
> CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
> unused.

You can also disable MACH_SUN8I, it should disable a significant
number of the clocks driver.

Maxime

Nicholas Piggin June 22, 2017, 3:50 p.m. UTC | #23

On Thu, 22 Jun 2017 08:18:38 +0200
Maxime Ripard <[e-mail]> wrote:

> On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
> > On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <[e-mail]> wrote:  
> > > On Wed, 21 Jun 2017 09:19:13 -0700
> > > Stephen Boyd <[e-mail]> wrote:  
> > >> On 06/21, Arnd Bergmann wrote:
> > >>
> > >> Ok. Can you send a revert patch to the list with some information
> > >> on why we can't do the hack anymore and also Cc lkml/kbuild
> > >> lists? The commit is in linux-next now as commit 06e226c7fb23
> > >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> > >>  
> > >
> > > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> > > with thin archives enabled and disabled with arm defconfig which ends up
> > > setting CONFIG_SUNXI_CCU=y.
> > >
> > > The patch makes no difference to vmlinux size whether using traditional
> > > incremental link, or thin archives (there is a small difference between
> > > inclink and thinarc but that's unrelated).
> > >
> > > So this thin archives change does not break your patch, it's just that
> > > it doesn't really do the right thing. I like the general idea, but we
> > > need to work out how to make it properly supported by the build system.
> > >
> > > With the patch applied, this is what the build system currently does:
> > >
> > > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
> > >
> > > This incremental link pulls in all your lib.a objects and links them
> > > into built-in.o. They can no longer be selectively linked.  
> > 
> > I think the ARM defconfig actually needs all those objects because it
> > enables all the high-level drivers. You could try disabling e.g. all except
> > CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
> > unused.  
> 
> You can also disable MACH_SUN8I, it should disable a significant
> number of the clocks driver.

   text	   data	    bss	    dec	    hex	filename
11655078	6018736	 418944	18092758	11412d6	vmlinux.inclink
11655078	6018736	 418944	18092758	11412d6	vmlinux.inclink.clk
11650534	6010288	 418616	18079438	113dece	vmlinux.thinarc
11650534	6010288	 418616	18079438	113dece	vmlinux.thinarc.clk

Same result for me when compiling defconfig minus MACH_SUN8I, comparing
+/- the clock patch and thin archives. No size savings. As I explained
already, I really can't see how there could be any saving if the lib.a is
incrementally linked into built-in.o.

--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Masahiro Yamada June 23, 2017, 5:31 a.m. UTC | #24

Hi Nicholas,

2017-06-23 0:50 GMT+09:00 Nicholas Piggin <[e-mail]>:
> On Thu, 22 Jun 2017 08:18:38 +0200
> Maxime Ripard <[e-mail]> wrote:
>
>> On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
>> > On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <[e-mail]> wrote:
>> > > On Wed, 21 Jun 2017 09:19:13 -0700
>> > > Stephen Boyd <[e-mail]> wrote:
>> > >> On 06/21, Arnd Bergmann wrote:
>> > >>
>> > >> Ok. Can you send a revert patch to the list with some information
>> > >> on why we can't do the hack anymore and also Cc lkml/kbuild
>> > >> lists? The commit is in linux-next now as commit 06e226c7fb23
>> > >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
>> > >>
>> > >
>> > > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
>> > > with thin archives enabled and disabled with arm defconfig which ends up
>> > > setting CONFIG_SUNXI_CCU=y.
>> > >
>> > > The patch makes no difference to vmlinux size whether using traditional
>> > > incremental link, or thin archives (there is a small difference between
>> > > inclink and thinarc but that's unrelated).
>> > >
>> > > So this thin archives change does not break your patch, it's just that
>> > > it doesn't really do the right thing. I like the general idea, but we
>> > > need to work out how to make it properly supported by the build system.
>> > >
>> > > With the patch applied, this is what the build system currently does:
>> > >
>> > > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
>> > >
>> > > This incremental link pulls in all your lib.a objects and links them
>> > > into built-in.o. They can no longer be selectively linked.
>> >
>> > I think the ARM defconfig actually needs all those objects because it
>> > enables all the high-level drivers. You could try disabling e.g. all except
>> > CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
>> > unused.
>>
>> You can also disable MACH_SUN8I, it should disable a significant
>> number of the clocks driver.
>
>    text    data     bss     dec     hex filename
> 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink
> 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink.clk
> 11650534        6010288  418616 18079438        113dece vmlinux.thinarc
> 11650534        6010288  418616 18079438        113dece vmlinux.thinarc.clk
>
> Same result for me when compiling defconfig minus MACH_SUN8I, comparing
> +/- the clock patch and thin archives. No size savings. As I explained
> already, I really can't see how there could be any saving if the lib.a is
> incrementally linked into built-in.o.

I think defconfig minus MACH_SUN8I
will give no difference to CONFIG_SUNXI_CCU*

As far as I tried,  minus MACH_SUN8I still enabled the following:
CONFIG_SUNXI_CCU=y
CONFIG_SUNXI_CCU_DIV=y
CONFIG_SUNXI_CCU_FRAC=y
CONFIG_SUNXI_CCU_GATE=y
CONFIG_SUNXI_CCU_MUX=y
CONFIG_SUNXI_CCU_MULT=y
CONFIG_SUNXI_CCU_PHASE=y
CONFIG_SUNXI_CCU_NK=y
CONFIG_SUNXI_CCU_NKM=y
CONFIG_SUNXI_CCU_NKMP=y
CONFIG_SUNXI_CCU_NM=y
CONFIG_SUNXI_CCU_MP=y

I think you need to disable some more MACH_SUN*I and
perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
then you will see difference in the result.

Maxime Ripard June 23, 2017, 2:57 p.m. UTC | #25

On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
> Hi Nicholas,
> 
> 2017-06-23 0:50 GMT+09:00 Nicholas Piggin <[e-mail]>:
> > On Thu, 22 Jun 2017 08:18:38 +0200
> > Maxime Ripard <[e-mail]> wrote:
> >
> >> On Wed, Jun 21, 2017 at 10:55:06PM +0200, Arnd Bergmann wrote:
> >> > On Wed, Jun 21, 2017 at 8:08 PM, Nicholas Piggin <[e-mail]> wrote:
> >> > > On Wed, 21 Jun 2017 09:19:13 -0700
> >> > > Stephen Boyd <[e-mail]> wrote:
> >> > >> On 06/21, Arnd Bergmann wrote:
> >> > >>
> >> > >> Ok. Can you send a revert patch to the list with some information
> >> > >> on why we can't do the hack anymore and also Cc lkml/kbuild
> >> > >> lists? The commit is in linux-next now as commit 06e226c7fb23
> >> > >> (clk: sunxi-ng: Move all clock types to a library, 2017-06-02).
> >> > >>
> >> > >
> >> > > I grabbed this patch and applied it to the kbuid/thin-ac tree. I tested
> >> > > with thin archives enabled and disabled with arm defconfig which ends up
> >> > > setting CONFIG_SUNXI_CCU=y.
> >> > >
> >> > > The patch makes no difference to vmlinux size whether using traditional
> >> > > incremental link, or thin archives (there is a small difference between
> >> > > inclink and thinarc but that's unrelated).
> >> > >
> >> > > So this thin archives change does not break your patch, it's just that
> >> > > it doesn't really do the right thing. I like the general idea, but we
> >> > > need to work out how to make it properly supported by the build system.
> >> > >
> >> > > With the patch applied, this is what the build system currently does:
> >> > >
> >> > > arm-linux-gnueabihf-ld -EL    -r -o drivers/clk/sunxi-ng/built-in.o drivers/clk/sunxi-ng/ccu-sun5i.o drivers/clk/sunxi-ng/ccu-sun6i-a31.o drivers/clk/sunxi-ng/ccu-sun8i-a23.o drivers/clk/sunxi-ng/ccu-sun8i-a33.o drivers/clk/sunxi-ng/ccu-sun8i-h3.o drivers/clk/sunxi-ng/ccu-sun8i-v3s.o drivers/clk/sunxi-ng/ccu-sun8i-r.o drivers/clk/sunxi-ng/ccu-sun9i-a80.o drivers/clk/sunxi-ng/ccu-sun9i-a80-de.o drivers/clk/sunxi-ng/ccu-sun9i-a80-usb.o drivers/clk/sunxi-ng/lib.a drivers/clk/sunxi-ng/lib-ksyms.o
> >> > >
> >> > > This incremental link pulls in all your lib.a objects and links them
> >> > > into built-in.o. They can no longer be selectively linked.
> >> >
> >> > I think the ARM defconfig actually needs all those objects because it
> >> > enables all the high-level drivers. You could try disabling e.g. all except
> >> > CONFIG_SUN8I_DE2_CCU if you want the objects to actually becomes
> >> > unused.
> >>
> >> You can also disable MACH_SUN8I, it should disable a significant
> >> number of the clocks driver.
> >
> >    text    data     bss     dec     hex filename
> > 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink
> > 11655078        6018736  418944 18092758        11412d6 vmlinux.inclink.clk
> > 11650534        6010288  418616 18079438        113dece vmlinux.thinarc
> > 11650534        6010288  418616 18079438        113dece vmlinux.thinarc.clk
> >
> > Same result for me when compiling defconfig minus MACH_SUN8I, comparing
> > +/- the clock patch and thin archives. No size savings. As I explained
> > already, I really can't see how there could be any saving if the lib.a is
> > incrementally linked into built-in.o.
> 
> I think defconfig minus MACH_SUN8I
> will give no difference to CONFIG_SUNXI_CCU*
> 
> As far as I tried,  minus MACH_SUN8I still enabled the following:
> CONFIG_SUNXI_CCU=y
> CONFIG_SUNXI_CCU_DIV=y
> CONFIG_SUNXI_CCU_FRAC=y
> CONFIG_SUNXI_CCU_GATE=y
> CONFIG_SUNXI_CCU_MUX=y
> CONFIG_SUNXI_CCU_MULT=y
> CONFIG_SUNXI_CCU_PHASE=y
> CONFIG_SUNXI_CCU_NK=y
> CONFIG_SUNXI_CCU_NKM=y
> CONFIG_SUNXI_CCU_NKMP=y
> CONFIG_SUNXI_CCU_NM=y
> CONFIG_SUNXI_CCU_MP=y
> 
> I think you need to disable some more MACH_SUN*I and
> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
> then you will see difference in the result.

Ah, right, it's all selected by the rest.

I guess to get a meaningful example you could disable all the
SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
SUN8I_R_CCU.

Most of the clock types shouldn't be used but div and gates.

Maxime

Arnd Bergmann June 23, 2017, 3:04 p.m. UTC | #26

On Fri, Jun 23, 2017 at 4:57 PM, Maxime Ripard
<[e-mail]> wrote:
> On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
>> As far as I tried,  minus MACH_SUN8I still enabled the following:
>> CONFIG_SUNXI_CCU=y
>> CONFIG_SUNXI_CCU_DIV=y
>> CONFIG_SUNXI_CCU_FRAC=y
>> CONFIG_SUNXI_CCU_GATE=y
>> CONFIG_SUNXI_CCU_MUX=y
>> CONFIG_SUNXI_CCU_MULT=y
>> CONFIG_SUNXI_CCU_PHASE=y
>> CONFIG_SUNXI_CCU_NK=y
>> CONFIG_SUNXI_CCU_NKM=y
>> CONFIG_SUNXI_CCU_NKMP=y
>> CONFIG_SUNXI_CCU_NM=y
>> CONFIG_SUNXI_CCU_MP=y
>>
>> I think you need to disable some more MACH_SUN*I and
>> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
>> then you will see difference in the result.
>
> Ah, right, it's all selected by the rest.
>
> I guess to get a meaningful example you could disable all the
> SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
> SUN8I_R_CCU.
>
> Most of the clock types shouldn't be used but div and gates.

Given that we appear to basically always pull in all (or most) of
the CCU parts anyway, and it's hard to even test with a subset
being disabled, I wonder how much we care about dropping
the unused objects at all:

Maybe the answer is that we just always build all of them into
the kernel and make only the SoC-specific objects configurable ;-)

Once we start linking with  --gc-sections, it will all be fine anyway.

       Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Masahiro Yamada June 25, 2017, 3:55 a.m. UTC | #27

2017-06-24 0:04 GMT+09:00 Arnd Bergmann <[e-mail]>:
> On Fri, Jun 23, 2017 at 4:57 PM, Maxime Ripard
> <[e-mail]> wrote:
>> On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
>>> As far as I tried,  minus MACH_SUN8I still enabled the following:
>>> CONFIG_SUNXI_CCU=y
>>> CONFIG_SUNXI_CCU_DIV=y
>>> CONFIG_SUNXI_CCU_FRAC=y
>>> CONFIG_SUNXI_CCU_GATE=y
>>> CONFIG_SUNXI_CCU_MUX=y
>>> CONFIG_SUNXI_CCU_MULT=y
>>> CONFIG_SUNXI_CCU_PHASE=y
>>> CONFIG_SUNXI_CCU_NK=y
>>> CONFIG_SUNXI_CCU_NKM=y
>>> CONFIG_SUNXI_CCU_NKMP=y
>>> CONFIG_SUNXI_CCU_NM=y
>>> CONFIG_SUNXI_CCU_MP=y
>>>
>>> I think you need to disable some more MACH_SUN*I and
>>> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
>>> then you will see difference in the result.
>>
>> Ah, right, it's all selected by the rest.
>>
>> I guess to get a meaningful example you could disable all the
>> SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
>> SUN8I_R_CCU.
>>
>> Most of the clock types shouldn't be used but div and gates.
>
> Given that we appear to basically always pull in all (or most) of
> the CCU parts anyway, and it's hard to even test with a subset
> being disabled, I wonder how much we care about dropping
> the unused objects at all:
>
> Maybe the answer is that we just always build all of them into
> the kernel and make only the SoC-specific objects configurable ;-)
>
> Once we start linking with  --gc-sections, it will all be fine anyway.
>

I agree!

Maxime Ripard June 27, 2017, 3:42 p.m. UTC | #28

On Fri, Jun 23, 2017 at 05:04:45PM +0200, Arnd Bergmann wrote:
> On Fri, Jun 23, 2017 at 4:57 PM, Maxime Ripard
> <[e-mail]> wrote:
> > On Fri, Jun 23, 2017 at 02:31:40PM +0900, Masahiro Yamada wrote:
> >> As far as I tried,  minus MACH_SUN8I still enabled the following:
> >> CONFIG_SUNXI_CCU=y
> >> CONFIG_SUNXI_CCU_DIV=y
> >> CONFIG_SUNXI_CCU_FRAC=y
> >> CONFIG_SUNXI_CCU_GATE=y
> >> CONFIG_SUNXI_CCU_MUX=y
> >> CONFIG_SUNXI_CCU_MULT=y
> >> CONFIG_SUNXI_CCU_PHASE=y
> >> CONFIG_SUNXI_CCU_NK=y
> >> CONFIG_SUNXI_CCU_NKM=y
> >> CONFIG_SUNXI_CCU_NKMP=y
> >> CONFIG_SUNXI_CCU_NM=y
> >> CONFIG_SUNXI_CCU_MP=y
> >>
> >> I think you need to disable some more MACH_SUN*I and
> >> perhaps disable some CONFIG_SUN*I_*_CCU explicitly,
> >> then you will see difference in the result.
> >
> > Ah, right, it's all selected by the rest.
> >
> > I guess to get a meaningful example you could disable all the
> > SUN*I_CCU (SUN5I, SUN8I, and the likes) options and while keeping
> > SUN8I_R_CCU.
> >
> > Most of the clock types shouldn't be used but div and gates.
> 
> Given that we appear to basically always pull in all (or most) of
> the CCU parts anyway, and it's hard to even test with a subset
> being disabled, I wonder how much we care about dropping
> the unused objects at all:
> 
> Maybe the answer is that we just always build all of them into
> the kernel and make only the SoC-specific objects configurable ;-)

I was trying to avoid that, but maybe that's not worth it, and it's
our only solution.

> Once we start linking with  --gc-sections, it will all be fine anyway.

That works for me. Who wants to send the patch?

Thanks!
Maxime

Patch

diff --git a/Documentation/kbuild/kbuild.txt b/Documentation/kbuild/kbuild.txt
index 0ff6a466a05b..ac2363ea05c5 100644
--- a/Documentation/kbuild/kbuild.txt
+++ b/Documentation/kbuild/kbuild.txt
@@ -236,5 +236,9 @@  Files specified with KBUILD_VMLINUX_INIT are linked first.
 KBUILD_VMLINUX_MAIN
 --------------------------------------------------
 All object files for the main part of vmlinux.
-KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN together specify
-all the object files used to link vmlinux.
+
+KBUILD_VMLINUX_LIBS
+--------------------------------------------------
+All .a "lib" files for vmlinux.
+KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, and KBUILD_VMLINUX_LIBS together
+specify all the object files used to link vmlinux.
diff --git a/Makefile b/Makefile
index 470bd4d9513a..a145a537ad42 100644
--- a/Makefile
+++ b/Makefile
@@ -952,19 +952,20 @@  core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
 drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
 net-y		:= $(patsubst %/, %/built-in.o, $(net-y))
 libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
-libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
+libs-y2		:= $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
 libs-y		:= $(libs-y1) $(libs-y2)
 virt-y		:= $(patsubst %/, %/built-in.o, $(virt-y))
 
 # Externally visible symbols (used by link-vmlinux.sh)
 export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
-export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(virt-y)
+export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)
+export KBUILD_VMLINUX_LIBS := $(libs-y1)
 export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
 export LDFLAGS_vmlinux
 # used by scripts/pacmage/Makefile
 export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) arch Documentation include samples scripts tools)
 
-vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN)
+vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)
 
 # Include targets which we want to execute sequentially if the rest of the
 # kernel build went well. If CONFIG_TRIM_UNUSED_KSYMS is set, this might be
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 2a062ea130b5..e7b7eee31538 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -3,9 +3,12 @@ 
 # link vmlinux
 #
 # vmlinux is linked from the objects selected by $(KBUILD_VMLINUX_INIT) and
-# $(KBUILD_VMLINUX_MAIN). Most are built-in.o files from top-level directories
-# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
-# Ordering when linking is important, and $(KBUILD_VMLINUX_INIT) must be first.
+# $(KBUILD_VMLINUX_MAIN) and $(KBUILD_VMLINUX_LIBS). Most are built-in.o files
+# from top-level directories in the kernel tree, others are specified in
+# arch/$(ARCH)/Makefile. Ordering when linking is important, and
+# $(KBUILD_VMLINUX_INIT) must be first. $(KBUILD_VMLINUX_LIBS) are archives
+# which are linked conditionally (not within --whole-archive), and do not
+# require symbol indexes added.
 #
 # vmlinux
 #   ^
@@ -16,6 +19,9 @@ 
 #   +--< $(KBUILD_VMLINUX_MAIN)
 #   |    +--< drivers/built-in.o mm/built-in.o + more
 #   |
+#   +--< $(KBUILD_VMLINUX_LIBS)
+#   |    +--< lib/lib.a + more
+#   |
 #   +-< ${kallsymso} (see description in KALLSYMS section)
 #
 # vmlinux version (uname -v) cannot be updated during normal
@@ -37,9 +43,10 @@  info()
 	fi
 }
 
-# Thin archive build here makes a final archive with
-# symbol table and indexes from vmlinux objects, which can be
-# used as input to linker.
+# Thin archive build here makes a final archive with symbol table and indexes
+# from vmlinux objects INIT and MAIN, which can be used as input to linker.
+# KBUILD_VMLINUX_LIBS archives should already have symbol table and indexes
+# added.
 #
 # Traditional incremental style of link does not require this step
 #
@@ -50,7 +57,7 @@  archive_builtin()
 	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
 		info AR built-in.o
 		rm -f built-in.o;
-		${AR} rcsT${KBUILD_ARFLAGS} built-in.o			\
+		${AR} rcsTP${KBUILD_ARFLAGS} built-in.o			\
 					${KBUILD_VMLINUX_INIT}		\
 					${KBUILD_VMLINUX_MAIN}
 	fi
@@ -63,11 +70,17 @@  modpost_link()
 	local objects
 
 	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-		objects="--whole-archive built-in.o --no-whole-archive"
+		objects="--whole-archive				\
+			built-in.o					\
+			--no-whole-archive				\
+			--start-group					\
+			${KBUILD_VMLINUX_LIBS}				\
+			--end-group"
 	else
 		objects="${KBUILD_VMLINUX_INIT}				\
 			--start-group					\
 			${KBUILD_VMLINUX_MAIN}				\
+			${KBUILD_VMLINUX_LIBS}				\
 			--end-group"
 	fi
 	${LD} ${LDFLAGS} -r -o ${1} ${objects}
@@ -83,11 +96,18 @@  vmlinux_link()
 
 	if [ "${SRCARCH}" != "um" ]; then
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-			objects="--whole-archive built-in.o ${1} --no-whole-archive"
+			objects="--whole-archive			\
+				built-in.o				\
+				--no-whole-archive			\
+				--start-group				\
+				${KBUILD_VMLINUX_LIBS}			\
+				--end-group				\
+				${1}"
 		else
 			objects="${KBUILD_VMLINUX_INIT}			\
 				--start-group				\
 				${KBUILD_VMLINUX_MAIN}			\
+				${KBUILD_VMLINUX_LIBS}			\
 				--end-group				\
 				${1}"
 		fi
@@ -96,11 +116,18 @@  vmlinux_link()
 			-T ${lds} ${objects}
 	else
 		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-			objects="-Wl,--whole-archive built-in.o ${1} -Wl,--no-whole-archive"
+			objects="-Wl,--whole-archive			\
+				built-in.o				\
+				-Wl,--no-whole-archive			\
+				-Wl,--start-group			\
+				${KBUILD_VMLINUX_LIBS}			\
+				-Wl,--end-group				\
+				${1}"
 		else
 			objects="${KBUILD_VMLINUX_INIT}			\
 				-Wl,--start-group			\
 				${KBUILD_VMLINUX_MAIN}			\
+				${KBUILD_VMLINUX_LIBS}			\
 				-Wl,--end-group				\
 				${1}"
 		fi