diff mbox

ARM: add boot image dependencies not to generate invalid images

Message ID 1436186224-6673-1-git-send-email-yamada.masahiro@socionext.com
State New, archived
Headers show

Commit Message

Masahiro Yamada July 6, 2015, 12:37 p.m. UTC
U-Boot is often used to boot the kernel on ARM boards, but uImage
is not built by "make all", so we are often inclined to do
"make all uImage" in a single command, but we should notice a
pitfall behind it.  In fact, "make all uImage" could generate an
invalid uImage if it is run with the parallel option (-j).

You can reproduce this problem with the following procedure:

[1] First, build "all" and "uImage" separately.
    You will get a valid uImage
  $ export CROSS_COMPILE=<your-tools-prefix>
  $ make -s -j8 ARCH=arm multi_v7_defconfig
  $ make -s -j8 ARCH=arm all
  $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 uImage
  Image Name:   Linux-4.2.0-rc1-00008-g1c4c715
  Created:      Mon Jul  6 17:49:52 2015
  Image Type:   ARM Linux Kernel Image (uncompressed)
  Data Size:    6145624 Bytes = 6001.59 kB = 5.86 MB
  Load Address: 80208000
  Entry Point:  80208000
  $ cat arch/arm/boot/uImage | wc
      17553  107879 6145688

[2] Update some source file(s)
  $ touch init/main.c

[3] Then, re-build "all" and "uImage" simultaneously.
    You will get an invalid uImage at random.
  $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
  Image Name:   Linux-4.2.0-rc1-00008-g1c4c715-d
  Created:      Mon Jul  6 17:52:22 2015
  Image Type:   ARM Linux Kernel Image (uncompressed)
  Data Size:    26768 Bytes = 26.14 kB = 0.03 MB
  Load Address: 80208000
  Entry Point:  80208000
  $ cat arch/arm/boot/uImage | wc
      266    1063   26832

Please notice the uImage is extremely small when this issue is
encountered.  The root cause of this is the race condition between
zImage and uImage targets.

"make uImage" could descend into arch/arm/boot/Makefile before
"make zImage" is completed because arch/arm/Makefile describes no
dependency among boot targets.

The same problem could happen on bootpImage.

Add correct dependencies among Image, zImage, uImage, and bootpImage
to eliminate this pit-fall.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 arch/arm/Makefile | 3 +++
 1 file changed, 3 insertions(+)

Comments

Russell King - ARM Linux July 10, 2015, 10:22 a.m. UTC | #1
On Mon, Jul 06, 2015 at 09:37:04PM +0900, Masahiro Yamada wrote:
> [3] Then, re-build "all" and "uImage" simultaneously.
>     You will get an invalid uImage at random.
>   $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>   Image Name:   Linux-4.2.0-rc1-00008-g1c4c715-d
>   Created:      Mon Jul  6 17:52:22 2015
>   Image Type:   ARM Linux Kernel Image (uncompressed)
>   Data Size:    26768 Bytes = 26.14 kB = 0.03 MB
>   Load Address: 80208000
>   Entry Point:  80208000

At no point in the above do I see an attempt to rebuild init/main.o,
which there should be as you touched the corresponding .c file -
because you're hiding that output with -s.  Please show what's going
on without using -s.

> "make uImage" could descend into arch/arm/boot/Makefile before
> "make zImage" is completed because arch/arm/Makefile describes no
> dependency among boot targets.

The uImage target should depend on vmlinux, which should force the
rebuild of init/main.o, and relink of the top-level vmlinux file
before decending into arch/arm/boot/Makefile for the
arch/arm/boot/uImage target.

That makefile contains the dependencies required to order things
correctly - the arch/arm/boot/uImage target depends on
arch/arm/boot/zImage, which in turn depends on
arch/arm/boot/compressed/vmlinux, and then arch/arm/boot/Image.

In other words, arch/arm/boot/Makefile deals with the dependencies
between the targets it's responsible for building itself.
Masahiro Yamada July 10, 2015, 4:41 p.m. UTC | #2
Hi Russel,


2015-07-10 19:22 GMT+09:00 Russell King - ARM Linux <linux@arm.linux.org.uk>:
> On Mon, Jul 06, 2015 at 09:37:04PM +0900, Masahiro Yamada wrote:
>> [3] Then, re-build "all" and "uImage" simultaneously.
>>     You will get an invalid uImage at random.
>>   $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>>   Image Name:   Linux-4.2.0-rc1-00008-g1c4c715-d
>>   Created:      Mon Jul  6 17:52:22 2015
>>   Image Type:   ARM Linux Kernel Image (uncompressed)
>>   Data Size:    26768 Bytes = 26.14 kB = 0.03 MB
>>   Load Address: 80208000
>>   Entry Point:  80208000
>
> At no point in the above do I see an attempt to rebuild init/main.o,
> which there should be as you touched the corresponding .c file -
> because you're hiding that output with -s.  Please show what's going
> on without using -s.

Please see the log attached at the end of this message.


>> "make uImage" could descend into arch/arm/boot/Makefile before
>> "make zImage" is completed because arch/arm/Makefile describes no
>> dependency among boot targets.
>
> The uImage target should depend on vmlinux, which should force the
> rebuild of init/main.o, and relink of the top-level vmlinux file
> before decending into arch/arm/boot/Makefile for the
> arch/arm/boot/uImage target.
>
> That makefile contains the dependencies required to order things
> correctly - the arch/arm/boot/uImage target depends on
> arch/arm/boot/zImage, which in turn depends on
> arch/arm/boot/compressed/vmlinux, and then arch/arm/boot/Image.
>
> In other words, arch/arm/boot/Makefile deals with the dependencies
> between the targets it's responsible for building itself.

No, you do not understand what is happening in here.


The dependencies among targets such as Image, zImage, uImage,
are fully described in arch/arm/boot/Makefile, but it is not enough.

Because arch/arm/boot/Makefile is _not_ included from the top-level Makefile,
the top-level build cannot know the dependency between zImage and uImage.

arch/arm/Makefile, which is included from the top Makefile,
only describes that zImage depends on vmlinux,
and uImage depends on vmlinux as well.
But, no dependency between zImage and uImage is written in arch/arm/Makefile.

Consequently, we run make with the parallel option, first Kbuild
updates vmlinux,
and then two different threads descends into arch/arm/boot/Makefile
almost at the same time, one for updating zImage and the other for uImage.

The is a race between the two threads.

While one thread is re-generating zImage and also uImage on top of that,
the other thread tries to re-generate zImage independently.

zImage is overwritten by the slower thread, and then uImage is
re-generated based on the broken zImage.

See my detailed build log below:

Please note:
"Kernel: arch/arm/boot/zImage is ready"
is displayed twice,
the first one is shown before the uImage log, and the second one is
after uImage.

zImage is correct, but uImage is extremely small when this problem happens.



$ git describe
v4.2-rc1-62-gc4b5fd3


$ touch init/main.c
$ make -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: `include/generated/mach-types.h' is up to date.
  CHK     include/generated/timeconst.h
  CHK     include/generated/bounds.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CC      init/main.o
  CHK     include/generated/compile.h
  LD      init/built-in.o
  LINK    vmlinux
  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map
  OBJCOPY arch/arm/boot/Image
  Building modules, stage 2.
  Kernel: arch/arm/boot/Image is ready
  GZIP    arch/arm/boot/compressed/piggy.gzip
  AS      arch/arm/boot/compressed/piggy.gzip.o
  Kernel: arch/arm/boot/Image is ready
  GZIP    arch/arm/boot/compressed/piggy.gzip
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
  UIMAGE  arch/arm/boot/uImage
Image Name:   Linux-4.2.0-rc1-00062-gc4b5fd3-d
Created:      Sat Jul 11 01:22:22 2015
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    26472 Bytes = 25.85 kB = 0.03 MB
Load Address: 80208000
Entry Point:  80208000
  Image arch/arm/boot/uImage is ready
  MODPOST 192 modules
  AS      arch/arm/boot/compressed/piggy.gzip.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
$ LANG=C ls -l arch/arm/boot/
total 19636
-rwxrwxr-x 1 masahiro masahiro 13766656 Jul 11 01:22 Image
-rw-rw-r-- 1 masahiro masahiro     3137 Jul  3 01:17 Makefile
drwxrwxr-x 2 masahiro masahiro     4096 Jul  3 01:17 bootp
drwxrwxr-x 2 masahiro masahiro     4096 Jul 11 01:22 compressed
drwxrwxr-x 3 masahiro masahiro   155648 Jul 11 00:58 dts
-rw-rw-r-- 1 masahiro masahiro     1648 Jul  3 01:17 install.sh
-rw-rw-r-- 1 masahiro masahiro    26536 Jul 11 01:22 uImage
-rwxrwxr-x 1 masahiro masahiro  6135048 Jul 11 01:22 zImage
Masahiro Yamada July 21, 2015, 6:58 a.m. UTC | #3
Hi Russel,

No more comment?
I answered your question.




2015-07-11 1:41 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> Hi Russel,
>
>
> 2015-07-10 19:22 GMT+09:00 Russell King - ARM Linux <linux@arm.linux.org.uk>:
>> On Mon, Jul 06, 2015 at 09:37:04PM +0900, Masahiro Yamada wrote:
>>> [3] Then, re-build "all" and "uImage" simultaneously.
>>>     You will get an invalid uImage at random.
>>>   $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>>>   Image Name:   Linux-4.2.0-rc1-00008-g1c4c715-d
>>>   Created:      Mon Jul  6 17:52:22 2015
>>>   Image Type:   ARM Linux Kernel Image (uncompressed)
>>>   Data Size:    26768 Bytes = 26.14 kB = 0.03 MB
>>>   Load Address: 80208000
>>>   Entry Point:  80208000
>>
>> At no point in the above do I see an attempt to rebuild init/main.o,
>> which there should be as you touched the corresponding .c file -
>> because you're hiding that output with -s.  Please show what's going
>> on without using -s.
>
> Please see the log attached at the end of this message.
>
>
>>> "make uImage" could descend into arch/arm/boot/Makefile before
>>> "make zImage" is completed because arch/arm/Makefile describes no
>>> dependency among boot targets.
>>
>> The uImage target should depend on vmlinux, which should force the
>> rebuild of init/main.o, and relink of the top-level vmlinux file
>> before decending into arch/arm/boot/Makefile for the
>> arch/arm/boot/uImage target.
>>
>> That makefile contains the dependencies required to order things
>> correctly - the arch/arm/boot/uImage target depends on
>> arch/arm/boot/zImage, which in turn depends on
>> arch/arm/boot/compressed/vmlinux, and then arch/arm/boot/Image.
>>
>> In other words, arch/arm/boot/Makefile deals with the dependencies
>> between the targets it's responsible for building itself.
>
> No, you do not understand what is happening in here.
>
>
> The dependencies among targets such as Image, zImage, uImage,
> are fully described in arch/arm/boot/Makefile, but it is not enough.
>
> Because arch/arm/boot/Makefile is _not_ included from the top-level Makefile,
> the top-level build cannot know the dependency between zImage and uImage.
>
> arch/arm/Makefile, which is included from the top Makefile,
> only describes that zImage depends on vmlinux,
> and uImage depends on vmlinux as well.
> But, no dependency between zImage and uImage is written in arch/arm/Makefile.
>
> Consequently, we run make with the parallel option, first Kbuild
> updates vmlinux,
> and then two different threads descends into arch/arm/boot/Makefile
> almost at the same time, one for updating zImage and the other for uImage.
>
> The is a race between the two threads.
>
> While one thread is re-generating zImage and also uImage on top of that,
> the other thread tries to re-generate zImage independently.
>
> zImage is overwritten by the slower thread, and then uImage is
> re-generated based on the broken zImage.
>
> See my detailed build log below:
>
> Please note:
> "Kernel: arch/arm/boot/zImage is ready"
> is displayed twice,
> the first one is shown before the uImage log, and the second one is
> after uImage.
>
> zImage is correct, but uImage is extremely small when this problem happens.
>
>
>
> $ git describe
> v4.2-rc1-62-gc4b5fd3
>
>
> $ touch init/main.c
> $ make -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>   CHK     include/config/kernel.release
>   CHK     include/generated/uapi/linux/version.h
>   CHK     include/generated/utsrelease.h
> make[1]: `include/generated/mach-types.h' is up to date.
>   CHK     include/generated/timeconst.h
>   CHK     include/generated/bounds.h
>   CHK     include/generated/asm-offsets.h
>   CALL    scripts/checksyscalls.sh
>   CC      init/main.o
>   CHK     include/generated/compile.h
>   LD      init/built-in.o
>   LINK    vmlinux
>   LD      vmlinux.o
>   MODPOST vmlinux.o
>   GEN     .version
>   CHK     include/generated/compile.h
>   UPD     include/generated/compile.h
>   CC      init/version.o
>   LD      init/built-in.o
>   KSYM    .tmp_kallsyms1.o
>   KSYM    .tmp_kallsyms2.o
>   LD      vmlinux
>   SORTEX  vmlinux
>   SYSMAP  System.map
>   OBJCOPY arch/arm/boot/Image
>   Building modules, stage 2.
>   Kernel: arch/arm/boot/Image is ready
>   GZIP    arch/arm/boot/compressed/piggy.gzip
>   AS      arch/arm/boot/compressed/piggy.gzip.o
>   Kernel: arch/arm/boot/Image is ready
>   GZIP    arch/arm/boot/compressed/piggy.gzip
>   LD      arch/arm/boot/compressed/vmlinux
>   OBJCOPY arch/arm/boot/zImage
>   Kernel: arch/arm/boot/zImage is ready
>   UIMAGE  arch/arm/boot/uImage
> Image Name:   Linux-4.2.0-rc1-00062-gc4b5fd3-d
> Created:      Sat Jul 11 01:22:22 2015
> Image Type:   ARM Linux Kernel Image (uncompressed)
> Data Size:    26472 Bytes = 25.85 kB = 0.03 MB
> Load Address: 80208000
> Entry Point:  80208000
>   Image arch/arm/boot/uImage is ready
>   MODPOST 192 modules
>   AS      arch/arm/boot/compressed/piggy.gzip.o
>   LD      arch/arm/boot/compressed/vmlinux
>   OBJCOPY arch/arm/boot/zImage
>   Kernel: arch/arm/boot/zImage is ready
> $ LANG=C ls -l arch/arm/boot/
> total 19636
> -rwxrwxr-x 1 masahiro masahiro 13766656 Jul 11 01:22 Image
> -rw-rw-r-- 1 masahiro masahiro     3137 Jul  3 01:17 Makefile
> drwxrwxr-x 2 masahiro masahiro     4096 Jul  3 01:17 bootp
> drwxrwxr-x 2 masahiro masahiro     4096 Jul 11 01:22 compressed
> drwxrwxr-x 3 masahiro masahiro   155648 Jul 11 00:58 dts
> -rw-rw-r-- 1 masahiro masahiro     1648 Jul  3 01:17 install.sh
> -rw-rw-r-- 1 masahiro masahiro    26536 Jul 11 01:22 uImage
> -rwxrwxr-x 1 masahiro masahiro  6135048 Jul 11 01:22 zImage
>
>
>
>
>
>
>
> --
> Best Regards
> Masahiro Yamada
Masahiro Yamada Aug. 8, 2015, 12:29 p.m. UTC | #4
Hi Russell,

I will rephrase the git-description and post v2.





2015-07-21 15:58 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> Hi Russel,
>
> No more comment?
> I answered your question.
>
>
>
>
> 2015-07-11 1:41 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>> Hi Russel,
>>
>>
>> 2015-07-10 19:22 GMT+09:00 Russell King - ARM Linux <linux@arm.linux.org.uk>:
>>> On Mon, Jul 06, 2015 at 09:37:04PM +0900, Masahiro Yamada wrote:
>>>> [3] Then, re-build "all" and "uImage" simultaneously.
>>>>     You will get an invalid uImage at random.
>>>>   $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>>>>   Image Name:   Linux-4.2.0-rc1-00008-g1c4c715-d
>>>>   Created:      Mon Jul  6 17:52:22 2015
>>>>   Image Type:   ARM Linux Kernel Image (uncompressed)
>>>>   Data Size:    26768 Bytes = 26.14 kB = 0.03 MB
>>>>   Load Address: 80208000
>>>>   Entry Point:  80208000
>>>
>>> At no point in the above do I see an attempt to rebuild init/main.o,
>>> which there should be as you touched the corresponding .c file -
>>> because you're hiding that output with -s.  Please show what's going
>>> on without using -s.
>>
>> Please see the log attached at the end of this message.
>>
>>
>>>> "make uImage" could descend into arch/arm/boot/Makefile before
>>>> "make zImage" is completed because arch/arm/Makefile describes no
>>>> dependency among boot targets.
>>>
>>> The uImage target should depend on vmlinux, which should force the
>>> rebuild of init/main.o, and relink of the top-level vmlinux file
>>> before decending into arch/arm/boot/Makefile for the
>>> arch/arm/boot/uImage target.
>>>
>>> That makefile contains the dependencies required to order things
>>> correctly - the arch/arm/boot/uImage target depends on
>>> arch/arm/boot/zImage, which in turn depends on
>>> arch/arm/boot/compressed/vmlinux, and then arch/arm/boot/Image.
>>>
>>> In other words, arch/arm/boot/Makefile deals with the dependencies
>>> between the targets it's responsible for building itself.
>>
>> No, you do not understand what is happening in here.
>>
>>
>> The dependencies among targets such as Image, zImage, uImage,
>> are fully described in arch/arm/boot/Makefile, but it is not enough.
>>
>> Because arch/arm/boot/Makefile is _not_ included from the top-level Makefile,
>> the top-level build cannot know the dependency between zImage and uImage.
>>
>> arch/arm/Makefile, which is included from the top Makefile,
>> only describes that zImage depends on vmlinux,
>> and uImage depends on vmlinux as well.
>> But, no dependency between zImage and uImage is written in arch/arm/Makefile.
>>
>> Consequently, we run make with the parallel option, first Kbuild
>> updates vmlinux,
>> and then two different threads descends into arch/arm/boot/Makefile
>> almost at the same time, one for updating zImage and the other for uImage.
>>
>> The is a race between the two threads.
>>
>> While one thread is re-generating zImage and also uImage on top of that,
>> the other thread tries to re-generate zImage independently.
>>
>> zImage is overwritten by the slower thread, and then uImage is
>> re-generated based on the broken zImage.
>>
>> See my detailed build log below:
>>
>> Please note:
>> "Kernel: arch/arm/boot/zImage is ready"
>> is displayed twice,
>> the first one is shown before the uImage log, and the second one is
>> after uImage.
>>
>> zImage is correct, but uImage is extremely small when this problem happens.
>>
>>
>>
>> $ git describe
>> v4.2-rc1-62-gc4b5fd3
>>
>>
>> $ touch init/main.c
>> $ make -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>>   CHK     include/config/kernel.release
>>   CHK     include/generated/uapi/linux/version.h
>>   CHK     include/generated/utsrelease.h
>> make[1]: `include/generated/mach-types.h' is up to date.
>>   CHK     include/generated/timeconst.h
>>   CHK     include/generated/bounds.h
>>   CHK     include/generated/asm-offsets.h
>>   CALL    scripts/checksyscalls.sh
>>   CC      init/main.o
>>   CHK     include/generated/compile.h
>>   LD      init/built-in.o
>>   LINK    vmlinux
>>   LD      vmlinux.o
>>   MODPOST vmlinux.o
>>   GEN     .version
>>   CHK     include/generated/compile.h
>>   UPD     include/generated/compile.h
>>   CC      init/version.o
>>   LD      init/built-in.o
>>   KSYM    .tmp_kallsyms1.o
>>   KSYM    .tmp_kallsyms2.o
>>   LD      vmlinux
>>   SORTEX  vmlinux
>>   SYSMAP  System.map
>>   OBJCOPY arch/arm/boot/Image
>>   Building modules, stage 2.
>>   Kernel: arch/arm/boot/Image is ready
>>   GZIP    arch/arm/boot/compressed/piggy.gzip
>>   AS      arch/arm/boot/compressed/piggy.gzip.o
>>   Kernel: arch/arm/boot/Image is ready
>>   GZIP    arch/arm/boot/compressed/piggy.gzip
>>   LD      arch/arm/boot/compressed/vmlinux
>>   OBJCOPY arch/arm/boot/zImage
>>   Kernel: arch/arm/boot/zImage is ready
>>   UIMAGE  arch/arm/boot/uImage
>> Image Name:   Linux-4.2.0-rc1-00062-gc4b5fd3-d
>> Created:      Sat Jul 11 01:22:22 2015
>> Image Type:   ARM Linux Kernel Image (uncompressed)
>> Data Size:    26472 Bytes = 25.85 kB = 0.03 MB
>> Load Address: 80208000
>> Entry Point:  80208000
>>   Image arch/arm/boot/uImage is ready
>>   MODPOST 192 modules
>>   AS      arch/arm/boot/compressed/piggy.gzip.o
>>   LD      arch/arm/boot/compressed/vmlinux
>>   OBJCOPY arch/arm/boot/zImage
>>   Kernel: arch/arm/boot/zImage is ready
>> $ LANG=C ls -l arch/arm/boot/
>> total 19636
>> -rwxrwxr-x 1 masahiro masahiro 13766656 Jul 11 01:22 Image
>> -rw-rw-r-- 1 masahiro masahiro     3137 Jul  3 01:17 Makefile
>> drwxrwxr-x 2 masahiro masahiro     4096 Jul  3 01:17 bootp
>> drwxrwxr-x 2 masahiro masahiro     4096 Jul 11 01:22 compressed
>> drwxrwxr-x 3 masahiro masahiro   155648 Jul 11 00:58 dts
>> -rw-rw-r-- 1 masahiro masahiro     1648 Jul  3 01:17 install.sh
>> -rw-rw-r-- 1 masahiro masahiro    26536 Jul 11 01:22 uImage
>> -rwxrwxr-x 1 masahiro masahiro  6135048 Jul 11 01:22 zImage
>>
>>
>>
>>
>>
>>
>>
>> --
>> Best Regards
>> Masahiro Yamada
>
>
>
> --
> Best Regards
> Masahiro Yamada
diff mbox

Patch

diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 07ab3d2..7451b44 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -312,6 +312,9 @@  INSTALL_TARGETS	= zinstall uinstall install
 
 PHONY += bzImage $(BOOT_TARGETS) $(INSTALL_TARGETS)
 
+bootpImage uImage: zImage
+zImage: Image
+
 $(BOOT_TARGETS): vmlinux
 	$(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $(boot)/$@