| Message ID | 20240414174139.3001175-1-masahiroy@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | kbuild: buildtar: add comments about inconsistent package generation | expand |
Masahiro Yamada wrote: > scripts/package/buildtar checks some kernel packages, and copies the > first image found. This may potentially produce an inconsistent (and > possibly wrong) package. > > For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, > and vmlinuz.efi, then copies the first image found, which might be a > stale image. > > When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, > 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the > expected behavior. > > If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created, > which will remain in the build directory unless you clean it. Even if > CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy > stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes > precedence over vmlinuz.efi. > > In summary, the code "[ -f ... ] && cp" does not consistently produce > the desired outcome. > > The other package scripts are deterministic; scripts/package/mkdebian, > for example, chooses a copied kernel image based on CONFIG options. > > I removed [ -f ... ] checks from x86, alpha, parisc, and the default > because they have a single kernel image to copy. If it is missing, it > should be an error. > > I did not modify the code for mips, arm64, riscv. Instead, I left some > comments. Eventually, someone may fix the code, or at the very least, > it may discourage the copy-pasting of incorrect code. > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Hi Masahiro, Thanks for the patch. I wonder why arm64 and riscv is not just using KBUILD_IMAGE as the source path of the image to copy like some other architectures do. In any case: Reviewed-by: Emil Renner Berthing <emil.renner.berthing@canonical.com> > --- > > scripts/package/buildtar | 16 ++++++++++++---- > 1 file changed, 12 insertions(+), 4 deletions(-) > > diff --git a/scripts/package/buildtar b/scripts/package/buildtar > index 72c91a1b832f..ed8d9b496305 100755 > --- a/scripts/package/buildtar > +++ b/scripts/package/buildtar > @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > # > # Install arch-specific kernel image(s) > # > +# Note: > +# mips, arm64, and riscv copy the first image found. This may not produce > +# the desired outcome because it may pick up a stale file remaining in the > +# build tree. > +# > case "${ARCH}" in > x86|i386|x86_64) > - [ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > + cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > ;; > alpha) > - [ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > + cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > ;; > parisc*) > - [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > + cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > [ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}" > ;; > mips) > + # Please note the following code may copy a stale file. > if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then > cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then > @@ -86,6 +92,7 @@ case "${ARCH}" in > fi > ;; > arm64) > + # Please note the following code may copy a stale file. > for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do > if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then > cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > @@ -94,6 +101,7 @@ case "${ARCH}" in > done > ;; > riscv) > + # Please note the following code may copy a stale file. > for i in Image.bz2 Image.gz Image; do > if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then > cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > @@ -102,7 +110,7 @@ case "${ARCH}" in > done > ;; > *) > - [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}" > + cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}" > echo "" >&2 > echo '** ** ** WARNING ** ** **' >&2 > echo "" >&2 > -- > 2.40.1 > > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv
On Mon, Apr 15, 2024 at 2:42 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > scripts/package/buildtar checks some kernel packages, and copies the > first image found. This may potentially produce an inconsistent (and > possibly wrong) package. > > For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, > and vmlinuz.efi, then copies the first image found, which might be a > stale image. > > When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, > 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the > expected behavior. > > If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created, The correct sentence is If you build the kernel with CONFIG_EFI_ZBOOT disabled, ... > which will remain in the build directory unless you clean it. Even if > CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy > stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes > precedence over vmlinuz.efi. > > In summary, the code "[ -f ... ] && cp" does not consistently produce > the desired outcome. > > The other package scripts are deterministic; scripts/package/mkdebian, > for example, chooses a copied kernel image based on CONFIG options. > > I removed [ -f ... ] checks from x86, alpha, parisc, and the default > because they have a single kernel image to copy. If it is missing, it > should be an error. > > I did not modify the code for mips, arm64, riscv. Instead, I left some > comments. Eventually, someone may fix the code, or at the very least, > it may discourage the copy-pasting of incorrect code. > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > scripts/package/buildtar | 16 ++++++++++++---- > 1 file changed, 12 insertions(+), 4 deletions(-) > > diff --git a/scripts/package/buildtar b/scripts/package/buildtar > index 72c91a1b832f..ed8d9b496305 100755 > --- a/scripts/package/buildtar > +++ b/scripts/package/buildtar > @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > # > # Install arch-specific kernel image(s) > # > +# Note: > +# mips, arm64, and riscv copy the first image found. This may not produce > +# the desired outcome because it may pick up a stale file remaining in the > +# build tree. > +# > case "${ARCH}" in > x86|i386|x86_64) > - [ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > + cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > ;; > alpha) > - [ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > + cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > ;; > parisc*) > - [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > + cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > [ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}" > ;; > mips) > + # Please note the following code may copy a stale file. > if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then > cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then > @@ -86,6 +92,7 @@ case "${ARCH}" in > fi > ;; > arm64) > + # Please note the following code may copy a stale file. > for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do > if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then > cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" > @@ -94,6 +101,7 @@ case "${ARCH}" in > done > ;; > riscv) > + # Please note the following code may copy a stale file. > for i in Image.bz2 Image.gz Image; do > if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then > cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" > @@ -102,7 +110,7 @@ case "${ARCH}" in > done > ;; > *) > - [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}" > + cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}" > echo "" >&2 > echo '** ** ** WARNING ** ** **' >&2 > echo "" >&2 > -- > 2.40.1 > -- Best Regards Masahiro Yamada
On Tue 16 Apr 2024 11:07:48 GMT, Masahiro Yamada wrote: > On Mon, Apr 15, 2024 at 2:42 AM Masahiro Yamada > <masahiroy@kernel.org> wrote: > > > > scripts/package/buildtar checks some kernel packages, and copies the > > first image found. This may potentially produce an inconsistent (and > > possibly wrong) package. > > > > For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, > > and vmlinuz.efi, then copies the first image found, which might be a > > stale image. > > > > When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, > > 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the > > expected behavior. > > > > If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created, > > > The correct sentence is > > If you build the kernel with CONFIG_EFI_ZBOOT disabled, ... Thanks, I appreciate the detailed descriptions and comments. Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> Kind regards, Nicolas
diff --git a/scripts/package/buildtar b/scripts/package/buildtar index 72c91a1b832f..ed8d9b496305 100755 --- a/scripts/package/buildtar +++ b/scripts/package/buildtar @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" # # Install arch-specific kernel image(s) # +# Note: +# mips, arm64, and riscv copy the first image found. This may not produce +# the desired outcome because it may pick up a stale file remaining in the +# build tree. +# case "${ARCH}" in x86|i386|x86_64) - [ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" + cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" ;; alpha) - [ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" + cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" ;; parisc*) - [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" + cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" [ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}" ;; mips) + # Please note the following code may copy a stale file. if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then @@ -86,6 +92,7 @@ case "${ARCH}" in fi ;; arm64) + # Please note the following code may copy a stale file. for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}" @@ -94,6 +101,7 @@ case "${ARCH}" in done ;; riscv) + # Please note the following code may copy a stale file. for i in Image.bz2 Image.gz Image; do if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}" @@ -102,7 +110,7 @@ case "${ARCH}" in done ;; *) - [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}" + cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}" echo "" >&2 echo '** ** ** WARNING ** ** **' >&2 echo "" >&2
scripts/package/buildtar checks some kernel packages, and copies the first image found. This may potentially produce an inconsistent (and possibly wrong) package. For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, and vmlinuz.efi, then copies the first image found, which might be a stale image. When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the expected behavior. If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created, which will remain in the build directory unless you clean it. Even if CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes precedence over vmlinuz.efi. In summary, the code "[ -f ... ] && cp" does not consistently produce the desired outcome. The other package scripts are deterministic; scripts/package/mkdebian, for example, chooses a copied kernel image based on CONFIG options. I removed [ -f ... ] checks from x86, alpha, parisc, and the default because they have a single kernel image to copy. If it is missing, it should be an error. I did not modify the code for mips, arm64, riscv. Instead, I left some comments. Eventually, someone may fix the code, or at the very least, it may discourage the copy-pasting of incorrect code. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/buildtar | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)