| Message ID | 20200605073955.6384-1-efremov@linux.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v5] kbuild: add variables for compression tools | expand |
On Fri, Jun 5, 2020 at 4:40 PM Denis Efremov <efremov@linux.com> wrote: > > Allow user to use alternative implementations of compression tools, > such as pigz, pbzip2, pxz. For example, multi-threaded tools to > speed up the build: > $ make GZIP=pigz BZIP2=pbzip2 > > Variables _GZIP, _BZIP2, _LZOP are used internally because original env > vars are reserved by the tools. The use of GZIP in gzip tool is obsolete > since 2015. However, alternative implementations (e.g., pigz) still rely > on it. BZIP2, BZIP, LZOP vars are not obsolescent. > > The credit goes to @grsecurity. > > As a sidenote, for multi-threaded lzma, xz compression one can use: > $ export XZ_OPT="--threads=0" > > Signed-off-by: Denis Efremov <efremov@linux.com> > --- > Changes in v2: > - _GZIP used instead of GZIP > - tar commands altered to use tools from the vars > Changes in v3: > - _BZIP2 used instead of BZIP2 > - _LZOP used instead of LZOP > Changes in v4: > - Unexports removed from Kbuild.include > - MAKEOVERRIDES used in top Makefile > - All variables checks removed from scripts > Changes in v5: > - Conflict with removed LASAT resolved Replaced. Thanks.
Hi, On Fri, Jun 05, 2020 at 10:39:55AM +0300, Denis Efremov wrote: > Allow user to use alternative implementations of compression tools, > such as pigz, pbzip2, pxz. For example, multi-threaded tools to > speed up the build: > $ make GZIP=pigz BZIP2=pbzip2 > > Variables _GZIP, _BZIP2, _LZOP are used internally because original env > vars are reserved by the tools. The use of GZIP in gzip tool is obsolete > since 2015. However, alternative implementations (e.g., pigz) still rely > on it. BZIP2, BZIP, LZOP vars are not obsolescent. > When building mips:defconfig, this patch results in: Building mips:defconfig ... failed -------------- Error log: /bin/sh: -n: command not found make[3]: *** [kernel/config_data.gz] Error 127 make[3]: *** Deleting file 'kernel/config_data.gz' make[3]: *** Waiting for unfinished jobs.... make[2]: *** [kernel] Error 2 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [autoksyms_recursive] Error 2 make: *** [__sub-make] Error 2 Reverting this patch fixes the problem. Bisect log is attached. Guenter --- # bad: [cf0c97f148e9e50aa5a7ddd1984a604dd2bde4af] Merge tag 'pinctrl-v5.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl # good: [aaa2faab4ed8e5fe0111e04d6e168c028fe2987f] Merge tag 'for-linus-5.8-ofs1' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux git bisect start 'HEAD' 'aaa2faab4ed8' # good: [77f55d1305c11fb729b88f2c3f7881ba0831fa6f] staging: rtl8723bs: Use common packet header constants git bisect good 77f55d1305c11fb729b88f2c3f7881ba0831fa6f # bad: [e611c0fe318c6d6827ee2bba660fbc23cf73f7dc] Merge tag 'usb-5.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb git bisect bad e611c0fe318c6d6827ee2bba660fbc23cf73f7dc # bad: [cff11abeca78aa782378401ca2800bd2194aa14e] Merge tag 'kbuild-v5.8' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild git bisect bad cff11abeca78aa782378401ca2800bd2194aa14e # good: [2bd81cd04a3f5eb873cc81fa16c469377be3b092] Merge branch 'remotes/lorenzo/pci/vmd' git bisect good 2bd81cd04a3f5eb873cc81fa16c469377be3b092 # good: [269a535ca931b754a40dda3ab60514e68773c759] modpost: generate vmlinux.symvers and reuse it for the second modpost git bisect good 269a535ca931b754a40dda3ab60514e68773c759 # good: [e542e0dc3ee3eafc46dd8e3073388079d69cace0] Merge branch 'dmi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jdelvare/staging git bisect good e542e0dc3ee3eafc46dd8e3073388079d69cace0 # good: [4de7b62936122570408357417f21072e78292926] modpost: remove is_vmlinux() helper git bisect good 4de7b62936122570408357417f21072e78292926 # good: [1ee18de92927f37e6948d5a6fc73cbf89f806905] Merge tag 'dma-mapping-5.8' of git://git.infradead.org/users/hch/dma-mapping git bisect good 1ee18de92927f37e6948d5a6fc73cbf89f806905 # bad: [8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294] kbuild: add variables for compression tools git bisect bad 8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294 # good: [c0901577e1dcc8d1c0fd1a11c8d571f650df845f] kbuild: doc: rename LDFLAGS to KBUILD_LDFLAGS git bisect good c0901577e1dcc8d1c0fd1a11c8d571f650df845f # good: [e0b250b57dcf403529081e5898a9de717f96b76b] Makefile: install modules.builtin even if CONFIG_MODULES=n git bisect good e0b250b57dcf403529081e5898a9de717f96b76b # first bad commit: [8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294] kbuild: add variables for compression tools
On Mon, Jun 8, 2020 at 10:30 AM Guenter Roeck <linux@roeck-us.net> wrote: > > Hi, > > On Fri, Jun 05, 2020 at 10:39:55AM +0300, Denis Efremov wrote: > > Allow user to use alternative implementations of compression tools, > > such as pigz, pbzip2, pxz. For example, multi-threaded tools to > > speed up the build: > > $ make GZIP=pigz BZIP2=pbzip2 > > > > Variables _GZIP, _BZIP2, _LZOP are used internally because original env > > vars are reserved by the tools. The use of GZIP in gzip tool is obsolete > > since 2015. However, alternative implementations (e.g., pigz) still rely > > on it. BZIP2, BZIP, LZOP vars are not obsolescent. > > > > When building mips:defconfig, this patch results in: > > Building mips:defconfig ... failed > -------------- > Error log: > /bin/sh: -n: command not found > make[3]: *** [kernel/config_data.gz] Error 127 > make[3]: *** Deleting file 'kernel/config_data.gz' > make[3]: *** Waiting for unfinished jobs.... > make[2]: *** [kernel] Error 2 > make[2]: *** Waiting for unfinished jobs.... > make[1]: *** [autoksyms_recursive] Error 2 > make: *** [__sub-make] Error 2 > > Reverting this patch fixes the problem. Bisect log is attached. > > Guenter Agh, this is because of CONFIG_TRIM_UNUSED_KSYMS. Also, the distro package builds are broken e.g. make GZIP=gzip bindeb-pkg Denis, I think we should go back to the original KGZIP, KBZIP2, KLZOP. > > --- > # bad: [cf0c97f148e9e50aa5a7ddd1984a604dd2bde4af] Merge tag 'pinctrl-v5.8-1' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl > # good: [aaa2faab4ed8e5fe0111e04d6e168c028fe2987f] Merge tag 'for-linus-5.8-ofs1' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux > git bisect start 'HEAD' 'aaa2faab4ed8' > # good: [77f55d1305c11fb729b88f2c3f7881ba0831fa6f] staging: rtl8723bs: Use common packet header constants > git bisect good 77f55d1305c11fb729b88f2c3f7881ba0831fa6f > # bad: [e611c0fe318c6d6827ee2bba660fbc23cf73f7dc] Merge tag 'usb-5.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb > git bisect bad e611c0fe318c6d6827ee2bba660fbc23cf73f7dc > # bad: [cff11abeca78aa782378401ca2800bd2194aa14e] Merge tag 'kbuild-v5.8' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild > git bisect bad cff11abeca78aa782378401ca2800bd2194aa14e > # good: [2bd81cd04a3f5eb873cc81fa16c469377be3b092] Merge branch 'remotes/lorenzo/pci/vmd' > git bisect good 2bd81cd04a3f5eb873cc81fa16c469377be3b092 > # good: [269a535ca931b754a40dda3ab60514e68773c759] modpost: generate vmlinux.symvers and reuse it for the second modpost > git bisect good 269a535ca931b754a40dda3ab60514e68773c759 > # good: [e542e0dc3ee3eafc46dd8e3073388079d69cace0] Merge branch 'dmi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jdelvare/staging > git bisect good e542e0dc3ee3eafc46dd8e3073388079d69cace0 > # good: [4de7b62936122570408357417f21072e78292926] modpost: remove is_vmlinux() helper > git bisect good 4de7b62936122570408357417f21072e78292926 > # good: [1ee18de92927f37e6948d5a6fc73cbf89f806905] Merge tag 'dma-mapping-5.8' of git://git.infradead.org/users/hch/dma-mapping > git bisect good 1ee18de92927f37e6948d5a6fc73cbf89f806905 > # bad: [8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294] kbuild: add variables for compression tools > git bisect bad 8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294 > # good: [c0901577e1dcc8d1c0fd1a11c8d571f650df845f] kbuild: doc: rename LDFLAGS to KBUILD_LDFLAGS > git bisect good c0901577e1dcc8d1c0fd1a11c8d571f650df845f > # good: [e0b250b57dcf403529081e5898a9de717f96b76b] Makefile: install modules.builtin even if CONFIG_MODULES=n > git bisect good e0b250b57dcf403529081e5898a9de717f96b76b > # first bad commit: [8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294] kbuild: add variables for compression tools
On 6/8/20 7:59 AM, Masahiro Yamada wrote: > On Mon, Jun 8, 2020 at 10:30 AM Guenter Roeck <linux@roeck-us.net> wrote: >> >> Hi, >> >> On Fri, Jun 05, 2020 at 10:39:55AM +0300, Denis Efremov wrote: >>> Allow user to use alternative implementations of compression tools, >>> such as pigz, pbzip2, pxz. For example, multi-threaded tools to >>> speed up the build: >>> $ make GZIP=pigz BZIP2=pbzip2 >>> >>> Variables _GZIP, _BZIP2, _LZOP are used internally because original env >>> vars are reserved by the tools. The use of GZIP in gzip tool is obsolete >>> since 2015. However, alternative implementations (e.g., pigz) still rely >>> on it. BZIP2, BZIP, LZOP vars are not obsolescent. >>> >> >> When building mips:defconfig, this patch results in: >> >> Building mips:defconfig ... failed >> -------------- >> Error log: >> /bin/sh: -n: command not found >> make[3]: *** [kernel/config_data.gz] Error 127 >> make[3]: *** Deleting file 'kernel/config_data.gz' >> make[3]: *** Waiting for unfinished jobs.... >> make[2]: *** [kernel] Error 2 >> make[2]: *** Waiting for unfinished jobs.... >> make[1]: *** [autoksyms_recursive] Error 2 >> make: *** [__sub-make] Error 2 >> >> Reverting this patch fixes the problem. Bisect log is attached. >> >> Guenter > I tried to reproduce it with cross-compilation on Fedora32. $ export ARCH=mips $ export CROSS_COMPILE=mips64-linux-gnu- $ make defconfig $ make -j12 And the kernel builds successfully. Could you please provide details about your compilation steps and environment, esp. what "env | grep ZIP" shows, "gzip --version", "sh --version", "bash --version"? This will be very helpful. Additionally: $ make GZIP=gzip -j12 # works $ make GZIP=pigz -j12 # works $ make GZIP=nosuchcommand -j12 # fails, as expected > > Agh, this is because of CONFIG_TRIM_UNUSED_KSYMS. > Hmm, it somehow works on my machine. But yes, this call looks like a problem for these env vars: autoksyms_recursive: descend modules.order $(Q)$(CONFIG_SHELL) $(srctree)/scripts/adjust_autoksyms.sh \ "$(MAKE) -f $(srctree)/Makefile vmlinux" > Also, the distro package builds are broken > e.g. make GZIP=gzip bindeb-pkg > Yes, thanks. > Denis, > > I think we should go back to the original > KGZIP, KBZIP2, KLZOP. > Given that the original patch is already in the Linus tree, I've sent a hotfix. Commit message is not perfect, because I didn't have enough time to deeply debug it. I just hope that the original patch didn't broke too many builds. Maybe later I will try to prepare a patch with GZIP again when I will fully debug these corner cases. Thanks, Denis
On 6/8/20 3:28 AM, Denis Efremov wrote: > > > On 6/8/20 7:59 AM, Masahiro Yamada wrote: >> On Mon, Jun 8, 2020 at 10:30 AM Guenter Roeck <linux@roeck-us.net> wrote: >>> >>> Hi, >>> >>> On Fri, Jun 05, 2020 at 10:39:55AM +0300, Denis Efremov wrote: >>>> Allow user to use alternative implementations of compression tools, >>>> such as pigz, pbzip2, pxz. For example, multi-threaded tools to >>>> speed up the build: >>>> $ make GZIP=pigz BZIP2=pbzip2 >>>> >>>> Variables _GZIP, _BZIP2, _LZOP are used internally because original env >>>> vars are reserved by the tools. The use of GZIP in gzip tool is obsolete >>>> since 2015. However, alternative implementations (e.g., pigz) still rely >>>> on it. BZIP2, BZIP, LZOP vars are not obsolescent. >>>> >>> >>> When building mips:defconfig, this patch results in: >>> >>> Building mips:defconfig ... failed >>> -------------- >>> Error log: >>> /bin/sh: -n: command not found >>> make[3]: *** [kernel/config_data.gz] Error 127 >>> make[3]: *** Deleting file 'kernel/config_data.gz' >>> make[3]: *** Waiting for unfinished jobs.... >>> make[2]: *** [kernel] Error 2 >>> make[2]: *** Waiting for unfinished jobs.... >>> make[1]: *** [autoksyms_recursive] Error 2 >>> make: *** [__sub-make] Error 2 >>> >>> Reverting this patch fixes the problem. Bisect log is attached. >>> >>> Guenter >> > > I tried to reproduce it with cross-compilation on Fedora32. > $ export ARCH=mips > $ export CROSS_COMPILE=mips64-linux-gnu- > $ make defconfig > $ make -j12 > > And the kernel builds successfully. Could you please provide details about your > compilation steps and environment, esp. what "env | grep ZIP" shows, > "gzip --version", "sh --version", "bash --version"? This will be very helpful. > > Additionally: > $ make GZIP=gzip -j12 # works > $ make GZIP=pigz -j12 # works > $ make GZIP=nosuchcommand -j12 # fails, as expected > I use make -j30 ARCH=mips CROSS_COMPILE=mips64-linux- ... ie I don't use environment variables. Using environment variables indeed makes the problem "disappear". Also, it is important to run "make mrproper" first to ensure that the generated file does not already exist. make -j30 ARCH=mips CROSS_COMPILE=mips64-linux- GZIP=gzip ... does not work for me. Guenter
diff --git a/Makefile b/Makefile index a7bc91cbac8f..7920404950c3 100644 --- a/Makefile +++ b/Makefile @@ -447,6 +447,26 @@ PYTHON = python PYTHON3 = python3 CHECK = sparse BASH = bash +GZIP = gzip +BZIP2 = bzip2 +LZOP = lzop +LZMA = lzma +LZ4 = lz4c +XZ = xz + +# GZIP, BZIP2, LZOP env vars are used by the tools. Support them as the command +# line interface, but use _GZIP, _BZIP2, _LZOP internally. +_GZIP := $(GZIP) +_BZIP2 := $(BZIP2) +_LZOP := $(LZOP) + +# Reset GZIP, BZIP2, LZOP in this Makefile +override GZIP= +override BZIP2= +override LZOP= + +# Reset GZIP, BZIP2, LZOP in recursive invocations +MAKEOVERRIDES += GZIP= BZIP2= LZOP= CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF) @@ -495,6 +515,7 @@ CLANG_FLAGS := export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX +export _GZIP _BZIP2 _LZOP LZMA LZ4 XZ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS @@ -1011,10 +1032,10 @@ export mod_strip_cmd mod_compress_cmd = true ifdef CONFIG_MODULE_COMPRESS ifdef CONFIG_MODULE_COMPRESS_GZIP - mod_compress_cmd = gzip -n -f + mod_compress_cmd = $(_GZIP) -n -f endif # CONFIG_MODULE_COMPRESS_GZIP ifdef CONFIG_MODULE_COMPRESS_XZ - mod_compress_cmd = xz -f + mod_compress_cmd = $(XZ) -f endif # CONFIG_MODULE_COMPRESS_XZ endif # CONFIG_MODULE_COMPRESS export mod_compress_cmd diff --git a/arch/arm/boot/deflate_xip_data.sh b/arch/arm/boot/deflate_xip_data.sh index 40937248cebe..739f0464321e 100755 --- a/arch/arm/boot/deflate_xip_data.sh +++ b/arch/arm/boot/deflate_xip_data.sh @@ -56,7 +56,7 @@ trap 'rm -f "$XIPIMAGE.tmp"; exit 1' 1 2 3 # substitute the data section by a compressed version $DD if="$XIPIMAGE" count=$data_start iflag=count_bytes of="$XIPIMAGE.tmp" $DD if="$XIPIMAGE" skip=$data_start iflag=skip_bytes | -gzip -9 >> "$XIPIMAGE.tmp" +$_GZIP -9 >> "$XIPIMAGE.tmp" # replace kernel binary mv -f "$XIPIMAGE.tmp" "$XIPIMAGE" diff --git a/arch/ia64/Makefile b/arch/ia64/Makefile index 32240000dc0c..f817f3d5e758 100644 --- a/arch/ia64/Makefile +++ b/arch/ia64/Makefile @@ -40,7 +40,7 @@ $(error Sorry, you need a newer version of the assember, one that is built from endif quiet_cmd_gzip = GZIP $@ -cmd_gzip = cat $(real-prereqs) | gzip -n -f -9 > $@ +cmd_gzip = cat $(real-prereqs) | $(_GZIP) -n -f -9 > $@ quiet_cmd_objcopy = OBJCOPY $@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@ diff --git a/arch/m68k/Makefile b/arch/m68k/Makefile index 5d9288384096..ce6db5e5a5a3 100644 --- a/arch/m68k/Makefile +++ b/arch/m68k/Makefile @@ -135,10 +135,10 @@ vmlinux.gz: vmlinux ifndef CONFIG_KGDB cp vmlinux vmlinux.tmp $(STRIP) vmlinux.tmp - gzip -9c vmlinux.tmp >vmlinux.gz + $(_GZIP) -9c vmlinux.tmp >vmlinux.gz rm vmlinux.tmp else - gzip -9c vmlinux >vmlinux.gz + $(_GZIP) -9c vmlinux >vmlinux.gz endif bzImage: vmlinux.bz2 @@ -148,10 +148,10 @@ vmlinux.bz2: vmlinux ifndef CONFIG_KGDB cp vmlinux vmlinux.tmp $(STRIP) vmlinux.tmp - bzip2 -1c vmlinux.tmp >vmlinux.bz2 + $(_BZIP2) -1c vmlinux.tmp >vmlinux.bz2 rm vmlinux.tmp else - bzip2 -1c vmlinux >vmlinux.bz2 + $(_BZIP2) -1c vmlinux >vmlinux.bz2 endif archclean: diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile index fadbbd010337..182a5bca3e2c 100644 --- a/arch/parisc/Makefile +++ b/arch/parisc/Makefile @@ -162,7 +162,7 @@ vmlinuz: bzImage $(OBJCOPY) $(boot)/bzImage $@ else vmlinuz: vmlinux - @gzip -cf -9 $< > $@ + @$(_GZIP) -cf -9 $< > $@ endif install: diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh index e13ca842eb7e..c1510f0ab3ea 100755 --- a/kernel/gen_kheaders.sh +++ b/kernel/gen_kheaders.sh @@ -88,7 +88,7 @@ find $cpio_dir -type f -print0 | find $cpio_dir -printf "./%P\n" | LC_ALL=C sort | \ tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \ --owner=0 --group=0 --numeric-owner --no-recursion \ - -Jcf $tarfile -C $cpio_dir/ -T - > /dev/null + -I $XZ -cf $tarfile -C $cpio_dir/ -T - > /dev/null echo $headers_md5 > kernel/kheaders.md5 echo "$this_file_md5" >> kernel/kheaders.md5 diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 4b799737722c..a1c1424e6b52 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -241,7 +241,7 @@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@ # --------------------------------------------------------------------------- quiet_cmd_gzip = GZIP $@ - cmd_gzip = cat $(real-prereqs) | gzip -n -f -9 > $@ + cmd_gzip = cat $(real-prereqs) | $(_GZIP) -n -f -9 > $@ # DTC # --------------------------------------------------------------------------- @@ -334,19 +334,19 @@ printf "%08x\n" $$dec_size | \ ) quiet_cmd_bzip2 = BZIP2 $@ - cmd_bzip2 = { cat $(real-prereqs) | bzip2 -9; $(size_append); } > $@ + cmd_bzip2 = { cat $(real-prereqs) | $(_BZIP2) -9; $(size_append); } > $@ # Lzma # --------------------------------------------------------------------------- quiet_cmd_lzma = LZMA $@ - cmd_lzma = { cat $(real-prereqs) | lzma -9; $(size_append); } > $@ + cmd_lzma = { cat $(real-prereqs) | $(LZMA) -9; $(size_append); } > $@ quiet_cmd_lzo = LZO $@ - cmd_lzo = { cat $(real-prereqs) | lzop -9; $(size_append); } > $@ + cmd_lzo = { cat $(real-prereqs) | $(_LZOP) -9; $(size_append); } > $@ quiet_cmd_lz4 = LZ4 $@ - cmd_lz4 = { cat $(real-prereqs) | lz4c -l -c1 stdin stdout; \ + cmd_lz4 = { cat $(real-prereqs) | $(LZ4) -l -c1 stdin stdout; \ $(size_append); } > $@ # U-Boot mkimage @@ -393,7 +393,7 @@ quiet_cmd_xzkern = XZKERN $@ $(size_append); } > $@ quiet_cmd_xzmisc = XZMISC $@ - cmd_xzmisc = cat $(real-prereqs) | xz --check=crc32 --lzma2=dict=1MiB > $@ + cmd_xzmisc = cat $(real-prereqs) | $(XZ) --check=crc32 --lzma2=dict=1MiB > $@ # ASM offsets # --------------------------------------------------------------------------- diff --git a/scripts/Makefile.package b/scripts/Makefile.package index 02135d2671a6..b2b6153af63a 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -45,7 +45,7 @@ if test "$(objtree)" != "$(srctree)"; then \ false; \ fi ; \ $(srctree)/scripts/setlocalversion --save-scmversion; \ -tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \ +tar -I $(_GZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \ --transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \ rm -f $(objtree)/.scmversion @@ -127,9 +127,9 @@ util/PERF-VERSION-GEN $(CURDIR)/$(perf-tar)/); \ tar rf $(perf-tar).tar $(perf-tar)/HEAD $(perf-tar)/PERF-VERSION-FILE; \ rm -r $(perf-tar); \ $(if $(findstring tar-src,$@),, \ -$(if $(findstring bz2,$@),bzip2, \ -$(if $(findstring gz,$@),gzip, \ -$(if $(findstring xz,$@),xz, \ +$(if $(findstring bz2,$@),$(_BZIP2), \ +$(if $(findstring gz,$@),$(_GZIP), \ +$(if $(findstring xz,$@),$(XZ), \ $(error unknown target $@)))) \ -f -9 $(perf-tar).tar) diff --git a/scripts/package/buildtar b/scripts/package/buildtar index 77c7caefede1..ad62c6879622 100755 --- a/scripts/package/buildtar +++ b/scripts/package/buildtar @@ -28,15 +28,15 @@ case "${1}" in opts= ;; targz-pkg) - opts=--gzip + opts="-I ${_GZIP}" tarball=${tarball}.gz ;; tarbz2-pkg) - opts=--bzip2 + opts="-I ${_BZIP2}" tarball=${tarball}.bz2 ;; tarxz-pkg) - opts=--xz + opts="-I ${XZ}" tarball=${tarball}.xz ;; *) diff --git a/scripts/xz_wrap.sh b/scripts/xz_wrap.sh index 7a2d372f4885..76e9cbcfbeab 100755 --- a/scripts/xz_wrap.sh +++ b/scripts/xz_wrap.sh @@ -20,4 +20,4 @@ case $SRCARCH in sparc) BCJ=--sparc ;; esac -exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB +exec $XZ --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB
Allow user to use alternative implementations of compression tools, such as pigz, pbzip2, pxz. For example, multi-threaded tools to speed up the build: $ make GZIP=pigz BZIP2=pbzip2 Variables _GZIP, _BZIP2, _LZOP are used internally because original env vars are reserved by the tools. The use of GZIP in gzip tool is obsolete since 2015. However, alternative implementations (e.g., pigz) still rely on it. BZIP2, BZIP, LZOP vars are not obsolescent. The credit goes to @grsecurity. As a sidenote, for multi-threaded lzma, xz compression one can use: $ export XZ_OPT="--threads=0" Signed-off-by: Denis Efremov <efremov@linux.com> --- Changes in v2: - _GZIP used instead of GZIP - tar commands altered to use tools from the vars Changes in v3: - _BZIP2 used instead of BZIP2 - _LZOP used instead of LZOP Changes in v4: - Unexports removed from Kbuild.include - MAKEOVERRIDES used in top Makefile - All variables checks removed from scripts Changes in v5: - Conflict with removed LASAT resolved Makefile | 25 +++++++++++++++++++++++-- arch/arm/boot/deflate_xip_data.sh | 2 +- arch/ia64/Makefile | 2 +- arch/m68k/Makefile | 8 ++++---- arch/parisc/Makefile | 2 +- kernel/gen_kheaders.sh | 2 +- scripts/Makefile.lib | 12 ++++++------ scripts/Makefile.package | 8 ++++---- scripts/package/buildtar | 6 +++--- scripts/xz_wrap.sh | 2 +- 10 files changed, 45 insertions(+), 24 deletions(-)