From patchwork Mon Mar 16 14:30:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Malat X-Patchwork-Id: 11440453 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 932BE13B1 for ; Mon, 16 Mar 2020 14:30:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 61DCC2051A for ; Mon, 16 Mar 2020 14:30:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=malat-biz.20150623.gappssmtp.com header.i=@malat-biz.20150623.gappssmtp.com header.b="qxWtyq5M" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731543AbgCPOaj (ORCPT ); Mon, 16 Mar 2020 10:30:39 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:43223 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731465AbgCPOai (ORCPT ); Mon, 16 Mar 2020 10:30:38 -0400 Received: by mail-wr1-f68.google.com with SMTP id b2so15265346wrj.10 for ; Mon, 16 Mar 2020 07:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=malat-biz.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g/dhNA3aNRvuvEsXtbiSUvtrpS+S/9UD1UxqNBwuDcQ=; b=qxWtyq5MBRY8apb3t42+Iz7Gi0OU//4zkGhKbYC3S7i3E+iMpkhz/+tevkHXnaLAJ3 5OMYowQkZap9gx6HcHygIryfYSd+qlj4GtW3/tciVRS+3g3r/ZmVsxSXWv7w4au8YBUn VEqX2m2A8RgvaxPHddsmJIiubtKXXrR4GCvkxB3fNKARkkJxrPm1EjdQbf4a0HXuFWtJ dZVv1Slir3cGguBRz4dir3EQsAszPZK7r5BfdzxUBVGG7Hoz2vUq0zAQfJFvzHN7Pjow mcGSZN/5oRB3WSFYcCEZ559aG7caEKcrctOB1g7XcRv1zm0VinMHLfOim2vz2zzfXzpJ gWug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g/dhNA3aNRvuvEsXtbiSUvtrpS+S/9UD1UxqNBwuDcQ=; b=amY3jqg+/NTrDWZdcDTgMqVmSaWNl6ASNLQhOTSTfYnVDK8hsgHlVrU0TTHSCt4to3 Wyxj1KGLwc8TPuA8O5H64aVSByEZmLHWuA3GDPjzOClhMqDpC/MfYQe2wILl4VLpSqCh vIqcV4wwPAnrVD7d6iEw0vNL5Fgbd4F+LLDpNbjWch2Ol8rqCXYwB4UI7hX9Ys19b8ZX OxUpjrpyvSjgnWOlFX3pf2b7Tr24qgqgoceE4OLxRuBHOVE/g2q+rs6slQ+owko97JWf t0zmnSS9Knbsx8PIFoSiJJRVWVLjKslIcnXXXPbP55DwRDg8a2X1gl+9HVgb5bYx01ei hlHg== X-Gm-Message-State: ANhLgQ0CaKUbepxEWRPRpUiu0uW3YyrmFStQ7EDUnThHFrQQj6dOHVKJ FHLsky/U+MvmwNVuqJqP9J39pPnioGgYFJsZ X-Google-Smtp-Source: ADFU+vvIOSAB0AsdRcUD6ucE8HedXZ2Nt6Bl7Qv1h5oNrkDS7C4cqzz+e40W4meA3nr+XOSYId967w== X-Received: by 2002:adf:e502:: with SMTP id j2mr16993870wrm.259.1584369032776; Mon, 16 Mar 2020 07:30:32 -0700 (PDT) Received: from ntb.Speedport_W_921V_1_46_000 (p57AF9474.dip0.t-ipconnect.de. [87.175.148.116]) by smtp.googlemail.com with ESMTPSA id z129sm31344330wmb.7.2020.03.16.07.30.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Mar 2020 07:30:31 -0700 (PDT) From: Petr Malat To: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: terrelln@fb.com, clm@fb.com, gregkh@linuxfoundation.org, keescook@chromium.org, Petr Malat Subject: [PATCH v2 1/2] lib: add support for ZSTD-compressed kernel Date: Mon, 16 Mar 2020 15:30:16 +0100 Message-Id: <20200316143018.1366-1-oss@malat.biz> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200316140745.GB4041840@kroah.com> References: <20200316140745.GB4041840@kroah.com> MIME-Version: 1.0 Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org Add support for extracting ZSTD-compressed kernel images, as well as ZSTD-compressed initramfs. ZSTD compression ratio is roughly 10% worst than xz, but the decompression is 10x faster. Currently, this is one of the optimal algorithms available in the kernel, as there isn't an algorithm, which would provide a better compression ratio and a shorter decompression time. Signed-off-by: Petr Malat --- include/linux/decompress/unzstd.h | 12 +++ init/Kconfig | 15 ++- lib/Kconfig | 4 + lib/Makefile | 1 + lib/decompress.c | 5 + lib/decompress_unzstd.c | 159 ++++++++++++++++++++++++++++++ lib/zstd/decompress.c | 2 + lib/zstd/fse_decompress.c | 4 +- scripts/Makefile.lib | 3 + usr/Kconfig | 24 +++++ 10 files changed, 227 insertions(+), 2 deletions(-) create mode 100644 include/linux/decompress/unzstd.h create mode 100644 lib/decompress_unzstd.c diff --git a/include/linux/decompress/unzstd.h b/include/linux/decompress/unzstd.h new file mode 100644 index 000000000000..dd2c49d47456 --- /dev/null +++ b/include/linux/decompress/unzstd.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef DECOMPRESS_UNZSTD_H +#define DECOMPRESS_UNZSTD_H + +int unzstd(unsigned char *inbuf, long len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *output, + long *pos, + void (*error)(char *x)); +#endif + diff --git a/init/Kconfig b/init/Kconfig index a34064a031a5..628eb3c290a2 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -172,13 +172,16 @@ config HAVE_KERNEL_LZO config HAVE_KERNEL_LZ4 bool +config HAVE_KERNEL_ZSTD + bool + config HAVE_KERNEL_UNCOMPRESSED bool choice prompt "Kernel compression mode" default KERNEL_GZIP - depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED + depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_ZSTD || HAVE_KERNEL_UNCOMPRESSED help The linux kernel is a kind of self-extracting executable. Several compression algorithms are available, which differ @@ -257,6 +260,16 @@ config KERNEL_LZ4 is about 8% bigger than LZO. But the decompression speed is faster than LZO. +config KERNEL_ZSTD + bool "ZSTD" + depends on HAVE_KERNEL_ZSTD + help + Its compression ratio is roughly 10% worst than xz, but the + decompression is 10x faster. Currently, this is one of the optimal + algorithms available in the kernel, as there isn't an algorithm, + which would provide a better compression ratio and a shorter + decompression time. + config KERNEL_UNCOMPRESSED bool "None" depends on HAVE_KERNEL_UNCOMPRESSED diff --git a/lib/Kconfig b/lib/Kconfig index 6e790dc55c5b..df301bd888d7 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -329,6 +329,10 @@ config DECOMPRESS_LZ4 select LZ4_DECOMPRESS tristate +config DECOMPRESS_ZSTD + select ZSTD_DECOMPRESS + tristate + # # Generic allocator support is selected if needed # diff --git a/lib/Makefile b/lib/Makefile index 93217d44237f..3ab9f4c31f8b 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -158,6 +158,7 @@ lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o lib-$(CONFIG_DECOMPRESS_XZ) += decompress_unxz.o lib-$(CONFIG_DECOMPRESS_LZO) += decompress_unlzo.o lib-$(CONFIG_DECOMPRESS_LZ4) += decompress_unlz4.o +lib-$(CONFIG_DECOMPRESS_ZSTD) += decompress_unzstd.o obj-$(CONFIG_TEXTSEARCH) += textsearch.o obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o diff --git a/lib/decompress.c b/lib/decompress.c index 857ab1af1ef3..ab3fc90ffc64 100644 --- a/lib/decompress.c +++ b/lib/decompress.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,9 @@ #ifndef CONFIG_DECOMPRESS_LZ4 # define unlz4 NULL #endif +#ifndef CONFIG_DECOMPRESS_ZSTD +# define unzstd NULL +#endif struct compress_format { unsigned char magic[2]; @@ -52,6 +56,7 @@ static const struct compress_format compressed_formats[] __initconst = { { {0xfd, 0x37}, "xz", unxz }, { {0x89, 0x4c}, "lzo", unlzo }, { {0x02, 0x21}, "lz4", unlz4 }, + { {0x28, 0xb5}, "zstd", unzstd }, { {0, 0}, NULL, NULL } }; diff --git a/lib/decompress_unzstd.c b/lib/decompress_unzstd.c new file mode 100644 index 000000000000..b8be89250033 --- /dev/null +++ b/lib/decompress_unzstd.c @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Wrapper for decompressing ZSTD-compressed kernel, initramfs, and initrd + * Based on decompress_unlz4.c + * + * Copyright (C) 2020, Petr Malat + */ + +#ifdef STATIC +#define PREBOOT +#include "zstd/zstd_internal.h" +#include "zstd/huf_decompress.c" +#include "zstd/entropy_common.c" +#include "zstd/fse_decompress.c" +#include "zstd/zstd_common.c" +#include "zstd/decompress.c" +#include "xxhash.c" +#else +#include +#include +#endif +#include +#include +#include + +STATIC inline int INIT unzstd(u8 *input, long in_len, + long (*fill)(void *, unsigned long), + long (*flush)(void *, unsigned long), + u8 *output, long *posp, + void (*error)(char *x)) +{ + int ret = -1, ws = 1 << ZSTD_WINDOWLOG_MAX; + u8 *inp, *outp; + ZSTD_DStream *zstd; + void *workspace; + size_t workspace_size; + ZSTD_outBuffer out; + ZSTD_inBuffer in; + unsigned long out_len; + unsigned long pos; + + if (output) { + out_len = ULONG_MAX; // Caller knows data will fit + outp = output; + } else if (!flush) { + error("NULL output pointer and no flush function provided"); + goto exit_0; + } else { + out_len = ZSTD_DStreamOutSize(); + outp = large_malloc(out_len); + if (!outp) { + error("Could not allocate output buffer"); + goto exit_0; + } + } + + if (input && fill) { + error("Both input pointer and fill function provided,"); + goto exit_1; + } else if (input) { + ZSTD_frameParams p; + + inp = input; + if (!ZSTD_getFrameParams(&p, input, in_len)) + ws = p.windowSize; + } else if (!fill) { + error("NULL input pointer and missing fill function"); + goto exit_1; + } else { + in_len = ZSTD_DStreamInSize(); + inp = large_malloc(in_len); + if (!inp) { + error("Could not allocate input buffer"); + goto exit_1; + } + } + + workspace_size = ZSTD_DStreamWorkspaceBound(ws); + workspace = large_malloc(workspace_size); + if (!workspace) { + error("Could not allocate workspace"); + goto exit_2; + } + + zstd = ZSTD_initDStream(ws, workspace, workspace_size); + if (!zstd) { + error("Could not initialize ZSTD"); + goto exit_3; + } + + in.src = inp; + in.size = in_len; + in.pos = 0; + if (posp) + *posp = 0; + + for (;;) { + if (fill) { + in.size = fill(inp, in_len); + if (in.size == 0) + break; + } else if (in.size == in.pos) { + break; + } +init: out.dst = outp; + out.size = out_len; + out.pos = 0; + pos = in.pos; + + ret = ZSTD_decompressStream(zstd, &out, &in); + if (posp) + *posp += in.pos - pos; + if (ZSTD_isError(ret)) { + error("Decompression failed"); + ret = -EIO; + goto exit_3; + } + + if (flush && out.pos) { + if (flush(out.dst, out.pos) != out.pos) { + ret = -EIO; + goto exit_3; + } + goto init; + } + + if (ret == 0) { + ret = ZSTD_resetDStream(zstd); + if (ZSTD_isError(ret)) { + ret = -EIO; + goto exit_3; + } + } + if (in.pos < in.size) + goto init; + } + + ret = 0; + +exit_3: large_free(workspace); +exit_2: if (!input) + large_free(inp); +exit_1: if (!output) + large_free(outp); +exit_0: return ret; +} + +#ifdef PREBOOT +STATIC int INIT __decompress(unsigned char *buf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *output, long out_len, + long *posp, + void (*error)(char *x) + ) +{ + return unzstd(buf, in_len, fill, flush, output, posp, error); +} +#endif diff --git a/lib/zstd/decompress.c b/lib/zstd/decompress.c index 269ee9a796c1..6a5e1ce22719 100644 --- a/lib/zstd/decompress.c +++ b/lib/zstd/decompress.c @@ -42,9 +42,11 @@ /*-************************************* * Macros ***************************************/ +#ifndef PREBOOT #define ZSTD_isError ERR_isError /* for inlining */ #define FSE_isError ERR_isError #define HUF_isError ERR_isError +#endif /*_******************************************************* * Memory operations diff --git a/lib/zstd/fse_decompress.c b/lib/zstd/fse_decompress.c index a84300e5a013..bd4e9c891d96 100644 --- a/lib/zstd/fse_decompress.c +++ b/lib/zstd/fse_decompress.c @@ -54,12 +54,13 @@ /* ************************************************************** * Error Management ****************************************************************/ -#define FSE_isError ERR_isError #define FSE_STATIC_ASSERT(c) \ { \ enum { FSE_static_assert = 1 / (int)(!!(c)) }; \ } /* use only *after* variable declarations */ +#ifndef PREBOOT +#define FSE_isError ERR_isError /* check and forward error code */ #define CHECK_F(f) \ { \ @@ -67,6 +68,7 @@ if (FSE_isError(e)) \ return e; \ } +#endif /* ************************************************************** * Templates diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 3fa32f83b2d7..1c2f2dc528dc 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -337,6 +337,9 @@ quiet_cmd_lz4 = LZ4 $@ cmd_lz4 = { cat $(real-prereqs) | lz4c -l -c1 stdin stdout; \ $(size_append); } > $@ +quiet_cmd_zstd = ZSTD $@ + cmd_zstd = { cat $(real-prereqs) | zstd -19 --zstd=wlog=21; $(size_append); } > $@ + # U-Boot mkimage # --------------------------------------------------------------------------- diff --git a/usr/Kconfig b/usr/Kconfig index a6b68503d177..892eb15957db 100644 --- a/usr/Kconfig +++ b/usr/Kconfig @@ -106,6 +106,15 @@ config RD_LZ4 Support loading of a LZ4 encoded initial ramdisk or cpio buffer If unsure, say N. +config RD_ZSTD + bool "Support initial ramdisk/ramfs compressed using ZSTD" + default y + depends on BLK_DEV_INITRD + select DECOMPRESS_ZSTD + help + Support loading of a ZSTD encoded initial ramdisk or cpio buffer + If unsure, say N. + choice prompt "Built-in initramfs compression mode" depends on INITRAMFS_SOURCE!="" @@ -214,6 +223,19 @@ config INITRAMFS_COMPRESSION_LZ4 If you choose this, keep in mind that most distros don't provide lz4 by default which could cause a build failure. +config INITRAMFS_COMPRESSION_ZSTD + bool "ZSTD" + depends on RD_ZSTD + help + Its compression ratio is roughly 10% worst than xz, but the + decompression is 10x faster. Currently, this is one of the optimal + algorithms available in the kernel, as there isn't an algorithm, + which would provide a better compression ratio and a shorter + decompression time. + + If you choose this, keep in mind that you may need to install the zstd + tool to be able to compress the initram. + endchoice config INITRAMFS_COMPRESSION @@ -226,10 +248,12 @@ config INITRAMFS_COMPRESSION default ".xz" if INITRAMFS_COMPRESSION_XZ default ".lzo" if INITRAMFS_COMPRESSION_LZO default ".lz4" if INITRAMFS_COMPRESSION_LZ4 + default ".zst" if INITRAMFS_COMPRESSION_ZSTD default ".gz" if RD_GZIP default ".lz4" if RD_LZ4 default ".lzo" if RD_LZO default ".xz" if RD_XZ default ".lzma" if RD_LZMA default ".bz2" if RD_BZIP2 + default ".zst" if RD_ZSTD default "" From patchwork Mon Mar 16 14:30:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Malat X-Patchwork-Id: 11440455 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 36FCB13B1 for ; Mon, 16 Mar 2020 14:30:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 17C0020674 for ; Mon, 16 Mar 2020 14:30:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=malat-biz.20150623.gappssmtp.com header.i=@malat-biz.20150623.gappssmtp.com header.b="pK4psSJa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731541AbgCPOam (ORCPT ); Mon, 16 Mar 2020 10:30:42 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:50193 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731465AbgCPOak (ORCPT ); Mon, 16 Mar 2020 10:30:40 -0400 Received: by mail-wm1-f68.google.com with SMTP id z13so1855355wml.0 for ; Mon, 16 Mar 2020 07:30:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=malat-biz.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R+Gvs9hVDDCMOY336QoblV4ZIg4hz1Qc4pgjDVSYrZw=; b=pK4psSJaWvRH2ShNntRanlxFZciHpDRYht0XxCrDqbpXASnkX8hwZGkm+pAGRmIRE5 EYCs81swbCNuLcnkFMX7lDn1MmRvSmA84ltkpxKWvKHFW1oFs+Sef7jDi7hfRFwsalKR SZGirVQLpL/Eh9zvjXI0s33o4t7okW7baFp/uSmbofDLUp8IunUjryBBbRraSND+HS8H dii3B7s2DSllNMsHgWVT0PZAxaA/A/QW50XMqp1n5ltKiXESHzKRZBUpY4Wq3goD/20X W1MqxSwts99LYy8f14jFSJkW2OUmB3kIoI/eko31fV7u5ExIVe5tQC6mjYAZAfQD+qvw PtEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R+Gvs9hVDDCMOY336QoblV4ZIg4hz1Qc4pgjDVSYrZw=; b=e3x4JOkyCMeFS2UR97qsYKZASXPBTbMUc7/U1qSE6MsCTpAv66Nx7S80+zPbtQHps4 DhNK2BQwQAUmcWQPO8jlcbL9c5BaecBbaqv5/qPo7TSynSfSsNX8GW73JTA3HAuXHGs0 qCS2XI/nXUWwvTy+Cj2B2jlRastmVSS5uHqDDrQ4OVbqj9uK0OWZ051DzeEO+csNQ5Fp fZbK3BAEPBzib9tnIohIWX8ZR75AjDvfbRUul4hxRtCgFOFDqZZAQdOnmOAPJOLtqxoM PTv0KkGmhGbNDKfecuJOYCI6Z/zpYefallrAcry34Q49mNnbqt00xS2/WbJATUrf+YWB k+mg== X-Gm-Message-State: ANhLgQ0kpjzGG798+6CacQxyp8jPKo8trcxheMqvRmvzIkErsnE762pV ne0qfFZXurS5xHB5oPDCBgB1BKkZDCtDfi5r X-Google-Smtp-Source: ADFU+vtV83xhjIA0Tdy7spH4K8paaf1O0RSR60YNVgjS1x/u7IoiaFyGaHwTE03P7JDLOrQr5vAnuw== X-Received: by 2002:a05:600c:2319:: with SMTP id 25mr28084151wmo.106.1584369036638; Mon, 16 Mar 2020 07:30:36 -0700 (PDT) Received: from ntb.Speedport_W_921V_1_46_000 (p57AF9474.dip0.t-ipconnect.de. [87.175.148.116]) by smtp.googlemail.com with ESMTPSA id z129sm31344330wmb.7.2020.03.16.07.30.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Mar 2020 07:30:35 -0700 (PDT) From: Petr Malat To: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: terrelln@fb.com, clm@fb.com, gregkh@linuxfoundation.org, keescook@chromium.org, Petr Malat Subject: [PATCH v2 2/2] x86: Enable support for ZSTD-compressed kernel Date: Mon, 16 Mar 2020 15:30:17 +0100 Message-Id: <20200316143018.1366-2-oss@malat.biz> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200316143018.1366-1-oss@malat.biz> References: <20200316140745.GB4041840@kroah.com> <20200316143018.1366-1-oss@malat.biz> MIME-Version: 1.0 Sender: linux-kbuild-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kbuild@vger.kernel.org ZSTD compression ratio is roughly 10% worst than xz, but the decompression is 10x faster. Currently, this is one of the optimal algorithms available in the kernel, as there isn't an algorithm, which would provide a better compression ratio and a shorter decompression time. Signed-off-by: Petr Malat --- arch/x86/Kconfig | 1 + arch/x86/boot/compressed/Makefile | 5 ++++- arch/x86/boot/compressed/misc.c | 4 ++++ arch/x86/include/asm/boot.h | 4 ++-- 4 files changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5e8949953660..b22312aae674 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -180,6 +180,7 @@ config X86 select HAVE_KERNEL_LZMA select HAVE_KERNEL_LZO select HAVE_KERNEL_XZ + select HAVE_KERNEL_ZSTD select HAVE_KPROBES select HAVE_KPROBES_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 1dac210f7d44..a87dc1e41772 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -24,7 +24,7 @@ OBJECT_FILES_NON_STANDARD := y KCOV_INSTRUMENT := n targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma \ - vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4 + vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4 vmlinux.bin.zst KBUILD_CFLAGS := -m$(BITS) -O2 KBUILD_CFLAGS += -fno-strict-aliasing $(call cc-option, -fPIE, -fPIC) @@ -145,6 +145,8 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE $(call if_changed,lzo) $(obj)/vmlinux.bin.lz4: $(vmlinux.bin.all-y) FORCE $(call if_changed,lz4) +$(obj)/vmlinux.bin.zst: $(vmlinux.bin.all-y) FORCE + $(call if_changed,zstd) suffix-$(CONFIG_KERNEL_GZIP) := gz suffix-$(CONFIG_KERNEL_BZIP2) := bz2 @@ -152,6 +154,7 @@ suffix-$(CONFIG_KERNEL_LZMA) := lzma suffix-$(CONFIG_KERNEL_XZ) := xz suffix-$(CONFIG_KERNEL_LZO) := lzo suffix-$(CONFIG_KERNEL_LZ4) := lz4 +suffix-$(CONFIG_KERNEL_ZSTD) := zst quiet_cmd_mkpiggy = MKPIGGY $@ cmd_mkpiggy = $(obj)/mkpiggy $< > $@ diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 9652d5c2afda..39e592d0e0b4 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -77,6 +77,10 @@ static int lines, cols; #ifdef CONFIG_KERNEL_LZ4 #include "../../../../lib/decompress_unlz4.c" #endif + +#ifdef CONFIG_KERNEL_ZSTD +#include "../../../../lib/decompress_unzstd.c" +#endif /* * NOTE: When adding a new decompressor, please update the analysis in * ../header.S. diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index 680c320363db..9838c183e9a8 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -24,9 +24,9 @@ # error "Invalid value for CONFIG_PHYSICAL_ALIGN" #endif -#ifdef CONFIG_KERNEL_BZIP2 +#if CONFIG_KERNEL_BZIP2 || CONFIG_KERNEL_ZSTD # define BOOT_HEAP_SIZE 0x400000 -#else /* !CONFIG_KERNEL_BZIP2 */ +#else /* !(CONFIG_KERNEL_BZIP2 || CONFIG_KERNEL_ZSTD) */ # define BOOT_HEAP_SIZE 0x10000 #endif