From patchwork Tue Nov 3 06:05:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BB76C388F7 for ; Tue, 3 Nov 2020 06:02:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B4E2222277 for ; Tue, 3 Nov 2020 06:02:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SZLN3AN3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727348AbgKCGCA (ORCPT ); Tue, 3 Nov 2020 01:02:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725958AbgKCGB6 (ORCPT ); Tue, 3 Nov 2020 01:01:58 -0500 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5298CC0617A6; Mon, 2 Nov 2020 22:01:58 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id t14so12861031pgg.1; Mon, 02 Nov 2020 22:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZZoF2o+tkvuq8VLagQ9LAmpzy23c60yClvw8C48QvBI=; b=SZLN3AN3zm5xSLI7gILhRwa4u4suQzvPFE4pgpPMr5FgJ0vG3RZy88QSEOkqeB+6Yl 2QdMYXxgCBmjKWX42O/b1TOeojmNLpi+rTZPMxuDQD5FrilB8+uF8CoqAudsyvkZSe36 Ug/dsQiTJ77rHm60rNXQlElI6m+dvHfF/sMMlgVzHwvzi8sQP6ErS+F6oj5xKIQMrVQn QrNGSpRXsfHay5qzCJpd++YBiFs0rCfESLninzIJQxHH5Uzy8jiqmGlSJtfu3kFdZe9e 8olBkN9BFBn7iOI8cFuklSu/ZKvZy+4o+nqUz3ohnGLI0IZMfbuVGzjXwmZCSLyjzrG9 1q1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZZoF2o+tkvuq8VLagQ9LAmpzy23c60yClvw8C48QvBI=; b=SUTKURbwP75gu3VMR5eqhfItEkFX9f6+1zTlZ8Oyw3AJAXIh3b6qbwiqMtHG82OD8C P9ShdHi5+P2TBszNoYJvxFrtN7g+TX0Y2rAO593mLv5qO6wA06VXNluk9YRUqTlrkvfA tLd0wkTw3IZ4SbkBf4iveuQ2zl9bbfRzuPA4W5pYIqmdOPgqbPB+Kv96iMveZ12jPWtc cDZN8PycqsbAGDj2Byo/YKRJStNIYPcvt9h8xanmygRlS+NbTpjijdocEsekAoLpqF4l kXQ0uozW7R18iRjeyQ8tAFBkzcXvAzh3i4Y+iYIpYorNAQpgZuGuW0bcd7fFnK23SQgh fMmA== X-Gm-Message-State: AOAM532tBTV+snRiQ2uxMH6a7xEiko3gI1IdSefrngL5Hx7FWvCcYvCR 9/Wu/jjzuG4JnSFURJcWDXQ= X-Google-Smtp-Source: ABdhPJxBVpR2ptBmfVgx2wM9Fvv6dTERN3mGOPced1qq0T1brHonHs8ZDXE7a7s2aGOjMj7K6tTx8g== X-Received: by 2002:a62:de44:0:b029:18a:ae59:4d74 with SMTP id h65-20020a62de440000b029018aae594d74mr13143074pfg.72.1604383317751; Mon, 02 Nov 2020 22:01:57 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:01:57 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 1/9] lib: zstd: Add zstd compatibility wrapper Date: Mon, 2 Nov 2020 22:05:27 -0800 Message-Id: <20201103060535.8460-2-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Adds zstd_compat.h which provides the necessary functions from the current zstd.h API. It is only active for zstd versions 1.4.6 and newer. That means it is disabled currently, but will become active when a later patch in this series updates the zstd library in the kernel to 1.4.6. This header allows the zstd upgrade to 1.4.6 without changing any callers, since they all include zstd through the compatibility wrapper. Later patches in this series transition each caller away from the compatibility wrapper. After all the callers have been transitioned away from the compatibility wrapper, the final patch in this series deletes it. Signed-off-by: Nick Terrell --- crypto/zstd.c | 2 +- fs/btrfs/zstd.c | 2 +- fs/f2fs/compress.c | 2 +- fs/squashfs/zstd_wrapper.c | 2 +- include/linux/zstd_compat.h | 116 ++++++++++++++++++++++++++++++++++++ lib/decompress_unzstd.c | 2 +- 6 files changed, 121 insertions(+), 5 deletions(-) create mode 100644 include/linux/zstd_compat.h diff --git a/crypto/zstd.c b/crypto/zstd.c index 1a3309f066f7..dcda3cad3b5c 100644 --- a/crypto/zstd.c +++ b/crypto/zstd.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include #include diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index 9a4871636c6c..a7367ff573d4 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include "misc.h" #include "compression.h" #include "ctree.h" diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c index 14262e0f1cd6..57a6360b9827 100644 --- a/fs/f2fs/compress.c +++ b/fs/f2fs/compress.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include #include "f2fs.h" #include "node.h" diff --git a/fs/squashfs/zstd_wrapper.c b/fs/squashfs/zstd_wrapper.c index b7cb1faa652d..f8c512a6204e 100644 --- a/fs/squashfs/zstd_wrapper.c +++ b/fs/squashfs/zstd_wrapper.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include #include #include "squashfs_fs.h" diff --git a/include/linux/zstd_compat.h b/include/linux/zstd_compat.h new file mode 100644 index 000000000000..cda9208bf04a --- /dev/null +++ b/include/linux/zstd_compat.h @@ -0,0 +1,116 @@ +/* + * Copyright (c) 2016-present, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under the BSD-style license found in the + * LICENSE file in the root directory of https://github.com/facebook/zstd. + * An additional grant of patent rights can be found in the PATENTS file in the + * same directory. + * + * This program is free software; you can redistribute it and/or modify it under + * the terms of the GNU General Public License version 2 as published by the + * Free Software Foundation. This program is dual-licensed; you may select + * either version 2 of the GNU General Public License ("GPL") or BSD license + * ("BSD"). + */ + +#ifndef ZSTD_COMPAT_H +#define ZSTD_COMPAT_H + +#include + +#if defined(ZSTD_VERSION_NUMBER) && (ZSTD_VERSION_NUMBER >= 10406) +/* + * This header provides backwards compatibility for the zstd-1.4.6 library + * upgrade. This header allows us to upgrade the zstd library version without + * modifying any callers. Then we will migrate callers from the compatibility + * wrapper one at a time until none remain. At which point we will delete this + * header. + * + * It is temporary and will be deleted once the upgrade is complete. + */ + +#include + +static inline size_t ZSTD_CCtxWorkspaceBound(ZSTD_compressionParameters compression_params) +{ + return ZSTD_estimateCCtxSize_usingCParams(compression_params); +} + +static inline size_t ZSTD_CStreamWorkspaceBound(ZSTD_compressionParameters compression_params) +{ + return ZSTD_estimateCStreamSize_usingCParams(compression_params); +} + +static inline size_t ZSTD_DCtxWorkspaceBound(void) +{ + return ZSTD_estimateDCtxSize(); +} + +static inline size_t ZSTD_DStreamWorkspaceBound(unsigned long long window_size) +{ + return ZSTD_estimateDStreamSize(window_size); +} + +static inline ZSTD_CCtx* ZSTD_initCCtx(void* wksp, size_t wksp_size) +{ + if (wksp == NULL) + return NULL; + return ZSTD_initStaticCCtx(wksp, wksp_size); +} + +static inline ZSTD_CStream* ZSTD_initCStream_compat(ZSTD_parameters params, uint64_t pledged_src_size, void* wksp, size_t wksp_size) +{ + ZSTD_CStream* cstream; + size_t ret; + + if (wksp == NULL) + return NULL; + + cstream = ZSTD_initStaticCStream(wksp, wksp_size); + if (cstream == NULL) + return NULL; + + /* 0 means unknown in old API but means 0 in new API */ + if (pledged_src_size == 0) + pledged_src_size = ZSTD_CONTENTSIZE_UNKNOWN; + + ret = ZSTD_initCStream_advanced(cstream, NULL, 0, params, pledged_src_size); + if (ZSTD_isError(ret)) + return NULL; + + return cstream; +} +#define ZSTD_initCStream ZSTD_initCStream_compat + +static inline ZSTD_DCtx* ZSTD_initDCtx(void* wksp, size_t wksp_size) +{ + if (wksp == NULL) + return NULL; + return ZSTD_initStaticDCtx(wksp, wksp_size); +} + +static inline ZSTD_DStream* ZSTD_initDStream_compat(unsigned long long window_size, void* wksp, size_t wksp_size) +{ + if (wksp == NULL) + return NULL; + (void)window_size; + return ZSTD_initStaticDStream(wksp, wksp_size); +} +#define ZSTD_initDStream ZSTD_initDStream_compat + +typedef ZSTD_frameHeader ZSTD_frameParams; + +static inline size_t ZSTD_getFrameParams(ZSTD_frameParams* frame_params, const void* src, size_t src_size) +{ + return ZSTD_getFrameHeader(frame_params, src, src_size); +} + +static inline size_t ZSTD_compressCCtx_compat(ZSTD_CCtx* cctx, void* dst, size_t dst_capacity, const void* src, size_t src_size, ZSTD_parameters params) +{ + return ZSTD_compress_advanced(cctx, dst, dst_capacity, src, src_size, NULL, 0, params); +} +#define ZSTD_compressCCtx ZSTD_compressCCtx_compat + +#endif /* ZSTD_VERSION_NUMBER >= 10406 */ +#endif /* ZSTD_COMPAT_H */ diff --git a/lib/decompress_unzstd.c b/lib/decompress_unzstd.c index 790abc472f5b..6bb805aeec08 100644 --- a/lib/decompress_unzstd.c +++ b/lib/decompress_unzstd.c @@ -77,7 +77,7 @@ #include #include -#include +#include /* 128MB is the maximum window size supported by zstd. */ #define ZSTD_WINDOWSIZE_MAX (1 << ZSTD_WINDOWLOG_MAX) From patchwork Tue Nov 3 06:05:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42EDBC388F7 for ; Tue, 3 Nov 2020 06:02:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD83B222EC for ; Tue, 3 Nov 2020 06:02:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JM0TrRYa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727452AbgKCGCE (ORCPT ); Tue, 3 Nov 2020 01:02:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725958AbgKCGCB (ORCPT ); Tue, 3 Nov 2020 01:02:01 -0500 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85069C061A47; Mon, 2 Nov 2020 22:02:01 -0800 (PST) Received: by mail-pl1-x644.google.com with SMTP id z1so8037691plo.12; Mon, 02 Nov 2020 22:02:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MWygwZ+ivEMIi25OaiUo1Zam2MncVHjO+BypAFXIYAo=; b=JM0TrRYa221OBNL+3xlD+e2qFNGW+oJCIa+nKENT/oK16ai1/3ZwfwHdCCnZqVlMYB E0bGwvZgYl0/v8cs1ZNEUjKZc35gP7wC+1gzWp/U5+9lLGjpLQwLbinx7EZOVtka9aqh uQqpNedOzM3YjO3AHViLpvAbo41ENlIS7TKOXGaXRMlaAuIHQnyQoVyML0Ukr6m10WHW GBDilgXFImL026l/ZahUnhrUvoCkxVD564hJ5CkoXicJDznfLp7wcpKbh2r71vd9Movt O8FkIttZkOI+aw3Oc6SekaaFb+q4R+5M5dK9b/qwhKYJfzeyE2SOubJ+2H5qtH+wPbug uZQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MWygwZ+ivEMIi25OaiUo1Zam2MncVHjO+BypAFXIYAo=; b=Iz009n+22Dw4LL/qVlMF7XOBOrUh52jxofvSVDa7ZgudubUeVipdcuXNvrqLLJDY1/ 5GSawlG7bvtjbpsdhG0M7HDmD/ZVf5Wlj/7aE+jyVx4kGmR/UhBQprX+MoBHkWSqTxdu My/GPGr/21ITYQGWgKZnqjE7IVZvTkKf1ndPIY1/h4EYhfGAomClZOLVm+tNEPMIVf5V JrAjJ27QX+9y3xa3Qjm52W+IlAsis/DeSforYtVJds1A6iDJ9WY4Vjo1fQJmt/dwMkMq 0x8CmyAi6ACN1X8KHZItgpyNUnC0NJTomhN0IzV2+8M7m0xRkKCoJb+Ie4DCC2Azkr57 4y7g== X-Gm-Message-State: AOAM532uMoGup6OrdPf0G1TzHSCCmUE0PyaqzqoWPZk22F9FmRau55mG 7mSWGJ8RxGmlDNHebhBnNGg= X-Google-Smtp-Source: ABdhPJwtMrMHoYdikDXTZBypvBnbSnctOVijmsKBxGO4u9Ixf+hOMd80RN9SqJluxhfUII1McD2Y7Q== X-Received: by 2002:a17:902:8693:b029:d5:d861:6f03 with SMTP id g19-20020a1709028693b02900d5d8616f03mr23725211plo.19.1604383320987; Mon, 02 Nov 2020 22:02:00 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.01.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:00 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 2/9] lib: zstd: Add decompress_sources.h for decompress_unzstd Date: Mon, 2 Nov 2020 22:05:28 -0800 Message-Id: <20201103060535.8460-3-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Adds decompress_sources.h which includes every .c file necessary for zstd decompression. This is used in decompress_unzstd.c so the internal structure of the library isn't exposed. This allows us to upgrade the zstd library version without modifying any callers. Instead we just need to update decompress_sources.h. Signed-off-by: Nick Terrell --- lib/decompress_unzstd.c | 6 +----- lib/zstd/decompress_sources.h | 14 ++++++++++++++ 2 files changed, 15 insertions(+), 5 deletions(-) create mode 100644 lib/zstd/decompress_sources.h diff --git a/lib/decompress_unzstd.c b/lib/decompress_unzstd.c index 6bb805aeec08..3c6ad01ffcd5 100644 --- a/lib/decompress_unzstd.c +++ b/lib/decompress_unzstd.c @@ -68,11 +68,7 @@ #ifdef STATIC # define UNZSTD_PREBOOT # include "xxhash.c" -# include "zstd/entropy_common.c" -# include "zstd/fse_decompress.c" -# include "zstd/huf_decompress.c" -# include "zstd/zstd_common.c" -# include "zstd/decompress.c" +# include "zstd/decompress_sources.h" #endif #include diff --git a/lib/zstd/decompress_sources.h b/lib/zstd/decompress_sources.h new file mode 100644 index 000000000000..ccb4960ea0cd --- /dev/null +++ b/lib/zstd/decompress_sources.h @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/* + * This file includes every .c file needed for decompression. + * It is used by lib/decompress_unzstd.c to include the decompression + * source into the translation-unit, so it can be used for kernel + * decompression. + */ + +#include "entropy_common.c" +#include "fse_decompress.c" +#include "huf_decompress.c" +#include "zstd_common.c" +#include "decompress.c" From patchwork Tue Nov 3 06:05:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA08BC388F9 for ; Tue, 3 Nov 2020 06:02:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 50A9422277 for ; Tue, 3 Nov 2020 06:02:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GSRu5L2P" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727489AbgKCGCU (ORCPT ); Tue, 3 Nov 2020 01:02:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727497AbgKCGCO (ORCPT ); Tue, 3 Nov 2020 01:02:14 -0500 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E97FC0617A6; Mon, 2 Nov 2020 22:02:14 -0800 (PST) Received: by mail-pg1-x544.google.com with SMTP id z24so12854388pgk.3; Mon, 02 Nov 2020 22:02:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5fKNeZSsXX3F75uWAvoclig0FCLwLLfStaZMUpC2Nwo=; b=GSRu5L2PQk84FeCeKEGbNFwcDIBC6ksTVwVMkVAwC2zdttpARRdY3h/s02DkFGdSf3 U4YQHvfYMojAb/457zl23L8sA37nL6Yqn5R3z0ncxLK/N7++XALuwmkNb7ElqwXGCAP2 r2fh/EEUiGZWmoaA/+dcWcsgN1dtNlGnBrUUTW4y9WHccgqneHCy0J5DL3FOQutjw0Ka f4xtF2vwzCiAST4tV0jiyRWkzymUmtgoL75rmdrjwpBHkXQfHDTMUPWcMIgyhBxoJmD7 J7NTYfM1tOsWVBzHQ6RSyLnDdYIacl8FDoad984Ey8Q3T3r66tnozU+RsMOLOenx1hiL YDeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5fKNeZSsXX3F75uWAvoclig0FCLwLLfStaZMUpC2Nwo=; b=GVZAzRsInAWuDvB7ABhzm4G3LZcsdYZLaYx/8lfgwTqrXaBWtzapdsygVht+Ng/L/Z N6RU2hMEepafWutf4ItlM+6BMTZxQwxmbIYTrG5twdKYXST+29ei1IWajMvXjXWzPx8N r9+eXDXKzee175HPwe8rFogWphwo1c0rHXSJzHd/pAgPvtvCMAjgZQIPN/0KYIohaEvN OfDYYMIaixY9q8AqSx+/qkSnXmbs1u2H4vHRNRQemVT65mI1iVfkh1WEHSeefyWLHlhj kvMabaNwMLmlXqz0RLef0LJgOYRjgul9kVwFhEMI3SbhMCTltsW2ULeMFOGYTT3oXMSN 8ioQ== X-Gm-Message-State: AOAM531PlobBUvRVtLmg34nLhbs7lTdf02gOxlXzbrtF5mZFqEtzssXu HSm37OCG7TE63FepA+UdPic= X-Google-Smtp-Source: ABdhPJydR/DseP0askKzZQYnimEkOFv3IDptlJLuF0BOY15TFxBu7o2wi88ZbnWG/atOCo+bJ49RYA== X-Received: by 2002:a63:174a:: with SMTP id 10mr16371117pgx.212.1604383333550; Mon, 02 Nov 2020 22:02:13 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.02.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:12 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 4/9] crypto: zstd: Switch to zstd-1.4.6 API Date: Mon, 2 Nov 2020 22:05:30 -0800 Message-Id: <20201103060535.8460-5-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is functionally equivalent. Signed-off-by: Nick Terrell --- crypto/zstd.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/crypto/zstd.c b/crypto/zstd.c index dcda3cad3b5c..767fe2fbe009 100644 --- a/crypto/zstd.c +++ b/crypto/zstd.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include #include @@ -24,16 +24,15 @@ struct zstd_ctx { void *dwksp; }; -static ZSTD_parameters zstd_params(void) -{ - return ZSTD_getParams(ZSTD_DEF_LEVEL, 0, 0); -} - static int zstd_comp_init(struct zstd_ctx *ctx) { int ret = 0; - const ZSTD_parameters params = zstd_params(); - const size_t wksp_size = ZSTD_CCtxWorkspaceBound(params.cParams); + const size_t wksp_size = ZSTD_estimateCCtxSize(ZSTD_DEF_LEVEL); + + if (ZSTD_isError(wksp_size)) { + ret = -EINVAL; + goto out_free; + } ctx->cwksp = vzalloc(wksp_size); if (!ctx->cwksp) { @@ -41,7 +40,7 @@ static int zstd_comp_init(struct zstd_ctx *ctx) goto out; } - ctx->cctx = ZSTD_initCCtx(ctx->cwksp, wksp_size); + ctx->cctx = ZSTD_initStaticCCtx(ctx->cwksp, wksp_size); if (!ctx->cctx) { ret = -EINVAL; goto out_free; @@ -56,7 +55,7 @@ static int zstd_comp_init(struct zstd_ctx *ctx) static int zstd_decomp_init(struct zstd_ctx *ctx) { int ret = 0; - const size_t wksp_size = ZSTD_DCtxWorkspaceBound(); + const size_t wksp_size = ZSTD_estimateDCtxSize(); ctx->dwksp = vzalloc(wksp_size); if (!ctx->dwksp) { @@ -64,7 +63,7 @@ static int zstd_decomp_init(struct zstd_ctx *ctx) goto out; } - ctx->dctx = ZSTD_initDCtx(ctx->dwksp, wksp_size); + ctx->dctx = ZSTD_initStaticDCtx(ctx->dwksp, wksp_size); if (!ctx->dctx) { ret = -EINVAL; goto out_free; @@ -152,9 +151,8 @@ static int __zstd_compress(const u8 *src, unsigned int slen, { size_t out_len; struct zstd_ctx *zctx = ctx; - const ZSTD_parameters params = zstd_params(); - out_len = ZSTD_compressCCtx(zctx->cctx, dst, *dlen, src, slen, params); + out_len = ZSTD_compressCCtx(zctx->cctx, dst, *dlen, src, slen, ZSTD_DEF_LEVEL); if (ZSTD_isError(out_len)) return -EINVAL; *dlen = out_len; From patchwork Tue Nov 3 06:05:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1302EC388F9 for ; Tue, 3 Nov 2020 06:02:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4E48222B9 for ; Tue, 3 Nov 2020 06:02:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iEqbPFcm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727457AbgKCGCd (ORCPT ); Tue, 3 Nov 2020 01:02:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727474AbgKCGCV (ORCPT ); Tue, 3 Nov 2020 01:02:21 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB323C0617A6; Mon, 2 Nov 2020 22:02:17 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id j18so13279423pfa.0; Mon, 02 Nov 2020 22:02:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2E/IwoJzArQOI0tLeIWcAV63W6gwJ92+ohjyeEw8pg8=; b=iEqbPFcmoBVXXCzrGQK2134vW+8Xz0YWFUGJNPGE6qJxp0tCM0nsNFXXC4h+P1xzAD RbBL+COQhh47jkqv5/sjqYoTs+bvTCVUYiVeJLSPeBykfty17A15g1dGz5KLHAwrnbnL jfvTqAeL/w3PsoiBSs4kOok9tzIiJPLdAgMdtRc3w3MZXnhWixUYI51c+QjDl8YrX7cE CUswOLRvdyxH4x/EqRZKpCMOjGikIZ3SgigzOwqhm3i9s0EVg6BbA8tpVlRSCnVYCyEt XCfTdYqaZXUBPJE+5ku4PMi6En22dNHhdVv82IsiWGYbeSNF2ra1lj/e0zSsQN/sNBy8 N/Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2E/IwoJzArQOI0tLeIWcAV63W6gwJ92+ohjyeEw8pg8=; b=K6ZFOjsOW/rR9Atxp04ih9lAosCe6LKjJaOkhtMbCIV09LelmCsJ/991TZuw5OzwaO 0QlY976DmbzPj1yaqSiRGCnBVMsOmK/dVScwUrM07NDXfRoCWxFw+sxxj1+pnxQMT0TR l9KCmEILMjlySPf8siqqssS0r9aEAJNBicjvCcKNovLzpzUPEvm49d0gaQevkG7FYD+S vOn7KSbRBKDL7TyzN4qTkeggNPUnv5L2Xlc2gSFWzGlhOssSu7mbvHg7hChKRxw9Orqk 6exL3q2xy51th2phdzU2+E5NIGuMj6qHvguqEQAVJSPY8UzgV694XD6aH/RzBluhwmkP JdWA== X-Gm-Message-State: AOAM5339836i1oUHfFj+la899Js4mlDNdR6rYeaLNPIfjzdt1Xd+cb67 YcTgSOsR8UfGIUenxqEwMHw= X-Google-Smtp-Source: ABdhPJyGVtb0kQhKxDOQ3rgCyFM7CMKxKblu6sRqk+tIeYGAbfG3LCMa/grSJQZbBh9QW8E7X+cIbQ== X-Received: by 2002:a63:c64:: with SMTP id 36mr5263358pgm.255.1604383337287; Mon, 02 Nov 2020 22:02:17 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.02.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:16 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 5/9] btrfs: zstd: Switch to the zstd-1.4.6 API Date: Mon, 2 Nov 2020 22:05:31 -0800 Message-Id: <20201103060535.8460-6-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is functionally equivalent. Signed-off-by: Nick Terrell Reviewed-by: Josef Bacik --- fs/btrfs/zstd.c | 48 ++++++++++++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index a7367ff573d4..6b466e090cd7 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include "misc.h" #include "compression.h" #include "ctree.h" @@ -159,8 +159,8 @@ static void zstd_calc_ws_mem_sizes(void) zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT); size_t level_size = max_t(size_t, - ZSTD_CStreamWorkspaceBound(params.cParams), - ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT)); + ZSTD_estimateCStreamSize_usingCParams(params.cParams), + ZSTD_estimateDStreamSize(ZSTD_BTRFS_MAX_INPUT)); max_size = max_t(size_t, max_size, level_size); zstd_ws_mem_sizes[level - 1] = max_size; @@ -389,13 +389,23 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, *total_in = 0; /* Initialize the stream */ - stream = ZSTD_initCStream(params, len, workspace->mem, - workspace->size); + stream = ZSTD_initStaticCStream(workspace->mem, workspace->size); if (!stream) { - pr_warn("BTRFS: ZSTD_initCStream failed\n"); + pr_warn("BTRFS: ZSTD_initStaticCStream failed\n"); ret = -EIO; goto out; } + { + size_t ret2; + + ret2 = ZSTD_initCStream_advanced(stream, NULL, 0, params, len); + if (ZSTD_isError(ret2)) { + pr_warn("BTRFS: ZSTD_initCStream_advanced returned %s\n", + ZSTD_getErrorName(ret2)); + ret = -EIO; + goto out; + } + } /* map in the first page of input data */ in_page = find_get_page(mapping, start >> PAGE_SHIFT); @@ -421,8 +431,8 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, ret2 = ZSTD_compressStream(stream, &workspace->out_buf, &workspace->in_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_compressStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_compressStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto out; } @@ -489,8 +499,8 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, ret2 = ZSTD_endStream(stream, &workspace->out_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_endStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_endStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto out; } @@ -557,10 +567,9 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb) unsigned long buf_start; unsigned long total_out = 0; - stream = ZSTD_initDStream( - ZSTD_BTRFS_MAX_INPUT, workspace->mem, workspace->size); + stream = ZSTD_initStaticDStream(workspace->mem, workspace->size); if (!stream) { - pr_debug("BTRFS: ZSTD_initDStream failed\n"); + pr_debug("BTRFS: ZSTD_initStaticDStream failed\n"); ret = -EIO; goto done; } @@ -579,8 +588,8 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb) ret2 = ZSTD_decompressStream(stream, &workspace->out_buf, &workspace->in_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_decompressStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_decompressStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto done; } @@ -633,10 +642,9 @@ int zstd_decompress(struct list_head *ws, unsigned char *data_in, unsigned long pg_offset = 0; char *kaddr; - stream = ZSTD_initDStream( - ZSTD_BTRFS_MAX_INPUT, workspace->mem, workspace->size); + stream = ZSTD_initStaticDStream(workspace->mem, workspace->size); if (!stream) { - pr_warn("BTRFS: ZSTD_initDStream failed\n"); + pr_warn("BTRFS: ZSTD_initStaticDStream failed\n"); ret = -EIO; goto finish; } @@ -667,8 +675,8 @@ int zstd_decompress(struct list_head *ws, unsigned char *data_in, ret2 = ZSTD_decompressStream(stream, &workspace->out_buf, &workspace->in_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_decompressStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_decompressStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto finish; } From patchwork Tue Nov 3 06:05:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2190C4742C for ; Tue, 3 Nov 2020 06:02:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7A3522227F for ; Tue, 3 Nov 2020 06:02:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gSSV2Azc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727632AbgKCGCd (ORCPT ); Tue, 3 Nov 2020 01:02:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727543AbgKCGCW (ORCPT ); Tue, 3 Nov 2020 01:02:22 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26982C061A47; Mon, 2 Nov 2020 22:02:21 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id g12so12837065pgm.8; Mon, 02 Nov 2020 22:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tfz5JoPFBe+5NZSwhHeml9GZoHiera3KaxD144JrSCQ=; b=gSSV2AzcSgKjWbA0/duEx/32yEqKcFXKR17Ctdoe1pcX/lfA9NvUrSBAIAJSl1Z7tj H2ui0Evi/5PTqJlwsDtcpVN419Jbs3OELOwS5RGAVngk2jRvNTKb/ZkAsm8Oe0/fXi4+ ckqV6Ge2q4B+j+A0/CRILjnvPQQHcR9Nw2Igf4qVYN/H3Rf8+LYVEcSdySSXengisu8U 64/db4Qsod9qGVW7k1Py7tK0/mg2MC2id1BrP7MklQRcBwyF3leReGVmsyiWK2RL8ZT/ d3x+aGj5IWmdxpL1LHBxjSXE4bFbeCYbSpdJXO5s+2vkwRlfEyRmCqBIVGgf8SoSrYxP NLxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tfz5JoPFBe+5NZSwhHeml9GZoHiera3KaxD144JrSCQ=; b=H1JevOqAZJa0HGPW2spY5mqIHK12Zja+YOEH4k1uRl505QDhZieL4E2J+QNHK0UHyp HG99BWXzwMo5LIhStkdlHAE7YHItwR5wkxJ4+bs4fUI9LqjQBLaOBB05Sq9N5IdXfOR+ gqndSzaLDv13EWltd5KAP3+5Fk5IkxxkvYz9pGCW2PNQoXABmmr3o1g89dQuript3eOW dkQuRPOOvYqiWPBBR3RxA3lWODb+kUrgBQVPrf3/cDiqSQ1R9D/nIrtdtHieY1w46GQK 4TuA6kSXdURQ+p4afUqQoC1LIkISZBio5eV1VbREJENcFdkLR7Azx8DmpsyNoJsH3T8k E6nw== X-Gm-Message-State: AOAM5315S1gSNEGEBmLY7+bXrZUXyA5YrOusPKPuMr/0n8X6KJE19O8I n6vuvFuYshTGomMpvpZGzgs= X-Google-Smtp-Source: ABdhPJypwJ893RanOWhi2oqXdSk3EUdYWLWR9gKh4oGpMiSgrcRpGfTkc49X7X3lz+5ApmK445SP+Q== X-Received: by 2002:a63:389:: with SMTP id 131mr3966874pgd.128.1604383340660; Mon, 02 Nov 2020 22:02:20 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.02.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:20 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 6/9] f2fs: zstd: Switch to the zstd-1.4.6 API Date: Mon, 2 Nov 2020 22:05:32 -0800 Message-Id: <20201103060535.8460-7-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is more efficient because it uses the single-pass API instead of the streaming API. The streaming API is not necessary because the whole input and output buffers are available. This saves memory because we don't need to allocate a buffer for the window. It is also more efficient because it saves unnecessary memcpy calls. Compression memory increases from 168 KB to 204 KB because upstream uses slightly more memory. Decompression memory decreases from 1.4 MB to 158 KB. Signed-off-by: Nick Terrell --- fs/f2fs/compress.c | 101 +++++++++++++++++---------------------------- 1 file changed, 37 insertions(+), 64 deletions(-) diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c index 57a6360b9827..8f8234877666 100644 --- a/fs/f2fs/compress.c +++ b/fs/f2fs/compress.c @@ -11,7 +11,8 @@ #include #include #include -#include +#include +#include #include "f2fs.h" #include "node.h" @@ -322,21 +323,21 @@ static const struct f2fs_compress_ops f2fs_lz4_ops = { static int zstd_init_compress_ctx(struct compress_ctx *cc) { ZSTD_parameters params; - ZSTD_CStream *stream; + ZSTD_CCtx *ctx; void *workspace; unsigned int workspace_size; params = ZSTD_getParams(F2FS_ZSTD_DEFAULT_CLEVEL, cc->rlen, 0); - workspace_size = ZSTD_CStreamWorkspaceBound(params.cParams); + workspace_size = ZSTD_estimateCCtxSize_usingCParams(params.cParams); workspace = f2fs_kvmalloc(F2FS_I_SB(cc->inode), workspace_size, GFP_NOFS); if (!workspace) return -ENOMEM; - stream = ZSTD_initCStream(params, 0, workspace, workspace_size); - if (!stream) { - printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_initCStream failed\n", + ctx = ZSTD_initStaticCCtx(workspace, workspace_size); + if (!ctx) { + printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_inittaticCStream failed\n", KERN_ERR, F2FS_I_SB(cc->inode)->sb->s_id, __func__); kvfree(workspace); @@ -344,7 +345,7 @@ static int zstd_init_compress_ctx(struct compress_ctx *cc) } cc->private = workspace; - cc->private2 = stream; + cc->private2 = ctx; cc->clen = cc->rlen - PAGE_SIZE - COMPRESS_HEADER_SIZE; return 0; @@ -359,66 +360,48 @@ static void zstd_destroy_compress_ctx(struct compress_ctx *cc) static int zstd_compress_pages(struct compress_ctx *cc) { - ZSTD_CStream *stream = cc->private2; - ZSTD_inBuffer inbuf; - ZSTD_outBuffer outbuf; - int src_size = cc->rlen; - int dst_size = src_size - PAGE_SIZE - COMPRESS_HEADER_SIZE; - int ret; - - inbuf.pos = 0; - inbuf.src = cc->rbuf; - inbuf.size = src_size; - - outbuf.pos = 0; - outbuf.dst = cc->cbuf->cdata; - outbuf.size = dst_size; + ZSTD_CCtx *ctx = cc->private2; + const size_t src_size = cc->rlen; + const size_t dst_size = src_size - PAGE_SIZE - COMPRESS_HEADER_SIZE; + ZSTD_parameters params = ZSTD_getParams(F2FS_ZSTD_DEFAULT_CLEVEL, src_size, 0); + size_t ret; - ret = ZSTD_compressStream(stream, &outbuf, &inbuf); + ret = ZSTD_compress_advanced( + ctx, cc->cbuf->cdata, dst_size, cc->rbuf, src_size, NULL, 0, params); if (ZSTD_isError(ret)) { - printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_compressStream failed, ret: %d\n", - KERN_ERR, F2FS_I_SB(cc->inode)->sb->s_id, - __func__, ZSTD_getErrorCode(ret)); - return -EIO; - } - - ret = ZSTD_endStream(stream, &outbuf); - if (ZSTD_isError(ret)) { - printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_endStream returned %d\n", + /* + * there is compressed data remained in intermediate buffer due to + * no more space in cbuf.cdata + */ + if (ZSTD_getErrorCode(ret) == ZSTD_error_dstSize_tooSmall) + return -EAGAIN; + /* other compression errors return -EIO */ + printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_compress_advanced failed, err: %s\n", KERN_ERR, F2FS_I_SB(cc->inode)->sb->s_id, - __func__, ZSTD_getErrorCode(ret)); + __func__, ZSTD_getErrorName(ret)); return -EIO; } - /* - * there is compressed data remained in intermediate buffer due to - * no more space in cbuf.cdata - */ - if (ret) - return -EAGAIN; - - cc->clen = outbuf.pos; + cc->clen = ret; return 0; } static int zstd_init_decompress_ctx(struct decompress_io_ctx *dic) { - ZSTD_DStream *stream; + ZSTD_DCtx *ctx; void *workspace; unsigned int workspace_size; - unsigned int max_window_size = - MAX_COMPRESS_WINDOW_SIZE(dic->log_cluster_size); - workspace_size = ZSTD_DStreamWorkspaceBound(max_window_size); + workspace_size = ZSTD_estimateDCtxSize(); workspace = f2fs_kvmalloc(F2FS_I_SB(dic->inode), workspace_size, GFP_NOFS); if (!workspace) return -ENOMEM; - stream = ZSTD_initDStream(max_window_size, workspace, workspace_size); - if (!stream) { - printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_initDStream failed\n", + ctx = ZSTD_initStaticDCtx(workspace, workspace_size); + if (!ctx) { + printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_initStaticDCtx failed\n", KERN_ERR, F2FS_I_SB(dic->inode)->sb->s_id, __func__); kvfree(workspace); @@ -426,7 +409,7 @@ static int zstd_init_decompress_ctx(struct decompress_io_ctx *dic) } dic->private = workspace; - dic->private2 = stream; + dic->private2 = ctx; return 0; } @@ -440,28 +423,18 @@ static void zstd_destroy_decompress_ctx(struct decompress_io_ctx *dic) static int zstd_decompress_pages(struct decompress_io_ctx *dic) { - ZSTD_DStream *stream = dic->private2; - ZSTD_inBuffer inbuf; - ZSTD_outBuffer outbuf; - int ret; - - inbuf.pos = 0; - inbuf.src = dic->cbuf->cdata; - inbuf.size = dic->clen; - - outbuf.pos = 0; - outbuf.dst = dic->rbuf; - outbuf.size = dic->rlen; + ZSTD_DCtx *ctx = dic->private2; + size_t ret; - ret = ZSTD_decompressStream(stream, &outbuf, &inbuf); + ret = ZSTD_decompressDCtx(ctx, dic->rbuf, dic->rlen, dic->cbuf->cdata, dic->clen); if (ZSTD_isError(ret)) { - printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_compressStream failed, ret: %d\n", + printk_ratelimited("%sF2FS-fs (%s): %s ZSTD_decompressDCtx failed, err: %s\n", KERN_ERR, F2FS_I_SB(dic->inode)->sb->s_id, - __func__, ZSTD_getErrorCode(ret)); + __func__, ZSTD_getErrorName(ret)); return -EIO; } - if (dic->rlen != outbuf.pos) { + if (dic->rlen != ret) { printk_ratelimited("%sF2FS-fs (%s): %s ZSTD invalid rlen:%zu, " "expected:%lu\n", KERN_ERR, F2FS_I_SB(dic->inode)->sb->s_id, From patchwork Tue Nov 3 06:05:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875979 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1787FC388F9 for ; Tue, 3 Nov 2020 06:02:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BB7352227F for ; Tue, 3 Nov 2020 06:02:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tmkhJaJU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727667AbgKCGCf (ORCPT ); Tue, 3 Nov 2020 01:02:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727243AbgKCGCY (ORCPT ); Tue, 3 Nov 2020 01:02:24 -0500 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F180C061A48; Mon, 2 Nov 2020 22:02:24 -0800 (PST) Received: by mail-pl1-x644.google.com with SMTP id z1so8038132plo.12; Mon, 02 Nov 2020 22:02:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=290Bo0sJTh/t678nA8yWOfHRoSh5E7Ekj1ME0hd0Klk=; b=tmkhJaJU3VHkDyTbru8lROAJsbWUowjJpfjeebweaYui9+dmpz9dp4CiqwVroJXu4r e8qFLTGO7BrrkBcyAJfn8s2nTfvG5gjJ1khl+BjW/1+PVJUxqn9EqV6ZrD0gvug7z8gA eY4KuYCaGxS6cQdPkeOmoQrc8TarN3zTBK2tbwqJ7YpAUJ56ov1ZOXJtfACGI+zExXPc Esk7bBmXTyf2iTDq9UrZRBWZgsW/jfXOYpNJA72ELOZo6kAHQHKxX3i+3VligPWKziIs cVthbjCjS21wLR2JkMrbXMLO8Y4DC/h+kZ4de4lLQ9n3XmynYbfmAnxyI0QNS8VeNWXW 0rjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=290Bo0sJTh/t678nA8yWOfHRoSh5E7Ekj1ME0hd0Klk=; b=iAuF+Tiz11XuyfnFEd+l2QYc2utjMo6848eimwuK7YqP5kDuDMK7jYsN123Zh5EzIM OAKMTU+qpudWgfd1zdaL7/2uxHiU2EIcbpQEVi/ibQpV0alWoaZ2zVJV+svWrIqCuHZT FJ69/43gPC76KGtTqRCFChpxoF3x1OQruWVXUYLlTP6K5j95/5YV3sRXkyvqENAqgvM2 FUZw1pOy6WYmLxdWq9Xce++awygBUiYIEqQdXT/6UI7Xlknm4B95ae2FSjjvR7bja75V OtJMxdF9WrSE2UbRGYs7D5qYJPrQMOGj+9FOLNZIOYd+MuYYDiaOgOOSjQFvuGjLGAVN ejAA== X-Gm-Message-State: AOAM532Hl7SzLUwJGmiHHcaywD5M0drAqnqbJf7DC4bueu0CL7aavtks MpllS0lJ6gw3MB4qVOrQHBdCIQudkkc= X-Google-Smtp-Source: ABdhPJxRj4C0s/7LRqkG1TAIaPpnRoWT0V5FJT5A21qtBTEslWRfNbGEXSp9WpAS5/zlFWRkm2d7KQ== X-Received: by 2002:a17:902:8d95:b029:d6:521f:2f2d with SMTP id v21-20020a1709028d95b02900d6521f2f2dmr24445166plo.76.1604383343728; Mon, 02 Nov 2020 22:02:23 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.02.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:23 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 7/9] squashfs: zstd: Switch to the zstd-1.4.6 API Date: Mon, 2 Nov 2020 22:05:33 -0800 Message-Id: <20201103060535.8460-8-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is functionally equivalent. Signed-off-by: Nick Terrell --- fs/squashfs/zstd_wrapper.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/squashfs/zstd_wrapper.c b/fs/squashfs/zstd_wrapper.c index f8c512a6204e..add582409866 100644 --- a/fs/squashfs/zstd_wrapper.c +++ b/fs/squashfs/zstd_wrapper.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include #include #include "squashfs_fs.h" @@ -34,7 +34,7 @@ static void *zstd_init(struct squashfs_sb_info *msblk, void *buff) goto failed; wksp->window_size = max_t(size_t, msblk->block_size, SQUASHFS_METADATA_SIZE); - wksp->mem_size = ZSTD_DStreamWorkspaceBound(wksp->window_size); + wksp->mem_size = ZSTD_estimateDStreamSize(wksp->window_size); wksp->mem = vmalloc(wksp->mem_size); if (wksp->mem == NULL) goto failed; @@ -71,7 +71,7 @@ static int zstd_uncompress(struct squashfs_sb_info *msblk, void *strm, struct bvec_iter_all iter_all = {}; struct bio_vec *bvec = bvec_init_iter_all(&iter_all); - stream = ZSTD_initDStream(wksp->window_size, wksp->mem, wksp->mem_size); + stream = ZSTD_initStaticDStream(wksp->mem, wksp->mem_size); if (!stream) { ERROR("Failed to initialize zstd decompressor\n"); @@ -122,8 +122,7 @@ static int zstd_uncompress(struct squashfs_sb_info *msblk, void *strm, break; if (ZSTD_isError(zstd_err)) { - ERROR("zstd decompression error: %d\n", - (int)ZSTD_getErrorCode(zstd_err)); + ERROR("zstd decompression error: %s\n", ZSTD_getErrorName(zstd_err)); error = -EIO; break; } From patchwork Tue Nov 3 06:05:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3D02C388F7 for ; Tue, 3 Nov 2020 06:02:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 628D122277 for ; Tue, 3 Nov 2020 06:02:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fqlEVbK3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727673AbgKCGCh (ORCPT ); Tue, 3 Nov 2020 01:02:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727308AbgKCGC2 (ORCPT ); Tue, 3 Nov 2020 01:02:28 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6388DC061A49; Mon, 2 Nov 2020 22:02:27 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id r10so12826703pgb.10; Mon, 02 Nov 2020 22:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z11dyIW9wYOT1WRGrY3n/HRYqfGSZJkJPYcfMlfSK1k=; b=fqlEVbK35EPh+UkmbfTOML9wO/Yu6UWciXnYiqqdEFBWDytHVrYmbYSbG12dv3lj29 ijQLun9BJcZFWXiFfb7KP6SyOle0F7XUCtEdyucxYLOXj8HjJeyih7omsHvYll7LqUXo Ezth8S3rW1LfHcLdV+yztMhArCw4MGetq8yF2xgV953ex+dYZdd95rMdMI/ZjReJM31v OLpBsuFM19OuIh9mQqk1Ibz47smDqth/3X5EdOHwsL24siNZb4S+19Bxoqf/gbZeNE66 tS4lJIZ8sg6y5/BkJic3oQgWYhl+xHOgAJppgO4t86xw4TwHcW3qcydvarNH2ufdjq/e DuxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z11dyIW9wYOT1WRGrY3n/HRYqfGSZJkJPYcfMlfSK1k=; b=fUrXlGrfrT0/chZXLdoDZkOTDjwEiZUf4mpPg9DjYVKHdoOWxxAcDnzTo8qZP3Lh62 LQ95O/8TKYdMdLD40bMj4YGkzb/vGbuAoEK6qoDj8rrV9oKXEwYJkIwTWS9Uk/z6XtmX 4fmH34LOLV9fzC/4KwhjdRjrNcC9S4F9TfmsCekKiqiyEajqQbgau3JqRkQwbV2V4eVx vIarYu30mce7yjDyieThOcONWgPvSS+0ORKHmbw7J+MRymmsBkFuJdC3RootFW9yc6fj vhamsluca5qBcdzYrPZ53nCN2exjQmHFer/j3ts4q0k2TPiuM1uS5IRvuNByQIg0S7TO TdOA== X-Gm-Message-State: AOAM531FNOAupjK58Oge71fGD+Ax3TOPlGq79IVfDKc1xmFBZQiKpSn5 LqOwv6xmczQu9QTd7SiZA1s= X-Google-Smtp-Source: ABdhPJzczT8OSnH39dRzgGhVfqu8jd5/AMVdwt8aIEY9Xprsdx50dQXpPz4sAoL7yZ+FMm8SFeMYSw== X-Received: by 2002:a62:4e0f:0:b029:156:13e0:efa7 with SMTP id c15-20020a624e0f0000b029015613e0efa7mr24248296pfb.73.1604383346862; Mon, 02 Nov 2020 22:02:26 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.02.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:26 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 8/9] lib: unzstd: Switch to the zstd-1.4.6 API Date: Mon, 2 Nov 2020 22:05:34 -0800 Message-Id: <20201103060535.8460-9-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is functionally equivalent. Signed-off-by: Nick Terrell --- lib/decompress_unzstd.c | 40 ++++++++++++++-------------------------- 1 file changed, 14 insertions(+), 26 deletions(-) diff --git a/lib/decompress_unzstd.c b/lib/decompress_unzstd.c index 3c6ad01ffcd5..efbe66501b34 100644 --- a/lib/decompress_unzstd.c +++ b/lib/decompress_unzstd.c @@ -73,7 +73,8 @@ #include #include -#include +#include +#include /* 128MB is the maximum window size supported by zstd. */ #define ZSTD_WINDOWSIZE_MAX (1 << ZSTD_WINDOWLOG_MAX) @@ -120,9 +121,9 @@ static int INIT decompress_single(const u8 *in_buf, long in_len, u8 *out_buf, long out_len, long *in_pos, void (*error)(char *x)) { - const size_t wksp_size = ZSTD_DCtxWorkspaceBound(); + const size_t wksp_size = ZSTD_estimateDCtxSize(); void *wksp = large_malloc(wksp_size); - ZSTD_DCtx *dctx = ZSTD_initDCtx(wksp, wksp_size); + ZSTD_DCtx *dctx = ZSTD_initStaticDCtx(wksp, wksp_size); int err; size_t ret; @@ -165,7 +166,6 @@ static int INIT __unzstd(unsigned char *in_buf, long in_len, { ZSTD_inBuffer in; ZSTD_outBuffer out; - ZSTD_frameParams params; void *in_allocated = NULL; void *out_allocated = NULL; void *wksp = NULL; @@ -234,36 +234,24 @@ static int INIT __unzstd(unsigned char *in_buf, long in_len, out.size = out_len; /* - * We need to know the window size to allocate the ZSTD_DStream. - * Since we are streaming, we need to allocate a buffer for the sliding - * window. The window size varies from 1 KB to ZSTD_WINDOWSIZE_MAX - * (8 MB), so it is important to use the actual value so as not to - * waste memory when it is smaller. + * Zstd determines the workspace size from the window size written + * into the frame header. This ensures that we use the minimum value + * possible, since the window size varies from 1 KB to ZSTD_WINDOWSIZE_MAX + * (1 GB), so it is very important to use the actual value. */ - ret = ZSTD_getFrameParams(¶ms, in.src, in.size); + wksp_size = ZSTD_estimateDStreamSize_fromFrame(in.src, in.size); err = handle_zstd_error(ret, error); if (err) goto out; - if (ret != 0) { - error("ZSTD-compressed data has an incomplete frame header"); - err = -1; - goto out; - } - if (params.windowSize > ZSTD_WINDOWSIZE_MAX) { - error("ZSTD-compressed data has too large a window size"); + wksp = large_malloc(wksp_size); + if (wksp == NULL) { + error("Out of memory while allocating ZSTD_DStream"); err = -1; goto out; } - - /* - * Allocate the ZSTD_DStream now that we know how much memory is - * required. - */ - wksp_size = ZSTD_DStreamWorkspaceBound(params.windowSize); - wksp = large_malloc(wksp_size); - dstream = ZSTD_initDStream(params.windowSize, wksp, wksp_size); + dstream = ZSTD_initStaticDStream(wksp, wksp_size); if (dstream == NULL) { - error("Out of memory while allocating ZSTD_DStream"); + error("ZSTD_initStaticDStream failed"); err = -1; goto out; } From patchwork Tue Nov 3 06:05:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 11875973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93CFBC388F7 for ; Tue, 3 Nov 2020 06:02:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4EE502227F for ; Tue, 3 Nov 2020 06:02:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="p7KlBBGL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727649AbgKCGCf (ORCPT ); Tue, 3 Nov 2020 01:02:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbgKCGCb (ORCPT ); Tue, 3 Nov 2020 01:02:31 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04A20C061A4A; Mon, 2 Nov 2020 22:02:31 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id i7so10906478pgh.6; Mon, 02 Nov 2020 22:02:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s2HCAxyO+XuYMLzto3YJY+q/fwyGyhZhVewL5HVF+qc=; b=p7KlBBGLft8dFPRJKfCutgbwfRd5yWhKlvBFON2+wv9VJrvR36DECWzNPpd/n36IED ZY0e3i7ASgkz8//vQFonbgL6nACypaK/9bB/rFS8/mbs+s99xOsmELOb7X76+AY87EkX rfN5NkpWRgMGfMNIN2yHfvMIluVdHup00vUJg3V0N+NaVvWtWdoDex4v71QSWSbtedcO +MhF/WSvq3paW4XfKrZKn+p0I3jhwaRoji0S1tA2iCFfy/ows40uFc/cGQVycZZ0Kr9F nt3F6WSayHQmY7NOWGOttU7E+CmTdBR9oQ3w1cC/n9+gzDo2m3549nYpRVaXg9VAm07U kkVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s2HCAxyO+XuYMLzto3YJY+q/fwyGyhZhVewL5HVF+qc=; b=dBoZqClT4gjQahzoTvToT2m3AfLWbMrHXsT2rdUIihy//hkMryzo+IANc0bI9EIT4c JjV55Kpy9Gsk2w1COu4oD7v59/wZvRZ/ivvgBrSmc0Ls5WTUgrsVGKfPiJpivb/Za1l4 D893GyV5toExcAcIKr1BF9FJZVRnRiduXGMrMUVbHG8JPd4we46odWglRZ1hHudZPQj8 /QT06OQAwvU6mfuA36C6vQ8r/7yYx51pX0Qcf4TnG0QqBKi1fH9C06dpUFN6MqD2YLGC TpSLtupv+XQMWNE3fhsBHGry143iWGDMfM0HC4tF3PlF4kXadphw2RukM+poj7ZD/Lvl 2dWQ== X-Gm-Message-State: AOAM531A5SojjedszKOwR9DE7CLPJbQ1vrMUotBhnbdsX/VpjTCR3mhL x8u8eFWz62frYjrArPEj+FM= X-Google-Smtp-Source: ABdhPJyyTjet0hB3i5D2u5kVje2efJwJ6tBaH7zXqSEo7vAbmIbW8YoDfZXAeBtfOOYzUqpvwNqNeQ== X-Received: by 2002:a63:d46:: with SMTP id 6mr9709661pgn.227.1604383349363; Mon, 02 Nov 2020 22:02:29 -0800 (PST) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id b16sm15647269pfp.195.2020.11.02.22.02.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 22:02:28 -0800 (PST) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v5 9/9] lib: zstd: Remove zstd compatibility wrapper Date: Mon, 2 Nov 2020 22:05:35 -0800 Message-Id: <20201103060535.8460-10-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201103060535.8460-1-nickrterrell@gmail.com> References: <20201103060535.8460-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Nick Terrell All callers have been transitioned to the new zstd-1.4.6 API. There are no more callers of the zstd compatibility wrapper, so delete it. Signed-off-by: Nick Terrell --- include/linux/zstd_compat.h | 116 ------------------------------------ 1 file changed, 116 deletions(-) delete mode 100644 include/linux/zstd_compat.h diff --git a/include/linux/zstd_compat.h b/include/linux/zstd_compat.h deleted file mode 100644 index cda9208bf04a..000000000000 --- a/include/linux/zstd_compat.h +++ /dev/null @@ -1,116 +0,0 @@ -/* - * Copyright (c) 2016-present, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under the BSD-style license found in the - * LICENSE file in the root directory of https://github.com/facebook/zstd. - * An additional grant of patent rights can be found in the PATENTS file in the - * same directory. - * - * This program is free software; you can redistribute it and/or modify it under - * the terms of the GNU General Public License version 2 as published by the - * Free Software Foundation. This program is dual-licensed; you may select - * either version 2 of the GNU General Public License ("GPL") or BSD license - * ("BSD"). - */ - -#ifndef ZSTD_COMPAT_H -#define ZSTD_COMPAT_H - -#include - -#if defined(ZSTD_VERSION_NUMBER) && (ZSTD_VERSION_NUMBER >= 10406) -/* - * This header provides backwards compatibility for the zstd-1.4.6 library - * upgrade. This header allows us to upgrade the zstd library version without - * modifying any callers. Then we will migrate callers from the compatibility - * wrapper one at a time until none remain. At which point we will delete this - * header. - * - * It is temporary and will be deleted once the upgrade is complete. - */ - -#include - -static inline size_t ZSTD_CCtxWorkspaceBound(ZSTD_compressionParameters compression_params) -{ - return ZSTD_estimateCCtxSize_usingCParams(compression_params); -} - -static inline size_t ZSTD_CStreamWorkspaceBound(ZSTD_compressionParameters compression_params) -{ - return ZSTD_estimateCStreamSize_usingCParams(compression_params); -} - -static inline size_t ZSTD_DCtxWorkspaceBound(void) -{ - return ZSTD_estimateDCtxSize(); -} - -static inline size_t ZSTD_DStreamWorkspaceBound(unsigned long long window_size) -{ - return ZSTD_estimateDStreamSize(window_size); -} - -static inline ZSTD_CCtx* ZSTD_initCCtx(void* wksp, size_t wksp_size) -{ - if (wksp == NULL) - return NULL; - return ZSTD_initStaticCCtx(wksp, wksp_size); -} - -static inline ZSTD_CStream* ZSTD_initCStream_compat(ZSTD_parameters params, uint64_t pledged_src_size, void* wksp, size_t wksp_size) -{ - ZSTD_CStream* cstream; - size_t ret; - - if (wksp == NULL) - return NULL; - - cstream = ZSTD_initStaticCStream(wksp, wksp_size); - if (cstream == NULL) - return NULL; - - /* 0 means unknown in old API but means 0 in new API */ - if (pledged_src_size == 0) - pledged_src_size = ZSTD_CONTENTSIZE_UNKNOWN; - - ret = ZSTD_initCStream_advanced(cstream, NULL, 0, params, pledged_src_size); - if (ZSTD_isError(ret)) - return NULL; - - return cstream; -} -#define ZSTD_initCStream ZSTD_initCStream_compat - -static inline ZSTD_DCtx* ZSTD_initDCtx(void* wksp, size_t wksp_size) -{ - if (wksp == NULL) - return NULL; - return ZSTD_initStaticDCtx(wksp, wksp_size); -} - -static inline ZSTD_DStream* ZSTD_initDStream_compat(unsigned long long window_size, void* wksp, size_t wksp_size) -{ - if (wksp == NULL) - return NULL; - (void)window_size; - return ZSTD_initStaticDStream(wksp, wksp_size); -} -#define ZSTD_initDStream ZSTD_initDStream_compat - -typedef ZSTD_frameHeader ZSTD_frameParams; - -static inline size_t ZSTD_getFrameParams(ZSTD_frameParams* frame_params, const void* src, size_t src_size) -{ - return ZSTD_getFrameHeader(frame_params, src, src_size); -} - -static inline size_t ZSTD_compressCCtx_compat(ZSTD_CCtx* cctx, void* dst, size_t dst_capacity, const void* src, size_t src_size, ZSTD_parameters params) -{ - return ZSTD_compress_advanced(cctx, dst, dst_capacity, src, src_size, NULL, 0, params); -} -#define ZSTD_compressCCtx ZSTD_compressCCtx_compat - -#endif /* ZSTD_VERSION_NUMBER >= 10406 */ -#endif /* ZSTD_COMPAT_H */