From patchwork Fri Nov 27 09:31:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B50DAC2D0E4 for ; Fri, 27 Nov 2020 09:33:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C950206DF for ; Fri, 27 Nov 2020 09:33:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C950206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60950 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kia8L-000454-6t for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:33:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6a-0002JH-FZ for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:31:54 -0500 Received: from szxga08-in.huawei.com ([45.249.212.255]:2238) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6T-0006eH-V8 for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:31:52 -0500 Received: from DGGEMM403-HUB.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4Cj8WQ6trMz13KvD; Fri, 27 Nov 2020 17:30:58 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM403-HUB.china.huawei.com (10.3.20.211) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:31:37 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:31:37 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 1/6] migration: Add multi-thread compress method Date: Fri, 27 Nov 2020 17:31:35 +0800 Message-ID: <20201127093135.2503-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.255; envelope-from=jinzeyu@huawei.com; helo=szxga08-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ying Fang , qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" A multi-thread compress method parameter is added to hold the method we are going to use. By default the 'zlib' method is used to maintain the compatibility as before. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang --- hw/core/qdev-properties-system.c | 11 +++++++++++ include/hw/qdev-properties.h | 4 ++++ migration/migration.c | 15 +++++++++++++++ monitor/hmp-cmds.c | 12 ++++++++++++ qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 67 insertions(+), 1 deletion(-) diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c index 9d80a07d26..a582721a7b 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -663,6 +663,17 @@ const PropertyInfo qdev_prop_multifd_compression = { .set_default_value = qdev_propinfo_set_default_value_enum, }; +/* --- CompressMethod --- */ +const PropertyInfo qdev_prop_compress_method = { + .name = "CompressMethod", + .description = "multi-thread compression method, " + "zlib", + .enum_table = &CompressMethod_lookup, + .get = qdev_propinfo_get_enum, + .set = qdev_propinfo_set_enum, + .set_default_value = qdev_propinfo_set_default_value_enum, +}; + /* --- Reserved Region --- */ /* diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h index 4437450065..4a943f7e80 100644 --- a/include/hw/qdev-properties.h +++ b/include/hw/qdev-properties.h @@ -23,6 +23,7 @@ extern const PropertyInfo qdev_prop_macaddr; extern const PropertyInfo qdev_prop_reserved_region; extern const PropertyInfo qdev_prop_on_off_auto; extern const PropertyInfo qdev_prop_multifd_compression; +extern const PropertyInfo qdev_prop_compress_method; extern const PropertyInfo qdev_prop_losttickpolicy; extern const PropertyInfo qdev_prop_blockdev_on_error; extern const PropertyInfo qdev_prop_bios_chs_trans; @@ -193,6 +194,9 @@ extern const PropertyInfo qdev_prop_pcie_link_width; #define DEFINE_PROP_MULTIFD_COMPRESSION(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_multifd_compression, \ MultiFDCompression) +#define DEFINE_PROP_COMPRESS_METHOD(_n, _s, _f, _d) \ + DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_compress_method, \ + CompressMethod) #define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_losttickpolicy, \ LostTickPolicy) diff --git a/migration/migration.c b/migration/migration.c index 87a9b59f83..bfbe48cc74 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -83,6 +83,7 @@ #define DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT 2 /*0: means nocompress, 1: best speed, ... 9: best compress ratio */ #define DEFAULT_MIGRATE_COMPRESS_LEVEL 1 +#define DEFAULT_MIGRATE_COMPRESS_METHOD COMPRESS_METHOD_ZLIB /* Define default autoconverge cpu throttle migration parameters */ #define DEFAULT_MIGRATE_THROTTLE_TRIGGER_THRESHOLD 50 #define DEFAULT_MIGRATE_CPU_THROTTLE_INITIAL 20 @@ -843,6 +844,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->compress_wait_thread = s->parameters.compress_wait_thread; params->has_decompress_threads = true; params->decompress_threads = s->parameters.decompress_threads; + params->has_compress_method = true; + params->compress_method = s->parameters.compress_method; params->has_throttle_trigger_threshold = true; params->throttle_trigger_threshold = s->parameters.throttle_trigger_threshold; params->has_cpu_throttle_initial = true; @@ -1407,6 +1410,10 @@ static void migrate_params_test_apply(MigrateSetParameters *params, dest->decompress_threads = params->decompress_threads; } + if (params->has_compress_method) { + dest->compress_method = params->compress_method; + } + if (params->has_throttle_trigger_threshold) { dest->throttle_trigger_threshold = params->throttle_trigger_threshold; } @@ -1504,6 +1511,10 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) s->parameters.decompress_threads = params->decompress_threads; } + if (params->has_compress_method) { + s->parameters.compress_method = params->compress_method; + } + if (params->has_throttle_trigger_threshold) { s->parameters.throttle_trigger_threshold = params->throttle_trigger_threshold; } @@ -3717,6 +3728,9 @@ static Property migration_properties[] = { DEFINE_PROP_UINT8("x-decompress-threads", MigrationState, parameters.decompress_threads, DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT), + DEFINE_PROP_COMPRESS_METHOD("compress-method", MigrationState, + parameters.compress_method, + DEFAULT_MIGRATE_COMPRESS_METHOD), DEFINE_PROP_UINT8("x-throttle-trigger-threshold", MigrationState, parameters.throttle_trigger_threshold, DEFAULT_MIGRATE_THROTTLE_TRIGGER_THRESHOLD), @@ -3831,6 +3845,7 @@ static void migration_instance_init(Object *obj) params->has_compress_level = true; params->has_compress_threads = true; params->has_decompress_threads = true; + params->has_compress_method = true; params->has_throttle_trigger_threshold = true; params->has_cpu_throttle_initial = true; params->has_cpu_throttle_increment = true; diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 65d8ff4849..a86574048c 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -420,6 +420,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) MigrationParameter_str(MIGRATION_PARAMETER_DECOMPRESS_THREADS), params->decompress_threads); assert(params->has_throttle_trigger_threshold); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_METHOD), + CompressMethod_str(params->compress_method)); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_THROTTLE_TRIGGER_THRESHOLD), params->throttle_trigger_threshold); @@ -1282,6 +1285,7 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) MigrateSetParameters *p = g_new0(MigrateSetParameters, 1); uint64_t valuebw = 0; uint64_t cache_size; + CompressMethod compress_method; Error *err = NULL; int val, ret; @@ -1307,6 +1311,14 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_decompress_threads = true; visit_type_int(v, param, &p->decompress_threads, &err); break; + case MIGRATION_PARAMETER_COMPRESS_METHOD: + p->has_compress_method = true; + visit_type_CompressMethod(v, param, &compress_method, &err); + if (err) { + break; + } + p->compress_method = compress_method; + break; case MIGRATION_PARAMETER_THROTTLE_TRIGGER_THRESHOLD: p->has_throttle_trigger_threshold = true; visit_type_int(v, param, &p->throttle_trigger_threshold, &err); diff --git a/qapi/migration.json b/qapi/migration.json index 3c75820527..2ed6a55b92 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -525,6 +525,19 @@ 'data': [ 'none', 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] } +## +# @CompressMethod: +# +# An enumeration of multi-thread compression methods. +# +# @zlib: use zlib compression method. +# +# Since: 6.0 +# +## +{ 'enum': 'CompressMethod', + 'data': [ 'zlib' ] } + ## # @BitmapMigrationBitmapAlias: # @@ -599,6 +612,9 @@ # compression, so set the decompress-threads to the number about 1/4 # of compress-threads is adequate. # +# @compress-method: Set compression method to use in multi-thread compression. +# Defaults to zlib. (Since 6.0) +# # @throttle-trigger-threshold: The ratio of bytes_dirty_period and bytes_xfer_period # to trigger throttling. It is expressed as percentage. # The default value is 50. (Since 5.0) @@ -722,7 +738,7 @@ 'data': ['announce-initial', 'announce-max', 'announce-rounds', 'announce-step', 'compress-level', 'compress-threads', 'decompress-threads', - 'compress-wait-thread', 'throttle-trigger-threshold', + 'compress-wait-thread', 'compress-method', 'throttle-trigger-threshold', 'cpu-throttle-initial', 'cpu-throttle-increment', 'cpu-throttle-tailslow', 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', @@ -759,6 +775,9 @@ # # @decompress-threads: decompression thread count # +# @compress-method: Which multi-thread compression method to use. +# Defaults to zlib. (Since 6.0) +# # @throttle-trigger-threshold: The ratio of bytes_dirty_period and bytes_xfer_period # to trigger throttling. It is expressed as percentage. # The default value is 50. (Since 5.0) @@ -889,6 +908,7 @@ '*compress-threads': 'int', '*compress-wait-thread': 'bool', '*decompress-threads': 'int', + '*compress-method': 'CompressMethod', '*throttle-trigger-threshold': 'int', '*cpu-throttle-initial': 'int', '*cpu-throttle-increment': 'int', @@ -953,6 +973,9 @@ # # @decompress-threads: decompression thread count # +# @compress-method: Which multi-thread compression method to use. +# Defaults to zlib. (Since 6.0) +# # @throttle-trigger-threshold: The ratio of bytes_dirty_period and bytes_xfer_period # to trigger throttling. It is expressed as percentage. # The default value is 50. (Since 5.0) @@ -1083,6 +1106,7 @@ '*compress-threads': 'uint8', '*compress-wait-thread': 'bool', '*decompress-threads': 'uint8', + '*compress-method': 'CompressMethod', '*throttle-trigger-threshold': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', From patchwork Fri Nov 27 09:31:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95E70C64E75 for ; Fri, 27 Nov 2020 09:33:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0AF83206DF for ; Fri, 27 Nov 2020 09:33:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0AF83206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60054 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kia83-0003hx-OI for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:33:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6i-0002NO-PM for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:00 -0500 Received: from szxga02-in.huawei.com ([45.249.212.188]:2120) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6d-0006i2-Kx for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:00 -0500 Received: from DGGEMM404-HUB.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Cj8X10x4pz4wx6; Fri, 27 Nov 2020 17:31:29 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM404-HUB.china.huawei.com (10.3.20.212) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:31:50 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:31:50 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 2/6] migration: Refactoring multi-thread compress migration Date: Fri, 27 Nov 2020 17:31:49 +0800 Message-ID: <20201127093149.2555-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.188; envelope-from=jinzeyu@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ying Fang , qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Code refactor for the compression procedure which includes: 1. Move qemu_compress_data and qemu_put_compression_data from qemu-file.c to ram.c, for the reason that most part of the code logical has nothing to do with qemu-file. Besides, the decompression code is located at ram.c only. 2. Simplify the function input arguments for compression and decompression. Wrap the input into the param structure which already exists. This change also makes the function much more flexible for other compression methods. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang --- migration/qemu-file.c | 62 ++++++------------------------- migration/qemu-file.h | 4 +- migration/ram.c | 86 ++++++++++++++++++++++++++++++------------- 3 files changed, 75 insertions(+), 77 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index be21518c57..1efb667aa1 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -737,56 +737,6 @@ uint64_t qemu_get_be64(QEMUFile *f) return v; } -/* return the size after compression, or negative value on error */ -static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len, - const uint8_t *source, size_t source_len) -{ - int err; - - err = deflateReset(stream); - if (err != Z_OK) { - return -1; - } - - stream->avail_in = source_len; - stream->next_in = (uint8_t *)source; - stream->avail_out = dest_len; - stream->next_out = dest; - - err = deflate(stream, Z_FINISH); - if (err != Z_STREAM_END) { - return -1; - } - - return stream->next_out - dest; -} - -/* Compress size bytes of data start at p and store the compressed - * data to the buffer of f. - * - * Since the file is dummy file with empty_ops, return -1 if f has no space to - * save the compressed data. - */ -ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, - const uint8_t *p, size_t size) -{ - ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t); - - if (blen < compressBound(size)) { - return -1; - } - - blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t), - blen, p, size); - if (blen < 0) { - return -1; - } - - qemu_put_be32(f, blen); - add_buf_to_iovec(f, blen); - return blen + sizeof(int32_t); -} - /* Put the data in the buffer of f_src to the buffer of f_des, and * then reset the buf_index of f_src to 0. */ @@ -846,3 +796,15 @@ void qemu_file_set_blocking(QEMUFile *f, bool block) f->ops->set_blocking(f->opaque, block, NULL); } } + +ssize_t qemu_put_compress_start(QEMUFile *f, uint8_t **dest_ptr) +{ + *dest_ptr = f->buf + f->buf_index + sizeof(int32_t); + return IO_BUF_SIZE - f->buf_index - sizeof(int32_t); +} + +void qemu_put_compress_end(QEMUFile *f, unsigned int v) +{ + qemu_put_be32(f, v); + add_buf_to_iovec(f, v); +} diff --git a/migration/qemu-file.h b/migration/qemu-file.h index a9b6d6ccb7..1ac1566460 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -138,8 +138,6 @@ bool qemu_file_is_writable(QEMUFile *f); size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t offset); size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size); -ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, - const uint8_t *p, size_t size); int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src); /* @@ -166,6 +164,8 @@ void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); +ssize_t qemu_put_compress_start(QEMUFile *f, uint8_t **dest_ptr); +void qemu_put_compress_end(QEMUFile *f, unsigned int v); /* Whenever this is found in the data stream, the flags * will be passed to ram_control_load_hook in the incoming-migration * side. This lets before_ram_iterate/after_ram_iterate add diff --git a/migration/ram.c b/migration/ram.c index 7811cde643..1818a56314 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -453,27 +453,22 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; -static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, - ram_addr_t offset, uint8_t *source_buf); +static bool do_compress_ram_page(CompressParam *param, RAMBlock *block); static void *do_data_compress(void *opaque) { CompressParam *param = opaque; RAMBlock *block; - ram_addr_t offset; bool zero_page; qemu_mutex_lock(¶m->mutex); while (!param->quit) { if (param->block) { block = param->block; - offset = param->offset; param->block = NULL; qemu_mutex_unlock(¶m->mutex); - zero_page = do_compress_ram_page(param->file, ¶m->stream, - block, offset, param->originbuf); - + zero_page = do_compress_ram_page(param, block); qemu_mutex_lock(&comp_done_lock); param->done = true; param->zero_page = zero_page; @@ -1214,28 +1209,73 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, return 1; } -static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, - ram_addr_t offset, uint8_t *source_buf) +/* + * Compress size bytes of data start at p and store the compressed + * data to the buffer of f. + * + * Since the file is dummy file with empty_ops, return -1 if f has no space to + * save the compressed data. + */ +static ssize_t qemu_put_compression_data(CompressParam *param, size_t size) +{ + int err; + uint8_t *dest = NULL; + z_stream *stream = ¶m->stream; + uint8_t *p = param->originbuf; + QEMUFile *f = f = param->file; + ssize_t blen = qemu_put_compress_start(f, &dest); + + if (blen < compressBound(size)) { + return -1; + } + + err = deflateReset(stream); + if (err != Z_OK) { + return -1; + } + + stream->avail_in = size; + stream->next_in = p; + stream->avail_out = blen; + stream->next_out = dest; + + err = deflate(stream, Z_FINISH); + if (err != Z_STREAM_END) { + return -1; + } + + blen = stream->next_out - dest; + if (blen < 0) { + return -1; + } + + qemu_put_compress_end(f, blen); + return blen + sizeof(int32_t); +} + +static bool do_compress_ram_page(CompressParam *param, RAMBlock *block) { RAMState *rs = ram_state; + ram_addr_t offset = param->offset; uint8_t *p = block->host + (offset & TARGET_PAGE_MASK); bool zero_page = false; int ret; - if (save_zero_page_to_file(rs, f, block, offset)) { + if (save_zero_page_to_file(rs, param->file, block, offset)) { zero_page = true; goto exit; } - save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(rs, param->file, block, + offset | RAM_SAVE_FLAG_COMPRESS_PAGE); /* * copy it to a internal buffer to avoid it being modified by VM * so that we can catch up the error during compression and * decompression */ - memcpy(source_buf, p, TARGET_PAGE_SIZE); - ret = qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_SIZE); + memcpy(param->originbuf, p, TARGET_PAGE_SIZE); + ret = qemu_put_compression_data(param, TARGET_PAGE_SIZE); if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); @@ -2826,19 +2866,20 @@ void ram_handle_compressed(void *host, uint8_t ch, uint64_t size) /* return the size after decompression, or negative value on error */ static int -qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, - const uint8_t *source, size_t source_len) +qemu_uncompress_data(DecompressParam *param, uint8_t *dest, size_t pagesize) { int err; + z_stream *stream = ¶m->stream; + err = inflateReset(stream); if (err != Z_OK) { return -1; } - stream->avail_in = source_len; - stream->next_in = (uint8_t *)source; - stream->avail_out = dest_len; + stream->avail_in = param->len; + stream->next_in = param->compbuf; + stream->avail_out = pagesize; stream->next_out = dest; err = inflate(stream, Z_NO_FLUSH); @@ -2852,22 +2893,17 @@ qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, static void *do_data_decompress(void *opaque) { DecompressParam *param = opaque; - unsigned long pagesize; uint8_t *des; - int len, ret; + int ret; qemu_mutex_lock(¶m->mutex); while (!param->quit) { if (param->des) { des = param->des; - len = param->len; param->des = 0; qemu_mutex_unlock(¶m->mutex); - pagesize = TARGET_PAGE_SIZE; - - ret = qemu_uncompress_data(¶m->stream, des, pagesize, - param->compbuf, len); + ret = qemu_uncompress_data(param, des, TARGET_PAGE_SIZE); if (ret < 0 && migrate_get_current()->decompress_error_check) { error_report("decompress data failed"); qemu_file_set_error(decomp_file, ret); From patchwork Fri Nov 27 09:32:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED698C2D0E4 for ; Fri, 27 Nov 2020 09:35:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 55609206DF for ; Fri, 27 Nov 2020 09:35:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 55609206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiaAT-0007J0-9v for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:35:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6s-0002fE-Ba for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:10 -0500 Received: from szxga02-in.huawei.com ([45.249.212.188]:2121) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6p-0006mb-Bz for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:10 -0500 Received: from DGGEMM401-HUB.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Cj8XG0cmfz4yrh; Fri, 27 Nov 2020 17:31:42 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM401-HUB.china.huawei.com (10.3.20.209) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:32:03 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:32:03 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 3/6] migration: Add multi-thread compress ops Date: Fri, 27 Nov 2020 17:32:02 +0800 Message-ID: <20201127093202.2607-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.188; envelope-from=jinzeyu@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add the MigrationCompressOps and MigrationDecompressOps structures to make the compression method configurable for multi-thread compression migration. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang parameters.decompress_threads; } +CompressMethod migrate_compress_method(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->parameters.compress_method; +} + bool migrate_dirty_bitmaps(void) { MigrationState *s; diff --git a/migration/migration.h b/migration/migration.h index d096b77f74..e22b2ef840 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -339,6 +339,7 @@ int migrate_compress_level(void); int migrate_compress_threads(void); int migrate_compress_wait_thread(void); int migrate_decompress_threads(void); +CompressMethod migrate_compress_method(void); bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); diff --git a/migration/ram.c b/migration/ram.c index 1818a56314..6f7fab7d4f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -419,8 +419,11 @@ struct CompressParam { ram_addr_t offset; /* internally used fields */ - z_stream stream; uint8_t *originbuf; + + /* for zlib compression */ + z_stream stream; + }; typedef struct CompressParam CompressParam; @@ -432,12 +435,29 @@ struct DecompressParam { void *des; uint8_t *compbuf; int len; + + /* for zlib compression */ z_stream stream; }; typedef struct DecompressParam DecompressParam; +typedef struct { + int (*save_setup)(CompressParam *param); + void (*save_cleanup)(CompressParam *param); + ssize_t (*compress_data)(CompressParam *param, size_t size); +} MigrationCompressOps; + +typedef struct { + int (*load_setup)(DecompressParam *param); + void (*load_cleanup)(DecompressParam *param); + int (*decompress_data)(DecompressParam *param, uint8_t *dest, size_t size); + int (*check_len)(int len); +} MigrationDecompressOps; + static CompressParam *comp_param; static QemuThread *compress_threads; +static MigrationCompressOps *compress_ops; +static MigrationDecompressOps *decompress_ops; /* comp_done_cond is used to wake up the migration thread when * one of the compression threads has finished the compression. * comp_done_lock is used to co-work with comp_done_cond. @@ -455,6 +475,157 @@ static QemuCond decomp_done_cond; static bool do_compress_ram_page(CompressParam *param, RAMBlock *block); +static int zlib_save_setup(CompressParam *param) +{ + if (deflateInit(¶m->stream, + migrate_compress_level()) != Z_OK) { + return -1; + } + + return 0; +} + +static ssize_t zlib_compress_data(CompressParam *param, size_t size) +{ + int err; + uint8_t *dest = NULL; + z_stream *stream = ¶m->stream; + uint8_t *p = param->originbuf; + QEMUFile *f = f = param->file; + ssize_t blen = qemu_put_compress_start(f, &dest); + + if (blen < compressBound(size)) { + return -1; + } + + err = deflateReset(stream); + if (err != Z_OK) { + return -1; + } + + stream->avail_in = size; + stream->next_in = p; + stream->avail_out = blen; + stream->next_out = dest; + + err = deflate(stream, Z_FINISH); + if (err != Z_STREAM_END) { + return -1; + } + + blen = stream->next_out - dest; + if (blen < 0) { + return -1; + } + + qemu_put_compress_end(f, blen); + return blen + sizeof(int32_t); +} + +static void zlib_save_cleanup(CompressParam *param) +{ + deflateEnd(¶m->stream); +} + +static int zlib_load_setup(DecompressParam *param) +{ + if (inflateInit(¶m->stream) != Z_OK) { + return -1; + } + + return 0; +} + +static int +zlib_decompress_data(DecompressParam *param, uint8_t *dest, size_t size) +{ + int err; + + z_stream *stream = ¶m->stream; + + err = inflateReset(stream); + if (err != Z_OK) { + return -1; + } + + stream->avail_in = param->len; + stream->next_in = param->compbuf; + stream->avail_out = size; + stream->next_out = dest; + + err = inflate(stream, Z_NO_FLUSH); + if (err != Z_STREAM_END) { + return -1; + } + + return stream->total_out; +} + +static void zlib_load_cleanup(DecompressParam *param) +{ + inflateEnd(¶m->stream); +} + +static int zlib_check_len(int len) +{ + return len < 0 || len > compressBound(TARGET_PAGE_SIZE); +} + +static int set_compress_ops(void) +{ + compress_ops = g_new0(MigrationCompressOps, 1); + + switch (migrate_compress_method()) { + case COMPRESS_METHOD_ZLIB: + compress_ops->save_setup = zlib_save_setup; + compress_ops->save_cleanup = zlib_save_cleanup; + compress_ops->compress_data = zlib_compress_data; + break; + default: + return -1; + } + + return 0; +} + +static int set_decompress_ops(void) +{ + decompress_ops = g_new0(MigrationDecompressOps, 1); + + switch (migrate_compress_method()) { + case COMPRESS_METHOD_ZLIB: + decompress_ops->load_setup = zlib_load_setup; + decompress_ops->load_cleanup = zlib_load_cleanup; + decompress_ops->decompress_data = zlib_decompress_data; + decompress_ops->check_len = zlib_check_len; + break; + default: + return -1; + } + + return 0; +} + +static void clean_compress_ops(void) +{ + compress_ops->save_setup = NULL; + compress_ops->save_cleanup = NULL; + compress_ops->compress_data = NULL; + + g_free(compress_ops); + compress_ops = NULL; +} + +static void clean_decompress_ops(void) +{ + decompress_ops->load_setup = NULL; + decompress_ops->load_cleanup = NULL; + decompress_ops->decompress_data = NULL; + + g_free(decompress_ops); + decompress_ops = NULL; +} + static void *do_data_compress(void *opaque) { CompressParam *param = opaque; @@ -511,7 +682,7 @@ static void compress_threads_save_cleanup(void) qemu_thread_join(compress_threads + i); qemu_mutex_destroy(&comp_param[i].mutex); qemu_cond_destroy(&comp_param[i].cond); - deflateEnd(&comp_param[i].stream); + compress_ops->save_cleanup(&comp_param[i]); g_free(comp_param[i].originbuf); qemu_fclose(comp_param[i].file); comp_param[i].file = NULL; @@ -522,6 +693,7 @@ static void compress_threads_save_cleanup(void) g_free(comp_param); compress_threads = NULL; comp_param = NULL; + clean_compress_ops(); } static int compress_threads_save_setup(void) @@ -531,6 +703,12 @@ static int compress_threads_save_setup(void) if (!migrate_use_compression()) { return 0; } + + if (set_compress_ops() < 0) { + clean_compress_ops(); + return -1; + } + thread_count = migrate_compress_threads(); compress_threads = g_new0(QemuThread, thread_count); comp_param = g_new0(CompressParam, thread_count); @@ -542,8 +720,7 @@ static int compress_threads_save_setup(void) goto exit; } - if (deflateInit(&comp_param[i].stream, - migrate_compress_level()) != Z_OK) { + if (compress_ops->save_setup(&comp_param[i]) < 0) { g_free(comp_param[i].originbuf); goto exit; } @@ -1209,50 +1386,6 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, return 1; } -/* - * Compress size bytes of data start at p and store the compressed - * data to the buffer of f. - * - * Since the file is dummy file with empty_ops, return -1 if f has no space to - * save the compressed data. - */ -static ssize_t qemu_put_compression_data(CompressParam *param, size_t size) -{ - int err; - uint8_t *dest = NULL; - z_stream *stream = ¶m->stream; - uint8_t *p = param->originbuf; - QEMUFile *f = f = param->file; - ssize_t blen = qemu_put_compress_start(f, &dest); - - if (blen < compressBound(size)) { - return -1; - } - - err = deflateReset(stream); - if (err != Z_OK) { - return -1; - } - - stream->avail_in = size; - stream->next_in = p; - stream->avail_out = blen; - stream->next_out = dest; - - err = deflate(stream, Z_FINISH); - if (err != Z_STREAM_END) { - return -1; - } - - blen = stream->next_out - dest; - if (blen < 0) { - return -1; - } - - qemu_put_compress_end(f, blen); - return blen + sizeof(int32_t); -} - static bool do_compress_ram_page(CompressParam *param, RAMBlock *block) { RAMState *rs = ram_state; @@ -1275,7 +1408,7 @@ static bool do_compress_ram_page(CompressParam *param, RAMBlock *block) * decompression */ memcpy(param->originbuf, p, TARGET_PAGE_SIZE); - ret = qemu_put_compression_data(param, TARGET_PAGE_SIZE); + ret = compress_ops->compress_data(param, TARGET_PAGE_SIZE); if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); @@ -2864,32 +2997,6 @@ void ram_handle_compressed(void *host, uint8_t ch, uint64_t size) } } -/* return the size after decompression, or negative value on error */ -static int -qemu_uncompress_data(DecompressParam *param, uint8_t *dest, size_t pagesize) -{ - int err; - - z_stream *stream = ¶m->stream; - - err = inflateReset(stream); - if (err != Z_OK) { - return -1; - } - - stream->avail_in = param->len; - stream->next_in = param->compbuf; - stream->avail_out = pagesize; - stream->next_out = dest; - - err = inflate(stream, Z_NO_FLUSH); - if (err != Z_STREAM_END) { - return -1; - } - - return stream->total_out; -} - static void *do_data_decompress(void *opaque) { DecompressParam *param = opaque; @@ -2903,7 +3010,7 @@ static void *do_data_decompress(void *opaque) param->des = 0; qemu_mutex_unlock(¶m->mutex); - ret = qemu_uncompress_data(param, des, TARGET_PAGE_SIZE); + ret = decompress_ops->decompress_data(param, des, TARGET_PAGE_SIZE); if (ret < 0 && migrate_get_current()->decompress_error_check) { error_report("decompress data failed"); qemu_file_set_error(decomp_file, ret); @@ -2973,7 +3080,7 @@ static void compress_threads_load_cleanup(void) qemu_thread_join(decompress_threads + i); qemu_mutex_destroy(&decomp_param[i].mutex); qemu_cond_destroy(&decomp_param[i].cond); - inflateEnd(&decomp_param[i].stream); + decompress_ops->load_cleanup(&decomp_param[i]); g_free(decomp_param[i].compbuf); decomp_param[i].compbuf = NULL; } @@ -2982,6 +3089,7 @@ static void compress_threads_load_cleanup(void) decompress_threads = NULL; decomp_param = NULL; decomp_file = NULL; + clean_decompress_ops(); } static int compress_threads_load_setup(QEMUFile *f) @@ -2992,6 +3100,11 @@ static int compress_threads_load_setup(QEMUFile *f) return 0; } + if (set_decompress_ops() < 0) { + clean_decompress_ops(); + return -1; + } + thread_count = migrate_decompress_threads(); decompress_threads = g_new0(QemuThread, thread_count); decomp_param = g_new0(DecompressParam, thread_count); @@ -2999,7 +3112,7 @@ static int compress_threads_load_setup(QEMUFile *f) qemu_cond_init(&decomp_done_cond); decomp_file = f; for (i = 0; i < thread_count; i++) { - if (inflateInit(&decomp_param[i].stream) != Z_OK) { + if (decompress_ops->load_setup(&decomp_param[i]) < 0) { goto exit; } @@ -3335,7 +3448,7 @@ static int ram_load_postcopy(QEMUFile *f) case RAM_SAVE_FLAG_COMPRESS_PAGE: all_zero = false; len = qemu_get_be32(f); - if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { + if (decompress_ops->check_len(len)) { error_report("Invalid compressed data length: %d", len); ret = -EINVAL; break; @@ -3602,7 +3715,7 @@ static int ram_load_precopy(QEMUFile *f) case RAM_SAVE_FLAG_COMPRESS_PAGE: len = qemu_get_be32(f); - if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { + if (decompress_ops->check_len(len)) { error_report("Invalid compressed data length: %d", len); ret = -EINVAL; break; From patchwork Fri Nov 27 09:32:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC77FC2D0E4 for ; Fri, 27 Nov 2020 09:36:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B417206DF for ; Fri, 27 Nov 2020 09:36:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B417206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiaAb-0007ZR-6y for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:36:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40704) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia77-0002ya-2T for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:26 -0500 Received: from szxga03-in.huawei.com ([45.249.212.189]:2182) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia73-0006rr-H2 for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:24 -0500 Received: from DGGEMM406-HUB.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4Cj8XP1RhKz4xBQ; Fri, 27 Nov 2020 17:31:49 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM406-HUB.china.huawei.com (10.3.20.214) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:32:17 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:32:17 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 4/6] migration: Add zstd support in multi-thread compression Date: Fri, 27 Nov 2020 17:32:15 +0800 Message-ID: <20201127093215.2659-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.189; envelope-from=jinzeyu@huawei.com; helo=szxga03-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ying Fang , qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This patch enables zstd option in multi-thread compression. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang --- hw/core/qdev-properties-system.c | 2 +- migration/ram.c | 128 ++++++++++++++++++++++++++++++- qapi/migration.json | 2 +- 3 files changed, 129 insertions(+), 3 deletions(-) diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c index a582721a7b..b369187bdc 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -667,7 +667,7 @@ const PropertyInfo qdev_prop_multifd_compression = { const PropertyInfo qdev_prop_compress_method = { .name = "CompressMethod", .description = "multi-thread compression method, " - "zlib", + "zlib/zstd", .enum_table = &CompressMethod_lookup, .get = qdev_propinfo_get_enum, .set = qdev_propinfo_set_enum, diff --git a/migration/ram.c b/migration/ram.c index 6f7fab7d4f..2550225d9a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -57,6 +57,10 @@ #include "qemu/iov.h" #include "multifd.h" +#ifdef CONFIG_ZSTD +#include +#include +#endif /***********************************************************/ /* ram save/restore */ @@ -424,6 +428,11 @@ struct CompressParam { /* for zlib compression */ z_stream stream; +#ifdef CONFIG_ZSTD + ZSTD_CStream *zstd_cs; + ZSTD_inBuffer in; + ZSTD_outBuffer out; +#endif }; typedef struct CompressParam CompressParam; @@ -438,6 +447,12 @@ struct DecompressParam { /* for zlib compression */ z_stream stream; + +#ifdef CONFIG_ZSTD + ZSTD_DStream *zstd_ds; + ZSTD_inBuffer in; + ZSTD_outBuffer out; +#endif }; typedef struct DecompressParam DecompressParam; @@ -571,6 +586,102 @@ static int zlib_check_len(int len) return len < 0 || len > compressBound(TARGET_PAGE_SIZE); } +#ifdef CONFIG_ZSTD +static int zstd_save_setup(CompressParam *param) +{ + int res; + param->zstd_cs = ZSTD_createCStream(); + if (!param->zstd_cs) { + return -1; + } + res = ZSTD_initCStream(param->zstd_cs, migrate_compress_level()); + if (ZSTD_isError(res)) { + return -1; + } + return 0; +} +static void zstd_save_cleanup(CompressParam *param) +{ + ZSTD_freeCStream(param->zstd_cs); + param->zstd_cs = NULL; +} +static ssize_t zstd_compress_data(CompressParam *param, size_t size) +{ + int ret; + uint8_t *dest = NULL; + uint8_t *p = param->originbuf; + QEMUFile *f = f = param->file; + ssize_t blen = qemu_put_compress_start(f, &dest); + if (blen < ZSTD_compressBound(size)) { + return -1; + } + param->out.dst = dest; + param->out.size = blen; + param->out.pos = 0; + param->in.src = p; + param->in.size = size; + param->in.pos = 0; + do { + ret = ZSTD_compressStream2(param->zstd_cs, ¶m->out, + ¶m->in, ZSTD_e_end); + } while (ret > 0 && (param->in.size - param->in.pos > 0) + && (param->out.size - param->out.pos > 0)); + if (ret > 0 && (param->in.size - param->in.pos > 0)) { + return -1; + } + if (ZSTD_isError(ret)) { + return -1; + } + blen = param->out.pos; + qemu_put_compress_end(f, blen); + return blen + sizeof(int32_t); +} +static int zstd_load_setup(DecompressParam *param) +{ + int ret; + param->zstd_ds = ZSTD_createDStream(); + if (!param->zstd_ds) { + return -1; + } + ret = ZSTD_initDStream(param->zstd_ds); + if (ZSTD_isError(ret)) { + return -1; + } + return 0; +} +static void zstd_load_cleanup(DecompressParam *param) +{ + ZSTD_freeDStream(param->zstd_ds); + param->zstd_ds = NULL; +} +static int +zstd_decompress_data(DecompressParam *param, uint8_t *dest, size_t size) +{ + int ret; + param->out.dst = dest; + param->out.size = size; + param->out.pos = 0; + param->in.src = param->compbuf; + param->in.size = param->len; + param->in.pos = 0; + do { + ret = ZSTD_decompressStream(param->zstd_ds, ¶m->out, ¶m->in); + } while (ret > 0 && (param->in.size - param->in.pos > 0) + && (param->out.size - param->out.pos > 0)); + if (ret > 0 && (param->in.size - param->in.pos > 0)) { + return -1; + } + if (ZSTD_isError(ret)) { + return -1; + } + return ret; +} +static int zstd_check_len(int len) +{ + return len < 0 || len > ZSTD_compressBound(TARGET_PAGE_SIZE); +} +#endif + static int set_compress_ops(void) { compress_ops = g_new0(MigrationCompressOps, 1); @@ -581,9 +692,16 @@ static int set_compress_ops(void) compress_ops->save_cleanup = zlib_save_cleanup; compress_ops->compress_data = zlib_compress_data; break; +#ifdef CONFIG_ZSTD + case COMPRESS_METHOD_ZSTD: + compress_ops->save_setup = zstd_save_setup; + compress_ops->save_cleanup = zstd_save_cleanup; + compress_ops->compress_data = zstd_compress_data; + break; +#endif default: return -1; - } + } return 0; } @@ -599,6 +717,14 @@ static int set_decompress_ops(void) decompress_ops->decompress_data = zlib_decompress_data; decompress_ops->check_len = zlib_check_len; break; +#ifdef CONFIG_ZSTD + case COMPRESS_METHOD_ZSTD: + decompress_ops->load_setup = zstd_load_setup; + decompress_ops->load_cleanup = zstd_load_cleanup; + decompress_ops->decompress_data = zstd_decompress_data; + decompress_ops->check_len = zstd_check_len; + break; +#endif default: return -1; } diff --git a/qapi/migration.json b/qapi/migration.json index 2ed6a55b92..883e597ae3 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -536,7 +536,7 @@ # ## { 'enum': 'CompressMethod', - 'data': [ 'zlib' ] } + 'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] } ## # @BitmapMigrationBitmapAlias: From patchwork Fri Nov 27 09:32:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7E9EC2D0E4 for ; Fri, 27 Nov 2020 09:36:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F14B221534 for ; Fri, 27 Nov 2020 09:36:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F14B221534 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42314 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiaAp-00083r-U5 for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:36:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40774) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia7K-00038J-VN for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:38 -0500 Received: from szxga02-in.huawei.com ([45.249.212.188]:2058) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia7H-0006wX-Tz for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:38 -0500 Received: from DGGEMM405-HUB.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Cj8Xs5mqSzQlNn; Fri, 27 Nov 2020 17:32:13 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM405-HUB.china.huawei.com (10.3.20.213) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:32:32 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:32:32 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 5/6] migration: Add compress_level sanity check Date: Fri, 27 Nov 2020 17:32:30 +0800 Message-ID: <20201127093230.2711-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.188; envelope-from=jinzeyu@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ying Fang , qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Zlib compression has level from 1 to 9. However Zstd compression has level from 1 to 22 (level >= 20 not recommanded). Let's do sanity check here to make sure a vaild compress_level is given by user. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang --- migration/migration.c | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index c11f137a8d..fab26084c2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1235,16 +1235,40 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, } } +static bool compress_level_check(MigrationParameters *params, Error **errp) +{ + switch (params->compress_method) { + case COMPRESS_METHOD_ZLIB: + if (params->compress_level > 9 || params->compress_level < 1) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "compress_level", + "a value in the range of 0 to 9 for Zlib method"); + return false; + } + break; +#ifdef CONFIG_ZSTD + case COMPRESS_METHOD_ZSTD: + if (params->compress_level > 19 || params->compress_level < 1) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "compress_level", + "a value in the range of 1 to 19 for Zstd method"); + return false; + } + break; +#endif + default: + error_setg(errp, "Checking compress_level failed for unknown reason"); + return false; + } + + return true; +} + /* * Check whether the parameters are valid. Error will be put into errp * (if provided). Return true if valid, otherwise false. */ static bool migrate_params_check(MigrationParameters *params, Error **errp) { - if (params->has_compress_level && - (params->compress_level > 9)) { - error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "compress_level", - "is invalid, it should be in the range of 0 to 9"); + if (params->has_compress_level && !compress_level_check(params, errp)) { return false; } From patchwork Fri Nov 27 09:32:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91DEFC63777 for ; Fri, 27 Nov 2020 09:34:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 09C23206DF for ; Fri, 27 Nov 2020 09:34:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09C23206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kia9Q-0005Nv-04 for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:34:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40828) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia7a-0003eu-Ew for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:54 -0500 Received: from szxga08-in.huawei.com ([45.249.212.255]:2239) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia7Y-00070p-7i for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:32:54 -0500 Received: from DGGEMM402-HUB.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4Cj8Xp0tt1z13MqM; Fri, 27 Nov 2020 17:32:10 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM402-HUB.china.huawei.com (10.3.20.210) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:32:48 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:32:48 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 6/6] doc: Update multi-thread compression doc Date: Fri, 27 Nov 2020 17:32:46 +0800 Message-ID: <20201127093246.2763-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.255; envelope-from=jinzeyu@huawei.com; helo=szxga08-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_PORT=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ying Fang , qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Modify the doc to fit the previous changes. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang --- docs/multi-thread-compression.txt | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/docs/multi-thread-compression.txt b/docs/multi-thread-compression.txt index bb88c6bdf1..d429963cb0 100644 --- a/docs/multi-thread-compression.txt +++ b/docs/multi-thread-compression.txt @@ -33,14 +33,15 @@ thread compression can be used to accelerate the compression process. The decompression speed of Zlib is at least 4 times as quick as compression, if the source and destination CPU have equal speed, -keeping the compression thread count 4 times the decompression -thread count can avoid resource waste. +and you choose Zlib as compression method, keeping the compression +thread count 4 times the decompression thread count can avoid resource waste. Compression level can be used to control the compression speed and the -compression ratio. High compression ratio will take more time, level 0 -stands for no compression, level 1 stands for the best compression -speed, and level 9 stands for the best compression ratio. Users can -select a level number between 0 and 9. +compression ratio. High compression ratio will take more time, +level 1 stands for the best compression speed, and higher level means higher +compression ration. For Zlib, users can select a level number between 0 and 9, +where level 0 stands for no compression. For Zstd, users can select a +level number between 1 and 22. When to use the multiple thread compression in live migration @@ -116,16 +117,19 @@ to support the multiple thread compression migration: 2. Activate compression on the source: {qemu} migrate_set_capability compress on -3. Set the compression thread count on source: +3. Set the compression method: + {qemu} migrate_set_parameter compress_method zstd + +4. Set the compression thread count on source: {qemu} migrate_set_parameter compress_threads 12 -4. Set the compression level on the source: +5. Set the compression level on the source: {qemu} migrate_set_parameter compress_level 1 -5. Set the decompression thread count on destination: +6. Set the decompression thread count on destination: {qemu} migrate_set_parameter decompress_threads 3 -6. Start outgoing migration: +7. Start outgoing migration: {qemu} migrate -d tcp:destination.host:4444 {qemu} info migrate Capabilities: ... compress: on @@ -136,6 +140,7 @@ The following are the default settings: compress_threads: 8 decompress_threads: 2 compress_level: 1 (which means best speed) + compress_method: zlib So, only the first two steps are required to use the multiple thread compression in migration. You can do more if the default @@ -143,7 +148,7 @@ settings are not appropriate. TODO ==== -Some faster (de)compression method such as LZ4 and Quicklz can help -to reduce the CPU consumption when doing (de)compression. If using -these faster (de)compression method, less (de)compression threads +Comparing to Zlib, Some faster (de)compression method such as LZ4 +and Quicklz can help to reduce the CPU consumption when doing (de)compression. +If using these faster (de)compression method, less (de)compression threads are needed when doing the migration.