From patchwork Fri Nov 27 09:31:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeyu Jin X-Patchwork-Id: 11935229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFC97C2D0E4 for ; Fri, 27 Nov 2020 09:33:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1ADB9206DF for ; Fri, 27 Nov 2020 09:33:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1ADB9206DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59600 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kia7j-0003WZ-9V for qemu-devel@archiver.kernel.org; Fri, 27 Nov 2020 04:33:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6M-0002DP-Vc for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:31:38 -0500 Received: from szxga02-in.huawei.com ([45.249.212.188]:2057) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kia6J-0006YE-Cq for qemu-devel@nongnu.org; Fri, 27 Nov 2020 04:31:38 -0500 Received: from DGGEMM402-HUB.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Cj8WZ25wNzQlGm; Fri, 27 Nov 2020 17:31:06 +0800 (CST) Received: from dggemi758-chm.china.huawei.com (10.1.198.144) by DGGEMM402-HUB.china.huawei.com (10.3.20.210) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 27 Nov 2020 17:31:23 +0800 Received: from localhost (10.174.187.211) by dggemi758-chm.china.huawei.com (10.1.198.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 27 Nov 2020 17:31:23 +0800 From: Zeyu Jin To: , Subject: [PATCH v2 0/6] Date: Fri, 27 Nov 2020 17:31:22 +0800 Message-ID: <20201127093122.2451-1-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.211] X-ClientProxiedBy: dggemi702-chm.china.huawei.com (10.3.20.101) To dggemi758-chm.china.huawei.com (10.1.198.144) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.188; envelope-from=jinzeyu@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Currently we have both multi-thread compression and multifd to optimize live migration in Qemu. Mulit-thread compression deals with the situation where network bandwith is limited but cpu resource adequate. Multifd instead aims to take full advantage of network bandwith. Moreover it supports both zlib and zstd compression on each channel. In this patch series, we did some code refactoring on multi-thread compression live migration and bring zstd compression method support for it. Below is the test result of multi-thread compression live migration with different compress methods. Test result shows that zstd outperforms zlib by about 70%. Migration Configuration: Guest 8U 32G compress-threads 8 decompress-threads 2 compress-level 1 bandwidth-limit 100Mbps Test Result: +---------------------+--------------+-------------+ | compress method | zlib | zstd | +---------------------+--------------+-------------+ | total time (ms) | 75256 | 44187 | +---------------------+--------------+-------------+ | downtime(ms) | 128 | 81 | +---------------------+--------------+-------------+ | transferred ram(kB)| 1576866 | 736117 | +---------------------+--------------+-------------+ | throughput(mbps) | 172.06 | 137.16 | +---------------------+--------------+-------------+ | total ram(kB) | 33685952 | 33685952 | +---------------------+--------------+-------------+ Zeyu Jin (6): migration: Add multi-thread compress method migration: Refactoring multi-thread compress migration migration: Add multi-thread compress ops migration: Add zstd support in multi-thread compression migration: Add compress_level sanity check doc: Update multi-thread compression doc docs/multi-thread-compression.txt | 31 ++- hw/core/qdev-properties-system.c | 11 + include/hw/qdev-properties.h | 4 + migration/migration.c | 56 ++++- migration/migration.h | 1 + migration/qemu-file.c | 62 +---- migration/qemu-file.h | 4 +- migration/ram.c | 381 +++++++++++++++++++++++++----- monitor/hmp-cmds.c | 12 + qapi/migration.json | 26 +- 10 files changed, 465 insertions(+), 123 deletions(-)