From patchwork Sun Apr 2 17:47:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13197502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A6B6C7619A for ; Sun, 2 Apr 2023 17:49:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pj1on-0003GF-Tq; Sun, 02 Apr 2023 13:48:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1om-0003G3-LP for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:48:40 -0400 Received: from mout.web.de ([217.72.192.78]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1ok-0002gp-RK for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:48:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1680457713; i=lukasstraub2@web.de; bh=zF1d+LYn7W0RpO/gzEs/u71icHjBIdWD5oHxcsVifYw=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject; b=j6+z4+THWxPiUXsWi289ilpsUxY/L7RrqosyMKq4Ilz8QDDQIjBGAV2cgaWPLuYgY Lhysck4HGyywLAxJS/2HzinG1B+tyWfqDixDKVADdUguACr/26FnLU4ElX9aeKLkia ssxMm2C1m2/ODqfV6I6/aPZDYNE8scYTFJSWEsWLveOyiOThwQWcDeYPtr8XTKhJ4S UIAZn5yLFdRK2TJ7277SmmWh0qvSog9ZnNY14E1SkYzEV6jRUUl0Dhi0zFO3JB/ZNR zLiuz36G5bOXROShlY6LU6KlHy6to/Bsw3ye1P3fpifam3m7lPlX4PVzaQB0FbwT0E mVFw6St207ysg== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.111]) by smtp.web.de (mrweb105 [213.165.67.124]) with ESMTPSA (Nemesis) id 1M2xrq-1phupB2flM-003kk2; Sun, 02 Apr 2023 19:48:33 +0200 Date: Sun, 2 Apr 2023 17:47:45 +0000 From: Lukas Straub To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Peter Xu , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH RESEND 1/2] qtest/migration-test.c: Add test with compress enabled Message-ID: MIME-Version: 1.0 X-Provags-ID: V03:K1:PxQwMhmIwR6K92ky9qrIa2hhqcRm2IfKE94mpR2gvnYaJuB8TvH P9MXqn5kwB8aJHCok9nEK7RrcuLEEavcDxYKmJ4Q7JuK+mQtdkVpaeTi4zQjIUIriB6Jab8 GrcphghDW8HkWH+7yZ55Ni84Y8ViT+QDPeZmwsUy+hDcx4okUyLwE68OytBBXutJb1yu5ki DupUAN0qfiEqd5fk964nQ== UI-OutboundReport: notjunk:1;M01:P0:1Iol1Uuf/EY=;dWtE46/FyDes9Or7S66kUzO1/3q FRy1WEs93MVu8igwtSaGi9r1WzEC8lgHzXWMRRg9UDE/mJFQlIz83j2zgldq6DZkCXpLcci/C 5kGK90d4RMJ0qyLQA8og4A05Mk5A+iX8jowqAif/6/lKowi5CxOPPOgv7/TEJ4+Xttdr4jBTf nNS6RXqcx+Ecw5llNpgRxQhvdKHOWQUCK+wnZ8I0xZjNtJ6/z3o3LNpEhqmCFzlyoczYkeefs Vtan1dm81Fs7muQ6eKvRV4Ja2EifL13FlTZOW7ruR3zBS5lANoFVkZrMVjG1Yu41YGJ9RvDw6 Md6GilGki62/x0R7FE5G7r3Das3NWXPfQx/G8L0/PCiNdUr5mLGWjiY/C0YOmDNiqilBHeK8m EzhJd0q+ibAb0K+H7uXmjMap66fV/ofmy9bjaSmWsYvuTghwI/ipaDqYCJNYzegv2lrXmKnl/ +RZCG31VyApbI8+shqOr8Mw4w8YmNdejSvMyT4QoneiZ5LJsuJrxjwH1+to/af+P9VgUuQ/a9 efmEtH2Ibg4GIYUp4CBEczOXfBdSeL5m/CEmR0i6+hE3OqUQVMDGC6R5HvPSSYxEizv/9+C6R dyIwAW8dk4v6b5elkqf1CMh+7TpNQYSdsEGbSPG/9pm0y+H9byfWBhpzawL4tECSwh79n6qkf G5E4SW4akkf+GFztYoMlM6aPZBgNMMCJoMVQ1paE/OjwTDNLU+A8TYLJo6nO9iz8cFVyHTX/1 eHmA86VsYGeXvw+7sze79W24lyitGEYFd0YMkW7wmAhJS5oJgkifGRoQBBRIYhtx2fIoc69yr EBPOifXn2guKWyjQGTPnuKmgHpNZR4wQ5yMMSPkXdGY+Cl8bB173k0djLrXZSIZKnJlw7z/Gf /P5EuCIB7/za6azNRxPmdrez6omQQiWVO/vtrCSLw1PfuhCsI6RjT8ZUOMRdvfR36y//WkQST KkFLaKRo/y3Vf9T7WJLVOciJQho= Received-SPF: pass client-ip=217.72.192.78; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There has never been a test for migration with compress enabled. Add a suitable test, testing with compress-wait-thread = false too. iterations = 2 is intentional, so it also tests that no invalid thread state is left over from the previous iteration. Signed-off-by: Lukas Straub Reviewed-by: Peter Xu --- tests/qtest/migration-test.c | 67 ++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 3b615b0da9..dbcab2e8ae 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -406,6 +406,41 @@ static void migrate_set_parameter_str(QTestState *who, const char *parameter, migrate_check_parameter_str(who, parameter, value); } +static long long migrate_get_parameter_bool(QTestState *who, + const char *parameter) +{ + QDict *rsp; + int result; + + rsp = wait_command(who, "{ 'execute': 'query-migrate-parameters' }"); + result = qdict_get_bool(rsp, parameter); + qobject_unref(rsp); + return !!result; +} + +static void migrate_check_parameter_bool(QTestState *who, const char *parameter, + int value) +{ + int result; + + result = migrate_get_parameter_bool(who, parameter); + g_assert_cmpint(result, ==, value); +} + +static void migrate_set_parameter_bool(QTestState *who, const char *parameter, + int value) +{ + QDict *rsp; + + rsp = qtest_qmp(who, + "{ 'execute': 'migrate-set-parameters'," + "'arguments': { %s: %i } }", + parameter, value); + g_assert(qdict_haskey(rsp, "return")); + qobject_unref(rsp); + migrate_check_parameter_bool(who, parameter, value); +} + static void migrate_ensure_non_converge(QTestState *who) { /* Can't converge with 1ms downtime + 3 mbs bandwidth limit */ @@ -1524,6 +1559,36 @@ static void test_precopy_unix_xbzrle(void) test_precopy_common(&args); } +static void * +test_migrate_compress_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "compress-level", 9); + migrate_set_parameter_int(from, "compress-threads", 1); + migrate_set_parameter_bool(from, "compress-wait-thread", false); + migrate_set_parameter_int(to, "decompress-threads", 1); + + migrate_set_capability(from, "compress", true); + migrate_set_capability(to, "compress", true); + + return NULL; +} + +static void test_precopy_unix_compress(void) +{ + g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = uri, + + .start_hook = test_migrate_compress_start, + + .iterations = 2, + }; + + test_precopy_common(&args); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -2515,6 +2580,8 @@ int main(int argc, char **argv) qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plain); qtest_add_func("/migration/precopy/unix/xbzrle", test_precopy_unix_xbzrle); + qtest_add_func("/migration/precopy/unix/compress", + test_precopy_unix_compress); #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); From patchwork Sun Apr 2 17:48:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13197503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37F4AC76196 for ; Sun, 2 Apr 2023 17:49:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pj1ov-0003In-HW; Sun, 02 Apr 2023 13:48:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1os-0003Gp-QO for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:48:47 -0400 Received: from mout.web.de ([212.227.17.11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pj1oq-0002h6-B1 for qemu-devel@nongnu.org; Sun, 02 Apr 2023 13:48:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1680457719; i=lukasstraub2@web.de; bh=g8CpH09MCnsv7m0q/+YFIHQE1GLIeIMiUJ3+xuDtBac=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=RBNjM5tzudU/v5v5fqsmig5/UhOM+nq/awDLUUB1JXRDysAmNFzNbxpA21bWh8fml Ap/JrJqR/iNe1Lf4eZyLBUnGQdYbThA1yl7NUiFPaRkqK7e+a+cueo88WGVu5ePxnm ZXrbdlHTOT8o+rKrhYCYfRBSdyewpTS/PKj7/VIRh0FoauZPQ95T8SHILd81JRZxhl YS5smBXuRzx4A8zG1BiN6Yzqqy5Zhzu0JACgA0EF6fTaxx62uCRtkpSGoFSlVtjn7u nlHVYddSdBPbkMEPFyILdUcP84iptKsk6u8ciX89yxcbm/9hn2Kp3LIi11i2bYFFCL 86AZC6zw99Slw== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.111]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MBjMM-1pZ57T2eho-00COLC; Sun, 02 Apr 2023 19:48:39 +0200 Date: Sun, 2 Apr 2023 17:48:38 +0000 From: Lukas Straub To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Peter Xu , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH RESEND 2/2] migration/ram.c: Fix migration with compress enabled Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:hBDurFHeYOZwIjj8l7DLAiHbQuT5o0U42fO11C6pRIgLehIZyw0 5/PbZ18jKaVMiPGoTw1qgQ0l1xlilWp1jIpmMMB9MHVIck94ZosZOw8tO+EGUYxUOUXd8BL mWSmAdZR6msf9IWf91mnD1wx5YM5Vlnhp9LZ+uq3WhtEZ5J0zUuBxa+tX3WUhMlN3l9MPe6 7JuQ2keW25s2kuyzWXQpg== UI-OutboundReport: notjunk:1;M01:P0:4d/CN6g3mPs=;TRb0limn6WQEXujp+ZaevjHaPjd IpDpZl3pT1XQRdkG/TqZMswUjuc9fnO2bgBDmZwesTcUC0rVC5aSlQIemfSvS+6eVzXWISQKR 4ehccBAUaTNeeS7rnnig0/kR2QV3IyYpaqJy+y9peaacP4upAsXyqPaAqYXHi/QGM3ybEtr5f Xoe4y09xq5vhrtQF7pZll3LGliHtxeMpod2ybPmLbf7PFQPVnE35sAd4VTS0CxR1BzgAja3TC DjhIFj65XcdKiTCQFDpHW64yhL4KfQLSIqZjrumOOxjDptU0ehBfSc1AbOZ07DLcmlhAHnttY TMG+M92r3wMU4SRm9auqtn5UOvq7xpC2VftQ7ZReHSyZivaJUSNu79zzA/Z1hQLVXVr80z3Xk hDRuHKQEkU3+6qbanHYIApjrhyH3IZzdrF0wqVB2qNc3DyQtkuj6JbSQbPjJG3gSmXxidTlJx Fgl7nr2NAPpRvebjRU8KrdERqdzQaieJ0qTWNQcKI6wS1uVhOOgGBncr+QTIKz7tK647wZ9r8 j9iNkag22K5qrqKM0ZH73l4rghT8Xe51lyUxXFrF47IOjSkgSMoU/ICIvF20Jc8iNfArZAXpd /CLSO555Gmk8Fho9+YrM1b0HmATsEVE7u/2MfFtSoxB8XB47/VjDDrWHmvfR+LpH+TkemdyaY h1BEf7M8Fzt/AEAep3PWW0OJGjUJIYakFgAUI8FgJc+MCcioV9EjvLrGf1/KXaScc3qiCa1YQ bqh5f+oYZxRDXT6C63x6jT6piECxlqrJ0M7T4f1YM8rcPTTLtC34+XbNRmWeNxg+Hm0lcmCiE xV6LXXmEo00iLN4vS32ss6l7E1k2EY2MX9qRgKCHNtPN+9sHHGyF4e69fA5X6A2aM6baLnNoY L1y90qxxI3QdsKtlPhdIbvDFO/ptL2TT3TUaINHesG/rT2MZZqlzH2QxSijnivneXEQgnXVCS yRZTFemlVir0zFch45TADVUwcJc= Received-SPF: pass client-ip=212.227.17.11; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Since ec6f3ab9, migration with compress enabled was broken, because the compress threads use a dummy QEMUFile which just acts as a buffer and that commit accidentally changed it to use the outgoing migration channel instead. Fix this by using the dummy file again in the compress threads. Signed-off-by: Lukas Straub Reviewed-by: Peter Xu --- migration/ram.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 96e8a19a58..9d1817ab7b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -688,12 +688,11 @@ exit: * @offset: offset inside the block for the page * in the lower bits, it contains flags */ -static size_t save_page_header(PageSearchStatus *pss, RAMBlock *block, - ram_addr_t offset) +static size_t save_page_header(PageSearchStatus *pss, QEMUFile *f, + RAMBlock *block, ram_addr_t offset) { size_t size, len; bool same_block = (block == pss->last_sent_block); - QEMUFile *f = pss->pss_channel; if (same_block) { offset |= RAM_SAVE_FLAG_CONTINUE; @@ -867,7 +866,7 @@ static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss, } /* Send XBZRLE based compressed page */ - bytes_xbzrle = save_page_header(pss, block, + bytes_xbzrle = save_page_header(pss, pss->pss_channel, block, offset | RAM_SAVE_FLAG_XBZRLE); qemu_put_byte(file, ENCODING_FLAG_XBZRLE); qemu_put_be16(file, encoded_len); @@ -1302,15 +1301,14 @@ void ram_release_page(const char *rbname, uint64_t offset) * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page_to_file(PageSearchStatus *pss, +static int save_zero_page_to_file(PageSearchStatus *pss, QEMUFile *file, RAMBlock *block, ram_addr_t offset) { uint8_t *p = block->host + offset; - QEMUFile *file = pss->pss_channel; int len = 0; if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { - len += save_page_header(pss, block, offset | RAM_SAVE_FLAG_ZERO); + len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO); qemu_put_byte(file, 0); len += 1; ram_release_page(block->idstr, offset); @@ -1327,10 +1325,10 @@ static int save_zero_page_to_file(PageSearchStatus *pss, * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(PageSearchStatus *pss, RAMBlock *block, +static int save_zero_page(PageSearchStatus *pss, QEMUFile *f, RAMBlock *block, ram_addr_t offset) { - int len = save_zero_page_to_file(pss, block, offset); + int len = save_zero_page_to_file(pss, f, block, offset); if (len) { stat64_add(&ram_atomic_counters.duplicate, 1); @@ -1394,7 +1392,7 @@ static int save_normal_page(PageSearchStatus *pss, RAMBlock *block, { QEMUFile *file = pss->pss_channel; - ram_transferred_add(save_page_header(pss, block, + ram_transferred_add(save_page_header(pss, pss->pss_channel, block, offset | RAM_SAVE_FLAG_PAGE)); if (async) { qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, @@ -1473,11 +1471,11 @@ static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, uint8_t *p = block->host + offset; int ret; - if (save_zero_page_to_file(pss, block, offset)) { + if (save_zero_page_to_file(pss, f, block, offset)) { return true; } - save_page_header(pss, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(pss, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); /* * copy it to a internal buffer to avoid it being modified by VM @@ -2355,7 +2353,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) return 1; } - res = save_zero_page(pss, block, offset); + res = save_zero_page(pss, pss->pss_channel, block, offset); if (res > 0) { /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale