From patchwork Fri Feb 28 12:17:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13996331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59F3FC19776 for ; Fri, 28 Feb 2025 12:18:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tnzJp-0004Bv-IO; Fri, 28 Feb 2025 07:18:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJk-0004BO-O9 for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJi-0001Fz-Tn for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740745088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oK/AOAOSw07oCFtCGtBGm4epHpYFuS5WO67rNDkiJh4=; b=Ct4XgzaQZflWQUkOvgK82uX1ChZLM5YN9eBoQGWqmZyiStVjKYypYpvxbZ35zqmZLn04Xt ZhGXvzE5T6nfY98JH8cmOR58DcgMY+xkip1198L6G3I2KQHuD6CvDozcM2VW2OXdPUigQK K2vGPAK3cUrySD6IeEIGa6pldb3dpto= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-319-WrUbm6UZOa6KTyfW8MvRiQ-1; Fri, 28 Feb 2025 07:18:07 -0500 X-MC-Unique: WrUbm6UZOa6KTyfW8MvRiQ-1 X-Mimecast-MFC-AGG-ID: WrUbm6UZOa6KTyfW8MvRiQ_1740745086 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C348B1955BFC; Fri, 28 Feb 2025 12:18:05 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.175]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 31A08180098C; Fri, 28 Feb 2025 12:18:01 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, berrange@redhat.com, Prasad Pandit Subject: [PATCH v7 1/5] migration/multifd: move macros to multifd header Date: Fri, 28 Feb 2025 17:47:45 +0530 Message-ID: <20250228121749.553184-2-ppandit@redhat.com> In-Reply-To: <20250228121749.553184-1-ppandit@redhat.com> References: <20250228121749.553184-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.438, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Move MULTIFD_ macros to the header file so that they are accessible from other source files. Reviewed-by: Fabiano Rosas Signed-off-by: Prasad Pandit --- migration/multifd.c | 5 ----- migration/multifd.h | 5 +++++ 2 files changed, 5 insertions(+), 5 deletions(-) v6: no change - https://lore.kernel.org/qemu-devel/20250215123119.814345-1-ppandit@redhat.com/T/#t diff --git a/migration/multifd.c b/migration/multifd.c index 215ad0414a..30e514785f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -33,11 +33,6 @@ #include "io/channel-socket.h" #include "yank_functions.h" -/* Multiple fd's */ - -#define MULTIFD_MAGIC 0x11223344U -#define MULTIFD_VERSION 1 - typedef struct { uint32_t magic; uint32_t version; diff --git a/migration/multifd.h b/migration/multifd.h index cf408ff721..bff867ca6b 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -49,6 +49,11 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offset); bool multifd_recv(void); MultiFDRecvData *multifd_get_recv_data(void); +/* Multiple fd's */ + +#define MULTIFD_MAGIC 0x11223344U +#define MULTIFD_VERSION 1 + /* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) From patchwork Fri Feb 28 12:17:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13996336 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85C0BC19776 for ; Fri, 28 Feb 2025 12:19:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tnzJt-0004Cf-ET; Fri, 28 Feb 2025 07:18:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJo-0004Bu-6G for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJl-0001GL-O8 for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740745092; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fQGZ+HC/OVwWaheXEX5ZIeSt5B4zBEPgqTFgyQhqQ+I=; b=RRAJTvPOFFGJGB6LDO58Uwmt9FVaaO7LSQyLxjbEHdkqI0y/nd9NwxgN1b555fluBKu7m5 qON0eAiTEathaBWcCRmL9BKiye+jNo9IEjaPWa1fZrRoBws3czI6PUOlOOBNHGYttVhthM 49mq4zlY1tjscPsJyL1isQaW/9uJGso= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-304-1AWUMTiVNVac5MlXsAqvsA-1; Fri, 28 Feb 2025 07:18:11 -0500 X-MC-Unique: 1AWUMTiVNVac5MlXsAqvsA-1 X-Mimecast-MFC-AGG-ID: 1AWUMTiVNVac5MlXsAqvsA_1740745090 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 79A6919540EE; Fri, 28 Feb 2025 12:18:10 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.175]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DF60E180087F; Fri, 28 Feb 2025 12:18:06 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, berrange@redhat.com, Prasad Pandit Subject: [PATCH v7 2/5] migration: enable multifd and postcopy together Date: Fri, 28 Feb 2025 17:47:46 +0530 Message-ID: <20250228121749.553184-3-ppandit@redhat.com> In-Reply-To: <20250228121749.553184-1-ppandit@redhat.com> References: <20250228121749.553184-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass client-ip=170.10.133.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.438, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Enable Multifd and Postcopy migration together. The migration_ioc_process_incoming() routine checks magic value sent on each channel and helps to properly setup multifd and postcopy channels. The Precopy and Multifd threads work during the initial guest RAM transfer. When migration moves to the Postcopy phase, the multifd threads are shutdown and Postcopy threads on the destination request/pull data from the source side. Signed-off-by: Prasad Pandit --- migration/migration.c | 133 +++++++++++++++++++++---------------- migration/multifd-nocomp.c | 3 +- migration/multifd.c | 11 ++- migration/options.c | 5 -- migration/ram.c | 7 +- 5 files changed, 89 insertions(+), 70 deletions(-) v6: update channel discovery part in migration_ioc_process_incoming() - https://lore.kernel.org/qemu-devel/20250215123119.814345-1-ppandit@redhat.com/T/#t diff --git a/migration/migration.c b/migration/migration.c index c597aa707e..5db9e18272 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -95,6 +95,9 @@ enum mig_rp_message_type { MIG_RP_MSG_MAX }; +/* Migration channel types */ +enum { CH_MAIN, CH_MULTIFD, CH_POSTCOPY }; + /* When we add fault tolerance, we could have several migrations at once. For now we don't need to add dynamic creation of migration */ @@ -947,28 +950,19 @@ void migration_fd_process_incoming(QEMUFile *f) migration_incoming_process(); } -/* - * Returns true when we want to start a new incoming migration process, - * false otherwise. - */ -static bool migration_should_start_incoming(bool main_channel) +static bool migration_has_main_and_multifd_channels(void) { - /* Multifd doesn't start unless all channels are established */ - if (migrate_multifd()) { - return migration_has_all_channels(); + MigrationIncomingState *mis = migration_incoming_get_current(); + if (!mis->from_src_file) { + /* main channel not established */ + return false; } - /* Preempt channel only starts when the main channel is created */ - if (migrate_postcopy_preempt()) { - return main_channel; + if (migrate_multifd() && !multifd_recv_all_channels_created()) { + return false; } - /* - * For all the rest types of migration, we should only reach here when - * it's the main channel that's being created, and we should always - * proceed with this channel. - */ - assert(main_channel); + /* main channel and all multifd channels are established */ return true; } @@ -977,59 +971,84 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) MigrationIncomingState *mis = migration_incoming_get_current(); Error *local_err = NULL; QEMUFile *f; - bool default_channel = true; + uint8_t channel; uint32_t channel_magic = 0; int ret = 0; - if (migrate_multifd() && !migrate_mapped_ram() && - !migrate_postcopy_ram() && - qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { - /* - * With multiple channels, it is possible that we receive channels - * out of order on destination side, causing incorrect mapping of - * source channels on destination side. Check channel MAGIC to - * decide type of channel. Please note this is best effort, postcopy - * preempt channel does not send any magic number so avoid it for - * postcopy live migration. Also tls live migration already does - * tls handshake while initializing main channel so with tls this - * issue is not possible. - */ - ret = migration_channel_read_peek(ioc, (void *)&channel_magic, - sizeof(channel_magic), errp); + if (!migration_has_main_and_multifd_channels()) { + if (qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + /* + * With multiple channels, it is possible that we receive channels + * out of order on destination side, causing incorrect mapping of + * source channels on destination side. Check channel MAGIC to + * decide type of channel. Please note this is best effort, + * postcopy preempt channel does not send any magic number so + * avoid it for postcopy live migration. Also tls live migration + * already does tls handshake while initializing main channel so + * with tls this issue is not possible. + */ + ret = migration_channel_read_peek(ioc, (void *)&channel_magic, + sizeof(channel_magic), errp); + if (ret != 0) { + return; + } - if (ret != 0) { + channel_magic = be32_to_cpu(channel_magic); + if (channel_magic == QEMU_VM_FILE_MAGIC) { + channel = CH_MAIN; + } else if (channel_magic == MULTIFD_MAGIC) { + channel = CH_MULTIFD; + } else if (!mis->from_src_file && + mis->state == MIGRATION_STATUS_POSTCOPY_PAUSED) { + /* reconnect main channel for postcopy recovery */ + channel = CH_MAIN; + } else { + error_setg(errp, "unknown channel magic: %u", channel_magic); + return; + } + } else if (mis->from_src_file && migrate_multifd()) { + /* + * Non-peekable channels like tls/file are processed as + * multifd channels when multifd is enabled. + */ + channel = CH_MULTIFD; + } else if (!mis->from_src_file) { + channel = CH_MAIN; + } else { + error_setg(errp, "non-peekable channel used without multifd"); return; } - - default_channel = (channel_magic == cpu_to_be32(QEMU_VM_FILE_MAGIC)); + } else if (mis->from_src_file) { + channel = CH_POSTCOPY; } else { - default_channel = !mis->from_src_file; + channel = CH_MAIN; } if (multifd_recv_setup(errp) != 0) { return; } - if (default_channel) { + if (channel == CH_MAIN) { f = qemu_file_new_input(ioc); migration_incoming_setup(f); - } else { + } else if (channel == CH_MULTIFD) { /* Multiple connections */ - assert(migration_needs_multiple_sockets()); if (migrate_multifd()) { multifd_recv_new_channel(ioc, &local_err); - } else { - assert(migrate_postcopy_preempt()); - f = qemu_file_new_input(ioc); - postcopy_preempt_new_channel(mis, f); } if (local_err) { error_propagate(errp, local_err); return; } + } else if (channel == CH_POSTCOPY) { + assert(migrate_postcopy_preempt()); + assert(!mis->postcopy_qemufile_dst); + f = qemu_file_new_input(ioc); + postcopy_preempt_new_channel(mis, f); + return; } - if (migration_should_start_incoming(default_channel)) { + if (migration_has_main_and_multifd_channels()) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; @@ -1046,20 +1065,15 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) */ bool migration_has_all_channels(void) { + if (!migration_has_main_and_multifd_channels()) { + return false; + } + MigrationIncomingState *mis = migration_incoming_get_current(); - - if (!mis->from_src_file) { + if (migrate_postcopy_preempt() && !mis->postcopy_qemufile_dst) { return false; } - if (migrate_multifd()) { - return multifd_recv_all_channels_created(); - } - - if (migrate_postcopy_preempt()) { - return mis->postcopy_qemufile_dst != NULL; - } - return true; } @@ -1470,6 +1484,8 @@ static void migration_cleanup(MigrationState *s) assert(!migration_is_active()); + file_cleanup_outgoing_migration(); + socket_cleanup_outgoing_migration(); if (s->state == MIGRATION_STATUS_CANCELLING) { migrate_set_state(&s->state, MIGRATION_STATUS_CANCELLING, MIGRATION_STATUS_CANCELLED); @@ -3382,8 +3398,11 @@ static MigIterateState migration_iteration_run(MigrationState *s) } /* Still a significant amount to transfer */ - if (!in_postcopy && must_precopy <= s->threshold_size && can_switchover && - qatomic_read(&s->start_postcopy)) { + if (!in_postcopy && must_precopy <= s->threshold_size + && can_switchover && qatomic_read(&s->start_postcopy)) { + if (migrate_multifd()) { + multifd_send_shutdown(); + } if (postcopy_start(s, &local_err)) { migrate_set_error(s, local_err); error_report_err(local_err); diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c index 1325dba97c..d0edec7cd1 100644 --- a/migration/multifd-nocomp.c +++ b/migration/multifd-nocomp.c @@ -16,6 +16,7 @@ #include "file.h" #include "multifd.h" #include "options.h" +#include "migration.h" #include "qapi/error.h" #include "qemu/cutils.h" #include "qemu/error-report.h" @@ -391,7 +392,7 @@ int multifd_ram_flush_and_sync(QEMUFile *f) MultiFDSyncReq req; int ret; - if (!migrate_multifd()) { + if (!migrate_multifd() || migration_in_postcopy()) { return 0; } diff --git a/migration/multifd.c b/migration/multifd.c index 30e514785f..2bd8604ca1 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -467,8 +467,6 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) static void multifd_send_cleanup_state(void) { - file_cleanup_outgoing_migration(); - socket_cleanup_outgoing_migration(); qemu_sem_destroy(&multifd_send_state->channels_created); qemu_sem_destroy(&multifd_send_state->channels_ready); g_free(multifd_send_state->params); @@ -481,7 +479,7 @@ void multifd_send_shutdown(void) { int i; - if (!migrate_multifd()) { + if (!migrate_multifd() || !multifd_send_state) { return; } @@ -1231,6 +1229,13 @@ static void *multifd_recv_thread(void *opaque) } if (has_data) { + /* + * multifd thread should not be active and receive data + * when migration is in the Postcopy phase. Two threads + * writing the same memory area could easily corrupt + * the guest state. + */ + assert(!migration_in_postcopy()); ret = multifd_recv_state->ops->recv(p, &local_err); if (ret != 0) { break; diff --git a/migration/options.c b/migration/options.c index bb259d192a..5d56591f9e 100644 --- a/migration/options.c +++ b/migration/options.c @@ -482,11 +482,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) error_setg(errp, "Postcopy is not compatible with ignore-shared"); return false; } - - if (new_caps[MIGRATION_CAPABILITY_MULTIFD]) { - error_setg(errp, "Postcopy is not yet compatible with multifd"); - return false; - } } if (new_caps[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { diff --git a/migration/ram.c b/migration/ram.c index 589b6505eb..2d8e593c90 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1297,7 +1297,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) pss->page = 0; pss->block = QLIST_NEXT_RCU(pss->block, next); if (!pss->block) { - if (multifd_ram_sync_per_round()) { + if (multifd_ram_sync_per_round() && !migration_in_postcopy()) { QEMUFile *f = rs->pss[RAM_CHANNEL_PRECOPY].pss_channel; int ret = multifd_ram_flush_and_sync(f); if (ret < 0) { @@ -1971,9 +1971,8 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) } } - if (migrate_multifd()) { - RAMBlock *block = pss->block; - return ram_save_multifd_page(block, offset); + if (migrate_multifd() && !migration_in_postcopy()) { + return ram_save_multifd_page(pss->block, offset); } if (control_save_page(pss, offset, &res)) { From patchwork Fri Feb 28 12:17:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13996334 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 714A3C282CD for ; Fri, 28 Feb 2025 12:19:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tnzJw-0004Kb-HK; Fri, 28 Feb 2025 07:18:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJu-0004H3-Sb for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJr-0001Gc-OX for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740745098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Uia+fes4pNtdusLHioa6C5P5V8u5kbWAm64xht2Ey9w=; b=cwkF5JAA6hA9CbpQUOOB/lWvG2DAA/tSlzU9MJ1p4A5igAnbr1ggtH/Cx9Qy+0uqg87THT GEStPtgEEFd/e1pHZNX5gWJ18+9BlJxoJONzyt+CO98PNSlvwzN4cUBZ1N0cDl6MS/R7Et Os8RQe4bxkufvToJHT/Q2xFM6803a1I= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-673-yn-wRK8FNgWV6n0_7UoEiQ-1; Fri, 28 Feb 2025 07:18:17 -0500 X-MC-Unique: yn-wRK8FNgWV6n0_7UoEiQ-1 X-Mimecast-MFC-AGG-ID: yn-wRK8FNgWV6n0_7UoEiQ_1740745096 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 516051944D01; Fri, 28 Feb 2025 12:18:16 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.175]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 912E1180087F; Fri, 28 Feb 2025 12:18:10 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, berrange@redhat.com, Prasad Pandit Subject: [PATCH v7 3/5] tests/qtest/migration: consolidate set capabilities Date: Fri, 28 Feb 2025 17:47:47 +0530 Message-ID: <20250228121749.553184-4-ppandit@redhat.com> In-Reply-To: <20250228121749.553184-1-ppandit@redhat.com> References: <20250228121749.553184-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.438, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Migration capabilities are set in multiple '.start_hook' functions for various tests. Instead, consolidate setting capabilities in 'migrate_start_set_capabilities()' function which is called from the 'migrate_start()' function. While simplifying the capabilities setting, it helps to declutter the qtest sources. Suggested-by: Fabiano Rosas Signed-off-by: Prasad Pandit --- tests/qtest/migration/compression-tests.c | 22 +++++-- tests/qtest/migration/cpr-tests.c | 6 +- tests/qtest/migration/file-tests.c | 58 ++++++++--------- tests/qtest/migration/framework.c | 76 ++++++++++++++++------- tests/qtest/migration/framework.h | 9 ++- tests/qtest/migration/misc-tests.c | 4 +- tests/qtest/migration/postcopy-tests.c | 8 ++- tests/qtest/migration/precopy-tests.c | 29 +++++---- tests/qtest/migration/tls-tests.c | 23 ++++++- 9 files changed, 151 insertions(+), 84 deletions(-) v6: Move .caps initialization to the .start object. Set default multifd threads to 4. - https://lore.kernel.org/qemu-devel/20250215123119.814345-1-ppandit@redhat.com/T/#t diff --git a/tests/qtest/migration/compression-tests.c b/tests/qtest/migration/compression-tests.c index 8b58401b84..41e79f031b 100644 --- a/tests/qtest/migration/compression-tests.c +++ b/tests/qtest/migration/compression-tests.c @@ -35,6 +35,9 @@ static void test_multifd_tcp_zstd(void) { MigrateCommon args = { .listen_uri = "defer", + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, .start_hook = migrate_hook_start_precopy_tcp_multifd_zstd, }; test_precopy_common(&args); @@ -56,6 +59,9 @@ static void test_multifd_tcp_qatzip(void) { MigrateCommon args = { .listen_uri = "defer", + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, .start_hook = migrate_hook_start_precopy_tcp_multifd_qatzip, }; test_precopy_common(&args); @@ -74,6 +80,9 @@ static void test_multifd_tcp_qpl(void) { MigrateCommon args = { .listen_uri = "defer", + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, .start_hook = migrate_hook_start_precopy_tcp_multifd_qpl, }; test_precopy_common(&args); @@ -92,6 +101,9 @@ static void test_multifd_tcp_uadk(void) { MigrateCommon args = { .listen_uri = "defer", + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, .start_hook = migrate_hook_start_precopy_tcp_multifd_uadk, }; test_precopy_common(&args); @@ -103,10 +115,6 @@ migrate_hook_start_xbzrle(QTestState *from, QTestState *to) { migrate_set_parameter_int(from, "xbzrle-cache-size", 33554432); - - migrate_set_capability(from, "xbzrle", true); - migrate_set_capability(to, "xbzrle", true); - return NULL; } @@ -118,6 +126,9 @@ static void test_precopy_unix_xbzrle(void) .listen_uri = uri, .start_hook = migrate_hook_start_xbzrle, .iterations = 2, + .start = { + .caps[MIGRATION_CAPABILITY_XBZRLE] = true, + }, /* * XBZRLE needs pages to be modified when doing the 2nd+ round * iteration to have real data pushed to the stream. @@ -146,6 +157,9 @@ static void test_multifd_tcp_zlib(void) { MigrateCommon args = { .listen_uri = "defer", + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, .start_hook = migrate_hook_start_precopy_tcp_multifd_zlib, }; test_precopy_common(&args); diff --git a/tests/qtest/migration/cpr-tests.c b/tests/qtest/migration/cpr-tests.c index 4758841824..5536e14610 100644 --- a/tests/qtest/migration/cpr-tests.c +++ b/tests/qtest/migration/cpr-tests.c @@ -24,9 +24,6 @@ static void *migrate_hook_start_mode_reboot(QTestState *from, QTestState *to) migrate_set_parameter_str(from, "mode", "cpr-reboot"); migrate_set_parameter_str(to, "mode", "cpr-reboot"); - migrate_set_capability(from, "x-ignore-shared", true); - migrate_set_capability(to, "x-ignore-shared", true); - return NULL; } @@ -39,6 +36,9 @@ static void test_mode_reboot(void) .connect_uri = uri, .listen_uri = "defer", .start_hook = migrate_hook_start_mode_reboot, + .start = { + .caps[MIGRATION_CAPABILITY_X_IGNORE_SHARED] = true, + }, }; test_file_common(&args, true); diff --git a/tests/qtest/migration/file-tests.c b/tests/qtest/migration/file-tests.c index f260e2871d..4d78ce0855 100644 --- a/tests/qtest/migration/file-tests.c +++ b/tests/qtest/migration/file-tests.c @@ -107,15 +107,6 @@ static void test_precopy_file_offset_bad(void) test_file_common(&args, false); } -static void *migrate_hook_start_mapped_ram(QTestState *from, - QTestState *to) -{ - migrate_set_capability(from, "mapped-ram", true); - migrate_set_capability(to, "mapped-ram", true); - - return NULL; -} - static void test_precopy_file_mapped_ram_live(void) { g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, @@ -123,7 +114,9 @@ static void test_precopy_file_mapped_ram_live(void) MigrateCommon args = { .connect_uri = uri, .listen_uri = "defer", - .start_hook = migrate_hook_start_mapped_ram, + .start = { + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + }, }; test_file_common(&args, false); @@ -136,26 +129,14 @@ static void test_precopy_file_mapped_ram(void) MigrateCommon args = { .connect_uri = uri, .listen_uri = "defer", - .start_hook = migrate_hook_start_mapped_ram, + .start = { + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + }, }; test_file_common(&args, true); } -static void *migrate_hook_start_multifd_mapped_ram(QTestState *from, - QTestState *to) -{ - migrate_hook_start_mapped_ram(from, to); - - migrate_set_parameter_int(from, "multifd-channels", 4); - migrate_set_parameter_int(to, "multifd-channels", 4); - - migrate_set_capability(from, "multifd", true); - migrate_set_capability(to, "multifd", true); - - return NULL; -} - static void test_multifd_file_mapped_ram_live(void) { g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, @@ -163,7 +144,10 @@ static void test_multifd_file_mapped_ram_live(void) MigrateCommon args = { .connect_uri = uri, .listen_uri = "defer", - .start_hook = migrate_hook_start_multifd_mapped_ram, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + }, }; test_file_common(&args, false); @@ -176,7 +160,10 @@ static void test_multifd_file_mapped_ram(void) MigrateCommon args = { .connect_uri = uri, .listen_uri = "defer", - .start_hook = migrate_hook_start_multifd_mapped_ram, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + }, }; test_file_common(&args, true); @@ -185,8 +172,6 @@ static void test_multifd_file_mapped_ram(void) static void *migrate_hook_start_multifd_mapped_ram_dio(QTestState *from, QTestState *to) { - migrate_hook_start_multifd_mapped_ram(from, to); - migrate_set_parameter_bool(from, "direct-io", true); migrate_set_parameter_bool(to, "direct-io", true); @@ -201,6 +186,10 @@ static void test_multifd_file_mapped_ram_dio(void) .connect_uri = uri, .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_mapped_ram_dio, + .start = { + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; if (!probe_o_direct_support(tmpfs)) { @@ -246,7 +235,6 @@ static void *migrate_hook_start_multifd_mapped_ram_fdset_dio(QTestState *from, fdset_add_fds(from, file, O_WRONLY, 2, true); fdset_add_fds(to, file, O_RDONLY, 2, true); - migrate_hook_start_multifd_mapped_ram(from, to); migrate_set_parameter_bool(from, "direct-io", true); migrate_set_parameter_bool(to, "direct-io", true); @@ -261,8 +249,6 @@ static void *migrate_hook_start_multifd_mapped_ram_fdset(QTestState *from, fdset_add_fds(from, file, O_WRONLY, 2, false); fdset_add_fds(to, file, O_RDONLY, 2, false); - migrate_hook_start_multifd_mapped_ram(from, to); - return NULL; } @@ -275,6 +261,10 @@ static void test_multifd_file_mapped_ram_fdset(void) .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_mapped_ram_fdset, .end_hook = migrate_hook_end_multifd_mapped_ram_fdset, + .start = { + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; test_file_common(&args, true); @@ -289,6 +279,10 @@ static void test_multifd_file_mapped_ram_fdset_dio(void) .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_mapped_ram_fdset_dio, .end_hook = migrate_hook_end_multifd_mapped_ram_fdset, + .start = { + .caps[MIGRATION_CAPABILITY_MAPPED_RAM] = true, + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; if (!probe_o_direct_support(tmpfs)) { diff --git a/tests/qtest/migration/framework.c b/tests/qtest/migration/framework.c index 10e1d04b58..be6c245843 100644 --- a/tests/qtest/migration/framework.c +++ b/tests/qtest/migration/framework.c @@ -30,6 +30,7 @@ #define QEMU_VM_FILE_MAGIC 0x5145564d #define QEMU_ENV_SRC "QTEST_QEMU_BINARY_SRC" #define QEMU_ENV_DST "QTEST_QEMU_BINARY_DST" +#define MULTIFD_TEST_CHANNELS 4 unsigned start_address; unsigned end_address; @@ -207,6 +208,52 @@ static QList *migrate_start_get_qmp_capabilities(const MigrateStart *args) return capabilities; } +static void migrate_start_set_capabilities(QTestState *from, QTestState *to, + MigrateStart *args) +{ + /* + * MigrationCapability_lookup and MIGRATION_CAPABILITY_ constants + * are from qapi-types-migration.h. + */ + for (uint8_t i = 0; i < MIGRATION_CAPABILITY__MAX; i++) + { + if (!args->caps[i]) { + continue; + } + if (from) { + migrate_set_capability(from, + MigrationCapability_lookup.array[i], true); + } + if (to) { + migrate_set_capability(to, + MigrationCapability_lookup.array[i], true); + } + } + + /* + * Always enable migration events. Libvirt always uses it, let's try + * to mimic as closer as that. + */ + migrate_set_capability(from, "events", true); + if (!args->defer_target_connect) { + migrate_set_capability(to, "events", true); + } + + /* + * Default number of channels should be fine for most + * tests. Individual tests can override by calling + * migrate_set_parameter() directly. + */ + if (args->caps[MIGRATION_CAPABILITY_MULTIFD]) { + migrate_set_parameter_int(from, "multifd-channels", + MULTIFD_TEST_CHANNELS); + migrate_set_parameter_int(to, "multifd-channels", + MULTIFD_TEST_CHANNELS); + } + + return; +} + int migrate_start(QTestState **from, QTestState **to, const char *uri, MigrateStart *args) { @@ -379,14 +426,7 @@ int migrate_start(QTestState **from, QTestState **to, const char *uri, unlink(shmem_path); } - /* - * Always enable migration events. Libvirt always uses it, let's try - * to mimic as closer as that. - */ - migrate_set_capability(*from, "events", true); - if (!args->defer_target_connect) { - migrate_set_capability(*to, "events", true); - } + migrate_start_set_capabilities(*from, *to, args); return 0; } @@ -432,6 +472,10 @@ static int migrate_postcopy_prepare(QTestState **from_ptr, { QTestState *from, *to; + /* set postcopy capabilities */ + args->start.caps[MIGRATION_CAPABILITY_POSTCOPY_BLOCKTIME] = true; + args->start.caps[MIGRATION_CAPABILITY_POSTCOPY_RAM] = true; + if (migrate_start(&from, &to, "defer", &args->start)) { return -1; } @@ -440,17 +484,7 @@ static int migrate_postcopy_prepare(QTestState **from_ptr, args->postcopy_data = args->start_hook(from, to); } - migrate_set_capability(from, "postcopy-ram", true); - migrate_set_capability(to, "postcopy-ram", true); - migrate_set_capability(to, "postcopy-blocktime", true); - - if (args->postcopy_preempt) { - migrate_set_capability(from, "postcopy-preempt", true); - migrate_set_capability(to, "postcopy-preempt", true); - } - migrate_ensure_non_converge(from); - migrate_prepare_for_dirty_mem(from); qtest_qmp_assert_success(to, "{ 'execute': 'migrate-incoming'," " 'arguments': { " @@ -948,15 +982,9 @@ void *migrate_hook_start_precopy_tcp_multifd_common(QTestState *from, QTestState *to, const char *method) { - migrate_set_parameter_int(from, "multifd-channels", 16); - migrate_set_parameter_int(to, "multifd-channels", 16); - migrate_set_parameter_str(from, "multifd-compression", method); migrate_set_parameter_str(to, "multifd-compression", method); - migrate_set_capability(from, "multifd", true); - migrate_set_capability(to, "multifd", true); - /* Start incoming migration from the 1st socket */ migrate_incoming_qmp(to, "tcp:127.0.0.1:0", NULL, "{}"); diff --git a/tests/qtest/migration/framework.h b/tests/qtest/migration/framework.h index e4a11870f6..01e425e64e 100644 --- a/tests/qtest/migration/framework.h +++ b/tests/qtest/migration/framework.h @@ -12,6 +12,7 @@ #define TEST_FRAMEWORK_H #include "libqtest.h" +#include #define FILE_TEST_FILENAME "migfile" #define FILE_TEST_OFFSET 0x1000 @@ -120,6 +121,13 @@ typedef struct { /* Do not connect to target monitor and qtest sockets in qtest_init */ bool defer_target_connect; + + /* + * Migration capabilities to be set in both source and + * destination. For unilateral capabilities, use + * migration_set_capabilities(). + */ + bool caps[MIGRATION_CAPABILITY__MAX]; } MigrateStart; typedef enum PostcopyRecoveryFailStage { @@ -207,7 +215,6 @@ typedef struct { /* Postcopy specific fields */ void *postcopy_data; - bool postcopy_preempt; PostcopyRecoveryFailStage postcopy_recovery_fail_stage; } MigrateCommon; diff --git a/tests/qtest/migration/misc-tests.c b/tests/qtest/migration/misc-tests.c index 2e612d9e38..54995256d8 100644 --- a/tests/qtest/migration/misc-tests.c +++ b/tests/qtest/migration/misc-tests.c @@ -98,6 +98,7 @@ static void test_ignore_shared(void) QTestState *from, *to; MigrateStart args = { .use_shmem = true, + .caps[MIGRATION_CAPABILITY_X_IGNORE_SHARED] = true, }; if (migrate_start(&from, &to, uri, &args)) { @@ -107,9 +108,6 @@ static void test_ignore_shared(void) migrate_ensure_non_converge(from); migrate_prepare_for_dirty_mem(from); - migrate_set_capability(from, "x-ignore-shared", true); - migrate_set_capability(to, "x-ignore-shared", true); - /* Wait for the first serial output from the source */ wait_for_serial("src_serial"); diff --git a/tests/qtest/migration/postcopy-tests.c b/tests/qtest/migration/postcopy-tests.c index 982457bed1..483e3ff99f 100644 --- a/tests/qtest/migration/postcopy-tests.c +++ b/tests/qtest/migration/postcopy-tests.c @@ -39,7 +39,9 @@ static void test_postcopy_suspend(void) static void test_postcopy_preempt(void) { MigrateCommon args = { - .postcopy_preempt = true, + .start = { + .caps[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT] = true, + }, }; test_postcopy_common(&args); @@ -73,7 +75,9 @@ static void test_postcopy_recovery_fail_reconnect(void) static void test_postcopy_preempt_recovery(void) { MigrateCommon args = { - .postcopy_preempt = true, + .start = { + .caps[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT] = true, + }, }; test_postcopy_recovery_common(&args); diff --git a/tests/qtest/migration/precopy-tests.c b/tests/qtest/migration/precopy-tests.c index ba273d10b9..f8404793b8 100644 --- a/tests/qtest/migration/precopy-tests.c +++ b/tests/qtest/migration/precopy-tests.c @@ -108,23 +108,14 @@ static void test_precopy_tcp_plain(void) test_precopy_common(&args); } -static void *migrate_hook_start_switchover_ack(QTestState *from, QTestState *to) -{ - - migrate_set_capability(from, "return-path", true); - migrate_set_capability(to, "return-path", true); - - migrate_set_capability(from, "switchover-ack", true); - migrate_set_capability(to, "switchover-ack", true); - - return NULL; -} - static void test_precopy_tcp_switchover_ack(void) { MigrateCommon args = { .listen_uri = "tcp:127.0.0.1:0", - .start_hook = migrate_hook_start_switchover_ack, + .start = { + .caps[MIGRATION_CAPABILITY_RETURN_PATH] = true, + .caps[MIGRATION_CAPABILITY_SWITCHOVER_ACK] = true, + }, /* * Source VM must be running in order to consider the switchover ACK * when deciding to do switchover or not. @@ -393,6 +384,9 @@ static void test_multifd_tcp_uri_none(void) MigrateCommon args = { .listen_uri = "defer", .start_hook = migrate_hook_start_precopy_tcp_multifd, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, /* * Multifd is more complicated than most of the features, it * directly takes guest page buffers when sending, make sure @@ -408,6 +402,9 @@ static void test_multifd_tcp_zero_page_legacy(void) MigrateCommon args = { .listen_uri = "defer", .start_hook = migrate_hook_start_precopy_tcp_multifd_zero_page_legacy, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, /* * Multifd is more complicated than most of the features, it * directly takes guest page buffers when sending, make sure @@ -423,6 +420,9 @@ static void test_multifd_tcp_no_zero_page(void) MigrateCommon args = { .listen_uri = "defer", .start_hook = migrate_hook_start_precopy_tcp_multifd_no_zero_page, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, /* * Multifd is more complicated than most of the features, it * directly takes guest page buffers when sending, make sure @@ -439,6 +439,9 @@ static void test_multifd_tcp_channels_none(void) .listen_uri = "defer", .start_hook = migrate_hook_start_precopy_tcp_multifd, .live = true, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, .connect_channels = ("[ { 'channel-type': 'main'," " 'addr': { 'transport': 'socket'," " 'type': 'inet'," diff --git a/tests/qtest/migration/tls-tests.c b/tests/qtest/migration/tls-tests.c index 2cb4a44bcd..72f44defbb 100644 --- a/tests/qtest/migration/tls-tests.c +++ b/tests/qtest/migration/tls-tests.c @@ -375,9 +375,11 @@ static void test_postcopy_tls_psk(void) static void test_postcopy_preempt_tls_psk(void) { MigrateCommon args = { - .postcopy_preempt = true, .start_hook = migrate_hook_start_tls_psk_match, .end_hook = migrate_hook_end_tls_psk, + .start = { + .caps[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT] = true, + }, }; test_postcopy_common(&args); @@ -397,9 +399,11 @@ static void test_postcopy_recovery_tls_psk(void) static void test_postcopy_preempt_all(void) { MigrateCommon args = { - .postcopy_preempt = true, .start_hook = migrate_hook_start_tls_psk_match, .end_hook = migrate_hook_end_tls_psk, + .start = { + .caps[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT] = true, + }, }; test_postcopy_recovery_common(&args); @@ -631,6 +635,9 @@ static void test_multifd_tcp_tls_psk_match(void) .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tcp_tls_psk_match, .end_hook = migrate_hook_end_tls_psk, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; test_precopy_common(&args); } @@ -640,6 +647,7 @@ static void test_multifd_tcp_tls_psk_mismatch(void) MigrateCommon args = { .start = { .hide_stderr = true, + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, }, .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tcp_tls_psk_mismatch, @@ -656,6 +664,9 @@ static void test_multifd_tcp_tls_x509_default_host(void) .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tls_x509_default_host, .end_hook = migrate_hook_end_tls_x509, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; test_precopy_common(&args); } @@ -666,6 +677,9 @@ static void test_multifd_tcp_tls_x509_override_host(void) .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tls_x509_override_host, .end_hook = migrate_hook_end_tls_x509, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; test_precopy_common(&args); } @@ -688,6 +702,7 @@ static void test_multifd_tcp_tls_x509_mismatch_host(void) MigrateCommon args = { .start = { .hide_stderr = true, + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, }, .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tls_x509_mismatch_host, @@ -703,6 +718,9 @@ static void test_multifd_tcp_tls_x509_allow_anon_client(void) .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tls_x509_allow_anon_client, .end_hook = migrate_hook_end_tls_x509, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, }; test_precopy_common(&args); } @@ -712,6 +730,7 @@ static void test_multifd_tcp_tls_x509_reject_anon_client(void) MigrateCommon args = { .start = { .hide_stderr = true, + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, }, .listen_uri = "defer", .start_hook = migrate_hook_start_multifd_tls_x509_reject_anon_client, From patchwork Fri Feb 28 12:17:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13996333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B7FD0C282CD for ; Fri, 28 Feb 2025 12:18:51 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tnzK2-0004UE-O3; Fri, 28 Feb 2025 07:18:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzK0-0004PH-6u for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzJx-0001H7-Vn for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740745104; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NspCqIF6EkMAo/PsJWgsVgmYR+MCIC1H39YMCmein1A=; b=WlYMoexO9aOxRCQJqJpf+hDnDxeN4nqzdFolKsRjY9fSLWIaPvAvM8vZfCw/TDmS9tsyUH 1Cgk0DPofREw8blIgzAgbvsKqffD0mNUSUCH2Q66K1t0B7rxz5ZRoRONQQ8+P46rtbjTfq LfQN00tOI8yYtm0qZndQAEujUdpMqHM= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-591-88KZq4OBOFywJSQx-F4pfA-1; Fri, 28 Feb 2025 07:18:22 -0500 X-MC-Unique: 88KZq4OBOFywJSQx-F4pfA-1 X-Mimecast-MFC-AGG-ID: 88KZq4OBOFywJSQx-F4pfA_1740745101 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 46C851800373; Fri, 28 Feb 2025 12:18:21 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.175]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 515AB180087F; Fri, 28 Feb 2025 12:18:16 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, berrange@redhat.com, Prasad Pandit Subject: [PATCH v7 4/5] tests/qtest/migration: add postcopy tests with multifd Date: Fri, 28 Feb 2025 17:47:48 +0530 Message-ID: <20250228121749.553184-5-ppandit@redhat.com> In-Reply-To: <20250228121749.553184-1-ppandit@redhat.com> References: <20250228121749.553184-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.438, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Add new qtests to run postcopy migration with multifd channels enabled. Signed-off-by: Prasad Pandit --- tests/qtest/migration/compression-tests.c | 15 ++++++++ tests/qtest/migration/postcopy-tests.c | 27 +++++++++++++ tests/qtest/migration/precopy-tests.c | 19 ++++++++++ tests/qtest/migration/tls-tests.c | 46 +++++++++++++++++++++++ 4 files changed, 107 insertions(+) v6: Move .caps initialisations to .start object. - https://lore.kernel.org/qemu-devel/20250215123119.814345-1-ppandit@redhat.com/T/#t diff --git a/tests/qtest/migration/compression-tests.c b/tests/qtest/migration/compression-tests.c index 41e79f031b..592e85d69d 100644 --- a/tests/qtest/migration/compression-tests.c +++ b/tests/qtest/migration/compression-tests.c @@ -42,6 +42,19 @@ static void test_multifd_tcp_zstd(void) }; test_precopy_common(&args); } + +static void test_multifd_postcopy_tcp_zstd(void) +{ + MigrateCommon args = { + .listen_uri = "defer", + .start = { + .caps[MIGRATION_CAPABILITY_POSTCOPY_RAM] = true, + }, + .start_hook = migrate_hook_start_precopy_tcp_multifd_zstd, + }; + + test_precopy_common(&args); +} #endif /* CONFIG_ZSTD */ #ifdef CONFIG_QATZIP @@ -184,6 +197,8 @@ void migration_test_add_compression(MigrationTestEnv *env) #ifdef CONFIG_ZSTD migration_test_add("/migration/multifd/tcp/plain/zstd", test_multifd_tcp_zstd); + migration_test_add("/migration/multifd+postcopy/tcp/plain/zstd", + test_multifd_postcopy_tcp_zstd); #endif #ifdef CONFIG_QATZIP diff --git a/tests/qtest/migration/postcopy-tests.c b/tests/qtest/migration/postcopy-tests.c index 483e3ff99f..eb637f94f7 100644 --- a/tests/qtest/migration/postcopy-tests.c +++ b/tests/qtest/migration/postcopy-tests.c @@ -94,6 +94,29 @@ static void migration_test_add_postcopy_smoke(MigrationTestEnv *env) } } +static void test_multifd_postcopy(void) +{ + MigrateCommon args = { + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, + }; + + test_postcopy_common(&args); +} + +static void test_multifd_postcopy_preempt(void) +{ + MigrateCommon args = { + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + .caps[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT] = true, + }, + }; + + test_postcopy_common(&args); +} + void migration_test_add_postcopy(MigrationTestEnv *env) { migration_test_add_postcopy_smoke(env); @@ -114,6 +137,10 @@ void migration_test_add_postcopy(MigrationTestEnv *env) "/migration/postcopy/recovery/double-failures/reconnect", test_postcopy_recovery_fail_reconnect); + migration_test_add("/migration/postcopy/multifd/plain", + test_multifd_postcopy); + migration_test_add("/migration/postcopy/multifd/preempt/plain", + test_multifd_postcopy_preempt); if (env->is_x86) { migration_test_add("/migration/postcopy/suspend", test_postcopy_suspend); diff --git a/tests/qtest/migration/precopy-tests.c b/tests/qtest/migration/precopy-tests.c index f8404793b8..b2b0db8076 100644 --- a/tests/qtest/migration/precopy-tests.c +++ b/tests/qtest/migration/precopy-tests.c @@ -34,6 +34,7 @@ #define DIRTYLIMIT_TOLERANCE_RANGE 25 /* MB/s */ static char *tmpfs; +static bool postcopy_ram = false; static void test_precopy_unix_plain(void) { @@ -476,6 +477,11 @@ static void test_multifd_tcp_cancel(void) migrate_ensure_non_converge(from); migrate_prepare_for_dirty_mem(from); + if (postcopy_ram) { + migrate_set_capability(from, "postcopy-ram", true); + migrate_set_capability(to, "postcopy-ram", true); + } + migrate_set_parameter_int(from, "multifd-channels", 16); migrate_set_parameter_int(to, "multifd-channels", 16); @@ -517,6 +523,10 @@ static void test_multifd_tcp_cancel(void) return; } + if (postcopy_ram) { + migrate_set_capability(to2, "postcopy-ram", true); + } + migrate_set_parameter_int(to2, "multifd-channels", 16); migrate_set_capability(to2, "multifd", true); @@ -540,6 +550,13 @@ static void test_multifd_tcp_cancel(void) migrate_end(from, to2, true); } +static void test_multifd_postcopy_tcp_cancel(void) +{ + postcopy_ram = true; + test_multifd_tcp_cancel(); + postcopy_ram = false; +} + static void test_cancel_src_after_failed(QTestState *from, QTestState *to, const char *uri, const char *phase) { @@ -1127,6 +1144,8 @@ static void migration_test_add_precopy_smoke(MigrationTestEnv *env) test_multifd_tcp_uri_none); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); + migration_test_add("/migration/multifd+postcopy/tcp/plain/cancel", + test_multifd_postcopy_tcp_cancel); } void migration_test_add_precopy(MigrationTestEnv *env) diff --git a/tests/qtest/migration/tls-tests.c b/tests/qtest/migration/tls-tests.c index 72f44defbb..81a2faf4c0 100644 --- a/tests/qtest/migration/tls-tests.c +++ b/tests/qtest/migration/tls-tests.c @@ -395,6 +395,19 @@ static void test_postcopy_recovery_tls_psk(void) test_postcopy_recovery_common(&args); } +static void test_multifd_postcopy_recovery_tls_psk(void) +{ + MigrateCommon args = { + .start_hook = migrate_hook_start_tls_psk_match, + .end_hook = migrate_hook_end_tls_psk, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, + }; + + test_postcopy_recovery_common(&args); +} + /* This contains preempt+recovery+tls test altogether */ static void test_postcopy_preempt_all(void) { @@ -409,6 +422,19 @@ static void test_postcopy_preempt_all(void) test_postcopy_recovery_common(&args); } +static void test_multifd_postcopy_preempt_recovery_tls_psk(void) +{ + MigrateCommon args = { + .start_hook = migrate_hook_start_tls_psk_match, + .end_hook = migrate_hook_end_tls_psk, + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, + }; + + test_postcopy_recovery_common(&args); +} + static void test_precopy_unix_tls_psk(void) { g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs); @@ -657,6 +683,20 @@ static void test_multifd_tcp_tls_psk_mismatch(void) test_precopy_common(&args); } +static void test_multifd_postcopy_tcp_tls_psk_match(void) +{ + MigrateCommon args = { + .start = { + .caps[MIGRATION_CAPABILITY_MULTIFD] = true, + }, + .listen_uri = "defer", + .start_hook = migrate_hook_start_multifd_tcp_tls_psk_match, + .end_hook = migrate_hook_end_tls_psk, + }; + + test_precopy_common(&args); +} + #ifdef CONFIG_TASN1 static void test_multifd_tcp_tls_x509_default_host(void) { @@ -774,6 +814,10 @@ void migration_test_add_tls(MigrationTestEnv *env) test_postcopy_preempt_tls_psk); migration_test_add("/migration/postcopy/preempt/recovery/tls/psk", test_postcopy_preempt_all); + migration_test_add("/migration/postcopy/multifd/recovery/tls/psk", + test_multifd_postcopy_recovery_tls_psk); + migration_test_add("/migration/postcopy/multifd/preempt/recovery/tls/psk", + test_multifd_postcopy_preempt_recovery_tls_psk); } #ifdef CONFIG_TASN1 migration_test_add("/migration/precopy/unix/tls/x509/default-host", @@ -805,6 +849,8 @@ void migration_test_add_tls(MigrationTestEnv *env) test_multifd_tcp_tls_psk_match); migration_test_add("/migration/multifd/tcp/tls/psk/mismatch", test_multifd_tcp_tls_psk_mismatch); + migration_test_add("/migration/multifd+postcopy/tcp/tls/psk/match", + test_multifd_postcopy_tcp_tls_psk_match); #ifdef CONFIG_TASN1 migration_test_add("/migration/multifd/tcp/tls/x509/default-host", test_multifd_tcp_tls_x509_default_host); From patchwork Fri Feb 28 12:17:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13996335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57E21C19776 for ; Fri, 28 Feb 2025 12:19:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tnzK3-0004Uc-Mi; Fri, 28 Feb 2025 07:18:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzK2-0004UN-L0 for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tnzK0-0001Ha-Mv for qemu-devel@nongnu.org; Fri, 28 Feb 2025 07:18:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740745108; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ymdbOcFU28t6i+NJy7WqEtgrLRGdGbzIInM2PsVA9IE=; b=Yqb4U1P2CPSHZ5pGJ9HMDH3dNGgU+akNbP4rcjWY4Ie5d3gvmDMIDVXbt19wyk3QXG4JLM 4nn1b+FP2j8/o+ejB1cglhsBLoF/LdeUidHC5zUqgrqxD0s9jJzpTjjLO4oDPZVfxbKVoO sHaQoZW5jrqn4RRp4C9S3M6Fq3OC9Wk= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-482-IgV3vDqING-v7uF3k0NgJQ-1; Fri, 28 Feb 2025 07:18:26 -0500 X-MC-Unique: IgV3vDqING-v7uF3k0NgJQ-1 X-Mimecast-MFC-AGG-ID: IgV3vDqING-v7uF3k0NgJQ_1740745105 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 922AE180036F; Fri, 28 Feb 2025 12:18:25 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.175]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0B2F2180087F; Fri, 28 Feb 2025 12:18:21 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, berrange@redhat.com, Prasad Pandit Subject: [PATCH v7 5/5] migration: add MULTIFD_RECV_SYNC migration command Date: Fri, 28 Feb 2025 17:47:49 +0530 Message-ID: <20250228121749.553184-6-ppandit@redhat.com> In-Reply-To: <20250228121749.553184-1-ppandit@redhat.com> References: <20250228121749.553184-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass client-ip=170.10.133.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.438, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit When Multifd and Postcopy migration is enabled together, before starting Postcopy phase, Multifd threads are shutdown. But before this shutdown, we need to flush the Multifd channels on the source side and notify the destination migration thread to synchronise with the Multifd 'recv_x' threads, so that all the Multifd data is received and processed on the destination side, before Multifd threads are shutdown. This synchronisation happens when the main migration thread waits for all the Multifd threads to receive their data. Add 'MULTIFD_RECV_SYNC' migration command to notify the destination side to synchronise with Multifd threads. Suggested-by: Peter Xu Signed-off-by: Prasad Pandit Signed-off-by: Peter Xu --- migration/migration.c | 3 +++ migration/multifd-nocomp.c | 21 +++++++++++++++------ migration/multifd.c | 1 + migration/multifd.h | 1 + migration/savevm.c | 13 +++++++++++++ migration/savevm.h | 1 + 6 files changed, 34 insertions(+), 6 deletions(-) v6: New patch, not from earlier versions. - https://lore.kernel.org/qemu-devel/20250215123119.814345-1-ppandit@redhat.com/T/#t diff --git a/migration/migration.c b/migration/migration.c index 5db9e18272..65fc4f5eed 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3401,6 +3401,9 @@ static MigIterateState migration_iteration_run(MigrationState *s) if (!in_postcopy && must_precopy <= s->threshold_size && can_switchover && qatomic_read(&s->start_postcopy)) { if (migrate_multifd()) { + multifd_send_flush(); + multifd_send_sync_main(MULTIFD_SYNC_LOCAL); + qemu_savevm_send_multifd_recv_sync(s->to_dst_file); multifd_send_shutdown(); } if (postcopy_start(s, &local_err)) { diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c index d0edec7cd1..bbe07e4f7e 100644 --- a/migration/multifd-nocomp.c +++ b/migration/multifd-nocomp.c @@ -334,7 +334,7 @@ retry: * After flush, always retry. */ if (pages->block != block || multifd_queue_full(pages)) { - if (!multifd_send(&multifd_ram_send)) { + if (multifd_send_flush() < 0) { return false; } goto retry; @@ -387,6 +387,18 @@ bool multifd_ram_sync_per_round(void) return !migrate_multifd_flush_after_each_section(); } +int multifd_send_flush(void) +{ + if (!multifd_payload_empty(multifd_ram_send)) { + if (!multifd_send(&multifd_ram_send)) { + error_report("%s: multifd_send fail", __func__); + return -1; + } + } + + return 0; +} + int multifd_ram_flush_and_sync(QEMUFile *f) { MultiFDSyncReq req; @@ -396,11 +408,8 @@ int multifd_ram_flush_and_sync(QEMUFile *f) return 0; } - if (!multifd_payload_empty(multifd_ram_send)) { - if (!multifd_send(&multifd_ram_send)) { - error_report("%s: multifd_send fail", __func__); - return -1; - } + if ((ret = multifd_send_flush()) < 0) { + return ret; } /* File migrations only need to sync with threads */ diff --git a/migration/multifd.c b/migration/multifd.c index 2bd8604ca1..8928ca2611 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1265,6 +1265,7 @@ static void *multifd_recv_thread(void *opaque) rcu_unregister_thread(); trace_multifd_recv_thread_end(p->id, p->packets_recved); + qemu_sem_post(&multifd_recv_state->sem_sync); return NULL; } diff --git a/migration/multifd.h b/migration/multifd.h index bff867ca6b..242b923633 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -361,6 +361,7 @@ static inline uint32_t multifd_ram_page_count(void) void multifd_ram_save_setup(void); void multifd_ram_save_cleanup(void); +int multifd_send_flush(void); int multifd_ram_flush_and_sync(QEMUFile *f); bool multifd_ram_sync_per_round(void); bool multifd_ram_sync_per_section(void); diff --git a/migration/savevm.c b/migration/savevm.c index 4046faf009..0b71e988ba 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -37,6 +37,7 @@ #include "migration/register.h" #include "migration/global_state.h" #include "migration/channel-block.h" +#include "multifd.h" #include "ram.h" #include "qemu-file.h" #include "savevm.h" @@ -90,6 +91,7 @@ enum qemu_vm_cmd { MIG_CMD_ENABLE_COLO, /* Enable COLO */ MIG_CMD_POSTCOPY_RESUME, /* resume postcopy on dest */ MIG_CMD_RECV_BITMAP, /* Request for recved bitmap on dst */ + MIG_CMD_MULTIFD_RECV_SYNC, /* Sync multifd recv_x and main threads */ MIG_CMD_MAX }; @@ -109,6 +111,7 @@ static struct mig_cmd_args { [MIG_CMD_POSTCOPY_RESUME] = { .len = 0, .name = "POSTCOPY_RESUME" }, [MIG_CMD_PACKAGED] = { .len = 4, .name = "PACKAGED" }, [MIG_CMD_RECV_BITMAP] = { .len = -1, .name = "RECV_BITMAP" }, + [MIG_CMD_MULTIFD_RECV_SYNC] = { .len = 0, .name = "MULTIFD_RECV_SYNC" }, [MIG_CMD_MAX] = { .len = -1, .name = "MAX" }, }; @@ -1201,6 +1204,12 @@ void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name) qemu_savevm_command_send(f, MIG_CMD_RECV_BITMAP, len + 1, (uint8_t *)buf); } +void qemu_savevm_send_multifd_recv_sync(QEMUFile *f) +{ + /* TBD: trace_savevm_send_multifd_recv_sync(); */ + qemu_savevm_command_send(f, MIG_CMD_MULTIFD_RECV_SYNC, 0, NULL); +} + bool qemu_savevm_state_blocked(Error **errp) { SaveStateEntry *se; @@ -2479,6 +2488,10 @@ static int loadvm_process_command(QEMUFile *f) case MIG_CMD_RECV_BITMAP: return loadvm_handle_recv_bitmap(mis, len); + case MIG_CMD_MULTIFD_RECV_SYNC: + multifd_recv_sync_main(); + break; + case MIG_CMD_ENABLE_COLO: return loadvm_process_enable_colo(mis); } diff --git a/migration/savevm.h b/migration/savevm.h index 7957460062..91ae703925 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -53,6 +53,7 @@ void qemu_savevm_send_postcopy_listen(QEMUFile *f); void qemu_savevm_send_postcopy_run(QEMUFile *f); void qemu_savevm_send_postcopy_resume(QEMUFile *f); void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name); +void qemu_savevm_send_multifd_recv_sync(QEMUFile *f); void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name, uint16_t len,