From patchwork Mon Oct 3 03:15:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9B88C433F5 for ; Mon, 3 Oct 2022 03:18:00 +0000 (UTC) Received: from localhost ([::1]:60222 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofBxv-0007X6-Ov for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:17:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58424) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwG-0004m4-CQ for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53339) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwE-0007VF-A9 for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WHTJSU5p2ydUkpzgW9zTbZ4QQbzhLEXPeT5VNILWTYQ=; b=gxptliZLRWifEK10JpRqGQhLb7nZwsISjgZUBAR00cL5l+viPmh7ZUmRAZ2Wnv5xkZPrKh TEtxCmrCDu6Exy577Z/Lnu0rV7bEy9jdY3vaLgok42H9P/Hr1dVqG+1arV0GSZkdO7iL+g Z/YKvAL5ZcmABi5qxd5E9/9lSnkAg+U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-614-7IqO6F2RPLWjbWAtVwuECA-1; Sun, 02 Oct 2022 23:16:09 -0400 X-MC-Unique: 7IqO6F2RPLWjbWAtVwuECA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2F84D101A528; Mon, 3 Oct 2022 03:16:09 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id DBFC640C206B; Mon, 3 Oct 2022 03:16:05 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 1/7] migration: Remove res_compatible parameter Date: Mon, 3 Oct 2022 05:15:54 +0200 Message-Id: <20221003031600.20084-2-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" It was only used for RAM, and in that case, it means that this amount of data was sent for memory. Just delete the field in all callers. Signed-off-by: Juan Quintela --- include/migration/register.h | 20 ++++++++++---------- migration/savevm.h | 4 +--- hw/s390x/s390-stattrib.c | 6 ++---- hw/vfio/migration.c | 10 ++++------ migration/block-dirty-bitmap.c | 7 +++---- migration/block.c | 7 +++---- migration/migration.c | 9 ++++----- migration/ram.c | 8 +++----- migration/savevm.c | 14 +++++--------- hw/vfio/trace-events | 2 +- migration/trace-events | 2 +- 11 files changed, 37 insertions(+), 52 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index c1dcff0f90..1950fee6a8 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -48,18 +48,18 @@ typedef struct SaveVMHandlers { int (*save_setup)(QEMUFile *f, void *opaque); void (*save_live_pending)(QEMUFile *f, void *opaque, uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only); + uint64_t *rest_precopy, + uint64_t *rest_postcopy); /* Note for save_live_pending: - * - res_precopy_only is for data which must be migrated in precopy phase - * or in stopped state, in other words - before target vm start - * - res_compatible is for data which may be migrated in any phase - * - res_postcopy_only is for data which must be migrated in postcopy phase - * or in stopped state, in other words - after source vm stop + * - res_precopy is for data which must be migrated in precopy + * phase or in stopped state, in other words - before target + * vm start + * - res_postcopy is for data which must be migrated in postcopy + * phase or in stopped state, in other words - after source vm + * stop * - * Sum of res_postcopy_only, res_compatible and res_postcopy_only is the - * whole amount of pending data. + * Sum of res_precopy and res_postcopy is the whole amount of + * pending data. */ diff --git a/migration/savevm.h b/migration/savevm.h index 6461342cb4..9bd55c336c 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -41,9 +41,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only); + uint64_t *res_precopy, uint64_t *res_postcopy); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index 9eda1c3b2a..ee60b53da4 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -183,16 +183,14 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) } static void cmma_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, uint64_t *res_postcopy) { S390StAttribState *sas = S390_STATTRIB(opaque); S390StAttribClass *sac = S390_STATTRIB_GET_CLASS(sas); long long res = sac->get_dirtycount(sas); if (res >= 0) { - *res_precopy_only += res; + *res_precopy += res; } } diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3de4252111..3423f113f0 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -458,9 +458,8 @@ static void vfio_save_cleanup(void *opaque) static void vfio_save_pending(QEMUFile *f, void *opaque, uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -471,10 +470,9 @@ static void vfio_save_pending(QEMUFile *f, void *opaque, return; } - *res_precopy_only += migration->pending_bytes; + *res_precopy += migration->pending_bytes; - trace_vfio_save_pending(vbasedev->name, *res_precopy_only, - *res_postcopy_only, *res_compatible); + trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); } static int vfio_save_iterate(QEMUFile *f, void *opaque) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 9aba7d9c22..dfea546330 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -763,9 +763,8 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { DBMSaveState *s = &((DBMState *)opaque)->save; SaveBitmapState *dbms; @@ -785,7 +784,7 @@ static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, trace_dirty_bitmap_save_pending(pending, max_size); - *res_postcopy_only += pending; + *res_postcopy += pending; } /* First occurrence of this bitmap. It should be created if doesn't exist */ diff --git a/migration/block.c b/migration/block.c index 3577c815a9..4ae8f837b0 100644 --- a/migration/block.c +++ b/migration/block.c @@ -863,9 +863,8 @@ static int block_save_complete(QEMUFile *f, void *opaque) } static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { /* Estimate pending number of bytes to send */ uint64_t pending; @@ -886,7 +885,7 @@ static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, trace_migration_block_save_pending(pending); /* We don't do postcopy */ - *res_precopy_only += pending; + *res_precopy += pending; } static int block_load(QEMUFile *f, void *opaque, int version_id) diff --git a/migration/migration.c b/migration/migration.c index bb8bbddfe4..440aa62f16 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3734,15 +3734,14 @@ typedef enum { */ static MigIterateState migration_iteration_run(MigrationState *s) { - uint64_t pending_size, pend_pre, pend_compat, pend_post; + uint64_t pending_size, pend_pre, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, - &pend_compat, &pend_post); - pending_size = pend_pre + pend_compat + pend_post; + &pend_post); + pending_size = pend_pre + pend_post; - trace_migrate_pending(pending_size, s->threshold_size, - pend_pre, pend_compat, pend_post); + trace_migrate_pending(pending_size, s->threshold_size, pend_pre, pend_post); if (pending_size && pending_size >= s->threshold_size) { /* Still a significant amount to transfer */ diff --git a/migration/ram.c b/migration/ram.c index dc1de9ddbc..20167e1102 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3435,9 +3435,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, uint64_t *res_postcopy) { RAMState **temp = opaque; RAMState *rs = *temp; @@ -3457,9 +3455,9 @@ static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, if (migrate_postcopy_ram()) { /* We can do postcopy, and all the data is postcopiable */ - *res_compatible += remaining_size; + *res_postcopy += remaining_size; } else { - *res_precopy_only += remaining_size; + *res_precopy += remaining_size; } } diff --git a/migration/savevm.c b/migration/savevm.c index 48e85c052c..a752ff9ea1 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1472,16 +1472,13 @@ flush: * for units that can't do postcopy. */ void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { SaveStateEntry *se; - *res_precopy_only = 0; - *res_compatible = 0; - *res_postcopy_only = 0; - + *res_precopy = 0; + *res_postcopy = 0; QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || !se->ops->save_live_pending) { @@ -1493,8 +1490,7 @@ void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, } } se->ops->save_live_pending(f, se->opaque, threshold_size, - res_precopy_only, res_compatible, - res_postcopy_only); + res_precopy, res_postcopy); } } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 73dffe9e00..a21cbd2a56 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -157,7 +157,7 @@ vfio_save_cleanup(const char *name) " (%s)" vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_size, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%"PRIx64 vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%"PRIx64 vfio_save_device_config_state(const char *name) " (%s)" -vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy, uint64_t compatible) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64" compatible 0x%"PRIx64 +vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64 vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" diff --git a/migration/trace-events b/migration/trace-events index 57003edcbd..f2a873fd6c 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -150,7 +150,7 @@ migrate_fd_cleanup(void) "" migrate_fd_error(const char *error_desc) "error=%s" migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) "in %s at 0x%zx len 0x%zx" -migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t compat, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " compat=%" PRIu64 " post=%" PRIu64 ")" +migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " post=%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%"PRIi64 migration_completion_file_err(void) "" From patchwork Mon Oct 3 03:15:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18424C433FE for ; Mon, 3 Oct 2022 03:21:34 +0000 (UTC) Received: from localhost ([::1]:51406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofC1N-0005yK-3z for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:21:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46504) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwQ-0005Av-Lt for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:34211) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwO-0007Wa-UI for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aZgmHTCR9bxJcEmUoPOTtendDNFE6ukZny7bykvV0ic=; b=eyzw1TLGMJRsMiSfSAVQYYPnmEMkg7N6hdfbG42Y2TL2nyNd3nM0pUTCPN01eeqvPs2npZ 88Hw7IdnQ6OD8pqfJAfxt1qUlridm1fequhcbmDfVC8odHjDlIizgS7cXJOo3k1yDL5fkq ifaobsgFBsN3ccSn3VZj12lP1GMycck= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-347-vAoQrRoMOcGqESXH2wtIPQ-1; Sun, 02 Oct 2022 23:16:13 -0400 X-MC-Unique: vAoQrRoMOcGqESXH2wtIPQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C14B18027ED; Mon, 3 Oct 2022 03:16:12 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A16840C206B; Mon, 3 Oct 2022 03:16:09 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 2/7] migration: No save_live_pending() method uses the QEMUFile parameter Date: Mon, 3 Oct 2022 05:15:55 +0200 Message-Id: <20221003031600.20084-3-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" So remove it everywhere. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- include/migration/register.h | 6 ++---- migration/savevm.h | 2 +- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 6 ++---- migration/block-dirty-bitmap.c | 5 ++--- migration/block.c | 2 +- migration/migration.c | 3 +-- migration/ram.c | 2 +- migration/savevm.c | 5 ++--- 9 files changed, 13 insertions(+), 20 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index 1950fee6a8..5b5424ed8f 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -46,10 +46,8 @@ typedef struct SaveVMHandlers { /* This runs outside the iothread lock! */ int (*save_setup)(QEMUFile *f, void *opaque); - void (*save_live_pending)(QEMUFile *f, void *opaque, - uint64_t threshold_size, - uint64_t *rest_precopy, - uint64_t *rest_postcopy); + void (*save_live_pending)(void *opaque, uint64_t threshold_size, + uint64_t *rest_precopy, uint64_t *rest_postcopy); /* Note for save_live_pending: * - res_precopy is for data which must be migrated in precopy * phase or in stopped state, in other words - before target diff --git a/migration/savevm.h b/migration/savevm.h index 9bd55c336c..98fae6f9b3 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,7 +40,7 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, +void qemu_savevm_state_pending(uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index ee60b53da4..9b74eeadf3 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -182,7 +182,7 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) return 0; } -static void cmma_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void cmma_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { S390StAttribState *sas = S390_STATTRIB(opaque); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3423f113f0..760d5f3c5c 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -456,10 +456,8 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_save_pending(QEMUFile *f, void *opaque, - uint64_t threshold_size, - uint64_t *res_precopy, - uint64_t *res_postcopy) +static void vfio_save_pending(void *opaque, uint64_t threshold_size, + uint64_t *res_precopy, uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index dfea546330..a445bdc3c3 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -761,9 +761,8 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) return 0; } -static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, - uint64_t max_size, - uint64_t *res_precopy, +static void dirty_bitmap_save_pending(void *opaque, uint64_t max_size, + uint64_t *res_precopy, uint64_t *res_postcopy) { DBMSaveState *s = &((DBMState *)opaque)->save; diff --git a/migration/block.c b/migration/block.c index 4ae8f837b0..b3d680af75 100644 --- a/migration/block.c +++ b/migration/block.c @@ -862,7 +862,7 @@ static int block_save_complete(QEMUFile *f, void *opaque) return 0; } -static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void block_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { diff --git a/migration/migration.c b/migration/migration.c index 440aa62f16..038fc58a96 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3737,8 +3737,7 @@ static MigIterateState migration_iteration_run(MigrationState *s) uint64_t pending_size, pend_pre, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; - qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, - &pend_post); + qemu_savevm_state_pending(s->threshold_size, &pend_pre, &pend_post); pending_size = pend_pre + pend_post; trace_migrate_pending(pending_size, s->threshold_size, pend_pre, pend_post); diff --git a/migration/ram.c b/migration/ram.c index 20167e1102..48a31b87c8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3434,7 +3434,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) return 0; } -static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void ram_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { RAMState **temp = opaque; diff --git a/migration/savevm.c b/migration/savevm.c index a752ff9ea1..d937ab0b2e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1471,8 +1471,7 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, - uint64_t *res_precopy, +void qemu_savevm_state_pending(uint64_t threshold_size, uint64_t *res_precopy, uint64_t *res_postcopy) { SaveStateEntry *se; @@ -1489,7 +1488,7 @@ void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, continue; } } - se->ops->save_live_pending(f, se->opaque, threshold_size, + se->ops->save_live_pending(se->opaque, threshold_size, res_precopy, res_postcopy); } } From patchwork Mon Oct 3 03:15:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD8CCC433FE for ; Mon, 3 Oct 2022 03:19:46 +0000 (UTC) Received: from localhost ([::1]:49186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofBzd-00030I-Pz for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:19:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwO-00053h-Dz for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49149) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwL-0007W4-P8 for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C8b+n7qI3SJroMtAL0rndE14xTRYBp9eMbtC+fFP/Aw=; b=D1TSkjoHnVWEGxllKE9y+UDEDmqASuCDm4dPYQrnj3+RDIXqnc2GRTnrA2VeSZuAERqmVr q5fNkFZ/EzyYE6I5E1XGQrD/h+H0KaQvhJQCbA7PzNOKFYa62eiWfjdZuvqKMitDpPL0FR tBMp+dOQEUXfILu5Mv9rLgEYotBgryk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-674-UfSUNzFCMhySYlSEtWPhZQ-1; Sun, 02 Oct 2022 23:16:16 -0400 X-MC-Unique: UfSUNzFCMhySYlSEtWPhZQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3AD8987B2A7; Mon, 3 Oct 2022 03:16:16 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 17B5140C206B; Mon, 3 Oct 2022 03:16:12 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 3/7] migration: Block migration comment or code is wrong Date: Mon, 3 Oct 2022 05:15:56 +0200 Message-Id: <20221003031600.20084-4-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" And it appears that what is wrong is the code. During bulk stage we need to make sure that some block is dirty, but no games with max_size at all. Signed-off-by: Juan Quintela Reviewed-by: Stefan Hajnoczi --- migration/block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/block.c b/migration/block.c index b3d680af75..39ce4003c6 100644 --- a/migration/block.c +++ b/migration/block.c @@ -879,8 +879,8 @@ static void block_save_pending(void *opaque, uint64_t max_size, blk_mig_unlock(); /* Report at least one block pending during bulk phase */ - if (pending <= max_size && !block_mig_state.bulk_completed) { - pending = max_size + BLK_MIG_BLOCK_SIZE; + if (!pending && !block_mig_state.bulk_completed) { + pending = BLK_MIG_BLOCK_SIZE; } trace_migration_block_save_pending(pending); From patchwork Mon Oct 3 03:15:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80B3BC433FE for ; Mon, 3 Oct 2022 03:19:53 +0000 (UTC) Received: from localhost ([::1]:39624 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofBzk-0003HJ-M3 for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:19:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46516) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwT-0005KZ-LR for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24465) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwR-0007XA-2M for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Oyv5/kE5hDxtUHpYrXHo8QcScwE8+qo3olvNiQkGJJ0=; b=LBU86meo6Q5y8o67Z7pEX2vhxQL+oAcbx67oX7e1y5+Tvr8Ouo5vUVpMTHwuZvnf3fWh/b soVPS9cykQ3Pa5O9/wjvexuimTJMQ5AacifOwjvQj6cafjzoGRVdaqHu5pfe25+4tcJii6 wY4X59vZEbdbJ6pT6S/KRbqItqWgdsk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-451-FGDp8GLWPAmM3rUxVCsa8A-1; Sun, 02 Oct 2022 23:16:20 -0400 X-MC-Unique: FGDp8GLWPAmM3rUxVCsa8A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C78A7101A528; Mon, 3 Oct 2022 03:16:19 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 80EB040C206B; Mon, 3 Oct 2022 03:16:16 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 4/7] migration: Split save_live_pending() into state_pending_* Date: Mon, 3 Oct 2022 05:15:57 +0200 Message-Id: <20221003031600.20084-5-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" We split the function into to: - state_pending_estimate: We estimate the remaining state size without stopping the machine. - state pending_exact: We calculate the exact amount of remaining state. The only "device" that implements different functions for _estimate() and _exact() is ram. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- docs/devel/migration.rst | 18 ++++++++++-------- docs/devel/vfio-migration.rst | 4 ++-- include/migration/register.h | 12 ++++++++---- migration/savevm.h | 8 ++++++-- hw/s390x/s390-stattrib.c | 7 ++++--- hw/vfio/migration.c | 9 +++++---- migration/block-dirty-bitmap.c | 11 ++++++----- migration/block.c | 11 ++++++----- migration/migration.c | 13 +++++++++---- migration/ram.c | 31 ++++++++++++++++++++++++------- migration/savevm.c | 34 +++++++++++++++++++++++++++++----- hw/vfio/trace-events | 2 +- migration/trace-events | 7 ++++--- 13 files changed, 114 insertions(+), 53 deletions(-) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index 3e9656d8e0..6f65c23b47 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -482,15 +482,17 @@ An iterative device must provide: - A ``load_setup`` function that initialises the data structures on the destination. - - A ``save_live_pending`` function that is called repeatedly and must - indicate how much more data the iterative data must save. The core - migration code will use this to determine when to pause the CPUs - and complete the migration. + - A ``state_pending_exact`` function that indicates how much more + data we must save. The core migration code will use this to + determine when to pause the CPUs and complete the migration. - - A ``save_live_iterate`` function (called after ``save_live_pending`` - when there is significant data still to be sent). It should send - a chunk of data until the point that stream bandwidth limits tell it - to stop. Each call generates one section. + - A ``state_pending_estimate`` function that indicates how much more + data we must save. When the estimated amount is smaller than the + threshold, we call ``state_pending_exact``. + + - A ``save_live_iterate`` function should send a chunk of data until + the point that stream bandwidth limits tell it to stop. Each call + generates one section. - A ``save_live_complete_precopy`` function that must transmit the last section for the device containing any remaining data. diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index 9ff6163c88..673057c90d 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -28,7 +28,7 @@ VFIO implements the device hooks for the iterative approach as follows: * A ``load_setup`` function that sets up the migration region on the destination and sets _RESUMING flag in the VFIO device state. -* A ``save_live_pending`` function that reads pending_bytes from the vendor +* A ``state_pending_exact`` function that reads pending_bytes from the vendor driver, which indicates the amount of data that the vendor driver has yet to save for the VFIO device. @@ -114,7 +114,7 @@ Live migration save path (RUNNING, _SETUP, _RUNNING|_SAVING) | (RUNNING, _ACTIVE, _RUNNING|_SAVING) - If device is active, get pending_bytes by .save_live_pending() + If device is active, get pending_bytes by .state_pending_exact() If total pending_bytes >= threshold_size, call .save_live_iterate() Data of VFIO device for pre-copy phase is copied Iterate till total pending bytes converge and are less than threshold diff --git a/include/migration/register.h b/include/migration/register.h index 5b5424ed8f..313b8e1c3b 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -46,9 +46,7 @@ typedef struct SaveVMHandlers { /* This runs outside the iothread lock! */ int (*save_setup)(QEMUFile *f, void *opaque); - void (*save_live_pending)(void *opaque, uint64_t threshold_size, - uint64_t *rest_precopy, uint64_t *rest_postcopy); - /* Note for save_live_pending: + /* Note for state_pending_*: * - res_precopy is for data which must be migrated in precopy * phase or in stopped state, in other words - before target * vm start @@ -59,7 +57,13 @@ typedef struct SaveVMHandlers { * Sum of res_precopy and res_postcopy is the whole amount of * pending data. */ - + /* This calculate the exact remaining data to transfer */ + void (*state_pending_exact)(void *opaque, uint64_t threshold_size, + uint64_t *rest_precopy, uint64_t *rest_postcopy); + /* This estimates the remaining data to transfer */ + void (*state_pending_estimate)(void *opaque, uint64_t threshold_size, + uint64_t *rest_precopy, + uint64_t *rest_postcopy); LoadStateHandler *load_state; int (*load_setup)(QEMUFile *f, void *opaque); diff --git a/migration/savevm.h b/migration/savevm.h index 98fae6f9b3..613f85e717 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,8 +40,12 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending(uint64_t max_size, - uint64_t *res_precopy, uint64_t *res_postcopy); +void qemu_savevm_state_pending_exact(uint64_t max_size, + uint64_t *res_precopy, + uint64_t *res_postcopy); +void qemu_savevm_state_pending_estimate(uint64_t max_size, + uint64_t *res_precopy, + uint64_t *res_postcopy); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index 9b74eeadf3..dfb95eb20c 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -182,8 +182,8 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) return 0; } -static void cmma_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy, uint64_t *res_postcopy) +static void cmma_state_pending(void *opaque, uint64_t *res_precopy, + uint64_t *res_postcopy) { S390StAttribState *sas = S390_STATTRIB(opaque); S390StAttribClass *sac = S390_STATTRIB_GET_CLASS(sas); @@ -369,7 +369,8 @@ static SaveVMHandlers savevm_s390_stattrib_handlers = { .save_setup = cmma_save_setup, .save_live_iterate = cmma_save_iterate, .save_live_complete_precopy = cmma_save_complete, - .save_live_pending = cmma_save_pending, + .state_pending_exact = cmma_state_pending, + .state_pending_estimate = cmma_state_pending, .save_cleanup = cmma_save_cleanup, .load_state = cmma_load, .is_active = cmma_active, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 760d5f3c5c..680cf4df6e 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -456,8 +456,8 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_save_pending(void *opaque, uint64_t threshold_size, - uint64_t *res_precopy, uint64_t *res_postcopy) +static void vfio_state_pending(void *opaque, uint64_t threshold_size, + uint64_t *res_precopy, uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -470,7 +470,7 @@ static void vfio_save_pending(void *opaque, uint64_t threshold_size, *res_precopy += migration->pending_bytes; - trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); + trace_vfio_state_pending(vbasedev->name, *res_precopy, *res_postcopy); } static int vfio_save_iterate(QEMUFile *f, void *opaque) @@ -681,7 +681,8 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) static SaveVMHandlers savevm_vfio_handlers = { .save_setup = vfio_save_setup, .save_cleanup = vfio_save_cleanup, - .save_live_pending = vfio_save_pending, + .state_pending_exact = vfio_state_pending, + .state_pending_estimate = vfio_state_pending, .save_live_iterate = vfio_save_iterate, .save_live_complete_precopy = vfio_save_complete_precopy, .save_state = vfio_save_state, diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index a445bdc3c3..5b24007650 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -761,9 +761,9 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) return 0; } -static void dirty_bitmap_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy, - uint64_t *res_postcopy) +static void dirty_bitmap_state_pending(void *opaque, uint64_t max_size, + uint64_t *res_precopy, + uint64_t *res_postcopy) { DBMSaveState *s = &((DBMState *)opaque)->save; SaveBitmapState *dbms; @@ -781,7 +781,7 @@ static void dirty_bitmap_save_pending(void *opaque, uint64_t max_size, qemu_mutex_unlock_iothread(); - trace_dirty_bitmap_save_pending(pending, max_size); + trace_dirty_bitmap_state_pending(pending); *res_postcopy += pending; } @@ -1250,7 +1250,8 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers = { .save_live_complete_postcopy = dirty_bitmap_save_complete, .save_live_complete_precopy = dirty_bitmap_save_complete, .has_postcopy = dirty_bitmap_has_postcopy, - .save_live_pending = dirty_bitmap_save_pending, + .state_pending_exact = dirty_bitmap_state_pending, + .state_pending_estimate = dirty_bitmap_state_pending, .save_live_iterate = dirty_bitmap_save_iterate, .is_active_iterate = dirty_bitmap_is_active_iterate, .load_state = dirty_bitmap_load, diff --git a/migration/block.c b/migration/block.c index 39ce4003c6..8e6ad1c468 100644 --- a/migration/block.c +++ b/migration/block.c @@ -862,9 +862,9 @@ static int block_save_complete(QEMUFile *f, void *opaque) return 0; } -static void block_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy, - uint64_t *res_postcopy) +static void block_state_pending(void *opaque, uint64_t max_size, + uint64_t *res_precopy, + uint64_t *res_postcopy) { /* Estimate pending number of bytes to send */ uint64_t pending; @@ -883,7 +883,7 @@ static void block_save_pending(void *opaque, uint64_t max_size, pending = BLK_MIG_BLOCK_SIZE; } - trace_migration_block_save_pending(pending); + trace_migration_block_state_pending(pending); /* We don't do postcopy */ *res_precopy += pending; } @@ -1018,7 +1018,8 @@ static SaveVMHandlers savevm_block_handlers = { .save_setup = block_save_setup, .save_live_iterate = block_save_iterate, .save_live_complete_precopy = block_save_complete, - .save_live_pending = block_save_pending, + .state_pending_exact = block_state_pending, + .state_pending_estimate = block_state_pending, .load_state = block_load, .save_cleanup = block_migration_cleanup, .is_active = block_is_active, diff --git a/migration/migration.c b/migration/migration.c index 038fc58a96..4676568699 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3734,13 +3734,18 @@ typedef enum { */ static MigIterateState migration_iteration_run(MigrationState *s) { - uint64_t pending_size, pend_pre, pend_post; + uint64_t pend_pre, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; - qemu_savevm_state_pending(s->threshold_size, &pend_pre, &pend_post); - pending_size = pend_pre + pend_post; + qemu_savevm_state_pending_estimate(s->threshold_size, &pend_pre, &pend_post); + uint64_t pending_size = pend_pre + pend_post; + trace_migrate_pending_estimate(pending_size, s->threshold_size, pend_pre, pend_post); - trace_migrate_pending(pending_size, s->threshold_size, pend_pre, pend_post); + if (pend_pre <= s->threshold_size) { + qemu_savevm_state_pending_exact(s->threshold_size, &pend_pre, &pend_post); + pending_size = pend_pre + pend_post; + trace_migrate_pending_exact(pending_size, s->threshold_size, pend_pre, pend_post); + } if (pending_size && pending_size >= s->threshold_size) { /* Still a significant amount to transfer */ diff --git a/migration/ram.c b/migration/ram.c index 48a31b87c8..8d989d51db 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3434,17 +3434,33 @@ static int ram_save_complete(QEMUFile *f, void *opaque) return 0; } -static void ram_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy, uint64_t *res_postcopy) +static void ram_state_pending_estimate(void *opaque, uint64_t max_size, + uint64_t *res_precopy, + uint64_t *res_postcopy) { RAMState **temp = opaque; RAMState *rs = *temp; - uint64_t remaining_size; - remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE; + uint64_t remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE; - if (!migration_in_postcopy() && - remaining_size < max_size) { + if (migrate_postcopy_ram()) { + /* We can do postcopy, and all the data is postcopiable */ + *res_postcopy += remaining_size; + } else { + *res_precopy += remaining_size; + } +} + +static void ram_state_pending_exact(void *opaque, uint64_t max_size, + uint64_t *res_precopy, + uint64_t *res_postcopy) +{ + RAMState **temp = opaque; + RAMState *rs = *temp; + + uint64_t remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE; + + if (!migration_in_postcopy()) { qemu_mutex_lock_iothread(); WITH_RCU_READ_LOCK_GUARD() { migration_bitmap_sync_precopy(rs); @@ -4600,7 +4616,8 @@ static SaveVMHandlers savevm_ram_handlers = { .save_live_complete_postcopy = ram_save_complete, .save_live_complete_precopy = ram_save_complete, .has_postcopy = ram_has_postcopy, - .save_live_pending = ram_save_pending, + .state_pending_exact = ram_state_pending_exact, + .state_pending_estimate = ram_state_pending_estimate, .load_state = ram_load, .save_cleanup = ram_save_cleanup, .load_setup = ram_load_setup, diff --git a/migration/savevm.c b/migration/savevm.c index d937ab0b2e..976ece3f3f 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1471,8 +1471,9 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending(uint64_t threshold_size, uint64_t *res_precopy, - uint64_t *res_postcopy) +void qemu_savevm_state_pending_exact(uint64_t threshold_size, + uint64_t *res_precopy, + uint64_t *res_postcopy) { SaveStateEntry *se; @@ -1480,7 +1481,7 @@ void qemu_savevm_state_pending(uint64_t threshold_size, uint64_t *res_precopy, *res_postcopy = 0; QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->save_live_pending) { + if (!se->ops || !se->ops->state_pending_exact) { continue; } if (se->ops->is_active) { @@ -1488,8 +1489,31 @@ void qemu_savevm_state_pending(uint64_t threshold_size, uint64_t *res_precopy, continue; } } - se->ops->save_live_pending(se->opaque, threshold_size, - res_precopy, res_postcopy); + se->ops->state_pending_exact(se->opaque, threshold_size, + res_precopy, res_postcopy); + } +} + +void qemu_savevm_state_pending_estimate(uint64_t threshold_size, + uint64_t *res_precopy, + uint64_t *res_postcopy) +{ + SaveStateEntry *se; + + *res_precopy = 0; + *res_postcopy = 0; + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (!se->ops || !se->ops->state_pending_estimate) { + continue; + } + if (se->ops->is_active) { + if (!se->ops->is_active(se->opaque)) { + continue; + } + } + se->ops->state_pending_estimate(se->opaque, threshold_size, + res_precopy, res_postcopy); } } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index a21cbd2a56..90a8aecb37 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -157,7 +157,7 @@ vfio_save_cleanup(const char *name) " (%s)" vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_size, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%"PRIx64 vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%"PRIx64 vfio_save_device_config_state(const char *name) " (%s)" -vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64 +vfio_state_pending(const char *name, uint64_t precopy, uint64_t postcopy) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64 vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" diff --git a/migration/trace-events b/migration/trace-events index f2a873fd6c..84352f310a 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -150,7 +150,8 @@ migrate_fd_cleanup(void) "" migrate_fd_error(const char *error_desc) "error=%s" migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) "in %s at 0x%zx len 0x%zx" -migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " post=%" PRIu64 ")" +migrate_pending_estimate(uint64_t size, uint64_t max, uint64_t pre, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " post=%" PRIu64 ")" +migrate_pending_exact(uint64_t size, uint64_t max, uint64_t pre, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " post=%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%"PRIi64 migration_completion_file_err(void) "" @@ -330,7 +331,7 @@ send_bitmap_bits(uint32_t flags, uint64_t start_sector, uint32_t nr_sectors, uin dirty_bitmap_save_iterate(int in_postcopy) "in postcopy: %d" dirty_bitmap_save_complete_enter(void) "" dirty_bitmap_save_complete_finish(void) "" -dirty_bitmap_save_pending(uint64_t pending, uint64_t max_size) "pending %" PRIu64 " max: %" PRIu64 +dirty_bitmap_state_pending(uint64_t pending) "pending %" PRIu64 dirty_bitmap_load_complete(void) "" dirty_bitmap_load_bits_enter(uint64_t first_sector, uint32_t nr_sectors) "chunk: %" PRIu64 " %" PRIu32 dirty_bitmap_load_bits_zeroes(void) "" @@ -355,7 +356,7 @@ migration_block_save_device_dirty(int64_t sector) "Error reading sector %" PRId6 migration_block_flush_blks(const char *action, int submitted, int read_done, int transferred) "%s submitted %d read_done %d transferred %d" migration_block_save(const char *mig_stage, int submitted, int transferred) "Enter save live %s submitted %d transferred %d" migration_block_save_complete(void) "Block migration completed" -migration_block_save_pending(uint64_t pending) "Enter save live pending %" PRIu64 +migration_block_state_pending(uint64_t pending) "Enter save live pending %" PRIu64 # page_cache.c migration_pagecache_init(int64_t max_num_items) "Setting cache buckets to %" PRId64 From patchwork Mon Oct 3 03:15:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2ADD0C433F5 for ; Mon, 3 Oct 2022 03:18:23 +0000 (UTC) Received: from localhost ([::1]:56704 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofByI-0008PD-80 for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:18:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwW-0005Xf-D8 for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:46067) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwU-0007Xv-H1 for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TuQYppfjMjYmOBpb1/3mlSFSjtXfiaf9/CdgUKmVikg=; b=QDwiXWdA4ZQTdjOpB8khUgDL5jqiSwG+8Uj6whj15bZq0ACGoh6ZY4/xlFvDEbiKRiGIfe icGX7sAwN0N8oXR7JMIEHRtnAsPEfASjhBEYK3Hg6teMOOg5b7nAx1loEtzmE1Li7QRw3Q 7G9RaR41WQM65osWb7esAPKCjRoUEVE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-184-z0luv5uHNQu67pQQHGHXGQ-1; Sun, 02 Oct 2022 23:16:23 -0400 X-MC-Unique: z0luv5uHNQu67pQQHGHXGQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 44E42185A79C; Mon, 3 Oct 2022 03:16:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1DEBD40C206B; Mon, 3 Oct 2022 03:16:19 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 5/7] migration: Remove unused threshold_size parameter Date: Mon, 3 Oct 2022 05:15:58 +0200 Message-Id: <20221003031600.20084-6-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Until previous commit, save_live_pending() was used for ram. Now with the split into state_pending_estimate() and state_pending_exact() it is not needed anymore, so remove them. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- include/migration/register.h | 7 +++---- migration/savevm.h | 6 ++---- hw/vfio/migration.c | 6 +++--- migration/block-dirty-bitmap.c | 3 +-- migration/block.c | 3 +-- migration/migration.c | 4 ++-- migration/ram.c | 6 ++---- migration/savevm.c | 12 ++++-------- 8 files changed, 18 insertions(+), 29 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index 313b8e1c3b..5f08204fb2 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -58,11 +58,10 @@ typedef struct SaveVMHandlers { * pending data. */ /* This calculate the exact remaining data to transfer */ - void (*state_pending_exact)(void *opaque, uint64_t threshold_size, - uint64_t *rest_precopy, uint64_t *rest_postcopy); + void (*state_pending_exact)(void *opaque, uint64_t *rest_precopy, + uint64_t *rest_postcopy); /* This estimates the remaining data to transfer */ - void (*state_pending_estimate)(void *opaque, uint64_t threshold_size, - uint64_t *rest_precopy, + void (*state_pending_estimate)(void *opaque, uint64_t *rest_precopy, uint64_t *rest_postcopy); LoadStateHandler *load_state; diff --git a/migration/savevm.h b/migration/savevm.h index 613f85e717..24f2d2a28b 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,11 +40,9 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending_exact(uint64_t max_size, - uint64_t *res_precopy, +void qemu_savevm_state_pending_exact(uint64_t *res_precopy, uint64_t *res_postcopy); -void qemu_savevm_state_pending_estimate(uint64_t max_size, - uint64_t *res_precopy, +void qemu_savevm_state_pending_estimate(uint64_t *res_precopy, uint64_t *res_postcopy); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 680cf4df6e..d9598ce070 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -456,8 +456,8 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_state_pending(void *opaque, uint64_t threshold_size, - uint64_t *res_precopy, uint64_t *res_postcopy) +static void vfio_state_pending(void *opaque, uint64_t *res_precopy, + uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -511,7 +511,7 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) } /* - * Reset pending_bytes as .save_live_pending is not called during savevm or + * Reset pending_bytes as .state_pending_* is not called during savevm or * snapshot case, in such case vfio_update_pending() at the start of this * function updates pending_bytes. */ diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 5b24007650..8a11577252 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -761,8 +761,7 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) return 0; } -static void dirty_bitmap_state_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy, +static void dirty_bitmap_state_pending(void *opaque, uint64_t *res_precopy, uint64_t *res_postcopy) { DBMSaveState *s = &((DBMState *)opaque)->save; diff --git a/migration/block.c b/migration/block.c index 8e6ad1c468..4f1f7c0f8d 100644 --- a/migration/block.c +++ b/migration/block.c @@ -862,8 +862,7 @@ static int block_save_complete(QEMUFile *f, void *opaque) return 0; } -static void block_state_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy, +static void block_state_pending(void *opaque, uint64_t *res_precopy, uint64_t *res_postcopy) { /* Estimate pending number of bytes to send */ diff --git a/migration/migration.c b/migration/migration.c index 4676568699..97fefd579e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3737,12 +3737,12 @@ static MigIterateState migration_iteration_run(MigrationState *s) uint64_t pend_pre, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; - qemu_savevm_state_pending_estimate(s->threshold_size, &pend_pre, &pend_post); + qemu_savevm_state_pending_estimate(&pend_pre, &pend_post); uint64_t pending_size = pend_pre + pend_post; trace_migrate_pending_estimate(pending_size, s->threshold_size, pend_pre, pend_post); if (pend_pre <= s->threshold_size) { - qemu_savevm_state_pending_exact(s->threshold_size, &pend_pre, &pend_post); + qemu_savevm_state_pending_exact(&pend_pre, &pend_post); pending_size = pend_pre + pend_post; trace_migrate_pending_exact(pending_size, s->threshold_size, pend_pre, pend_post); } diff --git a/migration/ram.c b/migration/ram.c index 8d989d51db..53a5dd64a8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3434,8 +3434,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) return 0; } -static void ram_state_pending_estimate(void *opaque, uint64_t max_size, - uint64_t *res_precopy, +static void ram_state_pending_estimate(void *opaque, uint64_t *res_precopy, uint64_t *res_postcopy) { RAMState **temp = opaque; @@ -3451,8 +3450,7 @@ static void ram_state_pending_estimate(void *opaque, uint64_t max_size, } } -static void ram_state_pending_exact(void *opaque, uint64_t max_size, - uint64_t *res_precopy, +static void ram_state_pending_exact(void *opaque, uint64_t *res_precopy, uint64_t *res_postcopy) { RAMState **temp = opaque; diff --git a/migration/savevm.c b/migration/savevm.c index 976ece3f3f..1e0328e088 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1471,8 +1471,7 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending_exact(uint64_t threshold_size, - uint64_t *res_precopy, +void qemu_savevm_state_pending_exact(uint64_t *res_precopy, uint64_t *res_postcopy) { SaveStateEntry *se; @@ -1489,13 +1488,11 @@ void qemu_savevm_state_pending_exact(uint64_t threshold_size, continue; } } - se->ops->state_pending_exact(se->opaque, threshold_size, - res_precopy, res_postcopy); + se->ops->state_pending_exact(se->opaque, res_precopy, res_postcopy); } } -void qemu_savevm_state_pending_estimate(uint64_t threshold_size, - uint64_t *res_precopy, +void qemu_savevm_state_pending_estimate(uint64_t *res_precopy, uint64_t *res_postcopy) { SaveStateEntry *se; @@ -1512,8 +1509,7 @@ void qemu_savevm_state_pending_estimate(uint64_t threshold_size, continue; } } - se->ops->state_pending_estimate(se->opaque, threshold_size, - res_precopy, res_postcopy); + se->ops->state_pending_estimate(se->opaque, res_precopy, res_postcopy); } } From patchwork Mon Oct 3 03:15:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997028 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5B9AEC433F5 for ; Mon, 3 Oct 2022 03:21:44 +0000 (UTC) Received: from localhost ([::1]:45200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofC1X-0006H8-D0 for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:21:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46524) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwY-0005iC-Sa for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26229) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwX-0007YE-5z for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LdQoPIYFC95etXl0XLn4BlX5ux6lHKo/6jSURRB0+cI=; b=d2ch6cccAjvXWNDjuYWdvhe4lwPlvGdVlv7ro+28gKt3Q0fbpHQTITEfQnUOkgiW7AXiMc PcYl1tIyUP+uRLS8qDM1M2PeiwJbjorqBjvj2ppCVwFSD7qr31ao8QWwHrcT0RrHm2m56a ZlLG1Kf5VAihfP6BCPKOikNGYSD81KQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-615-OVt_AwmHPGaY-0OBCNkdoA-1; Sun, 02 Oct 2022 23:16:27 -0400 X-MC-Unique: OVt_AwmHPGaY-0OBCNkdoA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3D01829AA3AD; Mon, 3 Oct 2022 03:16:27 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8EFF340C206B; Mon, 3 Oct 2022 03:16:23 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 6/7] migration: simplify migration_iteration_run() Date: Mon, 3 Oct 2022 05:15:59 +0200 Message-Id: <20221003031600.20084-7-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 97fefd579e..35e512887a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3747,23 +3747,23 @@ static MigIterateState migration_iteration_run(MigrationState *s) trace_migrate_pending_exact(pending_size, s->threshold_size, pend_pre, pend_post); } - if (pending_size && pending_size >= s->threshold_size) { - /* Still a significant amount to transfer */ - if (!in_postcopy && pend_pre <= s->threshold_size && - qatomic_read(&s->start_postcopy)) { - if (postcopy_start(s)) { - error_report("%s: postcopy failed to start", __func__); - } - return MIG_ITERATE_SKIP; - } - /* Just another iteration step */ - qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); - } else { + if (pending_size < s->threshold_size) { trace_migration_thread_low_pending(pending_size); migration_completion(s); return MIG_ITERATE_BREAK; } + /* Still a significant amount to transfer */ + if (!in_postcopy && pend_pre <= s->threshold_size && + qatomic_read(&s->start_postcopy)) { + if (postcopy_start(s)) { + error_report("%s: postcopy failed to start", __func__); + } + return MIG_ITERATE_SKIP; + } + + /* Just another iteration step */ + qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); return MIG_ITERATE_RESUME; } From patchwork Mon Oct 3 03:16:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 12997029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E2EFC433F5 for ; Mon, 3 Oct 2022 03:23:27 +0000 (UTC) Received: from localhost ([::1]:51230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofC3C-0000e8-Ia for qemu-devel@archiver.kernel.org; Sun, 02 Oct 2022 23:23:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60438) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwd-00061N-GL for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:38940) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofBwb-0007Yv-VG for qemu-devel@nongnu.org; Sun, 02 Oct 2022 23:16:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664766996; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DSlfKfX7T5ftvQhXO6YGaAgDqum1wou6/eQjCrOiJa4=; b=gJYXwi9ZuyiE1IGNBcMedX19pJ5QCaFR4Ym1a3OgAA6ASf63DwQEEjXKAVoc+F+yoVV6wE iWbVgCyxvyWQx7zbX0q8hXuanv+7l2m6DRIPYL0WDyqKWB8Vohqxn7ePMAW0Wrqj775CUJ xJSXopgkyG+eX50OLJEQrPHtcshKo5M= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-219-a64570PfN1G7FGkV3o39pg-1; Sun, 02 Oct 2022 23:16:31 -0400 X-MC-Unique: a64570PfN1G7FGkV3o39pg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AA8AE85A583; Mon, 3 Oct 2022 03:16:30 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 875C940C206B; Mon, 3 Oct 2022 03:16:27 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Alex Williamson , Eric Blake , Stefan Hajnoczi , Fam Zheng , qemu-s390x@nongnu.org, Cornelia Huck , Thomas Huth , Vladimir Sementsov-Ogievskiy , Laurent Vivier , John Snow , "Dr. David Alan Gilbert" , Christian Borntraeger , Halil Pasic , Juan Quintela , Paolo Bonzini , qemu-block@nongnu.org, Eric Farman , Richard Henderson , David Hildenbrand Subject: [RFC 7/7] migration: call qemu_savevm_state_pending_exact() with the guest stopped Date: Mon, 3 Oct 2022 05:16:00 +0200 Message-Id: <20221003031600.20084-8-quintela@redhat.com> In-Reply-To: <20221003031600.20084-1-quintela@redhat.com> References: <20221003031600.20084-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" HACK ahead. There are devices that require the guest to be stopped to tell us what is the size of its state. So we need to stop the vm "before" we cal the functions. It is a hack because: - we are "starting" the guest again to stop it in migration_complete() I know, I know, but it is not trivial to get all the information easily to migration_complete(), so this hack. - auto_converge test fails with this hack. I think that it is related to previous problem. We start the guest when it is supposed to be stopped for convergence reasons. - All experiments that I did to do the proper thing failed with having the iothread_locked() or try to unlock() it when not locked. - several of the pending functions are using the iothread lock themselves, so I need to split it to have two versions (one for the _estimate() case with the iothread lock), and another for the _exact() case without the iothread_lock(). I want comments about this approach before I try to continue on this direction. Signed-off-by: Juan Quintela --- migration/migration.c | 13 +++++++++++++ tests/qtest/migration-test.c | 3 ++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 35e512887a..7374884818 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3742,7 +3742,20 @@ static MigIterateState migration_iteration_run(MigrationState *s) trace_migrate_pending_estimate(pending_size, s->threshold_size, pend_pre, pend_post); if (pend_pre <= s->threshold_size) { + int old_state = s->state; + qemu_mutex_lock_iothread(); + // is this really necessary? it works for me both ways. + qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL); + s->vm_was_running = runstate_is_running(); + vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); + qemu_mutex_unlock_iothread(); qemu_savevm_state_pending_exact(&pend_pre, &pend_post); + qemu_mutex_lock_iothread(); + runstate_set(old_state); + if (s->vm_was_running) { + vm_start(); + } + qemu_mutex_unlock_iothread(); pending_size = pend_pre + pend_post; trace_migrate_pending_exact(pending_size, s->threshold_size, pend_pre, pend_post); } diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 0d153d6b5e..0541a842ec 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2564,7 +2564,8 @@ int main(int argc, char **argv) qtest_add_func("/migration/validate_uuid_dst_not_set", test_validate_uuid_dst_not_set); - qtest_add_func("/migration/auto_converge", test_migrate_auto_converge); + if (0) + qtest_add_func("/migration/auto_converge", test_migrate_auto_converge); qtest_add_func("/migration/multifd/tcp/plain/none", test_multifd_tcp_none); qtest_add_func("/migration/multifd/tcp/plain/cancel",