[RFC,v2,3/3] migration: reduce time of loading non-iterable vmstate

Commit Message

Chuang Xu Dec. 12, 2022, 4:49 p.m. UTC

The duration of loading non-iterable vmstate accounts for a significant
portion of downtime (starting with the timestamp of source qemu stop and
ending with the timestamp of target qemu start). Most of the time is spent
committing memory region changes repeatedly.

This patch packs all the changes to memory region during the period of
loading non-iterable vmstate in a single memory transaction. With the
increase of devices, this patch will greatly improve the performance.

Here are the test results:
test vm info:
- 32 CPUs 128GB RAM
- 8 16-queue vhost-net device
- 16 4-queue vhost-user-blk device.

	time of loading non-iterable vmstate
before		about 210 ms
after		about 40 ms

Signed-off-by: Chuang Xu <[e-mail]>
---
 migration/savevm.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

Patch

diff --git a/migration/savevm.c b/migration/savevm.c
index a0cdb714f7..68a7a99b79 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2617,6 +2617,9 @@  int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis)
     uint8_t section_type;
     int ret = 0;
 
+    /* call memory_region_transaction_begin() before loading vmstate */
+    memory_region_transaction_begin();
+
 retry:
     while (true) {
         section_type = qemu_get_byte(f);
@@ -2684,6 +2687,16 @@  out:
             goto retry;
         }
     }
+
+    /*
+     * call memory_region_transaction_commit() after loading non-iterable
+     * vmstate, make sure the migration_enable_load_check_delay flag is
+     * true during commit.
+     */
+    migration_enable_load_check_delay = true;
+    memory_region_transaction_commit();
+    migration_enable_load_check_delay = false;
+
     return ret;
 }