[RFC,00/10] migration: auto-converge refinements for huge VM

Message

Yong Huang Sept. 9, 2024, 1:47 p.m. UTC

Currently, a huge VM with high memory overload may take a long time
to increase its maximum throttle percentage. The root cause is that
the current auto-converge throttle logic doesn't look like it will
scale because migration_trigger_throttle() is only called for each
iteration, so it won't be invoked for a long time if one iteration
can take a long time.

This patchset provides two refinements aiming at the above case.

1: The periodic CPU throttle. As Peter points out, "throttle only
   for each sync, sync for each iteration" may make sense in the
   old days, but perhaps not anymore. So we introduce perioidic
   CPU throttle implementation for migration, which is a trade-off
   between synchronization overhead and CPU throttle impact.

2: The responsive CPU throttle. We present new criteria called
   "dirty ratio" to help improve the detection accuracy and hence
   accelerate the throttle's invocation.

The RFC version of the refinement may be a rudimentary implementation,
I would appreciate hearing more feedback.

Yong, thanks.

Hyman Huang (10):
  migration: Introduce structs for periodic CPU throttle
  migration: Refine util functions to support periodic CPU throttle
  qapi/migration: Introduce periodic CPU throttling parameters
  qapi/migration: Introduce the iteration-count
  migration: Introduce util functions for periodic CPU throttle
  migration: Support periodic CPU throttle
  tests/migration-tests: Add test case for periodic throttle
  migration: Introduce cpu-responsive-throttle parameter
  migration: Support responsive CPU throttle
  tests/migration-tests: Add test case for responsive CPU throttle

 include/exec/ram_addr.h        | 117 ++++++++++++++++--
 include/exec/ramblock.h        |  45 +++++++
 migration/migration-hmp-cmds.c |  25 ++++
 migration/migration-stats.h    |   4 +
 migration/migration.c          |  12 ++
 migration/options.c            |  74 +++++++++++
 migration/options.h            |   3 +
 migration/ram.c                | 218 ++++++++++++++++++++++++++++++---
 migration/ram.h                |   4 +
 migration/trace-events         |   4 +
 qapi/migration.json            |  45 ++++++-
 tests/qtest/migration-test.c   |  77 +++++++++++-
 12 files changed, 600 insertions(+), 28 deletions(-)