[RFC,V1,00/14] precreate phase

Message

Steven Sistare Oct. 17, 2024, 3:14 p.m. UTC

Define a new qemu initialization phase called 'precreate' which occurs
before most backends or devices have been created.  The only exception
is monitor and qtest devices and their associated chardevs.

QEMU runs in the main loop during this phase.  Monitor connections are
active and can receive migration configuration commands.  QEMU starts
listening on the normal migration URI during this phase, which can come
from either the QEMU command line or from a migrate_incoming command.
Thus the user can issue query-migrate to get the socket-address for
dynamically allocated port numbers during precreate.

In this series QEMU passes through and does not linger in the precreate
phase, and the user sees no change in behavior.  The cpr-transfer series
will linger in the phase for an incoming CPR operation, and exit the phase
when the migrate command is send to source QEMU and causes destination QEMU
to read CPR state.

A future series may wish to add a command-line argument and monitor
command that enters and exits the precreate phase for general purpose use.
That would enable the user to issue monitor commands that specify backend
creation parameters.

Now that both the monitor and migration are initialized early, one could
implement a future scheme in which very few parameters are passed on the
command line, and all backend and device creation parameters are passed
in the migration stream.

Patches 1-5 untangle dependencies between accelerator and migration
initialization, allowing the latter to be called earlier.  Currently
configure_accelerators chooses the accelerator, defines compatability
properties needed to initialize migration, and creates the machine in
accel_init_machine.  The patches split configure_accelerators into a first
part which probes for accelerators and defines properties, and a second
part which creates the machine.  The second part is called after the
precreate phase.  The patches also set machine options earlier, so they
are known when migration is initialized.

Patch 6 defines the boundaries of the precreate phase.

Patches 7-12 define and use helper functions to extract monitor and qtest
options from the command line, find their associated chardevs, and initialize
the monitor and qtest devices early, prior to qemu_create_early_backends.

Patches 13-14 contain miscellany.

Steve Sistare (14):
  accel: encapsulate search state
  accel: accel preinit function
  accel: split configure_accelerators
  accel: set accelerator and machine props earlier
  migration: init and listen during precreate
  vl: precreate phase
  monitor: chardev name
  qom: get properties
  qemu-option: filtered foreach
  qemu-options: pass object to filter
  monitor: connect in precreate
  qtest: connect in precreate
  net: cleanup for precreate phase
  migration: allow commands during precreate and preconfig

 accel/accel-system.c            |   2 -
 accel/kvm/kvm-all.c             |  58 +++++----
 accel/xen/xen-all.c             |  11 +-
 hmp-commands.hx                 |   2 +
 include/monitor/monitor.h       |   1 +
 include/qapi/visitor.h          |   1 +
 include/qemu/accel.h            |   1 +
 include/qemu/option.h           |   5 +
 include/qom/object_interfaces.h |   8 ++
 monitor/monitor.c               |  21 ++++
 net/net.c                       |   4 +-
 qapi/migration.json             |  16 ++-
 qapi/misc.json                  |   3 +-
 qom/object_interfaces.c         |  27 ++--
 system/vl.c                     | 270 ++++++++++++++++++++++++++++++----------
 target/i386/nvmm/nvmm-all.c     |  10 +-
 target/i386/whpx/whpx-all.c     |  14 ++-
 util/qemu-option.c              |  25 ++++
 18 files changed, 358 insertions(+), 121 deletions(-)

Comments

Steven Sistare Oct. 17, 2024, 3:19 p.m. UTC | #1

On 10/17/2024 11:14 AM, Steve Sistare wrote:
> Define a new qemu initialization phase called 'precreate' which occurs
> before most backends or devices have been created.  The only exception
> is monitor and qtest devices and their associated chardevs.
> 
> QEMU runs in the main loop during this phase.  Monitor connections are
> active and can receive migration configuration commands.  QEMU starts
> listening on the normal migration URI during this phase, which can come
> from either the QEMU command line or from a migrate_incoming command.
> Thus the user can issue query-migrate to get the socket-address for
> dynamically allocated port numbers during precreate.
> 
> In this series QEMU passes through and does not linger in the precreate
> phase, and the user sees no change in behavior.  The cpr-transfer series
> will linger in the phase for an incoming CPR operation, and exit the phase
> when the migrate command is send to source QEMU and causes destination QEMU
> to read CPR state.

Hi Peter, I rebased the cpr-transfer series on precreate.  The
cpr-transfer migration-test now works.  Do you want to see cpr-transfer
V3 now, or wait until we get feedback on precreate?  The only significant
change is that I deleted the HUP synchronization, and I post an async
listen for the incoming cpr-uri connection.

- Steve

Peter Xu Oct. 17, 2024, 3:53 p.m. UTC | #2

On Thu, Oct 17, 2024 at 11:19:51AM -0400, Steven Sistare wrote:
> On 10/17/2024 11:14 AM, Steve Sistare wrote:
> > Define a new qemu initialization phase called 'precreate' which occurs
> > before most backends or devices have been created.  The only exception
> > is monitor and qtest devices and their associated chardevs.
> > 
> > QEMU runs in the main loop during this phase.  Monitor connections are
> > active and can receive migration configuration commands.  QEMU starts
> > listening on the normal migration URI during this phase, which can come
> > from either the QEMU command line or from a migrate_incoming command.
> > Thus the user can issue query-migrate to get the socket-address for
> > dynamically allocated port numbers during precreate.
> > 
> > In this series QEMU passes through and does not linger in the precreate
> > phase, and the user sees no change in behavior.  The cpr-transfer series
> > will linger in the phase for an incoming CPR operation, and exit the phase
> > when the migrate command is send to source QEMU and causes destination QEMU
> > to read CPR state.
> 
> Hi Peter, I rebased the cpr-transfer series on precreate.  The
> cpr-transfer migration-test now works.  Do you want to see cpr-transfer
> V3 now, or wait until we get feedback on precreate?  The only significant
> change is that I deleted the HUP synchronization, and I post an async
> listen for the incoming cpr-uri connection.

Maybe you can still send it, because I remember there're some other
discussion that may not settled yet (e.g. how anon-memfd is applied, iirc),
then it can be reviewed and discussed concurrently when proper with the
precreate series.

Thanks,