[v3,00/16] migration/mapped-ram: Add direct-io support

Message

Fabiano Rosas June 17, 2024, 6:57 p.m. UTC

Hi,

Not many changes since v2:

- new patch 1: drop IOC reference on offset check failure

- dropped 2 patches adding direct-io to the single threaded
  migration. We have agreed to use multifd with 1 channel for that
  use-case.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/1335719164

Thanks

v2:
https://lore.kernel.org/r/[e-mail]

v1:
https://lore.kernel.org/r/[e-mail]

Hi everyone, here's the rest of the migration "mapped-ram" feature
that didn't get merged for 9.0. This series adds support for direct
I/O, the missing piece to get the desired performance improvements.

There's 3 parts to this:

1- The plumbing for the new "direct-io" migration parameter. With this
   we can already use direct-io with the file transport + multifd +
   mapped-ram. Patches 1-3.

Due to the alignment requirements of O_DIRECT and the fact that
multifd runs the channels in parallel with the migration thread, we
must open the migration file two times, one with O_DIRECT set and
another with it clear.

If the user is not passing in a file name which QEMU can open at will,
we must then require that the user pass the two file descriptors with
the flags already properly set. We'll use the already existing fdset +
QMP add-fd infrastructure for this.

2- Changes to the fdset infrastructure to support O_DIRECT. We need
   those to be able to select from the user-provided fdset the file
   descriptor that contains the O_DIRECT flag. Patches 4-5.

3- Some fdset validation to make sure the two-fds requirement is being
   met. Patches 6-7.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/1269352083

Fabiano Rosas (15):
  migration: Drop reference to QIOChannel if file seeking fails
  migration: Fix file migration with fdset
  tests/qtest/migration: Fix file migration offset check
  tests/qtest/migration: Add a precopy file test with fdset
  monitor: Introduce monitor_fdset_*free
  monitor: Stop removing non-duplicated fds
  monitor: Simplify fdset and fd removal
  monitor: Report errors from monitor_fdset_dup_fd_add
  io: Stop using qemu_open_old in channel-file
  migration: Add direct-io parameter
  migration/multifd: Add direct-io support
  tests/qtest/migration: Add tests for file migration with direct-io
  monitor: fdset: Match against O_DIRECT
  migration: Add documentation for fdset with multifd + file
  tests/qtest/migration: Add a test for mapped-ram with passing of fds

Peter Xu (1):
  monitor: Drop monitor_fdset_dup_fd_find/_remove()

 docs/devel/migration/main.rst       |  24 ++-
 docs/devel/migration/mapped-ram.rst |   6 +-
 include/monitor/monitor.h           |   3 +-
 include/qemu/osdep.h                |   2 +
 io/channel-file.c                   |   8 +-
 migration/file.c                    |  45 ++++-
 migration/file.h                    |   1 -
 migration/migration-hmp-cmds.c      |  11 ++
 migration/migration.c               |  23 +++
 migration/options.c                 |  35 ++++
 migration/options.h                 |   1 +
 monitor/fds.c                       |  96 +++++-----
 monitor/hmp.c                       |   2 -
 monitor/monitor-internal.h          |   1 -
 monitor/monitor.c                   |   1 -
 monitor/qmp.c                       |   2 -
 qapi/migration.json                 |  21 ++-
 stubs/fdset.c                       |   7 +-
 tests/qtest/migration-helpers.c     |  44 +++++
 tests/qtest/migration-helpers.h     |   8 +
 tests/qtest/migration-test.c        | 263 +++++++++++++++++++++++++---
 util/osdep.c                        |  34 ++--
 22 files changed, 508 insertions(+), 130 deletions(-)


base-commit: 05ad1440b8428b0ade9b8e5c01469adb8fbf83e3