[RFC,00/14] *** multifd for RDMA v2 ***

Message

fengzhimin Feb. 13, 2020, 9:37 a.m. UTC

Hi

This is a version against current code. It is based on the multifd work.
we can use the multifd parameters for rdma transport. All data is
transported by the multifd RDMA channels and the main channel is only
used to distribute its to the different multifd channels.

Zhimin Feng (14):
  migration: add the 'migrate_use_rdma_pin_all' function
  migration: judge whether or not the RDMA is used for migration
  migration/rdma: Create multiFd migration threads
  migration/rdma: Export the RDMAContext struct
  migration/rdma: Create the multifd channels for RDMA
  migration/rdma: Transmit initial packet
  migration/rdma: Export the 'qemu_rdma_registration_handle' and
    'qemu_rdma_exchange_send' functions
  migration/rdma: Add the function for dynamic page registration
  migration/rdma: register memory for multifd RDMA channels
  migration/rdma: Wait for all multifd to complete registration
  migration/rdma: use multifd to migrate VM for rdma-pin-all mode
  migration/rdma: use multifd to migrate VM for NOT rdma-pin-all mode
  migration/rdma: only register the memory for multifd channels
  migration/rdma: RDMA cleanup for multifd migration

 migration/migration.c |  19 ++
 migration/migration.h |   2 +
 migration/multifd.c   | 192 +++++++++++++-
 migration/multifd.h   |  12 +
 migration/qemu-file.c |   5 +
 migration/qemu-file.h |   1 +
 migration/rdma.c      | 579 +++++++++++++++++++++++-------------------
 migration/rdma.h      | 268 +++++++++++++++++++
 8 files changed, 804 insertions(+), 274 deletions(-)

Comments

[e-mail] Feb. 13, 2020, 10:14 a.m. UTC | #1

Patchew URL: https://patchew.org/QEMU/[e-mail]/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

/tmp/qemu-test/src/migration/multifd.c:663: undefined reference to `multifd_channel_rdma_connect'
../migration/multifd.o: In function `multifd_load_cleanup':
/tmp/qemu-test/src/migration/multifd.c:843: undefined reference to `qemu_rdma_cleanup'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:206: qemu-system-x86_64w.exe] Error 1
make: *** [Makefile:497: x86_64-softmmu/all] Error 2
make: *** Waiting for unfinished jobs....
../migration/multifd.o: In function `multifd_rdma_recv_thread':
/tmp/qemu-test/src/migration/multifd.c:898: undefined reference to `qemu_rdma_registration_handle'
---
/tmp/qemu-test/src/migration/multifd.c:663: undefined reference to `multifd_channel_rdma_connect'
../migration/multifd.o: In function `multifd_load_cleanup':
/tmp/qemu-test/src/migration/multifd.c:843: undefined reference to `qemu_rdma_cleanup'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:206: qemu-system-aarch64w.exe] Error 1
make: *** [Makefile:497: aarch64-softmmu/all] Error 2
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=88473d634d6543ea992045cbe9a806e1', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-yiq7aevf/src/docker-src.2020-02-13-05.11.35.1374:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=88473d634d6543ea992045cbe9a806e1
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-yiq7aevf/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    2m35.791s
user    0m7.717s


The full log is available at
http://patchew.org/logs/[e-mail]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [e-mail]

Dr. David Alan Gilbert Feb. 14, 2020, 1:23 p.m. UTC | #2

Make sure that it compiles OK with RDMA compiled out; I think this is a
windows cross build that's failing, but more generally even a Linux box
with no-RDMA libraries.

Dave

* [e-mail] ([e-mail]) wrote:
> Patchew URL: https://patchew.org/QEMU/[e-mail]/
> 
> 
> 
> Hi,
> 
> This series failed the docker-mingw@fedora build test. Please find the testing commands and
> their output below. If you have Docker installed, you can probably reproduce it
> locally.
> 
> === TEST SCRIPT BEGIN ===
> #! /bin/bash
> export ARCH=x86_64
> make docker-image-fedora V=1 NETWORK=1
> time make docker-test-mingw@fedora J=14 NETWORK=1
> === TEST SCRIPT END ===
> 
> /tmp/qemu-test/src/migration/multifd.c:663: undefined reference to `multifd_channel_rdma_connect'
> ../migration/multifd.o: In function `multifd_load_cleanup':
> /tmp/qemu-test/src/migration/multifd.c:843: undefined reference to `qemu_rdma_cleanup'
> collect2: error: ld returned 1 exit status
> make[1]: *** [Makefile:206: qemu-system-x86_64w.exe] Error 1
> make: *** [Makefile:497: x86_64-softmmu/all] Error 2
> make: *** Waiting for unfinished jobs....
> ../migration/multifd.o: In function `multifd_rdma_recv_thread':
> /tmp/qemu-test/src/migration/multifd.c:898: undefined reference to `qemu_rdma_registration_handle'
> ---
> /tmp/qemu-test/src/migration/multifd.c:663: undefined reference to `multifd_channel_rdma_connect'
> ../migration/multifd.o: In function `multifd_load_cleanup':
> /tmp/qemu-test/src/migration/multifd.c:843: undefined reference to `qemu_rdma_cleanup'
> collect2: error: ld returned 1 exit status
> make[1]: *** [Makefile:206: qemu-system-aarch64w.exe] Error 1
> make: *** [Makefile:497: aarch64-softmmu/all] Error 2
> Traceback (most recent call last):
>   File "./tests/docker/docker.py", line 664, in <module>
>     sys.exit(main())
> ---
>     raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=88473d634d6543ea992045cbe9a806e1', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-yiq7aevf/src/docker-src.2020-02-13-05.11.35.1374:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
> filter=--filter=label=com.qemu.instance.uuid=88473d634d6543ea992045cbe9a806e1
> make[1]: *** [docker-run] Error 1
> make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-yiq7aevf/src'
> make: *** [docker-run-test-mingw@fedora] Error 2
> 
> real    2m35.791s
> user    0m7.717s
> 
> 
> The full log is available at
> http://patchew.org/logs/[e-mail]/testing.docker-mingw@fedora/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to [e-mail]
--
Dr. David Alan Gilbert / [e-mail] / Manchester, UK