| Message ID | 20231016100706.2551-1-quintela@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote: > > The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56: > > Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400) > > are available in the Git repository at: > > https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request > > for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2: > > migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200) > > ---------------------------------------------------------------- > Migration Pull request (20231016) > > In this pull request: > - rdma cleanups > - removal of QEMUFileHook > - test for analyze-migration.py > - test for multifd file > - multifd cleanups > - available switchover bandwidth > - lots of cleanups. > > CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829 > > Please, apply. This CI failure looks migration-related: MALLOC_PERTURB_=96 PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3 QTEST_QEMU_BINARY=./qemu-system-i386 G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh QTEST_QEMU_IMG=./qemu-img QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test --tap -k ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― stderr: ** ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook: assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 == 1363498573) https://gitlab.com/qemu-project/qemu/-/jobs/5301793548 Stefan > > ---------------------------------------------------------------- > > Dmitry Frolov (1): > migration: fix RAMBlock add NULL check > > Elena Ufimtseva (3): > migration: check for rate_limit_max for RATE_LIMIT_DISABLED > multifd: fix counters in multifd_send_thread > multifd: reset next_packet_len after sending pages > > Fabiano Rosas (13): > migration: Fix analyze-migration.py 'configuration' parsing > migration: Add capability parsing to analyze-migration.py > migration: Fix analyze-migration.py when ignore-shared is used > migration: Fix analyze-migration read operation signedness > tests/qtest/migration: Add a test for the analyze-migration script > tests/qtest: migration-test: Add tests for file-based migration > migration/ram: Remove RAMState from xbzrle_cache_zero_page > migration/ram: Stop passing QEMUFile around in save_zero_page > migration/ram: Move xbzrle zero page handling into save_zero_page > migration/ram: Merge save_zero_page functions > migration/multifd: Remove direct "socket" references > migration/multifd: Unify multifd_send_thread error paths > migration/multifd: Clarify Error usage in multifd_channel_connect > > Fiona Ebner (1): > migration: hold the BQL during setup > > Juan Quintela (15): > migration: Non multifd migration don't care about multifd flushes > migration: Create migrate_rdma() > migration/rdma: Unfold ram_control_before_iterate() > migration/rdma: Unfold ram_control_after_iterate() > migration/rdma: Remove all uses of RAM_CONTROL_HOOK > migration/rdma: Unfold hook_ram_load() > migration/rdma: Create rdma_control_save_page() > qemu-file: Remove QEMUFileHooks > migration/rdma: Move rdma constants from qemu-file.h to rdma.h > migration/rdma: Remove qemu_ prefix from exported functions > migration/rdma: Check sooner if we are in postcopy for save_page() > migration/rdma: Use i as for index instead of idx > migration/rdma: Declare for index variables local > migration/rdma: Remove all "ret" variables that are used only once > migration: Improve json and formatting > > Nikolay Borisov (2): > migration: Add the configuration vmstate to the json writer > migration/ram: Refactor precopy ram loading code > > Peter Xu (1): > migration: Allow user to specify available switchover bandwidth > > Philippe Mathieu-Daudé (1): > migration: Use g_autofree to simplify ram_dirty_bitmap_reload() > > Wei Wang (1): > migration: refactor migration_completion > > qapi/migration.json | 41 ++++- > include/migration/register.h | 2 +- > migration/migration.h | 4 +- > migration/options.h | 2 + > migration/qemu-file.h | 49 ------ > migration/rdma.h | 42 +++++ > migration/block-dirty-bitmap.c | 3 - > migration/block.c | 5 - > migration/migration-hmp-cmds.c | 14 ++ > migration/migration-stats.c | 9 +- > migration/migration.c | 199 +++++++++++++-------- > migration/multifd.c | 101 +++++------ > migration/options.c | 35 ++++ > migration/qemu-file.c | 61 +------ > migration/ram.c | 306 ++++++++++++++++++--------------- > migration/rdma.c | 259 ++++++++++++---------------- > migration/savevm.c | 22 ++- > tests/qtest/migration-test.c | 207 ++++++++++++++++++++++ > migration/trace-events | 33 ++-- > scripts/analyze-migration.py | 67 +++++++- > tests/qtest/meson.build | 2 + > 21 files changed, 895 insertions(+), 568 deletions(-) > > -- > 2.41.0 > >
Stefan Hajnoczi <stefanha@gmail.com> writes: > On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote: >> >> The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56: >> >> Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400) >> >> are available in the Git repository at: >> >> https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request >> >> for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2: >> >> migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200) >> >> ---------------------------------------------------------------- >> Migration Pull request (20231016) >> >> In this pull request: >> - rdma cleanups >> - removal of QEMUFileHook >> - test for analyze-migration.py >> - test for multifd file >> - multifd cleanups >> - available switchover bandwidth >> - lots of cleanups. >> >> CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829 >> >> Please, apply. > > This CI failure looks migration-related: > > MALLOC_PERTURB_=96 > PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3 > QTEST_QEMU_BINARY=./qemu-system-i386 > G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh > QTEST_QEMU_IMG=./qemu-img > QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon > /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test > --tap -k > ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― > stderr: > ** > ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook: > assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 == > 1363498573) That's the test for the file: transport which got merged in the last PR. I'll look into it.
On Mon, 16 Oct 2023 at 13:13, Fabiano Rosas <farosas@suse.de> wrote: > > Stefan Hajnoczi <stefanha@gmail.com> writes: > > > On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote: > >> > >> The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56: > >> > >> Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400) > >> > >> are available in the Git repository at: > >> > >> https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request > >> > >> for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2: > >> > >> migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200) > >> > >> ---------------------------------------------------------------- > >> Migration Pull request (20231016) > >> > >> In this pull request: > >> - rdma cleanups > >> - removal of QEMUFileHook > >> - test for analyze-migration.py > >> - test for multifd file > >> - multifd cleanups > >> - available switchover bandwidth > >> - lots of cleanups. > >> > >> CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829 > >> > >> Please, apply. > > > > This CI failure looks migration-related: > > > > MALLOC_PERTURB_=96 > > PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3 > > QTEST_QEMU_BINARY=./qemu-system-i386 > > G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh > > QTEST_QEMU_IMG=./qemu-img > > QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon > > /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test > > --tap -k > > ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― > > stderr: > > ** > > ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook: > > assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 == > > 1363498573) > > That's the test for the file: transport which got merged in the last > PR. I'll look into it. I have dropped the 20231016 pull request for now and the tests passed without it. Maybe there is an interaction with the test you recently added and this pull request? Stefan
Stefan Hajnoczi <stefanha@gmail.com> writes: > On Mon, 16 Oct 2023 at 13:13, Fabiano Rosas <farosas@suse.de> wrote: >> >> Stefan Hajnoczi <stefanha@gmail.com> writes: >> >> > On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote: >> >> >> >> The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56: >> >> >> >> Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400) >> >> >> >> are available in the Git repository at: >> >> >> >> https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request >> >> >> >> for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2: >> >> >> >> migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200) >> >> >> >> ---------------------------------------------------------------- >> >> Migration Pull request (20231016) >> >> >> >> In this pull request: >> >> - rdma cleanups >> >> - removal of QEMUFileHook >> >> - test for analyze-migration.py >> >> - test for multifd file >> >> - multifd cleanups >> >> - available switchover bandwidth >> >> - lots of cleanups. >> >> >> >> CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829 >> >> >> >> Please, apply. >> > >> > This CI failure looks migration-related: >> > >> > MALLOC_PERTURB_=96 >> > PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3 >> > QTEST_QEMU_BINARY=./qemu-system-i386 >> > G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh >> > QTEST_QEMU_IMG=./qemu-img >> > QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon >> > /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test >> > --tap -k >> > ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― >> > stderr: >> > ** >> > ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook: >> > assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 == >> > 1363498573) >> >> That's the test for the file: transport which got merged in the last >> PR. I'll look into it. > > I have dropped the 20231016 pull request for now and the tests passed > without it. Maybe there is an interaction with the test you recently > added and this pull request? Sorry, I expressed myself poorly. The test _is_ what is breaking this pull request. The feature was already merged and is working fine. I commented with a fix on the patch that adds the test. [PULL 11/38] tests/qtest: migration-test: Add tests for file-based migration https://lore.kernel.org/r/20231016100706.2551-12-quintela@redhat.com
Stefan Hajnoczi <stefanha@gmail.com> wrote: > On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote: >> >> The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56: >> >> Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400) >> >> are available in the Git repository at: >> >> https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request >> >> for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2: >> >> migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200) >> >> ---------------------------------------------------------------- >> Migration Pull request (20231016) >> >> In this pull request: >> - rdma cleanups >> - removal of QEMUFileHook >> - test for analyze-migration.py >> - test for multifd file >> - multifd cleanups >> - available switchover bandwidth >> - lots of cleanups. >> >> CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829 >> >> Please, apply. > > This CI failure looks migration-related: It is. > MALLOC_PERTURB_=96 > PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3 > QTEST_QEMU_BINARY=./qemu-system-i386 > G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh > QTEST_QEMU_IMG=./qemu-img > QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon > /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test > --tap -k > ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― > stderr: > ** > ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook: > assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 == > 1363498573) > > https://gitlab.com/qemu-project/qemu/-/jobs/5301793548 But what I am doing wrong here? This time I even posnted the CI link that I passed O:-) I thought that if I pass the: $ git push -o ci.variable=QEMU_CI=2 on my branch, I was doing all the testing that you are doing. Clearly not. Thanks, Juan. > Stefan > >> >> ---------------------------------------------------------------- >> >> Dmitry Frolov (1): >> migration: fix RAMBlock add NULL check >> >> Elena Ufimtseva (3): >> migration: check for rate_limit_max for RATE_LIMIT_DISABLED >> multifd: fix counters in multifd_send_thread >> multifd: reset next_packet_len after sending pages >> >> Fabiano Rosas (13): >> migration: Fix analyze-migration.py 'configuration' parsing >> migration: Add capability parsing to analyze-migration.py >> migration: Fix analyze-migration.py when ignore-shared is used >> migration: Fix analyze-migration read operation signedness >> tests/qtest/migration: Add a test for the analyze-migration script >> tests/qtest: migration-test: Add tests for file-based migration >> migration/ram: Remove RAMState from xbzrle_cache_zero_page >> migration/ram: Stop passing QEMUFile around in save_zero_page >> migration/ram: Move xbzrle zero page handling into save_zero_page >> migration/ram: Merge save_zero_page functions >> migration/multifd: Remove direct "socket" references >> migration/multifd: Unify multifd_send_thread error paths >> migration/multifd: Clarify Error usage in multifd_channel_connect >> >> Fiona Ebner (1): >> migration: hold the BQL during setup >> >> Juan Quintela (15): >> migration: Non multifd migration don't care about multifd flushes >> migration: Create migrate_rdma() >> migration/rdma: Unfold ram_control_before_iterate() >> migration/rdma: Unfold ram_control_after_iterate() >> migration/rdma: Remove all uses of RAM_CONTROL_HOOK >> migration/rdma: Unfold hook_ram_load() >> migration/rdma: Create rdma_control_save_page() >> qemu-file: Remove QEMUFileHooks >> migration/rdma: Move rdma constants from qemu-file.h to rdma.h >> migration/rdma: Remove qemu_ prefix from exported functions >> migration/rdma: Check sooner if we are in postcopy for save_page() >> migration/rdma: Use i as for index instead of idx >> migration/rdma: Declare for index variables local >> migration/rdma: Remove all "ret" variables that are used only once >> migration: Improve json and formatting >> >> Nikolay Borisov (2): >> migration: Add the configuration vmstate to the json writer >> migration/ram: Refactor precopy ram loading code >> >> Peter Xu (1): >> migration: Allow user to specify available switchover bandwidth >> >> Philippe Mathieu-Daudé (1): >> migration: Use g_autofree to simplify ram_dirty_bitmap_reload() >> >> Wei Wang (1): >> migration: refactor migration_completion >> >> qapi/migration.json | 41 ++++- >> include/migration/register.h | 2 +- >> migration/migration.h | 4 +- >> migration/options.h | 2 + >> migration/qemu-file.h | 49 ------ >> migration/rdma.h | 42 +++++ >> migration/block-dirty-bitmap.c | 3 - >> migration/block.c | 5 - >> migration/migration-hmp-cmds.c | 14 ++ >> migration/migration-stats.c | 9 +- >> migration/migration.c | 199 +++++++++++++-------- >> migration/multifd.c | 101 +++++------ >> migration/options.c | 35 ++++ >> migration/qemu-file.c | 61 +------ >> migration/ram.c | 306 ++++++++++++++++++--------------- >> migration/rdma.c | 259 ++++++++++++---------------- >> migration/savevm.c | 22 ++- >> tests/qtest/migration-test.c | 207 ++++++++++++++++++++++ >> migration/trace-events | 33 ++-- >> scripts/analyze-migration.py | 67 +++++++- >> tests/qtest/meson.build | 2 + >> 21 files changed, 895 insertions(+), 568 deletions(-) >> >> -- >> 2.41.0 >> >>
On 17/10/2023 09.24, Juan Quintela wrote: > Stefan Hajnoczi <stefanha@gmail.com> wrote: >> On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote: >>> >>> The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56: >>> >>> Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400) >>> >>> are available in the Git repository at: >>> >>> https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request >>> >>> for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2: >>> >>> migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200) >>> >>> ---------------------------------------------------------------- >>> Migration Pull request (20231016) >>> >>> In this pull request: >>> - rdma cleanups >>> - removal of QEMUFileHook >>> - test for analyze-migration.py >>> - test for multifd file >>> - multifd cleanups >>> - available switchover bandwidth >>> - lots of cleanups. >>> >>> CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829 >>> >>> Please, apply. >> >> This CI failure looks migration-related: > > It is. > >> MALLOC_PERTURB_=96 >> PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3 >> QTEST_QEMU_BINARY=./qemu-system-i386 >> G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh >> QTEST_QEMU_IMG=./qemu-img >> QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon >> /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test >> --tap -k >> ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― >> stderr: >> ** >> ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook: >> assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 == >> 1363498573) >> >> https://gitlab.com/qemu-project/qemu/-/jobs/5301793548 > > But what I am doing wrong here? > This time I even posnted the CI link that I passed O:-) > > I thought that if I pass the: > > $ git push -o ci.variable=QEMU_CI=2 > > on my branch, I was doing all the testing that you are doing. Clearly > not. The failure happened on a s390x host, so nothing you can test by default [*] - these checks are only running in the qemu-project itself. Thomas * If you still have a github repo, too, you can test with Travis-CI there, it features tests on s390x, aarch64 and ppc64le hosts.