mbox series

[v6,00/14] Reverse debugging

Message ID 160137726426.31007.12061315974029139983.stgit@pasha-ThinkPad-X280 (mailing list archive)
Headers show
Series Reverse debugging | expand

Message

Pavel Dovgalyuk Sept. 29, 2020, 11:01 a.m. UTC
GDB remote protocol supports reverse debugging of the targets.
It includes 'reverse step' and 'reverse continue' operations.
The first one finds the previous step of the execution,
and the second one is intended to stop at the last breakpoint that
would happen when the program is executed normally.

Reverse debugging is possible in the replay mode, when at least
one snapshot was created at the record or replay phase.
QEMU can use these snapshots for travelling back in time with GDB.

Running the execution in replay mode allows using GDB reverse debugging
commands:
 - reverse-stepi (or rsi): Steps one instruction to the past.
   QEMU loads on of the prior snapshots and proceeds to the desired
   instruction forward. When that step is reaches, execution stops.
 - reverse-continue (or rc): Runs execution "backwards".
   QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
   and replaying the execution. Then QEMU loads snapshots again and
   replays to the latest breakpoint. When there are no breakpoints in
   the examined section of the execution, QEMU finds one more snapshot
   and tries again. After the first snapshot is processed, execution
   stops at this snapshot.

The set of patches include the following modifications:
 - gdbstub update for reverse debugging support
 - functions that automatically perform reverse step and reverse
   continue operations
 - hmp/qmp commands for manipulating the replay process
 - improvement of the snapshotting for saving the execution step
   in the snapshot parameters
 - avocado-based acceptance tests for reverse debugging

The patches are available in the repository:
https://github.com/ispras/qemu/tree/rr-200901

v6 changes:
 - removed passing err variable without checking it's value after
v5 changes:
 - disabled reverse debugging tests for gitlab-based testing
   due to the unidentified timeout problem
v4 changes:
 - added VM snapshot creation on gdb connect (suggested by Alex Bennée)
 - removed useless calls to error_free
 - updated poll interrupt processing
 - minor changes
v3 changes:
 - rebased to support the new build system
 - bumped avocado framework version for using fixed remote gdb client
v2 changes:
 - rebased to the latest upstream version
 - fixed replaying of the POLL interrupts after the latest debug changes

---

Pavel Dovgaluk (11):
      replay: provide an accessor for rr filename
      qcow2: introduce icount field for snapshots
      qapi: introduce replay.json for record/replay-related stuff
      replay: introduce info hmp/qmp command
      replay: introduce breakpoint at the specified step
      replay: implement replay-seek command
      replay: flush rr queue before loading the vmstate
      gdbstub: add reverse step support in replay mode
      gdbstub: add reverse continue support in replay mode
      replay: describe reverse debugging in docs/replay.txt
      tests/acceptance: add reverse debugging test

Pavel Dovgalyuk (3):
      replay: don't record interrupt poll
      migration: introduce icount field for snapshots
      replay: create temporary snapshot at debugger connection


 MAINTAINERS                           |    2 
 accel/tcg/cpu-exec.c                  |   21 ++
 accel/tcg/translator.c                |    1 
 block/qapi.c                          |   18 +-
 block/qcow2-snapshot.c                |    9 +
 block/qcow2.h                         |    3 
 blockdev.c                            |   10 +
 docs/interop/qcow2.txt                |    5 
 docs/replay.txt                       |   46 +++++
 exec.c                                |    8 +
 gdbstub.c                             |   64 ++++++
 hmp-commands-info.hx                  |   11 +
 hmp-commands.hx                       |   50 +++++
 include/block/snapshot.h              |    1 
 include/monitor/hmp.h                 |    4 
 include/sysemu/replay.h               |   26 +++
 migration/savevm.c                    |   17 +-
 qapi/block-core.json                  |   11 +
 qapi/meson.build                      |    1 
 qapi/misc.json                        |   18 --
 qapi/qapi-schema.json                 |    1 
 qapi/replay.json                      |  121 ++++++++++++
 replay/meson.build                    |    1 
 replay/replay-debugging.c             |  332 +++++++++++++++++++++++++++++++++
 replay/replay-events.c                |    4 
 replay/replay-internal.h              |    6 -
 replay/replay.c                       |   22 ++
 softmmu/cpus.c                        |   19 ++
 stubs/replay.c                        |   15 +
 tests/acceptance/reverse_debugging.py |  208 +++++++++++++++++++++
 tests/qemu-iotests/267.out            |   48 ++---
 31 files changed, 1039 insertions(+), 64 deletions(-)
 create mode 100644 qapi/replay.json
 create mode 100644 replay/replay-debugging.c
 create mode 100644 tests/acceptance/reverse_debugging.py

--
Pavel Dovgalyuk

Comments

no-reply@patchew.org Sept. 29, 2020, 3:51 p.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/160137726426.31007.12061315974029139983.stgit@pasha-ThinkPad-X280/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 160137726426.31007.12061315974029139983.stgit@pasha-ThinkPad-X280
Subject: [PATCH v6 00/14] Reverse debugging

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
42ed9ff tests/acceptance: add reverse debugging test
9e6312b replay: create temporary snapshot at debugger connection
e5b95b0 replay: describe reverse debugging in docs/replay.txt
47c1dc5 gdbstub: add reverse continue support in replay mode
9f812ce gdbstub: add reverse step support in replay mode
47580b0 replay: flush rr queue before loading the vmstate
3a46821 replay: implement replay-seek command
333560c replay: introduce breakpoint at the specified step
912d979 replay: introduce info hmp/qmp command
f174f33 qapi: introduce replay.json for record/replay-related stuff
fa8bc32 migration: introduce icount field for snapshots
1785cc5 qcow2: introduce icount field for snapshots
5d53fea replay: provide an accessor for rr filename
84a429f replay: don't record interrupt poll

=== OUTPUT BEGIN ===
1/14 Checking commit 84a429fe86d2 (replay: don't record interrupt poll)
2/14 Checking commit 5d53feae3272 (replay: provide an accessor for rr filename)
3/14 Checking commit 1785cc5df6f7 (qcow2: introduce icount field for snapshots)
4/14 Checking commit fa8bc32691b4 (migration: introduce icount field for snapshots)
ERROR: trailing whitespace
#226: FILE: tests/qemu-iotests/267.out:37:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#237: FILE: tests/qemu-iotests/267.out:48:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#248: FILE: tests/qemu-iotests/267.out:73:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#259: FILE: tests/qemu-iotests/267.out:98:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#270: FILE: tests/qemu-iotests/267.out:109:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#281: FILE: tests/qemu-iotests/267.out:123:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#292: FILE: tests/qemu-iotests/267.out:138:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#303: FILE: tests/qemu-iotests/267.out:149:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#312: FILE: tests/qemu-iotests/267.out:156:
+1         snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#323: FILE: tests/qemu-iotests/267.out:170:
+--        snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#332: FILE: tests/qemu-iotests/267.out:177:
+1         snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

ERROR: trailing whitespace
#338: FILE: tests/qemu-iotests/267.out:181:
+1         snap0                SIZE yyyy-mm-dd hh:mm:ss 00:00:00.000           $

total: 12 errors, 0 warnings, 259 lines checked

Patch 4/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

5/14 Checking commit f174f339f352 (qapi: introduce replay.json for record/replay-related stuff)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#93: 
new file mode 100644

total: 0 errors, 1 warnings, 78 lines checked

Patch 5/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/14 Checking commit 912d9793f5a5 (replay: introduce info hmp/qmp command)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#122: 
new file mode 100644

total: 0 errors, 1 warnings, 120 lines checked

Patch 6/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/14 Checking commit 333560c38d6a (replay: introduce breakpoint at the specified step)
8/14 Checking commit 3a46821e71ef (replay: implement replay-seek command)
9/14 Checking commit 47580b0a465e (replay: flush rr queue before loading the vmstate)
10/14 Checking commit 9f812ced7a0a (gdbstub: add reverse step support in replay mode)
WARNING: line over 80 characters
#220: FILE: replay/replay-debugging.c:237:
+        replay_seek(replay_get_current_icount() - 1, replay_stop_vm_debug, &err);

total: 0 errors, 1 warnings, 201 lines checked

Patch 10/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/14 Checking commit 47c1dc583443 (gdbstub: add reverse continue support in replay mode)
WARNING: line over 80 characters
#150: FILE: replay/replay-debugging.c:300:
+        replay_seek(replay_get_current_icount() - 1, replay_continue_stop, &err);

total: 0 errors, 1 warnings, 146 lines checked

Patch 11/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/14 Checking commit e5b95b028f0d (replay: describe reverse debugging in docs/replay.txt)
13/14 Checking commit 9e6312b2ec00 (replay: create temporary snapshot at debugger connection)
14/14 Checking commit 42ed9ffcae51 (tests/acceptance: add reverse debugging test)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#40: 
new file mode 100644

total: 0 errors, 1 warnings, 215 lines checked

Patch 14/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/160137726426.31007.12061315974029139983.stgit@pasha-ThinkPad-X280/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Paolo Bonzini Oct. 2, 2020, 3:39 p.m. UTC | #2
On 29/09/20 13:01, Pavel Dovgalyuk wrote:
> GDB remote protocol supports reverse debugging of the targets.
> It includes 'reverse step' and 'reverse continue' operations.
> The first one finds the previous step of the execution,
> and the second one is intended to stop at the last breakpoint that
> would happen when the program is executed normally.
> 
> Reverse debugging is possible in the replay mode, when at least
> one snapshot was created at the record or replay phase.
> QEMU can use these snapshots for travelling back in time with GDB.
> 
> Running the execution in replay mode allows using GDB reverse debugging
> commands:
>  - reverse-stepi (or rsi): Steps one instruction to the past.
>    QEMU loads on of the prior snapshots and proceeds to the desired
>    instruction forward. When that step is reaches, execution stops.
>  - reverse-continue (or rc): Runs execution "backwards".
>    QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
>    and replaying the execution. Then QEMU loads snapshots again and
>    replays to the latest breakpoint. When there are no breakpoints in
>    the examined section of the execution, QEMU finds one more snapshot
>    and tries again. After the first snapshot is processed, execution
>    stops at this snapshot.
> 
> The set of patches include the following modifications:
>  - gdbstub update for reverse debugging support
>  - functions that automatically perform reverse step and reverse
>    continue operations
>  - hmp/qmp commands for manipulating the replay process
>  - improvement of the snapshotting for saving the execution step
>    in the snapshot parameters
>  - avocado-based acceptance tests for reverse debugging
> 
> The patches are available in the repository:
> https://github.com/ispras/qemu/tree/rr-200901

Hi Pavel,

I'm still seeing failures in "make check-block":

https://gitlab.com/bonzini/qemu/-/jobs/769653852

Paolo

> v6 changes:
>  - removed passing err variable without checking it's value after
> v5 changes:
>  - disabled reverse debugging tests for gitlab-based testing
>    due to the unidentified timeout problem
> v4 changes:
>  - added VM snapshot creation on gdb connect (suggested by Alex Bennée)
>  - removed useless calls to error_free
>  - updated poll interrupt processing
>  - minor changes
> v3 changes:
>  - rebased to support the new build system
>  - bumped avocado framework version for using fixed remote gdb client
> v2 changes:
>  - rebased to the latest upstream version
>  - fixed replaying of the POLL interrupts after the latest debug changes
> 
> ---
> 
> Pavel Dovgaluk (11):
>       replay: provide an accessor for rr filename
>       qcow2: introduce icount field for snapshots
>       qapi: introduce replay.json for record/replay-related stuff
>       replay: introduce info hmp/qmp command
>       replay: introduce breakpoint at the specified step
>       replay: implement replay-seek command
>       replay: flush rr queue before loading the vmstate
>       gdbstub: add reverse step support in replay mode
>       gdbstub: add reverse continue support in replay mode
>       replay: describe reverse debugging in docs/replay.txt
>       tests/acceptance: add reverse debugging test
> 
> Pavel Dovgalyuk (3):
>       replay: don't record interrupt poll
>       migration: introduce icount field for snapshots
>       replay: create temporary snapshot at debugger connection
> 
> 
>  MAINTAINERS                           |    2 
>  accel/tcg/cpu-exec.c                  |   21 ++
>  accel/tcg/translator.c                |    1 
>  block/qapi.c                          |   18 +-
>  block/qcow2-snapshot.c                |    9 +
>  block/qcow2.h                         |    3 
>  blockdev.c                            |   10 +
>  docs/interop/qcow2.txt                |    5 
>  docs/replay.txt                       |   46 +++++
>  exec.c                                |    8 +
>  gdbstub.c                             |   64 ++++++
>  hmp-commands-info.hx                  |   11 +
>  hmp-commands.hx                       |   50 +++++
>  include/block/snapshot.h              |    1 
>  include/monitor/hmp.h                 |    4 
>  include/sysemu/replay.h               |   26 +++
>  migration/savevm.c                    |   17 +-
>  qapi/block-core.json                  |   11 +
>  qapi/meson.build                      |    1 
>  qapi/misc.json                        |   18 --
>  qapi/qapi-schema.json                 |    1 
>  qapi/replay.json                      |  121 ++++++++++++
>  replay/meson.build                    |    1 
>  replay/replay-debugging.c             |  332 +++++++++++++++++++++++++++++++++
>  replay/replay-events.c                |    4 
>  replay/replay-internal.h              |    6 -
>  replay/replay.c                       |   22 ++
>  softmmu/cpus.c                        |   19 ++
>  stubs/replay.c                        |   15 +
>  tests/acceptance/reverse_debugging.py |  208 +++++++++++++++++++++
>  tests/qemu-iotests/267.out            |   48 ++---
>  31 files changed, 1039 insertions(+), 64 deletions(-)
>  create mode 100644 qapi/replay.json
>  create mode 100644 replay/replay-debugging.c
>  create mode 100644 tests/acceptance/reverse_debugging.py
> 
> --
> Pavel Dovgalyuk
>
Pavel Dovgalyuk Oct. 2, 2020, 4:01 p.m. UTC | #3
On 02.10.2020 18:39, Paolo Bonzini wrote:
> On 29/09/20 13:01, Pavel Dovgalyuk wrote:
>> GDB remote protocol supports reverse debugging of the targets.
>> It includes 'reverse step' and 'reverse continue' operations.
>> The first one finds the previous step of the execution,
>> and the second one is intended to stop at the last breakpoint that
>> would happen when the program is executed normally.
>>
>> Reverse debugging is possible in the replay mode, when at least
>> one snapshot was created at the record or replay phase.
>> QEMU can use these snapshots for travelling back in time with GDB.
>>
>> Running the execution in replay mode allows using GDB reverse debugging
>> commands:
>>   - reverse-stepi (or rsi): Steps one instruction to the past.
>>     QEMU loads on of the prior snapshots and proceeds to the desired
>>     instruction forward. When that step is reaches, execution stops.
>>   - reverse-continue (or rc): Runs execution "backwards".
>>     QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
>>     and replaying the execution. Then QEMU loads snapshots again and
>>     replays to the latest breakpoint. When there are no breakpoints in
>>     the examined section of the execution, QEMU finds one more snapshot
>>     and tries again. After the first snapshot is processed, execution
>>     stops at this snapshot.
>>
>> The set of patches include the following modifications:
>>   - gdbstub update for reverse debugging support
>>   - functions that automatically perform reverse step and reverse
>>     continue operations
>>   - hmp/qmp commands for manipulating the replay process
>>   - improvement of the snapshotting for saving the execution step
>>     in the snapshot parameters
>>   - avocado-based acceptance tests for reverse debugging
>>
>> The patches are available in the repository:
>> https://github.com/ispras/qemu/tree/rr-200901
> 
> Hi Pavel,
> 
> I'm still seeing failures in "make check-block":
> 
> https://gitlab.com/bonzini/qemu/-/jobs/769653852

I saw a message from Alex about it, but I don't see any errors when run 
'make check' myself. Could it be caused by new patches that also were 
added to your branch?

It seems, that some 'valid' test output should be updated, because 
patches changed the qcow2 snapshot extra data size.


> 
> Paolo
> 
>> v6 changes:
>>   - removed passing err variable without checking it's value after
>> v5 changes:
>>   - disabled reverse debugging tests for gitlab-based testing
>>     due to the unidentified timeout problem
>> v4 changes:
>>   - added VM snapshot creation on gdb connect (suggested by Alex Bennée)
>>   - removed useless calls to error_free
>>   - updated poll interrupt processing
>>   - minor changes
>> v3 changes:
>>   - rebased to support the new build system
>>   - bumped avocado framework version for using fixed remote gdb client
>> v2 changes:
>>   - rebased to the latest upstream version
>>   - fixed replaying of the POLL interrupts after the latest debug changes
>>
>> ---
>>
>> Pavel Dovgaluk (11):
>>        replay: provide an accessor for rr filename
>>        qcow2: introduce icount field for snapshots
>>        qapi: introduce replay.json for record/replay-related stuff
>>        replay: introduce info hmp/qmp command
>>        replay: introduce breakpoint at the specified step
>>        replay: implement replay-seek command
>>        replay: flush rr queue before loading the vmstate
>>        gdbstub: add reverse step support in replay mode
>>        gdbstub: add reverse continue support in replay mode
>>        replay: describe reverse debugging in docs/replay.txt
>>        tests/acceptance: add reverse debugging test
>>
>> Pavel Dovgalyuk (3):
>>        replay: don't record interrupt poll
>>        migration: introduce icount field for snapshots
>>        replay: create temporary snapshot at debugger connection
>>
>>
>>   MAINTAINERS                           |    2
>>   accel/tcg/cpu-exec.c                  |   21 ++
>>   accel/tcg/translator.c                |    1
>>   block/qapi.c                          |   18 +-
>>   block/qcow2-snapshot.c                |    9 +
>>   block/qcow2.h                         |    3
>>   blockdev.c                            |   10 +
>>   docs/interop/qcow2.txt                |    5
>>   docs/replay.txt                       |   46 +++++
>>   exec.c                                |    8 +
>>   gdbstub.c                             |   64 ++++++
>>   hmp-commands-info.hx                  |   11 +
>>   hmp-commands.hx                       |   50 +++++
>>   include/block/snapshot.h              |    1
>>   include/monitor/hmp.h                 |    4
>>   include/sysemu/replay.h               |   26 +++
>>   migration/savevm.c                    |   17 +-
>>   qapi/block-core.json                  |   11 +
>>   qapi/meson.build                      |    1
>>   qapi/misc.json                        |   18 --
>>   qapi/qapi-schema.json                 |    1
>>   qapi/replay.json                      |  121 ++++++++++++
>>   replay/meson.build                    |    1
>>   replay/replay-debugging.c             |  332 +++++++++++++++++++++++++++++++++
>>   replay/replay-events.c                |    4
>>   replay/replay-internal.h              |    6 -
>>   replay/replay.c                       |   22 ++
>>   softmmu/cpus.c                        |   19 ++
>>   stubs/replay.c                        |   15 +
>>   tests/acceptance/reverse_debugging.py |  208 +++++++++++++++++++++
>>   tests/qemu-iotests/267.out            |   48 ++---
>>   31 files changed, 1039 insertions(+), 64 deletions(-)
>>   create mode 100644 qapi/replay.json
>>   create mode 100644 replay/replay-debugging.c
>>   create mode 100644 tests/acceptance/reverse_debugging.py
>>
>> --
>> Pavel Dovgalyuk
>>
>
Pavel Dovgalyuk Oct. 2, 2020, 4:25 p.m. UTC | #4
On 02.10.2020 18:39, Paolo Bonzini wrote:
> On 29/09/20 13:01, Pavel Dovgalyuk wrote:
>> GDB remote protocol supports reverse debugging of the targets.
>> It includes 'reverse step' and 'reverse continue' operations.
>> The first one finds the previous step of the execution,
>> and the second one is intended to stop at the last breakpoint that
>> would happen when the program is executed normally.
>>
>> Reverse debugging is possible in the replay mode, when at least
>> one snapshot was created at the record or replay phase.
>> QEMU can use these snapshots for travelling back in time with GDB.
>>
>> Running the execution in replay mode allows using GDB reverse debugging
>> commands:
>>   - reverse-stepi (or rsi): Steps one instruction to the past.
>>     QEMU loads on of the prior snapshots and proceeds to the desired
>>     instruction forward. When that step is reaches, execution stops.
>>   - reverse-continue (or rc): Runs execution "backwards".
>>     QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
>>     and replaying the execution. Then QEMU loads snapshots again and
>>     replays to the latest breakpoint. When there are no breakpoints in
>>     the examined section of the execution, QEMU finds one more snapshot
>>     and tries again. After the first snapshot is processed, execution
>>     stops at this snapshot.
>>
>> The set of patches include the following modifications:
>>   - gdbstub update for reverse debugging support
>>   - functions that automatically perform reverse step and reverse
>>     continue operations
>>   - hmp/qmp commands for manipulating the replay process
>>   - improvement of the snapshotting for saving the execution step
>>     in the snapshot parameters
>>   - avocado-based acceptance tests for reverse debugging
>>
>> The patches are available in the repository:
>> https://github.com/ispras/qemu/tree/rr-200901
> 
> Hi Pavel,
> 
> I'm still seeing failures in "make check-block":
> 
> https://gitlab.com/bonzini/qemu/-/jobs/769653852

Ok, now I can reproduce it with manually running 'check -qcow2 261'.
I'll send a patch soon.

> 
> Paolo
> 
>> v6 changes:
>>   - removed passing err variable without checking it's value after
>> v5 changes:
>>   - disabled reverse debugging tests for gitlab-based testing
>>     due to the unidentified timeout problem
>> v4 changes:
>>   - added VM snapshot creation on gdb connect (suggested by Alex Bennée)
>>   - removed useless calls to error_free
>>   - updated poll interrupt processing
>>   - minor changes
>> v3 changes:
>>   - rebased to support the new build system
>>   - bumped avocado framework version for using fixed remote gdb client
>> v2 changes:
>>   - rebased to the latest upstream version
>>   - fixed replaying of the POLL interrupts after the latest debug changes
>>
>> ---
>>
>> Pavel Dovgaluk (11):
>>        replay: provide an accessor for rr filename
>>        qcow2: introduce icount field for snapshots
>>        qapi: introduce replay.json for record/replay-related stuff
>>        replay: introduce info hmp/qmp command
>>        replay: introduce breakpoint at the specified step
>>        replay: implement replay-seek command
>>        replay: flush rr queue before loading the vmstate
>>        gdbstub: add reverse step support in replay mode
>>        gdbstub: add reverse continue support in replay mode
>>        replay: describe reverse debugging in docs/replay.txt
>>        tests/acceptance: add reverse debugging test
>>
>> Pavel Dovgalyuk (3):
>>        replay: don't record interrupt poll
>>        migration: introduce icount field for snapshots
>>        replay: create temporary snapshot at debugger connection
>>
>>
>>   MAINTAINERS                           |    2
>>   accel/tcg/cpu-exec.c                  |   21 ++
>>   accel/tcg/translator.c                |    1
>>   block/qapi.c                          |   18 +-
>>   block/qcow2-snapshot.c                |    9 +
>>   block/qcow2.h                         |    3
>>   blockdev.c                            |   10 +
>>   docs/interop/qcow2.txt                |    5
>>   docs/replay.txt                       |   46 +++++
>>   exec.c                                |    8 +
>>   gdbstub.c                             |   64 ++++++
>>   hmp-commands-info.hx                  |   11 +
>>   hmp-commands.hx                       |   50 +++++
>>   include/block/snapshot.h              |    1
>>   include/monitor/hmp.h                 |    4
>>   include/sysemu/replay.h               |   26 +++
>>   migration/savevm.c                    |   17 +-
>>   qapi/block-core.json                  |   11 +
>>   qapi/meson.build                      |    1
>>   qapi/misc.json                        |   18 --
>>   qapi/qapi-schema.json                 |    1
>>   qapi/replay.json                      |  121 ++++++++++++
>>   replay/meson.build                    |    1
>>   replay/replay-debugging.c             |  332 +++++++++++++++++++++++++++++++++
>>   replay/replay-events.c                |    4
>>   replay/replay-internal.h              |    6 -
>>   replay/replay.c                       |   22 ++
>>   softmmu/cpus.c                        |   19 ++
>>   stubs/replay.c                        |   15 +
>>   tests/acceptance/reverse_debugging.py |  208 +++++++++++++++++++++
>>   tests/qemu-iotests/267.out            |   48 ++---
>>   31 files changed, 1039 insertions(+), 64 deletions(-)
>>   create mode 100644 qapi/replay.json
>>   create mode 100644 replay/replay-debugging.c
>>   create mode 100644 tests/acceptance/reverse_debugging.py
>>
>> --
>> Pavel Dovgalyuk
>>
>
Pavel Dovgalyuk Oct. 3, 2020, 5:19 p.m. UTC | #5
On 02.10.2020 18:39, Paolo Bonzini wrote:
> On 29/09/20 13:01, Pavel Dovgalyuk wrote:
>> GDB remote protocol supports reverse debugging of the targets.
>> It includes 'reverse step' and 'reverse continue' operations.
>> The first one finds the previous step of the execution,
>> and the second one is intended to stop at the last breakpoint that
>> would happen when the program is executed normally.
>>
>> Reverse debugging is possible in the replay mode, when at least
>> one snapshot was created at the record or replay phase.
>> QEMU can use these snapshots for travelling back in time with GDB.
>>
>> Running the execution in replay mode allows using GDB reverse debugging
>> commands:
>>   - reverse-stepi (or rsi): Steps one instruction to the past.
>>     QEMU loads on of the prior snapshots and proceeds to the desired
>>     instruction forward. When that step is reaches, execution stops.
>>   - reverse-continue (or rc): Runs execution "backwards".
>>     QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
>>     and replaying the execution. Then QEMU loads snapshots again and
>>     replays to the latest breakpoint. When there are no breakpoints in
>>     the examined section of the execution, QEMU finds one more snapshot
>>     and tries again. After the first snapshot is processed, execution
>>     stops at this snapshot.
>>
>> The set of patches include the following modifications:
>>   - gdbstub update for reverse debugging support
>>   - functions that automatically perform reverse step and reverse
>>     continue operations
>>   - hmp/qmp commands for manipulating the replay process
>>   - improvement of the snapshotting for saving the execution step
>>     in the snapshot parameters
>>   - avocado-based acceptance tests for reverse debugging
>>
>> The patches are available in the repository:
>> https://github.com/ispras/qemu/tree/rr-200901
> 
> Hi Pavel,
> 
> I'm still seeing failures in "make check-block":
> 
> https://gitlab.com/bonzini/qemu/-/jobs/769653852

Please, find the new version.
The patches 3 and 4 were updated.

However, is there any reason that not all the tests are executed?
I ran 'check -qcow2' without list of the tests, and encountered some 
problems with test 286 (that are fixed now).
But this test was not executed in your gitlab repository and with 'make 
check' on my local machine.

> 
> Paolo
> 
>> v6 changes:
>>   - removed passing err variable without checking it's value after
>> v5 changes:
>>   - disabled reverse debugging tests for gitlab-based testing
>>     due to the unidentified timeout problem
>> v4 changes:
>>   - added VM snapshot creation on gdb connect (suggested by Alex Bennée)
>>   - removed useless calls to error_free
>>   - updated poll interrupt processing
>>   - minor changes
>> v3 changes:
>>   - rebased to support the new build system
>>   - bumped avocado framework version for using fixed remote gdb client
>> v2 changes:
>>   - rebased to the latest upstream version
>>   - fixed replaying of the POLL interrupts after the latest debug changes
>>
>> ---
>>
>> Pavel Dovgaluk (11):
>>        replay: provide an accessor for rr filename
>>        qcow2: introduce icount field for snapshots
>>        qapi: introduce replay.json for record/replay-related stuff
>>        replay: introduce info hmp/qmp command
>>        replay: introduce breakpoint at the specified step
>>        replay: implement replay-seek command
>>        replay: flush rr queue before loading the vmstate
>>        gdbstub: add reverse step support in replay mode
>>        gdbstub: add reverse continue support in replay mode
>>        replay: describe reverse debugging in docs/replay.txt
>>        tests/acceptance: add reverse debugging test
>>
>> Pavel Dovgalyuk (3):
>>        replay: don't record interrupt poll
>>        migration: introduce icount field for snapshots
>>        replay: create temporary snapshot at debugger connection
>>
>>
>>   MAINTAINERS                           |    2
>>   accel/tcg/cpu-exec.c                  |   21 ++
>>   accel/tcg/translator.c                |    1
>>   block/qapi.c                          |   18 +-
>>   block/qcow2-snapshot.c                |    9 +
>>   block/qcow2.h                         |    3
>>   blockdev.c                            |   10 +
>>   docs/interop/qcow2.txt                |    5
>>   docs/replay.txt                       |   46 +++++
>>   exec.c                                |    8 +
>>   gdbstub.c                             |   64 ++++++
>>   hmp-commands-info.hx                  |   11 +
>>   hmp-commands.hx                       |   50 +++++
>>   include/block/snapshot.h              |    1
>>   include/monitor/hmp.h                 |    4
>>   include/sysemu/replay.h               |   26 +++
>>   migration/savevm.c                    |   17 +-
>>   qapi/block-core.json                  |   11 +
>>   qapi/meson.build                      |    1
>>   qapi/misc.json                        |   18 --
>>   qapi/qapi-schema.json                 |    1
>>   qapi/replay.json                      |  121 ++++++++++++
>>   replay/meson.build                    |    1
>>   replay/replay-debugging.c             |  332 +++++++++++++++++++++++++++++++++
>>   replay/replay-events.c                |    4
>>   replay/replay-internal.h              |    6 -
>>   replay/replay.c                       |   22 ++
>>   softmmu/cpus.c                        |   19 ++
>>   stubs/replay.c                        |   15 +
>>   tests/acceptance/reverse_debugging.py |  208 +++++++++++++++++++++
>>   tests/qemu-iotests/267.out            |   48 ++---
>>   31 files changed, 1039 insertions(+), 64 deletions(-)
>>   create mode 100644 qapi/replay.json
>>   create mode 100644 replay/replay-debugging.c
>>   create mode 100644 tests/acceptance/reverse_debugging.py
>>
>> --
>> Pavel Dovgalyuk
>>
>