| Message ID | 20160301110752.10104.65295.stgit@PASHA-ISP (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 01/03/2016 12:07, Pavel Dovgalyuk wrote: > qemu_clock_warp function is called to update virtual clock when CPU > is sleeping. This function includes replay checkpoint to make execution > deterministic in icount mode. > Record/replay module flushes async event queue at checkpoints. > Some of the events (e.g., block devices operations) include interaction > with hardware. E.g., APIC polled by block devices sets one of IRQ flags. > Flag to be set depends on currently executed thread (CPU or iothread). > Therefore in replay mode we have to process the checkpoints in the same thread > as they were recorded. > qemu_clock_warp function (and its checkpoint) may be called from different > thread. This patch introduces new checkpoint which distinguished warp > checkpoint calls from different threads. > > Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru> I think we need two different kinds of "warp" behavior, one to start the warp timer (from the main loop and when a timer is set) and one to end it (from the CPUs). Then the need for two checkpoints is much clearer, though I suggestnaming them without a reference to TCG; something like CHECKPOINT_CLOCK_WARP_START and CHECKPOINT_CLOCK_WARP_ACCOUNT for example. The start would be where you call qemu_clock_warp(QEMU_CLOCK_VIRTUAL, false): if (!use_icount) { return; } if (!runstate_is_running()) { return; } if (!replay_checkpoint(CHECKPOINT_CLOCK_WARP_START)) { return; } /* I think calling icount_warp_rt here is unnecessary. */ if (!all_cpu_threads_idle()) { return; } if (qtest_enabled()) { /* When testing, qtest commands advance icount. */ return; } /* We want to use the earliest deadline from ALL vm_clocks */ clock = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL_RT); deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL); ... The end or account function, instead, would be called from tcg_exec_all() and also from icount_dummy_timer() (this is what makes the call to icount_warp_rt unnecessary above): if (!use_icount || !icount_isleep) { return; } if (!runstate_is_running()) { return; } if (!replay_checkpoint(CHECKPOINT_CLOCK_WARP_END)) { return; } timer_del(icount_warp_timer); /* * If the CPUs have been sleeping, advance QEMU_CLOCK_VIRTUAL timer now. * This ensures that the deadline for the timer is computed correctly * below. * This also makes sure that the insn counter is synchronized before * the CPU starts running, in case the CPU is woken by an event other * than the earliest QEMU_CLOCK_VIRTUAL timer. */ // ...include icount_warp_rt function here... qemu_clock_warp would only be called from qemu-timer.c, and it would be simply be if (type == QEMU_CLOCK_VIRTUAL) { qemu_start_warp_timer(); } Separating the two boundaries of the warp this way would make the code much easier to understand, and would also make the need for a new checkpoint more obvious. Paolo
> From: Paolo Bonzini [mailto:pbonzini@redhat.com] > On 01/03/2016 12:07, Pavel Dovgalyuk wrote: > > qemu_clock_warp function is called to update virtual clock when CPU > > is sleeping. This function includes replay checkpoint to make execution > > deterministic in icount mode. > > Record/replay module flushes async event queue at checkpoints. > > Some of the events (e.g., block devices operations) include interaction > > with hardware. E.g., APIC polled by block devices sets one of IRQ flags. > > Flag to be set depends on currently executed thread (CPU or iothread). > > Therefore in replay mode we have to process the checkpoints in the same thread > > as they were recorded. > > qemu_clock_warp function (and its checkpoint) may be called from different > > thread. This patch introduces new checkpoint which distinguished warp > > checkpoint calls from different threads. > > > > Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru> > > I think we need two different kinds of "warp" behavior, one to start the > warp timer (from the main loop and when a timer is set) and one to end it > (from the CPUs). > > Then the need for two checkpoints is much clearer, though I suggestnaming > them without a reference to TCG; something like CHECKPOINT_CLOCK_WARP_START > and CHECKPOINT_CLOCK_WARP_ACCOUNT for example. Thanks, this seems reasonable. > The start would be where you call qemu_clock_warp(QEMU_CLOCK_VIRTUAL, false): > > if (!use_icount) { > return; > } > if (!runstate_is_running()) { > return; > } > if (!replay_checkpoint(CHECKPOINT_CLOCK_WARP_START)) { > return; > } > /* I think calling icount_warp_rt here is unnecessary. */ > if (!all_cpu_threads_idle()) { > return; > } > if (qtest_enabled()) { > /* When testing, qtest commands advance icount. */ > return; > } > > /* We want to use the earliest deadline from ALL vm_clocks */ > clock = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL_RT); > deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL); > ... > > The end or account function, instead, would be called from tcg_exec_all() > and also from icount_dummy_timer() (this is what makes the call to > icount_warp_rt unnecessary above): Why icount_warp_rt is unnecessary? There is no code to proceed the virtual clock. Then qemu_start_warp_timer will forever setup the timer without any progress. Pavel Dovgalyuk
On 10/03/2016 10:10, Pavel Dovgalyuk wrote: >> > The start would be where you call qemu_clock_warp(QEMU_CLOCK_VIRTUAL, false): >> > >> > if (!use_icount) { >> > return; >> > } >> > if (!runstate_is_running()) { >> > return; >> > } >> > if (!replay_checkpoint(CHECKPOINT_CLOCK_WARP_START)) { >> > return; >> > } >> > /* I think calling icount_warp_rt here is unnecessary. */ >> > if (!all_cpu_threads_idle()) { >> > return; >> > } >> > if (qtest_enabled()) { >> > /* When testing, qtest commands advance icount. */ >> > return; >> > } >> > >> > /* We want to use the earliest deadline from ALL vm_clocks */ >> > clock = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL_RT); >> > deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL); >> > ... >> > >> > The end or account function, instead, would be called from tcg_exec_all() >> > and also from icount_dummy_timer() (this is what makes the call to >> > icount_warp_rt unnecessary above): > Why icount_warp_rt is unnecessary? There is no code to proceed the virtual clock. > Then qemu_start_warp_timer will forever setup the timer without any progress. If icount_warp_rt is called from icount_dummy_timer(), the virtual clock will be updated as soon as the VIRTUAL_RT clock reaches the deadline. It's a much more reasonable place to call icount_warp_rt from (if it works...). Paolo
diff --git a/cpus.c b/cpus.c index 01c9809..c2d9cfe 100644 --- a/cpus.c +++ b/cpus.c @@ -396,7 +396,7 @@ void qtest_clock_warp(int64_t dest) qemu_clock_notify(QEMU_CLOCK_VIRTUAL); } -void qemu_clock_warp(QEMUClockType type) +void qemu_clock_warp(QEMUClockType type, bool in_tcg) { int64_t clock; int64_t deadline; @@ -418,7 +418,8 @@ void qemu_clock_warp(QEMUClockType type) } /* warp clock deterministically in record/replay mode */ - if (!replay_checkpoint(CHECKPOINT_CLOCK_WARP)) { + if (!replay_checkpoint(in_tcg ? CHECKPOINT_CLOCK_WARP_TCG + : CHECKPOINT_CLOCK_WARP)) { return; } @@ -1496,7 +1497,7 @@ static void tcg_exec_all(void) int r; /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ - qemu_clock_warp(QEMU_CLOCK_VIRTUAL); + qemu_clock_warp(QEMU_CLOCK_VIRTUAL, true); if (next_cpu == NULL) { next_cpu = first_cpu; diff --git a/include/qemu/timer.h b/include/qemu/timer.h index d0946cb..c58192c 100644 --- a/include/qemu/timer.h +++ b/include/qemu/timer.h @@ -212,10 +212,11 @@ void qemu_clock_enable(QEMUClockType type, bool enabled); /** * qemu_clock_warp: * @type: the clock type + * @in_tcg: true if function is called from TCG CPU thread * * Warp a clock to a new value */ -void qemu_clock_warp(QEMUClockType type); +void qemu_clock_warp(QEMUClockType type, bool in_tcg); /** * qemu_clock_register_reset_notifier: diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h index 3c4a988..c879231 100644 --- a/include/sysemu/replay.h +++ b/include/sysemu/replay.h @@ -31,6 +31,7 @@ typedef enum ReplayClockKind ReplayClockKind; /* IDs of the checkpoints */ enum ReplayCheckpoint { CHECKPOINT_CLOCK_WARP, + CHECKPOINT_CLOCK_WARP_TCG, CHECKPOINT_RESET_REQUESTED, CHECKPOINT_SUSPEND_REQUESTED, CHECKPOINT_CLOCK_VIRTUAL, diff --git a/main-loop.c b/main-loop.c index 19beae7..cd8415f 100644 --- a/main-loop.c +++ b/main-loop.c @@ -509,7 +509,7 @@ int main_loop_wait(int nonblocking) /* CPU thread can infinitely wait for event after missing the warp */ - qemu_clock_warp(QEMU_CLOCK_VIRTUAL); + qemu_clock_warp(QEMU_CLOCK_VIRTUAL, false); qemu_clock_run_all_timers(); return ret; diff --git a/qemu-timer.c b/qemu-timer.c index e98ecc9..980fe7e 100644 --- a/qemu-timer.c +++ b/qemu-timer.c @@ -394,7 +394,7 @@ static bool timer_mod_ns_locked(QEMUTimerList *timer_list, static void timerlist_rearm(QEMUTimerList *timer_list) { /* Interrupt execution to force deadline recalculation. */ - qemu_clock_warp(timer_list->clock->type); + qemu_clock_warp(timer_list->clock->type, false); timerlist_notify(timer_list); } diff --git a/stubs/clock-warp.c b/stubs/clock-warp.c index 5ae32b9..24ae0f8 100644 --- a/stubs/clock-warp.c +++ b/stubs/clock-warp.c @@ -2,7 +2,7 @@ #include "qemu-common.h" #include "qemu/timer.h" -void qemu_clock_warp(QEMUClockType type) +void qemu_clock_warp(QEMUClockType type, bool in_tcg) { }
qemu_clock_warp function is called to update virtual clock when CPU is sleeping. This function includes replay checkpoint to make execution deterministic in icount mode. Record/replay module flushes async event queue at checkpoints. Some of the events (e.g., block devices operations) include interaction with hardware. E.g., APIC polled by block devices sets one of IRQ flags. Flag to be set depends on currently executed thread (CPU or iothread). Therefore in replay mode we have to process the checkpoints in the same thread as they were recorded. qemu_clock_warp function (and its checkpoint) may be called from different thread. This patch introduces new checkpoint which distinguished warp checkpoint calls from different threads. Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru> --- cpus.c | 7 ++++--- include/qemu/timer.h | 3 ++- include/sysemu/replay.h | 1 + main-loop.c | 2 +- qemu-timer.c | 2 +- stubs/clock-warp.c | 2 +- 6 files changed, 10 insertions(+), 7 deletions(-)