| Message ID | 20180907161520.26349-13-kwolf@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Fix some jobs/drain/aio_poll related hangs | expand |
On Fri, 09/07 18:15, Kevin Wolf wrote: > Block jobs claim in .drained_poll() that they are in a quiescent state > as soon as job->deferred_to_main_loop is true. This is obviously wrong, > they still have a completion BH to run. We only get away with this > because commit 91af091f923 added an unconditional aio_poll(false) to the > drain functions, but this is bypassing the regular drain mechanisms. > > However, just removing this and telling that the job is still active > doesn't work either: The completion callbacks themselves call drain > functions (directly, or indirectly with bdrv_reopen), so they would > deadlock then. > > As a better lie, tell that the job is active as long as the BH is > pending, but falsely call it quiescent from the point in the BH when the > completion callback is called. At this point, nested drain calls won't > deadlock because they ignore the job, and outer drains will wait for the > job to really reach a quiescent state because the callback is already > running. > > Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com>
diff --git a/include/qemu/job.h b/include/qemu/job.h index 8ac48dbd28..5a49d6f9a5 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -76,6 +76,9 @@ typedef struct Job { * Set to false by the job while the coroutine has yielded and may be * re-entered by job_enter(). There may still be I/O or event loop activity * pending. Accessed under block_job_mutex (in blockjob.c). + * + * When the job is deferred to the main loop, busy is true as long as the + * bottom half is still pending. */ bool busy; diff --git a/blockjob.c b/blockjob.c index 8d27e8e1ea..617d86fe93 100644 --- a/blockjob.c +++ b/blockjob.c @@ -164,7 +164,7 @@ static bool child_job_drained_poll(BdrvChild *c) /* An inactive or completed job doesn't have any pending requests. Jobs * with !job->busy are either already paused or have a pause point after * being reentered, so no job driver code will run before they pause. */ - if (!job->busy || job_is_completed(job) || job->deferred_to_main_loop) { + if (!job->busy || job_is_completed(job)) { return false; } diff --git a/job.c b/job.c index 8480eda188..d05795843b 100644 --- a/job.c +++ b/job.c @@ -978,6 +978,13 @@ static void job_defer_to_main_loop_bh(void *opaque) AioContext *aio_context = job->aio_context; aio_context_acquire(aio_context); + + /* This is a lie, we're not quiescent, but still doing the completion + * callbacks. However, completion callbacks tend to involve operations that + * drain block nodes, and if .drained_poll still returned true, we would + * deadlock. */ + job->busy = false; + data->fn(data->job, data->opaque); aio_context_release(aio_context); @@ -991,6 +998,7 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque) data->fn = fn; data->opaque = opaque; job->deferred_to_main_loop = true; + job->busy = true; aio_bh_schedule_oneshot(qemu_get_aio_context(), job_defer_to_main_loop_bh, data);
Block jobs claim in .drained_poll() that they are in a quiescent state as soon as job->deferred_to_main_loop is true. This is obviously wrong, they still have a completion BH to run. We only get away with this because commit 91af091f923 added an unconditional aio_poll(false) to the drain functions, but this is bypassing the regular drain mechanisms. However, just removing this and telling that the job is still active doesn't work either: The completion callbacks themselves call drain functions (directly, or indirectly with bdrv_reopen), so they would deadlock then. As a better lie, tell that the job is active as long as the BH is pending, but falsely call it quiescent from the point in the BH when the completion callback is called. At this point, nested drain calls won't deadlock because they ignore the job, and outer drains will wait for the job to really reach a quiescent state because the callback is already running. Signed-off-by: Kevin Wolf <kwolf@redhat.com> --- include/qemu/job.h | 3 +++ blockjob.c | 2 +- job.c | 8 ++++++++ 3 files changed, 12 insertions(+), 1 deletion(-)