From patchwork Thu Sep 20 16:19:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEB4114BD for ; Thu, 20 Sep 2018 16:22:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8F392DDB5 for ; Thu, 20 Sep 2018 16:22:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF5872E053; Thu, 20 Sep 2018 16:22:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4B4BA2E12F for ; Thu, 20 Sep 2018 16:22:00 +0000 (UTC) Received: from localhost ([::1]:51948 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31iJ-0005zK-3d for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:21:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45455) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31gg-0004b6-MB for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31gf-0005VY-Rx for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:1923) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gd-0005RM-13; Thu, 20 Sep 2018 12:20:15 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4FB9A30014B2; Thu, 20 Sep 2018 16:20:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43E4687E2C; Thu, 20 Sep 2018 16:20:11 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:40 +0200 Message-Id: <20180920161958.27453-2-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 20 Sep 2018 16:20:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 01/19] job: Fix missing locking due to mismerge X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP job_completed() had a problem with double locking that was recently fixed independently by two different commits: "job: Fix nested aio_poll() hanging in job_txn_apply" "jobs: add exit shim" One fix removed the first aio_context_acquire(), the other fix removed the other one. Now we have a bug again and the code is run without any locking. Add it back in one of the places. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: John Snow --- job.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/job.c b/job.c index 82b46929bd..5c4e84f007 100644 --- a/job.c +++ b/job.c @@ -847,7 +847,11 @@ static void job_completed(Job *job) static void job_exit(void *opaque) { Job *job = (Job *)opaque; + AioContext *ctx = job->aio_context; + + aio_context_acquire(ctx); job_completed(job); + aio_context_release(ctx); } /** From patchwork Thu Sep 20 16:19:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26CCD14BD for ; Thu, 20 Sep 2018 16:22:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1658F2E100 for ; Thu, 20 Sep 2018 16:22:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1419D2E111; Thu, 20 Sep 2018 16:22:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6FA162E070 for ; Thu, 20 Sep 2018 16:22:12 +0000 (UTC) Received: from localhost ([::1]:51949 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31iV-0006AD-HA for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:22:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45501) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31gi-0004d4-TF for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31gh-0005Yr-KE for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49510) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31ge-0005Ry-V1; Thu, 20 Sep 2018 12:20:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 566683001DCA; Thu, 20 Sep 2018 16:20:16 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D63585A33; Thu, 20 Sep 2018 16:20:14 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:41 +0200 Message-Id: <20180920161958.27453-3-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Thu, 20 Sep 2018 16:20:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 02/19] blockjob: Wake up BDS when job becomes idle X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP In the context of draining a BDS, the .drained_poll callback of block jobs is called. If this returns true (i.e. there is still some activity pending), the drain operation may call aio_poll() with blocking=true to wait for completion. As soon as the pending activity is completed and the job finally arrives in a quiescent state (i.e. its coroutine either yields with busy=false or terminates), the block job must notify the aio_poll() loop to wake up, otherwise we get a deadlock if both are running in different threads. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng Reviewed-by: Max Reitz --- include/block/blockjob.h | 13 +++++++++++++ include/qemu/job.h | 3 +++ blockjob.c | 18 ++++++++++++++++++ job.c | 7 +++++++ 4 files changed, 41 insertions(+) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 32c00b7dc0..2290bbb824 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -70,6 +70,9 @@ typedef struct BlockJob { /** Called when the job transitions to READY */ Notifier ready_notifier; + /** Called when the job coroutine yields or terminates */ + Notifier idle_notifier; + /** BlockDriverStates that are involved in this block job */ GSList *nodes; } BlockJob; @@ -119,6 +122,16 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, void block_job_remove_all_bdrv(BlockJob *job); /** + * block_job_wakeup_all_bdrv: + * @job: The block job + * + * Calls bdrv_wakeup() for all BlockDriverStates that have been added to the + * job. This function is to be called whenever child_job_drained_poll() would + * go from true to false to notify waiting drain requests. + */ +void block_job_wakeup_all_bdrv(BlockJob *job); + +/** * block_job_set_speed: * @job: The job to set the speed for. * @speed: The new value diff --git a/include/qemu/job.h b/include/qemu/job.h index 5cb0681834..b4a784d3cc 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -156,6 +156,9 @@ typedef struct Job { /** Notifiers called when the job transitions to READY */ NotifierList on_ready; + /** Notifiers called when the job coroutine yields or terminates */ + NotifierList on_idle; + /** Element of the list of jobs */ QLIST_ENTRY(Job) job_list; diff --git a/blockjob.c b/blockjob.c index be5903aa96..8d27e8e1ea 100644 --- a/blockjob.c +++ b/blockjob.c @@ -221,6 +221,22 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, return 0; } +void block_job_wakeup_all_bdrv(BlockJob *job) +{ + GSList *l; + + for (l = job->nodes; l; l = l->next) { + BdrvChild *c = l->data; + bdrv_wakeup(c->bs); + } +} + +static void block_job_on_idle(Notifier *n, void *opaque) +{ + BlockJob *job = opaque; + block_job_wakeup_all_bdrv(job); +} + bool block_job_is_internal(BlockJob *job) { return (job->job.id == NULL); @@ -419,6 +435,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, job->finalize_completed_notifier.notify = block_job_event_completed; job->pending_notifier.notify = block_job_event_pending; job->ready_notifier.notify = block_job_event_ready; + job->idle_notifier.notify = block_job_on_idle; notifier_list_add(&job->job.on_finalize_cancelled, &job->finalize_cancelled_notifier); @@ -426,6 +443,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, &job->finalize_completed_notifier); notifier_list_add(&job->job.on_pending, &job->pending_notifier); notifier_list_add(&job->job.on_ready, &job->ready_notifier); + notifier_list_add(&job->job.on_idle, &job->idle_notifier); error_setg(&job->blocker, "block device is in use by block job: %s", job_type_str(&job->job)); diff --git a/job.c b/job.c index 5c4e84f007..48a767c102 100644 --- a/job.c +++ b/job.c @@ -402,6 +402,11 @@ static void job_event_ready(Job *job) notifier_list_notify(&job->on_ready, job); } +static void job_event_idle(Job *job) +{ + notifier_list_notify(&job->on_idle, job); +} + void job_enter_cond(Job *job, bool(*fn)(Job *job)) { if (!job_started(job)) { @@ -447,6 +452,7 @@ static void coroutine_fn job_do_yield(Job *job, uint64_t ns) timer_mod(&job->sleep_timer, ns); } job->busy = false; + job_event_idle(job); job_unlock(); qemu_coroutine_yield(); @@ -865,6 +871,7 @@ static void coroutine_fn job_co_entry(void *opaque) assert(job && job->driver && job->driver->run); job_pause_point(job); job->ret = job->driver->run(job, &job->err); + job_event_idle(job); job->deferred_to_main_loop = true; aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); } From patchwork Thu Sep 20 16:19:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608081 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C4C80913 for ; Thu, 20 Sep 2018 16:25:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B401B2E145 for ; Thu, 20 Sep 2018 16:25:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A78FC2E18E; Thu, 20 Sep 2018 16:25:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5C0E92E145 for ; Thu, 20 Sep 2018 16:25:14 +0000 (UTC) Received: from localhost ([::1]:51968 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31lR-0000p1-FH for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:25:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45522) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31gk-0004dT-8S for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31gj-0005au-Gc for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47310) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gh-0005Xs-CU; Thu, 20 Sep 2018 12:20:19 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B6473003D28; Thu, 20 Sep 2018 16:20:18 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id A21CE85A33; Thu, 20 Sep 2018 16:20:16 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:42 +0200 Message-Id: <20180920161958.27453-4-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Thu, 20 Sep 2018 16:20:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 03/19] aio-wait: Increase num_waiters even in home thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Even if AIO_WAIT_WHILE() is called in the home context of the AioContext, we still want to allow the condition to change depending on other threads as long as they kick the AioWait. Specfically block jobs can be running in an I/O thread and should then be able to kick a drain in the main loop context. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng --- include/block/aio-wait.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index c85a62f798..600fad183e 100644 --- a/include/block/aio-wait.h +++ b/include/block/aio-wait.h @@ -76,6 +76,8 @@ typedef struct { bool waited_ = false; \ AioWait *wait_ = (wait); \ AioContext *ctx_ = (ctx); \ + /* Increment wait_->num_waiters before evaluating cond. */ \ + atomic_inc(&wait_->num_waiters); \ if (ctx_ && in_aio_context_home_thread(ctx_)) { \ while ((cond)) { \ aio_poll(ctx_, true); \ @@ -84,8 +86,6 @@ typedef struct { } else { \ assert(qemu_get_current_aio_context() == \ qemu_get_aio_context()); \ - /* Increment wait_->num_waiters before evaluating cond. */ \ - atomic_inc(&wait_->num_waiters); \ while ((cond)) { \ if (ctx_) { \ aio_context_release(ctx_); \ @@ -96,8 +96,8 @@ typedef struct { } \ waited_ = true; \ } \ - atomic_dec(&wait_->num_waiters); \ } \ + atomic_dec(&wait_->num_waiters); \ waited_; }) /** From patchwork Thu Sep 20 16:19:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73859161F for ; Thu, 20 Sep 2018 16:25:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63DC12DDB5 for ; Thu, 20 Sep 2018 16:25:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57EDA2E00F; Thu, 20 Sep 2018 16:25:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CC25D2DDB5 for ; Thu, 20 Sep 2018 16:25:31 +0000 (UTC) Received: from localhost ([::1]:51974 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31lj-0001CN-4U for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:25:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45599) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31gv-0004pS-IR for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31gr-0005ew-MH for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53159) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gj-0005Zr-DN; Thu, 20 Sep 2018 12:20:21 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 920F034C9; Thu, 20 Sep 2018 16:20:20 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB59885A43; Thu, 20 Sep 2018 16:20:18 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:43 +0200 Message-Id: <20180920161958.27453-5-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 20 Sep 2018 16:20:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 04/19] test-bdrv-drain: Drain with block jobs in an I/O thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This extends the existing drain test with a block job to include variants where the block job runs in a different AioContext. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng --- tests/test-bdrv-drain.c | 92 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 86 insertions(+), 6 deletions(-) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index 89ac15e88a..57da22a096 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -174,6 +174,28 @@ static void do_drain_end(enum drain_type drain_type, BlockDriverState *bs) } } +static void do_drain_begin_unlocked(enum drain_type drain_type, BlockDriverState *bs) +{ + if (drain_type != BDRV_DRAIN_ALL) { + aio_context_acquire(bdrv_get_aio_context(bs)); + } + do_drain_begin(drain_type, bs); + if (drain_type != BDRV_DRAIN_ALL) { + aio_context_release(bdrv_get_aio_context(bs)); + } +} + +static void do_drain_end_unlocked(enum drain_type drain_type, BlockDriverState *bs) +{ + if (drain_type != BDRV_DRAIN_ALL) { + aio_context_acquire(bdrv_get_aio_context(bs)); + } + do_drain_end(drain_type, bs); + if (drain_type != BDRV_DRAIN_ALL) { + aio_context_release(bdrv_get_aio_context(bs)); + } +} + static void test_drv_cb_common(enum drain_type drain_type, bool recursive) { BlockBackend *blk; @@ -785,11 +807,13 @@ BlockJobDriver test_job_driver = { }, }; -static void test_blockjob_common(enum drain_type drain_type) +static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) { BlockBackend *blk_src, *blk_target; BlockDriverState *src, *target; BlockJob *job; + IOThread *iothread = NULL; + AioContext *ctx; int ret; src = bdrv_new_open_driver(&bdrv_test, "source", BDRV_O_RDWR, @@ -797,21 +821,31 @@ static void test_blockjob_common(enum drain_type drain_type) blk_src = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); blk_insert_bs(blk_src, src, &error_abort); + if (use_iothread) { + iothread = iothread_new(); + ctx = iothread_get_aio_context(iothread); + blk_set_aio_context(blk_src, ctx); + } else { + ctx = qemu_get_aio_context(); + } + target = bdrv_new_open_driver(&bdrv_test, "target", BDRV_O_RDWR, &error_abort); blk_target = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); blk_insert_bs(blk_target, target, &error_abort); + aio_context_acquire(ctx); job = block_job_create("job0", &test_job_driver, NULL, src, 0, BLK_PERM_ALL, 0, 0, NULL, NULL, &error_abort); block_job_add_bdrv(job, "target", target, 0, BLK_PERM_ALL, &error_abort); job_start(&job->job); + aio_context_release(ctx); g_assert_cmpint(job->job.pause_count, ==, 0); g_assert_false(job->job.paused); g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ - do_drain_begin(drain_type, src); + do_drain_begin_unlocked(drain_type, src); if (drain_type == BDRV_DRAIN_ALL) { /* bdrv_drain_all() drains both src and target */ @@ -822,7 +856,14 @@ static void test_blockjob_common(enum drain_type drain_type) g_assert_true(job->job.paused); g_assert_false(job->job.busy); /* The job is paused */ - do_drain_end(drain_type, src); + do_drain_end_unlocked(drain_type, src); + + if (use_iothread) { + /* paused is reset in the I/O thread, wait for it */ + while (job->job.paused) { + aio_poll(qemu_get_aio_context(), false); + } + } g_assert_cmpint(job->job.pause_count, ==, 0); g_assert_false(job->job.paused); @@ -841,32 +882,64 @@ static void test_blockjob_common(enum drain_type drain_type) do_drain_end(drain_type, target); + if (use_iothread) { + /* paused is reset in the I/O thread, wait for it */ + while (job->job.paused) { + aio_poll(qemu_get_aio_context(), false); + } + } + g_assert_cmpint(job->job.pause_count, ==, 0); g_assert_false(job->job.paused); g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + aio_context_acquire(ctx); ret = job_complete_sync(&job->job, &error_abort); g_assert_cmpint(ret, ==, 0); + if (use_iothread) { + blk_set_aio_context(blk_src, qemu_get_aio_context()); + } + aio_context_release(ctx); + blk_unref(blk_src); blk_unref(blk_target); bdrv_unref(src); bdrv_unref(target); + + if (iothread) { + iothread_join(iothread); + } } static void test_blockjob_drain_all(void) { - test_blockjob_common(BDRV_DRAIN_ALL); + test_blockjob_common(BDRV_DRAIN_ALL, false); } static void test_blockjob_drain(void) { - test_blockjob_common(BDRV_DRAIN); + test_blockjob_common(BDRV_DRAIN, false); } static void test_blockjob_drain_subtree(void) { - test_blockjob_common(BDRV_SUBTREE_DRAIN); + test_blockjob_common(BDRV_SUBTREE_DRAIN, false); +} + +static void test_blockjob_iothread_drain_all(void) +{ + test_blockjob_common(BDRV_DRAIN_ALL, true); +} + +static void test_blockjob_iothread_drain(void) +{ + test_blockjob_common(BDRV_DRAIN, true); +} + +static void test_blockjob_iothread_drain_subtree(void) +{ + test_blockjob_common(BDRV_SUBTREE_DRAIN, true); } @@ -1338,6 +1411,13 @@ int main(int argc, char **argv) g_test_add_func("/bdrv-drain/blockjob/drain_subtree", test_blockjob_drain_subtree); + g_test_add_func("/bdrv-drain/blockjob/iothread/drain_all", + test_blockjob_iothread_drain_all); + g_test_add_func("/bdrv-drain/blockjob/iothread/drain", + test_blockjob_iothread_drain); + g_test_add_func("/bdrv-drain/blockjob/iothread/drain_subtree", + test_blockjob_iothread_drain_subtree); + g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); g_test_add_func("/bdrv-drain/detach/drain_all", test_detach_by_drain_all); g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); From patchwork Thu Sep 20 16:19:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608051 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23F106CB for ; Thu, 20 Sep 2018 16:22:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14F6A2E018 for ; Thu, 20 Sep 2018 16:22:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 084BD2E08C; Thu, 20 Sep 2018 16:22:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A4B7E2E082 for ; Thu, 20 Sep 2018 16:22:17 +0000 (UTC) Received: from localhost ([::1]:51951 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31ia-0006EP-PG for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:22:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45659) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31gz-0004qX-Mk for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31gv-0005mI-Op for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52481) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gm-0005by-26; Thu, 20 Sep 2018 12:20:24 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AC7F43164981; Thu, 20 Sep 2018 16:20:22 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAD8785A33; Thu, 20 Sep 2018 16:20:20 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:44 +0200 Message-Id: <20180920161958.27453-6-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 20 Sep 2018 16:20:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 05/19] test-blockjob: Acquire AioContext around job_cancel_sync() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP All callers in QEMU proper hold the AioContext lock when calling job_finish_sync(). test-blockjob should do the same when it calls the function indirectly through job_cancel_sync(). Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng --- include/qemu/job.h | 6 ++++++ tests/test-blockjob.c | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index b4a784d3cc..63c60ef1ba 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -524,6 +524,8 @@ void job_user_cancel(Job *job, bool force, Error **errp); * * Returns the return value from the job if the job actually completed * during the call, or -ECANCELED if it was canceled. + * + * Callers must hold the AioContext lock of job->aio_context. */ int job_cancel_sync(Job *job); @@ -541,6 +543,8 @@ void job_cancel_sync_all(void); * function). * * Returns the return value from the job. + * + * Callers must hold the AioContext lock of job->aio_context. */ int job_complete_sync(Job *job, Error **errp); @@ -566,6 +570,8 @@ void job_dismiss(Job **job, Error **errp); * * Returns 0 if the job is successfully completed, -ECANCELED if the job was * cancelled before completing, and -errno in other error cases. + * + * Callers must hold the AioContext lock of job->aio_context. */ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp); diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c index de4c1c20aa..652d1e8359 100644 --- a/tests/test-blockjob.c +++ b/tests/test-blockjob.c @@ -223,6 +223,10 @@ static void cancel_common(CancelJob *s) BlockJob *job = &s->common; BlockBackend *blk = s->blk; JobStatus sts = job->job.status; + AioContext *ctx; + + ctx = job->job.aio_context; + aio_context_acquire(ctx); job_cancel_sync(&job->job); if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { @@ -232,6 +236,8 @@ static void cancel_common(CancelJob *s) assert(job->job.status == JOB_STATUS_NULL); job_unref(&job->job); destroy_blk(blk); + + aio_context_release(ctx); } static void test_cancel_created(void) From patchwork Thu Sep 20 16:19:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608135 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13B43913 for ; Thu, 20 Sep 2018 16:28:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 046BC2AC15 for ; Thu, 20 Sep 2018 16:28:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EC9B02DFD0; Thu, 20 Sep 2018 16:28:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9B0552AC15 for ; Thu, 20 Sep 2018 16:28:31 +0000 (UTC) Received: from localhost ([::1]:51994 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31oc-0004LU-M8 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:28:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45766) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h3-0004sC-QY for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h2-0005rg-ST for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45656) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gp-0005ca-RI; Thu, 20 Sep 2018 12:20:27 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CA6681E308; Thu, 20 Sep 2018 16:20:24 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 01E2485A33; Thu, 20 Sep 2018 16:20:22 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:45 +0200 Message-Id: <20180920161958.27453-7-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 20 Sep 2018 16:20:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 06/19] job: Use AIO_WAIT_WHILE() in job_finish_sync() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP job_finish_sync() needs to release the AioContext lock of the job before calling aio_poll(). Otherwise, callbacks called by aio_poll() would possibly take the lock a second time and run into a deadlock with a nested AIO_WAIT_WHILE() call. Also, job_drain() without aio_poll() isn't necessarily enough to make progress on a job, it could depend on bottom halves to be executed. Combine both open-coded while loops into a single AIO_WAIT_WHILE() call that solves both of these problems. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng Reviewed-by: Max Reitz --- job.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/job.c b/job.c index 48a767c102..fa74558ba0 100644 --- a/job.c +++ b/job.c @@ -29,6 +29,7 @@ #include "qemu/job.h" #include "qemu/id.h" #include "qemu/main-loop.h" +#include "block/aio-wait.h" #include "trace-root.h" #include "qapi/qapi-events-job.h" @@ -962,6 +963,7 @@ void job_complete(Job *job, Error **errp) int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) { Error *local_err = NULL; + AioWait dummy_wait = {}; int ret; job_ref(job); @@ -974,14 +976,10 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) job_unref(job); return -EBUSY; } - /* job_drain calls job_enter, and it should be enough to induce progress - * until the job completes or moves to the main thread. */ - while (!job->deferred_to_main_loop && !job_is_completed(job)) { - job_drain(job); - } - while (!job_is_completed(job)) { - aio_poll(qemu_get_aio_context(), true); - } + + AIO_WAIT_WHILE(&dummy_wait, job->aio_context, + (job_drain(job), !job_is_completed(job))); + ret = (job_is_cancelled(job) && job->ret == 0) ? -ECANCELED : job->ret; job_unref(job); return ret; From patchwork Thu Sep 20 16:19:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608163 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC3E415A6 for ; Thu, 20 Sep 2018 16:32:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC0F52DE87 for ; Thu, 20 Sep 2018 16:32:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BCFCE2DFF0; Thu, 20 Sep 2018 16:32:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 52A382DE87 for ; Thu, 20 Sep 2018 16:32:03 +0000 (UTC) Received: from localhost ([::1]:52015 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31s2-00070u-A9 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:32:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45895) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h9-0004ze-CP for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h5-0005v8-Lj for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46832) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gr-0005dC-NB; Thu, 20 Sep 2018 12:20:31 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DD9EB30C6190; Thu, 20 Sep 2018 16:20:26 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D13385A39; Thu, 20 Sep 2018 16:20:24 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:46 +0200 Message-Id: <20180920161958.27453-8-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 20 Sep 2018 16:20:26 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 07/19] test-bdrv-drain: Test AIO_WAIT_WHILE() in completion callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is a regression test for a deadlock that occurred in block job completion callbacks (via job_defer_to_main_loop) because the AioContext lock was taken twice: once in job_finish_sync() and then again in job_defer_to_main_loop_bh(). This would cause AIO_WAIT_WHILE() to hang. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng --- tests/test-bdrv-drain.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index 57da22a096..c3c17b9ff7 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -774,6 +774,15 @@ typedef struct TestBlockJob { bool should_complete; } TestBlockJob; +static int test_job_prepare(Job *job) +{ + TestBlockJob *s = container_of(job, TestBlockJob, common.job); + + /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ + blk_flush(s->common.blk); + return 0; +} + static int coroutine_fn test_job_run(Job *job, Error **errp) { TestBlockJob *s = container_of(job, TestBlockJob, common.job); @@ -804,6 +813,7 @@ BlockJobDriver test_job_driver = { .drain = block_job_drain, .run = test_job_run, .complete = test_job_complete, + .prepare = test_job_prepare, }, }; From patchwork Thu Sep 20 16:19:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608083 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CE2D161F for ; Thu, 20 Sep 2018 16:25:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DAC12E0B9 for ; Thu, 20 Sep 2018 16:25:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8147A2E145; Thu, 20 Sep 2018 16:25:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1A4E32E0B9 for ; Thu, 20 Sep 2018 16:25:16 +0000 (UTC) Received: from localhost ([::1]:51969 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31lU-0000s2-7G for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:25:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45779) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h4-0004sy-6P for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h2-0005rl-U9 for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41842) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gx-0005e7-Fl; Thu, 20 Sep 2018 12:20:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0577E8764A; Thu, 20 Sep 2018 16:20:29 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 35B4685A33; Thu, 20 Sep 2018 16:20:27 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:47 +0200 Message-Id: <20180920161958.27453-9-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Thu, 20 Sep 2018 16:20:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 08/19] block: Add missing locking in bdrv_co_drain_bh_cb() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP bdrv_do_drained_begin/end() assume that they are called with the AioContext lock of bs held. If we call drain functions from a coroutine with the AioContext lock held, we yield and schedule a BH to move out of coroutine context. This means that the lock for the home context of the coroutine is released and must be re-acquired in the bottom half. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- include/qemu/coroutine.h | 5 +++++ block/io.c | 15 +++++++++++++++ util/qemu-coroutine.c | 5 +++++ 3 files changed, 25 insertions(+) diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h index 6f8a487041..9801e7f5a4 100644 --- a/include/qemu/coroutine.h +++ b/include/qemu/coroutine.h @@ -90,6 +90,11 @@ void qemu_aio_coroutine_enter(AioContext *ctx, Coroutine *co); void coroutine_fn qemu_coroutine_yield(void); /** + * Get the AioContext of the given coroutine + */ +AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co); + +/** * Get the currently executing coroutine */ Coroutine *coroutine_fn qemu_coroutine_self(void); diff --git a/block/io.c b/block/io.c index 7100344c7b..914ba78f1a 100644 --- a/block/io.c +++ b/block/io.c @@ -288,6 +288,18 @@ static void bdrv_co_drain_bh_cb(void *opaque) BlockDriverState *bs = data->bs; if (bs) { + AioContext *ctx = bdrv_get_aio_context(bs); + AioContext *co_ctx = qemu_coroutine_get_aio_context(co); + + /* + * When the coroutine yielded, the lock for its home context was + * released, so we need to re-acquire it here. If it explicitly + * acquired a different context, the lock is still held and we don't + * want to lock it a second time (or AIO_WAIT_WHILE() would hang). + */ + if (ctx == co_ctx) { + aio_context_acquire(ctx); + } bdrv_dec_in_flight(bs); if (data->begin) { bdrv_do_drained_begin(bs, data->recursive, data->parent, @@ -296,6 +308,9 @@ static void bdrv_co_drain_bh_cb(void *opaque) bdrv_do_drained_end(bs, data->recursive, data->parent, data->ignore_bds_parents); } + if (ctx == co_ctx) { + aio_context_release(ctx); + } } else { assert(data->begin); bdrv_drain_all_begin(); diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c index 1ba4191b84..2295928d33 100644 --- a/util/qemu-coroutine.c +++ b/util/qemu-coroutine.c @@ -198,3 +198,8 @@ bool qemu_coroutine_entered(Coroutine *co) { return co->caller; } + +AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co) +{ + return co->ctx; +} From patchwork Thu Sep 20 16:19:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C808161F for ; Thu, 20 Sep 2018 16:28:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6BEE72DFC2 for ; Thu, 20 Sep 2018 16:28:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5DDA42DFC5; Thu, 20 Sep 2018 16:28:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0FF0D2DFC2 for ; Thu, 20 Sep 2018 16:28:23 +0000 (UTC) Received: from localhost ([::1]:51993 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31oU-0004By-6l for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:28:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45777) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h4-0004sv-5g for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h2-0005rt-Vw for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:3322) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gv-0005gk-Of; Thu, 20 Sep 2018 12:20:34 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2046230A5A75; Thu, 20 Sep 2018 16:20:31 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4EE5785A43; Thu, 20 Sep 2018 16:20:29 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:48 +0200 Message-Id: <20180920161958.27453-10-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Thu, 20 Sep 2018 16:20:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 09/19] block-backend: Add .drained_poll callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP A bdrv_drain operation must ensure that all parents are quiesced, this includes BlockBackends. Otherwise, callbacks called by requests that are completed on the BDS layer, but not quite yet on the BlockBackend layer could still create new requests. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng Reviewed-by: Max Reitz --- block/block-backend.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index fa120630be..efa8d8011c 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -121,6 +121,7 @@ static void blk_root_inherit_options(int *child_flags, QDict *child_options, abort(); } static void blk_root_drained_begin(BdrvChild *child); +static bool blk_root_drained_poll(BdrvChild *child); static void blk_root_drained_end(BdrvChild *child); static void blk_root_change_media(BdrvChild *child, bool load); @@ -294,6 +295,7 @@ static const BdrvChildRole child_root = { .get_parent_desc = blk_root_get_parent_desc, .drained_begin = blk_root_drained_begin, + .drained_poll = blk_root_drained_poll, .drained_end = blk_root_drained_end, .activate = blk_root_activate, @@ -2191,6 +2193,13 @@ static void blk_root_drained_begin(BdrvChild *child) } } +static bool blk_root_drained_poll(BdrvChild *child) +{ + BlockBackend *blk = child->opaque; + assert(blk->quiesce_counter); + return !!blk->in_flight; +} + static void blk_root_drained_end(BdrvChild *child) { BlockBackend *blk = child->opaque; From patchwork Thu Sep 20 16:19:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608173 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B87615A6 for ; Thu, 20 Sep 2018 16:34:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 196A22E0AC for ; Thu, 20 Sep 2018 16:34:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0D8EA2E0BA; Thu, 20 Sep 2018 16:34:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B10902E0AC for ; Thu, 20 Sep 2018 16:34:51 +0000 (UTC) Received: from localhost ([::1]:52031 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31ul-0001bU-1T for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:34:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45894) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h9-0004zd-CS for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h5-0005vN-Nt for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:30031) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gx-0005lz-En; Thu, 20 Sep 2018 12:20:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4148C30BAC4B; Thu, 20 Sep 2018 16:20:33 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A2B585A33; Thu, 20 Sep 2018 16:20:31 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:49 +0200 Message-Id: <20180920161958.27453-11-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Thu, 20 Sep 2018 16:20:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 10/19] block-backend: Fix potential double blk_delete() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP blk_unref() first decreases the refcount of the BlockBackend and calls blk_delete() if the refcount reaches zero. Requests can still be in flight at this point, they are only drained during blk_delete(): At this point, arbitrary callbacks can run. If any callback takes a temporary BlockBackend reference, it will first increase the refcount to 1 and then decrease it to 0 again, triggering another blk_delete(). This will cause a use-after-free crash in the outer blk_delete(). Fix it by draining the BlockBackend before decreasing to refcount to 0. Assert in blk_ref() that it never takes the first refcount (which would mean that the BlockBackend is already being deleted). Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng Reviewed-by: Max Reitz --- block/block-backend.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/block/block-backend.c b/block/block-backend.c index efa8d8011c..63eb2d10cd 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -435,6 +435,7 @@ int blk_get_refcnt(BlockBackend *blk) */ void blk_ref(BlockBackend *blk) { + assert(blk->refcnt > 0); blk->refcnt++; } @@ -447,7 +448,13 @@ void blk_unref(BlockBackend *blk) { if (blk) { assert(blk->refcnt > 0); - if (!--blk->refcnt) { + if (blk->refcnt > 1) { + blk->refcnt--; + } else { + blk_drain(blk); + /* blk_drain() cannot resurrect blk, nobody held a reference */ + assert(blk->refcnt == 1); + blk->refcnt = 0; blk_delete(blk); } } From patchwork Thu Sep 20 16:19:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608091 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E7EA913 for ; Thu, 20 Sep 2018 16:25:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F3C62DDB5 for ; Thu, 20 Sep 2018 16:25:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 532A12E00F; Thu, 20 Sep 2018 16:25:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0F2D12DDB5 for ; Thu, 20 Sep 2018 16:25:24 +0000 (UTC) Received: from localhost ([::1]:51973 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31lc-000136-4t for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:25:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45764) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h3-0004s8-OK for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h2-0005rb-RW for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:16566) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31gx-0005nT-VA; Thu, 20 Sep 2018 12:20:36 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5AF4231524C5; Thu, 20 Sep 2018 16:20:35 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87D5285A39; Thu, 20 Sep 2018 16:20:33 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:50 +0200 Message-Id: <20180920161958.27453-12-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 20 Sep 2018 16:20:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 11/19] block-backend: Decrease in_flight only after callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Request callbacks can do pretty much anything, including operations that will yield from the coroutine (such as draining the backend). In that case, a decreased in_flight would be visible to other code and could lead to a drain completing while the callback hasn't actually completed yet. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng Reviewed-by: Max Reitz Reviewed-by: Paolo Bonzini --- block/block-backend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/block-backend.c b/block/block-backend.c index 63eb2d10cd..5e5e60ff8d 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1340,8 +1340,8 @@ static const AIOCBInfo blk_aio_em_aiocb_info = { static void blk_aio_complete(BlkAioEmAIOCB *acb) { if (acb->has_returned) { - blk_dec_in_flight(acb->rwco.blk); acb->common.cb(acb->common.opaque, acb->rwco.ret); + blk_dec_in_flight(acb->rwco.blk); qemu_aio_unref(acb); } } From patchwork Thu Sep 20 16:19:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608161 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA73C15A6 for ; Thu, 20 Sep 2018 16:31:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9754E2E2CA for ; Thu, 20 Sep 2018 16:31:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 958D12E2C7; Thu, 20 Sep 2018 16:31:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1DA1C2E2D0 for ; Thu, 20 Sep 2018 16:31:39 +0000 (UTC) Received: from localhost ([::1]:52014 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31rf-0006kI-73 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:31:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45806) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h4-0004tn-Sk for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h3-0005sP-Mm for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45832) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31h0-0005ov-2B; Thu, 20 Sep 2018 12:20:38 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 714AA2552; Thu, 20 Sep 2018 16:20:37 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0B8485A44; Thu, 20 Sep 2018 16:20:35 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:51 +0200 Message-Id: <20180920161958.27453-13-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 20 Sep 2018 16:20:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 12/19] blockjob: Lie better in child_job_drained_poll() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Block jobs claim in .drained_poll() that they are in a quiescent state as soon as job->deferred_to_main_loop is true. This is obviously wrong, they still have a completion BH to run. We only get away with this because commit 91af091f923 added an unconditional aio_poll(false) to the drain functions, but this is bypassing the regular drain mechanisms. However, just removing this and telling that the job is still active doesn't work either: The completion callbacks themselves call drain functions (directly, or indirectly with bdrv_reopen), so they would deadlock then. As a better lie, tell that the job is active as long as the BH is pending, but falsely call it quiescent from the point in the BH when the completion callback is called. At this point, nested drain calls won't deadlock because they ignore the job, and outer drains will wait for the job to really reach a quiescent state because the callback is already running. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- include/qemu/job.h | 3 +++ blockjob.c | 2 +- job.c | 11 ++++++++++- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index 63c60ef1ba..9e7cd1e4a0 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -76,6 +76,9 @@ typedef struct Job { * Set to false by the job while the coroutine has yielded and may be * re-entered by job_enter(). There may still be I/O or event loop activity * pending. Accessed under block_job_mutex (in blockjob.c). + * + * When the job is deferred to the main loop, busy is true as long as the + * bottom half is still pending. */ bool busy; diff --git a/blockjob.c b/blockjob.c index 8d27e8e1ea..617d86fe93 100644 --- a/blockjob.c +++ b/blockjob.c @@ -164,7 +164,7 @@ static bool child_job_drained_poll(BdrvChild *c) /* An inactive or completed job doesn't have any pending requests. Jobs * with !job->busy are either already paused or have a pause point after * being reentered, so no job driver code will run before they pause. */ - if (!job->busy || job_is_completed(job) || job->deferred_to_main_loop) { + if (!job->busy || job_is_completed(job)) { return false; } diff --git a/job.c b/job.c index fa74558ba0..00a1cd128d 100644 --- a/job.c +++ b/job.c @@ -857,7 +857,16 @@ static void job_exit(void *opaque) AioContext *ctx = job->aio_context; aio_context_acquire(ctx); + + /* This is a lie, we're not quiescent, but still doing the completion + * callbacks. However, completion callbacks tend to involve operations that + * drain block nodes, and if .drained_poll still returned true, we would + * deadlock. */ + job->busy = false; + job_event_idle(job); + job_completed(job); + aio_context_release(ctx); } @@ -872,8 +881,8 @@ static void coroutine_fn job_co_entry(void *opaque) assert(job && job->driver && job->driver->run); job_pause_point(job); job->ret = job->driver->run(job, &job->err); - job_event_idle(job); job->deferred_to_main_loop = true; + job->busy = true; aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); } From patchwork Thu Sep 20 16:19:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608139 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F913913 for ; Thu, 20 Sep 2018 16:28:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3097B2DFD1 for ; Thu, 20 Sep 2018 16:28:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 249F92DFD4; Thu, 20 Sep 2018 16:28:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CC00F2DFD1 for ; Thu, 20 Sep 2018 16:28:56 +0000 (UTC) Received: from localhost ([::1]:51995 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31p2-0004dc-38 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:28:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45820) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31h5-0004uB-7e for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h4-0005t6-6X for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:64587) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31h2-0005qE-4Z; Thu, 20 Sep 2018 12:20:40 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B0A931500B2; Thu, 20 Sep 2018 16:20:39 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB1A085A33; Thu, 20 Sep 2018 16:20:37 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:52 +0200 Message-Id: <20180920161958.27453-14-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 20 Sep 2018 16:20:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 13/19] block: Remove aio_poll() in bdrv_drain_poll variants X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP bdrv_drain_poll_top_level() was buggy because it didn't release the AioContext lock of the node to be drained before calling aio_poll(). This way, callbacks called by aio_poll() would possibly take the lock a second time and run into a deadlock with a nested AIO_WAIT_WHILE() call. However, it turns out that the aio_poll() call isn't actually needed any more. It was introduced in commit 91af091f923, which is effectively reverted by this patch. The cases it was supposed to fix are now covered by bdrv_drain_poll(), which waits for block jobs to reach a quiescent state. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng Reviewed-by: Max Reitz --- block/io.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/block/io.c b/block/io.c index 914ba78f1a..8b81ff3913 100644 --- a/block/io.c +++ b/block/io.c @@ -268,10 +268,6 @@ bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, static bool bdrv_drain_poll_top_level(BlockDriverState *bs, bool recursive, BdrvChild *ignore_parent) { - /* Execute pending BHs first and check everything else only after the BHs - * have executed. */ - while (aio_poll(bs->aio_context, false)); - return bdrv_drain_poll(bs, recursive, ignore_parent, false); } @@ -511,10 +507,6 @@ static bool bdrv_drain_all_poll(void) BlockDriverState *bs = NULL; bool result = false; - /* Execute pending BHs first (may modify the graph) and check everything - * else only after the BHs have executed. */ - while (aio_poll(qemu_get_aio_context(), false)); - /* bdrv_drain_poll() can't make changes to the graph and we are holding the * main AioContext lock, so iterating bdrv_next_all_states() is safe. */ while ((bs = bdrv_next_all_states(bs))) { From patchwork Thu Sep 20 16:19:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608129 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B94C1913 for ; Thu, 20 Sep 2018 16:28:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A51202DFC1 for ; Thu, 20 Sep 2018 16:28:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 96E552DFC4; Thu, 20 Sep 2018 16:28:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 46BF82DFC1 for ; Thu, 20 Sep 2018 16:28:14 +0000 (UTC) Received: from localhost ([::1]:51992 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31oL-00045d-3v for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:28:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45941) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31hA-00051j-Us for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31h9-00063t-By for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:25242) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31h4-0005sY-B4; Thu, 20 Sep 2018 12:20:42 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AAB8D81DE3; Thu, 20 Sep 2018 16:20:41 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id D46EE85A33; Thu, 20 Sep 2018 16:20:39 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:53 +0200 Message-Id: <20180920161958.27453-15-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 20 Sep 2018 16:20:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 14/19] test-bdrv-drain: Test nested poll in bdrv_drain_poll_top_level() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is a regression test for a deadlock that could occur in callbacks called from the aio_poll() in bdrv_drain_poll_top_level(). The AioContext lock wasn't released and therefore would be taken a second time in the callback. This would cause a possible AIO_WAIT_WHILE() in the callback to hang. Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng --- tests/test-bdrv-drain.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index c3c17b9ff7..e105c0ae84 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -636,6 +636,17 @@ static void test_iothread_aio_cb(void *opaque, int ret) qemu_event_set(&done_event); } +static void test_iothread_main_thread_bh(void *opaque) +{ + struct test_iothread_data *data = opaque; + + /* Test that the AioContext is not yet locked in a random BH that is + * executed during drain, otherwise this would deadlock. */ + aio_context_acquire(bdrv_get_aio_context(data->bs)); + bdrv_flush(data->bs); + aio_context_release(bdrv_get_aio_context(data->bs)); +} + /* * Starts an AIO request on a BDS that runs in the AioContext of iothread 1. * The request involves a BH on iothread 2 before it can complete. @@ -705,6 +716,8 @@ static void test_iothread_common(enum drain_type drain_type, int drain_thread) aio_context_acquire(ctx_a); } + aio_bh_schedule_oneshot(ctx_a, test_iothread_main_thread_bh, &data); + /* The request is running on the IOThread a. Draining its block device * will make sure that it has completed as far as the BDS is concerned, * but the drain in this thread can continue immediately after From patchwork Thu Sep 20 16:19:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608165 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7484A913 for ; Thu, 20 Sep 2018 16:32:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63F4F2DFF0 for ; Thu, 20 Sep 2018 16:32:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57ACF2E020; Thu, 20 Sep 2018 16:32:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 03A6A2DFF0 for ; Thu, 20 Sep 2018 16:32:04 +0000 (UTC) Received: from localhost ([::1]:52016 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31s4-00071p-5E for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:32:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45967) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31hC-000537-1e for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31hA-000668-Qi for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:32854) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31h6-0005vT-Cf; Thu, 20 Sep 2018 12:20:44 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C1C8633BF0; Thu, 20 Sep 2018 16:20:43 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id F27D985A33; Thu, 20 Sep 2018 16:20:41 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:54 +0200 Message-Id: <20180920161958.27453-16-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 20 Sep 2018 16:20:43 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 15/19] job: Avoid deadlocks in job_completed_txn_abort() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Amongst others, job_finalize_single() calls the .prepare/.commit/.abort callbacks of the individual job driver. Recently, their use was adapted for all block jobs so that they involve code calling AIO_WAIT_WHILE() now. Such code must be called under the AioContext lock for the respective job, but without holding any other AioContext lock. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- job.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/job.c b/job.c index 00a1cd128d..0b021867da 100644 --- a/job.c +++ b/job.c @@ -718,6 +718,7 @@ static void job_cancel_async(Job *job, bool force) static void job_completed_txn_abort(Job *job) { + AioContext *outer_ctx = job->aio_context; AioContext *ctx; JobTxn *txn = job->txn; Job *other_job; @@ -731,23 +732,26 @@ static void job_completed_txn_abort(Job *job) txn->aborting = true; job_txn_ref(txn); - /* We are the first failed job. Cancel other jobs. */ - QLIST_FOREACH(other_job, &txn->jobs, txn_list) { - ctx = other_job->aio_context; - aio_context_acquire(ctx); - } + /* We can only hold the single job's AioContext lock while calling + * job_finalize_single() because the finalization callbacks can involve + * calls of AIO_WAIT_WHILE(), which could deadlock otherwise. */ + aio_context_release(outer_ctx); /* Other jobs are effectively cancelled by us, set the status for * them; this job, however, may or may not be cancelled, depending * on the caller, so leave it. */ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { if (other_job != job) { + ctx = other_job->aio_context; + aio_context_acquire(ctx); job_cancel_async(other_job, false); + aio_context_release(ctx); } } while (!QLIST_EMPTY(&txn->jobs)) { other_job = QLIST_FIRST(&txn->jobs); ctx = other_job->aio_context; + aio_context_acquire(ctx); if (!job_is_completed(other_job)) { assert(job_is_cancelled(other_job)); job_finish_sync(other_job, NULL, NULL); @@ -756,6 +760,8 @@ static void job_completed_txn_abort(Job *job) aio_context_release(ctx); } + aio_context_acquire(outer_ctx); + job_txn_unref(txn); } From patchwork Thu Sep 20 16:19:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608185 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3140E913 for ; Thu, 20 Sep 2018 16:37:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 213822E056 for ; Thu, 20 Sep 2018 16:37:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 150F22E0D5; Thu, 20 Sep 2018 16:37:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 76B122E056 for ; Thu, 20 Sep 2018 16:37:46 +0000 (UTC) Received: from localhost ([::1]:52052 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31xZ-0003zs-PO for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:37:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45999) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31hE-00056L-Mv for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31hB-00066w-Rj for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39975) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31h8-00060i-GP; Thu, 20 Sep 2018 12:20:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E13A2308403C; Thu, 20 Sep 2018 16:20:45 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A44185A43; Thu, 20 Sep 2018 16:20:43 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:55 +0200 Message-Id: <20180920161958.27453-17-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Thu, 20 Sep 2018 16:20:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 16/19] test-bdrv-drain: AIO_WAIT_WHILE() in job .commit/.abort X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This adds tests for calling AIO_WAIT_WHILE() in the .commit and .abort callbacks. Both reasons why .abort could be called for a single job are tested: Either .run or .prepare could return an error. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- tests/test-bdrv-drain.c | 116 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 104 insertions(+), 12 deletions(-) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index e105c0ae84..bbc56a055f 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -784,6 +784,8 @@ static void test_iothread_drain_subtree(void) typedef struct TestBlockJob { BlockJob common; + int run_ret; + int prepare_ret; bool should_complete; } TestBlockJob; @@ -793,7 +795,23 @@ static int test_job_prepare(Job *job) /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ blk_flush(s->common.blk); - return 0; + return s->prepare_ret; +} + +static void test_job_commit(Job *job) +{ + TestBlockJob *s = container_of(job, TestBlockJob, common.job); + + /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ + blk_flush(s->common.blk); +} + +static void test_job_abort(Job *job) +{ + TestBlockJob *s = container_of(job, TestBlockJob, common.job); + + /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ + blk_flush(s->common.blk); } static int coroutine_fn test_job_run(Job *job, Error **errp) @@ -809,7 +827,7 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) job_pause_point(&s->common.job); } - return 0; + return s->run_ret; } static void test_job_complete(Job *job, Error **errp) @@ -827,14 +845,24 @@ BlockJobDriver test_job_driver = { .run = test_job_run, .complete = test_job_complete, .prepare = test_job_prepare, + .commit = test_job_commit, + .abort = test_job_abort, }, }; -static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) +enum test_job_result { + TEST_JOB_SUCCESS, + TEST_JOB_FAIL_RUN, + TEST_JOB_FAIL_PREPARE, +}; + +static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + enum test_job_result result) { BlockBackend *blk_src, *blk_target; BlockDriverState *src, *target; BlockJob *job; + TestBlockJob *tjob; IOThread *iothread = NULL; AioContext *ctx; int ret; @@ -858,9 +886,23 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) blk_insert_bs(blk_target, target, &error_abort); aio_context_acquire(ctx); - job = block_job_create("job0", &test_job_driver, NULL, src, 0, BLK_PERM_ALL, - 0, 0, NULL, NULL, &error_abort); + tjob = block_job_create("job0", &test_job_driver, NULL, src, + 0, BLK_PERM_ALL, + 0, 0, NULL, NULL, &error_abort); + job = &tjob->common; block_job_add_bdrv(job, "target", target, 0, BLK_PERM_ALL, &error_abort); + + switch (result) { + case TEST_JOB_SUCCESS: + break; + case TEST_JOB_FAIL_RUN: + tjob->run_ret = -EIO; + break; + case TEST_JOB_FAIL_PREPARE: + tjob->prepare_ret = -EIO; + break; + } + job_start(&job->job); aio_context_release(ctx); @@ -918,7 +960,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) aio_context_acquire(ctx); ret = job_complete_sync(&job->job, &error_abort); - g_assert_cmpint(ret, ==, 0); + g_assert_cmpint(ret, ==, (result == TEST_JOB_SUCCESS ? 0 : -EIO)); if (use_iothread) { blk_set_aio_context(blk_src, qemu_get_aio_context()); @@ -937,32 +979,68 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) static void test_blockjob_drain_all(void) { - test_blockjob_common(BDRV_DRAIN_ALL, false); + test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_SUCCESS); } static void test_blockjob_drain(void) { - test_blockjob_common(BDRV_DRAIN, false); + test_blockjob_common(BDRV_DRAIN, false, TEST_JOB_SUCCESS); } static void test_blockjob_drain_subtree(void) { - test_blockjob_common(BDRV_SUBTREE_DRAIN, false); + test_blockjob_common(BDRV_SUBTREE_DRAIN, false, TEST_JOB_SUCCESS); +} + +static void test_blockjob_error_drain_all(void) +{ + test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_FAIL_RUN); + test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_FAIL_PREPARE); +} + +static void test_blockjob_error_drain(void) +{ + test_blockjob_common(BDRV_DRAIN, false, TEST_JOB_FAIL_RUN); + test_blockjob_common(BDRV_DRAIN, false, TEST_JOB_FAIL_PREPARE); +} + +static void test_blockjob_error_drain_subtree(void) +{ + test_blockjob_common(BDRV_SUBTREE_DRAIN, false, TEST_JOB_FAIL_RUN); + test_blockjob_common(BDRV_SUBTREE_DRAIN, false, TEST_JOB_FAIL_PREPARE); } static void test_blockjob_iothread_drain_all(void) { - test_blockjob_common(BDRV_DRAIN_ALL, true); + test_blockjob_common(BDRV_DRAIN_ALL, true, TEST_JOB_SUCCESS); } static void test_blockjob_iothread_drain(void) { - test_blockjob_common(BDRV_DRAIN, true); + test_blockjob_common(BDRV_DRAIN, true, TEST_JOB_SUCCESS); } static void test_blockjob_iothread_drain_subtree(void) { - test_blockjob_common(BDRV_SUBTREE_DRAIN, true); + test_blockjob_common(BDRV_SUBTREE_DRAIN, true, TEST_JOB_SUCCESS); +} + +static void test_blockjob_iothread_error_drain_all(void) +{ + test_blockjob_common(BDRV_DRAIN_ALL, true, TEST_JOB_FAIL_RUN); + test_blockjob_common(BDRV_DRAIN_ALL, true, TEST_JOB_FAIL_PREPARE); +} + +static void test_blockjob_iothread_error_drain(void) +{ + test_blockjob_common(BDRV_DRAIN, true, TEST_JOB_FAIL_RUN); + test_blockjob_common(BDRV_DRAIN, true, TEST_JOB_FAIL_PREPARE); +} + +static void test_blockjob_iothread_error_drain_subtree(void) +{ + test_blockjob_common(BDRV_SUBTREE_DRAIN, true, TEST_JOB_FAIL_RUN); + test_blockjob_common(BDRV_SUBTREE_DRAIN, true, TEST_JOB_FAIL_PREPARE); } @@ -1434,6 +1512,13 @@ int main(int argc, char **argv) g_test_add_func("/bdrv-drain/blockjob/drain_subtree", test_blockjob_drain_subtree); + g_test_add_func("/bdrv-drain/blockjob/error/drain_all", + test_blockjob_error_drain_all); + g_test_add_func("/bdrv-drain/blockjob/error/drain", + test_blockjob_error_drain); + g_test_add_func("/bdrv-drain/blockjob/error/drain_subtree", + test_blockjob_error_drain_subtree); + g_test_add_func("/bdrv-drain/blockjob/iothread/drain_all", test_blockjob_iothread_drain_all); g_test_add_func("/bdrv-drain/blockjob/iothread/drain", @@ -1441,6 +1526,13 @@ int main(int argc, char **argv) g_test_add_func("/bdrv-drain/blockjob/iothread/drain_subtree", test_blockjob_iothread_drain_subtree); + g_test_add_func("/bdrv-drain/blockjob/iothread/error/drain_all", + test_blockjob_iothread_error_drain_all); + g_test_add_func("/bdrv-drain/blockjob/iothread/error/drain", + test_blockjob_iothread_error_drain); + g_test_add_func("/bdrv-drain/blockjob/iothread/error/drain_subtree", + test_blockjob_iothread_error_drain_subtree); + g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); g_test_add_func("/bdrv-drain/detach/drain_all", test_detach_by_drain_all); g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); From patchwork Thu Sep 20 16:19:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608159 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEA10913 for ; Thu, 20 Sep 2018 16:31:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD4342E263 for ; Thu, 20 Sep 2018 16:31:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB0F92E2B6; Thu, 20 Sep 2018 16:31:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 643412E263 for ; Thu, 20 Sep 2018 16:31:20 +0000 (UTC) Received: from localhost ([::1]:52013 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31rL-0006V9-ID for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:31:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46023) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31hI-0005At-Gq for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31hE-00068r-K2 for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:20:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38556) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31hA-00064x-QD; Thu, 20 Sep 2018 12:20:49 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 02019804E4; Thu, 20 Sep 2018 16:20:48 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 34C2985A39; Thu, 20 Sep 2018 16:20:46 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:56 +0200 Message-Id: <20180920161958.27453-18-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 20 Sep 2018 16:20:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 17/19] test-bdrv-drain: Fix outdated comments X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Commit 89bd030533e changed the test case from using job_sleep_ns() to using qemu_co_sleep_ns() instead. Also, block_job_sleep_ns() became job_sleep_ns() in commit 5d43e86e11f. In both cases, some comments in the test case were not updated. Do that now. Reported-by: Max Reitz Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/test-bdrv-drain.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index bbc56a055f..526978a8ac 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -820,9 +820,9 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) job_transition_to_ready(&s->common.job); while (!s->should_complete) { - /* Avoid block_job_sleep_ns() because it marks the job as !busy. We - * want to emulate some actual activity (probably some I/O) here so - * that drain has to wait for this acitivity to stop. */ + /* Avoid job_sleep_ns() because it marks the job as !busy. We want to + * emulate some actual activity (probably some I/O) here so that drain + * has to wait for this acitivity to stop. */ qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); job_pause_point(&s->common.job); } @@ -908,7 +908,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, g_assert_cmpint(job->job.pause_count, ==, 0); g_assert_false(job->job.paused); - g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ do_drain_begin_unlocked(drain_type, src); @@ -956,7 +956,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, g_assert_cmpint(job->job.pause_count, ==, 0); g_assert_false(job->job.paused); - g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ aio_context_acquire(ctx); ret = job_complete_sync(&job->job, &error_abort); From patchwork Thu Sep 20 16:19:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608169 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4558B913 for ; Thu, 20 Sep 2018 16:34:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32C472E051 for ; Thu, 20 Sep 2018 16:34:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25C852E0B4; Thu, 20 Sep 2018 16:34:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 497CE2E051 for ; Thu, 20 Sep 2018 16:34:45 +0000 (UTC) Received: from localhost ([::1]:52030 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31ue-0001Zi-1n for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:34:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46142) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31hU-0005Kj-D7 for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:21:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31hQ-0006Hs-4N for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:21:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55278) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31hE-00067C-FZ; Thu, 20 Sep 2018 12:20:52 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 23093750D7; Thu, 20 Sep 2018 16:20:50 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4ECFC85A33; Thu, 20 Sep 2018 16:20:48 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:57 +0200 Message-Id: <20180920161958.27453-19-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 20 Sep 2018 16:20:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 18/19] block: Use a single global AioWait X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP When draining a block node, we recurse to its parent and for subtree drains also to its children. A single AIO_WAIT_WHILE() is then used to wait for bdrv_drain_poll() to become true, which depends on all of the nodes we recursed to. However, if the respective child or parent becomes quiescent and calls bdrv_wakeup(), only the AioWait of the child/parent is checked, while AIO_WAIT_WHILE() depends on the AioWait of the original node. Fix this by using a single AioWait for all callers of AIO_WAIT_WHILE(). This may mean that the draining thread gets a few more unnecessary wakeups because an unrelated operation got completed, but we already wake it up when something _could_ have changed rather than only if it has certainly changed. Apart from that, drain is a slow path anyway. In theory it would be possible to use wakeups more selectively and still correctly, but the gains are likely not worth the additional complexity. In fact, this patch is a nice simplification for some places in the code. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- include/block/aio-wait.h | 11 +++++------ include/block/block.h | 6 +----- include/block/block_int.h | 3 --- include/block/blockjob.h | 10 ---------- block.c | 5 ----- block/block-backend.c | 11 ++++------- block/io.c | 7 ++----- blockjob.c | 13 +------------ job.c | 3 +-- util/aio-wait.c | 11 ++++++----- 10 files changed, 20 insertions(+), 60 deletions(-) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index 600fad183e..46f86f948b 100644 --- a/include/block/aio-wait.h +++ b/include/block/aio-wait.h @@ -54,9 +54,10 @@ typedef struct { unsigned num_waiters; } AioWait; +extern AioWait global_aio_wait; + /** * AIO_WAIT_WHILE: - * @wait: the aio wait object * @ctx: the aio context, or NULL if multiple aio contexts (for which the * caller does not hold a lock) are involved in the polling condition. * @cond: wait while this conditional expression is true @@ -72,9 +73,9 @@ typedef struct { * wait on conditions between two IOThreads since that could lead to deadlock, * go via the main loop instead. */ -#define AIO_WAIT_WHILE(wait, ctx, cond) ({ \ +#define AIO_WAIT_WHILE(ctx, cond) ({ \ bool waited_ = false; \ - AioWait *wait_ = (wait); \ + AioWait *wait_ = &global_aio_wait; \ AioContext *ctx_ = (ctx); \ /* Increment wait_->num_waiters before evaluating cond. */ \ atomic_inc(&wait_->num_waiters); \ @@ -102,14 +103,12 @@ typedef struct { /** * aio_wait_kick: - * @wait: the aio wait object that should re-evaluate its condition - * * Wake up the main thread if it is waiting on AIO_WAIT_WHILE(). During * synchronous operations performed in an IOThread, the main thread lets the * IOThread's event loop run, waiting for the operation to complete. A * aio_wait_kick() call will wake up the main thread. */ -void aio_wait_kick(AioWait *wait); +void aio_wait_kick(void); /** * aio_wait_bh_oneshot: diff --git a/include/block/block.h b/include/block/block.h index 4e0871aaf9..4edc1e8afa 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -410,13 +410,9 @@ void bdrv_drain_all_begin(void); void bdrv_drain_all_end(void); void bdrv_drain_all(void); -/* Returns NULL when bs == NULL */ -AioWait *bdrv_get_aio_wait(BlockDriverState *bs); - #define BDRV_POLL_WHILE(bs, cond) ({ \ BlockDriverState *bs_ = (bs); \ - AIO_WAIT_WHILE(bdrv_get_aio_wait(bs_), \ - bdrv_get_aio_context(bs_), \ + AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) int bdrv_pdiscard(BdrvChild *child, int64_t offset, int bytes); diff --git a/include/block/block_int.h b/include/block/block_int.h index 4000d2af45..92ecbd866e 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -794,9 +794,6 @@ struct BlockDriverState { unsigned int in_flight; unsigned int serialising_in_flight; - /* Kicked to signal main loop when a request completes. */ - AioWait wait; - /* counter for nested bdrv_io_plug. * Accessed with atomic ops. */ diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 2290bbb824..ede0bd8dcb 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -122,16 +122,6 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, void block_job_remove_all_bdrv(BlockJob *job); /** - * block_job_wakeup_all_bdrv: - * @job: The block job - * - * Calls bdrv_wakeup() for all BlockDriverStates that have been added to the - * job. This function is to be called whenever child_job_drained_poll() would - * go from true to false to notify waiting drain requests. - */ -void block_job_wakeup_all_bdrv(BlockJob *job); - -/** * block_job_set_speed: * @job: The job to set the speed for. * @speed: The new value diff --git a/block.c b/block.c index a381c8ece8..c298ca6a19 100644 --- a/block.c +++ b/block.c @@ -4886,11 +4886,6 @@ AioContext *bdrv_get_aio_context(BlockDriverState *bs) return bs ? bs->aio_context : qemu_get_aio_context(); } -AioWait *bdrv_get_aio_wait(BlockDriverState *bs) -{ - return bs ? &bs->wait : NULL; -} - void bdrv_coroutine_enter(BlockDriverState *bs, Coroutine *co) { aio_co_enter(bdrv_get_aio_context(bs), co); diff --git a/block/block-backend.c b/block/block-backend.c index 5e5e60ff8d..e943b2cc40 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -88,7 +88,6 @@ struct BlockBackend { * Accessed with atomic ops. */ unsigned int in_flight; - AioWait wait; }; typedef struct BlockBackendAIOCB { @@ -1299,7 +1298,7 @@ static void blk_inc_in_flight(BlockBackend *blk) static void blk_dec_in_flight(BlockBackend *blk) { atomic_dec(&blk->in_flight); - aio_wait_kick(&blk->wait); + aio_wait_kick(); } static void error_callback_bh(void *opaque) @@ -1600,9 +1599,8 @@ void blk_drain(BlockBackend *blk) } /* We may have -ENOMEDIUM completions in flight */ - AIO_WAIT_WHILE(&blk->wait, - blk_get_aio_context(blk), - atomic_mb_read(&blk->in_flight) > 0); + AIO_WAIT_WHILE(blk_get_aio_context(blk), + atomic_mb_read(&blk->in_flight) > 0); if (bs) { bdrv_drained_end(bs); @@ -1621,8 +1619,7 @@ void blk_drain_all(void) aio_context_acquire(ctx); /* We may have -ENOMEDIUM completions in flight */ - AIO_WAIT_WHILE(&blk->wait, ctx, - atomic_mb_read(&blk->in_flight) > 0); + AIO_WAIT_WHILE(ctx, atomic_mb_read(&blk->in_flight) > 0); aio_context_release(ctx); } diff --git a/block/io.c b/block/io.c index 8b81ff3913..bd9d688f8b 100644 --- a/block/io.c +++ b/block/io.c @@ -38,8 +38,6 @@ /* Maximum bounce buffer for copy-on-read and write zeroes, in bytes */ #define MAX_BOUNCE_BUFFER (32768 << BDRV_SECTOR_BITS) -static AioWait drain_all_aio_wait; - static void bdrv_parent_cb_resize(BlockDriverState *bs); static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, BdrvRequestFlags flags); @@ -557,7 +555,7 @@ void bdrv_drain_all_begin(void) } /* Now poll the in-flight requests */ - AIO_WAIT_WHILE(&drain_all_aio_wait, NULL, bdrv_drain_all_poll()); + AIO_WAIT_WHILE(NULL, bdrv_drain_all_poll()); while ((bs = bdrv_next_all_states(bs))) { bdrv_drain_assert_idle(bs); @@ -713,8 +711,7 @@ void bdrv_inc_in_flight(BlockDriverState *bs) void bdrv_wakeup(BlockDriverState *bs) { - aio_wait_kick(bdrv_get_aio_wait(bs)); - aio_wait_kick(&drain_all_aio_wait); + aio_wait_kick(); } void bdrv_dec_in_flight(BlockDriverState *bs) diff --git a/blockjob.c b/blockjob.c index 617d86fe93..06f2429ef0 100644 --- a/blockjob.c +++ b/blockjob.c @@ -221,20 +221,9 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, return 0; } -void block_job_wakeup_all_bdrv(BlockJob *job) -{ - GSList *l; - - for (l = job->nodes; l; l = l->next) { - BdrvChild *c = l->data; - bdrv_wakeup(c->bs); - } -} - static void block_job_on_idle(Notifier *n, void *opaque) { - BlockJob *job = opaque; - block_job_wakeup_all_bdrv(job); + aio_wait_kick(); } bool block_job_is_internal(BlockJob *job) diff --git a/job.c b/job.c index 0b021867da..ed4da6f810 100644 --- a/job.c +++ b/job.c @@ -978,7 +978,6 @@ void job_complete(Job *job, Error **errp) int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) { Error *local_err = NULL; - AioWait dummy_wait = {}; int ret; job_ref(job); @@ -992,7 +991,7 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) return -EBUSY; } - AIO_WAIT_WHILE(&dummy_wait, job->aio_context, + AIO_WAIT_WHILE(job->aio_context, (job_drain(job), !job_is_completed(job))); ret = (job_is_cancelled(job) && job->ret == 0) ? -ECANCELED : job->ret; diff --git a/util/aio-wait.c b/util/aio-wait.c index b8a8f86dba..b4877493f8 100644 --- a/util/aio-wait.c +++ b/util/aio-wait.c @@ -26,21 +26,22 @@ #include "qemu/main-loop.h" #include "block/aio-wait.h" +AioWait global_aio_wait; + static void dummy_bh_cb(void *opaque) { /* The point is to make AIO_WAIT_WHILE()'s aio_poll() return */ } -void aio_wait_kick(AioWait *wait) +void aio_wait_kick(void) { /* The barrier (or an atomic op) is in the caller. */ - if (atomic_read(&wait->num_waiters)) { + if (atomic_read(&global_aio_wait.num_waiters)) { aio_bh_schedule_oneshot(qemu_get_aio_context(), dummy_bh_cb, NULL); } } typedef struct { - AioWait wait; bool done; QEMUBHFunc *cb; void *opaque; @@ -54,7 +55,7 @@ static void aio_wait_bh(void *opaque) data->cb(data->opaque); data->done = true; - aio_wait_kick(&data->wait); + aio_wait_kick(); } void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque) @@ -67,5 +68,5 @@ void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque) assert(qemu_get_current_aio_context() == qemu_get_aio_context()); aio_bh_schedule_oneshot(ctx, aio_wait_bh, &data); - AIO_WAIT_WHILE(&data.wait, ctx, !data.done); + AIO_WAIT_WHILE(ctx, !data.done); } From patchwork Thu Sep 20 16:19:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10608179 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4A19913 for ; Thu, 20 Sep 2018 16:35:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C49182E01F for ; Thu, 20 Sep 2018 16:35:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B89E52E0AA; Thu, 20 Sep 2018 16:35:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 340412E01F for ; Thu, 20 Sep 2018 16:35:41 +0000 (UTC) Received: from localhost ([::1]:52040 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31vY-0002IK-FE for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Sep 2018 12:35:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46143) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g31hU-0005Kl-DY for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:21:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g31hQ-0006Hr-4N for qemu-devel@nongnu.org; Thu, 20 Sep 2018 12:21:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:12550) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g31hH-00068H-5Z; Thu, 20 Sep 2018 12:20:56 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3B720CD4D2; Thu, 20 Sep 2018 16:20:52 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-231.ams2.redhat.com [10.36.116.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6D3F485A43; Thu, 20 Sep 2018 16:20:50 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 20 Sep 2018 18:19:58 +0200 Message-Id: <20180920161958.27453-20-kwolf@redhat.com> In-Reply-To: <20180920161958.27453-1-kwolf@redhat.com> References: <20180920161958.27453-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 20 Sep 2018 16:20:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 19/19] test-bdrv-drain: Test draining job source child and parent X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, slp@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP For the block job drain test, don't only test draining the source and the target node, but create a backing chain for the source (source_backing <- source <- source_overlay) and test draining each of the nodes in it. When using iothreads, the source node (and therefore the job) is in a different AioContext than the drain, which happens from the main thread. This way, the main thread waits in AIO_WAIT_WHILE() for the iothread to make process and aio_wait_kick() is required to notify it. The test validates that calling bdrv_wakeup() for a child or a parent node will actually notify AIO_WAIT_WHILE() instead of letting it hang. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- tests/test-bdrv-drain.c | 75 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 67 insertions(+), 8 deletions(-) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index 526978a8ac..193b88d340 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -786,6 +786,7 @@ typedef struct TestBlockJob { BlockJob common; int run_ret; int prepare_ret; + bool running; bool should_complete; } TestBlockJob; @@ -818,12 +819,17 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) { TestBlockJob *s = container_of(job, TestBlockJob, common.job); + /* We are running the actual job code past the pause point in + * job_co_entry(). */ + s->running = true; + job_transition_to_ready(&s->common.job); while (!s->should_complete) { /* Avoid job_sleep_ns() because it marks the job as !busy. We want to * emulate some actual activity (probably some I/O) here so that drain * has to wait for this acitivity to stop. */ - qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000); + job_pause_point(&s->common.job); } @@ -856,11 +862,19 @@ enum test_job_result { TEST_JOB_FAIL_PREPARE, }; -static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, - enum test_job_result result) +enum test_job_drain_node { + TEST_JOB_DRAIN_SRC, + TEST_JOB_DRAIN_SRC_CHILD, + TEST_JOB_DRAIN_SRC_PARENT, +}; + +static void test_blockjob_common_drain_node(enum drain_type drain_type, + bool use_iothread, + enum test_job_result result, + enum test_job_drain_node drain_node) { BlockBackend *blk_src, *blk_target; - BlockDriverState *src, *target; + BlockDriverState *src, *src_backing, *src_overlay, *target, *drain_bs; BlockJob *job; TestBlockJob *tjob; IOThread *iothread = NULL; @@ -869,8 +883,30 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, src = bdrv_new_open_driver(&bdrv_test, "source", BDRV_O_RDWR, &error_abort); + src_backing = bdrv_new_open_driver(&bdrv_test, "source-backing", + BDRV_O_RDWR, &error_abort); + src_overlay = bdrv_new_open_driver(&bdrv_test, "source-overlay", + BDRV_O_RDWR, &error_abort); + + bdrv_set_backing_hd(src_overlay, src, &error_abort); + bdrv_unref(src); + bdrv_set_backing_hd(src, src_backing, &error_abort); + bdrv_unref(src_backing); + blk_src = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); - blk_insert_bs(blk_src, src, &error_abort); + blk_insert_bs(blk_src, src_overlay, &error_abort); + + switch (drain_node) { + case TEST_JOB_DRAIN_SRC: + drain_bs = src; + break; + case TEST_JOB_DRAIN_SRC_CHILD: + drain_bs = src_backing; + break; + case TEST_JOB_DRAIN_SRC_PARENT: + drain_bs = src_overlay; + break; + } if (use_iothread) { iothread = iothread_new(); @@ -906,11 +942,21 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, job_start(&job->job); aio_context_release(ctx); + if (use_iothread) { + /* job_co_entry() is run in the I/O thread, wait for the actual job + * code to start (we don't want to catch the job in the pause point in + * job_co_entry(). */ + while (!tjob->running) { + aio_poll(qemu_get_aio_context(), false); + } + } + g_assert_cmpint(job->job.pause_count, ==, 0); g_assert_false(job->job.paused); + g_assert_true(tjob->running); g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ - do_drain_begin_unlocked(drain_type, src); + do_drain_begin_unlocked(drain_type, drain_bs); if (drain_type == BDRV_DRAIN_ALL) { /* bdrv_drain_all() drains both src and target */ @@ -921,7 +967,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, g_assert_true(job->job.paused); g_assert_false(job->job.busy); /* The job is paused */ - do_drain_end_unlocked(drain_type, src); + do_drain_end_unlocked(drain_type, drain_bs); if (use_iothread) { /* paused is reset in the I/O thread, wait for it */ @@ -969,7 +1015,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, blk_unref(blk_src); blk_unref(blk_target); - bdrv_unref(src); + bdrv_unref(src_overlay); bdrv_unref(target); if (iothread) { @@ -977,6 +1023,19 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, } } +static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + enum test_job_result result) +{ + test_blockjob_common_drain_node(drain_type, use_iothread, result, + TEST_JOB_DRAIN_SRC); + test_blockjob_common_drain_node(drain_type, use_iothread, result, + TEST_JOB_DRAIN_SRC_CHILD); + if (drain_type == BDRV_SUBTREE_DRAIN) { + test_blockjob_common_drain_node(drain_type, use_iothread, result, + TEST_JOB_DRAIN_SRC_PARENT); + } +} + static void test_blockjob_drain_all(void) { test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_SUCCESS);