From patchwork Thu Jun 9 16:25:20 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 9167413 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EA6066048F for ; Thu, 9 Jun 2016 16:27:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DBBED28338 for ; Thu, 9 Jun 2016 16:27:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE99F28359; Thu, 9 Jun 2016 16:27:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D9F5828338 for ; Thu, 9 Jun 2016 16:27:36 +0000 (UTC) Received: from localhost ([::1]:35633 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bB2nv-0008Jq-L4 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 09 Jun 2016 12:27:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43271) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bB2nS-0008H8-1R for qemu-devel@nongnu.org; Thu, 09 Jun 2016 12:27:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bB2nN-0004F4-Pk for qemu-devel@nongnu.org; Thu, 09 Jun 2016 12:27:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59674) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bB2nN-0004Eq-Hq for qemu-devel@nongnu.org; Thu, 09 Jun 2016 12:27:01 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6F52F64070; Thu, 9 Jun 2016 16:27:00 +0000 (UTC) Received: from localhost (ovpn-112-34.ams2.redhat.com [10.36.112.34]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u59GQwVW004613; Thu, 9 Jun 2016 12:26:59 -0400 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Date: Thu, 9 Jun 2016 17:25:20 +0100 Message-Id: <1465489520-27115-1-git-send-email-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 09 Jun 2016 16:27:00 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH] mirror: follow AioContext change gracefully X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Jeff Cody , Fam Zheng , Stefan Hajnoczi , jjherne@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP When dataplane is enabled or disabled the drive switches to a new AioContext. The mirror block job must also move to the new AioContext so that drive accesses are always made within its AioContext. This work-in-progress patch partially achieves that by draining s->target of in-flight write requests to reach a quiescent point. The job is resumed in the new AioContext after moving s->target into the new AioContext. Unsolved cases include block_job_sleep_ns(), bdrv_is_allocated_above(), and bdrv_get_block_status_above(). Currently they continue executing in the old AioContext. Perhaps something similar to block_job_is_cancelled() checking is necessary so the coroutine can move itself to the new AioContext. Cc: Fam Zheng Cc: Paolo Bonzini Cc: Jeff Cody Signed-off-by: Stefan Hajnoczi --- Jason: Please try this patch and see if it fixes the abort. It has fixed migration after drive_mirror for me. --- block/mirror.c | 41 +++++++++++++++++++++++++++++++++++------ 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 80fd3c7..62d948d 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -63,6 +63,7 @@ typedef struct MirrorBlockJob { int ret; bool unmap; bool waiting_for_io; + bool detached; /* temporarily detached from AioContext, don't do I/O */ int target_cluster_sectors; int max_iov; } MirrorBlockJob; @@ -119,7 +120,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret) qemu_iovec_destroy(&op->qiov); g_free(op); - if (s->waiting_for_io) { + if (s->waiting_for_io && !s->detached) { qemu_coroutine_enter(s->common.co, NULL); } } @@ -375,7 +376,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) assert(!(sector_num % sectors_per_chunk)); ret = bdrv_get_block_status_above(source, NULL, sector_num, nb_chunks * sectors_per_chunk, - &io_sectors, &file); + &io_sectors, &file); /* TODO blocking */ if (ret < 0) { io_sectors = nb_chunks * sectors_per_chunk; } @@ -442,6 +443,29 @@ static void mirror_drain(MirrorBlockJob *s) } } +static void mirror_attached_aio_context(AioContext *new_context, void *opaque) +{ + MirrorBlockJob *s = opaque; + + blk_set_aio_context(s->target, new_context); + + /* Resume execution */ + s->detached = false; + if (s->waiting_for_io) { + qemu_coroutine_enter(s->common.co, NULL); + } +} + +static void mirror_detach_aio_context(void *opaque) +{ + MirrorBlockJob *s = opaque; + + s->detached = true; + + /* Complete pending write requests */ + blk_drain(s->target); +} + typedef struct { int ret; } MirrorExitData; @@ -491,6 +515,8 @@ static void mirror_exit(BlockJob *job, void *opaque) if (replace_aio_context) { aio_context_release(replace_aio_context); } + blk_remove_aio_context_notifier(s->common.blk, mirror_attached_aio_context, + mirror_detach_aio_context, s); g_free(s->replaces); bdrv_op_unblock_all(target_bs, s->common.blocker); blk_unref(s->target); @@ -580,14 +606,14 @@ static void coroutine_fn mirror_run(void *opaque) if (now - last_pause_ns > SLICE_TIME) { last_pause_ns = now; - block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, 0); + block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, 0); /* TODO blocking */ } if (block_job_is_cancelled(&s->common)) { goto immediate_exit; } - ret = bdrv_is_allocated_above(bs, base, sector_num, nb_sectors, &n); + ret = bdrv_is_allocated_above(bs, base, sector_num, nb_sectors, &n); /* TODO blocking */ if (ret < 0) { goto immediate_exit; @@ -680,13 +706,13 @@ static void coroutine_fn mirror_run(void *opaque) ret = 0; trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); if (!s->synced) { - block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns); + block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns); /* TODO blocking */ if (block_job_is_cancelled(&s->common)) { break; } } else if (!should_complete) { delay_ns = (s->in_flight == 0 && cnt == 0 ? SLICE_TIME : 0); - block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns); + block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns); /* TODO blocking */ } else if (cnt == 0) { /* The two disks are in sync. Exit and report successful * completion. @@ -851,6 +877,9 @@ static void mirror_start_job(BlockDriverState *bs, BlockDriverState *target, bdrv_op_block_all(target, s->common.blocker); + blk_add_aio_context_notifier(s->common.blk, mirror_attached_aio_context, + mirror_detach_aio_context, s); + s->common.co = qemu_coroutine_create(mirror_run); trace_mirror_start(bs, s, s->common.co, opaque); qemu_coroutine_enter(s->common.co, s);