From patchwork Sun Jun 12 06:51:04 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fam Zheng X-Patchwork-Id: 9171351 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9A9CF60574 for ; Sun, 12 Jun 2016 06:51:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 809D921BED for ; Sun, 12 Jun 2016 06:51:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 744D827C0C; Sun, 12 Jun 2016 06:51:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A59C621BED for ; Sun, 12 Jun 2016 06:51:38 +0000 (UTC) Received: from localhost ([::1]:49965 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bBzFB-0004oW-MY for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 Jun 2016 02:51:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48287) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bBzEw-0004nq-It for qemu-devel@nongnu.org; Sun, 12 Jun 2016 02:51:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bBzEu-0004VJ-BT for qemu-devel@nongnu.org; Sun, 12 Jun 2016 02:51:21 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55192) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bBzEo-0004Un-HJ; Sun, 12 Jun 2016 02:51:14 -0400 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D9A26C05091E; Sun, 12 Jun 2016 06:51:13 +0000 (UTC) Received: from ad.usersys.redhat.com (dhcp-15-133.nay.redhat.com [10.66.15.133]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u5C6pAQs028064; Sun, 12 Jun 2016 02:51:10 -0400 From: Fam Zheng To: qemu-devel@nongnu.org Date: Sun, 12 Jun 2016 14:51:04 +0800 Message-Id: <20160612065104.13856-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Sun, 12 Jun 2016 06:51:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2] mirror: follow AioContext change gracefully X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Fam Zheng , jjherne@linux.vnet.ibm.com, qemu-block@nongnu.org, Jeff Cody , mreitz@redhat.com, Stefan Hajnoczi , Paolo Bonzini Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Stefan Hajnoczi When dataplane is enabled or disabled the drive switches to a new AioContext. The mirror block job must also move to the new AioContext so that drive accesses are always made within its AioContext. This patch partially achieves that by draining target and source requests to reach a quiescent point. The job is resumed in the new AioContext after moving s->target into the new AioContext. The quiesce_requested flag is added to deal with yield points in block_job_sleep_ns(), bdrv_is_allocated_above(), and bdrv_get_block_status_above(). Previously they continue executing in the old AioContext. The nested aio_poll in mirror_detach_aio_context will drive the mirror coroutine upto fixed yield points, where mirror_check_for_quiesce is called. Cc: Fam Zheng Cc: Paolo Bonzini Cc: Jeff Cody Signed-off-by: Stefan Hajnoczi [Drain source as well, and add s->quiesce_requested flag. -- Fam] Signed-off-by: Fam Zheng --- v2: Picked up Stefan's RFC patch and move on towards a more complete fix. Please review! Jason: it would be nice if you could test this version again. It differs from the previous version. --- block/mirror.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/block/mirror.c b/block/mirror.c index 80fd3c7..142199a 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -63,6 +63,8 @@ typedef struct MirrorBlockJob { int ret; bool unmap; bool waiting_for_io; + bool quiesce_requested; /* temporarily detached to move AioContext, + don't do more I/O */ int target_cluster_sectors; int max_iov; } MirrorBlockJob; @@ -119,7 +121,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret) qemu_iovec_destroy(&op->qiov); g_free(op); - if (s->waiting_for_io) { + if (s->waiting_for_io && !s->quiesce_requested) { qemu_coroutine_enter(s->common.co, NULL); } } @@ -307,6 +309,14 @@ static void mirror_do_zero_or_discard(MirrorBlockJob *s, } } +static void coroutine_fn mirror_check_for_quiesce(MirrorBlockJob *s) +{ + if (s->quiesce_requested) { + s->quiesce_requested = false; + qemu_coroutine_yield(); + } +} + static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) { BlockDriverState *source = blk_bs(s->common.blk); @@ -331,6 +341,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) mirror_wait_for_io(s); } + mirror_check_for_quiesce(s); /* Find the number of consective dirty chunks following the first dirty * one, and wait for in flight requests in them. */ while (nb_chunks * sectors_per_chunk < (s->buf_size >> BDRV_SECTOR_BITS)) { @@ -442,6 +453,31 @@ static void mirror_drain(MirrorBlockJob *s) } } +static void mirror_attached_aio_context(AioContext *new_context, void *opaque) +{ + MirrorBlockJob *s = opaque; + + blk_set_aio_context(s->target, new_context); + + /* Resume execution */ + assert(!s->quiesce_requested); + if (s->waiting_for_io) { + qemu_coroutine_enter(s->common.co, NULL); + } +} + +static void mirror_detach_aio_context(void *opaque) +{ + MirrorBlockJob *s = opaque; + + /* Complete pending write requests */ + assert(!s->quiesce_requested); + s->quiesce_requested = true; + while (s->quiesce_requested || s->in_flight) { + aio_poll(blk_get_aio_context(s->common.blk), true); + } +} + typedef struct { int ret; } MirrorExitData; @@ -491,6 +527,8 @@ static void mirror_exit(BlockJob *job, void *opaque) if (replace_aio_context) { aio_context_release(replace_aio_context); } + blk_remove_aio_context_notifier(s->common.blk, mirror_attached_aio_context, + mirror_detach_aio_context, s); g_free(s->replaces); bdrv_op_unblock_all(target_bs, s->common.blocker); blk_unref(s->target); @@ -583,6 +621,7 @@ static void coroutine_fn mirror_run(void *opaque) block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, 0); } + mirror_check_for_quiesce(s); if (block_job_is_cancelled(&s->common)) { goto immediate_exit; } @@ -612,6 +651,7 @@ static void coroutine_fn mirror_run(void *opaque) goto immediate_exit; } + mirror_check_for_quiesce(s); cnt = bdrv_get_dirty_count(s->dirty_bitmap); /* s->common.offset contains the number of bytes already processed so * far, cnt is the number of dirty sectors remaining and @@ -851,6 +891,9 @@ static void mirror_start_job(BlockDriverState *bs, BlockDriverState *target, bdrv_op_block_all(target, s->common.blocker); + blk_add_aio_context_notifier(s->common.blk, mirror_attached_aio_context, + mirror_detach_aio_context, s); + s->common.co = qemu_coroutine_create(mirror_run); trace_mirror_start(bs, s, s->common.co, opaque); qemu_coroutine_enter(s->common.co, s);