From patchwork Sun Oct 30 10:46:55 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amit Shah X-Patchwork-Id: 9404411 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2A49160586 for ; Sun, 30 Oct 2016 10:56:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19ED328FD7 for ; Sun, 30 Oct 2016 10:56:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0EE4528FD9; Sun, 30 Oct 2016 10:56:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 757EB28FD7 for ; Sun, 30 Oct 2016 10:56:16 +0000 (UTC) Received: from localhost ([::1]:57748 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c0nmh-0003EK-Lp for patchwork-qemu-devel@patchwork.kernel.org; Sun, 30 Oct 2016 06:56:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37396) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c0neA-0005HA-KB for qemu-devel@nongnu.org; Sun, 30 Oct 2016 06:47:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c0ne9-0001Fc-4c for qemu-devel@nongnu.org; Sun, 30 Oct 2016 06:47:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43152) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c0ne8-0001FW-TL for qemu-devel@nongnu.org; Sun, 30 Oct 2016 06:47:25 -0400 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 23F973B3C7; Sun, 30 Oct 2016 10:47:24 +0000 (UTC) Received: from localhost (ovpn-116-76.phx2.redhat.com [10.3.116.76]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u9UAlMWJ023849; Sun, 30 Oct 2016 06:47:23 -0400 From: Amit Shah To: Peter Maydell Date: Sun, 30 Oct 2016 16:16:55 +0530 Message-Id: <1477824430-1460-4-git-send-email-amit.shah@redhat.com> In-Reply-To: <1477824430-1460-1-git-send-email-amit.shah@redhat.com> References: <1477824430-1460-1-git-send-email-amit.shah@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Sun, 30 Oct 2016 10:47:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/18] migration: Enter into COLO mode after migration if COLO is enabled X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu list , zhang.zhanghailiang@huawei.com, "Dr. David Alan Gilbert" , Juan Quintela Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: zhanghailiang Add a new migration state: MIGRATION_STATUS_COLO. Migration source side enters this state after the first live migration successfully finished if COLO is enabled by command 'migrate_set_capability x-colo on'. We reuse migration thread, so the process of checkpointing will be handled in migration thread. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Amit Shah Signed-off-by: Amit Shah --- include/migration/colo.h | 3 +++ migration/colo.c | 29 +++++++++++++++++++++++++++++ migration/migration.c | 38 +++++++++++++++++++++++++++++++++----- migration/trace-events | 3 +++ qapi-schema.json | 4 +++- stubs/migration-colo.c | 9 +++++++++ 6 files changed, 80 insertions(+), 6 deletions(-) diff --git a/include/migration/colo.h b/include/migration/colo.h index 1c899a0..bf84b99 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -19,4 +19,7 @@ bool colo_supported(void); void colo_info_init(void); +void migrate_start_colo_process(MigrationState *s); +bool migration_in_colo_state(void); + #endif diff --git a/migration/colo.c b/migration/colo.c index d215057..f480431 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -11,9 +11,38 @@ */ #include "qemu/osdep.h" +#include "sysemu/sysemu.h" #include "migration/colo.h" +#include "trace.h" bool colo_supported(void) { return false; } + +bool migration_in_colo_state(void) +{ + MigrationState *s = migrate_get_current(); + + return (s->state == MIGRATION_STATUS_COLO); +} + +static void colo_process_checkpoint(MigrationState *s) +{ + qemu_mutex_lock_iothread(); + vm_start(); + qemu_mutex_unlock_iothread(); + trace_colo_vm_state_change("stop", "run"); + + /* TODO: COLO checkpoint savevm loop */ + +} + +void migrate_start_colo_process(MigrationState *s) +{ + qemu_mutex_unlock_iothread(); + migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_COLO); + colo_process_checkpoint(s); + qemu_mutex_lock_iothread(); +} diff --git a/migration/migration.c b/migration/migration.c index c7869e5..aff3eb4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -695,6 +695,10 @@ MigrationInfo *qmp_query_migrate(Error **errp) get_xbzrle_cache_stats(info); break; + case MIGRATION_STATUS_COLO: + info->has_status = true; + /* TODO: display COLO specific information (checkpoint info etc.) */ + break; case MIGRATION_STATUS_COMPLETED: get_xbzrle_cache_stats(info); @@ -1113,7 +1117,8 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, params.shared = has_inc && inc; if (migration_is_setup_or_active(s->state) || - s->state == MIGRATION_STATUS_CANCELLING) { + s->state == MIGRATION_STATUS_CANCELLING || + s->state == MIGRATION_STATUS_COLO) { error_setg(errp, QERR_MIGRATION_ACTIVE); return; } @@ -1661,7 +1666,11 @@ static void migration_completion(MigrationState *s, int current_active_state, if (!ret) { ret = vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); - if (ret >= 0) { + /* + * Don't mark the image with BDRV_O_INACTIVE flag if + * we will go into COLO stage later. + */ + if (ret >= 0 && !migrate_colo_enabled()) { ret = bdrv_inactivate_all(); } if (ret >= 0) { @@ -1703,8 +1712,11 @@ static void migration_completion(MigrationState *s, int current_active_state, goto fail_invalidate; } - migrate_set_state(&s->state, current_active_state, - MIGRATION_STATUS_COMPLETED); + if (!migrate_colo_enabled()) { + migrate_set_state(&s->state, current_active_state, + MIGRATION_STATUS_COMPLETED); + } + return; fail_invalidate: @@ -1749,6 +1761,7 @@ static void *migration_thread(void *opaque) bool entered_postcopy = false; /* The active state we expect to be in; ACTIVE or POSTCOPY_ACTIVE */ enum MigrationStatus current_active_state = MIGRATION_STATUS_ACTIVE; + bool enable_colo = migrate_colo_enabled(); rcu_register_thread(); @@ -1857,7 +1870,13 @@ static void *migration_thread(void *opaque) end_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_mutex_lock_iothread(); - qemu_savevm_state_cleanup(); + /* + * The resource has been allocated by migration will be reused in COLO + * process, so don't release them. + */ + if (!enable_colo) { + qemu_savevm_state_cleanup(); + } if (s->state == MIGRATION_STATUS_COMPLETED) { uint64_t transferred_bytes = qemu_ftell(s->to_dst_file); s->total_time = end_time - s->total_time; @@ -1870,6 +1889,15 @@ static void *migration_thread(void *opaque) } runstate_set(RUN_STATE_POSTMIGRATE); } else { + if (s->state == MIGRATION_STATUS_ACTIVE && enable_colo) { + migrate_start_colo_process(s); + qemu_savevm_state_cleanup(); + /* + * Fixme: we will run VM in COLO no matter its old running state. + * After exited COLO, we will keep running. + */ + old_vm_running = true; + } if (old_vm_running && !entered_postcopy) { vm_start(); } else { diff --git a/migration/trace-events b/migration/trace-events index dfee75a..a29e5a0 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -207,3 +207,6 @@ migration_tls_outgoing_handshake_complete(void) "" migration_tls_incoming_handshake_start(void) "" migration_tls_incoming_handshake_error(const char *err) "err=%s" migration_tls_incoming_handshake_complete(void) "" + +# migration/colo.c +colo_vm_state_change(const char *old, const char *new) "Change '%s' => '%s'" diff --git a/qapi-schema.json b/qapi-schema.json index 0fb4d7e..8b21215 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -459,12 +459,14 @@ # # @failed: some error occurred during migration process. # +# @colo: VM is in the process of fault tolerance. (since 2.8) +# # Since: 2.3 # ## { 'enum': 'MigrationStatus', 'data': [ 'none', 'setup', 'cancelling', 'cancelled', - 'active', 'postcopy-active', 'completed', 'failed' ] } + 'active', 'postcopy-active', 'completed', 'failed', 'colo' ] } ## # @MigrationInfo diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c index d215057..0c8eef4 100644 --- a/stubs/migration-colo.c +++ b/stubs/migration-colo.c @@ -17,3 +17,12 @@ bool colo_supported(void) { return false; } + +bool migration_in_colo_state(void) +{ + return false; +} + +void migrate_start_colo_process(MigrationState *s) +{ +}