From patchwork Wed Dec 30 02:37:47 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 7931551 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 4ABCF9F350 for ; Wed, 30 Dec 2015 02:40:28 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 16977201FA for ; Wed, 30 Dec 2015 02:40:27 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AF79E201EF for ; Wed, 30 Dec 2015 02:40:25 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aE6fB-00035R-HW; Wed, 30 Dec 2015 02:38:57 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aE6f8-0002yl-8h for xen-devel@lists.xen.org; Wed, 30 Dec 2015 02:38:54 +0000 Received: from [85.158.143.35] by server-3.bemta-4.messagelabs.com id 8E/4A-31122-DB343865; Wed, 30 Dec 2015 02:38:53 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-6.tower-21.messagelabs.com!1451443105!7658664!7 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 52646 invoked from network); 30 Dec 2015 02:38:51 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-6.tower-21.messagelabs.com with SMTP; 30 Dec 2015 02:38:51 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="2080190" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 30 Dec 2015 10:38:36 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id B338141890FD; Wed, 30 Dec 2015 10:38:17 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Wed, 30 Dec 2015 10:38:17 +0800 From: Wen Congyang To: xen devel , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Wed, 30 Dec 2015 10:37:47 +0800 Message-ID: <1451443075-27428-18-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451443075-27428-1-git-send-email-wency@cn.fujitsu.com> References: <1451443075-27428-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: B338141890FD.A8AA1 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v9 17/25] implement the cmdline for COLO X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new option -c to the command 'xl remus'. If you want to use COLO HA instead of Remus HA, please use -c option. Update man pages to reflect the addition of a new option to 'xl remus' command. Also add a new option -c to the internal command 'xl migrate-receive'. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang --- docs/man/xl.pod.1 | 12 +++++++-- tools/libxl/libxl.c | 23 ++++++++++++++++-- tools/libxl/xl_cmdimpl.c | 62 ++++++++++++++++++++++++++++++++++++----------- tools/libxl/xl_cmdtable.c | 4 ++- 4 files changed, 82 insertions(+), 19 deletions(-) diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1 index 4279c7c..1c6dd87 100644 --- a/docs/man/xl.pod.1 +++ b/docs/man/xl.pod.1 @@ -447,12 +447,15 @@ Print huge (!) amount of debug during the migration process. =item B [I] I I -Enable Remus HA for domain. By default B relies on ssh as a transport -mechanism between the two hosts. +Enable Remus HA or COLO HA for domain. By default B relies on ssh as a +transport mechanism between the two hosts. N.B: Remus support in xl is still in experimental (proof-of-concept) phase. Disk replication support is limited to DRBD disks. + COLO support in xl is still in experimental (proof-of-concept) phase. + There is no support for network or disk at the moment. + B =over 4 @@ -498,6 +501,11 @@ Disable network output buffering. Requires enabling unsafe mode. Disable disk replication. Requires enabling unsafe mode. +=item B<-c> + +Enable COLO HA. This conflicts with B<-i> and B<-b>, and memory +checkpoint compression must be disabled. + =back =item B I diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 7d227d7..ca6bca1 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -851,12 +851,28 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, goto out; } + /* The caller must set this defbool */ + if (libxl_defbool_is_default(info->colo)) { + LOG(ERROR, "colo mode must be enabled/disabled"); + rc = ERROR_FAIL; + goto out; + } + libxl_defbool_setdefault(&info->allow_unsafe, false); libxl_defbool_setdefault(&info->blackhole, false); - libxl_defbool_setdefault(&info->compression, true); + libxl_defbool_setdefault(&info->compression, + !libxl_defbool_val(info->colo)); libxl_defbool_setdefault(&info->netbuf, true); libxl_defbool_setdefault(&info->diskbuf, true); + if (libxl_defbool_val(info->colo)) { + if (libxl_defbool_val(info->compression)) { + LOG(ERROR, "cannot use memory checkpoint compression in COLO mode"); + rc = ERROR_FAIL; + goto out; + } + } + if (!libxl_defbool_val(info->allow_unsafe) && (libxl_defbool_val(info->blackhole) || !libxl_defbool_val(info->netbuf) || @@ -878,7 +894,10 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->live = 1; dss->debug = 0; dss->remus = info; - dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; + if (libxl_defbool_val(info->colo)) + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_COLO; + else + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; assert(info); diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 6580b59..70b8b82 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -4435,6 +4435,8 @@ static void migrate_receive(int debug, int daemonize, int monitor, char rc_buf; char *migration_domname; struct domain_create dom_info; + const char *ha = checkpointed == LIBXL_CHECKPOINTED_STREAM_COLO ? + "COLO" : "Remus"; signal(SIGPIPE, SIG_IGN); /* if we get SIGPIPE we'd rather just have it as an error */ @@ -4455,6 +4457,9 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.send_fd = send_fd; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; + if (checkpointed == LIBXL_CHECKPOINTED_STREAM_COLO) + /* COLO uses stdout to send control message to master */ + dom_info.quiet = 1; rc = create_domain(&dom_info); if (rc < 0) { @@ -4467,11 +4472,12 @@ static void migrate_receive(int debug, int daemonize, int monitor, switch (checkpointed) { case LIBXL_CHECKPOINTED_STREAM_REMUS: + case LIBXL_CHECKPOINTED_STREAM_COLO: /* If we are here, it means that the sender (primary) has crashed. * TODO: Split-Brain Check. */ - fprintf(stderr, "migration target: Remus Failover for domain %u\n", - domid); + fprintf(stderr, "migration target: %s Failover for domain %u\n", + ha, domid); /* * If domain renaming fails, lets just continue (as we need the domain @@ -4487,16 +4493,20 @@ static void migrate_receive(int debug, int daemonize, int monitor, rc = libxl_domain_rename(ctx, domid, migration_domname, common_domname); if (rc) - fprintf(stderr, "migration target (Remus): " + fprintf(stderr, "migration target (%s): " "Failed to rename domain from %s to %s:%d\n", - migration_domname, common_domname, rc); + ha, migration_domname, common_domname, rc); } + if (checkpointed == LIBXL_CHECKPOINTED_STREAM_COLO) + /* The guest is running after failover in COLO mode */ + exit(rc ? -ERROR_FAIL: 0); + rc = libxl_domain_unpause(ctx, domid); if (rc) - fprintf(stderr, "migration target (Remus): " + fprintf(stderr, "migration target (%s): " "Failed to unpause domain %s (id: %u):%d\n", - common_domname, domid, rc); + ha, common_domname, domid, rc); exit(rc ? -ERROR_FAIL: 0); default: @@ -4644,7 +4654,7 @@ int main_migrate_receive(int argc, char **argv) libxl_checkpointed_stream checkpointed = LIBXL_CHECKPOINTED_STREAM_NONE; int opt; - SWITCH_FOREACH_OPT(opt, "Fedr", NULL, "migrate-receive", 0) { + SWITCH_FOREACH_OPT(opt, "Fedrc", NULL, "migrate-receive", 0) { case 'F': daemonize = 0; break; @@ -4658,6 +4668,9 @@ int main_migrate_receive(int argc, char **argv) case 'r': checkpointed = LIBXL_CHECKPOINTED_STREAM_REMUS; break; + case 'c': + checkpointed = LIBXL_CHECKPOINTED_STREAM_COLO; + break; } if (argc-optind != 0) { @@ -8023,11 +8036,8 @@ int main_remus(int argc, char **argv) int config_len; memset(&r_info, 0, sizeof(libxl_domain_remus_info)); - /* Defaults */ - r_info.interval = 200; - libxl_defbool_setdefault(&r_info.blackhole, false); - SWITCH_FOREACH_OPT(opt, "Fbundi:s:N:e", NULL, "remus", 2) { + SWITCH_FOREACH_OPT(opt, "Fbundi:s:N:ec", NULL, "remus", 2) { case 'i': r_info.interval = atoi(optarg); break; @@ -8055,11 +8065,32 @@ int main_remus(int argc, char **argv) case 'e': daemonize = 0; break; + case 'c': + libxl_defbool_set(&r_info.colo, true); } domid = find_domain(argv[optind]); host = argv[optind + 1]; + /* Defaults */ + libxl_defbool_setdefault(&r_info.blackhole, false); + libxl_defbool_setdefault(&r_info.colo, false); + if (!libxl_defbool_val(r_info.colo) && !r_info.interval) + r_info.interval = 200; + + if (libxl_defbool_val(r_info.colo)) { + if (r_info.interval || libxl_defbool_val(r_info.blackhole)) { + perror("Option -c conflicts with -i or -b"); + exit(-1); + } + + if (libxl_defbool_is_default(r_info.compression)) { + perror("COLO can't be used with memory compression. " + "Disable memory checkpoint compression now..."); + libxl_defbool_set(&r_info.compression, false); + } + } + if (!r_info.netbufscript) r_info.netbufscript = default_remus_netbufscript; @@ -8074,8 +8105,9 @@ int main_remus(int argc, char **argv) if (!ssh_command[0]) { rune = host; } else { - xasprintf(&rune, "exec %s %s xl migrate-receive -r %s", + xasprintf(&rune, "exec %s %s xl migrate-receive %s %s", ssh_command, host, + libxl_defbool_val(r_info.colo) ? "-c" : "-r", daemonize ? "" : " -e"); } @@ -8103,7 +8135,8 @@ int main_remus(int argc, char **argv) * domain to force failover */ if (libxl_domain_info(ctx, 0, domid)) { - fprintf(stderr, "Remus: Primary domain has been destroyed.\n"); + fprintf(stderr, "%s: Primary domain has been destroyed.\n", + libxl_defbool_val(r_info.colo) ? "COLO" : "Remus"); close(send_fd); return 0; } @@ -8115,7 +8148,8 @@ int main_remus(int argc, char **argv) if (rc == ERROR_GUEST_TIMEDOUT) fprintf(stderr, "Failed to suspend domain at primary.\n"); else { - fprintf(stderr, "Remus: Backup failed? resuming domain at primary.\n"); + fprintf(stderr, "%s: Backup failed? resuming domain at primary.\n", + libxl_defbool_val(r_info.colo) ? "COLO" : "Remus"); libxl_domain_resume(ctx, domid, 1, 0); } diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c index fdc1ac6..b6b630c 100644 --- a/tools/libxl/xl_cmdtable.c +++ b/tools/libxl/xl_cmdtable.c @@ -499,7 +499,9 @@ struct cmd_spec cmd_table[] = { "-b Replicate memory checkpoints to /dev/null (blackhole).\n" " Works only in unsafe mode.\n" "-n Disable network output buffering. Works only in unsafe mode.\n" - "-d Disable disk replication. Works only in unsafe mode." + "-d Disable disk replication. Works only in unsafe mode.\n" + "-c Enable COLO HA. It is conflict with -i and -b, and memory\n" + " checkpoint must be disabled" }, #endif { "devd",