From patchwork Wed Mar 23 08:06:17 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changlong Xie X-Patchwork-Id: 8647761 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 130529F36E for ; Wed, 23 Mar 2016 08:07:00 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C862620398 for ; Wed, 23 Mar 2016 08:06:58 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 77FE72038D for ; Wed, 23 Mar 2016 08:06:57 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aidmO-0001uI-Rv; Wed, 23 Mar 2016 08:04:36 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aidmN-0001tU-1P for xen-devel@lists.xen.org; Wed, 23 Mar 2016 08:04:35 +0000 Received: from [193.109.254.147] by server-10.bemta-14.messagelabs.com id 02/8D-02986-21E42F65; Wed, 23 Mar 2016 08:04:34 +0000 X-Env-Sender: xiecl.fnst@cn.fujitsu.com X-Msg-Ref: server-5.tower-27.messagelabs.com!1458720265!33319081!3 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 52545 invoked from network); 23 Mar 2016 08:04:32 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-5.tower-27.messagelabs.com with SMTP; 23 Mar 2016 08:04:32 -0000 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="4882426" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 23 Mar 2016 16:04:23 +0800 Received: from G08CNEXCHPEKD03.g08.fujitsu.local (unknown [10.167.33.85]) by cn.fujitsu.com (Postfix) with ESMTP id 2FC85408D262; Wed, 23 Mar 2016 16:04:22 +0800 (CST) Received: from changlox.g08.fujitsu.local (10.167.225.55) by G08CNEXCHPEKD03.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.279.2; Wed, 23 Mar 2016 16:04:21 +0800 From: Changlong Xie To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Wed, 23 Mar 2016 16:06:17 +0800 Message-ID: <1458720400-4699-4-git-send-email-xiecl.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1458720400-4699-1-git-send-email-xiecl.fnst@cn.fujitsu.com> References: <1458720400-4699-1-git-send-email-xiecl.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.55] X-yoursite-MailScanner-ID: 2FC85408D262.A1B7E X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: xiecl.fnst@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Li Zhijian , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Anthony Perard , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v12 03/26] tools/libxl: Add back channel to allow migration target send data back X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wen Congyang In COLO mode, secondary needs to send the following data to primary: 1. In libxl Secondary sends the following CHECKPOINT_CONTEXT to primary: CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED 2. In libxc Secondary sends the dirty pfn list to primary But the io_fd only can be written in primary, and only can be read in secondary. Save recv_fd in domain_suspend_state, and send_fd in domain_create_state. Extend libxl_domain_create_restore API, add a send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate the API change. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Changlong Xie Acked-by: Wei Liu --- tools/libxl/libxl.c | 3 +-- tools/libxl/libxl.h | 29 ++++++++++++++++++++++++++++- tools/libxl/libxl_create.c | 11 +++++++---- tools/libxl/libxl_internal.h | 2 ++ tools/libxl/xl_cmdimpl.c | 8 +++++++- tools/ocaml/libs/xl/xenlight_stubs.c | 2 +- 6 files changed, 46 insertions(+), 9 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 93e228d..83a419e 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -834,7 +834,6 @@ out: static void remus_failover_cb(libxl__egc *egc, libxl__domain_save_state *dss, int rc); -/* TODO: Explicit Checkpoint acknowledgements via recv_fd. */ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, uint32_t domid, int send_fd, int recv_fd, const libxl_asyncop_how *ao_how) @@ -871,7 +870,7 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->callback = remus_failover_cb; dss->domid = domid; dss->fd = send_fd; - /* TODO do something with recv_fd */ + dss->recv_fd = recv_fd; dss->type = type; dss->live = 1; dss->debug = 0; diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index f9e3ef5..4b75d52 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -639,6 +639,15 @@ typedef struct libxl__ctx libxl_ctx; #define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1 /* + * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_BACK_FD 1 + * + * If this is defined, libxl_domain_create_restore()'s API includes the + * send_back_fd param. This is used only with COLO, for the libxl migration + * back channel; other callers should pass -1. + */ +#define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_BACK_FD 1 + +/* * LIBXL_HAVE_CREATEINFO_PVH * If this is defined, then libxl supports creation of a PVH guest. */ @@ -1165,6 +1174,7 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, LIBXL_EXTERNAL_CALLERS_ONLY; int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, uint32_t *domid, int restore_fd, + int send_back_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1185,7 +1195,7 @@ int static inline libxl_domain_create_restore_0x040200( libxl_domain_restore_params_init(¶ms); ret = libxl_domain_create_restore( - ctx, d_config, domid, restore_fd, ¶ms, ao_how, aop_console_how); + ctx, d_config, domid, restore_fd, -1, ¶ms, ao_how, aop_console_how); libxl_domain_restore_params_dispose(¶ms); return ret; @@ -1193,6 +1203,23 @@ int static inline libxl_domain_create_restore_0x040200( #define libxl_domain_create_restore libxl_domain_create_restore_0x040200 +#elif defined(LIBXL_API_VERSION) && LIBXL_API_VERSION >= 0x040400 \ + && LIBXL_API_VERSION < 0x040700 + +int static inline libxl_domain_create_restore_0x040400( + libxl_ctx *ctx, libxl_domain_config *d_config, + uint32_t *domid, int restore_fd, + const libxl_domain_restore_params *params, + const libxl_asyncop_how *ao_how, + const libxl_asyncprogress_how *aop_console_how) + LIBXL_EXTERNAL_CALLERS_ONLY +{ + return libxl_domain_create_restore(ctx, d_config, domid, restore_fd, + -1, params, ao_how, aop_console_how); +} + +#define libxl_domain_create_restore libxl_domain_create_restore_0x040400 + #endif int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index f1028bc..e72f829 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1572,7 +1572,7 @@ static void domain_create_cb(libxl__egc *egc, int rc, uint32_t domid); static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_back_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1587,6 +1587,7 @@ static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_domain_config_init(&cdcs->dcs.guest_config_saved); libxl_domain_config_copy(ctx, &cdcs->dcs.guest_config_saved, d_config); cdcs->dcs.restore_fd = cdcs->dcs.libxc_fd = restore_fd; + cdcs->dcs.send_back_fd = send_back_fd; if (restore_fd > -1) { cdcs->dcs.restore_params = *params; rc = libxl__fd_flags_modify_save(gc, cdcs->dcs.restore_fd, @@ -1765,18 +1766,20 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, -1, NULL, + return do_domain_create(ctx, d_config, domid, -1, -1, NULL, ao_how, aop_console_how); } int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, uint32_t *domid, int restore_fd, + int send_back_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, restore_fd, params, - ao_how, aop_console_how); + assert(send_back_fd == -1); + return do_domain_create(ctx, d_config, domid, restore_fd, send_back_fd, + params, ao_how, aop_console_how); } int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 15b9508..5314522 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3132,6 +3132,7 @@ struct libxl__domain_save_state { uint32_t domid; int fd; int fdfl; /* original flags on fd */ + int recv_fd; libxl_domain_type type; int live; int debug; @@ -3465,6 +3466,7 @@ struct libxl__domain_create_state { libxl_domain_config guest_config_saved; /* vanilla config */ int restore_fd, libxc_fd; int restore_fdfl; /* original flags of restore_fd */ + int send_back_fd; libxl_domain_restore_params restore_params; uint32_t domid_soft_reset; libxl__domain_create_cb *callback; diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 990d3c9..680aeb0 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -159,6 +159,7 @@ struct domain_create { char *extra_config; /* extra config string */ const char *restore_file; int migrate_fd; /* -1 means none */ + int send_back_fd; /* -1 means none */ char **migration_domname_r; /* from malloc */ }; @@ -2691,6 +2692,7 @@ static uint32_t create_domain(struct domain_create *dom_info) int config_len = 0; int restore_fd = -1; int restore_fd_to_close = -1; + int send_back_fd = -1; const libxl_asyncprogress_how *autoconnect_console_how; struct save_file_header hdr; uint32_t domid_soft_reset = INVALID_DOMID; @@ -2708,6 +2710,7 @@ static uint32_t create_domain(struct domain_create *dom_info) if (migrate_fd >= 0) { restore_source = ""; restore_fd = migrate_fd; + send_back_fd = dom_info->send_back_fd; } else { restore_source = restore_file; restore_fd = open(restore_file, O_RDONLY); @@ -2896,7 +2899,7 @@ start: ret = libxl_domain_create_restore(ctx, &d_config, &domid, restore_fd, - ¶ms, + send_back_fd, ¶ms, 0, autoconnect_console_how); libxl_domain_restore_params_dispose(¶ms); @@ -4459,6 +4462,7 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.monitor = monitor; dom_info.paused = 1; dom_info.migrate_fd = recv_fd; + dom_info.send_back_fd = -1; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; @@ -4632,6 +4636,7 @@ int main_restore(int argc, char **argv) dom_info.config_file = config_file; dom_info.restore_file = checkpoint_file; dom_info.migrate_fd = -1; + dom_info.send_back_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; @@ -5099,6 +5104,7 @@ int main_create(int argc, char **argv) dom_info.quiet = quiet; dom_info.config_file = filename; dom_info.migrate_fd = -1; + dom_info.send_back_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c b/tools/ocaml/libs/xl/xenlight_stubs.c index 4133527..98b52b9 100644 --- a/tools/ocaml/libs/xl/xenlight_stubs.c +++ b/tools/ocaml/libs/xl/xenlight_stubs.c @@ -538,7 +538,7 @@ value stub_libxl_domain_create_restore(value ctx, value domain_config, value par caml_enter_blocking_section(); ret = libxl_domain_create_restore(CTX, &c_dconfig, &c_domid, restore_fd, - &c_params, ao_how, NULL); + -1, &c_params, ao_how, NULL); caml_leave_blocking_section(); free(ao_how);