From patchwork Thu Dec 17 07:48:15 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 7870241 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id D84D3BEEED for ; Thu, 17 Dec 2015 07:51:44 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 653FD2040F for ; Thu, 17 Dec 2015 07:51:43 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95070203E3 for ; Thu, 17 Dec 2015 07:51:41 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1a9TJL-0002dw-0m; Thu, 17 Dec 2015 07:49:15 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1a9TJI-0002ZT-GP for xen-devel@lists.xen.org; Thu, 17 Dec 2015 07:49:12 +0000 Received: from [85.158.139.211] by server-4.bemta-5.messagelabs.com id 70/27-24856-7F862765; Thu, 17 Dec 2015 07:49:11 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-12.tower-206.messagelabs.com!1450338539!11275817!6 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 11802 invoked from network); 17 Dec 2015 07:49:10 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-12.tower-206.messagelabs.com with SMTP; 17 Dec 2015 07:49:10 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="1683640" Received: from bogon (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 17 Dec 2015 15:48:56 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 25DCB409256D; Thu, 17 Dec 2015 15:48:49 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Thu, 17 Dec 2015 15:48:48 +0800 From: Wen Congyang To: xen devel , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Thu, 17 Dec 2015 15:48:15 +0800 Message-ID: <1450338502-27335-12-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1450338502-27335-1-git-send-email-wency@cn.fujitsu.com> References: <1450338502-27335-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: 25DCB409256D.AA90C X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v5 COLOPre 11/18] tools/libxl: Add back channel to allow migration target send data back X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yang Hongyang In colo mode, slave needs to send data to master, but the io_fd only can be written in master, and only can be read in slave. Save recv_fd in domain_suspend_state, and send_fd in domain_create_state. Extend libxl_domain_create_restore API, add a send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate the API change. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang --- tools/libxl/libxl.c | 2 +- tools/libxl/libxl.h | 30 ++++++++++++++++++++++++++++-- tools/libxl/libxl_create.c | 9 +++++---- tools/libxl/libxl_internal.h | 2 ++ tools/libxl/libxl_types.idl | 1 + tools/libxl/xl_cmdimpl.c | 8 +++++++- tools/ocaml/libs/xl/xenlight_stubs.c | 2 +- 7 files changed, 45 insertions(+), 9 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 2faea4d..69c8047 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -872,7 +872,7 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->callback = remus_failover_cb; dss->domid = domid; dss->fd = send_fd; - /* TODO do something with recv_fd */ + dss->recv_fd = recv_fd; dss->type = type; dss->live = 1; dss->debug = 0; diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index a01e448..67a4ad7 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -630,6 +630,15 @@ typedef struct libxl__ctx libxl_ctx; #define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1 /* + * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_FD 1 + * + * If this is defined, libxl_domain_create_restore()'s API has changed to + * include a send_fd param which used for libxl migration back channel + * during COLO FT. + */ +#define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_FD 1 + +/* * LIBXL_HAVE_CREATEINFO_PVH * If this is defined, then libxl supports creation of a PVH guest. */ @@ -1143,7 +1152,7 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncprogress_how *aop_console_how) LIBXL_EXTERNAL_CALLERS_ONLY; int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1164,7 +1173,7 @@ int static inline libxl_domain_create_restore_0x040200( libxl_domain_restore_params_init(¶ms); ret = libxl_domain_create_restore( - ctx, d_config, domid, restore_fd, ¶ms, ao_how, aop_console_how); + ctx, d_config, domid, restore_fd, -1, ¶ms, ao_how, aop_console_how); libxl_domain_restore_params_dispose(¶ms); return ret; @@ -1172,6 +1181,23 @@ int static inline libxl_domain_create_restore_0x040200( #define libxl_domain_create_restore libxl_domain_create_restore_0x040200 +#elif defined(LIBXL_API_VERSION) && LIBXL_API_VERSION >= 0x040400 \ + && LIBXL_API_VERSION < 0x040600 + +int static inline libxl_domain_create_restore_0x040400( + libxl_ctx *ctx, libxl_domain_config *d_config, + uint32_t *domid, int restore_fd, + const libxl_domain_restore_params *params, + const libxl_asyncop_how *ao_how, + const libxl_asyncprogress_how *aop_console_how) + LIBXL_EXTERNAL_CALLERS_ONLY +{ + return libxl_domain_create_restore(ctx, d_config, domid, restore_fd, + -1, params, ao_how, aop_console_how); +} + +#define libxl_domain_create_restore libxl_domain_create_restore_0x040400 + #endif int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index fdf3b46..6aa6525 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1548,7 +1548,7 @@ static void domain_create_cb(libxl__egc *egc, int rc, uint32_t domid); static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1563,6 +1563,7 @@ static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_domain_config_init(&cdcs->dcs.guest_config_saved); libxl_domain_config_copy(ctx, &cdcs->dcs.guest_config_saved, d_config); cdcs->dcs.restore_fd = cdcs->dcs.libxc_fd = restore_fd; + cdcs->dcs.send_fd = send_fd; if (restore_fd > -1) { cdcs->dcs.restore_params = *params; rc = libxl__fd_flags_modify_save(gc, cdcs->dcs.restore_fd, @@ -1741,17 +1742,17 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, -1, NULL, + return do_domain_create(ctx, d_config, domid, -1, -1, NULL, ao_how, aop_console_how); } int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, restore_fd, params, + return do_domain_create(ctx, d_config, domid, restore_fd, send_fd, params, ao_how, aop_console_how); } diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index e9c297d..ea57ce7 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3116,6 +3116,7 @@ struct libxl__domain_save_state { uint32_t domid; int fd; int fdfl; /* original flags on fd */ + int recv_fd; libxl_domain_type type; int live; int debug; @@ -3453,6 +3454,7 @@ struct libxl__domain_create_state { libxl_domain_config guest_config_saved; /* vanilla config */ int restore_fd, libxc_fd; int restore_fdfl; /* original flags of restore_fd */ + int send_fd; libxl_domain_restore_params restore_params; uint32_t domid_soft_reset; libxl__domain_create_cb *callback; diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 9aa94be..c5d5d40 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -232,6 +232,7 @@ libxl_hdtype = Enumeration("hdtype", [ libxl_checkpointed_stream = Enumeration("checkpointed_stream", [ (0, "NONE"), (1, "REMUS"), + (2, "COLO"), ]) # diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index c1cd696..6580b59 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -159,6 +159,7 @@ struct domain_create { char *extra_config; /* extra config string */ const char *restore_file; int migrate_fd; /* -1 means none */ + int send_fd; /* -1 means none */ char **migration_domname_r; /* from malloc */ }; @@ -2686,6 +2687,7 @@ static uint32_t create_domain(struct domain_create *dom_info) int config_len = 0; int restore_fd = -1; int restore_fd_to_close = -1; + int send_fd = -1; const libxl_asyncprogress_how *autoconnect_console_how; struct save_file_header hdr; uint32_t domid_soft_reset = INVALID_DOMID; @@ -2703,6 +2705,7 @@ static uint32_t create_domain(struct domain_create *dom_info) if (migrate_fd >= 0) { restore_source = ""; restore_fd = migrate_fd; + send_fd = dom_info->send_fd; } else { restore_source = restore_file; restore_fd = open(restore_file, O_RDONLY); @@ -2893,7 +2896,7 @@ start: ret = libxl_domain_create_restore(ctx, &d_config, &domid, restore_fd, - ¶ms, + send_fd, ¶ms, 0, autoconnect_console_how); libxl_domain_restore_params_dispose(¶ms); @@ -4449,6 +4452,7 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.monitor = monitor; dom_info.paused = 1; dom_info.migrate_fd = recv_fd; + dom_info.send_fd = send_fd; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; @@ -4622,6 +4626,7 @@ int main_restore(int argc, char **argv) dom_info.config_file = config_file; dom_info.restore_file = checkpoint_file; dom_info.migrate_fd = -1; + dom_info.send_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; @@ -5088,6 +5093,7 @@ int main_create(int argc, char **argv) dom_info.quiet = quiet; dom_info.config_file = filename; dom_info.migrate_fd = -1; + dom_info.send_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c b/tools/ocaml/libs/xl/xenlight_stubs.c index 4133527..1c52c2a 100644 --- a/tools/ocaml/libs/xl/xenlight_stubs.c +++ b/tools/ocaml/libs/xl/xenlight_stubs.c @@ -537,7 +537,7 @@ value stub_libxl_domain_create_restore(value ctx, value domain_config, value par restore_fd = Int_val(Field(params, 0)); caml_enter_blocking_section(); - ret = libxl_domain_create_restore(CTX, &c_dconfig, &c_domid, restore_fd, + ret = libxl_domain_create_restore(CTX, &c_dconfig, &c_domid, restore_fd, -1, &c_params, ao_how, NULL); caml_leave_blocking_section();