From patchwork Wed Mar 23 08:06:25 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changlong Xie X-Patchwork-Id: 8647681 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id B09D99F36E for ; Wed, 23 Mar 2016 08:06:43 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 923CE203AA for ; Wed, 23 Mar 2016 08:06:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5C7BA2038D for ; Wed, 23 Mar 2016 08:06:41 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aidmU-00021b-0b; Wed, 23 Mar 2016 08:04:42 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aidmS-0001u5-QF for xen-devel@lists.xen.org; Wed, 23 Mar 2016 08:04:40 +0000 Received: from [193.109.254.147] by server-3.bemta-14.messagelabs.com id 38/97-03304-81E42F65; Wed, 23 Mar 2016 08:04:40 +0000 X-Env-Sender: xiecl.fnst@cn.fujitsu.com X-Msg-Ref: server-5.tower-27.messagelabs.com!1458720265!33319081!7 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 53496 invoked from network); 23 Mar 2016 08:04:39 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-5.tower-27.messagelabs.com with SMTP; 23 Mar 2016 08:04:39 -0000 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="4882440" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 23 Mar 2016 16:04:33 +0800 Received: from G08CNEXCHPEKD03.g08.fujitsu.local (unknown [10.167.33.85]) by cn.fujitsu.com (Postfix) with ESMTP id 93A83408D260; Wed, 23 Mar 2016 16:04:30 +0800 (CST) Received: from changlox.g08.fujitsu.local (10.167.225.55) by G08CNEXCHPEKD03.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.279.2; Wed, 23 Mar 2016 16:04:30 +0800 From: Changlong Xie To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Wed, 23 Mar 2016 16:06:25 +0800 Message-ID: <1458720400-4699-12-git-send-email-xiecl.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1458720400-4699-1-git-send-email-xiecl.fnst@cn.fujitsu.com> References: <1458720400-4699-1-git-send-email-xiecl.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.55] X-yoursite-MailScanner-ID: 93A83408D260.AFA95 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: xiecl.fnst@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Li Zhijian , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Anthony Perard , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v12 11/26] tools/libxl: add back channel support to read stream X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wen Congyang This is used by primay to read records sent by secondary. Note: The function libxl__stream_read_checkpoint_state() will be used in later patches called "secondary vm suspend/resume/checkpoint code" and "primary vm suspend/resume/checkpoint code". Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- tools/libxl/libxl_create.c | 1 + tools/libxl/libxl_internal.h | 4 ++ tools/libxl/libxl_stream_read.c | 94 +++++++++++++++++++++++++++++++++++++---- 3 files changed, 91 insertions(+), 8 deletions(-) diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index e72f829..55f1427 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1030,6 +1030,7 @@ static void domcreate_bootloader_done(libxl__egc *egc, dcs->srs.dcs = dcs; dcs->srs.fd = restore_fd; dcs->srs.legacy = (dcs->restore_params.stream_version == 1); + dcs->srs.back_channel = false; dcs->srs.completion_callback = domcreate_stream_done; if (restore_fd >= 0) { diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index fb8b2fa..6307b71 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3418,6 +3418,7 @@ struct libxl__stream_read_state { libxl__domain_create_state *dcs; int fd; bool legacy; + bool back_channel; void (*completion_callback)(libxl__egc *egc, libxl__stream_read_state *srs, int rc); @@ -3429,6 +3430,7 @@ struct libxl__stream_read_state { bool running; bool in_checkpoint; bool sync_teardown; /* Only used to coordinate shutdown on error path. */ + bool in_checkpoint_state; libxl__save_helper_state shs; libxl__conversion_helper_state chs; @@ -3456,6 +3458,8 @@ _hidden void libxl__stream_read_start(libxl__egc *egc, libxl__stream_read_state *stream); _hidden void libxl__stream_read_start_checkpoint(libxl__egc *egc, libxl__stream_read_state *stream); +_hidden void libxl__stream_read_checkpoint_state(libxl__egc *egc, + libxl__stream_read_state *stream); _hidden void libxl__stream_read_abort(libxl__egc *egc, libxl__stream_read_state *stream, int rc); static inline bool diff --git a/tools/libxl/libxl_stream_read.c b/tools/libxl/libxl_stream_read.c index f4781eb..302ae53 100644 --- a/tools/libxl/libxl_stream_read.c +++ b/tools/libxl/libxl_stream_read.c @@ -118,6 +118,15 @@ * record, and therefore the buffered state is inconsistent. In * libxl__xc_domain_restore_done(), we just complete the stream and * stream->completion_callback() will be called to resume the guest + * + * For back channel stream: + * - libxl__stream_read_start() + * - Set up the stream to running state + * + * - libxl__stream_read_continue() + * - Set up reading the next record from a started stream. + * Add some codes to process_record() to handle the record. + * Then call stream->checkpoint_callback() to return. */ /* Success/error/cleanup handling. */ @@ -157,6 +166,10 @@ static void write_emulator_done(libxl__egc *egc, libxl__datacopier_state *dc, int rc, int onwrite, int errnoval); +/* Handlers for checkpoint state mini-loop */ +static void checkpoint_state_done(libxl__egc *egc, + libxl__stream_read_state *stream, int rc); + /*----- Helpers -----*/ /* Helper to set up reading some data from the stream. */ @@ -221,6 +234,14 @@ void libxl__stream_read_start(libxl__egc *egc, stream->running = true; stream->phase = SRS_PHASE_NORMAL; + dc->ao = stream->ao; + dc->copywhat = "restore v2 stream"; + dc->readfd = stream->fd; + dc->writefd = -1; + + if (stream->back_channel) + return; + if (stream->legacy) { /* Convert the legacy stream. */ libxl__conversion_helper_state *chs = &stream->chs; @@ -241,12 +262,6 @@ void libxl__stream_read_start(libxl__egc *egc, stream->fd = libxl__carefd_fd(stream->chs.v2_carefd); stream->dcs->libxc_fd = stream->fd; } - /* stream->fd is now a v2 stream. */ - - dc->ao = stream->ao; - dc->copywhat = "restore v2 stream"; - dc->readfd = stream->fd; - dc->writefd = -1; /* Start reading the stream header. */ rc = setup_read(stream, "stream header", @@ -532,6 +547,7 @@ static bool process_record(libxl__egc *egc, STATE_AO_GC(stream->ao); libxl__domain_create_state *dcs = stream->dcs; libxl__sr_record_buf *rec; + libxl_sr_checkpoint_state *srcs; bool further_action_needed = false; int rc = 0; @@ -602,6 +618,17 @@ static bool process_record(libxl__egc *egc, checkpoint_done(egc, stream, 0); break; + case REC_TYPE_CHECKPOINT_STATE: + if (!stream->in_checkpoint_state) { + LOG(ERROR, "Unexpected CHECKPOINT_STATE record in stream"); + rc = ERROR_FAIL; + goto err; + } + + srcs = rec->body; + checkpoint_state_done(egc, stream, srcs->id); + break; + default: LOG(ERROR, "Unrecognised record 0x%08x", rec->hdr.type); rc = ERROR_FAIL; @@ -713,6 +740,21 @@ static void stream_complete(libxl__egc *egc, return; } + if (stream->in_checkpoint_state) { + assert(rc); + + /* + * If an error is encountered while in a checkpoint, pass it + * back to libxc. The failure will come back around to us via + * 1. normal stream + * libxl__xc_domain_restore_done() + * 2. back_channel stream + * libxl__stream_read_abort() + */ + checkpoint_state_done(egc, stream, rc); + return; + } + stream_done(egc, stream, rc); } @@ -743,6 +785,7 @@ static void stream_done(libxl__egc *egc, assert(stream->running); assert(!stream->in_checkpoint); + assert(!stream->in_checkpoint_state); stream->running = false; if (stream->incoming_record) @@ -762,7 +805,19 @@ static void stream_done(libxl__egc *egc, LIBXL_STAILQ_FOREACH_SAFE(rec, &stream->record_queue, entry, trec) free_record(rec); - check_all_finished(egc, stream, rc); + if (!stream->back_channel) { + /* + * 1. In stream_done(), stream->running is set to false, so + * the stream itself is not in use. + * 2. Read stream is a back channel stream, this means it is + * only used by primary(save side) to read records sent by + * secondary(restore side), so it doesn't have restore helper. + * 3. Back channel stream doesn't support legacy stream, so + * there is no conversion helper. + * So we don't need invoke check_all_finished here + */ + check_all_finished(egc, stream, rc); + } } void libxl__xc_domain_restore_done(libxl__egc *egc, void *dcs_void, @@ -862,7 +917,30 @@ static void check_all_finished(libxl__egc *egc, libxl__conversion_helper_inuse(&stream->chs)) return; - stream->completion_callback(egc, stream, stream->rc); + if (stream->completion_callback) + /* back channel stream doesn't have completion_callback() */ + stream->completion_callback(egc, stream, stream->rc); +} + +/*----- Checkpoint state handlers -----*/ + +void libxl__stream_read_checkpoint_state(libxl__egc *egc, + libxl__stream_read_state *stream) +{ + assert(stream->running); + assert(!stream->in_checkpoint); + assert(!stream->in_checkpoint_state); + stream->in_checkpoint_state = true; + + setup_read_record(egc, stream); +} + +static void checkpoint_state_done(libxl__egc *egc, + libxl__stream_read_state *stream, int rc) +{ + assert(stream->in_checkpoint_state); + stream->in_checkpoint_state = false; + stream->checkpoint_callback(egc, stream, rc); } /*