From patchwork Wed Mar 30 15:10:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konrad Rzeszutek Wilk X-Patchwork-Id: 8698291 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id E7A599F3D1 for ; Wed, 30 Mar 2016 15:13:12 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 003BE2038D for ; Wed, 30 Mar 2016 15:13:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E9F772038A for ; Wed, 30 Mar 2016 15:13:10 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1alHlu-0001Np-HK; Wed, 30 Mar 2016 15:11:02 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1alHlu-0001Nj-2G for xen-devel@lists.xenproject.org; Wed, 30 Mar 2016 15:11:02 +0000 Received: from [85.158.137.68] by server-5.bemta-3.messagelabs.com id 02/FF-03651-58CEBF65; Wed, 30 Mar 2016 15:11:01 +0000 X-Env-Sender: konrad@char.us.oracle.com X-Msg-Ref: server-7.tower-31.messagelabs.com!1459350658!24617879!1 X-Originating-IP: [156.151.31.81] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTU2LjE1MS4zMS44MSA9PiAyODgzMzk=\n X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 20263 invoked from network); 30 Mar 2016 15:11:00 -0000 Received: from userp1040.oracle.com (HELO userp1040.oracle.com) (156.151.31.81) by server-7.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 30 Mar 2016 15:11:00 -0000 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u2UFAa06011861 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 30 Mar 2016 15:10:37 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u2UFAawI020167 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 30 Mar 2016 15:10:36 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u2UFAZ1X005720; Wed, 30 Mar 2016 15:10:35 GMT Received: from char.us.oracle.com (/10.137.176.158) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 Mar 2016 08:10:35 -0700 Received: by char.us.oracle.com (Postfix, from userid 1000) id 1EE176A00D9; Wed, 30 Mar 2016 11:10:30 -0400 (EDT) From: Konrad Rzeszutek Wilk To: xen-devel@lists.xenproject.org, wei.liu2@citrix.com Date: Wed, 30 Mar 2016 11:10:23 -0400 Message-Id: <1459350623-29548-1-git-send-email-konrad.wilk@oracle.com> X-Mailer: git-send-email 2.5.0 X-Source-IP: aserv0022.oracle.com [141.146.126.234] Cc: Konrad Rzeszutek Wilk , ian.jackson@eu.citrix.com, wency@cn.fujitsu.com, jbeulich@suse.com, hongyang.yang@easystack.cn Subject: [Xen-devel] [PATCH] libxc: Document xc_domain_resume X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Document the save and suspend mechanism. Signed-off-by: Konrad Rzeszutek Wilk --- v2: Wei update on wording. --- tools/libxc/include/xenctrl.h | 52 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h index 150d727..096ff5c 100644 --- a/tools/libxc/include/xenctrl.h +++ b/tools/libxc/include/xenctrl.h @@ -565,6 +565,58 @@ int xc_domain_destroy(xc_interface *xch, * This function resumes a suspended domain. The domain should have * been previously suspended. * + * Note that there are 'xc_domain_suspend' as suspending a domain + * is quite the endeavour. As such this long comment will describe the + * suspend and resume path. + * + * For the purpose of this explanation there are three guests: + * PV (using hypercalls for privilgied operations), HVM + * (fully hardware virtualized guests using emulated devices for everything), + * and PVHVM (hardware virtualized guest with PV drivers). + * + * HVM guest are the simplest - they suspend via S3 and resume from + * S3. Upon resume they have to re-negotiate with the emulated devices. + * + * PV and PVHVM communicate via via hypercalls for suspend (and resume). + * For suspend the toolstack initiaties the process by writting an value in + * XenBus "control/shutdown" with the string "suspend". + * + * The PV guest stashes anything it deems neccessary in 'struct start_info' + * in case of failure (PVHVM may ignore this) and calls the + * SCHEDOP_shutdown::SHUTDOWN_suspend hypercall (for PV as argument it + * passes the MFN to 'struct start_info'). + * + * And then the guest is suspended. + * + * At this point the guest may be resumed on the same host under the same + * domain (checkpointing or suspending failed), or on a different host. + * + * The checkpointing or notifying an guest that the suspend failed is by + * having the SCHEDOP_shutdown::SHUTDOWN_suspend hypercall return a non-zero + * value. + * + * The PV and PVHVM resume path are similar. For PV it would be similar to bootup + * - figure out where the 'struct start_info' is (or if the suspend was + * cancelled aka checkpointed - reuse the saved values). + * + * From here on they differ depending whether the guest is PV or PVHVM + * in specifics but follow overall the same path: + * - PV: Bringing up the vCPUS, + * - PVHVM: Setup vector callback, + * - Bring up vCPU runstates, + * - Remap the grant tables if checkpointing or setup from scratch, + * + * + * If the resume was not checkpointing (or if suspend was succesful) we would + * setup the PV timers and the different PV events. Lastly the PV drivers + * re-negotiate with the backend. + * + * This function would return before the guest started resuming. That is + * the guest would be in non-running state and its vCPU context would be + * in the the SCHEDOP_shutdown::SHUTDOWN_suspend hypercall return path + * (for PV and PVHVM). For HVM it would be in would be in QEMU emulated + * BIOS handling S3 suspend. + * * @parm xch a handle to an open hypervisor interface * @parm domid the domain id to resume * @parm fast use cooperative resume (guest must support this)