From patchwork Mon Feb 29 19:59:26 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konrad Rzeszutek Wilk X-Patchwork-Id: 8457771 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 30AB09F2F0 for ; Mon, 29 Feb 2016 20:02:38 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 4B2C6201C8 for ; Mon, 29 Feb 2016 20:02:37 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61F112017D for ; Mon, 29 Feb 2016 20:02:36 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.84) (envelope-from ) id 1aaTyx-0001k7-DJ; Mon, 29 Feb 2016 19:59:51 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.84) (envelope-from ) id 1aaTyw-0001k1-15 for xen-devel@lists.xenproject.org; Mon, 29 Feb 2016 19:59:50 +0000 Received: from [85.158.139.211] by server-4.bemta-5.messagelabs.com id 82/B8-04025-533A4D65; Mon, 29 Feb 2016 19:59:49 +0000 X-Env-Sender: konrad@char.us.oracle.com X-Msg-Ref: server-14.tower-206.messagelabs.com!1456775987!25470556!1 X-Originating-IP: [156.151.31.81] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTU2LjE1MS4zMS44MSA9PiAyODgzMzk=\n X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 9445 invoked from network); 29 Feb 2016 19:59:48 -0000 Received: from userp1040.oracle.com (HELO userp1040.oracle.com) (156.151.31.81) by server-14.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 29 Feb 2016 19:59:48 -0000 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u1TJxVcg020416 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 29 Feb 2016 19:59:32 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u1TJxVwr026927 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 29 Feb 2016 19:59:31 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by aserv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u1TJxTWB028876; Mon, 29 Feb 2016 19:59:29 GMT Received: from char.us.oracle.com (/10.137.176.158) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Feb 2016 11:59:29 -0800 Received: by char.us.oracle.com (Postfix, from userid 1000) id A9C086A0042; Mon, 29 Feb 2016 14:59:28 -0500 (EST) From: Konrad Rzeszutek Wilk To: wency@cn.fujitsu.com, hongyang.yang@easystack.cn, xen-devel@lists.xenproject.org Date: Mon, 29 Feb 2016 14:59:26 -0500 Message-Id: <1456775966-8475-1-git-send-email-konrad.wilk@oracle.com> X-Mailer: git-send-email 2.4.3 X-Source-IP: aserv0022.oracle.com [141.146.126.234] Cc: Wei Liu , Stefano Stabellini , Ian Jackson , Konrad Rzeszutek Wilk Subject: [Xen-devel] [PATCH RFC] libxc: Document xc_domain_resume X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Document the save and suspend mechanism. Signed-off-by: Konrad Rzeszutek Wilk --- tools/libxc/include/xenctrl.h | 52 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h index 150d727..9778947 100644 --- a/tools/libxc/include/xenctrl.h +++ b/tools/libxc/include/xenctrl.h @@ -565,6 +565,58 @@ int xc_domain_destroy(xc_interface *xch, * This function resumes a suspended domain. The domain should have * been previously suspended. * + * Note that there are 'xc_domain_suspend' as suspending a domain + * is quite the endeavour. As such this long comment will describe the + * suspend and resume path. + * + * For the purpose of this explanation there are three guests: + * PV (using hypercalls for privilgied operations), HVM + * (fully hardware virtualized guests using emulated devices for everything), + * and PVHVM (hardware virtualized guest with PV drivers). + * + * HVM guest are the simplest - they suspend via S3 and resume from + * S3. Upon resume they have to re-negotiate with the emulated devices. + * + * PV and PVHVM communate via via hypercalls for suspend (and resume). + * For suspend the toolstack initiaties the process by writting an value in + * XenBus "control/shutdown" with the string "suspend". + * + * The PV guest stashes anything it deems neccessary in 'struct start_info' + * in case of failure (PVHVM may ignore this) and calls the + * SCHEDOP_shutdown::SHUTDOWN_suspend hypercall (for PV as argument it + * passes the MFN to 'struct start_info'). + * + * And then the guest is suspended. + * + * At this point the guest may be resumed on the same host under the same + * domain (checkpointing or suspending failed), or on a different host. + * + * The checkpointing or notifying an guest that the suspend failed is by + * having the SCHEDOP_shutdown::SHUTDOWN_suspend hypercall return a non-zero + * value. + * + * The PV and PVHVM resume path are similar. For PV it would be similar to bootup + * - figure out where the 'struct start_info' is (or if the suspend was + * cancelled aka checkpointed - reuse the saved values). + * + * From here on they differ depending whether the guest is PV or PVHVM + * in specifics but follow overall the same path: + * - PV: Bringing up the vCPUS, + * - PVHVM: Setup vector callback, + * - Bring up vCPU runstates, + * - Remap the grant tables if checkpointing or setup from scratch, + * + * + * If the resume was not checkpointing (or if suspend was succesful) we would + * setup the PV timers and the different PV events. Lastly the PV drivers + * re-negotiate with the backend. + * + * This function would return before the guest started resuming. That is + * the guest would be in non-running state and its vCPU context would be + * in the the SCHEDOP_shutdown::SHUTDOWN_suspend hypercall return path + * (for PV and PVHVM). For HVM it would be in would be in QEMU emulated + * BIOS handling S3 suspend. + * * @parm xch a handle to an open hypervisor interface * @parm domid the domain id to resume * @parm fast use cooperative resume (guest must support this)