From patchwork Wed Jun 16 12:50:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25033C48BE5 for ; Wed, 16 Jun 2021 12:51:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C501B60FDB for ; Wed, 16 Jun 2021 12:51:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C501B60FDB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142936.263600 (Exim 4.92) (envelope-from ) id 1ltV1D-0007QE-2e; Wed, 16 Jun 2021 12:51:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142936.263600; Wed, 16 Jun 2021 12:51:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1C-0007P0-TD; Wed, 16 Jun 2021 12:51:42 +0000 Received: by outflank-mailman (input) for mailman id 142936; Wed, 16 Jun 2021 12:51:41 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1A-0006lZ-WD for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:41 +0000 Received: from mo4-p00-ob.smtp.rzone.de (unknown [81.169.146.220]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 4e6699dc-b1cf-43a2-81bb-a6bd85cbf143; Wed, 16 Jun 2021 12:51:39 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpXtlo (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:33 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4e6699dc-b1cf-43a2-81bb-a6bd85cbf143 ARC-Seal: i=1; a=rsa-sha256; t=1623847893; cv=none; d=strato.com; s=strato-dkim-0002; b=Cdxlo0nJQMRYmW173WlSYKFWf7/BTOjrKQt8Kj6rjd66akrj/b/65Gqx0zAZhprFEw b9MfAK4ql7VcFbOxknAtUQF4kP/445oU9FW6neSBS9rLYdduVeT1mV33QYzbp6RMZQcq dvjCcN0zAF0llJlX40JdmnW5SKgQoqqz/KjZPwsijKfIB2HBZHxQCaH8tYNAzBcbfokI AaHIBlHBHsE6aMZw6xKr/+jy2vAcsyvcsRe4MdAEKiA57KxzBBAcPuTMGGEmj3P7fNks zO+z2d2kPOr/J6iMstJD781c8pihaeG0mRHUAYh4csCzTSMQ61WmFebAJ2T/6oDUeaWh Ko/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847893; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=fttDAmpegUqwpOBB61SFRLpHBJB9wPmVo+V4iKyUqQw=; b=jwlIGgbZKKQpnjD9HSvE2RFElERcmHymT3lbI6MRI/8WtMQLgP/8QqVy9khLXRP9nx LaIIBB/sqeFglBzBuW43cCiCrWtb+fhF/LQ9ekSnMaPs7fOtugjuvJIcHD1RAeA7W0uc O+uViMr2ixMvHJEdeAhZFLyaXJQv5BYL4uKCiU5NiCv8o/teSTnJrYjYb81cySBD6drS 7sBaxUX0Nxi93Dr8hltVhfqfNh1BZc+5I4l/xnTsfQM1wYdZeTOvchyWX7Ep3mYLBokp C6QtuPS+3pGYTn6+MvpTCjONG9a1k2DbhwmpvkBrN0k/TCt+rqPse4cf78Wc7nnzCArD orOw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847893; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=fttDAmpegUqwpOBB61SFRLpHBJB9wPmVo+V4iKyUqQw=; b=OjG0qsM2uRqMdqvr+J6VYuyfYZx+2bMTe2yoHEx5YW4o/f/enW5cCNozCLK+2rskLR 91s0v4n92LNh/vJTe9y/0HSQeDR+x9soRnWkIbNawgmdSLT2p9ZqljbDEaMAOpsnF6nW 5CRSYtkNGAiVQHy8w7wfM/8fNG/YpYfAXrh8QzJ/NsKRe/aRG1/ci6r7PcUTaxkFpDzl griQpPW8S6Kwkl7PxdQfQZhUM7WFoVERQODYRm08VHSufgKuaITlAE3K0dIsQmz68S+G xS7JSQy46bAsmICo3W0JcHagcUJPBWQJN4h2EX6wMSp04v3NjxuLBSksbyN7brLpE/SH OPKA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu Subject: [PATCH v20210616 01/36] hotplug/Linux: fix starting of xenstored with restarting systemd Date: Wed, 16 Jun 2021 14:50:54 +0200 Message-Id: <20210616125129.26563-2-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 A hard to trigger race with another unrelated systemd service and xenstored.service unveiled a bug in the way how xenstored is launched with systemd. launch-xenstore may start either a daemon or a domain. In case a domain is used, systemd-notify was called. If another service triggered a restart of systemd while xenstored.service was executed, systemd may temporary lose track of services with Type=notify. As a result, xenstored.service would be marked as failed and units that depend on it will not be started. This breaks the enire Xen toolstack. The chain of events is basically: xenstored.service sends the notification to systemd, this is a one-way event. Then systemd may be restarted by the other unit. During this time, xenstored.service is done and exits. Once systemd is done with its restart, it collects the pending notifications and childs. If it does not find the unit which sent the notification it will declare it as failed. A workaround for this scenario is to leave the child processes running for a short time after sending the "READY=1" notification. If systemd happens to restart it will still find the unit it launched. Adjust the callers of launch-xenstore to specifiy the init system: Do not fork xenstored with systemd, preserve pid. This wil also avoid the need for a sleep because the process which sent the "READY=1" (the previously forked child) is still alive. Remove the --pid-file in the systemd case because the pid of the child is known, and the file had probably little effect anyway due to lack of PidFile= and Type=forking in the unit file. Be verbose about xenstored startup only with sysv to avoid interleaved output in systemd journal. Do the same also for domain case, even if is not strictly needed because init-xenstore-domain has no output. The fix for upstream systemd which is supposed to fix it: 575b300b795b6 ("pid1: rework how we dispatch SIGCHLD and other signals") Signed-off-by: Olaf Hering --- v04: - do mkdir unconditionally because init-xenstore-domain writes the domid to xenstored.pid v03: - remove run_xenstored function, follow style of shell built-in test function v02: - preserve Type=notify --- tools/hotplug/Linux/init.d/xencommons.in | 2 +- tools/hotplug/Linux/launch-xenstore.in | 40 ++++++++++++++----- .../Linux/systemd/xenstored.service.in | 2 +- 3 files changed, 31 insertions(+), 13 deletions(-) diff --git a/tools/hotplug/Linux/init.d/xencommons.in b/tools/hotplug/Linux/init.d/xencommons.in index 7fd6903b98..dcb0ce4b73 100644 --- a/tools/hotplug/Linux/init.d/xencommons.in +++ b/tools/hotplug/Linux/init.d/xencommons.in @@ -60,7 +60,7 @@ do_start () { mkdir -m700 -p ${XEN_LOCK_DIR} mkdir -p ${XEN_LOG_DIR} - @XEN_SCRIPT_DIR@/launch-xenstore || exit 1 + @XEN_SCRIPT_DIR@/launch-xenstore 'sysv' || exit 1 echo Setting domain 0 name, domid and JSON config... ${LIBEXEC_BIN}/xen-init-dom0 ${XEN_DOM0_UUID} diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in index 019f9d6f4d..d40c66482a 100644 --- a/tools/hotplug/Linux/launch-xenstore.in +++ b/tools/hotplug/Linux/launch-xenstore.in @@ -15,6 +15,17 @@ # License along with this library; If not, see . # +initd=$1 + +case "$initd" in + sysv) nonl='-n' ;; + systemd) nonl= ;; + *) + echo "first argument must be 'sysv' or 'systemd'" + exit 1 + ;; +esac + XENSTORED=@XENSTORED@ . @XEN_SCRIPT_DIR@/hotplugpath.sh @@ -44,14 +55,16 @@ timeout_xenstore () { return 0 } -test_xenstore && exit 0 +mkdir -p @XEN_RUN_DIR@ + +if test "$initd" = 'sysv' ; then + test_xenstore && exit 0 +fi test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons [ "$XENSTORETYPE" = "" ] && XENSTORETYPE=daemon -/bin/mkdir -p @XEN_RUN_DIR@ - [ "$XENSTORETYPE" = "daemon" ] && { [ -z "$XENSTORED_TRACE" ] || XENSTORED_ARGS="$XENSTORED_ARGS -T @XEN_LOG_DIR@/xenstored-trace.log" [ -z "$XENSTORED" ] && XENSTORED=@XENSTORED@ @@ -59,13 +72,15 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF echo "No xenstored found" exit 1 } + [ "$initd" = 'sysv' ] && { + echo $nonl Starting $XENSTORED... + $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS + timeout_xenstore $XENSTORED || exit 1 + exit 0 + } - echo -n Starting $XENSTORED... - $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS - - systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1 - - exit 0 + exec $XENSTORED -N $XENSTORED_ARGS + exit 1 } [ "$XENSTORETYPE" = "domain" ] && { @@ -75,9 +90,12 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE" [ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE" - echo -n Starting $XENSTORE_DOMAIN_KERNEL... + echo $nonl Starting $XENSTORE_DOMAIN_KERNEL... ${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1 - systemd-notify --ready 2>/dev/null + [ "$initd" = 'systemd' ] && { + systemd-notify --ready + sleep 9 + } exit 0 } diff --git a/tools/hotplug/Linux/systemd/xenstored.service.in b/tools/hotplug/Linux/systemd/xenstored.service.in index 80c1d408a5..c226eb3635 100644 --- a/tools/hotplug/Linux/systemd/xenstored.service.in +++ b/tools/hotplug/Linux/systemd/xenstored.service.in @@ -11,7 +11,7 @@ Type=notify NotifyAccess=all RemainAfterExit=true ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities -ExecStart=@XEN_SCRIPT_DIR@/launch-xenstore +ExecStart=@XEN_SCRIPT_DIR@/launch-xenstore 'systemd' [Install] WantedBy=multi-user.target From patchwork Wed Jun 16 12:50:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BA16C48BE5 for ; Wed, 16 Jun 2021 12:51:47 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 07DBA6023D for ; Wed, 16 Jun 2021 12:51:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07DBA6023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142934.263585 (Exim 4.92) (envelope-from ) id 1ltV1B-00076S-LZ; Wed, 16 Jun 2021 12:51:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142934.263585; Wed, 16 Jun 2021 12:51:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1B-00076F-HT; Wed, 16 Jun 2021 12:51:41 +0000 Received: by outflank-mailman (input) for mailman id 142934; Wed, 16 Jun 2021 12:51:40 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1A-00075D-8n for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:40 +0000 Received: from mo4-p01-ob.smtp.rzone.de (unknown [81.169.146.164]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1e989e65-495b-4d05-8988-fae5248b83ef; Wed, 16 Jun 2021 12:51:39 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpXtlp (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:33 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1e989e65-495b-4d05-8988-fae5248b83ef ARC-Seal: i=1; a=rsa-sha256; t=1623847894; cv=none; d=strato.com; s=strato-dkim-0002; b=FeGjylmrxkl+Ci1PpMJGW1Tax2bnjr7O4fv8cyrOoEZvjjd28zNOYElLMC/KY6smkZ MR/Ql/hggF3Op+FrIVRcBDIT0Xjd53gWafL8smW/Ze4Ais3Qn6WkRwNFyEV7rques3NI 3kWV42VBxgxLKEKiTebKmgHaC8/ugSPPwlSZnGT7tsOtT2EvMIck+oxUMs7tQzPyi/QY cLpdl19/1WkfVvwKvuDQkZVQOgq/DSk3s1D9uGJ6qNgw6+e5RkncKEXiCFOfmzENnztA Ihg+zfDU3UIcjvZkxOpQVg2fn2lVcgZMlZUVXRf/z8BMGgmUgAp8h4wKkGFMd7kDq654 kJ1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847894; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=DqV9Bza4xh4lmngXwj1J3LSRM3kpJQsPapKBO4Z/YvI=; b=kh+V1rAx7xgK2zFpG6lcaLCQakgeMdWrwOYnXxg8ivaAECy3YwjOPQ617eESscoi8d 31+z2zGp4Nd8vEaoj8h00aVnVZTdPSzsBL4XWvGOac3xjFx4oIkfBuswAH83QrHL2pSm 0EYkkAMZLD4ZpWms/eLy33qxc6oXwFCVdbuw6iACNwmOv/WKNLPCpVRVMgDYJV0TUCDe EeMggkNZ4Mby7iJw0gEYIVXQkPOSZYM167QgaD8F42jdndQSR82EYkEfg3oFlt7D2j8r UyluyDkkau2rCTNlgNnNmh/Z3pqihoJSEVChtoDm1BN2AmSLG2gUM6s3+DaM2vDUXf/f M0wQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847894; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=DqV9Bza4xh4lmngXwj1J3LSRM3kpJQsPapKBO4Z/YvI=; b=ChJfkLUGcpV13DyNrpdOMxO0gJbRtLCxPMzv0Spyvh1kCIfFls2vLalZ4Mdeg1GSDG mo4tAr7da5bWH5afDonQsTr7EAUfYdKNAqKktQS9YbRSVXp8EuNcqHQJBS2HgJoudS7l MHa5q/KfOoa0USuMnvUQKM/RdidnYaaoQ+DZjGw1KysmVq1zhujC5e/p382DutvOVQbZ H5u1fSJv7zJz7vIxIdzmRVp9hMFfoV3b6USJGzrIKD/j6hxles3STf8pYZMynfZ6VxjA /KIQHWTE5UFpFwaaOlAxLLv/jfVuGuSMPjRmWzxwvrAzlMxRJAOJ2KXvjFv6cp9YkB39 Dm6g== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 02/36] tools: add API to work with sevaral bits at once Date: Wed, 16 Jun 2021 14:50:55 +0200 Message-Id: <20210616125129.26563-3-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Introduce new API to test if a fixed number of bits is clear or set, and clear or set them all at once. The caller has to make sure the input bitnumber is a multiple of BITS_PER_LONG. This API avoids the loop over each bit in a known range just to see if all of them are either clear or set. Signed-off-by: Olaf Hering v02: - change return type from int to bool (jgross) --- tools/libs/ctrl/xc_bitops.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tools/libs/ctrl/xc_bitops.h b/tools/libs/ctrl/xc_bitops.h index f0bac4a071..8e8c6efb45 100644 --- a/tools/libs/ctrl/xc_bitops.h +++ b/tools/libs/ctrl/xc_bitops.h @@ -3,6 +3,7 @@ /* bitmap operations for single threaded access */ +#include #include #include @@ -77,4 +78,31 @@ static inline void bitmap_or(void *_dst, const void *_other, dst[i] |= other[i]; } +static inline bool test_bit_long_set(unsigned long nr_base, const void *_addr) +{ + const unsigned long *addr = _addr; + unsigned long val = addr[nr_base / BITS_PER_LONG]; + + return val == ~0; +} + +static inline bool test_bit_long_clear(unsigned long nr_base, const void *_addr) +{ + const unsigned long *addr = _addr; + unsigned long val = addr[nr_base / BITS_PER_LONG]; + + return val == 0; +} + +static inline void clear_bit_long(unsigned long nr_base, void *_addr) +{ + unsigned long *addr = _addr; + addr[nr_base / BITS_PER_LONG] = 0; +} + +static inline void set_bit_long(unsigned long nr_base, void *_addr) +{ + unsigned long *addr = _addr; + addr[nr_base / BITS_PER_LONG] = ~0; +} #endif /* XC_BITOPS_H */ From patchwork Wed Jun 16 12:50:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53AE9C48BE5 for ; Wed, 16 Jun 2021 12:51:54 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 23CBC60FDB for ; Wed, 16 Jun 2021 12:51:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 23CBC60FDB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142938.263622 (Exim 4.92) (envelope-from ) id 1ltV1H-000881-Ly; Wed, 16 Jun 2021 12:51:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142938.263622; Wed, 16 Jun 2021 12:51:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1H-00087k-HT; Wed, 16 Jun 2021 12:51:47 +0000 Received: by outflank-mailman (input) for mailman id 142938; Wed, 16 Jun 2021 12:51:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1G-0006lZ-0K for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:46 +0000 Received: from mo4-p01-ob.smtp.rzone.de (unknown [85.215.255.53]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id d1943abb-3cfd-4d03-9314-0c71412d5751; Wed, 16 Jun 2021 12:51:41 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpYtlq (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:34 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d1943abb-3cfd-4d03-9314-0c71412d5751 ARC-Seal: i=1; a=rsa-sha256; t=1623847894; cv=none; d=strato.com; s=strato-dkim-0002; b=soFrmfUzg9UNLI8yVxGJmyKafXyp1k0pP6Tz7C6qrjRjIjjInvmcIq7NDWzoXEpYpa is5J4weSbVdhj+Zv+exqhxzn9JCh70HsxVuioyvtd3/icKwIxal2s1iOhU6IiAqeshN4 GE5cuALgLBx29/FjVhUoAtIn17oRmmHc9ygwS581gl/kQ3KK8otGIGOjZeyH+g1lf0Mf wM4lwrrI/6tKseCltsc33Y2MpZNXO/SNS6sdL7AazagYfsUSCfQMy1Rbv4tjcU0Fj6sO o0EsWuYJzd1QLdrcqzgL/jozmedbbylz7MjX5SaezjDDFoHX4XOh3T8nyMx6jn/333eg eoTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847894; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=WwrXv/hX+8hePyd4mOYFfrU0MUgrTtUZ8Gjzx55scrs=; b=CumTvA1YqjZBixzbp99+/N+2MDfv6RVYWAZjCKD6Rf5H1R1lNbsSY6YZObLLUE5Owt 32biwzycgSL2YhhRgHJ8y1+pMc2ek0NExN0ijFfLlNqajcTRu1fINPOMMPCoW+4BSuJP GQnDgF9MWrzopCxL4ZfOuCwsZ4DOvxkHlt4PmuNoeOozH4WT/mrIzH2etYic/7of6/Gq TL8WTiBGP0mT6fz/Of+0rZJr2y0W6svOguW22N+B7dYQ/+JjX7kvrgdZTVk/pCGIlaQS cCJkTCU26Y1hbQZEZ3Xe3PF2xycaGsuyLBJ7EJVSBJJXLNkF16Hf/z5ZQE2XTRl0HZU/ 1cWw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847894; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=WwrXv/hX+8hePyd4mOYFfrU0MUgrTtUZ8Gjzx55scrs=; b=Hdyru/O388OBCl1cJ0YWnVg/2XeuMH7WAZa6zCOYBj2Y0mqdK1mO+sA1FldUjL2AL0 b0snEqEad8XYecbuLL55rwYjFhNs5PeRbVqPttbvjWLEmb7JjURaPEvNMiVFVLuhnDYy kAv6XnK34DIhKUwixo5hwHPBdgrfMA//xNS2j7t+SxN0HEpP1Tlg2TS9g6RJPJAolpxg gzu14toyeV8UqfP8LLUhmnzfvLHgrzSRTvSIYdhpLMhjEHt2il73AVt63UUAQ4GHw8jc Bd28M4cLNrCzsnrkfRKdaoRXblmkSeQ8afmaRpQsF9ZgkaDy6XFOhDfSkIdXEWAw8FRd QY7Q== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu , Anthony PERARD Subject: [PATCH v20210616 03/36] xl: fix description of migrate --debug Date: Wed, 16 Jun 2021 14:50:56 +0200 Message-Id: <20210616125129.26563-4-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 xl migrate --debug used to track every pfn in every batch of pages. But these times are gone. The code in xc_domain_save is the consumer of this knob, but it considers it only for the remus and colo case. Adjust the help text to tell what --debug does today: Nothing. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross v02: - the option has no effect anymore --- docs/man/xl.1.pod.in | 2 +- tools/xl/xl_cmdtable.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index e2176bd696..70a6ebf438 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -481,7 +481,7 @@ domain. =item B<--debug> -Display huge (!) amount of debug information during the migration process. +This option has no effect. It is preserved for compatibility reasons. =item B<-p> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 661323d488..ca1dfa3525 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -172,7 +172,7 @@ const struct cmd_spec cmd_table[] = { " migrate-receive [-d -e]\n" "-e Do not wait in the background (on ) for the death\n" " of the domain.\n" - "--debug Print huge (!) amount of debug during the migration process.\n" + "--debug Ignored.\n" "-p Do not unpause domain after migrating it.\n" "-D Preserve the domain id" }, From patchwork Wed Jun 16 12:50:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C264BC48BE5 for ; Wed, 16 Jun 2021 13:07:00 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 594F16024A for ; Wed, 16 Jun 2021 13:07:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 594F16024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142987.263798 (Exim 4.92) (envelope-from ) id 1ltVFl-0003Bj-VM; Wed, 16 Jun 2021 13:06:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142987.263798; Wed, 16 Jun 2021 13:06:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVFl-0003Bc-RT; Wed, 16 Jun 2021 13:06:45 +0000 Received: by outflank-mailman (input) for mailman id 142987; Wed, 16 Jun 2021 13:06:44 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2D-00075D-9S for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:45 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.171]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0da998a0-f389-4c10-bd1c-49a8d0dc6afe; Wed, 16 Jun 2021 12:51:53 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpYtls (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:34 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0da998a0-f389-4c10-bd1c-49a8d0dc6afe ARC-Seal: i=1; a=rsa-sha256; t=1623847895; cv=none; d=strato.com; s=strato-dkim-0002; b=nlXGn4eAj+/o8cz1jwkwxkYwqZj4Dr4F3XxtmexjzjhsiTBWvfgmgK5bomoEHqWlJa PiUMT8zfvgr6EAAanYx25PEI4Y15fF34UjD/SE+WbAr25BnE1Zz0UI2N5DmltSVsLw+N 7YYCFN0OoBI1WuGBbNYWRNU97SysCiXg06NOUeSzPSJVy+Y6iWmFq3v5c6Uos0piDh+/ jFK5g9f8tWZmcL7utME32WUv0Wo/Y2jt+AirQy0b2ri6RDuLVGE+nOvX6v+sQaFj6a5g nOd4/SZZ4Vaws2zrGV55QYp+uP1THIblELU8UDe8rjbS6tmI7EvZ8w6GzSqGU2ld5d9H sXQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847895; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=31z4r3p8KG3e4KFRiT8bYpR5AIuurjrWpdtWj4tqq5A=; b=b/BtIFTCY9djLKWONeenwbsHzlpRkls0TDQXBh5pl6Jeqz2Ehp9uG1rjZt2EL76E2f wSl9yFL7EtHLQmedHfXcWr0qgOLt1zedkSBpVa2r59xFwxmZD/0OHDsa7diHhzibCKlG w9VOeGbq3zd9TDMTAm7ANYl+5q3qMn26M4hB2Pyznv/w5kbYd0oyo4n1IlwsKBpxTr00 /MBByCzdyvQOG2PzLXUKG/ecw4ZLUhooPsbYm1xgzbyE95VaZuoZVWnDXiiq60MnQ7xf beF4F8IP9WDj5ChW6Gf0zEQApcDJqXV+xMTrLy5K2WnNSCtD6t0DZUSehHFI65SuyOFl 8tXQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847895; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=31z4r3p8KG3e4KFRiT8bYpR5AIuurjrWpdtWj4tqq5A=; b=bkwIWCRIKq4BG8o2IXVVrJLwM1/4ph7G66ofLvqqM/39E8EddtrHv5YUdtmkpGwQg9 /AdYkgexVkhfKqCLwqDvhOCjnOcdHWIFWx1vmK9G1R8qb7yMHmcqzglPx+IQHWxFRvEd Afa/u574eJGlCwaBNGDEaO0jmcJ9J27Ua0THsUvx+K2LK9ilbgNFKcV9RLynQ+pAH2To GMnQvDbTprTESVfklHMl3vp9wdL0ts9gyBMRRH1cj8cMoHz8ddFZCMw6YTLDCOeIyJN/ yC/eg+g63ctZ2AEXSkP3N+2IAkGxd1WJ6JrBJK7pR5YcNf5xfnWuYqzsb5JqB2y7ij/p 3ybw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Wei Liu , Andrew Cooper , George Dunlap , Ian Jackson , Jan Beulich , Julien Grall , Stefano Stabellini , Juergen Gross , Anthony PERARD Subject: [PATCH v20210616 04/36] tools: create libxensaverestore Date: Wed, 16 Jun 2021 14:50:57 +0200 Message-Id: <20210616125129.26563-5-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Move all save/restore related code from libxenguest.so into a separate library libxensaverestore.so. The only consumer is libxl-save-helper. There is no need to have the moved code mapped all the time in binaries where libxenguest.so is used. According to size(1) the change is: text data bss dec hex filename 187183 4304 48 191535 2ec2f guest/libxenguest.so.4.15.0 124106 3376 48 127530 1f22a guest/libxenguest.so.4.15.0 67841 1872 8 69721 11059 saverestore/libxensaverestore.so.4.15.0 While touching the files anyway, take the opportunity to drop the redundant xg_sr_ filename prefix. Signed-off-by: Olaf Hering Acked-by: Wei Liu v5: - fix spelling in description v4: - drop xg_ prefix from filenames (jgross) - drop sr_ prefix from filenames (jbeulich) v3: - repost in time for 4.16 v2: - copy also license header - move xg_nomigrate.c - add size(1) output to commit msg - remove change from libxl_create.c --- .gitignore | 2 + tools/include/xenguest.h | 186 ---------------- tools/include/xensaverestore.h | 208 ++++++++++++++++++ tools/libs/Makefile | 1 + tools/libs/guest/Makefile | 11 - tools/libs/guest/xg_offline_page.c | 1 - tools/libs/light/Makefile | 4 +- tools/libs/light/libxl_internal.h | 1 + tools/libs/light/libxl_save_helper.c | 1 + tools/libs/light/libxl_save_msgs_gen.pl | 2 +- tools/libs/saverestore/Makefile | 38 ++++ .../xg_sr_common.c => saverestore/common.c} | 2 +- .../xg_sr_common.h => saverestore/common.h} | 16 +- .../common_x86.c} | 2 +- .../common_x86.h} | 2 +- .../common_x86_pv.c} | 2 +- .../common_x86_pv.h} | 2 +- .../nomigrate.c} | 0 .../xg_sr_restore.c => saverestore/restore.c} | 2 +- .../restore_x86_hvm.c} | 2 +- .../restore_x86_pv.c} | 2 +- .../xg_sr_save.c => saverestore/save.c} | 2 +- .../save_restore.h} | 2 - .../save_x86_hvm.c} | 2 +- .../save_x86_pv.c} | 2 +- .../stream_format.h} | 0 tools/libs/uselibs.mk | 4 +- 27 files changed, 282 insertions(+), 217 deletions(-) create mode 100644 tools/include/xensaverestore.h create mode 100644 tools/libs/saverestore/Makefile rename tools/libs/{guest/xg_sr_common.c => saverestore/common.c} (99%) rename tools/libs/{guest/xg_sr_common.h => saverestore/common.h} (98%) rename tools/libs/{guest/xg_sr_common_x86.c => saverestore/common_x86.c} (99%) rename tools/libs/{guest/xg_sr_common_x86.h => saverestore/common_x86.h} (98%) rename tools/libs/{guest/xg_sr_common_x86_pv.c => saverestore/common_x86_pv.c} (99%) rename tools/libs/{guest/xg_sr_common_x86_pv.h => saverestore/common_x86_pv.h} (98%) rename tools/libs/{guest/xg_nomigrate.c => saverestore/nomigrate.c} (100%) rename tools/libs/{guest/xg_sr_restore.c => saverestore/restore.c} (99%) rename tools/libs/{guest/xg_sr_restore_x86_hvm.c => saverestore/restore_x86_hvm.c} (99%) rename tools/libs/{guest/xg_sr_restore_x86_pv.c => saverestore/restore_x86_pv.c} (99%) rename tools/libs/{guest/xg_sr_save.c => saverestore/save.c} (99%) rename tools/libs/{guest/xg_save_restore.h => saverestore/save_restore.h} (98%) rename tools/libs/{guest/xg_sr_save_x86_hvm.c => saverestore/save_x86_hvm.c} (99%) rename tools/libs/{guest/xg_sr_save_x86_pv.c => saverestore/save_x86_pv.c} (99%) rename tools/libs/{guest/xg_sr_stream_format.h => saverestore/stream_format.h} (100%) diff --git a/.gitignore b/.gitignore index 38a085e398..08a321e995 100644 --- a/.gitignore +++ b/.gitignore @@ -147,6 +147,8 @@ tools/libs/light/test_timedereg tools/libs/light/test_fdderegrace tools/libs/light/tmp.* tools/libs/light/xenlight.pc +tools/libs/saverestore/libxensaverestore.map +tools/libs/saverestore/xensaverestore.pc tools/libs/stat/_paths.h tools/libs/stat/headers.chk tools/libs/stat/libxenstat.map diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h index 61d0a82f48..7417675b3b 100644 --- a/tools/include/xenguest.h +++ b/tools/include/xenguest.h @@ -24,9 +24,6 @@ #define XC_NUMA_NO_NODE (~0U) -#define XCFLAGS_LIVE (1 << 0) -#define XCFLAGS_DEBUG (1 << 1) - #define X86_64_B_SIZE 64 #define X86_32_B_SIZE 32 @@ -433,189 +430,6 @@ static inline xen_pfn_t xc_dom_p2m(struct xc_dom_image *dom, xen_pfn_t pfn) */ struct xenevtchn_handle; -/* For save's precopy_policy(). */ -struct precopy_stats -{ - unsigned int iteration; - unsigned long total_written; - long dirty_count; /* -1 if unknown */ -}; - -/* - * A precopy_policy callback may not be running in the same address - * space as libxc an so precopy_stats is passed by value. - */ -typedef int (*precopy_policy_t)(struct precopy_stats, void *); - -/* callbacks provided by xc_domain_save */ -struct save_callbacks { - /* - * Called after expiration of checkpoint interval, - * to suspend the guest. - */ - int (*suspend)(void *data); - - /* - * Called before and after every batch of page data sent during - * the precopy phase of a live migration to ask the caller what - * to do next based on the current state of the precopy migration. - * - * Should return one of the values listed below: - */ -#define XGS_POLICY_ABORT (-1) /* Abandon the migration entirely - * and tidy up. */ -#define XGS_POLICY_CONTINUE_PRECOPY 0 /* Remain in the precopy phase. */ -#define XGS_POLICY_STOP_AND_COPY 1 /* Immediately suspend and transmit the - * remaining dirty pages. */ - precopy_policy_t precopy_policy; - - /* - * Called after the guest's dirty pages have been - * copied into an output buffer. - * Callback function resumes the guest & the device model, - * returns to xc_domain_save. - * xc_domain_save then flushes the output buffer, while the - * guest continues to run. - */ - int (*postcopy)(void *data); - - /* - * Called after the memory checkpoint has been flushed - * out into the network. Typical actions performed in this - * callback include: - * (a) send the saved device model state (for HVM guests), - * (b) wait for checkpoint ack - * (c) release the network output buffer pertaining to the acked checkpoint. - * (c) sleep for the checkpoint interval. - * - * returns: - * 0: terminate checkpointing gracefully - * 1: take another checkpoint - */ - int (*checkpoint)(void *data); - - /* - * Called after the checkpoint callback. - * - * returns: - * 0: terminate checkpointing gracefully - * 1: take another checkpoint - */ - int (*wait_checkpoint)(void *data); - - /* Enable qemu-dm logging dirty pages to xen */ - int (*switch_qemu_logdirty)(uint32_t domid, unsigned enable, void *data); /* HVM only */ - - /* to be provided as the last argument to each callback function */ - void *data; -}; - -/* Type of stream. Plain, or using a continuous replication protocol? */ -typedef enum { - XC_STREAM_PLAIN, - XC_STREAM_REMUS, - XC_STREAM_COLO, -} xc_stream_type_t; - -/** - * This function will save a running domain. - * - * @param xch a handle to an open hypervisor interface - * @param io_fd the file descriptor to save a domain to - * @param dom the id of the domain - * @param flags XCFLAGS_xxx - * @param stream_type XC_STREAM_PLAIN if the far end of the stream - * doesn't use checkpointing - * @param recv_fd Only used for XC_STREAM_COLO. Contains backchannel from - * the destination side. - * @return 0 on success, -1 on failure - */ -int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, - uint32_t flags, struct save_callbacks *callbacks, - xc_stream_type_t stream_type, int recv_fd); - -/* callbacks provided by xc_domain_restore */ -struct restore_callbacks { - /* - * Called once the STATIC_DATA_END record has been received/inferred. - * - * For compatibility with older streams, provides a list of static data - * expected to be found in the stream, which was missing. A higher level - * toolstack is responsible for providing any necessary compatibiltiy. - */ -#define XGR_SDD_MISSING_CPUID (1 << 0) -#define XGR_SDD_MISSING_MSR (1 << 1) - int (*static_data_done)(unsigned int missing, void *data); - - /* Called after a new checkpoint to suspend the guest. */ - int (*suspend)(void *data); - - /* - * Called after the secondary vm is ready to resume. - * Callback function resumes the guest & the device model, - * returns to xc_domain_restore. - */ - int (*postcopy)(void *data); - - /* - * A checkpoint record has been found in the stream. - * returns: - */ -#define XGR_CHECKPOINT_ERROR 0 /* Terminate processing */ -#define XGR_CHECKPOINT_SUCCESS 1 /* Continue reading more data from the stream */ -#define XGR_CHECKPOINT_FAILOVER 2 /* Failover and resume VM */ - int (*checkpoint)(void *data); - - /* - * Called after the checkpoint callback. - * - * returns: - * 0: terminate checkpointing gracefully - * 1: take another checkpoint - */ - int (*wait_checkpoint)(void *data); - - /* - * callback to send store gfn and console gfn to xl - * if we want to resume vm before xc_domain_save() - * exits. - */ - void (*restore_results)(xen_pfn_t store_gfn, xen_pfn_t console_gfn, - void *data); - - /* to be provided as the last argument to each callback function */ - void *data; -}; - -/** - * This function will restore a saved domain. - * - * Domain is restored in a suspended state ready to be unpaused. - * - * @param xch a handle to an open hypervisor interface - * @param io_fd the file descriptor to restore a domain from - * @param dom the id of the domain - * @param store_evtchn the xenstore event channel for this domain to use - * @param store_mfn filled with the gfn of the store page - * @param store_domid the backend domain for xenstore - * @param console_evtchn the console event channel for this domain to use - * @param console_mfn filled with the gfn of the console page - * @param console_domid the backend domain for xenconsole - * @param stream_type XC_STREAM_PLAIN if the far end of the stream is using - * checkpointing - * @param callbacks non-NULL to receive a callback to restore toolstack - * specific data - * @param send_back_fd Only used for XC_STREAM_COLO. Contains backchannel to - * the source side. - * @return 0 on success, -1 on failure - */ -int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, - unsigned int store_evtchn, unsigned long *store_mfn, - uint32_t store_domid, unsigned int console_evtchn, - unsigned long *console_mfn, uint32_t console_domid, - xc_stream_type_t stream_type, - struct restore_callbacks *callbacks, int send_back_fd); - /** * This function will create a domain for a paravirtualized Linux * using file names pointing to kernel and ramdisk diff --git a/tools/include/xensaverestore.h b/tools/include/xensaverestore.h new file mode 100644 index 0000000000..0410f0469e --- /dev/null +++ b/tools/include/xensaverestore.h @@ -0,0 +1,208 @@ +/****************************************************************************** + * A library for guest domain save/restore/migration in Xen. + * + * Copyright (c) 2003-2004, K A Fraser. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; If not, see . + */ +#ifndef XENSAVERESTORE_H +#define XENSAVERESTORE_H + +#define XCFLAGS_LIVE (1 << 0) +#define XCFLAGS_DEBUG (1 << 1) + +/* For save's precopy_policy(). */ +struct precopy_stats +{ + unsigned int iteration; + unsigned long total_written; + long dirty_count; /* -1 if unknown */ +}; + +/* + * A precopy_policy callback may not be running in the same address + * space as libxc an so precopy_stats is passed by value. + */ +typedef int (*precopy_policy_t)(struct precopy_stats, void *); + +/* callbacks provided by xc_domain_save */ +struct save_callbacks { + /* + * Called after expiration of checkpoint interval, + * to suspend the guest. + */ + int (*suspend)(void *data); + + /* + * Called before and after every batch of page data sent during + * the precopy phase of a live migration to ask the caller what + * to do next based on the current state of the precopy migration. + * + * Should return one of the values listed below: + */ +#define XGS_POLICY_ABORT (-1) /* Abandon the migration entirely + * and tidy up. */ +#define XGS_POLICY_CONTINUE_PRECOPY 0 /* Remain in the precopy phase. */ +#define XGS_POLICY_STOP_AND_COPY 1 /* Immediately suspend and transmit the + * remaining dirty pages. */ + precopy_policy_t precopy_policy; + + /* + * Called after the guest's dirty pages have been + * copied into an output buffer. + * Callback function resumes the guest & the device model, + * returns to xc_domain_save. + * xc_domain_save then flushes the output buffer, while the + * guest continues to run. + */ + int (*postcopy)(void *data); + + /* + * Called after the memory checkpoint has been flushed + * out into the network. Typical actions performed in this + * callback include: + * (a) send the saved device model state (for HVM guests), + * (b) wait for checkpoint ack + * (c) release the network output buffer pertaining to the acked checkpoint. + * (c) sleep for the checkpoint interval. + * + * returns: + * 0: terminate checkpointing gracefully + * 1: take another checkpoint + */ + int (*checkpoint)(void *data); + + /* + * Called after the checkpoint callback. + * + * returns: + * 0: terminate checkpointing gracefully + * 1: take another checkpoint + */ + int (*wait_checkpoint)(void *data); + + /* Enable qemu-dm logging dirty pages to xen */ + int (*switch_qemu_logdirty)(uint32_t domid, unsigned enable, void *data); /* HVM only */ + + /* to be provided as the last argument to each callback function */ + void *data; +}; + +/* Type of stream. Plain, or using a continuous replication protocol? */ +typedef enum { + XC_STREAM_PLAIN, + XC_STREAM_REMUS, + XC_STREAM_COLO, +} xc_stream_type_t; + +/** + * This function will save a running domain. + * + * @param xch a handle to an open hypervisor interface + * @param io_fd the file descriptor to save a domain to + * @param dom the id of the domain + * @param flags XCFLAGS_xxx + * @param stream_type XC_STREAM_PLAIN if the far end of the stream + * doesn't use checkpointing + * @param recv_fd Only used for XC_STREAM_COLO. Contains backchannel from + * the destination side. + * @return 0 on success, -1 on failure + */ +int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, + uint32_t flags, struct save_callbacks *callbacks, + xc_stream_type_t stream_type, int recv_fd); + +/* callbacks provided by xc_domain_restore */ +struct restore_callbacks { + /* + * Called once the STATIC_DATA_END record has been received/inferred. + * + * For compatibility with older streams, provides a list of static data + * expected to be found in the stream, which was missing. A higher level + * toolstack is responsible for providing any necessary compatibiltiy. + */ +#define XGR_SDD_MISSING_CPUID (1 << 0) +#define XGR_SDD_MISSING_MSR (1 << 1) + int (*static_data_done)(unsigned int missing, void *data); + + /* Called after a new checkpoint to suspend the guest. */ + int (*suspend)(void *data); + + /* + * Called after the secondary vm is ready to resume. + * Callback function resumes the guest & the device model, + * returns to xc_domain_restore. + */ + int (*postcopy)(void *data); + + /* + * A checkpoint record has been found in the stream. + * returns: + */ +#define XGR_CHECKPOINT_ERROR 0 /* Terminate processing */ +#define XGR_CHECKPOINT_SUCCESS 1 /* Continue reading more data from the stream */ +#define XGR_CHECKPOINT_FAILOVER 2 /* Failover and resume VM */ + int (*checkpoint)(void *data); + + /* + * Called after the checkpoint callback. + * + * returns: + * 0: terminate checkpointing gracefully + * 1: take another checkpoint + */ + int (*wait_checkpoint)(void *data); + + /* + * callback to send store gfn and console gfn to xl + * if we want to resume vm before xc_domain_save() + * exits. + */ + void (*restore_results)(xen_pfn_t store_gfn, xen_pfn_t console_gfn, + void *data); + + /* to be provided as the last argument to each callback function */ + void *data; +}; + +/** + * This function will restore a saved domain. + * + * Domain is restored in a suspended state ready to be unpaused. + * + * @param xch a handle to an open hypervisor interface + * @param io_fd the file descriptor to restore a domain from + * @param dom the id of the domain + * @param store_evtchn the xenstore event channel for this domain to use + * @param store_mfn filled with the gfn of the store page + * @param store_domid the backend domain for xenstore + * @param console_evtchn the console event channel for this domain to use + * @param console_mfn filled with the gfn of the console page + * @param console_domid the backend domain for xenconsole + * @param stream_type XC_STREAM_PLAIN if the far end of the stream is using + * checkpointing + * @param callbacks non-NULL to receive a callback to restore toolstack + * specific data + * @param send_back_fd Only used for XC_STREAM_COLO. Contains backchannel to + * the source side. + * @return 0 on success, -1 on failure + */ +int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, + unsigned int store_evtchn, unsigned long *store_mfn, + uint32_t store_domid, unsigned int console_evtchn, + unsigned long *console_mfn, uint32_t console_domid, + xc_stream_type_t stream_type, + struct restore_callbacks *callbacks, int send_back_fd); + +#endif /* XENSAVERESTORE_H */ diff --git a/tools/libs/Makefile b/tools/libs/Makefile index 1afcd12e2b..ca43c66777 100644 --- a/tools/libs/Makefile +++ b/tools/libs/Makefile @@ -12,6 +12,7 @@ SUBDIRS-y += devicemodel SUBDIRS-y += ctrl SUBDIRS-y += guest SUBDIRS-y += hypfs +SUBDIRS-y += saverestore SUBDIRS-y += store SUBDIRS-y += stat SUBDIRS-$(CONFIG_Linux) += vchan diff --git a/tools/libs/guest/Makefile b/tools/libs/guest/Makefile index 2ce92d247e..4cf5459bb1 100644 --- a/tools/libs/guest/Makefile +++ b/tools/libs/guest/Makefile @@ -11,18 +11,7 @@ SRCS-y += xg_domain.c SRCS-y += xg_suspend.c SRCS-y += xg_resume.c ifeq ($(CONFIG_MIGRATE),y) -SRCS-y += xg_sr_common.c -SRCS-$(CONFIG_X86) += xg_sr_common_x86.c -SRCS-$(CONFIG_X86) += xg_sr_common_x86_pv.c -SRCS-$(CONFIG_X86) += xg_sr_restore_x86_pv.c -SRCS-$(CONFIG_X86) += xg_sr_restore_x86_hvm.c -SRCS-$(CONFIG_X86) += xg_sr_save_x86_pv.c -SRCS-$(CONFIG_X86) += xg_sr_save_x86_hvm.c -SRCS-y += xg_sr_restore.c -SRCS-y += xg_sr_save.c SRCS-y += xg_offline_page.c -else -SRCS-y += xg_nomigrate.c endif SRCS-y += xg_core.c SRCS-$(CONFIG_X86) += xg_core_x86.c diff --git a/tools/libs/guest/xg_offline_page.c b/tools/libs/guest/xg_offline_page.c index cfe0e2d537..92b65243b1 100644 --- a/tools/libs/guest/xg_offline_page.c +++ b/tools/libs/guest/xg_offline_page.c @@ -29,7 +29,6 @@ #include "xc_private.h" #include "xg_private.h" -#include "xg_save_restore.h" struct pte_backup_entry { diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile index 7d8c51d492..68e51dd13c 100644 --- a/tools/libs/light/Makefile +++ b/tools/libs/light/Makefile @@ -179,7 +179,7 @@ $(ACPI_OBJS) $(ACPI_PIC_OBJS): CFLAGS += -I. -DLIBACPI_STDUTILS=\"$(CURDIR)/libx $(TEST_PROG_OBJS) _libxl.api-for-check: CFLAGS += $(CFLAGS_libxentoollog) $(CFLAGS_libxentoolcore) libxl_dom.o libxl_dom.opic: CFLAGS += -I$(XEN_ROOT)/tools # include libacpi/x86.h libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -I$(XEN_ROOT)/tools -$(SAVE_HELPER_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenevtchn) $(CFLAGS_libxenguest) +$(SAVE_HELPER_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenevtchn) $(CFLAGS_libxensaverestore) testidl.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight) testidl.c: libxl_types.idl gentest.py $(XEN_INCLUDE)/libxl.h $(AUTOINCS) @@ -241,7 +241,7 @@ test_%: test_%.o test_common.o libxenlight_test.so $(CC) $(LDFLAGS) -o $@ $^ $(filter-out %libxenlight.so, $(LDLIBS_libxenlight)) $(LDLIBS_libxentoollog) $(LDLIBS_libxentoolcore) -lyajl $(APPEND_LDFLAGS) libxl-save-helper: $(SAVE_HELPER_OBJS) libxenlight.so - $(CC) $(LDFLAGS) -o $@ $(SAVE_HELPER_OBJS) $(LDLIBS_libxentoollog) $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest) $(LDLIBS_libxentoolcore) $(APPEND_LDFLAGS) + $(CC) $(LDFLAGS) -o $@ $(SAVE_HELPER_OBJS) $(LDLIBS_libxentoollog) $(LDLIBS_libxenctrl) $(LDLIBS_libxensaverestore) $(LDLIBS_libxentoolcore) $(APPEND_LDFLAGS) testidl: testidl.o libxenlight.so $(CC) $(LDFLAGS) -o $@ testidl.o $(LDLIBS_libxenlight) $(LDLIBS_libxentoollog) $(LDLIBS_libxentoolcore) $(APPEND_LDFLAGS) diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index 0b4671318c..439c654733 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -56,6 +56,7 @@ #define XC_WANT_COMPAT_MAP_FOREIGN_API #include #include +#include #include #include diff --git a/tools/libs/light/libxl_save_helper.c b/tools/libs/light/libxl_save_helper.c index 65dff389bf..896e845a2f 100644 --- a/tools/libs/light/libxl_save_helper.c +++ b/tools/libs/light/libxl_save_helper.c @@ -48,6 +48,7 @@ #include "xenctrl.h" #include "xenguest.h" +#include "xensaverestore.h" #include "_libxl_save_msgs_helper.h" /*----- logger -----*/ diff --git a/tools/libs/light/libxl_save_msgs_gen.pl b/tools/libs/light/libxl_save_msgs_gen.pl index 9d425b1dee..f263ee01bb 100755 --- a/tools/libs/light/libxl_save_msgs_gen.pl +++ b/tools/libs/light/libxl_save_msgs_gen.pl @@ -72,7 +72,7 @@ END_BOTH END_CALLOUT #include -#include +#include #include "_libxl_save_msgs_${ah}.h" END_HELPER diff --git a/tools/libs/saverestore/Makefile b/tools/libs/saverestore/Makefile new file mode 100644 index 0000000000..48728b3be2 --- /dev/null +++ b/tools/libs/saverestore/Makefile @@ -0,0 +1,38 @@ +XEN_ROOT = $(CURDIR)/../../.. +include $(XEN_ROOT)/tools/Rules.mk + +ifeq ($(CONFIG_MIGRATE),y) +SRCS-y += common.c +SRCS-$(CONFIG_X86) += common_x86.c +SRCS-$(CONFIG_X86) += common_x86_pv.c +SRCS-$(CONFIG_X86) += restore_x86_pv.c +SRCS-$(CONFIG_X86) += restore_x86_hvm.c +SRCS-$(CONFIG_X86) += save_x86_pv.c +SRCS-$(CONFIG_X86) += save_x86_hvm.c +SRCS-y += restore.c +SRCS-y += save.c +else +SRCS-y += nomigrate.c +endif + +CFLAGS += -I$(XEN_libxenctrl) +CFLAGS += -I$(XEN_libxenguest) + +-include $(XEN_TARGET_ARCH)/Makefile + +CFLAGS += -Werror -Wmissing-prototypes +CFLAGS += -I. -I./include $(CFLAGS_xeninclude) +CFLAGS += -D__XEN_TOOLS__ +CFLAGS += -include $(XEN_ROOT)/tools/config.h +# Needed for asprintf() +CFLAGS-$(CONFIG_Linux) += -D_GNU_SOURCE + +LIBHEADER := xensaverestore.h + +NO_HEADERS_CHK := y + +include $(XEN_ROOT)/tools/libs/libs.mk + +.PHONY: cleanlocal +cleanlocal: + rm -f libxensaverestore.map diff --git a/tools/libs/guest/xg_sr_common.c b/tools/libs/saverestore/common.c similarity index 99% rename from tools/libs/guest/xg_sr_common.c rename to tools/libs/saverestore/common.c index 17567ab133..77128bc747 100644 --- a/tools/libs/guest/xg_sr_common.c +++ b/tools/libs/saverestore/common.c @@ -1,6 +1,6 @@ #include -#include "xg_sr_common.h" +#include "common.h" #include diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/saverestore/common.h similarity index 98% rename from tools/libs/guest/xg_sr_common.h rename to tools/libs/saverestore/common.h index e2994e18ac..ca2eb47a4f 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/saverestore/common.h @@ -1,13 +1,25 @@ #ifndef __COMMON__H #define __COMMON__H +#include +#include #include +#include +#include +#include +#include +#include +#include + +#include "xc_private.h" +#include "xenguest.h" +#include "xensaverestore.h" #include "xg_private.h" -#include "xg_save_restore.h" +#include "save_restore.h" #include "xc_bitops.h" -#include "xg_sr_stream_format.h" +#include "stream_format.h" /* String representation of Domain Header types. */ const char *dhdr_type_to_str(uint32_t type); diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/saverestore/common_x86.c similarity index 99% rename from tools/libs/guest/xg_sr_common_x86.c rename to tools/libs/saverestore/common_x86.c index 563b4f0168..f1beb234ae 100644 --- a/tools/libs/guest/xg_sr_common_x86.c +++ b/tools/libs/saverestore/common_x86.c @@ -1,4 +1,4 @@ -#include "xg_sr_common_x86.h" +#include "common_x86.h" int write_x86_tsc_info(struct xc_sr_context *ctx) { diff --git a/tools/libs/guest/xg_sr_common_x86.h b/tools/libs/saverestore/common_x86.h similarity index 98% rename from tools/libs/guest/xg_sr_common_x86.h rename to tools/libs/saverestore/common_x86.h index b55758c96d..3a2d91dcb8 100644 --- a/tools/libs/guest/xg_sr_common_x86.h +++ b/tools/libs/saverestore/common_x86.h @@ -1,7 +1,7 @@ #ifndef __COMMON_X86__H #define __COMMON_X86__H -#include "xg_sr_common.h" +#include "common.h" /* * Obtains a domains TSC information from Xen and writes a X86_TSC_INFO record diff --git a/tools/libs/guest/xg_sr_common_x86_pv.c b/tools/libs/saverestore/common_x86_pv.c similarity index 99% rename from tools/libs/guest/xg_sr_common_x86_pv.c rename to tools/libs/saverestore/common_x86_pv.c index c0acf00f90..cfe1b24bed 100644 --- a/tools/libs/guest/xg_sr_common_x86_pv.c +++ b/tools/libs/saverestore/common_x86_pv.c @@ -1,6 +1,6 @@ #include -#include "xg_sr_common_x86_pv.h" +#include "common_x86_pv.h" xen_pfn_t mfn_to_pfn(struct xc_sr_context *ctx, xen_pfn_t mfn) { diff --git a/tools/libs/guest/xg_sr_common_x86_pv.h b/tools/libs/saverestore/common_x86_pv.h similarity index 98% rename from tools/libs/guest/xg_sr_common_x86_pv.h rename to tools/libs/saverestore/common_x86_pv.h index 953b5bfb8d..a9f8c970e3 100644 --- a/tools/libs/guest/xg_sr_common_x86_pv.h +++ b/tools/libs/saverestore/common_x86_pv.h @@ -1,7 +1,7 @@ #ifndef __COMMON_X86_PV_H #define __COMMON_X86_PV_H -#include "xg_sr_common_x86.h" +#include "common_x86.h" /* Virtual address ranges reserved for hypervisor. */ #define HYPERVISOR_VIRT_START_X86_64 0xFFFF800000000000ULL diff --git a/tools/libs/guest/xg_nomigrate.c b/tools/libs/saverestore/nomigrate.c similarity index 100% rename from tools/libs/guest/xg_nomigrate.c rename to tools/libs/saverestore/nomigrate.c diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/saverestore/restore.c similarity index 99% rename from tools/libs/guest/xg_sr_restore.c rename to tools/libs/saverestore/restore.c index b57a787519..be259a1c6b 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/saverestore/restore.c @@ -2,7 +2,7 @@ #include -#include "xg_sr_common.h" +#include "common.h" /* * Read and validate the Image and Domain headers. diff --git a/tools/libs/guest/xg_sr_restore_x86_hvm.c b/tools/libs/saverestore/restore_x86_hvm.c similarity index 99% rename from tools/libs/guest/xg_sr_restore_x86_hvm.c rename to tools/libs/saverestore/restore_x86_hvm.c index d6ea6f3012..bd63bd2818 100644 --- a/tools/libs/guest/xg_sr_restore_x86_hvm.c +++ b/tools/libs/saverestore/restore_x86_hvm.c @@ -1,7 +1,7 @@ #include #include -#include "xg_sr_common_x86.h" +#include "common_x86.h" /* * Process an HVM_CONTEXT record from the stream. diff --git a/tools/libs/guest/xg_sr_restore_x86_pv.c b/tools/libs/saverestore/restore_x86_pv.c similarity index 99% rename from tools/libs/guest/xg_sr_restore_x86_pv.c rename to tools/libs/saverestore/restore_x86_pv.c index dc50b0f5a8..96608e5231 100644 --- a/tools/libs/guest/xg_sr_restore_x86_pv.c +++ b/tools/libs/saverestore/restore_x86_pv.c @@ -1,6 +1,6 @@ #include -#include "xg_sr_common_x86_pv.h" +#include "common_x86_pv.h" static xen_pfn_t pfn_to_mfn(const struct xc_sr_context *ctx, xen_pfn_t pfn) { diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/saverestore/save.c similarity index 99% rename from tools/libs/guest/xg_sr_save.c rename to tools/libs/saverestore/save.c index 2ba7c3200c..ae3e8797d0 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/saverestore/save.c @@ -1,7 +1,7 @@ #include #include -#include "xg_sr_common.h" +#include "common.h" /* * Writes an Image header and Domain header into the stream. diff --git a/tools/libs/guest/xg_save_restore.h b/tools/libs/saverestore/save_restore.h similarity index 98% rename from tools/libs/guest/xg_save_restore.h rename to tools/libs/saverestore/save_restore.h index 3dbbc8dcd2..20bd3d30a5 100644 --- a/tools/libs/guest/xg_save_restore.h +++ b/tools/libs/saverestore/save_restore.h @@ -15,8 +15,6 @@ * License along with this library; If not, see . */ -#include "xc_private.h" - #include #include diff --git a/tools/libs/guest/xg_sr_save_x86_hvm.c b/tools/libs/saverestore/save_x86_hvm.c similarity index 99% rename from tools/libs/guest/xg_sr_save_x86_hvm.c rename to tools/libs/saverestore/save_x86_hvm.c index 1634a7bc43..91c2cb99ab 100644 --- a/tools/libs/guest/xg_sr_save_x86_hvm.c +++ b/tools/libs/saverestore/save_x86_hvm.c @@ -1,6 +1,6 @@ #include -#include "xg_sr_common_x86.h" +#include "common_x86.h" #include diff --git a/tools/libs/guest/xg_sr_save_x86_pv.c b/tools/libs/saverestore/save_x86_pv.c similarity index 99% rename from tools/libs/guest/xg_sr_save_x86_pv.c rename to tools/libs/saverestore/save_x86_pv.c index 4964f1f7b8..92f77fad0f 100644 --- a/tools/libs/guest/xg_sr_save_x86_pv.c +++ b/tools/libs/saverestore/save_x86_pv.c @@ -1,7 +1,7 @@ #include #include -#include "xg_sr_common_x86_pv.h" +#include "common_x86_pv.h" /* Check a 64 bit virtual address for being canonical. */ static inline bool is_canonical_address(xen_vaddr_t vaddr) diff --git a/tools/libs/guest/xg_sr_stream_format.h b/tools/libs/saverestore/stream_format.h similarity index 100% rename from tools/libs/guest/xg_sr_stream_format.h rename to tools/libs/saverestore/stream_format.h diff --git a/tools/libs/uselibs.mk b/tools/libs/uselibs.mk index efd7a475ba..62a2990b95 100644 --- a/tools/libs/uselibs.mk +++ b/tools/libs/uselibs.mk @@ -20,6 +20,8 @@ LIBS_LIBS += ctrl USELIBS_ctrl := toollog call evtchn gnttab foreignmemory devicemodel LIBS_LIBS += guest USELIBS_guest := evtchn ctrl +LIBS_LIBS += saverestore +USELIBS_saverestore := guest ctrl LIBS_LIBS += store USELIBS_store := toolcore LIBS_LIBS += vchan @@ -27,7 +29,7 @@ USELIBS_vchan := toollog store gnttab evtchn LIBS_LIBS += stat USELIBS_stat := ctrl store LIBS_LIBS += light -USELIBS_light := toollog evtchn toolcore ctrl store hypfs guest +USELIBS_light := toollog evtchn toolcore ctrl store hypfs guest saverestore LIBS_LIBS += util USELIBS_util := light FILENAME_util := xlutil From patchwork Wed Jun 16 12:50:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98C73C48BE6 for ; Wed, 16 Jun 2021 13:08:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6C75E61166 for ; Wed, 16 Jun 2021 13:08:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C75E61166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143102.263948 (Exim 4.92) (envelope-from ) id 1ltVHR-0002U7-DL; Wed, 16 Jun 2021 13:08:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143102.263948; Wed, 16 Jun 2021 13:08:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHR-0002Rg-47; Wed, 16 Jun 2021 13:08:29 +0000 Received: by outflank-mailman (input) for mailman id 143102; Wed, 16 Jun 2021 13:08:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV24-0006lZ-1j for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:36 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.168]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id fb714658-f09d-4355-abea-3ce246a2aade; Wed, 16 Jun 2021 12:51:46 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpZtlt (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:35 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fb714658-f09d-4355-abea-3ce246a2aade ARC-Seal: i=1; a=rsa-sha256; t=1623847895; cv=none; d=strato.com; s=strato-dkim-0002; b=qAkzC+afFgUE2AnwHG51nJsrR/LZDNYKYoEMoNM4kCUpKNtHmro7Np4B815L2h4/1g OeqgG/tiTpj2WQAwTgrLRBZ4aZZHL81XM36sgup0bRzZeKPPX8DUCTNMbRRpdJwDxEZV Nlq4v33jpLivhyAfEc7dSo+vrORLONwqD/s64oqCusBkrqhELfSZyqxm6gNjfsuRYpQA FDHjiup36JBAq4NKf3jC4muOjHk2qJKkyUDjwcKNc+4Lw7vnvxwhYXJI5bnEQ+NLyKaO 0FFyjQJWuehqYfvIim9VPmTPIZF9XBngFkPKerw3Kl52eP4L/duQz/bnZy2rNRE+Ccz3 wUAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847895; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=5OMdhD/BLm6v061HzJKSHTlMezBek4cccWDJ4F+kDME=; b=PQ+LkxJnNdz6yZBR0CqNxUAh4SuxdOqYH1yC8YUESK0EbnzPzPjltUA7pNeWvMb8Iy L/dNpzTQGTCxLOHxPxNhQuDB2xCQthIFsmIa3kcfzSW45uqVUZStGTdiCRZI0QXJbmTg QQplmIyAnaww8FUQj8YoqgsweJ1GLHxZ3Vi+xBE47BTIVBxMXlBSvyIcwgHs7oZWyt+i zkO2sHjYW3tMhgZujijdPkRsYe8AIsoJbLGTbA0j5YkODZ8hU9DJu+//1YLTgssHHW13 YH4K0+TCQJF591Vie2PXeyijbabry8ZuSmBEfEPE2JRIiMSrqaDRs60/l61Ejbv0ZeMw Qz/w== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847895; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=5OMdhD/BLm6v061HzJKSHTlMezBek4cccWDJ4F+kDME=; b=PbR5bZJ6nLqe9xCJmbIqduxFqGtJHgwz3s6xQG465ikfrxnVkdMmkbIAo8PvIuH6IL 92LHUy9u0lYUp9FYSpTefqPdLKQa+1THa/qtlfu0PGlN95RBe3jeZZ2CCKlAInHG4bt+ EwxrdqvSugw1uIKvr502hcjlfCsanzoDY8m/GzvsXASx+e7Y9Et0oSxZi4NY1jkmi4e2 QS26aZNyMSpwiCnzztnDTVbtYorO3O0IaYvRJqAtum+qYXjKdmzgdDC2rEAq5YJlTB/g B078TaGMtHuw3Ffvb9gHtHKzFdV1aMmBxaR2nKLR3KQnWsLkc3EQFxgaPJJJ8iI/3btw 84uw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Andrew Cooper , George Dunlap , Ian Jackson , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v20210616 05/36] MAINTAINERS: add myself as saverestore maintainer Date: Wed, 16 Jun 2021 14:50:58 +0200 Message-Id: <20210616125129.26563-6-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 I touched it last. Signed-off-by: Olaf Hering --- MAINTAINERS | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 39750bb75d..dbb8f56ab3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -381,6 +381,12 @@ R: Juergen Gross S: Supported F: tools/libs/ +LIBSAVERESTORE: +M: Olaf Hering +S: Supported +F: tools/include/xensaverestore.h +F: tools/libs/saverestore/ + LIBXENLIGHT M: Ian Jackson M: Wei Liu From patchwork Wed Jun 16 12:50:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2540AC48BE8 for ; Wed, 16 Jun 2021 12:51:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E48F96023D for ; Wed, 16 Jun 2021 12:51:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E48F96023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142937.263611 (Exim 4.92) (envelope-from ) id 1ltV1E-0007hV-DW; Wed, 16 Jun 2021 12:51:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142937.263611; Wed, 16 Jun 2021 12:51:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1E-0007h6-7G; Wed, 16 Jun 2021 12:51:44 +0000 Received: by outflank-mailman (input) for mailman id 142937; Wed, 16 Jun 2021 12:51:43 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1D-00075D-3i for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:43 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.171]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 71b8fcae-6fc2-487b-a5d1-2823a9646c44; Wed, 16 Jun 2021 12:51:42 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpatlu (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:36 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 71b8fcae-6fc2-487b-a5d1-2823a9646c44 ARC-Seal: i=1; a=rsa-sha256; t=1623847896; cv=none; d=strato.com; s=strato-dkim-0002; b=gg+/47bZpm4YB6I1hv7GH0kgAMdek5epu5Z7Cwt0HbMxD0mv2NJDtrvIS/eJUdH9Qd ZiybM/Yt17xfoQGSa6wYbBmqLbvB/pxNUvndEP6KTGsfIsS2ZG79Dk4RxrTmHibYjcg+ 8H/ZbGhHgEgAfQpexEViICbi8NheNQKlRTo6bnXMe5tL99sE109F2PIuLdKZ9tnBeqSi wMxb2h9te3iLP/zkRNGQmdCKG1H/zGt/P4NOCkz+VmpUTa83mfwcJ5HTYBZGmm7MaM2F qgOy+5DoQ4CjvBjEszbAb5COq8CDl7PDPyEsrSTM/78MGURuE2fdHQlt6M+Gshkh1Gn5 o0RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847896; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2reXKEzUM07y7ty9jLpkdpjD7wjbIcJf0B0ZkXQLZnc=; b=noUc6A2hE/RWnNf1+XuBctR6/Mytab/IRCwImeUw18mDKqqnrgPeu+7MzVAvTB03hN rEG688W0NlVcQRBaYy6Lx/ih5DbKkOm7LcTXbawrnGuboA2OvUYtJiSNv8rfA3773O3Q FG+lDUuyEyReBBaa4Wx0tQNf6HoKwYZRl5ftbwll1RvFF45WGuXOLWNDA7sDV0eOtB/1 Nb3Ghmo2E8Hnz9q8F66AKHYAHzqT+ZyQbr3Rkh+UI31AP5Iwez/4yF/tGli4H5B66v0d uR4yhi3ZQNL0J6rwVlPstvmlnkuQr7yD9d1V0DbaGaPdXlwd1gR9PhLIhC23bd+t/b0c 7qKw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847896; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2reXKEzUM07y7ty9jLpkdpjD7wjbIcJf0B0ZkXQLZnc=; b=oQ6Ua7JrtYUy0TVkX1AMIVu4rj1i2r4SR9AoWp9b1P2B8U13DXbBSJMBR1axLkAq/7 u968g+YGpSbMeujc1bTK8kmN5rcz9zy5evUojInagHDIigYu2pFDVMZEJwCCsPexyo79 gXg6U+cySyKwRQ8VCOD40ZcLDKp5skuPUBwzpVvMC5SJUgdR/jqlX0qfE+1Udjq1azjx MZZWpYrpNJ5HarkwBph81RsEAT8bNLapCjGooGmxCK6eSMdc4Uo+HoiSuw7wbrTCqayh 48CKdAPnvWtLLIq96r6KB8UwyBMN2NeOjAPM6NbPzCdRskFRCfz8tI/c9HnhQUG9oXvA irHA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 06/36] tools: add readv_exact to libxenctrl Date: Wed, 16 Jun 2021 14:50:59 +0200 Message-Id: <20210616125129.26563-7-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Read a batch of iovec's. Short reads are the common case, finish the trailing iov with read_exact. Signed-off-by: Olaf Hering v2: - add comment to short-read handling --- tools/libs/ctrl/xc_private.c | 57 +++++++++++++++++++++++++++++++++++- tools/libs/ctrl/xc_private.h | 1 + 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/tools/libs/ctrl/xc_private.c b/tools/libs/ctrl/xc_private.c index d94f846686..da58c3d9ba 100644 --- a/tools/libs/ctrl/xc_private.c +++ b/tools/libs/ctrl/xc_private.c @@ -659,8 +659,23 @@ int write_exact(int fd, const void *data, size_t size) #if defined(__MINIOS__) /* - * MiniOS's libc doesn't know about writev(). Implement it as multiple write()s. + * MiniOS's libc doesn't know about readv/writev(). + * Implement it as multiple read/write()s. */ +int readv_exact(int fd, const struct iovec *iov, int iovcnt) +{ + int rc, i; + + for ( i = 0; i < iovcnt; ++i ) + { + rc = read_exact(fd, iov[i].iov_base, iov[i].iov_len); + if ( rc ) + return rc; + } + + return 0; +} + int writev_exact(int fd, const struct iovec *iov, int iovcnt) { int rc, i; @@ -675,6 +690,46 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt) return 0; } #else +int readv_exact(int fd, const struct iovec *iov, int iovcnt) +{ + int rc = 0, idx = 0; + ssize_t len; + + while ( idx < iovcnt ) + { + len = readv(fd, &iov[idx], min(iovcnt - idx, IOV_MAX)); + if ( len == -1 && errno == EINTR ) + continue; + if ( len <= 0 ) + { + rc = -1; + goto out; + } + + /* Finish a potential short read in the last iov */ + while ( len > 0 && idx < iovcnt ) + { + if ( len >= iov[idx].iov_len ) + { + len -= iov[idx].iov_len; + } + else + { + void *p = iov[idx].iov_base + len; + size_t l = iov[idx].iov_len - len; + + rc = read_exact(fd, p, l); + if ( rc ) + goto out; + len = 0; + } + idx++; + } + } +out: + return rc; +} + int writev_exact(int fd, const struct iovec *iov, int iovcnt) { struct iovec *local_iov = NULL; diff --git a/tools/libs/ctrl/xc_private.h b/tools/libs/ctrl/xc_private.h index 3e299b943f..66086ef19f 100644 --- a/tools/libs/ctrl/xc_private.h +++ b/tools/libs/ctrl/xc_private.h @@ -410,6 +410,7 @@ int xc_flush_mmu_updates(xc_interface *xch, struct xc_mmu *mmu); /* Return 0 on success; -1 on error setting errno. */ int read_exact(int fd, void *data, size_t size); /* EOF => -1, errno=0 */ +int readv_exact(int fd, const struct iovec *iov, int iovcnt); int write_exact(int fd, const void *data, size_t size); int writev_exact(int fd, const struct iovec *iov, int iovcnt); From patchwork Wed Jun 16 12:51:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 650B8C48BE6 for ; Wed, 16 Jun 2021 12:52:06 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 253F660FE9 for ; Wed, 16 Jun 2021 12:52:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 253F660FE9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142944.263664 (Exim 4.92) (envelope-from ) id 1ltV1S-0001WF-1i; Wed, 16 Jun 2021 12:51:58 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142944.263664; Wed, 16 Jun 2021 12:51:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1R-0001Tj-MP; Wed, 16 Jun 2021 12:51:57 +0000 Received: by outflank-mailman (input) for mailman id 142944; Wed, 16 Jun 2021 12:51:56 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1Q-0006lZ-0A for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:56 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.83]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id cece85ed-2d27-4fdd-8717-480416772f0b; Wed, 16 Jun 2021 12:51:43 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpatlv (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:36 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cece85ed-2d27-4fdd-8717-480416772f0b ARC-Seal: i=1; a=rsa-sha256; t=1623847896; cv=none; d=strato.com; s=strato-dkim-0002; b=dxOtudDpDUxA61h6q5gaxkGxLyyUZ0Qhf56ZBw5pvtCPzZyGAT+05ORT39AQQSr2GL cOJ6ZQ+jCQOg6eWlv4mIPMeZwPXlLrUjhw40KsAgIVPchaLrVNwOSsvZ26CanAJ5GDn1 e3lyeeON91UPhwe+gTQpxToQRTj7Tu5bE7Z1CDguN+O3sjVy2cA7OqYNmgp0YIojewEN pXRkAItVbuHNTwsh0MO7jc+G6EnlEJUiFL0TYf+0rFGHHyJ1d60pzWPSjkMrrFRJwAks OqF2ASJoSouRjLhCM4oAeVO3vX123nYGdsINbfyx+CfEUdmh4lb0smVwYK+8JvrcDi3V OMcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847896; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ikZ54MUbm6J0+L4XW5ARSvIze4KQJX0wbR+SClf730w=; b=Li9M5NuoXwVzfog2OtC6dmydfnROU2UPs+KqlXq/sizYlcp2z8JA1F1Fa7HPKCTfQF Cdf6ROhmdEJtMN7UeEf/BOI/r6KYbHTTXegtQksvsO/OIHkAisdDcCZ0tiwPqpuxIs10 RrRHP/OMaMfNq47rot0BJL7SU+pSMqi11mCd9W7zZyc8BhS6nSEdtS17beoy3vugwAGF NLEGRJbGP4AkEjtjiuV7PtjA4LPq0lXBRZcGXIRAhxt3zAn/d5vtfdQmU+ggh1AUnJIj x5nM0HhegNmyIaGQENmES70rr4jjGH8tLlPEEjUnN/qTQGzV6unni4r0I3xCKNmFfbNA 4JiA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847896; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ikZ54MUbm6J0+L4XW5ARSvIze4KQJX0wbR+SClf730w=; b=UIsPIfYTqJ11BJg4cVQSTqrMfNiTuot30qjQsOw8l2Zh0GFtCmMhXoLh+mS7NmkjR9 KsOOPifUa9IHi4sxnQPt+CuIlzPk5Z2wpeBspqFbXJQ+T40WPYFOHvzYydzu/PJlU0w+ vsqQvAW83UgmmIDfDCo5M5lMQKBF7i24W0tUQp350Omma930GaS4sTbUwQkamAEpnq2O GjjTvU3t8D/spCCn9oP7gPSiZmkjZbRAwnSjhGU9yRtrauVWgB9Ppg15cBkoWlid50jS r5Nfvbm3NyQgxiOdVBb3vHaWh25+ZlhHmP220QJTlJsn0mnlW+noM8fmZXTQhSOAJCVX 3IAA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 07/36] tools: add sr_is_known_page_type to libsaverestore Date: Wed, 16 Jun 2021 14:51:00 +0200 Message-Id: <20210616125129.26563-8-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Users of xc_get_pfn_type_batch may want to sanity check the data returned by Xen. Add a simple helper for this purpose. Signed-off-by: Olaf Hering v02: - rename xc_is_known_page_type to sr_is_known_page_type - move from ctrl/xc_private.h to saverestore/common.h (jgross) --- tools/libs/saverestore/common.h | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index ca2eb47a4f..c9cc4206e5 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -467,6 +467,39 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, /* Handle a STATIC_DATA_END record. */ int handle_static_data_end(struct xc_sr_context *ctx); +/* Sanitiy check for types returned by Xen */ +static inline bool sr_is_known_page_type(xen_pfn_t type) +{ + bool ret; + + switch (type) + { + case XEN_DOMCTL_PFINFO_NOTAB: + + case XEN_DOMCTL_PFINFO_L1TAB: + case XEN_DOMCTL_PFINFO_L1TAB | XEN_DOMCTL_PFINFO_LPINTAB: + + case XEN_DOMCTL_PFINFO_L2TAB: + case XEN_DOMCTL_PFINFO_L2TAB | XEN_DOMCTL_PFINFO_LPINTAB: + + case XEN_DOMCTL_PFINFO_L3TAB: + case XEN_DOMCTL_PFINFO_L3TAB | XEN_DOMCTL_PFINFO_LPINTAB: + + case XEN_DOMCTL_PFINFO_L4TAB: + case XEN_DOMCTL_PFINFO_L4TAB | XEN_DOMCTL_PFINFO_LPINTAB: + + case XEN_DOMCTL_PFINFO_XTAB: + case XEN_DOMCTL_PFINFO_XALLOC: + case XEN_DOMCTL_PFINFO_BROKEN: + ret = true; + break; + default: + ret = false; + break; + } + return ret; +} + #endif /* * Local variables: From patchwork Wed Jun 16 12:51:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5E42C48BE8 for ; Wed, 16 Jun 2021 12:51:59 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 771E96109D for ; Wed, 16 Jun 2021 12:51:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 771E96109D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142942.263644 (Exim 4.92) (envelope-from ) id 1ltV1M-0000Wl-SY; Wed, 16 Jun 2021 12:51:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142942.263644; Wed, 16 Jun 2021 12:51:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1M-0000WM-Nr; Wed, 16 Jun 2021 12:51:52 +0000 Received: by outflank-mailman (input) for mailman id 142942; Wed, 16 Jun 2021 12:51:51 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1L-0006lZ-0A for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:51 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.83]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 43aa6e05-a993-430c-9968-2ac1ce447547; Wed, 16 Jun 2021 12:51:42 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpbtlx (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:37 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 43aa6e05-a993-430c-9968-2ac1ce447547 ARC-Seal: i=1; a=rsa-sha256; t=1623847897; cv=none; d=strato.com; s=strato-dkim-0002; b=ODGNr+uLUxhmfATeLKkGIHOLU7hPqdY/15IH6h60god/q+pChYN/+cU/VtTjPuTghE jO3D94kvRZV6Ghcif00Qn7nAg5e6aZ24xw5UuRhJneWquP5pzoRrnlrLhyf5ef8fiFM7 X00POAishpBGFDOoOnR+wLUfUfGm2WO7MbeDjCnwSQL5G9mm5rsYzh3FfUQv5NiBtmUz E3j2vwEWDaQlZRVeWPnZXFLDYnib22vpQML1mYppvhx/y6dJuAXSW2Bpizg+dgTJKmZF AV8db9yNq6tkbHzr/stXZB7a1DcJYEVGcHxh0E2fhDE6HdRFON18uBiaffB7DKIh3DIR m3oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847897; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ex4r4d11Mg1k2Zn9qFLItgRI9LdfJn43Y1sVEepvWiU=; b=ElVLRXcEYGZ3OxVQGsKXyhDRMLIeRmWr/Lkb/L3jeDK4+X10Bth8quN5EG31PNmK6N MT/ImowTtg+rh41UDlCSHPGXI6AMt6Y4dC/zB1xQu5x/nAEg/0SXlxT3c2LfoTDZLSA5 hD0cTPXB/a0xPaND1wObq1JViiEJTf1647uG+UON/Wd/YtqRXb5AYPgm1L9p7sa2KUXN 4Cn+UgXWFAxZIGhvAKVkIOVJ5wR6M63A9NXClgqGJ1KYJm/LqrgcbOPUsylFop8qU3ah ZeC5i8Qzfd8H6UdET/GYRPRMXR/hbFYy0l6JgKx1+43KGE1PDry4SSMILYrtvgo91F/W zR7w== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847897; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ex4r4d11Mg1k2Zn9qFLItgRI9LdfJn43Y1sVEepvWiU=; b=rdPVMskwM4FKzs88imzG5wXXgdzUPInSgw0NFn1sl+Dzq3JBU3VUXKySfJ6qe2EzVz 2ZDqzXJOgseB/2eka0fb4Z60lHbMp27kxYSX/UjquR6H8+1xEmwKM1FlI9d4cYQEYCB5 IYZK6hnWD4msa5fEnx83JgfMkyLHZSDPrGuGW2y0quwR86rL7jztf+w+kb6X8+uE4E1/ ELJWvIJmzCxcPiBOTeR9pR4nZ15S5JF2HEp057EUeAU/TMhUSrBljTT1wpb8NC7StN9u 9qYjlaiREmgnFxRnwjvtGuq/LMMcr7FGHcOqx0S20QlqEgdQszkC+65poKZoonPinfQQ ocBQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 08/36] tools: use sr_is_known_page_type Date: Wed, 16 Jun 2021 14:51:01 +0200 Message-Id: <20210616125129.26563-9-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Verify pfn type on sending side, also verify incoming batch of pfns. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross v02: - use sr_is_known_page_type instead of xc_is_known_page_type --- tools/libs/saverestore/restore.c | 3 +-- tools/libs/saverestore/save.c | 6 ++++++ 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index be259a1c6b..324b9050e2 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -406,8 +406,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) } type = (pages->pfn[i] & PAGE_DATA_TYPE_MASK) >> 32; - if ( ((type >> XEN_DOMCTL_PFINFO_LTAB_SHIFT) >= 5) && - ((type >> XEN_DOMCTL_PFINFO_LTAB_SHIFT) <= 8) ) + if ( sr_is_known_page_type(type) == false ) { ERROR("Invalid type %#"PRIx32" for pfn %#"PRIpfn" (index %u)", type, pfn, i); diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index ae3e8797d0..6f820ea432 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -147,6 +147,12 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; i < nr_pfns; ++i ) { + if ( sr_is_known_page_type(types[i]) == false ) + { + ERROR("Wrong type %#"PRIpfn" for pfn %#"PRIpfn, types[i], mfns[i]); + goto err; + } + switch ( types[i] ) { case XEN_DOMCTL_PFINFO_BROKEN: From patchwork Wed Jun 16 12:51:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46C83C48BE8 for ; Wed, 16 Jun 2021 12:52:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EFDE56023D for ; Wed, 16 Jun 2021 12:52:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFDE56023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142948.263683 (Exim 4.92) (envelope-from ) id 1ltV1X-0002T6-3G; Wed, 16 Jun 2021 12:52:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142948.263683; Wed, 16 Jun 2021 12:52:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1W-0002SG-LF; Wed, 16 Jun 2021 12:52:02 +0000 Received: by outflank-mailman (input) for mailman id 142948; Wed, 16 Jun 2021 12:52:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1V-0006lZ-0M for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:01 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.171]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a13aee9a-ffdd-4c72-83ef-ec013995368e; Wed, 16 Jun 2021 12:51:43 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpbtly (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:37 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a13aee9a-ffdd-4c72-83ef-ec013995368e ARC-Seal: i=1; a=rsa-sha256; t=1623847897; cv=none; d=strato.com; s=strato-dkim-0002; b=DYoFmxn0Dtbm+VbaZKlDCbmsImsyeyVIbmdVja/WYH7Yh7b3VxzyV+95Rv/od0I/cz yz5v+ihbmp6qtr+klmkkX01fY3hxfjDnHfHy82NnPZkcFtuTgOHOvwxSr3wEgXANEMIL 9vano6TGgNPAjgJ6A26sqYCTr+FtzFAaLdLlib64ZsvtD8UJGPZ5jWklAtDkf8CqqWgI NhVoMJ9ZxY4oM2AKtBImnUrA5r8MOYK3gWUZgRHbM/zThMNEN5eklers7DqmTQ8MqZAh LEM+/N/41WuijmQ27TTLid+o5rNMAVqVa5eaTZL/7YYlDv/kPRbLuoJqqRObjVrNsn6d 6eWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847897; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=VuUrMSQO4CJqxpoErSuXThYpl5ESMJog6Ej/du7Jhm8=; b=L7BIenlD2gLFJY33mpjNEtwdvigxvmiQu8kcfsKPRIaLNvBAF0MhNd3rn7RaNc4F+S Iwk3Iff43fgyXwqIo3e+JG/9f8eoWGSR1QVY44AiV42Tv9mn2D+AALn/110YsgRbJF/7 HUbWydzTUDZQYMWhV86LHLRjJFdriIlXI5/62DRsel2b3g5ALbiohbzX8yDTbZuG0OYo 2vjvgVJOSDRCm9nTNlngBefJc0QYwsrXZUPs5wY/HQYYz6WUNXxIkAB/Inb5b/rQkIHS prRgL18wVgHo1pcWpKYV8SrWVdQHa3RA/NZtYgotR/OEIhwa3QJdaTW+ax+2iOFnWcud Bu6g== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847897; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=VuUrMSQO4CJqxpoErSuXThYpl5ESMJog6Ej/du7Jhm8=; b=fgVuiq6lNo14eMaaSHNN964UZYKMmBml6lOvEjdDo157lZMXSwttn3KdVtF/ybsVr0 V/1rNHX9ojTlroR5uczwa5SglFU2HT0cvO3VTN+xnF/kXb7/v3Y86wkExfAhM49LTU44 naNCWgKSBgJqtYDjrzEzsPXh7BQnPXNELk2GhIN2qANRGy8XPKq9k/ZlUsPRcwo/WN43 HnpOfBzpqr30GsKcBchylGKkbEgiaYju/NEaAGwGZhg4uZorNoAKz0yELULez/AUL/dQ Aa8yILmfNI6g5c4dDmOLKZ3c+S7OQZL7cJpgOAwgErSZoQVwo3IeahqISgUuZWtQ1tGA yZ1w== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 09/36] tools: unify type checking for data pfns in migration stream Date: Wed, 16 Jun 2021 14:51:02 +0200 Message-Id: <20210616125129.26563-10-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Introduce a helper which decides if a given pfn type has data for the migration stream. No change in behavior intended. Signed-off-by: Olaf Hering --- tools/libs/saverestore/common.h | 17 ++++++++++++++++ tools/libs/saverestore/restore.c | 34 +++++--------------------------- tools/libs/saverestore/save.c | 14 ++----------- 3 files changed, 24 insertions(+), 41 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index c9cc4206e5..08bbe902b9 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -500,6 +500,23 @@ static inline bool sr_is_known_page_type(xen_pfn_t type) return ret; } +static inline bool page_type_has_stream_data(uint32_t type) +{ + bool ret; + + switch (type) + { + case XEN_DOMCTL_PFINFO_XTAB: + case XEN_DOMCTL_PFINFO_XALLOC: + case XEN_DOMCTL_PFINFO_BROKEN: + ret = false; + break; + default: + ret = true; + break; + } + return ret; +} #endif /* * Local variables: diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 324b9050e2..70c92eaadc 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -152,9 +152,8 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, for ( i = 0; i < count; ++i ) { - if ( (!types || (types && - (types[i] != XEN_DOMCTL_PFINFO_XTAB && - types[i] != XEN_DOMCTL_PFINFO_BROKEN))) && + if ( (!types || + (types && page_type_has_stream_data(types[i]) == true)) && !pfn_is_populated(ctx, original_pfns[i]) ) { rc = pfn_set_populated(ctx, original_pfns[i]); @@ -233,25 +232,8 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, { ctx->restore.ops.set_page_type(ctx, pfns[i], types[i]); - switch ( types[i] ) - { - case XEN_DOMCTL_PFINFO_NOTAB: - - case XEN_DOMCTL_PFINFO_L1TAB: - case XEN_DOMCTL_PFINFO_L1TAB | XEN_DOMCTL_PFINFO_LPINTAB: - - case XEN_DOMCTL_PFINFO_L2TAB: - case XEN_DOMCTL_PFINFO_L2TAB | XEN_DOMCTL_PFINFO_LPINTAB: - - case XEN_DOMCTL_PFINFO_L3TAB: - case XEN_DOMCTL_PFINFO_L3TAB | XEN_DOMCTL_PFINFO_LPINTAB: - - case XEN_DOMCTL_PFINFO_L4TAB: - case XEN_DOMCTL_PFINFO_L4TAB | XEN_DOMCTL_PFINFO_LPINTAB: - + if ( page_type_has_stream_data(types[i]) == true ) mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]); - break; - } } /* Nothing to do? */ @@ -271,14 +253,8 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, for ( i = 0, j = 0; i < count; ++i ) { - switch ( types[i] ) - { - case XEN_DOMCTL_PFINFO_XTAB: - case XEN_DOMCTL_PFINFO_BROKEN: - case XEN_DOMCTL_PFINFO_XALLOC: - /* No page data to deal with. */ + if ( page_type_has_stream_data(types[i]) == false ) continue; - } if ( map_errs[j] ) { @@ -413,7 +389,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - if ( type < XEN_DOMCTL_PFINFO_BROKEN ) + if ( page_type_has_stream_data(type) == true ) /* NOTAB and all L1 through L4 tables (including pinned) should * have a page worth of data in the record. */ pages_of_data++; diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 6f820ea432..12598bd4e2 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -153,13 +153,8 @@ static int write_batch(struct xc_sr_context *ctx) goto err; } - switch ( types[i] ) - { - case XEN_DOMCTL_PFINFO_BROKEN: - case XEN_DOMCTL_PFINFO_XALLOC: - case XEN_DOMCTL_PFINFO_XTAB: + if ( page_type_has_stream_data(types[i]) == false ) continue; - } mfns[nr_pages++] = mfns[i]; } @@ -177,13 +172,8 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0, p = 0; i < nr_pfns; ++i ) { - switch ( types[i] ) - { - case XEN_DOMCTL_PFINFO_BROKEN: - case XEN_DOMCTL_PFINFO_XALLOC: - case XEN_DOMCTL_PFINFO_XTAB: + if ( page_type_has_stream_data(types[i]) == false ) continue; - } if ( errors[p] ) { From patchwork Wed Jun 16 12:51:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D759C48BE8 for ; Wed, 16 Jun 2021 12:52:16 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1337860FE9 for ; Wed, 16 Jun 2021 12:52:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1337860FE9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142951.263704 (Exim 4.92) (envelope-from ) id 1ltV1b-0003Zt-G9; Wed, 16 Jun 2021 12:52:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142951.263704; Wed, 16 Jun 2021 12:52:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1b-0003Xu-3l; Wed, 16 Jun 2021 12:52:07 +0000 Received: by outflank-mailman (input) for mailman id 142951; Wed, 16 Jun 2021 12:52:06 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1a-0006lZ-0S for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:06 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.84]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1862b455-a593-4217-a96b-2c8baf4754e0; Wed, 16 Jun 2021 12:51:44 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpctlz (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:38 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1862b455-a593-4217-a96b-2c8baf4754e0 ARC-Seal: i=1; a=rsa-sha256; t=1623847898; cv=none; d=strato.com; s=strato-dkim-0002; b=ag0e/5sNW6+i/kXyXoqt1VWKzYhGkvW0AiLGi5eTmr7Em384H6YwHHk9NueZhAldqx Jyl+j2c36db5IiIN4u1QAyhX1KukuIMi3YGVKKJW5/1zg62cXvVrKjWONLG7vzXV2bdQ 6eRBuxuXW/bLCVKD9MWId3jcIDrBrqAKxzBkE7GbTnMXSWcaapFr4nwqzUdfyPJy/f5c CL0ZuNrWYSbiz4BYzJLpf+fjqQreDGX6LWPbdoMluAgV8mCVF4QR+s3ibQPKwh3gYZVM 74PiscTSFgHaThxQv4wobdcIX2qiqxHiLZAO550Ry7vp0jByztRWwz1d4VvxK4tjHXsn aVoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847898; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=Zji+2Vj186gOQDCiA8MR4SQwrN68K3CPojQMJJRtxVk=; b=Sqxqycp1IAHnBrceTsX6/60pWhzCIFxBPvyU4pzVlVcPo/G5vnjTvQIHTGcohglMzn tuhpSWpV7Rds8Tj389fql61styvu47nTWkAD5xwFmCEHbjYHUUEhVt0RL36cA3WEck1j hJYPlLlD0s9zwo25eLhgblPwK9xT1BOOEj2nr8/HUsZtayN13MUmivKNhhs8E7n0uzCR 1yZ2p987jl/fJf6+t2Vjwx9ud92HyBPBMWdgwvhqQrghxirMy+LiS6b9B7oQ8u9QCpMb l79RzrF8f1GZz+qscsndw85QqQpYj7tIVWsLvrNYgl6jnqsUe2mLDjmx3gi8pQyDJDPi tJGg== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847898; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=Zji+2Vj186gOQDCiA8MR4SQwrN68K3CPojQMJJRtxVk=; b=naAFTL1rKhbPf85zZPnuCga7UufnYhJaiCPJ0EROBqh0RuNZfq3kV28HyN1Gq0Q4V3 pK70e3O5ZB1+IzWWrS1w0gtK+YlOhQz2NmlumxQ1gf+Vc781A7bK/2/yAkXQ7AqABPE+ vY284A8IDgmtp5vNpzL2/+Wf1QFSyxXNU/wVUmZdl7PoSVX+uJ9/VmJ1Sq83UGrU5Qc7 xtpHQ61QC65WVEEeDWyNp5VhzWSPqArVfwh6YrnE8VZ3Ftjl1mwFL0w5PmA8yUHwjfPw FRql2yVtjyYZg17Iuj0MZ8YlwqS4kHHR0NyL6xsAjG30HOpK5kbMYwVHdak7ZsTnqO8b l1uw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 10/36] tools: show migration transfer rate in send_dirty_pages Date: Wed, 16 Jun 2021 14:51:03 +0200 Message-Id: <20210616125129.26563-11-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Show how fast domU pages are transferred in each iteration. The relevant data is how fast the pfns travel, not so much how much protocol overhead exists. So the reported MiB/sec is just for pfns. Signed-off-by: Olaf Hering v02: - rearrange MiB_sec calculation (jgross) --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 46 +++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 08bbe902b9..d61569e1a6 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -250,6 +250,8 @@ struct xc_sr_context bool debug; unsigned long p2m_size; + size_t pages_sent; + size_t overhead_sent; struct precopy_stats stats; diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 12598bd4e2..f8fbe7a742 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -1,5 +1,6 @@ #include #include +#include #include "common.h" @@ -238,6 +239,8 @@ static int write_batch(struct xc_sr_context *ctx) iov[3].iov_len = nr_pfns * sizeof(*rec_pfns); iovcnt = 4; + ctx->save.pages_sent += nr_pages; + ctx->save.overhead_sent += sizeof(rec) + sizeof(hdr) + nr_pfns * sizeof(*rec_pfns); if ( nr_pages ) { @@ -357,6 +360,42 @@ static int suspend_domain(struct xc_sr_context *ctx) return 0; } +static void show_transfer_rate(struct xc_sr_context *ctx, struct timespec *start) +{ + xc_interface *xch = ctx->xch; + struct timespec end = {}, diff = {}; + size_t ms, MiB_sec; + + if (!ctx->save.pages_sent) + return; + + if ( clock_gettime(CLOCK_MONOTONIC, &end) ) + PERROR("clock_gettime"); + + if ( (end.tv_nsec - start->tv_nsec) < 0 ) + { + diff.tv_sec = end.tv_sec - start->tv_sec - 1; + diff.tv_nsec = end.tv_nsec - start->tv_nsec + (1000U*1000U*1000U); + } + else + { + diff.tv_sec = end.tv_sec - start->tv_sec; + diff.tv_nsec = end.tv_nsec - start->tv_nsec; + } + + ms = (diff.tv_nsec / (1000U*1000U)); + ms += (diff.tv_sec * 1000U); + if (!ms) + ms = 1; + + MiB_sec = (ctx->save.pages_sent * PAGE_SIZE * 1000U) / ms / (1024U*1024U); + + errno = 0; + IPRINTF("%s: %zu bytes + %zu pages in %ld.%09ld sec, %zu MiB/sec", __func__, + ctx->save.overhead_sent, ctx->save.pages_sent, + diff.tv_sec, diff.tv_nsec, MiB_sec); +} + /* * Send a subset of pages in the guests p2m, according to the dirty bitmap. * Used for each subsequent iteration of the live migration loop. @@ -370,9 +409,15 @@ static int send_dirty_pages(struct xc_sr_context *ctx, xen_pfn_t p; unsigned long written; int rc; + struct timespec start = {}; DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap, &ctx->save.dirty_bitmap_hbuf); + ctx->save.pages_sent = 0; + ctx->save.overhead_sent = 0; + if ( clock_gettime(CLOCK_MONOTONIC, &start) ) + PERROR("clock_gettime"); + for ( p = 0, written = 0; p < ctx->save.p2m_size; ++p ) { if ( !test_bit(p, dirty_bitmap) ) @@ -396,6 +441,7 @@ static int send_dirty_pages(struct xc_sr_context *ctx, if ( written > entries ) DPRINTF("Bitmap contained more entries than expected..."); + show_transfer_rate(ctx, &start); xc_report_progress_step(xch, entries, entries); return ctx->save.ops.check_vm_state(ctx); From patchwork Wed Jun 16 12:51:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A708C48BE6 for ; Wed, 16 Jun 2021 12:52:26 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CCC2461166 for ; Wed, 16 Jun 2021 12:52:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCC2461166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142958.263748 (Exim 4.92) (envelope-from ) id 1ltV1l-0005tQ-P6; Wed, 16 Jun 2021 12:52:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142958.263748; Wed, 16 Jun 2021 12:52:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1l-0005r5-Bm; Wed, 16 Jun 2021 12:52:17 +0000 Received: by outflank-mailman (input) for mailman id 142958; Wed, 16 Jun 2021 12:52:16 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1k-0006lZ-0y for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:16 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.168]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 60011629-2e09-443f-87df-86cf30994ee6; Wed, 16 Jun 2021 12:51:44 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpctm0 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:38 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 60011629-2e09-443f-87df-86cf30994ee6 ARC-Seal: i=1; a=rsa-sha256; t=1623847898; cv=none; d=strato.com; s=strato-dkim-0002; b=pFTQeRYsr2MTd3019r4AEMFlQuer/MMi9ihfJb0tTG6q5Zpz0ChAs4/r+ClZFhRFrg EUuvnfONNvYhxxTMsXP9LkSTK74641BL60nFoHuC1h/nLTBB9bKvP9VYBanQx5rA4ghf nErv0iQ5I5UkWbGszs/xx3kU4TTqcEOY3WpEnM2bISjvq2meu8NUmcvO6x/D1WaUp9+B ZVdf9DNVr6wXgz7eK4wsWJDflVC+ba5WAl5dtQu5J0BGy/59T0RR+rWmwYmSx1aHSeND /5HyH/pxK2uiuAygJKCHWhesmKSqiQjSP/+4gZLn07KpA3DJ4RrGViwvPwPTzcTApXG3 o+Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847898; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=LAtpQH5Nzzg8VriDH1a+nFDqqWS5bmIvzEWNNDGz4ss=; b=an9UrxYlf/qKTISLIAosLyLVy24jAp/pJxHtf1nf8iB61a6GhgjQG1Utr8j8rEFi5Q IwdCQgVMzhWU7G6syZLblQQYfuHCkiJyqMUuCS3WOLShL9H6AvDTEIIFdmjI8ALheS1P ZjSx2P50ntV8z2uSOMf66XhTnwQ8N8sZUFnQZMZsb8tQxF146sVAIVO5SxFyrF2qyQcT VRprS/Taaf9ZBQUhND3ZIhP08lmQH4aByc1u4F/mUKQvRoV4xNRstFHMRVCWdNVEw/P1 9JyMIw5WDSRCx/cAnx7Zv4hFkkzcOiM9MLYkm24/RRgqsRO+D8r98KHTdtumpl8wLIhG C+kw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847898; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=LAtpQH5Nzzg8VriDH1a+nFDqqWS5bmIvzEWNNDGz4ss=; b=DHAkqf8vlK/aDQZNR6eWescyrHhCQzm6nKxrdI0no+Bm59v6PO9S+Vx636nAgTyMs4 tSfDujVUOiErBYIj6gMS/RrvtSfAFjWlj9I7we72tBiAWa3xfvTPZVV2RgAUxyXtpnRf vzv35+ZQ+q9KyIb4zgntvyrz6wkKgr18sl4C6Hcbm3mhN31030EVwEfuLoRnM3uW1GSC toAc0i38DKXSFwDl481uSTrYcXrnLgQT/CvZ6m1RUXmPm3ZzO6itHBC2KZcwMxkGcdO9 AJHyYw8r0EtycmO7V01VfC2OHUeUCf9Iqs6yCl7oLanwuA08+9+kwMXhVa663sTfZVcP eaPw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 11/36] tools: prepare to allocate saverestore arrays once Date: Wed, 16 Jun 2021 14:51:04 +0200 Message-Id: <20210616125129.26563-12-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 The hotpath 'send_dirty_pages' is supposed to do just one thing: sending. The other end 'handle_page_data' is supposed to do just receiving. But instead both do other costly work like memory allocations and data moving. Do the allocations once, the array sizes are a compiletime constant. Avoid unneeded copying of data by receiving data directly into mapped guest memory. This patch is just prepartion, subsequent changes will populate the arrays. Once all changes are applied, migration of a busy HVM domU changes like that: Without this series, from sr650 to sr950 (xen-4.15.20201027T173911.16a20963b3 xen_testing): 2020-10-29 10:23:10.711+0000: xc: show_transfer_rate: 23663128 bytes + 2879563 pages in 55.324905335 sec, 203 MiB/sec: Internal error 2020-10-29 10:23:35.115+0000: xc: show_transfer_rate: 16829632 bytes + 2097552 pages in 24.401179720 sec, 335 MiB/sec: Internal error 2020-10-29 10:23:59.436+0000: xc: show_transfer_rate: 16829032 bytes + 2097478 pages in 24.319025928 sec, 336 MiB/sec: Internal error 2020-10-29 10:24:23.844+0000: xc: show_transfer_rate: 16829024 bytes + 2097477 pages in 24.406992500 sec, 335 MiB/sec: Internal error 2020-10-29 10:24:48.292+0000: xc: show_transfer_rate: 16828912 bytes + 2097463 pages in 24.446489027 sec, 335 MiB/sec: Internal error 2020-10-29 10:25:01.816+0000: xc: show_transfer_rate: 16836080 bytes + 2098356 pages in 13.447091818 sec, 609 MiB/sec: Internal error With this series, from sr650 to sr950 (xen-4.15.20201027T173911.16a20963b3 xen_unstable): 2020-10-28 21:26:05.074+0000: xc: show_transfer_rate: 23663128 bytes + 2879563 pages in 52.564054368 sec, 213 MiB/sec: Internal error 2020-10-28 21:26:23.527+0000: xc: show_transfer_rate: 16830040 bytes + 2097603 pages in 18.450592015 sec, 444 MiB/sec: Internal error 2020-10-28 21:26:41.926+0000: xc: show_transfer_rate: 16830944 bytes + 2097717 pages in 18.397862306 sec, 445 MiB/sec: Internal error 2020-10-28 21:27:00.339+0000: xc: show_transfer_rate: 16829176 bytes + 2097498 pages in 18.411973339 sec, 445 MiB/sec: Internal error 2020-10-28 21:27:18.643+0000: xc: show_transfer_rate: 16828592 bytes + 2097425 pages in 18.303326695 sec, 447 MiB/sec: Internal error 2020-10-28 21:27:26.289+0000: xc: show_transfer_rate: 16835952 bytes + 2098342 pages in 7.579846749 sec, 1081 MiB/sec: Internal error Note: the performance improvement depends on the used network cards, wirespeed and the host: - No improvement is expected with a 1G link. - Improvement can be seen as shown above on a 10G link. - Just a slight improvment can be seen on a 100G link. This change also populates sr_save_arrays with "batch_pfns", and sr_restore_arrays with "pfns" to make sure malloc is always called with a non-zero value. Signed-off-by: Olaf Hering v02: - rename xc_sr_save_arrays to sr_save_arrays - rename xc_sr_restore_arrays to sr_restore_arrays - merge handling of "batch_pfns" and "pfns" to make sure malloc is called with a non-zero size value (jgross) --- tools/libs/saverestore/common.h | 12 +++++++++++- tools/libs/saverestore/restore.c | 14 ++++++++++---- tools/libs/saverestore/save.c | 27 +++++++++++++-------------- 3 files changed, 34 insertions(+), 19 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index d61569e1a6..b3941af537 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -223,6 +223,15 @@ static inline int update_blob(struct xc_sr_blob *blob, return 0; } +struct sr_save_arrays { + xen_pfn_t batch_pfns[MAX_BATCH_SIZE]; +}; + +struct sr_restore_arrays { + /* handle_page_data */ + xen_pfn_t pfns[MAX_BATCH_SIZE]; +}; + struct xc_sr_context { xc_interface *xch; @@ -255,11 +264,11 @@ struct xc_sr_context struct precopy_stats stats; - xen_pfn_t *batch_pfns; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; xc_hypercall_buffer_t dirty_bitmap_hbuf; + struct sr_save_arrays *m; } save; struct /* Restore data. */ @@ -311,6 +320,7 @@ struct xc_sr_context /* Sender has invoked verify mode on the stream. */ bool verify; + struct sr_restore_arrays *m; } restore; }; diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 70c92eaadc..e18a03b381 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -315,7 +315,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) unsigned int i, pages_of_data = 0; int rc = -1; - xen_pfn_t *pfns = NULL, pfn; + xen_pfn_t *pfns = ctx->restore.m->pfns, pfn; uint32_t *types = NULL, type; /* @@ -363,9 +363,8 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - pfns = malloc(pages->count * sizeof(*pfns)); types = malloc(pages->count * sizeof(*types)); - if ( !pfns || !types ) + if ( !types ) { ERROR("Unable to allocate enough memory for %u pfns", pages->count); @@ -412,7 +411,6 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) &pages->pfn[pages->count]); err: free(types); - free(pfns); return rc; } @@ -739,6 +737,13 @@ static int setup(struct xc_sr_context *ctx) } ctx->restore.allocated_rec_num = DEFAULT_BUF_RECORDS; + ctx->restore.m = malloc(sizeof(*ctx->restore.m)); + if ( !ctx->restore.m ) { + ERROR("Unable to allocate memory for arrays"); + rc = -1; + goto err; + } + err: return rc; } @@ -757,6 +762,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages( xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size))); + free(ctx->restore.m); free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index f8fbe7a742..e29b6e1d66 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -77,7 +77,7 @@ static int write_checkpoint_record(struct xc_sr_context *ctx) /* * Writes a batch of memory as a PAGE_DATA record into the stream. The batch - * is constructed in ctx->save.batch_pfns. + * is constructed in ctx->save.m->batch_pfns. * * This function: * - gets the types for each pfn in the batch. @@ -128,12 +128,12 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; i < nr_pfns; ++i ) { types[i] = mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, - ctx->save.batch_pfns[i]); + ctx->save.m->batch_pfns[i]); /* Likely a ballooned page. */ if ( mfns[i] == INVALID_MFN ) { - set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages); + set_bit(ctx->save.m->batch_pfns[i], ctx->save.deferred_pages); ++ctx->save.nr_deferred_pages; } } @@ -179,7 +179,7 @@ static int write_batch(struct xc_sr_context *ctx) if ( errors[p] ) { ERROR("Mapping of pfn %#"PRIpfn" (mfn %#"PRIpfn") failed %d", - ctx->save.batch_pfns[i], mfns[p], errors[p]); + ctx->save.m->batch_pfns[i], mfns[p], errors[p]); goto err; } @@ -193,7 +193,7 @@ static int write_batch(struct xc_sr_context *ctx) { if ( rc == -1 && errno == EAGAIN ) { - set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages); + set_bit(ctx->save.m->batch_pfns[i], ctx->save.deferred_pages); ++ctx->save.nr_deferred_pages; types[i] = XEN_DOMCTL_PFINFO_XTAB; --nr_pages; @@ -224,7 +224,7 @@ static int write_batch(struct xc_sr_context *ctx) rec.length += nr_pages * PAGE_SIZE; for ( i = 0; i < nr_pfns; ++i ) - rec_pfns[i] = ((uint64_t)(types[i]) << 32) | ctx->save.batch_pfns[i]; + rec_pfns[i] = ((uint64_t)(types[i]) << 32) | ctx->save.m->batch_pfns[i]; iov[0].iov_base = &rec.type; iov[0].iov_len = sizeof(rec.type); @@ -296,9 +296,9 @@ static int flush_batch(struct xc_sr_context *ctx) if ( !rc ) { - VALGRIND_MAKE_MEM_UNDEFINED(ctx->save.batch_pfns, + VALGRIND_MAKE_MEM_UNDEFINED(ctx->save.m->batch_pfns, MAX_BATCH_SIZE * - sizeof(*ctx->save.batch_pfns)); + sizeof(*ctx->save.m->batch_pfns)); } return rc; @@ -315,7 +315,7 @@ static int add_to_batch(struct xc_sr_context *ctx, xen_pfn_t pfn) rc = flush_batch(ctx); if ( rc == 0 ) - ctx->save.batch_pfns[ctx->save.nr_batch_pfns++] = pfn; + ctx->save.m->batch_pfns[ctx->save.nr_batch_pfns++] = pfn; return rc; } @@ -849,13 +849,12 @@ static int setup(struct xc_sr_context *ctx) dirty_bitmap = xc_hypercall_buffer_alloc_pages( xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); - ctx->save.batch_pfns = malloc(MAX_BATCH_SIZE * - sizeof(*ctx->save.batch_pfns)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); + ctx->save.m = malloc(sizeof(*ctx->save.m)); - if ( !ctx->save.batch_pfns || !dirty_bitmap || !ctx->save.deferred_pages ) + if ( !ctx->save.m || !dirty_bitmap || !ctx->save.deferred_pages ) { - ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" + ERROR("Unable to allocate memory for dirty bitmaps and" " deferred pages"); rc = -1; errno = ENOMEM; @@ -884,7 +883,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); - free(ctx->save.batch_pfns); + free(ctx->save.m); } /* From patchwork Wed Jun 16 12:51:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86B72C48BE5 for ; Wed, 16 Jun 2021 12:52:19 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58F046023D for ; Wed, 16 Jun 2021 12:52:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58F046023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142954.263723 (Exim 4.92) (envelope-from ) id 1ltV1g-0004jG-CQ; Wed, 16 Jun 2021 12:52:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142954.263723; Wed, 16 Jun 2021 12:52:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1g-0004hl-7T; Wed, 16 Jun 2021 12:52:12 +0000 Received: by outflank-mailman (input) for mailman id 142954; Wed, 16 Jun 2021 12:52:11 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1f-0006lZ-0a for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:11 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.80]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 34cb16d1-5820-4808-abfe-5c8756b5df6d; Wed, 16 Jun 2021 12:51:44 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpctm2 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:38 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 34cb16d1-5820-4808-abfe-5c8756b5df6d ARC-Seal: i=1; a=rsa-sha256; t=1623847899; cv=none; d=strato.com; s=strato-dkim-0002; b=ks+H/9Lq48itLgjkf43MKm2w6YqTU5BIgnXYJ95Akrg3HFrIyzoYuFeJAD8DuBVYlW gk/LNSYNDIbE2Ms4+0hEnvUIUBDwg6dRVLN97pBqIFh2BSlA4O/3sSRLjl7xEq+YcVAO I7KZNsDnWzgH3didlsLqY+3Ze9NjmrXitsiVCBfk9I4/eBl9Zb1HRf8T4L+TiROYvdbk VBF5Ice1XM9iA/V+FQrGuBcrOAmAEOhNLwnxjm1rLcm5MdBK7eT1dIknANUQtNw1H0SN CrmljYo5r/LJbgNxRIBpFK2+ZihzvovaQnrrQQLVDZwUB1dTBLeO3NFhhfKA08otRFV3 I63g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847899; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=aebmW30p8aSy8ZbiG8FVV1BV3qJw8YQmzxiu0pIwFxU=; b=c/MaKwTpb25k2Ii/WPKi5bRa3dhl2xMCIc44Gpim9vNbWGOCL1TnR1es7KF0X/Grbb 6cumaaxEYSHr3bzlDNYvGvn53Zvjt65NE+KW3PHVEAq3tqdtEpiKs1UE8f8svC8ysRJB Q+9BToDThANxVxIU9Whc3+4qSegFTLSWwJvHMDgr2QufxEYRpW0LN+ohxkwUfCf0iyJC Dc4v1UQOnoyUzSLkaxDSybfjpC6xRN/QfIKLhNrJPs8kQYMC8O26AYVAEv3I0I+vqTTs MGIO0LXhyLdBkUWOO5xDlPONpeEeBkxfJZjDxe9T1iywjcgkykwD8ax/5jYi+nLY6Pvq YU8g== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847899; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=aebmW30p8aSy8ZbiG8FVV1BV3qJw8YQmzxiu0pIwFxU=; b=I0fK91BTFb+YLpjAxmw4SLrJF10jLgNtr2YzluulSr0rNp+lZvDkd/6A2v19c/5y3M iEHRslKvBIUfw94aXpnZfywrZay0EbLX04diucJ2ZaEmM+1WWNEREygAyyssI5Q4lHvi oDfXeE3reWqHLeN8tUJRam366HvfHempVythqpZWBQasaXmVfV8HfafGoR3aZqwXNVLT Cy35d9X3Ju3GDw44OzzAeanSobNQ5lgoJ7WuZhQsiw/remW0bS/Cuf+RpRJ/QsIoQLGy Ye2S3/J0nYPOP738PpZNlnvqcHBtcxlGAadMtUOAVEZ/R6ZgWbMDG9OX6hcy6ooiq6qf Jd5A== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 12/36] tools: save: move mfns array Date: Wed, 16 Jun 2021 14:51:05 +0200 Message-Id: <20210616125129.26563-13-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move mfns array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++----- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index b3941af537..6129710a3f 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -225,6 +225,8 @@ static inline int update_blob(struct xc_sr_blob *blob, struct sr_save_arrays { xen_pfn_t batch_pfns[MAX_BATCH_SIZE]; + /* write_batch: Mfns of the batch pfns. */ + xen_pfn_t mfns[MAX_BATCH_SIZE]; }; struct sr_restore_arrays { diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index e29b6e1d66..6b09784be8 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -88,7 +88,7 @@ static int write_checkpoint_record(struct xc_sr_context *ctx) static int write_batch(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = NULL, *types = NULL; + xen_pfn_t *mfns = ctx->save.m->mfns, *types = NULL; void *guest_mapping = NULL; void **guest_data = NULL; void **local_pages = NULL; @@ -105,8 +105,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Mfns of the batch pfns. */ - mfns = malloc(nr_pfns * sizeof(*mfns)); /* Types of the batch pfns. */ types = malloc(nr_pfns * sizeof(*types)); /* Errors from attempting to map the gfns. */ @@ -118,7 +116,7 @@ static int write_batch(struct xc_sr_context *ctx) /* iovec[] for writev(). */ iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !mfns || !types || !errors || !guest_data || !local_pages || !iov ) + if ( !types || !errors || !guest_data || !local_pages || !iov ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -277,7 +275,6 @@ static int write_batch(struct xc_sr_context *ctx) free(guest_data); free(errors); free(types); - free(mfns); return rc; } From patchwork Wed Jun 16 12:51:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A62F1C48BE6 for ; Wed, 16 Jun 2021 12:51:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 652076023D for ; Wed, 16 Jun 2021 12:51:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 652076023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142939.263627 (Exim 4.92) (envelope-from ) id 1ltV1I-0008DI-4M; Wed, 16 Jun 2021 12:51:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142939.263627; Wed, 16 Jun 2021 12:51:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1H-0008BS-T8; Wed, 16 Jun 2021 12:51:47 +0000 Received: by outflank-mailman (input) for mailman id 142939; Wed, 16 Jun 2021 12:51:46 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1G-00075D-58 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:46 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.84]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id f18633fe-d4db-43d5-bfab-8fd491179c21; Wed, 16 Jun 2021 12:51:45 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpdtm3 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:39 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f18633fe-d4db-43d5-bfab-8fd491179c21 ARC-Seal: i=1; a=rsa-sha256; t=1623847899; cv=none; d=strato.com; s=strato-dkim-0002; b=V9RtZbf8C1wCglPYTnqfkxIgmvIai9m1C0WragE6L8TdkzfjYumF7NNu+mG+JUHAk4 QXWqeR2xquVh1uVG/VkbLL2j4nGs57IJ1xwFck5cvPZbKkNO0QvysEfAyIgbwT3l9Y4Z dJZPv4YXeDdDdhJhRIHptkE/HTeegGIfOaSX+wzIjdesa1YWoFjY+3GheapIdaOJJZL0 d0Iq+se2UH0Br2RQBI73EsqjeK8WZJGg/sLxcpXGZBYND2PjovgbaIyL8nFcuIU3myOG J/Awj6LCZq5uCLxRdR/PEN+JTAa+MT6t3ZVSe1K7m30vtb4Dd6s2mK+qxTTEHLLFWtsX FDkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847899; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=6QhLsTHKnGLq8ej3+19WMGRz5FDjlHYnvJVyc5tVB4g=; b=h0z+eiF/8OC+LqIJRm8ILXtBw7ot3Z8f+f6U6rVifrn5AlWSjCOFCLemkoRi+37BfG 698ydSzAHVBHMHx5PWd4EPzAmexuuhyA+6ED2c/HXOIhgSjGOmGmxFWwvhP8eZ/DbLyH BKoVPTcOLU5gDwVUnLjWmnognqBp8a3e7Ds7JRd9laVwwlrTR4xpaMFO8YixXK9tEVX/ wXskzK7shBvG+54OXBGB3+85HBLpuJUvd4qTgpZRh0spnhDHbqDNLhnrTAaLZBRzeKRA xZzYc5GwZGdJW/c7Lri9BW/FWfzNBiPG/x7dy5ALMmS4yRC+GpqATtYl3FtaCn0LuiOE qsBQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847899; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=6QhLsTHKnGLq8ej3+19WMGRz5FDjlHYnvJVyc5tVB4g=; b=olx76e+wLmQTdJGVtYcWr7CZ+zPZc2Px6jO/FrjyPpGQpK3OA4lw2zvWycFRaokYNK 3yeJKYqQj6FvNf/EQRtio/ooBjYBBvvwDpVrdp4oUc5uxMaIwbPpUakOmLR90585EF2Y CNIWA29se7WZKhW1tkj63t87JZiGLqWBQm9H9w3/d7h9PqF9IPvNPruSI32xXxqlVpLS ZoymyoBb/ZqTWzGG+F6zsZPsL07md2yjbkxiCkiSTDKYWIUzHG1PwYsmb9ju3gmkeyk8 fDNJDQ0YVSqpicsIz70bRRhtDhju+GxQHXSGySp05PoULlFDcZfghtcSP5sCmFUenNPl oTcw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 13/36] tools: save: move types array Date: Wed, 16 Jun 2021 14:51:06 +0200 Message-Id: <20210616125129.26563-14-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move types array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++----- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 6129710a3f..1df684acb9 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -227,6 +227,8 @@ struct sr_save_arrays { xen_pfn_t batch_pfns[MAX_BATCH_SIZE]; /* write_batch: Mfns of the batch pfns. */ xen_pfn_t mfns[MAX_BATCH_SIZE]; + /* write_batch: Types of the batch pfns. */ + xen_pfn_t types[MAX_BATCH_SIZE]; }; struct sr_restore_arrays { diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 6b09784be8..0883c1fac0 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -88,7 +88,7 @@ static int write_checkpoint_record(struct xc_sr_context *ctx) static int write_batch(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = ctx->save.m->mfns, *types = NULL; + xen_pfn_t *mfns = ctx->save.m->mfns, *types = ctx->save.m->types; void *guest_mapping = NULL; void **guest_data = NULL; void **local_pages = NULL; @@ -105,8 +105,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Types of the batch pfns. */ - types = malloc(nr_pfns * sizeof(*types)); /* Errors from attempting to map the gfns. */ errors = malloc(nr_pfns * sizeof(*errors)); /* Pointers to page data to send. Mapped gfns or local allocations. */ @@ -116,7 +114,7 @@ static int write_batch(struct xc_sr_context *ctx) /* iovec[] for writev(). */ iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !types || !errors || !guest_data || !local_pages || !iov ) + if ( !errors || !guest_data || !local_pages || !iov ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -274,7 +272,6 @@ static int write_batch(struct xc_sr_context *ctx) free(local_pages); free(guest_data); free(errors); - free(types); return rc; } From patchwork Wed Jun 16 12:51:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FCD6C49EA2 for ; Wed, 16 Jun 2021 13:07:09 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C66126024A for ; Wed, 16 Jun 2021 13:07:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C66126024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142991.263809 (Exim 4.92) (envelope-from ) id 1ltVG1-0003fQ-BR; Wed, 16 Jun 2021 13:07:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142991.263809; Wed, 16 Jun 2021 13:07:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVG1-0003fF-7Z; Wed, 16 Jun 2021 13:07:01 +0000 Received: by outflank-mailman (input) for mailman id 142991; Wed, 16 Jun 2021 13:07:00 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1p-0006lZ-1E for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:21 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.81]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1f7850ed-248e-4334-9e79-bd485c10ed00; Wed, 16 Jun 2021 12:51:45 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpdtm5 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:39 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1f7850ed-248e-4334-9e79-bd485c10ed00 ARC-Seal: i=1; a=rsa-sha256; t=1623847900; cv=none; d=strato.com; s=strato-dkim-0002; b=jXnxuKXXi9YSOUvEDsqfQ/lU2hM8dPHWhZy85fULVb+ZPaiZ+lpuA9lAo88wEyIQ7X SqoIxwwxav2z+uy+BRdlvTVehdwmPUH7zW9Z0B47ZwVMyaFPIJiLpeEYq8oYVr9Y+J+R UKpIVVoTxy9VjrObf2j9mE7OJV1+mTk3+aj5O81tvzpZrH+GxtvQhXA06cTEXUBZrJfP uGqfrmV4onf5fDnNXXS+n2Gix+snjS+G5/MKkvSoiBWwto7+qeFd1/HPOnNyorIYh/2Q Mo7VfqwgETTUTBQh9FADRR+NH5D1dQwg+9gbsYTQq43ijBiinc3TQNW6mjsD7S/Q7bU5 Y2jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847900; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=SqZyfuP6OhwWmlaC4WpxM23i9gjc0SfkH6LfRu5qTXI=; b=ZhoNi07QQy07VhaFaQT6l33vJ2GGOtub+4UaWcd2qHkgHyqF6oVMrF/d5JS6P0vtYH V5DcIu5wxkZ+exkhZApDDJ+pW8QX9GULXdNtiwwBybAsggfmdn/ALuRtNAGdIR66HWNF tfswfypCnwNjmtF1h23SRCaDjyTAnhnS1Kb0Dca4vh/AlhE0+b9Pc3eNfscTugUctrl8 g72PFUTY61N7p2SDAh6AnwQWrkX944K3FioBAZpuJ3B7jxDiemHGzo20SIF/hIj66Gbb MkTx6cO+9awO2ezgU8VBrxQCIqXPq0NFNt7Z7Aq9cxcfIXkTjiG8iU0Bx9rt4lu1RkU5 FOKw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847900; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=SqZyfuP6OhwWmlaC4WpxM23i9gjc0SfkH6LfRu5qTXI=; b=WgvLDIHXTCDnHlevSZ58X7Z9r2oezcfXPemDhnz2t28Br521maZfAbtHKs5lVmssTI SZiiQv0THduwxlXrLR/07iQtBMgI4Cri3E3rbgrxCmgvndBi/esg4hyXQhdFo+v2XSve g3aHDuDOPUzuapJI0Tv81UKB5fBD9OJH1L807eZ+6wZtU5okXhff9U6c0IkM2YI6WHMj YE4Aa8GDPPpwh+VvxjxCGb7WTBXVhQw9u1yQ6BlmQsjbmLCnrrcrn0tXRUbErFbdfdO9 FDE6mTEeRhFLsE0uTzSS4HO8jv0EimfRw4t1V9MkgypbiDlf+wLgHJqcky8HmR7XHj6F xgLw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 14/36] tools: save: move errors array Date: Wed, 16 Jun 2021 14:51:07 +0200 Message-Id: <20210616125129.26563-15-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move errors array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++----- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 1df684acb9..558b5fbf06 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -229,6 +229,8 @@ struct sr_save_arrays { xen_pfn_t mfns[MAX_BATCH_SIZE]; /* write_batch: Types of the batch pfns. */ xen_pfn_t types[MAX_BATCH_SIZE]; + /* write_batch: Errors from attempting to map the gfns. */ + int errors[MAX_BATCH_SIZE]; }; struct sr_restore_arrays { diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 0883c1fac0..9ebbf00ce7 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -92,7 +92,7 @@ static int write_batch(struct xc_sr_context *ctx) void *guest_mapping = NULL; void **guest_data = NULL; void **local_pages = NULL; - int *errors = NULL, rc = -1; + int *errors = ctx->save.m->errors, rc = -1; unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; unsigned int nr_pfns = ctx->save.nr_batch_pfns; void *page, *orig_page; @@ -105,8 +105,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Errors from attempting to map the gfns. */ - errors = malloc(nr_pfns * sizeof(*errors)); /* Pointers to page data to send. Mapped gfns or local allocations. */ guest_data = calloc(nr_pfns, sizeof(*guest_data)); /* Pointers to locally allocated pages. Need freeing. */ @@ -114,7 +112,7 @@ static int write_batch(struct xc_sr_context *ctx) /* iovec[] for writev(). */ iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !errors || !guest_data || !local_pages || !iov ) + if ( !guest_data || !local_pages || !iov ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -271,7 +269,6 @@ static int write_batch(struct xc_sr_context *ctx) free(iov); free(local_pages); free(guest_data); - free(errors); return rc; } From patchwork Wed Jun 16 12:51:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 239FEC48BE5 for ; Wed, 16 Jun 2021 13:08:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DACD161166 for ; Wed, 16 Jun 2021 13:08:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DACD161166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143103.263963 (Exim 4.92) (envelope-from ) id 1ltVHT-00034l-MJ; Wed, 16 Jun 2021 13:08:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143103.263963; Wed, 16 Jun 2021 13:08:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHT-00034c-H8; Wed, 16 Jun 2021 13:08:31 +0000 Received: by outflank-mailman (input) for mailman id 143103; Wed, 16 Jun 2021 13:08:29 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1u-0006lZ-1Q for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:26 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.80]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1d87af7a-fcd5-438d-ab62-34ee038e5189; Wed, 16 Jun 2021 12:51:45 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpetm6 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:40 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1d87af7a-fcd5-438d-ab62-34ee038e5189 ARC-Seal: i=1; a=rsa-sha256; t=1623847900; cv=none; d=strato.com; s=strato-dkim-0002; b=qqndapM6fq0DVojuSeYOeqUo244Ivu4gAa4rSpcedg8YERJY7/OLOi6XWeyI845Uxs zBNVMAcdUAhMd6qaxhh+NdNE9aHaxpZ/yVg0zLAzoilDVYcabOyCyJ4SY++Z8/y9UQyT loToNlG4zIBEAs24WvEu4K32b/t/1n+2dtiYP/uRptnyEbDXBSBqPsUTXrPnXcoOTsh+ 7o8BM2E12cQJFOKnv00bClK3RshwRq+wqM/CwNwGh//cxCHz1lUvok+q1DnrKHRFLnbh EYA63dif5twORXHM4K5DTPS3ruswNOpHA3eTc8ef7wDr8f+z5wzkJfRtEuOvhdQwW+A/ 42yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847900; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=XGzxV8H0rtxBrYjtA99frZPaR/R3Z7Ebv988Uk2fYO0=; b=W22pArx+bx+xlzcrGw3A+cdqm96bzRnLx2oAT6Prjp36W5Ol69D4zNnOAYiBX1MMmy eFP93tipOYV9IjoT9DUsifnqz9j7wSRECx17AZWitUD3OMldAzlWRW/+B9ZH4DbO7P3e Na+g+TncdD9xx3yqFin3mtGg1eXeFAmx1Hwl3BOvgdqT7+mAvqNUjXJa7LL4eFoQNzg9 Qeq9iqt48s1dEck9LA5s+3NBikUwuO753UDcUCaTJK0XLMRPdsmO1G+HWmMiFo0H5z3K m0Ot/vrEdZ4eFrDFFzmIWm8iR0WHBpNUF2FHxmoaK4Mqg156xJsQ4+6q8JT4KC6XWNfa LiuA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847900; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=XGzxV8H0rtxBrYjtA99frZPaR/R3Z7Ebv988Uk2fYO0=; b=X8IO+QK3/HCFVRHjeCoMJqDkTfz+7zRmaDSFeq5Ll3aHQwW8Bw1dU7zILDykfzwXAP 3FK3+F9Ba+uUAXuUc9hcDbQLz3Y5DHR2a6oXhgkN161wz3XXqU6V97ilZt1GbMG2x461 qDPeQtbrMnXylsNaBavhbRE+tBlh5GzZA3a7L+QWL8LkrMiF/5wZZmJpK5t+jxfHvNZO u38EflqMHcHDHyJSJMQPlJinDy8aLCiRsPl7hZ4SXQpQEtTzqzpelTvLL91ZcISCF/ho anmnlZHdWGVSD6zTDMJQJkhAtWIqWv/XQuDmpIeiXBr2dpItR5DZNrHYvJV0Uxc+nf5B 3e3w== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 15/36] tools: save: move iov array Date: Wed, 16 Jun 2021 14:51:08 +0200 Message-Id: <20210616125129.26563-16-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move iov array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++----- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 558b5fbf06..ae87954364 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -231,6 +231,8 @@ struct sr_save_arrays { xen_pfn_t types[MAX_BATCH_SIZE]; /* write_batch: Errors from attempting to map the gfns. */ int errors[MAX_BATCH_SIZE]; + /* write_batch: iovec[] for writev(). */ + struct iovec iov[MAX_BATCH_SIZE + 4]; }; struct sr_restore_arrays { diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 9ebbf00ce7..1a5f3d29ea 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -97,7 +97,7 @@ static int write_batch(struct xc_sr_context *ctx) unsigned int nr_pfns = ctx->save.nr_batch_pfns; void *page, *orig_page; uint64_t *rec_pfns = NULL; - struct iovec *iov = NULL; int iovcnt = 0; + struct iovec *iov = ctx->save.m->iov; int iovcnt = 0; struct xc_sr_rec_page_data_header hdr = { 0 }; struct xc_sr_record rec = { .type = REC_TYPE_PAGE_DATA, @@ -109,10 +109,8 @@ static int write_batch(struct xc_sr_context *ctx) guest_data = calloc(nr_pfns, sizeof(*guest_data)); /* Pointers to locally allocated pages. Need freeing. */ local_pages = calloc(nr_pfns, sizeof(*local_pages)); - /* iovec[] for writev(). */ - iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !guest_data || !local_pages || !iov ) + if ( !guest_data || !local_pages ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -266,7 +264,6 @@ static int write_batch(struct xc_sr_context *ctx) xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped); for ( i = 0; local_pages && i < nr_pfns; ++i ) free(local_pages[i]); - free(iov); free(local_pages); free(guest_data); From patchwork Wed Jun 16 12:51:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1312EC48BE6 for ; Wed, 16 Jun 2021 13:08:30 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CEB2461356 for ; Wed, 16 Jun 2021 13:08:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CEB2461356 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143090.263930 (Exim 4.92) (envelope-from ) id 1ltVHL-0001mJ-BK; Wed, 16 Jun 2021 13:08:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143090.263930; Wed, 16 Jun 2021 13:08:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHL-0001m7-87; Wed, 16 Jun 2021 13:08:23 +0000 Received: by outflank-mailman (input) for mailman id 143090; Wed, 16 Jun 2021 13:08:22 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1z-0006lZ-1f for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:31 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.168]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 65af41dd-a6c9-42f1-bf3a-f6f8f41f3654; Wed, 16 Jun 2021 12:51:46 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpetm7 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:40 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 65af41dd-a6c9-42f1-bf3a-f6f8f41f3654 ARC-Seal: i=1; a=rsa-sha256; t=1623847901; cv=none; d=strato.com; s=strato-dkim-0002; b=Y7kTzV20Ryphv7nsbuvcPoIymod2LecV5RXqfm85m6o0JAABVhouZMriYYidwcN8EW 5OgB6B1FrI8ofK6nne/kDnK5IH+Aw2rD5N+mDtWs+5IIiX81F1XfRbjqYc06Z1PpN7C0 7nr5hAgrKpq2/ffaI3pu7uaOkLdDDFKmLQRkthc9fL+H6mT/PEnOGtnaLBiy5rFNvyqC OwQp4gdqSQpQLrdNbgMI9ba0Go1qx41r7lD7DamchIukCS2rJCo4RAAnf37ZCJ8dqM3h 079Se9Qjr3dB0kkYSAK2NlKjmYhXCSXKQ+NUCXQ1Dc7FiFmxFFA3tReN/O1YYgNQgxFs twew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847901; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=YoJCymJybJhr3GQavdsstKKb1/Nvxc490XeqP35kyEU=; b=IIuwyi298hRmgHzgfUA0RTSK18HEz9PFjl49wvz4GNtijls/hM1T1lViA5ZZ4FomUo dwzf6yJII2jbYgU0OH9x/aOuMv8vCzJBXg3V3I4ErpUNHuNU3Kg/37kSVusFe1UMCqPn i/Ucnfv1foekWiHBx7o7H/QkYtGRcabBfS37/dp/XxcmwON/xZG2qzpELliypDrye3MO SJtgtUYlGbJ3/0w5/1PSMZPQDGemGYO95P+gZTRBlWASmwk7hqqYDl6VZqHpkd0L2Y6Q O1DTTCFb1YCzR8dYzNdyW9p8M/TfTWux7d86Of8/EKkmXJGXA1SJntMrL8wgsctgMiLi +xTw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847901; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=YoJCymJybJhr3GQavdsstKKb1/Nvxc490XeqP35kyEU=; b=KrSUsPk9X2uvD0iK5qKfS6WX1d0Oa4/IkJJIXNSKYMb1lfM3Jcb5oz17Lg2wQ+DKBK b3IPkUnj0nDY6OEfxPY/ETIXMQEgLZ5VeEyEB7KXOyjsuJ4eXgc+vJ+00in9mq5YjvKj V32FvQ666+Mnn8lJDAeGJLZPPSsaTbIwyZWucHpViSoSVWX1AoajBP2uY92DIEGTmofg fMSG53R9gDG4pTJkeKkBUc3vnG80eXabYSLyFXwKJct+1LKle9bJ13NVP0cO6Rx2xf5x o0g4ubXe+t6ngvjuds5X8u6iCRC/uY/r1ylFnid1Uke7pSPD6d4FUYqM3t63ttJuRewD IrUg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 16/36] tools: save: move rec_pfns array Date: Wed, 16 Jun 2021 14:51:09 +0200 Message-Id: <20210616125129.26563-17-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move rec_pfns array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 11 +---------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index ae87954364..2950947f1d 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -233,6 +233,8 @@ struct sr_save_arrays { int errors[MAX_BATCH_SIZE]; /* write_batch: iovec[] for writev(). */ struct iovec iov[MAX_BATCH_SIZE + 4]; + /* write_batch */ + uint64_t rec_pfns[MAX_BATCH_SIZE]; }; struct sr_restore_arrays { diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 1a5f3d29ea..0f02988ff9 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -96,7 +96,7 @@ static int write_batch(struct xc_sr_context *ctx) unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; unsigned int nr_pfns = ctx->save.nr_batch_pfns; void *page, *orig_page; - uint64_t *rec_pfns = NULL; + uint64_t *rec_pfns = ctx->save.m->rec_pfns; struct iovec *iov = ctx->save.m->iov; int iovcnt = 0; struct xc_sr_rec_page_data_header hdr = { 0 }; struct xc_sr_record rec = { @@ -201,14 +201,6 @@ static int write_batch(struct xc_sr_context *ctx) } } - rec_pfns = malloc(nr_pfns * sizeof(*rec_pfns)); - if ( !rec_pfns ) - { - ERROR("Unable to allocate %zu bytes of memory for page data pfn list", - nr_pfns * sizeof(*rec_pfns)); - goto err; - } - hdr.count = nr_pfns; rec.length = sizeof(hdr); @@ -259,7 +251,6 @@ static int write_batch(struct xc_sr_context *ctx) rc = ctx->save.nr_batch_pfns = 0; err: - free(rec_pfns); if ( guest_mapping ) xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped); for ( i = 0; local_pages && i < nr_pfns; ++i ) From patchwork Wed Jun 16 12:51:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D97F1C48BE5 for ; Wed, 16 Jun 2021 12:52:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A638860FDB for ; Wed, 16 Jun 2021 12:52:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A638860FDB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142943.263655 (Exim 4.92) (envelope-from ) id 1ltV1Q-00016I-DG; Wed, 16 Jun 2021 12:51:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142943.263655; Wed, 16 Jun 2021 12:51:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1Q-000166-70; Wed, 16 Jun 2021 12:51:56 +0000 Received: by outflank-mailman (input) for mailman id 142943; Wed, 16 Jun 2021 12:51:55 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1P-00075D-7R for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:51:55 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.103]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 4e9aa579-d54f-4978-9ff7-95a20453cfd6; Wed, 16 Jun 2021 12:51:46 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpftm8 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:41 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4e9aa579-d54f-4978-9ff7-95a20453cfd6 ARC-Seal: i=1; a=rsa-sha256; t=1623847901; cv=none; d=strato.com; s=strato-dkim-0002; b=qCP7K0VPh2lHPt79phYQtfJ3EltvJWbyp0vzsiw+Fw+2qvCl8ffjTn5xcfLXt0xc1D QQa6oMySIG0UkhNDF/8UaRvKrzz/l3SOuJDrg1VnJRQc81kiazRCt7W4FdQdIFW062m7 gWsqAHy7u5fy4HE4VJR/H3XYBkztAOBUtgcih/DXoe5KazWq+CDGvEPUMYn8/FiR9JEX KmarUltqJQJXmxsu27htAqEMsmIDhWZJzo34EmYkZyxI4l22B0w4kWszBiPLbqENv6mZ SGRNDG/mxdQDVizfJWYLYAYVU4Luvd0qFnkS7sEh1eMEPTplU1OvwjGyiM3Ns5YdV545 xwWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847901; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=1Q2pJ96Q87Yp6O1o39i2QtRnldFx2Zb75m3igVfGdxc=; b=k4gvs2RDpyNND1f4GpB6oSepQQYDcTFhqb2RBKuk6bOCbCRTnyYsgOUhLEGNsQ6AwI SqQoXzr/oGmQUw6qps4VpbuuO4rCldWmBmGgLjucP0RA2p4GFSKqK9FiNRevnqzIQ7M1 iHRygBWy+Q/kBtb0FStU36HO4kFotnlDFvJZi7455wKEPECB4n1YDKt2289j71VLFVLv DeqZAN7JlUk85Ld/ctIF0AA1sXdKw5Yi0muUGIqwOZqg8KJfSGQfRMn/yVHWNUcFIfC4 lUw3yd9Gbpr2Nq+zJIx5jMlskDcod/4Qd3bpP6TNp+uqN3Dwtna3DSkIemOzR67iog4s 2JKg== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847901; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=1Q2pJ96Q87Yp6O1o39i2QtRnldFx2Zb75m3igVfGdxc=; b=QKkWE1Bzcmtf0jW7m9uSoGmmmOUeB8TXyU+CG6V7YfLCe5aRu7G2qU8FSbOr/RS+VL g5AcZrl6G7mBr13qHK1wNdPaB65vIkj18GJo8T5lA9f7/KKIMymaFDxMmpDEgFLJvGuO wUi1HTcaKS6pfaIvYISq4d+gi2yc+XyZW7zeAn8SPTYvbfCKljlhHh/fetlqc/C5tDCP 8V30LgxIVgKukdE9UygjrTG2nRQPuIDSqjzF8ak4mz0TRUFGYzJKFOx38EjzBBdjAHQR DwomF2T4j1F0ehToUyHRYinv0J8JAEfs/lDfihm2A1LDt/MZIT15ysSngVk0AVTIsTZM P/Qw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 17/36] tools: save: move guest_data array Date: Wed, 16 Jun 2021 14:51:10 +0200 Message-Id: <20210616125129.26563-18-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move guest_data array into preallocated space. Because this was allocated with calloc: Adjust the loop to clear unused entries as needed. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 11 ++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 2950947f1d..c4ab843c77 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -235,6 +235,8 @@ struct sr_save_arrays { struct iovec iov[MAX_BATCH_SIZE + 4]; /* write_batch */ uint64_t rec_pfns[MAX_BATCH_SIZE]; + /* write_batch: Pointers to page data to send. Mapped gfns or local allocations. */ + void *guest_data[MAX_BATCH_SIZE]; }; struct sr_restore_arrays { diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index 0f02988ff9..ea04cb1a74 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -90,7 +90,7 @@ static int write_batch(struct xc_sr_context *ctx) xc_interface *xch = ctx->xch; xen_pfn_t *mfns = ctx->save.m->mfns, *types = ctx->save.m->types; void *guest_mapping = NULL; - void **guest_data = NULL; + void **guest_data = ctx->save.m->guest_data; void **local_pages = NULL; int *errors = ctx->save.m->errors, rc = -1; unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; @@ -105,12 +105,10 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Pointers to page data to send. Mapped gfns or local allocations. */ - guest_data = calloc(nr_pfns, sizeof(*guest_data)); /* Pointers to locally allocated pages. Need freeing. */ local_pages = calloc(nr_pfns, sizeof(*local_pages)); - if ( !guest_data || !local_pages ) + if ( !local_pages ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -166,7 +164,10 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0, p = 0; i < nr_pfns; ++i ) { if ( page_type_has_stream_data(types[i]) == false ) + { + guest_data[i] = NULL; continue; + } if ( errors[p] ) { @@ -183,6 +184,7 @@ static int write_batch(struct xc_sr_context *ctx) if ( rc ) { + guest_data[i] = NULL; if ( rc == -1 && errno == EAGAIN ) { set_bit(ctx->save.m->batch_pfns[i], ctx->save.deferred_pages); @@ -256,7 +258,6 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; local_pages && i < nr_pfns; ++i ) free(local_pages[i]); free(local_pages); - free(guest_data); return rc; } From patchwork Wed Jun 16 12:51:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90EADC48BE5 for ; Wed, 16 Jun 2021 13:07:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4D9E361166 for ; Wed, 16 Jun 2021 13:07:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D9E361166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143023.263853 (Exim 4.92) (envelope-from ) id 1ltVGT-0005zA-S1; Wed, 16 Jun 2021 13:07:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143023.263853; Wed, 16 Jun 2021 13:07:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGT-0005yz-Np; Wed, 16 Jun 2021 13:07:29 +0000 Received: by outflank-mailman (input) for mailman id 143023; Wed, 16 Jun 2021 13:07:28 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV29-0006lZ-21 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:41 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.104]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id d81fa4ca-4f08-400e-94e6-11372cd20c32; Wed, 16 Jun 2021 12:51:48 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpftm9 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:41 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d81fa4ca-4f08-400e-94e6-11372cd20c32 ARC-Seal: i=1; a=rsa-sha256; t=1623847902; cv=none; d=strato.com; s=strato-dkim-0002; b=LHRmdgbB2P9FHXnH1+3pKYiiMR4798eMpFlLXZXIOO5bIvV64ZvF5OLbijo/uikpKU M5ZH5UCmwHXnOuhoCfRGyNBA339A6lRDZkIEB/tHPeNOAGL5mA4LpvBb08FfGLDWE70l iOiY5TpKwfIFCWStoRK0e0NyRDb2r2UZDdcohB8JSniP2vv23FM/mcdDTyII4jPfpZ1K UsFi9OAK5u4uVD1eUIKn8NDs+u0xijWVdRCOdHjwB2QXCrQcFnlkZshoAawddneWTrcQ KyA3EjhWYFBlXuwjyIeOW5LZAUQL2Fd8+FV2gVeg7p8thJgTJLRxJH83L5TphCIvugee vRmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847902; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+1yexgYfAf4Dkla6Khf3T41pK7uLyCVzzVYxTrxfJ48=; b=BWPrXQpnG32M8F4fD+kGPisOlk9owLjbQdlPOK+aBT1bbQ/v4vfHu894RVk+Rq4A3a oD9cI1H0XxHM9jV5y5h4rY3E+qqhkIV2BseburvAl5SFkro4B2B8qtLGi7Dlrg33Advv 1RDreYpgKOcjlng262z1yk/S2GhpKEE9rN89KgqnJuhP1qE2BbFsxAfk+yoPTeVLf2jD +lpmPhEysPhF8nnQKJ2wz6je/YgSdBzHE9PmTsIvGYiBpWWIX1+8Mrfk+spStG8+EpQB sjcSNjdIOcZjxY6mdBi3xZdBfssZ92KdFiJRGRtFZ5nWmoPQOoV6KZK1vN5yaxq8NFTQ f5sA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847902; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+1yexgYfAf4Dkla6Khf3T41pK7uLyCVzzVYxTrxfJ48=; b=B89XTjnLcCrMRxIrW6HPZ6cnntI76bCASJh8Pi6CX0Bw+499Rl0RNtlKESneF/fMPN TSMINec8+KVobD8SplsEJYJooC2hg0ruvEVgEyfvOynJJkZkT8ukvFsSj6MHDZ90DJzk n2GD8p173JYT9zj9z788E0rW0/hSn8385o9aAVCLoZEyEr54L9MHSWlFtsMNWHOY7JjJ 4oyfludkdNQrvHw087R91XMlnlO31IlEilZDSXls4LB5E0ehkcEo8s4tUl0qQYUXTY9/ cn4tTkCzkTDBQ2SzLfktBOG/23kUNUQCj3W6riqe5fkfGY64z1aZNtrAAUk6+jlfXE2p WsgA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 18/36] tools: save: move local_pages array Date: Wed, 16 Jun 2021 14:51:11 +0200 Message-Id: <20210616125129.26563-19-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move local_pages array into preallocated space. Adjust the code to use the src page as is in case of HVM. In case of PV the page may need to be normalised, use a private memory area for this purpose. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 22 ++++++++++--------- tools/libs/saverestore/save.c | 25 +++------------------ tools/libs/saverestore/save_x86_hvm.c | 5 +++-- tools/libs/saverestore/save_x86_pv.c | 31 ++++++++++++++++++--------- 4 files changed, 39 insertions(+), 44 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index c4ab843c77..96ae0904fc 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -45,16 +45,12 @@ struct xc_sr_save_ops * Optionally transform the contents of a page from being specific to the * sending environment, to being generic for the stream. * - * The page of data at the end of 'page' may be a read-only mapping of a - * running guest; it must not be modified. If no transformation is - * required, the callee should leave '*pages' untouched. + * The page of data '*src' may be a read-only mapping of a running guest; + * it must not be modified. If no transformation is required, the callee + * should leave '*src' untouched, and return it via '**ptr'. * - * If a transformation is required, the callee should allocate themselves - * a local page using malloc() and return it via '*page'. - * - * The caller shall free() '*page' in all cases. In the case that the - * callee encounters an error, it should *NOT* free() the memory it - * allocated for '*page'. + * If a transformation is required, the callee should provide the + * transformed page in a private buffer and return it via '**ptr'. * * It is valid to fail with EAGAIN if the transformation is not able to be * completed at this point. The page shall be retried later. @@ -62,7 +58,7 @@ struct xc_sr_save_ops * @returns 0 for success, -1 for failure, with errno appropriately set. */ int (*normalise_page)(struct xc_sr_context *ctx, xen_pfn_t type, - void **page); + void *src, unsigned int idx, void **ptr); /** * Set up local environment to save a domain. (Typically querying @@ -385,6 +381,12 @@ struct xc_sr_context union { + struct + { + /* Used by write_batch for modified pages. */ + void *normalised_pages; + } save; + struct { /* State machine for the order of received records. */ diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index ea04cb1a74..fa83648f9a 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -91,11 +91,10 @@ static int write_batch(struct xc_sr_context *ctx) xen_pfn_t *mfns = ctx->save.m->mfns, *types = ctx->save.m->types; void *guest_mapping = NULL; void **guest_data = ctx->save.m->guest_data; - void **local_pages = NULL; int *errors = ctx->save.m->errors, rc = -1; unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; unsigned int nr_pfns = ctx->save.nr_batch_pfns; - void *page, *orig_page; + void *src; uint64_t *rec_pfns = ctx->save.m->rec_pfns; struct iovec *iov = ctx->save.m->iov; int iovcnt = 0; struct xc_sr_rec_page_data_header hdr = { 0 }; @@ -105,16 +104,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Pointers to locally allocated pages. Need freeing. */ - local_pages = calloc(nr_pfns, sizeof(*local_pages)); - - if ( !local_pages ) - { - ERROR("Unable to allocate arrays for a batch of %u pages", - nr_pfns); - goto err; - } - for ( i = 0; i < nr_pfns; ++i ) { types[i] = mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, @@ -176,11 +165,8 @@ static int write_batch(struct xc_sr_context *ctx) goto err; } - orig_page = page = guest_mapping + (p * PAGE_SIZE); - rc = ctx->save.ops.normalise_page(ctx, types[i], &page); - - if ( orig_page != page ) - local_pages[i] = page; + src = guest_mapping + (p * PAGE_SIZE); + rc = ctx->save.ops.normalise_page(ctx, types[i], src, i, &guest_data[i]); if ( rc ) { @@ -195,8 +181,6 @@ static int write_batch(struct xc_sr_context *ctx) else goto err; } - else - guest_data[i] = page; rc = -1; ++p; @@ -255,9 +239,6 @@ static int write_batch(struct xc_sr_context *ctx) err: if ( guest_mapping ) xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped); - for ( i = 0; local_pages && i < nr_pfns; ++i ) - free(local_pages[i]); - free(local_pages); return rc; } diff --git a/tools/libs/saverestore/save_x86_hvm.c b/tools/libs/saverestore/save_x86_hvm.c index 91c2cb99ab..26f49ee267 100644 --- a/tools/libs/saverestore/save_x86_hvm.c +++ b/tools/libs/saverestore/save_x86_hvm.c @@ -129,9 +129,10 @@ static xen_pfn_t x86_hvm_pfn_to_gfn(const struct xc_sr_context *ctx, return pfn; } -static int x86_hvm_normalise_page(struct xc_sr_context *ctx, - xen_pfn_t type, void **page) +static int x86_hvm_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type, + void *src, unsigned int idx, void **ptr) { + *ptr = src; return 0; } diff --git a/tools/libs/saverestore/save_x86_pv.c b/tools/libs/saverestore/save_x86_pv.c index 92f77fad0f..159ff59480 100644 --- a/tools/libs/saverestore/save_x86_pv.c +++ b/tools/libs/saverestore/save_x86_pv.c @@ -999,29 +999,31 @@ static xen_pfn_t x86_pv_pfn_to_gfn(const struct xc_sr_context *ctx, * save_ops function. Performs pagetable normalisation on appropriate pages. */ static int x86_pv_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type, - void **page) + void *src, unsigned int idx, void **ptr) { xc_interface *xch = ctx->xch; - void *local_page; int rc; + void *dst; type &= XEN_DOMCTL_PFINFO_LTABTYPE_MASK; if ( type < XEN_DOMCTL_PFINFO_L1TAB || type > XEN_DOMCTL_PFINFO_L4TAB ) + { + *ptr = src; return 0; + } - local_page = malloc(PAGE_SIZE); - if ( !local_page ) + if ( idx >= MAX_BATCH_SIZE ) { - ERROR("Unable to allocate scratch page"); - rc = -1; - goto out; + ERROR("idx %u out of range", idx); + errno = ERANGE; + return -1; } - rc = normalise_pagetable(ctx, *page, local_page, type); - *page = local_page; + dst = ctx->x86.pv.save.normalised_pages + idx * PAGE_SIZE; + rc = normalise_pagetable(ctx, src, dst, type); + *ptr = dst; - out: return rc; } @@ -1031,8 +1033,16 @@ static int x86_pv_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type, */ static int x86_pv_setup(struct xc_sr_context *ctx) { + xc_interface *xch = ctx->xch; int rc; + ctx->x86.pv.save.normalised_pages = malloc(MAX_BATCH_SIZE * PAGE_SIZE); + if ( !ctx->x86.pv.save.normalised_pages ) + { + PERROR("Failed to allocate normalised_pages"); + return -1; + } + rc = x86_pv_domain_info(ctx); if ( rc ) return rc; @@ -1118,6 +1128,7 @@ static int x86_pv_check_vm_state(struct xc_sr_context *ctx) static int x86_pv_cleanup(struct xc_sr_context *ctx) { + free(ctx->x86.pv.save.normalised_pages); free(ctx->x86.pv.p2m_pfns); if ( ctx->x86.pv.p2m ) From patchwork Wed Jun 16 12:51:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0340C48BE5 for ; Wed, 16 Jun 2021 12:52:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 833C66023D for ; Wed, 16 Jun 2021 12:52:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 833C66023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142946.263677 (Exim 4.92) (envelope-from ) id 1ltV1W-0002Ni-Da; Wed, 16 Jun 2021 12:52:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142946.263677; Wed, 16 Jun 2021 12:52:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1W-0002Mr-7m; Wed, 16 Jun 2021 12:52:02 +0000 Received: by outflank-mailman (input) for mailman id 142946; Wed, 16 Jun 2021 12:52:00 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1U-00075D-81 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:00 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 21727138-bae6-4aea-a9cd-17eb42f3cbf2; Wed, 16 Jun 2021 12:51:48 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpgtmA (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:42 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 21727138-bae6-4aea-a9cd-17eb42f3cbf2 ARC-Seal: i=1; a=rsa-sha256; t=1623847902; cv=none; d=strato.com; s=strato-dkim-0002; b=jSZ2Z9L7tplcMdULHYPib6jr8P3O27PjFRi3DaMtcyaivK632dkPPGhq0kdpV3n4Ke Ll9sPl0CkDZrFVZWnOKuyHz/G5EsWAHZpZGc24pW4iQ448b2y0LUGLcrGVoYy3QwK/ld MkhJ8fFHOemkCAoMSLkdr6LSLIIa4D23B8mjii49uUAIefMR0sq0Nd6lxkFk8nI8bKkJ bRnZmGamDGiBnG7I1R2i3mj0vvMQJTTaqYYpjUxs7jC2Ku7iHEvtcE3XrJQe8vDa8Z95 xiYUTFiCcNR57gDc/1xRquzvncRhqw+S43VFkrZKTidJknSPW7nU7wXcJGfqYYJnVPGK C8Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847902; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+zNYdRT311M2lTYh9UyGXJAkdHDCHae4YkiiVz2nJTk=; b=YMS9icgO95j0umh569k+Un5aJGs6dFWpsZCBYy3KmGCN/UW8obv+2y6cx/La7dD8EA j9xVhhfqAYPfxa4A5EWAKt58F72chToPGPOxqRIueqKyb/mWALz3tmA84i1BIU5FBFXy lKdInWBa7Ra8ZqLT0OWQX03fnqNgwb5WkaGqZY4zwsFL58xCIoaKXUaGzpe2RgurAafj KNx6APWzvA+8bnh9bfjyZk1WVJ7Q476iDHVoT1paw6qrzSCWzf/J9WPQ5kVuGcTmLC7a BvM91e0QYYJj46/XgXFwIVIRu8DcL5ymi2leA/7xBJ2msDIn7WT3V/RW2w9AepFWPpRi 4YMA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847902; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+zNYdRT311M2lTYh9UyGXJAkdHDCHae4YkiiVz2nJTk=; b=I565a8Paxp67rMuXqxcCzHUGgYOTV7Qh04/11K40ZE9TdvE5EZGK1CDOtZD1ZhJJJ0 IRtgquJKznOCTY7oJjMdfm++zrGTTFCwgqk3iww144KlZPJqkf4py3rD52QldjONCo84 Dyjjg04JTp9XmC2NITklkz8oIPs7FaUNv2OhCrJWzRZrXFcJ/jJ4IgVLAXhYFUBVCW6U UP3VvT1yX+BpZgmIgtqI1CN7vwASvmEzzyx0xEcuZYlVLoAZprvhhKnzRnAxQHgH5u1P YWBwFn4Ggkrw1ASXJi1sYOthJUYd+/2mTIPyAAZiXj/W4bdaMwgJgU7GDxYuAreEbyIl n8RQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 19/36] tools: restore: move types array Date: Wed, 16 Jun 2021 14:51:12 +0200 Message-Id: <20210616125129.26563-20-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move types array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 12 +----------- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 96ae0904fc..fe44302eac 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -238,6 +238,7 @@ struct sr_save_arrays { struct sr_restore_arrays { /* handle_page_data */ xen_pfn_t pfns[MAX_BATCH_SIZE]; + uint32_t types[MAX_BATCH_SIZE]; }; struct xc_sr_context diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index e18a03b381..d460a2b2b5 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -316,7 +316,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) int rc = -1; xen_pfn_t *pfns = ctx->restore.m->pfns, pfn; - uint32_t *types = NULL, type; + uint32_t *types = ctx->restore.m->types, type; /* * v2 compatibility only exists for x86 streams. This is a bit of a @@ -363,14 +363,6 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - types = malloc(pages->count * sizeof(*types)); - if ( !types ) - { - ERROR("Unable to allocate enough memory for %u pfns", - pages->count); - goto err; - } - for ( i = 0; i < pages->count; ++i ) { pfn = pages->pfn[i] & PAGE_DATA_PFN_MASK; @@ -410,8 +402,6 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) rc = process_page_data(ctx, pages->count, pfns, types, &pages->pfn[pages->count]); err: - free(types); - return rc; } From patchwork Wed Jun 16 12:51:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43103C48BE5 for ; Wed, 16 Jun 2021 12:52:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0932E6023D for ; Wed, 16 Jun 2021 12:52:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0932E6023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142949.263699 (Exim 4.92) (envelope-from ) id 1ltV1a-0003T6-QC; Wed, 16 Jun 2021 12:52:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142949.263699; Wed, 16 Jun 2021 12:52:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1a-0003Sk-JJ; Wed, 16 Jun 2021 12:52:06 +0000 Received: by outflank-mailman (input) for mailman id 142949; Wed, 16 Jun 2021 12:52:05 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1Z-00075D-82 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:05 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.102]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 440b148c-5a47-47e7-8268-4fec67cd4e3c; Wed, 16 Jun 2021 12:51:48 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpgtmC (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:42 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 440b148c-5a47-47e7-8268-4fec67cd4e3c ARC-Seal: i=1; a=rsa-sha256; t=1623847903; cv=none; d=strato.com; s=strato-dkim-0002; b=opDqhO4a1Vq4ddVcj+k8tqaWyd7iEl35jDu6mAo2Qt4tu3vXYHrlskuG3lPRYff0ov y443MOvX2wXF0kBESl9kaSX8E/AF7MORqBMUI2k0vhRoSUnqrzB09kJ3F+y7YG7LhZ3o g0frzi7M7n7ZEhoqv3SccUbu1XdjqdvuS9taPrW3zCAJ/c0BXG0XJ7dU8fQq9S5/ZaEl 4LJBD/B7XnT8w8kRlGdf3ozpGAu8cLr9nzTxqSFqvO9FKRS5rIQVfdi6WSx7rnqQWVrc RAGbuHfYazO1mmNG8JeWeb+JxeM2AqHnxL/uMXS1CeTRPwaV39GFpFVR6Z4WNNtudfQ6 HCnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847903; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=Nst2DxeDxoQRaBpmHCGwCg/XwLbUQH8n2F7XGaSzYSY=; b=THuKMd0uiY8blfLD/vzBCt8CiQ7ynxcB7hOV8Oxqi8BWA7HBC3wUB7Kl7Z1sdXrzM0 DWYoQvThi6ZX0oD/Yhjwi8w9huvWWniZP73zhjlvzG0vcErO9ANjbQyEDm86iZ6LDmtM 7ZaI4Aq+d4RowYfo5U8oNd0kOd2i+felN+y7zJWEsXFAYaF+pTUGvRmG8dKdwq3R83Fk Xfs8mErtxq5Jx48BYee3841s/5wrSkzDl2U8Gc8qZ7NxmMdhL5JLXPDxSaqSMahq0Axt kbEBe1sYMKXH6P6gI2PasDzOlx9o3Zv1qMmXHSwHkHR8grFF2cFQworQ6GR8QvAGDvvu mweQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847903; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=Nst2DxeDxoQRaBpmHCGwCg/XwLbUQH8n2F7XGaSzYSY=; b=tX+SlobqcYRUw8DVNaMA6Zdbbft4TT8fAIUkIPGgXORmYpajCecrKedu0kgLaOCfds nKCIO6rQo2jOj2NwcV0xTUF08Rh6GKXcGFyYQdP6FgmgNWIsxSulJH2gwjtprAPSCqee ia8rlaZbuMoqapjc6ahFsXkBrKDqgjSnb4snFm0x80hoMN9ptzziYn2KREpVfmoYY/JH HDUa33N6zy4VAaGjArJuY1/BccqjgYIa888jm88kcXm/LaUtpgcVmCeC+l+8BNyGyELp McljlGSAN26T8QNOvwXKPYQDtf/T7URtuTQx0gByye+vSJ9/aSoyeCLlAN0A51UJk5XX +kmA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 20/36] tools: restore: move mfns array Date: Wed, 16 Jun 2021 14:51:13 +0200 Message-Id: <20210616125129.26563-21-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move mfns array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/restore.c | 5 ++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index fe44302eac..54352f5427 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -239,6 +239,8 @@ struct sr_restore_arrays { /* handle_page_data */ xen_pfn_t pfns[MAX_BATCH_SIZE]; uint32_t types[MAX_BATCH_SIZE]; + /* process_page_data */ + xen_pfn_t mfns[MAX_BATCH_SIZE]; }; struct xc_sr_context diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index d460a2b2b5..1a7cfbcd47 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -205,7 +205,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, xen_pfn_t *pfns, uint32_t *types, void *page_data) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = malloc(count * sizeof(*mfns)); + xen_pfn_t *mfns = ctx->restore.m->mfns; int *map_errs = malloc(count * sizeof(*map_errs)); int rc; void *mapping = NULL, *guest_page = NULL; @@ -213,7 +213,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, j, /* j indexes the subset of pfns we decide to map. */ nr_pages = 0; - if ( !mfns || !map_errs ) + if ( !map_errs ) { rc = -1; ERROR("Failed to allocate %zu bytes to process page data", @@ -299,7 +299,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, xenforeignmemory_unmap(xch->fmem, mapping, nr_pages); free(map_errs); - free(mfns); return rc; } From patchwork Wed Jun 16 12:51:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86D6DC48BE8 for ; Wed, 16 Jun 2021 12:52:19 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F9F560FE9 for ; Wed, 16 Jun 2021 12:52:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F9F560FE9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142952.263720 (Exim 4.92) (envelope-from ) id 1ltV1f-0004dN-V1; Wed, 16 Jun 2021 12:52:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142952.263720; Wed, 16 Jun 2021 12:52:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1f-0004d3-QC; Wed, 16 Jun 2021 12:52:11 +0000 Received: by outflank-mailman (input) for mailman id 142952; Wed, 16 Jun 2021 12:52:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1e-00075D-88 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:10 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.102]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id c3133b3a-05ca-476e-a4d8-03107f301f32; Wed, 16 Jun 2021 12:51:48 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCphtmF (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:43 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c3133b3a-05ca-476e-a4d8-03107f301f32 ARC-Seal: i=1; a=rsa-sha256; t=1623847903; cv=none; d=strato.com; s=strato-dkim-0002; b=ohS/KAgS1qnECx4K8vUIAIRYQKCo0QYE9XXnqRt6loKSUMq5Af3ZMwucSSyU4hidku mQlWc/m0IPMaMuS60Kz+vXTUjg6az/PvkNew1U0vwz05NTofNK5EHKCCtpKPQVQk+vGY SaBcqIg+XXdVL3rCg5mNFhX2VIvPZNswDCoKUNXzxqEsI1FWVpJAnS3ketn9gxYc58WR 2jfMZUmrNSHS8rsMksgvsD+9fAKYBfV85U/QcJqk9teC6XQX19yHms13x5I5FfnV+3Yc uqu54dHC2WlJOaiulF6eXl65vw7Owj1joqeII3kqa5b+YL2m74W75yuyBeJARZvbtXYL MRbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847903; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=jECKf3sL2DTnpGqAznPVbUN7tLXZQbgjgWqGsEtA4iA=; b=GjR90vz9RCP0DIoOSuG1FugVmpRs1Gl2p0tn70f7eDGTIuc12jdOGUDAYVhAWPTWct x1eFWvAXGaZH33DWYP90UNrhm01LI7cjr7kjVNfZkB2rjo1q2Lj4y9A45gQweWNmJIKJ jWOqvrk4kokNbUhawJMK/k0CiTJAhRz2M7y3YjQZgd23lrtwvhnxxn6qDtPsFtHMUN8y bMYXJUWpd+EGBbQ+t6gC+WW3cq43iNCL3qywx1rILWsv6m6bGILhjHvvlzDq5Ec47Z8Y nVTqpFz4jcUOEL7sE7qJiYvwLFeumJWHAzkCdVNnCw3YWeqFij97Br6d+T5GFy+85ANr HFkA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847903; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=jECKf3sL2DTnpGqAznPVbUN7tLXZQbgjgWqGsEtA4iA=; b=F54RWxRU+WpmRT4Yk7Xy3BUNbHQQpm2uV1w+WnAQVsgZK2YZvBeQak043u5WfqVdTc XowzbQlDxibs0LBDp2jMjKmjI6Cg87lDGHxm1MuG95q0hhy8DLxeOsBYd08fTFsJu4MR Jh6/TOl9mx/CE7/Jkco/bY7/sy9zkNCp2chtOXjF7iB365k5LXIoR9DLNDDmIMsXaPNa xuQl6UlWTKuPKTHWZh0tzy7eLhLVsFmK2TS7jeAgFDZ9TR4+z01XryyEznJ2d321OS+S sV3xTazczZiYlWjY8lxyck5VSx56nCtowrNo+TRn11K7QjFoNnbG+v9/unXH3dIWSr0H ylNg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 21/36] tools: restore: move map_errs array Date: Wed, 16 Jun 2021 14:51:14 +0200 Message-Id: <20210616125129.26563-22-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move map_errs array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 12 +----------- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 54352f5427..34042c2b90 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -241,6 +241,7 @@ struct sr_restore_arrays { uint32_t types[MAX_BATCH_SIZE]; /* process_page_data */ xen_pfn_t mfns[MAX_BATCH_SIZE]; + int map_errs[MAX_BATCH_SIZE]; }; struct xc_sr_context diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 1a7cfbcd47..6eb955423c 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -206,21 +206,13 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, { xc_interface *xch = ctx->xch; xen_pfn_t *mfns = ctx->restore.m->mfns; - int *map_errs = malloc(count * sizeof(*map_errs)); + int *map_errs = ctx->restore.m->map_errs; int rc; void *mapping = NULL, *guest_page = NULL; unsigned int i, /* i indexes the pfns from the record. */ j, /* j indexes the subset of pfns we decide to map. */ nr_pages = 0; - if ( !map_errs ) - { - rc = -1; - ERROR("Failed to allocate %zu bytes to process page data", - count * (sizeof(*mfns) + sizeof(*map_errs))); - goto err; - } - rc = populate_pfns(ctx, count, pfns, types); if ( rc ) { @@ -298,8 +290,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, if ( mapping ) xenforeignmemory_unmap(xch->fmem, mapping, nr_pages); - free(map_errs); - return rc; } From patchwork Wed Jun 16 12:51:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44A88C48BE5 for ; Wed, 16 Jun 2021 12:52:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16CB46023D for ; Wed, 16 Jun 2021 12:52:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16CB46023D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142957.263743 (Exim 4.92) (envelope-from ) id 1ltV1k-0005kx-Sw; Wed, 16 Jun 2021 12:52:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142957.263743; Wed, 16 Jun 2021 12:52:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1k-0005kQ-OG; Wed, 16 Jun 2021 12:52:16 +0000 Received: by outflank-mailman (input) for mailman id 142957; Wed, 16 Jun 2021 12:52:15 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1j-00075D-8G for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:15 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 369b540b-50dd-4914-92b8-e2f355492428; Wed, 16 Jun 2021 12:51:49 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCphtmH (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:43 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 369b540b-50dd-4914-92b8-e2f355492428 ARC-Seal: i=1; a=rsa-sha256; t=1623847903; cv=none; d=strato.com; s=strato-dkim-0002; b=N9SFQiycwUqMVroio62f217jPoVGGdQJPf+D4dC5ewl1zRi6GWZOq5/LTLwxJbNuF1 9BfpbqPX6LQPB4vOcV5UPcNv4ZDrHaYcxfQhfG9hmP7QCeTMYYGIZVZQvtap80KRLHqX Jj4kVAMFAjF+X9iA05oIcmhnXZK1FqFzaoe5/j4wLIjeaNR/glhGnGrD0VQZESX+opfM GDAq4bwCedM48Jo8Na7rwEAciY81rkFfrUNAE78psUmav+Ruz1cWMZRS064C9wXrGOlL 5eBChx/a46rzypa89VzkNt9S1hdPVtwm+OZn3v/VUCEEN2nGbI1CjOUSd/yg7mRIlG58 /AUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847903; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2Gn1MT0Bi1Jqgmn6EP44HuJomJlyOc6/uRvNu601HFs=; b=TTyH7AS0EYebV694Lh3ak/7t7MpNELS967/d4eOPxKcUOU8PGLBzIivhH89OA/R3jC vz9f3/ke1SiOubqneeASXXLI/T9s4cH05Djx7giPr3m6x2keMKisCzwPZGEWxVQAR0Fd NQHBuTBuHu3HQhcb12hNY/WVcB3lfKHGzlR72v63f6hq6pQ0gGCCYC5XjIf255A6kAk2 ytL2PW6LB09Uc2uWJPWcZ3FsbVFDvaezeOcN9MrP7w9Ix/yrFKelf8PzqLYx8m0GhFdE ivA3x4o+bXEFXA5KeJyyaDr+zpq3J24MmwHK3kvWHp+bIIVIy08xDg2K/iWox1hWyWMS y2eQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847903; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2Gn1MT0Bi1Jqgmn6EP44HuJomJlyOc6/uRvNu601HFs=; b=HsPEmMwcLZBl7ViPln50aU/C9dmPPsiFdX7Gu+BIHYlvtmUwrzRTVn5aUVIOotoJUa xY+D0/ZdQEMjoBTz+nMkMBKTcWgY2JwXMrbUImBIcgmN/sIo+/Tr66fychQ1WT/tqkjH G0tQzl3iZ849Ok4KZ9vi50aRFysBJ/6fmswu2BbCg/PeftVFLIkBUAytM4VgSAd07o00 VKy9vYaxcL0Ay9/a33XngyJL0YFAHrVEprB392l6WRypprNQdUVdqCABDbDjIgHWJfbC GNj5Bjdn/QD0x5V32rlAjVN3DDMLQEsFv4NotZMHtCvQDsbCIQM+0gUO/N5zMqTgrUTL eoTQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 22/36] tools: restore: move mfns array in populate_pfns Date: Wed, 16 Jun 2021 14:51:15 +0200 Message-Id: <20210616125129.26563-23-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move populate_pfns mfns array into preallocated space. Use some prefix to avoid conflict with an array used in handle_page_data. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/restore.c | 5 ++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 34042c2b90..3cfb23861f 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -242,6 +242,8 @@ struct sr_restore_arrays { /* process_page_data */ xen_pfn_t mfns[MAX_BATCH_SIZE]; int map_errs[MAX_BATCH_SIZE]; + /* populate_pfns */ + xen_pfn_t pp_mfns[MAX_BATCH_SIZE]; }; struct xc_sr_context diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 6eb955423c..0c29478ccb 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -138,12 +138,12 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, const xen_pfn_t *original_pfns, const uint32_t *types) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = malloc(count * sizeof(*mfns)), + xen_pfn_t *mfns = ctx->restore.m->pp_mfns, *pfns = malloc(count * sizeof(*pfns)); unsigned int i, nr_pfns = 0; int rc = -1; - if ( !mfns || !pfns ) + if ( !pfns ) { ERROR("Failed to allocate %zu bytes for populating the physmap", 2 * count * sizeof(*mfns)); @@ -191,7 +191,6 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, err: free(pfns); - free(mfns); return rc; } From patchwork Wed Jun 16 12:51:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FD6CC48BE5 for ; Wed, 16 Jun 2021 12:52:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5A28661166 for ; Wed, 16 Jun 2021 12:52:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A28661166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142961.263764 (Exim 4.92) (envelope-from ) id 1ltV1p-0006pg-UW; Wed, 16 Jun 2021 12:52:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142961.263764; Wed, 16 Jun 2021 12:52:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1p-0006p1-Oa; Wed, 16 Jun 2021 12:52:21 +0000 Received: by outflank-mailman (input) for mailman id 142961; Wed, 16 Jun 2021 12:52:20 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1o-00075D-8h for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:20 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.103]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id a55ae0b3-93b5-4fad-bf8e-247239aca60e; Wed, 16 Jun 2021 12:51:50 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpitmI (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:44 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a55ae0b3-93b5-4fad-bf8e-247239aca60e ARC-Seal: i=1; a=rsa-sha256; t=1623847904; cv=none; d=strato.com; s=strato-dkim-0002; b=LTxVl9Oyg2idIwCSzLv4XMhSMg50poqnIqEXBORyknOrl2MMSI8beM3alR1r8dAiNq qQfKyOwHUDeig5sb4IxCh8JoJYfeG+8N8odQsng/yKsGvQqpo2ptViKYqtC/Xzo5G5nJ 5rPBsCgXIsNDynCFvKqv9xIX/YbMCyuBZAXUeAXhABJPZ52VZOqtKnJLYd26Xzagk5rp GrkRAX8yk5p+ZOQr4/fqYvZmh3EJrfoEquOCGKFDnhps7jrTi+SHpFY/Mr9UDejr2Phi FMgXxfgo4wGBjdWZchBjwZ5wM1pbXDU/YG5JUvfGHbswerXw5v8Y7MkfGRcuvGFgxfNM lVvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847904; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=pnLF16VUG8E4Jp5fWe4D439r7jQrrghaZKOdiE+JFI0=; b=t1ElC85hwloXfK5N6nrxainkrTPKhRZHdvPq6dL7JDtPKhDFDmDwz3Jdbpi33IVN+B ZHt7hvT3IcBtT/zhhe7WP2SuBbMhqEkYjShsDFuy8JzXQymbXn5pP14l+PMIOu5Mk8IE L2s4nWXoAO+1H3V/NwRbw3zUSOozMY2Xso2gyVL32e2MECs7C4UG0nRY+j16xh9RYsqD FBhieSIpl/MZciH0KcH3f+lGdZos3fpLTm8jCl4ZGR0ou2t84iSShLXhEjuPhMB+o/Zp 2luAbl6KpFxhKQszOy/tZXz4JDtMJXI1HFNEfGOaxS2O1/bf1FPK5g1SEZuI8nkmIAmo 9OxA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847904; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=pnLF16VUG8E4Jp5fWe4D439r7jQrrghaZKOdiE+JFI0=; b=gCyQuUTTqDyZMhlmYIN9tY4A0P4yNzjeUiLbPrNZBns0jWdbQR/4EIAVBkDiKWm6v8 +z1bs71Tjm8IgH7dP9k/QuoUljLADS5U+d+ddz8KAqhkIAr2EM0kts06fWJPCuMigebL dSNE1S2V61S9fBxiFSRzcDSVsoE6QLr4/J6pZl25d+Wk9KJFGEUz9seNYkVFqSQ+iPpz 4pZ1Ou9KKgvPt2LOwSndsnDCDNLwGLIKhag4y/LYr2TPWBRjBnpOQlI4u+BGiZa+/oXx RWEQH3fboZ+i1pDcrhdc3FyG1dW83flQhyrPENqhh4agYg3xlcQDOQFeIYmKHzDeqpxJ W5Sw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 23/36] tools: restore: move pfns array in populate_pfns Date: Wed, 16 Jun 2021 14:51:16 +0200 Message-Id: <20210616125129.26563-24-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Remove allocation from hotpath, move populate_pfns' pfns array into preallocated space. Use some prefix to avoid conflict with an array used in handle_page_data. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 11 +---------- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 3cfb23861f..379887e149 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -244,6 +244,7 @@ struct sr_restore_arrays { int map_errs[MAX_BATCH_SIZE]; /* populate_pfns */ xen_pfn_t pp_mfns[MAX_BATCH_SIZE]; + xen_pfn_t pp_pfns[MAX_BATCH_SIZE]; }; struct xc_sr_context diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 0c29478ccb..f2234eac55 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -139,17 +139,10 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, { xc_interface *xch = ctx->xch; xen_pfn_t *mfns = ctx->restore.m->pp_mfns, - *pfns = malloc(count * sizeof(*pfns)); + *pfns = ctx->restore.m->pp_pfns; unsigned int i, nr_pfns = 0; int rc = -1; - if ( !pfns ) - { - ERROR("Failed to allocate %zu bytes for populating the physmap", - 2 * count * sizeof(*mfns)); - goto err; - } - for ( i = 0; i < count; ++i ) { if ( (!types || @@ -190,8 +183,6 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, rc = 0; err: - free(pfns); - return rc; } From patchwork Wed Jun 16 12:51:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F257C48BE5 for ; Wed, 16 Jun 2021 13:08:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0CDA56024A for ; Wed, 16 Jun 2021 13:08:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CDA56024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143096.263941 (Exim 4.92) (envelope-from ) id 1ltVHQ-0002Ls-NO; Wed, 16 Jun 2021 13:08:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143096.263941; Wed, 16 Jun 2021 13:08:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHQ-0002LZ-Jc; Wed, 16 Jun 2021 13:08:28 +0000 Received: by outflank-mailman (input) for mailman id 143096; Wed, 16 Jun 2021 13:08:26 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2T-0006lZ-2q for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:53:01 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.102]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 046d4327-6b6c-4c35-882b-9d3651f4494a; Wed, 16 Jun 2021 12:51:51 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpitmJ (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:44 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 046d4327-6b6c-4c35-882b-9d3651f4494a ARC-Seal: i=1; a=rsa-sha256; t=1623847904; cv=none; d=strato.com; s=strato-dkim-0002; b=FBnmkbb9LZm/91iFiW2q8M9FboL1OUtmmvM7VGSguLzkiWperTNUhojbegLeR7lS3v Maix+C5WnemiZeKLPz27BPVfF0D/muHRgGPFQNrtVjL0AyykIlx+eAivdmDorzen2pTc istP1d73k1oVB9g+kwnc28I7ZYXqLOFpYiqk8go1coDS66n7LkkKPqmX4LENDlOdYPwD GFmtXQENh9IjOTm78CjORfHoLtVqpZjj6Qawe06TF//smjR8QMxAZ6OKgA6McR8TJLSW 0pJJCeGi4EJWbXopCuRj+dIEYuSk0JnAFZBYc6CRthdLkjF0rbkhQ8o9fU4Au7WEYPhp XPww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847904; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2FAZoK8lmXx9wrot1ueh4V2OeQ0OeMkG5/FHU6cLwr4=; b=PzMQRnTORgigITH0zv8GFr0IbWYUNZbO01DzrX3il4hf47u5Y74Y+NyvWapCIbYmxd yHqz6xGkoFa4VlwiAsZyfBiFNudsijQ3IBiUT/0kK4gKRq85EXcSIUBWl95Bb3D3ZFTE FaeXX1pWamcEBq1Jc6WbnIo8YFVcyZHk9Bn6jkVVDBqnBVaNQjTMPDN4qZ4AxtHv8VZb i19bjwGYC5c1jd2xDJgF2xC04DLz7dEOIyfv/7jSSsdvNKuV590IIK9ErWASM2fpl82z iA0lKPAlMMq9gAUr8Pj4q9VvBThM4YgXso7nkjOEcPYD8BcU1AStArLyH3cIvx6pYLo8 w74A== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847904; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2FAZoK8lmXx9wrot1ueh4V2OeQ0OeMkG5/FHU6cLwr4=; b=T4A/lccgXKvjZLLHc/E21dQfvrcQjrAgebXECbhRInmR+Xb+JTwEteNz2nAXu1x9Q1 1d0QDBugkXLKUmSjbbbtVt0Cwf/bX5bxQvKmkMyCgPvBa2uio/lmIo8fKcHFngjWBjIM M3oWLC1nXs6hfe9motmiDsd8JQO94yMCiZ4rUvwzrctp7LV4TfvsclsMTii6mUlSKmvB xAJ6/WLMXnHNS95AMSBz9/swblmq2iHhED8b/9A5jq5wBvgMqIVeTbyfrP7qxtZEw/Bj jnI09XkzIbFCemZk7aA7DZQVbQc4KJ/r8+cFv9plRj41ZlAucfDi0RTEpfeGv2ekQBwe ATUA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210616 24/36] tools: restore: split record processing Date: Wed, 16 Jun 2021 14:51:17 +0200 Message-Id: <20210616125129.26563-25-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 handle_page_data must be able to read directly into mapped guest memory. This will avoid unneccesary memcpy calls for data which can be consumed verbatim. Rearrange the code to allow decisions based on the incoming record. This change is preparation for future changes in handle_page_data, no change in behavior is intended. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.c | 33 ++++++++++++--------- tools/libs/saverestore/common.h | 4 ++- tools/libs/saverestore/restore.c | 49 ++++++++++++++++++++++---------- tools/libs/saverestore/save.c | 7 ++++- 4 files changed, 63 insertions(+), 30 deletions(-) diff --git a/tools/libs/saverestore/common.c b/tools/libs/saverestore/common.c index 77128bc747..7da7fa4e2c 100644 --- a/tools/libs/saverestore/common.c +++ b/tools/libs/saverestore/common.c @@ -91,26 +91,33 @@ int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec, return -1; } -int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) +int read_record_header(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr) { xc_interface *xch = ctx->xch; - struct xc_sr_rhdr rhdr; - size_t datasz; - if ( read_exact(fd, &rhdr, sizeof(rhdr)) ) + if ( read_exact(fd, rhdr, sizeof(*rhdr)) ) { PERROR("Failed to read Record Header from stream"); return -1; } - if ( rhdr.length > REC_LENGTH_MAX ) + if ( rhdr->length > REC_LENGTH_MAX ) { - ERROR("Record (0x%08x, %s) length %#x exceeds max (%#x)", rhdr.type, - rec_type_to_str(rhdr.type), rhdr.length, REC_LENGTH_MAX); + ERROR("Record (0x%08x, %s) length %#x exceeds max (%#x)", rhdr->type, + rec_type_to_str(rhdr->type), rhdr->length, REC_LENGTH_MAX); return -1; } - datasz = ROUNDUP(rhdr.length, REC_ALIGN_ORDER); + return 0; +} + +int read_record_data(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr, + struct xc_sr_record *rec) +{ + xc_interface *xch = ctx->xch; + size_t datasz; + + datasz = ROUNDUP(rhdr->length, REC_ALIGN_ORDER); if ( datasz ) { @@ -119,7 +126,7 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) if ( !rec->data ) { ERROR("Unable to allocate %zu bytes for record data (0x%08x, %s)", - datasz, rhdr.type, rec_type_to_str(rhdr.type)); + datasz, rhdr->type, rec_type_to_str(rhdr->type)); return -1; } @@ -128,18 +135,18 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) free(rec->data); rec->data = NULL; PERROR("Failed to read %zu bytes of data for record (0x%08x, %s)", - datasz, rhdr.type, rec_type_to_str(rhdr.type)); + datasz, rhdr->type, rec_type_to_str(rhdr->type)); return -1; } } else rec->data = NULL; - rec->type = rhdr.type; - rec->length = rhdr.length; + rec->type = rhdr->type; + rec->length = rhdr->length; return 0; -}; +} static void __attribute__((unused)) build_assertions(void) { diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 379887e149..2ced6f100d 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -487,7 +487,9 @@ static inline int write_record(struct xc_sr_context *ctx, * * On failure, the contents of the record structure are undefined. */ -int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec); +int read_record_header(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr); +int read_record_data(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr, + struct xc_sr_record *rec); /* * This would ideally be private in restore.c, but is needed by diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index f2234eac55..2409c8d603 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -471,7 +471,7 @@ static int send_checkpoint_dirty_pfn_list(struct xc_sr_context *ctx) return rc; } -static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec); +static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_record *rec); static int handle_checkpoint(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; @@ -510,7 +510,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx) for ( i = 0; i < ctx->restore.buffered_rec_num; i++ ) { - rc = process_record(ctx, &ctx->restore.buffered_records[i]); + rc = process_buffered_record(ctx, &ctx->restore.buffered_records[i]); if ( rc ) goto err; } @@ -571,10 +571,11 @@ static int handle_checkpoint(struct xc_sr_context *ctx) return rc; } -static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) +static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_rhdr *rhdr) { xc_interface *xch = ctx->xch; unsigned int new_alloc_num; + struct xc_sr_record rec; struct xc_sr_record *p; if ( ctx->restore.buffered_rec_num >= ctx->restore.allocated_rec_num ) @@ -592,8 +593,13 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) ctx->restore.allocated_rec_num = new_alloc_num; } + if ( read_record_data(ctx, ctx->fd, rhdr, &rec) ) + { + return -1; + } + memcpy(&ctx->restore.buffered_records[ctx->restore.buffered_rec_num++], - rec, sizeof(*rec)); + &rec, sizeof(rec)); return 0; } @@ -624,7 +630,7 @@ int handle_static_data_end(struct xc_sr_context *ctx) return rc; } -static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) +static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; int rc = 0; @@ -662,6 +668,19 @@ static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) return rc; } +static int process_incoming_record_header(struct xc_sr_context *ctx, struct xc_sr_rhdr *rhdr) +{ + struct xc_sr_record rec; + int rc; + + rc = read_record_data(ctx, ctx->fd, rhdr, &rec); + if ( rc ) + return rc; + + return process_buffered_record(ctx, &rec); +} + + static int setup(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; @@ -745,7 +764,7 @@ static void cleanup(struct xc_sr_context *ctx) static int restore(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - struct xc_sr_record rec; + struct xc_sr_rhdr rhdr; int rc, saved_rc = 0, saved_errno = 0; IPRINTF("Restoring domain"); @@ -756,7 +775,7 @@ static int restore(struct xc_sr_context *ctx) do { - rc = read_record(ctx, ctx->fd, &rec); + rc = read_record_header(ctx, ctx->fd, &rhdr); if ( rc ) { if ( ctx->restore.buffer_all_records ) @@ -766,25 +785,25 @@ static int restore(struct xc_sr_context *ctx) } if ( ctx->restore.buffer_all_records && - rec.type != REC_TYPE_END && - rec.type != REC_TYPE_CHECKPOINT ) + rhdr.type != REC_TYPE_END && + rhdr.type != REC_TYPE_CHECKPOINT ) { - rc = buffer_record(ctx, &rec); + rc = buffer_record(ctx, &rhdr); if ( rc ) goto err; } else { - rc = process_record(ctx, &rec); + rc = process_incoming_record_header(ctx, &rhdr); if ( rc == RECORD_NOT_PROCESSED ) { - if ( rec.type & REC_TYPE_OPTIONAL ) + if ( rhdr.type & REC_TYPE_OPTIONAL ) DPRINTF("Ignoring optional record %#x (%s)", - rec.type, rec_type_to_str(rec.type)); + rhdr.type, rec_type_to_str(rhdr.type)); else { ERROR("Mandatory record %#x (%s) not handled", - rec.type, rec_type_to_str(rec.type)); + rhdr.type, rec_type_to_str(rhdr.type)); rc = -1; goto err; } @@ -795,7 +814,7 @@ static int restore(struct xc_sr_context *ctx) goto err; } - } while ( rec.type != REC_TYPE_END ); + } while ( rhdr.type != REC_TYPE_END ); remus_failover: if ( ctx->stream_type == XC_STREAM_COLO ) diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index fa83648f9a..e486bce96f 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -589,6 +589,7 @@ static int send_memory_live(struct xc_sr_context *ctx) static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; + struct xc_sr_rhdr rhdr; struct xc_sr_record rec; uint64_t *pfns = NULL; uint64_t pfn; @@ -597,7 +598,11 @@ static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx) DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap, &ctx->save.dirty_bitmap_hbuf); - rc = read_record(ctx, ctx->save.recv_fd, &rec); + rc = read_record_header(ctx, ctx->save.recv_fd, &rhdr); + if ( rc ) + goto err; + + rc = read_record_data(ctx, ctx->save.recv_fd, &rhdr, &rec); if ( rc ) goto err; From patchwork Wed Jun 16 12:51:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29B3EC48BE5 for ; Wed, 16 Jun 2021 13:08:47 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D09FA6024A for ; Wed, 16 Jun 2021 13:08:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D09FA6024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143114.263983 (Exim 4.92) (envelope-from ) id 1ltVHc-00049v-G9; Wed, 16 Jun 2021 13:08:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143114.263983; Wed, 16 Jun 2021 13:08:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHc-00048f-Bd; Wed, 16 Jun 2021 13:08:40 +0000 Received: by outflank-mailman (input) for mailman id 143114; Wed, 16 Jun 2021 13:08:38 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1y-00075D-8o for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:30 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id a6f8c5a5-da29-4a23-8ff5-586f6ef15fae; Wed, 16 Jun 2021 12:51:52 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpjtmK (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:45 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a6f8c5a5-da29-4a23-8ff5-586f6ef15fae ARC-Seal: i=1; a=rsa-sha256; t=1623847905; cv=none; d=strato.com; s=strato-dkim-0002; b=JvlnntSKYv9tF9aoHR4YcbHinXeXv7WSGPTF4iEOj3hEgaZ3rfYK71Z+c1u/P9oqXP ygpI2DAhmADpkQjxV7nIy4Nl6T9C6yg4tDpufy7R5mP+18i3dNBfoIpuVB5DcSmrGKJd Q1JJV1P+F5f53g89lP/qCWI0UeBb/Wi7Okc/ZR9/rFf9vxReF2goTbAb9BapTDPzZbkG uI/8EdXhHw3bnVr9ZSZqbUprSS2+/e1ItAUXhABtzOpSMkWBcn6HkhYKSid34+pM0XVF T4mv1PoEej+efCX4C99jewUswe36nTVWhCbZfO03vAnHEeDEWmCiRxgd5R7C3OIXCfem kGwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847905; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=hD2Ug5waXHQrkHDlWstdnaBGpKn51RUN9z6BofzxctY=; b=NN9G0ZUM6yI8Gy7r02RYTVweKFCGoz1PK1+mndvDLbuszw92lldN86tr8zyviSCxpy ngHevMpvqH004PlVQc2aYk48z8IwTkQsXPBcUTrgtumFCJRtuMF0ExtzkTFjFzoED+tR Ka94LAsMlrlNkzSBCFoF8pvotQFKcMTUOrcpbTOIsuDd5h/5qbm2KIGc7AiRk2QaDvoh ztEwWWiwbFbBxtcVqjGrElq/E7Ggw+XRwlADRbYuyym7DCJ37omw22SFFRQQ5gXVw4fn mgO+8kg8UlwU1Uox7+JdfnmpyW5HDBNnthlUY7y0jZN90DSAM/DceN6aaOGeekRjoJUC Oe6A== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847905; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=hD2Ug5waXHQrkHDlWstdnaBGpKn51RUN9z6BofzxctY=; b=oIIgINgdysp5e2Ft3jMn8MxLIF1+rBoIenWMzeKeJJkk9UmTE8HEZ72sknf5SHBHOa vD7+xM6CmoLdF2lXes3AeMGOouESDswyoY5uQdO8wRaVmRj0ScOQ6JzIvdd59fg1WNa/ 0UJ+UtXbRmyUV+RI+GSyg/E4WZy0r07M7otyy8bzWD2pBr3tVnA75udXorURoU9JU7Uu OoTP0QhMVOyAvJAvG1AXIydMnd++Dy82qX1neuEdXE3WjSaRfCBY5B5Ms566TbprjUrD rJybBnxLw7VQ+MtYmK41GoEeU85HXlKmqxunqTzsT0GgkkTb/yTEzi3xsWmiVhRy51Fu Ou5A== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 25/36] tools: restore: split handle_page_data Date: Wed, 16 Jun 2021 14:51:18 +0200 Message-Id: <20210616125129.26563-26-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 handle_page_data must be able to read directly into mapped guest memory. This will avoid unneccesary memcpy calls for data that can be consumed verbatim. Split the various steps of record processing: - move processing to handle_buffered_page_data - adjust xenforeignmemory_map to set errno in case of failure - adjust verify mode to set errno in case of failure This change is preparation for future changes in handle_page_data, no change in behavior is intended. Signed-off-by: Olaf Hering --- tools/libs/saverestore/common.h | 9 + tools/libs/saverestore/restore.c | 343 ++++++++++++++++++++----------- 2 files changed, 231 insertions(+), 121 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 2ced6f100d..d479f1a918 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -242,9 +242,14 @@ struct sr_restore_arrays { /* process_page_data */ xen_pfn_t mfns[MAX_BATCH_SIZE]; int map_errs[MAX_BATCH_SIZE]; + void *guest_data[MAX_BATCH_SIZE]; + /* populate_pfns */ xen_pfn_t pp_mfns[MAX_BATCH_SIZE]; xen_pfn_t pp_pfns[MAX_BATCH_SIZE]; + + /* Must be the last member */ + struct xc_sr_rec_page_data_header pages; }; struct xc_sr_context @@ -335,7 +340,11 @@ struct xc_sr_context /* Sender has invoked verify mode on the stream. */ bool verify; + void *verify_buf; + struct sr_restore_arrays *m; + void *guest_mapping; + uint32_t nr_mapped_pages; } restore; }; diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 2409c8d603..877fd19a9b 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -186,123 +186,18 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, return rc; } -/* - * Given a list of pfns, their types, and a block of page data from the - * stream, populate and record their types, map the relevant subset and copy - * the data into the guest. - */ -static int process_page_data(struct xc_sr_context *ctx, unsigned int count, - xen_pfn_t *pfns, uint32_t *types, void *page_data) +static int handle_static_data_end_v2(struct xc_sr_context *ctx) { - xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = ctx->restore.m->mfns; - int *map_errs = ctx->restore.m->map_errs; - int rc; - void *mapping = NULL, *guest_page = NULL; - unsigned int i, /* i indexes the pfns from the record. */ - j, /* j indexes the subset of pfns we decide to map. */ - nr_pages = 0; - - rc = populate_pfns(ctx, count, pfns, types); - if ( rc ) - { - ERROR("Failed to populate pfns for batch of %u pages", count); - goto err; - } - - for ( i = 0; i < count; ++i ) - { - ctx->restore.ops.set_page_type(ctx, pfns[i], types[i]); - - if ( page_type_has_stream_data(types[i]) == true ) - mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]); - } - - /* Nothing to do? */ - if ( nr_pages == 0 ) - goto done; - - mapping = guest_page = xenforeignmemory_map( - xch->fmem, ctx->domid, PROT_READ | PROT_WRITE, - nr_pages, mfns, map_errs); - if ( !mapping ) - { - rc = -1; - PERROR("Unable to map %u mfns for %u pages of data", - nr_pages, count); - goto err; - } - - for ( i = 0, j = 0; i < count; ++i ) - { - if ( page_type_has_stream_data(types[i]) == false ) - continue; - - if ( map_errs[j] ) - { - rc = -1; - ERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed with %d", - pfns[i], mfns[j], types[i], map_errs[j]); - goto err; - } - - /* Undo page normalisation done by the saver. */ - rc = ctx->restore.ops.localise_page(ctx, types[i], page_data); - if ( rc ) - { - ERROR("Failed to localise pfn %#"PRIpfn" (type %#"PRIx32")", - pfns[i], types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); - goto err; - } - - if ( ctx->restore.verify ) - { - /* Verify mode - compare incoming data to what we already have. */ - if ( memcmp(guest_page, page_data, PAGE_SIZE) ) - ERROR("verify pfn %#"PRIpfn" failed (type %#"PRIx32")", - pfns[i], types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); - } - else - { - /* Regular mode - copy incoming data into place. */ - memcpy(guest_page, page_data, PAGE_SIZE); - } - - ++j; - guest_page += PAGE_SIZE; - page_data += PAGE_SIZE; - } - - done: - rc = 0; - - err: - if ( mapping ) - xenforeignmemory_unmap(xch->fmem, mapping, nr_pages); - - return rc; -} + int rc = 0; -/* - * Validate a PAGE_DATA record from the stream, and pass the results to - * process_page_data() to actually perform the legwork. - */ -static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) -{ +#if defined(__i386__) || defined(__x86_64__) xc_interface *xch = ctx->xch; - struct xc_sr_rec_page_data_header *pages = rec->data; - unsigned int i, pages_of_data = 0; - int rc = -1; - - xen_pfn_t *pfns = ctx->restore.m->pfns, pfn; - uint32_t *types = ctx->restore.m->types, type; - /* * v2 compatibility only exists for x86 streams. This is a bit of a * bodge, but it is less bad than duplicating handle_page_data() between * different architectures. */ -#if defined(__i386__) || defined(__x86_64__) + /* v2 compat. Infer the position of STATIC_DATA_END. */ if ( ctx->restore.format_version < 3 && !ctx->restore.seen_static_data_end ) { @@ -320,12 +215,26 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) ERROR("No STATIC_DATA_END seen"); goto err; } + + rc = 0; +err: #endif - if ( rec->length < sizeof(*pages) ) + return rc; +} + +static bool verify_rec_page_hdr(struct xc_sr_context *ctx, uint32_t rec_length, + struct xc_sr_rec_page_data_header *pages) +{ + xc_interface *xch = ctx->xch; + bool ret = false; + + errno = EINVAL; + + if ( rec_length < sizeof(*pages) ) { ERROR("PAGE_DATA record truncated: length %u, min %zu", - rec->length, sizeof(*pages)); + rec_length, sizeof(*pages)); goto err; } @@ -335,13 +244,35 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - if ( rec->length < sizeof(*pages) + (pages->count * sizeof(uint64_t)) ) + if ( pages->count > MAX_BATCH_SIZE ) + { + ERROR("pfn count %u in PAGE_DATA record too large", pages->count); + errno = E2BIG; + goto err; + } + + if ( rec_length < sizeof(*pages) + (pages->count * sizeof(uint64_t)) ) { ERROR("PAGE_DATA record (length %u) too short to contain %u" - " pfns worth of information", rec->length, pages->count); + " pfns worth of information", rec_length, pages->count); goto err; } + ret = true; + +err: + return ret; +} + +static bool verify_rec_page_pfns(struct xc_sr_context *ctx, uint32_t rec_length, + struct xc_sr_rec_page_data_header *pages) +{ + xc_interface *xch = ctx->xch; + uint32_t i, pages_of_data = 0; + xen_pfn_t pfn; + uint32_t type; + bool ret = false; + for ( i = 0; i < pages->count; ++i ) { pfn = pages->pfn[i] & PAGE_DATA_PFN_MASK; @@ -364,23 +295,183 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) * have a page worth of data in the record. */ pages_of_data++; - pfns[i] = pfn; - types[i] = type; + ctx->restore.m->pfns[i] = pfn; + ctx->restore.m->types[i] = type; } - if ( rec->length != (sizeof(*pages) + + if ( rec_length != (sizeof(*pages) + (sizeof(uint64_t) * pages->count) + (PAGE_SIZE * pages_of_data)) ) { ERROR("PAGE_DATA record wrong size: length %u, expected " - "%zu + %zu + %lu", rec->length, sizeof(*pages), + "%zu + %zu + %lu", rec_length, sizeof(*pages), (sizeof(uint64_t) * pages->count), (PAGE_SIZE * pages_of_data)); goto err; } - rc = process_page_data(ctx, pages->count, pfns, types, - &pages->pfn[pages->count]); + ret = true; + +err: + return ret; +} + +/* + * Populate pfns, if required + * Fill m->guest_data with either mapped address or NULL + * The caller must unmap guest_mapping + */ +static int map_guest_pages(struct xc_sr_context *ctx, + struct xc_sr_rec_page_data_header *pages) +{ + xc_interface *xch = ctx->xch; + struct sr_restore_arrays *m = ctx->restore.m; + uint32_t i, p; + int rc; + + rc = populate_pfns(ctx, pages->count, m->pfns, m->types); + if ( rc ) + { + ERROR("Failed to populate pfns for batch of %u pages", pages->count); + goto err; + } + + ctx->restore.nr_mapped_pages = 0; + + for ( i = 0; i < pages->count; i++ ) + { + ctx->restore.ops.set_page_type(ctx, m->pfns[i], m->types[i]); + + if ( page_type_has_stream_data(m->types[i]) == false ) + { + m->guest_data[i] = NULL; + continue; + } + + m->mfns[ctx->restore.nr_mapped_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, m->pfns[i]); + } + + /* Nothing to do? */ + if ( ctx->restore.nr_mapped_pages == 0 ) + goto done; + + ctx->restore.guest_mapping = xenforeignmemory_map(xch->fmem, ctx->domid, + PROT_READ | PROT_WRITE, ctx->restore.nr_mapped_pages, + m->mfns, m->map_errs); + if ( !ctx->restore.guest_mapping ) + { + rc = -1; + PERROR("Unable to map %u mfns for %u pages of data", + ctx->restore.nr_mapped_pages, pages->count); + goto err; + } + + /* Verify mapping, and assign address to pfn data */ + for ( i = 0, p = 0; i < pages->count; i++ ) + { + if ( page_type_has_stream_data(m->types[i]) == false ) + continue; + + if ( m->map_errs[p] == 0 ) + { + m->guest_data[i] = ctx->restore.guest_mapping + (p * PAGE_SIZE); + p++; + continue; + } + + errno = m->map_errs[p]; + rc = -1; + PERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed", + m->pfns[i], m->mfns[p], m->types[i]); + goto err; + } + +done: + rc = 0; + +err: + return rc; +} + +/* + * Handle PAGE_DATA record from an existing buffer + * Given a list of pfns, their types, and a block of page data from the + * stream, populate and record their types, map the relevant subset and copy + * the data into the guest. + */ +static int handle_buffered_page_data(struct xc_sr_context *ctx, + struct xc_sr_record *rec) +{ + xc_interface *xch = ctx->xch; + struct xc_sr_rec_page_data_header *pages = rec->data; + struct sr_restore_arrays *m = ctx->restore.m; + void *p; + uint32_t i; + int rc = -1, idx; + + rc = handle_static_data_end_v2(ctx); + if ( rc ) + goto err; + + /* First read and verify the header */ + if ( verify_rec_page_hdr(ctx, rec->length, pages) == false ) + { + rc = -1; + goto err; + } + + /* Then read and verify the pfn numbers */ + if ( verify_rec_page_pfns(ctx, rec->length, pages) == false ) + { + rc = -1; + goto err; + } + + /* Map the target pfn */ + rc = map_guest_pages(ctx, pages); + if ( rc ) + goto err; + + for ( i = 0, idx = 0; i < pages->count; i++ ) + { + if ( !m->guest_data[i] ) + continue; + + p = &pages->pfn[pages->count] + (idx * PAGE_SIZE); + rc = ctx->restore.ops.localise_page(ctx, m->types[i], p); + if ( rc ) + { + ERROR("Failed to localise pfn %#"PRIpfn" (type %#"PRIx32")", + m->pfns[i], m->types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + goto err; + + } + + if ( ctx->restore.verify ) + { + if ( memcmp(m->guest_data[i], p, PAGE_SIZE) ) + { + errno = EIO; + ERROR("verify pfn %#"PRIpfn" failed (type %#"PRIx32")", + m->pfns[i], m->types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + goto err; + } + } + else + { + memcpy(m->guest_data[i], p, PAGE_SIZE); + } + + idx++; + } + + rc = 0; + err: + if ( ctx->restore.guest_mapping ) + { + xenforeignmemory_unmap(xch->fmem, ctx->restore.guest_mapping, ctx->restore.nr_mapped_pages); + ctx->restore.guest_mapping = NULL; + } return rc; } @@ -641,12 +732,21 @@ static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_recor break; case REC_TYPE_PAGE_DATA: - rc = handle_page_data(ctx, rec); + rc = handle_buffered_page_data(ctx, rec); break; case REC_TYPE_VERIFY: DPRINTF("Verify mode enabled"); ctx->restore.verify = true; + if ( !ctx->restore.verify_buf ) + { + ctx->restore.verify_buf = malloc(MAX_BATCH_SIZE * PAGE_SIZE); + if ( !ctx->restore.verify_buf ) + { + rc = -1; + PERROR("Unable to allocate verify_buf"); + } + } break; case REC_TYPE_CHECKPOINT: @@ -725,7 +825,8 @@ static int setup(struct xc_sr_context *ctx) } ctx->restore.allocated_rec_num = DEFAULT_BUF_RECORDS; - ctx->restore.m = malloc(sizeof(*ctx->restore.m)); + ctx->restore.m = malloc(sizeof(*ctx->restore.m) + + (sizeof(*ctx->restore.m->pages.pfn) * MAX_BATCH_SIZE)); if ( !ctx->restore.m ) { ERROR("Unable to allocate memory for arrays"); rc = -1; From patchwork Wed Jun 16 12:51:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AFCBC48BE5 for ; Wed, 16 Jun 2021 13:07:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 275686024A for ; Wed, 16 Jun 2021 13:07:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 275686024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143004.263820 (Exim 4.92) (envelope-from ) id 1ltVGC-0004Lm-LA; Wed, 16 Jun 2021 13:07:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143004.263820; Wed, 16 Jun 2021 13:07:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGC-0004Ld-Hf; Wed, 16 Jun 2021 13:07:12 +0000 Received: by outflank-mailman (input) for mailman id 143004; Wed, 16 Jun 2021 13:07:11 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2J-0006lZ-2P for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:51 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.173]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1066a928-1d2a-49f9-b44c-1f3a18eed590; Wed, 16 Jun 2021 12:51:51 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpjtmL (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:45 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1066a928-1d2a-49f9-b44c-1f3a18eed590 ARC-Seal: i=1; a=rsa-sha256; t=1623847905; cv=none; d=strato.com; s=strato-dkim-0002; b=bANuIUngkLsBnEZY2I+0vBdL+kR3ZOiAGbA14jWxgDEHBCVpyWfW+yAuoW8rHW1Ez4 fh3DD0wY3yoNJcHlGB+NEHaU99w4wtPbY8MCu/tDS0nQGTE3TRjco6g9xmErPuWMFTbD Md8EVmmNEdg3bLfhW8rwOocIF28q5DHxwsALxipzmD4IkN2sHG6exKiiEhLUXDdRkHoA TWuNWfD+zcAVreUHyVOxZkzhlzeCqrKHKrxAt+Zvrxummz3Oy785D8bZ/NF1j8yWXFey szCXCc5WJjS3ShjrdPjH0t/uNX1Ar/bNsRp+XmaXTFsCfddSRbzcDxL/w8kNKyLyq1wN e1Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847905; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=XvzNAYL4Bz6FS15Jp93i8O47PVWqmiH8eMe8cuJSWuw=; b=GtJEASD9ixYM8kwE/+pjHDoFO61iRoeMYDGQE7gtB1iAGP3z+P/F/7YErteofWL/2V B3iyQ698+k9IrcOJ5VrOKsd30fgH/qxo1NEPvLPlqM3oPvKnmOVzdHn3gabr+BHmySRd rqGfEhP5eYFtUjBG7HJ+me6ucNS2e8hTyBVTp/9/RaW2CWjTdrPdLPO1DU3+2GWteF8z kRirjomPtwCKL3FO6f+gkLgXPOw5T63Gds/4fg1pQPqANJAHGYXSDE4Dz1NiBNayEaHL ZxhtD1pX88KLyP2lYYsUZqmhRQ9GdaRzhm6vRjTiDx1xb4/v56mcW1ioc2ioXfrT5jvb fDYQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847905; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=XvzNAYL4Bz6FS15Jp93i8O47PVWqmiH8eMe8cuJSWuw=; b=EXhLBPz/onKs0O5mn7+1kh6DniN7HJd6hIGmA5MmHqQsmZ8tEp9ubhaW9FSyISWoSp gFxMvmLF/lMcVsGZbnCQgGku0AYAEgde1LLnc8lQcOLNgjXl/GB8WcaCvzHDEobu/mfS j2c/Ivbks06pgF2QYlEyiazfcNYzxq5c4rv2ti/AzbaaNC1ufl4ehGV4UY5+CsZIkDSW lXbbQLt/MU1WTDgJvspuAHk7ADM0V45qGnG8g6/KZh9r+jS6oLaJQPb6yVUqfSUJS+l4 rWUmEXNz8zwXYNOR0VXzUe/iCRzR1Vai6F81fsA7hh/zj3kiNJ4LhotGoa9CBsXl75El DteA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 26/36] tools: restore: write data directly into guest Date: Wed, 16 Jun 2021 14:51:19 +0200 Message-Id: <20210616125129.26563-27-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Read incoming migration stream directly into the guest memory. This avoids the memory allocation and copying, and the resulting performance penalty. Signed-off-by: Olaf Hering --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 132 ++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index d479f1a918..5c440f28ec 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -243,6 +243,7 @@ struct sr_restore_arrays { xen_pfn_t mfns[MAX_BATCH_SIZE]; int map_errs[MAX_BATCH_SIZE]; void *guest_data[MAX_BATCH_SIZE]; + struct iovec iov[MAX_BATCH_SIZE]; /* populate_pfns */ xen_pfn_t pp_mfns[MAX_BATCH_SIZE]; diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 877fd19a9b..d0148606bf 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -392,6 +392,122 @@ err: return rc; } +/* + * Handle PAGE_DATA record from the stream. + * Given a list of pfns, their types, and a block of page data from the + * stream, populate and record their types, map the relevant subset and copy + * the data into the guest. + */ +static int handle_incoming_page_data(struct xc_sr_context *ctx, + struct xc_sr_rhdr *rhdr) +{ + xc_interface *xch = ctx->xch; + struct sr_restore_arrays *m = ctx->restore.m; + struct xc_sr_rec_page_data_header *pages = &m->pages; + uint64_t *pfn_nums = m->pages.pfn; + uint32_t i; + int rc, iov_idx; + + rc = handle_static_data_end_v2(ctx); + if ( rc ) + goto err; + + /* First read and verify the header */ + rc = read_exact(ctx->fd, pages, sizeof(*pages)); + if ( rc ) + { + PERROR("Could not read rec_pfn header"); + goto err; + } + + if ( verify_rec_page_hdr(ctx, rhdr->length, pages) == false ) + { + rc = -1; + goto err; + } + + /* Then read and verify the incoming pfn numbers */ + rc = read_exact(ctx->fd, pfn_nums, sizeof(*pfn_nums) * pages->count); + if ( rc ) + { + PERROR("Could not read rec_pfn data"); + goto err; + } + + if ( verify_rec_page_pfns(ctx, rhdr->length, pages) == false ) + { + rc = -1; + goto err; + } + + /* Finally read and verify the incoming pfn data */ + rc = map_guest_pages(ctx, pages); + if ( rc ) + goto err; + + /* Prepare read buffers, either guest or throw away memory */ + for ( i = 0, iov_idx = 0; i < pages->count; i++ ) + { + if ( !m->guest_data[i] ) + continue; + + m->iov[iov_idx].iov_len = PAGE_SIZE; + if ( ctx->restore.verify ) + m->iov[iov_idx].iov_base = ctx->restore.verify_buf + i * PAGE_SIZE; + else + m->iov[iov_idx].iov_base = m->guest_data[i]; + iov_idx++; + } + + if ( !iov_idx ) + goto done; + + rc = readv_exact(ctx->fd, m->iov, iov_idx); + if ( rc ) + { + PERROR("read of %d pages failed", iov_idx); + goto err; + } + + /* Post-processing of pfn data */ + for ( i = 0, iov_idx = 0; i < pages->count; i++ ) + { + if ( !m->guest_data[i] ) + continue; + + rc = ctx->restore.ops.localise_page(ctx, m->types[i], m->iov[iov_idx].iov_base); + if ( rc ) + { + ERROR("Failed to localise pfn %#"PRIpfn" (type %#"PRIx32")", + m->pfns[i], m->types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + goto err; + + } + + if ( ctx->restore.verify ) + { + if ( memcmp(m->guest_data[i], m->iov[iov_idx].iov_base, PAGE_SIZE) ) + { + ERROR("verify pfn %#"PRIpfn" failed (type %#"PRIx32")", + m->pfns[i], m->types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + } + } + + iov_idx++; + } + +done: + rc = 0; + +err: + if ( ctx->restore.guest_mapping ) + { + xenforeignmemory_unmap(xch->fmem, ctx->restore.guest_mapping, ctx->restore.nr_mapped_pages); + ctx->restore.guest_mapping = NULL; + } + return rc; +} + /* * Handle PAGE_DATA record from an existing buffer * Given a list of pfns, their types, and a block of page data from the @@ -773,11 +889,19 @@ static int process_incoming_record_header(struct xc_sr_context *ctx, struct xc_s struct xc_sr_record rec; int rc; - rc = read_record_data(ctx, ctx->fd, rhdr, &rec); - if ( rc ) - return rc; + switch ( rhdr->type ) + { + case REC_TYPE_PAGE_DATA: + rc = handle_incoming_page_data(ctx, rhdr); + break; + default: + rc = read_record_data(ctx, ctx->fd, rhdr, &rec); + if ( rc == 0 ) + rc = process_buffered_record(ctx, &rec);; + break; + } - return process_buffered_record(ctx, &rec); + return rc; } From patchwork Wed Jun 16 12:51:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3ACCC49361 for ; Wed, 16 Jun 2021 13:08:47 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC8EB61166 for ; Wed, 16 Jun 2021 13:08:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC8EB61166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143115.263990 (Exim 4.92) (envelope-from ) id 1ltVHd-0004Gs-7l; Wed, 16 Jun 2021 13:08:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143115.263990; Wed, 16 Jun 2021 13:08:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHc-0004Fr-Sd; Wed, 16 Jun 2021 13:08:40 +0000 Received: by outflank-mailman (input) for mailman id 143115; Wed, 16 Jun 2021 13:08:38 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV1t-00075D-8k for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:25 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id fd38271b-afa3-43fb-bb08-840dca73930b; Wed, 16 Jun 2021 12:51:50 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpjtmM (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:45 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fd38271b-afa3-43fb-bb08-840dca73930b ARC-Seal: i=1; a=rsa-sha256; t=1623847906; cv=none; d=strato.com; s=strato-dkim-0002; b=Vbs1j7mJbricsA1IIQ8f5z4T87A3bI9ApYAk+bMRpD8bhxcTw5qvgllKCvFNm+WwCw WmOKlZMV9uGIwJg03kHSni+u88xYHW9DMdPtis4Q9gXtaQgY9xbbplBw5+X3BwqZe59w VkvV/xN9uEtJNjE18co5YigmoL1BPFlF2RAQG3NJyHpK7YGEowhl5f93iEbj3MoAGMSJ fOjOPiNLSdVQoKSulFYSp9mWLF8VikPXhZSPTJNK85EmXNOxJnuAMGtBvqiBKRP3Yiqi 2P6Irr3/kB0KEnCZZFvnk1cNpKjCQ8q3pxlPwYNHDI4OkxykCWdAo2iMqZ9PE9pcENzd 4mOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847906; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=0LMXyjqr5dhJU6Kt86LbarPPbfCTtmro+NAOq1cH7Ts=; b=pLJdI34sA4mQh8XdxM4ZJwGOprEg6rp0dENeOEVsTbec9S87BDRDWvSYINYY+MvbSv jwRvEPnnWuNfJ91oRSnr/1zGp3uI5Rt+pvRH4sCJzy/jum3qIETlctAlLXVH3snHFyOQ c7p7qLFz3L5s5dDCal9ApishyyhVgMjir9anBkAKd6XycAnhyKYFI0pKz49WnBtkMyJp nuin+YUvbYR4iKQJgm7+544NQDJnUjZQi7zMlBnwSywmFe87HlGD3gmupMFXdP5NvD6q OaB6rJ2VXaXxYzLF94l0r8EYg/B23qSunWp/+6WQ/Yyj3hmXBzfPIDJPxHw4J8tm2yx+ ewtw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847906; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=0LMXyjqr5dhJU6Kt86LbarPPbfCTtmro+NAOq1cH7Ts=; b=iYOUWSZIPy/UeeZszze/3scZsU10jMdAzh7SXzV8Ou6jDvEfVIfruOS0enhMOI1WnQ +3tTWcJdMz5iKbR4gQX/ZrQXs7uUvDpyQfn5HhXOUyrVtNlIFw7eg7v9mFb5WTRjnWYY Rsmhw6Ao1Uem1OgyRvZbFbUnLyFLaEpOsa/Kjo4dAjd9wifCjt/KjwUDYIsBe2JRI3B/ worMaYRqYW/RmV5KgKDJpCLhwCIHibY7FYcAQJJ+bDHxl5etIiPgxK8patbwwzcVvSkV BAckxj5Wghl0WjZwzjtlD0HCAgXTaNsdbGCrlzogg9eLVnq5T9uo1NDS3XfVH6FvW+bo TcLg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu Subject: [PATCH v20210616 27/36] tools: recognize LIBXL_API_VERSION for 4.16 Date: Wed, 16 Jun 2021 14:51:20 +0200 Message-Id: <20210616125129.26563-28-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 This is required by upcoming API changes. Signed-off-by: Olaf Hering --- tools/include/libxl.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index ae7fe27c1f..29931626a2 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -729,7 +729,8 @@ typedef struct libxl__ctx libxl_ctx; #if LIBXL_API_VERSION != 0x040200 && LIBXL_API_VERSION != 0x040300 && \ LIBXL_API_VERSION != 0x040400 && LIBXL_API_VERSION != 0x040500 && \ LIBXL_API_VERSION != 0x040700 && LIBXL_API_VERSION != 0x040800 && \ - LIBXL_API_VERSION != 0x041300 && LIBXL_API_VERSION != 0x041400 + LIBXL_API_VERSION != 0x041300 && LIBXL_API_VERSION != 0x041400 && \ + LIBXL_API_VERSION != 0x041600 #error Unknown LIBXL_API_VERSION #endif #endif From patchwork Wed Jun 16 12:51:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9277C48BE6 for ; Wed, 16 Jun 2021 13:08:46 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6CBEA6024A for ; Wed, 16 Jun 2021 13:08:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6CBEA6024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143112.263974 (Exim 4.92) (envelope-from ) id 1ltVHb-0003ox-3R; Wed, 16 Jun 2021 13:08:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143112.263974; Wed, 16 Jun 2021 13:08:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVHa-0003oT-UN; Wed, 16 Jun 2021 13:08:38 +0000 Received: by outflank-mailman (input) for mailman id 143112; Wed, 16 Jun 2021 13:08:38 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2s-0006lZ-3z for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:53:26 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.175]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 836fdc85-aa4f-4342-9e78-cb0882912229; Wed, 16 Jun 2021 12:52:01 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpktmN (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:46 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 836fdc85-aa4f-4342-9e78-cb0882912229 ARC-Seal: i=1; a=rsa-sha256; t=1623847906; cv=none; d=strato.com; s=strato-dkim-0002; b=Bb02/uYdYI3v3QAK8agr/MyAFv/G7Oa2Bx7a+FYVforjXXtyAQMg+Q0iqz0l0bqefP /lnqy5lhD55+fZcRhOp0dKREn/o1okFQfsRHdNGvHo6Lgex5rGsh0Hitb09cXi4IzRJO eeRMtn40kAnmG6bwOc5iikT2XLb4CRET14Etyez4ckvBL28Ila9AT3boSSqe4sarKbSn Nd+ORIevFXOQPL5BCw1T49q8D7nEU03VzXmzFTXScaRFYrPtyDPo8bdMli/IT6l9895w 9+mzysccjy+OQQAnFcj8uaftHeC+lfCqZ8q9QzM6hT5iWOvIMvN4JI6CdfGO6XXQieDG XJgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847906; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=n0d0gwWXlJ/Ban5eVrK506Di/gOt/MD61K+N2RGQBbA=; b=q1UKio5xKBtR+KdxSh2Yno0DYwGIVeXHUBzHinABh6f2lymqVTcENFragEmMOlCYXj CT8T9OMHy8B/f+qQ9SLFQqraQw73s+ycFM47rF1imI6qwXcZYIINA6gVibgn6ZJa0YTl GRJfy3zeRhgAp8TRyjwNWlyV6FPws6FI/dmpk6swQyFWtG7jVEP+YNV8fjuv18tliD/q +c4WZHqo8nCuuq3ZKcmH6FYuH6Mt9dg1h9Yxgn2lrqwE9g0QSdWTQUeS/6ZGqLp4gFgl drSnIlEoYzFUJKjX1cihsbP5M8MGMjuqEdbiGD/5QgAglMNpmXR/2gLwJs0gXXBy3ZTg bCqA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847906; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=n0d0gwWXlJ/Ban5eVrK506Di/gOt/MD61K+N2RGQBbA=; b=g3wCKnOJqhlu289qUqGeYqy9RDAUEaOV7Yw1QOo/Y/prbvl4AlhGdYNtBPV3koqqxd 5uXUaYW0z9j7h1+jH/krk/hO4rvCxVYiFzwKxwXbhRAAbcoYRmAQkOOUF/cv5gWVZv0a rJ4Xw08/2hkOEX43CKJG92FbvATCekbqj7nmJUmroaqWmC5QQaZ7YozFKqEMn4Jz5sDC 0QR1DWVHXs+N/Jt1GirmjX2M2jfOZ9ep4bXQrC23hrF/yMEDX1vWKUsLPdZcqxtsClgi GC8WwDpJtzxmk1fhYuH1lUXd32BavjwbFcKhyN0pm6Fxxftep1sLkhdd9zNeivGIKxmB U0OA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Christian Lindig , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross , David Scott Subject: [PATCH v20210616 28/36] tools: adjust libxl_domain_suspend to receive a struct props Date: Wed, 16 Jun 2021 14:51:21 +0200 Message-Id: <20210616125129.26563-29-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Upcoming changes will pass more knobs down to xc_domain_save. Adjust the libxl_domain_suspend API to allow easy adding of additional knobs. No change in behavior intented. Signed-off-by: Olaf Hering Acked-by: Christian Lindig --- tools/include/libxl.h | 26 +++++++++++++++++++++++--- tools/libs/light/libxl_domain.c | 7 ++++--- tools/ocaml/libs/xl/xenlight_stubs.c | 3 ++- tools/xl/xl_migrate.c | 9 ++++++--- tools/xl/xl_saverestore.c | 3 ++- 5 files changed, 37 insertions(+), 11 deletions(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 29931626a2..9a4d7514ed 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1706,12 +1706,32 @@ static inline int libxl_retrieve_domain_configuration_0x041200( libxl_retrieve_domain_configuration_0x041200 #endif +/* + * LIBXL_HAVE_DOMAIN_SUSPEND_PROPS indicates that the + * libxl_domain_suspend_props() function takes a props struct. + */ +#define LIBXL_HAVE_DOMAIN_SUSPEND_PROPS 1 + +typedef struct { + uint32_t flags; /* LIBXL_SUSPEND_* */ +} libxl_domain_suspend_props; +#define LIBXL_SUSPEND_DEBUG 1 +#define LIBXL_SUSPEND_LIVE 2 + int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, - int flags, /* LIBXL_SUSPEND_* */ + libxl_domain_suspend_props *props, const libxl_asyncop_how *ao_how) LIBXL_EXTERNAL_CALLERS_ONLY; -#define LIBXL_SUSPEND_DEBUG 1 -#define LIBXL_SUSPEND_LIVE 2 +#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x041600 +static inline int libxl_domain_suspend_0x041500(libxl_ctx *ctx, uint32_t domid, + int fd, int flags, /* LIBXL_SUSPEND_* */ + const libxl_asyncop_how *ao_how) +{ + libxl_domain_suspend_props props = { .flags = flags, }; + return libxl_domain_suspend(ctx, domid, fd, &props, ao_how); +} +#define libxl_domain_suspend libxl_domain_suspend_0x041500 +#endif /* * Only suspend domain, do not save its state to file, do not destroy it. diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index c00c36c928..5dbd27900f 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -505,7 +505,8 @@ static void domain_suspend_cb(libxl__egc *egc, } -int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags, +int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, + libxl_domain_suspend_props *props, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); @@ -526,8 +527,8 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags, dss->domid = domid; dss->fd = fd; dss->type = type; - dss->live = flags & LIBXL_SUSPEND_LIVE; - dss->debug = flags & LIBXL_SUSPEND_DEBUG; + dss->live = props->flags & LIBXL_SUSPEND_LIVE; + dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; rc = libxl__fd_flags_modify_save(gc, dss->fd, diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c b/tools/ocaml/libs/xl/xenlight_stubs.c index 352a00134d..eaf7bce35a 100644 --- a/tools/ocaml/libs/xl/xenlight_stubs.c +++ b/tools/ocaml/libs/xl/xenlight_stubs.c @@ -614,10 +614,11 @@ value stub_libxl_domain_suspend(value ctx, value domid, value fd, value async, v int ret; uint32_t c_domid = Int_val(domid); int c_fd = Int_val(fd); + libxl_domain_suspend_props props = {}; libxl_asyncop_how *ao_how = aohow_val(async); caml_enter_blocking_section(); - ret = libxl_domain_suspend(CTX, c_domid, c_fd, 0, ao_how); + ret = libxl_domain_suspend(CTX, c_domid, c_fd, &props, ao_how); caml_leave_blocking_section(); free(ao_how); diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index b8594f44a5..144890924f 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -186,7 +186,10 @@ static void migrate_domain(uint32_t domid, int preserve_domid, char *away_domname; char rc_buf; uint8_t *config_data; - int config_len, flags = LIBXL_SUSPEND_LIVE; + int config_len; + libxl_domain_suspend_props props = { + .flags = LIBXL_SUSPEND_LIVE, + }; save_domain_core_begin(domid, preserve_domid, override_config_file, &config_data, &config_len); @@ -205,8 +208,8 @@ static void migrate_domain(uint32_t domid, int preserve_domid, xtl_stdiostream_adjust_flags(logger, XTL_STDIOSTREAM_HIDE_PROGRESS, 0); if (debug) - flags |= LIBXL_SUSPEND_DEBUG; - rc = libxl_domain_suspend(ctx, domid, send_fd, flags, NULL); + props.flags |= LIBXL_SUSPEND_DEBUG; + rc = libxl_domain_suspend(ctx, domid, send_fd, &props, NULL); if (rc) { fprintf(stderr, "migration sender: libxl_domain_suspend failed" " (rc=%d)\n", rc); diff --git a/tools/xl/xl_saverestore.c b/tools/xl/xl_saverestore.c index 953d791d1a..476d4d9a6a 100644 --- a/tools/xl/xl_saverestore.c +++ b/tools/xl/xl_saverestore.c @@ -130,6 +130,7 @@ static int save_domain(uint32_t domid, int preserve_domid, int fd; uint8_t *config_data; int config_len; + libxl_domain_suspend_props props = {}; save_domain_core_begin(domid, preserve_domid, override_config_file, &config_data, &config_len); @@ -146,7 +147,7 @@ static int save_domain(uint32_t domid, int preserve_domid, save_domain_core_writeconfig(fd, filename, config_data, config_len); - int rc = libxl_domain_suspend(ctx, domid, fd, 0, NULL); + int rc = libxl_domain_suspend(ctx, domid, fd, &props, NULL); close(fd); if (rc < 0) { From patchwork Wed Jun 16 12:51:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6CDDC48BE8 for ; Wed, 16 Jun 2021 13:06:58 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9ADE761359 for ; Wed, 16 Jun 2021 13:06:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9ADE761359 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.142983.263787 (Exim 4.92) (envelope-from ) id 1ltVFk-0002up-Jo; Wed, 16 Jun 2021 13:06:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 142983.263787; Wed, 16 Jun 2021 13:06:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVFk-0002ui-FX; Wed, 16 Jun 2021 13:06:44 +0000 Received: by outflank-mailman (input) for mailman id 142983; Wed, 16 Jun 2021 13:06:43 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV23-00075D-90 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:35 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1d52c234-137d-4e0e-94cb-eb330f4ac413; Wed, 16 Jun 2021 12:51:52 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpltmO (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:47 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1d52c234-137d-4e0e-94cb-eb330f4ac413 ARC-Seal: i=1; a=rsa-sha256; t=1623847907; cv=none; d=strato.com; s=strato-dkim-0002; b=LMTXMECnOOeKSJF1PJhPqO/25SF9NGK1THJsaFw/1J/70Y45J1kHvpSqyapwdqlfjS ipM49hs3lVxcpWlVpQCaMx1E1TVPLYp9rfnlTyD2pCNXF8+Vuw+1xzcx/52WMl/kfAGN OgP2Qz73BnsH4oRTZwDMCwjtrQJde4n0ufC7AJsXyUIJj9fYLbFgXK4whg1f09IH/qaK xKdt58Rs0VEXrFVKRPYARh9Z5lhA5skY6f7Q38KncPPagZtKKsc8O0/c9FFoeX/U4leC n8E57UTQjQOTsXRLzT5CkaJtgyPTbFviN4njGZ3F3D3x0PGKzwIaff3yNtfo3PQfgaJ/ URrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847907; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=YGLyCDpmfEm63K5TIs+xnzmQnbcLd4l6eUuaqJ8OGLg=; b=jXRFafS3o7BXy7rTxn/pEYaFG0E8WY33NmNWTFSQaRBis0744oY8Pyv0hHr/3I2lOZ Gax4JSMzlmxlp04trR7+k6VAmxz6VUKsOZtPKAnIAA4OwzpgUoy13onu4FLzkHz9dPZK KctNTiLN4mKeUK4kJwVt6Q8B35icMQNhdGYzyLQ3hIX0zu3OhSt8Ljr+ccskJijWmSUo 7WZfxSyaw6WQhqAmBS4LbMhwVxOX9f2p4CY/NLa+x5HSGqneJetffaJtOIG5LOBllmJJ 7s0Go3Fwe9q1wm6WwFohapArhlnDBZ9bqIf+IijghCMYnh7dTWO2M/yLHqlSWVhaJTgC WTLg== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847907; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=YGLyCDpmfEm63K5TIs+xnzmQnbcLd4l6eUuaqJ8OGLg=; b=q8THds8HGfoUSPajomJ9wIW1uH8sUibkJEb3bkBcdaOlZUohpfD7m5LcOugPhyFwog 104Cp3Lts5G+EAu4WBi+LB4oMuJIeTTdLKuzsXZcWSZaZnzHDbD/bu6D9ivTGM5G8t54 5Fxj3RSR5JxEsfQ96QmBUPPTV8fvuY5wALKZV6H7f2aftX7bIgdCeoiAJMMfFIx5Ajpd FLXgzOAsxM7hJ9PUIy9GgapNYw6NSa6sk1Oq3Fd/ICR80oMZdedzBquBzVTfruekinfg 2p1op6igS7JFuwBPgKdH/ASiPY56snontUIX+7riJ3KBqSqNcPOF6qVVq9Q2oQCS4yJe R6Og== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 29/36] tools: change struct precopy_stats to precopy_stats_t Date: Wed, 16 Jun 2021 14:51:22 +0200 Message-Id: <20210616125129.26563-30-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 This will help libxl_save_msgs_gen.pl to copy the struct as a region of memory. No change in behavior intented. Signed-off-by: Olaf Hering --- tools/include/xensaverestore.h | 7 +++---- tools/libs/saverestore/common.h | 2 +- tools/libs/saverestore/save.c | 6 +++--- 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/tools/include/xensaverestore.h b/tools/include/xensaverestore.h index 0410f0469e..dca0134605 100644 --- a/tools/include/xensaverestore.h +++ b/tools/include/xensaverestore.h @@ -23,18 +23,17 @@ #define XCFLAGS_DEBUG (1 << 1) /* For save's precopy_policy(). */ -struct precopy_stats -{ +typedef struct { unsigned int iteration; unsigned long total_written; long dirty_count; /* -1 if unknown */ -}; +} precopy_stats_t; /* * A precopy_policy callback may not be running in the same address * space as libxc an so precopy_stats is passed by value. */ -typedef int (*precopy_policy_t)(struct precopy_stats, void *); +typedef int (*precopy_policy_t)(precopy_stats_t, void *); /* callbacks provided by xc_domain_save */ struct save_callbacks { diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 5c440f28ec..60bbba6aa9 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -283,7 +283,7 @@ struct xc_sr_context size_t pages_sent; size_t overhead_sent; - struct precopy_stats stats; + precopy_stats_t stats; unsigned int nr_batch_pfns; unsigned long *deferred_pages; diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c index e486bce96f..537b977ba8 100644 --- a/tools/libs/saverestore/save.c +++ b/tools/libs/saverestore/save.c @@ -488,7 +488,7 @@ static int update_progress_string(struct xc_sr_context *ctx, char **str) #define SPP_MAX_ITERATIONS 5 #define SPP_TARGET_DIRTY_COUNT 50 -static int simple_precopy_policy(struct precopy_stats stats, void *user) +static int simple_precopy_policy(precopy_stats_t stats, void *user) { return ((stats.dirty_count >= 0 && stats.dirty_count < SPP_TARGET_DIRTY_COUNT) || @@ -515,13 +515,13 @@ static int send_memory_live(struct xc_sr_context *ctx) precopy_policy_t precopy_policy = ctx->save.callbacks->precopy_policy; void *data = ctx->save.callbacks->data; - struct precopy_stats *policy_stats; + precopy_stats_t *policy_stats; rc = update_progress_string(ctx, &progress_str); if ( rc ) goto out; - ctx->save.stats = (struct precopy_stats){ + ctx->save.stats = (precopy_stats_t){ .dirty_count = ctx->save.p2m_size, }; policy_stats = &ctx->save.stats; From patchwork Wed Jun 16 12:51:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DDE0C48BE6 for ; Wed, 16 Jun 2021 13:08:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 191166024A for ; Wed, 16 Jun 2021 13:08:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 191166024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143052.263886 (Exim 4.92) (envelope-from ) id 1ltVGq-0007vq-0X; Wed, 16 Jun 2021 13:07:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143052.263886; Wed, 16 Jun 2021 13:07:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGp-0007vg-St; Wed, 16 Jun 2021 13:07:51 +0000 Received: by outflank-mailman (input) for mailman id 143052; Wed, 16 Jun 2021 13:07:50 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2N-00075D-9W for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:55 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.103]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id f2b453df-01fb-4b67-9fad-bdacdf928dd0; Wed, 16 Jun 2021 12:51:55 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpltmP (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:47 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f2b453df-01fb-4b67-9fad-bdacdf928dd0 ARC-Seal: i=1; a=rsa-sha256; t=1623847907; cv=none; d=strato.com; s=strato-dkim-0002; b=kIuewaBaUo7N4cc0L8a9lIZxZRuflXX7CS3wZNNzDXoBBAFZK8OgO5Vgn7jTiXKbHu F4D58+c7d5+4Z2ezbhWTlNpJF0l0Y6XER3UScRfd52ggzeIE+iaFltig9uXMSEwCAcUm zbPBypeK9qBKXuvulxrB2zYStI+7hi3uA2tYoQ5QfTX/BfskBO8mc7kv1Z2w23zgb2Y1 cPPTPHbnbp/qmYBvkvsBfrD+TNmzrBAiEG212yqlb/i/6Cwm6Sas4wKuMxm7GNEvGXl9 HUQJl0tzAtosl9LEExc50fO5k+e/W9OxtBsCo/fv+o2BKnjX3HezgR6+2GZSP2GMRMSh PlWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847907; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=am1tugVzg71doIb/LdiNWtayiDvPFoCjZNyMmciOrqY=; b=GqFb2vbB+Jh89wB/deQ9VqUiTbVkXdOAaLyOfBMuvAOeFlEEJQCYaBIfsFIiYwEo7u KAwSemkL/0AstwiAPeaGhNv9uC7QnrUR1TbD+AmUtD6t3n+2PhejtIvKccKBHRz666sw S5iZKwghbJVzONTNoYPeFWNGcl6778xol3DgrXtn2Rcl38Ob7MYxjSc2RAe1aA6qM5G+ mZroe03cbZ1o/sCwIXK11WhYdCpIAftpb+gVYC/+oOWjFGAAlMeEIE6F3KVVyy1zvGTJ aB0b2vU1m8I0huk4P3Nu17YIGc4/4bFOxceFvtKNGMLTp4FN1YDWjfrFB23AJ4h7jV+u HUBQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847907; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=am1tugVzg71doIb/LdiNWtayiDvPFoCjZNyMmciOrqY=; b=ZyPflmDE/hyYGB/vXZcl7u7B+KonkHuWE4Mfv2YzW2JwqivRJ0Zrfz8oNrJEYVF9xX crKYZV8633x4FUi6IQpDkEKP0/8al9TL6vmmoe5RJgUBirwguDpNMHeEIyhBPTYIP4jC E4ZK+JbnP513hgwumJ504JhFWRvvAJm6fyNyTwbLVicuPejYmqpZFVbjRq5kjUWNLNeg BWTxwa9FWaE7hjzFQVZ5catJMO5nSelGGbvnQW16LaW0+PGZMxyPZNG+10d6tWdGLoP9 +oQQ0koGIFuI3mlgF8zRRNRP4gE9jpSyJWV8Sa6GtlX0FOpBVNYH+8CFcBHQu9EbOhCM jpYg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210616 30/36] tools: add callback to libxl for precopy_policy and precopy_stats_t Date: Wed, 16 Jun 2021 14:51:23 +0200 Message-Id: <20210616125129.26563-31-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 This duplicates simple_precopy_policy. To recap its purpose: - do up to 5 iterations of copying dirty domU memory to target, including the initial copying of all domU memory, excluding the final copying while the domU is suspended - do fewer iterations in case the domU dirtied less than 50 pages Take the opportunity to also move xen_pfn_t into qw(). Signed-off-by: Olaf Hering --- tools/libs/light/libxl_dom_save.c | 19 +++++++++++++++++++ tools/libs/light/libxl_internal.h | 2 ++ tools/libs/light/libxl_save_msgs_gen.pl | 3 ++- 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 32e3cb5a13..3f3cff0342 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -373,6 +373,24 @@ int libxl__save_emulator_xenstore_data(libxl__domain_save_state *dss, return rc; } +static int libxl__domain_save_precopy_policy(precopy_stats_t stats, void *user) +{ + libxl__save_helper_state *shs = user; + libxl__domain_save_state *dss = shs->caller_state; + STATE_AO_GC(dss->ao); + + LOGD(DEBUG, shs->domid, "iteration %u dirty_count %ld total_written %lu", + stats.iteration, stats.dirty_count, stats.total_written); + if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT) + goto stop_copy; + if (stats.iteration >= LIBXL_XGS_POLICY_MAX_ITERATIONS) + goto stop_copy; + return XGS_POLICY_CONTINUE_PRECOPY; + +stop_copy: + return XGS_POLICY_STOP_AND_COPY; +} + /*----- main code for saving, in order of execution -----*/ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) @@ -430,6 +448,7 @@ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) callbacks->suspend = libxl__domain_suspend_callback; callbacks->switch_qemu_logdirty = libxl__domain_suspend_common_switch_qemu_logdirty; + callbacks->precopy_policy = libxl__domain_save_precopy_policy; dss->sws.ao = dss->ao; dss->sws.dss = dss; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index 439c654733..57d7e4b4b8 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -125,6 +125,8 @@ #define DOMID_XS_PATH "domid" #define PVSHIM_BASENAME "xen-shim" #define PVSHIM_CMDLINE "pv-shim console=xen,pv" +#define LIBXL_XGS_POLICY_MAX_ITERATIONS 5 +#define LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT 50 /* Size macros. */ #define __AC(X,Y) (X##Y) diff --git a/tools/libs/light/libxl_save_msgs_gen.pl b/tools/libs/light/libxl_save_msgs_gen.pl index f263ee01bb..ab55c81644 100755 --- a/tools/libs/light/libxl_save_msgs_gen.pl +++ b/tools/libs/light/libxl_save_msgs_gen.pl @@ -23,6 +23,7 @@ our @msgs = ( STRING doing_what), 'unsigned long', 'done', 'unsigned long', 'total'] ], + [ 'scxW', "precopy_policy", ['precopy_stats_t', 'stats'] ], [ 'srcxA', "suspend", [] ], [ 'srcxA', "postcopy", [] ], [ 'srcxA', "checkpoint", [] ], @@ -142,7 +143,7 @@ static void bytes_put(unsigned char *const buf, int *len, END -foreach my $simpletype (qw(int uint16_t uint32_t unsigned), 'unsigned long', 'xen_pfn_t') { +foreach my $simpletype (qw(int uint16_t uint32_t unsigned precopy_stats_t xen_pfn_t), 'unsigned long') { my $typeid = typeid($simpletype); $out_body{'callout'} .= < X-Patchwork-Id: 12325173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5794BC48BE5 for ; Wed, 16 Jun 2021 13:07:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 10BA761166 for ; Wed, 16 Jun 2021 13:07:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10BA761166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143029.263875 (Exim 4.92) (envelope-from ) id 1ltVGY-0006px-Lh; Wed, 16 Jun 2021 13:07:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143029.263875; Wed, 16 Jun 2021 13:07:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGY-0006pc-Ho; Wed, 16 Jun 2021 13:07:34 +0000 Received: by outflank-mailman (input) for mailman id 143029; Wed, 16 Jun 2021 13:07:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2S-00075D-9q for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:53:00 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 95efd68c-aecb-4df5-81e1-862432e556f6; Wed, 16 Jun 2021 12:51:56 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpmtmR (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:48 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 95efd68c-aecb-4df5-81e1-862432e556f6 ARC-Seal: i=1; a=rsa-sha256; t=1623847908; cv=none; d=strato.com; s=strato-dkim-0002; b=cHIyIkmuhvUDNSy/ndXRUCW+1wkpVW7xVLVw6QYGgg8ZFKtrTjVR0kuYWyZbxNWHyz 1cCvLRiVNJ47kWv6rN0JzfpOi3CWwF5PR2W4EpVsCacVY1AFWZVv71uEQkVeqpqXvCsr ngOw7RTpHM9y5FfxOOs2yVwHTSgP0Vh1Eqo+YrXdAse+LS1gI8gp2vHI9LR/Cddv36ny IwmTmDOKPf+ARZsXhmCcJlBc69guJOtLvXMrCycDgd/FxJYyW1KQlErrloh6cXpP6CIS TTEMbmLulxivexXgp1lAUZLFSHBAWXfa5wdD4QW9HG6IFNsosgt6AYkZPPpXXdUGqFGJ OWcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847908; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=6LebzrvgpkRPMXPrQvQqUEqJ9oCT3tBlpGnE42P4LCk=; b=lmir08Rm5poB1h9YZdUUyI82Vm670RNQ5lbdEDt6mvlI2N1LVvVTnDYkjvjcZd94da SA6ctKVSltK3cfFmg+X1LkOOh/T8kYz9MOoRWL3M7bAtqgb/Vf768T5ZD4LBJIRtM1+u dmte6Ae+Xvv4l8iPjgcsHepokmZ+kbSt5aPr9Q2oM0g231B6FzkPOzkNSdAy8/Tma5f1 4zOj00HTkjc0qlfwtyyvCuPUnwCBdjH/YXj2CSUMztRRv8IwZn/G+9r+0vIBbpj0Yr5a KX0B54xfbznJPo5siGlqkPS77wArfWCQ6JdLFGJBJai83fiXNq7BeG5BVLVvzgGHcXk/ dHpA== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847908; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=6LebzrvgpkRPMXPrQvQqUEqJ9oCT3tBlpGnE42P4LCk=; b=hKrESaYSskxqbrwZ/gQCQYM/Cof6kuuYncYKGb2HW7qY7+k8Ysk54N5YHqE3QqyHmP cO8/I8XpWaPnkeZjaJDooigmKSEsoSOXEbEcX02Fa6DVlIbag+l/KHiLOYBCLAbqgM60 2dJWURDk9251OlsT2hPyYc0btejRsB7x6w/+Rc5GM7CKrbpNH50tSvfcUBBg2AeJnO3r a22NE7XVlvT50bo2lEacRLXPtLgN+rMEp8mowE31KNJCOd/zmFi3hJ62bvzXt8U9hu7c tOJHVNmWUM24xj76OUmsUN5Fzh7utoS5wtPYqTWI3DZXIwkFugF3Sq51DllrmTAUl61u rEcg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210616 31/36] tools: add --max_iters to libxl_domain_suspend Date: Wed, 16 Jun 2021 14:51:24 +0200 Message-Id: <20210616125129.26563-32-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Migrating a large, and potentially busy, domU will take more time than neccessary due to excessive number of copying iterations. Allow to host admin to control the number of iterations which copy cumulated domU dirty pages to the target host. The default remains 5, which means one initial iteration to copy the entire domU memory, and up to 4 additional iterations to copy dirty memory from the still running domU. After the given number of iterations the domU is suspended, remaining dirty memory is copied and the domU is finally moved to the target host. This patch adjusts xl(1) and the libxl API. External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility of the new .max_iters property. Signed-off-by: Olaf Hering --- docs/man/xl.1.pod.in | 4 ++++ tools/include/libxl.h | 1 + tools/libs/light/libxl_dom_save.c | 2 +- tools/libs/light/libxl_domain.c | 1 + tools/libs/light/libxl_internal.h | 1 + tools/xl/xl_cmdtable.c | 3 ++- tools/xl/xl_migrate.c | 10 +++++++++- 7 files changed, 19 insertions(+), 3 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index 70a6ebf438..594387bcf4 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -494,6 +494,10 @@ such that it will be identical on the destination host, unless that configuration is overridden using the B<-C> option. Note that it is not possible to use this option for a 'localhost' migration. +=item B<--max_iters> I + +Number of copy iterations before final suspend+move (default: 5) + =back =item B [I] I I diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 9a4d7514ed..bf77da0524 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1714,6 +1714,7 @@ static inline int libxl_retrieve_domain_configuration_0x041200( typedef struct { uint32_t flags; /* LIBXL_SUSPEND_* */ + uint32_t max_iters; } libxl_domain_suspend_props; #define LIBXL_SUSPEND_DEBUG 1 #define LIBXL_SUSPEND_LIVE 2 diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 3f3cff0342..938c0127f3 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -383,7 +383,7 @@ static int libxl__domain_save_precopy_policy(precopy_stats_t stats, void *user) stats.iteration, stats.dirty_count, stats.total_written); if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT) goto stop_copy; - if (stats.iteration >= LIBXL_XGS_POLICY_MAX_ITERATIONS) + if (stats.iteration >= dss->max_iters) goto stop_copy; return XGS_POLICY_CONTINUE_PRECOPY; diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 5dbd27900f..9f98cd7f2b 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -527,6 +527,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, dss->domid = domid; dss->fd = fd; dss->type = type; + dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS; dss->live = props->flags & LIBXL_SUSPEND_LIVE; dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index 57d7e4b4b8..8cbcc5282c 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3649,6 +3649,7 @@ struct libxl__domain_save_state { int live; int debug; int checkpointed_stream; + uint32_t max_iters; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index ca1dfa3525..9b6b3c99aa 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -174,7 +174,8 @@ const struct cmd_spec cmd_table[] = { " of the domain.\n" "--debug Ignored.\n" "-p Do not unpause domain after migrating it.\n" - "-D Preserve the domain id" + "-D Preserve the domain id\n" + "--max_iters N Number of copy iterations before final stop+move" }, { "restore", &main_restore, 0, 1, diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index 144890924f..af117d4d56 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -178,6 +178,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child, static void migrate_domain(uint32_t domid, int preserve_domid, const char *rune, int debug, + uint32_t max_iters, const char *override_config_file) { pid_t child = -1; @@ -189,6 +190,7 @@ static void migrate_domain(uint32_t domid, int preserve_domid, int config_len; libxl_domain_suspend_props props = { .flags = LIBXL_SUSPEND_LIVE, + .max_iters = max_iters, }; save_domain_core_begin(domid, preserve_domid, override_config_file, @@ -542,8 +544,10 @@ int main_migrate(int argc, char **argv) char *host; int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0; int preserve_domid = 0; + uint32_t max_iters = 0; static struct option opts[] = { {"debug", 0, 0, 0x100}, + {"max_iters", 1, 0, 0x101}, {"live", 0, 0, 0x200}, COMMON_LONG_OPTS }; @@ -571,6 +575,9 @@ int main_migrate(int argc, char **argv) case 0x100: /* --debug */ debug = 1; break; + case 0x101: /* --max_iters */ + max_iters = atoi(optarg); + break; case 0x200: /* --live */ /* ignored for compatibility with xm */ break; @@ -605,7 +612,8 @@ int main_migrate(int argc, char **argv) pause_after_migration ? " -p" : ""); } - migrate_domain(domid, preserve_domid, rune, debug, config_filename); + migrate_domain(domid, preserve_domid, rune, debug, + max_iters, config_filename); return EXIT_SUCCESS; } From patchwork Wed Jun 16 12:51:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325169 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6561C48BE5 for ; Wed, 16 Jun 2021 13:07:26 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C59561166 for ; Wed, 16 Jun 2021 13:07:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C59561166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143007.263842 (Exim 4.92) (envelope-from ) id 1ltVGH-00055l-BH; Wed, 16 Jun 2021 13:07:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143007.263842; Wed, 16 Jun 2021 13:07:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGH-00055a-6c; Wed, 16 Jun 2021 13:07:17 +0000 Received: by outflank-mailman (input) for mailman id 143007; Wed, 16 Jun 2021 13:07:15 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2i-0006lZ-3Z for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:53:16 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.103]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 05e4d266-6a6e-47b8-9f53-83a45b230a53; Wed, 16 Jun 2021 12:51:56 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpmtmS (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:48 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 05e4d266-6a6e-47b8-9f53-83a45b230a53 ARC-Seal: i=1; a=rsa-sha256; t=1623847908; cv=none; d=strato.com; s=strato-dkim-0002; b=ZbwVa8IxdPe/2U3g4S2BClLR/Cr7UXEm8XD35g25MdfwgOrqBZpW96bhOttC6rb6gK 9HckDwzfn+BKqN+1vnCvN4CoobKCcGEphFzFv1VPU1zzS7Swmql2Fo1imdSWDaNZYvR0 PlbQZY8/mdiFAyso+hVGBIlamek6TRRRp5/DSrdmbEfL1z76EmCJRi+MZe2g+W09//zr asgdVc5f9AChs1wOqHyUT9qoVFJI2FdPKADui8UsEfHSed1c8uiSMvgcbvnhmi+6VRHy LRJVYucZbDiweZu5r2ZXEUlJQTroq0z9KpU/gAY0JOkbse7D6jOEqi7HW/Jxv28f4QST el/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847908; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=qkM58ppJmICGaCKigNV7+3gfD18mpe/eRIOurb3atVw=; b=Z5AwzOS5tlCd6dyHSwzl3gKPr3NJ5sEnwkYAZ0wVtfyAezpPtqoI1xzm/YQ0wLvuR9 ucTuuNxURAYY1+CIIiHwlB+8zwwKvFJryu08NSs85n3ZtWPuL4jhT0Sr8eU1rKOB36XZ JF+nwOnb5ZRa2/IjNJZHMnCH6U387/YI2rGi07RwjF7tOIlTGnpBVzn++SV4AgC9Hzok CPjPPrQ0DlqvMirP44BWXae5LHvspeFF0w0laPbW1R8QQLGaagla8hzyfDKTpPe629+G 3akUkjx1soX91T0+IJBH+cm4sptvmvvzLH0HyDcd9I5xtAI0KVLOedGOZWdKyKE2XhUj mPCQ== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847908; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=qkM58ppJmICGaCKigNV7+3gfD18mpe/eRIOurb3atVw=; b=UPJ6B0D3F5GZ72Gb2NPrMvTFuZ2VqtnzA+jWaSQOCpM56nRClCnuDI3h7Zv2Xh3nQL kFOqtIWDllAFCnoLQaF9i2pdSLrlaxYXN9PYDIpda4oYaYJtGQ34F/HIOlGitFKg3G1b aLfYt+cg6qnuOBBQscePbVXGMuto1Nh1jwMjsG8CGmfJG4AnTn9hmCbBBhT/LKSN78Or R+JnyDxYw741UAFhj8CxcM2bMPwV6wwawXxwESHSoyVmzEblwb6JDVYjvNCw/6GuAWK8 7EzxPRkHGHAo/t857KxO17FBg9Q2tP3UO/Dkl2D3aXETo9EQs1MtoYwuxVpXXwa6PTHP +adw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210616 32/36] tools: add --min_remaining to libxl_domain_suspend Date: Wed, 16 Jun 2021 14:51:25 +0200 Message-Id: <20210616125129.26563-33-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 The decision to stop+move a domU to the new host must be based on two factors: - the available network bandwidth for the migration stream - the maximum time a workload within a domU can be savely suspended Both values define how many dirty pages a workload may produce prior the final stop+move. The default value of 50 pages is much too low with todays network bandwidths. On an idle 1GiB link these 200K will be transferred within ~2ms. Give the admin a knob to adjust the point when the final stop+move will be done, so he can base this decision on his own needs. This patch adjusts xl(1) and the libxl API. External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility of the new .min_remaining property. Signed-off-by: Olaf Hering --- docs/man/xl.1.pod.in | 8 ++++++++ tools/include/libxl.h | 1 + tools/libs/light/libxl_dom_save.c | 2 +- tools/libs/light/libxl_domain.c | 1 + tools/libs/light/libxl_internal.h | 1 + tools/xl/xl_cmdtable.c | 23 ++++++++++++----------- tools/xl/xl_migrate.c | 9 ++++++++- 7 files changed, 32 insertions(+), 13 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index 594387bcf4..09e866ad87 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -498,6 +498,14 @@ possible to use this option for a 'localhost' migration. Number of copy iterations before final suspend+move (default: 5) +=item B<--min_remaing> I + +Number of remaining dirty pages. If the number of dirty pages drops that +low, the guest is suspended and the domU will finally be moved to I. + +This allows the host admin to control for how long the domU will likely +be suspended during transit. + =back =item B [I] I I diff --git a/tools/include/libxl.h b/tools/include/libxl.h index bf77da0524..28d70b1078 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1715,6 +1715,7 @@ static inline int libxl_retrieve_domain_configuration_0x041200( typedef struct { uint32_t flags; /* LIBXL_SUSPEND_* */ uint32_t max_iters; + uint32_t min_remaining; } libxl_domain_suspend_props; #define LIBXL_SUSPEND_DEBUG 1 #define LIBXL_SUSPEND_LIVE 2 diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 938c0127f3..ad5df89b2c 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -381,7 +381,7 @@ static int libxl__domain_save_precopy_policy(precopy_stats_t stats, void *user) LOGD(DEBUG, shs->domid, "iteration %u dirty_count %ld total_written %lu", stats.iteration, stats.dirty_count, stats.total_written); - if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT) + if (stats.dirty_count >= 0 && stats.dirty_count < dss->min_remaining) goto stop_copy; if (stats.iteration >= dss->max_iters) goto stop_copy; diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 9f98cd7f2b..06ca7a7df6 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -528,6 +528,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, dss->fd = fd; dss->type = type; dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS; + dss->min_remaining = props->min_remaining ?: LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT; dss->live = props->flags & LIBXL_SUSPEND_LIVE; dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index 8cbcc5282c..e4bfb34085 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3650,6 +3650,7 @@ struct libxl__domain_save_state { int debug; int checkpointed_stream; uint32_t max_iters; + uint32_t min_remaining; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 9b6b3c99aa..2cb4980c80 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -165,17 +165,18 @@ const struct cmd_spec cmd_table[] = { &main_migrate, 0, 1, "Migrate a domain to another host", "[options] ", - "-h Print this help.\n" - "-C Send instead of config file from creation.\n" - "-s Use instead of ssh. String will be passed\n" - " to sh. If empty, run instead of ssh xl\n" - " migrate-receive [-d -e]\n" - "-e Do not wait in the background (on ) for the death\n" - " of the domain.\n" - "--debug Ignored.\n" - "-p Do not unpause domain after migrating it.\n" - "-D Preserve the domain id\n" - "--max_iters N Number of copy iterations before final stop+move" + "-h Print this help.\n" + "-C Send instead of config file from creation.\n" + "-s Use instead of ssh. String will be passed\n" + " to sh. If empty, run instead of ssh xl\n" + " migrate-receive [-d -e]\n" + "-e Do not wait in the background (on ) for the death\n" + " of the domain.\n" + "--debug Ignored.\n" + "-p Do not unpause domain after migrating it.\n" + "-D Preserve the domain id\n" + "--max_iters N Number of copy iterations before final stop+move\n" + "--min_remaining N Number of remaining dirty pages before final stop+move" }, { "restore", &main_restore, 0, 1, diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index af117d4d56..14feb2b7ec 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -179,6 +179,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child, static void migrate_domain(uint32_t domid, int preserve_domid, const char *rune, int debug, uint32_t max_iters, + uint32_t min_remaining, const char *override_config_file) { pid_t child = -1; @@ -191,6 +192,7 @@ static void migrate_domain(uint32_t domid, int preserve_domid, libxl_domain_suspend_props props = { .flags = LIBXL_SUSPEND_LIVE, .max_iters = max_iters, + .min_remaining = min_remaining, }; save_domain_core_begin(domid, preserve_domid, override_config_file, @@ -545,9 +547,11 @@ int main_migrate(int argc, char **argv) int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0; int preserve_domid = 0; uint32_t max_iters = 0; + uint32_t min_remaining = 0; static struct option opts[] = { {"debug", 0, 0, 0x100}, {"max_iters", 1, 0, 0x101}, + {"min_remaining", 1, 0, 0x102}, {"live", 0, 0, 0x200}, COMMON_LONG_OPTS }; @@ -578,6 +582,9 @@ int main_migrate(int argc, char **argv) case 0x101: /* --max_iters */ max_iters = atoi(optarg); break; + case 0x102: /* --min_remaining */ + min_remaining = atoi(optarg); + break; case 0x200: /* --live */ /* ignored for compatibility with xm */ break; @@ -613,7 +620,7 @@ int main_migrate(int argc, char **argv) } migrate_domain(domid, preserve_domid, rune, debug, - max_iters, config_filename); + max_iters, min_remaining, config_filename); return EXIT_SUCCESS; } From patchwork Wed Jun 16 12:51:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C85A8C48BE6 for ; Wed, 16 Jun 2021 13:07:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 81D086024A for ; Wed, 16 Jun 2021 13:07:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81D086024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143005.263825 (Exim 4.92) (envelope-from ) id 1ltVGD-0004OP-0S; Wed, 16 Jun 2021 13:07:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143005.263825; Wed, 16 Jun 2021 13:07:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGC-0004Ne-QM; Wed, 16 Jun 2021 13:07:12 +0000 Received: by outflank-mailman (input) for mailman id 143005; Wed, 16 Jun 2021 13:07:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2c-00075D-AK for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:53:10 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.101]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 43676f1c-25e8-4adf-bb2b-2befd94fe2ff; Wed, 16 Jun 2021 12:51:57 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpntmT (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:49 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 43676f1c-25e8-4adf-bb2b-2befd94fe2ff ARC-Seal: i=1; a=rsa-sha256; t=1623847909; cv=none; d=strato.com; s=strato-dkim-0002; b=FzMJyk/OgO8egTJoevrQLeC8WIWy98hhXt9CyBmgw5d6lttT9+go9w/CwXmMyiisx7 RFA12X8FSlTS+yQFTxgKeWk5AU3KNhvUxkUGj96tY8z1KfiwA4FwSXo09le0hFOTBMaZ W798aP3BgQ8j8WrYy7L0FHnL38vTCY55xzUsovUqhYY/xml+0jGb5fb9Y/QTvN5iXmS+ 5r92VmV+X7nedj474psHGPmaaTddCCrV02Oa4rFs+gfRBcsBGX2u7a6M+/YdfEqFgfyY E8t6WS8RpMW1QFWWs4Qp6LFaWptcQm0bAhl34m+em9Jph2wtu018XzDGBn1KHhTnkqzF hUCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847909; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+vXVfBW28MRAf3KK7IPzTuDzNHY/m8g7e2aJIg9hVrs=; b=r7yfjeR8GYJC4Y94AfVV0JXweeAYZJl3+bjgVUlMUyUTk0R7TSPdYh66Ezw3Q11bsX 5KzKtPiz4DYuQjLDWSlz7P0+YZw6LdQTejzpEsa/bSvlLztTPnRchvH6c2SFE+i/HKfR /IsMbm1VQ88am8IBIHJM34DvCUjq1sp8aqE9lgHXTfgJQV15OODXlnyBk3lEFGeKIQs6 18vzH2cgYZ0mUV90Zt7Qm9S4KvqJYis30/5yIQ4vJn3UXu+9Jq/MHD3OuzeuyQwdCiAb 2pH+JnxXlY59zw2CZJChz+5A3SUyLN3MVlAEiqmnopfqyIeqkW4VBJ7ppGQcYMtHixKJ PGFw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847909; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+vXVfBW28MRAf3KK7IPzTuDzNHY/m8g7e2aJIg9hVrs=; b=bWXogRscBUHUFRe1lcJZLupNeRT7jS5RjUDObCHmXcy04k4cG/LIEthkDCF/tQdBOq YwRxLR5WT2/XRBhL/2/giudwdX/+xs+ZhczS5ca72DnUmhtmQ7lVRvV9113+ELVKM8iE O0Pp70opATjLgfjmaq/am/5RQs7dCOqBkrRfKUaYyspYa0CxT+CBr7XgaeDzs5bcRJsK elBWQv/xM1sBqP3V44JvTwpDyjXtp9+oU0EpYkD3Nr+C2L8w2YrD8qI1iwXOE0HehpQs BO8WUbpCO2qdKoi6TJxRwr5CkLvzSvap98KXYBDnbcvT3xl0qSAN5Dpu+wKcY+/Y1P/K 8mfw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210616 33/36] tools: add --abort_if_busy to libxl_domain_suspend Date: Wed, 16 Jun 2021 14:51:26 +0200 Message-Id: <20210616125129.26563-34-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Provide a knob to the host admin to abort the live migration of a running domU if the downtime during final transit will be too long for the workload within domU. Adjust error reporting. Add ERROR_MIGRATION_ABORTED to allow callers of libxl_domain_suspend to distinguish between errors and the requested constraint. Adjust precopy_policy to simplify reporting of remaining dirty pages. The loop in send_memory_live populates ->dirty_count in a different place than ->iteration. Let it proceeed one more time to provide the desired information before leaving the loop. This patch adjusts xl(1) and the libxl API. External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility of the new .abort_if_busy property. Signed-off-by: Olaf Hering --- docs/man/xl.1.pod.in | 8 +++++++ tools/include/libxl.h | 1 + tools/libs/light/libxl_dom_save.c | 7 ++++++- tools/libs/light/libxl_domain.c | 1 + tools/libs/light/libxl_internal.h | 2 ++ tools/libs/light/libxl_stream_write.c | 9 +++++++- tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_cmdtable.c | 6 +++++- tools/xl/xl_migrate.c | 30 ++++++++++++++++++++------- 9 files changed, 55 insertions(+), 10 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index 09e866ad87..37267c9171 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -506,6 +506,14 @@ low, the guest is suspended and the domU will finally be moved to I. This allows the host admin to control for how long the domU will likely be suspended during transit. +=item B<--abort_if_busy> + +Abort migration instead of doing final suspend/move/resume if the +guest produced more than I dirty pages during th number +of I iterations. +This avoids long periods of time where the guest is suspended, which +may confuse the workload within domU. + =back =item B [I] I I diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 28d70b1078..cc056ed627 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1719,6 +1719,7 @@ typedef struct { } libxl_domain_suspend_props; #define LIBXL_SUSPEND_DEBUG 1 #define LIBXL_SUSPEND_LIVE 2 +#define LIBXL_SUSPEND_ABORT_IF_BUSY 4 int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, libxl_domain_suspend_props *props, diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index ad5df89b2c..1999a8997f 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -383,11 +383,16 @@ static int libxl__domain_save_precopy_policy(precopy_stats_t stats, void *user) stats.iteration, stats.dirty_count, stats.total_written); if (stats.dirty_count >= 0 && stats.dirty_count < dss->min_remaining) goto stop_copy; - if (stats.iteration >= dss->max_iters) + if (stats.dirty_count >= 0 && stats.iteration >= dss->max_iters) goto stop_copy; return XGS_POLICY_CONTINUE_PRECOPY; stop_copy: + if (dss->abort_if_busy) + { + dss->remaining_dirty_pages = stats.dirty_count; + return XGS_POLICY_ABORT; + } return XGS_POLICY_STOP_AND_COPY; } diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 06ca7a7df6..e4740b063e 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -529,6 +529,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, dss->type = type; dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS; dss->min_remaining = props->min_remaining ?: LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT; + dss->abort_if_busy = props->flags & LIBXL_SUSPEND_ABORT_IF_BUSY; dss->live = props->flags & LIBXL_SUSPEND_LIVE; dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index e4bfb34085..905d5179ba 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3648,9 +3648,11 @@ struct libxl__domain_save_state { libxl_domain_type type; int live; int debug; + int abort_if_busy; int checkpointed_stream; uint32_t max_iters; uint32_t min_remaining; + long remaining_dirty_pages; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/libs/light/libxl_stream_write.c b/tools/libs/light/libxl_stream_write.c index 634f3240d1..1ab3943f3e 100644 --- a/tools/libs/light/libxl_stream_write.c +++ b/tools/libs/light/libxl_stream_write.c @@ -344,11 +344,18 @@ void libxl__xc_domain_save_done(libxl__egc *egc, void *dss_void, goto err; if (retval) { + if (dss->remaining_dirty_pages) { + LOGD(NOTICE, dss->domid, "saving domain: aborted," + " %ld remaining dirty pages.", dss->remaining_dirty_pages); + } else { LOGEVD(ERROR, errnoval, dss->domid, "saving domain: %s", dss->dsps.guest_responded ? "domain responded to suspend request" : "domain did not respond to suspend request"); - if (!dss->dsps.guest_responded) + } + if (dss->remaining_dirty_pages) + rc = ERROR_MIGRATION_ABORTED; + else if(!dss->dsps.guest_responded) rc = ERROR_GUEST_TIMEDOUT; else if (dss->rc) rc = dss->rc; diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index f45adddab0..b91769ee10 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -76,6 +76,7 @@ libxl_error = Enumeration("error", [ (-30, "QMP_DEVICE_NOT_ACTIVE"), # a device has failed to be become active (-31, "QMP_DEVICE_NOT_FOUND"), # the requested device has not been found (-32, "QEMU_API"), # QEMU's replies don't contains expected members + (-33, "MIGRATION_ABORTED"), ], value_namespace = "") libxl_domain_type = Enumeration("domain_type", [ diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 2cb4980c80..322a47c2bc 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -176,7 +176,11 @@ const struct cmd_spec cmd_table[] = { "-p Do not unpause domain after migrating it.\n" "-D Preserve the domain id\n" "--max_iters N Number of copy iterations before final stop+move\n" - "--min_remaining N Number of remaining dirty pages before final stop+move" + "--min_remaining N Number of remaining dirty pages before final stop+move\n" + "--abort_if_busy Abort migration instead of doing final stop+move,\n" + " if the number of dirty pages is higher than \n" + " after iterations. Otherwise the amount of memory\n" + " to be transfered would exceed maximum allowed domU downtime." }, { "restore", &main_restore, 0, 1, diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index 14feb2b7ec..f523746e5b 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -177,7 +177,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child, } static void migrate_domain(uint32_t domid, int preserve_domid, - const char *rune, int debug, + const char *rune, int debug, int abort_if_busy, uint32_t max_iters, uint32_t min_remaining, const char *override_config_file) @@ -213,14 +213,20 @@ static void migrate_domain(uint32_t domid, int preserve_domid, if (debug) props.flags |= LIBXL_SUSPEND_DEBUG; + if (abort_if_busy) + props.flags |= LIBXL_SUSPEND_ABORT_IF_BUSY; rc = libxl_domain_suspend(ctx, domid, send_fd, &props, NULL); if (rc) { fprintf(stderr, "migration sender: libxl_domain_suspend failed" " (rc=%d)\n", rc); - if (rc == ERROR_GUEST_TIMEDOUT) - goto failed_suspend; - else - goto failed_resume; + switch (rc) { + case ERROR_GUEST_TIMEDOUT: + goto failed_suspend; + case ERROR_MIGRATION_ABORTED: + goto failed_busy; + default: + goto failed_resume; + } } //fprintf(stderr, "migration sender: Transfer complete.\n"); @@ -302,6 +308,12 @@ static void migrate_domain(uint32_t domid, int preserve_domid, fprintf(stderr, "Migration failed, failed to suspend at sender.\n"); exit(EXIT_FAILURE); + failed_busy: + close(send_fd); + migration_child_report(recv_fd); + fprintf(stderr, "Migration aborted as requested, domain is too busy.\n"); + exit(EXIT_FAILURE); + failed_resume: close(send_fd); migration_child_report(recv_fd); @@ -545,13 +557,14 @@ int main_migrate(int argc, char **argv) char *rune = NULL; char *host; int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0; - int preserve_domid = 0; + int preserve_domid = 0, abort_if_busy = 0; uint32_t max_iters = 0; uint32_t min_remaining = 0; static struct option opts[] = { {"debug", 0, 0, 0x100}, {"max_iters", 1, 0, 0x101}, {"min_remaining", 1, 0, 0x102}, + {"abort_if_busy", 0, 0, 0x103}, {"live", 0, 0, 0x200}, COMMON_LONG_OPTS }; @@ -585,6 +598,9 @@ int main_migrate(int argc, char **argv) case 0x102: /* --min_remaining */ min_remaining = atoi(optarg); break; + case 0x103: /* --abort_if_busy */ + abort_if_busy = 1; + break; case 0x200: /* --live */ /* ignored for compatibility with xm */ break; @@ -619,7 +635,7 @@ int main_migrate(int argc, char **argv) pause_after_migration ? " -p" : ""); } - migrate_domain(domid, preserve_domid, rune, debug, + migrate_domain(domid, preserve_domid, rune, debug, abort_if_busy, max_iters, min_remaining, config_filename); return EXIT_SUCCESS; } From patchwork Wed Jun 16 12:51:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D49F9C48BE5 for ; Wed, 16 Jun 2021 13:08:13 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8976E61166 for ; Wed, 16 Jun 2021 13:08:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8976E61166 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143066.263908 (Exim 4.92) (envelope-from ) id 1ltVH4-0000X8-In; Wed, 16 Jun 2021 13:08:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143066.263908; Wed, 16 Jun 2021 13:08:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVH4-0000WP-FD; Wed, 16 Jun 2021 13:08:06 +0000 Received: by outflank-mailman (input) for mailman id 143066; Wed, 16 Jun 2021 13:08:04 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV28-00075D-99 for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:40 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.173]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1e7ffb82-5835-4a90-9d45-0c35951b5587; Wed, 16 Jun 2021 12:51:55 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpntmU (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:49 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1e7ffb82-5835-4a90-9d45-0c35951b5587 ARC-Seal: i=1; a=rsa-sha256; t=1623847909; cv=none; d=strato.com; s=strato-dkim-0002; b=q8VD0jm7Rew2fXNFlWlOaluFjoDypQ44Ob95CamwqiR+wFbK3aoPdeWBNP7Adyjzne yy2SZMgDjWgqkigG4/sdsBn/oGb12CzvBW+3Qb0Qq9ufdvKNKw85v41ikb8g8rnt+o4b M7EwndFm5bcRBLyW1EfDdkrEG2V/u9XrbEr5HBD6XwVfHcbSg/YVxGY9LtaDuKZ6O/TA KEID/JqBJVq057/DMijJlIsy7b3KZkozr8FcxRKmXP3zeQrVEjeG0En//q9cHI5E5CPB eYk4XJBzRjoeGv7kqf9hM4o7yL1r5JH8a1T7JkB6YawWDINI3pxdH3bq8lO55OF1kEzR hkyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847909; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=zRM6/H2oQQ+NxJB9cpmsAqIVMl48wKfvMp6dg0xJ0jg=; b=ViIRyMbhJ2GR4h7fsY7ePjIySyiPiqm73DE6zoKgQkE/O4VDkNLugmMiD+Vvyp5V9x WCwZsVjXfvjSN1hMmKBDFwhBbcvB95Lu/jf6th14rlrDL4fREJ5WiYCkAAo2I/Pd2wqM 3cGrx86CSgW5OZFexKeCXM94BHF6G9zb5yt65YNgP0ZJmWrIk1mZdbVuErqcqSgdbxkx rVrSiaa3WGEk/3GRm5X9tMfiIhHVniPWoopomhrpiS7aiFhW/3Cr7E7UQHNhHqLKfjRW ELtrQk/gVKSKbVzv8BUh6X9U3EX1UcAXDbom7rjaPKeVuHNKwwcFF3fQm3BPMfTjesxB NHHw== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847909; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=zRM6/H2oQQ+NxJB9cpmsAqIVMl48wKfvMp6dg0xJ0jg=; b=FyYXt8+mf0DOlOUBmaijcaYECM/KBmEV/qHq+vM9apZ2vEz/9dbrx5zMW4NwsK32Av IbZrxGT2tm0DciNpmZ76RpPlvNL4jTF6zDJUzZfQgIoWNZ3hpV4v56u55k2D9xha6Jfj 6dHb0xTMZ3UgMbgv8YxQYDpzpNI6igcnmJu8WKFD7iVTs1ODe8rxrleursPbIpxkwXRc 6fXS1k1oDbl5hb7Rn6aNOR/97+JyPWaSAQbxO+jWIiJcPsU5XIXjTIhTGmTbMkcu7G34 9KivIKBrVt+sFYa48GAgS1ssJZ54TE+TluJHI2w5kvnCoHUZdnJ99l5bPARgxZZBUaar TLoA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 34/36] tools: add API for expandable bitmaps Date: Wed, 16 Jun 2021 14:51:27 +0200 Message-Id: <20210616125129.26563-35-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Since the incoming migration stream lacks info about what the highest pfn will be, data structures can not be allocated upfront. Add an API for expandable bitmaps, loosely based on pfn_set_populated. Signed-off-by: Olaf Hering v02: - remove xg_ prefix from functions --- tools/libs/saverestore/common.c | 40 ++++++++++++++++++++ tools/libs/saverestore/common.h | 67 +++++++++++++++++++++++++++++++++ 2 files changed, 107 insertions(+) diff --git a/tools/libs/saverestore/common.c b/tools/libs/saverestore/common.c index 7da7fa4e2c..8b4e402df5 100644 --- a/tools/libs/saverestore/common.c +++ b/tools/libs/saverestore/common.c @@ -163,6 +163,46 @@ static void __attribute__((unused)) build_assertions(void) BUILD_BUG_ON(sizeof(struct xc_sr_rec_hvm_params) != 8); } +/* + * Expand the tracking structures as needed. + * To avoid realloc()ing too excessively, the size increased to the nearest + * power of two large enough to contain the required number of bits. + */ +bool _sr_bitmap_expand(struct sr_bitmap *bm, unsigned long bits) +{ + size_t new_max; + size_t old_sz, new_sz; + void *p; + + if (bits <= bm->bits) + return true; + + /* Round up to the nearest power of two larger than bit, less 1. */ + new_max = bits; + new_max |= new_max >> 1; + new_max |= new_max >> 2; + new_max |= new_max >> 4; + new_max |= new_max >> 8; + new_max |= new_max >> 16; + if ( sizeof(unsigned long) > 4 ) + new_max |= new_max >> 32; + + /* Allocate units of unsigned long */ + new_max = (new_max + BITS_PER_LONG - 1) & ~(BITS_PER_LONG - 1); + + old_sz = bitmap_size(bm->bits); + new_sz = bitmap_size(new_max); + p = realloc(bm->p, new_sz); + if (!p) + return false; + + memset(p + old_sz, 0, new_sz - old_sz); + bm->p = p; + bm->bits = new_max; + + return true; +} + /* * Local variables: * mode: C diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 60bbba6aa9..43aa1a7b86 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -30,6 +30,73 @@ const char *rec_type_to_str(uint32_t type); struct xc_sr_context; struct xc_sr_record; +struct sr_bitmap +{ + void *p; + unsigned long bits; +}; + +extern bool _sr_bitmap_expand(struct sr_bitmap *bm, unsigned long bits); + +static inline bool sr_bitmap_expand(struct sr_bitmap *bm, unsigned long bits) +{ + if (bits > bm->bits) + return _sr_bitmap_expand(bm, bits); + return true; +} + +static inline void sr_bitmap_free(struct sr_bitmap *bm) +{ + free(bm->p); + bm->p = NULL; +} + +static inline bool sr_set_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (sr_bitmap_expand(bm, bit) == false) + return false; + + set_bit(bit, bm->p); + return true; +} + +static inline bool sr_test_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_bit(bit, bm->p); +} + +static inline void sr_clear_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit <= bm->bits) + clear_bit(bit, bm->p); +} + +static inline bool sr_test_and_clear_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_and_clear_bit(bit, bm->p); +} + +/* No way to report potential allocation error, bitmap must be expanded prior usage */ +static inline bool sr_test_and_set_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_and_set_bit(bit, bm->p); +} + +static inline bool sr_set_long_bit(unsigned long base_bit, struct sr_bitmap *bm) +{ + if (sr_bitmap_expand(bm, base_bit + BITS_PER_LONG) == false) + return false; + + set_bit_long(base_bit, bm->p); + return true; +} + /** * Save operations. To be implemented for each type of guest, for use by the * common save algorithm. From patchwork Wed Jun 16 12:51:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A91B0C48BE6 for ; Wed, 16 Jun 2021 13:08:13 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 590BE6024A for ; Wed, 16 Jun 2021 13:08:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 590BE6024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143063.263897 (Exim 4.92) (envelope-from ) id 1ltVH3-0000Fz-9s; Wed, 16 Jun 2021 13:08:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143063.263897; Wed, 16 Jun 2021 13:08:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVH3-0000Fp-6J; Wed, 16 Jun 2021 13:08:05 +0000 Received: by outflank-mailman (input) for mailman id 143063; Wed, 16 Jun 2021 13:08:03 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2I-00075D-9Y for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:52:50 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.102]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e6b65d89-6720-4ffd-a184-2f781514df3e; Wed, 16 Jun 2021 12:51:55 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpotmV (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:50 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e6b65d89-6720-4ffd-a184-2f781514df3e ARC-Seal: i=1; a=rsa-sha256; t=1623847910; cv=none; d=strato.com; s=strato-dkim-0002; b=FBHkTgZXolibcwjpysZzh8M2qH2k+aSKsoY4a7M4Oow0Xs9T0Wm2Haj6mmlIlJy0Kr dDh16bRAqyGU1vzTj7yR6KTbR1apuQAYKJAVc/lS2CqovzKJUO5BGO74TihGU2JH/SAn 0Notaabw9CAj4nbXVpSUixRlvcFdE8CvTynRchOxS6LW1Q9v0F5HPs8HFnTzKPx++GAX 5i/Y0SdT12lbo6Gg4LKX5i5Np04j26SPb3jqYZNvTjTgND0qTYi6j2q2l7khuhOd67HJ 6gi4E8PzZlZLm4wk17b9iQgZgXW6cTr6g6XkshHIE6acYVHMc2b/QLOHHleBNvki+dld JRxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847910; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=b3yBWw7yU2epDTrjSW6D9+v+xfIfJnT+XxyS94GOcl8=; b=H2SI/FxhNxisGvyhqeDCfOM2s2MYMvmWSqLHovX+WYbylzW+XxiEfLT5On1kI+bD3v TL7TTzr+XWcg5KdutNQZAh4R6/x2oWT1XKsYfwYq3FjRNkudN4Tx7/vVvJK2fp/SNQdd mrcIoUA1YZJzecTc/O+4fiqxMOG6oLHzjsOCIrU03HG21nYXYMzumBZfpNhQjG90FcfT Nc+lsDGudDPX0EPkbgpMfHFyFlgY2ay17tAfs21gWR/JKyIZ50yqet+E3vNAOoP3CbKe jPXV9MVdAmPgHOiItbMB+3beqaGSdAQ6UnDJHRlwivJrEMH5zDVgfzJXA0Xca/p8QwIt dj0A== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847910; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=b3yBWw7yU2epDTrjSW6D9+v+xfIfJnT+XxyS94GOcl8=; b=Ln01R/NUGT1zY8Rr1c8rijll3Cl+j/E0G1PEoVLjpe3cDyQ1pIxIZ3rY/1lsOfI7i8 PUWanNj6lNj52oXIq53XVA4cAJgTtZXVrT4O8KnaF+EBi9oS0K1wyalqOqiKX1uoCuTG 98Xi5sWV8EEWby7BhXjt8LW8pHY75yUmQ3QSVUX7OqHJ5Ha3CDWL/Jb9n/gZ7+SGhMK+ RmBxTjuKgmX0IixB+cuLPQVCnPvb2ILoLMzcq0Iy9sT93gm/iJICw9DOSw1wksBsGEiV LAqolLLUwXC+hERsAt5Q7A38e1Mhh6E4cyFUQNze3RASCjO1km6MXqR7Gph9nHcaUCYw 9Q9Q== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 35/36] tools: use sr_bitmap for populated_pfns Date: Wed, 16 Jun 2021 14:51:28 +0200 Message-Id: <20210616125129.26563-36-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 Signed-off-by: Olaf Hering v02: - remove xg_ prefix from called functions --- tools/libs/saverestore/common.h | 21 +++++++- tools/libs/saverestore/restore.c | 69 ------------------------ tools/libs/saverestore/restore_x86_hvm.c | 9 ++++ tools/libs/saverestore/restore_x86_pv.c | 7 +++ 4 files changed, 35 insertions(+), 71 deletions(-) diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 43aa1a7b86..43a31f9aa5 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -403,8 +403,7 @@ struct xc_sr_context uint32_t xenstore_domid, console_domid; /* Bitmap of currently populated PFNs during restore. */ - unsigned long *populated_pfns; - xen_pfn_t max_populated_pfn; + struct sr_bitmap populated_pfns; /* Sender has invoked verify mode on the stream. */ bool verify; @@ -629,6 +628,24 @@ static inline bool page_type_has_stream_data(uint32_t type) } return ret; } + +static inline bool pfn_is_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + return sr_test_bit(pfn, &ctx->restore.populated_pfns); +} + +static inline int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xc_interface *xch = ctx->xch; + + if ( sr_set_bit(pfn, &ctx->restore.populated_pfns) == false ) + { + PERROR("Failed to realloc populated_pfns bitmap"); + errno = ENOMEM; + return -1; + } + return 0; +} #endif /* * Local variables: diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index d0148606bf..8f7bce2585 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -71,64 +71,6 @@ static int read_headers(struct xc_sr_context *ctx) return 0; } -/* - * Is a pfn populated? - */ -static bool pfn_is_populated(const struct xc_sr_context *ctx, xen_pfn_t pfn) -{ - if ( pfn > ctx->restore.max_populated_pfn ) - return false; - return test_bit(pfn, ctx->restore.populated_pfns); -} - -/* - * Set a pfn as populated, expanding the tracking structures if needed. To - * avoid realloc()ing too excessively, the size increased to the nearest power - * of two large enough to contain the required pfn. - */ -static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) -{ - xc_interface *xch = ctx->xch; - - if ( pfn > ctx->restore.max_populated_pfn ) - { - xen_pfn_t new_max; - size_t old_sz, new_sz; - unsigned long *p; - - /* Round up to the nearest power of two larger than pfn, less 1. */ - new_max = pfn; - new_max |= new_max >> 1; - new_max |= new_max >> 2; - new_max |= new_max >> 4; - new_max |= new_max >> 8; - new_max |= new_max >> 16; -#ifdef __x86_64__ - new_max |= new_max >> 32; -#endif - - old_sz = bitmap_size(ctx->restore.max_populated_pfn + 1); - new_sz = bitmap_size(new_max + 1); - p = realloc(ctx->restore.populated_pfns, new_sz); - if ( !p ) - { - ERROR("Failed to realloc populated bitmap"); - errno = ENOMEM; - return -1; - } - - memset((uint8_t *)p + old_sz, 0x00, new_sz - old_sz); - - ctx->restore.populated_pfns = p; - ctx->restore.max_populated_pfn = new_max; - } - - assert(!test_bit(pfn, ctx->restore.populated_pfns)); - set_bit(pfn, ctx->restore.populated_pfns); - - return 0; -} - /* * Given a set of pfns, obtain memory from Xen to fill the physmap for the * unpopulated subset. If types is NULL, no page type checking is performed @@ -929,16 +871,6 @@ static int setup(struct xc_sr_context *ctx) if ( rc ) goto err; - ctx->restore.max_populated_pfn = (32 * 1024 / 4) - 1; - ctx->restore.populated_pfns = bitmap_alloc( - ctx->restore.max_populated_pfn + 1); - if ( !ctx->restore.populated_pfns ) - { - ERROR("Unable to allocate memory for populated_pfns bitmap"); - rc = -1; - goto err; - } - ctx->restore.buffered_records = malloc( DEFAULT_BUF_RECORDS * sizeof(struct xc_sr_record)); if ( !ctx->restore.buffered_records ) @@ -977,7 +909,6 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.m); free(ctx->restore.buffered_records); - free(ctx->restore.populated_pfns); if ( ctx->restore.ops.cleanup(ctx) ) PERROR("Failed to clean up"); diff --git a/tools/libs/saverestore/restore_x86_hvm.c b/tools/libs/saverestore/restore_x86_hvm.c index bd63bd2818..97e7e0f48c 100644 --- a/tools/libs/saverestore/restore_x86_hvm.c +++ b/tools/libs/saverestore/restore_x86_hvm.c @@ -136,6 +136,7 @@ static int x86_hvm_localise_page(struct xc_sr_context *ctx, static int x86_hvm_setup(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; + unsigned long max_pfn; if ( ctx->restore.guest_type != DHDR_TYPE_X86_HVM ) { @@ -161,6 +162,13 @@ static int x86_hvm_setup(struct xc_sr_context *ctx) } #endif + max_pfn = max(ctx->restore.p2m_size, ctx->dominfo.max_memkb >> (PAGE_SHIFT-10)); + if ( !sr_bitmap_expand(&ctx->restore.populated_pfns, max_pfn) ) + { + PERROR("Unable to allocate memory for populated_pfns bitmap"); + return -1; + } + return 0; } @@ -241,6 +249,7 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx) static int x86_hvm_cleanup(struct xc_sr_context *ctx) { + sr_bitmap_free(&ctx->restore.populated_pfns); free(ctx->x86.hvm.restore.context.ptr); free(ctx->x86.restore.cpuid.ptr); diff --git a/tools/libs/saverestore/restore_x86_pv.c b/tools/libs/saverestore/restore_x86_pv.c index 96608e5231..c73a3cd99f 100644 --- a/tools/libs/saverestore/restore_x86_pv.c +++ b/tools/libs/saverestore/restore_x86_pv.c @@ -1060,6 +1060,12 @@ static int x86_pv_setup(struct xc_sr_context *ctx) if ( rc ) return rc; + if ( !sr_bitmap_expand(&ctx->restore.populated_pfns, 32 * 1024 / 4) ) + { + PERROR("Unable to allocate memory for populated_pfns bitmap"); + return -1; + } + ctx->x86.pv.restore.nr_vcpus = ctx->dominfo.max_vcpu_id + 1; ctx->x86.pv.restore.vcpus = calloc(sizeof(struct xc_sr_x86_pv_restore_vcpu), ctx->x86.pv.restore.nr_vcpus); @@ -1153,6 +1159,7 @@ static int x86_pv_stream_complete(struct xc_sr_context *ctx) */ static int x86_pv_cleanup(struct xc_sr_context *ctx) { + sr_bitmap_free(&ctx->restore.populated_pfns); free(ctx->x86.pv.p2m); free(ctx->x86.pv.p2m_pfns); From patchwork Wed Jun 16 12:51:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12325175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0209BC48BE6 for ; Wed, 16 Jun 2021 13:07:43 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F1A26024A for ; Wed, 16 Jun 2021 13:07:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F1A26024A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.143025.263863 (Exim 4.92) (envelope-from ) id 1ltVGV-0006JO-5H; Wed, 16 Jun 2021 13:07:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 143025.263863; Wed, 16 Jun 2021 13:07:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltVGV-0006IV-1B; Wed, 16 Jun 2021 13:07:31 +0000 Received: by outflank-mailman (input) for mailman id 143025; Wed, 16 Jun 2021 13:07:29 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ltV2h-00075D-AQ for xen-devel@lists.xenproject.org; Wed, 16 Jun 2021 12:53:15 +0000 Received: from mo4-p04-ob.smtp.rzone.de (unknown [81.169.146.178]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 91e5a4a4-21aa-4503-93d0-c098ebfb81b5; Wed, 16 Jun 2021 12:51:59 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.27.2 AUTH) with ESMTPSA id j0415bx5GCpotmW (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Wed, 16 Jun 2021 14:51:50 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 91e5a4a4-21aa-4503-93d0-c098ebfb81b5 ARC-Seal: i=1; a=rsa-sha256; t=1623847911; cv=none; d=strato.com; s=strato-dkim-0002; b=VCr/2PZxuem7ziOl0SvYUxtu2I1JCHZc1ppdDwAIg22ZqdZnYeVyQLbrGR0oQ6uu4H stB2H3kgGsdz5JS00Ia/o1JYHyfQo7LyVBUO8BU3J/6GcYF4A/uM2Xp+A/1Hf68wMiwc Pma3fNeP6mim28jfdA/zwPu2J81jpvN1bPpfhG9lARFQ4sKiqUilwJMMrgN3BjD4zKfc yxgf8qsVwe20OSYhSIiubB3TrzsFhPxdGy9e5SUVUAqWoN4anUTXgrh8jRokFtA3bMeT AcUX3RspEySfbtjqF5b8ryOKpQA3bbIANFgT36sfcThNPduxtY6nTRgVBJeHct1+DWEQ VSJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847911; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=v7F+g3JSzDmZZTOZdDO+rS/XRfxqZ/ug2X6EnF8j6XQ=; b=QPf9+ju5uY2SxuUQDlHbY+HL/uWI2y94DzIjAGALOOCmoL7sl5Z7KI3lQD6qqbPiCl 5prcoABg+YJs0xivxuG3uB+P5sxHKTr/DJHwa4s5nyzExpHyHD3jxmyb7iPMYGOCgfDx o3BSytem3D7YCs7EGTgkzuUSXwZvGaLMVmW63r/kwu6vQfrlTt+Hu53Dui8ctMx1zUpl mtv0XAFXgmlD2WYdrWYgwfHqUz1ADGgxU237krCTBvan75mW19OvmiMPZhI17OTmMHcR ZvxnVYICI8U2ro+VaMKSN2212EWxd7dy+aYMQruFfnQqwUeby9Z55DWqknjtCD/iMXn5 CR1w== ARC-Authentication-Results: i=1; strato.com; dkim=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1623847910; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=v7F+g3JSzDmZZTOZdDO+rS/XRfxqZ/ug2X6EnF8j6XQ=; b=UGgWGQEPOlSwB3aEFnGTthuNM4J51eERuVa6sFgn5Rn/7eC9HsaPBfZ2O2JGn3MXDH KDjgevsBGd7Gq+BdKhZH3qEH00w8PRhoyS8ey6Ndy81zp5kChWkiGzy8iKJnQHiiQFOu 5/evSxKQdDu2jCwVhDzPqEfwBwnZZQzjayjXDR8UT+BCOcw6XZ0qt5lR7OnguZg3somx pdsU66CKlC442tSTeJj9LU1xjz2mOfZNWdsOfo6KSegrLdf13gf6pLVhFTBs2aS3oD5f AqYtRIpjVDVL4RZmCky3uYK4dA/wae8zNl81xo+0hoQTIXzhbUhX2GBAkjGAkDWWmk5P lamQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAg5l+Vz7FJgt8+TgOd9sTrMwXjWWExsBKQCrpnqhqg==" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210616 36/36] tools: use superpages during restore of HVM guest Date: Wed, 16 Jun 2021 14:51:29 +0200 Message-Id: <20210616125129.26563-37-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210616125129.26563-1-olaf@aepfle.de> References: <20210616125129.26563-1-olaf@aepfle.de> MIME-Version: 1.0 During creating of a HVM domU meminit_hvm() tries to map superpages. After save/restore or migration this mapping is lost, everything is allocated in single pages. This causes a performance degradation after migration. Add neccessary code to preallocate a superpage for an incoming chunk of pfns. In case a pfn was not populated on the sending side, it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering v02: - remove xg_ prefix from called functions --- tools/libs/guest/xg_dom_x86.c | 5 - tools/libs/guest/xg_private.h | 5 + tools/libs/saverestore/common.c | 1 - tools/libs/saverestore/common.h | 28 +- tools/libs/saverestore/restore.c | 62 +--- tools/libs/saverestore/restore_x86_hvm.c | 370 ++++++++++++++++++++++- tools/libs/saverestore/restore_x86_pv.c | 61 +++- 7 files changed, 455 insertions(+), 77 deletions(-) diff --git a/tools/libs/guest/xg_dom_x86.c b/tools/libs/guest/xg_dom_x86.c index d2eb89ce01..ec0d18fd60 100644 --- a/tools/libs/guest/xg_dom_x86.c +++ b/tools/libs/guest/xg_dom_x86.c @@ -44,11 +44,6 @@ #define SUPERPAGE_BATCH_SIZE 512 -#define SUPERPAGE_2MB_SHIFT 9 -#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT) -#define SUPERPAGE_1GB_SHIFT 18 -#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT) - #define X86_CR0_PE 0x01 #define X86_CR0_ET 0x10 diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h index 28441ee13f..b7372e6bd5 100644 --- a/tools/libs/guest/xg_private.h +++ b/tools/libs/guest/xg_private.h @@ -179,4 +179,9 @@ struct xc_cpu_policy { }; #endif /* x86 */ +#define SUPERPAGE_2MB_SHIFT 9 +#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT) +#define SUPERPAGE_1GB_SHIFT 18 +#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT) + #endif /* XG_PRIVATE_H */ diff --git a/tools/libs/saverestore/common.c b/tools/libs/saverestore/common.c index 8b4e402df5..5c659aa55b 100644 --- a/tools/libs/saverestore/common.c +++ b/tools/libs/saverestore/common.c @@ -1,5 +1,4 @@ #include - #include "common.h" #include diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h index 43a31f9aa5..8e67989bbf 100644 --- a/tools/libs/saverestore/common.h +++ b/tools/libs/saverestore/common.h @@ -219,6 +219,16 @@ struct xc_sr_restore_ops */ int (*setup)(struct xc_sr_context *ctx); + /** + * Populate PFNs + * + * Given a set of pfns, obtain memory from Xen to fill the physmap for the + * unpopulated subset. + */ + int (*populate_pfns)(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, const uint32_t *types); + + /** * Process an individual record from the stream. The caller shall take * care of processing common records (e.g. END, PAGE_DATA). @@ -366,6 +376,8 @@ struct xc_sr_context int send_back_fd; unsigned long p2m_size; + unsigned long max_pages; + unsigned long tot_pages; xc_hypercall_buffer_t dirty_bitmap_hbuf; /* From Image Header. */ @@ -503,6 +515,14 @@ struct xc_sr_context { /* HVM context blob. */ struct xc_sr_blob context; + + /* Bitmap of currently allocated PFNs during restore. */ + struct sr_bitmap attempted_1g; + struct sr_bitmap attempted_2m; + struct sr_bitmap allocated_pfns; + xen_pfn_t prev_populated_pfn; + xen_pfn_t iteration_tracker_pfn; + unsigned long iteration; } restore; }; } hvm; @@ -567,14 +587,6 @@ int read_record_header(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhd int read_record_data(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr, struct xc_sr_record *rec); -/* - * This would ideally be private in restore.c, but is needed by - * x86_pv_localise_page() if we receive pagetables frames ahead of the - * contents of the frames they point at. - */ -int populate_pfns(struct xc_sr_context *ctx, unsigned int count, - const xen_pfn_t *original_pfns, const uint32_t *types); - /* Handle a STATIC_DATA_END record. */ int handle_static_data_end(struct xc_sr_context *ctx); diff --git a/tools/libs/saverestore/restore.c b/tools/libs/saverestore/restore.c index 8f7bce2585..5ad3df49ba 100644 --- a/tools/libs/saverestore/restore.c +++ b/tools/libs/saverestore/restore.c @@ -71,63 +71,6 @@ static int read_headers(struct xc_sr_context *ctx) return 0; } -/* - * Given a set of pfns, obtain memory from Xen to fill the physmap for the - * unpopulated subset. If types is NULL, no page type checking is performed - * and all unpopulated pfns are populated. - */ -int populate_pfns(struct xc_sr_context *ctx, unsigned int count, - const xen_pfn_t *original_pfns, const uint32_t *types) -{ - xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = ctx->restore.m->pp_mfns, - *pfns = ctx->restore.m->pp_pfns; - unsigned int i, nr_pfns = 0; - int rc = -1; - - for ( i = 0; i < count; ++i ) - { - if ( (!types || - (types && page_type_has_stream_data(types[i]) == true)) && - !pfn_is_populated(ctx, original_pfns[i]) ) - { - rc = pfn_set_populated(ctx, original_pfns[i]); - if ( rc ) - goto err; - pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; - ++nr_pfns; - } - } - - if ( nr_pfns ) - { - rc = xc_domain_populate_physmap_exact( - xch, ctx->domid, nr_pfns, 0, 0, mfns); - if ( rc ) - { - PERROR("Failed to populate physmap"); - goto err; - } - - for ( i = 0; i < nr_pfns; ++i ) - { - if ( mfns[i] == INVALID_MFN ) - { - ERROR("Populate physmap failed for pfn %u", i); - rc = -1; - goto err; - } - - ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]); - } - } - - rc = 0; - - err: - return rc; -} - static int handle_static_data_end_v2(struct xc_sr_context *ctx) { int rc = 0; @@ -270,7 +213,7 @@ static int map_guest_pages(struct xc_sr_context *ctx, uint32_t i, p; int rc; - rc = populate_pfns(ctx, pages->count, m->pfns, m->types); + rc = ctx->restore.ops.populate_pfns(ctx, pages->count, m->pfns, m->types); if ( rc ) { ERROR("Failed to populate pfns for batch of %u pages", pages->count); @@ -1077,6 +1020,9 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, return -1; } + /* See xc_domain_getinfo */ + ctx.restore.max_pages = ctx.dominfo.max_memkb >> (PAGE_SHIFT-10); + ctx.restore.tot_pages = ctx.dominfo.nr_pages; ctx.restore.p2m_size = nr_pfns; ctx.restore.ops = ctx.dominfo.hvm ? restore_ops_x86_hvm : restore_ops_x86_pv; diff --git a/tools/libs/saverestore/restore_x86_hvm.c b/tools/libs/saverestore/restore_x86_hvm.c index 97e7e0f48c..7ed438e1be 100644 --- a/tools/libs/saverestore/restore_x86_hvm.c +++ b/tools/libs/saverestore/restore_x86_hvm.c @@ -130,6 +130,25 @@ static int x86_hvm_localise_page(struct xc_sr_context *ctx, return 0; } +static bool x86_hvm_expand_sp_bitmaps(struct xc_sr_context *ctx, unsigned long max_pfn) +{ + struct sr_bitmap *bm; + + bm = &ctx->x86.hvm.restore.attempted_1g; + if ( !sr_bitmap_expand(bm, max_pfn >> SUPERPAGE_1GB_SHIFT) ) + return false; + + bm = &ctx->x86.hvm.restore.attempted_2m; + if ( !sr_bitmap_expand(bm, max_pfn >> SUPERPAGE_2MB_SHIFT) ) + return false; + + bm = &ctx->x86.hvm.restore.allocated_pfns; + if ( !sr_bitmap_expand(bm, max_pfn) ) + return false; + + return true; +} + /* * restore_ops function. Confirms the stream matches the domain. */ @@ -164,12 +183,21 @@ static int x86_hvm_setup(struct xc_sr_context *ctx) max_pfn = max(ctx->restore.p2m_size, ctx->dominfo.max_memkb >> (PAGE_SHIFT-10)); if ( !sr_bitmap_expand(&ctx->restore.populated_pfns, max_pfn) ) - { - PERROR("Unable to allocate memory for populated_pfns bitmap"); - return -1; - } + goto out; + + if ( !x86_hvm_expand_sp_bitmaps(ctx, max_pfn) ) + goto out; + + /* FIXME: distinguish between PVH and HVM */ + /* No superpage in 1st 2MB due to VGA hole */ + sr_set_bit(0, &ctx->x86.hvm.restore.attempted_1g); + sr_set_bit(0, &ctx->x86.hvm.restore.attempted_2m); return 0; + +out: + PERROR("Unable to allocate memory for pfn bitmaps"); + return -1; } /* @@ -250,6 +278,9 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx) static int x86_hvm_cleanup(struct xc_sr_context *ctx) { sr_bitmap_free(&ctx->restore.populated_pfns); + sr_bitmap_free(&ctx->x86.hvm.restore.attempted_1g); + sr_bitmap_free(&ctx->x86.hvm.restore.attempted_2m); + sr_bitmap_free(&ctx->x86.hvm.restore.allocated_pfns); free(ctx->x86.hvm.restore.context.ptr); free(ctx->x86.restore.cpuid.ptr); @@ -258,6 +289,336 @@ static int x86_hvm_cleanup(struct xc_sr_context *ctx) return 0; } +/* + * Set a range of pfns as allocated + */ +static void pfn_set_long_allocated(struct xc_sr_context *ctx, xen_pfn_t base_pfn) +{ + sr_set_long_bit(base_pfn, &ctx->x86.hvm.restore.allocated_pfns); +} + +static void pfn_set_allocated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + sr_set_bit(pfn, &ctx->x86.hvm.restore.allocated_pfns); +} + +struct x86_hvm_sp { + xen_pfn_t pfn; + xen_pfn_t base_pfn; + unsigned long index; + unsigned long count; +}; + +/* + * Try to allocate a 1GB page for this pfn, but avoid Over-allocation. + * If this succeeds, mark the range of 2MB pages as busy. + */ +static bool x86_hvm_alloc_1g(struct xc_sr_context *ctx, struct x86_hvm_sp *sp) +{ + xc_interface *xch = ctx->xch; + unsigned int order; + int i, done; + xen_pfn_t extent; + + /* Only one attempt to avoid overlapping allocation */ + if ( sr_test_and_set_bit(sp->index, &ctx->x86.hvm.restore.attempted_1g) ) + return false; + + order = SUPERPAGE_1GB_SHIFT; + sp->count = SUPERPAGE_1GB_NR_PFNS; + + /* Allocate only if there is room for another superpage */ + if ( ctx->restore.tot_pages + sp->count > ctx->restore.max_pages ) + return false; + + extent = sp->base_pfn = (sp->pfn >> order) << order; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extent); + if ( done < 0 ) { + PERROR("populate_physmap failed."); + return false; + } + if ( done == 0 ) + return false; + + DPRINTF("1G %" PRI_xen_pfn "\n", sp->base_pfn); + + /* Mark all 2MB pages as done to avoid overlapping allocation */ + for ( i = 0; i < (SUPERPAGE_1GB_NR_PFNS/SUPERPAGE_2MB_NR_PFNS); i++ ) + sr_set_bit((sp->base_pfn >> SUPERPAGE_2MB_SHIFT) + i, &ctx->x86.hvm.restore.attempted_2m); + + return true; +} + +/* Allocate a 2MB page if x86_hvm_alloc_1g failed, avoid Over-allocation. */ +static bool x86_hvm_alloc_2m(struct xc_sr_context *ctx, struct x86_hvm_sp *sp) +{ + xc_interface *xch = ctx->xch; + unsigned int order; + int done; + xen_pfn_t extent; + + /* Only one attempt to avoid overlapping allocation */ + if ( sr_test_and_set_bit(sp->index, &ctx->x86.hvm.restore.attempted_2m) ) + return false; + + order = SUPERPAGE_2MB_SHIFT; + sp->count = SUPERPAGE_2MB_NR_PFNS; + + /* Allocate only if there is room for another superpage */ + if ( ctx->restore.tot_pages + sp->count > ctx->restore.max_pages ) + return false; + + extent = sp->base_pfn = (sp->pfn >> order) << order; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extent); + if ( done < 0 ) { + PERROR("populate_physmap failed."); + return false; + } + if ( done == 0 ) + return false; + + DPRINTF("2M %" PRI_xen_pfn "\n", sp->base_pfn); + return true; +} + +/* Allocate a single page if x86_hvm_alloc_2m failed. */ +static bool x86_hvm_alloc_4k(struct xc_sr_context *ctx, struct x86_hvm_sp *sp) +{ + xc_interface *xch = ctx->xch; + unsigned int order; + int done; + xen_pfn_t extent; + + order = 0; + sp->count = 1UL; + + /* Allocate only if there is room for another page */ + if ( ctx->restore.tot_pages + sp->count > ctx->restore.max_pages ) { + errno = E2BIG; + return false; + } + + extent = sp->base_pfn = (sp->pfn >> order) << order; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extent); + if ( done < 0 ) { + PERROR("populate_physmap failed."); + return false; + } + if ( done == 0 ) { + errno = ENOMEM; + return false; + } + + DPRINTF("4K %" PRI_xen_pfn "\n", sp->base_pfn); + return true; +} +/* + * Attempt to allocate a superpage where the pfn resides. + */ +static int x86_hvm_allocate_pfn(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + bool success; + unsigned long idx_1g, idx_2m; + struct x86_hvm_sp sp = { + .pfn = pfn + }; + + if ( sr_test_bit(pfn, &ctx->x86.hvm.restore.allocated_pfns) ) + return 0; + + idx_1g = pfn >> SUPERPAGE_1GB_SHIFT; + idx_2m = pfn >> SUPERPAGE_2MB_SHIFT; + + sp.index = idx_1g; + success = x86_hvm_alloc_1g(ctx, &sp); + + if ( success == false ) { + sp.index = idx_2m; + success = x86_hvm_alloc_2m(ctx, &sp); + } + + if ( success == false ) { + sp.index = 0; + success = x86_hvm_alloc_4k(ctx, &sp); + } + + if ( success == false ) + return -1; + + do { + if ( sp.count >= BITS_PER_LONG ) { + sp.count -= BITS_PER_LONG; + ctx->restore.tot_pages += BITS_PER_LONG; + pfn_set_long_allocated(ctx, sp.base_pfn + sp.count); + } else { + sp.count--; + ctx->restore.tot_pages++; + pfn_set_allocated(ctx, sp.base_pfn + sp.count); + } + } while ( sp.count ); + + return 0; +} + +/* + * Deallocate memory. + * There was likely an optimistic superpage allocation. + * This means more pages may have been allocated past gap_end. + * This range is not freed now. Incoming higher pfns will release it. + */ +static int x86_hvm_punch_hole(struct xc_sr_context *ctx, + xen_pfn_t gap_start, xen_pfn_t gap_end) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t _pfn, pfn; + uint32_t domid, freed = 0; + int rc; + + pfn = gap_start >> SUPERPAGE_1GB_SHIFT; + do + { + sr_set_bit(pfn, &ctx->x86.hvm.restore.attempted_1g); + } while (++pfn <= gap_end >> SUPERPAGE_1GB_SHIFT); + + pfn = gap_start >> SUPERPAGE_2MB_SHIFT; + do + { + sr_set_bit(pfn, &ctx->x86.hvm.restore.attempted_2m); + } while (++pfn <= gap_end >> SUPERPAGE_2MB_SHIFT); + + pfn = gap_start; + + while ( pfn <= gap_end ) + { + if ( sr_test_and_clear_bit(pfn, &ctx->x86.hvm.restore.allocated_pfns) ) + { + domid = ctx->domid; + _pfn = pfn; + rc = xc_domain_decrease_reservation_exact(xch, domid, 1, 0, &_pfn); + if ( rc ) + { + PERROR("Failed to release pfn %" PRI_xen_pfn, pfn); + return -1; + } + ctx->restore.tot_pages--; + freed++; + } + pfn++; + } + if ( freed ) + DPRINTF("freed %u between %" PRI_xen_pfn " %" PRI_xen_pfn "\n", + freed, gap_start, gap_end); + return 0; +} + +static int x86_hvm_unpopulate_page(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + sr_clear_bit(pfn, &ctx->restore.populated_pfns); + return x86_hvm_punch_hole(ctx, pfn, pfn); +} + +static int x86_hvm_populate_page(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xen_pfn_t gap_start, gap_end; + bool has_gap, first_iteration; + int rc; + + /* + * Check for a gap between the previous populated pfn and this pfn. + * In case a gap exists, it is required to punch a hole to release memory, + * starting after the previous pfn and before this pfn. + * + * But: this can be done only during the first iteration, which is the + * only place there superpage allocations are attempted. All following + * iterations lack the info to properly maintain prev_populated_pfn. + */ + has_gap = ctx->x86.hvm.restore.prev_populated_pfn + 1 < pfn; + first_iteration = ctx->x86.hvm.restore.iteration == 0; + if ( has_gap && first_iteration ) + { + gap_start = ctx->x86.hvm.restore.prev_populated_pfn + 1; + gap_end = pfn - 1; + + rc = x86_hvm_punch_hole(ctx, gap_start, gap_end); + if ( rc ) + goto err; + } + + rc = x86_hvm_allocate_pfn(ctx, pfn); + if ( rc ) + goto err; + pfn_set_populated(ctx, pfn); + ctx->x86.hvm.restore.prev_populated_pfn = pfn; + + rc = 0; +err: + return rc; +} + +/* + * Try to allocate superpages. + * This works without memory map because the pfns arrive in incremental order. + * All pfn numbers and their type are submitted. + * Only pfns with data will have also pfn content transmitted. + */ +static int x86_hvm_populate_pfns(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, + const uint32_t *types) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t pfn, min_pfn, max_pfn; + bool has_data, populated; + unsigned i = count; + int rc = 0; + + min_pfn = count ? original_pfns[0] : 0; + max_pfn = count ? original_pfns[count - 1] : 0; + DPRINTF("batch of %u pfns between %" PRI_xen_pfn " %" PRI_xen_pfn "\n", + count, min_pfn, max_pfn); + + if ( !x86_hvm_expand_sp_bitmaps(ctx, max_pfn) ) + { + ERROR("Unable to allocate memory for pfn bitmaps"); + return -1; + } + + /* + * There is no indicator for a new iteration. + * Simulate it by checking if a lower pfn is coming in. + * In the end it matters only to know if this iteration is the first one. + */ + if ( min_pfn < ctx->x86.hvm.restore.iteration_tracker_pfn ) + ctx->x86.hvm.restore.iteration++; + ctx->x86.hvm.restore.iteration_tracker_pfn = min_pfn; + + for ( i = 0; i < count; ++i ) + { + pfn = original_pfns[i]; + + has_data = page_type_has_stream_data(types[i]); + populated = pfn_is_populated(ctx, pfn); + + /* + * page has data, pfn populated: nothing to do + * page has data, pfn not populated: likely never seen before + * page has no data, pfn populated: likely ballooned out during migration + * page has no data, pfn not populated: nothing to do + */ + if ( has_data && !populated ) + { + rc = x86_hvm_populate_page(ctx, pfn); + } else if ( !has_data && populated ) + { + rc = x86_hvm_unpopulate_page(ctx, pfn); + } + if ( rc ) + break; + } + + return rc; +} + + struct xc_sr_restore_ops restore_ops_x86_hvm = { .pfn_is_valid = x86_hvm_pfn_is_valid, @@ -266,6 +627,7 @@ struct xc_sr_restore_ops restore_ops_x86_hvm = .set_page_type = x86_hvm_set_page_type, .localise_page = x86_hvm_localise_page, .setup = x86_hvm_setup, + .populate_pfns = x86_hvm_populate_pfns, .process_record = x86_hvm_process_record, .static_data_complete = x86_static_data_complete, .stream_complete = x86_hvm_stream_complete, diff --git a/tools/libs/saverestore/restore_x86_pv.c b/tools/libs/saverestore/restore_x86_pv.c index c73a3cd99f..244f1da218 100644 --- a/tools/libs/saverestore/restore_x86_pv.c +++ b/tools/libs/saverestore/restore_x86_pv.c @@ -959,6 +959,64 @@ static void x86_pv_set_gfn(struct xc_sr_context *ctx, xen_pfn_t pfn, ((uint32_t *)ctx->x86.pv.p2m)[pfn] = mfn; } +/* + * Given a set of pfns, obtain memory from Xen to fill the physmap for the + * unpopulated subset. If types is NULL, no page type checking is performed + * and all unpopulated pfns are populated. + */ +static int x86_pv_populate_pfns(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, + const uint32_t *types) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t *mfns = ctx->restore.m->pp_mfns, + *pfns = ctx->restore.m->pp_pfns; + unsigned int i, nr_pfns = 0; + int rc = -1; + + for ( i = 0; i < count; ++i ) + { + if ( (!types || + (types && page_type_has_stream_data(types[i]) == true)) && + !pfn_is_populated(ctx, original_pfns[i]) ) + { + rc = pfn_set_populated(ctx, original_pfns[i]); + if ( rc ) + goto err; + pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; + ++nr_pfns; + } + } + + if ( nr_pfns ) + { + rc = xc_domain_populate_physmap_exact( + xch, ctx->domid, nr_pfns, 0, 0, mfns); + if ( rc ) + { + PERROR("Failed to populate physmap"); + goto err; + } + + for ( i = 0; i < nr_pfns; ++i ) + { + if ( mfns[i] == INVALID_MFN ) + { + ERROR("Populate physmap failed for pfn %u", i); + rc = -1; + goto err; + } + + ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]); + } + } + + rc = 0; + + err: + return rc; +} + /* * restore_ops function. Convert pfns back to mfns in pagetables. Possibly * needs to populate new frames if a PTE is found referring to a frame which @@ -1003,7 +1061,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx, } } - if ( to_populate && populate_pfns(ctx, to_populate, pfns, NULL) ) + if ( to_populate && x86_pv_populate_pfns(ctx, to_populate, pfns, NULL) ) return -1; for ( i = 0; i < (PAGE_SIZE / sizeof(uint64_t)); ++i ) @@ -1200,6 +1258,7 @@ struct xc_sr_restore_ops restore_ops_x86_pv = .set_gfn = x86_pv_set_gfn, .localise_page = x86_pv_localise_page, .setup = x86_pv_setup, + .populate_pfns = x86_pv_populate_pfns, .process_record = x86_pv_process_record, .static_data_complete = x86_static_data_complete, .stream_complete = x86_pv_stream_complete,