From patchwork Wed Aug 2 13:45:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 9876809 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 350EB602BC for ; Wed, 2 Aug 2017 13:48:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 251D2287B7 for ; Wed, 2 Aug 2017 13:48:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19DDA287B9; Wed, 2 Aug 2017 13:48:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7BE6C287B7 for ; Wed, 2 Aug 2017 13:48:22 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dcty3-0003uE-I1; Wed, 02 Aug 2017 13:45:43 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dcty1-0003u8-Bj for xen-devel@lists.xen.org; Wed, 02 Aug 2017 13:45:41 +0000 Received: from [85.158.143.35] by server-4.bemta-6.messagelabs.com id 0D/2A-02962-487D1895; Wed, 02 Aug 2017 13:45:40 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrOLMWRWlGSWpSXmKPExsUSuHLSAt3m642 RBjNOqVks+biYxYHR4+ju30wBjFGsmXlJ+RUJrBmdmy4zF2xfzlhxebp0A+OKyi5GLg4WgSdM Eg/OHmfuYuTkkBDIlbixr5mti5EDyBaRePI/DSQsJHCQSWLrKSUQm01ASWLvweOMILaIQKrE8 fYJrCA2s4CCxIvnW5lAbGEBb4lZ3S/AalgEVCU6Hm9lAbF5BYwlJj/czAqxSl5iYu80xgmM3A sYGVYxahSnFpWlFukaGeslFWWmZ5TkJmbm6BoamOnlphYXJ6an5iQmFesl5+duYgR6lwEIdjD +mR94iFGSg0lJlFexpz5SiC8pP6UyI7E4I76oNCe1+BCjDAeHkgRv7dXGSCHBotT01Iq0zBxg mMGkJTh4lER4ea8BpXmLCxJzizPTIVKnGI05Nqxe/4WJ49WE/9+YhFjy8vNSpcR5NUFKBUBKM 0rz4AbBwv8So6yUMC8j0GlCPAWpRbmZJajyrxjFORiVhHmng9zDk5lXArfvFdApTECn/KkDO6 UkESEl1cDIzvTb7dneg68D3zYk9e0699jrg69GydeVavfuHDoo9btH4NuBLZnMEzXm/RQMPt1 089ZPl4s3rmXOEQlw/Tfx0hXWU+Ypjl2zd67nYjFwnnMg8unal31ewX8jutpU3L7vn+oXX7xU N1pS6PC/SSUJSz7ltJuxmxpKts0P3nopYtbRuxODaxcGKbEUZyQaajEXFScCABdgeip6AgAA X-Env-Sender: olaf@aepfle.de X-Msg-Ref: server-6.tower-21.messagelabs.com!1501681539!53097070!1 X-Originating-IP: [81.169.146.160] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: QmFkIElQOiA4MS4xNjkuMTQ2LjE2MCA9PiA1NTc3MTg=\n,sa_preprocessor: QmFkIElQOiA4MS4xNjkuMTQ2LjE2MCA9PiA1NTc3MTg=\n X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18948 invoked from network); 2 Aug 2017 13:45:39 -0000 Received: from mo4-p00-ob.smtp.rzone.de (HELO mo4-p00-ob.smtp.rzone.de) (81.169.146.160) by server-6.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 2 Aug 2017 13:45:39 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1501681539; l=19062; s=domk; d=aepfle.de; h=Date:Subject:Cc:To:From; bh=ji3we27Ls03naLq7LvZZe/GLaD0WnVITUZJyvs+7yhI=; b=eaByhT1dC8lvXmJCbger8HzDzQtufCthi+Uzt6M1kQoXqZ+GHEehnAIiBMqwUiNteB +vs5Xohyfb0+4kmCjiXfbHU53Fh9MUTyp55bS2+Xwy92qEB9DJOJDBeHPLBJwZFXUzxb dXPHuLq7VvRbJBxvCJfCe6uBJKoGWqsQl+Czs= X-RZG-AUTH: :P2EQZWCpfu+qG7CngxMFH1J+yackYocTD1iAi8x+OWi/zfN1cLnAYQ78mmFeN6Aqp4B5pIVlTUWUm3u3pWh1+4rEzpLBkg== X-RZG-CLASS-ID: mo00 Received: from sender ([2001:a61:102e:7fff:27ee:68a5:7ddc:d3db]) by smtp.strato.de (RZmta 41.1 AUTH) with ESMTPSA id Q06ec7t72Djad1r (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Wed, 2 Aug 2017 15:45:36 +0200 (CEST) From: Olaf Hering To: Ian Jackson , Wei Liu , xen-devel@lists.xen.org Date: Wed, 2 Aug 2017 15:45:25 +0200 Message-Id: <20170802134525.19597-1-olaf@aepfle.de> X-Mailer: git-send-email 2.13.3 Cc: Olaf Hering Subject: [Xen-devel] [PATCH v1] tools/libxc: use superpages during restore of HVM guest X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP During creating of a HVM domU meminit_hvm() tries to map superpages. After save/restore or migration this mapping is lost, everything is allocated in single pages. This causes a performance degradition after migration. Add neccessary code to preallocate a superpage for the chunk of pfns that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering --- based on RELEASE-4.9.0 tools/libxc/xc_sr_common.c | 41 ++++++++ tools/libxc/xc_sr_common.h | 79 +++++++++++++++- tools/libxc/xc_sr_restore.c | 135 +------------------------- tools/libxc/xc_sr_restore_x86_hvm.c | 183 ++++++++++++++++++++++++++++++++++++ tools/libxc/xc_sr_restore_x86_pv.c | 72 +++++++++++++- 5 files changed, 376 insertions(+), 134 deletions(-) diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c index 48fa676f4e..9b68a064eb 100644 --- a/tools/libxc/xc_sr_common.c +++ b/tools/libxc/xc_sr_common.c @@ -156,6 +156,47 @@ static void __attribute__((unused)) build_assertions(void) } /* + * Expand the tracking structures as needed. + * To avoid realloc()ing too excessively, the size increased to the nearest power + * of two large enough to contain the required number of bits. + */ +bool _xc_sr_bitmap_resize(struct xc_sr_bitmap *bm, unsigned long bits) +{ + if (bits > bm->bits) + { + size_t new_max; + size_t old_sz, new_sz; + void *p; + + /* Round up to the nearest power of two larger than bit, less 1. */ + new_max = bits; + new_max |= new_max >> 1; + new_max |= new_max >> 2; + new_max |= new_max >> 4; + new_max |= new_max >> 8; + new_max |= new_max >> 16; +#ifdef __x86_64__ + new_max |= new_max >> 32; +#endif + + old_sz = bitmap_size(bm->bits + 1); + new_sz = bitmap_size(new_max + 1); + p = realloc(bm->p, new_sz); + if (!p) + return false; + + if (bm->p) + memset(p + old_sz, 0, new_sz - old_sz); + else + memset(p, 0, new_sz); + + bm->p = p; + bm->bits = new_max; + } + return true; +} + +/* * Local variables: * mode: C * c-file-style: "BSD" diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h index a83f22af4e..ad1a2e6e02 100644 --- a/tools/libxc/xc_sr_common.h +++ b/tools/libxc/xc_sr_common.h @@ -140,6 +140,13 @@ struct xc_sr_restore_ops int (*setup)(struct xc_sr_context *ctx); /** + * Populate PFNs + * + */ + int (*populate_pfns)(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, const uint32_t *types); + + /** * Process an individual record from the stream. The caller shall take * care of processing common records (e.g. END, PAGE_DATA). * @@ -172,6 +179,12 @@ struct xc_sr_x86_pv_restore_vcpu size_t basicsz, extdsz, xsavesz, msrsz; }; +struct xc_sr_bitmap +{ + void *p; + unsigned long bits; +}; + struct xc_sr_context { xc_interface *xch; @@ -255,8 +268,7 @@ struct xc_sr_context domid_t xenstore_domid, console_domid; /* Bitmap of currently populated PFNs during restore. */ - unsigned long *populated_pfns; - xen_pfn_t max_populated_pfn; + struct xc_sr_bitmap populated_pfns; /* Sender has invoked verify mode on the stream. */ bool verify; @@ -331,6 +343,11 @@ struct xc_sr_context /* HVM context blob. */ void *context; size_t contextsz; + + /* Bitmap of currently allocated PFNs during restore. */ + struct xc_sr_bitmap attempted_1g; + struct xc_sr_bitmap attempted_2m; + struct xc_sr_bitmap allocated_pfns; } restore; }; } x86_hvm; @@ -343,6 +360,64 @@ extern struct xc_sr_save_ops save_ops_x86_hvm; extern struct xc_sr_restore_ops restore_ops_x86_pv; extern struct xc_sr_restore_ops restore_ops_x86_hvm; +extern bool _xc_sr_bitmap_resize(struct xc_sr_bitmap *bm, unsigned long bits); + +static inline bool xc_sr_bitmap_resize(struct xc_sr_bitmap *bm, unsigned long bits) +{ + if (bits > bm->bits) + return _xc_sr_bitmap_resize(bm, bits); + return true; +} + +static inline void xc_sr_bitmap_free(struct xc_sr_bitmap *bm) +{ + free(bm->p); +} + +static inline bool xc_sr_set_bit(unsigned long bit, struct xc_sr_bitmap *bm) +{ + if (!xc_sr_bitmap_resize(bm, bit)) + return false; + + set_bit(bit, bm->p); + return true; +} + +static inline bool xc_sr_test_bit(unsigned long bit, struct xc_sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_bit(bit, bm->p); +} + +static inline int xc_sr_test_and_clear_bit(unsigned long bit, struct xc_sr_bitmap *bm) +{ + return test_and_clear_bit(bit, bm->p); +} + +static inline int xc_sr_test_and_set_bit(unsigned long bit, struct xc_sr_bitmap *bm) +{ + return test_and_set_bit(bit, bm->p); +} + +static inline bool pfn_is_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + return xc_sr_test_bit(pfn, &ctx->restore.populated_pfns); +} + +static inline int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xc_interface *xch = ctx->xch; + + if ( !xc_sr_set_bit(pfn, &ctx->restore.populated_pfns) ) + { + ERROR("Failed to realloc populated_pfns bitmap"); + errno = ENOMEM; + return -1; + } + return 0; +} + struct xc_sr_record { uint32_t type; diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c index 3549f0a1ae..9888acb05f 100644 --- a/tools/libxc/xc_sr_restore.c +++ b/tools/libxc/xc_sr_restore.c @@ -68,131 +68,6 @@ static int read_headers(struct xc_sr_context *ctx) return 0; } -/* - * Is a pfn populated? - */ -static bool pfn_is_populated(const struct xc_sr_context *ctx, xen_pfn_t pfn) -{ - if ( pfn > ctx->restore.max_populated_pfn ) - return false; - return test_bit(pfn, ctx->restore.populated_pfns); -} - -/* - * Set a pfn as populated, expanding the tracking structures if needed. To - * avoid realloc()ing too excessively, the size increased to the nearest power - * of two large enough to contain the required pfn. - */ -static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) -{ - xc_interface *xch = ctx->xch; - - if ( pfn > ctx->restore.max_populated_pfn ) - { - xen_pfn_t new_max; - size_t old_sz, new_sz; - unsigned long *p; - - /* Round up to the nearest power of two larger than pfn, less 1. */ - new_max = pfn; - new_max |= new_max >> 1; - new_max |= new_max >> 2; - new_max |= new_max >> 4; - new_max |= new_max >> 8; - new_max |= new_max >> 16; -#ifdef __x86_64__ - new_max |= new_max >> 32; -#endif - - old_sz = bitmap_size(ctx->restore.max_populated_pfn + 1); - new_sz = bitmap_size(new_max + 1); - p = realloc(ctx->restore.populated_pfns, new_sz); - if ( !p ) - { - ERROR("Failed to realloc populated bitmap"); - errno = ENOMEM; - return -1; - } - - memset((uint8_t *)p + old_sz, 0x00, new_sz - old_sz); - - ctx->restore.populated_pfns = p; - ctx->restore.max_populated_pfn = new_max; - } - - assert(!test_bit(pfn, ctx->restore.populated_pfns)); - set_bit(pfn, ctx->restore.populated_pfns); - - return 0; -} - -/* - * Given a set of pfns, obtain memory from Xen to fill the physmap for the - * unpopulated subset. If types is NULL, no page type checking is performed - * and all unpopulated pfns are populated. - */ -int populate_pfns(struct xc_sr_context *ctx, unsigned count, - const xen_pfn_t *original_pfns, const uint32_t *types) -{ - xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = malloc(count * sizeof(*mfns)), - *pfns = malloc(count * sizeof(*pfns)); - unsigned i, nr_pfns = 0; - int rc = -1; - - if ( !mfns || !pfns ) - { - ERROR("Failed to allocate %zu bytes for populating the physmap", - 2 * count * sizeof(*mfns)); - goto err; - } - - for ( i = 0; i < count; ++i ) - { - if ( (!types || (types && - (types[i] != XEN_DOMCTL_PFINFO_XTAB && - types[i] != XEN_DOMCTL_PFINFO_BROKEN))) && - !pfn_is_populated(ctx, original_pfns[i]) ) - { - rc = pfn_set_populated(ctx, original_pfns[i]); - if ( rc ) - goto err; - pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; - ++nr_pfns; - } - } - - if ( nr_pfns ) - { - rc = xc_domain_populate_physmap_exact( - xch, ctx->domid, nr_pfns, 0, 0, mfns); - if ( rc ) - { - PERROR("Failed to populate physmap"); - goto err; - } - - for ( i = 0; i < nr_pfns; ++i ) - { - if ( mfns[i] == INVALID_MFN ) - { - ERROR("Populate physmap failed for pfn %u", i); - rc = -1; - goto err; - } - - ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]); - } - } - - rc = 0; - - err: - free(pfns); - free(mfns); - - return rc; -} /* * Given a list of pfns, their types, and a block of page data from the @@ -219,7 +94,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned count, goto err; } - rc = populate_pfns(ctx, count, pfns, types); + rc = ctx->restore.ops.populate_pfns(ctx, count, pfns, types); if ( rc ) { ERROR("Failed to populate pfns for batch of %u pages", count); @@ -684,10 +559,8 @@ static int setup(struct xc_sr_context *ctx) if ( rc ) goto err; - ctx->restore.max_populated_pfn = (32 * 1024 / 4) - 1; - ctx->restore.populated_pfns = bitmap_alloc( - ctx->restore.max_populated_pfn + 1); - if ( !ctx->restore.populated_pfns ) + rc = !xc_sr_bitmap_resize(&ctx->restore.populated_pfns, 32 * 1024 / 4); + if ( rc ) { ERROR("Unable to allocate memory for populated_pfns bitmap"); rc = -1; @@ -722,7 +595,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size))); free(ctx->restore.buffered_records); - free(ctx->restore.populated_pfns); + xc_sr_bitmap_free(&ctx->restore.populated_pfns); if ( ctx->restore.ops.cleanup(ctx) ) PERROR("Failed to clean up"); } diff --git a/tools/libxc/xc_sr_restore_x86_hvm.c b/tools/libxc/xc_sr_restore_x86_hvm.c index 1dca85354a..c32a318bc4 100644 --- a/tools/libxc/xc_sr_restore_x86_hvm.c +++ b/tools/libxc/xc_sr_restore_x86_hvm.c @@ -3,6 +3,10 @@ #include "xc_sr_common_x86.h" +#define SUPERPAGE_2MB_SHIFT 9 +#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT) +#define SUPERPAGE_1GB_SHIFT 18 +#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT) /* * Process an HVM_CONTEXT record from the stream. */ @@ -135,6 +139,8 @@ static int x86_hvm_localise_page(struct xc_sr_context *ctx, static int x86_hvm_setup(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; + struct xc_sr_bitmap *bm; + unsigned long bits; if ( ctx->restore.guest_type != DHDR_TYPE_X86_HVM ) { @@ -149,7 +155,30 @@ static int x86_hvm_setup(struct xc_sr_context *ctx) return -1; } + bm = &ctx->x86_hvm.restore.attempted_1g; + bits = (ctx->restore.p2m_size >> SUPERPAGE_1GB_SHIFT) + 1; + if ( xc_sr_bitmap_resize(bm, bits) == false ) + goto out; + + bm = &ctx->x86_hvm.restore.attempted_2m; + bits = (ctx->restore.p2m_size >> SUPERPAGE_2MB_SHIFT) + 1; + if ( xc_sr_bitmap_resize(bm, bits) == false ) + goto out; + + bm = &ctx->x86_hvm.restore.allocated_pfns; + bits = ctx->restore.p2m_size + 1; + if ( xc_sr_bitmap_resize(bm, bits) == false ) + goto out; + + /* No superpage in 1st 2MB due to VGA hole */ + xc_sr_set_bit(0, &ctx->x86_hvm.restore.attempted_1g); + xc_sr_set_bit(0, &ctx->x86_hvm.restore.attempted_2m); + return 0; + +out: + ERROR("Unable to allocate memory for pfn bitmaps"); + return -1; } /* @@ -224,10 +253,163 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx) static int x86_hvm_cleanup(struct xc_sr_context *ctx) { free(ctx->x86_hvm.restore.context); + xc_sr_bitmap_free(&ctx->x86_hvm.restore.attempted_1g); + xc_sr_bitmap_free(&ctx->x86_hvm.restore.attempted_2m); + xc_sr_bitmap_free(&ctx->x86_hvm.restore.allocated_pfns); return 0; } +/* + * Set a pfn as allocated, expanding the tracking structures if needed. + */ +static int pfn_set_allocated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xc_interface *xch = ctx->xch; + + if ( !xc_sr_set_bit(pfn, &ctx->x86_hvm.restore.allocated_pfns) ) + { + ERROR("Failed to realloc allocated_pfns bitmap"); + errno = ENOMEM; + return -1; + } + return 0; +} + +/* + * Attempt to allocate a superpage where the pfn resides. + */ +static int x86_hvm_allocate_pfn(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xc_interface *xch = ctx->xch; + bool success = false; + int rc = -1, done; + unsigned int order; + unsigned long i; + unsigned long stat_1g = 0, stat_2m = 0, stat_4k = 0; + unsigned long idx_1g, idx_2m; + unsigned long count; + xen_pfn_t base_pfn = 0, extnt; + + if (xc_sr_test_bit(pfn, &ctx->x86_hvm.restore.allocated_pfns)) + return 0; + + idx_1g = pfn >> SUPERPAGE_1GB_SHIFT; + idx_2m = pfn >> SUPERPAGE_2MB_SHIFT; + if (!xc_sr_bitmap_resize(&ctx->x86_hvm.restore.attempted_1g, idx_1g)) + { + PERROR("Failed to realloc attempted_1g"); + return -1; + } + if (!xc_sr_bitmap_resize(&ctx->x86_hvm.restore.attempted_2m, idx_2m)) + { + PERROR("Failed to realloc attempted_2m"); + return -1; + } + DPRINTF("idx_1g %lu idx_2m %lu\n", idx_1g, idx_2m); + if (!xc_sr_test_and_set_bit(idx_1g, &ctx->x86_hvm.restore.attempted_1g)) { + order = SUPERPAGE_1GB_SHIFT; + count = 1UL << order; + base_pfn = (pfn >> order) << order; + extnt = base_pfn; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extnt); + DPRINTF("1G base_pfn %" PRI_xen_pfn " done %d\n", base_pfn, done); + if (done > 0) { + struct xc_sr_bitmap *bm = &ctx->x86_hvm.restore.attempted_2m; + success = true; + stat_1g = done; + for (i = 0; i < (count >> SUPERPAGE_2MB_SHIFT); i++) + xc_sr_set_bit((base_pfn >> SUPERPAGE_2MB_SHIFT) + i, bm); + } + } + + if (!xc_sr_test_and_set_bit(idx_2m, &ctx->x86_hvm.restore.attempted_2m)) { + order = SUPERPAGE_2MB_SHIFT; + count = 1UL << order; + base_pfn = (pfn >> order) << order; + extnt = base_pfn; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extnt); + DPRINTF("2M base_pfn %" PRI_xen_pfn " done %d\n", base_pfn, done); + if (done > 0) { + success = true; + stat_2m = done; + } + } + if (success == false) { + count = 1; + extnt = base_pfn = pfn; + done = xc_domain_populate_physmap(xch, ctx->domid, count, 0, 0, &extnt); + if (done > 0) { + success = true; + stat_4k = count; + } + } + DPRINTF("count %lu 1G %lu 2M %lu 4k %lu\n", count, stat_1g, stat_2m, stat_4k); + if (success == true) { + do { + count--; + rc = pfn_set_allocated(ctx, base_pfn + count); + if (rc) + break; + } while (count); + } + return rc; +} + +static int x86_hvm_populate_pfns(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, + const uint32_t *types) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t min_pfn = original_pfns[0], max_pfn = original_pfns[0]; + unsigned i; + int rc = -1; + + for ( i = 0; i < count; ++i ) + { + if (original_pfns[i] < min_pfn) + min_pfn = original_pfns[i]; + if (original_pfns[i] > max_pfn) + max_pfn = original_pfns[i]; + if ( (types[i] != XEN_DOMCTL_PFINFO_XTAB && + types[i] != XEN_DOMCTL_PFINFO_BROKEN) && + !pfn_is_populated(ctx, original_pfns[i]) ) + { + rc = x86_hvm_allocate_pfn(ctx, original_pfns[i]); + if ( rc ) + goto err; + rc = pfn_set_populated(ctx, original_pfns[i]); + if ( rc ) + goto err; + } + } + + while (min_pfn < max_pfn) + { + if (!xc_sr_bitmap_resize(&ctx->x86_hvm.restore.allocated_pfns, min_pfn)) + { + PERROR("Failed to realloc allocated_pfns %" PRI_xen_pfn, min_pfn); + goto err; + } + if (!pfn_is_populated(ctx, min_pfn) && xc_sr_test_and_clear_bit(min_pfn, &ctx->x86_hvm.restore.allocated_pfns)) { + xen_pfn_t pfn = min_pfn; + rc = xc_domain_decrease_reservation_exact(xch, ctx->domid, 1, 0, &pfn); + if ( rc ) + { + PERROR("Failed to release pfn %" PRI_xen_pfn, min_pfn); + goto err; + } + } + min_pfn++; + } + + rc = 0; + + err: + return rc; +} + + struct xc_sr_restore_ops restore_ops_x86_hvm = { .pfn_is_valid = x86_hvm_pfn_is_valid, @@ -236,6 +418,7 @@ struct xc_sr_restore_ops restore_ops_x86_hvm = .set_page_type = x86_hvm_set_page_type, .localise_page = x86_hvm_localise_page, .setup = x86_hvm_setup, + .populate_pfns = x86_hvm_populate_pfns, .process_record = x86_hvm_process_record, .stream_complete = x86_hvm_stream_complete, .cleanup = x86_hvm_cleanup, diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c index 50e25c162c..87957559bc 100644 --- a/tools/libxc/xc_sr_restore_x86_pv.c +++ b/tools/libxc/xc_sr_restore_x86_pv.c @@ -937,6 +937,75 @@ static void x86_pv_set_gfn(struct xc_sr_context *ctx, xen_pfn_t pfn, } /* + * Given a set of pfns, obtain memory from Xen to fill the physmap for the + * unpopulated subset. If types is NULL, no page type checking is performed + * and all unpopulated pfns are populated. + */ +static int x86_pv_populate_pfns(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, + const uint32_t *types) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t *mfns = malloc(count * sizeof(*mfns)), + *pfns = malloc(count * sizeof(*pfns)); + unsigned i, nr_pfns = 0; + int rc = -1; + + if ( !mfns || !pfns ) + { + ERROR("Failed to allocate %zu bytes for populating the physmap", + 2 * count * sizeof(*mfns)); + goto err; + } + + for ( i = 0; i < count; ++i ) + { + if ( (!types || (types && + (types[i] != XEN_DOMCTL_PFINFO_XTAB && + types[i] != XEN_DOMCTL_PFINFO_BROKEN))) && + !pfn_is_populated(ctx, original_pfns[i]) ) + { + rc = pfn_set_populated(ctx, original_pfns[i]); + if ( rc ) + goto err; + pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; + ++nr_pfns; + } + } + + if ( nr_pfns ) + { + rc = xc_domain_populate_physmap_exact( + xch, ctx->domid, nr_pfns, 0, 0, mfns); + if ( rc ) + { + PERROR("Failed to populate physmap"); + goto err; + } + + for ( i = 0; i < nr_pfns; ++i ) + { + if ( mfns[i] == INVALID_MFN ) + { + ERROR("Populate physmap failed for pfn %u", i); + rc = -1; + goto err; + } + + ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]); + } + } + + rc = 0; + + err: + free(pfns); + free(mfns); + + return rc; +} + +/* * restore_ops function. Convert pfns back to mfns in pagetables. Possibly * needs to populate new frames if a PTE is found referring to a frame which * hasn't yet been seen from PAGE_DATA records. @@ -980,7 +1049,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx, } } - if ( to_populate && populate_pfns(ctx, to_populate, pfns, NULL) ) + if ( to_populate && x86_pv_populate_pfns(ctx, to_populate, pfns, NULL) ) return -1; for ( i = 0; i < (PAGE_SIZE / sizeof(uint64_t)); ++i ) @@ -1160,6 +1229,7 @@ struct xc_sr_restore_ops restore_ops_x86_pv = .set_gfn = x86_pv_set_gfn, .localise_page = x86_pv_localise_page, .setup = x86_pv_setup, + .populate_pfns = x86_pv_populate_pfns, .process_record = x86_pv_process_record, .stream_complete = x86_pv_stream_complete, .cleanup = x86_pv_cleanup,