From patchwork Thu Mar 9 18:38:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 9613843 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6346460414 for ; Thu, 9 Mar 2017 18:41:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 437D4285E4 for ; Thu, 9 Mar 2017 18:41:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 38036285E8; Thu, 9 Mar 2017 18:41:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8198C285E6 for ; Thu, 9 Mar 2017 18:41:14 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cm2xN-0005Qs-7l; Thu, 09 Mar 2017 18:38:33 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cm2xM-0005Qm-CC for xen-devel@lists.xenproject.org; Thu, 09 Mar 2017 18:38:32 +0000 Received: from [85.158.139.211] by server-2.bemta-5.messagelabs.com id EE/F6-20080-721A1C85; Thu, 09 Mar 2017 18:38:31 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBLMWRWlGSWpSXmKPExsWyU9JRQldt4cE Ig9lPzC2+b5nM5MDocfjDFZYAxijWzLyk/IoE1oxLu1azF6x2r7j0dx5zA+NTsy5GTg4JAT+J DRtesYHYbAIGEqc2LWIBsUUELCWeru9gB7GZBbIkulcuZ+5i5OAQFnCX+NWqCBJmEVCR+HT/P Fg5r4CnxLr7f1ghRspJ3DzXyQwRF5Q4OfMJC8QYCYmDL16AxYUE1CSOdu1igahPlzgw7xvjBE aeWUhaZiFpWcDItIpRozi1qCy1SNfIWC+pKDM9oyQ3MTNH19DAVC83tbg4MT01JzGpWC85P3c TIzBE6hkYGHcw7mj3O8QoycGkJMr7ee7BCCG+pPyUyozE4oz4otKc1OJDjDIcHEoSvGwLgHKC RanpqRVpmTnAYIVJS3DwKInwWoKkeYsLEnOLM9MhUqcYFaXEeblBEgIgiYzSPLg2WIRcYpSVE uZlZGBgEOIpSC3KzSxBlX/FKM7BqCTMywQyhSczrwRu+iugxUxAi6fxgS0uSURISTUwuqZNWV C6lPPmFPX3+8+tfXPuWRHDEyvLdpusT7N45sducjSQ2vukRaOltWGuT/uGlUeOnDPLjFjq6d+ wSNR1eXVL9f70slhBpyezm8venH33p+NbsfGzbXXnT6tctLSffT/wpvdu/ZsnOtL8p0x9++G8 9LfeX+HicxWzzkUqrVEtZH/I9zzkgRJLcUaioRZzUXEiANDu4cOLAgAA X-Env-Sender: prvs=234c6cd0c=igor.druzhinin@citrix.com X-Msg-Ref: server-4.tower-206.messagelabs.com!1489084710!88675370!1 X-Originating-IP: [185.25.65.24] X-SpamReason: No, hits=0.0 required=7.0 tests=received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 3036 invoked from network); 9 Mar 2017 18:38:30 -0000 Received: from smtp.ctxuk.citrix.com (HELO SMTP.EU.CITRIX.COM) (185.25.65.24) by server-4.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 9 Mar 2017 18:38:30 -0000 X-IronPort-AV: E=Sophos;i="5.36,136,1486425600"; d="scan'208";a="42181562" From: Igor Druzhinin To: , Date: Thu, 9 Mar 2017 18:38:09 +0000 Message-ID: <1489084689-19008-1-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-ClientProxiedBy: FTLPEX02CAS01.citrite.net (10.13.99.120) To AMSPEX02CL03.citrite.net (10.69.22.127) Cc: xen-devel@lists.xenproject.org, qemu-devel@nongnu.org, Igor Druzhinin Subject: [Xen-devel] [PATCH] xen: don't save/restore the physmap on VM save/restore X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Saving/restoring the physmap to/from xenstore was introduced to QEMU majorly in order to cover up the VRAM region restore issue. The sequence of restore operations implies that we should know the effective guest VRAM address *before* we have the VRAM region restored (which happens later). Unfortunately, in Xen environment VRAM memory does actually belong to a guest - not QEMU itself - which means the position of this region is unknown beforehand and can't be mapped into QEMU address space immediately. Previously, recreating xenstore keys, holding the physmap, by the toolstack helped to get this information in place at the right moment ready to be consumed by QEMU to map the region properly. The extraneous complexity of having those keys transferred by the toolstack and unnecessary redundancy prompted us to propose a solution which doesn't require any extra data in xenstore. The idea is to defer the VRAM region mapping till the point we actually know the effective address and able to map it. To that end, we initially only register the pointer to the framebuffer without actual mapping. Then, during the memory region restore phase, we perform the mapping of the known address and update the VRAM region metadata (including previously registered pointer) accordingly. Signed-off-by: Igor Druzhinin --- exec.c | 3 ++ hw/display/vga.c | 2 +- include/hw/xen/xen.h | 2 +- xen-hvm.c | 114 ++++++++++++--------------------------------------- 4 files changed, 32 insertions(+), 89 deletions(-) diff --git a/exec.c b/exec.c index aabb035..5f2809e 100644 --- a/exec.c +++ b/exec.c @@ -2008,6 +2008,9 @@ void *qemu_map_ram_ptr(RAMBlock *ram_block, ram_addr_t addr) } block->host = xen_map_cache(block->offset, block->max_length, 1); + if (block->host == NULL) { + return NULL; + } } return ramblock_ptr(block, addr); } diff --git a/hw/display/vga.c b/hw/display/vga.c index 69c3e1d..be554c2 100644 --- a/hw/display/vga.c +++ b/hw/display/vga.c @@ -2163,7 +2163,7 @@ void vga_common_init(VGACommonState *s, Object *obj, bool global_vmstate) memory_region_init_ram(&s->vram, obj, "vga.vram", s->vram_size, &error_fatal); vmstate_register_ram(&s->vram, global_vmstate ? NULL : DEVICE(obj)); - xen_register_framebuffer(&s->vram); + xen_register_framebuffer(&s->vram, &s->vram_ptr); s->vram_ptr = memory_region_get_ram_ptr(&s->vram); s->get_bpp = vga_get_bpp; s->get_offsets = vga_get_offsets; diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h index 09c2ce5..3831843 100644 --- a/include/hw/xen/xen.h +++ b/include/hw/xen/xen.h @@ -45,6 +45,6 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, struct MemoryRegion *mr, Error **errp); void xen_modified_memory(ram_addr_t start, ram_addr_t length); -void xen_register_framebuffer(struct MemoryRegion *mr); +void xen_register_framebuffer(struct MemoryRegion *mr, uint8_t **ptr); #endif /* QEMU_HW_XEN_H */ diff --git a/xen-hvm.c b/xen-hvm.c index 5043beb..ea5ed24 100644 --- a/xen-hvm.c +++ b/xen-hvm.c @@ -41,6 +41,7 @@ static MemoryRegion ram_memory, ram_640k, ram_lo, ram_hi; static MemoryRegion *framebuffer; +static uint8_t **framebuffer_ptr; static bool xen_in_migration; /* Compatibility with older version */ @@ -302,7 +303,6 @@ static hwaddr xen_phys_offset_to_gaddr(hwaddr start_addr, return physmap->start_addr; } } - return start_addr; } @@ -317,7 +317,6 @@ static int xen_add_to_physmap(XenIOState *state, XenPhysmap *physmap = NULL; hwaddr pfn, start_gpfn; hwaddr phys_offset = memory_region_get_ram_addr(mr); - char path[80], value[17]; const char *mr_name; if (get_physmapping(state, start_addr, size)) { @@ -340,6 +339,27 @@ go_physmap: DPRINTF("mapping vram to %"HWADDR_PRIx" - %"HWADDR_PRIx"\n", start_addr, start_addr + size); + mr_name = memory_region_name(mr); + + physmap = g_malloc(sizeof (XenPhysmap)); + + physmap->start_addr = start_addr; + physmap->size = size; + physmap->name = mr_name; + physmap->phys_offset = phys_offset; + + QLIST_INSERT_HEAD(&state->physmap, physmap, list); + + if (runstate_check(RUN_STATE_INMIGRATE)) + { + /* At this point we have a physmap entry for the framebuffer region + * established during the restore phase so we can safely update the + * registered framebuffer address here. */ + if (mr == framebuffer) + *framebuffer_ptr = memory_region_get_ram_ptr(framebuffer); + return 0; + } + pfn = phys_offset >> TARGET_PAGE_BITS; start_gpfn = start_addr >> TARGET_PAGE_BITS; for (i = 0; i < size >> TARGET_PAGE_BITS; i++) { @@ -350,49 +370,17 @@ go_physmap: if (rc) { DPRINTF("add_to_physmap MFN %"PRI_xen_pfn" to PFN %" PRI_xen_pfn" failed: %d (errno: %d)\n", idx, gpfn, rc, errno); + + QLIST_REMOVE(physmap, list); + g_free(physmap); return -rc; } } - mr_name = memory_region_name(mr); - - physmap = g_malloc(sizeof (XenPhysmap)); - - physmap->start_addr = start_addr; - physmap->size = size; - physmap->name = mr_name; - physmap->phys_offset = phys_offset; - - QLIST_INSERT_HEAD(&state->physmap, physmap, list); - xc_domain_pin_memory_cacheattr(xen_xc, xen_domid, start_addr >> TARGET_PAGE_BITS, (start_addr + size - 1) >> TARGET_PAGE_BITS, XEN_DOMCTL_MEM_CACHEATTR_WB); - - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap/%"PRIx64"/start_addr", - xen_domid, (uint64_t)phys_offset); - snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)start_addr); - if (!xs_write(state->xenstore, 0, path, value, strlen(value))) { - return -1; - } - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap/%"PRIx64"/size", - xen_domid, (uint64_t)phys_offset); - snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)size); - if (!xs_write(state->xenstore, 0, path, value, strlen(value))) { - return -1; - } - if (mr_name) { - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap/%"PRIx64"/name", - xen_domid, (uint64_t)phys_offset); - if (!xs_write(state->xenstore, 0, path, mr_name, strlen(mr_name))) { - return -1; - } - } - return 0; } @@ -1152,54 +1140,6 @@ static void xen_exit_notifier(Notifier *n, void *data) xs_daemon_close(state->xenstore); } -static void xen_read_physmap(XenIOState *state) -{ - XenPhysmap *physmap = NULL; - unsigned int len, num, i; - char path[80], *value = NULL; - char **entries = NULL; - - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap", xen_domid); - entries = xs_directory(state->xenstore, 0, path, &num); - if (entries == NULL) - return; - - for (i = 0; i < num; i++) { - physmap = g_malloc(sizeof (XenPhysmap)); - physmap->phys_offset = strtoull(entries[i], NULL, 16); - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap/%s/start_addr", - xen_domid, entries[i]); - value = xs_read(state->xenstore, 0, path, &len); - if (value == NULL) { - g_free(physmap); - continue; - } - physmap->start_addr = strtoull(value, NULL, 16); - free(value); - - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap/%s/size", - xen_domid, entries[i]); - value = xs_read(state->xenstore, 0, path, &len); - if (value == NULL) { - g_free(physmap); - continue; - } - physmap->size = strtoull(value, NULL, 16); - free(value); - - snprintf(path, sizeof(path), - "/local/domain/0/device-model/%d/physmap/%s/name", - xen_domid, entries[i]); - physmap->name = xs_read(state->xenstore, 0, path, &len); - - QLIST_INSERT_HEAD(&state->physmap, physmap, list); - } - free(entries); -} - static void xen_wakeup_notifier(Notifier *notifier, void *data) { xc_set_hvm_param(xen_xc, xen_domid, HVM_PARAM_ACPI_S_STATE, 0); @@ -1339,7 +1279,6 @@ void xen_hvm_init(PCMachineState *pcms, MemoryRegion **ram_memory) goto err; } xen_be_register_common(); - xen_read_physmap(state); /* Disable ACPI build because Xen handles it */ pcms->acpi_build_enabled = false; @@ -1374,9 +1313,10 @@ void destroy_hvm_domain(bool reboot) } } -void xen_register_framebuffer(MemoryRegion *mr) +void xen_register_framebuffer(MemoryRegion *mr, uint8_t **ptr) { framebuffer = mr; + framebuffer_ptr = ptr; } void xen_shutdown_fatal_error(const char *fmt, ...)