From patchwork Thu Jan 30 16:13:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F412139A for ; Thu, 30 Jan 2020 16:16:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EFA7620707 for ; Thu, 30 Jan 2020 16:16:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="NVcU3iEl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFA7620707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCST-0004Su-O5; Thu, 30 Jan 2020 16:14:21 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSS-0004SM-Nl for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:20 +0000 X-Inumbo-ID: 7570e732-437b-11ea-8396-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7570e732-437b-11ea-8396-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=cii2sqUGqqIXyUMQJswpEp7YbOA1HrjdrYUaabsuQ94=; b=NVcU3iEluDtLRkLrSlEhx/5SNC lnvYOy5/XRKC77W1rv+QXAm0v6PRNcjR4JsE7tk0vJ6rbd+8hYDRXX4bFZbQ6lq6wdHOJtl42b9Nk z6aJ81Z09iDvqxmwomAy/tX7uN0PsszjUUpzlH9IWZUSFKFQ4Dl7vCgQH8A8M8hti20ALrGt68MDD Jxb23YF+WO3O1SKgK6YEf2MH1GKkhNBGYr1VXMmaqOTC1YZ5xXSMSFH3pAc17SPmJ9cr+HbesmSb+ 6bpaAhh1fs2GXeLg1vrfOXJIc75Ihho3698ghhy7rWbtt2412dfZRHxIz6eWiT0GB8bmnR/DhQKJs 9jMKcyjQ==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-00059w-F8; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kcr-7u; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:09 +0000 Message-Id: <20200130161330.2324143-1-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: References: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 01/22] x86/setup: Don't skip 2MiB underneath relocated Xen image X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Set 'e' correctly to reflect the location that Xen is actually relocated to from its default 2MiB location. Not 2MiB below that. This is only vaguely a bug fix. The "missing" 2MiB would have been used in the end, and fed to the allocator. It's just that other things don't get to sit right up *next* to the Xen image, and it isn't very tidy. For live update, I'd quite like a single contiguous region for the reserved bootmem and Xen, allowing the 'slack' in the former to be used when Xen itself grows larger. Let's not allow 2MiB of random heap pages to get in the way... Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index d858883404..2677f127b9 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1080,9 +1080,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) unsigned long pte_update_limit; /* Select relocation address. */ - e = end - reloc_size; - xen_phys_start = e; - bootsym(trampoline_xen_phys_start) = e; + xen_phys_start = end - reloc_size; + e = xen_phys_start + XEN_IMG_OFFSET; + bootsym(trampoline_xen_phys_start) = xen_phys_start; /* * No PTEs pointing above this address are candidates for relocation. @@ -1090,7 +1090,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) * and the beginning of region for destination image some PTEs may * point to addresses in range [e, e + XEN_IMG_OFFSET). */ - pte_update_limit = PFN_DOWN(e + XEN_IMG_OFFSET); + pte_update_limit = PFN_DOWN(e); /* * Perform relocation to new physical address. @@ -1099,7 +1099,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) * data until after we have switched to the relocated pagetables! */ barrier(); - move_memory(e + XEN_IMG_OFFSET, XEN_IMG_OFFSET, _end - _start, 1); + move_memory(e, XEN_IMG_OFFSET, _end - _start, 1); /* Walk initial pagetables, relocating page directory entries. */ pl4e = __va(__pa(idle_pg_table)); From patchwork Thu Jan 30 16:13:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35D2F112B for ; Thu, 30 Jan 2020 16:16:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 064AF20707 for ; Thu, 30 Jan 2020 16:16:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="h9HZ/LhI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 064AF20707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTW-0005P8-LU; Thu, 30 Jan 2020 16:15:26 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTV-0005O2-QY for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:15:25 +0000 X-Inumbo-ID: 78caeb58-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78caeb58-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Il/bpITmb6KpASVHrjGn6HaimgeHELl2mWb6bWqwlCo=; b=h9HZ/LhIe1tvrcpCSyBBVpT6Bc S8ULnWIUEmGKjEeNkqckCrnXZPNbEza0NmPlyreKAZVLeqP+C3XEGU/6vmHy9egYvdDEtHINCje7k d+KSD/7CwD7fbRgLLvzIkYWjfPHtsREX29IHHdJi5Hp/pgrP3OJQORexjkKc1SniY9rfdzxarLRT2 3y10jaY80FgWK+VTaWs0oCMqjO93y+sShv0YqQHqwnbyCToljBGhCLyZpQ60SALI6uuq1si9pQ6D7 teEZedP4Sg5NNXK9sYjN8C8b3hLmeD0RzcSW1sVV0UIHCjxeXCzOteZi4HMA6dBrbWlzjX9ckkjbz 6x2+vgww==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tf-2L; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kcu-8X; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:10 +0000 Message-Id: <20200130161330.2324143-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 02/22] x86/boot: Reserve live update boot memory X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse For live update to work, it will need a region of memory that can be given to the boot allocator while it parses the state information from the previous Xen and works out which of the other pages of memory it can consume. Reserve that like the crashdump region, and accept it on the command line. Use only that region for early boot, and register the remaining RAM (all of it for now, until the real live update happens) later. Signed-off-by: David Woodhouse --- docs/misc/xen-command-line.pandoc | 9 +++ xen/arch/x86/setup.c | 120 ++++++++++++++++++++++++++++-- xen/include/asm-x86/config.h | 1 + 3 files changed, 123 insertions(+), 7 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 5eb3a07276..61524f1056 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1402,6 +1402,15 @@ This option is intended for debugging purposes only. Enable MSR_DEBUGCTL.LBR in hypervisor context to be able to dump the Last Interrupt/Exception To/From record with other registers. +### liveupdate +> `= [@]` + +Specify size and optionally placement of the boot memory reserved for +Xen live update. The size must be a multiple of 2MiB. + +A trailing `@` specifies the exact address this area should be +placed at, which must be below 4GiB. + ### loglvl > `= [/]` where level is `none | error | warning | info | debug | all` diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 2677f127b9..63f06d4856 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -681,6 +681,47 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li /* How much of the directmap is prebuilt at compile time. */ #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT) +unsigned long lu_bootmem_start, lu_bootmem_size; +static unsigned long lu_breadcrumb_phys; + +static int __init parse_liveupdate(const char *str) +{ + const char *cur; + + lu_bootmem_size = parse_size_and_unit(cur = str, &str); + if ( !lu_bootmem_size || cur == str ) + return -EINVAL; + + if ( lu_bootmem_size & ((MB(2) - 1)) ) + { + printk(XENLOG_WARNING "Live update size must be a multiple of 2MiB\n"); + return -EINVAL; + } + + if (!*str) { + printk(XENLOG_INFO "Live update size 0x%lx\n", lu_bootmem_size); + return 0; + } + + if (*str != '@') + return -EINVAL; + + lu_bootmem_start = parse_size_and_unit(cur = str + 1, &str); + if ( !lu_bootmem_start || cur == str ) + return -EINVAL; + + printk(XENLOG_INFO "Live update area 0x%lx-0x%lx (0x%lx)\n", lu_bootmem_start, + lu_bootmem_start + lu_bootmem_size, lu_bootmem_size); + + /* + * If present, the breadcrumb leading to the migration data stream is + * in the very beginning of the reserved bootmem region. + */ + lu_breadcrumb_phys = lu_bootmem_start; + return 0; +} +custom_param("liveupdate", parse_liveupdate); + void __init noreturn __start_xen(unsigned long mbi_p) { char *memmap_type = NULL; @@ -690,7 +731,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) module_t *mod; unsigned long nr_pages, raw_max_page, modules_headroom, module_map[1]; int i, j, e820_warn = 0, bytes = 0; - bool acpi_boot_table_init_done = false, relocated = false; + bool acpi_boot_table_init_done = false, relocated = false, lu_reserved = false; int ret; struct ns16550_defaults ns16550 = { .data_bits = 8, @@ -980,6 +1021,22 @@ void __init noreturn __start_xen(unsigned long mbi_p) set_kexec_crash_area_size((u64)nr_pages << PAGE_SHIFT); kexec_reserve_area(&boot_e820); + if ( lu_bootmem_start ) + { + /* XX: Check it's in usable memory first */ + reserve_e820_ram(&boot_e820, lu_bootmem_start, lu_bootmem_start + lu_bootmem_size); + + /* Since it will already be out of the e820 map by the time the first + * loop over physical memory, map it manually already. */ + set_pdx_range(lu_bootmem_start >> PAGE_SHIFT, + (lu_bootmem_start + lu_bootmem_size) >> PAGE_SHIFT); + map_pages_to_xen((unsigned long)__va(lu_bootmem_start), + maddr_to_mfn(lu_bootmem_start), + PFN_DOWN(lu_bootmem_size), PAGE_HYPERVISOR); + + lu_reserved = true; + } + initial_images = mod; nr_initial_images = mbi->mods_count; @@ -1207,6 +1264,16 @@ void __init noreturn __start_xen(unsigned long mbi_p) printk("New Xen image base address: %#lx\n", xen_phys_start); } + /* Is the region suitable for the live update bootmem region? */ + if ( lu_bootmem_size && ! lu_bootmem_start && e < limit ) + { + end = consider_modules(s, e, lu_bootmem_size, mod, mbi->mods_count + relocated, -1); + if ( end ) + { + e = lu_bootmem_start = end - lu_bootmem_size; + } + } + /* Is the region suitable for relocating the multiboot modules? */ for ( j = mbi->mods_count - 1; j >= 0; j-- ) { @@ -1270,6 +1337,15 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( !xen_phys_start ) panic("Not enough memory to relocate Xen\n"); + if ( lu_bootmem_start ) + { + if ( !lu_reserved ) + reserve_e820_ram(&boot_e820, lu_bootmem_start, lu_bootmem_start + lu_bootmem_size); + printk("LU bootmem: 0x%lx - 0x%lx\n", lu_bootmem_start, lu_bootmem_start + lu_bootmem_size); + init_boot_pages(lu_bootmem_start, lu_bootmem_start + lu_bootmem_size); + lu_reserved = true; + } + /* This needs to remain in sync with xen_in_range(). */ reserve_e820_ram(&boot_e820, __pa(_stext), __pa(__2M_rwdata_end)); @@ -1281,8 +1357,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) xenheap_max_mfn(PFN_DOWN(highmem_start - 1)); /* - * Walk every RAM region and map it in its entirety (on x86/64, at least) - * and notify it to the boot allocator. + * Walk every RAM region and map it in its entirety and (unless in + * live update mode) notify it to the boot allocator. */ for ( i = 0; i < boot_e820.nr_map; i++ ) { @@ -1335,6 +1411,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) printk(XENLOG_WARNING "Ignoring inaccessible memory range" " %013"PRIx64"-%013"PRIx64"\n", s, e); + reserve_e820_ram(&boot_e820, s, e); continue; } map_e = e; @@ -1342,6 +1419,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) printk(XENLOG_WARNING "Ignoring inaccessible memory range" " %013"PRIx64"-%013"PRIx64"\n", e, map_e); + reserve_e820_ram(&boot_e820, e, map_e); } set_pdx_range(s >> PAGE_SHIFT, e >> PAGE_SHIFT); @@ -1352,7 +1430,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT); /* Pass mapped memory to allocator /before/ creating new mappings. */ - init_boot_pages(s, min(map_s, e)); + if ( !lu_reserved) + init_boot_pages(s, min(map_s, e)); + s = map_s; if ( s < map_e ) { @@ -1360,7 +1440,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) map_s = (s + mask) & ~mask; map_e &= ~mask; - init_boot_pages(map_s, map_e); + if ( !lu_reserved) + init_boot_pages(map_s, map_e); } if ( map_s > map_e ) @@ -1376,7 +1457,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) { map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), PFN_DOWN(end - map_e), PAGE_HYPERVISOR); - init_boot_pages(map_e, end); + if ( !lu_reserved) + init_boot_pages(map_e, end); map_e = end; } } @@ -1391,7 +1473,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) { map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s), PFN_DOWN(map_s - s), PAGE_HYPERVISOR); - init_boot_pages(s, map_s); + if ( !lu_reserved) + init_boot_pages(s, map_s); } } @@ -1489,6 +1572,29 @@ void __init noreturn __start_xen(unsigned long mbi_p) numa_initmem_init(0, raw_max_page); + if ( lu_bootmem_start ) + { + unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); + uint64_t mask = PAGE_SIZE - 1; + + for ( i = 0; i < boot_e820.nr_map; i++ ) + { + uint64_t s, e; + + if ( boot_e820.map[i].type != E820_RAM ) + continue; + s = (boot_e820.map[i].addr + mask) & ~mask; + e = (boot_e820.map[i].addr + boot_e820.map[i].size) & ~mask; + s = max_t(uint64_t, s, 1<<20); + if ( PFN_DOWN(s) > limit ) + continue; + if ( PFN_DOWN(e) > limit ) + e = pfn_to_paddr(limit); + + init_boot_pages(s, e); + } + } + if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) ) { unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); diff --git a/xen/include/asm-x86/config.h b/xen/include/asm-x86/config.h index d0cfbb70a8..d59e5101c3 100644 --- a/xen/include/asm-x86/config.h +++ b/xen/include/asm-x86/config.h @@ -279,6 +279,7 @@ extern unsigned char boot_edid_info[128]; #ifndef __ASSEMBLY__ extern unsigned long xen_phys_start; +extern unsigned long lu_bootmem_start, lu_bootmem_size; #endif /* GDT/LDT shadow mapping area. The first per-domain-mapping sub-area. */ From patchwork Thu Jan 30 16:13:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 245EC924 for ; Thu, 30 Jan 2020 16:14:54 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0040520707 for ; Thu, 30 Jan 2020 16:14:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="i620Il+T" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0040520707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCRq-000472-IR; Thu, 30 Jan 2020 16:13:42 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCRo-00046e-NR for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:13:40 +0000 X-Inumbo-ID: 7570db34-437b-11ea-8396-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7570db34-437b-11ea-8396-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=cAqt49tEv3OXC9NUP1Cdh946E3NHiyr/43ygC7DmgYI=; b=i620Il+ThnWYPuGaMbCAH5mEYm 522t9pFHowCWbDW6ZHGjBOHgmqQYgv6D/+j61Xv3zHrq8szmCDKgI2Qq+3acsyMTuFZbGEiRNDa5e FLbiYTtTglRKyorkZ/l45di7RD/rDZxgwYv3dILbhLpw8ZvFucu4k6mk87azfXglgKAE5401diR8d ykLtSQ2FqrwlsKbRrznKQ+1ga/Nx8bPUC0/t2nW/z9uWBFNfyBw5roREMz49FVOmS2kB2kxaVauIR IsAe7JvQppJQbpjSO5BSPrF0pO2JAYotdMZ2v+NqhdvtAsjWYskHb9tbtn2S4fC6+3cOIpTfAg3Re GlcG4Lyw==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-00059x-G8; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kcz-94; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:11 +0000 Message-Id: <20200130161330.2324143-3-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 03/22] Reserve live update memory regions X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse The live update handover requires that a region of memory be reserved for the new Xen to use in its boot allocator. The original Xen may use that memory but not for any pages which are mapped to domains, or which would need to be preserved across the live update for any other reason. The same constraints apply to initmem pages freed from the Xen image, since the new Xen will be loaded into the same physical location as the previous Xen. There is separate work ongoing which will make the xenheap meet this requirement by eliminating share_xen_page_with_guest(). For the meantime, just don't add those pages to the heap at all in the live update case. Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 12 ++++++++++- xen/common/page_alloc.c | 45 +++++++++++++++++++++++++++++++++++++++++ xen/include/xen/mm.h | 2 ++ 3 files changed, 58 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 63f06d4856..dba8c3f0a1 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -611,7 +611,7 @@ static void noinline init_done(void) } destroy_xen_mappings(start, end); - init_xenheap_pages(__pa(start), __pa(end)); + init_lu_reserved_pages(__pa(start), __pa(end)); printk("Freed %lukB init memory\n", (end - start) >> 10); startup_cpu_idle_loop(); @@ -1577,6 +1577,16 @@ void __init noreturn __start_xen(unsigned long mbi_p) unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); uint64_t mask = PAGE_SIZE - 1; + /* + * Pages in the reserved LU region must not be used for anything which + * will need to persist across a live update. There is ongoing work to + * eliminate or limit the use of share_xen_page_with_guest() and get + * to a point where we can actually honour that promise, but for now + * just *don't* add those pages to the heap. Clear the boot allocator + * out completely, before adding the non-reserved ranges. + */ + clear_boot_allocator(); + for ( i = 0; i < boot_e820.nr_map; i++ ) { uint64_t s, e; diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 919a270587..a74bf02559 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -1879,6 +1879,51 @@ void __init end_boot_allocator(void) printk("\n"); } +/* + * Called when live update is supported. The memory ranges currently + * still free in the boot allocator must be added to the reserved + * heap, distinct from the xenheap in that pages from it MUST NOT be + * used for anything which will be mapped to a domain or otherwise + * need to survive a live update. + */ +void __init clear_boot_allocator(void) +{ + unsigned int i; + + /* Add at least one range on node zero first, if we can. */ + for ( i = 0; i < nr_bootmem_regions; i++ ) + { + struct bootmem_region *r = &bootmem_region_list[i]; + if ( (r->s < r->e) && + (phys_to_nid(pfn_to_paddr(r->s)) == cpu_to_node(0)) ) + { + init_lu_reserved_pages(r->s << PAGE_SHIFT, r->e << PAGE_SHIFT); + r->e = r->s; + break; + } + } + for ( i = nr_bootmem_regions; i-- > 0; ) + { + struct bootmem_region *r = &bootmem_region_list[i]; + if ( r->s < r->e ) + init_lu_reserved_pages(r->s << PAGE_SHIFT, r->e << PAGE_SHIFT); + } + nr_bootmem_regions = 0; +} + +void init_lu_reserved_pages(paddr_t ps, paddr_t pe) +{ + if (!lu_bootmem_start) + init_xenheap_pages(ps, pe); + + /* There is ongoing work for other reasons to eliminate the use of + * share_xen_page_with_guest() and get to a point where the normal + * xenheap actually meets the requirement we need for live update + * reserved memory, that nothing allocated from it will be mapped + * to a guest and/or need to be preserved over a live update. + * Until then, we simply don't use these pages after boot. */ +} + static void __init smp_scrub_heap_pages(void *data) { unsigned long mfn, start, end; diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index d0d095d9c7..d120d84d23 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -158,8 +158,10 @@ struct domain *__must_check page_get_owner_and_reference(struct page_info *); void init_boot_pages(paddr_t ps, paddr_t pe); mfn_t alloc_boot_pages(unsigned long nr_pfns, unsigned long pfn_align); void end_boot_allocator(void); +void clear_boot_allocator(void); /* Xen suballocator. These functions are interrupt-safe. */ +void init_lu_reserved_pages(paddr_t ps, paddr_t pe); void init_xenheap_pages(paddr_t ps, paddr_t pe); void xenheap_max_mfn(unsigned long mfn); void *alloc_xenheap_pages(unsigned int order, unsigned int memflags); From patchwork Thu Jan 30 16:13:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D324112B for ; Thu, 30 Jan 2020 16:14:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58C9520707 for ; Thu, 30 Jan 2020 16:14:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ZFwA8Wod" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58C9520707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSA-0004Fr-4x; Thu, 30 Jan 2020 16:14:02 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCS8-0004FI-NM for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:00 +0000 X-Inumbo-ID: 7576cf8a-437b-11ea-b211-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7576cf8a-437b-11ea-b211-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=1y2PqV6WbRypv8RR0KrTco7MDDMIC2Om05DEmv5EIF0=; b=ZFwA8WodtpJ3kkyXCPUYsVDM+x Kh0FnAgUUUGurXeiBJz67Kahnvj9Y7//nIYEQnN1+bNF2dUZw3d0K6CEdR+otm5giZOiQfoTjAgPA /iSZPv1NFzIBh59TJkXMPeaUFVqB3gbjE4WPLeAjqOvdOyIM69TUaT8DWWPEMRaF+RV36YqoHI0Lu udXYm7zxlKtsh98nkNxdGbO6OzD4+KmAacWd/Rua0tDqz9Kp3dFqecb6tz9yzHEcdhighY8uNUL3l rB8JGpMRA7jYOD361W7Qq68f0ThkkncyyF0lAwP3LjCB9GMpQ664vpXxG53OyFy5E6RkBsJs4p+lK X5B6IOYw==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-00059y-Gp; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kd4-9n; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:12 +0000 Message-Id: <20200130161330.2324143-4-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 04/22] Add KEXEC_RANGE_MA_LIVEUPDATE X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse This allows kexec userspace to tell the next Xen where the range is, on its command line. Signed-off-by: David Woodhouse --- xen/arch/x86/machine_kexec.c | 13 ++++++++++--- xen/include/public/kexec.h | 1 + 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/machine_kexec.c b/xen/arch/x86/machine_kexec.c index b70d5a6a86..273bc20664 100644 --- a/xen/arch/x86/machine_kexec.c +++ b/xen/arch/x86/machine_kexec.c @@ -186,9 +186,16 @@ void machine_kexec(struct kexec_image *image) int machine_kexec_get(xen_kexec_range_t *range) { - if (range->range != KEXEC_RANGE_MA_XEN) - return -EINVAL; - return machine_kexec_get_xen(range); + switch (range->range) { + case KEXEC_RANGE_MA_XEN: + return machine_kexec_get_xen(range); + case KEXEC_RANGE_MA_LIVEUPDATE: + range->start = lu_bootmem_start; + range->size = lu_bootmem_size; + return 0; + default: + return -EINVAL; + } } void arch_crash_save_vmcoreinfo(void) diff --git a/xen/include/public/kexec.h b/xen/include/public/kexec.h index 3f2a118381..298381af8d 100644 --- a/xen/include/public/kexec.h +++ b/xen/include/public/kexec.h @@ -150,6 +150,7 @@ typedef struct xen_kexec_load_v1 { #define KEXEC_RANGE_MA_EFI_MEMMAP 5 /* machine address and size of * of the EFI Memory Map */ #define KEXEC_RANGE_MA_VMCOREINFO 6 /* machine address and size of vmcoreinfo */ +#define KEXEC_RANGE_MA_LIVEUPDATE 7 /* Boot mem for live update */ /* * Find the address and size of certain memory areas From patchwork Thu Jan 30 16:13:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3654B14E3 for ; Thu, 30 Jan 2020 16:16:06 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1229C20707 for ; Thu, 30 Jan 2020 16:16:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ly+2Ete9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1229C20707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCT8-00050a-9g; Thu, 30 Jan 2020 16:15:02 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCT6-0004zM-Oq for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:15:00 +0000 X-Inumbo-ID: 78caef0e-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78caef0e-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=qmmRLb0NdTnw1b2mbJsYwt47rxkECNQNxPoVyBberGo=; b=Ly+2Ete9oEEq6xuSf8WUSJoHg1 yI3+YrqQJmxvKaUy/TDj/ee/TX5jCtfPanff4H89hrLiCSc4KuuJ8yoQGdMiK7a1THuF8UAlb//lc Z/LCDyOghyXNJYEytQi0rMSpCy5mMfOJpBfv7bc703Yb6VQm7QhCKTHstiteTyOQQLbw0s9+caHZS gXu2woPTzxMwm29gvGkBXbwdDDI+kxJZbbI/ELVN3CQyEMQ6UF3W7mRavGnpdjQHn5QkpSTR6bcB6 1sAfEVjTrDc9XpMwe9dl2U2JjA9EBthh4AuTWx4mbLnvcjf9ru1FGDenSVMUk0IBuC7N5rrb3FWif XKBbf/ZA==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tg-42; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kd9-AK; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:13 +0000 Message-Id: <20200130161330.2324143-5-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 05/22] Add KEXEC_TYPE_LIVE_UPDATE X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse This is identical to the default case... for now. Signed-off-by: David Woodhouse --- xen/common/kexec.c | 18 ++++++++++++++++++ xen/include/public/kexec.h | 12 ++++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/xen/common/kexec.c b/xen/common/kexec.c index a262cc5a18..a78aa4f5b0 100644 --- a/xen/common/kexec.c +++ b/xen/common/kexec.c @@ -404,6 +404,19 @@ static long kexec_reboot(void *_image) return 0; } +static long kexec_live_update(void *_image) +{ + struct kexec_image *image = _image; + + kexecing = TRUE; + + kexec_common_shutdown(); + machine_reboot_kexec(image); + + BUG(); + return 0; +} + static void do_crashdump_trigger(unsigned char key) { printk("'%c' pressed -> triggering crashdump\n", key); @@ -736,6 +749,7 @@ static int kexec_load_get_bits(int type, int *base, int *bit) switch ( type ) { case KEXEC_TYPE_DEFAULT: + case KEXEC_TYPE_LIVE_UPDATE: *base = KEXEC_IMAGE_DEFAULT_BASE; *bit = KEXEC_FLAG_DEFAULT_POS; break; @@ -837,6 +851,10 @@ static int kexec_exec(XEN_GUEST_HANDLE_PARAM(void) uarg) image = kexec_image[base + pos]; ret = continue_hypercall_on_cpu(0, kexec_reboot, image); break; + case KEXEC_TYPE_LIVE_UPDATE: + image = kexec_image[base + pos]; + ret = continue_hypercall_on_cpu(0, kexec_live_update, image); + break; case KEXEC_TYPE_CRASH: kexec_crash(); /* Does not return */ break; diff --git a/xen/include/public/kexec.h b/xen/include/public/kexec.h index 298381af8d..f5230286d3 100644 --- a/xen/include/public/kexec.h +++ b/xen/include/public/kexec.h @@ -71,18 +71,22 @@ */ /* - * Kexec supports two types of operation: + * Kexec supports three types of operation: * - kexec into a regular kernel, very similar to a standard reboot * - KEXEC_TYPE_DEFAULT is used to specify this type * - kexec into a special "crash kernel", aka kexec-on-panic * - KEXEC_TYPE_CRASH is used to specify this type * - parts of our system may be broken at kexec-on-panic time * - the code should be kept as simple and self-contained as possible + * - Live update into a new Xen, preserving all running domains + * - KEXEC_TYPE_LIVE_UPDATE is used to specify this type + * - Xen performs guest-transparent live migration and stores live + * update state in memory, passing it to the new Xen. */ -#define KEXEC_TYPE_DEFAULT 0 -#define KEXEC_TYPE_CRASH 1 - +#define KEXEC_TYPE_DEFAULT 0 +#define KEXEC_TYPE_CRASH 1 +#define KEXEC_TYPE_LIVE_UPDATE 2 /* The kexec implementation for Xen allows the user to load two * types of kernels, KEXEC_TYPE_DEFAULT and KEXEC_TYPE_CRASH. From patchwork Thu Jan 30 16:13:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB018139A for ; Thu, 30 Jan 2020 16:15:52 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B492E20707 for ; Thu, 30 Jan 2020 16:15:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="omlMB7VK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B492E20707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSy-0004rS-FN; Thu, 30 Jan 2020 16:14:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSw-0004qG-Op for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:50 +0000 X-Inumbo-ID: 78cadfaa-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78cadfaa-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=p233OatuRQuoWS6LOUmWPlwIXjbBEnNCI12DOuMnnC8=; b=omlMB7VKOb0lN6S+/niIOHxA1Q 0aONWvBgNIP1l53seXDxs1oOkiPDyw/T4IIFi+zILxo78iu3I8iRr49H9Cylq+dIX8snjOgQSO0nO S1jLhBS34GCFZYdvih2vAzFPazTAEmi982WhXo0t6AjnbLR1gzRKCupsgSFwTK9c72Y1YXZ5hHQq9 z4EalRZtO67jl72qBcteomsg34vc+LmiEafd0Gy/eZ3yCjcdK7zxpHpxKfhGvET8ZCrdC/q71f7ae DQQKDNfJnAz0bC/Q1vSY1joVlvpR5/2AxbllScAzO9K6q/fVf/jdY1gILaVKN8QLb4jgdGlSeO1lg pQVXkILw==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Th-4V; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdE-At; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:14 +0000 Message-Id: <20200130161330.2324143-6-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 06/22] Add IND_WRITE64 primitive to kexec kimage X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse This allows a single page-aligned physical address to be written to the current destination, intended to pass the location of the live update data stream from one Xen to the next. Signed-off-by: David Woodhouse --- xen/arch/x86/x86_64/kexec_reloc.S | 9 ++++++++- xen/include/xen/kimage.h | 1 + 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/x86_64/kexec_reloc.S b/xen/arch/x86/x86_64/kexec_reloc.S index d488d127cf..a75f989926 100644 --- a/xen/arch/x86/x86_64/kexec_reloc.S +++ b/xen/arch/x86/x86_64/kexec_reloc.S @@ -131,11 +131,18 @@ is_source: jmp next_entry is_zero: testb $IND_ZERO, %cl - jz next_entry + jz is_write64 movl $(PAGE_SIZE / 8), %ecx /* Zero the destination page. */ xorl %eax, %eax rep stosq jmp next_entry +is_write64: + testb $IND_WRITE64, %cl + jz next_entry + andq $PAGE_MASK, %rcx + movq %rcx, %rax + stosq + jmp next_entry done: popq %rbx ret diff --git a/xen/include/xen/kimage.h b/xen/include/xen/kimage.h index cbfb9e9054..e94839d7c3 100644 --- a/xen/include/xen/kimage.h +++ b/xen/include/xen/kimage.h @@ -6,6 +6,7 @@ #define IND_DONE 0x4 #define IND_SOURCE 0x8 #define IND_ZERO 0x10 +#define IND_WRITE64 0x20 #ifndef __ASSEMBLY__ From patchwork Thu Jan 30 16:13:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5ABE7139A for ; Thu, 30 Jan 2020 16:16:17 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 36B5620707 for ; Thu, 30 Jan 2020 16:16:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="uQsXGNFK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36B5620707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSP-0004Pt-AI; Thu, 30 Jan 2020 16:14:17 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSN-0004Ob-NX for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:15 +0000 X-Inumbo-ID: 758ad0ca-437b-11ea-a933-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 758ad0ca-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=P1JBbhQ8GJUIqzOoGTJM/rEnqMJxaYbZHtfoRu2ij9A=; b=uQsXGNFKRAQjhQu4gczJ6ykM9a qPjTFzzOeB9kNOSVOEqiXHoz7Nxim1uXoriSwyAaUfQaB13zbt1GbJh0/3LRJCvp+DGTkGPmSLdj5 0PcOOTeXeYXqCv5nW5pter/mQUrNiYpDEQw7XMrWnln7TRl6b3GV/wOG31Lr8brNcWVpbwqYtnb7R caK2BmEujfXz3uqMyDQhmqvqDwREvwUDo0oZ9aLW/kg/CneCXLhINiot5mBIgxML4kTXLeSiJtppW 5XPBvWwTS3vV5bu+51jPjcwYm5YvFKplnWrZShX0nPRoaYy3iVYuX4XkXx51UseIZAVJYZCr0Axzs U31SFj3w==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-00059z-IS; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdJ-Ba; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:15 +0000 Message-Id: <20200130161330.2324143-7-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 07/22] Add basic live update stream creation X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/common/Makefile | 1 + xen/common/lu/Makefile | 1 + xen/common/lu/stream.c | 135 +++++++++++++++++++++++++++++++++++++++++ xen/include/xen/lu.h | 29 +++++++++ 4 files changed, 166 insertions(+) create mode 100644 xen/common/lu/Makefile create mode 100644 xen/common/lu/stream.c create mode 100644 xen/include/xen/lu.h diff --git a/xen/common/Makefile b/xen/common/Makefile index 2abb8250b0..60502bb909 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -72,3 +72,4 @@ subdir-$(CONFIG_UBSAN) += ubsan subdir-$(CONFIG_NEEDS_LIBELF) += libelf subdir-$(CONFIG_HAS_DEVICE_TREE) += libfdt +subdir-y += lu diff --git a/xen/common/lu/Makefile b/xen/common/lu/Makefile new file mode 100644 index 0000000000..68991b3ca4 --- /dev/null +++ b/xen/common/lu/Makefile @@ -0,0 +1 @@ +obj-y += stream.o diff --git a/xen/common/lu/stream.c b/xen/common/lu/stream.c new file mode 100644 index 0000000000..10e123a466 --- /dev/null +++ b/xen/common/lu/stream.c @@ -0,0 +1,135 @@ +/* + * Live update data stream handling. + * + * During live update, one version of Xen (Xen#1) performs a kexec into + * a new version of Xen (Xen#2), performing guest-transparent live + * migration of all existing domains. + * + * Xen#2 must avoid scribbling on any pages which may belong to existing + * domains. In order to achieve this, we reserve a contiguous area of + * physical memory to be used by the boot allocator in Xen#2. Xen must + * not allocate pages from that region which are later shared with + * guests or need to persist across live update. + * + * The live update bootmem region is reserved by the first Xen to boot, + * and userspace can obtain its address using KEXEC_CMD_kexec_get_range + * with the new KEXEC_RANGE_MA_LIVEUPDATE type. Userspace kexec(8) + * appends the appropriate 'liveupdate=' parameter to the command line + * of Xen#2 when setting up the kexec image. + * + * At the time of kexec, Xen#1 serialises the domain state into buffers + * allocated from its own heap., then creates a single physically + * contiguous scatter-gather list containing the MFNs of those data + * pages (which Xen#2 can then trivially vmap()). In a system with + * 4KiB pages, the MFN list for the live update data stream will fit + * into a single page until the total size of the live update data + * exceeds 2MiB. + * + * The physical address of the MFN list is passed to Xen#2 by placing + * it at the start of the reserved live update bootmem region, with a + * magic number to avoid false positives. + */ + +#include +#include +#include + +static int lu_stream_extend(struct lu_stream *stream, int nr_pages) +{ + int order = get_order_from_bytes((nr_pages + 1) * sizeof(mfn_t)); + int old_order = get_order_from_bytes((stream->nr_pages + 1) * sizeof(mfn_t)); + + if ( !stream->nr_pages || order > old_order ) + { + mfn_t *new_pglist = alloc_xenheap_pages(order, 0); + + if ( !new_pglist ) + return -ENOMEM; + + if ( stream->nr_pages ) + { + memcpy(new_pglist, stream->pagelist, + stream->nr_pages * sizeof(mfn_t)); + free_xenheap_pages(stream->pagelist, old_order); + } + stream->pagelist = new_pglist; + } + while ( stream->nr_pages < nr_pages ) + { + struct page_info *pg = alloc_domheap_page(NULL, MEMF_no_owner); + + if ( !pg ) + { + /* Ensure the cleanup frees the correct order of pagelist */ + stream->nr_pages++; + + return -ENOMEM; + } + stream->pagelist[stream->nr_pages++] = page_to_mfn(pg); + stream->pagelist[stream->nr_pages] = INVALID_MFN; + } + + if ( stream->data ) + vunmap(stream->data); + stream->data = vmap(stream->pagelist, stream->nr_pages); + if ( !stream->data ) + return -ENOMEM; + + return 0; +} + +void *lu_stream_reserve(struct lu_stream *stream, size_t size) +{ + int nr_pages = (stream->len + size + PAGE_SIZE - 1) >> PAGE_SHIFT; + + if ( stream->nr_pages < nr_pages && lu_stream_extend(stream, nr_pages) ) + return NULL; + + return stream->data + stream->len; +} + +void lu_stream_end_reservation(struct lu_stream *stream, size_t size) +{ + stream->len += size; +} + +int lu_stream_append(struct lu_stream *stream, const void *data, size_t size) +{ + void *p = lu_stream_reserve(stream, size); + + if ( !p ) + return -ENOMEM; + memcpy(p, data, size); + lu_stream_end_reservation(stream, size); + + return 0; +} + +void lu_stream_free(struct lu_stream *stream) +{ + unsigned int order = get_order_from_bytes((stream->nr_pages + 1) * sizeof(mfn_t)); + unsigned int i; + + if ( stream->data ) + vunmap(stream->data); + + if ( stream->pagelist ) + { + for ( i = 0; i < stream->nr_pages; i++ ) + { + if (mfn_valid(stream->pagelist[i])) + free_domheap_page(mfn_to_page(stream->pagelist[i])); + } + free_xenheap_pages(stream->pagelist, order); + } +} + +/* + * local variables: + * mode: c + * c-file-style: "bsd" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * end: + */ diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h new file mode 100644 index 0000000000..abb30545fe --- /dev/null +++ b/xen/include/xen/lu.h @@ -0,0 +1,29 @@ +#ifndef __XEN_LU_H__ +#define __XEN_LU_H__ + +#include +#include + +struct lu_stream { + mfn_t *pagelist; + size_t len; + int nr_pages; + char *data; +}; + +void *lu_stream_reserve(struct lu_stream *stream, size_t size); +void lu_stream_end_reservation(struct lu_stream *stream, size_t size); +int lu_stream_append(struct lu_stream *stream, const void *data, size_t size); +void lu_stream_free(struct lu_stream *stream); + +#endif /* __XEN_LU_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Thu Jan 30 16:13:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358321 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0B2C112B for ; Thu, 30 Jan 2020 16:16:43 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CCC8320707 for ; Thu, 30 Jan 2020 16:16:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="0i7G+cHL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCC8320707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCT3-0004wC-Rc; Thu, 30 Jan 2020 16:14:57 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCT1-0004uO-OI for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:55 +0000 X-Inumbo-ID: 78cad7e4-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78cad7e4-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=j0bRNgj/mkTr+3RVAh2A0VZOBTd/VSZMQJdSs4KPZ0I=; b=0i7G+cHLZGpPJpeYPH6bFwyPU7 ZGIAtDNG7ulsMnGt0hr5a7vi7jPXD4O7dOHquOAPPKbu4kbz3dRJ1Rm4jy/fGKYoQ7RHh5Tu0ww+t 91QWMXlbs1zuEfDcMb4/NbNfiudsIEvHyxTYy779YOUJWaGF4RFpIV+DaojGSjCDGEy4MJ5ZSMZW8 cCRSvcw8f4bfnE1ZU1V/m+oZh5cPa/1fwcdwVsfothtsXJ0ObY6VmGnEbPKo8+S5nowy+4qQKKa4y 6W4+hC0CMdXdVSns2LfrVcFdg3nO0V8kEcn0xK8/PvHXWnv2uc71+M2CFieUUxp+zG9m76TpoFy6B BhaE+EPQ==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Ti-63; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdO-CI; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:16 +0000 Message-Id: <20200130161330.2324143-8-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 08/22] Add kimage_add_live_update_data() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/common/kimage.c | 34 ++++++++++++++++++++++++++++++++++ xen/include/xen/kimage.h | 3 +++ xen/include/xen/lu.h | 3 +++ 3 files changed, 40 insertions(+) diff --git a/xen/common/kimage.c b/xen/common/kimage.c index 210241dfb7..86d2797cbc 100644 --- a/xen/common/kimage.c +++ b/xen/common/kimage.c @@ -20,6 +20,7 @@ #include #include #include +#include #include @@ -938,6 +939,39 @@ done: return ret; } +int kimage_add_live_update_data(struct kexec_image *image, mfn_t data, int nr_mfns) +{ + int ret; + + /* + * For live update, we place the physical location of 'data' + * into the first 64 bits of the reserved live update bootmem + * region. At 'data' is an MFN list of pages containing the + * actual live update stream, which the new Xen can vmap(). + * + * Append IND_WRITE64 operations to the end of the kimage stream + * to store the live update magic and the address of 'data' for + * the new Xen to see. + */ + if (!lu_bootmem_start || kimage_dst_used(image, lu_bootmem_start)) + return -EINVAL; + + ret = machine_kexec_add_page(image, lu_bootmem_start, lu_bootmem_start); + if ( ret < 0 ) + return ret; + + ret = kimage_set_destination(image, lu_bootmem_start); + if (!ret) + ret = kimage_add_entry(image, LIVE_UPDATE_MAGIC | IND_WRITE64); + if (!ret) + ret = kimage_add_entry(image, mfn_to_maddr(data) | IND_WRITE64); + if (!ret) + ret = kimage_add_entry(image, (nr_mfns << PAGE_SHIFT) | IND_WRITE64); + + kimage_terminate(image); + + return ret; +} /* * Local variables: * mode: C diff --git a/xen/include/xen/kimage.h b/xen/include/xen/kimage.h index e94839d7c3..1e0e378afd 100644 --- a/xen/include/xen/kimage.h +++ b/xen/include/xen/kimage.h @@ -54,6 +54,9 @@ unsigned long kimage_entry_ind(kimage_entry_t *entry, bool_t compat); int kimage_build_ind(struct kexec_image *image, mfn_t ind_mfn, bool_t compat); +int kimage_add_live_update_data(struct kexec_image *image, mfn_t data, + int nr_mfns); + #endif /* __ASSEMBLY__ */ #endif /* __XEN_KIMAGE_H__ */ diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h index abb30545fe..21ee1825d3 100644 --- a/xen/include/xen/lu.h +++ b/xen/include/xen/lu.h @@ -1,9 +1,12 @@ #ifndef __XEN_LU_H__ #define __XEN_LU_H__ + #include #include +#define LIVE_UPDATE_MAGIC (0x4c69766555706461UL & PAGE_MASK) + struct lu_stream { mfn_t *pagelist; size_t len; From patchwork Thu Jan 30 16:13:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 401C6112B for ; Thu, 30 Jan 2020 16:15:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C15920707 for ; Thu, 30 Jan 2020 16:15:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="zgFQVEzg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C15920707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSo-0004io-DV; Thu, 30 Jan 2020 16:14:42 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSm-0004hg-NL for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:40 +0000 X-Inumbo-ID: 78caf2ba-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78caf2ba-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ZQFPxvjbG/EwWLKgg6TYeH0pv/puTiqYoV1kR249sSU=; b=zgFQVEzgsQJyVaVsS8xoOhqMF4 Ab/RzYf/MeImA12hEreW19krDoLRE/piY5yiR9ZDnDTyGnG+Adv4puHGY4WXK0boQSyPVRNW17m1s Ay8ayT3KedPQYdxAcYQuHw08igydv+I2w1t5uRO+5F0CoKVrxdVeQd2oscLHwkhtQQIVGTSY++9Fr +eJ8gedinb0A10nk0NlIm4M8QY11gS1pHymebjwdWt+FZ76V6go08zEeiGyyogAZukLm/QI72IMKR Hm5ctlb+27a/plOklIVKew0Y5mORR7OF/FgEypP4hZ33vhV8hftuGin2DguOQpkW4uuJy2eV2u0WO RFS6t1Vw==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tj-6S; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdT-Co; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:17 +0000 Message-Id: <20200130161330.2324143-9-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 09/22] Add basic lu_save_all() shell X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse --- xen/common/kexec.c | 6 +++++ xen/common/lu/Makefile | 2 +- xen/common/lu/save.c | 56 ++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/lu.h | 3 +++ 4 files changed, 66 insertions(+), 1 deletion(-) create mode 100644 xen/common/lu/save.c diff --git a/xen/common/kexec.c b/xen/common/kexec.c index a78aa4f5b0..658fe3d3d4 100644 --- a/xen/common/kexec.c +++ b/xen/common/kexec.c @@ -29,6 +29,7 @@ #include #include #include +#include #ifdef CONFIG_COMPAT #include #endif @@ -407,6 +408,11 @@ static long kexec_reboot(void *_image) static long kexec_live_update(void *_image) { struct kexec_image *image = _image; + int ret; + + ret = lu_save_all(image); + if (ret) + return ret; kexecing = TRUE; diff --git a/xen/common/lu/Makefile b/xen/common/lu/Makefile index 68991b3ca4..7b7d975f65 100644 --- a/xen/common/lu/Makefile +++ b/xen/common/lu/Makefile @@ -1 +1 @@ -obj-y += stream.o +obj-y += stream.o save.o diff --git a/xen/common/lu/save.c b/xen/common/lu/save.c new file mode 100644 index 0000000000..c43962c44e --- /dev/null +++ b/xen/common/lu/save.c @@ -0,0 +1,56 @@ + +#include +#include +#include +#include +#include + +int lu_save_global(struct lu_stream *stream) +{ + return 0; +} + + +int lu_save_domain(struct lu_stream *stream, struct domain *d) +{ + return 0; +} + +int lu_save_all(struct kexec_image *image) +{ + struct lu_stream stream; + struct domain *d; + int ret; + + memset(&stream, 0, sizeof(stream)); + + ret = lu_save_global(&stream); + + for_each_domain ( d ) + { + if (ret) + break; + + ret = lu_save_domain(&stream, d); + } + + if (!ret) + ret = kimage_add_live_update_data(image, + _mfn(virt_to_mfn(stream.pagelist)), + stream.nr_pages); + + if (ret) + lu_stream_free(&stream); + + return ret; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h index 21ee1825d3..f232a56950 100644 --- a/xen/include/xen/lu.h +++ b/xen/include/xen/lu.h @@ -19,6 +19,9 @@ void lu_stream_end_reservation(struct lu_stream *stream, size_t size); int lu_stream_append(struct lu_stream *stream, const void *data, size_t size); void lu_stream_free(struct lu_stream *stream); +struct kexec_image; +int lu_save_all(struct kexec_image *image); + #endif /* __XEN_LU_H__ */ /* From patchwork Thu Jan 30 16:13:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 75C50112B for ; Thu, 30 Jan 2020 16:15:19 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5137520707 for ; Thu, 30 Jan 2020 16:15:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="JFyYXbnV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5137520707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSd-0004aV-KV; Thu, 30 Jan 2020 16:14:31 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSc-0004Zw-NO for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:30 +0000 X-Inumbo-ID: 757f79b4-437b-11ea-b211-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 757f79b4-437b-11ea-b211-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=7m1dAFoMUtNjEe58VPqWPcnanUtrN5ymOelPdQ5HWM8=; b=JFyYXbnVd5Wp0vUhUeRFtFK6Gx c1yAQUwr3Fn9fI0gQzYM350wAdb8K8B9iYdCNltdRl/3KL8RoMH6sgY2/6aFgy3qLmBxebBGhitms u7gt/G6exj6OLjam4gumJ8i9ya8ffMkYMBpD3YYCbMYuR9lcxEm+9USSHUVZzqGqwxIpMW+FVOqp2 tCWJ5bSIBYOe0xTPFzWdXCGU/tZ0loXZaz4wyvDRUx34FNlkWiZrBafFvX0P8iqHqhDMQwdcKdkw1 3aJQubmXR2b93rzWSVihImqgiY7zM36D7oGbKgztCsWeQLWsso45uapXqGK4r0HUuytL0ZLJEs0OA wSQlN/pg==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A0-Ku; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdY-DY; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:18 +0000 Message-Id: <20200130161330.2324143-10-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 10/22] Don't add bad pages above HYPERVISOR_VIRT_END to the domheap X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/common/page_alloc.c | 83 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 3 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index a74bf02559..4ada4412dd 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -1758,6 +1758,18 @@ int query_page_offline(mfn_t mfn, uint32_t *status) return 0; } +static unsigned long contig_avail_pages(struct page_info *pg, unsigned long max_pages) +{ + unsigned long i; + + for ( i = 0 ; i < max_pages; i++) + { + if ( pg[i].count_info & (PGC_broken | PGC_allocated) ) + break; + } + return i; +} + /* * Hand the specified arbitrary page range to the specified heap zone * checking the node_id of the previous page. If they differ and the @@ -1799,18 +1811,24 @@ static void init_heap_pages( { unsigned int nid = phys_to_nid(page_to_maddr(pg+i)); + /* If the (first) page is already marked bad, or allocated in advance + * due to live update, don't add it to the heap. */ + if (pg[i].count_info & (PGC_broken | PGC_allocated)) + continue; + if ( unlikely(!avail[nid]) ) { + unsigned long contig_nr_pages = contig_avail_pages(pg + i, nr_pages); unsigned long s = mfn_x(page_to_mfn(pg + i)); - unsigned long e = mfn_x(mfn_add(page_to_mfn(pg + nr_pages - 1), 1)); + unsigned long e = mfn_x(mfn_add(page_to_mfn(pg + i + contig_nr_pages - 1), 1)); bool use_tail = (nid == phys_to_nid(pfn_to_paddr(e - 1))) && !(s & ((1UL << MAX_ORDER) - 1)) && (find_first_set_bit(e) <= find_first_set_bit(s)); unsigned long n; - n = init_node_heap(nid, mfn_x(page_to_mfn(pg + i)), nr_pages - i, + n = init_node_heap(nid, mfn_x(page_to_mfn(pg + i)), contig_nr_pages, &use_tail); - BUG_ON(i + n > nr_pages); + BUG_ON(n > contig_nr_pages); if ( n && !use_tail ) { i += n - 1; @@ -1846,6 +1864,63 @@ static unsigned long avail_heap_pages( return free_pages; } +static void mark_bad_pages(void) +{ + unsigned long bad_spfn, bad_epfn; + const char *p; + struct page_info *pg; +#ifdef CONFIG_X86 + const struct platform_bad_page *badpage; + unsigned int i, j, array_size; + + badpage = get_platform_badpages(&array_size); + if ( badpage ) + { + for ( i = 0; i < array_size; i++ ) + { + for ( j = 0; j < 1UL << badpage->order; j++ ) + { + if ( mfn_valid(_mfn(badpage->mfn + j)) ) + { + pg = mfn_to_page(_mfn(badpage->mfn + j)); + pg->count_info |= PGC_broken; + page_list_add_tail(pg, &page_broken_list); + } + } + } + } +#endif + + /* Check new pages against the bad-page list. */ + p = opt_badpage; + while ( *p != '\0' ) + { + bad_spfn = simple_strtoul(p, &p, 0); + bad_epfn = bad_spfn; + + if ( *p == '-' ) + { + p++; + bad_epfn = simple_strtoul(p, &p, 0); + if ( bad_epfn < bad_spfn ) + bad_epfn = bad_spfn; + } + + if ( *p == ',' ) + p++; + else if ( *p != '\0' ) + break; + + while ( mfn_valid(_mfn(bad_spfn)) && bad_spfn < bad_epfn ) + { + pg = mfn_to_page(_mfn(bad_spfn)); + pg->count_info |= PGC_broken; + page_list_add_tail(pg, &page_broken_list); + bad_spfn++; + } + } +} + void __init end_boot_allocator(void) { unsigned int i; @@ -1870,6 +1945,8 @@ void __init end_boot_allocator(void) } nr_bootmem_regions = 0; + mark_bad_pages(); + if ( !dma_bitsize && (num_online_nodes() > 1) ) dma_bitsize = arch_get_dma_bitsize(); From patchwork Thu Jan 30 16:13:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF027112B for ; Thu, 30 Jan 2020 16:16:07 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CB07020707 for ; Thu, 30 Jan 2020 16:16:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="wx+jdvar" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB07020707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTC-00054M-Ot; Thu, 30 Jan 2020 16:15:06 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTB-00053f-OL for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:15:05 +0000 X-Inumbo-ID: 78c916f2-437b-11ea-b211-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78c916f2-437b-11ea-b211-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=L2O922a9WZmlPredpfqcFeznyIaN0AM+ucM06nCll7Q=; b=wx+jdvarg4k55FV/CqRUtTbbkW Im1xINsAXlL4yIrSs3M2o0iZsMDtAij65qm4e0yudCktQ44WjruFRhR2GjEIcRejPdAXDNYVtZBqs yBG6+/3Djx1VXRzU6DloekHRHmbGlUXF8yWc4b7usn9/IQd1gK6v1pl47pUcrJ3+MUbyxsrwlgzME bVmwNZ9ZEsuGfLtSraROTS3bhXF8EhPjBPQAPnnG1PwmW3POIUc7vpzfTzHVKDpOArkwxpTLz+V+n moeRWdtN9PGYq+5lmm6XNHhA7ur7LDUW6QByU5x/0pqP8ewBNuGmM/BJrhbOpdqYdHUzQz6CbwGA4 VkntaJMg==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tk-7v; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdd-E9; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:19 +0000 Message-Id: <20200130161330.2324143-11-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 11/22] xen/vmap: allow vm_init_type to be called during early_boot X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Wei Liu We want to move vm_init, which calls vm_init_type under the hood, to early boot stage. Add a path to get page from boot allocator instead. Add an emacs block to that file while I was there. Signed-off-by: Wei Liu --- xen/common/vmap.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/xen/common/vmap.c b/xen/common/vmap.c index faebc1ddf1..37922f735b 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -34,9 +34,15 @@ void __init vm_init_type(enum vmap_region type, void *start, void *end) for ( i = 0, va = (unsigned long)vm_bitmap(type); i < nr; ++i, va += PAGE_SIZE ) { - struct page_info *pg = alloc_domheap_page(NULL, 0); - - map_pages_to_xen(va, page_to_mfn(pg), 1, PAGE_HYPERVISOR); + mfn_t mfn; + int rc; + + if ( system_state == SYS_STATE_early_boot ) + mfn = alloc_boot_pages(1, 1); + else + mfn = page_to_mfn(alloc_domheap_page(NULL, 0)); + rc = map_pages_to_xen(va, mfn, 1, PAGE_HYPERVISOR); + BUG_ON(rc); clear_page((void *)va); } bitmap_fill(vm_bitmap(type), vm_low[type]); @@ -330,3 +336,13 @@ void vfree(void *va) free_domheap_page(pg); } #endif + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Thu Jan 30 16:13:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE23E139A for ; Thu, 30 Jan 2020 16:16:18 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9E8C20707 for ; Thu, 30 Jan 2020 16:16:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="r7fq0Sbt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9E8C20707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSZ-0004XP-96; Thu, 30 Jan 2020 16:14:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSX-0004WE-Ot for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:25 +0000 X-Inumbo-ID: 758a083e-437b-11ea-ad98-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 758a083e-437b-11ea-ad98-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=8bCC1czvIgY7PeMeaG5CbgEmu47OJWNzV1dFnhwyx6o=; b=r7fq0SbtTsPG2fwyLwzfpSYH0Z FhRGmoe46pwUiZnNPUdYys775LcrTc/j4YqBhbVPmWDSly5v+TLGAgvHCJjncJNLMjrkAnWfNNsAp 8Y/ar538d4tqhCMtgeAMwMM9ijx+OC0MqCODibJnFD7SeyweznN4A7W5d4G1ys82IrzFfKym/R/p8 Q0X5Ei34VbY11mBFAKiVLP6YD0fmEkkIVt4VTio3rDIXhmNjJX3wLzYhpRNXcxdXbkjKWB4bCJocr VLqvZaLrKPUR7tSfpzWmTQTeysiEpyi5/etRYSinnwkSDAuicIpKeN7m9pfCi3amkL2j9qTcn79Tn 2bnFdc0w==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A1-LD; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdi-Ej; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:20 +0000 Message-Id: <20200130161330.2324143-12-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 12/22] xen/vmap: allow vmap() to be called during early boot X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/common/vmap.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 37922f735b..8343460794 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -68,7 +68,7 @@ static void *vm_alloc(unsigned int nr, unsigned int align, spin_lock(&vm_lock); for ( ; ; ) { - struct page_info *pg; + mfn_t mfn; ASSERT(vm_low[t] == vm_top[t] || !test_bit(vm_low[t], vm_bitmap(t))); for ( start = vm_low[t]; start < vm_top[t]; ) @@ -103,9 +103,17 @@ static void *vm_alloc(unsigned int nr, unsigned int align, if ( vm_top[t] >= vm_end[t] ) return NULL; - pg = alloc_domheap_page(NULL, 0); - if ( !pg ) - return NULL; + if ( system_state == SYS_STATE_early_boot ) + { + mfn = alloc_boot_pages(1, 1); + } + else + { + struct page_info *pg = alloc_domheap_page(NULL, 0); + if ( !pg ) + return NULL; + mfn = page_to_mfn(pg); + } spin_lock(&vm_lock); @@ -113,7 +121,7 @@ static void *vm_alloc(unsigned int nr, unsigned int align, { unsigned long va = (unsigned long)vm_bitmap(t) + vm_top[t] / 8; - if ( !map_pages_to_xen(va, page_to_mfn(pg), 1, PAGE_HYPERVISOR) ) + if ( !map_pages_to_xen(va, mfn, 1, PAGE_HYPERVISOR) ) { clear_page((void *)va); vm_top[t] += PAGE_SIZE * 8; @@ -123,7 +131,10 @@ static void *vm_alloc(unsigned int nr, unsigned int align, } } - free_domheap_page(pg); + if ( system_state == SYS_STATE_early_boot ) + init_boot_pages(mfn_to_maddr(mfn), mfn_to_maddr(mfn) + PAGE_SIZE); + else + free_domheap_page(mfn_to_page(mfn)); if ( start >= vm_top[t] ) { From patchwork Thu Jan 30 16:13:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94FF9139A for ; Thu, 30 Jan 2020 16:15:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 707E020CC7 for ; Thu, 30 Jan 2020 16:15:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="3jirisKv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 707E020CC7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSj-0004eo-23; Thu, 30 Jan 2020 16:14:37 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSh-0004dz-NA for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:35 +0000 X-Inumbo-ID: 78cd6306-437b-11ea-b211-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78cd6306-437b-11ea-b211-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=IlalkARsClixWlIYZ3dNEz91vQZ1wmUKqx0bvL84HHo=; b=3jirisKvFxxzCsna4kXLVhU/s7 k/i7spQa4qNDgi57ADfn6/5gSVgqcuS2+2pMKT5EVIaOZOLJuF9v0jXTooO3hbdp85NF8FNIWjxof cbRBIumAdeAXOVJLda/LRFd9AGGyNfFDkfHKbCb8WzMNS3UlD+m36yagNHnOXSUi1kicxAztmnxgn 5yObEuBNgSzA51+mh5AsydS6bYjyrCt1QbTJLdthFuTpdw5Pwme6Ov2XC5+eWVMX8zXJaMYEkTGq1 oZduYT5epY7dI02eOlFtVJr4Gbxp3JQk8sbKPYkOtIayYtbBMcYZaGQr4IdFw93wFy0lT1varRiC8 +7u1OLhQ==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tl-93; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdn-FO; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:21 +0000 Message-Id: <20200130161330.2324143-13-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 13/22] x86/setup: move vm_init() before end_boot_allocator() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse We would like to be able to use vmap() to map the live update data, and we need to do a first pass of the live update data before we prime the heap because we need to know which pages need to be preserved. Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index dba8c3f0a1..ea3f423b4c 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1572,6 +1572,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) numa_initmem_init(0, raw_max_page); + vm_init(); + if ( lu_bootmem_start ) { unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); @@ -1635,12 +1637,6 @@ void __init noreturn __start_xen(unsigned long mbi_p) end_boot_allocator(); system_state = SYS_STATE_boot; - /* - * No calls involving ACPI code should go between the setting of - * SYS_STATE_boot and vm_init() (or else acpi_os_{,un}map_memory() - * will break). - */ - vm_init(); console_init_ring(); vesa_init(); From patchwork Thu Jan 30 16:13:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B730924 for ; Thu, 30 Jan 2020 16:14:48 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 479EB20707 for ; Thu, 30 Jan 2020 16:14:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="LqTlxPcy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 479EB20707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCRz-0004AC-7P; Thu, 30 Jan 2020 16:13:51 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCRy-00049z-Mv for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:13:50 +0000 X-Inumbo-ID: 75a02cae-437b-11ea-8396-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 75a02cae-437b-11ea-8396-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=t0fgs4Yd9bAMIIi/KkocOUn98/R6lbHiWa3xZZC5cZY=; b=LqTlxPcy9l+8pTvBVeJjbVhP7o k0eV0kZ0Q0/HuXatvFK9IvlXszvvusuZJuGkKYf3vKSn3osTwyk2ucVbWO4ld8/RbQFvUZSBSFXQp XZzOcL5DwCoXJO+YxH6ztzUinL8crc2Xfs3H4i1rtXrTQUCRAU/PhkMo6p3U2GcZx3PrS7O+jaBg5 Yt2I3W2w7+NiMfV01qykq1z2rkqAXBunHH7kRCP+7uEh3zTEc05WRGMxONfh5Ovt6PKRPGiv8TcO0 siUCqfBzjAkCUELRjENTU504WKlTOSyv2kLMfTvxXdbyxlAqOT2N2EBjdHE8jpJQdv1XoTgItZOY9 usgMC7+A==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A2-NO; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kds-G8; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:22 +0000 Message-Id: <20200130161330.2324143-14-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 14/22] Detect live update breadcrumb at boot and map data stream X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 35 +++++++++++++++++++++++++++++++++-- xen/common/lu/stream.c | 34 ++++++++++++++++++++++++++++++++++ xen/include/xen/lu.h | 2 ++ 3 files changed, 69 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index ea3f423b4c..eea670e03b 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -745,6 +746,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) .max_maptrack_frames = -1, }; const char *hypervisor_name; + uint64_t lu_mfnlist_phys = 0, lu_nr_pages = 0; + struct lu_stream lu_stream; /* Critical region without IDT or TSS. Any fault is deadly! */ @@ -889,9 +892,16 @@ void __init noreturn __start_xen(unsigned long mbi_p) printk(" Found %d EDD information structures\n", bootsym(boot_edd_info_nr)); - /* Check that we have at least one Multiboot module. */ if ( !(mbi->flags & MBI_MODULES) || (mbi->mods_count == 0) ) - panic("dom0 kernel not specified. Check bootloader configuration\n"); + { + if ( !lu_breadcrumb_phys ) + panic("dom0 kernel not specified. Check bootloader configuration\n"); + } + else + { + /* If modules are provided, don't even look for live update data. */ + lu_breadcrumb_phys = 0; + } /* Check that we don't have a silly number of modules. */ if ( mbi->mods_count > sizeof(module_map) * 8 ) @@ -1337,6 +1347,22 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( !xen_phys_start ) panic("Not enough memory to relocate Xen\n"); + /* Check for the state breadcrumb before giving it to the boot allocator */ + if ( lu_breadcrumb_phys ) + { + uint64_t *breadcrumb = maddr_to_virt(lu_breadcrumb_phys); + + lu_mfnlist_phys = breadcrumb[1]; + lu_nr_pages = breadcrumb[2] >> PAGE_SHIFT; + + if ( breadcrumb[0] == LIVE_UPDATE_MAGIC && lu_nr_pages) { + printk("%ld pages of live update data at 0x%lx\n", lu_nr_pages, lu_mfnlist_phys); + } else { + panic("Live update breadcrumb not found: %lx %lx %lx at %lx\n", + breadcrumb[0], breadcrumb[1], breadcrumb[2], lu_breadcrumb_phys); + } + } + if ( lu_bootmem_start ) { if ( !lu_reserved ) @@ -1574,6 +1600,11 @@ void __init noreturn __start_xen(unsigned long mbi_p) vm_init(); + if ( lu_breadcrumb_phys ) + { + lu_stream_map(&lu_stream, lu_mfnlist_phys, lu_nr_pages); + } + if ( lu_bootmem_start ) { unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); diff --git a/xen/common/lu/stream.c b/xen/common/lu/stream.c index 10e123a466..8c44a4eb37 100644 --- a/xen/common/lu/stream.c +++ b/xen/common/lu/stream.c @@ -124,6 +124,40 @@ void lu_stream_free(struct lu_stream *stream) } } +void lu_stream_map(struct lu_stream *stream, unsigned long mfns_phys, int nr_pages) +{ + unsigned int order = get_order_from_bytes((nr_pages + 1) * sizeof(mfn_t)); + unsigned int i; + + memset(stream, 0, sizeof(*stream)); + + stream->len = nr_pages << PAGE_SHIFT; + stream->nr_pages = nr_pages; + stream->pagelist = __va(mfns_phys); + + map_pages_to_xen((unsigned long)stream->pagelist, maddr_to_mfn(mfns_phys), + 1 << order, PAGE_HYPERVISOR); + + /* Reserve the pages used for the pagelist itself. */ + for ( i = 0; i < (1 << order); i++ ) + { + maddr_to_page(mfns_phys + (i << PAGE_SHIFT))->count_info |= PGC_allocated; + } + + /* Validate and reserve the data pages */ + for ( i = 0; i < nr_pages; i++ ) + { + if (!mfn_valid(stream->pagelist[i])) + panic("Invalid MFN %lx in live update stream\n", mfn_x(stream->pagelist[i])); + + mfn_to_page(stream->pagelist[i])->count_info |= PGC_allocated; + } + + stream->data = vmap(stream->pagelist, nr_pages); + if (!stream->data) + panic("Failed to map live update data\n"); +} + /* * local variables: * mode: c diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h index f232a56950..21abace130 100644 --- a/xen/include/xen/lu.h +++ b/xen/include/xen/lu.h @@ -22,6 +22,8 @@ void lu_stream_free(struct lu_stream *stream); struct kexec_image; int lu_save_all(struct kexec_image *image); +void lu_stream_map(struct lu_stream *stream, unsigned long mfns_phys, int nr_pages); + #endif /* __XEN_LU_H__ */ /* From patchwork Thu Jan 30 16:13:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7E40112B for ; Thu, 30 Jan 2020 16:15:07 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 892CD2082E for ; Thu, 30 Jan 2020 16:15:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="FtQvH+Qu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 892CD2082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCS4-0004Ce-K7; Thu, 30 Jan 2020 16:13:56 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCS3-0004CD-OH for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:13:55 +0000 X-Inumbo-ID: 75b9116a-437b-11ea-b211-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 75b9116a-437b-11ea-b211-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=4t52AQJGtKP6dJkAT1LYIGVd9TufHSmi14aL7qYAa5w=; b=FtQvH+QuZwmXTwwLKTV5c5l/rX 7v9PFj2T6iyQgnPWdNF0aCwvnpp4AyVboB8z6dmny93R7IxVLzwMnb34PoE1MDep0OJcAHGhTtk42 xXRh6jEz2jsFUoS8lCvbIy0AeKTFCyHqzFfE2MyRjmt18Ng+ykqkMdEAwnTi84C9Zh/QnEX331Qu7 0Bv7CaZ18Xm+8byudtKD/U7F8p5Q+MaHEiu+rH5VPEe1BxUqs7IMjBdxR5haQAfpXiVBuvItKZS87 EepDBpueuueQAZ3vBv556CXQ3zguUfVdLevLCykYputGgRKxVteodpNyURtjcGFZ7ips8kFaoYwh5 8WXvQsWA==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A3-Nm; Thu, 30 Jan 2020 16:13:32 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009kdx-Gk; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:23 +0000 Message-Id: <20200130161330.2324143-15-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 15/22] Start documenting the live update handover X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- docs/specs/libxc-migration-stream.pandoc | 19 +- docs/specs/live-update-handover.pandoc | 371 +++++++++++++++++++++++ 2 files changed, 388 insertions(+), 2 deletions(-) create mode 100644 docs/specs/live-update-handover.pandoc diff --git a/docs/specs/libxc-migration-stream.pandoc b/docs/specs/libxc-migration-stream.pandoc index a7a8a08936..9a6679f3de 100644 --- a/docs/specs/libxc-migration-stream.pandoc +++ b/docs/specs/libxc-migration-stream.pandoc @@ -227,12 +227,18 @@ type 0x00000000: END 0x0000000F: CHECKPOINT_DIRTY_PFN_LIST (Secondary -> Primary) - 0x00000010 - 0x7FFFFFFF: Reserved for future _mandatory_ + 0x00000010 - 0x3FFFFFFF: Reserved for future _mandatory_ records. - 0x80000000 - 0xFFFFFFFF: Reserved for future _optional_ + 0x40000000 - 0x7FFFFFFF: Reserved for future _mandatory_ + live update records. + + 0x80000000 - 0xBFFFFFFF: Reserved for future _optional_ records. + 0xC0000000 - 0xFFFFFFFF: Reserved for future _optional_ + live update records. + body_length Length in octets of the record body. body Content of the record. @@ -246,6 +252,15 @@ Records may be _mandatory_ or _optional_. Optional records have bit unsupported mandatory record must fail. The contents of optional records may be ignored during a restore. +Note: This basic record format,. and some of the record types defined here, +are also used for Live Update, as discussed in the Live Update Handover +document: `docs/specs/live-update-handover.pandoc`. + +Records defined for live update have bit 30 set in their type value, +are defined in that document, and are out of scope for this document. +Such records shall not appear in the Domain Image Format as defined by +this document. + The following sub-sections specify the record body format for each of the record types. diff --git a/docs/specs/live-update-handover.pandoc b/docs/specs/live-update-handover.pandoc new file mode 100644 index 0000000000..31d23c7c90 --- /dev/null +++ b/docs/specs/live-update-handover.pandoc @@ -0,0 +1,371 @@ +% Live Update Handover Protocol +% David Woodhouse <> +% Revision 1 + +Introduction +============ + +Purpose +------- + +Live update performs a _kexec_ from one running version of Xen to +another, preserving all running domains in a form of guest-transparent +live migration. + +This document outlines the memory layout requirements and data stream +used in handover protocol, to ensure that pages used by running +domains are preserved during the transition from one version of Xen +to the next. + + +Compatibility +------------- + +It cannot be repeated often enough that information passed over live +update is an ABI. It is expected that live update can be performed from +one major version of Xen to another, or even hypothetically to a system +which is not Xen at all. + +It is necessary that some data are handed over "in place"; in +particular the memory pages of the running domains. However, no +internal Xen data structures may be transferred in this fashion; at +least not without retrospectively declaring them to be ABI, with the +restrictions that places on subsequent changes. + + + +Handover +======== + + +Memory Usage Restrictions +------------------------- + +The new Xen must take care not to use any memory pages which already +belong to guests. To facilitate this, a contiguous region of memory +is reserved for the boot allocator, known as _live update bootmem_. + +This region is reserved by the original Xen during its own boot, and +the location made available to the _kexec(8)_ user space tool +through the `kexec_get_range` hypercall using a new region type +`KEXEC_RANGE_MA_LIVEUPDATE`. It is passed to the new Xen on the +command line, using the `liveupdate=` parameter. + +The new Xen must not use any pages outside this region until it has +consumed the live update data stream and determined which pages are +already in use by running domains. + +At run time, Xen may use memory from the reserved region for any +purpose that does not require preservation over a live update; in +particular it must not be mapped to a domain. + +The new Xen executable image must be loaded by kexec to the same +physical location as the running Xen, since that region of memory is +known to be available. For that reason, freed init memory from the +Xen image is also treated as reserved _live update bootmem_. + + +Live Update Data Stream +----------------------- + +During handover, the running Xen pauses all domains and creates a +_live update data stream_ containing all the information required by +the new Xen to restore them. This is largely the same as guest +transparent live migration. + +Data pages for this stream may be allocated anywhere in physical +memory outside the _live update bootmem_ regions. + +Xen creates a physically contiguous array of MFNs of the allocated +data pages, suitable for passing to `vmap()` to obtain a virtually +contiguous mapping of the whole data stream. + + +Breadcrumb +---------- + +Since the live update data stream is created during the final `kexec_exec` +hypercall, its address cannot be passed on the command line to the +new Xen since the command line needs to have been set up by `kexec(8)` +in userspace long beforehand. + +Thus, to allow the new Xen to find the data stream, the old Xen places +a _breadcrumb_ in the first words of the _live update bootmem_, containing +the number of data pages, and the physical address of the contiguous MFN +array. + +The breadcrumb is written as the last action of the `kexec_reloc()` +routine during the `kexec` handover, so cannot overwrite anything +important by virtue of the existing guarantee that Xen will not place +any data in that region which needs to survive across a live update. + +A restriction of the `kexec_reloc()` mechanism for writing the breadcrumb +is that the values are host-endian and are masked with PAGE_MASK; the low +bits are zeroed. This is actually perfect for the magic value used +to recognise a live update breadcrumb, since it neatly prevents any attempt +to live update to a Xen which uses a different endianness or page size. + +For the physical address of the MFN list it's perfectly fine, since +that list is page-aligned anyway. For the number of pages, it means +the value must be shifted accordingly. Hence the use of `shifted_nr_pages` +in the breadcrumb structure below: + + + 0 1 2 3 4 5 6 7 octet + +-------------------------------------------------+ + | live_update_magic | + +-------------------------------------------------+ + | mfn_array_physaddr | + +-------------------------------------------------+ + | shifted_nr_pages | + +-------------------------------------------------+ + +-------------------------------------------------------------------- +Field Description +------------------- ------------------------------------------------ +live_update_magic "LiveUpda" (0x4c69766555706461) stored in the the host + endianness and masked with PAGE_MASK. + For example on x86_64: `00 60 70 55 65 76 89 4c`. + +mfn_array_physaddr Machine address of MFN list for data streaes. + +shift_nr_pages Number of data pages, shifted by PAGE_SHIFT to + avoid the limitation of kexec_reloc(). +-------------------------------------------------------------------- + + +IOMMU +----- + +Where devices are passed through to domains, it may not be possible +to quiesce those devices for the purpose of performing the update. + +If performing live update with assigned devices, the original Xen will +leave the IOMMU mappings active during the handover (thus implying +that IOMMU page tables may not be allocated in the `live update +bootmem` region either). + +The new Xen must resume control of the IOMMU without causing those mappings +to become invalid even for a short period of time. On hardware which does not +support Posted Interrupts, interrupts may need to be generated on resume. + +_This section will be expanded once we actually have it working._ + +\clearpage + +Data Stream Overview +==================== + +Once discovered and mapped, the live update data stream forms a +virtually contiguous stream of records following the basic form +documented in the LibXenCtrl Domain Image Format at +`docs/specs/libxc-migration-stream.pandoc`. + +Some record types from the LibXenCtrl Domain Image format are used +as-is, such as the `X86_PV_INFO`, `X86_PV_VCPU_BASIC`, `HVM_CONTEXT` +and other records containing domain-specific data. + +The Domain Header from that document is not used in that form, and a new +record of type `LU_DOMAIN_INFO` is defined below. + +Other new record types specific to the live update process are defined in +this document. Of those, some contain global state such as the M2P table +information, while others are domain-specific. + +The live update data stream starts with records containing global +information, followed any number of times by a `LU_DOMAIN_INFO` record +and subsequent domain-specific records for that domain. + +There is a single `END` record at the end of the live update data stream, +indicating that no more `DOMAIN_INFO` records are present. + +\clearpage + +As defined in the LibXenCtrl Domain Image format document, a record +has the following structure. Record type values defined for live update +have bit 30 set, and are thus in the range 0x40000000-0x7FFFFFFF for +mandatory live update records, and 0xC0000000-0xFFFFFFFF for optional +live update records _(of which there are none at the present time)_. + + + 0 1 2 3 4 5 6 7 octet + +-----------------------+-------------------------+ + | type | body_length | + +-----------+-----------+-------------------------+ + | body... | + ... + | | padding (0 to 7 octets) | + +-----------+-------------------------------------+ + +-------------------------------------------------------------------- +Field Description +----------- ------------------------------------------------------- +type 0x40000000: LU_VERSION + + 0x40000001: LU_M2P + + 0x40000002: LU_M2P_COMPAT + + 0x40000003: LU_DOMAIN_INFO + + 0x40000004 - 0x7FFFFFFF: Reserved for future _mandatory_ + live update records. + + 0xC0000000 - 0xFFFFFFFF: Reserved for future _optional_ + live update records. + +body_length Length in octets of the record body. + +body Content of the record. + +padding 0 to 7 octets of zeros to pad the whole record to a multiple + of 8 octets. +-------------------------------------------------------------------- + + +\clearpage + +Global Records +============== + +LU_VERSION +---------- + +The version field indicates the version of Xen from which the system +is live updating. In theory this should never be relevant, but it +allows for version-specific workarounds to be implementing in the receiving +Xen should they become necessary. + + 0 1 2 3 4 5 6 7 octet + +-----------------------+-----------+-------------+ + | xen_major | xen_minor | + +-----------------------+-------------------------+ + + +-------------------------------------------------------------------- +Field Description +----------- -------------------------------------------------------- +xen_major The Xen major version from which the system is updating. + +xen_minor The Xen minor version from which the system is updating. +-------------------------------------------------------------------- + +\clearpage + +LU_M2P / LU_M2P_COMPAT +---------------------- + +The M2P and compatibility M2P records contain a scatter/gather list of +pages containing native or 32-bit M2P data. + + + 0 1 2 3 4 5 6 7 octet + +-----------------------+-------------------------+ + | m2p_page_data[0]... | + ... + +-------------------------------------------------+ + | m2p_page_data[N-1]... | + ... + +-------------------------------------------------+ + +-------------------------------------------------------------------- +Field Description +----------- -------------------------------------------------------- +m2p_page_data A 64-bit value containing the physical address of the + next page of M2P data, encoding the _order_ of the page + into the low 12 bits. Thus, a 1GiB page at 0x4C0000000 + would be encoded as 0x4C000001E. + + In case the M2P does not contiguously cover pages starting + from MFN zero, a discontiguity is indicated by a field + with order set to zero. The high bits of the field then + provide the MFN for which the subsequent M2P data page + provides data. + +-------------------------------------------------------------------- + +\clearpage + +Domain Specific Records +======================= + + +LU_DOMAIN_INFO +-------------- + +The domain info record contains general properties necessary to +recreate a domain in the receiving Xen, and marks the start of a set +of other domain-specific records pertaining to that domain. + + 0 1 2 3 4 5 6 7 octet + +-----------------------+-----------+-------------+ + | type | page_shift| domain_id | + +-----------------------+-----------+-------------+ + | domain_handle[0-7] | + +-------------------------------------------------+ + | domain_handle[8-15] | + +-----------------------+-------------------------+ + | ssidref | flags | + +-----------------------+-------------------------+ + | max_vcpus | emulation_flags | + +-----------------------+-------------------------+ + | extra_flags | (padding) | + +-----------------------+-------------------------+ + + +-------------------------------------------------------------------- +Field Description +--------------- -------------------------------------------------------- +type 0x0000: Reserved. + + 0x0001: x86 PV. + + 0x0002: x86 HVM. + + 0x0003 - 0xFFFFFFFF: Reserved. + +page_shift Size of a guest page as a power of two. + + i.e., page size = 2 ^page_shift^. + +domain_id Domain ID + + +domain_handle UUID domain handle. + +ssidref Security Identifier Index + +flags Domain flags using `XEN_DOMCTL_CTF_` + +max_vcpus Maximum vCPUs for domain. + +emulation_flags Emulation flags using `XEN_X86_EMU_` + +extra_flags Additional flags: + + 0x00000001: Is privileged + +-------------------------------------------------------------------- + +\clearpage + +Future Extensions +================= + +All changes to this specification should bump the revision number in +the title block. + +All changes to the image or domain headers require the image version +to be increased. + +The format may be extended by adding additional record types. + +Extending an existing record type must be done by adding a new record +type. This allows old images with the old record to still be +restored. + +The image header may only be extended by _appending_ additional +fields. In particular, the `marker`, `id` and `version` fields must +never change size or location. + + From patchwork Thu Jan 30 16:13:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23A7C112B for ; Thu, 30 Jan 2020 16:16:23 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8D472082E for ; Thu, 30 Jan 2020 16:16:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="orrwA0Pn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8D472082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTI-00059J-55; Thu, 30 Jan 2020 16:15:12 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTG-00057u-ON for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:15:10 +0000 X-Inumbo-ID: 78cae3ba-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78cae3ba-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=dvssONFM41Bo5NgJdCUQIY/j3A4Avtsk6txMcpFUyOE=; b=orrwA0Pn5HHPg1zQsfb4R/x7Nd hA9G9GRmC/rhcuv0ekAUXOgXyPV/gFQxT+d5wkALtIw28lFWfIm7XxX+fK6UAc3SPviAQfEOTYkJH uJGJ5j0ZtJBF6Hlre48o4/Vod4FpbvNug5xYe0W+UviSwVTrfnbVQL4k8jIw3oZzGkwzd3Lo33bZJ 2jBb+15k9FJDKo+MDgMYR1+gn41Ytl75dglkwHCBRu92s17HxUkBYDwPL0+Ug+eqlHh4mChd+dlNs duGx/pHS4eKRsICgk3pAtbsQcyNN9Ip0+xVCumJgHRwKWrOoy6jCnQ3cfSahPTDrqekzKwaW2FZml fZxSFDPw==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tm-BC; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009ke2-HN; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:24 +0000 Message-Id: <20200130161330.2324143-16-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 16/22] Migrate migration stream definitions into Xen public headers X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- tools/libxc/xc_sr_common.c | 20 ++-- tools/libxc/xc_sr_common_x86.c | 4 +- tools/libxc/xc_sr_restore.c | 2 +- tools/libxc/xc_sr_restore_x86_hvm.c | 4 +- tools/libxc/xc_sr_restore_x86_pv.c | 8 +- tools/libxc/xc_sr_save.c | 2 +- tools/libxc/xc_sr_save_x86_hvm.c | 4 +- tools/libxc/xc_sr_save_x86_pv.c | 12 +-- tools/libxc/xc_sr_stream_format.h | 97 +------------------- xen/include/Makefile | 2 +- xen/include/public/migration_stream.h | 126 ++++++++++++++++++++++++++ 11 files changed, 157 insertions(+), 124 deletions(-) create mode 100644 xen/include/public/migration_stream.h diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c index dd9a11b4b5..92f9332e73 100644 --- a/tools/libxc/xc_sr_common.c +++ b/tools/libxc/xc_sr_common.c @@ -91,7 +91,7 @@ int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec, int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rhdr rhdr; + struct mr_rhdr rhdr; size_t datasz; if ( read_exact(fd, &rhdr, sizeof(rhdr)) ) @@ -142,15 +142,15 @@ static void __attribute__((unused)) build_assertions(void) { BUILD_BUG_ON(sizeof(struct xc_sr_ihdr) != 24); BUILD_BUG_ON(sizeof(struct xc_sr_dhdr) != 16); - BUILD_BUG_ON(sizeof(struct xc_sr_rhdr) != 8); - - BUILD_BUG_ON(sizeof(struct xc_sr_rec_page_data_header) != 8); - BUILD_BUG_ON(sizeof(struct xc_sr_rec_x86_pv_info) != 8); - BUILD_BUG_ON(sizeof(struct xc_sr_rec_x86_pv_p2m_frames) != 8); - BUILD_BUG_ON(sizeof(struct xc_sr_rec_x86_pv_vcpu_hdr) != 8); - BUILD_BUG_ON(sizeof(struct xc_sr_rec_x86_tsc_info) != 24); - BUILD_BUG_ON(sizeof(struct xc_sr_rec_hvm_params_entry) != 16); - BUILD_BUG_ON(sizeof(struct xc_sr_rec_hvm_params) != 8); + BUILD_BUG_ON(sizeof(struct mr_rhdr) != 8); + + BUILD_BUG_ON(sizeof(struct mr_page_data_header) != 8); + BUILD_BUG_ON(sizeof(struct mr_x86_pv_info) != 8); + BUILD_BUG_ON(sizeof(struct mr_x86_pv_p2m_frames) != 8); + BUILD_BUG_ON(sizeof(struct mr_x86_pv_vcpu_hdr) != 8); + BUILD_BUG_ON(sizeof(struct mr_x86_tsc_info) != 24); + BUILD_BUG_ON(sizeof(struct mr_hvm_params_entry) != 16); + BUILD_BUG_ON(sizeof(struct mr_hvm_params) != 8); } /* diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c index 011684df97..1627ff72d6 100644 --- a/tools/libxc/xc_sr_common_x86.c +++ b/tools/libxc/xc_sr_common_x86.c @@ -3,7 +3,7 @@ int write_x86_tsc_info(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_x86_tsc_info tsc = {}; + struct mr_x86_tsc_info tsc = {}; struct xc_sr_record rec = { .type = REC_TYPE_X86_TSC_INFO, .length = sizeof(tsc), @@ -23,7 +23,7 @@ int write_x86_tsc_info(struct xc_sr_context *ctx) int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_x86_tsc_info *tsc = rec->data; + struct mr_x86_tsc_info *tsc = rec->data; if ( rec->length != sizeof(*tsc) ) { diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c index 5e31908ca8..29c264ecc7 100644 --- a/tools/libxc/xc_sr_restore.c +++ b/tools/libxc/xc_sr_restore.c @@ -335,7 +335,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_page_data_header *pages = rec->data; + struct mr_page_data_header *pages = rec->data; unsigned int i, pages_of_data = 0; int rc = -1; diff --git a/tools/libxc/xc_sr_restore_x86_hvm.c b/tools/libxc/xc_sr_restore_x86_hvm.c index 3f78248f32..e5b25f4280 100644 --- a/tools/libxc/xc_sr_restore_x86_hvm.c +++ b/tools/libxc/xc_sr_restore_x86_hvm.c @@ -25,8 +25,8 @@ static int handle_hvm_params(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_hvm_params *hdr = rec->data; - struct xc_sr_rec_hvm_params_entry *entry = hdr->param; + struct mr_hvm_params *hdr = rec->data; + struct mr_hvm_params_entry *entry = hdr->param; unsigned int i; int rc; diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c index 16e738884e..8e43ddcfd7 100644 --- a/tools/libxc/xc_sr_restore_x86_pv.c +++ b/tools/libxc/xc_sr_restore_x86_pv.c @@ -585,7 +585,7 @@ static int update_guest_p2m(struct xc_sr_context *ctx) * Cross-check the legitimate combinations. */ static bool valid_x86_pv_info_combination( - const struct xc_sr_rec_x86_pv_info *info) + const struct mr_x86_pv_info *info) { switch ( info->guest_width ) { @@ -602,7 +602,7 @@ static int handle_x86_pv_info(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_x86_pv_info *info = rec->data; + struct mr_x86_pv_info *info = rec->data; if ( ctx->x86.pv.restore.seen_pv_info ) { @@ -675,7 +675,7 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_x86_pv_p2m_frames *data = rec->data; + struct mr_x86_pv_p2m_frames *data = rec->data; unsigned int start, end, x, fpp = PAGE_SIZE / ctx->x86.pv.width; int rc; @@ -734,7 +734,7 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_x86_pv_vcpu_hdr *vhdr = rec->data; + struct mr_x86_pv_vcpu_hdr *vhdr = rec->data; struct xc_sr_x86_pv_restore_vcpu *vcpu; const char *rec_name; size_t blobsz; diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c index fa736a311f..41af26909e 100644 --- a/tools/libxc/xc_sr_save.c +++ b/tools/libxc/xc_sr_save.c @@ -87,7 +87,7 @@ static int write_batch(struct xc_sr_context *ctx) void *page, *orig_page; uint64_t *rec_pfns = NULL; struct iovec *iov = NULL; int iovcnt = 0; - struct xc_sr_rec_page_data_header hdr = { 0 }; + struct mr_page_data_header hdr = { 0 }; struct xc_sr_record rec = { .type = REC_TYPE_PAGE_DATA, }; diff --git a/tools/libxc/xc_sr_save_x86_hvm.c b/tools/libxc/xc_sr_save_x86_hvm.c index d99efe65e5..c4dc42479f 100644 --- a/tools/libxc/xc_sr_save_x86_hvm.c +++ b/tools/libxc/xc_sr_save_x86_hvm.c @@ -80,8 +80,8 @@ static int write_hvm_params(struct xc_sr_context *ctx) }; xc_interface *xch = ctx->xch; - struct xc_sr_rec_hvm_params_entry entries[ARRAY_SIZE(params)]; - struct xc_sr_rec_hvm_params hdr = { + struct mr_hvm_params_entry entries[ARRAY_SIZE(params)]; + struct mr_hvm_params hdr = { .count = 0, }; struct xc_sr_record rec = { diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c index f3ccf5bb4b..916c5aad41 100644 --- a/tools/libxc/xc_sr_save_x86_pv.c +++ b/tools/libxc/xc_sr_save_x86_pv.c @@ -485,7 +485,7 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id) unsigned int i, gdt_count; int rc = -1; vcpu_guest_context_any_t vcpu; - struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = { + struct mr_x86_pv_vcpu_hdr vhdr = { .vcpu_id = id, }; struct xc_sr_record rec = { @@ -583,7 +583,7 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id) static int write_one_vcpu_extended(struct xc_sr_context *ctx, uint32_t id) { xc_interface *xch = ctx->xch; - struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = { + struct mr_x86_pv_vcpu_hdr vhdr = { .vcpu_id = id, }; struct xc_sr_record rec = { @@ -620,7 +620,7 @@ static int write_one_vcpu_xsave(struct xc_sr_context *ctx, uint32_t id) xc_interface *xch = ctx->xch; int rc = -1; DECLARE_HYPERCALL_BUFFER(void, buffer); - struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = { + struct mr_x86_pv_vcpu_hdr vhdr = { .vcpu_id = id, }; struct xc_sr_record rec = { @@ -686,7 +686,7 @@ static int write_one_vcpu_msrs(struct xc_sr_context *ctx, uint32_t id) int rc = -1; size_t buffersz; DECLARE_HYPERCALL_BUFFER(void, buffer); - struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = { + struct mr_x86_pv_vcpu_hdr vhdr = { .vcpu_id = id, }; struct xc_sr_record rec = { @@ -793,7 +793,7 @@ static int write_all_vcpu_information(struct xc_sr_context *ctx) */ static int write_x86_pv_info(struct xc_sr_context *ctx) { - struct xc_sr_rec_x86_pv_info info = { + struct mr_x86_pv_info info = { .guest_width = ctx->x86.pv.width, .pt_levels = ctx->x86.pv.levels, }; @@ -816,7 +816,7 @@ static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx) int rc; unsigned int i; size_t datasz = ctx->x86.pv.p2m_frames * sizeof(uint64_t); uint64_t *data = NULL; - struct xc_sr_rec_x86_pv_p2m_frames hdr = { + struct mr_x86_pv_p2m_frames hdr = { .end_pfn = ctx->x86.pv.max_pfn, }; struct xc_sr_record rec = { diff --git a/tools/libxc/xc_sr_stream_format.h b/tools/libxc/xc_sr_stream_format.h index 37a7da6eab..0700cde54f 100644 --- a/tools/libxc/xc_sr_stream_format.h +++ b/tools/libxc/xc_sr_stream_format.h @@ -1,6 +1,8 @@ #ifndef __STREAM_FORMAT__H #define __STREAM_FORMAT__H +#include + /* * C structures for the Migration v2 stream format. * See docs/specs/libxc-migration-stream.pandoc @@ -41,101 +43,6 @@ struct xc_sr_dhdr uint32_t xen_minor; }; -#define DHDR_TYPE_X86_PV 0x00000001U -#define DHDR_TYPE_X86_HVM 0x00000002U - -/* - * Record Header - */ -struct xc_sr_rhdr -{ - uint32_t type; - uint32_t length; -}; - -/* All records must be aligned up to an 8 octet boundary */ -#define REC_ALIGN_ORDER (3U) -/* Somewhat arbitrary - 128MB */ -#define REC_LENGTH_MAX (128U << 20) - -#define REC_TYPE_END 0x00000000U -#define REC_TYPE_PAGE_DATA 0x00000001U -#define REC_TYPE_X86_PV_INFO 0x00000002U -#define REC_TYPE_X86_PV_P2M_FRAMES 0x00000003U -#define REC_TYPE_X86_PV_VCPU_BASIC 0x00000004U -#define REC_TYPE_X86_PV_VCPU_EXTENDED 0x00000005U -#define REC_TYPE_X86_PV_VCPU_XSAVE 0x00000006U -#define REC_TYPE_SHARED_INFO 0x00000007U -#define REC_TYPE_X86_TSC_INFO 0x00000008U -#define REC_TYPE_HVM_CONTEXT 0x00000009U -#define REC_TYPE_HVM_PARAMS 0x0000000aU -#define REC_TYPE_TOOLSTACK 0x0000000bU -#define REC_TYPE_X86_PV_VCPU_MSRS 0x0000000cU -#define REC_TYPE_VERIFY 0x0000000dU -#define REC_TYPE_CHECKPOINT 0x0000000eU -#define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST 0x0000000fU - -#define REC_TYPE_OPTIONAL 0x80000000U - -/* PAGE_DATA */ -struct xc_sr_rec_page_data_header -{ - uint32_t count; - uint32_t _res1; - uint64_t pfn[0]; -}; - -#define PAGE_DATA_PFN_MASK 0x000fffffffffffffULL -#define PAGE_DATA_TYPE_MASK 0xf000000000000000ULL - -/* X86_PV_INFO */ -struct xc_sr_rec_x86_pv_info -{ - uint8_t guest_width; - uint8_t pt_levels; - uint8_t _res[6]; -}; - -/* X86_PV_P2M_FRAMES */ -struct xc_sr_rec_x86_pv_p2m_frames -{ - uint32_t start_pfn; - uint32_t end_pfn; - uint64_t p2m_pfns[0]; -}; - -/* X86_PV_VCPU_{BASIC,EXTENDED,XSAVE,MSRS} */ -struct xc_sr_rec_x86_pv_vcpu_hdr -{ - uint32_t vcpu_id; - uint32_t _res1; - uint8_t context[0]; -}; - -/* X86_TSC_INFO */ -struct xc_sr_rec_x86_tsc_info -{ - uint32_t mode; - uint32_t khz; - uint64_t nsec; - uint32_t incarnation; - uint32_t _res1; -}; - -/* HVM_PARAMS */ -struct xc_sr_rec_hvm_params_entry -{ - uint64_t index; - uint64_t value; -}; - -struct xc_sr_rec_hvm_params -{ - uint32_t count; - uint32_t _res1; - struct xc_sr_rec_hvm_params_entry param[0]; -}; - #endif /* * Local variables: diff --git a/xen/include/Makefile b/xen/include/Makefile index c3e0283d34..9161716e8f 100644 --- a/xen/include/Makefile +++ b/xen/include/Makefile @@ -101,7 +101,7 @@ all: headers.chk headers99.chk headers++.chk PUBLIC_HEADERS := $(filter-out public/arch-% public/dom0_ops.h, $(wildcard public/*.h public/*/*.h) $(public-y)) PUBLIC_C99_HEADERS := public/io/9pfs.h public/io/pvcalls.h -PUBLIC_ANSI_HEADERS := $(filter-out public/%ctl.h public/xsm/% public/%hvm/save.h $(PUBLIC_C99_HEADERS), $(PUBLIC_HEADERS)) +PUBLIC_ANSI_HEADERS := $(filter-out public/%ctl.h public/migration_stream.h public/xsm/% public/%hvm/save.h $(PUBLIC_C99_HEADERS), $(PUBLIC_HEADERS)) public/io/9pfs.h-prereq := string public/io/pvcalls.h-prereq := string diff --git a/xen/include/public/migration_stream.h b/xen/include/public/migration_stream.h new file mode 100644 index 0000000000..92dd119f9f --- /dev/null +++ b/xen/include/public/migration_stream.h @@ -0,0 +1,126 @@ +#ifndef __XEN_MIGRATION_STREAM_H__ +#define __XEN_MIGRATION_STREAM_H__ + +#if !defined(__XEN__) && !defined(__XEN_TOOLS__) +#error "Migration stream definitions are intended for use by node control tools only" +#endif + +/* + * C structures for the Migration and Live Update. + * See docs/specs/libxc-migration-stream.pandoc + * abd docs/specs/live-update-handover.pandoc + */ + +#include "xen.h" + +/* + * Domain types are used in the libxc stream domain header as well + * as in the live update REC_TYPE_DOMAIN_INFO record. + */ +#define DHDR_TYPE_X86_PV 0x00000001U +#define DHDR_TYPE_X86_HVM 0x00000002U + +/* + * Record Header + */ +struct mr_rhdr +{ + uint32_t type; + uint32_t length; +}; + +/* All records must be aligned up to an 8 octet boundary */ +#define REC_ALIGN_ORDER (3U) +/* Somewhat arbitrary - 128MB */ +#define REC_LENGTH_MAX (128U << 20) + +#define REC_TYPE_END 0x00000000U +#define REC_TYPE_PAGE_DATA 0x00000001U +#define REC_TYPE_X86_PV_INFO 0x00000002U +#define REC_TYPE_X86_PV_P2M_FRAMES 0x00000003U +#define REC_TYPE_X86_PV_VCPU_BASIC 0x00000004U +#define REC_TYPE_X86_PV_VCPU_EXTENDED 0x00000005U +#define REC_TYPE_X86_PV_VCPU_XSAVE 0x00000006U +#define REC_TYPE_SHARED_INFO 0x00000007U +#define REC_TYPE_X86_TSC_INFO 0x00000008U +#define REC_TYPE_HVM_CONTEXT 0x00000009U +#define REC_TYPE_HVM_PARAMS 0x0000000aU +#define REC_TYPE_TOOLSTACK 0x0000000bU +#define REC_TYPE_X86_PV_VCPU_MSRS 0x0000000cU +#define REC_TYPE_VERIFY 0x0000000dU +#define REC_TYPE_CHECKPOINT 0x0000000eU +#define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST 0x0000000fU + +#define REC_TYPE_OPTIONAL 0x80000000U +#define REC_TYPE_LIVE_UPDATE 0x40000000U + +/* PAGE_DATA */ +struct mr_page_data_header +{ + uint32_t count; + uint32_t _res1; + uint64_t pfn[0]; +}; + +#define PAGE_DATA_PFN_MASK 0x000fffffffffffffULL +#define PAGE_DATA_TYPE_MASK 0xf000000000000000ULL + +/* X86_PV_INFO */ +struct mr_x86_pv_info +{ + uint8_t guest_width; + uint8_t pt_levels; + uint8_t _res[6]; +}; + +/* X86_PV_P2M_FRAMES */ +struct mr_x86_pv_p2m_frames +{ + uint32_t start_pfn; + uint32_t end_pfn; + uint64_t p2m_pfns[0]; +}; + +/* X86_PV_VCPU_{BASIC,EXTENDED,XSAVE,MSRS} */ +struct mr_x86_pv_vcpu_hdr +{ + uint32_t vcpu_id; + uint32_t _res1; + uint8_t context[0]; +}; + +/* X86_TSC_INFO */ +struct mr_x86_tsc_info +{ + uint32_t mode; + uint32_t khz; + uint64_t nsec; + uint32_t incarnation; + uint32_t _res1; +}; + +/* HVM_PARAMS */ +struct mr_hvm_params_entry +{ + uint64_t index; + uint64_t value; +}; + +struct mr_hvm_params +{ + uint32_t count; + uint32_t _res1; + struct mr_hvm_params_entry param[0]; +}; + +#endif /* __XEN_MIGRATION_STREAM_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Thu Jan 30 16:13:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4BEB9112B for ; Thu, 30 Jan 2020 16:16:11 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25EB920707 for ; Thu, 30 Jan 2020 16:16:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="dmjZp24W" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25EB920707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTM-0005FB-Qb; Thu, 30 Jan 2020 16:15:16 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTL-0005DW-Q1 for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:15:15 +0000 X-Inumbo-ID: 78cb9918-437b-11ea-8396-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78cb9918-437b-11ea-8396-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=v/QUZkpN6XQ5mb9yTgHgXdzpdT/hSpe4dmLFXOsSGm4=; b=dmjZp24WTkiVupuuMxd+JNv3fp XuS/edGhPMTk+AtIjr36KBTfCxbr1kyeXSUwFG+m0fzvIRDTireRXgr4GubB0Jl9bAwTSEuAg3seq xLDGDFMBQrk0x3iWYU/961vvLV7glm0ZwMwm4yYV0OL0cYv2qlI6ptuE7WrbWBxTazgIsWc5xa6lF Wo7AD3Rbwk+BlzC4GeqlSHY4v0bW0JRtdapV/ZLjzaRfrnctzLS8H9WZiF00nNu01/8gRnOvc4ufU phWYnmMM8BKN21X3zsLuKz0EzrEy1coVqxCcTBKNR8aIyzLvFsngqRAMxDzet+M7xL7+PLzjnA2FP nt/TblvQ==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tn-Bi; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009ke7-I7; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:25 +0000 Message-Id: <20200130161330.2324143-17-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 17/22] Add lu_stream_{open, close, append}_record() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/common/lu/stream.c | 50 ++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/lu.h | 5 +++++ 2 files changed, 55 insertions(+) diff --git a/xen/common/lu/stream.c b/xen/common/lu/stream.c index 8c44a4eb37..2ee870e80a 100644 --- a/xen/common/lu/stream.c +++ b/xen/common/lu/stream.c @@ -33,6 +33,7 @@ #include #include #include +#include static int lu_stream_extend(struct lu_stream *stream, int nr_pages) { @@ -105,6 +106,55 @@ int lu_stream_append(struct lu_stream *stream, const void *data, size_t size) return 0; } +int lu_stream_open_record(struct lu_stream *stream, unsigned int type) +{ + struct mr_rhdr *hdr; + + stream->last_hdr = stream->len; + + hdr = lu_stream_reserve(stream, sizeof(hdr)); + if (!hdr) + return -ENOMEM; + + hdr->type = type; + hdr->length = 0; + + lu_stream_end_reservation(stream, sizeof(*hdr)); + + return 0; +} + +int lu_stream_close_record(struct lu_stream *stream) +{ + uint64_t zeroes = 0; + struct mr_rhdr *hdr; + int rc = 0; + + hdr = (struct mr_rhdr *)(stream->data + stream->last_hdr); + + hdr->length = stream->len - stream->last_hdr - sizeof(*hdr); + + if (stream->len & 7) + rc = lu_stream_append(stream, &zeroes, 8 - (stream->len & 7)); + + return rc; +} + +int lu_stream_append_record(struct lu_stream *stream, unsigned int type, + void *rec, size_t len) +{ + int rc; + + + rc = lu_stream_open_record(stream, type); + if (!rc && len) + rc = lu_stream_append(stream, rec, len); + if (!rc) + rc = lu_stream_close_record(stream); + + return 0; +} + void lu_stream_free(struct lu_stream *stream) { unsigned int order = get_order_from_bytes((stream->nr_pages + 1) * sizeof(mfn_t)); diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h index 21abace130..c02268e414 100644 --- a/xen/include/xen/lu.h +++ b/xen/include/xen/lu.h @@ -9,6 +9,7 @@ struct lu_stream { mfn_t *pagelist; + size_t last_hdr; size_t len; int nr_pages; char *data; @@ -17,6 +18,10 @@ struct lu_stream { void *lu_stream_reserve(struct lu_stream *stream, size_t size); void lu_stream_end_reservation(struct lu_stream *stream, size_t size); int lu_stream_append(struct lu_stream *stream, const void *data, size_t size); +int lu_stream_open_record(struct lu_stream *stream, unsigned int type); +int lu_stream_close_record(struct lu_stream *stream); +int lu_stream_append_record(struct lu_stream *stream, unsigned int type, + void *rec, size_t len); void lu_stream_free(struct lu_stream *stream); struct kexec_image; From patchwork Thu Jan 30 16:13:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88670139A for ; Thu, 30 Jan 2020 16:15:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6433820707 for ; Thu, 30 Jan 2020 16:15:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pd6yl2vf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6433820707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCRu-00048a-TG; Thu, 30 Jan 2020 16:13:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCRt-00048D-MS for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:13:45 +0000 X-Inumbo-ID: 75906d96-437b-11ea-8396-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 75906d96-437b-11ea-8396-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ZTgfDMXt3n597mGD9I/q6Dgx1CUvT+O8Ay/FP5gtSHI=; b=pd6yl2vfbxOutyLDfuee0eJIzX R+e6fU09g+WvwiU2Jx2AvWnq0r4dqmbX9KQ0tmBxMhB9wOWXdEpHZFSvl1uaV4iOQuUyQDdryROt+ O4mu4+aY4/XCKeOls56JPoY5a6MomA/VqYaWMTBWt6pJT0WVTogaeK1mNJIXyaxKkMH26MqXFzF+m gEM4z1+NoGRNwoKstqCTEGOdsx/OAkOMrzA3sYfixrR+Dd0z5M9mMh2Ds4L3ILYg7pr6R8KX83WwN haUSuzdqV4cdP7uaiDLBiMqNhBCza6tLZDZnTF8/j2jD8xp+tKqGlhdgLO3E2jaTBpEmmKfiEoUB1 IxAqRhTA==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A4-Pi; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009keC-Ig; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:26 +0000 Message-Id: <20200130161330.2324143-18-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 18/22] Add LU_VERSION and LU_END records to live update stream X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Signed-off-by: David Woodhouse --- xen/common/lu/save.c | 13 ++++++++++++- xen/include/public/migration_stream.h | 9 +++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/xen/common/lu/save.c b/xen/common/lu/save.c index c43962c44e..84852da35e 100644 --- a/xen/common/lu/save.c +++ b/xen/common/lu/save.c @@ -4,10 +4,17 @@ #include #include #include +#include +#include int lu_save_global(struct lu_stream *stream) { - return 0; + struct mr_lu_version ver_rec; + + ver_rec.xen_major = xen_major_version(); + ver_rec.xen_minor = xen_minor_version(); + return lu_stream_append_record(stream, REC_TYPE_LU_VERSION, + &ver_rec, sizeof(ver_rec)); } @@ -34,6 +41,10 @@ int lu_save_all(struct kexec_image *image) ret = lu_save_domain(&stream, d); } + if (!ret) + ret = lu_stream_append_record(&stream, REC_TYPE_END, + NULL, 0); + if (!ret) ret = kimage_add_live_update_data(image, _mfn(virt_to_mfn(stream.pagelist)), diff --git a/xen/include/public/migration_stream.h b/xen/include/public/migration_stream.h index 92dd119f9f..29ed8cc2b5 100644 --- a/xen/include/public/migration_stream.h +++ b/xen/include/public/migration_stream.h @@ -51,6 +51,8 @@ struct mr_rhdr #define REC_TYPE_CHECKPOINT 0x0000000eU #define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST 0x0000000fU +#define REC_TYPE_LU_VERSION 0x40000000U + #define REC_TYPE_OPTIONAL 0x80000000U #define REC_TYPE_LIVE_UPDATE 0x40000000U @@ -113,6 +115,13 @@ struct mr_hvm_params struct mr_hvm_params_entry param[0]; }; +/* LU_VERSION */ +struct mr_lu_version +{ + uint32_t xen_major; + uint32_t xen_minor; +}; + #endif /* __XEN_MIGRATION_STREAM_H__ */ /* From patchwork Thu Jan 30 16:13:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C729114E3 for ; Thu, 30 Jan 2020 16:15:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A38742082E for ; Thu, 30 Jan 2020 16:15:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="X7iQOJbg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A38742082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSt-0004nL-Ph; Thu, 30 Jan 2020 16:14:47 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSr-0004ll-O9 for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:45 +0000 X-Inumbo-ID: 78c91d78-437b-11ea-b211-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78c91d78-437b-11ea-b211-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+EvjtNtQ/Zt64mtfAp1QVm2tBB6SY7x7nrE15LbB/oA=; b=X7iQOJbgC6Oc8YF9TDnqAqDLnt eZ4f1UjLXYkpmDxs4D7utHH3PylsHwl15lL/hn2FRbtv5QI1XnZuFMGhak4krfJYl4RI2/Tm8WGRD VKAxzgllworT5oEoIEmXu4zFrDG5cEIMPZJ4VlGRvniIsQK3nqAHlF8X6K+lw5whh/HPD8uIfmUb0 X7AGnyUh9Hf78QkfrWBK8kTCnA7NjL2yz1eNfHKNw/iWHdVbZoyefc6ub4Kzx5oRXsxXQgtlkHKyC FzTtWKcVOvo+sfIXnfr2+1x6ltQ8mwz7fUylFeJv8em+ugfM8nvXDdauJMo7h2Zt3CWUQOJybJuFY RNNDQwhw==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005To-D6; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009keH-JI; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:27 +0000 Message-Id: <20200130161330.2324143-19-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 19/22] Add shell of lu_reserve_pages() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse This currently only iterates over the records and prints the version of Xen that we're live updating from. In the fullness of time, it will also reserve the pages passed over as M2P as well as the pages belonging to preserved domains. Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 2 ++ xen/common/lu/Makefile | 2 +- xen/common/lu/restore.c | 34 ++++++++++++++++++++++++++++++++++ xen/include/xen/lu.h | 18 ++++++++++++++++++ 4 files changed, 55 insertions(+), 1 deletion(-) create mode 100644 xen/common/lu/restore.c diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index eea670e03b..f789713b1b 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1603,6 +1603,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( lu_breadcrumb_phys ) { lu_stream_map(&lu_stream, lu_mfnlist_phys, lu_nr_pages); + + lu_reserve_pages(&lu_stream); } if ( lu_bootmem_start ) diff --git a/xen/common/lu/Makefile b/xen/common/lu/Makefile index 7b7d975f65..592c72e1ec 100644 --- a/xen/common/lu/Makefile +++ b/xen/common/lu/Makefile @@ -1 +1 @@ -obj-y += stream.o save.o +obj-y += stream.o save.o restore.o diff --git a/xen/common/lu/restore.c b/xen/common/lu/restore.c new file mode 100644 index 0000000000..f52bb660d2 --- /dev/null +++ b/xen/common/lu/restore.c @@ -0,0 +1,34 @@ +#include +#include +#include +#include +#include + +#include + +void lu_reserve_pages(struct lu_stream *stream) +{ + struct mr_rhdr *hdr; + + while ( (hdr = lu_next_record(stream)) && hdr->type != REC_TYPE_END ) + { + if ( hdr->type == REC_TYPE_LU_VERSION && + hdr->length == sizeof(struct mr_lu_version) ) + { + struct mr_lu_version *vers = LU_REC_DATA(hdr); + + printk("Live update from Xen %d.%d\n", + vers->xen_major, vers->xen_minor); + } + } +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h index c02268e414..588f2dd137 100644 --- a/xen/include/xen/lu.h +++ b/xen/include/xen/lu.h @@ -5,6 +5,8 @@ #include #include +#include + #define LIVE_UPDATE_MAGIC (0x4c69766555706461UL & PAGE_MASK) struct lu_stream { @@ -28,6 +30,22 @@ struct kexec_image; int lu_save_all(struct kexec_image *image); void lu_stream_map(struct lu_stream *stream, unsigned long mfns_phys, int nr_pages); +void lu_reserve_pages(struct lu_stream *stream); + +/* Pointer to the data immediately following a record header */ +#define LU_REC_DATA(hdr) ((void *)&(hdr)[1]) + +static inline struct mr_rhdr *lu_next_record(struct lu_stream *stream) +{ + struct mr_rhdr *hdr = (struct mr_rhdr *)(stream->data + stream->last_hdr); + + if (stream->len < stream->last_hdr + sizeof(*hdr) || + stream->len < stream->last_hdr + sizeof(*hdr) + hdr->length) + return NULL; + + stream->last_hdr += sizeof(*hdr) + ROUNDUP(hdr->length, 1< X-Patchwork-Id: 11358317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16149139A for ; Thu, 30 Jan 2020 16:16:23 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DAE7B20707 for ; Thu, 30 Jan 2020 16:16:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ml6Qa2Bi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DAE7B20707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTS-0005K8-6U; Thu, 30 Jan 2020 16:15:22 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCTQ-0005Iv-Q3 for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:15:20 +0000 X-Inumbo-ID: 78cae798-437b-11ea-a933-bc764e2007e4 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 78cae798-437b-11ea-a933-bc764e2007e4; Thu, 30 Jan 2020 16:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=h+oTXoHnIIrP6hDWy90HME6sh9qFOM8efyxFfaYTYHA=; b=ml6Qa2BimcK1mSml0OacL9n6Ci OHEbZKZxl2PzoHdT/i7lLXwsXFpA7qBIm96O5ZlLJBeOom17RbIHLBThzFTP6ySgnnLrxII7DCHMJ m0Q7uNQVI+NjN0Kh2O73PI5BGDzXheyy53ucPa0rYr00hOxDvT6zL+CqfsaGxSZHMBBSvG9Jrtijd vUX2QFX7ay7KEcw8oK8qLYQ1esmj2fXxxm95o2PTN/azprEaRsdJlhQY78DOqIHwnPxsttEaQTMFm 87u89PjdSuV7THXAkZ2fAdtN22UGjEHjaMaUt9iqwEDPzhnzG++T35K2DYxRazAbV/IC8DLDcpre1 P3HpAyRA==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005Tp-Dt; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009keM-K6; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:28 +0000 Message-Id: <20200130161330.2324143-20-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 20/22] x86/setup: lift dom0 creation out into create_dom0 function X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse It's about to become optional as __start_xen() grows a different path for live update, so move it out of the way. Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 173 +++++++++++++++++++++++-------------------- 1 file changed, 94 insertions(+), 79 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index f789713b1b..ac93965be4 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -679,6 +679,92 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li return n; } +static struct domain * __init create_dom0(const module_t *image, + unsigned long headroom, + module_t *initrd, char *kextra, + char *loader) +{ + struct xen_domctl_createdomain dom0_cfg = { + .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0, + .max_evtchn_port = -1, + .max_grant_frames = -1, + .max_maptrack_frames = -1, + }; + struct domain *d; + char *cmdline; + + if ( opt_dom0_pvh ) + { + dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm | + ((hvm_hap_supported() && !opt_dom0_shadow) ? + XEN_DOMCTL_CDF_hap : 0)); + + dom0_cfg.arch.emulation_flags |= + XEN_X86_EMU_LAPIC | XEN_X86_EMU_IOAPIC | XEN_X86_EMU_VPCI; + } + dom0_cfg.max_vcpus = dom0_max_vcpus(); + + if ( iommu_enabled ) + dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu; + + /* Create initial domain 0. */ + d = domain_create(get_initial_domain_id(), &dom0_cfg, !pv_shim); + if ( IS_ERR(d) || (alloc_dom0_vcpu0(d) == NULL) ) + panic("Error creating domain 0\n"); + + /* Grab the DOM0 command line. */ + cmdline = (char *)(image->string ? __va(image->string) : NULL); + if ( (cmdline != NULL) || (kextra != NULL) ) + { + static char __initdata dom0_cmdline[MAX_GUEST_CMDLINE]; + + cmdline = cmdline_cook(cmdline, loader); + safe_strcpy(dom0_cmdline, cmdline); + + if ( kextra != NULL ) + /* kextra always includes exactly one leading space. */ + safe_strcat(dom0_cmdline, kextra); + + /* Append any extra parameters. */ + if ( skip_ioapic_setup && !strstr(dom0_cmdline, "noapic") ) + safe_strcat(dom0_cmdline, " noapic"); + if ( (strlen(acpi_param) == 0) && acpi_disabled ) + { + printk("ACPI is disabled, notifying Domain 0 (acpi=off)\n"); + safe_strcpy(acpi_param, "off"); + } + if ( (strlen(acpi_param) != 0) && !strstr(dom0_cmdline, "acpi=") ) + { + safe_strcat(dom0_cmdline, " acpi="); + safe_strcat(dom0_cmdline, acpi_param); + } + + cmdline = dom0_cmdline; + } + + /* + * Temporarily clear SMAP in CR4 to allow user-accesses in construct_dom0(). + * This saves a large number of corner cases interactions with + * copy_from_user(). + */ + if ( cpu_has_smap ) + { + cr4_pv32_mask &= ~X86_CR4_SMAP; + write_cr4(read_cr4() & ~X86_CR4_SMAP); + } + + if ( construct_dom0(d, image, headroom, initrd, cmdline) != 0 ) + panic("Could not construct domain 0\n"); + + if ( cpu_has_smap ) + { + write_cr4(read_cr4() | X86_CR4_SMAP); + cr4_pv32_mask |= X86_CR4_SMAP; + } + + return d; +} + /* How much of the directmap is prebuilt at compile time. */ #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT) @@ -739,12 +825,6 @@ void __init noreturn __start_xen(unsigned long mbi_p) .parity = 'n', .stop_bits = 1 }; - struct xen_domctl_createdomain dom0_cfg = { - .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0, - .max_evtchn_port = -1, - .max_grant_frames = -1, - .max_maptrack_frames = -1, - }; const char *hypervisor_name; uint64_t lu_mfnlist_phys = 0, lu_nr_pages = 0; struct lu_stream lu_stream; @@ -1889,94 +1969,29 @@ void __init noreturn __start_xen(unsigned long mbi_p) init_guest_cpuid(); init_guest_msr_policy(); - if ( opt_dom0_pvh ) - { - dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm | - ((hvm_hap_supported() && !opt_dom0_shadow) ? - XEN_DOMCTL_CDF_hap : 0)); - - dom0_cfg.arch.emulation_flags |= - XEN_X86_EMU_LAPIC | XEN_X86_EMU_IOAPIC | XEN_X86_EMU_VPCI; - } - dom0_cfg.max_vcpus = dom0_max_vcpus(); - - if ( iommu_enabled ) - dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu; - - /* Create initial domain 0. */ - dom0 = domain_create(get_initial_domain_id(), &dom0_cfg, !pv_shim); - if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) ) - panic("Error creating domain 0\n"); - - /* Grab the DOM0 command line. */ - cmdline = (char *)(mod[0].string ? __va(mod[0].string) : NULL); - if ( (cmdline != NULL) || (kextra != NULL) ) - { - static char __initdata dom0_cmdline[MAX_GUEST_CMDLINE]; - - cmdline = cmdline_cook(cmdline, loader); - safe_strcpy(dom0_cmdline, cmdline); - - if ( kextra != NULL ) - /* kextra always includes exactly one leading space. */ - safe_strcat(dom0_cmdline, kextra); - - /* Append any extra parameters. */ - if ( skip_ioapic_setup && !strstr(dom0_cmdline, "noapic") ) - safe_strcat(dom0_cmdline, " noapic"); - if ( (strlen(acpi_param) == 0) && acpi_disabled ) - { - printk("ACPI is disabled, notifying Domain 0 (acpi=off)\n"); - safe_strcpy(acpi_param, "off"); - } - if ( (strlen(acpi_param) != 0) && !strstr(dom0_cmdline, "acpi=") ) - { - safe_strcat(dom0_cmdline, " acpi="); - safe_strcat(dom0_cmdline, acpi_param); - } - - cmdline = dom0_cmdline; - } - if ( xen_cpuidle ) xen_processor_pmbits |= XEN_PROCESSOR_PM_CX; + printk("%sNX (Execute Disable) protection %sactive\n", + cpu_has_nx ? XENLOG_INFO : XENLOG_WARNING "Warning: ", + cpu_has_nx ? "" : "not "); + initrdidx = find_first_bit(module_map, mbi->mods_count); if ( bitmap_weight(module_map, mbi->mods_count) > 1 ) printk(XENLOG_WARNING "Multiple initrd candidates, picking module #%u\n", initrdidx); - /* - * Temporarily clear SMAP in CR4 to allow user-accesses in construct_dom0(). - * This saves a large number of corner cases interactions with - * copy_from_user(). - */ - if ( cpu_has_smap ) - { - cr4_pv32_mask &= ~X86_CR4_SMAP; - write_cr4(read_cr4() & ~X86_CR4_SMAP); - } - - printk("%sNX (Execute Disable) protection %sactive\n", - cpu_has_nx ? XENLOG_INFO : XENLOG_WARNING "Warning: ", - cpu_has_nx ? "" : "not "); - /* * We're going to setup domain0 using the module(s) that we stashed safely * above our heap. The second module, if present, is an initrd ramdisk. */ - if ( construct_dom0(dom0, mod, modules_headroom, - (initrdidx > 0) && (initrdidx < mbi->mods_count) - ? mod + initrdidx : NULL, cmdline) != 0) + dom0 = create_dom0(mod, modules_headroom, + (initrdidx > 0) && (initrdidx < mbi->mods_count) + ? mod + initrdidx : NULL, kextra, loader); + if ( dom0 == NULL ) panic("Could not set up DOM0 guest OS\n"); - if ( cpu_has_smap ) - { - write_cr4(read_cr4() | X86_CR4_SMAP); - cr4_pv32_mask |= X86_CR4_SMAP; - } - heap_init_late(); init_trace_bufs(); From patchwork Thu Jan 30 16:13:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0FAD139A for ; Thu, 30 Jan 2020 16:15:02 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C18520707 for ; Thu, 30 Jan 2020 16:15:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ivhdTqTw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C18520707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSJ-0004Lz-UI; Thu, 30 Jan 2020 16:14:11 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSI-0004LD-NT for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:10 +0000 X-Inumbo-ID: 75a8636a-437b-11ea-8396-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 75a8636a-437b-11ea-8396-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=lUrLxpsmvhxnZ4nJpEXnAKJQpB1NwPaxNMGSqfQEiMI=; b=ivhdTqTwXTEt52uLg9U9VhjQ8f BsQ7ogoOaM15fsQW57tNSEUJtWZI/IMAdo7E0lpuVJlLRM6vx48r9MYv+n1vchH7EAzyNMCfOivvZ KEbUL/dUpsuWo7q1no0DO7eaebMzqCXtTcvcrgucf7eZSwQXHt4yddllm+AP3B5/t16NOHw5iR41m Me9pXlyw+mdvxGe7TRbfV/G9ClQcxL2bMjPKloT0z0jlNq9XR0PP2U3rCief1OwCp6pPSyk8QO5R0 DdF8Qj0KEXROCHIq879FDgDppFixLEPF7DFdh4WxsPFcr05gpW8s+Yc8eE5iHUmpIYcy8ZZbTOzvI 9LtRUOEQ==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A5-Rk; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009keR-Kh; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:29 +0000 Message-Id: <20200130161330.2324143-21-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 21/22] x86/setup: finish plumbing in live update path through __start_xen() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse With this we are fairly much done hacking up __start_xen() to support live update. The live update functions themselves are still stubs, but now we can start populating those with actual save/restore of domain information. Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 52 +++++++++++++++++++++++++++-------------- xen/common/lu/restore.c | 5 ++++ xen/include/xen/lu.h | 2 ++ 3 files changed, 41 insertions(+), 18 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index ac93965be4..53f7b9ced4 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -816,7 +816,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) unsigned int initrdidx, num_parked = 0; multiboot_info_t *mbi; module_t *mod; - unsigned long nr_pages, raw_max_page, modules_headroom, module_map[1]; + unsigned long nr_pages, raw_max_page, modules_headroom = 0, module_map[1]; int i, j, e820_warn = 0, bytes = 0; bool acpi_boot_table_init_done = false, relocated = false, lu_reserved = false; int ret; @@ -992,7 +992,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) } bitmap_fill(module_map, mbi->mods_count); - __clear_bit(0, module_map); /* Dom0 kernel is always first */ + if ( !lu_breadcrumb_phys ) + __clear_bit(0, module_map); /* Dom0 kernel is always first */ if ( pvh_boot ) { @@ -1151,8 +1152,12 @@ void __init noreturn __start_xen(unsigned long mbi_p) mod[mbi->mods_count].mod_end = __2M_rwdata_end - _stext; } - modules_headroom = bzimage_headroom(bootstrap_map(mod), mod->mod_end); - bootstrap_map(NULL); + + if ( !lu_breadcrumb_phys ) + { + modules_headroom = bzimage_headroom(bootstrap_map(mod), mod->mod_end); + bootstrap_map(NULL); + } #ifndef highmem_start /* Don't allow split below 4Gb. */ @@ -1976,21 +1981,32 @@ void __init noreturn __start_xen(unsigned long mbi_p) cpu_has_nx ? XENLOG_INFO : XENLOG_WARNING "Warning: ", cpu_has_nx ? "" : "not "); - initrdidx = find_first_bit(module_map, mbi->mods_count); - if ( bitmap_weight(module_map, mbi->mods_count) > 1 ) - printk(XENLOG_WARNING - "Multiple initrd candidates, picking module #%u\n", - initrdidx); - /* - * We're going to setup domain0 using the module(s) that we stashed safely - * above our heap. The second module, if present, is an initrd ramdisk. - */ - dom0 = create_dom0(mod, modules_headroom, - (initrdidx > 0) && (initrdidx < mbi->mods_count) - ? mod + initrdidx : NULL, kextra, loader); - if ( dom0 == NULL ) - panic("Could not set up DOM0 guest OS\n"); + if ( lu_breadcrumb_phys ) + { + dom0 = lu_restore_domains(&lu_stream); + if ( dom0 == NULL ) + panic("No DOM0 found in live update data\n"); + + lu_stream_free(&lu_stream); + } + else + { + initrdidx = find_first_bit(module_map, mbi->mods_count); + if ( bitmap_weight(module_map, mbi->mods_count) > 1 ) + printk(XENLOG_WARNING + "Multiple initrd candidates, picking module #%u\n", + initrdidx); + /* + * We're going to setup domain0 using the module(s) that we stashed + * safely above our heap. The second module, if present, is an initrd. + */ + dom0 = create_dom0(mod, modules_headroom, + (initrdidx > 0) && (initrdidx < mbi->mods_count) + ? mod + initrdidx : NULL, kextra, loader); + if ( dom0 == NULL ) + panic("Could not set up DOM0 guest OS\n"); + } heap_init_late(); diff --git a/xen/common/lu/restore.c b/xen/common/lu/restore.c index f52bb660d2..163827f5de 100644 --- a/xen/common/lu/restore.c +++ b/xen/common/lu/restore.c @@ -23,6 +23,11 @@ void lu_reserve_pages(struct lu_stream *stream) } } +struct domain *lu_restore_domains(struct lu_stream *stream) +{ + panic("Implement me!\n"); +} + /* * Local variables: * mode: C diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h index 588f2dd137..817a88b77a 100644 --- a/xen/include/xen/lu.h +++ b/xen/include/xen/lu.h @@ -31,6 +31,8 @@ int lu_save_all(struct kexec_image *image); void lu_stream_map(struct lu_stream *stream, unsigned long mfns_phys, int nr_pages); void lu_reserve_pages(struct lu_stream *stream); +/* Returns Dom0 in case the architecture needs to do anything special to it */ +struct domain *lu_restore_domains(struct lu_stream *stream); /* Pointer to the data immediately following a record header */ #define LU_REC_DATA(hdr) ((void *)&(hdr)[1]) From patchwork Thu Jan 30 16:13:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11358287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AB6414E3 for ; Thu, 30 Jan 2020 16:15:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1658320707 for ; Thu, 30 Jan 2020 16:15:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="G1/6HPsM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1658320707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSE-0004Ib-HS; Thu, 30 Jan 2020 16:14:06 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixCSD-0004I9-MT for xen-devel@lists.xenproject.org; Thu, 30 Jan 2020 16:14:05 +0000 X-Inumbo-ID: 75a285c6-437b-11ea-ad98-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 75a285c6-437b-11ea-ad98-bc764e2007e4; Thu, 30 Jan 2020 16:13:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=T0s0/VAvwl0g9rp0gJ4yY/77JC3YwThEFwhhDArumQc=; b=G1/6HPsMgAzG3s+MOV47+gA3NV sfGWXzmNR3s5kh9gAonDoxdtbbYQcRmRhp3tRXi/xsftM8i8cjTcXMUi021fDJPsNqj9YRZ6/2BsY 5viv9TgQhUyeDogP1g/HzbWdjEjJJxrRfqSdH3fWgVVcOzy8wDgrOWXM+B3vQ9guTQLcQ2srQqV9g QQ4AfCYsSv307IN8Eox8ArlbNrEy5dspmlat49/EKdkFOT1tsFK7T7IyRPkg+8iadi35Ut3aEyXTQ P1zofnPq1Hv1epqfSy7zcN5rvUedscczOTzYfywxRrvUV6prUkhf3dPWgN4yrr0iDZ0IdoUysCw7C OFOSN1UQ==; Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixCRf-0005A6-SI; Thu, 30 Jan 2020 16:13:31 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1ixCRe-009keW-LF; Thu, 30 Jan 2020 16:13:30 +0000 From: David Woodhouse To: Xen-devel Date: Thu, 30 Jan 2020 16:13:30 +0000 Message-Id: <20200130161330.2324143-22-dwmw2@infradead.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200130161330.2324143-1-dwmw2@infradead.org> References: <20200130161330.2324143-1-dwmw2@infradead.org> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 22/22] x86/setup: simplify handling of initrdidx when no initrd present X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Remove a ternary operator that made my brain hurt and replace it with something simpler that makes it clearer that the >= mbi->mods_count is because of what find_first_bit() returns when it doesn't find anything. Just have a simple condition to set initrdidx to zero in that case, and a much simpler ternary operator in the create_dom0() call. Signed-off-by: David Woodhouse --- xen/arch/x86/setup.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 53f7b9ced4..6b3a5777cb 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1993,6 +1993,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) else { initrdidx = find_first_bit(module_map, mbi->mods_count); + if ( initrdidx >= mbi->mods_count ) + initrdidx = 0; + if ( bitmap_weight(module_map, mbi->mods_count) > 1 ) printk(XENLOG_WARNING "Multiple initrd candidates, picking module #%u\n", @@ -2002,8 +2005,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) * safely above our heap. The second module, if present, is an initrd. */ dom0 = create_dom0(mod, modules_headroom, - (initrdidx > 0) && (initrdidx < mbi->mods_count) - ? mod + initrdidx : NULL, kextra, loader); + initrdidx ? mod + initrdidx : NULL, + kextra, loader); if ( dom0 == NULL ) panic("Could not set up DOM0 guest OS\n"); } From patchwork Fri Jan 31 17:16:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11360299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 04FB514E3 for ; Fri, 31 Jan 2020 17:18:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CF946206D5 for ; Fri, 31 Jan 2020 17:18:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="WJxXJ+1I" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF946206D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZun-0003CE-Iu; Fri, 31 Jan 2020 17:17:09 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZun-0003C9-1m for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 17:17:09 +0000 X-Inumbo-ID: 7a3f2a8a-444d-11ea-8396-bc764e2007e4 Received: from bombadil.infradead.org (unknown [2607:7c80:54:e::133]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7a3f2a8a-444d-11ea-8396-bc764e2007e4; Fri, 31 Jan 2020 17:16:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Mime-Version:Content-Type:References: In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=pONlBV0QK/OoxCrphrHtXXUvEnLCkpzr5mmlIsqWVEo=; b=WJxXJ+1Iqtbe4dFhY73DHDKvp I9oW2frW7fr2ZaGOWHAmAFWPSMRgItBRm06n/LPTutClkC6WXwJdObEwPA9hfd8ZyETKXYWH667Dg xIz4CkKvmlewCBriNtvcC8G9VtjfUFjS6SiXXv9O41FgJbq0f2F9nvsVtPLuHIp17fUCcxMzIimx0 IQX0CadSaAsTZddP684FxGs1f+fz+3eMRJv0PpUDUuDPhGJyLvfWALS6w1XnLIUln8eIg0aChEqhA o9IYJV4LP2GoJBeELuORqrmnBw4BBvXTrctMAPjmit4GoaQAZhl4BX4+OQ+A3Htho+o6urjSOg7Dz G8w7GFSOw==; Received: from [54.239.6.185] (helo=vpn-10-85-96-156.fra54.corp.amazon.com) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ixZuW-0005sC-Im; Fri, 31 Jan 2020 17:16:52 +0000 Message-ID: From: David Woodhouse To: Xen-devel Date: Fri, 31 Jan 2020 17:16:48 +0000 In-Reply-To: References: X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.1 Mime-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Subject: [Xen-devel] [RFC PATCH v3 23/22] x86/smp: reset x2apic_enabled in smp_send_stop() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Varad Gautam , paul@xen.org, Ian Jackson , Hongyan Xia , Amit Shah , Roger Pau =?iso-8859-1?q?Monn=E9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: David Woodhouse Just before smp_send_stop() re-enables interrupts when shutting down for reboot or kexec, it calls __stop_this_cpu() which in turn calls disable_local_APIC(), which puts the APIC back in to the mode Xen found it in at boot. If that means turning x2APIC off and going back into xAPIC mode, then a timer interrupt occurring just after interrupts come back on will lead to a GP# when apic_timer_interrupt() attempts to ack the IRQ through the EOI register in x2APIC MSR 0x80b: (XEN) Executing kexec image on cpu0 (XEN) ----[ Xen-4.14-unstable x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[] apic_timer_interrupt+0x29/0x40 (XEN) RFLAGS: 0000000000010046 CONTEXT: hypervisor (XEN) rax: 0000000000000000 rbx: 00000000000000fa rcx: 000000000000080b … (XEN) Xen code around (apic_timer_interrupt+0x29/0x40): (XEN) c0 b9 0b 08 00 00 89 c2 <0f> 30 31 ff e9 0e c9 fb ff 0f 1f 40 00 66 2e 0f … (XEN) Xen call trace: (XEN) [] R apic_timer_interrupt+0x29/0x40 (XEN) [] S do_IRQ+0x95/0x750 … (XEN) [] S smp_send_stop+0x42/0xd0 We can't clear the global x2apic_enabled variable in disable_local_APIC() itself because that runs on each CPU. Instead, correct it (by using current_local_apic_mode()) in smp_send_stop() while interrupts are still disabled immediately after calling __stop_this_cpu() for the boot CPU, after all other CPUs have been stopped. cf: d639bdd9bbe ("x86/apic: Disable the LAPIC later in smp_send_stop()") ... which didn't quite fix it completely. Signed-off-by: David Woodhouse --- xen/arch/x86/smp.c | 1 + 1 file changed, 1 insertion(+) diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c index 65eb7cbda8..fac295fa6f 100644 --- a/xen/arch/x86/smp.c +++ b/xen/arch/x86/smp.c @@ -354,6 +354,7 @@ void smp_send_stop(void) disable_IO_APIC(); hpet_disable(); __stop_this_cpu(); + x2apic_enabled = (current_local_apic_mode() == APIC_MODE_X2APIC); local_irq_enable(); } }