From patchwork Thu Apr 30 20:44:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D5D251575 for ; Thu, 30 Apr 2020 20:46:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B264420836 for ; Thu, 30 Apr 2020 20:46:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="5iNBkOqM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B264420836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG32-0002op-IS; Thu, 30 Apr 2020 20:44:44 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG31-0002ok-Rk for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:43 +0000 X-Inumbo-ID: 691dd6bc-8b23-11ea-9aaf-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 691dd6bc-8b23-11ea-9aaf-12813bfff9fa; Thu, 30 Apr 2020 20:44:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=N6y7WNNIWMO3WocYdovup6f1egnxn9RTSj105NrfG5Q=; b=5iNBkOqMsqkMJcNQR3NM3JHRRb 2aO9ma52JHL8rPL36/oljF/PGHY5SZKIQQ0BFSJFXel6loih+ewgqTI6TlDfFduTOxG2/d/MGFbSi W706jUyccJkaU4kKJxiJoqsA5Ech9GyThky18TeWGyHfDCdcj0Il4eiEWpF8rbL9NJ5U=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG2w-0004Kt-MH; Thu, 30 Apr 2020 20:44:38 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG2w-0005wj-Bb; Thu, 30 Apr 2020 20:44:38 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 01/16] x86/setup: move vm_init() before acpi calls Date: Thu, 30 Apr 2020 21:44:10 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , julien@xen.org, Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Wei Liu After the direct map removal, pages from the boot allocator are not mapped at all in the direct map. Although we have map_domain_page, they are ephemeral and are less helpful for mappings that are more than a page, so we want a mechanism to globally map a range of pages, which is what vmap is for. Therefore, we bring vm_init into early boot stage. To allow vmap to be initialised and used in early boot, we need to modify vmap to receive pages from the boot allocator during early boot stage. Signed-off-by: Wei Liu Signed-off-by: David Woodhouse Signed-off-by: Hongyan Xia --- xen/arch/arm/setup.c | 4 ++-- xen/arch/x86/setup.c | 31 ++++++++++++++++++++----------- xen/common/vmap.c | 37 +++++++++++++++++++++++++++++-------- 3 files changed, 51 insertions(+), 21 deletions(-) diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 7968cee47d..8f0ac87419 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -822,6 +822,8 @@ void __init start_xen(unsigned long boot_phys_offset, setup_mm(); + vm_init(); + /* Parse the ACPI tables for possible boot-time configuration */ acpi_boot_table_init(); @@ -833,8 +835,6 @@ void __init start_xen(unsigned long boot_phys_offset, */ system_state = SYS_STATE_boot; - vm_init(); - if ( acpi_disabled ) { printk("Booting using Device Tree\n"); diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index fc0a6e5fcc..faca8c9758 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -695,6 +695,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) int i, j, e820_warn = 0, bytes = 0; bool acpi_boot_table_init_done = false, relocated = false; int ret; + bool vm_init_done = false; struct ns16550_defaults ns16550 = { .data_bits = 8, .parity = 'n', @@ -1301,12 +1302,23 @@ void __init noreturn __start_xen(unsigned long mbi_p) continue; if ( !acpi_boot_table_init_done && - s >= (1ULL << 32) && - !acpi_boot_table_init() ) + s >= (1ULL << 32) ) { - acpi_boot_table_init_done = true; - srat_parse_regions(s); - setup_max_pdx(raw_max_page); + /* + * We only initialise vmap and acpi after going through the bottom + * 4GiB, so that we have enough pages in the boot allocator. + */ + if ( !vm_init_done ) + { + vm_init(); + vm_init_done = true; + } + if ( !acpi_boot_table_init() ) + { + acpi_boot_table_init_done = true; + srat_parse_regions(s); + setup_max_pdx(raw_max_page); + } } if ( pfn_to_pdx((e - 1) >> PAGE_SHIFT) >= max_pdx ) @@ -1483,6 +1495,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) init_frametable(); + if ( !vm_init_done ) + vm_init(); + if ( !acpi_boot_table_init_done ) acpi_boot_table_init(); @@ -1520,12 +1535,6 @@ void __init noreturn __start_xen(unsigned long mbi_p) end_boot_allocator(); system_state = SYS_STATE_boot; - /* - * No calls involving ACPI code should go between the setting of - * SYS_STATE_boot and vm_init() (or else acpi_os_{,un}map_memory() - * will break). - */ - vm_init(); console_init_ring(); vesa_init(); diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 9964ab2096..e8533a8a80 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -35,9 +35,20 @@ void __init vm_init_type(enum vmap_region type, void *start, void *end) for ( i = 0, va = (unsigned long)vm_bitmap(type); i < nr; ++i, va += PAGE_SIZE ) { - struct page_info *pg = alloc_domheap_page(NULL, 0); + mfn_t mfn; + int rc; - map_pages_to_xen(va, page_to_mfn(pg), 1, PAGE_HYPERVISOR); + if ( system_state == SYS_STATE_early_boot ) + mfn = alloc_boot_pages(1, 1); + else + { + struct page_info *pg = alloc_domheap_page(NULL, 0); + + BUG_ON(!pg); + mfn = page_to_mfn(pg); + } + rc = map_pages_to_xen(va, mfn, 1, PAGE_HYPERVISOR); + BUG_ON(rc); clear_page((void *)va); } bitmap_fill(vm_bitmap(type), vm_low[type]); @@ -63,7 +74,7 @@ static void *vm_alloc(unsigned int nr, unsigned int align, spin_lock(&vm_lock); for ( ; ; ) { - struct page_info *pg; + mfn_t mfn; ASSERT(vm_low[t] == vm_top[t] || !test_bit(vm_low[t], vm_bitmap(t))); for ( start = vm_low[t]; start < vm_top[t]; ) @@ -98,9 +109,16 @@ static void *vm_alloc(unsigned int nr, unsigned int align, if ( vm_top[t] >= vm_end[t] ) return NULL; - pg = alloc_domheap_page(NULL, 0); - if ( !pg ) - return NULL; + if ( system_state == SYS_STATE_early_boot ) + mfn = alloc_boot_pages(1, 1); + else + { + struct page_info *pg = alloc_domheap_page(NULL, 0); + + if ( !pg ) + return NULL; + mfn = page_to_mfn(pg); + } spin_lock(&vm_lock); @@ -108,7 +126,7 @@ static void *vm_alloc(unsigned int nr, unsigned int align, { unsigned long va = (unsigned long)vm_bitmap(t) + vm_top[t] / 8; - if ( !map_pages_to_xen(va, page_to_mfn(pg), 1, PAGE_HYPERVISOR) ) + if ( !map_pages_to_xen(va, mfn, 1, PAGE_HYPERVISOR) ) { clear_page((void *)va); vm_top[t] += PAGE_SIZE * 8; @@ -118,7 +136,10 @@ static void *vm_alloc(unsigned int nr, unsigned int align, } } - free_domheap_page(pg); + if ( system_state == SYS_STATE_early_boot ) + init_boot_pages(mfn_to_maddr(mfn), mfn_to_maddr(mfn) + PAGE_SIZE); + else + free_domheap_page(mfn_to_page(mfn)); if ( start >= vm_top[t] ) { From patchwork Thu Apr 30 20:44:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E2471575 for ; Thu, 30 Apr 2020 20:46:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1AC7520836 for ; Thu, 30 Apr 2020 20:46:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="qCRIOJB3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1AC7520836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG31-0002oe-7q; Thu, 30 Apr 2020 20:44:43 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG2z-0002oZ-Gl for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:41 +0000 X-Inumbo-ID: 6a0b6dc8-8b23-11ea-b9cf-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 6a0b6dc8-8b23-11ea-b9cf-bc764e2007e4; Thu, 30 Apr 2020 20:44:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=mny4fPGI0W4kUckCGEUBvQNC9MU0BAlbEexMC4JDbrU=; b=qCRIOJB3HVKUWxuaCufWT128qC L1aqst12J/+1zHfzX8myHDSFQDCOCuKkgc5zDiDbEd8QTjWaXYlS5tiRH9JMfcrfWdTmJ5NyJ9vG+ d092LdAOlkOX5j3wDWzaG3EQ+zglhFK812VPu0KH2cHEk1rpOLNL8l1+emPs5w4O0Uw8=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG2y-0004L2-CB; Thu, 30 Apr 2020 20:44:40 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG2y-0005wj-2Q; Thu, 30 Apr 2020 20:44:40 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 02/16] acpi: vmap pages in acpi_os_alloc_memory Date: Thu, 30 Apr 2020 21:44:11 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , julien@xen.org, Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia Also, introduce a wrapper around vmap that maps a contiguous range for boot allocations. Signed-off-by: Hongyan Xia --- xen/drivers/acpi/osl.c | 9 ++++++++- xen/include/xen/vmap.h | 5 +++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/xen/drivers/acpi/osl.c b/xen/drivers/acpi/osl.c index 4c8bb7839e..d0762dad4e 100644 --- a/xen/drivers/acpi/osl.c +++ b/xen/drivers/acpi/osl.c @@ -219,7 +219,11 @@ void *__init acpi_os_alloc_memory(size_t sz) void *ptr; if (system_state == SYS_STATE_early_boot) - return mfn_to_virt(mfn_x(alloc_boot_pages(PFN_UP(sz), 1))); + { + mfn_t mfn = alloc_boot_pages(PFN_UP(sz), 1); + + return vmap_boot_pages(mfn, PFN_UP(sz)); + } ptr = xmalloc_bytes(sz); ASSERT(!ptr || is_xmalloc_memory(ptr)); @@ -244,5 +248,8 @@ void __init acpi_os_free_memory(void *ptr) if (is_xmalloc_memory(ptr)) xfree(ptr); else if (ptr && system_state == SYS_STATE_early_boot) + { + vunmap(ptr); init_boot_pages(__pa(ptr), __pa(ptr) + PAGE_SIZE); + } } diff --git a/xen/include/xen/vmap.h b/xen/include/xen/vmap.h index 369560e620..c70801e195 100644 --- a/xen/include/xen/vmap.h +++ b/xen/include/xen/vmap.h @@ -23,6 +23,11 @@ void *vmalloc_xen(size_t size); void *vzalloc(size_t size); void vfree(void *va); +static inline void *vmap_boot_pages(mfn_t mfn, unsigned int nr_pages) +{ + return __vmap(&mfn, nr_pages, 1, 1, PAGE_HYPERVISOR, VMAP_DEFAULT); +} + void __iomem *ioremap(paddr_t, size_t); static inline void iounmap(void __iomem *va) From patchwork Thu Apr 30 20:44:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521579 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD9A815E6 for ; Thu, 30 Apr 2020 20:46:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8A41720836 for ; Thu, 30 Apr 2020 20:46:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="439Wd2ro" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A41720836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG36-0002pZ-2Q; Thu, 30 Apr 2020 20:44:48 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG34-0002pO-Ev for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:46 +0000 X-Inumbo-ID: 6ad52000-8b23-11ea-ae69-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 6ad52000-8b23-11ea-ae69-bc764e2007e4; Thu, 30 Apr 2020 20:44:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=OnO+pVh1ubCsK0ipJDW4CA7CINr9chMuRxxDESUPwi4=; b=439Wd2ro1oGLaPPX0JahId3fzs 2UZjHr0o8rVmfbRuWbS7GpBZCoEd1PAGXxdfto+n1F2uWiEjq5qn4kL5lvCz0NR9EMQGJQYWPuIQz tmGc8oqYeouUa9D4QxWaoDQIiJh0JBe4ZuFGdz7L1nX6UgpjPr1SrSMOZMc5klNivpW4=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG2z-0004LB-Q1; Thu, 30 Apr 2020 20:44:41 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG2z-0005wj-GN; Thu, 30 Apr 2020 20:44:41 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 03/16] x86/numa: vmap the pages for memnodemap Date: Thu, 30 Apr 2020 21:44:12 +0100 Message-Id: <9a007a9965461127e2a3361cc1fde6a2d217ef48.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia This avoids the assumption that there is a direct map and boot pages fall inside the direct map. Clean up the variables so that mfn actually stores a type-safe mfn. Signed-off-by: Hongyan Xia --- xen/arch/x86/numa.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index f1066c59c7..51eca3f3fc 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -100,13 +100,13 @@ static int __init populate_memnodemap(const struct node *nodes, static int __init allocate_cachealigned_memnodemap(void) { unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap)); - unsigned long mfn = mfn_x(alloc_boot_pages(size, 1)); + mfn_t mfn = alloc_boot_pages(size, 1); - memnodemap = mfn_to_virt(mfn); - mfn <<= PAGE_SHIFT; + memnodemap = vmap_boot_pages(mfn, size); + BUG_ON(!memnodemap); size <<= PAGE_SHIFT; printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n", - mfn, mfn + size); + mfn_to_maddr(mfn), mfn_to_maddr(mfn) + size); memnodemapsize = size / sizeof(*memnodemap); return 0; From patchwork Thu Apr 30 20:44:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521577 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C3DE17EF for ; Thu, 30 Apr 2020 20:46:11 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 489B720836 for ; Thu, 30 Apr 2020 20:46:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="uYUxtIHD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 489B720836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG38-0002pu-CF; Thu, 30 Apr 2020 20:44:50 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG36-0002pi-Ro for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:48 +0000 X-Inumbo-ID: 6b9ad03e-8b23-11ea-9aaf-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 6b9ad03e-8b23-11ea-9aaf-12813bfff9fa; Thu, 30 Apr 2020 20:44:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=lV7eCfKft8Va6m4uNBBmoRi9XzqEneVzvGHlaIMLHds=; b=uYUxtIHD8MR/+KYumRVW16F+UA YB51HnVj3v7KqMEsddmEI8nPi+vxoKYfqM+BP5L7PANMjASGaMa6pcOdFqqRuSwTQzI0binfimCJo /e3ehVqSE2MyPpxxG9Xb3oYsvTQxLj8r80tfxjgnhMVQWR8TIH54CDxjtes744lpQQtQ=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG31-0004LH-7y; Thu, 30 Apr 2020 20:44:43 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG30-0005wj-US; Thu, 30 Apr 2020 20:44:43 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 04/16] x86/srat: vmap the pages for acpi_slit Date: Thu, 30 Apr 2020 21:44:13 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia This avoids the assumption that boot pages are in the direct map. Signed-off-by: Hongyan Xia --- xen/arch/x86/srat.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 506a56d66b..9a84c6c8a8 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -196,7 +196,8 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit) return; } mfn = alloc_boot_pages(PFN_UP(slit->header.length), 1); - acpi_slit = mfn_to_virt(mfn_x(mfn)); + acpi_slit = vmap_boot_pages(mfn, PFN_UP(slit->header.length)); + BUG_ON(!acpi_slit); memcpy(acpi_slit, slit, slit->header.length); } From patchwork Thu Apr 30 20:44:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521575 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D57801575 for ; Thu, 30 Apr 2020 20:46:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1AF120836 for ; Thu, 30 Apr 2020 20:46:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="cdbbpPvF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1AF120836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3A-0002qt-O6; Thu, 30 Apr 2020 20:44:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG39-0002qK-EF for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:51 +0000 X-Inumbo-ID: 6c920ebc-8b23-11ea-b9cf-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 6c920ebc-8b23-11ea-b9cf-bc764e2007e4; Thu, 30 Apr 2020 20:44:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=WTEV0oPtcbmzQwnNtr2I2AVsKwc6yGr7IZ4NoXd1nT4=; b=cdbbpPvFM/mqtoRn44qvXJUJ4E SAz077vwekyoVNG4sLAzgzr8i2FsuLB8SvwFl0rw55f/u0ElP8VlofANR8/aRzyeJOtHaQBSzKarT PiJ8oEW5NYkGxTPvhqeuh7OVM/GrUQcqaVtacXAAc/XRSz5Kxlryon/f9Swfq02cHHyA=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG32-0004LP-Mo; Thu, 30 Apr 2020 20:44:44 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG32-0005wj-Cc; Thu, 30 Apr 2020 20:44:44 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 05/16] x86: map/unmap pages in restore_all_guests. Date: Thu, 30 Apr 2020 21:44:14 +0100 Message-Id: <97a7680a1be26f6e34b91d29551747afa5235555.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia Before, it assumed the pv cr3 could be accessed via a direct map. This is no longer true. Note that we do not map and unmap root_pgt for now since it is still a xenheap page. Signed-off-by: Hongyan Xia --- xen/arch/x86/x86_64/entry.S | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index d55453f3f3..110cd0394f 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -154,7 +154,24 @@ restore_all_guest: and %rsi, %rdi and %r9, %rsi add %rcx, %rdi - add %rcx, %rsi + + /* + * Without a direct map, we have to map first before copying. We only + * need to map the guest root table but not the per-CPU root_pgt, + * because the latter is still a xenheap page. + */ + pushq %r9 + pushq %rdx + pushq %rax + pushq %rdi + mov %rsi, %rdi + shr $PAGE_SHIFT, %rdi + callq map_domain_page + mov %rax, %rsi + popq %rdi + /* Stash the pointer for unmapping later. */ + pushq %rax + mov $ROOT_PAGETABLE_FIRST_XEN_SLOT, %ecx mov root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rsi), %r8 mov %r8, root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rdi) @@ -166,6 +183,14 @@ restore_all_guest: sub $(ROOT_PAGETABLE_FIRST_XEN_SLOT - \ ROOT_PAGETABLE_LAST_XEN_SLOT - 1) * 8, %rdi rep movsq + + /* Unmap the page. */ + popq %rdi + callq unmap_domain_page + popq %rax + popq %rdx + popq %r9 + .Lrag_copy_done: mov %r9, STACK_CPUINFO_FIELD(xen_cr3)(%rdx) movb $1, STACK_CPUINFO_FIELD(use_pv_cr3)(%rdx) From patchwork Thu Apr 30 20:44:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521583 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F22EE1575 for ; Thu, 30 Apr 2020 20:46:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CEA1D20836 for ; Thu, 30 Apr 2020 20:46:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="j45FVkk1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CEA1D20836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3D-0002tG-3T; Thu, 30 Apr 2020 20:44:55 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3B-0002sV-S0 for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:53 +0000 X-Inumbo-ID: 6d691b32-8b23-11ea-9aaf-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 6d691b32-8b23-11ea-9aaf-12813bfff9fa; Thu, 30 Apr 2020 20:44:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=X5SxQC62czR9+3XEiKLbmZlfT+782Zyy4B2JOemF0eA=; b=j45FVkk1uPTyI4beeHYTMJtL1z cAUwlKi4AHWpsW4uytEQGmJBL73hKK9YgJKQkYOPVT536BsDSv774j3mGtEmYZMDLNq08gqzVhl4q xtvpSix5xmaT8beOkXfhCJO6MdHxw5Ll21HTa8sDNXhT9nYhmaNdew61fpVrnQwkKkKA=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG34-0004LV-3Z; Thu, 30 Apr 2020 20:44:46 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG33-0005wj-QZ; Thu, 30 Apr 2020 20:44:46 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 06/16] x86/pv: domheap pages should be mapped while relocating initrd Date: Thu, 30 Apr 2020 21:44:15 +0100 Message-Id: <535925f046bcc38fa26e2d5fd1c47c58f4e41b37.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Wei Liu Xen shouldn't use domheap page as if they were xenheap pages. Map and unmap pages accordingly. Signed-off-by: Wei Liu Signed-off-by: Wei Wang --- xen/arch/x86/pv/dom0_build.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index 3522eb0114..b052f13462 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -515,18 +515,31 @@ int __init dom0_construct_pv(struct domain *d, if ( d->arch.physaddr_bitsize && ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) ) { + unsigned long nr_pages; + unsigned long len = initrd_len; + order = get_order_from_pages(count); page = alloc_domheap_pages(d, order, MEMF_no_scrub); if ( !page ) panic("Not enough RAM for domain 0 initrd\n"); + + nr_pages = 1UL << order; for ( count = -count; order--; ) if ( count & (1UL << order) ) { free_domheap_pages(page, order); page += 1UL << order; + nr_pages -= 1UL << order; } - memcpy(page_to_virt(page), mfn_to_virt(initrd->mod_start), - initrd_len); + + for ( i = 0; i < nr_pages; i++, len -= PAGE_SIZE ) + { + void *p = __map_domain_page(page + i); + memcpy(p, mfn_to_virt(initrd_mfn + i), + min(len, (unsigned long)PAGE_SIZE)); + unmap_domain_page(p); + } + mpt_alloc = (paddr_t)initrd->mod_start << PAGE_SHIFT; init_domheap_pages(mpt_alloc, mpt_alloc + PAGE_ALIGN(initrd_len)); From patchwork Thu Apr 30 20:44:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521593 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2658A1575 for ; Thu, 30 Apr 2020 20:47:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 014A720836 for ; Thu, 30 Apr 2020 20:47:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="mGheIJdi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 014A720836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3F-0002uu-GP; Thu, 30 Apr 2020 20:44:57 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3E-0002uE-EI for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:56 +0000 X-Inumbo-ID: 6e33b0fe-8b23-11ea-b07b-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 6e33b0fe-8b23-11ea-b07b-bc764e2007e4; Thu, 30 Apr 2020 20:44:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=9eF+qb5V9Z9xxRC1WCmcjTYdHVKPY6+x95YE0mwwEYA=; b=mGheIJdijdnAzN7nyVUP750uWf MJSDx6ufAsJ9VQBVZLr42scaVu1w5KybOeVRGaDszwKaqWl+0SOPu0I7is5xmXlfVs1fS/VyiVEzK i2XNeB4bWSsmRjzm+bjMHQ/f+CsD90G0AzYJW56DgbT4jbXMp/DH9Y7PVd8n5msiAWVk=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG35-0004Lb-I3; Thu, 30 Apr 2020 20:44:47 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG35-0005wj-8Z; Thu, 30 Apr 2020 20:44:47 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 07/16] x86/pv: rewrite how building PV dom0 handles domheap mappings Date: Thu, 30 Apr 2020 21:44:16 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia Building a PV dom0 is allocating from the domheap but uses it like the xenheap. This is clearly wrong. Fix. Signed-off-by: Hongyan Xia --- xen/arch/x86/pv/dom0_build.c | 58 ++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index b052f13462..adaa6afda2 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -309,6 +309,10 @@ int __init dom0_construct_pv(struct domain *d, l3_pgentry_t *l3tab = NULL, *l3start = NULL; l2_pgentry_t *l2tab = NULL, *l2start = NULL; l1_pgentry_t *l1tab = NULL, *l1start = NULL; + mfn_t l4start_mfn = INVALID_MFN; + mfn_t l3start_mfn = INVALID_MFN; + mfn_t l2start_mfn = INVALID_MFN; + mfn_t l1start_mfn = INVALID_MFN; /* * This fully describes the memory layout of the initial domain. All @@ -535,6 +539,7 @@ int __init dom0_construct_pv(struct domain *d, for ( i = 0; i < nr_pages; i++, len -= PAGE_SIZE ) { void *p = __map_domain_page(page + i); + memcpy(p, mfn_to_virt(initrd_mfn + i), min(len, (unsigned long)PAGE_SIZE)); unmap_domain_page(p); @@ -610,23 +615,32 @@ int __init dom0_construct_pv(struct domain *d, v->arch.pv.event_callback_cs = FLAT_COMPAT_KERNEL_CS; } +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \ +do { \ + UNMAP_DOMAIN_PAGE(virt_var); \ + mfn_var = maddr_to_mfn(maddr); \ + maddr += PAGE_SIZE; \ + virt_var = map_domain_page(mfn_var); \ +} while ( false ) + /* WARNING: The new domain must have its 'processor' field filled in! */ if ( !is_pv_32bit_domain(d) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table; - l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc); + l4tab = l4start; clear_page(l4tab); - init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)), - d, INVALID_MFN, true); - v->arch.guest_table = pagetable_from_paddr(__pa(l4start)); + init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true); + v->arch.guest_table = pagetable_from_mfn(l4start_mfn); } else { /* Monitor table already created by switch_compat(). */ - l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table)); + l4start_mfn = pagetable_get_mfn(v->arch.guest_table); + l4start = l4tab = map_domain_page(l4start_mfn); /* See public/xen.h on why the following is needed. */ maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table; - l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc); } l4tab += l4_table_offset(v_start); @@ -636,14 +650,16 @@ int __init dom0_construct_pv(struct domain *d, if ( !((unsigned long)l1tab & (PAGE_SIZE-1)) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l1_page_table; - l1start = l1tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l1start_mfn, l1start, mpt_alloc); + l1tab = l1start; clear_page(l1tab); if ( count == 0 ) l1tab += l1_table_offset(v_start); if ( !((unsigned long)l2tab & (PAGE_SIZE-1)) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table; - l2start = l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc); + l2tab = l2start; clear_page(l2tab); if ( count == 0 ) l2tab += l2_table_offset(v_start); @@ -653,19 +669,19 @@ int __init dom0_construct_pv(struct domain *d, { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table; - l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc); } l3tab = l3start; clear_page(l3tab); if ( count == 0 ) l3tab += l3_table_offset(v_start); - *l4tab = l4e_from_paddr(__pa(l3start), L4_PROT); + *l4tab = l4e_from_mfn(l3start_mfn, L4_PROT); l4tab++; } - *l3tab = l3e_from_paddr(__pa(l2start), L3_PROT); + *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT); l3tab++; } - *l2tab = l2e_from_paddr(__pa(l1start), L2_PROT); + *l2tab = l2e_from_mfn(l1start_mfn, L2_PROT); l2tab++; } if ( count < initrd_pfn || count >= initrd_pfn + PFN_UP(initrd_len) ) @@ -692,9 +708,9 @@ int __init dom0_construct_pv(struct domain *d, if ( !l3e_get_intpte(*l3tab) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table; - l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; - clear_page(l2tab); - *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT); + UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc); + clear_page(l2start); + *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT); } if ( i == 3 ) l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2; @@ -705,9 +721,17 @@ int __init dom0_construct_pv(struct domain *d, unmap_domain_page(l2t); } +#undef UNMAP_MAP_AND_ADVANCE + + UNMAP_DOMAIN_PAGE(l1start); + UNMAP_DOMAIN_PAGE(l2start); + UNMAP_DOMAIN_PAGE(l3start); + /* Pages that are part of page tables must be read only. */ mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages); + UNMAP_DOMAIN_PAGE(l4start); + /* Mask all upcalls... */ for ( i = 0; i < XEN_LEGACY_MAX_VCPUS; i++ ) shared_info(d, vcpu_info[i].evtchn_upcall_mask) = 1; @@ -869,8 +893,12 @@ int __init dom0_construct_pv(struct domain *d, * !CONFIG_VIDEO case so the logic here can be simplified. */ if ( pv_shim ) + { + l4start = map_domain_page(l4start_mfn); pv_shim_setup_dom(d, l4start, v_start, vxenstore_start, vconsole_start, vphysmap_start, si); + UNMAP_DOMAIN_PAGE(l4start); + } if ( is_pv_32bit_domain(d) ) xlat_start_info(si, pv_shim ? XLAT_start_info_console_domU From patchwork Thu Apr 30 20:44:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521571 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 518AC1575 for ; Thu, 30 Apr 2020 20:45:17 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D8A020774 for ; Thu, 30 Apr 2020 20:45:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="Dw8P8ANR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D8A020774 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3H-0002wh-Sp; Thu, 30 Apr 2020 20:44:59 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3G-0002w3-Ry for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:44:58 +0000 X-Inumbo-ID: 6f1a72e6-8b23-11ea-9aaf-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 6f1a72e6-8b23-11ea-9aaf-12813bfff9fa; Thu, 30 Apr 2020 20:44:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=aO60ViVBWEWDKEHryOEaMwDjalPzlWCpuejw56ZNsYY=; b=Dw8P8ANR0KLecQjeUa+KKEQz93 ch1rbKMtixX5Z80yFeFqneXkG5PjZDPNCPUUdWDfhLW9277bgqsxg5/EBnV7KzRJfbtsD6/QPWNrY pN4b74Fx87xXCF1D/JO90ahcNTiVhBF0uAKWn3ow7BAqHu6WTlLRhUcTuOZwXe8irHVg=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG36-0004Li-Vm; Thu, 30 Apr 2020 20:44:48 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG36-0005wj-MQ; Thu, 30 Apr 2020 20:44:48 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 08/16] x86: add Persistent Map (PMAP) infrastructure Date: Thu, 30 Apr 2020 21:44:17 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Wei Liu The basic idea is like Persistent Kernel Map (PKMAP) in linux. We pre-populate all the relevant page tables before system is fully set up. It is needed to bootstrap map domain page infrastructure -- we need some way to map pages to set up the mapcache without a direct map. This infrastructure is not lock-protected therefore can only be used before smpboot. After smpboot, mapcache has to be used. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia --- xen/arch/x86/Makefile | 1 + xen/arch/x86/pmap.c | 87 ++++++++++++++++++++++++++++++++++++ xen/include/asm-x86/fixmap.h | 3 ++ xen/include/asm-x86/pmap.h | 10 +++++ 4 files changed, 101 insertions(+) create mode 100644 xen/arch/x86/pmap.c create mode 100644 xen/include/asm-x86/pmap.h diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile index 44137d919b..c8e565867b 100644 --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -52,6 +52,7 @@ obj-y += pci.o obj-y += percpu.o obj-y += physdev.o x86_64/physdev.o obj-y += platform_hypercall.o x86_64/platform_hypercall.o +obj-bin-y += pmap.init.o obj-y += psr.o obj-y += setup.o obj-y += shutdown.o diff --git a/xen/arch/x86/pmap.c b/xen/arch/x86/pmap.c new file mode 100644 index 0000000000..44d02ece89 --- /dev/null +++ b/xen/arch/x86/pmap.c @@ -0,0 +1,87 @@ +#include +#include +#include + +#include +#include +#include + +/* + * Simple mapping infrastructure to map / unmap pages in fixed map. + * This is used to set up the page table for mapcache, which is used + * by map domain page infrastructure. + * + * This structure is not protected by any locks, so it must not be used after + * smp bring-up. + */ + +/* Bitmap to track which slot is used */ +static unsigned long __initdata inuse; + +void *__init pmap_map(mfn_t mfn) +{ + unsigned long flags; + unsigned int idx; + void *linear = NULL; + enum fixed_addresses slot; + l1_pgentry_t *pl1e; + + BUILD_BUG_ON(sizeof(inuse) * BITS_PER_LONG < NUM_FIX_PMAP); + + ASSERT(system_state < SYS_STATE_smp_boot); + + local_irq_save(flags); + + idx = find_first_zero_bit(&inuse, NUM_FIX_PMAP); + if ( idx == NUM_FIX_PMAP ) + panic("Out of PMAP slots\n"); + + __set_bit(idx, &inuse); + + slot = idx + FIX_PMAP_BEGIN; + ASSERT(slot >= FIX_PMAP_BEGIN && slot <= FIX_PMAP_END); + + linear = fix_to_virt(slot); + /* + * We cannot use set_fixmap() here. We use PMAP when there is no direct map, + * so map_pages_to_xen() called by set_fixmap() needs to map pages on + * demand, which then calls pmap() again, resulting in a loop. Modify the + * PTEs directly instead. The same is true for pmap_unmap(). + */ + pl1e = &l1_fixmap[l1_table_offset((unsigned long)linear)]; + l1e_write_atomic(pl1e, l1e_from_mfn(mfn, PAGE_HYPERVISOR)); + + local_irq_restore(flags); + + return linear; +} + +void __init pmap_unmap(void *p) +{ + unsigned long flags; + unsigned int idx; + l1_pgentry_t *pl1e; + enum fixed_addresses slot = __virt_to_fix((unsigned long)p); + + ASSERT(system_state < SYS_STATE_smp_boot); + ASSERT(slot >= FIX_PMAP_BEGIN && slot <= FIX_PMAP_END); + + idx = slot - FIX_PMAP_BEGIN; + local_irq_save(flags); + + __clear_bit(idx, &inuse); + pl1e = &l1_fixmap[l1_table_offset((unsigned long)p)]; + l1e_write_atomic(pl1e, l1e_empty()); + flush_tlb_one_local(p); + + local_irq_restore(flags); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/asm-x86/fixmap.h b/xen/include/asm-x86/fixmap.h index 8330097a74..000f3b3375 100644 --- a/xen/include/asm-x86/fixmap.h +++ b/xen/include/asm-x86/fixmap.h @@ -24,6 +24,7 @@ #include #include #include +#include #include /* @@ -49,6 +50,8 @@ enum fixed_addresses { FIX_XEN_SHARED_INFO, #endif /* CONFIG_XEN_GUEST */ /* Everything else should go further down. */ + FIX_PMAP_BEGIN, + FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1, FIX_APIC_BASE, FIX_IO_APIC_BASE_0, FIX_IO_APIC_BASE_END = FIX_IO_APIC_BASE_0 + MAX_IO_APICS-1, diff --git a/xen/include/asm-x86/pmap.h b/xen/include/asm-x86/pmap.h new file mode 100644 index 0000000000..790cd71fb3 --- /dev/null +++ b/xen/include/asm-x86/pmap.h @@ -0,0 +1,10 @@ +#ifndef __X86_PMAP_H__ +#define __X86_PMAP_H__ + +/* Large enough for mapping 5 levels of page tables with some headroom */ +#define NUM_FIX_PMAP 8 + +void *pmap_map(mfn_t mfn); +void pmap_unmap(void *p); + +#endif /* __X86_PMAP_H__ */ From patchwork Thu Apr 30 20:44:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BFB4015E6 for ; Thu, 30 Apr 2020 20:46:26 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9CA1D214D8 for ; Thu, 30 Apr 2020 20:46:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="Tycjx5Ye" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9CA1D214D8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3K-0002zB-HR; Thu, 30 Apr 2020 20:45:02 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG3J-0002yJ-F3 for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:45:01 +0000 X-Inumbo-ID: 701ee8de-8b23-11ea-b07b-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 701ee8de-8b23-11ea-b07b-bc764e2007e4; Thu, 30 Apr 2020 20:44:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=wgpps+8lZONEwS5fLM70YV18IA19BevUWAZAtXCq3Wg=; b=Tycjx5YewosH0ZNVQ/Ai+rio1p sRQC/0oEorEhrqHDpNUyb9p7BJo/des8e7rOv6UuFAfSP9+nIxTNe0FT2TZOw/7qlW1ZM/tx/tpCt dxmMkwXkZfljbUe+crVha8JlCQBA/jA1mlUxFRaEyFHFQVfEZLa/IxXcjmxxA+vDI6PE=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG38-0004Lq-Dy; Thu, 30 Apr 2020 20:44:50 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG38-0005wj-4w; Thu, 30 Apr 2020 20:44:50 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 09/16] x86: lift mapcache variable to the arch level Date: Thu, 30 Apr 2020 21:44:18 +0100 Message-Id: <0794bccd94b7613dafab85ecc30a832229af7997.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Wei Liu It is going to be needed by HVM and idle domain as well, because without the direct map, both need a mapcache to map pages. This only lifts the mapcache variable up. Whether we populate the mapcache for a domain is unchanged in this patch. Signed-off-by: Wei Liu Signed-off-by: Wei Wang Signed-off-by: Hongyan Xia --- xen/arch/x86/domain.c | 4 ++-- xen/arch/x86/domain_page.c | 22 ++++++++++------------ xen/include/asm-x86/domain.h | 12 ++++++------ 3 files changed, 18 insertions(+), 20 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index a4428190d5..e73f1efe85 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -634,6 +634,8 @@ int arch_domain_create(struct domain *d, psr_domain_init(d); + mapcache_domain_init(d); + if ( is_hvm_domain(d) ) { if ( (rc = hvm_domain_initialise(d)) != 0 ) @@ -641,8 +643,6 @@ int arch_domain_create(struct domain *d, } else if ( is_pv_domain(d) ) { - mapcache_domain_init(d); - if ( (rc = pv_domain_initialise(d)) != 0 ) goto fail; } diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 3a244bb500..7b22e7c6ed 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn) #endif v = mapcache_current_vcpu(); - if ( !v || !is_pv_vcpu(v) ) + if ( !v ) return mfn_to_virt(mfn_x(mfn)); - dcache = &v->domain->arch.pv.mapcache; - vcache = &v->arch.pv.mapcache; + dcache = &v->domain->arch.mapcache; + vcache = &v->arch.mapcache; if ( !dcache->inuse ) return mfn_to_virt(mfn_x(mfn)); @@ -187,14 +187,14 @@ void unmap_domain_page(const void *ptr) ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); v = mapcache_current_vcpu(); - ASSERT(v && is_pv_vcpu(v)); + ASSERT(v); - dcache = &v->domain->arch.pv.mapcache; + dcache = &v->domain->arch.mapcache; ASSERT(dcache->inuse); idx = PFN_DOWN(va - MAPCACHE_VIRT_START); mfn = l1e_get_pfn(MAPCACHE_L1ENT(idx)); - hashent = &v->arch.pv.mapcache.hash[MAPHASH_HASHFN(mfn)]; + hashent = &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)]; local_irq_save(flags); @@ -233,11 +233,9 @@ void unmap_domain_page(const void *ptr) int mapcache_domain_init(struct domain *d) { - struct mapcache_domain *dcache = &d->arch.pv.mapcache; + struct mapcache_domain *dcache = &d->arch.mapcache; unsigned int bitmap_pages; - ASSERT(is_pv_domain(d)); - #ifdef NDEBUG if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) return 0; @@ -261,12 +259,12 @@ int mapcache_domain_init(struct domain *d) int mapcache_vcpu_init(struct vcpu *v) { struct domain *d = v->domain; - struct mapcache_domain *dcache = &d->arch.pv.mapcache; + struct mapcache_domain *dcache = &d->arch.mapcache; unsigned long i; unsigned int ents = d->max_vcpus * MAPCACHE_VCPU_ENTRIES; unsigned int nr = PFN_UP(BITS_TO_LONGS(ents) * sizeof(long)); - if ( !is_pv_vcpu(v) || !dcache->inuse ) + if ( !dcache->inuse ) return 0; if ( ents > dcache->entries ) @@ -293,7 +291,7 @@ int mapcache_vcpu_init(struct vcpu *v) BUILD_BUG_ON(MAPHASHENT_NOTINUSE < MAPCACHE_ENTRIES); for ( i = 0; i < MAPHASH_ENTRIES; i++ ) { - struct vcpu_maphash_entry *hashent = &v->arch.pv.mapcache.hash[i]; + struct vcpu_maphash_entry *hashent = &v->arch.mapcache.hash[i]; hashent->mfn = ~0UL; /* never valid to map */ hashent->idx = MAPHASHENT_NOTINUSE; diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 5b6d909266..1cee04c0c5 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -271,9 +271,6 @@ struct pv_domain /* Mitigate L1TF with shadow/crashing? */ bool check_l1tf; - /* map_domain_page() mapping cache. */ - struct mapcache_domain mapcache; - struct cpuidmasks *cpuidmasks; }; @@ -306,6 +303,9 @@ struct arch_domain uint32_t pci_cf8; uint8_t cmos_idx; + /* map_domain_page() mapping cache. */ + struct mapcache_domain mapcache; + union { struct pv_domain pv; struct hvm_domain hvm; @@ -482,9 +482,6 @@ struct arch_domain struct pv_vcpu { - /* map_domain_page() mapping cache. */ - struct mapcache_vcpu mapcache; - unsigned int vgc_flags; struct trap_info *trap_ctxt; @@ -567,6 +564,9 @@ struct arch_vcpu #define async_exception_state(t) async_exception_state[(t)-1] uint8_t async_exception_mask; + /* map_domain_page() mapping cache. */ + struct mapcache_vcpu mapcache; + /* Virtual Machine Extensions */ union { struct pv_vcpu pv; From patchwork Thu Apr 30 20:44:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E8A01575 for ; Thu, 30 Apr 2020 20:51:00 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1AC8A20836 for ; Thu, 30 Apr 2020 20:51:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="TtOV/Rvo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1AC8A20836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8V-0004SI-Aj; Thu, 30 Apr 2020 20:50:23 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8T-0004Rl-SK for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:21 +0000 X-Inumbo-ID: 2e6e8f8a-8b24-11ea-9ab3-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2e6e8f8a-8b24-11ea-9ab3-12813bfff9fa; Thu, 30 Apr 2020 20:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=oWkkTLJs5F+dmSX55OPdYYCmMvWzu4b2BjRHw0fEjkE=; b=TtOV/RvoLDyCPdZUmHtj1s0Jid vRyibSHL+ihp465M1yP98RBGRCnp5xgGzDAJYO3TqcqSYhu0+hO8dhtJmYYcQazwyIw3wkoGAzylc jm42BGa4sLwwe2Hfj5Gy8ugfTNFpxFeJxh3K9ZFk0DzD9OpmSJdmQziZ1l5qlaJyC9q4=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004Ub-MK; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG39-0005wj-JL; Thu, 30 Apr 2020 20:44:51 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 10/16] x86/mapcache: initialise the mapcache for the idle domain Date: Thu, 30 Apr 2020 21:44:19 +0100 Message-Id: <17fd1408a5f9e3ff2ed1f8d7f4f68427448df417.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia In order to use the mapcache in the idle domain, we also have to populate its page tables in the PERDOMAIN region, and we need to move mapcache_domain_init() earlier in arch_domain_create(). Note, commit 'x86: lift mapcache variable to the arch level' has initialised the mapcache for HVM domains. With this patch, PV, HVM, idle domains now all initialise the mapcache. Signed-off-by: Wei Wang Signed-off-by: Hongyan Xia --- xen/arch/x86/domain.c | 4 ++-- xen/arch/x86/mm.c | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index e73f1efe85..c7e90c50e6 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -539,6 +539,8 @@ int arch_domain_create(struct domain *d, spin_lock_init(&d->arch.e820_lock); + mapcache_domain_init(d); + /* Minimal initialisation for the idle domain. */ if ( unlikely(is_idle_domain(d)) ) { @@ -634,8 +636,6 @@ int arch_domain_create(struct domain *d, psr_domain_init(d); - mapcache_domain_init(d); - if ( is_hvm_domain(d) ) { if ( (rc = hvm_domain_initialise(d)) != 0 ) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index a17ae0004a..b3530d2763 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5828,6 +5828,9 @@ int create_perdomain_mapping(struct domain *d, unsigned long va, l3tab = __map_domain_page(pg); clear_page(l3tab); d->arch.perdomain_l3_pg = pg; + if ( is_idle_domain(d) ) + idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] = + l4e_from_page(pg, __PAGE_HYPERVISOR_RW); if ( !nr ) { unmap_domain_page(l3tab); From patchwork Thu Apr 30 20:44:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2BF2A1575 for ; Thu, 30 Apr 2020 20:51:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0883820836 for ; Thu, 30 Apr 2020 20:51:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="ygXMYmWs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0883820836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8K-0004PE-Je; Thu, 30 Apr 2020 20:50:12 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8J-0004Ow-U7 for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:11 +0000 X-Inumbo-ID: 2ed536a2-8b24-11ea-9ab3-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2ed536a2-8b24-11ea-9ab3-12813bfff9fa; Thu, 30 Apr 2020 20:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=PcycubMAuSqGSuNs6t/y8u3H4ytKIm03X5Wiy+SUCcU=; b=ygXMYmWskqVSzW591ZTOFCGThF YklHwIBhupZhlHTHjsqn+y/Un9pUUSrRKv+c5C36KmMe7ZWmbp/jWMhmTbaM67vmZ0fDRJQhUJzy4 8OZg8fxFJ0iFV3WJXsUQC8HKjLLm3NZTRLSm/3IaNEsxjuOWsFs74oiqKt9Pnb7Jk+cY=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004UV-Es; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG3B-0005wj-IK; Thu, 30 Apr 2020 20:44:53 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 11/16] x86: add a boot option to enable and disable the direct map Date: Thu, 30 Apr 2020 21:44:20 +0100 Message-Id: <7360b59e8fd39796fee56430a437b20c948d08c2.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , julien@xen.org, Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia Also add a helper function to retrieve it. Change arch_mfn_in_direct_map to check this option before returning. This is added as a boot command line option, not a Kconfig. We do not produce different builds for EC2 so this is not introduced as a compile-time configuration. Signed-off-by: Hongyan Xia --- docs/misc/xen-command-line.pandoc | 12 ++++++++++++ xen/arch/x86/mm.c | 3 +++ xen/arch/x86/setup.c | 2 ++ xen/include/asm-arm/mm.h | 5 +++++ xen/include/asm-x86/mm.h | 17 ++++++++++++++++- 5 files changed, 38 insertions(+), 1 deletion(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index ee12b0f53f..7027e3a15c 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -652,6 +652,18 @@ Specify the size of the console debug trace buffer. By specifying `cpu:` additionally a trace buffer of the specified size is allocated per cpu. The debug trace feature is only enabled in debugging builds of Xen. +### directmap (x86) +> `= ` + +> Default: `true` + +Enable or disable the direct map region in Xen. + +By default, Xen creates the direct map region which maps physical memory +in that region. Setting this to no will remove the direct map, blocking +exploits that leak secrets via speculative memory access in the direct +map. + ### dma_bits > `= ` diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index b3530d2763..64da997764 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -162,6 +162,9 @@ l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) l1_fixmap_x[L1_PAGETABLE_ENTRIES]; +bool __read_mostly opt_directmap = true; +boolean_param("directmap", opt_directmap); + paddr_t __read_mostly mem_hotplug; /* Frame table size in pages. */ diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index faca8c9758..60fc4038be 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1282,6 +1282,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( highmem_start ) xenheap_max_mfn(PFN_DOWN(highmem_start - 1)); + printk("Booting with directmap %s\n", arch_has_directmap() ? "on" : "off"); + /* * Walk every RAM region and map it in its entirety (on x86/64, at least) * and notify it to the boot allocator. diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h index 7df91280bc..e6fd934113 100644 --- a/xen/include/asm-arm/mm.h +++ b/xen/include/asm-arm/mm.h @@ -366,6 +366,11 @@ int arch_acquire_resource(struct domain *d, unsigned int type, unsigned int id, return -EOPNOTSUPP; } +static inline bool arch_has_directmap(void) +{ + return true; +} + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index ef7a20ac7d..7ff99ee8e3 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -454,6 +454,8 @@ int check_descriptor(const struct domain *d, seg_desc_t *desc); extern paddr_t mem_hotplug; +extern bool opt_directmap; + /****************************************************************************** * With shadow pagetables, the different kinds of address start * to get get confusing. @@ -637,13 +639,26 @@ extern const char zero_page[]; /* Build a 32bit PSE page table using 4MB pages. */ void write_32bit_pse_identmap(uint32_t *l2); +static inline bool arch_has_directmap(void) +{ + return opt_directmap; +} + /* * x86 maps part of physical memory via the directmap region. * Return whether the input MFN falls in that range. + * + * When boot command line sets directmap=no, we will not have a direct map at + * all so this will always return false. */ static inline bool arch_mfn_in_directmap(unsigned long mfn) { - unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END); + unsigned long eva; + + if ( !arch_has_directmap() ) + return false; + + eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END); return mfn <= (virt_to_mfn(eva - 1) + 1); } From patchwork Thu Apr 30 20:44:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFF8115E6 for ; Thu, 30 Apr 2020 20:51:18 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ACAA020836 for ; Thu, 30 Apr 2020 20:51:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="myk0BmGz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ACAA020836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8f-0004Z0-8q; Thu, 30 Apr 2020 20:50:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8d-0004YD-Sn for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:31 +0000 X-Inumbo-ID: 2ef3fdc6-8b24-11ea-9ab3-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2ef3fdc6-8b24-11ea-9ab3-12813bfff9fa; Thu, 30 Apr 2020 20:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Pf2FpaoK+GE72TCueex6j/niFJJvks21euODf44DpKQ=; b=myk0BmGzVF8fak/lksv7yEF7pT Mp4P90HlVzy+aldv4qpMsqb3GS79SKDnwbvhBvYs9haauxpfTXMD/J+xFHgXBS95YqVnglhAUQl0I lHqbeROnHMkm1cgsVEMu1bymFqhvS+dfVuYXNAgcyzdqOPhLSnvtQBDSsysu/+0vsqJ4=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004Uh-QR; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG3C-0005wj-Vj; Thu, 30 Apr 2020 20:44:55 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 12/16] x86/domain_page: remove the fast paths when mfn is not in the directmap Date: Thu, 30 Apr 2020 21:44:21 +0100 Message-Id: <72c1d30ea95756ff6c8e14b084d7dd2e74b83043.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia When mfn is not in direct map, never use mfn_to_virt for any mappings. We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with arch_mfn_in_direct_map(mfn) because these two are equivalent. The extra comparison in arch_mfn_in_direct_map() looks different but because DIRECTMAP_VIRT_END is always higher, it does not make any difference. Signed-off-by: Hongyan Xia --- xen/arch/x86/domain_page.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 7b22e7c6ed..fc705f056e 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -14,8 +14,10 @@ #include #include #include +#include #include #include +#include #include static DEFINE_PER_CPU(struct vcpu *, override); @@ -35,10 +37,11 @@ static inline struct vcpu *mapcache_current_vcpu(void) /* * When using efi runtime page tables, we have the equivalent of the idle * domain's page tables but current may point at another domain's VCPU. - * Return NULL as though current is not properly set up yet. + * Return the idle domains's vcpu on that core because the efi per-domain + * region (where the mapcache is) is in-sync with the idle domain. */ if ( efi_rs_using_pgtables() ) - return NULL; + return idle_vcpu[smp_processor_id()]; /* * If guest_table is NULL, and we are running a paravirtualised guest, @@ -77,18 +80,24 @@ void *map_domain_page(mfn_t mfn) struct vcpu_maphash_entry *hashent; #ifdef NDEBUG - if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfn_in_directmap(mfn_x(mfn)) ) return mfn_to_virt(mfn_x(mfn)); #endif v = mapcache_current_vcpu(); - if ( !v ) - return mfn_to_virt(mfn_x(mfn)); + if ( !v || !v->domain->arch.mapcache.inuse ) + { + if ( arch_mfn_in_directmap(mfn_x(mfn)) ) + return mfn_to_virt(mfn_x(mfn)); + else + { + BUG_ON(system_state >= SYS_STATE_smp_boot); + return pmap_map(mfn); + } + } dcache = &v->domain->arch.mapcache; vcache = &v->arch.mapcache; - if ( !dcache->inuse ) - return mfn_to_virt(mfn_x(mfn)); perfc_incr(map_domain_page_count); @@ -184,6 +193,12 @@ void unmap_domain_page(const void *ptr) if ( va >= DIRECTMAP_VIRT_START ) return; + if ( va >= FIXADDR_START && va < FIXADDR_TOP ) + { + pmap_unmap((void *)ptr); + return; + } + ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); v = mapcache_current_vcpu(); @@ -237,7 +252,7 @@ int mapcache_domain_init(struct domain *d) unsigned int bitmap_pages; #ifdef NDEBUG - if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( !mem_hotplug && arch_mfn_in_directmap(max_page) ) return 0; #endif @@ -308,7 +323,7 @@ void *map_domain_page_global(mfn_t mfn) local_irq_is_enabled())); #ifdef NDEBUG - if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfn_in_directmap(mfn_x(mfn)) ) return mfn_to_virt(mfn_x(mfn)); #endif From patchwork Thu Apr 30 20:44:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B66EE1575 for ; Thu, 30 Apr 2020 20:51:13 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92ABA20836 for ; Thu, 30 Apr 2020 20:51:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="aey+DHDq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92ABA20836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8Z-0004Vq-Sn; Thu, 30 Apr 2020 20:50:27 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8Y-0004VJ-Sb for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:26 +0000 X-Inumbo-ID: 2e6e8f89-8b24-11ea-9ab3-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2e6e8f89-8b24-11ea-9ab3-12813bfff9fa; Thu, 30 Apr 2020 20:50:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=uiGyNUH2p33cZ+IUo5Oln93DkD9Vx7DyiuUT3pjDMT0=; b=aey+DHDqSODtkbq0t9Hwwo/d3g EP6M4/fDeLHgbGAz+AerEAn1WQ5DNbCAzIQQz0lmxW78Y5aSCThHx7v4Xvhq/TTlvKhO/ly8+KIo5 GdSKM2A33MZigBhvfQoJ10nafkunwzZbG9KpaKSH6sfagQ3b73ZKT/PVzl7baGsWDFA8=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004UX-I4; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG3E-0005wj-LT; Thu, 30 Apr 2020 20:44:56 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 13/16] xen/page_alloc: add a path for xenheap when there is no direct map Date: Thu, 30 Apr 2020 21:44:22 +0100 Message-Id: <32ae7c14babf7e78b60febb53095a74c5e865456.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , julien@xen.org, Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia When there is not an always-mapped direct map, xenheap allocations need to be mapped and unmapped on-demand. Signed-off-by: Hongyan Xia --- xen/common/page_alloc.c | 45 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 10b7aeca48..1285fc5977 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -2143,6 +2143,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe) void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) { struct page_info *pg; + void *ret; ASSERT(!in_irq()); @@ -2151,14 +2152,27 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) if ( unlikely(pg == NULL) ) return NULL; - memguard_unguard_range(page_to_virt(pg), 1 << (order + PAGE_SHIFT)); + ret = page_to_virt(pg); - return page_to_virt(pg); + if ( !arch_has_directmap() && + map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order, + PAGE_HYPERVISOR) ) + { + /* Failed to map xenheap pages. */ + free_heap_pages(pg, order, false); + return NULL; + } + + memguard_unguard_range(ret, 1 << (order + PAGE_SHIFT)); + + return ret; } void free_xenheap_pages(void *v, unsigned int order) { + unsigned long va = (unsigned long)v & PAGE_MASK; + ASSERT(!in_irq()); if ( v == NULL ) @@ -2166,6 +2180,12 @@ void free_xenheap_pages(void *v, unsigned int order) memguard_guard_range(v, 1 << (order + PAGE_SHIFT)); + if ( !arch_has_directmap() && + destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) ) + dprintk(XENLOG_WARNING, + "Error while destroying xenheap mappings at %p, order %u\n", + v, order) + free_heap_pages(virt_to_page(v), order, false); } @@ -2189,6 +2209,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) { struct page_info *pg; unsigned int i; + void *ret; ASSERT(!in_irq()); @@ -2201,16 +2222,28 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) if ( unlikely(pg == NULL) ) return NULL; + ret = page_to_virt(pg); + + if ( !arch_has_directmap() && + map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order, + PAGE_HYPERVISOR) ) + { + /* Failed to map xenheap pages. */ + free_domheap_pages(pg, order); + return NULL; + } + for ( i = 0; i < (1u << order); i++ ) pg[i].count_info |= PGC_xen_heap; - return page_to_virt(pg); + return ret; } void free_xenheap_pages(void *v, unsigned int order) { struct page_info *pg; unsigned int i; + unsigned long va = (unsigned long)v & PAGE_MASK; ASSERT(!in_irq()); @@ -2222,6 +2255,12 @@ void free_xenheap_pages(void *v, unsigned int order) for ( i = 0; i < (1u << order); i++ ) pg[i].count_info &= ~PGC_xen_heap; + if ( !arch_has_directmap() && + destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) ) + dprintk(XENLOG_WARNING, + "Error while destroying xenheap mappings at %p, order %u\n", + v, order); + free_heap_pages(pg, order, true); } From patchwork Thu Apr 30 20:44:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521611 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4A5E15E6 for ; Thu, 30 Apr 2020 20:51:43 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0E2820836 for ; Thu, 30 Apr 2020 20:51:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="HZ3Q9E4S" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0E2820836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8j-0004bx-KG; Thu, 30 Apr 2020 20:50:37 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8i-0004bO-Sh for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:36 +0000 X-Inumbo-ID: 2dcb9211-8b24-11ea-9ab3-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2dcb9211-8b24-11ea-9ab3-12813bfff9fa; Thu, 30 Apr 2020 20:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=tF+dQ9SLZTmEavdmofAbgYKyBZJqPPcUUiyb5AWHxN8=; b=HZ3Q9E4S5yiVy8k/Mfc9UAUXsi O1sw7XgIi6nF741qocRN/95JoYovNWdGd3bBrj+aNrItkexAW57YfkTs4HkGKyEaNkKkFFYXdfVhU eMd1JKLU1yKKp4Z4KPUjwrS/sMnihHtyOyD16eQyYgRoJFaTzFqR1qwJ94oMjm7IhaEY=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004Un-SN; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG3G-0005wj-3A; Thu, 30 Apr 2020 20:44:58 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 14/16] x86/setup: leave early boot slightly earlier Date: Thu, 30 Apr 2020 21:44:23 +0100 Message-Id: <446c0b3b1811574d155cb84827e4fc64e3425413.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia When we do not have a direct map, memory for metadata of heap nodes in init_node_heap() is allocated from xenheap, which needs to be mapped and unmapped on demand. However, we cannot just take memory from the boot allocator to create the PTEs while we are passing memory to the heap allocator. To solve this race, we leave early boot slightly sooner so that Xen PTE pages are allocated from the heap instead of the boot allocator. We can do this because the metadata for the 1st node is statically allocated, and by the time we need memory to create mappings for the 2nd node, we already have enough memory in the heap allocator in the 1st node. Signed-off-by: Hongyan Xia --- xen/arch/x86/setup.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 60fc4038be..dbb2ac1c8f 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1507,6 +1507,22 @@ void __init noreturn __start_xen(unsigned long mbi_p) numa_initmem_init(0, raw_max_page); + /* + * When we do not have a direct map, memory for metadata of heap nodes in + * init_node_heap() is allocated from xenheap, which needs to be mapped and + * unmapped on demand. However, we cannot just take memory from the boot + * allocator to create the PTEs while we are passing memory to the heap + * allocator during end_boot_allocator(). + * + * To solve this race, we need to leave early boot before + * end_boot_allocator() so that Xen PTE pages are allocated from the heap + * instead of the boot allocator. We can do this because the metadata for + * the 1st node is statically allocated, and by the time we need memory to + * create mappings for the 2nd node, we already have enough memory in the + * heap allocator in the 1st node. + */ + system_state = SYS_STATE_boot; + if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) ) { unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); @@ -1536,8 +1552,6 @@ void __init noreturn __start_xen(unsigned long mbi_p) else end_boot_allocator(); - system_state = SYS_STATE_boot; - console_init_ring(); vesa_init(); From patchwork Thu Apr 30 20:44:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 343EF1575 for ; Thu, 30 Apr 2020 20:51:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 10DE520836 for ; Thu, 30 Apr 2020 20:51:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="DShHz1Nz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10DE520836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8K-0004P5-9n; Thu, 30 Apr 2020 20:50:12 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8J-0004Ov-OR for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:11 +0000 X-Inumbo-ID: 2ee5217a-8b24-11ea-9887-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2ee5217a-8b24-11ea-9887-bc764e2007e4; Thu, 30 Apr 2020 20:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Uv/yzMUx1QdBCN124eu+OU2bGCwQI9wMhMOGicg+2lw=; b=DShHz1Nz/o6zRMx8Lqm66YrFlZ yXOu6yAXfPlYM+hbeb8Rxsx0m+4okLioSESpWgPDtPww7uMPO+3wQ11/iy2Cbkx3QnDyEZ6Yrbo3X dMAtlAquopm2dJcrZ8z8/eouK0P2bTEpTtZpP644hLZ/O5Ll/QLrkaSpXvq1T8kwaUnA=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004Uf-Od; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG3H-0005wj-Pa; Thu, 30 Apr 2020 20:45:00 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 15/16] x86/setup: vmap heap nodes when they are outside the direct map Date: Thu, 30 Apr 2020 21:44:24 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , julien@xen.org, Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia When we do not have a direct map, arch_mfn_in_direct_map() will always return false, thus init_node_heap() will allocate xenheap pages from an existing node for the metadata of a new node. This means that the metadata of a new node is in a different node, slowing down heap allocation. Since we now have early vmap, vmap the metadata locally in the new node. Signed-off-by: Hongyan Xia --- xen/common/page_alloc.c | 40 ++++++++++++++++++++++++++++++++-------- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 1285fc5977..1e18b45caf 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -584,22 +584,46 @@ static unsigned long init_node_heap(int node, unsigned long mfn, needed = 0; } else if ( *use_tail && nr >= needed && - arch_mfn_in_directmap(mfn + nr) && (!xenheap_bits || !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn + nr - needed); - avail[node] = mfn_to_virt(mfn + nr - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; + if ( arch_mfn_in_directmap(mfn + nr) ) + { + _heap[node] = mfn_to_virt(mfn + nr - needed); + avail[node] = mfn_to_virt(mfn + nr - 1) + + PAGE_SIZE - sizeof(**avail) * NR_ZONES; + } + else + { + mfn_t needed_start = _mfn(mfn + nr - needed); + + _heap[node] = __vmap(&needed_start, needed, 1, 1, PAGE_HYPERVISOR, + VMAP_DEFAULT); + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; + } } else if ( nr >= needed && - arch_mfn_in_directmap(mfn + needed) && (!xenheap_bits || !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn); - avail[node] = mfn_to_virt(mfn + needed - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; + if ( arch_mfn_in_directmap(mfn + needed) ) + { + _heap[node] = mfn_to_virt(mfn); + avail[node] = mfn_to_virt(mfn + needed - 1) + + PAGE_SIZE - sizeof(**avail) * NR_ZONES; + } + else + { + mfn_t needed_start = _mfn(mfn); + + _heap[node] = __vmap(&needed_start, needed, 1, 1, PAGE_HYPERVISOR, + VMAP_DEFAULT); + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; + } *use_tail = false; } else if ( get_order_from_bytes(sizeof(**_heap)) == From patchwork Thu Apr 30 20:44:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 11521599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62DF81575 for ; Thu, 30 Apr 2020 20:50:33 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3F98020836 for ; Thu, 30 Apr 2020 20:50:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="NUlVPCW6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F98020836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8P-0004QP-VA; Thu, 30 Apr 2020 20:50:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8O-0004Q7-SE for xen-devel@lists.xenproject.org; Thu, 30 Apr 2020 20:50:16 +0000 X-Inumbo-ID: 2ed536a3-8b24-11ea-9ab3-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2ed536a3-8b24-11ea-9ab3-12813bfff9fa; Thu, 30 Apr 2020 20:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=3jn+1WoLnrWmwaD8Flpv3LUTCB5BEWlqT9Jy2nqV9e4=; b=NUlVPCW6rftBkpIbssTQnRUstn cCBO6ewSmywoum6FKCJ+/2dCwHa79TXlIKq6QdlJMjtGcIKwLgGhXQd3PHJh4EP6SxE++v+0SShBD 3mY4ZiNRk5HVg8vdBzrirflwkF8xxyA3Eb2NsU2qu9F1qryOEgLRSieWxSo587W1LRKg=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUG8I-0004UZ-KR; Thu, 30 Apr 2020 20:50:10 +0000 Received: from 54-240-197-234.amazon.com ([54.240.197.234] helo=u1bbd043a57dd5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jUG3J-0005wj-7Z; Thu, 30 Apr 2020 20:45:01 +0000 From: Hongyan Xia To: xen-devel@lists.xenproject.org Subject: [PATCH 16/16] x86/setup: do not create valid mappings when directmap=no Date: Thu, 30 Apr 2020 21:44:25 +0100 Message-Id: <6e03a111f564366ca4a5d145dde2cd46e757935b.1588278317.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , julien@xen.org, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Hongyan Xia Create empty mappings in the second e820 pass. Also, destroy existing direct map mappings created in the first pass. To make xenheap pages visible in guests, it is necessary to create empty L3 tables in the direct map even when directmap=no, since guest cr3s copy idle domain's L4 entries, which means they will share mappings in the direct map if we pre-populate idle domain's L4 entries and L3 tables. A helper is introduced for this. Also, after the direct map is actually gone, we need to stop updating the direct map in update_xen_mappings(). Signed-off-by: Hongyan Xia --- xen/arch/x86/mm.c | 2 +- xen/arch/x86/setup.c | 74 +++++++++++++++++++++++++++++++++++++++----- 2 files changed, 68 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 64da997764..33b7e3a003 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -798,7 +798,7 @@ static int update_xen_mappings(unsigned long mfn, unsigned int cacheattr) if ( unlikely(alias) && cacheattr ) err = map_pages_to_xen(xen_va, _mfn(mfn), 1, 0); - if ( !err ) + if ( arch_has_directmap() && !err ) err = map_pages_to_xen((unsigned long)mfn_to_virt(mfn), _mfn(mfn), 1, PAGE_HYPERVISOR | cacheattr_to_pte_flags(cacheattr)); if ( unlikely(alias) && !cacheattr && !err ) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index dbb2ac1c8f..13c37f435b 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -684,6 +684,57 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li /* How much of the directmap is prebuilt at compile time. */ #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT) +/* + * This either populates a valid direct map, or allocates empty L3 tables and + * creates the L4 entries for virtual address between [start, end) in the + * direct map depending on arch_has_directmap(); + * + * When directmap=no, we still need to populate empty L3 tables in the + * direct map region. The reason is that on-demand xenheap mappings are + * created in the idle domain's page table but must be seen by + * everyone. Since all domains share the direct map L4 entries, they + * will share xenheap mappings if we pre-populate the L4 entries and L3 + * tables in the direct map region for all RAM. We also rely on the fact + * that L3 tables are never freed. + */ +static void __init populate_directmap(uint64_t pstart, uint64_t pend, + unsigned int flags) +{ + unsigned long vstart = (unsigned long)__va(pstart); + unsigned long vend = (unsigned long)__va(pend); + + if ( pstart >= pend ) + return; + + BUG_ON(vstart < DIRECTMAP_VIRT_START); + BUG_ON(vend > DIRECTMAP_VIRT_END); + + if ( arch_has_directmap() ) + /* Populate valid direct map. */ + BUG_ON(map_pages_to_xen(vstart, maddr_to_mfn(pstart), + PFN_DOWN(pend - pstart), flags)); + else + { + /* Create empty L3 tables. */ + unsigned long vaddr = vstart & ~((1UL << L4_PAGETABLE_SHIFT) - 1); + + for ( ; vaddr < vend; vaddr += (1UL << L4_PAGETABLE_SHIFT) ) + { + l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(vaddr)]; + + if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) ) + { + mfn_t mfn = alloc_boot_pages(1, 1); + void *v = map_domain_page(mfn); + + clear_page(v); + UNMAP_DOMAIN_PAGE(v); + l4e_write(pl4e, l4e_from_mfn(mfn, __PAGE_HYPERVISOR)); + } + } + } +} + void __init noreturn __start_xen(unsigned long mbi_p) { char *memmap_type = NULL; @@ -1366,8 +1417,17 @@ void __init noreturn __start_xen(unsigned long mbi_p) map_e = min_t(uint64_t, e, ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT); - /* Pass mapped memory to allocator /before/ creating new mappings. */ + /* + * Pass mapped memory to allocator /before/ creating new mappings. + * The direct map for the bottom 4GiB has been populated in the first + * e820 pass. In the second pass, we make sure those existing mappings + * are destroyed when directmap=no. + */ init_boot_pages(s, min(map_s, e)); + if ( !arch_has_directmap() ) + destroy_xen_mappings((unsigned long)__va(s), + (unsigned long)__va(min(map_s, e))); + s = map_s; if ( s < map_e ) { @@ -1376,6 +1436,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) map_s = (s + mask) & ~mask; map_e &= ~mask; init_boot_pages(map_s, map_e); + if ( !arch_has_directmap() ) + destroy_xen_mappings((unsigned long)__va(map_s), + (unsigned long)__va(map_e)); } if ( map_s > map_e ) @@ -1389,8 +1452,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( map_e < end ) { - map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), - PFN_DOWN(end - map_e), PAGE_HYPERVISOR); + populate_directmap(map_e, end, PAGE_HYPERVISOR); init_boot_pages(map_e, end); map_e = end; } @@ -1399,13 +1461,11 @@ void __init noreturn __start_xen(unsigned long mbi_p) { /* This range must not be passed to the boot allocator and * must also not be mapped with _PAGE_GLOBAL. */ - map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), - PFN_DOWN(e - map_e), __PAGE_HYPERVISOR_RW); + populate_directmap(map_e, e, __PAGE_HYPERVISOR_RW); } if ( s < map_s ) { - map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s), - PFN_DOWN(map_s - s), PAGE_HYPERVISOR); + populate_directmap(s, map_s, PAGE_HYPERVISOR); init_boot_pages(s, map_s); } }