From patchwork Thu Jan 19 03:52:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas Goldstein X-Patchwork-Id: 9525139 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3727060437 for ; Thu, 19 Jan 2017 03:55:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1430128648 for ; Thu, 19 Jan 2017 03:55:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08D5128653; Thu, 19 Jan 2017 03:55:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 49B3D28648 for ; Thu, 19 Jan 2017 03:55:19 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cU3mZ-00054h-OM; Thu, 19 Jan 2017 03:53:03 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cU3mY-00053Z-HM for xen-devel@lists.xen.org; Thu, 19 Jan 2017 03:53:02 +0000 Received: from [193.109.254.147] by server-10.bemta-6.messagelabs.com id B0/41-13192-D1830885; Thu, 19 Jan 2017 03:53:01 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrPIsWRWlGSWpSXmKPExsVyMXThEV0Zi4Y Ig7sLbSyWfFzM4sDocXT3b6YAxijWzLyk/IoE1ow1e2+xFjT6VLRda2BvYFxg28XIxSEkMJFR YubBXjYQh0VgP7PE/VP3WUAcCYFlrBI/PrUzdjFyAjkxEqsOfGLvYuQAsisl1s7WBgkLCShI/ J5wiAli0komiWNvDzCBJNgENCSe/JrBBmKLCEhLXPt8mRGkiFngKpPExhs3wYqEBewkth+GWM AioCpx/fIEsAZegTiJeV2P2CEWy0ssPH8ErJ5TIFWir7WPBWJzisTGHbfZcYlPYBRawMiwilG jOLWoLLVI18hUL6koMz2jJDcxM0fX0MBMLze1uDgxPTUnMalYLzk/dxMjMBgZgGAH46oFgYcY JTmYlER5z+k3RAjxJeWnVGYkFmfEF5XmpBYfYpTh4FCS4OUzB8oJFqWmp1akZeYA4wImLcHBo yTCywaS5i0uSMwtzkyHSJ1iNObo6Tr9koljz67LL5mEWPLy81KlxHlZQEoFQEozSvPgBsHi9R KjrJQwLyPQaUI8BalFuZklqPKvGMU5GJWEec+ZAU3hycwrgdv3CugUJqBTrJTrQU4pSURISTU wLlrWULXh1Jbl/UW+t29nLJSafdv8zKdDLgckH1x4t1bZki89cXnCKus/x8u6jDMaYm2q5Xp1 ltzQs/q6vsmG+UPb7ql7qqMzD/t376lXT53+Rcn5nshjYQO2ernfizgfhjcuYXYNSdILYXj+V Po+f4xm7qV0saAnc0xC7mZWPZqW/eJqfESUEktxRqKhFnNRcSIArE4TNNICAAA= X-Env-Sender: cardoe@cardoe.com X-Msg-Ref: server-7.tower-27.messagelabs.com!1484797979!78023248!1 X-Originating-IP: [209.85.161.196] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 439 invoked from network); 19 Jan 2017 03:53:00 -0000 Received: from mail-yw0-f196.google.com (HELO mail-yw0-f196.google.com) (209.85.161.196) by server-7.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 19 Jan 2017 03:53:00 -0000 Received: by mail-yw0-f196.google.com with SMTP id u68so2871149ywg.0 for ; Wed, 18 Jan 2017 19:53:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cardoe.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=c/I3uNY0URPEmX076YJPxYCv6gz1OX2bew/iCzt/yGU=; b=C4gepiwNlMb61ghJA8z2Z+5U/75bDg3lSjEqMj6QSc3UKL9IqOcKDpJxTXth2IunYj khhknuwlwwSlR+DSSqn9WeLqHr7ix+GiRoparTBpBZlgatBACcJcW2ZphO0URBs0S92s umlLKdXV8Qo5cUH9+E/0l7sDE2gEvS9ujPcZg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=c/I3uNY0URPEmX076YJPxYCv6gz1OX2bew/iCzt/yGU=; b=WxOU1WGznR/KO/lIUgxg82qoGCh+llrneZyAySQ2/WtbVYo4rjBkaQOEjg+JqWfnXw Wpxsgcm+7qJealt1FzGGoIhajvMWpmmfMgqwQXnYHuoBtWYzlMD39wsqvk8Z3VIcJu1h 8qbtPa6t3x2wT7vSFixH6aBdifaQRkpXR9bkBiSZ8z7qbzXTYyQ+jTAK2L8AkLqSnBLo PvOuaR4g3ju1QSTE1Iwoo2f35EepOp9Lwi+FtEfzXdFLt3ca5XC5LY410QsE97XzE0ng sHOWB7V0ICwBelCRWNZwWVXQfBLFkxjToclJbGVJdvmr8WMLVLAtIstJx5egLpdVmkXi UplQ== X-Gm-Message-State: AIkVDXLAOA0INVFiVPNLGjmn+zZVyG7AZRWeOHtQ5CkKu+ArO6yIP7vK+OBR4S3YPxGB/w== X-Received: by 10.129.93.66 with SMTP id r63mr5734232ywb.211.1484797979465; Wed, 18 Jan 2017 19:52:59 -0800 (PST) Received: from swanson.attlocal.net (104-179-196-18.lightspeed.brhmal.sbcglobal.net. [104.179.196.18]) by smtp.gmail.com with ESMTPSA id g133sm1203373ywe.19.2017.01.18.19.52.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 18 Jan 2017 19:52:58 -0800 (PST) From: Doug Goldstein To: xen-devel@lists.xen.org Date: Wed, 18 Jan 2017 21:52:43 -0600 Message-Id: <9bf8a5f94e58ec80e61c0d93a8bd93da618af2a4.1484797961.git-series.cardoe@cardoe.com> X-Mailer: git-send-email 2.10.2 In-Reply-To: References: In-Reply-To: References: Cc: jgross@suse.com, sstabellini@kernel.org, konrad.wilk@oracle.com, Andrew Cooper , daniel.kiper@oracle.com, Doug Goldstein , pgnet.dev@gmail.com, ning.sun@intel.com, julien.grall@arm.com, Jan Beulich , qiaowei.ren@intel.com, gang.wei@intel.com, fu.wei@linaro.org Subject: [Xen-devel] [PATCH v5 3/5] efi: create new early memory allocator X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Daniel Kiper There is a problem with place_string() which is used as early memory allocator. It gets memory chunks starting from start symbol and goes down. Sadly this does not work when Xen is loaded using multiboot2 protocol because then the start lives on 1 MiB address and we should not allocate a memory from below of it. So, I tried to use mem_lower address calculated by GRUB2. However, this solution works only on some machines. There are machines in the wild (e.g. Dell PowerEdge R820) which uses first ~640 KiB for boot services code or data... :-((( Hence, we need new memory allocator for Xen EFI boot code which is quite simple and generic and could be used by place_string() and efi_arch_allocate_mmap_buffer(). I think about following solutions: 1) We could use native EFI allocation functions (e.g. AllocatePool() or AllocatePages()) to get memory chunk. However, later (somewhere in __start_xen()) we must copy its contents to safe place or reserve it in e820 memory map and map it in Xen virtual address space. This means that the code referring to Xen command line, loaded modules and EFI memory map, mostly in __start_xen(), will be further complicated and diverge from legacy BIOS cases. Additionally, both former things have to be placed below 4 GiB because their addresses are stored in multiboot_info_t structure which has 32-bit relevant members. 2) We may allocate memory area statically somewhere in Xen code which could be used as memory pool for early dynamic allocations. Looks quite simple. Additionally, it would not depend on EFI at all and could be used on legacy BIOS platforms if we need it. However, we must carefully choose size of this pool. We do not want increase Xen binary size too much and waste too much memory but also we must fit at least memory map on x86 EFI platforms. As I saw on small machine, e.g. IBM System x3550 M2 with 8 GiB RAM, memory map may contain more than 200 entries. Every entry on x86-64 platform is 40 bytes in size. So, it means that we need more than 8 KiB for EFI memory map only. Additionally, if we use this memory pool for Xen and modules command line storage (it would be used when xen.efi is executed as EFI application) then we should add, I think, about 1 KiB. In this case, to be on safe side, we should assume at least 64 KiB pool for early memory allocations. Which is about 4 times of our earlier calculations. However, during discussion on Xen-devel Jan Beulich suggested that just in case we should use 1 MiB memory pool like it is in original place_string() implementation. So, let's use 1 MiB as it was proposed. If we think that we should not waste unallocated memory in the pool on running system then we can mark this region as __initdata and move all required data to dynamically allocated places somewhere in __start_xen(). 2a) We could put memory pool into .bss.page_aligned section. Then allocate memory chunks starting from the lowest address. After init phase we can free unused portion of the memory pool as in case of .init.text or .init.data sections. This way we do not need to allocate any space in image file and freeing of unused area in the memory pool is very simple. Now #2a solution is implemented because it is quite simple and requires limited number of changes, especially in __start_xen(). The new allocator is quite generic and can be used on ARM platforms too. Signed-off-by: Daniel Kiper Acked-by: Jan Beulich Acked-by: Julien Grall Reviewed-by: Doug Goldstein --- Doug v1 - removed stale paragraph v11 - suggestions/fixes: - #ifdef only EBMALLOC_SIZE from ebmalloc machinery (suggested by Jan Beulich). v10 - suggestions/fixes: - remove unneeded ARM free_ebmalloc_unused_mem() stub. v9 - suggestions/fixes: - call free_ebmalloc_unused_mem() from efi_init_memory() instead of xen/arch/arm/setup.c:init_done() (suggested by Jan Beulich), - improve comments. v8 - suggestions/fixes: - disable whole ebmalloc machinery on ARM platforms, - add comment saying what should be done before enabling ebmalloc on ARM, (suggested by Julien Grall), - move ebmalloc code before efi-boot.h inclusion and remove unneeded forward declaration (suggested by Jan Beulich), - remove free_ebmalloc_unused_mem() call from xen/arch/arm/setup.c:init_done() (suggested by Julien Grall), - improve commit message. v7 - suggestions/fixes: - enable most of ebmalloc machinery on ARM platforms (suggested by Jan Beulich), - remove unneeded cast (suggested by Jan Beulich), - wrap long line (suggested by Jan Beulich), - improve commit message. v6 - suggestions/fixes: - optimize ebmalloc allocator, - move ebmalloc machinery to xen/common/efi/boot.c (suggested by Jan Beulich), - enforce PAGE_SIZE ebmalloc_mem alignment (suggested by Jan Beulich), - ebmalloc() must allocate properly aligned memory regions (suggested by Jan Beulich), - printk() should use XENLOG_INFO (suggested by Jan Beulich). v4 - suggestions/fixes: - move from #2 solution to #2a solution, - improve commit message. --- --- xen/arch/x86/efi/efi-boot.h | 11 ++------ xen/arch/x86/setup.c | 3 +-- xen/common/efi/boot.c | 50 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 54 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h index 388c4ea..62c010e 100644 --- a/xen/arch/x86/efi/efi-boot.h +++ b/xen/arch/x86/efi/efi-boot.h @@ -114,7 +114,7 @@ static void __init relocate_trampoline(unsigned long phys) static void __init place_string(u32 *addr, const char *s) { - static char *__initdata alloc = start; + char *alloc = NULL; if ( s && *s ) { @@ -122,7 +122,7 @@ static void __init place_string(u32 *addr, const char *s) const char *old = (char *)(long)*addr; size_t len2 = *addr ? strlen(old) + 1 : 0; - alloc -= len1 + len2; + alloc = ebmalloc(len1 + len2); /* * Insert new string before already existing one. This is needed * for options passed on the command line to override options from @@ -205,12 +205,7 @@ static void __init efi_arch_process_memory_map(EFI_SYSTEM_TABLE *SystemTable, static void *__init efi_arch_allocate_mmap_buffer(UINTN map_size) { - place_string(&mbi.mem_upper, NULL); - mbi.mem_upper -= map_size; - mbi.mem_upper &= -__alignof__(EFI_MEMORY_DESCRIPTOR); - if ( mbi.mem_upper < xen_phys_start ) - return NULL; - return (void *)(long)mbi.mem_upper; + return ebmalloc(map_size); } static void __init efi_arch_pre_exit_boot(void) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 0ccef1d..d4b7d25 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1124,8 +1124,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( !xen_phys_start ) panic("Not enough memory to relocate Xen."); - reserve_e820_ram(&boot_e820, efi_enabled(EFI_LOADER) ? mbi->mem_upper : __pa(&_start), - __pa(&_end)); + reserve_e820_ram(&boot_e820, __pa(&_start), __pa(&_end)); /* Late kexec reservation (dynamic start address). */ kexec_reserve_area(&boot_e820); diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c index df8c702..36dbb71 100644 --- a/xen/common/efi/boot.c +++ b/xen/common/efi/boot.c @@ -98,6 +98,54 @@ static CHAR16 __initdata newline[] = L"\r\n"; #define PrintStr(s) StdOut->OutputString(StdOut, s) #define PrintErr(s) StdErr->OutputString(StdErr, s) +#ifdef CONFIG_ARM +/* + * TODO: Enable EFI boot allocator on ARM. + * This code can be common for x86 and ARM. + * Things TODO on ARM before enabling ebmalloc: + * - estimate required EBMALLOC_SIZE value, + * - where (in which section) ebmalloc_mem[] should live; if in + * .bss.page_aligned, as it is right now, then whole BSS zeroing + * have to be disabled in xen/arch/arm/arm64/head.S; though BSS + * should be initialized somehow before use of variables living there, + * - use ebmalloc() in ARM/common EFI boot code, + * - call free_ebmalloc_unused_mem() somewhere in init code. + */ +#define EBMALLOC_SIZE MB(0) +#else +#define EBMALLOC_SIZE MB(1) +#endif + +static char __section(".bss.page_aligned") __aligned(PAGE_SIZE) + ebmalloc_mem[EBMALLOC_SIZE]; +static unsigned long __initdata ebmalloc_allocated; + +/* EFI boot allocator. */ +static void __init __maybe_unused *ebmalloc(size_t size) +{ + void *ptr = ebmalloc_mem + ebmalloc_allocated; + + ebmalloc_allocated += (size + sizeof(void *) - 1) & ~(sizeof(void *) - 1); + + if ( ebmalloc_allocated > sizeof(ebmalloc_mem) ) + blexit(L"Out of static memory\r\n"); + + return ptr; +} + +static void __init __maybe_unused free_ebmalloc_unused_mem(void) +{ + unsigned long start, end; + + start = (unsigned long)ebmalloc_mem + PAGE_ALIGN(ebmalloc_allocated); + end = (unsigned long)ebmalloc_mem + sizeof(ebmalloc_mem); + + destroy_xen_mappings(start, end); + init_xenheap_pages(__pa(start), __pa(end)); + + printk(XENLOG_INFO "Freed %lukB unused BSS memory\n", (end - start) >> 10); +} + /* * Include architecture specific implementation here, which references the * static globals defined above. @@ -1251,6 +1299,8 @@ void __init efi_init_memory(void) } *extra, *extra_head = NULL; #endif + free_ebmalloc_unused_mem(); + if ( !efi_enabled(EFI_BOOT) ) return;