From patchwork Wed Nov 13 09:30:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13873341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 35F51D41C23 for ; Wed, 13 Nov 2024 09:31:15 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.835152.1250988 (Exim 4.92) (envelope-from ) id 1tB9iO-0008AS-Gi; Wed, 13 Nov 2024 09:31:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 835152.1250988; Wed, 13 Nov 2024 09:31:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tB9iO-0008AL-Da; Wed, 13 Nov 2024 09:31:08 +0000 Received: by outflank-mailman (input) for mailman id 835152; Wed, 13 Nov 2024 09:31:06 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tB9iM-0007YE-9p for xen-devel@lists.xenproject.org; Wed, 13 Nov 2024 09:31:06 +0000 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [2a00:1450:4864:20::62f]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 006b0a28-a1a2-11ef-a0c7-8be0dac302b0; Wed, 13 Nov 2024 10:31:03 +0100 (CET) Received: by mail-ej1-x62f.google.com with SMTP id a640c23a62f3a-a9aa8895facso1222392366b.2 for ; Wed, 13 Nov 2024 01:31:03 -0800 (PST) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5cf03bb7530sm6929630a12.43.2024.11.13.01.31.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2024 01:31:01 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 006b0a28-a1a2-11ef-a0c7-8be0dac302b0 X-Custom-Connection: eyJyZW1vdGVpcCI6IjJhMDA6MTQ1MDo0ODY0OjIwOjo2MmYiLCJoZWxvIjoibWFpbC1lajEteDYyZi5nb29nbGUuY29tIn0= X-Custom-Transaction: eyJpZCI6IjAwNmIwYTI4LWExYTItMTFlZi1hMGM3LThiZTBkYWMzMDJiMCIsInRzIjoxNzMxNDkwMjYzLjcyMjMxMywic2VuZGVyIjoiYW5kcmV3LmNvb3BlckBjbG91ZC5jb20iLCJyZWNpcGllbnQiOiJ4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmcifQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1731490263; x=1732095063; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zs7n1P5X7Jke+I4bmzjJ2iKQwDiqHhXQ1kiL+k6H1ts=; b=hnIKhJA9/EYKcsWU7HD0b2ZPs4dAp2vOi/1L8KgZToZ3dDMmrU2mIEIjCusPnFmo+Z w0PO4PWyvahpmY3pwB/UQfTO7Wdpq4OGAVikMZhY3pvhamKfz0/8MEBbr/mse3jWXyCw FV596PZayhvMvfyyRXAePHO5snWFDWIjTFw+A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731490263; x=1732095063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zs7n1P5X7Jke+I4bmzjJ2iKQwDiqHhXQ1kiL+k6H1ts=; b=hAhivLCkRiWDVe+wKJ3EDCNrNNVkfoOHGL9atw9/B26yI9y6VOPGy1sL3axbChTUzK B3Y6woKxwqaVJXeVwWsE0h9m85jFxi4v7kgjeKJN3Up5rdCgGf0lTYoywat+tH2qbAhw sihvfUYg4q+hbcruIpzqHAbnPv41cKdC2MvTnPHsbVPqs+GG8HO34/y47wB8Nnu+XZUO WlfN9Ic+qxtTrrUozQT9Lf1eVMiv7HZsysXSEkGM4TW8EqlEYnzs0kVsSxK87GuWEBgq uJktAsmzYyRd/my3LS+t1SPLuWVrTVIBWexkAyHueBnonPBD8BsETv2QtnQFgYX1Kbg8 eQsQ== X-Gm-Message-State: AOJu0YySM+SHRcwMPnxjSo6e3JghzIKgk0WLuJ6BvlCt4tWmBPNhesJv 60HtkXoMRTHm41RcZbP0eASGykV42YzNrrxwm+HszzcJ4Rp89ZIdlzqSU1yrcDC+L44nsuk7Cji / X-Google-Smtp-Source: AGHT+IGF8u7M6B5/UBHjGhmB4aekj0WVqm3/KJd6Nrn1prSwc8n5Tj1nm9shu46HJCAPnxDeM9Rocw== X-Received: by 2002:a17:907:7f1f:b0:a99:f4be:7a6a with SMTP id a640c23a62f3a-a9eeffdc630mr2031668866b.47.1731490262648; Wed, 13 Nov 2024 01:31:02 -0800 (PST) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , "Daniel P . Smith" , Frediano Ziglio , Alejandro Vallejo Subject: [PATCH 1/2] x86/trampoline: Document how the trampoline is laid out Date: Wed, 13 Nov 2024 09:30:57 +0000 Message-Id: <20241113093058.1562447-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241113093058.1562447-1-andrew.cooper3@citrix.com> References: <20241113093058.1562447-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 This is, to the best of my knowledge, accurate. I am providing no comment on how sane I believe it to be. At the time of writing, the sizes of the regions are: offset size AP: 0x0000 0x00b0 S3: 0x00b0 0x0140 Boot: 0x01f0 0x1780 Heap: 0x1970 0xe690 Stack: 0xf000 0x1000 and wakeup_stack overlays boot_edd_info. Signed-off-by: Andrew Cooper Reviewed-by: Frediano Ziglio --- CC: Jan Beulich CC: Roger Pau Monné CC: Daniel P. Smith CC: Frediano Ziglio CC: Alejandro Vallejo --- xen/arch/x86/include/asm/trampoline.h | 55 ++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 2 deletions(-) base-commit: 38febce5b35a7a4391906499b2046f01ec0129ca diff --git a/xen/arch/x86/include/asm/trampoline.h b/xen/arch/x86/include/asm/trampoline.h index 8c1e0b48c2c9..d801bea400dc 100644 --- a/xen/arch/x86/include/asm/trampoline.h +++ b/xen/arch/x86/include/asm/trampoline.h @@ -37,12 +37,63 @@ * manually as part of placement. */ +/* + * Layout of the trampoline. Logical areas, in ascending order: + * + * 1) AP boot: + * + * The INIT-SIPI-SIPI entrypoint. This logic is stack-less so the identity + * mapping (which must be executable) can at least be Read Only. + * + * 2) S3 resume: + * + * The S3 wakeup logic may need to interact with the BIOS, so needs a + * stack. The stack pointer is set to trampoline_phys + 4k and clobbers an + * undefined part of the the boot trampoline. The stack is only used with + * paging disabled. + * + * 3) Boot trampoline: + * + * This region houses various data used by the AP/S3 paths too. The boot + * trampoline collects data from the BIOS (E820/EDD/EDID/etc), so needs a + * stack. The stack pointer is set to trampoline_phys + 64k and has 4k + * space reserved. + * + * 4) Heap space: + * + * The first 1k of heap space is statically allocated for VESA information. + * + * The remainder of the heap is used by reloc(), logic which is otherwise + * outside of the trampoline, to collect the bootloader metadata (cmdline, + * module list, etc). It does so with a bump allocator starting from the + * end of the heap and allocating backwards. + * + * 5) Boot stack: + * + * 4k of space is reserved for the boot stack, at trampoline_phys + 64k. + * + * Therefore, when placed, it looks somewhat like this: + * + * +--- trampoline_phys + * v + * |<-------------------------------64K------------------------------->| + * |<-----4K----->| |<---4K--->| + * +----+----+----+-+---------------------------------------+----------+ + * | AP | S3 | Boot | Heap | Stack | + * +----+----+------+---------------------------------------+----------+ + * ^ <~~^ ^ <~~^ <~~^ + * | | +- trampoline_end[] | | + * | +--- S3 Stack reloc() allocator -+ | + * +------------------ trampoline_start[] Boot Stack ------------+ + */ + #include #include /* - * Start and end of the trampoline section, as linked into Xen. It is within - * the .init section and reclaimed after boot. + * Start and end of the trampoline section, as linked into Xen. This covers + * the AP, S3 and Boot regions, but not the heap or stack. It is within the + * .init section and reclaimed after boot. */ /* SAF-0-safe */ extern char trampoline_start[], trampoline_end[]; From patchwork Wed Nov 13 09:30:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13873342 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F1BFD41C26 for ; Wed, 13 Nov 2024 09:31:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.835154.1250993 (Exim 4.92) (envelope-from ) id 1tB9iO-0008F6-Sv; Wed, 13 Nov 2024 09:31:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 835154.1250993; Wed, 13 Nov 2024 09:31:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tB9iO-0008EP-Pz; Wed, 13 Nov 2024 09:31:08 +0000 Received: by outflank-mailman (input) for mailman id 835154; Wed, 13 Nov 2024 09:31:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tB9iN-0007YE-BZ for xen-devel@lists.xenproject.org; Wed, 13 Nov 2024 09:31:07 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 013e22a6-a1a2-11ef-a0c7-8be0dac302b0; Wed, 13 Nov 2024 10:31:05 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-a99fa009adcso441029266b.0 for ; Wed, 13 Nov 2024 01:31:05 -0800 (PST) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5cf03bb7530sm6929630a12.43.2024.11.13.01.31.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2024 01:31:03 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 013e22a6-a1a2-11ef-a0c7-8be0dac302b0 X-Custom-Connection: eyJyZW1vdGVpcCI6IjJhMDA6MTQ1MDo0ODY0OjIwOjo2MmQiLCJoZWxvIjoibWFpbC1lajEteDYyZC5nb29nbGUuY29tIn0= X-Custom-Transaction: eyJpZCI6IjAxM2UyMmE2LWExYTItMTFlZi1hMGM3LThiZTBkYWMzMDJiMCIsInRzIjoxNzMxNDkwMjY1LjA5NzkzNSwic2VuZGVyIjoiYW5kcmV3LmNvb3BlckBjbG91ZC5jb20iLCJyZWNpcGllbnQiOiJ4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmcifQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1731490264; x=1732095064; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a9O53tsUH9KI5NmtcGog/gKO969NUIAyapBgZEp49gQ=; b=f6K3wvzeg1Bp9OpNbZbz3bgsxrlb9ZvNgRV1/u6FlDPhUGysEfFgxgyDaxZnmYOg4T Y0ae5uEqkZDy43OIDXnJ5KLJfN4GAgbt7UIn64ffijkhIF7kj6llkvody2vPswPBuiYh KOhPThBK4TdBStoZ5f+wkF+g9WVKcTiR+UKMI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731490264; x=1732095064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a9O53tsUH9KI5NmtcGog/gKO969NUIAyapBgZEp49gQ=; b=i0Pp58PMznHMAaG2EfzKCUULBDytF/6mVG52xSD3Qag/eEv89q2oSV+4f6tP58Dl/s Gw03trzdWyzd806hiYYUSCklmHJb/MMd4bL+LQ87u8PBFjPG7RUtlMRf9obGyUfUH//I MReOacN4wzvmQanmde9CpMEawAQ9c3kGhypnKsWrkMoik9iP5D4iTGyBLB8FLVDfIYEk VhVzKpX19VmlB0SBzIAxqJi0i2O2L4v2e+sl1j5FAT6iF70dQeEYvN8veXXmE44yNXT/ 5lmlbbEJPKF21O22qTfV4TuHq0LDgHC4qIynA5kGMiPYPHkeqA5RphgNrdx+iM0f3FHK 0mWw== X-Gm-Message-State: AOJu0Yyt7XxQyLpYv2ENVl8Mxz1NVZpUxNOjftSTGni/zJ+vJ3bI7Zeb qHDzySMa73zGRKTqjpq1Etu1mvHm2Z6yClLHBIgqurd97f+99nHHUUcA9JIRXy1cOWGTxR3O/K0 V X-Google-Smtp-Source: AGHT+IHuBv0pvJp20rHG2r+QXP/gVZ5tZwxn4lKkF4ZW6jyudMS8KM2jb1ZYyNxOOBF1egs+drrIog== X-Received: by 2002:a05:6402:510f:b0:5c9:44bc:f9b3 with SMTP id 4fb4d7f45d1cf-5cf0a31116amr23492905a12.11.1731490264093; Wed, 13 Nov 2024 01:31:04 -0800 (PST) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , "Daniel P . Smith" , Frediano Ziglio , Alejandro Vallejo Subject: [PATCH 2/2] x86/trampoline: Rationalise the constants to describe the size Date: Wed, 13 Nov 2024 09:30:58 +0000 Message-Id: <20241113093058.1562447-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241113093058.1562447-1-andrew.cooper3@citrix.com> References: <20241113093058.1562447-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 The logic is far more sane to follow with a total size, and the position of the end of the heap. Remove or fix the the remaining descriptions of how the trampoline is laid out. No functional change. The compiled binary is identical. Signed-off-by: Andrew Cooper Reviewed-by: Frediano Ziglio --- CC: Jan Beulich CC: Roger Pau Monné CC: Daniel P. Smith CC: Frediano Ziglio CC: Alejandro Vallejo --- xen/arch/x86/boot/head.S | 21 ++------------------- xen/arch/x86/boot/reloc.c | 5 ++--- xen/arch/x86/efi/efi-boot.h | 2 +- xen/arch/x86/include/asm/config.h | 5 +++-- xen/arch/x86/xen.lds.S | 2 +- 5 files changed, 9 insertions(+), 26 deletions(-) diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S index dcda91cfda49..b31cf83758c1 100644 --- a/xen/arch/x86/boot/head.S +++ b/xen/arch/x86/boot/head.S @@ -494,7 +494,7 @@ trampoline_bios_setup: 2: /* Reserve memory for the trampoline and the low-memory stack. */ - sub $((TRAMPOLINE_SPACE+TRAMPOLINE_STACK_SPACE)>>4),%ecx + sub $TRAMPOLINE_SIZE >> 4, %ecx /* From arch/x86/smpboot.c: start_eip had better be page-aligned! */ xor %cl, %cl @@ -525,23 +525,6 @@ trampoline_setup: mov %eax, sym_esi(multiboot_ptr) 2: - /* - * Now trampoline_phys points to the following structure (lowest address - * is at the bottom): - * - * +------------------------+ - * | TRAMPOLINE_STACK_SPACE | - * +------------------------+ - * | Data (MBI / PVH) | - * +- - - - - - - - - - - - + - * | TRAMPOLINE_SPACE | - * +------------------------+ - * - * Data grows downwards from the highest address of TRAMPOLINE_SPACE - * region to the end of the trampoline. The rest of TRAMPOLINE_SPACE is - * reserved for trampoline code and data. - */ - /* Interrogate CPU extended features via CPUID. */ mov $1, %eax cpuid @@ -713,7 +696,7 @@ trampoline_setup: 1: /* Switch to low-memory stack which lives at the end of trampoline region. */ mov sym_esi(trampoline_phys), %edi - lea TRAMPOLINE_SPACE+TRAMPOLINE_STACK_SPACE(%edi),%esp + lea TRAMPOLINE_SIZE(%edi), %esp lea trampoline_boot_cpu_entry-trampoline_start(%edi),%eax pushl $BOOT_CS32 push %eax diff --git a/xen/arch/x86/boot/reloc.c b/xen/arch/x86/boot/reloc.c index e50e161b2740..1f47e10f7fa6 100644 --- a/xen/arch/x86/boot/reloc.c +++ b/xen/arch/x86/boot/reloc.c @@ -65,7 +65,7 @@ typedef struct memctx { /* * Simple bump allocator. * - * It starts from the base of the trampoline and allocates downwards. + * It starts from end of of the trampoline heap and allocates downwards. */ uint32_t ptr; } memctx; @@ -349,8 +349,7 @@ static multiboot_info_t *mbi2_reloc(uint32_t mbi_in, memctx *ctx) /* SAF-1-safe */ void *reloc(uint32_t magic, uint32_t in) { - /* Get bottom-most low-memory stack address. */ - memctx ctx = { trampoline_phys + TRAMPOLINE_SPACE }; + memctx ctx = { trampoline_phys + TRAMPOLINE_HEAP_END }; switch ( magic ) { diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h index 7930b7c73892..9d3f2b71447e 100644 --- a/xen/arch/x86/efi/efi-boot.h +++ b/xen/arch/x86/efi/efi-boot.h @@ -633,7 +633,7 @@ static void __init efi_arch_memory_setup(void) if ( efi_enabled(EFI_LOADER) ) cfg.size = trampoline_end - trampoline_start; else - cfg.size = TRAMPOLINE_SPACE + TRAMPOLINE_STACK_SPACE; + cfg.size = TRAMPOLINE_SIZE; status = efi_bs->AllocatePages(AllocateMaxAddress, EfiLoaderData, PFN_UP(cfg.size), &cfg.addr); diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h index f8a5a4913b07..20141ede31a1 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -51,8 +51,9 @@ #define IST_SHSTK_SIZE 1024 -#define TRAMPOLINE_STACK_SPACE PAGE_SIZE -#define TRAMPOLINE_SPACE (KB(64) - TRAMPOLINE_STACK_SPACE) +/* See asm/trampoline.h */ +#define TRAMPOLINE_SIZE KB(64) +#define TRAMPOLINE_HEAP_END (TRAMPOLINE_SIZE - PAGE_SIZE) #define WAKEUP_STACK_MIN 3072 #define MBI_SPACE_MIN (2 * PAGE_SIZE) diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S index 35693f6e3380..e7d93d1f4ac3 100644 --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -410,7 +410,7 @@ ASSERT(!SIZEOF(.plt), ".plt non-empty") ASSERT(!SIZEOF(.rela), "leftover relocations") #endif -ASSERT((trampoline_end - trampoline_start) < TRAMPOLINE_SPACE - MBI_SPACE_MIN, +ASSERT((trampoline_end - trampoline_start) < TRAMPOLINE_HEAP_END - MBI_SPACE_MIN, "not enough room for trampoline and mbi data") ASSERT((wakeup_stack - wakeup_stack_start) >= WAKEUP_STACK_MIN, "wakeup stack too small")