From patchwork Mon May 13 11:11:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elias El Yandouzi X-Patchwork-Id: 13663338 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68F11C25B75 for ; Mon, 13 May 2024 11:12:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.720811.1123782 (Exim 4.92) (envelope-from ) id 1s6Tb4-0004yx-31; Mon, 13 May 2024 11:11:58 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 720811.1123782; Mon, 13 May 2024 11:11:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s6Tb3-0004yq-Vg; Mon, 13 May 2024 11:11:57 +0000 Received: by outflank-mailman (input) for mailman id 720811; Mon, 13 May 2024 11:11:56 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s6Tb2-00036L-Dh for xen-devel@lists.xenproject.org; Mon, 13 May 2024 11:11:56 +0000 Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com [99.78.197.219]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 99746a71-1119-11ef-b4bb-af5377834399; Mon, 13 May 2024 13:11:54 +0200 (CEST) Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-80008.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2024 11:11:49 +0000 Received: from EX19MTAUEB001.ant.amazon.com [10.0.44.209:31352] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.17:2525] with esmtp (Farcaster) id 27d3b449-45c7-420e-af1d-caf6889650d5; Mon, 13 May 2024 11:11:49 +0000 (UTC) Received: from EX19D008UEC003.ant.amazon.com (10.252.135.194) by EX19MTAUEB001.ant.amazon.com (10.252.135.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 13 May 2024 11:11:48 +0000 Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by EX19D008UEC003.ant.amazon.com (10.252.135.194) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Mon, 13 May 2024 11:11:48 +0000 Received: from dev-dsk-eliasely-1a-fd74790f.eu-west-1.amazon.com (10.253.91.118) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28 via Frontend Transport; Mon, 13 May 2024 11:11:47 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 99746a71-1119-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1715598714; x=1747134714; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FfhF1uGZGyTejdo4gJox/0kvgwcb9PzofNFjsbGWbZM=; b=A/GDk4m5tw5i08QZifo33Ex5KhH+zWL+SAEcium2mEUqNrQB0mRSX1u3 1ru4Yz8o5aVk1tqx0l7aTVCp2p5zwScAq7ESz5LHXWUAyjZgPbZMQS6kg VoE9Nt6BxgxReTkN7UsoOHwXjsakAV41aISV9VQyD/ArowpEgqjDoJTke g=; X-IronPort-AV: E=Sophos;i="6.08,158,1712620800"; d="scan'208";a="88736852" X-Farcaster-Flow-ID: 27d3b449-45c7-420e-af1d-caf6889650d5 From: Elias El Yandouzi To: CC: , , , Hongyan Xia , Julien Grall , Elias El Yandouzi Subject: [PATCH V3 11/19] x86/setup: Leave early boot slightly earlier Date: Mon, 13 May 2024 11:11:09 +0000 Message-ID: <20240513111117.68828-12-eliasely@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240513111117.68828-1-eliasely@amazon.com> References: <20240513111117.68828-1-eliasely@amazon.com> MIME-Version: 1.0 From: Hongyan Xia When we do not have a direct map, memory for metadata of heap nodes in init_node_heap() is allocated from xenheap, which needs to be mapped and unmapped on demand. However, we cannot just take memory from the boot allocator to create the PTEs while we are passing memory to the heap allocator. To solve this race, we leave early boot slightly sooner so that Xen PTE pages are allocated from the heap instead of the boot allocator. We can do this because the metadata for the 1st node is statically allocated, and by the time we need memory to create mappings for the 2nd node, we already have enough memory in the heap allocator in the 1st node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index bd6b1184f5..f26c9799e4 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1751,6 +1751,22 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p) numa_initmem_init(0, raw_max_page); + /* + * When we do not have a direct map, memory for metadata of heap nodes in + * init_node_heap() is allocated from xenheap, which needs to be mapped and + * unmapped on demand. However, we cannot just take memory from the boot + * allocator to create the PTEs while we are passing memory to the heap + * allocator during end_boot_allocator(). + * + * To solve this race, we need to leave early boot before + * end_boot_allocator() so that Xen PTE pages are allocated from the heap + * instead of the boot allocator. We can do this because the metadata for + * the 1st node is statically allocated, and by the time we need memory to + * create mappings for the 2nd node, we already have enough memory in the + * heap allocator in the 1st node. + */ + system_state = SYS_STATE_boot; + if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) ) { unsigned long lo = virt_to_mfn(HYPERVISOR_VIRT_END - 1); @@ -1782,8 +1798,6 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p) else end_boot_allocator(); - system_state = SYS_STATE_boot; - bsp_stack = cpu_alloc_stack(0); if ( !bsp_stack ) panic("No memory for BSP stack\n");