From patchwork Wed Jan 8 15:18:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6C3A0E7719A for ; Wed, 8 Jan 2025 15:19:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867480.1279014 (Exim 4.92) (envelope-from ) id 1tVXpq-0000A7-BL; Wed, 08 Jan 2025 15:19:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867480.1279014; Wed, 08 Jan 2025 15:19:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpq-00009y-7s; Wed, 08 Jan 2025 15:19:06 +0000 Received: by outflank-mailman (input) for mailman id 867480; Wed, 08 Jan 2025 15:19:04 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpo-0008Ue-QQ for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:04 +0000 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [2a00:1450:4864:20::630]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id e4744bb5-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:02 +0100 (CET) Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-aaec61d0f65so285320266b.1 for ; Wed, 08 Jan 2025 07:19:02 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:01 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e4744bb5-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349542; x=1736954342; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XjrHpoafwPytgBLLE9UWiWZhowYUy5r4Ft9RDVMVLUE=; b=BsK/awXxL3rqIgDMLE74aaE63ujSQ2fhVdgh/kWRIynhd6ZsrLfipJqNuF49i4mcA/ h5RQKPKH2e4t5hRYQAHXTvUfK/tND+84iil7xP0SPoYH8dT983wZU2O5gZulYbvpORwO 2sUUzPge1ahR8V34KGNIMNvD/72Ntc6/jZvG4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349542; x=1736954342; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XjrHpoafwPytgBLLE9UWiWZhowYUy5r4Ft9RDVMVLUE=; b=TnLdQ3GsSmzH2pWP7mFdaX6I+fpr1/MO0MZxNLRw79F33Crhnjp0evIT0gGQPy7p5A jd8Uhi4GFecFpBr5T1mBgQE2vpFipaTf+ESwVRUgt7djyjvAt2FSc/BtSii4WbQlNYqF LR4OpgHiSY6mW8y6GHmEoPSOWRJGwwsf0N6OFPSgR3psln6J8+oKt3XTOrEV697zl2ET DOyCiPc0pAQk8dlZ0Mc4FjQTixK1oOeLquJLgBwLVVslsISEDgXZyn2e4Xsl5nCemY07 2c5EdcY6IR7dBjqSrW9a6ajPzuEa74tYS4E1TFMFM7nGLmZ5S8Me6lgz0QryUrtkW4j1 yOfg== X-Gm-Message-State: AOJu0YyfeUFCI2EeQ7dktdc9agOInHY8pvHe1L6R4DXMN4KeJ+Xb9WBA h6X8nf2eCJ1MU8RCKzk9crPLvv4TzWKcCtN+pvRi8cFjr+XmS6YAJUDOw2+hRQgcmHiG3b00nr9 Cyq2UQQ== X-Gm-Gg: ASbGncs+SRfjSGY6gz+F/sVqefvmLVaDoOtvT2pRix9wxyTHMcZVOHl0wm/s3BQkaHc zdZ4Jr87a4USk/Ch2BCWtAhOhhxqus27twt2DhsWWJkxg67KQm8LgZ97eUq8PfjXDW21pSTslmt Lt34SdnGe3vyEwAxacNmDekmI2Bslo5zcEcMe240AEUKZQK7/iP2GNCzj0n83NX9VAkNZ0jHTFk Gp5CQsC3aUHm4VDb5gsCKkfI5g4mhA77zBYOMCXSEM+1IMDGUnutXnEfJRPPIT05yukR/cW4z2l 5zw= X-Google-Smtp-Source: AGHT+IFy2qJHO7J67PwIOe0KP7iXXVioCmlPY6UKUmkZiRaVm0XX4FKWhIb/MWinoqrJ0CU6MUY5UA== X-Received: by 2002:a17:907:1c93:b0:aa6:894c:84b9 with SMTP id a640c23a62f3a-ab2ab6a4da6mr267067066b.23.1736349541822; Wed, 08 Jan 2025 07:19:01 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 01/15] x86: Create per-domain mapping for guest_root_pt Date: Wed, 8 Jan 2025 15:18:08 +0000 Message-ID: <20250108151822.16030-2-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia This patch introduces a per-domain mapping for the `guest_root_pt` in PV guests as part of the effort to remove the direct map in Xen. For the time being, the `root_pgt` is not mapped or unmapped, as it remains a Xenheap page. This will be addressed in subsequent patches. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * bugfix: s/FREE_XENHEAP_PAGE/XFREE/ on destroying root_pt_l1tab. * Add a few extra comments to the declarations for sanity's sake. * Refactor pv_root_pt macro to use L1_PAGETABLE_ENTRIES instead. * Removed extra newline in pv_destory_root_pt_l1tab() v3->v4: * Fix over-allocation issue * Update the mappings when switching from kernel to user-mode v2->v3: * Rename SHADOW_ROOT * Haven't addressed the potentially over-allocation issue as I don't get it v1->v2: * Rework the shadow perdomain mapping solution in the follow-up patches Changes since Hongyan's version: * Remove the final dot in the commit title Signed-off-by: Alejandro Vallejo --- xen/arch/x86/include/asm/config.h | 10 ++++++- xen/arch/x86/include/asm/domain.h | 3 +++ xen/arch/x86/mm.c | 13 +++++++++ xen/arch/x86/pv/domain.c | 44 ++++++++++++++++++++++++++++--- xen/arch/x86/x86_64/asm-offsets.c | 1 + xen/arch/x86/x86_64/entry.S | 9 ++++++- 6 files changed, 74 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h index 19746f956ec3..42c8a120e7dc 100644 --- a/xen/arch/x86/include/asm/config.h +++ b/xen/arch/x86/include/asm/config.h @@ -168,7 +168,7 @@ /* Slot 260: per-domain mappings (including map cache). */ #define PERDOMAIN_VIRT_START (PML4_ADDR(260)) #define PERDOMAIN_SLOT_MBYTES (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER)) -#define PERDOMAIN_SLOTS 3 +#define PERDOMAIN_SLOTS 4 #define PERDOMAIN_VIRT_SLOT(s) (PERDOMAIN_VIRT_START + (s) * \ (PERDOMAIN_SLOT_MBYTES << 20)) /* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). */ @@ -282,6 +282,14 @@ extern unsigned long xen_phys_start; #define ARG_XLAT_START(v) \ (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT)) +/* pv_root_pt mapping area. The fourth per-domain-mapping sub-area */ +#define PV_ROOT_PT_MAPPING_VIRT_START PERDOMAIN_VIRT_SLOT(3) +#define PV_ROOT_PT_MAPPING_ENTRIES MAX_VIRT_CPUS + +/* The address of a particular VCPU's PV_ROOT_PT */ +#define PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v) \ + (PV_ROOT_PT_MAPPING_VIRT_START + ((v)->vcpu_id * PAGE_SIZE)) + #define ELFSIZE 64 #define ARCH_CRASH_SAVE_VMCOREINFO diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h index b79d6badd71c..b5a14991ca0b 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -273,6 +273,9 @@ struct pv_domain { l1_pgentry_t **gdt_ldt_l1tab; + /* Array of pointers to the l1 PTs holding PV root PTs of each vCPU */ + l1_pgentry_t **root_pt_l1tab; + atomic_t nr_l4_pages; /* Is a 32-bit PV guest? */ diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index fa21903eb25a..adcc0b5ff328 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -527,6 +527,14 @@ void make_cr3(struct vcpu *v, mfn_t mfn) v->arch.cr3 |= get_pcid_bits(v, false); } +/* Index of the l1 PT that maps v's root PT */ +#define pv_root_pt_idx(v) ((v)->vcpu_id / L1_PAGETABLE_ENTRIES) + +/* Pointer to the PTE that maps v's root PT in the perdomain area */ +#define pv_root_pt_pte(v) \ + ((v)->domain->arch.pv.root_pt_l1tab[pv_root_pt_idx(v)] + \ + ((v)->vcpu_id & (L1_PAGETABLE_ENTRIES - 1))) + void write_ptbase(struct vcpu *v) { const struct domain *d = v->domain; @@ -538,11 +546,16 @@ void write_ptbase(struct vcpu *v) if ( is_pv_domain(d) && d->arch.pv.xpti ) { + mfn_t guest_root_pt = _mfn(MASK_EXTR(v->arch.cr3, X86_CR3_ADDR_MASK)); + l1_pgentry_t *pte = pv_root_pt_pte(v); + cpu_info->root_pgt_changed = true; cpu_info->pv_cr3 = __pa(this_cpu(root_pgt)); if ( new_cr4 & X86_CR4_PCIDE ) cpu_info->pv_cr3 |= get_pcid_bits(v, true); switch_cr3_cr4(v->arch.cr3, new_cr4); + + l1e_write(pte, l1e_from_mfn(guest_root_pt, __PAGE_HYPERVISOR_RO)); } else { diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 7aef628f55be..4b85a02d910e 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -289,6 +289,20 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v) 1U << GDT_LDT_VCPU_SHIFT); } +static int pv_create_root_pt_l1tab(const struct vcpu *v) +{ + return create_perdomain_mapping(v->domain, + PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v), + 1, v->domain->arch.pv.root_pt_l1tab, + NULL); +} + +static void pv_destroy_root_pt_l1tab(const struct vcpu *v) +{ + destroy_perdomain_mapping(v->domain, + PV_ROOT_PT_MAPPING_VCPU_VIRT_START(v), 1); +} + void pv_vcpu_destroy(struct vcpu *v) { if ( is_pv_32bit_vcpu(v) ) @@ -298,6 +312,7 @@ void pv_vcpu_destroy(struct vcpu *v) } pv_destroy_gdt_ldt_l1tab(v); + pv_destroy_root_pt_l1tab(v); XFREE(v->arch.pv.trap_ctxt); } @@ -312,6 +327,13 @@ int pv_vcpu_initialise(struct vcpu *v) if ( rc ) return rc; + if ( v->domain->arch.pv.xpti ) + { + rc = pv_create_root_pt_l1tab(v); + if ( rc ) + goto done; + } + BUILD_BUG_ON(X86_NR_VECTORS * sizeof(*v->arch.pv.trap_ctxt) > PAGE_SIZE); v->arch.pv.trap_ctxt = xzalloc_array(struct trap_info, X86_NR_VECTORS); @@ -347,10 +369,12 @@ void pv_domain_destroy(struct domain *d) destroy_perdomain_mapping(d, GDT_LDT_VIRT_START, GDT_LDT_MBYTES << (20 - PAGE_SHIFT)); + destroy_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START, d->max_vcpus); XFREE(d->arch.pv.cpuidmasks); FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab); + XFREE(d->arch.pv.root_pt_l1tab); } void noreturn cf_check continue_pv_domain(void); @@ -376,8 +400,22 @@ int pv_domain_initialise(struct domain *d) (d->arch.pv.cpuidmasks = xmemdup(&cpuidmask_defaults)) == NULL ) goto fail; + rc = create_perdomain_mapping(d, PV_ROOT_PT_MAPPING_VIRT_START, + d->max_vcpus, NULL, NULL); + if ( rc ) + goto fail; + d->arch.ctxt_switch = &pv_csw; + if ( d->arch.pv.xpti ) + { + d->arch.pv.root_pt_l1tab = + xzalloc_array(l1_pgentry_t *, + DIV_ROUND_UP(d->max_vcpus, L1_PAGETABLE_ENTRIES)); + if ( !d->arch.pv.root_pt_l1tab ) + goto fail; + } + if ( !is_pv_32bit_domain(d) && use_invpcid && cpu_has_pcid ) switch ( ACCESS_ONCE(opt_pcid) ) { @@ -451,7 +489,8 @@ static void _toggle_guest_pt(struct vcpu *v) guest_update = false; } } - write_cr3(cr3); + + write_ptbase(v); if ( !pagetable_is_null(old_shadow) ) shadow_put_top_level(v->domain, old_shadow); @@ -491,9 +530,6 @@ void toggle_guest_mode(struct vcpu *v) { struct cpu_info *cpu_info = get_cpu_info(); - cpu_info->root_pgt_changed = true; - cpu_info->pv_cr3 = __pa(this_cpu(root_pgt)) | - (d->arch.pv.pcid ? get_pcid_bits(v, true) : 0); /* * As in _toggle_guest_pt() the XPTI CR3 write needs to be a TLB- * flushing one too for shadow mode guests. diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c index 630bdc39451d..c1ae5013af96 100644 --- a/xen/arch/x86/x86_64/asm-offsets.c +++ b/xen/arch/x86/x86_64/asm-offsets.c @@ -80,6 +80,7 @@ void __dummy__(void) #undef OFFSET_EF + OFFSET(VCPU_id, struct vcpu, vcpu_id); OFFSET(VCPU_processor, struct vcpu, processor); OFFSET(VCPU_domain, struct vcpu, domain); OFFSET(VCPU_vcpu_info, struct vcpu, vcpu_info_area.map); diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 40d094d5b2ee..4de41ce743c7 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -170,9 +170,16 @@ FUNC_LOCAL(restore_all_guest) movabs $PADDR_MASK & PAGE_MASK, %rsi movabs $DIRECTMAP_VIRT_START, %rcx and %rsi, %rdi - and %r9, %rsi add %rcx, %rdi + + /* + * The address in the vCPU cr3 is always mapped in the per-domain + * pv_root_pt virt area. + */ + imul $PAGE_SIZE, VCPU_id(%rbx), %esi + movabs $PV_ROOT_PT_MAPPING_VIRT_START, %rcx add %rcx, %rsi + mov $ROOT_PAGETABLE_FIRST_XEN_SLOT, %ecx mov root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rsi), %r8 mov %r8, root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rdi) From patchwork Wed Jan 8 15:18:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 886E6E7719E for ; Wed, 8 Jan 2025 15:19:22 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867482.1279029 (Exim 4.92) (envelope-from ) id 1tVXpr-0000SZ-T0; Wed, 08 Jan 2025 15:19:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867482.1279029; Wed, 08 Jan 2025 15:19:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpr-0000S8-NK; Wed, 08 Jan 2025 15:19:07 +0000 Received: by outflank-mailman (input) for mailman id 867482; Wed, 08 Jan 2025 15:19:05 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpp-0008Ue-GZ for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:05 +0000 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [2a00:1450:4864:20::536]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id e5088e42-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:03 +0100 (CET) Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-5d3d479b1e6so24091031a12.2 for ; Wed, 08 Jan 2025 07:19:03 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:02 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e5088e42-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349543; x=1736954343; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wbGRQTGcHs+jb1boNth07gwQLtq4Kdirf6gaOSDnoIc=; b=EI8IcDPLbUmkegEKo/JEVosAAlseqKivtVJrNKCpj4A/Yjv3RwQ3VriuiidP+rvcJa Z2ixumAufKwRZsMwmCkxaAJvLFaH/ElJPyWQfYA8oyNH76ag1/n+/SuiLdiHqGU7Ucur kEA6VTGmOKUumzF8j5ujBAr/5pnQmq7nx22OA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349543; x=1736954343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wbGRQTGcHs+jb1boNth07gwQLtq4Kdirf6gaOSDnoIc=; b=RWtxGUgGlFVRKspVhGJNBwz0124JrtTA4OaUA/DOeMbfUgHZBuMfDGAQss7I+GRS9h 5T5niFhGBrjVq6PP89pzj6i1/fy+ZJxJB0PDqNy1nJ1xW/yf640FvzYR0zsFdUUtMN7K h7jAjpMox73xgv8incDRLq0r781Svo0hmMzYkNo9pinY4sMfwyxWWUX8PtSv+ZGUDzT8 03CGVX9qyuz9xW3PPeAjmoM0xJT3p/ticrmb069G08RVyjKghUZv+l2fCmX5PBeE40hS BwFm+/W9F99LCisYnv2OKfDH6IOp2voKirDH8Wqn4RaB4ytAXb6ViEGkKWxszjpvID86 lYjQ== X-Gm-Message-State: AOJu0YyX/8WPJCAYdBzRnm5YDTVENdzL/+msCnuvdnKLlulkkJDq0hdP 8BN0KWxsxN0WBbhgIz8RCgFtuDva5cInqmf32ITj+uLmUk3CGUb7FZ2EygK37pFuZU5AfRTrS/F W6t081w== X-Gm-Gg: ASbGnctQT8VI5eAmzC5uV0jQKF66MBNivNApujtE2cpyQzCn+5BS93zPd0usKPLDrNv 0bW7mjESxDisWKEVrBzD/KtcRkwVHHoQU7gt1fvlCjR3n7aV7B+s85/9/UTqBcHuD8OUGhISY0G zmBIM7owLEIx4KZTHgDLi2k7hXoHEaYkvI6OyvCwCidf/4xmfTliuzX6rl8EYl+VydZF05oJNMG PX/0P8vFCObzlIMYXh9pxd1WRbGaBaP9UPgyciJS8gq80c9BUrGol/VI25xTwVvCH4zAmHaQuvu W44= X-Google-Smtp-Source: AGHT+IGkaiMbxkSh85J0AOmJK4ePyzmat6eWHelGZ7J77qfccD/S6Yzj6V5PY3eDPIIRCuv82vjIQg== X-Received: by 2002:a05:6402:2548:b0:5d4:2ef7:1c with SMTP id 4fb4d7f45d1cf-5d972e639e4mr6963135a12.24.1736349542864; Wed, 08 Jan 2025 07:19:02 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Wei Liu , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Wang , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 02/15] x86/pv: Use copy_domain_page() to manage domheap pages during initrd relocation Date: Wed, 8 Jan 2025 15:18:09 +0000 Message-ID: <20250108151822.16030-3-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Wei Liu Replace the manual copying logic with a call to `copy_domain_page()` while relocating intird which map and unmap pages accordingly. Signed-off-by: Wei Liu Signed-off-by: Wei Wang Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * No changes v3->v4: * Use the count of pages instead of calculating from order for nr_pages initialization v2->v3: * Rename commit title * Rework the for loop copying the pages v1->v2: * Get rid of mfn_to_virt * Don't open code copy_domain_page() Changes since Hongyan's version: * Add missing newline after the variable declaration --- xen/arch/x86/pv/dom0_build.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index f54d1da5c6f4..08a4534d5c19 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -637,17 +637,24 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) if ( d->arch.physaddr_bitsize && ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) ) { + unsigned int nr_pages = count; + order = get_order_from_pages(count); page = alloc_domheap_pages(d, order, MEMF_no_scrub); if ( !page ) panic("Not enough RAM for domain 0 initrd\n"); + for ( count = -count; order--; ) if ( count & (1UL << order) ) { free_domheap_pages(page, order); page += 1UL << order; + nr_pages -= 1UL << order; } - memcpy(page_to_virt(page), __va(initrd->start), initrd_len); + + for ( ; nr_pages-- ; page++, mfn++ ) + copy_domain_page(page_to_mfn(page), _mfn(mfn)); + /* * The initrd was copied but the initrd variable is reused in the * calculations below. As to not leak the memory used for the From patchwork Wed Jan 8 15:18:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 88CF7C02181 for ; Wed, 8 Jan 2025 15:19:22 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867481.1279024 (Exim 4.92) (envelope-from ) id 1tVXpr-0000Pg-IZ; Wed, 08 Jan 2025 15:19:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867481.1279024; Wed, 08 Jan 2025 15:19:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpr-0000PX-F0; Wed, 08 Jan 2025 15:19:07 +0000 Received: by outflank-mailman (input) for mailman id 867481; Wed, 08 Jan 2025 15:19:05 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpp-0008Lg-Cp for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:05 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e5a4a2a2-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:04 +0100 (CET) Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-ab2b72fb3c9so123438366b.0 for ; Wed, 08 Jan 2025 07:19:04 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:03 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e5a4a2a2-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349544; x=1736954344; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b5kW+L9or4UchYGbtNWwT1tJvUFiadtexJoE68LPlKw=; b=U7ARSjPCGO0znmMI9r/qNh4LJXhhHoPd824PeTWz9iG/byXWT2cW06pEUttR8QdncM +e8Mm3bcrp3PyK/slRqr/GxIwbMao9BQqE2p3kIl7/0ynk5306BpgmfgZja21IcXXrAg FPBzfZysJRa7sxDcjy90jk5B1c4dqf4S4sdFA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349544; x=1736954344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b5kW+L9or4UchYGbtNWwT1tJvUFiadtexJoE68LPlKw=; b=uvZr6epX7KHQLAHi19TUDS+49Fgyu4Uq7LT2HONtd+jgfLCESAj5ds9gKo5nLHUWmR 4PZGlwkRGc4FuIPoWH8ypHtDlkShuzmBbOEDzvaO2s3jUkXP6NPxAOSRfIM7SqDuTOOE es+cFfbEzGf6Uuroa8QpK4dUUlqeAzu36dT2gPddlK8KnDrwdR4A578upVIdIMeI23Kt Y9vYi8RIFz/DIqD/q2c0xAuxednh4AcRsL+JInB6yhf+lbVXpOwKLlYLPFd0s6iev3Bo qXLmDHCmQw29h0KIcfR9QeZ4jSlBK/W7RQwtPvBCAKyGKJ2hO079+Yjn7HhpPHkhE7y7 CiNw== X-Gm-Message-State: AOJu0YwaVl4HAK9yTGsJCIa+tD2rdqlVbswyez3vpfuP9/3xi3SX5u8E OTxJ2pfR9Z0pYMam+WUg5Kzif2yTXbLY0TgNQTB+v3klid8DqtvUZVN5jblUS+luKrq3RFOHO7b dMr8Ltg== X-Gm-Gg: ASbGncsAEiodk1DlQj9vVBTC1Cuv6zFwVg+zojaVZzhNXgVIqWEuxpNYOXEuryUCBji dwg8rzGC2xaCkR842m6EAz2C3pvmFWLkU6IZ/kygSLZNA2cCyQRiOVPw0wrRaxv0HyFpxIQu689 k7XhePjogVfjTBf/j7t31GFBMzUy62SQ1oL0NzeHLU90RZvi2zLQ+eBx+0tt3ALgpQzH9Qtvyn3 oBhc/iz/OBMDnfFwgOFydOtc1f88G6IXLyV3WBVT+OpGQfk1dxvOEjdZlic6YlRKd45KdR1vNOd yh0= X-Google-Smtp-Source: AGHT+IHV6oa2te4PnkoUSSr5SDjwRGUrfSPwGyilCbkNwbSum59kdIk8gEKqbOSWy3RmfhUHpU0ykQ== X-Received: by 2002:a17:907:7f08:b0:aab:9258:488 with SMTP id a640c23a62f3a-ab2904ff9d7mr611013366b.10.1736349543895; Wed, 08 Jan 2025 07:19:03 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 03/15] x86/pv: Rewrite how building PV dom0 handles domheap mappings Date: Wed, 8 Jan 2025 15:18:10 +0000 Message-ID: <20250108151822.16030-4-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia Building a PV dom0 is allocating from the domheap but uses it like the xenheap. Use the pages as they should be. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Bugfix: Don't map l4start with a mapcache and unmap with another. This is a revert to how it originally was in the series. i.e: UNMAP_DOMAIN_PAGE(l4start) before overriding current and then re-mapping on the idle PTs if needed. * Simplify UNMAP_MAP_AND_ADVANCE removing the do-while(). It's not needed with the ({ x }) expression. Assignments return the assigned value, so the last line was not needed either. v3->v4: * Reduce the scope of l{1,2,4}start_mfn variables * Make the macro `UNMAP_MAP_AND_ADVANCE` return the new virtual address v2->v3: * Fold following patch 'x86/pv: Map L4 page table for shim domain' v1->v2: * Clarify the commit message * Break the patch in two parts Changes since Hongyan's version: * Rebase * Remove spurious newline --- xen/arch/x86/pv/dom0_build.c | 61 +++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 19 deletions(-) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index 08a4534d5c19..649412590e72 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -384,6 +384,8 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) l3_pgentry_t *l3tab = NULL, *l3start = NULL; l2_pgentry_t *l2tab = NULL, *l2start = NULL; l1_pgentry_t *l1tab = NULL, *l1start = NULL; + mfn_t l3start_mfn = INVALID_MFN; + mfn_t l4start_mfn = INVALID_MFN; /* * This fully describes the memory layout of the initial domain. All @@ -738,22 +740,30 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) v->arch.pv.event_callback_cs = FLAT_COMPAT_KERNEL_CS; } +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) ({ \ + unmap_domain_page(virt_var); \ + mfn_var = maddr_to_mfn(maddr); \ + maddr += PAGE_SIZE; \ + virt_var = map_domain_page(mfn_var); \ +}) + if ( !compat ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table; - l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + l4tab = UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc); clear_page(l4tab); - init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)), - d, INVALID_MFN, true); - v->arch.guest_table = pagetable_from_paddr(__pa(l4start)); + init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true); + v->arch.guest_table = pagetable_from_mfn(l4start_mfn); } else { /* Monitor table already created by switch_compat(). */ - l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table)); + l4start_mfn = pagetable_get_mfn(v->arch.guest_table); + l4start = l4tab = map_domain_page(l4start_mfn); /* See public/xen.h on why the following is needed. */ maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table; l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc); } l4tab += l4_table_offset(v_start); @@ -762,15 +772,17 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) { if ( !((unsigned long)l1tab & (PAGE_SIZE-1)) ) { + mfn_t l1start_mfn; maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l1_page_table; - l1start = l1tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + l1tab = UNMAP_MAP_AND_ADVANCE(l1start_mfn, l1start, mpt_alloc); clear_page(l1tab); if ( count == 0 ) l1tab += l1_table_offset(v_start); if ( !((unsigned long)l2tab & (PAGE_SIZE-1)) ) { + mfn_t l2start_mfn; maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table; - l2start = l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + l2tab = UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc); clear_page(l2tab); if ( count == 0 ) l2tab += l2_table_offset(v_start); @@ -780,19 +792,19 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table; - l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc); } l3tab = l3start; clear_page(l3tab); if ( count == 0 ) l3tab += l3_table_offset(v_start); - *l4tab = l4e_from_paddr(__pa(l3start), L4_PROT); + *l4tab = l4e_from_mfn(l3start_mfn, L4_PROT); l4tab++; } - *l3tab = l3e_from_paddr(__pa(l2start), L3_PROT); + *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT); l3tab++; } - *l2tab = l2e_from_paddr(__pa(l1start), L2_PROT); + *l2tab = l2e_from_mfn(l1start_mfn, L2_PROT); l2tab++; } if ( count < initrd_pfn || count >= initrd_pfn + PFN_UP(initrd_len) ) @@ -811,30 +823,37 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) if ( compat ) { - l2_pgentry_t *l2t; - /* Ensure the first four L3 entries are all populated. */ for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab ) { if ( !l3e_get_intpte(*l3tab) ) { + mfn_t l2start_mfn; maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table; - l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; - clear_page(l2tab); - *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT); + UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc); + clear_page(l2start); + *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT); } if ( i == 3 ) l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2; } - l2t = map_l2t_from_l3e(l3start[3]); - init_xen_pae_l2_slots(l2t, d); - unmap_domain_page(l2t); + UNMAP_DOMAIN_PAGE(l2start); + l2start = map_l2t_from_l3e(l3start[3]); + init_xen_pae_l2_slots(l2start, d); } +#undef UNMAP_MAP_AND_ADVANCE + + UNMAP_DOMAIN_PAGE(l1start); + UNMAP_DOMAIN_PAGE(l2start); + UNMAP_DOMAIN_PAGE(l3start); + /* Pages that are part of page tables must be read only. */ mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages, &flush_flags); + UNMAP_DOMAIN_PAGE(l4start); + /* Mask all upcalls... */ for ( i = 0; i < XEN_LEGACY_MAX_VCPUS; i++ ) shared_info(d, vcpu_info[i].evtchn_upcall_mask) = 1; @@ -1003,8 +1022,12 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d) * !CONFIG_VIDEO case so the logic here can be simplified. */ if ( pv_shim ) + { + l4start = map_domain_page(l4start_mfn); pv_shim_setup_dom(d, l4start, v_start, vxenstore_start, vconsole_start, vphysmap_start, si); + UNMAP_DOMAIN_PAGE(l4start); + } #ifdef CONFIG_COMPAT if ( compat ) From patchwork Wed Jan 8 15:18:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DBB75E7719B for ; Wed, 8 Jan 2025 15:19:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867483.1279037 (Exim 4.92) (envelope-from ) id 1tVXps-0000bx-ES; Wed, 08 Jan 2025 15:19:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867483.1279037; Wed, 08 Jan 2025 15:19:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXps-0000YG-5J; Wed, 08 Jan 2025 15:19:08 +0000 Received: by outflank-mailman (input) for mailman id 867483; Wed, 08 Jan 2025 15:19:06 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpq-0008Lg-H6 for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:06 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e63db730-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:05 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aa69107179cso2812353366b.0 for ; Wed, 08 Jan 2025 07:19:05 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:04 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e63db730-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349545; x=1736954345; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3YsfPYnnI6FWUYgHfjtYUL/QekhFikCxvRDQSMOAmr4=; b=CP3lwKLnbK8m7BwkCP6XQHqCxwhpL2tmYCLq2a8Q+TVguTyR5aMDjO1Xs70puHmLWx GRQ0T20vQ2vQsbByyrEvGojbEA8gljByCgOtIYNpjuMGMrMhlfWvmryIw7t4YS0yjA8K ZU43p05ieyVX06KRjiKLvQBrf6K28UmvwU19Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349545; x=1736954345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3YsfPYnnI6FWUYgHfjtYUL/QekhFikCxvRDQSMOAmr4=; b=A8Eo6oF0eChv+iKtFS14M2zuTVF9XwX8tNBFhF6XrkWjbXo69/DlW3QURtF2I7KT5i IzhyYZrltDx/b1+sAlruiGHzyf5Y7m+j5cmv7OQOwglmbY71eXzVdVcvBH0nP/RFS1XW OABLtKlTxZ8L5bPmby4Drqkri/iwXNAptnhu0maNPeB3T+3SHv7fcjZi1p1Xs/RehNU7 +cxPbwODZbGIkVpmBx44Tl+EAzMmZpy74yhq6KTXUChgH1rV/cXGOJWSno80EPtFYg2j 4MR69NHPMLe8GndQllwdh+Ng9cwAb2/574isjEk8qLMapkU1W0iwoKn0EQh0cupa+ihl oZtQ== X-Gm-Message-State: AOJu0Yye+YXNmEu7WpjbtyIVzwLMNRLw4iFwBbvsUF+XU0l9vVqHEBGs UzcZzd1xvDp0RyxW1Iz0Sb7FvFmu+t0+UoJkNu59SISgvp7FCX7605AztIUugiSI8XGJkAn+Li4 QFiGnHw== X-Gm-Gg: ASbGncuhskvXfBFQIDYKTzncA50EMdEEIrZdbboeBYWvCUQNSegRsHddKGtvSYKPuJA Xkcyqp/Z3IRIM7FRDnny/O133WHggRRl6OGE9cfIceeelHk8RoABlmvgYEQEZhi956gfDN/zMIw kbNFPMxh0tyL+8sZDCLw0bQYQKNgaaNHN/AqNp1qLFzpveDht1agpbnMVclDn71fVPkcXh2o7zw UkZD7gWe2w2lGjwDhbLDn+76Kc6C5J9k1/5ixDjALxckAioRBdlOGWhaTFHSZ+C9gquD/bNofmI kNY= X-Google-Smtp-Source: AGHT+IEUj4gOUwHUn/rIAicijRfStFxoaCwoTk6QHyfvmCd+pbRo3s4LpEwHMg36vX8h3/K60AU/og== X-Received: by 2002:a17:907:3da1:b0:aaf:3f57:9d2e with SMTP id a640c23a62f3a-ab2aaaf6571mr264290166b.0.1736349544953; Wed, 08 Jan 2025 07:19:04 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Wei Liu , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Wang , Hongyan Xia , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 04/15] x86: Initialize mapcache for PV, HVM, and idle domains Date: Wed, 8 Jan 2025 15:18:11 +0000 Message-ID: <20250108151822.16030-5-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Wei Liu To support the transition away from the direct map, the mapcache will now be used by HVM and idle domains as well. This patch lifts the `mapcache` to the arch level and moves its initialization to `arch_domain_create()`. For the idle domain to utilize the mapcache, this patch also populates the mapcache page tables within the `PERDOMAIN` region and adjusts the initialization sequence in `arch_idle_init()`, as it's no longer covered by `arch_domain_create()` With this change, mapcache initialization is now unified across all domain types—PV, HVM, and idle. Signed-off-by: Wei Liu Signed-off-by: Wei Wang Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Move mapcache initialization and cleanup back to arch-specific code and reword commit message to reflect it. Since v3, the idle domain gained its own arch-init function, so initialise the mapcache there instead. Panic on failure to initialise it for the idle domain, as there's no possible recovery. v2->v4: * Reword the commit message * Rebase it on top of staging * The logic for the creation of the domain has been reworked so introduced #ifdef CONFIG_X86 in the common code to initialise the mapcache v1->v2: * Free resources if mapcache initialisation fails * Remove `is_idle_domain()` check from `create_perdomain_mappings()` --- xen/arch/x86/domain.c | 13 ++++++++++--- xen/arch/x86/domain_page.c | 22 ++++++++++------------ xen/arch/x86/include/asm/domain.h | 12 ++++++------ 3 files changed, 26 insertions(+), 21 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 78a13e6812c9..307ec0f11fed 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -777,6 +777,12 @@ void __init arch_init_idle_domain(struct domain *d) }; d->arch.ctxt_switch = &idle_csw; + + BUG_ON(mapcache_domain_init(d)); + + /* Slot 260: Per-domain mappings. */ + idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] = + l4e_from_page(d->arch.perdomain_l3_pg, __PAGE_HYPERVISOR_RW); } int arch_domain_create(struct domain *d, @@ -832,6 +838,10 @@ int arch_domain_create(struct domain *d, spec_ctrl_init_domain(d); + rc = mapcache_domain_init(d); + if ( rc ) + goto fail; + if ( (rc = paging_domain_init(d)) != 0 ) goto fail; paging_initialised = true; @@ -870,9 +880,6 @@ int arch_domain_create(struct domain *d, } else if ( is_pv_domain(d) ) { - if ( (rc = mapcache_domain_init(d)) != 0 ) - goto fail; - if ( (rc = pv_domain_initialise(d)) != 0 ) goto fail; } diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index eac5e3304fb8..55e337aaf703 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn) #endif v = mapcache_current_vcpu(); - if ( !v || !is_pv_vcpu(v) ) + if ( !v ) return mfn_to_virt(mfn_x(mfn)); - dcache = &v->domain->arch.pv.mapcache; - vcache = &v->arch.pv.mapcache; + dcache = &v->domain->arch.mapcache; + vcache = &v->arch.mapcache; if ( !dcache->inuse ) return mfn_to_virt(mfn_x(mfn)); @@ -187,14 +187,14 @@ void unmap_domain_page(const void *ptr) ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); v = mapcache_current_vcpu(); - ASSERT(v && is_pv_vcpu(v)); + ASSERT(v); - dcache = &v->domain->arch.pv.mapcache; + dcache = &v->domain->arch.mapcache; ASSERT(dcache->inuse); idx = PFN_DOWN(va - MAPCACHE_VIRT_START); mfn = l1e_get_pfn(MAPCACHE_L1ENT(idx)); - hashent = &v->arch.pv.mapcache.hash[MAPHASH_HASHFN(mfn)]; + hashent = &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)]; local_irq_save(flags); @@ -233,11 +233,9 @@ void unmap_domain_page(const void *ptr) int mapcache_domain_init(struct domain *d) { - struct mapcache_domain *dcache = &d->arch.pv.mapcache; + struct mapcache_domain *dcache = &d->arch.mapcache; unsigned int bitmap_pages; - ASSERT(is_pv_domain(d)); - #ifdef NDEBUG if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) return 0; @@ -261,12 +259,12 @@ int mapcache_domain_init(struct domain *d) int mapcache_vcpu_init(struct vcpu *v) { struct domain *d = v->domain; - struct mapcache_domain *dcache = &d->arch.pv.mapcache; + struct mapcache_domain *dcache = &d->arch.mapcache; unsigned long i; unsigned int ents = d->max_vcpus * MAPCACHE_VCPU_ENTRIES; unsigned int nr = PFN_UP(BITS_TO_LONGS(ents) * sizeof(long)); - if ( !is_pv_vcpu(v) || !dcache->inuse ) + if ( !dcache->inuse ) return 0; if ( ents > dcache->entries ) @@ -293,7 +291,7 @@ int mapcache_vcpu_init(struct vcpu *v) BUILD_BUG_ON(MAPHASHENT_NOTINUSE < MAPCACHE_ENTRIES); for ( i = 0; i < MAPHASH_ENTRIES; i++ ) { - struct vcpu_maphash_entry *hashent = &v->arch.pv.mapcache.hash[i]; + struct vcpu_maphash_entry *hashent = &v->arch.mapcache.hash[i]; hashent->mfn = ~0UL; /* never valid to map */ hashent->idx = MAPHASHENT_NOTINUSE; diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h index b5a14991ca0b..470192646b50 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -287,9 +287,6 @@ struct pv_domain /* Mitigate L1TF with shadow/crashing? */ bool check_l1tf; - /* map_domain_page() mapping cache. */ - struct mapcache_domain mapcache; - struct cpuidmasks *cpuidmasks; }; @@ -328,6 +325,9 @@ struct arch_domain uint8_t scf; /* See SCF_DOM_MASK */ + /* map_domain_page() mapping cache. */ + struct mapcache_domain mapcache; + union { struct pv_domain pv; struct hvm_domain hvm; @@ -518,9 +518,6 @@ struct arch_domain struct pv_vcpu { - /* map_domain_page() mapping cache. */ - struct mapcache_vcpu mapcache; - unsigned int vgc_flags; struct trap_info *trap_ctxt; @@ -614,6 +611,9 @@ struct arch_vcpu #define async_exception_state(t) async_exception_state[(t)-1] uint8_t async_exception_mask; + /* map_domain_page() mapping cache. */ + struct mapcache_vcpu mapcache; + /* Virtual Machine Extensions */ union { struct pv_vcpu pv; From patchwork Wed Jan 8 15:18:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60A06E77199 for ; Wed, 8 Jan 2025 15:19:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867484.1279053 (Exim 4.92) (envelope-from ) id 1tVXpt-00015m-M1; Wed, 08 Jan 2025 15:19:09 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867484.1279053; Wed, 08 Jan 2025 15:19:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpt-00013u-GA; Wed, 08 Jan 2025 15:19:09 +0000 Received: by outflank-mailman (input) for mailman id 867484; Wed, 08 Jan 2025 15:19:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpr-0008Lg-Lh for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:07 +0000 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [2a00:1450:4864:20::62a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e6f60577-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:07 +0100 (CET) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-aa6a92f863cso532874266b.1 for ; Wed, 08 Jan 2025 07:19:06 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:05 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e6f60577-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349546; x=1736954346; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JwBhhGsIi6j0431N7dPj7kwiLad+N3xTknrKun361rM=; b=M8n8xpYd9vewjvtm5b76xNCSnGbb1XXttXuIjHEpNx/D4qdLC7YaX2HvC/lGCPTtqh h7F20sgMxTr1a863CknvsiC0FEpxogkLYVq5+iYwNvoftU1bXGPKP3Q5fUeg11tKDNM8 njDRjSelwbthUFejJjz6KnctQinwPDt5wtfVg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349546; x=1736954346; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JwBhhGsIi6j0431N7dPj7kwiLad+N3xTknrKun361rM=; b=FD9mCh9T9ccYg26fyaY5RhNSDoJtBUgFZUeQf9b3d+8G3JLsyFOvEpLzSO8GuVQub/ M4taYRzQ2roo2LIJmqzi7qZ63NKZiExiq4av4/oVX6eh6nY/9vzKrgfkJkdrrtNZflFh Xj6MHPgWh8Wakuk1btrpcvFfO2d752dk25TZXZedvGlypGoJMrTDCj8ApF99C5XXecqf QpiKU5Wzs7ZDWfyPTfV4CW1xcZLd6/Qe0HBVyPzEW4UdGzFAOprOUh+4XHkKxFWQ5MNW uBxBGBG4H7y51Heay2ubPdSmvFa3Q6lbvdCxp0ss3WtbFyrI59Jfc3KMCAvDTuVywxcv no2A== X-Gm-Message-State: AOJu0Yx0EEbnRzRMjALD284nIHpu02mqrYo/pmpsCelGji5H2FthiyR3 AjbwQ4y7kS0UnCMNXiLFTaY/kz10dmHwosxyz2OrChlzvjcpoELnO1Z0YFiRoHl9brLfAFrJlHr DKDAPBg== X-Gm-Gg: ASbGncsqeXJJInbWIdRG+UkAIuH125muNoyqyTxgITcsu6h5TRkI0uLTpnb1b4SwCB6 TCMVrBPJuYe3kH4wEK6RnjqvhTbEaO8JnZIvtAU7pe4tSV1YAKhCD7PL/0/oyxP9nUb4gJz5Z6a wRN5DYUNxRMCiUpd+CcLflAEgmYzpMoFEuHD6xDNGgSnie/dO23er5TUKyouidUf2wzWzIa9fUM yud2SI/Vt32XYmffgpkzVsVmH0RETZ0lET6N6U0Id3UK98i3yfXSiCSok+vbLOAClXwIyUNsRZU YnU= X-Google-Smtp-Source: AGHT+IEmRfZTn8az4OYaE9fsLSrrsmECKXw/OTjUIfdtxZkfacPZRnaTOoT2mNfX9WgazQmGZBl7sA== X-Received: by 2002:a17:907:7fa5:b0:aa6:5d30:d976 with SMTP id a640c23a62f3a-ab2ab6a84f7mr275128066b.10.1736349546043; Wed, 08 Jan 2025 07:19:06 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 05/15] x86: Add a boot option to enable and disable the direct map Date: Wed, 8 Jan 2025 15:18:12 +0000 Message-ID: <20250108151822.16030-6-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia This is added as a Kconfig option as well as a boot command line option. While being generic, the Kconfig option is only usable for x86 at the moment. Note that there remains some users of the directmap at this point. The option is introduced now as it will be needed in follow-up patches. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- There's a major change compared with v4. directmap= turned into asi= for compatibility with Roger's ASI series. In particular, after everything is done we expect the commandline to have somewhat different effects per arch depending on the extent of ASI implemented. e.g: x86 might implement distinct toggles for PV and HVM, which are nonsensical for anything but x86. With the intent of parsing the commandline differently I've modified this patch to parse it on arch-specific code from the get-go. I also adjusted the commandline text to make it a lot more similar to the final text expected on Roger's series. v4->v5: * Moved cmdline parsing to arch-specific code and turned directmap= to asi= (where asi == !directmap). This is for compatibility with Roger's work, which appends extra suboptions to asi= on x86. * Moved the printk() highlighting the sparse directmap to spec-ctrl with the rest of speculative mitigations. Further additions to asi= will interact with XPTI, so it makes sense to have it all there. * Moved CONFIG_ONDEMAND_DIRECTMAP to the speculation hardening section of Kconfig. * Reworded the help message to align it with other rows in that section and make it more ameanable for end users. * Simplified #ifdef hunk defining has_directmap() * Used #ifdef CONFIG_ONDEMAND_DIRECTMAP rather than CONFIG_HAS_ONDEMAND_DIRECTMAP. The former selects the feature, while the latter signals support for it on the arch. v3->v4: * Rename the Kconfig options * Set opt_directmap to true if CONFIG_HAS_ONDEMAND_DIRECTMAP is not enabled v2->v3: * No changes. v1->v2: * Introduce a Kconfig option * Reword the commit message * Make opt_directmap and helper generic Changes since Hongyan's version: * Reword the commit message * opt_directmap is only modified during boot so mark it as __ro_after_init --- docs/misc/xen-command-line.pandoc | 11 +++++++++++ xen/arch/x86/Kconfig | 1 + xen/arch/x86/include/asm/mm.h | 6 ++++++ xen/arch/x86/spec_ctrl.c | 7 +++++++ xen/common/Kconfig | 21 +++++++++++++++++++++ xen/include/xen/mm.h | 11 +++++++++++ 6 files changed, 57 insertions(+) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 08b0053f9ced..6a1351b6c09b 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -202,6 +202,17 @@ to appropriate auditing by Xen. Argo is disabled by default. This option is disabled by default, to protect domains from a DoS by a buggy or malicious other domain spamming the ring. +### asi (x86) +> `= ` + +> Default: `false` + +Offers control over whether the hypervisor will engage in Address Space +Isolation, by not having potentially sensitive information permanently mapped +in the directmap. Enabling this option populates the directmap sparsely on +demand, blocking exploits that leak secrets via speculative memory access in +the directmap. + ### asid (x86) > `= ` diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 9cdd04721afa..55f1e8702ab9 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -23,6 +23,7 @@ config X86 select HAS_IOPORTS select HAS_KEXEC select HAS_NS16550 + select HAS_ONDEMAND_DIRECTMAP select HAS_PASSTHROUGH select HAS_PCI select HAS_PCI_MSI diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 6c7e66ee21ab..4c82428b8b3e 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -643,11 +643,17 @@ void write_32bit_pse_identmap(uint32_t *l2); /* * x86 maps part of physical memory via the directmap region. * Return whether the range of MFN falls in the directmap region. + * + * Enabling ASI on the commandline (i.e: using the `asi=` option) causes the + * directmap to be mostly empty, so this always returns false in that case. */ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) { unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END); + if ( !has_directmap() ) + return false; + return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1); } diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index ced84750015c..f67e0139b81e 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -85,6 +85,11 @@ static int8_t __initdata opt_gds_mit = -1; static int8_t __initdata opt_div_scrub = -1; bool __ro_after_init opt_bp_spec_reduce = true; +#ifdef CONFIG_ONDEMAND_DIRECTMAP +bool __ro_after_init opt_ondemand_dmap; +boolean_param("asi", opt_ondemand_dmap); +#endif + static int __init cf_check parse_spec_ctrl(const char *s) { const char *ss; @@ -633,6 +638,8 @@ static void __init print_details(enum ind_thunk thunk) cpu_has_bug_l1tf ? "" : " not", l1d_maxphysaddr, paddr_bits, l1tf_safe_maddr); + printk(" ASI: %s", !has_directmap() ? "enabled" : "disabled"); + /* * Alternatives blocks for protecting against and/or virtualising * mitigation support for guests. diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 6166327f4d14..1ee498a3e9a7 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -74,6 +74,9 @@ config HAS_KEXEC config HAS_LLC_COLORING bool +config HAS_ONDEMAND_DIRECTMAP + bool + config HAS_PIRQ bool @@ -214,6 +217,24 @@ config SPECULATIVE_HARDEN_LOCK If unsure, say Y. +config ONDEMAND_DIRECTMAP + bool "On-Demand Directmap" + depends on HAS_ONDEMAND_DIRECTMAP + help + Contemporary processors may use speculative execution as a + performance optimisation, but this can potentially be abused by an + attacker to leak data via speculative sidechannels. + + When enabled, this option provides defense in depth by preventing + most RAM from being constantly mapped on the hypervisor, thereby + greatly reducing the scope of data leaks after a successful + speculative attack. + + This option is disabled by default at run time, and needs to be + enabled on the command line. + + If unsure, say N. + endmenu config DIT_DEFAULT diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index 16f733281af3..fe73057e1781 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -169,6 +169,17 @@ extern unsigned long max_page; extern unsigned long total_pages; extern paddr_t mem_hotplug; +#ifdef CONFIG_ONDEMAND_DIRECTMAP +extern bool opt_ondemand_dmap; + +static inline bool has_directmap(void) +{ + return !opt_ondemand_dmap; +} +#else +#define has_directmap() true +#endif + /* * Extra fault info types which are used to further describe * the source of an access violation. From patchwork Wed Jan 8 15:18:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA01BE77188 for ; Wed, 8 Jan 2025 15:19:24 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867485.1279065 (Exim 4.92) (envelope-from ) id 1tVXpv-0001OW-19; Wed, 08 Jan 2025 15:19:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867485.1279065; Wed, 08 Jan 2025 15:19:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpu-0001Nk-Nm; Wed, 08 Jan 2025 15:19:10 +0000 Received: by outflank-mailman (input) for mailman id 867485; Wed, 08 Jan 2025 15:19:08 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXps-0008Lg-GL for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:08 +0000 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [2a00:1450:4864:20::635]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e78e75fd-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:08 +0100 (CET) Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-aa67ac42819so2455543566b.0 for ; Wed, 08 Jan 2025 07:19:07 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:06 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e78e75fd-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349547; x=1736954347; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yjMmCnPvYhGWKY9jq+47lrrCTBXbD9mgdkXdbRy6L+U=; b=T93PYqB+WYv9aZXqGBBSHyzcJ12AJYpjmKiI8tLy0XIN3Ff2sBCULlOJTRnpg+kt3t EayJEpoioR6l92BSCOdvF1KA/fkfVAiL6eP1mxr5BLGxA7A5c+hMlDo3VYnRRrB2I3WH q24QD4WPuoJTFlv//KWTU25yySKyhYeOHa4OE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349547; x=1736954347; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yjMmCnPvYhGWKY9jq+47lrrCTBXbD9mgdkXdbRy6L+U=; b=Pb5k16uYU5fPwzxXi/yiEaPQRN5YPbFEgnkYAo86jLLTdodT8nJxSV6imgF+v4ujsf e9DFptFEV5q6w5udN7tjhmJTDcIYGVQp7nkpbxlf8jkJw3lw3TwE3IDm8bAQyUP0MoH2 EQ8dPz1yNAd7rC3cWf70z/IQhM4aNGuPAafaBhfue0k1RDDMkeqZJo+eG1/5vUjCWyIc AIsfia7SYRlDy9Zr3nvJGrOCy0fLaWZjFcHxVPs/wHhDDZpoiMU5RPLDAT+Ie9FCcg60 LWEwly3PJzUz0NUN9u6yMa/BRepARL6h3WCnvl+OHb0bqMX3EpIE8khrne71bqyGjXti XKvw== X-Gm-Message-State: AOJu0Yw9W6tB21VJ634nXKI2jDk1kvQRJ4UKl+O5D4wKMXNguKCt2u+h WGOcgan975qlVowuCRoIjrb8cIpsXNJuhvZsIp5415zQZX0BHDm8yl8UJ0L5ThswC3PyLsLK7wC Idmigrg== X-Gm-Gg: ASbGncsEG3s6loO5wSrzJw13jMdmIIT+mtqAqF2Fn9UezBX6HXpkW4PF6h7xNddHoI7 tf9yBiLTmZfu8depvCGLlGNk+Wpk08KacXfUevGZYP4KrH4E6DrysLGvZTHnSiZpXdIXgRftUsw YgLewGqjmbxa/+0qVLMP+XglF02+LJaNN9b1I9VIlNakM/y6yWK2cZ6uZjTvaoVqbqFvFjOby5E n8w6sFVAuQfHalya8rrpXNV+/P2LgqljnfopNaQIgkIwmOdG526ZH9hLJrFH7n9Cper5KhF0Wyp 2a4= X-Google-Smtp-Source: AGHT+IHm+GlwZHnqvLphieP9LH0FVfXA/XTlt6gMtPQsfzpf8oMG9ZAYuETNdwTvWKs+ujchLnpC2w== X-Received: by 2002:a17:907:94c8:b0:aaf:7321:f05a with SMTP id a640c23a62f3a-ab2abc6e773mr279314766b.46.1736349547208; Wed, 08 Jan 2025 07:19:07 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Julien Grall , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 06/15] xen/x86: Add support for the PMAP Date: Wed, 8 Jan 2025 15:18:13 +0000 Message-ID: <20250108151822.16030-7-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Julien Grall PMAP will be used in a follow-up patch to bootstrap map domain page infrastructure -- we need some way to map pages to setup the mapcache without a direct map. The functions pmap_{map, unmap} open code {set, clear}_fixmap to break the loop. Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * No changes. v3->v4: * Select PMAP KConfig option iff ONDEMAND_DIRECTMAP is used v2->v3: * No changes. v1->v2: * Declare PMAP entries earlier in fixed_addresses * Reword the commit message The PMAP infrastructure was upstream separately for Arm since Hongyan sent the secret-free hypervisor series. So this is a new patch to plumb the feature on x86. --- xen/arch/x86/include/asm/fixmap.h | 6 ++++++ xen/arch/x86/include/asm/pmap.h | 35 +++++++++++++++++++++++++++++++ xen/common/Kconfig | 1 + 3 files changed, 42 insertions(+) create mode 100644 xen/arch/x86/include/asm/pmap.h diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h index 516ec3fa6c95..80b7b74fd816 100644 --- a/xen/arch/x86/include/asm/fixmap.h +++ b/xen/arch/x86/include/asm/fixmap.h @@ -21,6 +21,8 @@ #include #include +#include + #include #include #include @@ -53,6 +55,10 @@ enum fixed_addresses { FIX_PV_CONSOLE, FIX_XEN_SHARED_INFO, #endif /* CONFIG_XEN_GUEST */ +#ifdef CONFIG_HAS_PMAP + FIX_PMAP_BEGIN, + FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP, +#endif /* Everything else should go further down. */ FIX_APIC_BASE, FIX_IO_APIC_BASE_0, diff --git a/xen/arch/x86/include/asm/pmap.h b/xen/arch/x86/include/asm/pmap.h new file mode 100644 index 000000000000..1b3b729b90b2 --- /dev/null +++ b/xen/arch/x86/include/asm/pmap.h @@ -0,0 +1,35 @@ +#ifndef __ASM_PMAP_H__ +#define __ASM_PMAP_H__ + +#include + +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) +{ + unsigned long linear = (unsigned long)fix_to_virt(slot); + l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)]; + + BUILD_BUG_ON(FIX_APIC_BASE - 1 > L1_PAGETABLE_ENTRIES - 1); + ASSERT(!(l1e_get_flags(*pl1e) & _PAGE_PRESENT)); + + l1e_write(pl1e, l1e_from_mfn(mfn, PAGE_HYPERVISOR)); +} + +static inline void arch_pmap_unmap(unsigned int slot) +{ + unsigned long linear = (unsigned long)fix_to_virt(slot); + l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)]; + + l1e_write(pl1e, l1e_empty()); + flush_tlb_one_local(linear); +} + +#endif /* __ASM_PMAP_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 1ee498a3e9a7..5b22b09a4fbc 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -220,6 +220,7 @@ config SPECULATIVE_HARDEN_LOCK config ONDEMAND_DIRECTMAP bool "On-Demand Directmap" depends on HAS_ONDEMAND_DIRECTMAP + select HAS_PMAP help Contemporary processors may use speculative execution as a performance optimisation, but this can potentially be abused by an From patchwork Wed Jan 8 15:18:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4BAF8E7719D for ; Wed, 8 Jan 2025 15:19:22 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867487.1279071 (Exim 4.92) (envelope-from ) id 1tVXpv-0001Vj-Jn; Wed, 08 Jan 2025 15:19:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867487.1279071; Wed, 08 Jan 2025 15:19:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpv-0001UP-9Z; Wed, 08 Jan 2025 15:19:11 +0000 Received: by outflank-mailman (input) for mailman id 867487; Wed, 08 Jan 2025 15:19:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpu-0008Ue-IL for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:10 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id e82d12c0-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:09 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aa692211331so197245766b.1 for ; Wed, 08 Jan 2025 07:19:09 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:07 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e82d12c0-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349548; x=1736954348; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jw7mWXVpCJfQ9bvcjiyKZM3RdMlwMcvwWhT8eTvVn6U=; b=gUNxLztGT+XuBuLXeJ9Y+muSqTAx9oG17WmI1QGVAMJ5mrCDqqjn/7kKVyNFDVIJ29 8wihzLbT9umRID8q2SfEcUHZIR+8BTI0tD8a+BbK86ZeO1ZiHUch4X773ENOj7R4lfwA mydv2YzQ0WCNANOEADXSgPpGpR6sPTIr9QOok= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349548; x=1736954348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jw7mWXVpCJfQ9bvcjiyKZM3RdMlwMcvwWhT8eTvVn6U=; b=SE1qVMkjSoaUscFxYV6HA0bCx8DIL1KAQQsBRZ7HC71GXqjlrgR6j64estYN9mEqDW NntetsQtlSJlfF5lxbYxLkXwWE7WbGwTP1/KhIQHA7dETNUGJJr2nxArYPTmWkofQ4H3 CH4eDx1pj/DYiR2bCnzDna497O/lWKs1plj78I5bVzHeSfJaJdEpAprDiHS+hsX0H7uc qimBI96vXi3LoU7aTqZ4UYI6AAWmb2PEeeV9XM3AGzK1vy1GVSAjoa1aoWAcmfdJiEbe T4oMdEn2qEHwgfggdQa1f1YqD2Av9hG+6o51qAWUCTxmuSeFd/dDhF5g9afu3rr3t1DJ CMtQ== X-Gm-Message-State: AOJu0YxsoiGaNL92kpM4ekLPUFtsH+H7bUTSIxF+7ZGpaT6wXfgodHUd /LvwuqJwUgBpbHobVNUXwZKRN2T9WqnYVTT0Q/9dy/l5ON07HCqv/00ZBnmDRa71IZwsnYYlNxd nA/gImQ== X-Gm-Gg: ASbGncu2cSnrP3zpgNxieNy8MxmbThKDbP/f9kyk5ENh8Q2V2m/xN7XdhRLzDoe4A44 NM+aHcv1veXiOJOZtBBovVJxtobGNSW/uNn+5/IbOPhCyiN1cCzmT1oYpKvC20Y9n4w2c0Ncj5D vDgzKFUKrTcuWRQuX2HOhQcurszOrXJHjjtihORhyAHT+MIOBNSS33RZS6F+09YSmIewvDj+h9/ KcOg+e9QWA8ojyrADlITDh33yteavBYpzzSIiAwoFGnnLUx/4A6+z9YsPnb2cLlP/Eo8UQEUscJ EwY= X-Google-Smtp-Source: AGHT+IHSHfjOlGcCZmim3Gu0rKS/1nvYqcoEGDp9YfwIaOq2RV4Rfts5cwCxRuUXRYrl6NitiJ0QjA== X-Received: by 2002:a17:907:2d0e:b0:aa6:945c:4531 with SMTP id a640c23a62f3a-ab2a79d1d18mr240135266b.7.1736349548147; Wed, 08 Jan 2025 07:19:08 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Alejandro Vallejo Subject: [PATCH v5 07/15] x86/domain_page: Remove the fast paths when mfn is not in the directmap Date: Wed, 8 Jan 2025 15:18:14 +0000 Message-ID: <20250108151822.16030-8-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia When mfn is not in direct map, never use mfn_to_virt for any mappings. We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with arch_mfns_in_direct_map(mfn, 1) because these two are equivalent. The extra comparison in arch_mfns_in_direct_map() looks different but because DIRECTMAP_VIRT_END is always higher, it does not make any difference. Lastly, domain_page_map_to_mfn() needs to gain to a special case for the PMAP. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Alejandro Vallejo --- v4->v5: * s/BUILD_BUG_ON/ASSERT/, or it won't build with gcc11. * Add CONFIG_HAS_PMAP guards as needed so the code builds without PMAP * Fix typos on 2 arch_mfns_in_directmap() calls in release config. v3->v4: * Introduce helper functions virt_is_fixmap and virt_in_fixmap_range Changes since Hongyan's version: * arch_mfn_in_direct_map() was renamed to arch_mfns_in_directmap() * add a special case for the PMAP in domain_page_map_to_mfn() --- xen/arch/x86/domain_page.c | 60 +++++++++++++++++++++++++------ xen/arch/x86/include/asm/fixmap.h | 25 +++++++++++++ 2 files changed, 75 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 55e337aaf703..9582bd63b5c3 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -14,8 +14,10 @@ #include #include #include +#include #include #include +#include #include static DEFINE_PER_CPU(struct vcpu *, override); @@ -24,6 +26,7 @@ static inline struct vcpu *mapcache_current_vcpu(void) { /* In the common case we use the mapcache of the running VCPU. */ struct vcpu *v = this_cpu(override) ?: current; + struct vcpu *idle_v = idle_vcpu[smp_processor_id()]; /* * When current isn't properly set up yet, this is equivalent to @@ -35,10 +38,11 @@ static inline struct vcpu *mapcache_current_vcpu(void) /* * When using efi runtime page tables, we have the equivalent of the idle * domain's page tables but current may point at another domain's VCPU. - * Return NULL as though current is not properly set up yet. + * Return the idle domains's vcpu on that core because the efi per-domain + * region (where the mapcache is) is in-sync with the idle domain. */ if ( efi_rs_using_pgtables() ) - return NULL; + return idle_v; /* * If guest_table is NULL, and we are running a paravirtualised guest, @@ -48,7 +52,7 @@ static inline struct vcpu *mapcache_current_vcpu(void) if ( unlikely(pagetable_is_null(v->arch.guest_table)) && is_pv_vcpu(v) ) { /* If we really are idling, perform lazy context switch now. */ - if ( (v = idle_vcpu[smp_processor_id()]) == current ) + if ( (v = idle_v) == current ) sync_local_execstate(); /* We must now be running on the idle page table. */ ASSERT(cr3_pa(read_cr3()) == __pa(idle_pg_table)); @@ -77,18 +81,28 @@ void *map_domain_page(mfn_t mfn) struct vcpu_maphash_entry *hashent; #ifdef NDEBUG - if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) return mfn_to_virt(mfn_x(mfn)); #endif v = mapcache_current_vcpu(); - if ( !v ) - return mfn_to_virt(mfn_x(mfn)); + if ( !v || !v->domain->arch.mapcache.inuse ) + { + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) + return mfn_to_virt(mfn_x(mfn)); + else + { +#ifdef CONFIG_HAS_PMAP + BUG_ON(system_state >= SYS_STATE_smp_boot); + return pmap_map(mfn); +#else + BUG(); +#endif + } + } dcache = &v->domain->arch.mapcache; vcache = &v->arch.mapcache; - if ( !dcache->inuse ) - return mfn_to_virt(mfn_x(mfn)); perfc_incr(map_domain_page_count); @@ -184,6 +198,14 @@ void unmap_domain_page(const void *ptr) if ( !va || va >= DIRECTMAP_VIRT_START ) return; +#ifdef CONFIG_HAS_PMAP + if ( virt_is_fixmap(va) ) + { + pmap_unmap(ptr); + return; + } +#endif + ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); v = mapcache_current_vcpu(); @@ -237,7 +259,7 @@ int mapcache_domain_init(struct domain *d) unsigned int bitmap_pages; #ifdef NDEBUG - if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( !mem_hotplug && arch_mfns_in_directmap(0, max_page) ) return 0; #endif @@ -308,7 +330,7 @@ void *map_domain_page_global(mfn_t mfn) local_irq_is_enabled())); #ifdef NDEBUG - if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) return mfn_to_virt(mfn_x(mfn)); #endif @@ -335,6 +357,24 @@ mfn_t domain_page_map_to_mfn(const void *ptr) if ( va >= DIRECTMAP_VIRT_START ) return _mfn(virt_to_mfn(ptr)); +#ifdef CONFIG_HAS_PMAP + /* + * The fixmap is stealing the top-end of the VMAP. So the check for + * the PMAP *must* happen first. + * + * Also, the fixmap translate a slot to an address backwards. The + * logic will rely on it to avoid any complexity. So check at + * compile time this will always hold. + */ + ASSERT(fix_to_virt(FIX_PMAP_BEGIN) > fix_to_virt(FIX_PMAP_END)); + + if ( virt_in_fixmap_range(va, FIX_PMAP_BEGIN, FIX_PMAP_END) ) + { + BUG_ON(system_state >= SYS_STATE_smp_boot); + return l1e_get_mfn(l1_fixmap[l1_table_offset(va)]); + } +#endif /* CONFIG_HAS_PMAP */ + if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END ) return vmap_to_mfn(va); diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h index 80b7b74fd816..381c95a8b11f 100644 --- a/xen/arch/x86/include/asm/fixmap.h +++ b/xen/arch/x86/include/asm/fixmap.h @@ -101,6 +101,31 @@ static inline unsigned long virt_to_fix(const unsigned long vaddr) return __virt_to_fix(vaddr); } +static inline bool virt_is_fixmap(const unsigned long vaddr) +{ + return vaddr >= FIXADDR_START && vaddr < FIXADDR_TOP; +} + +static inline bool virt_in_fixmap_range( + const unsigned long vaddr, + const unsigned int start_idx, + const unsigned int end_idx +) +{ + unsigned long start_addr = (unsigned long)fix_to_virt(start_idx); + unsigned long end_addr = (unsigned long)fix_to_virt(end_idx); + + /* + * The check ensures that the virtual address (vaddr) is within the + * fixmap range. The addresses are allocated backwards, meaning the + * start address is higher than the end address. As a result, the + * check ensures that the virtual address is greater than or equal to + * the end address, and less than or equal to the start address, which + * may appear counterintuitive due to the reverse allocation order. + */ + return ((vaddr & PAGE_MASK) <= start_addr) && (vaddr >= end_addr); +} + enum fixed_addresses_x { /* Index 0 is reserved since fix_x_to_virt(0) == FIXADDR_X_TOP. */ FIX_X_RESERVED, From patchwork Wed Jan 8 15:18:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74ADCE77188 for ; Wed, 8 Jan 2025 15:19:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867488.1279082 (Exim 4.92) (envelope-from ) id 1tVXpw-0001rT-Ql; Wed, 08 Jan 2025 15:19:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867488.1279082; Wed, 08 Jan 2025 15:19:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpw-0001qr-Is; Wed, 08 Jan 2025 15:19:12 +0000 Received: by outflank-mailman (input) for mailman id 867488; Wed, 08 Jan 2025 15:19:10 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpu-0008Lg-Qr for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:10 +0000 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [2a00:1450:4864:20::630]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e8ece51c-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:10 +0100 (CET) Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-aaf60d85238so1272582266b.0 for ; Wed, 08 Jan 2025 07:19:10 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:09 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e8ece51c-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349549; x=1736954349; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OW3IEcDQIGgERvNx7U/UfHJTPkVf/pejI3zoKLzHcZ4=; b=O72zaF7NS/cBoKnHQexNJbYarf9+dYvh96M2k8qFN+wUvi7IkFykhW2VmNbksaR9RH 7GITDaWY15+x0ChXWeLV2tTNQK9wr5MF5fpkFKzk50tnh8b63vY4FmjPzfZwaWevFmsc HZd4+Na3kmOUh0Bm+T8GN2w4ZIGWXWWJz/g+A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349549; x=1736954349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OW3IEcDQIGgERvNx7U/UfHJTPkVf/pejI3zoKLzHcZ4=; b=ubMthTtbcUycdSEFddVBcOLar8m3BhdxFLsC+JrBm8kwnMD8UIbmiHCXebqMtCpAMb 0N250MvhkD8MMCJQ25ny2Hsl0uhFVbKVBycAnhyYWJydg/SitilFDK3w1Muq/0TDGnls yRN5FjLnJ8wzA+Ff8peQ1nrfGzGEwnbaBS8ZKZAsnJ+CUWyvvBXQc4r0g0JmcSiacuwh VBkefj689YhIJvArELCN4CLx6GVRe8+Yw0ePhmmnPFcM95k2tVVH2vmxdxs/Lptzrr4x yk861rZ/GQCC2d6jj5ZyEg2w7H15wtANX7n7+TR9qmBBq1zZCe5yOLWrWxSxMT4J4/kW 9E2w== X-Gm-Message-State: AOJu0YzjYDDbIjY6KIW+QfbS8rlNMFZvwf5ERV+SquDIDEMguUlka7Y2 bCWhoxZIXjavxBMZko1UIB5vqn3g8ufLrGP0GYLvgQBXnTPyuVqD+ajrR3u7J5K2RYb607bQU4J hANFuoQ== X-Gm-Gg: ASbGncvzo5Oj155b46FsIDMSDAghELqsQYQsXnXFn5YxM4usWrbSB4G99/grBwuoYoT 1KTp1SeJF6PFq9w2cZsmep6Z+avuWkmgKKnIcfyuEbWf9fY/X8OOBgmo687Z1KRqeJnTYRZMFP0 PRZyIAAyin8UI8AxLNkCkdndwgeC1z2DtlmU3bHe/BcKUkmwCoc/jJWhpclR8RAyNp9iyPUeJ/5 VblwUkN5KT7O2hSw236f7jeKYzZZN52bWBx3IzNwMcJ38SysdcCJovdWHIgomDw8bLoueOUig8g Xvc= X-Google-Smtp-Source: AGHT+IEHcvVuDBftNzgvsSJjl9/9ijtvIIRWM0K6YyDq9Uo03CAZE0TpiiTY8VB9i1idNueu4C4Prg== X-Received: by 2002:a17:907:80c:b0:aab:daf9:972 with SMTP id a640c23a62f3a-ab2ab70c9afmr294425966b.28.1736349549423; Wed, 08 Jan 2025 07:19:09 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 08/15] xen/page_alloc: Add a path for xenheap when there is no direct map Date: Wed, 8 Jan 2025 15:18:15 +0000 Message-ID: <20250108151822.16030-9-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia When there is not an always-mapped direct map, xenheap allocations need to be mapped and unmapped on-demand. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Remove stray comma in printk() after XENLOG_WARNING. Elias @ v4: I have left the call to map_pages_to_xen() and destroy_xen_mappings() in the split heap for now. I am not entirely convinced this is necessary because in that setup only the xenheap would be always mapped and this doesn't contain any guest memory (aside the grant-table). So map/unmapping for every allocation seems unnecessary. v3->v4: * Call printk instead of dprintk() v1->v2: * Fix remaining wrong indentation in alloc_xenheap_pages() Changes since Hongyan's version: * Rebase * Fix indentation in alloc_xenheap_pages() * Fix build for arm32 --- xen/common/page_alloc.c | 43 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 41 insertions(+), 2 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 1bf070c8c5df..1c01332b6cb0 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -2435,6 +2435,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe) void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) { struct page_info *pg; + void *virt_addr; ASSERT_ALLOC_CONTEXT(); @@ -2443,17 +2444,36 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) if ( unlikely(pg == NULL) ) return NULL; - return page_to_virt(pg); + virt_addr = page_to_virt(pg); + + if ( !has_directmap() && + map_pages_to_xen((unsigned long)virt_addr, page_to_mfn(pg), 1UL << order, + PAGE_HYPERVISOR) ) + { + /* Failed to map xenheap pages. */ + free_heap_pages(pg, order, false); + return NULL; + } + + return virt_addr; } void free_xenheap_pages(void *v, unsigned int order) { + unsigned long va = (unsigned long)v & PAGE_MASK; + ASSERT_ALLOC_CONTEXT(); if ( v == NULL ) return; + if ( !has_directmap() && + destroy_xen_mappings(va, va + (PAGE_SIZE << order)) ) + printk(XENLOG_WARNING + "Error while destroying xenheap mappings at %p, order %u\n", + v, order); + free_heap_pages(virt_to_page(v), order, false); } @@ -2477,6 +2497,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) { struct page_info *pg; unsigned int i; + void *virt_addr; ASSERT_ALLOC_CONTEXT(); @@ -2489,16 +2510,28 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) if ( unlikely(pg == NULL) ) return NULL; + virt_addr = page_to_virt(pg); + + if ( !has_directmap() && + map_pages_to_xen((unsigned long)virt_addr, page_to_mfn(pg), 1UL << order, + PAGE_HYPERVISOR) ) + { + /* Failed to map xenheap pages. */ + free_domheap_pages(pg, order); + return NULL; + } + for ( i = 0; i < (1u << order); i++ ) pg[i].count_info |= PGC_xen_heap; - return page_to_virt(pg); + return virt_addr; } void free_xenheap_pages(void *v, unsigned int order) { struct page_info *pg; unsigned int i; + unsigned long va = (unsigned long)v & PAGE_MASK; ASSERT_ALLOC_CONTEXT(); @@ -2510,6 +2543,12 @@ void free_xenheap_pages(void *v, unsigned int order) for ( i = 0; i < (1u << order); i++ ) pg[i].count_info &= ~PGC_xen_heap; + if ( !has_directmap() && + destroy_xen_mappings(va, va + (PAGE_SIZE << order)) ) + printk(XENLOG_WARNING + "Error while destroying xenheap mappings at %p, order %u\n", + v, order); + free_heap_pages(pg, order, true); } From patchwork Wed Jan 8 15:18:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF06DE7719C for ; Wed, 8 Jan 2025 15:19:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867489.1279092 (Exim 4.92) (envelope-from ) id 1tVXpy-0002El-EL; Wed, 08 Jan 2025 15:19:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867489.1279092; Wed, 08 Jan 2025 15:19:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpy-0002DT-4b; Wed, 08 Jan 2025 15:19:14 +0000 Received: by outflank-mailman (input) for mailman id 867489; Wed, 08 Jan 2025 15:19:11 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpv-0008Lg-Ro for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:11 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e99c4ecb-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:11 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aa68b513abcso3100450766b.0 for ; Wed, 08 Jan 2025 07:19:11 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:10 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e99c4ecb-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349550; x=1736954350; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B1o5uv9Ja6v4/kcOJ8sZumXTXa5wvZv/gHrWxF6fu20=; b=ImEbpo7Rpm9n0aWOQ9skx4qvVdzUoRCxEVh9CJM1cOhiTmZ3YGJG/jEKeXF2dB8evM 7vVqq5EZZz7UO/tm1NJL5Dr0+RldfOxPXv10MXFKHyTXWDEOr+zmxrAX5PKObJJJMUef +7ylvgud9kTo/yml/lHomVdIwtIgSiG2/o6mI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349550; x=1736954350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B1o5uv9Ja6v4/kcOJ8sZumXTXa5wvZv/gHrWxF6fu20=; b=dP93HiOXpi0H6LLmRwfRKdolD2BBd3giREzMBgbhMx+52WFVM0yMVJ8r5kmmjPvXCY hr/0BUfGeGuOKLBnxX8k+5WIuXUdc0X4sVwwbXJIGTWl4e9y6YmsUmIQGGHjnNK0fPvz OUF6DunkzmTD9xSqCoqDIjAPUxPkzwIc3CPOTsrntp0IJ8brAHHzN+yts0pYQqMTXUNK BYYKMNq9L/TrTe0SpkgxADW+0taEr0YxX+r4IEUEIkllt6VPMB1wUm+enZRNfp4SQy/D XuFMzbu6vkdE+XHPQuFolt2nynLCEUcsoBTTRdovO2hdtYkM4D1hajeHbqFozTOc01L4 D91Q== X-Gm-Message-State: AOJu0YzPT4VQd+yZKP5l7V46F83y/+6bHCADx5UXAPjN47VMmv1cEKcj KdYaZcmA63eZijJh4M0p1jYVyMiq7hC5iHll2iLViFPEMIqi+FZ6WS0ioMtAZWbQKemUnCO6tNq vzIqtOA== X-Gm-Gg: ASbGnct0WZ5HM5U2i1ScDyyP222JB3RDyb1siH5NE9FBUJshrKsw4zkysKFjp4psLSc 0bosRp91DVG2wSTOimSpf1irCO3UNy1/5pJ57fOInR9+IQyWL6RUe/+UnYVpPMmaaO37egbRCM3 AKRQfkg7WpvVywWBrlrKEUg/hXUFji7CGr6v9424wVTOduJ2ak7M+OBElhSJXhRXspOxkP2joIg GkOHG/YfXSjJZCx0dRcl7p1SxpS7lD4YibvyjYuO0/BpobfP59E3ScuHOYhL8plki+Vl+8VZclv 55s= X-Google-Smtp-Source: AGHT+IFmAfbrkaRmO1XEvxpuHTtJZiGpHLiJR0cpXNNmmNnQWLh9ejYwG1frtDsro+4RNSQ6b49m0g== X-Received: by 2002:a17:907:7286:b0:aa6:89b9:e9c0 with SMTP id a640c23a62f3a-ab2ab6a7624mr229884666b.8.1736349550502; Wed, 08 Jan 2025 07:19:10 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 09/15] x86/setup: Leave early boot slightly earlier Date: Wed, 8 Jan 2025 15:18:16 +0000 Message-ID: <20250108151822.16030-10-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia When we do not have a direct map, memory for metadata of heap nodes in init_node_heap() is allocated from xenheap, which needs to be mapped and unmapped on demand. However, we cannot just take memory from the boot allocator to create the PTEs while we are passing memory to the heap allocator. To solve this race, we leave early boot slightly sooner so that Xen PTE pages are allocated from the heap instead of the boot allocator. We can do this because the metadata for the 1st node is statically allocated, and by the time we need memory to create mappings for the 2nd node, we already have enough memory in the heap allocator in the 1st node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * No changes. v3->v4: * Fix indentation * Refactor the code to reduce code duplication --- xen/arch/x86/setup.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 8ebe5a9443f3..609ec4cf07f2 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1831,6 +1831,22 @@ void asmlinkage __init noreturn __start_xen(void) numa_initmem_init(0, raw_max_page); + /* + * When we do not have a direct map, memory for metadata of heap nodes in + * init_node_heap() is allocated from xenheap, which needs to be mapped and + * unmapped on demand. However, we cannot just take memory from the boot + * allocator to create the PTEs while we are passing memory to the heap + * allocator during end_boot_allocator(). + * + * To solve this race, we need to leave early boot before + * end_boot_allocator() so that Xen PTE pages are allocated from the heap + * instead of the boot allocator. We can do this because the metadata for + * the 1st node is statically allocated, and by the time we need memory to + * create mappings for the 2nd node, we already have enough memory in the + * heap allocator in the 1st node. + */ + system_state = SYS_STATE_boot; + if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) ) { unsigned long lo = virt_to_mfn(HYPERVISOR_VIRT_END - 1); @@ -1862,8 +1878,6 @@ void asmlinkage __init noreturn __start_xen(void) else end_boot_allocator(); - system_state = SYS_STATE_boot; - bsp_stack = cpu_alloc_stack(0); if ( !bsp_stack ) panic("No memory for BSP stack\n"); From patchwork Wed Jan 8 15:18:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7874C02184 for ; Wed, 8 Jan 2025 15:19:24 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867490.1279102 (Exim 4.92) (envelope-from ) id 1tVXpz-0002Xo-W2; Wed, 08 Jan 2025 15:19:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867490.1279102; Wed, 08 Jan 2025 15:19:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpz-0002X1-Ka; Wed, 08 Jan 2025 15:19:15 +0000 Received: by outflank-mailman (input) for mailman id 867490; Wed, 08 Jan 2025 15:19:14 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpy-0008Ue-05 for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:14 +0000 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [2a00:1450:4864:20::62f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ea3f2af6-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:12 +0100 (CET) Received: by mail-ej1-x62f.google.com with SMTP id a640c23a62f3a-aaee0b309adso2115822366b.3 for ; Wed, 08 Jan 2025 07:19:12 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:11 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ea3f2af6-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349552; x=1736954352; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4dAl5hiiumxZdSL6vCO/VaNqrRA8aywqXxbQG/mKtGY=; b=Hmh+CMApb3O8FVO4bwKJSJMyHrpRmdeiirVhb4uxxifc3B0XpmYesVCsMIgr+xx4Lc a2lz/ukJhHz6QcVV4fFABgoy8K3jHsbAHMd/5F9WszbzEOce1Um3G+yofi+qWZ49sZoc 3BxutmNm50Vw4ZsSC1NsWS55fO4n+NRkgdKuk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349552; x=1736954352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4dAl5hiiumxZdSL6vCO/VaNqrRA8aywqXxbQG/mKtGY=; b=kZTkqYatzkthByTWjAhvCEMdYJNkpn2U8J930MseWSXE+sAAb8oF5m70rO8QIBEGt5 mwjTN4VRJO62ZUqelP0bop8f8GiL7GPQ96q/mQpRmhPyp3o7Abh/LIISNGwUryZM/nPs OVV0JIQahfoBYe7gMx+F53xPZjQdBZYSBRMFJmIT1ZqW3GnzlfFi2+29+K8aRH3+bu9Q QWOj1l3srNavi71gJjMsi0hn1UwhUFjOWaosAW3PZLCrvXA9CKhsPuXhTyYtemaLrE7+ ovTb6F4wxvWLlSdNdzW8wJo41ThupJJ8jOWdSH7IidAqRKnPTtAQozbmnRQnpr2gpPn6 O0QA== X-Gm-Message-State: AOJu0YwGupN/Nf04gQjGhAldnAov8KxdPp+aZPYlbugIROTqya5S+Ij2 EGBsonaUI+2cz/hZz/WVXRThEZbsREnnavcgcaOGRNK431QrFozjzHd8x9tsoM3J3al1/o6jWgR Ffs6f9Q== X-Gm-Gg: ASbGncuktTLp95nko05sR9HAephjFcmBCzAGwo8N7IMbDaycp3IOv+jpW1zDdjI7chn q1soFU6X4JduksKwN9Zv0HCnbeO5YEpEr9qJu58xRiBWqd+e/z4B67wP9r7ip9YQzzScv7wYLU8 3GYJFm2HE2bjTWr4pgUmOVtPV4FYUXkBUgQSgYz3SwXWNLrhorCDZd6MybdWz2VzK20kU5orMiv lwY9B2wyzyNsZ4iyPQWlxBGv7cl3GWHRQNAEXgVXS5d3bHbH658jH83D/byqjilc3+wnVrFnVK1 oqY= X-Google-Smtp-Source: AGHT+IFxfaz1of1Dy+QRE/RL4Nf+b/dHh+YwZM7uFybv3bH9J3u0daWHFg1k3w3NgvYUSIrBHUHeaQ== X-Received: by 2002:a17:907:70c:b0:aa6:9176:61ed with SMTP id a640c23a62f3a-ab2abc6d42emr300126166b.48.1736349551627; Wed, 08 Jan 2025 07:19:11 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 10/15] xen/page_alloc: vmap heap nodes when they are outside the direct map Date: Wed, 8 Jan 2025 15:18:17 +0000 Message-ID: <20250108151822.16030-11-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia When we do not have a direct map, archs_mfn_in_direct_map() will always return false, thus init_node_heap() will allocate xenheap pages from an existing node for the metadata of a new node. This means that the metadata of a new node is in a different node, slowing down heap allocation. Since we now have early vmap, vmap the metadata locally in the new node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Fix bug introduced in v4 by which node metadata would be unconditionally mapped at the tail of the heap node. * Remove extra space in conditional v3->v4: * Change type of the parameters to paddr_t * Use clear_domain_page() instead of open-coding it v1->v2: * vmap_contig_pages() was renamed to vmap_contig() * Fix indentation and coding style Changes from Hongyan's version: * arch_mfn_in_direct_map() was renamed to arch_mfns_in_direct_map() * Use vmap_contig_pages() rather than __vmap(...). * Add missing include (xen/vmap.h) so it compiles on Arm --- xen/common/page_alloc.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 1c01332b6cb0..3af86a213c4e 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -139,6 +139,7 @@ #include #include #include +#include #include #include @@ -615,22 +616,30 @@ static unsigned long init_node_heap(int node, unsigned long mfn, needed = 0; } else if ( *use_tail && nr >= needed && - arch_mfns_in_directmap(mfn + nr - needed, needed) && (!xenheap_bits || !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn + nr - needed); - avail[node] = mfn_to_virt(mfn + nr - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; + if ( arch_mfns_in_directmap(mfn + nr - needed, needed) ) + _heap[node] = mfn_to_virt(mfn + nr - needed); + else + _heap[node] = vmap_contig(_mfn(mfn + nr - needed), needed); + + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; } else if ( nr >= needed && - arch_mfns_in_directmap(mfn, needed) && (!xenheap_bits || !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn); - avail[node] = mfn_to_virt(mfn + needed - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; + if ( arch_mfns_in_directmap(mfn, needed) ) + _heap[node] = mfn_to_virt(mfn); + else + _heap[node] = vmap_contig(_mfn(mfn), needed); + + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; *use_tail = false; } else if ( get_order_from_bytes(sizeof(**_heap)) == From patchwork Wed Jan 8 15:18:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68C47E77199 for ; Wed, 8 Jan 2025 15:19:26 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867493.1279117 (Exim 4.92) (envelope-from ) id 1tVXq2-0002zc-JD; Wed, 08 Jan 2025 15:19:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867493.1279117; Wed, 08 Jan 2025 15:19:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq1-0002x2-SM; Wed, 08 Jan 2025 15:19:17 +0000 Received: by outflank-mailman (input) for mailman id 867493; Wed, 08 Jan 2025 15:19:15 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpz-0008Ue-N1 for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:15 +0000 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [2a00:1450:4864:20::530]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ead458f6-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:13 +0100 (CET) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-5d3d14336f0so10201984a12.3 for ; Wed, 08 Jan 2025 07:19:13 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:12 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ead458f6-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349553; x=1736954353; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t3Cqxub2b0rcz5hsX+nKFFTBraQpc6Cf0wNJ5J2Y5Kk=; b=cOR7jUI+eFa3+6GqKNoZDW9t62TZUmubaP5YaQQs7ImmW3zr9En5TZdiKRrGTHIhgE VatN+IbZXyyUg3LJ69qV5fYS1KC1/QukHP4xiIiQaNW6oY/Lm6MmXAsKVkAbwvHB9eW7 W+Liz95yF/wlb7Epy13g2732UIyf88a993vL4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349553; x=1736954353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t3Cqxub2b0rcz5hsX+nKFFTBraQpc6Cf0wNJ5J2Y5Kk=; b=QkFCDD00ec2y29pv8ePj7/Qw2OOt+SxRFksgSkyYfq01A6NU4IsZ3LYGMTJZAg+TzS FZQW4lyLQBpm4tTGc6UCo+t6by3vsdJ/SfQZwnRNYkRk2fjbOCK8SkZ9jdc7Z7aYmKAF HHLCDJU9Uc28RG8A3qaMPrB8t99ZCUNq4yTvwOrPguaSGG76RWvhZ1FcFXAwJIoOZ/CF 2Qhezkm1J0eDdTBkq+1GMOI+iy6PCKjLigLmNy6B0ogtOvLldavTmDUu8PDQH7kkh1yf 5mJMYGkBrh07ForlKvBUhLduTHKQ5KZKcq4gbotUuPusflo6PAYexDjloWihEzXVCe8Y 78uA== X-Gm-Message-State: AOJu0Yz5br1KLIjODl8cSxZLvpim99Sj3C+LHqEPFXIuo9XvZCui8PBL 53Kv4pH/DteJXFgwQRI73zT584FYYXYO9i0Z9XOmYC+fEDnYebLW6lAOPrHScRB6Hbi7Gm6yoLu l9MnlQQ== X-Gm-Gg: ASbGnctj6efBGsPnNvXwCVYV3xU5ohpOR1KPV6hjG88yETjy15+DocYCZQAw4iXFDbT YolXx4djXAZwvSiAk/1DG7lR4WQ8cZYQJ/wjreL5GPIJVtIeds5iMwNJIGX47mLp+ZVyxRVmoJh +WMdRwe4Tl9k3ZB8pjjl9G7z1KIiYG7lG7Squ9N4VUGY2n/mS33uQH+RR59oxwP4syJDVtvtKVD Qo5NXJCZbpcUNCk3kP+Ol/EjuX2qsmbTVFCxQ5Z5RC6dy9PanWnjTwpojU6ojhcYr/Th9B7dHsN avY= X-Google-Smtp-Source: AGHT+IHqrn/5VYtgEnnsa6k46BuFXzyBRGCwDHd32H/cc0VkKHYs5ezd33Pd3pAtPdvk2DJ4qBaqdQ== X-Received: by 2002:a17:907:3f92:b0:aac:439:10ce with SMTP id a640c23a62f3a-ab2ab73e812mr245573266b.27.1736349552593; Wed, 08 Jan 2025 07:19:12 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Julien Grall , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 11/15] x86/setup: Do not create valid mappings when directmap=no Date: Wed, 8 Jan 2025 15:18:18 +0000 Message-ID: <20250108151822.16030-12-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Hongyan Xia Create empty mappings in the second e820 pass. Also, destroy existing direct map mappings created in the first pass. To make xenheap pages visible in guests, it is necessary to create empty L3 tables in the direct map even when directmap=no, since guest cr3s copy idle domain's L4 entries, which means they will share mappings in the direct map if we pre-populate idle domain's L4 entries and L3 tables. A helper is introduced for this. Also, after the direct map is actually gone, we need to stop updating the direct map in update_xen_mappings(). Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * No changes. --- xen/arch/x86/setup.c | 73 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 66 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 609ec4cf07f2..23b77f13bc10 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1060,6 +1060,56 @@ static struct domain *__init create_dom0(struct boot_info *bi) return d; } +/* + * This either populates a valid direct map, or allocates empty L3 tables and + * creates the L4 entries for virtual address between [start, end) in the + * direct map depending on has_directmap(); + * + * When directmap=no, we still need to populate empty L3 tables in the + * direct map region. The reason is that on-demand xenheap mappings are + * created in the idle domain's page table but must be seen by + * everyone. Since all domains share the direct map L4 entries, they + * will share xenheap mappings if we pre-populate the L4 entries and L3 + * tables in the direct map region for all RAM. We also rely on the fact + * that L3 tables are never freed. + */ +static void __init populate_directmap(paddr_t pstart, paddr_t pend, + unsigned int flags) +{ + unsigned long vstart = (unsigned long)__va(pstart); + unsigned long vend = (unsigned long)__va(pend); + + if ( pstart >= pend ) + return; + + BUG_ON(vstart < DIRECTMAP_VIRT_START); + BUG_ON(vend > DIRECTMAP_VIRT_END); + + if ( has_directmap() ) + /* Populate valid direct map. */ + BUG_ON(map_pages_to_xen(vstart, maddr_to_mfn(pstart), + PFN_DOWN(pend - pstart), flags)); + else + { + /* Create empty L3 tables. */ + unsigned long vaddr = vstart & ~((1UL << L4_PAGETABLE_SHIFT) - 1); + + for ( unsigned long idx = l4_table_offset(vaddr); + idx <= l4_table_offset(vend); idx++ ) + { + l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(idx)]; + + if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) ) + { + mfn_t mfn = alloc_boot_pages(1, 1); + + clear_domain_page(mfn); + l4e_write(pl4e, l4e_from_mfn(mfn, __PAGE_HYPERVISOR)); + } + } + } +} + /* How much of the directmap is prebuilt at compile time. */ #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT) @@ -1681,8 +1731,17 @@ void asmlinkage __init noreturn __start_xen(void) map_e = min_t(uint64_t, e, ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT); - /* Pass mapped memory to allocator /before/ creating new mappings. */ + /* + * Pass mapped memory to allocator /before/ creating new mappings. + * The direct map for the bottom 4GiB has been populated in the first + * e820 pass. In the second pass, we make sure those existing mappings + * are destroyed when directmap=no. + */ init_boot_pages(s, min(map_s, e)); + if ( !has_directmap() ) + destroy_xen_mappings((unsigned long)__va(s), + (unsigned long)__va(min(map_s, e))); + s = map_s; if ( s < map_e ) { @@ -1690,6 +1749,9 @@ void asmlinkage __init noreturn __start_xen(void) map_s = (s + mask) & ~mask; map_e &= ~mask; init_boot_pages(map_s, map_e); + if ( !has_directmap() ) + destroy_xen_mappings((unsigned long)__va(map_s), + (unsigned long)__va(map_e)); } if ( map_s > map_e ) @@ -1703,8 +1765,7 @@ void asmlinkage __init noreturn __start_xen(void) if ( map_e < end ) { - map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), - PFN_DOWN(end - map_e), PAGE_HYPERVISOR); + populate_directmap(map_e, end, PAGE_HYPERVISOR); init_boot_pages(map_e, end); map_e = end; } @@ -1713,13 +1774,11 @@ void asmlinkage __init noreturn __start_xen(void) { /* This range must not be passed to the boot allocator and * must also not be mapped with _PAGE_GLOBAL. */ - map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), - PFN_DOWN(e - map_e), __PAGE_HYPERVISOR_RW); + populate_directmap(map_e, e, __PAGE_HYPERVISOR_RW); } if ( s < map_s ) { - map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s), - PFN_DOWN(map_s - s), PAGE_HYPERVISOR); + populate_directmap(s, map_s, PAGE_HYPERVISOR); init_boot_pages(s, map_s); } } From patchwork Wed Jan 8 15:18:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D6CE7C02183 for ; Wed, 8 Jan 2025 15:19:26 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867492.1279110 (Exim 4.92) (envelope-from ) id 1tVXq1-0002s0-I0; Wed, 08 Jan 2025 15:19:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867492.1279110; Wed, 08 Jan 2025 15:19:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq1-0002r2-CQ; Wed, 08 Jan 2025 15:19:17 +0000 Received: by outflank-mailman (input) for mailman id 867492; Wed, 08 Jan 2025 15:19:15 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXpz-0008Lg-IS for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:15 +0000 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [2a00:1450:4864:20::62a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id eb7b0015-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:14 +0100 (CET) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-aaf60d85238so1272591666b.0 for ; Wed, 08 Jan 2025 07:19:14 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:13 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: eb7b0015-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349554; x=1736954354; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mBjkyLdsM17iRKfOvcTvCdENHWOQ8GdEz5R+mPR7wOI=; b=OsHcG82ueMHXdbQVs94HoeEMORhwZ7cUMwx9u3+gHCIDprc1cQroBeUZae/XNlU7O2 CbncZ7V1gB59SuHpwbuiULUAtP8uM18bwjjNJ/ENPVXaMohcPjVpgelZaC4AQOW8WpA/ 6FvucMMpkQd5rCiZsy3dR4w36Dr+ctZ9QhenU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349554; x=1736954354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mBjkyLdsM17iRKfOvcTvCdENHWOQ8GdEz5R+mPR7wOI=; b=NNw7p9GA8gBr0FRruG7Ynk5WDmRKcHJn7HgeeBlJhIB6n+sWnFyFv3sNP1Me5Z1B66 qfCw4pbEMO86O+5fSa8W0J6G6k6+Wc1Cf+1l2XPUJBNocgtSYXEUCX9cMOq5OKr2cnK5 TziD6KJqcTK59LSwQt1tR7fm7IttDNGRzHHM/hLAWqNdo2o2ukRjJ1CZbUK8FN1r/hWr FfE04zT5epJ4Es98HCoCnxXTqW5OH0eLZaWRw+XrqbPWYRr+ES0GhyISdrnwU3ZFFPor 9CElo+d/vXo5LTvvkBkJ7P3JZjYPsndUnjTLPuHkRjfOPmXmLb27sJrj9m8Z8HUithrb lKcg== X-Gm-Message-State: AOJu0YwDtheWfaEVyatbIPdtCi4iCIHml2Av3Q3fR/+BZLwCGHCelGmt Jtd4Dv8XIoFBhQFeQEyU8KF0idKXcVwHG56xPHFd2hCa9DDyEgBaJ8IB4GVirvvWvK5tlscdR3F uaJAUAQ== X-Gm-Gg: ASbGncv+0oTllUgdVf47aF6vT+vqFs8aU9eNYj1Yo5rKn4Mix29blmyd+3NMV7nDSyh XWIbUL6KgTtGrOeQ/cjVLXHkCFVeRt9+ZRaVf9RIMTbk5iCCBdhniTzpPFyiCM1RYbqFBBOJcSH EUNYkgEoMTj2i65Ebi3U3DseMneoElxjny4S0fAayBkI/D/pRZWTmn+0AJzEJggvr3jeCfLuRcs BgavPwL70MtpNEai/y1aIXtYUoEdLXc9sPgIfHCc+ejyS69msC7Jpi8K7mdflRy5Jv+eMWaor8t Btg= X-Google-Smtp-Source: AGHT+IFr/4vVsl04wRMj0lsbKLbG9ZbWt/CJpvwrdHYprt+TFDd3PrslNZSj1LFhnlLTVEjrn+bd9Q== X-Received: by 2002:a17:907:1b12:b0:aae:85b4:a07 with SMTP id a640c23a62f3a-ab2ab675c58mr266911366b.8.1736349553655; Wed, 08 Jan 2025 07:19:13 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 12/15] xen/arm64: mm: Use per-pCPU page-tables Date: Wed, 8 Jan 2025 15:18:19 +0000 Message-ID: <20250108151822.16030-13-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Julien Grall At the moment, on Arm64, every pCPU is sharing the same page-tables. In a follow-up patch, we will allow the possibility to remove the direct map and therefore it will be necessary to have a mapcache. While we have plenty of spare virtual address space to reserve part for each pCPU, it means that temporary mappings (e.g. guest memory) could be accessible by every pCPU. In order to increase our security posture, it would be better if those mappings are only accessible by the pCPU doing the temporary mapping. In addition to that, a per-pCPU page-tables opens the way to have per-domain mapping area. Arm32 is already using per-pCPU page-tables so most of the code can be re-used. Arm64 doesn't yet have support for the mapcache, so a stub is provided (moved to its own header asm/domain_page.h). Take the opportunity to fix a typo in a comment that is modified. Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Added missing asm/domain_page.h header to arm32. Compilation fails otherwise. * NOTE: I rebased this patch over the LLC coloring as best as I could and may have messed it up. Please do double check. --- xen/arch/arm/arm32/mmu/mm.c | 1 + xen/arch/arm/arm64/mmu/mm.c | 3 ++- xen/arch/arm/include/asm/arm32/mm.h | 8 -------- xen/arch/arm/include/asm/domain_page.h | 13 +++++++++++++ xen/arch/arm/include/asm/mm.h | 3 +++ xen/arch/arm/include/asm/mmu/mm.h | 2 ++ xen/arch/arm/mmu/pt.c | 6 +++--- xen/arch/arm/mmu/setup.c | 23 ++++++++++------------- xen/arch/arm/mmu/smpboot.c | 16 +--------------- xen/arch/arm/setup.c | 1 + 10 files changed, 36 insertions(+), 40 deletions(-) create mode 100644 xen/arch/arm/include/asm/domain_page.h diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c index 956693232a1b..60b7f4f40512 100644 --- a/xen/arch/arm/arm32/mmu/mm.c +++ b/xen/arch/arm/arm32/mmu/mm.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c index 26361c4fe4c0..7de5885cc776 100644 --- a/xen/arch/arm/arm64/mmu/mm.c +++ b/xen/arch/arm/arm64/mmu/mm.c @@ -77,6 +77,7 @@ static void __init prepare_runtime_identity_mapping(void) paddr_t id_addr = virt_to_maddr(_start); lpae_t pte; DECLARE_OFFSETS(id_offsets, id_addr); + lpae_t *root = this_cpu(xen_pgtable); if ( id_offsets[0] >= IDENTITY_MAPPING_AREA_NR_L0 ) panic("Cannot handle ID mapping above %uTB\n", @@ -87,7 +88,7 @@ static void __init prepare_runtime_identity_mapping(void) pte.pt.table = 1; pte.pt.xn = 0; - write_pte(&xen_pgtable[id_offsets[0]], pte); + write_pte(&root[id_offsets[0]], pte); /* Link second ID table */ pte = pte_of_xenaddr((vaddr_t)xen_second_id); diff --git a/xen/arch/arm/include/asm/arm32/mm.h b/xen/arch/arm/include/asm/arm32/mm.h index 856f2dbec4ad..87a315db013d 100644 --- a/xen/arch/arm/include/asm/arm32/mm.h +++ b/xen/arch/arm/include/asm/arm32/mm.h @@ -1,12 +1,6 @@ #ifndef __ARM_ARM32_MM_H__ #define __ARM_ARM32_MM_H__ -#include - -#include - -DECLARE_PER_CPU(lpae_t *, xen_pgtable); - /* * Only a limited amount of RAM, called xenheap, is always mapped on ARM32. * For convenience always return false. @@ -16,8 +10,6 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) return false; } -bool init_domheap_mappings(unsigned int cpu); - static inline void arch_setup_page_tables(void) { } diff --git a/xen/arch/arm/include/asm/domain_page.h b/xen/arch/arm/include/asm/domain_page.h new file mode 100644 index 000000000000..e9f52685e2ec --- /dev/null +++ b/xen/arch/arm/include/asm/domain_page.h @@ -0,0 +1,13 @@ +#ifndef __ASM_ARM_DOMAIN_PAGE_H__ +#define __ASM_ARM_DOMAIN_PAGE_H__ + +#ifdef CONFIG_ARCH_MAP_DOMAIN_PAGE +bool init_domheap_mappings(unsigned int cpu); +#else +static inline bool init_domheap_mappings(unsigned int cpu) +{ + return true; +} +#endif + +#endif /* __ASM_ARM_DOMAIN_PAGE_H__ */ diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index f91ff088f6b1..07329a17fffa 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -2,6 +2,9 @@ #define __ARCH_ARM_MM__ #include +#include + +#include #include #include #include diff --git a/xen/arch/arm/include/asm/mmu/mm.h b/xen/arch/arm/include/asm/mmu/mm.h index f5a00558c47b..5a8fde313693 100644 --- a/xen/arch/arm/include/asm/mmu/mm.h +++ b/xen/arch/arm/include/asm/mmu/mm.h @@ -2,6 +2,8 @@ #ifndef __ARM_MMU_MM_H__ #define __ARM_MMU_MM_H__ +DECLARE_PER_CPU(lpae_t *, xen_pgtable); + /* Non-boot CPUs use this to find the correct pagetables. */ extern uint64_t init_ttbr; diff --git a/xen/arch/arm/mmu/pt.c b/xen/arch/arm/mmu/pt.c index da28d669e796..1ed1a53ab1f2 100644 --- a/xen/arch/arm/mmu/pt.c +++ b/xen/arch/arm/mmu/pt.c @@ -607,9 +607,9 @@ static int xen_pt_update(unsigned long virt, unsigned long left = nr_mfns; /* - * For arm32, page-tables are different on each CPUs. Yet, they share - * some common mappings. It is assumed that only common mappings - * will be modified with this function. + * Page-tables are different on each CPU. Yet, they share some common + * mappings. It is assumed that only common mappings will be modified + * with this function. * * XXX: Add a check. */ diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c index 30afe9778194..d9308e0475ff 100644 --- a/xen/arch/arm/mmu/setup.c +++ b/xen/arch/arm/mmu/setup.c @@ -34,17 +34,15 @@ * PCPUs. */ -#ifdef CONFIG_ARM_64 -DEFINE_PAGE_TABLE(xen_pgtable); -static DEFINE_PAGE_TABLE(xen_first); -#define THIS_CPU_PGTABLE xen_pgtable -#else /* Per-CPU pagetable pages */ /* xen_pgtable == root of the trie (zeroeth level on 64-bit, first on 32-bit) */ DEFINE_PER_CPU(lpae_t *, xen_pgtable); #define THIS_CPU_PGTABLE this_cpu(xen_pgtable) /* Root of the trie for cpu0, other CPU's PTs are dynamically allocated */ static DEFINE_PAGE_TABLE(cpu0_pgtable); + +#ifdef CONFIG_ARM_64 +static DEFINE_PAGE_TABLE(xen_first); #endif /* Common pagetable leaves */ @@ -368,17 +366,20 @@ void __init setup_pagetables(void) if ( llc_coloring_enabled ) create_llc_coloring_mappings(); + p = cpu0_pgtable; + + /* arch_setup_page_tables() may need to access the root page-tables. */ + per_cpu(xen_pgtable, 0) = cpu0_pgtable; + arch_setup_page_tables(); #ifdef CONFIG_ARM_64 pte = pte_of_xenaddr((uintptr_t)xen_first); pte.pt.table = 1; pte.pt.xn = 0; - xen_pgtable[zeroeth_table_offset(XEN_VIRT_START)] = pte; + p[zeroeth_table_offset(XEN_VIRT_START)] = pte; - p = (void *) xen_first; -#else - p = (void *) cpu0_pgtable; + p = xen_first; #endif /* Map xen second level page-table */ @@ -415,10 +416,6 @@ void __init setup_pagetables(void) pte.pt.table = 1; xen_second[second_table_offset(FIXMAP_ADDR(0))] = pte; -#ifdef CONFIG_ARM_32 - per_cpu(xen_pgtable, 0) = cpu0_pgtable; -#endif - if ( llc_coloring_enabled ) { ttbr = virt_to_maddr(virt_to_reloc_virt(THIS_CPU_PGTABLE)); diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index 37e91d72b785..e4bde31605bd 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -7,6 +7,7 @@ #include +#include #include /* Override macros from asm/page.h to make them work with mfn_t */ @@ -93,20 +94,6 @@ static void set_init_ttbr(lpae_t *root) unmap_domain_page(ptr); } -#ifdef CONFIG_ARM_64 -int prepare_secondary_mm(int cpu) -{ - clear_boot_pagetables(); - - /* - * Set init_ttbr for this CPU coming up. All CPUs share a single setof - * pagetables, but rewrite it each time for consistency with 32 bit. - */ - set_init_ttbr(xen_pgtable); - - return 0; -} -#else int prepare_secondary_mm(int cpu) { lpae_t *root = alloc_xenheap_page(); @@ -136,7 +123,6 @@ int prepare_secondary_mm(int cpu) return 0; } -#endif /* * Local variables: diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index c1f2d1b89d43..3b1ab6be3fbd 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include From patchwork Wed Jan 8 15:18:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 52717E7719C for ; Wed, 8 Jan 2025 15:19:28 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867496.1279124 (Exim 4.92) (envelope-from ) id 1tVXq3-00039u-SC; Wed, 08 Jan 2025 15:19:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867496.1279124; Wed, 08 Jan 2025 15:19:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq2-00037b-Q0; Wed, 08 Jan 2025 15:19:18 +0000 Received: by outflank-mailman (input) for mailman id 867496; Wed, 08 Jan 2025 15:19:17 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq1-0008Ue-2k for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:17 +0000 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [2a00:1450:4864:20::533]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ec0bb7f6-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:15 +0100 (CET) Received: by mail-ed1-x533.google.com with SMTP id 4fb4d7f45d1cf-5d982de9547so395569a12.2 for ; Wed, 08 Jan 2025 07:19:15 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:14 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec0bb7f6-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349555; x=1736954355; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LC/g8x4OedY7trloFSvWpnFUkcUT8czZLIGCqOW++Ak=; b=ZMH4rZ5n/W52oYgp1tkLEVCeBzzJWq7QFWI2CSKUE9u337bxBbx4XQffWax71gQv+S 1L5uinKZa68Y0XWmDHO9pOG0dr5bJVSovlnoRw6XWWkqnBHGs+E6zNHk8hYenb9/ReGF UbyqZ7uv922V4HetHW1khkjajK81LlJGPvSDI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349555; x=1736954355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LC/g8x4OedY7trloFSvWpnFUkcUT8czZLIGCqOW++Ak=; b=iCx/+mYxicilCDAmCCxM+GLx9G6e29a+Yp7hYrajwf3QJ0TMSK3J615Eax9yszpDeD 39ZEq2Uijf6R56Z3MdMf21xK3VdXz/cPSV8KSmqXdrtqGpOa77NOVUn+QE1CpWzHQNzr kLw5D6ETy/adigkyzS6C9hiA7DTvP3hv25aYws0nxlgRV/tKEL/Hb4tMyptyQ4Pqbrf6 RR7oiKhSdy4OxJzn2r8/iwOlCFMy3MyRj30zT4ew37fBea+b6mlckQJIT3b+bnwP20oE G0pbqAYQTQiK1I9jibxb7jsP7c4jUMyt1z2ocouignBGLPcxp7RfoXibvNCkUGKsd9I1 BvlA== X-Gm-Message-State: AOJu0YxVZ+ZHO01U22dXywOGSpaIXwZcgdD46NyhqRV0mVLgA9LRGZry s96pp00aLbiz/wsrzg28Tra3DD3w9/bbNkMn2ci11IbR7FfUTaHL0X232Jzizho/JhdGQvb3pw8 VU3VNNQ== X-Gm-Gg: ASbGncsboEhwv5a7Lx6imknQjPkoiNh3IrEDvf+jq8mc/bsRuUUV+2AxH+hVfQBv8kK nxvuoYZih0geYpJVh8Pkg4zPTW8fPQECd8TbnqvTCDPOhp7VWR1mdS3d1tMRaMzSQruXyPEKt5i fpKzUUVN1Ik2KVGz9HbzKSTGpdeLkBRuheF/1L9b8tP7vLv86RSD8IQCHmwnQ6WCVaeCrDQNaKm +Q2oT6gyqF8/Q4uq/k8+F7KsFuY/zSlcSO9u+E6phDJ/0DP8sXOBmAqwAOdZDhUXgl8PLxhyDGL uQk= X-Google-Smtp-Source: AGHT+IFgF0QSUh74KgWW/mU+J3nBtJOl0SmmwvnWpBL0fo639DqkdvXF9jxD0MEjJF6UsgiJnul6MQ== X-Received: by 2002:a05:6402:354c:b0:5d3:ba42:e9d6 with SMTP id 4fb4d7f45d1cf-5d972e14828mr3116152a12.17.1736349554733; Wed, 08 Jan 2025 07:19:14 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v5 13/15] xen/arm32: Hardwire zeroeth_table_offset to 0 on ARM_32 Date: Wed, 8 Jan 2025 15:18:20 +0000 Message-ID: <20250108151822.16030-14-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Include arm32 in 7c72147baa22("xen/arm: Restrict zeroeth_table_offset for ARM_64"). Otherwise `va` overflows on shift in DECLARE_OFFSETS(). Fixes: 7c72147baa22("xen/arm: Restrict zeroeth_table_offset for ARM_64") Signed-off-by: Alejandro Vallejo --- xen/arch/arm/include/asm/lpae.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/arch/arm/include/asm/lpae.h b/xen/arch/arm/include/asm/lpae.h index 4a1679cb3334..d07456ffc8e3 100644 --- a/xen/arch/arm/include/asm/lpae.h +++ b/xen/arch/arm/include/asm/lpae.h @@ -259,7 +259,7 @@ lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr); #define first_table_offset(va) TABLE_OFFSET(first_linear_offset(va)) #define second_table_offset(va) TABLE_OFFSET(second_linear_offset(va)) #define third_table_offset(va) TABLE_OFFSET(third_linear_offset(va)) -#ifdef CONFIG_PHYS_ADDR_T_32 +#if defined(CONFIG_PHYS_ADDR_T_32) || defined(CONFIG_ARM_32) #define zeroeth_table_offset(va) 0 #else #define zeroeth_table_offset(va) TABLE_OFFSET(zeroeth_linear_offset(va)) From patchwork Wed Jan 8 15:18:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59B5DE7719A for ; Wed, 8 Jan 2025 15:19:31 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867501.1279136 (Exim 4.92) (envelope-from ) id 1tVXq6-0003tS-4Y; Wed, 08 Jan 2025 15:19:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867501.1279136; Wed, 08 Jan 2025 15:19:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq5-0003rk-Pf; Wed, 08 Jan 2025 15:19:21 +0000 Received: by outflank-mailman (input) for mailman id 867501; Wed, 08 Jan 2025 15:19:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq4-0008Ue-0w for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:20 +0000 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [2a00:1450:4864:20::532]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ecba1b49-cdd3-11ef-99a4-01e77a169b0f; Wed, 08 Jan 2025 16:19:16 +0100 (CET) Received: by mail-ed1-x532.google.com with SMTP id 4fb4d7f45d1cf-5d3e9f60bf4so30149211a12.3 for ; Wed, 08 Jan 2025 07:19:16 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:15 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ecba1b49-cdd3-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349556; x=1736954356; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TcWHO2t2N7UwLnGLQCw7cWTCMCFtS4oV7aIoPrHjvaM=; b=UdXmqDiOmVrAZ2I/sa/7gqtokXWMtdp0UMvvyA4ZZ5n2f+DHsD/K32V+wxV+xTqeHP 0xZebVgylrTnAeR93o+qzKqytvIHByDP2yjl/zHiFeIqapVXMCW+BNsp/++c7/GEUg+M C2sNDDg0pmcg+8CNCB3maT1Dqag5ztXPznUpc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349556; x=1736954356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TcWHO2t2N7UwLnGLQCw7cWTCMCFtS4oV7aIoPrHjvaM=; b=ORMaKUXaSyTif+tA+eOtamvpaRTtPs8HbQGFrEWeM/Nghx2Vk5pBkgm9SvjPf9VhfN kom3OSgYzKgXMdgxAavnAzLRxqzBMka3bp0jeb5xlXzUKoNUc+lgIyx4r8rWTJTXUqdt q45lfWQuDJT7pTrRwMWPBw1LiaO6d3Y8d3dEg4iWOtWldfiUR8q7gNZtfnZ5ZYxSB3fk C/J9hkEhh6H5j7z/B2do2RRGxMFtTg+HcwEY7O9DfRdtxE11x2V24WYUQ1cunpWa1OiI 49HKzm1z9edsfgIqOspLCkXwL7WHIfnIu6xSZ3tbpWcJHBxc6iI0cPzAL+p+tRr3GY26 KkPw== X-Gm-Message-State: AOJu0YxIKdEy8KMzMsGIRBJSjTxAXeToyU4eLiAUC3qn7bz+TiYjaMgK Q6IuLKE4BglOVqiJ/WcwHXqOBvdn73uQ544AYuPFD2x6Gi6Ppn+WhoDRL1KYAjIFmUlaMvxKgwT gEU8/vQ== X-Gm-Gg: ASbGncvRw7OhMwlNpzb1tMKZxHWMJRtfl1bOInE0sqLjs18VT6TDqKfU7K8FtN0znPE HlQ+jARqhQKP87aygI30dolTwRFCTKytQCpAOVjUjKWuwTNDm3Fv8azHbxi7TyjOZyyMDT/xsZf OsZMvWNvcXiMN9oPA2NXiYK6jFNqx35UOzv3VJrG1DFjVjrZ317EX8xkPrt5OTT4HyHgRwfbZNj 0K2yyOAkclD90hjc+iHgAzaf0ZTGKmtkt3y2kr96HNGHCadOF4upMNq+aeorI76pAS2lLKWXLWP /bE= X-Google-Smtp-Source: AGHT+IF9fxsdMLev/SrCilzHh6Pzzv4UGiliFO6+R9GQV0Y5+wSNaw2EGspt7tbA+yvRoGwZ7vZbYg== X-Received: by 2002:a05:6402:388a:b0:5d1:2631:b897 with SMTP id 4fb4d7f45d1cf-5d972e08403mr3094678a12.14.1736349555696; Wed, 08 Jan 2025 07:19:15 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 14/15] xen/arm64: Implement a mapcache for arm64 Date: Wed, 8 Jan 2025 15:18:21 +0000 Message-ID: <20250108151822.16030-15-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Julien Grall At the moment, on arm64, map_domain_page() is implemented using virt_to_mfn(). Therefore it is relying on the directmap. In a follow-up patch, we will allow the admin to remove the directmap. Therefore we want to implement a mapcache. Thanksfully there is already one for arm32. So select ARCH_ARM_DOMAIN_PAGE and add the necessary boiler plate to support 64-bit: - The page-table start at level 0, so we need to allocate the level 1 page-table - map_domain_page() should check if the page is in the directmap. If yes, then use virt_to_mfn() to limit the performance impact when the directmap is still enabled (this will be selectable on the command line). Take the opportunity to replace first_table_offset(...) with offsets[...]. Note that, so far, arch_mfns_in_directmap() always return true on arm64. So the mapcache is not yet used. This will change in a follow-up patch. Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Add missing "select ARCH_MAP_DOMAIN_PAGE". It was weirdly dropped from v2. * Bugfix: Unwrap mfn_t before passing it to mfn_to_virt() in map_domain_page(). Elias @ v4: There are a few TODOs: - It is becoming more critical to fix the mapcache implementation (this is not compliant with the Arm Arm) - Evaluate the performance --- xen/arch/arm/Kconfig | 2 +- xen/arch/arm/arm64/mmu/mm.c | 9 ++++++ xen/arch/arm/include/asm/mm.h | 5 +++ xen/arch/arm/include/asm/mmu/layout.h | 13 +++++++- xen/arch/arm/mmu/domain_page.c | 45 ++++++++++++++++++++++++--- xen/arch/arm/mmu/pt.c | 6 ++-- 6 files changed, 71 insertions(+), 9 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index a26d3e11827c..5c31bb616608 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -1,7 +1,6 @@ config ARM_32 def_bool y depends on "$(ARCH)" = "arm32" - select ARCH_MAP_DOMAIN_PAGE config ARM_64 def_bool y @@ -12,6 +11,7 @@ config ARM_64 config ARM def_bool y + select ARCH_MAP_DOMAIN_PAGE select FUNCTION_ALIGNMENT_4B select GENERIC_UART_INIT select HAS_ALTERNATIVE if HAS_VMAP diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c index 7de5885cc776..8e121e5ffe8d 100644 --- a/xen/arch/arm/arm64/mmu/mm.c +++ b/xen/arch/arm/arm64/mmu/mm.c @@ -5,6 +5,7 @@ #include #include +#include #include #include #include @@ -283,6 +284,14 @@ void __init setup_mm(void) setup_frametable_mappings(ram_start, ram_end); max_page = PFN_DOWN(ram_end); + /* + * The allocators may need to use map_domain_page() (such as for + * scrubbing pages). So we need to prepare the domheap area first. + */ + if ( !init_domheap_mappings(smp_processor_id()) ) + panic("CPU%u: Unable to prepare the domheap page-tables\n", + smp_processor_id()); + init_staticmem_pages(); init_sharedmem_pages(); } diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 07329a17fffa..0a4dc53a6050 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -434,6 +434,11 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn) } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x ); } +/* Helpers to allocate, map and unmap a Xen page-table */ +int create_xen_table(lpae_t *entry); +lpae_t *xen_map_table(mfn_t mfn); +void xen_unmap_table(const lpae_t *table); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/include/asm/mmu/layout.h b/xen/arch/arm/include/asm/mmu/layout.h index 19c0ec63a59a..35f4204ce76a 100644 --- a/xen/arch/arm/include/asm/mmu/layout.h +++ b/xen/arch/arm/include/asm/mmu/layout.h @@ -36,9 +36,13 @@ * * 32G - 64G Frametable: 56 bytes per page for 2TB of RAM * - * 0x00000a8000000000 - 0x00007fffffffffff (512GB+117TB, L0 slots [21..255]) + * 0x00000a8000000000 - 0x00007f7fffffffff (117TB, L0 slots [21..254]) * Unused * + * 0x00007f8000000000 - 0x00007fffffffffff (512GB, L0 slot [255]) + * (Relative offsets) + * 0 - 2G Domheap: on-demand-mapped + * * 0x0000800000000000 - 0x000084ffffffffff (5TB, L0 slots [256..265]) * 1:1 mapping of RAM * @@ -133,6 +137,13 @@ #define FRAMETABLE_SIZE GB(32) #define FRAMETABLE_NR (FRAMETABLE_SIZE / sizeof(*frame_table)) +#define DOMHEAP_VIRT_START SLOT0(255) +#define DOMHEAP_VIRT_SIZE GB(2) + +#define DOMHEAP_ENTRIES 1024 /* 1024 2MB mapping slots */ +/* Number of domheap pagetable pages required at the second level (2MB mappings) */ +#define DOMHEAP_SECOND_PAGES (DOMHEAP_VIRT_SIZE >> FIRST_SHIFT) + #define DIRECTMAP_VIRT_START SLOT0(256) #define DIRECTMAP_SIZE (SLOT0_ENTRY_SIZE * (266 - 256)) #define DIRECTMAP_VIRT_END (DIRECTMAP_VIRT_START + DIRECTMAP_SIZE - 1) diff --git a/xen/arch/arm/mmu/domain_page.c b/xen/arch/arm/mmu/domain_page.c index 3a43601623f0..7276c2b3b868 100644 --- a/xen/arch/arm/mmu/domain_page.c +++ b/xen/arch/arm/mmu/domain_page.c @@ -29,13 +29,30 @@ bool init_domheap_mappings(unsigned int cpu) { unsigned int order = get_order_from_pages(DOMHEAP_SECOND_PAGES); lpae_t *root = per_cpu(xen_pgtable, cpu); + lpae_t *first; unsigned int i, first_idx; lpae_t *domheap; mfn_t mfn; + /* Convenience aliases */ + DECLARE_OFFSETS(offsets, DOMHEAP_VIRT_START); + ASSERT(root); ASSERT(!per_cpu(xen_dommap, cpu)); + /* + * On Arm64, the root is at level 0. Therefore we need an extra step + * to allocate the first level page-table. + */ +#ifdef CONFIG_ARM_64 + if ( create_xen_table(&root[offsets[0]]) ) + return false; + + first = xen_map_table(lpae_get_mfn(root[offsets[0]])); +#else + first = root; +#endif + /* * The domheap for cpu0 is initialized before the heap is initialized. * So we need to use pre-allocated pages. @@ -56,16 +73,20 @@ bool init_domheap_mappings(unsigned int cpu) * domheap mapping pages. */ mfn = virt_to_mfn(domheap); - first_idx = first_table_offset(DOMHEAP_VIRT_START); + first_idx = offsets[1]; for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ ) { lpae_t pte = mfn_to_xen_entry(mfn_add(mfn, i), MT_NORMAL); pte.pt.table = 1; - write_pte(&root[first_idx + i], pte); + write_pte(&first[first_idx + i], pte); } per_cpu(xen_dommap, cpu) = domheap; +#ifdef CONFIG_ARM_64 + xen_unmap_table(first); +#endif + return true; } @@ -89,6 +110,10 @@ void *map_domain_page(mfn_t mfn) lpae_t pte; int i, slot; + /* Bypass the mapcache if the page is in the directmap */ + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) + return mfn_to_virt(mfn_x(mfn)); + local_irq_save(flags); /* The map is laid out as an open-addressed hash table where each @@ -151,13 +176,25 @@ void *map_domain_page(mfn_t mfn) /* Release a mapping taken with map_domain_page() */ void unmap_domain_page(const void *ptr) { + unsigned long va = (unsigned long)ptr; unsigned long flags; lpae_t *map = this_cpu(xen_dommap); - int slot = ((unsigned long)ptr - DOMHEAP_VIRT_START) >> SECOND_SHIFT; + unsigned int slot; + + /* Below we assume that the domheap area doesn't start at 0 */ + BUILD_BUG_ON(DOMHEAP_VIRT_START == 0); - if ( !ptr ) + /* + * map_domain_page() may not have mapped anything if the address + * is part of the directmap. So ignore anything outside of the + * domheap. + */ + if ( (va < DOMHEAP_VIRT_START) || + ((va - DOMHEAP_VIRT_START) >= DOMHEAP_VIRT_SIZE) ) return; + slot = (va - DOMHEAP_VIRT_START) >> SECOND_SHIFT; + local_irq_save(flags); ASSERT(slot >= 0 && slot < DOMHEAP_ENTRIES); diff --git a/xen/arch/arm/mmu/pt.c b/xen/arch/arm/mmu/pt.c index 1ed1a53ab1f2..da33c6c52e39 100644 --- a/xen/arch/arm/mmu/pt.c +++ b/xen/arch/arm/mmu/pt.c @@ -33,7 +33,7 @@ mm_printk(const char *fmt, ...) {} #define HYP_PT_ROOT_LEVEL 1 #endif -static lpae_t *xen_map_table(mfn_t mfn) +lpae_t *xen_map_table(mfn_t mfn) { /* * During early boot, map_domain_page() may be unusable. Use the @@ -45,7 +45,7 @@ static lpae_t *xen_map_table(mfn_t mfn) return map_domain_page(mfn); } -static void xen_unmap_table(const lpae_t *table) +void xen_unmap_table(const lpae_t *table) { /* * During early boot, xen_map_table() will not use map_domain_page() @@ -228,7 +228,7 @@ void *ioremap(paddr_t pa, size_t len) return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE); } -static int create_xen_table(lpae_t *entry) +int create_xen_table(lpae_t *entry) { mfn_t mfn; void *p; From patchwork Wed Jan 8 15:18:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13931151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57E28E77188 for ; Wed, 8 Jan 2025 15:21:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.867551.1279154 (Exim 4.92) (envelope-from ) id 1tVXri-0000jk-9z; Wed, 08 Jan 2025 15:21:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 867551.1279154; Wed, 08 Jan 2025 15:21:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXri-0000jd-5u; Wed, 08 Jan 2025 15:21:02 +0000 Received: by outflank-mailman (input) for mailman id 867551; Wed, 08 Jan 2025 15:21:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tVXq2-0008Lg-Bx for xen-devel@lists.xenproject.org; Wed, 08 Jan 2025 15:19:18 +0000 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [2a00:1450:4864:20::531]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ed63cabb-cdd3-11ef-a0df-8be0dac302b0; Wed, 08 Jan 2025 16:19:17 +0100 (CET) Received: by mail-ed1-x531.google.com with SMTP id 4fb4d7f45d1cf-5d122cf8dd1so29422630a12.2 for ; Wed, 08 Jan 2025 07:19:17 -0800 (PST) Received: from localhost.localdomain ([66.81.170.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0eae71desm2488412166b.89.2025.01.08.07.19.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 07:19:16 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ed63cabb-cdd3-11ef-a0df-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1736349557; x=1736954357; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XzrS2Bw0H8USJct01IYqEGdeeRooPpKhcpnuOPYIt28=; b=T9EE7M546zVv71S6qXEjvuyB1KHnNnR7rJCvJfMgr7/IzWXoXOKonOiAXWA6KNxLcv Rl4vzX0I85fsIAYYZUT9uVoIdbEYgZVeLvF/N/rMKZn9Xog9Q7teN0YJ7SGukVsn36Gv OP6Hf/SxFAZ3XKvwUdjqIOOY5J/h7LjVQJRqg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736349557; x=1736954357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XzrS2Bw0H8USJct01IYqEGdeeRooPpKhcpnuOPYIt28=; b=qrsALIhSM/uwb6qyrnvz8vVXOqYvzeeRlEfjkZH/hcdSeRkb+v7OGmZi5mUJni3p7E U2d4PBWy/XtvsV4n6ikUxChBxpGCTJa6Y/hjptffW7o/VkHwMDW9m789ux/VQ871263O dCBDqrXm+bAg/iQv7p6AQqYdDu/eQoqIyge7m6dnFXCccPQfqGheKrUI3YtsusAC3jYs qv9MPG8RJ73Wx5we2ejq/2tVHCczuliVsLFn3A6u7HDhRenU5bepUFRRx98kKzHaWZFX kfQytqlyAKD2wVVNfj/mgkjmC6ZNYt/j7FZqTgFSE/esCvIjcp4PmC0FDSUfpNnTFpsk vadA== X-Gm-Message-State: AOJu0YxNO+G1HdW3kqwiUJGPLtTxLbPBUL0+zav2KZiRtDA6qh6ADppZ Z97KpmMmQX7BCWAsjmXlhtSXLz9L3nifUveKWFY6mmU5KZqb++geLYUZa8rwBSxDdqWdCtQyRj3 bwZTpbg== X-Gm-Gg: ASbGncvCCMzlzsWHdf00mY+atR85PGBepjbPunnD/465QjWBCC/RpjIh9XMLdoQQxu5 P4sZ+lrXvRU2SMmKGqHXv5hAq8ldMRbue1ezejhGMn0RYOg/eR9emYjVAdxk5PFsASzMg/Idwdh SDnIO7WxVu7GnIuDymKCzYBTmg1PVpmG/MYojbaG5Flb6HUTZrFSY7Ip9LxZ+YiFVebfzayVniS ZgXhaQ3z8SS7cC3FCZ3cDLJkMdM2EZccXBzDG9cnufGIDvsytBPUDRfEWf0z3Gky+asR+1DYjLY ilM= X-Google-Smtp-Source: AGHT+IEzh2vXVXXW5B5Ed5ZTzK8fKIgHzlWWpwrWKLYvwwb30s4uKM+ShVikFKtNzNtPnQJDrTJRfg== X-Received: by 2002:a05:6402:1ed4:b0:5d0:ed71:3ce4 with SMTP id 4fb4d7f45d1cf-5d972dfea2amr2948651a12.6.1736349556914; Wed, 08 Jan 2025 07:19:16 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Julien Grall , Andrew Cooper , Anthony PERARD , Michal Orzel , Jan Beulich , Julien Grall , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Elias El Yandouzi , Alejandro Vallejo Subject: [PATCH v5 15/15] xen/arm64: Allow the admin to enable/disable the directmap Date: Wed, 8 Jan 2025 15:18:22 +0000 Message-ID: <20250108151822.16030-16-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250108151822.16030-1-alejandro.vallejo@cloud.com> References: <20250108151822.16030-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 From: Julien Grall Implement the same command line option as x86 to enable/disable the directmap. By default this is kept enabled. Also modify setup_directmap_mappings() to populate the L0 entries related to the directmap area. Signed-off-by: Julien Grall Signed-off-by: Elias El Yandouzi Signed-off-by: Alejandro Vallejo --- v4->v5: * Fixed typo in comment. s/fdirect/direct/ * Adjusted comment so 's/directmap=no/asi=true' * Adjusted printk() so 's/on/full' and 's/off/on demand' * s/HAS_SECRET_HIDING/HAS_ONDEMAND_DIRECTMAP. Otherwise CONFIG_ONDEMAND_DIRECTMAP won't appear on the menu for ARM (Note: I didn't test ARM because I have no boxes to do so) v1->v2: * Rely on the Kconfig option to enable Secret Hiding on Arm64 * Use generic helper instead of arch_has_directmap() --- docs/misc/xen-command-line.pandoc | 2 +- xen/arch/arm/Kconfig | 1 + xen/arch/arm/arm64/mmu/mm.c | 39 +++++++++++++++++++++++++++-- xen/arch/arm/include/asm/arm64/mm.h | 7 ++---- xen/arch/arm/setup.c | 2 ++ 5 files changed, 43 insertions(+), 8 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 6a1351b6c09b..68cbaf17e768 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -202,7 +202,7 @@ to appropriate auditing by Xen. Argo is disabled by default. This option is disabled by default, to protect domains from a DoS by a buggy or malicious other domain spamming the ring. -### asi (x86) +### asi (arm64, x86) > `= ` > Default: `false` diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 5c31bb616608..ec9536a1111e 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -8,6 +8,7 @@ config ARM_64 select 64BIT select HAS_FAST_MULTIPLY select HAS_LLC_COLORING if !NUMA + select HAS_ONDEMAND_DIRECTMAP config ARM def_bool y diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c index 8e121e5ffe8d..99f14ce17878 100644 --- a/xen/arch/arm/arm64/mmu/mm.c +++ b/xen/arch/arm/arm64/mmu/mm.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include @@ -14,6 +15,11 @@ #undef virt_to_mfn #define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) +#ifdef CONFIG_ONDEMAND_DIRECTMAP +bool __ro_after_init opt_ondemand_dmap; +boolean_param("asi", opt_ondemand_dmap); +#endif + static DEFINE_PAGE_TABLE(xen_first_id); static DEFINE_PAGE_TABLE(xen_second_id); static DEFINE_PAGE_TABLE(xen_third_id); @@ -204,16 +210,27 @@ void __init switch_ttbr(uint64_t ttbr) update_identity_mapping(false); } -/* Map the region in the directmap area. */ +/* + * This either populate a valid directmap, or allocates empty L1 tables + * and creates the L0 entries for the given region in the direct map + * depending on has_directmap(). + * + * When asi=true, we still need to populate empty L1 tables in the + * directmap region. The reason is that the root page-table (i.e. L0) + * is per-CPU and secondary CPUs will initialize their root page-table + * based on the pCPU0 one. So L0 entries will be shared if they are + * pre-populated. We also rely on the fact that L1 tables are never + * freed. + */ static void __init setup_directmap_mappings(unsigned long base_mfn, unsigned long nr_mfns) { + unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1); int rc; /* First call sets the directmap physical and virtual offset. */ if ( mfn_eq(directmap_mfn_start, INVALID_MFN) ) { - unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1); directmap_mfn_start = _mfn(base_mfn); directmap_base_pdx = mfn_to_pdx(_mfn(base_mfn)); @@ -234,6 +251,24 @@ static void __init setup_directmap_mappings(unsigned long base_mfn, panic("cannot add directmap mapping at %lx below heap start %lx\n", base_mfn, mfn_x(directmap_mfn_start)); + if ( !has_directmap() ) + { + vaddr_t vaddr = (vaddr_t)__mfn_to_virt(base_mfn); + lpae_t *root = this_cpu(xen_pgtable); + unsigned int i, slot; + + slot = first_table_offset(vaddr); + nr_mfns += base_mfn - mfn_gb; + for ( i = 0; i < nr_mfns; i += BIT(XEN_PT_LEVEL_ORDER(0), UL), slot++ ) + { + lpae_t *entry = &root[slot]; + + if ( !lpae_is_valid(*entry) && !create_xen_table(entry) ) + panic("Unable to populate zeroeth slot %u\n", slot); + } + return; + } + rc = map_pages_to_xen((vaddr_t)__mfn_to_virt(base_mfn), _mfn(base_mfn), nr_mfns, PAGE_HYPERVISOR_RW | _PAGE_BLOCK); diff --git a/xen/arch/arm/include/asm/arm64/mm.h b/xen/arch/arm/include/asm/arm64/mm.h index b4f7545d2c87..2b1140a6b994 100644 --- a/xen/arch/arm/include/asm/arm64/mm.h +++ b/xen/arch/arm/include/asm/arm64/mm.h @@ -3,13 +3,10 @@ extern DEFINE_PAGE_TABLE(xen_pgtable); -/* - * On ARM64, all the RAM is currently direct mapped in Xen. - * Hence return always true. - */ +/* On Arm64, the user can chose whether all the RAM is directmap. */ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) { - return true; + return has_directmap(); } void arch_setup_page_tables(void); diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 3b1ab6be3fbd..e3505dca8889 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -346,6 +346,8 @@ void asmlinkage __init start_xen(unsigned long fdt_paddr) device_tree_flattened = early_fdt_map(fdt_paddr); setup_mm(); + printk("Booting with directmap: %s\n", + has_directmap() ? "full" : "on demand"); vm_init();