From patchwork Tue Apr 6 11:05:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A489C433B4 for ; Tue, 6 Apr 2021 11:06:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1FACD61055 for ; Tue, 6 Apr 2021 11:06:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FACD61055 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105799.202356 (Exim 4.92) (envelope-from ) id 1lTjXG-0004Ag-2u; Tue, 06 Apr 2021 11:06:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105799.202356; Tue, 06 Apr 2021 11:06:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXF-0004AZ-W1; Tue, 06 Apr 2021 11:06:17 +0000 Received: by outflank-mailman (input) for mailman id 105799; Tue, 06 Apr 2021 11:06:16 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXE-0004A9-BL for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:16 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXD-00052k-KV; Tue, 06 Apr 2021 11:06:15 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXC-0002qD-JB; Tue, 06 Apr 2021 11:06:15 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=3wDZcWq9SJYWjnqWjE3dFEtmqXMHubwG3MqOcCsU4aU=; b=5 UxuyJklDF3sz1uf8PFlyA8V4VTqD3XH44FIRo4FRiRYCjyg0JjvkJSFuBAnHXj3OI3nIOzteEKAuI dJw1uuBnOeLrQmFbweQso4czFmliZdxNZJivLmu3KVHSOpEgg4zP3Ez29fdkaGv97AXMZOY3VukZR gfG3YECzKlx9smew=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini Subject: [PATCH v9 01/13] x86/mm: rewrite virt_to_xen_l*e Date: Tue, 6 Apr 2021 12:05:49 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Rewrite those functions to use the new APIs. Modify its callers to unmap the pointer returned. Since alloc_xen_pagetable_new() is almost never useful unless accompanied by page clearing and a mapping, introduce a helper alloc_map_clear_xen_pt() for this sequence. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia --- Changed in v9: - use domain_page_map_to_mfn() around the L3 table locking logic. - remove vmap_to_mfn() changes since we now use xen_map_to_mfn(). Changed in v8: - s/virtual address/linear address/. - BUG_ON() on NULL return in vmap_to_mfn(). Changed in v7: - remove a comment. - use l1e_get_mfn() instead of converting things back and forth. - add alloc_map_clear_xen_pt(). - unmap before the next mapping to reduce mapcache pressure. - use normal unmap calls instead of the macro in error paths because unmap can handle NULL now. --- xen/arch/x86/mm.c | 102 +++++++++++++++++++++++++++------------ xen/common/vmap.c | 1 + xen/include/asm-x86/mm.h | 1 + 3 files changed, 73 insertions(+), 31 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index afb4febf6f4e..9705fed195f1 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -4915,8 +4915,28 @@ void free_xen_pagetable_new(mfn_t mfn) free_xenheap_page(mfn_to_virt(mfn_x(mfn))); } +void *alloc_map_clear_xen_pt(mfn_t *pmfn) +{ + mfn_t mfn = alloc_xen_pagetable_new(); + void *ret; + + if ( mfn_eq(mfn, INVALID_MFN) ) + return NULL; + + if ( pmfn ) + *pmfn = mfn; + ret = map_domain_page(mfn); + clear_page(ret); + + return ret; +} + static DEFINE_SPINLOCK(map_pgdir_lock); +/* + * For virt_to_xen_lXe() functions, they take a linear address and return a + * pointer to Xen's LX entry. Caller needs to unmap the pointer. + */ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v) { l4_pgentry_t *pl4e; @@ -4925,33 +4945,33 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v) if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) ) { bool locking = system_state > SYS_STATE_boot; - l3_pgentry_t *l3t = alloc_xen_pagetable(); + mfn_t l3mfn; + l3_pgentry_t *l3t = alloc_map_clear_xen_pt(&l3mfn); if ( !l3t ) return NULL; - clear_page(l3t); + UNMAP_DOMAIN_PAGE(l3t); if ( locking ) spin_lock(&map_pgdir_lock); if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) ) { - l4_pgentry_t l4e = l4e_from_paddr(__pa(l3t), __PAGE_HYPERVISOR); + l4_pgentry_t l4e = l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR); l4e_write(pl4e, l4e); efi_update_l4_pgtable(l4_table_offset(v), l4e); - l3t = NULL; + l3mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); - if ( l3t ) - free_xen_pagetable(l3t); + free_xen_pagetable_new(l3mfn); } - return l4e_to_l3e(*pl4e) + l3_table_offset(v); + return map_l3t_from_l4e(*pl4e) + l3_table_offset(v); } static l2_pgentry_t *virt_to_xen_l2e(unsigned long v) { - l3_pgentry_t *pl3e; + l3_pgentry_t *pl3e, l3e; pl3e = virt_to_xen_l3e(v); if ( !pl3e ) @@ -4960,31 +4980,37 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v) if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) { bool locking = system_state > SYS_STATE_boot; - l2_pgentry_t *l2t = alloc_xen_pagetable(); + mfn_t l2mfn; + l2_pgentry_t *l2t = alloc_map_clear_xen_pt(&l2mfn); if ( !l2t ) + { + unmap_domain_page(pl3e); return NULL; - clear_page(l2t); + } + UNMAP_DOMAIN_PAGE(l2t); if ( locking ) spin_lock(&map_pgdir_lock); if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) { - l3e_write(pl3e, l3e_from_paddr(__pa(l2t), __PAGE_HYPERVISOR)); - l2t = NULL; + l3e_write(pl3e, l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR)); + l2mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); - if ( l2t ) - free_xen_pagetable(l2t); + free_xen_pagetable_new(l2mfn); } BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE); - return l3e_to_l2e(*pl3e) + l2_table_offset(v); + l3e = *pl3e; + unmap_domain_page(pl3e); + + return map_l2t_from_l3e(l3e) + l2_table_offset(v); } l1_pgentry_t *virt_to_xen_l1e(unsigned long v) { - l2_pgentry_t *pl2e; + l2_pgentry_t *pl2e, l2e; pl2e = virt_to_xen_l2e(v); if ( !pl2e ) @@ -4993,26 +5019,32 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v) if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { bool locking = system_state > SYS_STATE_boot; - l1_pgentry_t *l1t = alloc_xen_pagetable(); + mfn_t l1mfn; + l1_pgentry_t *l1t = alloc_map_clear_xen_pt(&l1mfn); if ( !l1t ) + { + unmap_domain_page(pl2e); return NULL; - clear_page(l1t); + } + UNMAP_DOMAIN_PAGE(l1t); if ( locking ) spin_lock(&map_pgdir_lock); if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { - l2e_write(pl2e, l2e_from_paddr(__pa(l1t), __PAGE_HYPERVISOR)); - l1t = NULL; + l2e_write(pl2e, l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR)); + l1mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); - if ( l1t ) - free_xen_pagetable(l1t); + free_xen_pagetable_new(l1mfn); } BUG_ON(l2e_get_flags(*pl2e) & _PAGE_PSE); - return l2e_to_l1e(*pl2e) + l1_table_offset(v); + l2e = *pl2e; + unmap_domain_page(pl2e); + + return map_l1t_from_l2e(l2e) + l1_table_offset(v); } /* Convert to from superpage-mapping flags for map_pages_to_xen(). */ @@ -5069,7 +5101,7 @@ mfn_t xen_map_to_mfn(unsigned long va) L3T_INIT(l3page); CHECK_MAPPED(pl3e); - l3page = virt_to_page(pl3e); + l3page = mfn_to_page(domain_page_map_to_mfn(pl3e)); L3T_LOCK(l3page); CHECK_MAPPED(l3e_get_flags(*pl3e) & _PAGE_PRESENT); @@ -5108,7 +5140,8 @@ int map_pages_to_xen( unsigned int flags) { bool locking = system_state > SYS_STATE_boot; - l2_pgentry_t *pl2e, ol2e; + l3_pgentry_t *pl3e = NULL, ol3e; + l2_pgentry_t *pl2e = NULL, ol2e; l1_pgentry_t *pl1e, ol1e; unsigned int i; int rc = -ENOMEM; @@ -5132,15 +5165,16 @@ int map_pages_to_xen( while ( nr_mfns != 0 ) { - l3_pgentry_t *pl3e, ol3e; - + /* Clean up the previous iteration. */ L3T_UNLOCK(current_l3page); + UNMAP_DOMAIN_PAGE(pl3e); + UNMAP_DOMAIN_PAGE(pl2e); pl3e = virt_to_xen_l3e(virt); if ( !pl3e ) goto out; - current_l3page = virt_to_page(pl3e); + current_l3page = mfn_to_page(domain_page_map_to_mfn(pl3e)); L3T_LOCK(current_l3page); ol3e = *pl3e; @@ -5305,6 +5339,8 @@ int map_pages_to_xen( pl1e = virt_to_xen_l1e(virt); if ( pl1e == NULL ) goto out; + + UNMAP_DOMAIN_PAGE(pl1e); } else if ( l2e_get_flags(*pl2e) & _PAGE_PSE ) { @@ -5482,6 +5518,7 @@ int map_pages_to_xen( out: L3T_UNLOCK(current_l3page); + unmap_domain_page(pl3e); return rc; } @@ -5505,6 +5542,7 @@ int populate_pt_range(unsigned long virt, unsigned long nr_mfns) int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) { bool locking = system_state > SYS_STATE_boot; + l3_pgentry_t *pl3e = NULL; l2_pgentry_t *pl2e; l1_pgentry_t *pl1e; unsigned int i; @@ -5523,15 +5561,15 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) while ( v < e ) { - l3_pgentry_t *pl3e; - + /* Clean up the previous iteration. */ L3T_UNLOCK(current_l3page); + UNMAP_DOMAIN_PAGE(pl3e); pl3e = virt_to_xen_l3e(v); if ( !pl3e ) goto out; - current_l3page = virt_to_page(pl3e); + current_l3page = mfn_to_page(domain_page_map_to_mfn(pl3e)); L3T_LOCK(current_l3page); if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) @@ -5761,6 +5799,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) out: L3T_UNLOCK(current_l3page); + unmap_domain_page(pl2e); + unmap_domain_page(pl3e); return rc; } diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 4fd6b3067ec1..c5140ac4f6a9 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -1,6 +1,7 @@ #ifdef VMAP_VIRT_START #include #include +#include #include #include #include diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 041c158f03f6..681aac5b7ac2 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -577,6 +577,7 @@ void *alloc_xen_pagetable(void); void free_xen_pagetable(void *v); mfn_t alloc_xen_pagetable_new(void); void free_xen_pagetable_new(mfn_t mfn); +void *alloc_map_clear_xen_pt(mfn_t *pmfn); l1_pgentry_t *virt_to_xen_l1e(unsigned long v); From patchwork Tue Apr 6 11:05:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 729E9C43470 for ; Tue, 6 Apr 2021 11:06:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 370FF61055 for ; Tue, 6 Apr 2021 11:06:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 370FF61055 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105800.202368 (Exim 4.92) (envelope-from ) id 1lTjXH-0004Cf-JR; Tue, 06 Apr 2021 11:06:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105800.202368; Tue, 06 Apr 2021 11:06:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXH-0004CU-EN; Tue, 06 Apr 2021 11:06:19 +0000 Received: by outflank-mailman (input) for mailman id 105800; Tue, 06 Apr 2021 11:06:18 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXG-0004Bv-Jw for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:18 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXG-00052s-6l; Tue, 06 Apr 2021 11:06:18 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXF-0002qD-Bi; Tue, 06 Apr 2021 11:06:18 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=/MFVGtrkdYKM0RBkkP2MlsOy4fUYar3Tz/PcBBeLT3s=; b=m prUc60fPcyPWGdBWk0CGC1KuqIhsOR1mgmaXY+1wcc4F972G/mod4zJTN7lnH5uZruOKbSwgfO8lh x5EdYy2XFTdYvl7k4NtaCtqKnvF2ZNjkgFzYSvnjh5J1IfIMSb1cyYsYnqWXQOoPNCIlaB4Wdg3Sm FSjNe6G4ykbdEUdU=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 02/13] x86/mm: switch to new APIs in map_pages_to_xen Date: Tue, 6 Apr 2021 12:05:50 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Page tables allocated in that function should be mapped and unmapped now. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- xen/arch/x86/mm.c | 60 ++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 24 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 9705fed195f1..c49e8554f9f7 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5199,7 +5199,7 @@ int map_pages_to_xen( } else { - l2_pgentry_t *l2t = l3e_to_l2e(ol3e); + l2_pgentry_t *l2t = map_l2t_from_l3e(ol3e); for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) { @@ -5211,10 +5211,11 @@ int map_pages_to_xen( else { unsigned int j; - const l1_pgentry_t *l1t = l2e_to_l1e(ol2e); + const l1_pgentry_t *l1t = map_l1t_from_l2e(ol2e); for ( j = 0; j < L1_PAGETABLE_ENTRIES; j++ ) flush_flags(l1e_get_flags(l1t[j])); + unmap_domain_page(l1t); } } flush_area(virt, flush_flags); @@ -5223,9 +5224,10 @@ int map_pages_to_xen( ol2e = l2t[i]; if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) && !(l2e_get_flags(ol2e) & _PAGE_PSE) ) - free_xen_pagetable(l2e_to_l1e(ol2e)); + free_xen_pagetable_new(l2e_get_mfn(ol2e)); } - free_xen_pagetable(l2t); + unmap_domain_page(l2t); + free_xen_pagetable_new(l3e_get_mfn(ol3e)); } } @@ -5242,6 +5244,7 @@ int map_pages_to_xen( unsigned int flush_flags = FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER); l2_pgentry_t *l2t; + mfn_t l2mfn; /* Skip this PTE if there is no change. */ if ( ((l3e_get_pfn(ol3e) & ~(L2_PAGETABLE_ENTRIES * @@ -5263,15 +5266,17 @@ int map_pages_to_xen( continue; } - l2t = alloc_xen_pagetable(); - if ( l2t == NULL ) + l2mfn = alloc_xen_pagetable_new(); + if ( mfn_eq(l2mfn, INVALID_MFN) ) goto out; + l2t = map_domain_page(l2mfn); for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) l2e_write(l2t + i, l2e_from_pfn(l3e_get_pfn(ol3e) + (i << PAGETABLE_ORDER), l3e_get_flags(ol3e))); + UNMAP_DOMAIN_PAGE(l2t); if ( l3e_get_flags(ol3e) & _PAGE_GLOBAL ) flush_flags |= FLUSH_TLB_GLOBAL; @@ -5281,15 +5286,15 @@ int map_pages_to_xen( if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && (l3e_get_flags(*pl3e) & _PAGE_PSE) ) { - l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t), - __PAGE_HYPERVISOR)); - l2t = NULL; + l3e_write_atomic(pl3e, + l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR)); + l2mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(virt, flush_flags); - if ( l2t ) - free_xen_pagetable(l2t); + + free_xen_pagetable_new(l2mfn); } pl2e = virt_to_xen_l2e(virt); @@ -5317,12 +5322,13 @@ int map_pages_to_xen( } else { - l1_pgentry_t *l1t = l2e_to_l1e(ol2e); + l1_pgentry_t *l1t = map_l1t_from_l2e(ol2e); for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) flush_flags(l1e_get_flags(l1t[i])); flush_area(virt, flush_flags); - free_xen_pagetable(l1t); + unmap_domain_page(l1t); + free_xen_pagetable_new(l2e_get_mfn(ol2e)); } } @@ -5347,6 +5353,7 @@ int map_pages_to_xen( unsigned int flush_flags = FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER); l1_pgentry_t *l1t; + mfn_t l1mfn; /* Skip this PTE if there is no change. */ if ( (((l2e_get_pfn(*pl2e) & ~(L1_PAGETABLE_ENTRIES - 1)) + @@ -5366,14 +5373,16 @@ int map_pages_to_xen( goto check_l3; } - l1t = alloc_xen_pagetable(); - if ( l1t == NULL ) + l1mfn = alloc_xen_pagetable_new(); + if ( mfn_eq(l1mfn, INVALID_MFN) ) goto out; + l1t = map_domain_page(l1mfn); for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) l1e_write(&l1t[i], l1e_from_pfn(l2e_get_pfn(*pl2e) + i, lNf_to_l1f(l2e_get_flags(*pl2e)))); + UNMAP_DOMAIN_PAGE(l1t); if ( l2e_get_flags(*pl2e) & _PAGE_GLOBAL ) flush_flags |= FLUSH_TLB_GLOBAL; @@ -5383,20 +5392,21 @@ int map_pages_to_xen( if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && (l2e_get_flags(*pl2e) & _PAGE_PSE) ) { - l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(l1t), + l2e_write_atomic(pl2e, l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR)); - l1t = NULL; + l1mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(virt, flush_flags); - if ( l1t ) - free_xen_pagetable(l1t); + + free_xen_pagetable_new(l1mfn); } - pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(virt); + pl1e = map_l1t_from_l2e(*pl2e) + l1_table_offset(virt); ol1e = *pl1e; l1e_write_atomic(pl1e, l1e_from_mfn(mfn, flags)); + UNMAP_DOMAIN_PAGE(pl1e); if ( (l1e_get_flags(ol1e) & _PAGE_PRESENT) ) { unsigned int flush_flags = FLUSH_TLB | FLUSH_ORDER(0); @@ -5440,12 +5450,13 @@ int map_pages_to_xen( goto check_l3; } - l1t = l2e_to_l1e(ol2e); + l1t = map_l1t_from_l2e(ol2e); base_mfn = l1e_get_pfn(l1t[0]) & ~(L1_PAGETABLE_ENTRIES - 1); for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) if ( (l1e_get_pfn(l1t[i]) != (base_mfn + i)) || (l1e_get_flags(l1t[i]) != flags) ) break; + UNMAP_DOMAIN_PAGE(l1t); if ( i == L1_PAGETABLE_ENTRIES ) { l2e_write_atomic(pl2e, l2e_from_pfn(base_mfn, @@ -5455,7 +5466,7 @@ int map_pages_to_xen( flush_area(virt - PAGE_SIZE, FLUSH_TLB_GLOBAL | FLUSH_ORDER(PAGETABLE_ORDER)); - free_xen_pagetable(l2e_to_l1e(ol2e)); + free_xen_pagetable_new(l2e_get_mfn(ol2e)); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5488,7 +5499,7 @@ int map_pages_to_xen( continue; } - l2t = l3e_to_l2e(ol3e); + l2t = map_l2t_from_l3e(ol3e); base_mfn = l2e_get_pfn(l2t[0]) & ~(L2_PAGETABLE_ENTRIES * L1_PAGETABLE_ENTRIES - 1); for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) @@ -5496,6 +5507,7 @@ int map_pages_to_xen( (base_mfn + (i << PAGETABLE_ORDER))) || (l2e_get_flags(l2t[i]) != l1f_to_lNf(flags)) ) break; + UNMAP_DOMAIN_PAGE(l2t); if ( i == L2_PAGETABLE_ENTRIES ) { l3e_write_atomic(pl3e, l3e_from_pfn(base_mfn, @@ -5505,7 +5517,7 @@ int map_pages_to_xen( flush_area(virt - PAGE_SIZE, FLUSH_TLB_GLOBAL | FLUSH_ORDER(2*PAGETABLE_ORDER)); - free_xen_pagetable(l3e_to_l2e(ol3e)); + free_xen_pagetable_new(l3e_get_mfn(ol3e)); } else if ( locking ) spin_unlock(&map_pgdir_lock); From patchwork Tue Apr 6 11:05:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F7D0C43460 for ; Tue, 6 Apr 2021 11:06:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DE51B613C7 for ; Tue, 6 Apr 2021 11:06:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE51B613C7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105801.202379 (Exim 4.92) (envelope-from ) id 1lTjXJ-0004Fh-TJ; Tue, 06 Apr 2021 11:06:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105801.202379; Tue, 06 Apr 2021 11:06:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXJ-0004Fa-P3; Tue, 06 Apr 2021 11:06:21 +0000 Received: by outflank-mailman (input) for mailman id 105801; Tue, 06 Apr 2021 11:06:20 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXI-0004ES-Lx for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:20 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXI-000530-Dl; Tue, 06 Apr 2021 11:06:20 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXI-0002qD-0j; Tue, 06 Apr 2021 11:06:20 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=e8oDiNP5pPlJ0YBPEORyPfE5UKBo6Kl37Wzeoi7OFlc=; b=4 zOw9QBWIZ5jZZKSSqPbTPJabxVY7pIhoFudwK+X782axj8X/1zKvoLHC+IEpmyqpBSjZ6a22NNNw0 QQCrIsad3bFSgukiNkZdhn9SbbJ90nDKLVWyZyof53dAc8Hzcc0+tF9w9VHkzWbDpOLYfJ9PhzqIL Pzb5jU5BTc7o+jAI=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 03/13] x86/mm: switch to new APIs in modify_xen_mappings Date: Tue, 6 Apr 2021 12:05:51 +0100 Message-Id: <4d45d83f7c4578647180baa0e7850ec3fa37de9e.1617706782.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Page tables allocated in that function should be mapped and unmapped now. Note that pl2e now maybe mapped and unmapped in different iterations, so we need to add clean-ups for that. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v7: - use normal unmap in the error path. --- xen/arch/x86/mm.c | 57 ++++++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index c49e8554f9f7..ababfffb3afc 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5530,6 +5530,7 @@ int map_pages_to_xen( out: L3T_UNLOCK(current_l3page); + unmap_domain_page(pl2e); unmap_domain_page(pl3e); return rc; } @@ -5555,7 +5556,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) { bool locking = system_state > SYS_STATE_boot; l3_pgentry_t *pl3e = NULL; - l2_pgentry_t *pl2e; + l2_pgentry_t *pl2e = NULL; l1_pgentry_t *pl1e; unsigned int i; unsigned long v = s; @@ -5575,6 +5576,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) { /* Clean up the previous iteration. */ L3T_UNLOCK(current_l3page); + UNMAP_DOMAIN_PAGE(pl2e); UNMAP_DOMAIN_PAGE(pl3e); pl3e = virt_to_xen_l3e(v); @@ -5597,6 +5599,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( l3e_get_flags(*pl3e) & _PAGE_PSE ) { l2_pgentry_t *l2t; + mfn_t l2mfn; if ( l2_table_offset(v) == 0 && l1_table_offset(v) == 0 && @@ -5613,35 +5616,38 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) } /* PAGE1GB: shatter the superpage and fall through. */ - l2t = alloc_xen_pagetable(); - if ( !l2t ) + l2mfn = alloc_xen_pagetable_new(); + if ( mfn_eq(l2mfn, INVALID_MFN) ) goto out; + l2t = map_domain_page(l2mfn); for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) l2e_write(l2t + i, l2e_from_pfn(l3e_get_pfn(*pl3e) + (i << PAGETABLE_ORDER), l3e_get_flags(*pl3e))); + UNMAP_DOMAIN_PAGE(l2t); + if ( locking ) spin_lock(&map_pgdir_lock); if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && (l3e_get_flags(*pl3e) & _PAGE_PSE) ) { - l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t), - __PAGE_HYPERVISOR)); - l2t = NULL; + l3e_write_atomic(pl3e, + l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR)); + l2mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); - if ( l2t ) - free_xen_pagetable(l2t); + + free_xen_pagetable_new(l2mfn); } /* * The L3 entry has been verified to be present, and we've dealt with * 1G pages as well, so the L2 table cannot require allocation. */ - pl2e = l3e_to_l2e(*pl3e) + l2_table_offset(v); + pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(v); if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { @@ -5669,41 +5675,45 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) else { l1_pgentry_t *l1t; - /* PSE: shatter the superpage and try again. */ - l1t = alloc_xen_pagetable(); - if ( !l1t ) + mfn_t l1mfn = alloc_xen_pagetable_new(); + + if ( mfn_eq(l1mfn, INVALID_MFN) ) goto out; + l1t = map_domain_page(l1mfn); for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) l1e_write(&l1t[i], l1e_from_pfn(l2e_get_pfn(*pl2e) + i, l2e_get_flags(*pl2e) & ~_PAGE_PSE)); + UNMAP_DOMAIN_PAGE(l1t); + if ( locking ) spin_lock(&map_pgdir_lock); if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && (l2e_get_flags(*pl2e) & _PAGE_PSE) ) { - l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(l1t), + l2e_write_atomic(pl2e, l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR)); - l1t = NULL; + l1mfn = INVALID_MFN; } if ( locking ) spin_unlock(&map_pgdir_lock); - if ( l1t ) - free_xen_pagetable(l1t); + + free_xen_pagetable_new(l1mfn); } } else { l1_pgentry_t nl1e, *l1t; + mfn_t l1mfn; /* * Ordinary 4kB mapping: The L2 entry has been verified to be * present, and we've dealt with 2M pages as well, so the L1 table * cannot require allocation. */ - pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(v); + pl1e = map_l1t_from_l2e(*pl2e) + l1_table_offset(v); /* Confirm the caller isn't trying to create new mappings. */ if ( !(l1e_get_flags(*pl1e) & _PAGE_PRESENT) ) @@ -5714,6 +5724,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) (l1e_get_flags(*pl1e) & ~FLAGS_MASK) | nf); l1e_write_atomic(pl1e, nl1e); + UNMAP_DOMAIN_PAGE(pl1e); v += PAGE_SIZE; /* @@ -5743,10 +5754,12 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) continue; } - l1t = l2e_to_l1e(*pl2e); + l1mfn = l2e_get_mfn(*pl2e); + l1t = map_domain_page(l1mfn); for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) if ( l1e_get_intpte(l1t[i]) != 0 ) break; + UNMAP_DOMAIN_PAGE(l1t); if ( i == L1_PAGETABLE_ENTRIES ) { /* Empty: zap the L2E and free the L1 page. */ @@ -5754,7 +5767,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(NULL, FLUSH_TLB_GLOBAL); /* flush before free */ - free_xen_pagetable(l1t); + free_xen_pagetable_new(l1mfn); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5785,11 +5798,13 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) { l2_pgentry_t *l2t; + mfn_t l2mfn = l3e_get_mfn(*pl3e); - l2t = l3e_to_l2e(*pl3e); + l2t = map_domain_page(l2mfn); for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) if ( l2e_get_intpte(l2t[i]) != 0 ) break; + UNMAP_DOMAIN_PAGE(l2t); if ( i == L2_PAGETABLE_ENTRIES ) { /* Empty: zap the L3E and free the L2 page. */ @@ -5797,7 +5812,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(NULL, FLUSH_TLB_GLOBAL); /* flush before free */ - free_xen_pagetable(l2t); + free_xen_pagetable_new(l2mfn); } else if ( locking ) spin_unlock(&map_pgdir_lock); From patchwork Tue Apr 6 11:05:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59FFBC433ED for ; Tue, 6 Apr 2021 11:06:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1FCF9613C7 for ; Tue, 6 Apr 2021 11:06:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FCF9613C7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105802.202392 (Exim 4.92) (envelope-from ) id 1lTjXM-0004JB-7B; Tue, 06 Apr 2021 11:06:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105802.202392; Tue, 06 Apr 2021 11:06:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXM-0004J2-2L; Tue, 06 Apr 2021 11:06:24 +0000 Received: by outflank-mailman (input) for mailman id 105802; Tue, 06 Apr 2021 11:06:22 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXK-0004Hr-ST for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:22 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXK-00053A-KC; Tue, 06 Apr 2021 11:06:22 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXK-0002qD-2L; Tue, 06 Apr 2021 11:06:22 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=s1xYw5JRA0hX+hVyYpDhdvIQg52C43B8RF1tdty6+gI=; b=q rvO47LJf+Qx69S/KtsguvWew2ZpWZu6weBHn63fYk6YFjXcT+Ge+BULkx4s/rtdrm2wSly1NafdKP J79mwlq8crEPjt9GOzE4fq8gmJxmwhtonCmBaNt9Z6rnzC5p6pQp/f6IjABwCySb+QiO56TlbrfzV qJfq8CmpFC2ba8M8=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 04/13] x86_64/mm: introduce pl2e in paging_init Date: Tue, 6 Apr 2021 12:05:52 +0100 Message-Id: <836cc0def0609a5f4495491b1c4994fc044cfbef.1617706782.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu We will soon map and unmap pages in paging_init(). Introduce pl2e so that we can use l2_ro_mpt to point to the page table itself. No functional change. Signed-off-by: Wei Liu Acked-by: Jan Beulich --- Changed in v7: - reword commit message. --- xen/arch/x86/x86_64/mm.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index bce1561e1a80..c5a47df01bde 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -496,7 +496,7 @@ void __init paging_init(void) unsigned long i, mpt_size, va; unsigned int n, memflags; l3_pgentry_t *l3_ro_mpt; - l2_pgentry_t *l2_ro_mpt = NULL; + l2_pgentry_t *pl2e = NULL, *l2_ro_mpt = NULL; struct page_info *l1_pg; /* @@ -546,7 +546,7 @@ void __init paging_init(void) (L2_PAGETABLE_SHIFT - 3 + PAGE_SHIFT))); if ( cpu_has_page1gb && - !((unsigned long)l2_ro_mpt & ~PAGE_MASK) && + !((unsigned long)pl2e & ~PAGE_MASK) && (mpt_size >> L3_PAGETABLE_SHIFT) > (i >> PAGETABLE_ORDER) ) { unsigned int k, holes; @@ -606,7 +606,7 @@ void __init paging_init(void) memset((void *)(RDWR_MPT_VIRT_START + (i << L2_PAGETABLE_SHIFT)), 0xFF, 1UL << L2_PAGETABLE_SHIFT); } - if ( !((unsigned long)l2_ro_mpt & ~PAGE_MASK) ) + if ( !((unsigned long)pl2e & ~PAGE_MASK) ) { if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL ) goto nomem; @@ -614,13 +614,14 @@ void __init paging_init(void) l3e_write(&l3_ro_mpt[l3_table_offset(va)], l3e_from_paddr(__pa(l2_ro_mpt), __PAGE_HYPERVISOR_RO | _PAGE_USER)); + pl2e = l2_ro_mpt; ASSERT(!l2_table_offset(va)); } /* NB. Cannot be GLOBAL: guest user mode should not see it. */ if ( l1_pg ) - l2e_write(l2_ro_mpt, l2e_from_page( + l2e_write(pl2e, l2e_from_page( l1_pg, /*_PAGE_GLOBAL|*/_PAGE_PSE|_PAGE_USER|_PAGE_PRESENT)); - l2_ro_mpt++; + pl2e++; } #undef CNT #undef MFN @@ -632,6 +633,7 @@ void __init paging_init(void) goto nomem; compat_idle_pg_table_l2 = l2_ro_mpt; clear_page(l2_ro_mpt); + pl2e = l2_ro_mpt; /* Allocate and map the compatibility mode machine-to-phys table. */ mpt_size = (mpt_size >> 1) + (1UL << (L2_PAGETABLE_SHIFT - 1)); @@ -649,7 +651,7 @@ void __init paging_init(void) sizeof(*compat_machine_to_phys_mapping)) BUILD_BUG_ON((sizeof(*frame_table) & ~sizeof(*frame_table)) % \ sizeof(*compat_machine_to_phys_mapping)); - for ( i = 0; i < (mpt_size >> L2_PAGETABLE_SHIFT); i++, l2_ro_mpt++ ) + for ( i = 0; i < (mpt_size >> L2_PAGETABLE_SHIFT); i++, pl2e++ ) { memflags = MEMF_node(phys_to_nid(i << (L2_PAGETABLE_SHIFT - 2 + PAGE_SHIFT))); @@ -671,7 +673,7 @@ void __init paging_init(void) (i << L2_PAGETABLE_SHIFT)), 0xFF, 1UL << L2_PAGETABLE_SHIFT); /* NB. Cannot be GLOBAL as the ptes get copied into per-VM space. */ - l2e_write(l2_ro_mpt, l2e_from_page(l1_pg, _PAGE_PSE|_PAGE_PRESENT)); + l2e_write(pl2e, l2e_from_page(l1_pg, _PAGE_PSE|_PAGE_PRESENT)); } #undef CNT #undef MFN From patchwork Tue Apr 6 11:05:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E38CC43461 for ; Tue, 6 Apr 2021 11:06:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0FFF61055 for ; Tue, 6 Apr 2021 11:06:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0FFF61055 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105803.202404 (Exim 4.92) (envelope-from ) id 1lTjXO-0004Mo-HO; Tue, 06 Apr 2021 11:06:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105803.202404; Tue, 06 Apr 2021 11:06:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXO-0004Ma-CS; Tue, 06 Apr 2021 11:06:26 +0000 Received: by outflank-mailman (input) for mailman id 105803; Tue, 06 Apr 2021 11:06:25 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXN-0004LF-1q for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:25 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXM-00053H-Sy; Tue, 06 Apr 2021 11:06:24 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXM-0002qD-DN; Tue, 06 Apr 2021 11:06:24 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=DhjC1F6SkPBikln4izzyHzb4RHHtmb3wCYV7VN5lv1s=; b=L oYPM4nf88I0nCVJj3cm+4fIRf+TGjHZNssBs4sXOnAfcyRegK8KSave6kScWHyYtRwb1ZWi7seoIy kRtATa7odCnpyvQDSvy9dvf3Ijv47kh8ih2u1l40KTanMGUmB+AQhPnuwrBzvpF1QOaH6qh8W8/oI yDJcV7DTeA3N9wZ0=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 05/13] x86_64/mm: switch to new APIs in paging_init Date: Tue, 6 Apr 2021 12:05:53 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Map and unmap pages instead of relying on the direct map. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v9: - remove an unnecessary l3mfn variable. Changed in v8: - replace l3/2_ro_mpt_mfn with just mfn since their lifetimes do not overlap Changed in v7: - use the new alloc_map_clear_xen_pt() helper. - move the unmap of pl3t up a bit. - remove the unmaps in the nomem path. --- xen/arch/x86/x86_64/mm.c | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index c5a47df01bde..da239f097703 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -498,6 +498,7 @@ void __init paging_init(void) l3_pgentry_t *l3_ro_mpt; l2_pgentry_t *pl2e = NULL, *l2_ro_mpt = NULL; struct page_info *l1_pg; + mfn_t mfn; /* * We setup the L3s for 1:1 mapping if host support memory hotplug @@ -510,22 +511,22 @@ void __init paging_init(void) if ( !(l4e_get_flags(idle_pg_table[l4_table_offset(va)]) & _PAGE_PRESENT) ) { - l3_pgentry_t *pl3t = alloc_xen_pagetable(); + l3_pgentry_t *pl3t = alloc_map_clear_xen_pt(&mfn); if ( !pl3t ) goto nomem; - clear_page(pl3t); + UNMAP_DOMAIN_PAGE(pl3t); l4e_write(&idle_pg_table[l4_table_offset(va)], - l4e_from_paddr(__pa(pl3t), __PAGE_HYPERVISOR_RW)); + l4e_from_mfn(mfn, __PAGE_HYPERVISOR_RW)); } } /* Create user-accessible L2 directory to map the MPT for guests. */ - if ( (l3_ro_mpt = alloc_xen_pagetable()) == NULL ) + l3_ro_mpt = alloc_map_clear_xen_pt(&mfn); + if ( !l3_ro_mpt ) goto nomem; - clear_page(l3_ro_mpt); l4e_write(&idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)], - l4e_from_paddr(__pa(l3_ro_mpt), __PAGE_HYPERVISOR_RO | _PAGE_USER)); + l4e_from_mfn(mfn, __PAGE_HYPERVISOR_RO | _PAGE_USER)); /* * Allocate and map the machine-to-phys table. @@ -608,12 +609,14 @@ void __init paging_init(void) } if ( !((unsigned long)pl2e & ~PAGE_MASK) ) { - if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL ) + UNMAP_DOMAIN_PAGE(l2_ro_mpt); + + l2_ro_mpt = alloc_map_clear_xen_pt(&mfn); + if ( !l2_ro_mpt ) goto nomem; - clear_page(l2_ro_mpt); + l3e_write(&l3_ro_mpt[l3_table_offset(va)], - l3e_from_paddr(__pa(l2_ro_mpt), - __PAGE_HYPERVISOR_RO | _PAGE_USER)); + l3e_from_mfn(mfn, __PAGE_HYPERVISOR_RO | _PAGE_USER)); pl2e = l2_ro_mpt; ASSERT(!l2_table_offset(va)); } @@ -625,15 +628,18 @@ void __init paging_init(void) } #undef CNT #undef MFN + UNMAP_DOMAIN_PAGE(l2_ro_mpt); + UNMAP_DOMAIN_PAGE(l3_ro_mpt); /* Create user-accessible L2 directory to map the MPT for compat guests. */ if ( opt_pv32 ) { - if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL ) + mfn = alloc_xen_pagetable_new(); + if ( mfn_eq(mfn, INVALID_MFN) ) goto nomem; - compat_idle_pg_table_l2 = l2_ro_mpt; - clear_page(l2_ro_mpt); - pl2e = l2_ro_mpt; + compat_idle_pg_table_l2 = map_domain_page_global(mfn); + clear_page(compat_idle_pg_table_l2); + pl2e = compat_idle_pg_table_l2; /* Allocate and map the compatibility mode machine-to-phys table. */ mpt_size = (mpt_size >> 1) + (1UL << (L2_PAGETABLE_SHIFT - 1)); From patchwork Tue Apr 6 11:05:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BB0FC433B4 for ; Tue, 6 Apr 2021 11:06:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E22A261055 for ; Tue, 6 Apr 2021 11:06:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E22A261055 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105804.202416 (Exim 4.92) (envelope-from ) id 1lTjXR-0004Rt-5d; Tue, 06 Apr 2021 11:06:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105804.202416; Tue, 06 Apr 2021 11:06:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXQ-0004Rc-Un; Tue, 06 Apr 2021 11:06:28 +0000 Received: by outflank-mailman (input) for mailman id 105804; Tue, 06 Apr 2021 11:06:27 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXP-0004PN-Bj for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:27 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXP-00053O-5g; Tue, 06 Apr 2021 11:06:27 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXO-0002qD-Oe; Tue, 06 Apr 2021 11:06:27 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=sENfbFuCu1+L3yhspWdtG93S3uUvA4Qf23dnlgpUlWw=; b=g 6/uFwrP0ztAxMAycJ3CwgsiAbXrnNZIGYO1Ks0rZSQPbqqK+pIA3CBbFDGdyu+B3BzMPK1ZGVl4fg a/oSn/Xi7pVhWL5i9yi1BDuXHVds0AIrw0j2Ds3eKEjQjezcYTVSoWyxJYAm3Y6CMv3sUM3CpRdd4 Yef6mOmhATHukP1c=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 06/13] x86_64/mm: switch to new APIs in setup_m2p_table Date: Tue, 6 Apr 2021 12:05:54 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu While doing so, avoid repetitive mapping of l2_ro_mpt by keeping it across loops, and only unmap and map it when crossing 1G boundaries. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v8: - re-structure if condition around l2_ro_mpt. - reword the commit message. Changed in v7: - avoid repetitive mapping of l2_ro_mpt. - edit commit message. - switch to alloc_map_clear_xen_pt(). --- xen/arch/x86/x86_64/mm.c | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index da239f097703..442f345b2a54 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -402,7 +402,8 @@ static int setup_m2p_table(struct mem_hotadd_info *info) ASSERT(l4e_get_flags(idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]) & _PAGE_PRESENT); - l3_ro_mpt = l4e_to_l3e(idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]); + l3_ro_mpt = map_l3t_from_l4e( + idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]); smap = (info->spfn & (~((1UL << (L2_PAGETABLE_SHIFT - 3)) -1))); emap = ((info->epfn + ((1UL << (L2_PAGETABLE_SHIFT - 3)) - 1 )) & @@ -420,6 +421,10 @@ static int setup_m2p_table(struct mem_hotadd_info *info) i = smap; while ( i < emap ) { + if ( (RO_MPT_VIRT_START + i * sizeof(*machine_to_phys_mapping)) & + ((1UL << L3_PAGETABLE_SHIFT) - 1) ) + UNMAP_DOMAIN_PAGE(l2_ro_mpt); + switch ( m2p_mapped(i) ) { case M2P_1G_MAPPED: @@ -455,32 +460,31 @@ static int setup_m2p_table(struct mem_hotadd_info *info) ASSERT(!(l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) & _PAGE_PSE)); - if ( l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) & - _PAGE_PRESENT ) - l2_ro_mpt = l3e_to_l2e(l3_ro_mpt[l3_table_offset(va)]) + - l2_table_offset(va); + if ( l2_ro_mpt ) + /* nothing */; + else if ( l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) & + _PAGE_PRESENT ) + l2_ro_mpt = map_l2t_from_l3e(l3_ro_mpt[l3_table_offset(va)]); else { - l2_ro_mpt = alloc_xen_pagetable(); + mfn_t l2_ro_mpt_mfn; + + l2_ro_mpt = alloc_map_clear_xen_pt(&l2_ro_mpt_mfn); if ( !l2_ro_mpt ) { ret = -ENOMEM; goto error; } - clear_page(l2_ro_mpt); l3e_write(&l3_ro_mpt[l3_table_offset(va)], - l3e_from_paddr(__pa(l2_ro_mpt), - __PAGE_HYPERVISOR_RO | _PAGE_USER)); - l2_ro_mpt += l2_table_offset(va); + l3e_from_mfn(l2_ro_mpt_mfn, + __PAGE_HYPERVISOR_RO | _PAGE_USER)); } /* NB. Cannot be GLOBAL: guest user mode should not see it. */ - l2e_write(l2_ro_mpt, l2e_from_mfn(mfn, + l2e_write(&l2_ro_mpt[l2_table_offset(va)], l2e_from_mfn(mfn, /*_PAGE_GLOBAL|*/_PAGE_PSE|_PAGE_USER|_PAGE_PRESENT)); } - if ( !((unsigned long)l2_ro_mpt & ~PAGE_MASK) ) - l2_ro_mpt = NULL; i += ( 1UL << (L2_PAGETABLE_SHIFT - 3)); } #undef CNT @@ -488,6 +492,8 @@ static int setup_m2p_table(struct mem_hotadd_info *info) ret = setup_compat_m2p_table(info); error: + unmap_domain_page(l2_ro_mpt); + unmap_domain_page(l3_ro_mpt); return ret; } From patchwork Tue Apr 6 11:05:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 981D4C433ED for ; Tue, 6 Apr 2021 11:06:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5908F61055 for ; Tue, 6 Apr 2021 11:06:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5908F61055 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105805.202428 (Exim 4.92) (envelope-from ) id 1lTjXS-0004Uz-F3; Tue, 06 Apr 2021 11:06:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105805.202428; Tue, 06 Apr 2021 11:06:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXS-0004Ul-9u; Tue, 06 Apr 2021 11:06:30 +0000 Received: by outflank-mailman (input) for mailman id 105805; Tue, 06 Apr 2021 11:06:28 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXQ-0004RG-LP for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:28 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXQ-00053V-FM; Tue, 06 Apr 2021 11:06:28 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXQ-0002qD-35; Tue, 06 Apr 2021 11:06:28 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=LFpwvb91UZUoU14PzilixiyXhwhbK+Ssko2aEV8OspU=; b=d tCkrh5Em0heROtY5F1o8msjHHMg4VJGceKwrKAfMdczs1c89z/qDBPzgak9d8od/844cCrLIfAx6C yGHve5ntFf3TfLz2dVb8jxEjowz0XkqOTXHh3rSPHzH5vzMV+ywXpvmBr2652xRGQB9gOdFd62I9M FSbtyBrcttub7geE=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich Subject: [PATCH v9 07/13] efi: use new page table APIs in copy_mapping Date: Tue, 6 Apr 2021 12:05:55 +0100 Message-Id: <7acd46b563df7c684342a467de1b703be1366a74.1617706782.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v8: - remove redundant commit message. - unmap l3src based on va instead of mfn. - re-structure if condition around l3dst. Changed in v7: - hoist l3 variables out of the loop to avoid repetitive mappings. --- xen/common/efi/boot.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c index 63e289ab8506..64b319d0013b 100644 --- a/xen/common/efi/boot.c +++ b/xen/common/efi/boot.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -1439,29 +1440,42 @@ static __init void copy_mapping(unsigned long mfn, unsigned long end, unsigned long emfn)) { unsigned long next; + l3_pgentry_t *l3src = NULL, *l3dst = NULL; for ( ; mfn < end; mfn = next ) { l4_pgentry_t l4e = efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)]; - l3_pgentry_t *l3src, *l3dst; unsigned long va = (unsigned long)mfn_to_virt(mfn); + if ( !(mfn & ((1UL << (L4_PAGETABLE_SHIFT - PAGE_SHIFT)) - 1)) ) + UNMAP_DOMAIN_PAGE(l3dst); + if ( !(va & ((1UL << L4_PAGETABLE_SHIFT) - 1)) ) + UNMAP_DOMAIN_PAGE(l3src); next = mfn + (1UL << (L3_PAGETABLE_SHIFT - PAGE_SHIFT)); if ( !is_valid(mfn, min(next, end)) ) continue; - if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) ) + + if ( l3dst ) + /* nothing */; + else if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) ) { - l3dst = alloc_xen_pagetable(); + mfn_t l3mfn; + + l3dst = alloc_map_clear_xen_pt(&l3mfn); BUG_ON(!l3dst); - clear_page(l3dst); efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)] = - l4e_from_paddr(virt_to_maddr(l3dst), __PAGE_HYPERVISOR); + l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR); } else - l3dst = l4e_to_l3e(l4e); - l3src = l4e_to_l3e(idle_pg_table[l4_table_offset(va)]); + l3dst = map_l3t_from_l4e(l4e); + + if ( !l3src ) + l3src = map_l3t_from_l4e(idle_pg_table[l4_table_offset(va)]); l3dst[l3_table_offset(mfn << PAGE_SHIFT)] = l3src[l3_table_offset(va)]; } + + unmap_domain_page(l3src); + unmap_domain_page(l3dst); } static bool __init ram_range_valid(unsigned long smfn, unsigned long emfn) From patchwork Tue Apr 6 11:05:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 954BCC43460 for ; Tue, 6 Apr 2021 11:06:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5FAC160FF0 for ; Tue, 6 Apr 2021 11:06:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FAC160FF0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105807.202440 (Exim 4.92) (envelope-from ) id 1lTjXT-0004YK-Rj; Tue, 06 Apr 2021 11:06:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105807.202440; Tue, 06 Apr 2021 11:06:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXT-0004Y6-NM; Tue, 06 Apr 2021 11:06:31 +0000 Received: by outflank-mailman (input) for mailman id 105807; Tue, 06 Apr 2021 11:06:30 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXS-0004WA-Qk for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:30 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXS-00053d-Jv; Tue, 06 Apr 2021 11:06:30 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXS-0002qD-6v; Tue, 06 Apr 2021 11:06:30 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=Bokmzti3DdezOdGif8N/czvpGiQImoEVzNR/uUs1QN8=; b=A vngtYDbyG7tFo8wyRVUrNeuWuFv7P8+SOiyYGBDp/Mf3zYscyUoh9KJ+nAXZyA4H9ZNENxy1/uQNE I/vxLqgkNrZxoYTqZkO7f/ziaKzsgKuFnSYJcEHdXurd5MIZVY5UkeUlXzPXlfScLaFbEuIisImIu zI30WG7B2NnaeYoE=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 08/13] efi: switch to new APIs in EFI code Date: Tue, 6 Apr 2021 12:05:56 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v7: - add blank line after declaration. - rename efi_l4_pgtable into efi_l4t. - pass the mapped efi_l4t to copy_mapping() instead of map it again. - use the alloc_map_clear_xen_pt() API. - unmap pl3e, pl2e, l1t earlier. --- xen/arch/x86/efi/runtime.h | 13 ++++++--- xen/common/efi/boot.c | 55 ++++++++++++++++++++++---------------- xen/common/efi/efi.h | 3 ++- xen/common/efi/runtime.c | 8 +++--- 4 files changed, 48 insertions(+), 31 deletions(-) diff --git a/xen/arch/x86/efi/runtime.h b/xen/arch/x86/efi/runtime.h index d9eb8f5c270f..77866c5f2178 100644 --- a/xen/arch/x86/efi/runtime.h +++ b/xen/arch/x86/efi/runtime.h @@ -1,12 +1,19 @@ +#include +#include #include #include #ifndef COMPAT -l4_pgentry_t *__read_mostly efi_l4_pgtable; +mfn_t __read_mostly efi_l4_mfn = INVALID_MFN_INITIALIZER; void efi_update_l4_pgtable(unsigned int l4idx, l4_pgentry_t l4e) { - if ( efi_l4_pgtable ) - l4e_write(efi_l4_pgtable + l4idx, l4e); + if ( !mfn_eq(efi_l4_mfn, INVALID_MFN) ) + { + l4_pgentry_t *efi_l4t = map_domain_page(efi_l4_mfn); + + l4e_write(efi_l4t + l4idx, l4e); + unmap_domain_page(efi_l4t); + } } #endif diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c index 64b319d0013b..f21ad5030f41 100644 --- a/xen/common/efi/boot.c +++ b/xen/common/efi/boot.c @@ -1437,14 +1437,15 @@ custom_param("efi", parse_efi_param); static __init void copy_mapping(unsigned long mfn, unsigned long end, bool (*is_valid)(unsigned long smfn, - unsigned long emfn)) + unsigned long emfn), + l4_pgentry_t *efi_l4t) { unsigned long next; l3_pgentry_t *l3src = NULL, *l3dst = NULL; for ( ; mfn < end; mfn = next ) { - l4_pgentry_t l4e = efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)]; + l4_pgentry_t l4e = efi_l4t[l4_table_offset(mfn << PAGE_SHIFT)]; unsigned long va = (unsigned long)mfn_to_virt(mfn); if ( !(mfn & ((1UL << (L4_PAGETABLE_SHIFT - PAGE_SHIFT)) - 1)) ) @@ -1463,7 +1464,7 @@ static __init void copy_mapping(unsigned long mfn, unsigned long end, l3dst = alloc_map_clear_xen_pt(&l3mfn); BUG_ON(!l3dst); - efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)] = + efi_l4t[l4_table_offset(mfn << PAGE_SHIFT)] = l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR); } else @@ -1496,6 +1497,7 @@ static bool __init rt_range_valid(unsigned long smfn, unsigned long emfn) void __init efi_init_memory(void) { unsigned int i; + l4_pgentry_t *efi_l4t; struct rt_extra { struct rt_extra *next; unsigned long smfn, emfn; @@ -1610,11 +1612,10 @@ void __init efi_init_memory(void) * Set up 1:1 page tables for runtime calls. See SetVirtualAddressMap() in * efi_exit_boot(). */ - efi_l4_pgtable = alloc_xen_pagetable(); - BUG_ON(!efi_l4_pgtable); - clear_page(efi_l4_pgtable); + efi_l4t = alloc_map_clear_xen_pt(&efi_l4_mfn); + BUG_ON(!efi_l4t); - copy_mapping(0, max_page, ram_range_valid); + copy_mapping(0, max_page, ram_range_valid, efi_l4t); /* Insert non-RAM runtime mappings inside the direct map. */ for ( i = 0; i < efi_memmap_size; i += efi_mdesc_size ) @@ -1630,58 +1631,64 @@ void __init efi_init_memory(void) copy_mapping(PFN_DOWN(desc->PhysicalStart), PFN_UP(desc->PhysicalStart + (desc->NumberOfPages << EFI_PAGE_SHIFT)), - rt_range_valid); + rt_range_valid, efi_l4t); } /* Insert non-RAM runtime mappings outside of the direct map. */ while ( (extra = extra_head) != NULL ) { unsigned long addr = extra->smfn << PAGE_SHIFT; - l4_pgentry_t l4e = efi_l4_pgtable[l4_table_offset(addr)]; + l4_pgentry_t l4e = efi_l4t[l4_table_offset(addr)]; l3_pgentry_t *pl3e; l2_pgentry_t *pl2e; l1_pgentry_t *l1t; if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) ) { - pl3e = alloc_xen_pagetable(); + mfn_t l3mfn; + + pl3e = alloc_map_clear_xen_pt(&l3mfn); BUG_ON(!pl3e); - clear_page(pl3e); - efi_l4_pgtable[l4_table_offset(addr)] = - l4e_from_paddr(virt_to_maddr(pl3e), __PAGE_HYPERVISOR); + efi_l4t[l4_table_offset(addr)] = + l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR); } else - pl3e = l4e_to_l3e(l4e); + pl3e = map_l3t_from_l4e(l4e); pl3e += l3_table_offset(addr); if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) { - pl2e = alloc_xen_pagetable(); + mfn_t l2mfn; + + pl2e = alloc_map_clear_xen_pt(&l2mfn); BUG_ON(!pl2e); - clear_page(pl2e); - *pl3e = l3e_from_paddr(virt_to_maddr(pl2e), __PAGE_HYPERVISOR); + *pl3e = l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR); } else { BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE); - pl2e = l3e_to_l2e(*pl3e); + pl2e = map_l2t_from_l3e(*pl3e); } + UNMAP_DOMAIN_PAGE(pl3e); pl2e += l2_table_offset(addr); if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { - l1t = alloc_xen_pagetable(); + mfn_t l1mfn; + + l1t = alloc_map_clear_xen_pt(&l1mfn); BUG_ON(!l1t); - clear_page(l1t); - *pl2e = l2e_from_paddr(virt_to_maddr(l1t), __PAGE_HYPERVISOR); + *pl2e = l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR); } else { BUG_ON(l2e_get_flags(*pl2e) & _PAGE_PSE); - l1t = l2e_to_l1e(*pl2e); + l1t = map_l1t_from_l2e(*pl2e); } + UNMAP_DOMAIN_PAGE(pl2e); for ( i = l1_table_offset(addr); i < L1_PAGETABLE_ENTRIES && extra->smfn < extra->emfn; ++i, ++extra->smfn ) l1t[i] = l1e_from_pfn(extra->smfn, extra->prot); + UNMAP_DOMAIN_PAGE(l1t); if ( extra->smfn == extra->emfn ) { @@ -1693,6 +1700,8 @@ void __init efi_init_memory(void) /* Insert Xen mappings. */ for ( i = l4_table_offset(HYPERVISOR_VIRT_START); i < l4_table_offset(DIRECTMAP_VIRT_END); ++i ) - efi_l4_pgtable[i] = idle_pg_table[i]; + efi_l4t[i] = idle_pg_table[i]; + + unmap_domain_page(efi_l4t); } #endif diff --git a/xen/common/efi/efi.h b/xen/common/efi/efi.h index 663a8b5000d9..c9aa65d506b1 100644 --- a/xen/common/efi/efi.h +++ b/xen/common/efi/efi.h @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -29,7 +30,7 @@ extern UINTN efi_memmap_size, efi_mdesc_size; extern void *efi_memmap; #ifdef CONFIG_X86 -extern l4_pgentry_t *efi_l4_pgtable; +extern mfn_t efi_l4_mfn; #endif extern const struct efi_pci_rom *efi_pci_roms; diff --git a/xen/common/efi/runtime.c b/xen/common/efi/runtime.c index 95367694b5f3..375b94229e13 100644 --- a/xen/common/efi/runtime.c +++ b/xen/common/efi/runtime.c @@ -85,7 +85,7 @@ struct efi_rs_state efi_rs_enter(void) static const u32 mxcsr = MXCSR_DEFAULT; struct efi_rs_state state = { .cr3 = 0 }; - if ( !efi_l4_pgtable ) + if ( mfn_eq(efi_l4_mfn, INVALID_MFN) ) return state; state.cr3 = read_cr3(); @@ -111,7 +111,7 @@ struct efi_rs_state efi_rs_enter(void) lgdt(&gdt_desc); } - switch_cr3_cr4(virt_to_maddr(efi_l4_pgtable), read_cr4()); + switch_cr3_cr4(mfn_to_maddr(efi_l4_mfn), read_cr4()); return state; } @@ -140,9 +140,9 @@ void efi_rs_leave(struct efi_rs_state *state) bool efi_rs_using_pgtables(void) { - return efi_l4_pgtable && + return !mfn_eq(efi_l4_mfn, INVALID_MFN) && (smp_processor_id() == efi_rs_on_cpu) && - (read_cr3() == virt_to_maddr(efi_l4_pgtable)); + (read_cr3() == mfn_to_maddr(efi_l4_mfn)); } unsigned long efi_get_time(void) From patchwork Tue Apr 6 11:05:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52F74C433ED for ; Tue, 6 Apr 2021 11:06:41 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1CAC360FF0 for ; Tue, 6 Apr 2021 11:06:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1CAC360FF0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105808.202452 (Exim 4.92) (envelope-from ) id 1lTjXW-0004dy-Br; Tue, 06 Apr 2021 11:06:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105808.202452; Tue, 06 Apr 2021 11:06:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXW-0004dl-61; Tue, 06 Apr 2021 11:06:34 +0000 Received: by outflank-mailman (input) for mailman id 105808; Tue, 06 Apr 2021 11:06:33 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXU-0004bL-TE for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:06:32 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjXU-00053k-OF; Tue, 06 Apr 2021 11:06:32 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXU-0002qD-7v; Tue, 06 Apr 2021 11:06:32 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=WyOLTWy30UAQ1i+d+4Gv5h1nt743T4iqL5WX21fKHJs=; b=F rbB/vL/o0AwCfSeS1ZtEsipXn5nqwQTNJcXuBgbzqVmr1GKCYDdjW2yvZjW7WAnMjjOrKDI8Uh0a8 swwf7Tff+tN0aS2yZbNrnH+60UGjC7JW5VdKYHEyawpeI6oJ+W6Uy+k428Z0GbXZKYTUNni7e8ckI V247oIHt5qEaWkUc=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 09/13] x86/smpboot: add exit path for clone_mapping() Date: Tue, 6 Apr 2021 12:05:57 +0100 Message-Id: <1277cb7cb71edbb03a13f9db8fe07810e68c7248.1617706782.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu We will soon need to clean up page table mappings in the exit path. No functional change. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Acked-by: Jan Beulich --- Changed in v7: - edit commit message. - begin with rc = 0 and set it to -ENOMEM ahead of if(). --- xen/arch/x86/smpboot.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 82c1012e892f..e90c4dfa8a88 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -696,6 +696,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) l3_pgentry_t *pl3e; l2_pgentry_t *pl2e; l1_pgentry_t *pl1e; + int rc = 0; /* * Sanity check 'linear'. We only allow cloning from the Xen virtual @@ -736,7 +737,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(linear); flags = l1e_get_flags(*pl1e); if ( !(flags & _PAGE_PRESENT) ) - return 0; + goto out; pfn = l1e_get_pfn(*pl1e); } } @@ -744,8 +745,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) if ( !(root_get_flags(rpt[root_table_offset(linear)]) & _PAGE_PRESENT) ) { pl3e = alloc_xen_pagetable(); + rc = -ENOMEM; if ( !pl3e ) - return -ENOMEM; + goto out; clear_page(pl3e); l4e_write(&rpt[root_table_offset(linear)], l4e_from_paddr(__pa(pl3e), __PAGE_HYPERVISOR)); @@ -758,8 +760,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) { pl2e = alloc_xen_pagetable(); + rc = -ENOMEM; if ( !pl2e ) - return -ENOMEM; + goto out; clear_page(pl2e); l3e_write(pl3e, l3e_from_paddr(__pa(pl2e), __PAGE_HYPERVISOR)); } @@ -774,8 +777,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { pl1e = alloc_xen_pagetable(); + rc = -ENOMEM; if ( !pl1e ) - return -ENOMEM; + goto out; clear_page(pl1e); l2e_write(pl2e, l2e_from_paddr(__pa(pl1e), __PAGE_HYPERVISOR)); } @@ -796,7 +800,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) else l1e_write(pl1e, l1e_from_pfn(pfn, flags)); - return 0; + rc = 0; + out: + return rc; } DEFINE_PER_CPU(root_pgentry_t *, root_pgt); From patchwork Tue Apr 6 11:05:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF3AAC433ED for ; Tue, 6 Apr 2021 11:14:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E25B613C8 for ; Tue, 6 Apr 2021 11:14:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E25B613C8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105861.202492 (Exim 4.92) (envelope-from ) id 1lTjfM-0006AT-Ip; Tue, 06 Apr 2021 11:14:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105861.202492; Tue, 06 Apr 2021 11:14:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfM-0006AI-9N; Tue, 06 Apr 2021 11:14:40 +0000 Received: by outflank-mailman (input) for mailman id 105861; Tue, 06 Apr 2021 11:14:39 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfL-00068F-3A for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:14:39 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-0005DF-PH; Tue, 06 Apr 2021 11:14:38 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXW-0002qD-Ef; Tue, 06 Apr 2021 11:06:34 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=ghfYvoztMTHTZUjJZnAAYnP+07ha7ktJ9bkRKWPH3C0=; b=p dpfQAPuqvalw97DlIN6QlIium63XHVzzicF8LwVLyNlh0Hi8+DeQWCo1P5ENQK2J2tbVPU7lqY642 2z7mlsjHc06h4v5cLRU86NpLqLDbGmUadgIYcx54wwpem6xk2ItsgUB4O1uruBCrX3tAoganK5McH R699Sw+0+P8FU5CY=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 10/13] x86/smpboot: switch clone_mapping() to new APIs Date: Tue, 6 Apr 2021 12:05:58 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v7: - change patch title - remove initialiser of pl3e. - combine the initialisation of pl3e into a single assignment. - use the new alloc_map_clear() helper. - use the normal map_domain_page() in the error path. --- xen/arch/x86/smpboot.c | 44 ++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 17 deletions(-) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index e90c4dfa8a88..9c5323977b25 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -694,8 +694,8 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) unsigned long linear = (unsigned long)ptr, pfn; unsigned int flags; l3_pgentry_t *pl3e; - l2_pgentry_t *pl2e; - l1_pgentry_t *pl1e; + l2_pgentry_t *pl2e = NULL; + l1_pgentry_t *pl1e = NULL; int rc = 0; /* @@ -710,7 +710,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) (linear >= XEN_VIRT_END && linear < DIRECTMAP_VIRT_START) ) return -EINVAL; - pl3e = l4e_to_l3e(idle_pg_table[root_table_offset(linear)]) + + pl3e = map_l3t_from_l4e(idle_pg_table[root_table_offset(linear)]) + l3_table_offset(linear); flags = l3e_get_flags(*pl3e); @@ -723,7 +723,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) } else { - pl2e = l3e_to_l2e(*pl3e) + l2_table_offset(linear); + pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(linear); flags = l2e_get_flags(*pl2e); ASSERT(flags & _PAGE_PRESENT); if ( flags & _PAGE_PSE ) @@ -734,7 +734,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) } else { - pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(linear); + pl1e = map_l1t_from_l2e(*pl2e) + l1_table_offset(linear); flags = l1e_get_flags(*pl1e); if ( !(flags & _PAGE_PRESENT) ) goto out; @@ -742,51 +742,58 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) } } + UNMAP_DOMAIN_PAGE(pl1e); + UNMAP_DOMAIN_PAGE(pl2e); + UNMAP_DOMAIN_PAGE(pl3e); + if ( !(root_get_flags(rpt[root_table_offset(linear)]) & _PAGE_PRESENT) ) { - pl3e = alloc_xen_pagetable(); + mfn_t l3mfn; + + pl3e = alloc_map_clear_xen_pt(&l3mfn); rc = -ENOMEM; if ( !pl3e ) goto out; - clear_page(pl3e); l4e_write(&rpt[root_table_offset(linear)], - l4e_from_paddr(__pa(pl3e), __PAGE_HYPERVISOR)); + l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR)); } else - pl3e = l4e_to_l3e(rpt[root_table_offset(linear)]); + pl3e = map_l3t_from_l4e(rpt[root_table_offset(linear)]); pl3e += l3_table_offset(linear); if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ) { - pl2e = alloc_xen_pagetable(); + mfn_t l2mfn; + + pl2e = alloc_map_clear_xen_pt(&l2mfn); rc = -ENOMEM; if ( !pl2e ) goto out; - clear_page(pl2e); - l3e_write(pl3e, l3e_from_paddr(__pa(pl2e), __PAGE_HYPERVISOR)); + l3e_write(pl3e, l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR)); } else { ASSERT(!(l3e_get_flags(*pl3e) & _PAGE_PSE)); - pl2e = l3e_to_l2e(*pl3e); + pl2e = map_l2t_from_l3e(*pl3e); } pl2e += l2_table_offset(linear); if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ) { - pl1e = alloc_xen_pagetable(); + mfn_t l1mfn; + + pl1e = alloc_map_clear_xen_pt(&l1mfn); rc = -ENOMEM; if ( !pl1e ) goto out; - clear_page(pl1e); - l2e_write(pl2e, l2e_from_paddr(__pa(pl1e), __PAGE_HYPERVISOR)); + l2e_write(pl2e, l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR)); } else { ASSERT(!(l2e_get_flags(*pl2e) & _PAGE_PSE)); - pl1e = l2e_to_l1e(*pl2e); + pl1e = map_l1t_from_l2e(*pl2e); } pl1e += l1_table_offset(linear); @@ -802,6 +809,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t *rpt) rc = 0; out: + unmap_domain_page(pl1e); + unmap_domain_page(pl2e); + unmap_domain_page(pl3e); return rc; } From patchwork Tue Apr 6 11:05:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3361C433B4 for ; Tue, 6 Apr 2021 11:14:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6166A613C6 for ; Tue, 6 Apr 2021 11:14:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6166A613C6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105859.202471 (Exim 4.92) (envelope-from ) id 1lTjfL-00068x-Oi; Tue, 06 Apr 2021 11:14:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105859.202471; Tue, 06 Apr 2021 11:14:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfL-00068l-I7; Tue, 06 Apr 2021 11:14:39 +0000 Received: by outflank-mailman (input) for mailman id 105859; Tue, 06 Apr 2021 11:14:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-000680-P3 for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:14:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-0005D9-JO; Tue, 06 Apr 2021 11:14:38 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXY-0002qD-NN; Tue, 06 Apr 2021 11:06:37 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=+kGlUuzwT+sbpOTQzUJ4YhmOCDHflLj7pEbE8cUbSww=; b=3 BcsruzTfPLWLzHMFQd5uoEJPijKW2T57w2g1Oh+0KXo0aBf3mHJBepbHErZrvr5kJtQmBELbD7Xw7 7qBlOVqvL5SNVSNgGHIgTF405NOpDpmsko7FmjYGNIiNC4Vk1b3cdRP2VBxmk2C03Z33G/4+0YZkA 7dGSVu37lmcYHieY=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 11/13] x86/mm: drop old page table APIs Date: Tue, 6 Apr 2021 12:05:59 +0100 Message-Id: <89e67a12a1037a7f897bb4d24eb05399e5f7e441.1617706782.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Hongyan Xia Two sets of old APIs, alloc/free_xen_pagetable() and lXe_to_lYe(), are now dropped to avoid the dependency on direct map. There are two special cases which still have not been re-written into the new APIs, thus need special treatment: rpt in smpboot.c cannot use ephemeral mappings yet. The problem is that rpt is read and written in context switch code, but the mapping infrastructure is NOT context-switch-safe, meaning we cannot map rpt in one domain and unmap in another. Before the mapping infrastructure supports context switches, rpt has to be globally mapped. Also, lXe_to_lYe() during Xen image relocation cannot be converted into map/unmap pairs. We cannot hold on to mappings while the mapping infrastructure is being relocated! It is enough to remove the direct map in the second e820 pass, so we still use the direct map (<4GiB) in Xen relocation (which is during the first e820 pass). Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- xen/arch/x86/mm.c | 14 -------------- xen/arch/x86/setup.c | 4 ++-- xen/arch/x86/smpboot.c | 4 ++-- xen/include/asm-x86/mm.h | 2 -- xen/include/asm-x86/page.h | 5 ----- 5 files changed, 4 insertions(+), 25 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index ababfffb3afc..a1ea1835d49b 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -4875,20 +4875,6 @@ int mmcfg_intercept_write( return X86EMUL_OKAY; } -void *alloc_xen_pagetable(void) -{ - mfn_t mfn = alloc_xen_pagetable_new(); - - return mfn_eq(mfn, INVALID_MFN) ? NULL : mfn_to_virt(mfn_x(mfn)); -} - -void free_xen_pagetable(void *v) -{ - mfn_t mfn = v ? virt_to_mfn(v) : INVALID_MFN; - - free_xen_pagetable_new(mfn); -} - /* * For these PTE APIs, the caller must follow the alloc-map-unmap-free * lifecycle, which means explicitly mapping the PTE pages before accessing diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 68454df8ed67..84f015bfa949 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1245,7 +1245,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) continue; *pl4e = l4e_from_intpte(l4e_get_intpte(*pl4e) + xen_phys_start); - pl3e = l4e_to_l3e(*pl4e); + pl3e = __va(l4e_get_paddr(*pl4e)); for ( j = 0; j < L3_PAGETABLE_ENTRIES; j++, pl3e++ ) { /* Not present, 1GB mapping, or already relocated? */ @@ -1255,7 +1255,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) continue; *pl3e = l3e_from_intpte(l3e_get_intpte(*pl3e) + xen_phys_start); - pl2e = l3e_to_l2e(*pl3e); + pl2e = __va(l3e_get_paddr(*pl3e)); for ( k = 0; k < L2_PAGETABLE_ENTRIES; k++, pl2e++ ) { /* Not present, PSE, or already relocated? */ diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 9c5323977b25..b91d1f37e223 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -830,7 +830,7 @@ static int setup_cpu_root_pgt(unsigned int cpu) if ( !opt_xpti_hwdom && !opt_xpti_domu ) return 0; - rpt = alloc_xen_pagetable(); + rpt = alloc_xenheap_page(); if ( !rpt ) return -ENOMEM; @@ -933,7 +933,7 @@ static void cleanup_cpu_root_pgt(unsigned int cpu) free_xen_pagetable_new(l3mfn); } - free_xen_pagetable(rpt); + free_xenheap_page(rpt); /* Also zap the stub mapping for this CPU. */ if ( stub_linear ) diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 681aac5b7ac2..b05ede721377 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -573,8 +573,6 @@ int vcpu_destroy_pagetables(struct vcpu *); void *do_page_walk(struct vcpu *v, unsigned long addr); /* Allocator functions for Xen pagetables. */ -void *alloc_xen_pagetable(void); -void free_xen_pagetable(void *v); mfn_t alloc_xen_pagetable_new(void); void free_xen_pagetable_new(mfn_t mfn); void *alloc_map_clear_xen_pt(mfn_t *pmfn); diff --git a/xen/include/asm-x86/page.h b/xen/include/asm-x86/page.h index 4c7f2cb70c69..1d080cffbe84 100644 --- a/xen/include/asm-x86/page.h +++ b/xen/include/asm-x86/page.h @@ -180,11 +180,6 @@ static inline l4_pgentry_t l4e_from_paddr(paddr_t pa, unsigned int flags) #define l4e_has_changed(x,y,flags) \ ( !!(((x).l4 ^ (y).l4) & ((PADDR_MASK&PAGE_MASK)|put_pte_flags(flags))) ) -/* Pagetable walking. */ -#define l2e_to_l1e(x) ((l1_pgentry_t *)__va(l2e_get_paddr(x))) -#define l3e_to_l2e(x) ((l2_pgentry_t *)__va(l3e_get_paddr(x))) -#define l4e_to_l3e(x) ((l3_pgentry_t *)__va(l4e_get_paddr(x))) - #define map_l1t_from_l2e(x) (l1_pgentry_t *)map_domain_page(l2e_get_mfn(x)) #define map_l2t_from_l3e(x) (l2_pgentry_t *)map_domain_page(l3e_get_mfn(x)) #define map_l3t_from_l4e(x) (l3_pgentry_t *)map_domain_page(l4e_get_mfn(x)) From patchwork Tue Apr 6 11:06:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1365EC43460 for ; Tue, 6 Apr 2021 11:14:51 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C5575613C6 for ; Tue, 6 Apr 2021 11:14:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C5575613C6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105858.202464 (Exim 4.92) (envelope-from ) id 1lTjfL-00068R-DM; Tue, 06 Apr 2021 11:14:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105858.202464; Tue, 06 Apr 2021 11:14:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfL-00068K-9w; Tue, 06 Apr 2021 11:14:39 +0000 Received: by outflank-mailman (input) for mailman id 105858; Tue, 06 Apr 2021 11:14:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-000681-R1 for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:14:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-0005D7-Fq; Tue, 06 Apr 2021 11:14:38 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXa-0002qD-SA; Tue, 06 Apr 2021 11:06:39 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=cdFFj/D4PvTt7YO7kl51y+p0i52EwU56TpClW+8LHuI=; b=F 2jRgC9IJIzQMIGtxE5ACXbzoI9wSmitm1+2hY6pBwK+mFcNLvuiFJYSCE8hJHqwnaEoarNs3uXmnw OqylJwMpcIH7+BG+51ZbqlHEfj7T+88QhU5BTCrnkr/UPsAf1ZBtvncVKjV/klMrvJLHE0G2UuJ9y 1Hh/n+BY/tvP3uus=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 12/13] x86: switch to use domheap page for page tables Date: Tue, 6 Apr 2021 12:06:00 +0100 Message-Id: <45e9b291c308e0da0783f5f192cbd9e96652b6fb.1617706782.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Hongyan Xia Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Reviewed-by: Jan Beulich --- Changed in v8: - const qualify pg in alloc_xen_pagetable_new(). --- xen/arch/x86/mm.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index a1ea1835d49b..03362448bd05 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -4885,10 +4885,10 @@ mfn_t alloc_xen_pagetable_new(void) { if ( system_state != SYS_STATE_early_boot ) { - void *ptr = alloc_xenheap_page(); + const struct page_info *pg = alloc_domheap_page(NULL, 0); - BUG_ON(!hardware_domain && !ptr); - return ptr ? virt_to_mfn(ptr) : INVALID_MFN; + BUG_ON(!hardware_domain && !pg); + return pg ? page_to_mfn(pg) : INVALID_MFN; } return alloc_boot_pages(1, 1); @@ -4898,7 +4898,7 @@ mfn_t alloc_xen_pagetable_new(void) void free_xen_pagetable_new(mfn_t mfn) { if ( system_state != SYS_STATE_early_boot && !mfn_eq(mfn, INVALID_MFN) ) - free_xenheap_page(mfn_to_virt(mfn_x(mfn))); + free_domheap_page(mfn_to_page(mfn)); } void *alloc_map_clear_xen_pt(mfn_t *pmfn) From patchwork Tue Apr 6 11:06:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyan Xia X-Patchwork-Id: 12184903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD7E3C43462 for ; Tue, 6 Apr 2021 11:14:51 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 75BC5613C6 for ; Tue, 6 Apr 2021 11:14:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75BC5613C6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.105860.202478 (Exim 4.92) (envelope-from ) id 1lTjfM-00069g-3d; Tue, 06 Apr 2021 11:14:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 105860.202478; Tue, 06 Apr 2021 11:14:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfL-00069N-S0; Tue, 06 Apr 2021 11:14:39 +0000 Received: by outflank-mailman (input) for mailman id 105860; Tue, 06 Apr 2021 11:14:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-00068A-UM for xen-devel@lists.xenproject.org; Tue, 06 Apr 2021 11:14:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lTjfK-0005DB-Mq; Tue, 06 Apr 2021 11:14:38 +0000 Received: from 54-240-197-238.amazon.com ([54.240.197.238] helo=ua82172827c7b5a.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjXc-0002qD-Rp; Tue, 06 Apr 2021 11:06:41 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From; bh=sBNlkWTy5F/z7jpVrLYtyYD1UdrsEQWq0TNXWq9eqMA=; b=5 u/aLFNiJgIVn0nb6QF6Zxs0xpUSfMdldRkBykCkHMoA9uPAw8sdhvmEDgyyDqc00Je3NYAhM7MQBN lk3NKmfd/EdHDMHBnC0w2fb30/NmPcf6G1c454faNSyjScS21AReZI0pljFqj4ghAOSj8acKa9Qr5 o7i4wvBcJFOcjPkA=; From: Hongyan Xia To: xen-devel@lists.xenproject.org Cc: jgrall@amazon.com, Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 13/13] x86/mm: drop _new suffix for page table APIs Date: Tue, 6 Apr 2021 12:06:01 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: From: Wei Liu No functional change. Signed-off-by: Wei Liu Signed-off-by: Hongyan Xia Acked-by: Jan Beulich --- xen/arch/x86/mm.c | 44 ++++++++++++++++++++-------------------- xen/arch/x86/smpboot.c | 6 +++--- xen/arch/x86/x86_64/mm.c | 2 +- xen/include/asm-x86/mm.h | 4 ++-- 4 files changed, 28 insertions(+), 28 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 03362448bd05..b90c2d5f8911 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -369,7 +369,7 @@ void __init arch_init_memory(void) ASSERT(root_pgt_pv_xen_slots < ROOT_PAGETABLE_PV_XEN_SLOTS); if ( l4_table_offset(split_va) == l4_table_offset(split_va - 1) ) { - mfn_t l3mfn = alloc_xen_pagetable_new(); + mfn_t l3mfn = alloc_xen_pagetable(); if ( !mfn_eq(l3mfn, INVALID_MFN) ) { @@ -4881,7 +4881,7 @@ int mmcfg_intercept_write( * them. The caller must check whether the allocation has succeeded, and only * pass valid MFNs to map_domain_page(). */ -mfn_t alloc_xen_pagetable_new(void) +mfn_t alloc_xen_pagetable(void) { if ( system_state != SYS_STATE_early_boot ) { @@ -4895,7 +4895,7 @@ mfn_t alloc_xen_pagetable_new(void) } /* mfn can be INVALID_MFN */ -void free_xen_pagetable_new(mfn_t mfn) +void free_xen_pagetable(mfn_t mfn) { if ( system_state != SYS_STATE_early_boot && !mfn_eq(mfn, INVALID_MFN) ) free_domheap_page(mfn_to_page(mfn)); @@ -4903,7 +4903,7 @@ void free_xen_pagetable_new(mfn_t mfn) void *alloc_map_clear_xen_pt(mfn_t *pmfn) { - mfn_t mfn = alloc_xen_pagetable_new(); + mfn_t mfn = alloc_xen_pagetable(); void *ret; if ( mfn_eq(mfn, INVALID_MFN) ) @@ -4949,7 +4949,7 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v) } if ( locking ) spin_unlock(&map_pgdir_lock); - free_xen_pagetable_new(l3mfn); + free_xen_pagetable(l3mfn); } return map_l3t_from_l4e(*pl4e) + l3_table_offset(v); @@ -4984,7 +4984,7 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v) } if ( locking ) spin_unlock(&map_pgdir_lock); - free_xen_pagetable_new(l2mfn); + free_xen_pagetable(l2mfn); } BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE); @@ -5023,7 +5023,7 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v) } if ( locking ) spin_unlock(&map_pgdir_lock); - free_xen_pagetable_new(l1mfn); + free_xen_pagetable(l1mfn); } BUG_ON(l2e_get_flags(*pl2e) & _PAGE_PSE); @@ -5210,10 +5210,10 @@ int map_pages_to_xen( ol2e = l2t[i]; if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) && !(l2e_get_flags(ol2e) & _PAGE_PSE) ) - free_xen_pagetable_new(l2e_get_mfn(ol2e)); + free_xen_pagetable(l2e_get_mfn(ol2e)); } unmap_domain_page(l2t); - free_xen_pagetable_new(l3e_get_mfn(ol3e)); + free_xen_pagetable(l3e_get_mfn(ol3e)); } } @@ -5252,7 +5252,7 @@ int map_pages_to_xen( continue; } - l2mfn = alloc_xen_pagetable_new(); + l2mfn = alloc_xen_pagetable(); if ( mfn_eq(l2mfn, INVALID_MFN) ) goto out; @@ -5280,7 +5280,7 @@ int map_pages_to_xen( spin_unlock(&map_pgdir_lock); flush_area(virt, flush_flags); - free_xen_pagetable_new(l2mfn); + free_xen_pagetable(l2mfn); } pl2e = virt_to_xen_l2e(virt); @@ -5314,7 +5314,7 @@ int map_pages_to_xen( flush_flags(l1e_get_flags(l1t[i])); flush_area(virt, flush_flags); unmap_domain_page(l1t); - free_xen_pagetable_new(l2e_get_mfn(ol2e)); + free_xen_pagetable(l2e_get_mfn(ol2e)); } } @@ -5359,7 +5359,7 @@ int map_pages_to_xen( goto check_l3; } - l1mfn = alloc_xen_pagetable_new(); + l1mfn = alloc_xen_pagetable(); if ( mfn_eq(l1mfn, INVALID_MFN) ) goto out; @@ -5386,7 +5386,7 @@ int map_pages_to_xen( spin_unlock(&map_pgdir_lock); flush_area(virt, flush_flags); - free_xen_pagetable_new(l1mfn); + free_xen_pagetable(l1mfn); } pl1e = map_l1t_from_l2e(*pl2e) + l1_table_offset(virt); @@ -5452,7 +5452,7 @@ int map_pages_to_xen( flush_area(virt - PAGE_SIZE, FLUSH_TLB_GLOBAL | FLUSH_ORDER(PAGETABLE_ORDER)); - free_xen_pagetable_new(l2e_get_mfn(ol2e)); + free_xen_pagetable(l2e_get_mfn(ol2e)); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5503,7 +5503,7 @@ int map_pages_to_xen( flush_area(virt - PAGE_SIZE, FLUSH_TLB_GLOBAL | FLUSH_ORDER(2*PAGETABLE_ORDER)); - free_xen_pagetable_new(l3e_get_mfn(ol3e)); + free_xen_pagetable(l3e_get_mfn(ol3e)); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5602,7 +5602,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) } /* PAGE1GB: shatter the superpage and fall through. */ - l2mfn = alloc_xen_pagetable_new(); + l2mfn = alloc_xen_pagetable(); if ( mfn_eq(l2mfn, INVALID_MFN) ) goto out; @@ -5626,7 +5626,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( locking ) spin_unlock(&map_pgdir_lock); - free_xen_pagetable_new(l2mfn); + free_xen_pagetable(l2mfn); } /* @@ -5662,7 +5662,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) { l1_pgentry_t *l1t; /* PSE: shatter the superpage and try again. */ - mfn_t l1mfn = alloc_xen_pagetable_new(); + mfn_t l1mfn = alloc_xen_pagetable(); if ( mfn_eq(l1mfn, INVALID_MFN) ) goto out; @@ -5686,7 +5686,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( locking ) spin_unlock(&map_pgdir_lock); - free_xen_pagetable_new(l1mfn); + free_xen_pagetable(l1mfn); } } else @@ -5753,7 +5753,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(NULL, FLUSH_TLB_GLOBAL); /* flush before free */ - free_xen_pagetable_new(l1mfn); + free_xen_pagetable(l1mfn); } else if ( locking ) spin_unlock(&map_pgdir_lock); @@ -5798,7 +5798,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) if ( locking ) spin_unlock(&map_pgdir_lock); flush_area(NULL, FLUSH_TLB_GLOBAL); /* flush before free */ - free_xen_pagetable_new(l2mfn); + free_xen_pagetable(l2mfn); } else if ( locking ) spin_unlock(&map_pgdir_lock); diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index b91d1f37e223..8659220416aa 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -922,15 +922,15 @@ static void cleanup_cpu_root_pgt(unsigned int cpu) continue; ASSERT(!(l2e_get_flags(l2t[i2]) & _PAGE_PSE)); - free_xen_pagetable_new(l2e_get_mfn(l2t[i2])); + free_xen_pagetable(l2e_get_mfn(l2t[i2])); } unmap_domain_page(l2t); - free_xen_pagetable_new(l2mfn); + free_xen_pagetable(l2mfn); } unmap_domain_page(l3t); - free_xen_pagetable_new(l3mfn); + free_xen_pagetable(l3mfn); } free_xenheap_page(rpt); diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 442f345b2a54..579ffa54c982 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -640,7 +640,7 @@ void __init paging_init(void) /* Create user-accessible L2 directory to map the MPT for compat guests. */ if ( opt_pv32 ) { - mfn = alloc_xen_pagetable_new(); + mfn = alloc_xen_pagetable(); if ( mfn_eq(mfn, INVALID_MFN) ) goto nomem; compat_idle_pg_table_l2 = map_domain_page_global(mfn); diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index b05ede721377..b2580c9ad073 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -573,8 +573,8 @@ int vcpu_destroy_pagetables(struct vcpu *); void *do_page_walk(struct vcpu *v, unsigned long addr); /* Allocator functions for Xen pagetables. */ -mfn_t alloc_xen_pagetable_new(void); -void free_xen_pagetable_new(mfn_t mfn); +mfn_t alloc_xen_pagetable(void); +void free_xen_pagetable(mfn_t mfn); void *alloc_map_clear_xen_pt(mfn_t *pmfn); l1_pgentry_t *virt_to_xen_l1e(unsigned long v);