From patchwork Tue Feb 14 05:07:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 13139374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E240DC6379F for ; Tue, 14 Feb 2023 05:08:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.494957.765177 (Exim 4.92) (envelope-from ) id 1pRnXN-0007GY-Mu; Tue, 14 Feb 2023 05:07:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 494957.765177; Tue, 14 Feb 2023 05:07:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pRnXN-0007GR-K1; Tue, 14 Feb 2023 05:07:29 +0000 Received: by outflank-mailman (input) for mailman id 494957; Tue, 14 Feb 2023 05:07:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pRnXM-0007GL-FW for xen-devel@lists.xenproject.org; Tue, 14 Feb 2023 05:07:28 +0000 Received: from MTA-05-3.privateemail.com (mta-05-3.privateemail.com [68.65.122.15]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 783a7726-ac25-11ed-933c-83870f6b2ba8; Tue, 14 Feb 2023 06:07:26 +0100 (CET) Received: from mta-05.privateemail.com (localhost [127.0.0.1]) by mta-05.privateemail.com (Postfix) with ESMTP id 0FBA418000AF; Tue, 14 Feb 2023 00:07:24 -0500 (EST) Received: from localhost.localdomain (c-73-126-95-14.hsd1.ma.comcast.net [73.126.95.14]) by mta-05.privateemail.com (Postfix) with ESMTPA id D068A18000B2; Tue, 14 Feb 2023 00:07:16 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 783a7726-ac25-11ed-933c-83870f6b2ba8 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=tklengyel.com; s=default; t=1676351243; bh=U/t2aKecr1XPTUqutXYj+EG7n3HpJaTp5C9X4aDmHqY=; h=From:To:Cc:Subject:Date:From; b=OvBMe5ElsruzOj1OegqEs8cgxoeOwFDp7FMDTO/LL9IP0TCRoFm4iFHFS1GOKEjt5 FeXJnOvPPY2KQeMujIqb7Jxko6tFDxvL79LTJnF9JETZUmC719wH69aJydeviHBK9T U/R4Ef2yB+xbqwl8jVkXM9nv4U6BOJFyHhAdlYoWx5uMb7LV2yZOpbVQpZmzSinBjm ANE9Kl5iPY1pGK5zQrsr1WRPJ8UVsQPmbBkFklb8Zjqopz0PEjDPpJNzjNwyE+vS5b VALydlWAA/tQnpBV8kbvbuBGNH1pLBAxYv6BF/PggodYpB/vm50J2zGKJr0a+AgvzC xjwZScl/DHohg== From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Cc: Tamas K Lengyel , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 1/2] x86: Perform mem_sharing teardown before paging teardown Date: Tue, 14 Feb 2023 00:07:06 -0500 Message-Id: <3af8dbf3134b48f6bbc8f917e5fecaf8daee1c3d.1676351034.git.tamas@tklengyel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP An assert failure has been observed at p2m-basic.c:173 when performing vm forking and then destroying the forked VM. The assert checks whether the domain's shared pages counter is 0. According to the patch that originally added the assert (7bedbbb5c31) the p2m_teardown should only happen after mem_sharing already relinquished all shared pages. In this patch we flip the order in which relinquish ops are called to avoid tripping the assert. Signed-off-by: Tamas K Lengyel --- Note: it is unclear why this assert hasn't tripped in the past. It hasn't been observed with 4.17-rc1 but it is in RELEASE-4.17.0 --- xen/arch/x86/domain.c | 52 +++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index db3ebf062d..453ec52b6a 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2310,6 +2310,32 @@ int domain_relinquish_resources(struct domain *d) if ( ret ) return ret; +#ifdef CONFIG_MEM_SHARING + PROGRESS(shared): + + if ( is_hvm_domain(d) ) + { + /* If the domain has shared pages, relinquish them allowing + * for preemption. */ + ret = relinquish_shared_pages(d); + if ( ret ) + return ret; + + /* + * If the domain is forked, decrement the parent's pause count + * and release the domain. + */ + if ( mem_sharing_is_fork(d) ) + { + struct domain *parent = d->parent; + + d->parent = NULL; + domain_unpause(parent); + put_domain(parent); + } + } +#endif + PROGRESS(paging): /* Tear down paging-assistance stuff. */ @@ -2350,32 +2376,6 @@ int domain_relinquish_resources(struct domain *d) d->arch.auto_unmask = 0; } -#ifdef CONFIG_MEM_SHARING - PROGRESS(shared): - - if ( is_hvm_domain(d) ) - { - /* If the domain has shared pages, relinquish them allowing - * for preemption. */ - ret = relinquish_shared_pages(d); - if ( ret ) - return ret; - - /* - * If the domain is forked, decrement the parent's pause count - * and release the domain. - */ - if ( mem_sharing_is_fork(d) ) - { - struct domain *parent = d->parent; - - d->parent = NULL; - domain_unpause(parent); - put_domain(parent); - } - } -#endif - spin_lock(&d->page_alloc_lock); page_list_splice(&d->arch.relmem_list, &d->page_list); INIT_PAGE_LIST_HEAD(&d->arch.relmem_list); From patchwork Tue Feb 14 05:07:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 13139375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0714C05027 for ; Tue, 14 Feb 2023 05:08:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.494958.765188 (Exim 4.92) (envelope-from ) id 1pRnXX-0007XO-TQ; Tue, 14 Feb 2023 05:07:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 494958.765188; Tue, 14 Feb 2023 05:07:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pRnXX-0007XH-Qh; Tue, 14 Feb 2023 05:07:39 +0000 Received: by outflank-mailman (input) for mailman id 494958; Tue, 14 Feb 2023 05:07:38 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pRnXW-0007We-Hh for xen-devel@lists.xenproject.org; Tue, 14 Feb 2023 05:07:38 +0000 Received: from MTA-05-3.privateemail.com (mta-05-3.privateemail.com [68.65.122.15]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 7e1664ff-ac25-11ed-93b5-47a8fe42b414; Tue, 14 Feb 2023 06:07:35 +0100 (CET) Received: from mta-05.privateemail.com (localhost [127.0.0.1]) by mta-05.privateemail.com (Postfix) with ESMTP id 18BBD18000B2; Tue, 14 Feb 2023 00:07:34 -0500 (EST) Received: from localhost.localdomain (c-73-126-95-14.hsd1.ma.comcast.net [73.126.95.14]) by mta-05.privateemail.com (Postfix) with ESMTPA id 6059718000B0; Tue, 14 Feb 2023 00:07:24 -0500 (EST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7e1664ff-ac25-11ed-93b5-47a8fe42b414 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=tklengyel.com; s=default; t=1676351254; bh=LPH/xf231W5X/KC3fQVF1Wp17WKoMNP0q+BN8zwfIbU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LolOQG8tMtpM4vaSAopvlzPqvCfHDsvbsXQrXXg9O/fyMX+Nqwb904Fl8CMpB3guc UIMESDru24/g5UBpwTgdwg23gO+a51yMDNxKCggOQDDeAHTR2RH4VFATpeQaxyTQLO sZv+1KT3m12kRAB56Cpw7uG0xUcnYDHJ96yNIcTk9QSDOXct39Ft4iVZiN/SIcfo2p lQgqQs6k5dR0WKBro5YTBjBaew+7KD8SRgV5WkJExsFzwXMknHX5R1DwB5agfnZg6Z MrR2AM0idUXdQUd9nQcWx4YkkptuqN0KZ7OT5AINegWN29ywmotEjcRxQYaQ72MGoW sixgYWPzkbjFQ== From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Cc: Tamas K Lengyel , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap Subject: [PATCH 2/2] x86/mem_sharing: Add extra variable to track fork progress Date: Tue, 14 Feb 2023 00:07:07 -0500 Message-Id: <348063358f2ded237334b3cec7498a36296fd408.1676351034.git.tamas@tklengyel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <3af8dbf3134b48f6bbc8f917e5fecaf8daee1c3d.1676351034.git.tamas@tklengyel.com> References: <3af8dbf3134b48f6bbc8f917e5fecaf8daee1c3d.1676351034.git.tamas@tklengyel.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP When VM forking is initiated a VM is not supposed to try to perform mem_sharing yet as the fork process hasn't completed all required steps. However, the vCPU bring-up paths trigger guest memory accesses that can lead to such premature sharing ops. However, the gating check to see whether a VM is a fork is set already (ie. the domain has a parent). In this patch we introduce a separate variable to store the information that forking has been initiated so that the non-restartable part of the forking op is not repeated and we avoid the premature sharing ops from triggering. Signed-off-by: Tamas K Lengyel --- xen/arch/x86/include/asm/hvm/domain.h | 8 ++++++++ xen/arch/x86/mm/mem_sharing.c | 10 +++++++--- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/include/asm/hvm/domain.h b/xen/arch/x86/include/asm/hvm/domain.h index 698455444e..76a08b55f9 100644 --- a/xen/arch/x86/include/asm/hvm/domain.h +++ b/xen/arch/x86/include/asm/hvm/domain.h @@ -33,6 +33,14 @@ struct mem_sharing_domain { bool enabled, block_interrupts; + /* + * We need to avoid trying to nominate pages for forking until the + * fork operation is completely finished. As parts of fork creation + * is restartable we mark here if the process started to skip the + * non-restartable portion. + */ + bool fork_started; + /* * When releasing shared gfn's in a preemptible manner, recall where * to resume the search. diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 649d93dc54..b55c5bccdd 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -1888,11 +1888,12 @@ static int copy_settings(struct domain *cd, struct domain *d) static int fork(struct domain *cd, struct domain *d) { int rc = -EBUSY; + struct mem_sharing_domain *msd = &cd->arch.hvm.mem_sharing; if ( !cd->controller_pause_count ) return rc; - if ( !cd->parent ) + if ( !msd->fork_started ) { if ( !get_domain(d) ) { @@ -1905,7 +1906,7 @@ static int fork(struct domain *cd, struct domain *d) *cd->arch.cpuid = *d->arch.cpuid; *cd->arch.msr = *d->arch.msr; cd->vmtrace_size = d->vmtrace_size; - cd->parent = d; + msd->fork_started = 1; } /* This is preemptible so it's the first to get done */ @@ -1918,8 +1919,11 @@ static int fork(struct domain *cd, struct domain *d) rc = copy_settings(cd, d); done: - if ( rc && rc != -ERESTART ) + if ( !rc ) + cd->parent = d; + else if ( rc != -ERESTART ) { + msd->fork_started = 0; cd->parent = NULL; domain_unpause(d); put_domain(d);