From patchwork Tue Apr 30 16:58:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13649687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6703EC4345F for ; Tue, 30 Apr 2024 16:59:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715128.1116656 (Exim 4.92) (envelope-from ) id 1s1qon-0005q3-PX; Tue, 30 Apr 2024 16:59:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715128.1116656; Tue, 30 Apr 2024 16:59:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s1qon-0005pw-Kg; Tue, 30 Apr 2024 16:59:01 +0000 Received: by outflank-mailman (input) for mailman id 715128; Tue, 30 Apr 2024 16:59:00 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s1qom-0005Xj-Jz for xen-devel@lists.xenproject.org; Tue, 30 Apr 2024 16:59:00 +0000 Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [2607:f8b0:4864:20::72a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ef9663b1-0712-11ef-b4bb-af5377834399; Tue, 30 Apr 2024 18:58:58 +0200 (CEST) Received: by mail-qk1-x72a.google.com with SMTP id af79cd13be357-78ecd752a7cso423523485a.0 for ; Tue, 30 Apr 2024 09:58:58 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id sg18-20020a05620a935200b00790fedd9262sm1420114qkn.42.2024.04.30.09.58.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Apr 2024 09:58:56 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ef9663b1-0712-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1714496337; x=1715101137; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3Kuc/dHNbs05v7aWRXT2GIYcN5J0aHj10Td0wtI6Les=; b=BEHmHmT3rngsjT83s2MXo/BeMwvPQroymTsD63uqp1BLtzE4oGgI+elFF9r3QlBMGL T7z5rcyaiCXQ3AEHrZM1OeV7SgvLXFVj7Mbavfss/7rTlzHYnz7YwZ5iIcRVwTKw/atH hio0IUsBF5Vg4IsNO6vHQtELrIaqPvys2tQZg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714496337; x=1715101137; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Kuc/dHNbs05v7aWRXT2GIYcN5J0aHj10Td0wtI6Les=; b=eePGQ/rcyLAY8iFHlikxHPD/qxHHqGgPSa75M94xpDcjYj3wt2pQm1aCt0qUpXWUrm WGGTL3zz8zAbHlIsJAaI2/psiAP6x4bVNAkSMCUDvHUxHmleyQbbAFEyjReLwwScA7SR wAfHgK9og7gsPQvBWXxMrcr1AkaGgr+snP66Jzb7UwiBvDevkyYl+vEHd2Y18D5Fn09T DcAtY4e0zOjwVoYhWQLTGSWvST8pQPbTIu6Uo4PcReSwhOVOjm7fPWPzUdftZCF+ScVl o0Y3cceTPNI4k0Vkqv/1vS9gkrAuvP7PlZwE4Pz3v1Cg4vNR5QKVjqmJEXX09L7tVSaU n5PA== X-Gm-Message-State: AOJu0YyAe6+mIOjGh8+Go4TpMBALZiXuMi00VmVkzPrBT6rT4IRanBAv 118xzEJ5r1kHBohGQpMCFOTlbnYMWZKzQwv4yWvVxeN3p+MHH1LRJ1oHam/pRhZfeB1Llf2JZy7 R X-Google-Smtp-Source: AGHT+IEb4pX+qJH52aPh72xor/1XOHlskv5DRNPZx/MfY56V0w/5FhTDuS9JiMb24jlmU9vDCiAHhg== X-Received: by 2002:a05:620a:5e48:b0:790:c81e:7840 with SMTP id ya8-20020a05620a5e4800b00790c81e7840mr10707432qkn.12.1714496336968; Tue, 30 Apr 2024 09:58:56 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , George Dunlap Subject: [PATCH for-4.19? 1/2] xen/x86: account for max guest gfn and number of foreign mappings in the p2m Date: Tue, 30 Apr 2024 18:58:44 +0200 Message-ID: <20240430165845.81696-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240430165845.81696-1-roger.pau@citrix.com> References: <20240430165845.81696-1-roger.pau@citrix.com> MIME-Version: 1.0 Keep track of the maximum gfn that has ever been populated into the p2m, and also account for the number of foreign mappings. Such information will be needed in order to remove foreign mappings during teardown for HVM guests. Right now the introduced counters are not consumed. Signed-off-by: Roger Pau Monné --- xen/arch/x86/include/asm/p2m.h | 11 +++++++++++ xen/arch/x86/mm/p2m.c | 2 ++ 2 files changed, 13 insertions(+) diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h index 111badf89a6e..d95341ef4242 100644 --- a/xen/arch/x86/include/asm/p2m.h +++ b/xen/arch/x86/include/asm/p2m.h @@ -380,6 +380,14 @@ struct p2m_domain { unsigned int flags; unsigned long entry_count; } ioreq; + + /* + * Max gfn possibly mapped into the guest p2m. Note max_gfn is not + * adjusted to account for removals from the p2m. + */ + gfn_t max_gfn; + /* Number of foreign mappings. */ + unsigned long nr_foreign; #endif /* CONFIG_HVM */ }; @@ -1049,6 +1057,8 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt, if ( !page_get_owner_and_reference(mfn_to_page(nfn)) ) return -EBUSY; + p2m->nr_foreign++; + break; default: @@ -1069,6 +1079,7 @@ static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt, return -EINVAL; } put_page(mfn_to_page(ofn)); + p2m->nr_foreign--; break; default: diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c index ce742c12e0de..05d8536adcd7 100644 --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -413,6 +413,8 @@ int p2m_set_entry(struct p2m_domain *p2m, gfn_t gfn, mfn_t mfn, set_rc = p2m->set_entry(p2m, gfn, mfn, order, p2mt, p2ma, -1); if ( set_rc ) rc = set_rc; + else + p2m->max_gfn = gfn_max(gfn_add(gfn, 1u << order), p2m->max_gfn); gfn = gfn_add(gfn, 1UL << order); if ( !mfn_eq(mfn, INVALID_MFN) ) From patchwork Tue Apr 30 16:58:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13649688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BFA5C10F16 for ; Tue, 30 Apr 2024 16:59:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715129.1116664 (Exim 4.92) (envelope-from ) id 1s1qop-00066F-UD; Tue, 30 Apr 2024 16:59:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715129.1116664; Tue, 30 Apr 2024 16:59:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s1qop-000667-Rh; Tue, 30 Apr 2024 16:59:03 +0000 Received: by outflank-mailman (input) for mailman id 715129; Tue, 30 Apr 2024 16:59:02 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s1qoo-0005Xj-Mf for xen-devel@lists.xenproject.org; Tue, 30 Apr 2024 16:59:02 +0000 Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com [2607:f8b0:4864:20::f2f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f0d61980-0712-11ef-b4bb-af5377834399; Tue, 30 Apr 2024 18:59:01 +0200 (CEST) Received: by mail-qv1-xf2f.google.com with SMTP id 6a1803df08f44-6a0c8fb3540so108646d6.1 for ; Tue, 30 Apr 2024 09:59:01 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id t7-20020a056214154700b006a0d2a52017sm507692qvw.146.2024.04.30.09.58.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Apr 2024 09:58:59 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f0d61980-0712-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1714496339; x=1715101139; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nlkGJPlOCvm5A8j9uk3pxOJP2pbiQ5Ni0MtAwWQpoKg=; b=vQtC+lh78nZXSoKxoYu7t4brPeVaK7cOeUEZvMi6SlDrIX5CX7Z0hvTSJBpKUq9ClK QeyJSHRwmkegRH4FQAQdAh34ZerasCu9JWJQ85BtQyqNUrJTnG/Bk3oHmOOFShC/MDWL 2Up5i+trtSc+vzdz18yu9/LOH1JyhjEq1U57U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714496339; x=1715101139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nlkGJPlOCvm5A8j9uk3pxOJP2pbiQ5Ni0MtAwWQpoKg=; b=GRXoTC+8sMrpl6gn6pfhFMkUx/r9A1k3jMrgvJ0KEsOawwkZwgANFaMFFerh4Y5mr4 UP+vdKezo0ZzJYJ852WAWc5YlQzhFRWwbmcSaagDN0Zu3Mp0tKqMf/YjcCMCURPXmVFU 1VfXN65/gqbSwefRrRXd6hwMsE075/5fZUNv9ME1YzB1CU9QTXtun0QGmQapXSFYtr0g lDSyiooTOOJOg999NR77Nc+NzEOT7PYcdvNbQiqnK8Qfx1anSijEz1scxTWpRdVnGfeo iybwa11d/+VtSeN7x8fuMzVFTcCRZJ8z2uGMKgajw35y102YHY1Yrxm2Wg8ncGUCPeCl rnPw== X-Gm-Message-State: AOJu0YyhXekXEumzqiMzcMq7zFyPeKR/fggsQ/KIpvS+Y5FzJBkvJofQ alWqEgG2bwLkc7qOTYESt4w7YqzvOceH6+R4QLct9H4yZ7FeN7QZsyeWDh4IskZJVJrwZ2+wyKL L X-Google-Smtp-Source: AGHT+IE7KqGFBYUX9ZPYWYnzz4myXyFu+S8SzOCPr4mHPh92LYx3apQjDVu3vBuW8d7AByh6AJLKsA== X-Received: by 2002:a05:6214:202c:b0:6a0:a8f2:5f8e with SMTP id 12-20020a056214202c00b006a0a8f25f8emr706140qvf.1.1714496339437; Tue, 30 Apr 2024 09:58:59 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Oleksii Kurochko , Community Manager , Jan Beulich , Andrew Cooper , George Dunlap Subject: [PATCH for-4.19? 2/2] xen/x86: remove foreign mappings from the p2m on teardown Date: Tue, 30 Apr 2024 18:58:45 +0200 Message-ID: <20240430165845.81696-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240430165845.81696-1-roger.pau@citrix.com> References: <20240430165845.81696-1-roger.pau@citrix.com> MIME-Version: 1.0 Iterate over the p2m based on the maximum recorded gfn and remove any foreign mappings, in order to drop the underlying page references and thus don't keep extra page references if a domain is destroyed while still having foreign mappings on it's p2m. The logic is similar to the one used on Arm. Signed-off-by: Roger Pau Monné --- I still have to test with destroying a guest that does have foreign mappings on it's p2m. --- CHANGELOG.md | 1 + xen/arch/x86/domain.c | 6 +++ xen/arch/x86/include/asm/p2m.h | 17 +++++---- xen/arch/x86/mm/p2m.c | 68 ++++++++++++++++++++++++++++++++-- 4 files changed, 81 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8041cfb7d243..09bdb9b97578 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - HVM PIRQs are disabled by default. - Reduce IOMMU setup time for hardware domain. - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device. + - Allow HVM/PVH domains to map foreign pages. ### Added - On x86: diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 20e83cf38bbd..5aa2d3744e6b 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2364,6 +2364,7 @@ int domain_relinquish_resources(struct domain *d) enum { PROG_iommu_pagetables = 1, PROG_shared, + PROG_mappings, PROG_paging, PROG_vcpu_pagetables, PROG_xen, @@ -2412,6 +2413,11 @@ int domain_relinquish_resources(struct domain *d) } #endif + PROGRESS(mappings): + ret = relinquish_p2m_mapping(d); + if ( ret ) + return ret; + PROGRESS(paging): /* Tear down paging-assistance stuff. */ diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h index d95341ef4242..8b3e6a473a0c 100644 --- a/xen/arch/x86/include/asm/p2m.h +++ b/xen/arch/x86/include/asm/p2m.h @@ -402,13 +402,7 @@ struct p2m_domain { static inline bool arch_acquire_resource_check(struct domain *d) { - /* - * FIXME: Until foreign pages inserted into the P2M are properly - * reference counted, it is unsafe to allow mapping of - * resource pages unless the caller is the hardware domain - * (see set_foreign_p2m_entry()). - */ - return !paging_mode_translate(d) || is_hardware_domain(d); + return true; } /* @@ -725,6 +719,10 @@ p2m_pod_offline_or_broken_hit(struct page_info *p); void p2m_pod_offline_or_broken_replace(struct page_info *p); +/* Perform cleanup of p2m mappings ahead of teardown. */ +int +relinquish_p2m_mapping(struct domain *d); + #else static inline bool @@ -753,6 +751,11 @@ static inline void p2m_pod_offline_or_broken_replace(struct page_info *p) ASSERT_UNREACHABLE(); } +static inline int relinquish_p2m_mapping(struct domain *d) +{ + return 0; +} + #endif diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c index 05d8536adcd7..fac41e5ec808 100644 --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -2335,10 +2335,6 @@ static int p2m_add_foreign(struct domain *tdom, unsigned long fgfn, int rc; struct domain *fdom; - /* - * hvm fixme: until support is added to p2m teardown code to cleanup any - * foreign entries, limit this to hardware domain only. - */ if ( !arch_acquire_resource_check(tdom) ) return -EPERM; @@ -2695,6 +2691,70 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx, return rc; } +/* + * Remove foreign mappings from the p2m, as that drops the page reference taken + * when mapped. + */ +int relinquish_p2m_mapping(struct domain *d) +{ + struct p2m_domain *p2m = p2m_get_hostp2m(d); + unsigned long gfn = gfn_x(p2m->max_gfn); + int rc = 0; + + if ( !paging_mode_translate(d) ) + return 0; + + BUG_ON(!d->is_dying); + + p2m_lock(p2m); + + /* Iterate over the whole p2m on debug builds to ensure correctness. */ + while ( gfn && (IS_ENABLED(CONFIG_DEBUG) || p2m->nr_foreign) ) + { + unsigned int order; + p2m_type_t t; + p2m_access_t a; + + _get_gfn_type_access(p2m, _gfn(gfn - 1), &t, &a, 0, &order, 0); + ASSERT(IS_ALIGNED(gfn, 1u << order)); + gfn -= 1 << order; + + if ( t == p2m_map_foreign ) + { + ASSERT(p2m->nr_foreign); + ASSERT(order == 0); + /* + * Foreign mappings can only be of order 0, hence there's no need + * to align the gfn to the entry order. Otherwise we would need to + * adjust gfn to point to the start of the page if order > 0. + */ + rc = p2m_set_entry(p2m, _gfn(gfn), INVALID_MFN, order, p2m_invalid, + p2m->default_access); + if ( rc ) + { + printk(XENLOG_ERR + "%pd: failed to unmap foreign page %" PRI_gfn " order %u error %d\n", + d, gfn, order, rc); + ASSERT_UNREACHABLE(); + break; + } + } + + if ( !(gfn & 0xfff) && hypercall_preempt_check() ) + { + rc = -ERESTART; + break; + } + } + + ASSERT(gfn || !p2m->nr_foreign); + p2m->max_gfn = _gfn(gfn); + + p2m_unlock(p2m); + + return rc; +} + /* * Local variables: * mode: C