From patchwork Tue Jul 9 13:20:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Roy X-Patchwork-Id: 13727953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 886C0C3DA47 for ; Tue, 9 Jul 2024 13:21:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8BB626B00B0; Tue, 9 Jul 2024 09:21:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F7526B00B1; Tue, 9 Jul 2024 09:21:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55DE06B00B2; Tue, 9 Jul 2024 09:21:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1AF676B00B0 for ; Tue, 9 Jul 2024 09:21:18 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D5770A503D for ; Tue, 9 Jul 2024 13:21:17 +0000 (UTC) X-FDA: 82320275394.21.30028D0 Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by imf30.hostedemail.com (Postfix) with ESMTP id DE90B8001D for ; Tue, 9 Jul 2024 13:21:15 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=amazon.co.uk header.s=amazon201209 header.b=IBGmho88; dmarc=pass (policy=quarantine) header.from=amazon.co.uk; spf=pass (imf30.hostedemail.com: domain of "prvs=913fd7204=roypat@amazon.co.uk" designates 207.171.184.29 as permitted sender) smtp.mailfrom="prvs=913fd7204=roypat@amazon.co.uk" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720531253; a=rsa-sha256; cv=none; b=lQiuocnswDsTMV6m0PNkM0VueYto4+5l9jsa0vXkVM7kNdmeJtTihBNcSP6MgpB2G+iJkn amtqkgJKLUsQiB8UmOX7e7Dm9+jgTHssoyTNHPnPopiyoU6ZWIX2gQLaIcaEBGuOlvUiQ1 VYRJ5Y3eePCUudqQx1e/+F/efHbx5ZI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=amazon.co.uk header.s=amazon201209 header.b=IBGmho88; dmarc=pass (policy=quarantine) header.from=amazon.co.uk; spf=pass (imf30.hostedemail.com: domain of "prvs=913fd7204=roypat@amazon.co.uk" designates 207.171.184.29 as permitted sender) smtp.mailfrom="prvs=913fd7204=roypat@amazon.co.uk" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720531253; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j7xh8Fp0rGL+JNyIcONb4iIMzBYUq25BVxGN/Q+52J4=; b=Z5LjGe5dMZquV3UrfdDLvQ/RDLxp305o9d2x3xW4BCj7lQc6syXZu3KUwnJAmbUxlV0qY+ lQ508aF2ZsMEUIJsZxSDXjy4iCS3ibNt3fha+PICfkWhsav0uXnPVr+b1Tju2hNKQjlbB7 Wq0mTxtS/PgE2jQcCk48xd4oLwVPywg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.co.uk; i=@amazon.co.uk; q=dns/txt; s=amazon201209; t=1720531276; x=1752067276; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=j7xh8Fp0rGL+JNyIcONb4iIMzBYUq25BVxGN/Q+52J4=; b=IBGmho88qkJBK1FznX1Awd2MCCKuFbT+w20JujnLxX1RJto2mAXbmuGX EZkpIakmXASfZfgGtkrDebXGFXaVfAFCeCb+2XKAYdqNt3juF7EzhT5ox s+W4NsgRiO8dKvqsMie6mKdgmNafALOBl0W3+e/Q0m5hrJUk8/DClH3cM I=; X-IronPort-AV: E=Sophos;i="6.09,195,1716249600"; d="scan'208";a="432897833" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-9102.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jul 2024 13:21:09 +0000 Received: from EX19MTAUEA001.ant.amazon.com [10.0.0.204:54070] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.19.12:2525] with esmtp (Farcaster) id 9adfff71-39c0-4eea-be63-6a4028520ef9; Tue, 9 Jul 2024 13:21:07 +0000 (UTC) X-Farcaster-Flow-ID: 9adfff71-39c0-4eea-be63-6a4028520ef9 Received: from EX19D008UEA004.ant.amazon.com (10.252.134.191) by EX19MTAUEA001.ant.amazon.com (10.252.134.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Tue, 9 Jul 2024 13:21:04 +0000 Received: from EX19MTAUEC001.ant.amazon.com (10.252.135.222) by EX19D008UEA004.ant.amazon.com (10.252.134.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Tue, 9 Jul 2024 13:21:04 +0000 Received: from ua2d7e1a6107c5b.ant.amazon.com (172.19.88.180) by mail-relay.amazon.com (10.252.135.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Tue, 9 Jul 2024 13:21:01 +0000 From: Patrick Roy To: , , , , , CC: Patrick Roy , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 3/8] kvm: pfncache: enlighten about gmem Date: Tue, 9 Jul 2024 14:20:31 +0100 Message-ID: <20240709132041.3625501-4-roypat@amazon.co.uk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240709132041.3625501-1-roypat@amazon.co.uk> References: <20240709132041.3625501-1-roypat@amazon.co.uk> MIME-Version: 1.0 X-Rspamd-Queue-Id: DE90B8001D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: k99fyeygtkxwkzhb8witxhrc7d4s1juz X-HE-Tag: 1720531275-574605 X-HE-Meta: U2FsdGVkX1/cP7J3nEbTUxWdNf4dEPOQDIF9kLpbD/rvzjP5WkYeNNx3KZr/o75gG3OVbL2hASrEdZV+biLzJO0QAjX9HL/CZ0YGd3B8LkzaQBteNC1q/rOiD/TX295t7F8YyyXytY72aRqtEp+XRQ61DSJEe3zDsUuz+m18Oa4wh8eNaYTXCGV+Y2aug5V9ah0iNcSXH77y/+uhV1PM2OYwE09t06rgVm1RcXCSiVhmxpkzDUvY/t7vBv0HPmo9oTdscy7N8r4goq3TGYOAxF4ipPSNnUQtZL289Jx+58rPgi5IkY9aZrb2MgsDHf5qMk9qA0xI9TZLvNaTHz53/ewteeOyn48UbG9rJLQqrimSC5aVHdijaJwMfoLtNJ9nMf5hXt4AXywoKju4yqB46L0QT27Sq4iYro4R//nB2ioZhI0sp/Lo++34hzW3nQa2HBUtVaGHPjRJb4dNFWfXcAdyQ0+Ap3NuP7ijSOv969TExXhFiuj/OaR1Mj6KbcHud70Ebw/nXd9S33XrY85PHiMdHF4HgXYRrbatNXukSx6Exx03jX2kV/iHPj542nxGB5z19CaLe+WKEfqf2CjXDbe2qLodlPEx5r/1Np7mIDch8pQdmb40BYhrKWAB47UfsxqDEP5OxjoejOBjB/jzNNLIKEp/a4GEpTIuprWAYb7EjhHBwImQx3z5k/KOXqkwsDoXCqhBJSsJKtPuunmQr5OQfqRFJJVZ/Ty7VcvdUhHvM4O/cO+SxDL0IIoJ0EcmIUzmonLPcWv1tqZeyyzod9oc/7Tv3BZuHN0W6Pyz5ttD/plhN054b4aSsS9TZ6ugLYWeUmHft+CRjMzOrNBgBhAGQk4ZWkQ+jxm5PVE2zkjkDSUVo6QivTXYDle0fNMTl4w96Bb+Z7SGHtGzbl2AOLQK0NAs/P10rRgqsDgOiLfQdwL3129NQo6utCeb31kMMTXSDMzeN/7s52axUjs 3vU/ecgX o0U9TRhE2ud3/1o0/Ux8W17SoTboshc2NiuOhooAH2x4Mgk5rDKkhc/cuWD0+/39/KXKi2SPX9jXot3H7MKClXe5NEKHiIQCSWtxsUZ1CC+V7l8Y70WcdBIxTFXgfV9h2bama1436LPD+6yLANA3IGWHJCeiSWdFbiPQIn47X6YIDcVQ1vz+FVlAnZQFnKdfLmlBn25pr0KiVXqSvmpjMlVkp1HJC2WHOsFgKUyZ0nUvdrpT0wvvpfayWiBO/rsqhJi5ddWawty+yKycNZuT3EHpxcFlhuzqQqhIImQh3qyCT8d/Px+1JgOSDjRroq4GJlOByvakZW/5gRsInK5iCSbM8b6nqxNI/nZ466MY1HbYGQRs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: KVM uses gfn_to_pfn_caches to cache translations from gfn all the way to the pfn (for example, kvm-clock caches the page storing the page used for guest/host communication this way). Unlike the gfn_to_hva_cache, where no equivalent caching semantics were possible to gmem-backed gfns (see also 858e8068a750 ("kvm: pfncache: enlighten about gmem")), here it is possible to simply cache the pfn returned by `kvm_gmem_get_pfn`. Additionally, gfn_to_pfn_caches now invalidate whenever a cached gfn's attributes are flipped from shared to private (or vice-versa). Signed-off-by: Patrick Roy --- include/linux/kvm_types.h | 1 + virt/kvm/pfncache.c | 41 +++++++++++++++++++++++++++++++++------ 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h index 827ecc0b7e10..8f85f01f6bb0 100644 --- a/include/linux/kvm_types.h +++ b/include/linux/kvm_types.h @@ -70,6 +70,7 @@ struct gfn_to_pfn_cache { kvm_pfn_t pfn; bool active; bool valid; + bool is_private; }; #ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index f0039efb9e1e..6430e0a49558 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -90,6 +90,9 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len) if (!kvm_gpc_is_valid_len(gpc->gpa, gpc->uhva, len)) return false; + if (gpc->is_private != kvm_mem_is_private(gpc->kvm, gpa_to_gfn(gpc->gpa))) + return false; + if (!gpc->valid) return false; @@ -159,6 +162,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + gfn_t gfn; lockdep_assert_held(&gpc->refresh_lock); @@ -173,6 +177,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) do { mmu_seq = gpc->kvm->mmu_invalidate_seq; + gfn = gpa_to_gfn(gpc->gpa); smp_rmb(); write_unlock_irq(&gpc->lock); @@ -197,10 +202,19 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, NULL, true, NULL); - if (is_error_noslot_pfn(new_pfn)) - goto out_error; + if (gpc->is_private) { + int r = kvm_gmem_get_pfn(gpc->kvm, gfn_to_memslot(gpc->kvm, gfn), gfn, + &new_pfn, NULL); + + if (r) + goto out_error; + } else { + /* We always request a writeable mapping */ + new_pfn = hva_to_pfn(gpc->uhva, false, false, NULL, + true, NULL); + if (is_error_noslot_pfn(new_pfn)) + goto out_error; + } /* * Obtain a new kernel mapping if KVM itself will access the @@ -252,6 +266,7 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l unsigned long old_uhva; kvm_pfn_t old_pfn; bool hva_change = false; + bool old_private; void *old_khva; int ret; @@ -271,8 +286,21 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l old_pfn = gpc->pfn; old_khva = (void *)PAGE_ALIGN_DOWN((uintptr_t)gpc->khva); old_uhva = PAGE_ALIGN_DOWN(gpc->uhva); + old_private = gpc->is_private; + + gpc->is_private = kvm_mem_is_private(gpc->kvm, gpa_to_gfn(gpa)); + + if (gpc->is_private && !kvm_can_access_gmem(gpc->kvm)) { + ret = -EFAULT; + goto out_unlock; + } if (kvm_is_error_gpa(gpa)) { + if (WARN_ON_ONCE(gpc->is_private)) { + ret = -EINVAL; + goto out_unlock; + } + page_offset = offset_in_page(uhva); gpc->gpa = INVALID_GPA; @@ -316,9 +344,10 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l /* * If the userspace HVA changed or the PFN was already invalid, - * drop the lock and do the HVA to PFN lookup again. + * drop the lock and do the HVA to PFN lookup again. Also + * recompute the pfn if the gfn changed from shared to private (or vice-versa). */ - if (!gpc->valid || hva_change) { + if (!gpc->valid || hva_change || gpc->is_private != old_private) { ret = hva_to_pfn_retry(gpc); } else { /*