From patchwork Thu Oct 10 18:23:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830666 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BAF21CEADD for ; Thu, 10 Oct 2024 18:24:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584693; cv=none; b=adJL5rfLgOAJSeHM+qdvAgy3GUJK4GLOCFJDT6tJbeOM4bhfi8hELdcAy1HTOEfGKqkWo1KP2tztPC+yDS+mvjD7PNd1HZgqVWaZOvdio6iEjCBFd8GSnu6A0KQbXCgBVlOudsOorf4Z32e87crx8MX3MOvoQWY+W4BSWRNWc74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584693; c=relaxed/simple; bh=VeqAt9xsRyQYjwTyqnYEHFfSJqQO2zWoId1ssBiaEMk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JnCCt7zIB1e1jz15QP05RZX0DAewyf5YMAw4+VkeEXkNKaSPZ4zNXirKsd0mFIM8tWcnktAcztFuQMBpAZCoASG4TgB4AdRUsxcfHaynwNTkUaOY0S+E9xCSABZiAMJv+zONcNDa+zRUEBh3nX4VHjGRtG64Dncjv94tEJ9VX2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=c6ulq1H/; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c6ulq1H/" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2ebab7abfso157716a91.0 for ; Thu, 10 Oct 2024 11:24:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584691; x=1729189491; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=c6ulq1H/+SVkO3snwnEpIDv6c51AZwFPT1LwqFNCVv4NoGTlUIwMtlx2CGv4OolC4z 5koadCL6jvV5FKPkcR7h2EcqLSGLIrxPFSpXWBBtr1Q7Qs67rbWfXAkZuygIXztR30jX vinelIxIHIA9mCJvKaK123LtWbg1NOboLMeiKUh0EpYwOSI95YlMnIXDtfqepFmlcWAS CnV3G1l9SuTBGmxsijDu+22rWmksmHX3fhLSJfqBck7S6jAczfD9MKD7ErhknpjpLCXD TxRiyYgzfFwgfGEmjn40VJXaAyNZX1rswlRL/erQ/34CTu9kx47GkXzSb8i1IPcncetk Akrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584691; x=1729189491; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=rcQJmXxraaArxc0aos0VMWHdJSYEMelyCJIMYx9Xux9uJ9cWZcdX6B0prMU82ZgfjK 8SJOc18Beqrg7opOB8zoOC2J2zSCz9cd0mYgBCApOv7aqnD6kFWRknWsoI8v0ItegQep +Z91LBPGYxhOu2RKKkQcMnvWEq2QJSIVTeBrdVw7tkn4h17mBP2XML7ZFNmKcu9Umqz4 iYUQZSPXMI0Qj89vXfZuYeiToHBxjc4KrOYc9m/ox0nXfgHuiIChG0BXF6WiDbwGOLqh hM1JOuv5qeaqy5MvDmxQczHhhxXI9xBu+Scx0ImkghC7I1CEThE5BmufowQcajH8Y5gC e8aw== X-Forwarded-Encrypted: i=1; AJvYcCX7JHfvNBRpPr3LJKOoLZ+Jcl01cQNLoKvzQpHv+Utav6HB/iCK+2RbodrbQZRAPkZpguYAASDVyngy@vger.kernel.org X-Gm-Message-State: AOJu0Yzf+re5t3LwZTHDdJq1slcgMcIaK59W0UIROLJzfQKDm9XsU4rT cxBRYWwa/Y/2Fml7XeTQ2/dENcAETMYJDqIaDn1lEEoIWnxnbJXbH1dQoDTXloHoGbU7NwiHf/t F3A== X-Google-Smtp-Source: AGHT+IG89v4lIFRhnNqxe9hFvIIxqDvldHGwH//6ib2L4odxYyk8SJA2JGOjXIHuxBeh8ylp7HgA6abaBbM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:eac4:b0:2e2:8f4d:457 with SMTP id 98e67ed59e1d1-2e2c80b5015mr7266a91.2.1728584689990; Thu, 10 Oct 2024 11:24:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-2-seanjc@google.com> Subject: [PATCH v13 01/85] KVM: Drop KVM_ERR_PTR_BAD_PAGE and instead return NULL to indicate an error From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Remove KVM_ERR_PTR_BAD_PAGE and instead return NULL, as "bad page" is just a leftover bit of weirdness from days of old when KVM stuffed a "bad" page into the guest instead of actually handling missing pages. See commit cea7bb21280e ("KVM: MMU: Make gfn_to_page() always safe"). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 2 +- arch/powerpc/kvm/book3s_xive_native.c | 2 +- arch/s390/kvm/vsie.c | 2 +- arch/x86/kvm/lapic.c | 2 +- include/linux/kvm_host.h | 7 ------- virt/kvm/kvm_main.c | 15 ++++++--------- 6 files changed, 10 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index 7b8ae509328f..d7721297b9b6 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -645,7 +645,7 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) int i; hpage = gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (is_error_page(hpage)) + if (!hpage) return; hpage_offset = pte->raddr & ~PAGE_MASK; diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c index 6e2ebbd8aaac..d9bf1bc3ff61 100644 --- a/arch/powerpc/kvm/book3s_xive_native.c +++ b/arch/powerpc/kvm/book3s_xive_native.c @@ -654,7 +654,7 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive, } page = gfn_to_page(kvm, gfn); - if (is_error_page(page)) { + if (!page) { srcu_read_unlock(&kvm->srcu, srcu_idx); pr_err("Couldn't get queue page %llx!\n", kvm_eq.qaddr); return -EINVAL; diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 89cafea4c41f..763a070f5955 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -661,7 +661,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa) struct page *page; page = gfn_to_page(kvm, gpa_to_gfn(gpa)); - if (is_error_page(page)) + if (!page) return -EINVAL; *hpa = (hpa_t)page_to_phys(page) + (gpa & ~PAGE_MASK); return 0; diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 2098dc689088..20526e4d6c62 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2664,7 +2664,7 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) } page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (is_error_page(page)) { + if (!page) { ret = -EFAULT; goto out; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index db567d26f7b9..ee186a1fbaad 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -153,13 +153,6 @@ static inline bool kvm_is_error_gpa(gpa_t gpa) return gpa == INVALID_GPA; } -#define KVM_ERR_PTR_BAD_PAGE (ERR_PTR(-ENOENT)) - -static inline bool is_error_page(struct page *page) -{ - return IS_ERR(page); -} - #define KVM_REQUEST_MASK GENMASK(7,0) #define KVM_REQUEST_NO_WAKEUP BIT(8) #define KVM_REQUEST_WAIT BIT(9) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 05cbb2548d99..4b659a649dfa 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3078,19 +3078,14 @@ EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - struct page *page; kvm_pfn_t pfn; pfn = gfn_to_pfn(kvm, gfn); if (is_error_noslot_pfn(pfn)) - return KVM_ERR_PTR_BAD_PAGE; + return NULL; - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return KVM_ERR_PTR_BAD_PAGE; - - return page; + return kvm_pfn_to_refcounted_page(pfn); } EXPORT_SYMBOL_GPL(gfn_to_page); @@ -3184,7 +3179,8 @@ static void kvm_set_page_accessed(struct page *page) void kvm_release_page_clean(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; kvm_set_page_accessed(page); put_page(page); @@ -3208,7 +3204,8 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; kvm_set_page_dirty(page); kvm_release_page_clean(page); From patchwork Thu Oct 10 18:23:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830667 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F8871CEEA6 for ; Thu, 10 Oct 2024 18:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584695; cv=none; b=WPliJmmDeCbNhFHEYN3NBTSweEi4HpNUVE+NtQs4rlRRZoWPvXMa2OqNkZCzyyfG9RU5UbIiaDIwQBXSmkGxuXsiCOu43LxGmJi8RnXkvRO1AnwbR/m3ukukpnhmbgpjlf60Z7MYIDFe5lGivg3RTi5Fxye19zUtYX45x1uRuhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584695; c=relaxed/simple; bh=GKQ1+9zVatIXCwDEsM6DNkbKcmF2enzdpw2dYi1+r2s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=auVa3MOmcK593IE1U/A3HzVDtADJou73M0JSjDvHlKQ3OU7EFLCjJx7SxSI6RWzXl/qWKl2fqxByza5hBU1V9nsCEhhTTtD+jNeP0D/LRvLOfe3s0DMYNIHSfNUXTdz/wPLohgLokNt8ac468m6BDz2uMNrX3wNVTKpXqmaNhgc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LbsR3s+e; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LbsR3s+e" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20b6144cc2aso12164455ad.0 for ; Thu, 10 Oct 2024 11:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584693; x=1729189493; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=LbsR3s+eI6WfEnRbM2uT3vT6R0rDFtwfUQWzpB0nPc/rlV2FAnol+zYCnI0BRzeuc9 6dvAAbxGV4LpiWM+0lW7vLCrRS/LH1TIj9aZ6qsziigEi04cwRAF6RIz64LkI/oouB/x v8nk2+H9lL8+D3jP+Pb7CYde9KT6zgK6VQyFbvnUfqmJel6dqRVM0mYatQEmtKjpv3I7 n6sf8oQlLqPt2heDQ2EePF2CK5+5jZiCSmRGtjwlxxgMXznVLy1FP6MirDNLNTbto9Ys ZOZtRZ1DreJkK526CJoZ1fyGnLu5fiFs1pj6y4ni5JpaLLTAkcgHbH+XaO5ny1q8AxUS Cliw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584693; x=1729189493; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=SvtMYghLjXS6te2r44ZzBOhW3ee+oS+BBjXa9k2L6pyJXp8HD3VDEf/CQtFneVSeo7 Gl7Y/50Fm3LB2BbNHoGgby11SMDx5mlkI2n5TWnvYw99G/t9d/jwZx7WWk2cEYGhkQ2k 8hcAAp+Wgd5f53RmBSY52BncTptFMEBnUWfu3/KGMUC/lDslcZlRBtqu4BOyCp16qrO5 o75OECNCTEp+7x2hmuOmU3m1K61CLtJLCL5qua2u+ycxXURSXJQiSrof2V5CpjmK1GUe MmhwL6Tw+GMqOk5anFNaw8QkJFTy2MVMC9u8FKXUs8FXv5YbgoZet1EXV1I4ii+sIeiB LeVQ== X-Forwarded-Encrypted: i=1; AJvYcCWKVuKXsAfOuQ56UZ2y7Tjzwxa3W3KSLEO2AtbtXyULqeWx8RU0pWR5svjODwY8srYra/mi7AW1/TmT@vger.kernel.org X-Gm-Message-State: AOJu0YxR2kJSP3xnzYmQd305cDUkMcJTr8sQ5F+okGAa0H2p1Zs45bMs lGqW6cuY2tgI46Pu0pPE8/jVKofMHF+/CFOlHdSZOYZfxlMdkUBI8KTLtu/9lcIA+SkdTPgQWmE pDQ== X-Google-Smtp-Source: AGHT+IHgBvu4/6UpOEaDnS659CK1Rn1/E5r8mTk00ZsUtMRUIXJafj8Ignnkw5WxXqtPYIrr50vXY4UIGHU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ff02:b0:205:58ee:1567 with SMTP id d9443c01a7336-20c80362278mr43175ad.0.1728584692824; Thu, 10 Oct 2024 11:24:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-3-seanjc@google.com> Subject: [PATCH v13 02/85] KVM: Allow calling kvm_release_page_{clean,dirty}() on a NULL page pointer From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Allow passing a NULL @page to kvm_release_page_{clean,dirty}(), there's no tangible benefit to forcing the callers to pre-check @page, and it ends up generating a lot of duplicate boilerplate code. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4b659a649dfa..2032292df0b0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3179,7 +3179,7 @@ static void kvm_set_page_accessed(struct page *page) void kvm_release_page_clean(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; kvm_set_page_accessed(page); @@ -3204,7 +3204,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; kvm_set_page_dirty(page); From patchwork Thu Oct 10 18:23:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830668 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB2DB1CF7C3 for ; Thu, 10 Oct 2024 18:24:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584697; cv=none; b=dKCTxD+WnxTlm/gEaaDPd84RaflOjngcMwhEFCWg4/LH11Ga5Etp68qo863+imQ7mWWPNdRBeYF5pJfZliyYZzd1x1c/uQxS8GvzBlXwRECHzgDBFA5xq6mH7ILpqi2OYLusmKKfVkX9yUGk1be1t34YLOlZAvqzhdAh0Jt1QbY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584697; c=relaxed/simple; bh=iBKdL35kBHi2YuWkjigWDtnk5mPkldKXaU7GHff52mQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uonubBpkXoXIME0P0qlY+O8o75NWNYdLPCAS7MFLAhrXPympCleMrgjulSKWBNlNw9IFUJRp/Ye54OY6CPGua5hn92ozFW4IMJOv5G3jbdwrHjIII08qq9FmzhE5Mxucmq2F2CeribrWgWwigvPc3tXV3EBiB+FfqxapEHE5GYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YefjeM7N; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YefjeM7N" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e284982a31so22414817b3.3 for ; Thu, 10 Oct 2024 11:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584695; x=1729189495; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=YefjeM7NlL5WJTyheSu3K4yw2vp2CkS7uAx7Z40RaC/a/T9mf5Ut339LIWtkQ2XuXQ 17CbefMXqiqBMTUljRvZMHgszGwbaPFU/tkQky5Clu/5W9+/PGYR9hUCa+QIZ1madUv6 GhU3c/TE8J0FNncSSvv8YuqrNrxaNbhW9YiVKnl29law8WUX4fWW35jK/IILY2eu6SQ7 snOZPsHIyng5WefoSyBjOo1ogSwiFFA5eg3ef7Npa6sKnfD/cEcNQbDShB+D61sKl22R AeIaMhHY8Bfyfnv2ZxOyGX3QoGGAPDZXoT3sLgxyIdmnWJQG6fwCEFa2beItHdM20glS GA2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584695; x=1729189495; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=t6tTgifZUu6UfzFlMP7A0hae9LPGiHmFrdl/XvBDlREgEfSVGUCal77Fvim3R3eN+3 5k/FuFW8MxIWKnXdz8vC7+IBMyZS/EE2thcPpPiIyH1/6dWbMsroKeL7pFJ/kwNbQYw6 kuHiH2xfIhbXFWn+KSFObuNYGNEee0KLCC/+58UzX0lAIuHdUJVbaCFdxTbhSP7VCkbI HXtANwWbuvOUFlC0Tn56EiVwRMW01g5Zn8Y2EW5k3cVUinFRbNllIzgsy/0C6POQBAKY l84plzxatKA5Xt7s0Hk6hOCjmiD5+7VN3Jjofv4W4sqIxT8SWCJM3gFnuq9vFUMyXqin 1oQA== X-Forwarded-Encrypted: i=1; AJvYcCVCtXTyLsahC3jHe28UEPaOdyoQYl1CMQ57oAS6mRJy22aRNmEHkx53aikPeBs1Pn8MFY9Osxrln7zv@vger.kernel.org X-Gm-Message-State: AOJu0YyE2AAo4OhhWjgHslmcxzlvVIY9/1O803PfM6yId5iKd4wW+yha 9OEp5cFS8Q8PVlbfmW2zwM/jrBKLXeNWTI6IB4H/fjiFSHEOxRGcpvL4mtOy7sAN4VDP6/vV5za XqQ== X-Google-Smtp-Source: AGHT+IGz6mODxZTH5agCMYiGZey+8PwFJu50se4JqpXBNG0SwyKrN9TWGGQJqqmZYCdc2Fnc7eka1IGyLLE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a81:ad16:0:b0:6c1:298e:5a7 with SMTP id 00721157ae682-6e3224662ffmr1632867b3.5.1728584694780; Thu, 10 Oct 2024 11:24:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-4-seanjc@google.com> Subject: [PATCH v13 03/85] KVM: Add kvm_release_page_unused() API to put pages that KVM never consumes From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Add an API to release an unused page, i.e. to put a page without marking it accessed or dirty. The API will be used when KVM faults-in a page but bails before installing the guest mapping (and other similar flows). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ee186a1fbaad..ab4485b2bddc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1216,6 +1216,15 @@ unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot, gfn_t gfn, bool *writable); + +static inline void kvm_release_page_unused(struct page *page) +{ + if (!page) + return; + + put_page(page); +} + void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); From patchwork Thu Oct 10 18:23:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830669 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A94AA1D0DCA for ; Thu, 10 Oct 2024 18:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584699; cv=none; b=J0SA1GQ4MPFjhUGXpN04GvwLekriZYc06iWNTaH6kf5Zlj1JaJqxVyAlDl93oRZuwCXrgpgz/hhNEMrgFkBXYv8VJmosdQOQ+01oEOeqBOFmppasHTuzO0LWMVDsyhvcz1cc9/ZNLRyyVonQF5ZdM3iR02GNj4VFzPsKBX2VymY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584699; c=relaxed/simple; bh=/+UXWBjoqTQ+9Qx4e881Uho0Hr1R01lQsdRPnyHLl9M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Qlu/VQHnx+cINKpJrZg/JMUtqkwpJaXChuzMPohdU6av0cSmEdShrC8chyXSsoFOHlNA7vOL4RJgljdtF5bb4QyTvXrsFCpZf+rlpwOZnSeBXiLr9huIliFrrVxQndL9LDL1/PglcY1V2EGG86D5ta9iXzhlL3x7a0wUobRbvS4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=f+oxcvBw; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="f+oxcvBw" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e26ba37314so24841907b3.0 for ; Thu, 10 Oct 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584697; x=1729189497; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=f+oxcvBwyIf0CPK4Z3H2v2Mp1oKcg8M4xtBPnL3mEbTKwPFddA2gYnQvOwqwhMPqcs 6tTJPsf3R4LXc1m3EABExF2dXv+LHXuszre/1aFscEjO6qmtW3xxqqIATZx3rvdG8rsT 1J3bs69TevQpiRIDsRnNKqiW/W7gMd1WhJDESUTWDMC5TzfnghveyjVNiiFf2Y1HjFiQ vPehYSYcIUDZNDcFnYRqNJsZTdr7XltICzxxbDvDzqP29NLHv3HJTa/ykSCOdCs7Xdvo siw+q8s3hy1UsZky6mk+tSYSCcSAXsDzMU8KrWmL1toG+e2nbzaxdcSmAXg/WDWx5nyc 3r6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584697; x=1729189497; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=kDeVhipCsRm+yedvj0jhg7sVvK061Zw7W0Os4jxCKFPOgVicmbKm2HOUTsBCQNf/AR YKx/rijaOzPLTaemsDmeFOIf37QV+SE67Ls+nNRVFv8avtkft68kz3xpJMgu3FkxrJ7X LA0NtpyycSUdxI04iPA6egbKIWe1eUtdPijIITRaeRxOzsLaIMLu5yzm2DVnTtCgJXHY Yxqt3cueeDl3R4SauTk/OgTK3wOOfVHAszy9G/ak8dcFS8XMuL+gbn+Rwhi12UKM4d9L dXE1r2aVygrrs549kZNE/OQqD+Tqfp2/S0bITmS9W2ZPWgkah0jP0eH229uLg/SRaj/j 038w== X-Forwarded-Encrypted: i=1; AJvYcCVyEXxZqWuaMNTGFSnAgnMnfohvtenBjxTgf0tD48vcpgWkBZHgghxGVoVZeOqRMyrRZ1Wf9n5a/BOU@vger.kernel.org X-Gm-Message-State: AOJu0Yxa/NrOOzw8FndNjIEp4DdPg5+0QoWKVf9+wWgG8B2y8rBexvHq Rmn+6trRQeCzHYkQc5Hd3MMtOIOCrDNR9yEnY6Cr0vltTfmPPo9qzHj7hkskQT6cCBcBQpiwJHT C6Q== X-Google-Smtp-Source: AGHT+IGrULPD5dT5NWC9dFAfgTsoL9JIWGzHU9t8+6e7flVyO5QHPCRlr/jDUZCu+nhXBHYl+WGzejB3KuQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:e486:0:b0:6e2:371f:4aef with SMTP id 00721157ae682-6e322168931mr204147b3.3.1728584696651; Thu, 10 Oct 2024 11:24:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-5-seanjc@google.com> Subject: [PATCH v13 04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Apply make_spte()'s optimization to skip trying to unsync shadow pages if and only if the old SPTE was a leaf SPTE, as non-leaf SPTEs in direct MMUs are always writable, i.e. could trigger a false positive and incorrectly lead to KVM creating a SPTE without write-protecting or marking shadow pages unsync. This bug only affects the TDP MMU, as the shadow MMU only overwrites a shadow-present SPTE when synchronizing SPTEs (and only 4KiB SPTEs can be unsync). Specifically, mmu_set_spte() drops any non-leaf SPTEs *before* calling make_spte(), whereas the TDP MMU can do a direct replacement of a page table with the leaf SPTE. Opportunistically update the comment to explain why skipping the unsync stuff is safe, as opposed to simply saying "it's someone else's problem". Cc: stable@vger.kernel.org Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8f7eb3ad88fc..5521608077ec 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -226,12 +226,20 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= PT_WRITABLE_MASK | shadow_mmu_writable_mask; /* - * Optimization: for pte sync, if spte was writable the hash - * lookup is unnecessary (and expensive). Write protection - * is responsibility of kvm_mmu_get_page / kvm_mmu_sync_roots. - * Same reasoning can be applied to dirty page accounting. + * When overwriting an existing leaf SPTE, and the old SPTE was + * writable, skip trying to unsync shadow pages as any relevant + * shadow pages must already be unsync, i.e. the hash lookup is + * unnecessary (and expensive). + * + * The same reasoning applies to dirty page/folio accounting; + * KVM will mark the folio dirty using the old SPTE, thus + * there's no need to immediately mark the new SPTE as dirty. + * + * Note, both cases rely on KVM not changing PFNs without first + * zapping the old SPTE, which is guaranteed by both the shadow + * MMU and the TDP MMU. */ - if (is_writable_pte(old_spte)) + if (is_last_spte(old_spte, level) && is_writable_pte(old_spte)) goto out; /* From patchwork Thu Oct 10 18:23:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830670 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC8961E22F7 for ; Thu, 10 Oct 2024 18:25:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584703; cv=none; b=Z02ectNwIYWN+fhvGAygYE6CtffXZsTouVTvY06L4IUb/0ChNi7qcus6cl7RBucy9SoRAbXOtF2PNGiOx0MkFy3nUv2R9nEqYxERPvWqX5it6p471be1hw/lYNjqOKuqnhBdXaqr5lMLJBQtBemMFHSJasLR3kpTL2UD43ltKgA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584703; c=relaxed/simple; bh=uAiI5uzFAMXWntu4OPmR0EhFBczJTAI1tjFERMQUX4o=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WGWG7BiKRhcE98heRO2hbluyTADLHOQsuba4CaXFTMovUGh9RX87Vfk3yowLQCAaqeK2WQcATk2SBfaAJR/mQey1C3BCL0NE61uDzCFeCfxzi6kcwrsyCRFmvtHOVG3fLJXebPeCUpfaFiCboqLtnI26uXB6xmLaP3w/1mkz1E4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zZnV9RMn; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zZnV9RMn" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e290947f6f8so1842476276.2 for ; Thu, 10 Oct 2024 11:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584701; x=1729189501; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=zZnV9RMn+Omcj8cSLLQ1OO0nfSJBegsKEALf2M2ho0vTZxhMugmk84GGd9BSEy8lfi 5bXdzgRyXEp9OOV73dHe9K+/mvumfArmQlwdAIIf098Xo0lDHqhVeyJufcDy6upSTUCg eJa0l/phgkxFHgmSe5dUQRSSAhpuDVME8CPxLTUOLKDPpIH8StTUDV6Gj9Aj32V4pyB7 hUXbYov9HOCq3yRTRJegHiHEPel6bi7MvYy78Fmk4PNgStPCSNCbeQWI874/CL75Nylh b8mC6E7bFg+6jKx8vU6UyFAv3zFXpZ6ePXNmjV/tqGGnu0kMfhBsyo2PJWG6sUybPPoq 1hOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584701; x=1729189501; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=Yio4rfCshy34Xu+Ok2MHIO0p237u4NTdXjV/Djybo7/MmnIFH4WjYX0+S+wL/i94sJ MffT3Oy1fERy9HWRQ5+NKb7VhN1MmPCxImaQtfGzK2U9xKnRY9gaw19auHGKnq5b9TpW Y8C6CR6SMEFimZX5It4ppPRuUM4BcYXnjP5saJjPoTMgXin/LXITrMF488vBu3JNNaHc KPnwy/3Iy3F4E6uJeg1eRW2Hvpa7qEtRHuTl/5K/Yvg8apXCgpJ/Km6yvnfckZT0hNgv 5VXL+zKoE278+7IlKHMqBjL90tXv8y4peJKIlOAgVLQqoM/nRq5aquo7AhyKsVE7IjYK S+8w== X-Forwarded-Encrypted: i=1; AJvYcCXscWoRKf/9JA4Vxsv5S80mmbrRObeZicYD6PmNjFO/4vkJm4ksRTvo2VXnTJ5/pNGIzjeKE5wAUDAc@vger.kernel.org X-Gm-Message-State: AOJu0Yx1zhL6nKVi6cmj+6kZ3F+3XQWxXKDnBuVt2AfAY+QDGtT+hJj3 K+l8KuZ2kaTGjNF4ybDNG/b24inpce21r+lpgUiCXbD7akAWEJKBcyTpouk4hHPfATl6u0NBhbw cIw== X-Google-Smtp-Source: AGHT+IHQgMXiDY3ifHkkle2NgOnYNSR5nXtws6LnwLsJ4J6Fqip1dtdD5j5lAqBgp/4MJdCcEXFALInKEMI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:69c3:0:b0:e1a:9ed2:67f4 with SMTP id 3f1490d57ef6-e28fe4313ffmr4949276.2.1728584700890; Thu, 10 Oct 2024 11:25:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-6-seanjc@google.com> Subject: [PATCH v13 05/85] KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Treat attempts to prefetch/prefault MMU SPTEs as spurious if there's an existing shadow-present SPTE, as overwriting a SPTE that may have been create by a "real" fault is at best confusing, and at worst potentially harmful. E.g. mmu_try_to_unsync_pages() doesn't unsync when prefetching, which creates a scenario where KVM could try to replace a Writable SPTE with a !Writable SPTE, as sp->unsync is checked prior to acquiring mmu_unsync_pages_lock. Note, this applies to three of the four flavors of "prefetch" in KVM: - KVM_PRE_FAULT_MEMORY - Async #PF (host or PV) - Prefetching The fourth flavor, SPTE synchronization, i.e. FNAME(sync_spte), _only_ overwrites shadow-present SPTEs when calling make_spte(). But SPTE synchronization specifically uses mmu_spte_update(), and so naturally avoids the @prefetch check in mmu_set_spte(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 3 +++ arch/x86/kvm/mmu/tdp_mmu.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a9a23e058555..a8c64069aa89 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2919,6 +2919,9 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } if (is_shadow_present_pte(*sptep)) { + if (prefetch) + return RET_PF_SPURIOUS; + /* * If we overwrite a PTE page pointer with a 2MB PMD, unlink * the parent of the now unreachable PTE. diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3b996c1fdaab..3c6583468742 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1026,6 +1026,9 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, if (WARN_ON_ONCE(sp->role.level != fault->goal_level)) return RET_PF_RETRY; + if (fault->prefetch && is_shadow_present_pte(iter->old_spte)) + return RET_PF_SPURIOUS; + if (unlikely(!fault->slot)) new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else From patchwork Thu Oct 10 18:23:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830671 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B6501E3773 for ; Thu, 10 Oct 2024 18:25:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584706; cv=none; b=uuMOkXMZv3I8vYavW+49wNywsoNnt1wzsL17htjFVk/AHzJweNRdYaIxeOKiSybv4AZgQO8trbSZ1X1LE2/DYE6l6UuWHWGPiFDtmPxm2YiJi+SR4MHV37KHuD1eqGgrjFGka6OEBjJkLoOI2t+kQbeiLS38VfwjbLREAR5X84I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584706; c=relaxed/simple; bh=V8Pz/YRM4TKMSvyoWeXbxNdYc7yxSwxWSxDHm25abTE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oljMALxgBUXeqoj6pIHuFzeq9aIyR1N9vd/2hJvcXn0PuocW8WMhJ8t1Bvm2PBJH4oALaj5WY+unUzd7kAIHcSsBfmk+KLd/FL+CyHHWI3vyI7S4tf67zKlbxw8vXv4FkvHQVXpGzGfoIXMWIk7c04dg8OYHuoN+s9AQ3pBTI+I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2Sdd9oap; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2Sdd9oap" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e165fc5d94fso1874978276.2 for ; Thu, 10 Oct 2024 11:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584704; x=1729189504; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=2Sdd9oap+q0MyhyBJfN/XXaLaeugLyBeVKfhNG2OdkEb/qrt0NkuxHYVoDsDdUZ9e1 pmV7SmQhjF9bAwgYB0J8zvebqLiGSyv20F8v9j0thW0q6ArX1s4TwXm8L3IHPzem3f8y DwxbUzTbCpK28hZiB1VxvTVS9uIMZOjdMOhgijGcTvQ/wCKV0SUyqlU1dK5YJ8IjBDbg b/lM5r6hSEmGcKJbLuF7of6iZwO86ElPJvMvme5CnHpYMePTLHn6R0KeLtgTbtl3JJy2 5t78epXtg7vnXBYurXwLP6RwMrqQnpppEpVK40dF3oMv/Bjakbx1TqmR49X4kE7/FaxV fqbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584704; x=1729189504; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=Usi3FGl79ubrZ1NiJnhVLaWS4bA21/LO3r1dGomxeK69R682FRVZKY2VCYsHSYmsKF NwOBRxxdOq1A2/IkpteeKa5OI3PB2EvfkkbN9OT0XrfJXva/di5jdaVMMasuaCPYa2rT dv6rRm+8LaADEeo6lPbDycZJl/dv3pQIaMz5pPRqsnl6c1qk6skQxKT0UWRVMJDZagAe eyjmeveNpRNQkp3Z3ahTi9UurVyBp8ZYIuN7JYn+KkoY1R3/UKwRrGJiHhkPKWpr+dgz YCdxtTG8ITK8k26sIRaMH47O5XV/0YXmyCwVt7VwpuX+bjQiAPqI24oTugXLCx1gzY99 oPvA== X-Forwarded-Encrypted: i=1; AJvYcCXUkPaXcyXgHsS1PqL3/3wKOBAn6l5m9wASXObSJddWnotH10W29t/tjEW9iizSORZUqJ3H4VRBywIf@vger.kernel.org X-Gm-Message-State: AOJu0Yxkn6bM2Wdx/NNcdJw6gwJO94Kjht6bSIixKG6/Yyp+f8HSNADl e/vz8gOqXcTMYaf5laWN4ZptYAQE6UdAxuX4ObELQ1OlFrjw917ggUSQDnhM//ErVAZJDUrrMZ1 CEg== X-Google-Smtp-Source: AGHT+IHLS32ZnUy4bY0gdVqRjJSlR5N1zmb3UxYk5drKGwVtxhq/Y8NIZIwMp1duovDzYwXPQsOfO5WBu4o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:1812:b0:e24:9f58:dd17 with SMTP id 3f1490d57ef6-e28fe32f042mr66754276.1.1728584702932; Thu, 10 Oct 2024 11:25:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-7-seanjc@google.com> Subject: [PATCH v13 06/85] KVM: x86/mmu: Invert @can_unsync and renamed to @synchronizing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Invert the polarity of "can_unsync" and rename the parameter to "synchronizing" to allow a future change to set the Accessed bit if KVM is synchronizing an existing SPTE. Querying "can_unsync" in that case is nonsensical, as the fact that KVM can't unsync SPTEs doesn't provide any justification for setting the Accessed bit. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 ++++++------ arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.c | 4 ++-- arch/x86/kvm/mmu/spte.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 4 ++-- 6 files changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a8c64069aa89..0f21d6f76cab 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2795,7 +2795,7 @@ static void kvm_unsync_page(struct kvm *kvm, struct kvm_mmu_page *sp) * be write-protected. */ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, - gfn_t gfn, bool can_unsync, bool prefetch) + gfn_t gfn, bool synchronizing, bool prefetch) { struct kvm_mmu_page *sp; bool locked = false; @@ -2810,12 +2810,12 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, /* * The page is not write-tracked, mark existing shadow pages unsync - * unless KVM is synchronizing an unsync SP (can_unsync = false). In - * that case, KVM must complete emulation of the guest TLB flush before - * allowing shadow pages to become unsync (writable by the guest). + * unless KVM is synchronizing an unsync SP. In that case, KVM must + * complete emulation of the guest TLB flush before allowing shadow + * pages to become unsync (writable by the guest). */ for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { - if (!can_unsync) + if (synchronizing) return -EPERM; if (sp->unsync) @@ -2941,7 +2941,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } wrprot = make_spte(vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefetch, - true, host_writable, &spte); + false, host_writable, &spte); if (*sptep == spte) { ret = RET_PF_SPURIOUS; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index c98827840e07..4da83544c4e1 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -164,7 +164,7 @@ static inline gfn_t gfn_round_for_level(gfn_t gfn, int level) } int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, - gfn_t gfn, bool can_unsync, bool prefetch); + gfn_t gfn, bool synchronizing, bool prefetch); void kvm_mmu_gfn_disallow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn); void kvm_mmu_gfn_allow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index ae7d39ff2d07..6e7bd8921c6f 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -963,7 +963,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int host_writable = spte & shadow_host_writable_mask; slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); make_spte(vcpu, sp, slot, pte_access, gfn, - spte_to_pfn(spte), spte, true, false, + spte_to_pfn(spte), spte, true, true, host_writable, &spte); return mmu_spte_update(sptep, spte); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 5521608077ec..0e47fea1a2d9 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -157,7 +157,7 @@ bool spte_has_volatile_bits(u64 spte) bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte) { int level = sp->role.level; @@ -248,7 +248,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, * e.g. it's write-tracked (upper-level SPs) or has one or more * shadow pages and unsync'ing pages is not allowed. */ - if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, can_unsync, prefetch)) { + if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, synchronizing, prefetch)) { wrprot = true; pte_access &= ~ACC_WRITE_MASK; spte &= ~(PT_WRITABLE_MASK | shadow_mmu_writable_mask); diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 2cb816ea2430..c81cac9358e0 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -499,7 +499,7 @@ bool spte_has_volatile_bits(u64 spte); bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte); u64 make_huge_page_split_spte(struct kvm *kvm, u64 huge_spte, union kvm_mmu_page_role role, int index); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3c6583468742..76bca7a726c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1033,8 +1033,8 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else wrprot = make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, - fault->pfn, iter->old_spte, fault->prefetch, true, - fault->map_writable, &new_spte); + fault->pfn, iter->old_spte, fault->prefetch, + false, fault->map_writable, &new_spte); if (new_spte == iter->old_spte) ret = RET_PF_SPURIOUS; From patchwork Thu Oct 10 18:23:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830672 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A6621E491C for ; Thu, 10 Oct 2024 18:25:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584708; cv=none; b=HAIFfF8QLbDunYQitcXaJ48FHvdoO2FZNs/HdAq0nLqzklkaM8i0P2e3MApDe/uk0h2DXCaYpxwz0PcHrkyRv3h+IqpDwx7xGLbpVqQ96OrpPqRhs1faZhGwk5wvQ5n8GRd/yghLnrs/IRiH6A+85TDkU+xC5c/mhU3x99s8UkE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584708; c=relaxed/simple; bh=zZ6qnR3p2Ej0Et0EkgjdDF1APVKDfMggaSwqeFMbAB0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=omrUU+ryjAoDPuSSilnH8PXLjXPFF//z3wnnxFw21Y27UE95fxhMsz5c0vzuKSW9s2bMNOS3JCZ81yoCuL6J2g0Fuh6uzh8+xqHEyO+dazauaeF+nAFLe/0EUZYRw/e1nT3lyMOILSJIxrPzmKm8bKk5gCqPizemJWmM9KvcRSE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IDkmO6HR; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IDkmO6HR" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e295559c37so1096522a91.0 for ; Thu, 10 Oct 2024 11:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584707; x=1729189507; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=IDkmO6HRVmabgOlLjqCh29oQFT1bfBQbkp3F1tpeC5Cl5RH4yKJjEzu/fRdCPtY2LJ 8alxmLe/e6YwD8u37nx2cwxVWQGc05WHH8rhdocEO3m0ZILY2McAVy6C7ivAMxZD63lA 5rm83irlLcGgCTPE12ygB+5N8x12ZhG5hm21exX/zmXXDmPJm1UnrJS16qHVBYfIHr2O AobpACM+F+Sg7RjUSd+L/3V/uhbi11gyOc4mNHGNjscaAcXfUW6mW3Qzc8IqRRoIOJrz mQEaeoC3Oe07Txd1htL8K8SnhqjKEfbT3WNRYv4Ca+fTxlJk2zb0kfgWcERlJ9Sm7PG8 lEzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584707; x=1729189507; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=Z8Ric4ICKC1R+amEb0aDUXglJFs48TfQF3O7do/DDfcjo5RXge1Kbms/hV6ZQzYgJ/ NSNjZ3W8l3oFx0pfEhx/pjRidFTt4ItsywRnwOM8pU/+1nJwHIfoj8G0Oax63yTqhj/4 NI95apBoVGJslno8xyKUIzSLBH+gOa1F1TG3WWKD7ED4SPF0pU6BI7ceBKBrVY9eOPr4 oPTQxo1eR+ZZbiOqMU6XVeCzJA1oumtpff0daVHPyRWxpeKqmWO0aS7h2AoBNjVitVGp V40ghz29u1KdX2z61dLV7zCEoJp3c48C1LasCrc/Mx5cBnGZaiOjk57ra7Mmdfiue0Ug l/fQ== X-Forwarded-Encrypted: i=1; AJvYcCUHEnCz5Ddg6J6dnBnzylq93Aau21v0HrBuNvz33mSq7oASxO1xVl30HtQHJnRv2VLi05D82h5A2ZU3@vger.kernel.org X-Gm-Message-State: AOJu0YxZ6G/58/HtaiBmsoAVwgIs8iyAcn9vRRiJ/TNllG8hfeqLQ3wA L6XWSwLVAYdM36L50XZamGN6jomO0aHRB8liu6dE5FbVFTOiPXI/Mpy833Yz1AVXYpGhumdRL/t D/g== X-Google-Smtp-Source: AGHT+IH8UDk7ruln14ZDsSWqbj1uDcx7028dIle1wpc1As3ec3XNWN4cizl8I1jAsp9rsgJdiTjtCNc1iNw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fa84:b0:2e2:da81:40c6 with SMTP id 98e67ed59e1d1-2e2f0a524d4mr44a91.2.1728584705592; Thu, 10 Oct 2024 11:25:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-8-seanjc@google.com> Subject: [PATCH v13 07/85] KVM: x86/mmu: Mark new SPTE as Accessed when synchronizing existing SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Set the Accessed bit when making a "new" SPTE during SPTE synchronization, as _clearing_ the Accessed bit is counter-productive, and even if the Accessed bit wasn't set in the old SPTE, odds are very good the guest will access the page in the near future, as the most common case where KVM synchronizes a shadow-present SPTE is when the guest is making the gPTE read-only for Copy-on-Write (CoW). Preserving the Accessed bit will allow dropping the logic that propagates the Accessed bit to the underlying struct page when overwriting an existing SPTE, without undue risk of regressing page aging. Note, KVM's current behavior is very deliberate, as SPTE synchronization was the only "speculative" access type as of commit 947da5383069 ("KVM: MMU: Set the accessed bit on non-speculative shadow ptes"). But, much has changed since 2008, and more changes are on the horizon. Spurious clearing of the Accessed (and Dirty) was mitigated by commit e6722d9211b2 ("KVM: x86/mmu: Reduce the update to the spte in FNAME(sync_spte)"), which changed FNAME(sync_spte) to only overwrite SPTEs if the protections are actually changing. I.e. KVM is already preserving Accessed information for SPTEs that aren't dropping protections. And with the aforementioned future change to NOT mark the page/folio as accessed, KVM's SPTEs will become the "source of truth" so to speak, in which case clearing the Accessed bit outside of page aging becomes very undesirable. Suggested-by: Yan Zhao Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 0e47fea1a2d9..618059b30b8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -178,7 +178,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= SPTE_TDP_AD_WRPROT_ONLY; spte |= shadow_present_mask; - if (!prefetch) + if (!prefetch || synchronizing) spte |= spte_shadow_accessed_mask(spte); /* @@ -259,7 +259,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= spte_shadow_dirty_mask(spte); out: - if (prefetch) + if (prefetch && !synchronizing) spte = mark_spte_for_access_track(spte); WARN_ONCE(is_rsvd_spte(&vcpu->arch.mmu->shadow_zero_check, spte, level), From patchwork Thu Oct 10 18:23:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830673 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4361F1E5731 for ; Thu, 10 Oct 2024 18:25:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584712; cv=none; b=cyob2VfCzOmtOHCCxzodo1ITbcWSvGzshlyrIjYYPPvWLKscT+SB++lUjhkTEoVU0acIBf7carVDSMn79IxROIEO2fKgW5BcT27LnK01R/Dtw37ic9kyWkqWon9LGWzpjD2JgD2SL9x/6+aGxQTuzsDi4V3TVMRACHp43+7kWKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584712; c=relaxed/simple; bh=VkZarLC+Eu//G+0PeVKNSmckYM6o8fYMsW2puWWES3Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g20mxCQdv9FB44HkEXLAu5CiK5igxrY5WF6tKBaTMSgySAdqTfaRXpNmEnY9vwOK1eT53/U/DGPwxn59iq/uhT9oMegsJONMFftXgE77tm78aDnMMQLvmlKdMSHwVEdyRTiS519GQQ18LBz4lyWyV08prz42PzLMpw48C/J12yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Kbknugbz; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Kbknugbz" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e18b6cd304so1250232a91.0 for ; Thu, 10 Oct 2024 11:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584709; x=1729189509; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=KbknugbzdNUXpNoKARsIRLpx8PRqlV/FblB8LCRY3jXCEL8YQWp2+5e1SPcnpU1anA /Un1XGepnDYvRE8uMn1M8sNXDKThkOYuLMTMY4dvsf3kPUPmzir5ncOXQFn7+bv1p4hy H2jRB4kxAnETZzMSlESoEbGU/tyiL4jS3pm3WzUHXkieibovGZ1NPgTsV7+P+Cesmr1y Fs2xH6iupE8taJ7d1TWULgnebCuHG6dRZWs4+QPnv+cXQLB2p6C1WBwO4XLulqfa6dO3 cUBm6987ZwMn/Cpfa5BrC69Oqg5CqfZ6FFkwpWMqFmwwGgxbzBJ359J9aVrp/AaSSVAN 8DSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584709; x=1729189509; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=cD54EfdZ+KVDubCAX75bKsEvOubQZ9ZTLRoSjA6jd9ZNIKXgpsbAndj6hgLKQbdqk0 py1tWEb2aM/fWngRGFCbXhjQFvZy68C/3OEreixg2l0bhOPzy3XWC7F+gN+I0VbWXsFm ekvjaOHG3V0e5QbrolV0Ao0GyC3Q8OVUdwvtzarQlb+RvUYGG+MYn4MoFfK5T4no4caJ XU+EpnMUwTu37Ly9YnprIvDyc/zAo16J1q/D6y+5HpxiMj4iitzQ8Tb3clu6XJWPThVa qACnfC8TIcHBps8zf9oSFMKfuPExUg4cv/78zbeGKq9CZqi9VLHfYN3KjLWWWFRsiEej rXrg== X-Forwarded-Encrypted: i=1; AJvYcCXz3tNuFsdynaa2KSwfuKM51Ah+P6VlQD+uha8Ry5IfptkquKvLFAkMo7jur7MBWBTYoTjdFcGVIi/H@vger.kernel.org X-Gm-Message-State: AOJu0YyEavoeD2f9r4h6QScFEUbF7wOP1iAR1CEqiZik4YjhJbTqqeeN iiRipjV1qHSAmWgVjP+wWAE9tHu3OOxQAn997190VyIo7dBHIwHa1Z9xuPrXxJGyU5BrCssKgW5 RhQ== X-Google-Smtp-Source: AGHT+IFvOj1YmixVIm2giqrxikolLV70YR7/zWjxlJNmhfeSXdlwSACqKd64r4Is1alzH3q275ZQuugRXZo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:153:b0:2e2:af66:c33e with SMTP id 98e67ed59e1d1-2e2f0ae73f3mr37a91.1.1728584708343; Thu, 10 Oct 2024 11:25:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-9-seanjc@google.com> Subject: [PATCH v13 08/85] KVM: x86/mmu: Mark folio dirty when creating SPTE, not when zapping/modifying From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages/folios dirty when creating SPTEs to map PFNs into the guest, not when zapping or modifying SPTEs, as marking folios dirty when zapping or modifying SPTEs can be extremely inefficient. E.g. when KVM is zapping collapsible SPTEs to reconstitute a hugepage after disbling dirty logging, KVM will mark every 4KiB pfn as dirty, even though _at least_ 512 pfns are guaranteed to be in a single folio (the SPTE couldn't potentially be huge if that weren't the case). The problem only becomes worse for 1GiB HugeTLB pages, as KVM can mark a single folio dirty 512*512 times. Marking a folio dirty when mapping is functionally safe as KVM drops all relevant SPTEs in response to an mmu_notifier invalidation, i.e. ensures that the guest can't dirty a folio after access has been removed. And because KVM already marks folios dirty when zapping/modifying SPTEs for KVM reasons, i.e. not in response to an mmu_notifier invalidation, there is no danger of "prematurely" marking a folio dirty. E.g. if a filesystems cleans a folio without first removing write access, then there already exists races where KVM could mark a folio dirty before remote TLBs are flushed, i.e. before guest writes are guaranteed to stop. Furthermore, x86 is literally the only architecture that marks folios dirty on the backend; every other KVM architecture marks folios dirty at map time. x86's unique behavior likely stems from the fact that x86's MMU predates mmu_notifiers. Long, long ago, before mmu_notifiers were added, marking pages dirty when zapping SPTEs was logical, and perhaps even necessary, as KVM held references to pages, i.e. kept a page's refcount elevated while the page was mapped into the guest. At the time, KVM's rmap_remove() simply did: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); else kvm_release_pfn_clean(pfn); i.e. dropped the refcount and marked the page dirty at the same time. After mmu_notifiers were introduced, commit acb66dd051d0 ("KVM: MMU: don't hold pagecount reference for mapped sptes pages") removed the refcount logic, but kept the dirty logic, i.e. converted the above to: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); And for KVM x86, that's essentially how things have stayed over the last ~15 years, without anyone revisiting *why* KVM marks pages/folios dirty at zap/modification time, e.g. the behavior was blindly carried forward to the TDP MMU. Practically speaking, the only downside to marking a folio dirty during mapping is that KVM could trigger writeback of memory that was never actually written. Except that can't actually happen if KVM marks folios dirty if and only if a writable SPTE is created (as done here), because KVM always marks writable SPTEs as dirty during make_spte(). See commit 9b51a63024bd ("KVM: MMU: Explicitly set D-bit for writable spte."), circa 2015. Note, KVM's access tracking logic for prefetched SPTEs is a bit odd. If a guest PTE is dirty and writable, KVM will create a writable SPTE, but then mark the SPTE for access tracking. Which isn't wrong, just a bit odd, as it results in _more_ precise dirty tracking for MMUs _without_ A/D bits. To keep things simple, mark the folio dirty before access tracking comes into play, as an access-tracked SPTE can be restored in the fast page fault path, i.e. without holding mmu_lock. While writing SPTEs and accessing memslots outside of mmu_lock is safe, marking a folio dirty is not. E.g. if the fast path gets interrupted _just_ after setting a SPTE, the primary MMU could theoretically invalidate and free a folio before KVM marks it dirty. Unlike the shadow MMU, which waits for CPUs to respond to an IPI, the TDP MMU only guarantees the page tables themselves won't be freed (via RCU). Opportunistically update a few stale comments. Cc: David Matlack Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 30 ++++-------------------------- arch/x86/kvm/mmu/paging_tmpl.h | 6 +++--- arch/x86/kvm/mmu/spte.c | 20 ++++++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 12 ------------ 4 files changed, 25 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0f21d6f76cab..1ae823ebd12b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -547,10 +547,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } - if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) { + if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush = true; - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - } return flush; } @@ -593,9 +591,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) if (is_accessed_spte(old_spte)) kvm_set_pfn_accessed(pfn); - if (is_dirty_spte(old_spte)) - kvm_set_pfn_dirty(pfn); - return old_spte; } @@ -1250,16 +1245,6 @@ static bool spte_clear_dirty(u64 *sptep) return mmu_spte_update(sptep, spte); } -static bool spte_wrprot_for_clear_dirty(u64 *sptep) -{ - bool was_writable = test_and_clear_bit(PT_WRITABLE_SHIFT, - (unsigned long *)sptep); - if (was_writable && !spte_ad_enabled(*sptep)) - kvm_set_pfn_dirty(spte_to_pfn(*sptep)); - - return was_writable; -} - /* * Gets the GFN ready for another round of dirty logging by clearing the * - D bit on ad-enabled SPTEs, and @@ -1275,7 +1260,8 @@ static bool __rmap_clear_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head, for_each_rmap_spte(rmap_head, &iter, sptep) if (spte_ad_need_write_protect(*sptep)) - flush |= spte_wrprot_for_clear_dirty(sptep); + flush |= test_and_clear_bit(PT_WRITABLE_SHIFT, + (unsigned long *)sptep); else flush |= spte_clear_dirty(sptep); @@ -1628,14 +1614,6 @@ static bool kvm_rmap_age_gfn_range(struct kvm *kvm, clear_bit((ffs(shadow_accessed_mask) - 1), (unsigned long *)sptep); } else { - /* - * Capture the dirty status of the page, so that - * it doesn't get lost when the SPTE is marked - * for access tracking. - */ - if (is_writable_pte(spte)) - kvm_set_pfn_dirty(spte_to_pfn(spte)); - spte = mark_spte_for_access_track(spte); mmu_spte_update_no_track(sptep, spte); } @@ -3415,7 +3393,7 @@ static bool fast_pf_fix_direct_spte(struct kvm_vcpu *vcpu, * harm. This also avoids the TLB flush needed after setting dirty bit * so non-PML cases won't be impacted. * - * Compare with set_spte where instead shadow_dirty_mask is set. + * Compare with make_spte() where instead shadow_dirty_mask is set. */ if (!try_cmpxchg64(sptep, &old_spte, new_spte)) return false; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 6e7bd8921c6f..fbaae040218b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -892,9 +892,9 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, /* * Using the information in sp->shadowed_translation (kvm_mmu_page_get_gfn()) is - * safe because: - * - The spte has a reference to the struct page, so the pfn for a given gfn - * can't change unless all sptes pointing to it are nuked first. + * safe because SPTEs are protected by mmu_notifiers and memslot generations, so + * the pfn for a given gfn can't change unless all SPTEs pointing to the gfn are + * nuked first. * * Returns * < 0: failed to sync spte diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 618059b30b8b..8e8d6ee79c8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -232,8 +232,8 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, * unnecessary (and expensive). * * The same reasoning applies to dirty page/folio accounting; - * KVM will mark the folio dirty using the old SPTE, thus - * there's no need to immediately mark the new SPTE as dirty. + * KVM marked the folio dirty when the old SPTE was created, + * thus there's no need to mark the folio dirty again. * * Note, both cases rely on KVM not changing PFNs without first * zapping the old SPTE, which is guaranteed by both the shadow @@ -266,12 +266,28 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, "spte = 0x%llx, level = %d, rsvd bits = 0x%llx", spte, level, get_rsvd_bits(&vcpu->arch.mmu->shadow_zero_check, spte, level)); + /* + * Mark the memslot dirty *after* modifying it for access tracking. + * Unlike folios, memslots can be safely marked dirty out of mmu_lock, + * i.e. in the fast page fault handler. + */ if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) { /* Enforced by kvm_mmu_hugepage_adjust. */ WARN_ON_ONCE(level > PG_LEVEL_4K); mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } + /* + * If the page that KVM got from the primary MMU is writable, i.e. if + * it's host-writable, mark the page/folio dirty. As alluded to above, + * folios can't be safely marked dirty in the fast page fault handler, + * and so KVM must (somewhat) speculatively mark the folio dirty even + * though it isn't guaranteed to be written as KVM won't mark the folio + * dirty if/when the SPTE is made writable. + */ + if (host_writable) + kvm_set_pfn_dirty(pfn); + *new_spte = spte; return wrprot; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 76bca7a726c1..517b384473c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -511,10 +511,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (is_leaf != was_leaf) kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1); - if (was_leaf && is_dirty_spte(old_spte) && - (!is_present || !is_dirty_spte(new_spte) || pfn_changed)) - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - /* * Recursively handle child PTs if the change removed a subtree from * the paging structure. Note the WARN on the PFN changing without the @@ -1249,13 +1245,6 @@ static bool age_gfn_range(struct kvm *kvm, struct tdp_iter *iter, iter->level); new_spte = iter->old_spte & ~shadow_accessed_mask; } else { - /* - * Capture the dirty status of the page, so that it doesn't get - * lost when the SPTE is marked for access tracking. - */ - if (is_writable_pte(iter->old_spte)) - kvm_set_pfn_dirty(spte_to_pfn(iter->old_spte)); - new_spte = mark_spte_for_access_track(iter->old_spte); iter->old_spte = kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte, new_spte, @@ -1596,7 +1585,6 @@ static void clear_dirty_pt_masked(struct kvm *kvm, struct kvm_mmu_page *root, trace_kvm_tdp_mmu_spte_changed(iter.as_id, iter.gfn, iter.level, iter.old_spte, iter.old_spte & ~dbit); - kvm_set_pfn_dirty(spte_to_pfn(iter.old_spte)); } rcu_read_unlock(); From patchwork Thu Oct 10 18:23:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830674 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DA7E1E8820 for ; Thu, 10 Oct 2024 18:25:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584714; cv=none; b=igynxGBPGO1OwVyW7DSiullKQ16Lb52hWVr50oE/K2CGbs79CJcivF6mFzSgjAK2vpvDybQhOS35Q9nieeERWng4w5tXJ3DmWImfaVlYml3XB0MHKpC8BfGG1YVUxxUKYlK3aD9baFTpf7XCi1M/sa7N8zYA0KWWUfuXG+PzlCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584714; c=relaxed/simple; bh=sB5G3PQtHd5Wp6fJfSwA75+Ou8nv+Xyz2zynkW7I+sM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jH76Q2fjg6OdH1x+Yt45xym5m+84Vmt4FaCSKyIIDROAvjpwYdF3NTh3daxkDCQvjUsLPubuRTesgNbrnd6EmBLCWjgojqDGgKRQBiObyNcv+1D4uDIzMWrv0Q7TaGgQlzzYWbwDLUNaT7B9LXwVpNj8fyRAVNi/7cbryidl5B0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YFnRBGee; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YFnRBGee" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e1fbe2a6b1so23249377b3.2 for ; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584711; x=1729189511; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=YFnRBGeeymk+2pmMBdw0wl0Yy+Pdr75lMEEb46Qj2R/ZMELxdBUC0VhXMaFoLMRZyw lM5BZATdJOfXIHC7Ufn9e5g1CVYCsM89XBgpEbeWnjle3YVc0JrKT5IY3d3WFOKNc03i tQo8sG3IUbpyxFHlOMrb7ZHLDldQnJqNa2RjhIHLewNO7ZU5NmsXMw8G548D7wuNl5BX KAXxb5IgcpvzTluv7+tQS9iUf8SuC1dnEWalfACIcBmgOeNyW3QjD8rsI9do0p1/7fwV MXEidCQsM6goBjTy2GmzJ7UCSunVjhh4Ev1K313o4BihBPVtWlNKHeeWHalpT8ZOONki 6nMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584711; x=1729189511; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=ZcQgUw7C2xvuDZsQn7RPdJVvja4mEZcNzvQY76IItUFEufxyIdcxTikoNk5uHrV2RN 7gJVvS2/WEKts1uRj4SGzv6meFCf10Pbtyp4PYE8IdKVcxhumOEyEl9u4VDiGBsxxZbV L5kY8NBwJKkT9cJ5JcU7nixSj4t/UFric91MGRmKhNuqP/wulP96ZvZxjogKiPChz2J8 ehIL2GjWmA3qmM7mYMHmrQWN+oqDWb2JGe7D4Uc1V6TK/jdzGVAIycrTx2Xkrw84LpQ7 Ph5S+A0IfpttKQ5d3sw1waCr+ZbSNZratnPlBJCbBdIvYUB+Es7JXz7ylJsZh+wqo7Nw n9tw== X-Forwarded-Encrypted: i=1; AJvYcCUpm2xxk3dvHU2ziG/P6waCvY+ypcsB/hB8OpuFwaQteiOkDM1l1+BJOABbcHwyXvTqlv0fcfi/jg4T@vger.kernel.org X-Gm-Message-State: AOJu0YwP+yCYgXLJyXW51RkqccMvkCN7DWL2lz5uCCPiTClo+sFGii3l KKq9MCah4QVAStalfY9Aa9CHpxTnwvx4OXuKJ9cIaD4QgQLKjBltq3R9Ie5KfIrs0GG1N5pobXj vtA== X-Google-Smtp-Source: AGHT+IGD2TfP2dloLIlDYfRlI1lvmWPD7YUtu8EYW3dxRXY7JxJbZQIAJnuRF8jcBfCYGhk25T1qJs7AAuY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5b41:b0:6db:c6ac:62a0 with SMTP id 00721157ae682-6e322305467mr243067b3.5.1728584711093; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-10-seanjc@google.com> Subject: [PATCH v13 09/85] KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Now that KVM doesn't clobber Accessed bits of shadow-present SPTEs, e.g. when prefetching, mark folios as accessed only when zapping leaf SPTEs, which is a rough heuristic for "only in response to an mmu_notifier invalidation". Page aging and LRUs are tolerant of false negatives, i.e. KVM doesn't need to be precise for correctness, and re-marking folios as accessed when zapping entire roots or when zapping collapsible SPTEs is expensive and adds very little value. E.g. when a VM is dying, all of its memory is being freed; marking folios accessed at that time provides no known value. Similarly, because KVM marks folios as accessed when creating SPTEs, marking all folios as accessed when userspace happens to delete a memslot doesn't add value. The folio was marked access when the old SPTE was created, and will be marked accessed yet again if a vCPU accesses the pfn again after reloading a new root. Zapping collapsible SPTEs is a similar story; marking folios accessed just because userspace disable dirty logging is a side effect of KVM behavior, not a deliberate goal. As an intermediate step, a.k.a. bisection point, towards *never* marking folios accessed when dropping SPTEs, mark folios accessed when the primary MMU might be invalidating mappings, as such zappings are not KVM initiated, i.e. might actually be related to page aging and LRU activity. Note, x86 is the only KVM architecture that "double dips"; every other arch marks pfns as accessed only when mapping into the guest, not when mapping into the guest _and_ when removing from the guest. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/locking.rst | 76 +++++++++++++++--------------- arch/x86/kvm/mmu/mmu.c | 4 +- arch/x86/kvm/mmu/tdp_mmu.c | 7 ++- 3 files changed, 43 insertions(+), 44 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 20a9a37d1cdd..3d8bf40ca448 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -147,49 +147,51 @@ Then, we can ensure the dirty bitmaps is correctly set for a gfn. 2) Dirty bit tracking -In the origin code, the spte can be fast updated (non-atomically) if the +In the original code, the spte can be fast updated (non-atomically) if the spte is read-only and the Accessed bit has already been set since the Accessed bit and Dirty bit can not be lost. But it is not true after fast page fault since the spte can be marked writable between reading spte and updating spte. Like below case: -+------------------------------------------------------------------------+ -| At the beginning:: | -| | -| spte.W = 0 | -| spte.Accessed = 1 | -+------------------------------------+-----------------------------------+ -| CPU 0: | CPU 1: | -+------------------------------------+-----------------------------------+ -| In mmu_spte_clear_track_bits():: | | -| | | -| old_spte = *spte; | | -| | | -| | | -| /* 'if' condition is satisfied. */| | -| if (old_spte.Accessed == 1 && | | -| old_spte.W == 0) | | -| spte = 0ull; | | -+------------------------------------+-----------------------------------+ -| | on fast page fault path:: | -| | | -| | spte.W = 1 | -| | | -| | memory write on the spte:: | -| | | -| | spte.Dirty = 1 | -+------------------------------------+-----------------------------------+ -| :: | | -| | | -| else | | -| old_spte = xchg(spte, 0ull) | | -| if (old_spte.Accessed == 1) | | -| kvm_set_pfn_accessed(spte.pfn);| | -| if (old_spte.Dirty == 1) | | -| kvm_set_pfn_dirty(spte.pfn); | | -| OOPS!!! | | -+------------------------------------+-----------------------------------+ ++-------------------------------------------------------------------------+ +| At the beginning:: | +| | +| spte.W = 0 | +| spte.Accessed = 1 | ++-------------------------------------+-----------------------------------+ +| CPU 0: | CPU 1: | ++-------------------------------------+-----------------------------------+ +| In mmu_spte_update():: | | +| | | +| old_spte = *spte; | | +| | | +| | | +| /* 'if' condition is satisfied. */ | | +| if (old_spte.Accessed == 1 && | | +| old_spte.W == 0) | | +| spte = new_spte; | | ++-------------------------------------+-----------------------------------+ +| | on fast page fault path:: | +| | | +| | spte.W = 1 | +| | | +| | memory write on the spte:: | +| | | +| | spte.Dirty = 1 | ++-------------------------------------+-----------------------------------+ +| :: | | +| | | +| else | | +| old_spte = xchg(spte, new_spte);| | +| if (old_spte.Accessed && | | +| !new_spte.Accessed) | | +| flush = true; | | +| if (old_spte.Dirty && | | +| !new_spte.Dirty) | | +| flush = true; | | +| OOPS!!! | | ++-------------------------------------+-----------------------------------+ The Dirty bit is lost in this case. diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1ae823ebd12b..04228a7da69a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -542,10 +542,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) * to guarantee consistency between TLB and page tables. */ - if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) { + if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) flush = true; - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); - } if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush = true; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 517b384473c1..8aa0d7a7602b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -520,10 +520,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (was_present && !was_leaf && (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared); - - if (was_leaf && is_accessed_spte(old_spte) && - (!is_present || !is_accessed_spte(new_spte) || pfn_changed)) - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } static inline int __must_check __tdp_mmu_set_spte_atomic(struct tdp_iter *iter, @@ -865,6 +861,9 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); + if (is_accessed_spte(iter.old_spte)) + kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); + /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. From patchwork Thu Oct 10 18:23:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830675 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D98231E9060 for ; Thu, 10 Oct 2024 18:25:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584715; cv=none; b=UqGSwTBVFRzfDifi8UpkDck4NCbVCoAqvcO97buE2Esjaj/jRKYAWeBh/+V7gsas+7GT8YEjlZe+Gv/EEX/vBQrLhqA946cfwtWS+YbPX+41fzTwKZ+MSrU7U3YqS+xVUGc07bPA4Jw0Nl18m5XPvBp8uZZRCoF1sOdqbbE2BeU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584715; c=relaxed/simple; bh=FKY/uzFyW8/uk7MX2MD3fh26WMEEPHDBpnIM2/GXcs8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=i7b2Ii3kgWz1+TdqGvBRHJGQSShkg2yEiZk9Bj3sQkEkruNYhcnkl+abc/ezSfajA2xNhPrzYN4Jyo24XgX4VltUsdV/YtfKtvuW2EO+S2uLjRxd4PvKo4Ny7wmxe3Viy0Tl8MWqL5EIV9FIDaQUkTrYTakcWNlT2QczPTShhe8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=w5tvPSoS; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="w5tvPSoS" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e2a29fa586so1347700a91.2 for ; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584713; x=1729189513; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=w5tvPSoSWJn+x5q8Oi0ZuA+6nHh716+OIuMRO32iJPXf1iRYDrVPRgGrl1ZJENQvwc mz1HBZPp97M1QM9TizwsEctlYX2RZjDkJzAVpK2ZQIgTitnr1b7IytQF678iRdMz5tHh FUUvh1DdiXPRS4vweuQEsdZZ3uyd14jwGvFphrv6qIvKpgPgKpu0kl1URoW7+fcvcr81 nju8c/yF+vmfJpUJVd3TRkyahxIySMZEM0QnjUZq6q+VDmv8lr9b9PnwJtV3Y0LsQFCN Kh43SB85epl5iaJqPu/GSek9p6Db9/pYBgGQeM4RcUQbVYdV2ebPlFBqlLfYqVf3l6HF sB7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584713; x=1729189513; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=r4vzzEUYljoCBozWCH6VEqtS6mM0ZhFnSa00ziST+43O/G1za/68xBon4ns/h+g3hr /lz89YahNiaQCXyI7g6wvecmA0ZeiN9wx+IpNm2QPy3OhI+tDGjw7ikKAt2sGddqA5WD ixzhDnfA3hamup1zxtoa5xEmZNX477XcgKSiFTyxL4fobt/JtQ7br8w/aCVEltF3bjLM JPC59hF51fcx+dv2iX+SskJ9jKqXEndPrUIL7KmeXanvNN26exKdGx22nvFDseioUnfG jzCfzReY9ZerC+Z9Km5z7+yhS896NXSePXWTtlXviGtFH0/oev7a29C+ykvDk1+YxW0k QJzw== X-Forwarded-Encrypted: i=1; AJvYcCVC+2HnPM/y5u9rNUbDgxtnyuVGTZa7gIr9XwuI1g1PZ59vuxafuIhjIzK4Kwu2UEq+eJyLSoXFcf+k@vger.kernel.org X-Gm-Message-State: AOJu0YzaNmKAevpVTE0iiBygZwBAMCQXA6995JOSUSsMirAf9WisavjU jxk6nw6zRC5nes3fH4K69v6EsxGMKAkWDgOuoPrMH5iNzqYxLvisFYNtoFGN6xXotP8wJXUxqJV Y1g== X-Google-Smtp-Source: AGHT+IEzZdX7U43FtBqfYwOSCvhQrFRs7MoNj08p1am6l1FbD4QaVhgMc/3KfB78OlyTOzXtpjGPG3KIosU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:ea0e:b0:2e2:d843:4880 with SMTP id 98e67ed59e1d1-2e2f0c82a2fmr30a91.5.1728584713017; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-11-seanjc@google.com> Subject: [PATCH v13 10/85] KVM: x86/mmu: Use gfn_to_page_many_atomic() when prefetching indirect PTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use gfn_to_page_many_atomic() instead of gfn_to_pfn_memslot_atomic() when prefetching indirect PTEs (direct_pte_prefetch_many() already uses the "to page" APIS). Functionally, the two are subtly equivalent, as the "to pfn" API short-circuits hva_to_pfn() if hva_to_pfn_fast() fails, i.e. is just a wrapper for get_user_page_fast_only()/get_user_pages_fast_only(). Switching to the "to page" API will allow dropping the @atomic parameter from the entire hva_to_pfn() callchain. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/paging_tmpl.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index fbaae040218b..36b2607280f0 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -535,8 +535,8 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, { struct kvm_memory_slot *slot; unsigned pte_access; + struct page *page; gfn_t gfn; - kvm_pfn_t pfn; if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) return false; @@ -549,12 +549,11 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (!slot) return false; - pfn = gfn_to_pfn_memslot_atomic(slot, gfn); - if (is_error_pfn(pfn)) + if (gfn_to_page_many_atomic(slot, gfn, &page, 1) != 1) return false; - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, pfn, NULL); - kvm_release_pfn_clean(pfn); + mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); + kvm_release_page_clean(page); return true; } From patchwork Thu Oct 10 18:23:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830676 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C6C11E909D for ; Thu, 10 Oct 2024 18:25:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584718; cv=none; b=J5JrVWtZyuY6I9hKom9TsbsLY6u6EakG/5omvRSIm5uZfYswZ8VfzaMe6arullGXcEiLiT/G42F9gNtCvUVvT+2SlR2cf/ZT+2Q7mxkGHMMn46QKCWCeIbQgJazdN9Ewi1qBOq2I0XnqLL6KzoqvMQ+4glrb4PavF7n1Qqb+45k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584718; c=relaxed/simple; bh=bXBvqUXIo6rFfz7xJp3m3CL9RbZG3WivseimjKRiNcE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Fk0SCoUi1jnvEh9LNTGHD2Zvy2/juqq0o5piLwDCKmmwOPcivMAJYf7X3q3MbhLzPAkIOs8EKhs9bZnBPwmTcuhBxg48kjduSasoLGNwEVdcx/+KZGDJtc4vYNO3ziXs3AA/mCNVFaPQRCVzbG3uhRWSrguH2VRNhVUJcy7cy3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=msluruMB; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="msluruMB" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20c94c1f692so4780955ad.1 for ; Thu, 10 Oct 2024 11:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584715; x=1729189515; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=msluruMB47yV+eMqqJKQqQc3MtTMadYZ8sWgl6QRmMr09qa3+feEPjjvGyBY+uif8W qpS0VwqWhyL0czQJzqH+B9niYon2tyJam8T8Wx431OsNSh0byA40a4gCFG9E/iQ4vl7a 6UoZSJP5zfbt83Sm9xMLY1LQdgh7R2IqI7So3Y4ciWiXWgSST91zKUp2NWSuxui2Udu2 XYV5f67I+k1MuchiObUt1AkpUZwZ9d/5lXZxbM2nMVGYJ3xTOCQSP/NlphrkSb4KDTES lsmR21Ah+UemyDSlKJ7g2DcTy675DQzG6u8F3vgdUqFS9ZvBqFBbQWAyPrrqdAw2BlVR 7WoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584715; x=1729189515; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=PAYHKdIZYhuBU3HSwi+kOq0Zgff793iawaqmUHbPbzhptKiq1/ix3vlIAE6x7U7pkZ ioMMWViKnItylt121DqKJI+8KTcQh8g3m0poi68iFm7M7yx1ZXAmv31D6ZVQQIqSy3ae divQSXrmDybFI4ouFA+F5ynmzUaQeL43izenR2Q3X53hFyoJ4HhdWACXU5FJ1JBDeTV+ 6YUCW01cNg3gJMH/FuQlGnDvKqooZp8zscPmEFu7kewUlRiVPyOmxBASicfWNFAXQI2+ HsRLJ4Eqdf4ZWIdteN1aXMMHdAz/E3K9vQZ8HJKPAn26czCWx3Z81VIPyPhXp5zvLcIF qM2Q== X-Forwarded-Encrypted: i=1; AJvYcCUIJRkGF40dBTJwh2ZG/Z4C6uc7w/Xx+mCPjv2bdlgdNDUSb0niAKFbO/E7bojNjY0UpF9CHdsc0XEZ@vger.kernel.org X-Gm-Message-State: AOJu0YylLWEWlZ8ok+p2VwUc/6c59jYMAWQ8I8Fvw++Kf/Z/2k8h0bcg DcaMjFAwe0pe/kEbPk3WphRzQSgARVI46GWIfVwuxi4nK/vKVmj/UVuagnCXj6TwSas4g25OnXr WiQ== X-Google-Smtp-Source: AGHT+IFdLsiMiQR9ssEjYgvfJ+TOiOlvv8h202qp7bAbI4MCMFrC9IRsz+IufRcGyfoerq3Ms7MkoMKw8Uo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:f143:b0:20c:5beb:9c6a with SMTP id d9443c01a7336-20c80510505mr43225ad.4.1728584715291; Thu, 10 Oct 2024 11:25:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-12-seanjc@google.com> Subject: [PATCH v13 11/85] KVM: Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() to try and communicate its true purpose, as the "atomic" aspect is essentially a side effect of the fact that x86 uses the API while holding mmu_lock. E.g. even if mmu_lock weren't held, KVM wouldn't want to fault-in pages, as the goal is to opportunistically grab surrounding pages that have already been accessed and/or dirtied by the host, and to do so quickly. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- include/linux/kvm_host.h | 4 ++-- virt/kvm/kvm_main.c | 6 +++--- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 04228a7da69a..5fe45ab0e818 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2958,7 +2958,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, if (!slot) return -1; - ret = gfn_to_page_many_atomic(slot, gfn, pages, end - start); + ret = kvm_prefetch_pages(slot, gfn, pages, end - start); if (ret <= 0) return -1; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 36b2607280f0..143b7e9f26dc 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -549,7 +549,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (!slot) return false; - if (gfn_to_page_many_atomic(slot, gfn, &page, 1) != 1) + if (kvm_prefetch_pages(slot, gfn, &page, 1) != 1) return false; mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ab4485b2bddc..56e7cde8c8b8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1207,8 +1207,8 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm); void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot); -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages); +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages); struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 2032292df0b0..957b4a6c9254 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3053,8 +3053,8 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages) +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages) { unsigned long addr; gfn_t entry = 0; @@ -3068,7 +3068,7 @@ int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, return get_user_pages_fast_only(addr, nr_pages, FOLL_WRITE, pages); } -EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); +EXPORT_SYMBOL_GPL(kvm_prefetch_pages); /* * Do not use this helper unless you are absolutely certain the gfn _must_ be From patchwork Thu Oct 10 18:23:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830677 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E4D51EABD2 for ; Thu, 10 Oct 2024 18:25:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584720; cv=none; b=pVggY4S+A146SXESSHzs4VFDMYFZT8J7t9xa2m/fETw3yalnWKpSMO4pEArwzZtvPmVL+3nrj2dIpukISZta7WjOwOUuwUWs+YeJk0S3RFXtaEWr9xWfcZXv/MN4ZPSK/mSYdgDryA4OpUhnAd8vI1RfnMczdtV3CJ0DlG06P7s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584720; c=relaxed/simple; bh=NgBQk0YRvUNmc13dbVZlntKigL70MLaajTeU3NGo9fA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fs9atesEcDy2Oe9UjqnGZa4nvvYRYFGNKSc30ZUY/oi9HODrpFh+A85bWGOIsh9fpmJ5PvA04LQQrMmcNizoM14ZxAPk1mMUFfBD72ZfVROB+6oK81Ebhe75k1dVBJkE7fjGkUI4y1+EkG06yPv0Z547vDqew4D9hHJceyqR+8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eihI10jr; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eihI10jr" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71df4f10044so1398120b3a.1 for ; Thu, 10 Oct 2024 11:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584717; x=1729189517; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=eihI10jru2n8ukWDIrc4V1BUWuR7kg0e1MklavXglc6950CPFgvqStVWPROKStYTp3 hNXNXKKlqyzDuLLGxwwWxlgdHdmPGCfXDNUg1NmPaTS44X9jIVXe/ExDP544smBEGPaF hk8AtmMAPGvv8BKjmHvXS6tiKMnZmeVClyLoX/BaV/Blwx/GCI6B2Yu3V02I2crBiNsk 1u0NCDnwPePOmtM2eaOQH2672Om7WlhN2DEagdo+me81XgGdIVBDEjPXYHoKNWdjMqCl Z1oc4pvNwUbuOmrWza21vh57XnX3eqw7MGRoIVnvl5bwzDuQdR41ncqUkNktv/loOESb L5dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584717; x=1729189517; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=Juw+MfpzAlDqUC5kbpQhg5RLZ/vxXy0F6OsLfBxyqm4L91iAG7cr/ZVVH/Tm5v9NUl 9agCQLnWRgqQV8eX5EilhsrbpU0w+x3SqOIOWEgEcRZynQ42CavZtAVb785nZJOqc3hk 22hAWIProodGqaaDY/SFpluJ2NNq6o2YCpiVONl1swcpclIpK/SKim3/SMxi83kkBEcH gaOgwwHOiexQ5eI+g7KCD+ZmYd7NmCr9MbIC3mHryz5VUpOqhOJewC2N54gXl9cWo3Pe yCzaMFHFnWl88EEcqRjUfBo69utQWoAI4OvUWQIKgT3aAaUv0R/uJmeEulMyOYIsKREy 3Cvw== X-Forwarded-Encrypted: i=1; AJvYcCU6rAWlhMHYt0yIDsRhO4BkbzVPBBL2QbpTt87/1bDzIMQfWYc1qMwxHeGnloMkk8MreO7QMl2/vWxM@vger.kernel.org X-Gm-Message-State: AOJu0Yzy+hX5BqNuoKg7gYErayQKZAVM7DauWuogKaQv9Bo+aQYeKDbH BjtFxWDSeKvywQ53TVlywAdmIzJYLepPwAOQcipoPKURpTHDT+orSg6rImEqXuHZefSXnah738q Kqg== X-Google-Smtp-Source: AGHT+IEgBAPk5NggidzeRIz4/KASJ69N4B17cGxFjG2BQTaLRrpMtimTBEcirbwLQ+H7ew0HtHwcTgFgXoI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:fc7c:4a34 with SMTP id d2e1a72fcca58-71e1dbff219mr24371b3a.6.1728584717336; Thu, 10 Oct 2024 11:25:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-13-seanjc@google.com> Subject: [PATCH v13 12/85] KVM: Drop @atomic param from gfn=>pfn and hva=>pfn APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop @atomic from the myriad "to_pfn" APIs now that all callers pass "false", and remove a comment blurb about KVM running only the "GUP fast" part in atomic context. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/locking.rst | 4 +-- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 12 ++++---- include/linux/kvm_host.h | 4 +-- virt/kvm/kvm_main.c | 39 ++++++-------------------- virt/kvm/kvm_mm.h | 4 +-- virt/kvm/pfncache.c | 2 +- 9 files changed, 23 insertions(+), 48 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 3d8bf40ca448..f463ac42ac7a 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -135,8 +135,8 @@ We dirty-log for gfn1, that means gfn2 is lost in dirty-bitmap. For direct sp, we can easily avoid it since the spte of direct sp is fixed to gfn. For indirect sp, we disabled fast page fault for simplicity. -A solution for indirect sp could be to pin the gfn, for example via -kvm_vcpu_gfn_to_pfn_atomic, before the cmpxchg. After the pinning: +A solution for indirect sp could be to pin the gfn before the cmpxchg. After +the pinning: - We have held the refcount of pfn; that means the pfn can not be freed and be reused for another gfn. diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a509b63bd4dd..a6e62cc9015c 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1569,7 +1569,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmu_seq = vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, write_fault, &writable, NULL); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 1b51b1c4713b..8cd02ca4b1b8 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -613,7 +613,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok = true; } else { /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, &write_ok, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 408d98f8a514..26a969e935e3 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -852,7 +852,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long pfn; /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, upgrade_p, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5fe45ab0e818..0e235f276ee5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4380,9 +4380,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); async = false; - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, false, - &async, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &async, + fault->write, &fault->map_writable, + &fault->hva); if (!async) return RET_PF_CONTINUE; /* *pfn has correct page already */ @@ -4402,9 +4402,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - NULL, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->write, &fault->map_writable, + &fault->hva); return RET_PF_CONTINUE; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 56e7cde8c8b8..2faafc7a56ae 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1232,9 +1232,8 @@ kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva); void kvm_release_pfn_clean(kvm_pfn_t pfn); @@ -1315,7 +1314,6 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 957b4a6c9254..0bc077213d3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2756,8 +2756,7 @@ static inline int check_user_page_hwpoison(unsigned long addr) /* * The fast path to get the writable pfn which will be stored in @pfn, - * true indicates success, otherwise false is returned. It's also the - * only part that runs if we can in atomic context. + * true indicates success, otherwise false is returned. */ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, bool *writable, kvm_pfn_t *pfn) @@ -2922,7 +2921,6 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, /* * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest - * @atomic: whether this function is forbidden from sleeping * @interruptible: whether the process can be interrupted by non-fatal signals * @async: whether this function need to wait IO complete if the * host page is not in the memory @@ -2934,22 +2932,16 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * 2): @write_fault = false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable) { struct vm_area_struct *vma; kvm_pfn_t pfn; int npages, r; - /* we can do it either atomically or asynchronously, not both */ - BUG_ON(atomic && async); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; - if (atomic) - return KVM_PFN_ERR_FAULT; - npages = hva_to_pfn_slow(addr, async, write_fault, interruptible, writable, &pfn); if (npages == 1) @@ -2986,7 +2978,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, } kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -3008,39 +3000,24 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, writable = NULL; } - return hva_to_pfn(addr, atomic, interruptible, async, write_fault, - writable); + return hva_to_pfn(addr, interruptible, async, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - NULL, write_fault, writable, NULL); + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, NULL, true, - NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - return __gfn_to_pfn_memslot(slot, gfn, true, false, NULL, true, - NULL, NULL); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot_atomic); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot_atomic(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn_atomic); - kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) { return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 715f19669d01..a3fa86f60d6c 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,8 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable); +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable); #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index f0039efb9e1e..58c706a610e5 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -198,7 +198,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) } /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, NULL, true, NULL); + new_pfn = hva_to_pfn(gpc->uhva, false, NULL, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830678 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 433241EBA09 for ; Thu, 10 Oct 2024 18:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584722; cv=none; b=GrTKTMmRjXWviJRqsWoEB1sXlisLtf45tgZVnZQDy5/1PPRwO3UdJn+wA+XL3QGXYJNXPXQwHWUehmklZhROaUBpQYNy+HeFw4i1bBIOPWW3gXgREhlmtxtXn/NvyD/aYsnOaRKQh9CB+XFxU+o4T5piX8CQWzCa4PjwBsBGwok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584722; c=relaxed/simple; bh=IF5BVNwy5lfsZS2zGuSAAv+TGmQDGgo7CuXbuO4ZMc4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OgtgR/nEnrelRHzkHRCS+417MJ3QnWBFr6P/dTedOHes1KXBK/zG/58Vyzm1u7DdZzerEy8IlNCmnOpdfKWlrU4fDTXVX5r/RY/vx9+11AJedIxViCl7WsF05hHmU+X32baX65P8E7HZAylKa0dxGIo3IsC4CT+l7hLiPnKqkug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OrSuHW3L; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OrSuHW3L" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e3245ed6b8so23826767b3.0 for ; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584719; x=1729189519; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=OrSuHW3LuAFGFcLFdBetAruGipimy71Kdd23mnkCvSb3kzL/ihIej8uTFHbxMiLtnA 6YXSI2J+unla/LKg7JIEqX4mCyvrr3VQ406BwPbBG3PiTMk+2hz8bnoRevRPodlhpmMn aYJOYXJWz/3t+SqCWv1rhWrcN4c2Ngbiu7CrMUkXV7A/egH9oRwfDsMuE+3yorA8VAOz djCQlkkb9mc9WG+6oBsuPymdYZpYY26x8ENW3hOt+jiqZzezffl3p0vFDG1wHBHrBrbq D5swTKZ9IT9IBSz41WC4R7QS/s7rDjGVXDZH7rNvgqH9bS1bMtz43t3X2ef5ejrnse9y 5+Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584719; x=1729189519; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=TJU4InK/uSAe/naoDtIbPvSrG5/VGYZUbbxgeEiO0A0LGG4GXM8X7U6EeohzgNwomA I9+H0xl7eptjZhHJGelauRJnBJSIJij8yUWhSGnGRMJ1swuE25A13e/Wiu/fRsHEHowz deXbZg7wcJUzdWy8w/pzW/pXKtQYjuobfclOnu8jgqeH6zaIzRzZ/AlRluZZq2WRqKub 9K2uk6KeAHiL+YCkOPwkatVZBJcLvZmWU4xfzjMDx0JqD47W/LkvSibuMCPCKy+mWIsF EmAdzE5wtH3JEWtM7vePe0G22dThEPy/IaKAshkfpqgywIdO/+vEuqZ0LRhjID8M4k2V xpoQ== X-Forwarded-Encrypted: i=1; AJvYcCU0/EQO7uH/3Vy4yamfPs72zThOpwG11S7p8OI72V0sAYwBYe+HXFMu47HzL7UoWOpSVS5YuPKAfPeQ@vger.kernel.org X-Gm-Message-State: AOJu0Yz3UpuX6skrBYZFTe84FuFGS2cBfFp0RsIe3OUrh3FL03U3cuc/ u+cnbhyJWSZU1FpI5FTusBRnPrncbVpadDj91BprPjPvv5tqsZUmCPTR/XcrXfQRoRv1+J975ud yaQ== X-Google-Smtp-Source: AGHT+IEeBu92qjGJ6/8zutZUBi8WOanw4UupdtztXpUfLE+vAeLHZHNkVuFOksmpHMU427MoGYN+Q9YZlwU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:3204:b0:62c:f976:a763 with SMTP id 00721157ae682-6e322185b22mr897567b3.1.1728584719334; Thu, 10 Oct 2024 11:25:19 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-14-seanjc@google.com> Subject: [PATCH v13 13/85] KVM: Annotate that all paths in hva_to_pfn() might sleep From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Now that hva_to_pfn() no longer supports being called in atomic context, move the might_sleep() annotation from hva_to_pfn_slow() to hva_to_pfn(). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0bc077213d3e..17acc75990a5 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2804,8 +2804,6 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, struct page *page; int npages; - might_sleep(); - if (writable) *writable = write_fault; @@ -2939,6 +2937,8 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, kvm_pfn_t pfn; int npages, r; + might_sleep(); + if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; From patchwork Thu Oct 10 18:23:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830679 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5F681EF0A7 for ; Thu, 10 Oct 2024 18:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584724; cv=none; b=tX1gBUMvuDf8bqeVt28CufGee3RyapM/HK/qrAQQ74v0K4e4erqrDzN+3T77fVK2zzJTXYslHvsyVtE2TWi2WvWRzW756eiDmnE1RbQt9Hvm4khIwgJ4eP/QnEfBbCAPNo/rpfp9TLHkdG3ToSwWA4ecD3YWm5bQ+cb7A67dgrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584724; c=relaxed/simple; bh=rFmsStC1d1lztIwj7oxh4zyq4MQj9meeqHdDcxeq/wk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BAe34OgeFBISsS2CLsGpS9Z0wFraAdc+uncTXnArcvK/7ur9OX/yE2Lp3VUp25JAWZOD5phr6Mvo++gL7OpLt4Do7nDr05O6Jpcv03P0WLvuwlRvC+CMXh8ZE7ER+HlOO78+YQRgPBB9PoitINGszHxu1B1zLxdrEevBETZU1f0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zBhsl50q; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zBhsl50q" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e2a013a01bso1334146a91.0 for ; Thu, 10 Oct 2024 11:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584722; x=1729189522; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=zBhsl50qhIyYlq5VgEtE/+OL3pX+IkWGN8RxGVp/QOIATD1A51kLNeCudHlw6tpDB0 TTnMuZ0YrHvKBYD5A8G+ycj8IUIKyFCIAvMqpCIDMUUrDrOdy+ab52MBoxiszJHwHzOm sj7/CT59ErmtRPL1Q4YUop0G579u9pMO68H1UQD9Q5tvvttPtuceH0HBENkiQygrEN+L UXFheIKzvN3cuQNMasUDogdtF4XkKlByJg1oeWEuvwjQVYTXh4u+Ukk61xBxYSl0CUBn zAffTJ/B4qMDCsY9MtrgvulhfXkVKD4XfiY8/udxWcKlNZIr+Ec1bJeX/OtZ25Pola2P ZJog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584722; x=1729189522; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=EuhcyLi9qZfNoQAHp7HPSUyFmbjBoeMiYaog6t0B88idNoqkHrd+Xh6E1Merpd6tLx mzGhzZW4RkchF1mKfuoF1uxaZw25/lZ5LDIejW+LXcLLTdrtQrk0b1Qq/NgMHnzMK3a5 VrRhRy6qixDgDi2RdwV2rewtkD5kxoUitbQBPIHhVr+wiIPQhp6kyJfEhmAjADVFx8XF C7MWP3UTrdwOl5l8D9OmXyMv75mUGg0KsQsaooJgCTAUnxUNfEtxLHwjGw1kzEOmrd4D X7VssbTHptsrNmp4g/goCB+riqiP6Zv0FzopOvM3tl5hfBJVEOSJl5+0dC3FmHLEbec3 wfFg== X-Forwarded-Encrypted: i=1; AJvYcCWrQU1VnAnFAE60D+WR9qvnEpMXUFMANCtXNdAzYNQbiGTAqo47ziTO9B46lPahttXt+7Rjj3euCTxL@vger.kernel.org X-Gm-Message-State: AOJu0YwP9sEPwklmozr0AwMWm/9JdtO95TLUJkxXD5MYb4edMDvB48lF QSXxVrjQ/4EiInbB3evccn36etXvoXxe5ckCjGETvSFOiD+Q/WWBcGD/2O36j7F6k8i5xhG4kCY GVw== X-Google-Smtp-Source: AGHT+IH031wjBK6zBugJiyQwbgM1S3WkZumhuZG0/47H7WWzE24O4tbIqX0eqBdD0TzKWzHgCchY3qP4ics= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr107a91.1.1728584720854; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-15-seanjc@google.com> Subject: [PATCH v13 14/85] KVM: Return ERR_SIGPENDING from hva_to_pfn() if GUP returns -EGAIN From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Treat an -EAGAIN return from GUP the same as -EINTR and immediately report to the caller that a signal is pending. GUP only returns -EAGAIN if the _initial_ mmap_read_lock_killable() fails, which in turn onnly fails if a signal is pending Note, rwsem_down_read_slowpath() actually returns -EINTR, so GUP is really just making life harder than it needs to be. And the call to mmap_read_lock_killable() in the retry path returns its -errno verbatim, i.e. GUP (and thus KVM) is already handling locking failure this way, but only some of the time. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 17acc75990a5..ebba5d22db2d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2946,7 +2946,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, writable, &pfn); if (npages == 1) return pfn; - if (npages == -EINTR) + if (npages == -EINTR || npages == -EAGAIN) return KVM_PFN_ERR_SIGPENDING; mmap_read_lock(current->mm); From patchwork Thu Oct 10 18:23:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830680 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4D761EF936 for ; Thu, 10 Oct 2024 18:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584726; cv=none; b=cwNVgq5gpLOVtbaRmz/5hbNKay5RfKsavnVqqV6IdqNboZsbuKSlXlTETwQ8dUTu9yRqLTVSUPv262vPNBkgSg5JaZAgqHbvjxieqj6CODiO6ZyT6a08DRrIGOVL8tU7tT10hoeCX6Fv+bdQ/hWZB/4BJque+8USmd01o8ntuYc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584726; c=relaxed/simple; bh=xKxLUn1wM/pJucT4/ZxvVNmNhzhyHMbiLj3YVf1FNHQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hIF4vwXmjczPA+1dll0kHaeh5kgazP1q6uL3i5pTc0MzEgKw6quKNj9hT3KWnJ6Z6zrQGfRxL9FMEEMJqtGz1nBJWStdRnb39ECFAqVQKEtm84uIvXkT2bTvuAMnUi6SAARTMNPmYukZuOsQi+M3wcGT/d+dZmqE6+a9QFRjMjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TYQRQk+w; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TYQRQk+w" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e32e920cf6so23628657b3.2 for ; Thu, 10 Oct 2024 11:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584724; x=1729189524; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=TYQRQk+w7SZ6hxu000ERFTE/19nobGDZp4qfI4lUGaKSkx65llk5i8OtSPaWktaPe+ 5XozsAEpjOmnyoNqatnrYMAZ60RWiIscI44p1xR028Kq4YBFHeziOBq1g5YMW369+ewe DBbnBBj7uyWKhzvNHIsKPh1xm7J41dcuSCTEYZDqgPyXGvJRHeKaxnjlU+Q7xVjJlTj+ Py4wWC7F64nR2FkwHexXSLRXrQRj5eudodYY+PhBwB9w0LCVub3wGLZ+EE1KRXqo1yqd P1/WIlzQG/PMilWwZm1sDFH2cYInsL65Bp0RatNOaVo01u0iAbnToWRI8vRt9jBA3Mti z40w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584724; x=1729189524; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=YaLNua7uaeT6PmNhfIV8ctwuxos/++A8uFgc2xzAgvIlGDh8oRhE5No2WHMl61+NF8 01SYrhTrVRvGU3QDYW+v0+gTPOBFX+hNidlb5pLWQnt2H3IUH+DMRKID1db9Ps4unhNR 3Iulqk5djxkQ701ykFhD5PcPJLIpOZrjJXGfe87U+IvLDUMjPt+c3bwlfDm1ExCBtZOj oSeNwpLiSX9XlDjCnh1IMSmSREiJym3XAwvkaNQLWRA41fEX1MXB6JF/nfaOWDW/MJpK ddkAx4zGInNH1wgyZBQliXbJDPfQp8K1I9TqB/g4oOpxcHyjHuwhloVDl0Cks0vsl+es OGKQ== X-Forwarded-Encrypted: i=1; AJvYcCVBkYdR2XUWgnNpTTHniQ/AfI3MW59HprCRIVqAyMhRYbEM7B8kocDPDNdaJzWeUftD/XajKN0c61RA@vger.kernel.org X-Gm-Message-State: AOJu0YzLBHukB2fKWXn/DoSkWhgPhwHfTpcFxjXNJxfxtdCIbtUnGKdt tpwLElyxC+O6QBnkZfEO3+7CtLYdfBaLmz8CHcczCkgTBtxo4yjY3ZQd9fDN/CY7XK1NLjAua93 PoQ== X-Google-Smtp-Source: AGHT+IGf82b/hxj2IWFe54/TAnEDS4XyhakmYGUWxryxrCLTzb83+cUU5J1DZhSXbLBNZQvipdjBbhmVkR8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:8e09:0:b0:e28:ef25:5f13 with SMTP id 3f1490d57ef6-e28fe0e7352mr200743276.0.1728584723790; Thu, 10 Oct 2024 11:25:23 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-16-seanjc@google.com> Subject: [PATCH v13 15/85] KVM: Drop extra GUP (via check_user_page_hwpoison()) to detect poisoned page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Remove check_user_page_hwpoison() as it's effectively dead code. Prior to commit 234b239bea39 ("kvm: Faults which trigger IO release the mmap_sem"), hva_to_pfn_slow() wasn't actually a slow path in all cases, i.e. would do get_user_pages_fast() without ever doing slow GUP with FOLL_HWPOISON. Now that hva_to_pfn_slow() is a straight shot to get_user_pages_unlocked(), and unconditionally passes FOLL_HWPOISON, it is impossible for hva_to_pfn() to get an -errno that needs to be morphed to -EHWPOISON. There are essentially four cases in KVM: - npages == 0, then FOLL_NOWAIT, a.k.a. @async, must be true, and thus check_user_page_hwpoison() will not be called - npages == 1 || npages == -EHWPOISON, all good - npages == -EINTR || npages == -EAGAIN, bail early, all good - everything else, including -EFAULT, can go down the vma_lookup() path, as npages < 0 means KVM went through hva_to_pfn_slow() which passes FOLL_HWPOISON Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ebba5d22db2d..87f81e74cbc0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,14 +2746,6 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } -static inline int check_user_page_hwpoison(unsigned long addr) -{ - int rc, flags = FOLL_HWPOISON | FOLL_WRITE; - - rc = get_user_pages(addr, 1, flags, NULL); - return rc == -EHWPOISON; -} - /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2948,14 +2940,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, return pfn; if (npages == -EINTR || npages == -EAGAIN) return KVM_PFN_ERR_SIGPENDING; + if (npages == -EHWPOISON) + return KVM_PFN_ERR_HWPOISON; mmap_read_lock(current->mm); - if (npages == -EHWPOISON || - (!async && check_user_page_hwpoison(addr))) { - pfn = KVM_PFN_ERR_HWPOISON; - goto exit; - } - retry: vma = vma_lookup(current->mm, addr); @@ -2972,7 +2960,6 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, *async = true; pfn = KVM_PFN_ERR_FAULT; } -exit: mmap_read_unlock(current->mm); return pfn; } From patchwork Thu Oct 10 18:23:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830681 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C56511F12E5 for ; Thu, 10 Oct 2024 18:25:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584728; cv=none; b=OvOHvmB3npxLuFwHuWyo91MOH5pHs3GcYWQDGk4a8ZRqqbJHQPzm8qoo+4Osweyxls1JzDOv+9drLnz3ZnTWReclXCHWtwzspvC5UbQeZqo6kiqloUeK6Ru7E2rZaZenhfi1YrvOhXlYyy4uCcB7xyX6fj5cHnUmKjtDXzYHO9c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584728; c=relaxed/simple; bh=42VPy3Va09DWIWyeI7bQhF4N8yIw+oK2ibHTdiPXTHw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=u4OLq5JR3BA0VFI7D7vFol9aBwOxC87TDwCUb+M63tPleHpjO+7tAIr9/sEL16nWF14ma9j+n8D+nrMtyExn184AKrw5snmSqkoL//Vi5nkyEUQe/vrmZs9MeASeSzI4H1PGFZqr2QPK8RQTtMMU2/HiDLZYVrzvQNi2bvjWOmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WvslKqNS; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WvslKqNS" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71a122bf521so1617045b3a.2 for ; Thu, 10 Oct 2024 11:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584726; x=1729189526; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=WvslKqNSO5DKemaosV5Ekaavvk/7eArebNvJycUuLBkZGB/XJaMFX6aV1iKkGUFwP0 ZNwz0bY08AVY+Ai+JzYMIfwVM/iwhmIFh6ypp19yQJ7xlLlH4E1/h5/i4lkEIEswxRX6 G1V1+MDdkMXxuUeQhS2FytA/DTRX5CMGC7SlmlvH24EV5qDMrezu16iTJ094fO53XMoJ CGl31UkFsqipIwaYzOCagq4DAomkEhANFb6QoDU2IIOMnJOpIdKn8veU7TKRO0XUkEuU U8S1Qx2j8B7VTqF/nKa6l80LAoqMwmyO08nUTDqCHn4jvA8Xb+kpNd8Sq1I3C8YK6lcV TOiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584726; x=1729189526; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=I4dZyF9CwuHBDubhlH20A63ZeC7vIPqSmuEj9cwCrTAUeng1G1zk7cFWSSNCen/Am1 Mto8+kt7vj63jBpdz1tiZaNPQtQsh4yctuZu+NN7URjB5KcWUKEsqxyp7VPx/xlSeb08 2D/mv9MNSm3XVYFS1f0j6YgbWovA5KAlsnVRq4Mc3rsPw2yzYfTHMVx/pSXXh1/G5pjR bbxXsvSROKbSSEmGY1UHIQALzdbJy0had0L+W7iL/yVLnMA+VI4tz66vQlbHWxRQWMnT ChalCiwBBFijdrdty/WqPZo7jbzpuo3HO+zob0Z4dUb3ZNRnU0vDK8aFYInTvaJGveZV CD3g== X-Forwarded-Encrypted: i=1; AJvYcCVTRo4QI2XBbYNG/9Yy9qZmeys9Qx3KZXrt/v/+3PLKGmxBzeGpysCTnmA+QTOZU+dsSBsX5uI2jxF/@vger.kernel.org X-Gm-Message-State: AOJu0YyirATMyfo9yQQqb9PIDbItl0eoA+Z/5x4HWSOzWyt7bUVtzzpR kwFeIrOnND8yOpXpn2ofY9yO7I/ICx0QmkUXYKllSJAzbf+sgTz8t//RqQ/mglXBDw953GGSFHn tNA== X-Google-Smtp-Source: AGHT+IEgY3dn5eo0Xr6pp3rAg0uVd1WuwbDag/Dk23UVvT7MMuxyKVAtWc2T42TlMp1H8D9GJ1Oltmo921o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8c17:b0:71d:f452:ee99 with SMTP id d2e1a72fcca58-71e1dbcac44mr6866b3a.3.1728584725637; Thu, 10 Oct 2024 11:25:25 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-17-seanjc@google.com> Subject: [PATCH v13 16/85] KVM: Replace "async" pointer in gfn=>pfn with "no_wait" and error code From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones From: David Stevens Add a pfn error code to communicate that hva_to_pfn() failed because I/O was needed and disallowed, and convert @async to a constant @no_wait boolean. This will allow eliminating the @no_wait param by having callers pass in FOLL_NOWAIT along with other FOLL_* flags. Tested-by: Alex Bennée Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 18 +++++++++++------- include/linux/kvm_host.h | 3 ++- virt/kvm/kvm_main.c | 27 ++++++++++++++------------- virt/kvm/kvm_mm.h | 2 +- virt/kvm/pfncache.c | 4 ++-- 5 files changed, 30 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0e235f276ee5..fa8f3fb7c14b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4374,17 +4374,21 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - bool async; - if (fault->is_private) return kvm_faultin_pfn_private(vcpu, fault); - async = false; - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &async, + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, &fault->hva); - if (!async) - return RET_PF_CONTINUE; /* *pfn has correct page already */ + + /* + * If resolving the page failed because I/O is needed to fault-in the + * page, then either set up an asynchronous #PF to do the I/O, or if + * doing an async #PF isn't possible, retry with I/O allowed. All + * other failures are terminal, i.e. retrying won't help. + */ + if (fault->pfn != KVM_PFN_ERR_NEEDS_IO) + return RET_PF_CONTINUE; if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { trace_kvm_try_async_get_page(fault->addr, fault->gfn); @@ -4402,7 +4406,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, &fault->hva); return RET_PF_CONTINUE; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2faafc7a56ae..071a0a1f1c60 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -97,6 +97,7 @@ #define KVM_PFN_ERR_HWPOISON (KVM_PFN_ERR_MASK + 1) #define KVM_PFN_ERR_RO_FAULT (KVM_PFN_ERR_MASK + 2) #define KVM_PFN_ERR_SIGPENDING (KVM_PFN_ERR_MASK + 3) +#define KVM_PFN_ERR_NEEDS_IO (KVM_PFN_ERR_MASK + 4) /* * error pfns indicate that the gfn is in slot but faild to @@ -1233,7 +1234,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva); void kvm_release_pfn_clean(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 87f81e74cbc0..dd5839abef6c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2778,7 +2778,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, +static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, bool interruptible, bool *writable, kvm_pfn_t *pfn) { /* @@ -2801,7 +2801,7 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, if (write_fault) flags |= FOLL_WRITE; - if (async) + if (no_wait) flags |= FOLL_NOWAIT; if (interruptible) flags |= FOLL_INTERRUPTIBLE; @@ -2912,8 +2912,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest * @interruptible: whether the process can be interrupted by non-fatal signals - * @async: whether this function need to wait IO complete if the - * host page is not in the memory + * @no_wait: whether or not this function need to wait IO complete if the + * host page is not in the memory * @write_fault: whether we should get a writable host page * @writable: whether it allows to map a writable host page for !@write_fault * @@ -2922,7 +2922,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * 2): @write_fault = false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable) { struct vm_area_struct *vma; @@ -2934,7 +2934,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; - npages = hva_to_pfn_slow(addr, async, write_fault, interruptible, + npages = hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, writable, &pfn); if (npages == 1) return pfn; @@ -2956,16 +2956,17 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { - if (async && vma_is_valid(vma, write_fault)) - *async = true; - pfn = KVM_PFN_ERR_FAULT; + if (no_wait && vma_is_valid(vma, write_fault)) + pfn = KVM_PFN_ERR_NEEDS_IO; + else + pfn = KVM_PFN_ERR_FAULT; } mmap_read_unlock(current->mm); return pfn; } kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -2987,21 +2988,21 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, writable = NULL; } - return hva_to_pfn(addr, interruptible, async, write_fault, writable); + return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index a3fa86f60d6c..51f3fee4ca3f 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,7 +20,7 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable); #ifdef CONFIG_HAVE_KVM_PFNCACHE diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 58c706a610e5..32dc61f48c81 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -197,8 +197,8 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, NULL, true, NULL); + /* We always request a writable mapping */ + new_pfn = hva_to_pfn(gpc->uhva, false, false, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830682 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 928051F1314 for ; Thu, 10 Oct 2024 18:25:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584730; cv=none; b=kELXYtqvwP2C5xW2f3AxA4KMbWmviQEyRmvudlCULSHQ/GyIOThXR6smbhfuKFi2gGJYOO3WlynSqaTcu9dpWLABpurQLnl8sQeP+EaBCjFNY4gEiB80pGGGmkewXOiKaiAeaXxWLVauyZr4pnQ6KV+A1wKjQjjbP77ELY5oA/4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584730; c=relaxed/simple; bh=vtDT6oLoOmF91AiioFXu5/ol9RZL+XplRHPVYvyqLOQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tXY2z1L3biN0pqJX9EgTIg1Czq9fuyHaet8DSB44dvmfiSofgx7RptE16cSuxRueBWtmlbj7IsbP0VzwLWwWsv807lt1s7irWbXzDVwfoytavkLERJxgJdqYdcczYdHzeixRwUXhcPQmq25nYRXTVdoho1+oMBuZXpIxBLiN1I0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Dha/A36a; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Dha/A36a" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20c56938075so15549895ad.2 for ; Thu, 10 Oct 2024 11:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584728; x=1729189528; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=Dha/A36ak1juWSy53/C5jnDKwMB3ibLmNWOqN2CaIPVTvc2bOsO+vIG4ji5VUNB+Fr gAcZshJlAnKB9IGnefOr8J2/sIsOnWzbaUSaKAYAfY0GdCKdUUDLPpPmylbsAc5fiJtQ e+FQSoUCLb8PwxOQTC2Wp4MvrQOZvR+GL6RpTuUF2Ws2zSXOwcR2KcfTnxlrY2LApBDX 1jDtMaJUCphK2q+yi25T6nAEQFpfTOKBCtnS6RvfK2LDAc5h9R7TmHsLd/jFQ02K/iU9 XMkUMOtKJ9vnvdqixw1TS95uc48IbgO3uWWv/2SyHoFjG6zpnpjdrM+rk6uPgJaCnthV Nb3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584728; x=1729189528; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=PRQplw7en+JAeEZHcZx1lexEHIzV1YABTJBC2g2Q7BgS3CYF+3e/xDvdV3egiySFjq 1FGzIAo5AePvf88c+3AwWHeTKTfo9AeHZo5yHbxke2My23s1MzQ9ZxXE1vqXWKPhCM5L nSi2VpyIPhwaxMkIGb5GiFDbVKVRy9I57bFLyBu+dq6Xpqdsm22qA4M1kY4yIwMkc5qe YLSvInUIDNoSg29guGgH80pT/TmDWdxa1yEuuCmYpl464CJetPlMQJnXPlWa/t41wU8v 17eMFliMEHs6Upu1+DmSyvz654So3DvPQaEfNAKeiiuOFb0CPH2h3GJbNIu8wegmC3gG 4yvg== X-Forwarded-Encrypted: i=1; AJvYcCUk28anNg/RF6fhKtng4bR74SlJdIBUZkRikFaT4zK+FHsZyegeWMgbzfxkKOSM9zgOnzjJBRYGJ16c@vger.kernel.org X-Gm-Message-State: AOJu0Yw8iWLgc4AhfIIh7xXg2s1OZ2Zsw16aHkqGBGjCjjOCeHe9wXbv hH8Ze8aHguLsfrD/ipgLsXBRqKkvLaNofE/lsGRMuRKTrGNyq+XJ/8K4S5sKit8ec5g18zl2S9D k+w== X-Google-Smtp-Source: AGHT+IHx3wOCIkhIacFyS0KkUCfogOtyOnmQIp9NOdA3rn83blMwYV/7exbhoH/uHIfky6alS3vCfOjmfpM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:d502:b0:20b:7ece:3225 with SMTP id d9443c01a7336-20c6358fdd3mr1169885ad.0.1728584727745; Thu, 10 Oct 2024 11:25:27 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-18-seanjc@google.com> Subject: [PATCH v13 17/85] KVM: x86/mmu: Drop kvm_page_fault.hva, i.e. don't track intermediate hva From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Remove kvm_page_fault.hva as it is never read, only written. This will allow removing the @hva param from __gfn_to_pfn_memslot(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/mmu_internal.h | 2 -- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index fa8f3fb7c14b..c67228b46bd5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3294,7 +3294,6 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, fault->slot = NULL; fault->pfn = KVM_PFN_NOSLOT; fault->map_writable = false; - fault->hva = KVM_HVA_ERR_BAD; /* * If MMIO caching is disabled, emulate immediately without @@ -4379,7 +4378,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4408,7 +4407,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault */ fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); return RET_PF_CONTINUE; } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 4da83544c4e1..633aedec3c2e 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,7 +238,6 @@ struct kvm_page_fault { /* Outputs of kvm_faultin_pfn. */ unsigned long mmu_seq; kvm_pfn_t pfn; - hva_t hva; bool map_writable; /* @@ -313,7 +312,6 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .is_private = err & PFERR_PRIVATE_ACCESS, .pfn = KVM_PFN_ERR_FAULT, - .hva = KVM_HVA_ERR_BAD, }; int r; From patchwork Thu Oct 10 18:23:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830683 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0418A1F4FAB for ; Thu, 10 Oct 2024 18:25:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584733; cv=none; b=VrRw74tiG9FlsScMMAscftaT2IQtiEIzPEbmVxTLpTGNIO/Ebp/bDkrGpe1vIHwoyzVVCA8y03UA9i8CAnRaJ/tO4nkDgf4BED1eLzh8J/MiUrXW8OHvpsQXkwzf9nPhwFqvhl5ZUGCOi/4q/8/Bya6Opwkqerwc2275UdP7OFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584733; c=relaxed/simple; bh=gORZY+kwKqxFhTI7jRiw4mW6KKJknMWDgC2YkpetElw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fOb38K0ptUaVHqSA7tD7Zligjj4RaAV6jLPm63Th5xbDof3M543YXm9VSk1O0XBnPpSuuKXgnSg0OmxLO5JZYgD8PIIoTinP3vVkRPaEVP3YzR5UiSdtNw3Dmx5rVTwGrEHeWNUzsVU9hoD/Nno807pxe3rDICdu1tU0ma384Pg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eIGj+Y80; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eIGj+Y80" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2eba0efc6so171537a91.3 for ; Thu, 10 Oct 2024 11:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584731; x=1729189531; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=eIGj+Y80OKxaBgravZE7eiui0FBQFj4LLKuhGsYIejJ/ru27Nu4bo6PYyo0RqdW6jx mlEsjNvzm4bt/om7CPDzDCIitHXfOAawPomD6hoIfarHR+jb/Vpik0cM7RCURqHtp9Th KS+pp/789ua5N645ObTpGNCJ5zY5VYGSoas2pjsYOGcz4D//yVxmivdlBgrIuPOsZsuD PsIcOCklR+rODFwgTw/JLAwVlh79y7eUR2dCyqq7sfL4r4jGaiVeE7naHt9ulZ5giDzT SQCuZV4v5mlU7zeVE9DYptsrrR1n7XGueT0FUJrtCmHgSugT/5LXEjrWUt41nO2Mw821 sr8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584731; x=1729189531; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=UpWk8zIH7Jexw70LdhUwzzVv5WX4beC9HCg8zIet/o1rRba2x1ydFHWgZdaBhsf2Xn jTd9pVjK622eZdFMNMkfbZnoNk+V6qDXEcHX/xfjAn3qyWa0bMwS4XHl+QrO38WK0EsT rQkMzAYReXOT/MJ5fY6fP3Nuv2MSZExZj8gWe+SEtthX1BEXGPz6rh2hh5HNOfGZBqfk 3lCX10inURxj+AC4CLaU0X8Jy+zTd3nW30Kr10HUZ/1iEk97OS3bw7kxYpErO/J3GrHq /7uP3M+B44cWoarUI2mQTSzj73JVVA70ObGb+R9y2XZbqLt/ipQoF2MvbeuFZqjO8s5w ZeMg== X-Forwarded-Encrypted: i=1; AJvYcCXpWDIJUX2iIXbcTXKuM7qqXOCjGgXblh9oyTD1lx1FE7xpJTAxvasX+ww5ryHZ0Ezs19N80M4+QBuC@vger.kernel.org X-Gm-Message-State: AOJu0YzAqSYK2K6vcpcYhiiDyU7zVLstxLk1QWhYrdo0Rj8+k758BlaP NkX7FIFh5lD/O1Yq7w4OcwVbmFwV7xn9Blc8eilPfbBu7uAzX2TwYJSe1nGHVyOh7rOMtCKXXzu vuQ== X-Google-Smtp-Source: AGHT+IGFsZ8G+8oqHMXYnd6l/b9oSo3/s957bEBlaeuh+YsQIWv/Tzia2RQhXQw4+rIMPn/3epGKWmgCn2Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr20a91.8.1728584729847; Thu, 10 Oct 2024 11:25:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-19-seanjc@google.com> Subject: [PATCH v13 18/85] KVM: Drop unused "hva" pointer from __gfn_to_pfn_memslot() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop @hva from __gfn_to_pfn_memslot() now that all callers pass NULL. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 6 ++---- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 9 +++------ 6 files changed, 9 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a6e62cc9015c..dd221587fcca 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1570,7 +1570,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmap_read_unlock(current->mm); pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable, NULL); + write_fault, &writable); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 8cd02ca4b1b8..2f1d58984b41 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -614,7 +614,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, } else { /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok, NULL); + writing, &write_ok); if (is_error_noslot_pfn(pfn)) return -EFAULT; page = NULL; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 26a969e935e3..8304b6f8fe45 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -853,7 +853,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p, NULL); + writing, upgrade_p); if (is_error_noslot_pfn(pfn)) return -EFAULT; page = NULL; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c67228b46bd5..28f2b842d6ca 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4377,8 +4377,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4406,8 +4405,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * get a page and a fatal signal, i.e. SIGKILL, is pending. */ fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); return RET_PF_CONTINUE; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 071a0a1f1c60..cbc7b9c04c14 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1235,7 +1235,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva); + bool write_fault, bool *writable); void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dd5839abef6c..10071f31b2ca 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2967,13 +2967,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva) + bool write_fault, bool *writable) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); - if (hva) - *hva = addr; - if (kvm_is_error_hva(addr)) { if (writable) *writable = false; @@ -2996,13 +2993,13 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable, NULL); + write_fault, writable); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); From patchwork Thu Oct 10 18:23:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830684 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0126C1F4FAC for ; Thu, 10 Oct 2024 18:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584736; cv=none; b=bySGWZDCuOmXGg/JQG/2176eJO4VThD6DK8IlvWIfe6QxmpT8R2W9jruFfHaiZvQU0A7tFakfYG+65vyE6calIpegei1Dk2U/uQw5aqShI45wxES/+6zpzgFz5jVJIw6sht6WotyDY5sbvg9qQXAiTEZ+fW5RLdAMlMUQvdncbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584736; c=relaxed/simple; bh=B6DKB14GzdU+gSziUKXJ1u64FTzTfv1AIh0aTxVhDrQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F2b2MxypgK/rb5l/PsXotv9WPxEUj/PsmX0UPdcUWUOSgtZdifMZpwjxfHZJwu50ywPJOxSYDSiG3UeqHsxyj8Iho6bY++yY9ovlTsL9z6lNywHd6Afmq1jf5M4+XmqeRklWH1pBvbOCGz+jh3iKbG68SAHWBYCpSPBqlS0bo60= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=g1s0phY3; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="g1s0phY3" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e23ee3110fso21882477b3.1 for ; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584733; x=1729189533; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=g1s0phY3wpDJ54yACpNQxa2jJ5a2WcyMg39r6mcL0Y5WvOsI7TPZmKy87sNNnninoG zinstTd6W23/g/DTDy9vblMtUSS/Mw0aCsL5WxcWrXK9YHs7OO15+K5dRJE45N88mZ5k bMgGbOPlYZoLPGfXxDDp3SS2NsWU9F9TnainvF/SLhaC9R/T8+lj/750qjFxMw/9Xfiy Zwf1LPVrkRC722UWTl+lDusREgs7UPSV2fkeMFMKdlJAT0pFqDWDX48hjEcLhHzWX+ly PthT23TBgnEc1ZqcJZE7l64Sk6YgJ5BG3NR4GOREQp1+pZoy+Ws+Z8Vq7urpfzDZJtdV ARmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584733; x=1729189533; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=Dm04nk0HTW+z3stZThq1Onlr1cLmSEetL5xmSidEILaw4izB9A/gfa0II9s1QRC0IJ +9/piCuTA0Z5GzTGpIaKVMFHW1qY/Cz5JYXp0KjGiK08g+LyZPDaqm5Q0GTXxfph8hdY llAUSPyQYvpqJGofFLkzyZuAPIHXT5p9sdHTPk265wbPZP4CMGJCZUouatNnx3gPnxfH 08AQ8mFY/mEUc6UIMm8OKxPOsrMJSG/KZKVc554RzsLQjvnOzmnurtcwSI1eDj8R0w+l dGKGSQkgEDxtj6DZqhoy+etjpOtvaD1ozu0mhitmkQt7JFjRbYGDpYWS1zL+e08IKQYX oi8w== X-Forwarded-Encrypted: i=1; AJvYcCUDbfuNMbj21zDb2PN3hYSwnzmvXq9cx/SOkXk3PBFqR5HaJkSfUGhN9vHfiV9tsBTKqSjhtGNiffmt@vger.kernel.org X-Gm-Message-State: AOJu0YzMXeg0WqE+PMklRdkNThCgr4DHOUVGNnAdNnshh3uWCC+CDuOp PPNutFrOFe0ZUk6qUK9ty8kVHYZNj5URavAyJ+GuD2tz/yIC+S4E2tEGGUriFnKj50IdZhQdeKY A4g== X-Google-Smtp-Source: AGHT+IHJGx3vYA/ciscOnZoyMGBDqN0A5fNCXYXCqQ4jImN8v4VZ+ud/0qiiFEbLMXTR4U3Rg3+GOt3HIs4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6db:d257:b98 with SMTP id 00721157ae682-6e32215253bmr563917b3.3.1728584733164; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-20-seanjc@google.com> Subject: [PATCH v13 19/85] KVM: Introduce kvm_follow_pfn() to eventually replace "gfn_to_pfn" APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones From: David Stevens Introduce kvm_follow_pfn() to eventually supplant the various "gfn_to_pfn" APIs, albeit by adding more wrappers. The primary motivation of the new helper is to pass a structure instead of an ever changing set of parameters, e.g. so that tweaking the behavior, inputs, and/or outputs of the "to pfn" helpers doesn't require churning half of KVM. In the more distant future, the APIs exposed to arch code could also follow suit, e.g. by adding something akin to x86's "struct kvm_page_fault" when faulting in guest memory. But for now, the goal is purely to clean up KVM's "internal" MMU code. As part of the conversion, replace the write_fault, interruptible, and no-wait boolean flags with FOLL_WRITE, FOLL_INTERRUPTIBLE, and FOLL_NOWAIT respectively. Collecting the various FOLL_* flags into a single field will again ease the pain of passing new flags. Tested-by: Alex Bennée Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 162 +++++++++++++++++++++++--------------------- virt/kvm/kvm_mm.h | 20 +++++- virt/kvm/pfncache.c | 9 ++- 3 files changed, 109 insertions(+), 82 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 10071f31b2ca..52629ac26119 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2750,8 +2750,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. */ -static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *pfn) +static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page[1]; @@ -2760,14 +2759,13 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * or the caller allows to map a writable pfn for a read fault * request. */ - if (!(write_fault || writable)) + if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(addr, FOLL_WRITE, page)) { + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { *pfn = page_to_pfn(page[0]); - - if (writable) - *writable = true; + if (kfp->map_writable) + *kfp->map_writable = true; return true; } @@ -2778,8 +2776,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, - bool interruptible, bool *writable, kvm_pfn_t *pfn) +static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { /* * When a VCPU accesses a page that is not mapped into the secondary @@ -2792,34 +2789,30 @@ static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, * Note that get_user_page_fast_only() and FOLL_WRITE for now * implicitly honor NUMA hinting faults and don't need this flag. */ - unsigned int flags = FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT; - struct page *page; + unsigned int flags = FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT | kfp->flags; + struct page *page, *wpage; int npages; - if (writable) - *writable = write_fault; - - if (write_fault) - flags |= FOLL_WRITE; - if (no_wait) - flags |= FOLL_NOWAIT; - if (interruptible) - flags |= FOLL_INTERRUPTIBLE; - - npages = get_user_pages_unlocked(addr, 1, &page, flags); + npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages != 1) return npages; + if (!kfp->map_writable) + goto out; + + if (kfp->flags & FOLL_WRITE) { + *kfp->map_writable = true; + goto out; + } + /* map read fault as writable if possible */ - if (unlikely(!write_fault) && writable) { - struct page *wpage; - - if (get_user_page_fast_only(addr, FOLL_WRITE, &wpage)) { - *writable = true; - put_page(page); - page = wpage; - } + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { + *kfp->map_writable = true; + put_page(page); + page = wpage; } + +out: *pfn = page_to_pfn(page); return npages; } @@ -2846,10 +2839,10 @@ static int kvm_try_get_pfn(kvm_pfn_t pfn) } static int hva_to_pfn_remapped(struct vm_area_struct *vma, - unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *p_pfn) + struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { - struct follow_pfnmap_args args = { .vma = vma, .address = addr }; + struct follow_pfnmap_args args = { .vma = vma, .address = kfp->hva }; + bool write_fault = kfp->flags & FOLL_WRITE; kvm_pfn_t pfn; int r; @@ -2860,7 +2853,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * not call the fault handler, so do it here. */ bool unlocked = false; - r = fixup_user_fault(current->mm, addr, + r = fixup_user_fault(current->mm, kfp->hva, (write_fault ? FAULT_FLAG_WRITE : 0), &unlocked); if (unlocked) @@ -2878,8 +2871,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, goto out; } - if (writable) - *writable = args.writable; + if (kfp->map_writable) + *kfp->map_writable = args.writable; pfn = args.pfn; /* @@ -2908,22 +2901,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, return r; } -/* - * Pin guest page in memory and return its pfn. - * @addr: host virtual address which maps memory to the guest - * @interruptible: whether the process can be interrupted by non-fatal signals - * @no_wait: whether or not this function need to wait IO complete if the - * host page is not in the memory - * @write_fault: whether we should get a writable host page - * @writable: whether it allows to map a writable host page for !@write_fault - * - * The function will map a writable host page for these two cases: - * 1): @write_fault = true - * 2): @write_fault = false && @writable, @writable will tell the caller - * whether the mapping is writable. - */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) { struct vm_area_struct *vma; kvm_pfn_t pfn; @@ -2931,11 +2909,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, might_sleep(); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; - npages = hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, - writable, &pfn); + npages = hva_to_pfn_slow(kfp, &pfn); if (npages == 1) return pfn; if (npages == -EINTR || npages == -EAGAIN) @@ -2945,18 +2922,19 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, mmap_read_lock(current->mm); retry: - vma = vma_lookup(current->mm, addr); + vma = vma_lookup(current->mm, kfp->hva); if (vma == NULL) pfn = KVM_PFN_ERR_FAULT; else if (vma->vm_flags & (VM_IO | VM_PFNMAP)) { - r = hva_to_pfn_remapped(vma, addr, write_fault, writable, &pfn); + r = hva_to_pfn_remapped(vma, kfp, &pfn); if (r == -EAGAIN) goto retry; if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { - if (no_wait && vma_is_valid(vma, write_fault)) + if ((kfp->flags & FOLL_NOWAIT) && + vma_is_valid(vma, kfp->flags & FOLL_WRITE)) pfn = KVM_PFN_ERR_NEEDS_IO; else pfn = KVM_PFN_ERR_FAULT; @@ -2965,41 +2943,69 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, return pfn; } +static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) +{ + kfp->hva = __gfn_to_hva_many(kfp->slot, kfp->gfn, NULL, + kfp->flags & FOLL_WRITE); + + if (kfp->hva == KVM_HVA_ERR_RO_BAD) + return KVM_PFN_ERR_RO_FAULT; + + if (kvm_is_error_hva(kfp->hva)) + return KVM_PFN_NOSLOT; + + if (memslot_is_readonly(kfp->slot) && kfp->map_writable) { + *kfp->map_writable = false; + kfp->map_writable = NULL; + } + + return hva_to_pfn(kfp); +} + kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, bool write_fault, bool *writable) { - unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); - - if (kvm_is_error_hva(addr)) { - if (writable) - *writable = false; - - return addr == KVM_HVA_ERR_RO_BAD ? KVM_PFN_ERR_RO_FAULT : - KVM_PFN_NOSLOT; - } - - /* Do not map writable pfn in the readonly memslot. */ - if (writable && memslot_is_readonly(slot)) { - *writable = false; - writable = NULL; - } - - return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .map_writable = writable, + }; + + if (write_fault) + kfp.flags |= FOLL_WRITE; + if (no_wait) + kfp.flags |= FOLL_NOWAIT; + if (interruptible) + kfp.flags |= FOLL_INTERRUPTIBLE; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable); + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(kvm, gfn), + .gfn = gfn, + .flags = write_fault ? FOLL_WRITE : 0, + .map_writable = writable, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .flags = FOLL_WRITE, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 51f3fee4ca3f..d5a215958f06 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,24 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable); + +struct kvm_follow_pfn { + const struct kvm_memory_slot *slot; + const gfn_t gfn; + + unsigned long hva; + + /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ + unsigned int flags; + + /* + * If non-NULL, try to get a writable mapping even for a read fault. + * Set to true if a writable mapping was obtained. + */ + bool *map_writable; +}; + +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 32dc61f48c81..067daf9ad6ef 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,6 +159,12 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + struct kvm_follow_pfn kfp = { + .slot = gpc->memslot, + .gfn = gpa_to_gfn(gpc->gpa), + .flags = FOLL_WRITE, + .hva = gpc->uhva, + }; lockdep_assert_held(&gpc->refresh_lock); @@ -197,8 +203,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, true, NULL); + new_pfn = hva_to_pfn(&kfp); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830685 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEE751F4FDB for ; Thu, 10 Oct 2024 18:25:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584738; cv=none; b=ucYtTRoqSr0xByBU55bdFZmY4Lu7TwgFDcXPxKuUkRkULQ9G7ywLVBmUvs030HppnChGEQHl7UU8i/9NX6c+3zEt/+JZ8f55htAFLFibPXU/w/nR8NpjsydC8stNCSV3sp8IGwwicNnsFR1CMNJHGytnaUd6i+D65ydWNVkUbuM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584738; c=relaxed/simple; bh=KvCWyyHWjrlulYIoOxtt/Dqt8bcDl+mOtHXiCiz5pRo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cxJe2hn0rBP+9cESM8Mpr1ufg5X93mjXOtOnXhWj0J+n/kqxfoLKBEN5FItjfcerWaFINajVUKFQfc2IDIGqPaRx5GfdnpFA7RTliOa7Z+BMgGJpMuBpltyG7AIoovu5OXPsTXAkgD1aT2Yv2DPKTdHNILu06Igs84vYcSARBD8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aZapr+Qp; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aZapr+Qp" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e18d5b9a25so1639313a91.2 for ; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584735; x=1729189535; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=aZapr+QpZhS7r/MQaYKqDvjfYRBORnxInsXEL92L+scGg5hKtUtQE7o8nOHIrYusqm 5EmvoFkDqXFzl2HpKgkfAiP5lENBwGYDxj5waWmwhFK3gp6/VX/zM/witRAv91QhuvDh hk9MOt+wYPDWmF57nqd7mgAbHZd24J/E0TKSFO7QbNz5i+Ba9ZoHcPZ0IONz8KHYSAGl 5BXGeZ4jCqP2voRrg95+ZwUWfHnXmR2Cq5/ywlxqLxLBk2lAB2BuSmj2RKdIIYKnlJk1 iEfZhsO1MnnSNipUT/G2uAr44IdPub7PgGYmTru8gHSf1ad+5Bp4aUAXkTFpHx/6ALns SViA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584735; x=1729189535; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=H6jgNlRUE28Ns0EeJqccKQfGmRWNhrlslVizoMbt9a499d/tsrlibC6mHL8G2Q3ti8 QF1fBtn9yQf5kBRkBLxtJHyccu245GYfilzFMOhOIqTi9ZF9FsVgn74ZKvnvrbgmiQPJ mz3BjXDPlqaptdisNyy33QQ35nawohGR6rhD0o7PYtmtdihgkohtzh9h2K1xkBWgW2Fu KdrudpX7kVSP9L9EuXo3j/w1g/QK4smdBGV0W0sWWF3KOuRw5yQRR8VsxrfTlnB2MebU yGXXXFlryZJ1bxx8oWnEne+wdh35zfIJ7B0zJU/dbiYNfe6k0uM2m1dxfgwmbULzjU4r Pm0g== X-Forwarded-Encrypted: i=1; AJvYcCXaZOcBOIpz3iVkmqAf+TQu6c4Tb5xg99S1fKF59UJGvZVwxexUrW0Tw6LbHTTtzrvLfEKNOVCAwXTF@vger.kernel.org X-Gm-Message-State: AOJu0Yyj8hOQWAiyoIExAuRNIq9CAOgBEWCFkYwbNNcuB83lB0GDJRI5 eWnBC1UJkhaSiYmNFG9wtroiW0/0prd8+lV/wM1MiAAuYUBqueNvBJ4CwjZcL/Tl8hRFI/aYICR gEw== X-Google-Smtp-Source: AGHT+IEnZEGZphhdHlQgb0iOcJfNRdGKMtxwSRuFd5MkDGpZIOhDcg8ohYIeF4ySSwO5bqKUu8yK4lYjZ+4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:8595:b0:2e2:d600:3862 with SMTP id 98e67ed59e1d1-2e2f0d9dbfemr89a91.8.1728584735013; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-21-seanjc@google.com> Subject: [PATCH v13 20/85] KVM: Remove pointless sanity check on @map param to kvm_vcpu_(un)map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop kvm_vcpu_{,un}map()'s useless checks on @map being non-NULL. The map is 100% kernel controlled, any caller that passes a NULL pointer is broken and needs to be fixed, i.e. a crash due to a NULL pointer dereference is desirable (though obviously not as desirable as not having a bug in the first place). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 52629ac26119..c7691bc40389 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3071,9 +3071,6 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) void *hva = NULL; struct page *page = KVM_UNMAPPED_PAGE; - if (!map) - return -EINVAL; - pfn = gfn_to_pfn(vcpu->kvm, gfn); if (is_error_noslot_pfn(pfn)) return -EINVAL; @@ -3101,9 +3098,6 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) { - if (!map) - return; - if (!map->hva) return; From patchwork Thu Oct 10 18:23:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830686 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C63771F7084 for ; Thu, 10 Oct 2024 18:25:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584739; cv=none; b=qDVnRwLJbtXoqPMq73eQ7BL1PABlI6B7PQkSSYOOirAVNflw66X/dZEbmt0uCO58OwXCUKfsxxOTk1t7l7uRPw+fld8Aj36Ukk6zDBkAvdi+1IDBTAmnqT9MU5VbFCYfJObNVQzCbQE/3I6adKOfH2JcJ3JIG2JIgQfC7vmCFAM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584739; c=relaxed/simple; bh=w6CtoHT4iiBJ8oKluaibldhSnM94wRba+mvcEkROyJo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=liPoGHhPf0hAr1yR8rrDIuu5MDNOpTLJe9qZz5C5oq0TGj6Nyd/R8G89XLNfVT2r9RPJH9BEbhpp4qek5BwDjyPQ4k1eID5hSbqWqXeVZ8fAtzc3ibk3vUhYst1Nhi7GE+fDTSOkxD4pLlh3tib2tNXW54Fo0hItfit3Ki2Gcws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=B4VD+bqQ; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="B4VD+bqQ" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e0082c1dd0so24640047b3.3 for ; Thu, 10 Oct 2024 11:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584737; x=1729189537; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=B4VD+bqQ1bv6Pf45zN8plQZ+1nlQIV+x2YTm+XE8/QAFyOT6pF5ri3atTDy7wmWK1W cBuNjj+RZcYpDljbhbOPdKgUEc6sxDbKrQJ1fcQGXI1ipHd31NNylsUJEh9353nqzmBe xErqAc5drpMhDXL6T2FAUVJtAVGGhL9gh8F57tSlbW7u9huTn8DSiVUEoNHxzP9tsAAL App72uklOszMQmpZcdIaLkjDcH7P4uuH5tw33U+NlFCh5D1Z5egpssNiIn5bx95a2D6u zh0dl5vUT4MDK4fiWa6tECFRTbzyA11CMn2YXNF224VvrXZcgmYLwkzQLjyWsiB0wLbP KAFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584737; x=1729189537; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=mxwZKaMPYtAgxqGrVqiPVsGQ8bRJjiCYsmBbge5asuhDc4tSgimFQkrr+dSk1lfKFI tgVgej4FjIjUHyrMF02XO6YQPGK2alb2G7i9NeND6HjWGWYVUZA6wzgUbqcZu165MXFV m1hsmkEhitoKDYXjLfuzrlDqcZDM2KVUEj78SPzQe9WeJ4UhexVpV9jrH7bYYCyxS/NP sUO+SIxrT7rw0RSjVXiH95JSNit/GHBbc/QeD1S6yMe08o+40eMOT7o/5+UUP8eoBHfB +DX1AhpaP/+owAPwloQQfdpzFQUzpuzxR4b+G2XjzDXP3aIfg8WN/FaMhRdu/0Cw8+VB ZH8w== X-Forwarded-Encrypted: i=1; AJvYcCXTxlo0Apaqf+mXrNDGn94mn6MARQajoGQJcLvOLBjkQUes/9rlzcweKRPIewkg31ykBlHIIXi/fcdG@vger.kernel.org X-Gm-Message-State: AOJu0YzwAozkzRCMMLfx5oG5DAqEkDGp5cV2YzHtmVjY8zi2lR40bAt4 dMOwTzi3xeMI301MnvNbX24jQDXYGYSiAINgXGjtXInQUGyHVoscQ3pKwCFufEmjhFC1A0dc09G VHg== X-Google-Smtp-Source: AGHT+IGlSN4X6DbrP5DPPe6+MPgoNlxMoOT3GZzpJV2jr+xnXsAJlzTjTlsnBYkwN+CsyuU9UJbLGW89hxk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:c70a:0:b0:e28:eae7:f84a with SMTP id 3f1490d57ef6-e28fe0feaecmr5507276.0.1728584736880; Thu, 10 Oct 2024 11:25:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-22-seanjc@google.com> Subject: [PATCH v13 21/85] KVM: Explicitly initialize all fields at the start of kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Explicitly initialize the entire kvm_host_map structure when mapping a pfn, as some callers declare their struct on the stack, i.e. don't zero-initialize the struct, which makes the map->hva in kvm_vcpu_unmap() *very* suspect. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c7691bc40389..f1c9a781315c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,32 +3067,24 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { - kvm_pfn_t pfn; - void *hva = NULL; - struct page *page = KVM_UNMAPPED_PAGE; - - pfn = gfn_to_pfn(vcpu->kvm, gfn); - if (is_error_noslot_pfn(pfn)) - return -EINVAL; - - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - hva = kmap(page); -#ifdef CONFIG_HAS_IOMEM - } else { - hva = memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB); -#endif - } - - if (!hva) - return -EFAULT; - - map->page = page; - map->hva = hva; - map->pfn = pfn; + map->page = KVM_UNMAPPED_PAGE; + map->hva = NULL; map->gfn = gfn; - return 0; + map->pfn = gfn_to_pfn(vcpu->kvm, gfn); + if (is_error_noslot_pfn(map->pfn)) + return -EINVAL; + + if (pfn_valid(map->pfn)) { + map->page = pfn_to_page(map->pfn); + map->hva = kmap(map->page); +#ifdef CONFIG_HAS_IOMEM + } else { + map->hva = memremap(pfn_to_hpa(map->pfn), PAGE_SIZE, MEMREMAP_WB); +#endif + } + + return map->hva ? 0 : -EFAULT; } EXPORT_SYMBOL_GPL(kvm_vcpu_map); From patchwork Thu Oct 10 18:23:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830687 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEA191F8909 for ; Thu, 10 Oct 2024 18:25:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584742; cv=none; b=XzKHnEr28zCxCaJR6rnyOeqsPI7tPavzXK776sjwhwA4FpB/6ip5ZtsIgeSydz587YGryQfsc/JOAueFHPkSCYo2ljVHzbA6b4YcKy+iMeq6qjWGTZQ95ZAQINxBblCY1e+LEwcWiESamSUeYglpqkUChpLZ3fm9nLseBzAP868= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584742; c=relaxed/simple; bh=ZlT2BBZrHMf0x3c0Y2iKgnaJviq4ilGlr1zNhk4Vq/U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=CwO3PPyP2G1oqqZP1M9Z9x28JAXs+zotxiMg2/tSTmVKjWxi4HdvzzSWJrzLDMRnbVxCRfqGVEct6+wEwSf2iAj4WjchY5kxpnF2Wr6xXOwcq2ji/IHBomtAc1kG3kfc6rwj22kyTTpoGRNx6cYRSEZmMcHn0IMIRPsDLHobyp8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CjJawX0g; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CjJawX0g" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e284982a31so22432707b3.3 for ; Thu, 10 Oct 2024 11:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584739; x=1729189539; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=CjJawX0gL3DlpHXrFl3UettsKvgKZhWv4SqQA90BYqhykfLcxkE6Wu1vh/woj9cKRe WBeA5xaSPtiO2n25gFk5BFhGTdTAIXI9+cKopSHheHDsvMSXgv9GRVTZNsFw8UHjUIyO bgz5EqJfA36MxGkk/wKO81Pr6UfjRrL4/iYWSOGDvEZr2PKMKXa9vGQ9qv7HktdxPuOA rYvrX4Mva7y9IcOyZ2keW7/rxD4tkbLD7VQOI/NAXhIe4EeyM29lHpbclTyhvbd6iFmV yktNhWnPAFFG1WSt+zWdIjGhy2UW8v7gJwCEuCC8hBDIry/zGPL/EdBbYHZWYL9FX1C3 sl0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584739; x=1729189539; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=eWmzjXk11sEdKo4dRxN+NsNg1QeNbX6Yn0RYl+WUatHTiXc/l5DB2zUssz57bJiKai fxhZf8lWwIRCcvWoZdlN75Ijv3gaUXi95EZux5Wwma9MdD+tFOZMSNmCi+DZKgK6jyXU ruii1pNSq5c2+Ubjw4l7UNhPaWsLUxuE/x6q38IS6xQ/u172h08aLFU5l/mj755mMTH3 zSgM2iizI/K0CQmGs6/w0XNCzaRE1GxdeY0ijghCk1i+DctpB66hhIugZN4I+tsreShM NYdHJM28Ljl0IDn/Yafpu7MRxy3kqnyFVOcmb7SAzvbDXbzTfWnclWaEK/0exNRab/P8 FYOg== X-Forwarded-Encrypted: i=1; AJvYcCWkF/R/rHvtD+IzVvYIQptfTfXqltuLsoIXtSm+1m/9gxaq8QoAbxVfxD5OrcsjXryf8i/w7XH5zBsp@vger.kernel.org X-Gm-Message-State: AOJu0YxbEnL8/62OIQZRt0wIXTE4TM4N7iiFA5llDo/w3TaDZiCy+g8O 2AL4XI0W0xkoERaSCx1bjRv9i8+huRcOYYCnigCtnwMkB7v3D9SRj8tp/fD5Zqj493RSkREPmyc Gqg== X-Google-Smtp-Source: AGHT+IGZ+3mddEgfwOZrB7Mg/yEB+clUKORcl3gwcx/GeFypDRI64NMuGnBtjzJ0gTzObJ64BsofGpKHxtA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ab6d:0:b0:e28:f8e6:f4c6 with SMTP id 3f1490d57ef6-e28fe465652mr58758276.2.1728584738891; Thu, 10 Oct 2024 11:25:38 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-23-seanjc@google.com> Subject: [PATCH v13 22/85] KVM: Use NULL for struct page pointer to indicate mremapped memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop yet another unnecessary magic page value from KVM, as there's zero reason to use a poisoned pointer to indicate "no page". If KVM uses a NULL page pointer, the kernel will explode just as quickly as if KVM uses a poisoned pointer. Never mind the fact that such usage would be a blatant and egregious KVM bug. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 4 ---- virt/kvm/kvm_main.c | 4 ++-- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cbc7b9c04c14..e3c01cbbc41a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -273,16 +273,12 @@ enum { READING_SHADOW_PAGE_TABLES, }; -#define KVM_UNMAPPED_PAGE ((void *) 0x500 + POISON_POINTER_DELTA) - struct kvm_host_map { /* * Only valid if the 'pfn' is managed by the host kernel (i.e. There is * a 'struct page' for it. When using mem= kernel parameter some memory * can be used as guest memory but they are not managed by host * kernel). - * If 'pfn' is not managed by the host kernel, this field is - * initialized to KVM_UNMAPPED_PAGE. */ struct page *page; void *hva; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f1c9a781315c..7acb1a8af2e4 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,7 +3067,7 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { - map->page = KVM_UNMAPPED_PAGE; + map->page = NULL; map->hva = NULL; map->gfn = gfn; @@ -3093,7 +3093,7 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (!map->hva) return; - if (map->page != KVM_UNMAPPED_PAGE) + if (map->page) kunmap(map->page); #ifdef CONFIG_HAS_IOMEM else From patchwork Thu Oct 10 18:23:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830688 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A6951F8938 for ; Thu, 10 Oct 2024 18:25:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584743; cv=none; b=bHPu+AuVRAaOhOU1arvuNFtesZ0vhon1vxZClgmEVZ5RfZuz1dB8EJdYUgetzdc12Byu1Hufs0Zo9nZHtY9xZ2/AMvvvCb4xWWNqcJ8towjBSvfqzFTcePa/0HMRkab+9maSeFp0ToOIKeHLk0RxsCpmh0mz/3azi5xbrI8DMcI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584743; c=relaxed/simple; bh=Z6RyJQ8NNQkeYyg1tN/wL4j+YrBUDlf6269dIZP90as=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tnrhrVgK4PTm358XA32p3uarhE3UFhOhnsfbx9AslYrYgwyouk9hb/C75fv/ZfIL6zP5+FBTNwJO86Sr2ADL+DnPggarUqJCvtLp6G/gIkohLSfDabLVYwmyroOza9Wb46Y3MlbGzXkHA/WAwGKwMDRrzKXPQ/mvzWfEMwmK1rc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qoM+EiSr; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qoM+EiSr" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20c8a637b77so13049525ad.2 for ; Thu, 10 Oct 2024 11:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584741; x=1729189541; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=qoM+EiSrBnuwmjbt/H8tKBvxJwaIx0hkbfvehONsdCk9vjqOswD4mvEj0K1OvuyMS8 Xefy/51XOxUXr3lsl7Gxr0tNwtj+ZPlYKogfR3q1PQN1U7OSpY3Nf2KyigCmie4XlfOu 2xoglyZSTIcyp+jlQO9uqGd3hDo0QHuTSL4kl7jncKIHd+kyxUPUGGTYKg6YzSg7q58o hz32KEqkByEhfBdPCd4ourTLtS05BCrpdXQYuWlkGQ0tWMIgXq7FyYT20RKpzbGXrhJW eTw/7JHF+f6c5jdj0TmxNhUAaWushq/8RPGlbiYwR+r7c/2SWrqGoB8HMLioJXH2uYJ/ aKWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584741; x=1729189541; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=r25jPQC5STPSynrSpkgMzCFYNFTwKpopENh7c9DAl9EvBIs821daelufZslft6WBm7 jX+SD1XuKfzuYKoT0X1G8WLsDPuxNeClSBab2RKaWF/k78muTFz3TRp2Oy8Ax28i+IP1 dKqk/Nv3HdFx6D48GFpYQRopOs0VA+nU75K9QrROJoP6XmdLU39CDkOVbqtAmnNegVTg /6LeA8x+VWmu60flvDv8DTX0H0QqVSc3bM3SwkhlmIkJJbCvKXIkiRf2TPrPAwniDhCs CVN95cZ+W3LwAemVxf7lKjBCGjzZkOSww5oh8V2D4AM6UgT36oB4T9xEpCNwhkR0oKwa GrgQ== X-Forwarded-Encrypted: i=1; AJvYcCW69BhhblbO2UWKacAnaul3laqKJMx/8bEP8JrKghGnaRgc8A3cytnFBzXhV38Pks7ZhuiYhZhYcBPj@vger.kernel.org X-Gm-Message-State: AOJu0YzVAUqARD+dawt70ZRYwFUWsMI7A2Du8+9moIN07+XQLnypBglH gsCQZ6EwWFPNAImDXPj29LZHgwMGg7ZW7PBrQlYAYUg6NhiDzLgoryl2j+ipUlGRwS8ILCMBuiN AeA== X-Google-Smtp-Source: AGHT+IFSsaj7YGs0xXvL7KvHpo4OHZS23TfW901WnGn1dNBw4UEq8HUBbyPz7tDSJuBwPv672jv+MkN6+4M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2349:b0:20b:9365:e6e4 with SMTP id d9443c01a7336-20c63782bdcmr1094885ad.9.1728584740809; Thu, 10 Oct 2024 11:25:40 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-24-seanjc@google.com> Subject: [PATCH v13 23/85] KVM: nVMX: Rely on kvm_vcpu_unmap() to track validity of eVMCS mapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Remove the explicit evmptr12 validity check when deciding whether or not to unmap the eVMCS pointer, and instead rely on kvm_vcpu_unmap() to play nice with a NULL map->hva, i.e. to do nothing if the map is invalid. Note, vmx->nested.hv_evmcs_map is zero-allocated along with the rest of vcpu_vmx, i.e. the map starts out invalid/NULL. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index a8e7bc04d9bf..e94a25373a59 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,11 +231,8 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); - if (nested_vmx_is_evmptr12_valid(vmx)) { - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); - vmx->nested.hv_evmcs = NULL; - } - + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + vmx->nested.hv_evmcs = NULL; vmx->nested.hv_evmcs_vmptr = EVMPTR_INVALID; if (hv_vcpu) { From patchwork Thu Oct 10 18:23:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830689 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3AFE1F8F17 for ; Thu, 10 Oct 2024 18:25:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584745; cv=none; b=YY1bWoeA7xS6G7CgUoLAwLEfjx3iYkapq9J4oOHQ8gH56DFPD6cK4n1xmmcnQPgP15R/XhmzT0S+PeccmBY63KA5PAYb2/M+Rpohz24jj84Is5FOeJ1yxGSN6JN9AEHrYQXZgQKTaY0Cpf6Y+EUW03CaoVoWVsv3OdJ0Q9XgYs0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584745; c=relaxed/simple; bh=QVyL+E+J2Fu+8nHNQtozyIIO+ywn/WvnTCVSYx092RU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=n+JFqhZa9hUc43g5KnXDLRtNby31r7NrKB7Hl4A4WtQgSiXewYhBU90NaxrwhAMvSmbv1WbRlhZSKzuTDcUx2U3OjJF8k2pgDH0CWAgfq9QWQLX2D4aLixWOpV+uP/SUQm9Xt5kE/IvfPUujJJl7G/WCXdOLTSJ+Wf+op/sCiRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o9hLcB8T; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o9hLcB8T" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e29097b84c8so1994363276.3 for ; Thu, 10 Oct 2024 11:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584743; x=1729189543; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=o9hLcB8TcgrAYeRwjVSZvm39dElO8cyvKzpeBbUmWSYLS3W2vdIeqdtka+VJm68gJL Mt1xP26lna3GmuKsA5M6XaLxLrcnmZpwiPqK97xslR+y8/OD8joZ6hTsWJkepTSytJ4/ 5qQ/cvFl9Lhgl/9BAV4EsrHv5625y4MIqZDYmniqgzndR8v9BPPB25oMftqfGBWKejna 3Sh6VKQ7Q19iuz1MdTuXO7tL450UYOL13DDXwlw+nZBSCcl5e+WElcxqudAEceZSwbPv HeAmyiYjnu6f8gH9Kx4SOq0jxKrIFi4JcIiq3wGhRqCUNlPtIfkJbZnXdKTR6rDaWD7o bGSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584743; x=1729189543; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=D98VDVwB3JU28CfkvRvGsai6ifSvJV1f2C/pYFw2KrgE2/n6Z58smatQHvSlr+o75/ 5h/oModGhc/AkxgY9dFqFXQCv4kfOhP+7JkMWbN7y7i6hWZzClMC6U7qSZ0ZXfFdIHU9 h7wMDyuKZNwgUJKhf7lpkzfKm+Bebp/PU8OTfS85gReRI3eoFN/9y9r476DWLnEbo1yI oM92/cDiwTcaEtOfQ2FHpp1tL4VJzN/oMbe64c/y64biafP8DHq6TV828PjTUc1GnrVL ICK5RrqJGdlOZ03OmTu27evf1EvNrcHsUqm1zVfNkSTcv87rmJ7zXBLqS1EqVhuM5qrS FyFQ== X-Forwarded-Encrypted: i=1; AJvYcCVR2reEXq7SeHnMjkd34UkynnXWNXDIsxVERGnsMOjXQs3Nk5ZtoeBtTYdY4N9qlcJnhTHD9P2Ia5dH@vger.kernel.org X-Gm-Message-State: AOJu0Yw8DeTkz8L9Xi4uvBx5UC+p9hHOE0mypRPTx/2O5lyqi12ix+3G +ngWuffyxOF6QCRUb3uOFbjFAApu5D66uRlnqX2rKZGGgedQg3+fuOJB0x2RKEEscLc4xrRVyZh 1fQ== X-Google-Smtp-Source: AGHT+IGUAGxjHNELfO1gnQBoxyShlnhLiinRslmifWPsD1igdN7s2BU3B9tdatRPL4jwPkA1Z0qI9X+JjYQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:9346:0:b0:e28:fc1b:66bb with SMTP id 3f1490d57ef6-e28fe4f0fddmr4836276.6.1728584742775; Thu, 10 Oct 2024 11:25:42 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-25-seanjc@google.com> Subject: [PATCH v13 24/85] KVM: nVMX: Drop pointless msr_bitmap_map field from struct nested_vmx From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Remove vcpu_vmx.msr_bitmap_map and instead use an on-stack structure in the one function that uses the map, nested_vmx_prepare_msr_bitmap(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 8 ++++---- arch/x86/kvm/vmx/vmx.h | 2 -- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e94a25373a59..fb37658b62c9 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -621,7 +621,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map *map = &vmx->nested.msr_bitmap_map; + struct kvm_host_map msr_bitmap_map; /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -644,10 +644,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, return true; } - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), map)) + if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) return false; - msr_bitmap_l1 = (unsigned long *)map->hva; + msr_bitmap_l1 = (unsigned long *)msr_bitmap_map.hva; /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -711,7 +711,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); - kvm_vcpu_unmap(vcpu, &vmx->nested.msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); vmx->nested.force_msr_bitmap_recalc = false; diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 2325f773a20b..40303b43da6c 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -200,8 +200,6 @@ struct nested_vmx { struct kvm_host_map virtual_apic_map; struct kvm_host_map pi_desc_map; - struct kvm_host_map msr_bitmap_map; - struct pi_desc *pi_desc; bool pi_pending; u16 posted_intr_nv; From patchwork Thu Oct 10 18:23:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830690 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 911471F9420 for ; Thu, 10 Oct 2024 18:25:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584747; cv=none; b=XUQaeZHEVj8ygN12yIMqTO3jgdrLj4CMOcyOWB+QsQaYGPg5nc6gF1FwIAoRPJgyHTEvDauSkRhErMGnHT+EFt9kcNwtbtVPX+5YK5VnXEgV4sYW1TfgE2w0wbNS9fYRma03hZfDAMUR+QYAAl09pRUsCFImNTbH1PUcpDvyq5o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584747; c=relaxed/simple; bh=H3zY/Xt0t+c4LMWpCCUO1umdZ2mHGDI4Sis4Gpw2f4E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=V0HI4gfCzEjFycC/+wOYIV22+VWWgInbTRcNeORlSqKsMJe75U4kZuxLXoV/PUWCLsTOniKXswhiV6ywrioK/mageKeMZ69iSQyA/r6SAKvnRpHIbnpWUUgC7TIZYshL7UX3BJJyCDytBp+kZXyWBqNGR4Z4S6je//iJgY3ApZs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dOB3fNA4; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dOB3fNA4" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6886cd07673so27431547b3.3 for ; Thu, 10 Oct 2024 11:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584745; x=1729189545; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=dOB3fNA47Ys/t2RsrBoFMYei+S0bW216ulRwWrnfXEdN8iMvrCGNtKMihXfXcvqKyO UwHBwAHc8CUabZz6ufuXBq9dDxqCduiUDhRaekZSGdsbD//eJIWik8hOLt/7s3rfOiEJ z1Uug1P8XEYf+oxXnQpeR8VnmbIOdQv4uB4mOAhxzWuOX9ZERNYGDzAES/UqOG5jy5PV rtiiw/BOtjrl63GJm9GZUYSl6rhmyGpAthr4RNSdMa+mvLJhz3Sbj/1MdqYhqoqW/Pe4 JhLDjNtSaQNO119lhq8ZFAG0O0b9ck7954dyAKa4UuBRfnmamD6eqSzGHFWMXD7lGYjg mLVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584745; x=1729189545; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=eFo2RoBeJZjidYJAZGtFa/evKREB64PJgYG8UcZ4HBFYZKt7IF5Tj/sjqp607EVqaj 0O7QXo3TZhLs7SmdVZLpfVtDp/nPDsyDlNBmK61Uz+1dGhkfyWdobiqGXSglKwWYKktF BPJtd8I38RmjVRd34qUaRis9bTHlYRqAFZOOIltGhOQGDBmMYCIV+iuCgFLSHcxP+r6+ 4VxZrKdI2aTEoFEt16snQGPF0g2D91mOHMnfeX9I4Bca/x9VWa7S3c3R69DTCiOFKb4l CGmbUunCXg7oQ4y9L3ibB0JLHKQ/J7jsbG3VmPzyFGPmp/wBcLZUbihXiaImW0TfgLm8 OxMA== X-Forwarded-Encrypted: i=1; AJvYcCXFc/uNwZbZw/3vxKIOiHJ7Nv5UMt8GTmcmqXSRFT1SvtUMzuCBkXgN6zSWXNEcngbzYPs9pF8tY7+H@vger.kernel.org X-Gm-Message-State: AOJu0Ywl/BV4SU8S7StDg8/YVVGeRLy+SaaqAWEYKc4oQSHiowQYkuVS VVJYuwoiJsu+v9JjQeyuQzt3mWQgFRz6CFoHO1FuLJrSxCg18B3bg5T9I2iHiEhoCiWrNpUtqYP +MQ== X-Google-Smtp-Source: AGHT+IEdX//xUJq0LlzHSFsWnzcNANrEE9KtnndGvFzP5BAu23zXYFVXA6eeBi51F5M5i08FgAsxiLe5HbY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:300b:b0:6e3:f32:5fc8 with SMTP id 00721157ae682-6e32217bbe0mr502817b3.1.1728584744665; Thu, 10 Oct 2024 11:25:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-26-seanjc@google.com> Subject: [PATCH v13 25/85] KVM: nVMX: Add helper to put (unmap) vmcs12 pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Add a helper to dedup unmapping the vmcs12 pages. This will reduce the amount of churn when a future patch refactors the kvm_vcpu_unmap() API. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index fb37658b62c9..81865db18e12 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -314,6 +314,21 @@ static void vmx_switch_vmcs(struct kvm_vcpu *vcpu, struct loaded_vmcs *vmcs) vcpu->arch.regs_dirty = 0; } +static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + + /* + * Unpin physical memory we referred to in the vmcs02. The APIC access + * page's backing page (yeah, confusing) shouldn't actually be accessed, + * and if it is written, the contents are irrelevant. + */ + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + vmx->nested.pi_desc = NULL; +} + /* * Free whatever needs to be freed from vmx->nested when L1 goes down, or * just stops using VMX. @@ -346,15 +361,8 @@ static void free_nested(struct kvm_vcpu *vcpu) vmx->nested.cached_vmcs12 = NULL; kfree(vmx->nested.cached_shadow_vmcs12); vmx->nested.cached_shadow_vmcs12 = NULL; - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc = NULL; + + nested_put_vmcs12_pages(vcpu); kvm_mmu_free_roots(vcpu->kvm, &vcpu->arch.guest_mmu, KVM_MMU_ROOTS_ALL); @@ -5010,11 +5018,7 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, vmx_update_cpu_dirty_logging(vcpu); } - /* Unpin physical memory we referred to in vmcs02 */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc = NULL; + nested_put_vmcs12_pages(vcpu); if (vmx->nested.reload_vmcs01_apic_access_page) { vmx->nested.reload_vmcs01_apic_access_page = false; From patchwork Thu Oct 10 18:23:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830691 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8AF21F9A8D for ; Thu, 10 Oct 2024 18:25:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584750; cv=none; b=L6XbSNki73oCTxhBghPwjB5E0UK39pbY+arRaBnzGGXaeoMztBTmZyuzG8RgygcvqQrsdS/bQsG4FFJH3cQFysidKjxnpVaQt+SeqvA3k2jpaPjMH/wXqhs2Vl4WWLwecoRpZdbG5DxyD3rlfisyDHM2Mazaf8D/tbz5nxh9pCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584750; c=relaxed/simple; bh=wQkcrwrrK4R8UDZylZK1jjbxKHOYfzFAR77NBuuXUsk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m/cs1cyEZlZ0qBRu0ZTl8aUq7515eTnNPdcEOesdqYZoeiczasnsMGJ2ZLdONoyucFF6Yz2JK8HOvDctanyTEO2YFSE8kXjnYQ5eAC0/v/ktgsFqZ6ZxCs1r3kVc6R+wmBIjxj2cVVYrxaq7iFrrd8voIlLxSGx9B5d84taXEBo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=be7NmGNy; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="be7NmGNy" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2ca4fb175so1247940a91.3 for ; Thu, 10 Oct 2024 11:25:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584747; x=1729189547; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=be7NmGNypoYKlIQHzeI5qmlCcUnBhnj66Rq1h3NtOxbOPwC+w2QWEewadPkTALHHw3 xPJKAp195KEPkglLi++7I4Q8+YOmOUjNKY8vER3KsZeeMIagBJvsGt5Pobsm74JYsbU6 LTrXc4Yk9chzFDVCyczjOYxiAQl3ChwlolnTSQxMAaBK84r8sAvRfe70ej1wxuwkkAso K642YYl0QtppHrwBVSCdBwaujah/UYJ3uFn3uGnTv6BDtvRNYAJCNnt+vptRV4XD1YyB zB9Nu1ry3UFXALvaKzFf8mG173wYfV1cR9LVTLoNYadPVMufgcCdJQL3tQsG1XncstE1 EjmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584747; x=1729189547; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=KzKAdiUmlPVxK2rh1kMYw+ZTfr4lf/hGb4u9zn+iir89AgVX0lK2IK3AEXImFKSSSD eRtjHjHNO2BRHW+bW9j2g6uexO2T4VquNGksGGBrPxbp75FcSifPrhOkRgo7ArArr9yV WUBMuSB5QC28wGo2X7g5Lp/e9iLYp+frIPlIeITOuvcOj+m9ELifl0A/r15/H8rROlOh DD3p26vuph+gN9jUVxubvedv+b7BmZExgoZI2pCF1oShQnZuCRBhnG15oteUFyay9fhX k+pWNGCuN3hUuk46Gjxo5V8WdVay23uXPzZ8QiHDr8AxQI+Ihmi0e5NgLWn319WJL1Pv U97A== X-Forwarded-Encrypted: i=1; AJvYcCWTLMo60nLEni7tcZmMS2stVvU2GYK71O55ovR9vpUb/JSyxZLyX99qk5E4VmyXMZgoZAwiZclx02FR@vger.kernel.org X-Gm-Message-State: AOJu0Yyz2Vi4/gl4PmPrHGIhuF/N22o/lwaKFCV53puLm1vbSiX9GkUy xEHS2LGYTEBUnO0G4xXSQcw1tWadtNXPnW5SS682WOcA1wOiFUk8MbnHUdysZnwMDSAiIuvucDr ZCg== X-Google-Smtp-Source: AGHT+IFGWHaFUEsQRxn4rcRWabj8vzxqpLHdNOXqOmiiQlX4dD6MZqF5BpShV2hVfQIOX/PTyNMVtEMSya8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr109a91.1.1728584746508; Thu, 10 Oct 2024 11:25:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:28 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-27-seanjc@google.com> Subject: [PATCH v13 26/85] KVM: Use plain "struct page" pointer instead of single-entry array From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use a single pointer instead of a single-entry array for the struct page pointer in hva_to_pfn_fast(). Using an array makes the code unnecessarily annoying to read and update. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7acb1a8af2e4..d3e48fcc4fb0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2752,7 +2752,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w */ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { - struct page *page[1]; + struct page *page; /* * Fast pin a writable pfn only if it is a write fault request @@ -2762,8 +2762,8 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { - *pfn = page_to_pfn(page[0]); + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + *pfn = page_to_pfn(page); if (kfp->map_writable) *kfp->map_writable = true; return true; From patchwork Thu Oct 10 18:23:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830692 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58FD41FA261 for ; Thu, 10 Oct 2024 18:25:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584752; cv=none; b=Kn1SW5PdTt7cF8r9pVfOygJxvK+ic+vnSdFEbsN5Cu1VRs8qG9T5xfxE61mVkOVXgJ3hCVt08AnZdxIEDB/bHFonOExihFBdcgNvPErPcK5q0aPPq2B9+1Ye7lS4QBXOuv/dCt+HIExVYlYgl3LlkpAReNHClWQxENjqTFYmMfQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584752; c=relaxed/simple; bh=5U+Xe396ie07CWKh5LcgsN1ZYAsnCq4iDqWBxnyRpWQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sssgNPwZhxP072SNuNUJutTkTzH/HZwX8IGfMsRnTPFXnYuYNep6ir56FtQQD2R2rBwfJHf4Xx8+YU7RxUIHpCJxGpC7/57EKaLlSCq2k50Zk+DLyjm/ZPK/G3BpxVNE3HthakeX8EN08G6oL0crbP1yYFx6a4Plj/sYpjGj03U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mJXsN75Y; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mJXsN75Y" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71de5bff982so1491001b3a.2 for ; Thu, 10 Oct 2024 11:25:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584750; x=1729189550; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=mJXsN75Yca/FiAuy9ZZmO+iv6xILrS0c143N4G74Zg1mRonN8SudEZyx8Mw0SXvCSD ca4pXlX/A6lQVXbuKE1V20Lz5c5rCM8UK6t4ptPeRFLLOOLCovrIkYCA5qjUvkyVmd6h b7W7ux2NIrfEY/UfWHi8QurEOCGya8rHmWll+dwIhtF3dvvrK+rPgxorFq+8NaICEejl SNSG3iwJQ0jfY7Pme/IAMy9untERtw3E9nukkjggK/aixQ9Dy4mmU7t/eO8SmvsfJj/p rBl1lE4jI0h3ZnunsVG363jXBaiJauyAPgZjFEMhRdBifBhSIzSxHpQSNFuS0dFWG8fq jQcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584750; x=1729189550; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=Ys2aehjfeabuLBkZlHcQAVeRFdf/ugpzfXAdDkNnhsuQGSiVLuckoAkUpfyA9c5P3R GqICJPGNPxFM+vLyoOQMrp/hR7LXV1XOsilLyZxQJjuORPIIryxnsRsPurgHDTiVfE4Q 53ET86kT+TNmBqYsNO+eQ0rzKySO3VvR5MISUbUWZWuWc6B1X8hAY7La1mthdktPxrQO Lhhme9ASIOzLYgvxpkvwrO0IhSJLeLsMi0od3UbKuFxqIIfp4MQt12b1tfEPNZfz47VH kvNbiNRk2aHyYg7Lwyex2wwnq5fJ7C6YfpfPcjqEcD2iHugQvwxUOFuI+R+4VKDsrG/L T4TQ== X-Forwarded-Encrypted: i=1; AJvYcCVc1Fwa34y+t6krmZMUtsTLimSwKgfTEzo43AtEqwRajZh5FbfjYcG4HgQpA8inqnRt2A98t3lPjV9y@vger.kernel.org X-Gm-Message-State: AOJu0Yy8w8Od1nWc9m0uM1lnYzEApVToQs1xFFwLNLjA+GYLz+Nj7EKo WyKL7SBeaDPwagknKrEMcmmzXGwK3cZtTP9R6QP/kwBKvANsFJjb5pYHHCNku5FanaHv68u4zBO kEg== X-Google-Smtp-Source: AGHT+IE/uLa8aldzd4vJMnq+De69DiThAuVt+3q8zF6rqxlBj2/ZKo7+FHCoJyy45GpCD4mZjJfGV3U05Tk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:f1f9:b982 with SMTP id d2e1a72fcca58-71e1dc00b9cmr38199b3a.6.1728584749206; Thu, 10 Oct 2024 11:25:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:29 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-28-seanjc@google.com> Subject: [PATCH v13 27/85] KVM: Provide refcounted page as output field in struct kvm_follow_pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Add kvm_follow_pfn.refcounted_page as an output for the "to pfn" APIs to "return" the struct page that is associated with the returned pfn (if KVM acquired a reference to the page). This will eventually allow removing KVM's hacky kvm_pfn_to_refcounted_page() code, which is error prone and can't detect pfns that are valid, but aren't (currently) refcounted. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 99 +++++++++++++++++++++------------------------ virt/kvm/kvm_mm.h | 9 +++++ 2 files changed, 56 insertions(+), 52 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d3e48fcc4fb0..e29f78ed6f48 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,46 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } +static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, + struct follow_pfnmap_args *map, bool writable) +{ + kvm_pfn_t pfn; + + WARN_ON_ONCE(!!page == !!map); + + if (kfp->map_writable) + *kfp->map_writable = writable; + + /* + * FIXME: Remove this once KVM no longer blindly calls put_page() on + * every pfn that points at a struct page. + * + * Get a reference for follow_pte() pfns if they happen to point at a + * struct page, as KVM will ultimately call kvm_release_pfn_clean() on + * the returned pfn, i.e. KVM expects to have a reference. + * + * Certain IO or PFNMAP mappings can be backed with valid struct pages, + * but be allocated without refcounting, e.g. tail pages of + * non-compound higher order allocations. Grabbing and putting a + * reference to such pages would cause KVM to prematurely free a page + * it doesn't own (KVM gets and puts the one and only reference). + * Don't allow those pages until the FIXME is resolved. + */ + if (map) { + pfn = map->pfn; + page = kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } else { + pfn = page_to_pfn(page); + } + + if (kfp->refcounted_page) + *kfp->refcounted_page = page; + + return pfn; +} + /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2763,9 +2803,7 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) return false; if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { - *pfn = page_to_pfn(page); - if (kfp->map_writable) - *kfp->map_writable = true; + *pfn = kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2797,23 +2835,15 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) if (npages != 1) return npages; - if (!kfp->map_writable) - goto out; - - if (kfp->flags & FOLL_WRITE) { - *kfp->map_writable = true; - goto out; - } - /* map read fault as writable if possible */ - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { - *kfp->map_writable = true; + if (!(flags & FOLL_WRITE) && kfp->map_writable && + get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { put_page(page); page = wpage; + flags |= FOLL_WRITE; } -out: - *pfn = page_to_pfn(page); + *pfn = kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -2828,22 +2858,11 @@ static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault) return true; } -static int kvm_try_get_pfn(kvm_pfn_t pfn) -{ - struct page *page = kvm_pfn_to_refcounted_page(pfn); - - if (!page) - return 1; - - return get_page_unless_zero(page); -} - static int hva_to_pfn_remapped(struct vm_area_struct *vma, struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { struct follow_pfnmap_args args = { .vma = vma, .address = kfp->hva }; bool write_fault = kfp->flags & FOLL_WRITE; - kvm_pfn_t pfn; int r; r = follow_pfnmap_start(&args); @@ -2867,37 +2886,13 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, } if (write_fault && !args.writable) { - pfn = KVM_PFN_ERR_RO_FAULT; + *p_pfn = KVM_PFN_ERR_RO_FAULT; goto out; } - if (kfp->map_writable) - *kfp->map_writable = args.writable; - pfn = args.pfn; - - /* - * Get a reference here because callers of *hva_to_pfn* and - * *gfn_to_pfn* ultimately call kvm_release_pfn_clean on the - * returned pfn. This is only needed if the VMA has VM_MIXEDMAP - * set, but the kvm_try_get_pfn/kvm_release_pfn_clean pair will - * simply do nothing for reserved pfns. - * - * Whoever called remap_pfn_range is also going to call e.g. - * unmap_mapping_range before the underlying pages are freed, - * causing a call to our MMU notifier. - * - * Certain IO or PFNMAP mappings can be backed with valid - * struct pages, but be allocated without refcounting e.g., - * tail pages of non-compound higher order allocations, which - * would then underflow the refcount when the caller does the - * required put_page. Don't allow those pages here. - */ - if (!kvm_try_get_pfn(pfn)) - r = -EFAULT; + *p_pfn = kvm_resolve_pfn(kfp, NULL, &args, args.writable); out: follow_pfnmap_end(&args); - *p_pfn = pfn; - return r; } diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d5a215958f06..d3ac1ba8ba66 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -35,6 +35,15 @@ struct kvm_follow_pfn { * Set to true if a writable mapping was obtained. */ bool *map_writable; + + /* + * Optional output. Set to a valid "struct page" if the returned pfn + * is for a refcounted or pinned struct page, NULL if the returned pfn + * has no struct page or if the struct page is not being refcounted + * (e.g. tail pages of non-compound higher order allocations from + * IO/PFNMAP mappings). + */ + struct page **refcounted_page; }; kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); From patchwork Thu Oct 10 18:23:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830693 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 123AE1FA27A for ; Thu, 10 Oct 2024 18:25:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584753; cv=none; b=vE2oi2jQakWW9P3w44Rq7ahkkWLbXQ+s46jisK9vfyGWzMK7Anr7BDJ9WSZHV17QYkTg9TOwfRZHLqcMAtDeY4vsj8eM4x7TRkKS4zZMpbcWuySkuvhkjY1h7MEfl+cGNQKhQjpt054+rzIWqR8PTWwotNigr1dg9jVTwe4M5Gc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584753; c=relaxed/simple; bh=CmHVeWZ83J0EQfDqM5RvAa04rCEwL0d1DQJqfJodvCg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g1tpmmO6DCucNtnSwdJJiWBKZPNFunZ6T3Nm1noOhd52aYmA3tXkWWW44TLwnJPLwdjkP3wDenCHMjUdO/EmcYZZwcGGVUVEarTz3HpJYUbPZ9oYBu+8EV4CvWRP7wiH8kuBAQQG3hyqcvV85PNagqtk9wxOhXEzg5S/z/AObnY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FXMkCkX5; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FXMkCkX5" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e1346983c8so1277787a91.0 for ; Thu, 10 Oct 2024 11:25:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584751; x=1729189551; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=FXMkCkX57wh+XHLq6n+XdE6kGmp0hkZZ6UA+5owP77BOfoNtvC7ytMt+Xm91GK2Ptt Z/XHN/Hk1/ZsHTw8wUK6ItSDN1PZ/Fd39gXJ0/20Uus/oeHrpT4bHO1ml/gJ2PjphReY RrNOOnUdwahhvvYNSbZQ/1IpMiASQC2EYHaf0+95rIjmy89mPbWSyHnf4XImQefy6klx RNtpXcGLTqbVqH6LtcwuGjG4TL0uJhOqs7pHAce0CgWIm6XbodXJ/ME1WZd0CQ9+NL1R U+EB7wNh59mYw7+giGo4f0SmCXTtVP1ct2vdXaUQ41Qoz11pFqCyoPvQon1ikHXhiajl 4aPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584751; x=1729189551; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=aZCD8kKnuS/g7SZ3ZEecqai6y8dSGcjAjI4pndPV1xNUUD4vfu4ZymT9QiEblrCea6 nEIK3YXMpq8D8RIf2SOE4dnOeJ3oGFC687SfLEId3Cd/tpfp5rDjMjPbdZyptbAnhrWe J+EPqbWHSkTARHV2MuRkaxw/1M5JwCOBqajE3/5MF8M33rnKICZO2Q/92PnISqN/31D0 tWIHywrAE/uWv/fWxWpOS+pjdF5gG4LNyB/txrMOdzgvgW82HpDFOKeBFwhhgpQGvUY2 77sAskDDDVkwtwQN5cW91p3Efs6XaLOwpyWfZktZr9WToVDFpxWAPdwqI8gSPVOYPDx1 b3dA== X-Forwarded-Encrypted: i=1; AJvYcCXjsTN9q8c71L82T4lHW8JeYgfdzC9HNDipYsY//Ox7sZbXQjiXRxQjV2mjlFlHc11cHqozfgFc4Dr6@vger.kernel.org X-Gm-Message-State: AOJu0YyJ8U3HrFFBEREZddXELyEbhA90VqP2hj/uXd4NXceVORi8qx8k O9ff3pCYpJDylnVrwWUA6Jq0nk0E6Qg1gsr9gIxL8Pg8Cg7qBCe3Ke8nqKdEZybYTRFgntzgplH EDg== X-Google-Smtp-Source: AGHT+IHahb+vMbBUybe9JXpaiwYyoag4sz2flsLCX6vY5d7t+wo3V9TaPVQYV32tHeuijXZHqATezWyezMk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:d913:b0:2e2:a810:c3e4 with SMTP id 98e67ed59e1d1-2e2c81d4d19mr7321a91.4.1728584751203; Thu, 10 Oct 2024 11:25:51 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:30 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-29-seanjc@google.com> Subject: [PATCH v13 28/85] KVM: Move kvm_{set,release}_page_{clean,dirty}() helpers up in kvm_main.c From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Hoist the kvm_{set,release}_page_{clean,dirty}() APIs further up in kvm_main.c so that they can be used by the kvm_follow_pfn family of APIs. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 82 ++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e29f78ed6f48..6cdbd0516d58 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,47 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } +static bool kvm_is_ad_tracked_page(struct page *page) +{ + /* + * Per page-flags.h, pages tagged PG_reserved "should in general not be + * touched (e.g. set dirty) except by its owner". + */ + return !PageReserved(page); +} + +static void kvm_set_page_dirty(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + SetPageDirty(page); +} + +static void kvm_set_page_accessed(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + mark_page_accessed(page); +} + +void kvm_release_page_clean(struct page *page) +{ + if (!page) + return; + + kvm_set_page_accessed(page); + put_page(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_clean); + +void kvm_release_page_dirty(struct page *page) +{ + if (!page) + return; + + kvm_set_page_dirty(page); + kvm_release_page_clean(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_dirty); + static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, struct follow_pfnmap_args *map, bool writable) { @@ -3105,37 +3146,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); -static bool kvm_is_ad_tracked_page(struct page *page) -{ - /* - * Per page-flags.h, pages tagged PG_reserved "should in general not be - * touched (e.g. set dirty) except by its owner". - */ - return !PageReserved(page); -} - -static void kvm_set_page_dirty(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - SetPageDirty(page); -} - -static void kvm_set_page_accessed(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - mark_page_accessed(page); -} - -void kvm_release_page_clean(struct page *page) -{ - if (!page) - return; - - kvm_set_page_accessed(page); - put_page(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_clean); - void kvm_release_pfn_clean(kvm_pfn_t pfn) { struct page *page; @@ -3151,16 +3161,6 @@ void kvm_release_pfn_clean(kvm_pfn_t pfn) } EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); -void kvm_release_page_dirty(struct page *page) -{ - if (!page) - return; - - kvm_set_page_dirty(page); - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_dirty); - void kvm_release_pfn_dirty(kvm_pfn_t pfn) { struct page *page; From patchwork Thu Oct 10 18:23:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830694 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6806C1FAC4C for ; Thu, 10 Oct 2024 18:25:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584757; cv=none; b=CXtfs9AoFlfWIsuM9pUiSstNCiyWpcaQakBQNscPs94UsKmGRqPf7VLsB+MJalnBQ3PRGSABNAWgni5YJDNZSuEZ72WiRuXadeNs0X81mPKSihNzeqn5BK8UJAz3//1ba0Rn7gveU3WKpPdc2oAUjZb21gvZ9WNZPP0KebnubMc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584757; c=relaxed/simple; bh=DZ/5QhAqV/AFQ+a9ItmswBEEvilydxqWBCs3ItnJewM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=SlKYsHjyTtCQDVoabZp6D5/JEjxK98er5ugNlB0eViccjM08DHD0FkaJdelWuj+RKKcXrNXDe3fbezQ0U1Y4BCdizAoMMv4cRULImMpw0TC6tISJjLY4VKDXxNqoPuGWAjUp+/C/u8zwzLdkxVksW544wD9d25wbzNHYHr4u+MQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hmIwJ/Iy; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hmIwJ/Iy" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e0353b731b8so1674283276.2 for ; Thu, 10 Oct 2024 11:25:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584753; x=1729189553; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=hmIwJ/IyhcXzHrpeXJmRadf+rJk2JLpf6qp/np19zFesdgrIj1iUPTmoJQNxJ6SI6X IHcMrrqOA4nE8IiNaHd2zwz6BvlPYPPf1ekepldTmgYUw+ak/bAI/v9hLzV0EYY6w0as Pd91JjhJ3PMfnyMtRXbJz4nddyfr9oLinVFCzp1mn4EqH4SlHOX9f3OICBNQavz6abuI pryFHHJbcxlbbdQzivVw5wyjGWIEhRw5l97zg8ZIES2+CB6VqneR/2wG6WO5fTubinUd HBmHOSTWDqWDcYOMtblw7eC8qj2WdaMLhZ9SkfWqev1nkC4dDfd9Onjs7JQevhMCO0rc 5ELQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584753; x=1729189553; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=cQLigScQXpX/MNUjlpUnSz5lATBjdCxbZzm6iKlMd77gxPsD1u91GywmSdPIEsaQqw wHq9Bo70RJLhsAcoBA2sz6Z2qHg/f/dYhzjfThiFB92TNKaotVVuQimR8P5aCdCUacEB u9FRDxrqAhEf9G+4xBBmZtawFmy9JckCII4IeVLVl3LOi31zHdlTdlx1gsUi9GSyA1Mb o6FNAa+3C62cwq+YGqvigZuGCBy0xyTLn8p9bxlZaU/HP+IQxM1pdiCBeCyYU1Z+lXs1 e3RjnorY84SnyQiBXRLIiAftNFt52EAk344Wz9YqXP4OKI+NwQQMvV+r6TMzN0km5/k6 yFKw== X-Forwarded-Encrypted: i=1; AJvYcCUDayiaZU8N6dHQbQ5li2laCMFTcK5BWfGGU8hcefduXIXP77mK8cMz0STu9xytA1bHxXSb/zSG38Qz@vger.kernel.org X-Gm-Message-State: AOJu0Yzg7h8rexabwMQeWMywksevJxzgE3k7Tao46wui2w2zCMnqO8IE tpy5TWhsLU7rNDBiF2Jlyb5UM8ve+hymIGCnPDmII5qnDBjhzdzImL5dOSxmnf5Gyq0Dvi1pVDf HlQ== X-Google-Smtp-Source: AGHT+IFqzRS1xOg2oJDncuDOdOtR7gITdwbfvBFuJ4xWEWgDQEplutdiZUKmbH0mIVak18Oaak6LM032atM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b205:0:b0:e25:17cb:352e with SMTP id 3f1490d57ef6-e28fe43f3f5mr4095276.9.1728584753336; Thu, 10 Oct 2024 11:25:53 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:31 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-30-seanjc@google.com> Subject: [PATCH v13 29/85] KVM: pfncache: Precisely track refcounted pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Track refcounted struct page memory using kvm_follow_pfn.refcounted_page instead of relying on kvm_release_pfn_clean() to correctly detect that the pfn is associated with a struct page. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/pfncache.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 067daf9ad6ef..728d2c1b488a 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,11 +159,14 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + struct page *page; + struct kvm_follow_pfn kfp = { .slot = gpc->memslot, .gfn = gpa_to_gfn(gpc->gpa), .flags = FOLL_WRITE, .hva = gpc->uhva, + .refcounted_page = &page, }; lockdep_assert_held(&gpc->refresh_lock); @@ -198,7 +201,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) if (new_khva != old_khva) gpc_unmap(new_pfn, new_khva); - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); cond_resched(); } @@ -218,7 +221,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) new_khva = gpc_map(new_pfn); if (!new_khva) { - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); goto out_error; } @@ -236,11 +239,11 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) gpc->khva = new_khva + offset_in_page(gpc->uhva); /* - * Put the reference to the _new_ pfn. The pfn is now tracked by the + * Put the reference to the _new_ page. The page is now tracked by the * cache and can be safely migrated, swapped, etc... as the cache will * invalidate any mappings in response to relevant mmu_notifier events. */ - kvm_release_pfn_clean(new_pfn); + kvm_release_page_clean(page); return 0; From patchwork Thu Oct 10 18:23:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830695 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 494DF1FB3D7 for ; Thu, 10 Oct 2024 18:25:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584758; cv=none; b=lS3sxYMt4BQvHN79hCWWg6+deAiXHP38JzLLbtfypd2XfDgCxuhAgvmBbJqJpYJITK+msj2UFBbe7HoCpagsF1q63Uyx/idVMGvcbMID7aWoqzbE/ftw2zXmPQcfPPtJXWO1tk6YmpF7q3pazN6Wc5nbKW2AbMqV+CO6YrfGOSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584758; c=relaxed/simple; bh=bnwD3uEfCD/YcddkLJu8d23/zYqKX46jf2vKJvYQlNE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fLpnJViU0yQced6zi03jcSDOCSwRPQ6DrjiSycslmYVVCVIEullpV//mlddLfcNGgJgbYzYBYu1Jr6kUAL3htxUJComNVdqEKHR9jcWGbmKyimm0xgm4ih/EFIJc2fdNMJBUzhtkWxPcBmPMVt7ooLHwe/tsHihY3gformHU0RY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u+lh0lZQ; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u+lh0lZQ" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e2270a147aso21765597b3.0 for ; Thu, 10 Oct 2024 11:25:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584755; x=1729189555; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=u+lh0lZQ0iXDXtDscH8PiFZR0maNS7DQsBMYikW3/rmnfcBXpI7TGMudK3dfPgUTEB P0isjFKFnrvh+aV0DTRRLJCNrk0qEXoNyKTpSlURgiPJyby8IfIe3wTsXycMslYF84Uu LxbGHqs7w+ucwTevI1iLIdHaxVjE7F3ssW3jQgSMXBln1jgkY2wR5qIbf7ZdkBZ9CO4J LKRV/cxsUGAhvttyWa4tlVFNCym+L/JIU+85sWhn7wEMpOen81nC7NpKHxGCzaFpRxd6 w/zKDX4OeUl0XtF7v4HJT2epxE2xdrp+lv8Fag4R953vA0Hy5rN8/x7w1xKjZA+XIYl8 Vqew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584755; x=1729189555; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=dkMnADEHf5NYZozhty/xlSfKu1k6NPG++BD7gRCZpY2tUDu7+4PVpywDo4CgQ7xiau 91cCo7338ts+HXXNKxtEZrIi6Vw6Yuq+VWHHVq92qlqQJ1GVtM7X4zNjBhDJh63NF7fP KcO8iEhY4QrAHin5/jzrv6bdiqk2yClTvppCltv27C5zmjhncJXLSERkbyhbfU9f4njr I3Q5oREQibWLQCF8b5SHy7LUV+kHBKcTt9sLAtuWQOehoSWLvSKkSA34CVCJTUU+boVg DOrS36fFk4mPeX3N+v8ZfDaHsvAxO7KOSSBdzBymEI8E9pYyT7c9EhWw4KC8WSWzzy45 m+ew== X-Forwarded-Encrypted: i=1; AJvYcCXPCMgf+vsMpw7JrK5zIcT6SJARvR8BrCEu13xLNT7DA9VAPvb2bH+l3Pnj6oz3pAv+IGJ6a7icbh48@vger.kernel.org X-Gm-Message-State: AOJu0Yxh1ylQa1whGRdX0QlS1tWnYJBh6qTBOGSoo/J7KJvJ0KFCs8HZ +8EyXN9AzjtC8gupZtMAJhFnToec5mwByHEIEZ6Wy+XIxjUO+HkyfQ9xynT6sAv1Qr5agTHD6cI Fqg== X-Google-Smtp-Source: AGHT+IHNZW39HKgOpo4XLx10sqcp5MldEDD35H2r/7z7SkgDatDPvbEuIFyvlgb9BtXWEl9gaR4EDWCFxGs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2093:b0:6d4:30f0:3a74 with SMTP id 00721157ae682-6e322171640mr1011347b3.6.1728584755409; Thu, 10 Oct 2024 11:25:55 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:32 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-31-seanjc@google.com> Subject: [PATCH v13 30/85] KVM: Migrate kvm_vcpu_map() to kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones From: David Stevens Migrate kvm_vcpu_map() to kvm_follow_pfn(), and have it track whether or not the map holds a refcounted struct page. Precisely tracking struct page references will eventually allow removing kvm_pfn_to_refcounted_page() and its various wrappers. Signed-off-by: David Stevens [sean: use a pointer instead of a boolean] Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 26 ++++++++++++++++---------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e3c01cbbc41a..02ab3a657aa6 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,6 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ + struct page *refcounted_page; struct page *page; void *hva; kvm_pfn_t pfn; @@ -1238,7 +1239,6 @@ void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_accessed(kvm_pfn_t pfn); -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty); int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6cdbd0516d58..b1c1b7e4f33a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3093,21 +3093,21 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) -{ - if (dirty) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); -} - int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(vcpu->kvm, gfn), + .gfn = gfn, + .flags = FOLL_WRITE, + .refcounted_page = &map->refcounted_page, + }; + + map->refcounted_page = NULL; map->page = NULL; map->hva = NULL; map->gfn = gfn; - map->pfn = gfn_to_pfn(vcpu->kvm, gfn); + map->pfn = kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) return -EINVAL; @@ -3139,10 +3139,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); - kvm_release_pfn(map->pfn, dirty); + if (map->refcounted_page) { + if (dirty) + kvm_release_page_dirty(map->refcounted_page); + else + kvm_release_page_clean(map->refcounted_page); + } map->hva = NULL; map->page = NULL; + map->refcounted_page = NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); From patchwork Thu Oct 10 18:23:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830696 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 184481FBC90 for ; Thu, 10 Oct 2024 18:25:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584761; cv=none; b=IeHH9E9PLRiZJZ4pPkRNL3MKs/bVL1nebKbYWMte4B4nu+R3fPmJ30DfsJjEAA/GH8njiKkvypE9lXXSGhKSRTOaiiSaSKhU6PQbj2DstZw17f5v7Sr36XHH7k+i2IXsMXMPV4bALkMsfXcrOqnXmDqLGhDBYzn9dF+tNHlMKTI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584761; c=relaxed/simple; bh=M2NCneg/Pye+UyLI2GGt5shIez0MQF8LN4Gs0ecBwhw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FSdzA7pzDQ7IQgVOitx7rKxkhEYU0DSGBzOxeJ1VadKTJGR9Ib+P1jezac29xKiGfbgFvqMQ+3gRHfQslTUmr77yPM8iHP5v6Llk0GsWmkyi01T0I9I+GZ+5fNtOtCnvJ58G7GE2HUgHrfUaBOa+i3hHhU6Z1Btgm6Jmum6rb1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CYNUenPL; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CYNUenPL" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e2605ce4276so2180319276.3 for ; Thu, 10 Oct 2024 11:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584757; x=1729189557; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=CYNUenPLiXIvtTyD74D8imBRAdC50F3hM1AGdNct9/duav3eSxT08JrxuH4uXDnr2p GlywDJgMcirPnzaX/MzQmoAche5Jq/wQ+s2jZ8bLV0HN01jab8FA4Xyx+wz08gu7Ei/1 aMZVlCmv09LzBHl5ILjwWuvSGgp907WUV3teWOwY2a5c3RZBVUymYlPJ+uSMbRo4gDHd BCHayDy3tx5zkYmvi0rcCgT1BlZperbNOxiXf9X2CrpcePe1UPYsgemnEtJcHtRrO8rT DTQAY2TnyFkSUZshaKMhikHuc//2hbWd69gYZuL5srQsImqM4AupFJHvu+dzaLsC46rc 9Ypw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584757; x=1729189557; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=U7pPL0VbE722yXMp6A0Ywn/itdWFWCJX6u6NxF/9OkQbTnfuBUlI5tV2AU8U2MptzP jFzS0Jix7Xc/q+/YnOiHZc6WvA0mCDLaEPItV+nxtwwQSd4x2DSAFeBTkpz8Oh5yoY/P AGz/A9vZJTcE58qpwwosJCA4dgFCh016WKIjbi1FI1VDAgz9pZ8DUx7LWSbXo8eJhYtn BZxBOBd89hqOnRzlrDNJIJ++GuQQb5xFxhPIKXTGFHx7Ej6kQ7RY+GCs/83KMdfX1QUB wyWz6jqYzY4RwjckaHp7UJkQslRLVTGN5KhK2U7r9jhblG8k4V2JjYuxCP3DPIklA1tr i9Vw== X-Forwarded-Encrypted: i=1; AJvYcCXz1xQYwDO9fWyFqaFT0GJKCBh12OY3PppUpJgK21Mg1amQm/q8Si+ydG/MUp72i8jKp76x3ZpvPgjb@vger.kernel.org X-Gm-Message-State: AOJu0YyBPcccEybZZ9CrAN9QbF8JsSsE1zkQqyCKZCIlMu/o+0NqXyjs /KWfAkLYcLK/lJD8OpV6x5L1RyLF1c+LeApVqNXK7TU1fcWuWDfW+AYTPGCGKijvaj3na7e0VKf DUg== X-Google-Smtp-Source: AGHT+IHY0yNiFmgeAXsfVKl7EbEBY2R3q1IXvjRBHtSPoxXhe2HbHq4twyB0Huv0R+/OHvP6V1ZW3nZzpzQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:80cb:0:b0:e20:2502:be14 with SMTP id 3f1490d57ef6-e28fe410672mr4860276.7.1728584757233; Thu, 10 Oct 2024 11:25:57 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:33 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-32-seanjc@google.com> Subject: [PATCH v13 31/85] KVM: Pin (as in FOLL_PIN) pages during kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Pin, as in FOLL_PIN, pages when mapping them for direct access by KVM. As per Documentation/core-api/pin_user_pages.rst, writing to a page that was gotten via FOLL_GET is explicitly disallowed. Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() Unfortunately, FOLL_PIN is a "private" flag, and so kvm_follow_pfn must use a one-off bool instead of being able to piggyback the "flags" field. Link: https://lwn.net/Articles/930667 Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 54 +++++++++++++++++++++++++++++----------- virt/kvm/kvm_mm.h | 7 ++++++ 3 files changed, 47 insertions(+), 16 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 02ab3a657aa6..8739b905d85b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,7 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ - struct page *refcounted_page; + struct page *pinned_page; struct page *page; void *hva; kvm_pfn_t pfn; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1c1b7e4f33a..40a59526d466 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2814,9 +2814,12 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, */ if (map) { pfn = map->pfn; - page = kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; + + if (!kfp->pin) { + page = kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } } else { pfn = page_to_pfn(page); } @@ -2834,16 +2837,24 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page; + bool r; /* - * Fast pin a writable pfn only if it is a write fault request - * or the caller allows to map a writable pfn for a read fault - * request. + * Try the fast-only path when the caller wants to pin/get the page for + * writing. If the caller only wants to read the page, KVM must go + * down the full, slow path in order to avoid racing an operation that + * breaks Copy-on-Write (CoW), e.g. so that KVM doesn't end up pointing + * at the old, read-only page while mm/ points at a new, writable page. */ if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + if (kfp->pin) + r = pin_user_pages_fast(kfp->hva, 1, FOLL_WRITE, &page) == 1; + else + r = get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page); + + if (r) { *pfn = kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2872,10 +2883,21 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) struct page *page, *wpage; int npages; - npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); + if (kfp->pin) + npages = pin_user_pages_unlocked(kfp->hva, 1, &page, flags); + else + npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages != 1) return npages; + /* + * Pinning is mutually exclusive with opportunistically mapping a read + * fault as writable, as KVM should never pin pages when mapping memory + * into the guest (pinning is only for direct accesses from KVM). + */ + if (WARN_ON_ONCE(kfp->map_writable && kfp->pin)) + goto out; + /* map read fault as writable if possible */ if (!(flags & FOLL_WRITE) && kfp->map_writable && get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { @@ -2884,6 +2906,7 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) flags |= FOLL_WRITE; } +out: *pfn = kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -3099,10 +3122,11 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) .slot = gfn_to_memslot(vcpu->kvm, gfn), .gfn = gfn, .flags = FOLL_WRITE, - .refcounted_page = &map->refcounted_page, + .refcounted_page = &map->pinned_page, + .pin = true, }; - map->refcounted_page = NULL; + map->pinned_page = NULL; map->page = NULL; map->hva = NULL; map->gfn = gfn; @@ -3139,16 +3163,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); - if (map->refcounted_page) { + if (map->pinned_page) { if (dirty) - kvm_release_page_dirty(map->refcounted_page); - else - kvm_release_page_clean(map->refcounted_page); + kvm_set_page_dirty(map->pinned_page); + kvm_set_page_accessed(map->pinned_page); + unpin_user_page(map->pinned_page); } map->hva = NULL; map->page = NULL; - map->refcounted_page = NULL; + map->pinned_page = NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d3ac1ba8ba66..acef3f5c582a 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -30,6 +30,13 @@ struct kvm_follow_pfn { /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ unsigned int flags; + /* + * Pin the page (effectively FOLL_PIN, which is an mm/ internal flag). + * The page *must* be pinned if KVM will write to the page via a kernel + * mapping, e.g. via kmap(), mremap(), etc. + */ + bool pin; + /* * If non-NULL, try to get a writable mapping even for a read fault. * Set to true if a writable mapping was obtained. From patchwork Thu Oct 10 18:23:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830697 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E23581FCC47 for ; Thu, 10 Oct 2024 18:25:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584762; cv=none; b=YeyXnlpUzOJC1lBdbk+wHpu0EuFbtk6Ym4TFoZ/POn+A8pSRfLSEbBMkE4qWXXZzrL7rpyCoDa4oIDbm1uHGfOAPqOvxpYm16MYA7VK71TeDJJHC5SJoFKJTjkAr+WF7l5PmtxsM5YpgP/K6Zagfy+0zeUtf1QXyLf4BRo0DW84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584762; c=relaxed/simple; bh=6AchJ9iF2qZs5ZSAOBrdf0urgGqdHLpa0Cd8lTPRDhM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Qtw93H/CluFdJZvwNOd92wJA1vW89SPv8IANMtWjPa0dWkm6aySO76E7neewkXBxpgzrAxwFMAwr1b/2EtKyiBe8ZKTVgdL3kTh7m68859qe+pkvH/mdzK/hwi3NYKw4MW7JqiD/Vw2WXH1r8bpgMUgzTOGppjPxGtLrh+MW4Z4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KeBY5py2; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KeBY5py2" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7e6e98892e1so1287966a12.0 for ; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584759; x=1729189559; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=KeBY5py261h3po2p7EcoPVpo67GiOj7+6dOP6EoAEriscYrzhTzO39NHb0NJsAW1jR n42KW4EwGgaUs3yx/oCZEuQvV/M1idASTdIADhBN6ON4GT5SGgjV1ooFQU62n4X7Qr84 /fRWijapZZzZVKePZxyFZegIWlOsafg0wDZM5BHf4vXa9CU5WL9dY+OB5HyQ609h9s7e VPCG5daQbQMIE3NYESeghcC9iMjOQZxPTOapKpidLBB73TVQIpNb1Ze+j7LMk9Bx3x2i vO15GDeuHOJlaco62A9bG0UiqB8QoTlfPHg1Bpr7sbnSv5dMGej5HVQzlTOoDtu38Q9N 6JtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584759; x=1729189559; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=YNOeuvQLQssEXYhUSHSak0rapgCv+OgqUJaNWKGT3foZUUFNu/97A6raPF0tW0zhtG QAGg/arrXXIxHMqffn8RxlSToa6e5Q7ciC0CSrXknYr9AACptpT572uGdqYBgr/ZX1Q1 aCNekkVL+zBXOk7AHKXufbiBo1YXq5ZecejEGOTQNmncYy/vqt3RAms67tRgbYzgSm2Y UBZk4ygK9UQG/7iJjAM6P0ghGDo46dq/Zx9RxFg+C5UgwXAsgQ92MO+dnkrHG6DMvV2S 7jEyoEgh21/o5UGHx9TNtBjRGFiFWokpaSlywhExDzc9wmE+9SZys7knxZRFtBE6J7tA ruHg== X-Forwarded-Encrypted: i=1; AJvYcCWlH8Q92OlFArpB1JUor3g3Tf3ZYblFLpxxJlR9eXcr0Tnsd4wVbcubzdAchKHLDSEn5OfexHUqvrkW@vger.kernel.org X-Gm-Message-State: AOJu0YxnJCxATuoB1u2lzB13zBCUfdOCwdM12CFb0wuC/WRJj33iYs59 Q+edXvXLM9Bv9KWPdgPHKgDavnHPvsJ6ensMCxlUK145SiB3OfPMKApSKPTL3ZOSsVpXKcV/acM v5g== X-Google-Smtp-Source: AGHT+IEekecUM0XVWoBmNjTczqp1a2/ixTMw1fOAmBrSm/Dp9XgHLEkyd54mJG+SlDPSB0szYk/zKOq5dxU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:678f:0:b0:7e9:f98c:e9f7 with SMTP id 41be03b00d2f7-7ea5359ed14mr32a12.10.1728584759044; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:34 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-33-seanjc@google.com> Subject: [PATCH v13 32/85] KVM: nVMX: Mark vmcs12's APIC access page dirty when unmapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark the APIC access page as dirty when unmapping it from KVM. The fact that the page _shouldn't_ be written doesn't guarantee the page _won't_ be written. And while the contents are likely irrelevant, the values _are_ visible to the guest, i.e. dropping writes would be visible to the guest (though obviously highly unlikely to be problematic in practice). Marking the map dirty will allow specifying the write vs. read-only when *mapping* the memory, which in turn will allow creating read-only maps. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 81865db18e12..ff83b56fe2fa 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -318,12 +318,7 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); vmx->nested.pi_desc = NULL; From patchwork Thu Oct 10 18:23:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830698 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9AE61FEFA6 for ; Thu, 10 Oct 2024 18:26:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584763; cv=none; b=mcvKGhOXSvCxgtjXrGsCoo1GhdySokneU0bKCDHWdgS42AUq/VwkD3NXgvJThUQYnDCdlePxvFl24BF2s5HQ8tIWVIfRyx3QSIX8Z7aCQ1j2Sji1aUKSMvVYTES3q3nWJOhdT5NSK6RjumtWVhN5QUqWXXUJzWdUx8oRL10Jhaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584763; c=relaxed/simple; bh=mQXhN5iZpOZWnnd2N5GwBNgcRKXl5xUfnEr6O0WaUwk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F4iCQ9tuPEJ02A/tHI0kttWW2SC1LisCg8njpssF6gfLXlEFZjvzg+16L6SCd7q5LEMJEXMK3oGWRbJdeqjeiUBs9LGG7YHSoHG1GrTjaYfJHJn8QTnjowDqY8+iYhxpTNV5FR/v4VzlwKdS/4Ryvxs6mO3F/P6f1PguCmuyHbI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fhtsmxz8; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fhtsmxz8" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-690404fd230so25174427b3.3 for ; Thu, 10 Oct 2024 11:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584761; x=1729189561; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=fhtsmxz8vpfa2op10hhZgWbQg3a/Ux0j9mEjjKDNpT+6H68jzG7G30Lrt0DgwS2OqA VAQch49mqfwm5cA16yd/W6I1emiwxouDMXVSvKt1yo8/wuYywAj8WmU3ri9BLGwOXV2O ev+52ekxcamGE3PinuEE6axN2+AjKb8EBbMunUi5MpzIPNmR7Cd+feQewcjH6urh4T2Y L7YmQKAT/bzQUgg9L16aEUHrMjmkiyjytv61vSD0Smo/6KXR/dS0NqmYK+EDeZ81iIUB BpN0M+BzPxJAzpf2q5xvJT+CdoJM7Ovvc/CKmFvaOg7Bed7+EqAlOAxWv/v1tDiHH+OX ghxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584761; x=1729189561; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=vI07f+Npzz2IbJXzwLBbveiEwbdxOZnLjt0UEOhoEHT4p/j5F8GtefMx7UEGFqzMBO qX3Mn/F/GF9Gpmoc57DeWzo3n/4Jvg3ERSe2tsUNo3VIQvBEPMd0TjCwaAMuP0JH34kv bcsa1sJNBnlRETGfj7aMjDhfRACBRQmEicWZCzFfdtz4JnQdXrVf+i7kq6N4XJMr/2A7 eaeKCSbNcddzvx+LJnQg+7vdPEBQRnBvgKMtchjlYOR7ejFku2FrsDE6MIkdeALQGKcz oUjklLEgpMFQ3znVh+jVWoXh0TDdZ6aDlfRK71kmDqI04JQEN26S5W6WJnIiUpnQiKkU fFWA== X-Forwarded-Encrypted: i=1; AJvYcCU1D40ZXhRY7XV8S3Ww8XZji9kxEeQEZtw120sCkt6E8PfPAtQKF7Pk5jQszWcc6TqRwdzgH1jK/OGt@vger.kernel.org X-Gm-Message-State: AOJu0Yx7xzbn+x4zl6fZCHQiJAKONc2IsLREeGN/HaM2pAzXt7gTm+W5 bwn9aTp+UsmgzCj4nwP1y6Tpu5ypxNB64kCygwyUNQIjPQBouhRjulecVqfxExZWA2Q2sjvxUwi 1Nw== X-Google-Smtp-Source: AGHT+IHYBe/gylmYYB76Y4JbCGQqBxyU7NvKFk8CzbtQRqqOSH0kk5njqmBLFW7wxt+Okx9MKc9CHg/F7Og= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4349:b0:6e2:1713:bdb5 with SMTP id 00721157ae682-6e32217cfd3mr118607b3.5.1728584760829; Thu, 10 Oct 2024 11:26:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:35 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-34-seanjc@google.com> Subject: [PATCH v13 33/85] KVM: Pass in write/dirty to kvm_vcpu_map(), not kvm_vcpu_unmap() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Now that all kvm_vcpu_{,un}map() users pass "true" for @dirty, have them pass "true" as a @writable param to kvm_vcpu_map(), and thus create a read-only mapping when possible. Note, creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN. But practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 8 ++++---- arch/x86/kvm/vmx/nested.c | 16 ++++++++-------- include/linux/kvm_host.h | 20 ++++++++++++++++++-- virt/kvm/kvm_main.c | 12 +++++++----- 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index d5314cb7dff4..9f9478bdecfc 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -922,7 +922,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) nested_svm_vmexit(svm); out: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } @@ -1126,7 +1126,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.exit_int_info_err, KVM_ISA_SVM); - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); nested_svm_transition_tlb_flush(vcpu); diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0b851ef937f2..4557ff3804ae 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3468,7 +3468,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm) sev_es_sync_to_ghcb(svm); - kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true); + kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map); svm->sev_es.ghcb = NULL; } diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9df3e1e5ae81..c1e29307826b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2299,7 +2299,7 @@ static int vmload_vmsave_interception(struct kvm_vcpu *vcpu, bool vmload) svm_copy_vmloadsave_state(vmcb12, svm->vmcb); } - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } @@ -4714,7 +4714,7 @@ static int svm_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) svm_copy_vmrun_state(map_save.hva + 0x400, &svm->vmcb01.ptr->save); - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); return 0; } @@ -4774,9 +4774,9 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) svm->nested.nested_run_pending = 1; unmap_save: - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); unmap_map: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index ff83b56fe2fa..259fe445e695 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,7 +231,7 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map); vmx->nested.hv_evmcs = NULL; vmx->nested.hv_evmcs_vmptr = EVMPTR_INVALID; @@ -318,9 +318,9 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map); vmx->nested.pi_desc = NULL; } @@ -624,7 +624,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map msr_bitmap_map; + struct kvm_host_map map; /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -647,10 +647,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, return true; } - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) + if (kvm_vcpu_map_readonly(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &map)) return false; - msr_bitmap_l1 = (unsigned long *)msr_bitmap_map.hva; + msr_bitmap_l1 = (unsigned long *)map.hva; /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -714,7 +714,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); - kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &map); vmx->nested.force_msr_bitmap_recalc = false; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 8739b905d85b..9263375d0362 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -285,6 +285,7 @@ struct kvm_host_map { void *hva; kvm_pfn_t pfn; kvm_pfn_t gfn; + bool writable; }; /* @@ -1312,8 +1313,23 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty); + +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, + bool writable); +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); + +static inline int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, true); +} + +static inline int kvm_vcpu_map_readonly(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, false); +} + unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *writable); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, int offset, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 40a59526d466..080740f65061 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3116,7 +3116,8 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, + bool writable) { struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(vcpu->kvm, gfn), @@ -3130,6 +3131,7 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) map->page = NULL; map->hva = NULL; map->gfn = gfn; + map->writable = writable; map->pfn = kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) @@ -3146,9 +3148,9 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) return map->hva ? 0 : -EFAULT; } -EXPORT_SYMBOL_GPL(kvm_vcpu_map); +EXPORT_SYMBOL_GPL(__kvm_vcpu_map); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) { if (!map->hva) return; @@ -3160,11 +3162,11 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) memunmap(map->hva); #endif - if (dirty) + if (map->writable) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); if (map->pinned_page) { - if (dirty) + if (map->writable) kvm_set_page_dirty(map->pinned_page); kvm_set_page_accessed(map->pinned_page); unpin_user_page(map->pinned_page); From patchwork Thu Oct 10 18:23:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830699 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BABBF1CBEAB for ; Thu, 10 Oct 2024 18:26:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584765; cv=none; b=A5UucXGw9aaCzWg+6DRz6z/S2ptDa75bxxWfTrLvecGXnl/7Q7Lhn6DLUIqvFWcK7nuY+0L27eudLYEU8M4D+7oMLv8HiS7A+g50/QnWk1V6wPaYYIcESIizBLloHkch0THA9LYyW56o6V4Zn5QguX7CPDRDdeJ6zXsl2HN6fM0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584765; c=relaxed/simple; bh=+qwur9UMvfPsT7xhW5+1hFXGI3F0T2CjnbFVAQFEZ9U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kldnrMEKnFjoUMApShCLwQU7TjEipFWpODG7YUdSr98LAzTp4P+JLpsF7rVLfGTYLc3MRCLfLBqOi1XZZxoSkXXL2UiKPd0hO4dtsGsLztX7b8dcPMY0Dype9O084c+BkOxNNpcj48+hrvYgLxiA7nQKc3NlEValOVxAOSV+YNc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qwMYhkdO; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qwMYhkdO" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71df051770aso1193822b3a.2 for ; Thu, 10 Oct 2024 11:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584763; x=1729189563; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=qwMYhkdO+k0O3GAu86CC0Oovaqx2YjaPKBv6LeLzcOh3PtNiOaQrcv5HBilI8icD9g y0Ltij7dFk7sin4usHA+6Wv1hFiwFPtq5Ajocz2SMLSw7EHhQ24kqq+Bt+H2kyBdKaty qPo1gORXTlpg4cwuwI2/Xd/UCp8kWbToIyady8yUqr5BbwLt+Kkm56JwyHrVdgysxAdq eCA0ipk6bTZMsprIdAiZDgjFHxpqT0t6hWRsy+Feo3csNnBxmMja133Zk9XVBeicQl/q 2iXz1SFCo5UtodgUyE2QoPpXLNLLkpEupG73DPzHBSBkCdaePTQJNXxdK3uBHhhZG/wU Akeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584763; x=1729189563; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=oo3oDRlNfO40uAviWZ2Z2Dx9H0gqtTFGnvNm0RTqi59Mb6DUoYRZzSkjC0TYsCEqnD WpOv6Vs4L8nNEIntU6BDPNZYR2ZD/juljBSflVZ+ehBTFrv5Y2+hBmJ9FPA2WvpSptB6 nZarbgJ5e0JNBbeC3SU8ccEFBZPGBe77mcqSQMMIdzVCno0FuBrPho5yFXhS6QDtXLGR 3qgwDylTUDtxgdwF5Bzsrri29kjvItnDe9HNpCfWYME/rkBSnVCCBXv3Lq8DvMxCo97w l85dtPsq55wiHAyqhZIMuShYM1aQUScGHij2ccSVYa5ZNCLp3iXNIR/6h7CFbsQ8GNqF dw7Q== X-Forwarded-Encrypted: i=1; AJvYcCVmUufDpuRdODP3iVUvqaxbph66CO0VK9OKuY5/77ICjq3cdh89L4ZWf7rYSexAA9yCNJvLxQ2paq5g@vger.kernel.org X-Gm-Message-State: AOJu0YyngzV0hkZATJsz+at73AKaPbjiVuDyS0uRP13xjkhk666dV1EZ VelmI3s2+0FcsDEdfa137oPNi9Qplnq0ar/NlHfoXsAxSzdO0UfzeegWXMqMrOqHwnZgJ9i2WQh TDA== X-Google-Smtp-Source: AGHT+IHthU1bxxWE76UmRU/cmSB/2b+tJYVulTyd1SGmZGLYBHQFEbukUNmDJAlwNs2CFth+SfvGFADhDCI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2f51:b0:71e:dd:8f9b with SMTP id d2e1a72fcca58-71e1dbe8750mr6976b3a.5.1728584762866; Thu, 10 Oct 2024 11:26:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:36 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-35-seanjc@google.com> Subject: [PATCH v13 34/85] KVM: Get writable mapping for __kvm_vcpu_map() only when necessary From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones When creating a memory map for read, don't request a writable pfn from the primary MMU. While creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN, practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 080740f65061..b845e9252633 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3122,7 +3122,7 @@ int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(vcpu->kvm, gfn), .gfn = gfn, - .flags = FOLL_WRITE, + .flags = writable ? FOLL_WRITE : 0, .refcounted_page = &map->pinned_page, .pin = true, }; From patchwork Thu Oct 10 18:23:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830700 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90F211FF7A7 for ; Thu, 10 Oct 2024 18:26:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584767; cv=none; b=B7yoINxrgrPGgoSmLzm3bFuTsC1zqQDHZHvAQgaYIS+HSPmf7Wv+polk9uEY3QxsHiG6aaeZZ0QthPlsvN4rUWTTQTKVIClnYPmuuLxv0GY2LGgA2+/SBcbWwvw2iZY0PVsOpp2Hh3SrqlFG5kVF0Dvg+Fti4cdCP14eCYIFXsg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584767; c=relaxed/simple; bh=JluxJpntN2XzeRJ5Yz03kdoesvuYdAA2z8LW9UOaBnA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Pr2BZivmeEMdSSGsDKK2NhYa13gGcZEaAAbJgP6XymxVVtlaestVDpd4VLkLnM+AUnKo/HT0T9DQQcngViHoV5AZq2biBnBF8yQn7wBmCbMbnuDhmXbhDgMC5g9roZCeNKoInwH0+dHMNaDtuS9xdn5oH21Etl6wAgRw/F9XWsU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VfKoGAUz; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VfKoGAUz" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7e9fb5352dfso1520880a12.3 for ; Thu, 10 Oct 2024 11:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584765; x=1729189565; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=VfKoGAUzy1tyfifoK0lzuC5EO6vmRc5H8EipIe+ZLEuWAXzAG8pU2LiyJzlZVu0hi0 mOO9VygcpFe8zOSwUmfQQHakO15699SyqMVOWWSAiUH2fJJdgC8rguMny3tmzuNm6qma xqCi/swqvFmWY9n9yk/3JuaiFkFnRNBPBb225gAWzmoJNnJjRf6EzDz+lob/Swy2Zd8F 5WPg5QuuVmEjsJhFvUHJUB4SSv4y2afPWrVKR+wB8gSLJdQ+AEm1DKIqe37EnkjyN8cB 4U807FVw2rgqm/ygIdqMTxDCTj1f1B9aQDFvY0AHA0uFNQgR8DeiTSbrOcm3suHvH+5h 7yIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584765; x=1729189565; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=DK04qgf84CGZV3CkqwVqvRmTR3QgwrcP52xfsNITeCV3zSt0OfurebJ+uKTpqL3Ex8 TnTYNXmC+fiHoOvC9bB3hUlxAAgoUappW0/1+xQ/M2pzefVxXzk2Neo5ImLjdZT/x2TY UVdxBfHJBO03DTShkiRKk/RZPFHndJU1v6ca7CdKOucLz8rBsp+Vf+w9+yQLUu9LWiKw xyydBjaQWQz+MdFghFU3w1Zx6jqH6OYkNAQ0sNjZFVUgq7inYigrK9pzMZqS02ASd9+v 3Up/x9vy0MoNJ6RKWTK2U9TChV1DamptNUrwDgo/cttLp5KWVr7NHG/6VneC39UEGid+ nclQ== X-Forwarded-Encrypted: i=1; AJvYcCVu09nHLQPdFvFPOKERYJafk6szW6qi/Xo6bPpPKnEPRFRRr5J0bh74NMNynDohkhe1tWzsdOXPe+in@vger.kernel.org X-Gm-Message-State: AOJu0YyF0rRBfvn0VW57rS5Z2KWURAHrKQbxGI6sJ0g4Q29sZtw09AlN lqDojR9S8RvAQM4DZdhOcht7V40ftJ4v09wOAK8XPcphcJ1cWoB0oC7tPfPwq9E9i42DVSUaOaY E6w== X-Google-Smtp-Source: AGHT+IE23QkhjcWr/3QASJ/S9+dapZUNy7Z4CCepJKqapzuUx8eBu6OhWGfM5laLrlsw47tRhKuFSYugMsQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:ff61:0:b0:7cb:c8c3:3811 with SMTP id 41be03b00d2f7-7ea5356a658mr52a12.5.1728584764791; Thu, 10 Oct 2024 11:26:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:37 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-36-seanjc@google.com> Subject: [PATCH v13 35/85] KVM: Disallow direct access (w/o mmu_notifier) to unpinned pfn by default From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Add an off-by-default module param to control whether or not KVM is allowed to map memory that isn't pinned, i.e. that KVM can't guarantee won't be freed while it is mapped into KVM and/or the guest. Don't remove the functionality entirely, as there are use cases where mapping unpinned memory is safe (as defined by the platform owner), e.g. when memory is hidden from the kernel and managed by userspace, in which case userspace is already fully trusted to not muck with guest memory mappings. But for more typical setups, mapping unpinned memory is wildly unsafe, and unnecessary. The APIs are used exclusively by x86's nested virtualization support, and there is no known (or sane) use case for mapping PFN-mapped memory a KVM guest _and_ letting the guest use it for virtualization structures. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b845e9252633..6dcb4f0eed3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -94,6 +94,13 @@ unsigned int halt_poll_ns_shrink = 2; module_param(halt_poll_ns_shrink, uint, 0644); EXPORT_SYMBOL_GPL(halt_poll_ns_shrink); +/* + * Allow direct access (from KVM or the CPU) without MMU notifier protection + * to unpinned pages. + */ +static bool allow_unsafe_mappings; +module_param(allow_unsafe_mappings, bool, 0444); + /* * Ordering of locks: * @@ -2811,6 +2818,9 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, * reference to such pages would cause KVM to prematurely free a page * it doesn't own (KVM gets and puts the one and only reference). * Don't allow those pages until the FIXME is resolved. + * + * Don't grab a reference for pins, callers that pin pages are required + * to check refcounted_page, i.e. must not blindly release the pfn. */ if (map) { pfn = map->pfn; @@ -2929,6 +2939,14 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, bool write_fault = kfp->flags & FOLL_WRITE; int r; + /* + * Remapped memory cannot be pinned in any meaningful sense. Bail if + * the caller wants to pin the page, i.e. access the page outside of + * MMU notifier protection, and unsafe umappings are disallowed. + */ + if (kfp->pin && !allow_unsafe_mappings) + return -EINVAL; + r = follow_pfnmap_start(&args); if (r) { /* From patchwork Thu Oct 10 18:23:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830701 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA19200112 for ; Thu, 10 Oct 2024 18:26:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584769; cv=none; b=qis6Zbak6h9+mWoM1DbPrWZE08Ct2cOnk1bjM+4Ku2h8IdF6UDqkXzbe4/VbL68Q+qvGZF88TZgboat4zVdKKlr7Fyp3JkRCUjOnrbTgPQA5AR3XQDpLaVwG3jDONvwrxu1THm3WokB7KXzKkv6KO3OOghroCAQjT9zC5m4R4ao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584769; c=relaxed/simple; bh=HP41e6SfsC4ul+oSDEyzHZ0hlt7iv8seYpnKNCv0jUQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aBINEimqQXBwx/RPQFM+xOb1Zi0PeECYJgUyf42SbHBU+YVp3KSwPpz1U6bA5Fmr5Xt7XVCPmfJu80gxGI08jrCKE1fOpbbR0a29aeyCsFCpGhH0pcnqpnbwg0LJHNotEw10oG0Z/Mjy3bs0WzUTpvbXgwchSUVmHI5s4Xfnn6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3iE+fjr+; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3iE+fjr+" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e04c42fecso1290002b3a.0 for ; Thu, 10 Oct 2024 11:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584767; x=1729189567; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=3iE+fjr+J+BhPA8v3iqwSMU7gQHtmXoLc2UoAeE8K7W7XowZPMedacJz0kDLjxUDYQ QBHxZX1NpRwXYSGnH0HL3vJtR2VFd/IfEGEWVjktv7EDSMjvcPUq75M3pmwLSNSY+vlM TTGcpYpnDBEcT6Yq4ZgXXOtSRMnft79vdJT2U4MoIAfvpL1GKwjhtjsMIhwYKH1sdUdg mA89AXe5f0SdxOYRU+ZmAG4KCc7jke6+CVqu1SVWGJg4hyIVrQYwExBFF/T3JmjQdnSL ThQ6E+6hJCDsmRcl4PwyBxiNMZtTBKOsC/MsBvMbt48BIiFEhaYo/ha2z7lutfruNTUk snKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584767; x=1729189567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=EjO3ghip14vw6Db90o9FlClGO4m4imTzmv1fG+XFYO7U+Q/wDsWaI+G+myzMJK9CXp AeU5oaXubA+i9wgM2EJH+XsTdP+absPE2j4g9PeQvfWvppWNRbc6oBnXy8s69E8IM6lw E2vffgmkaj3kGUGPz9CYmA8fLx8XALf1WfliWdm4CUQy0muRcVQBpmnhOfJJIPvXzHqM uG7Ki/A1TEypWeHJpKZYcT5qTFbui90SdATF9DiL6aozT6IxeUIdhOOsYR4fMk4lDJS9 DygHuByGVUBYgPH3GY1+nwdAc05Hjl1X4JH8cCXILBH3xTpLmlRjOADOJLpEpRhmKr0p Iq9A== X-Forwarded-Encrypted: i=1; AJvYcCWKwKQ7EeMZCuBAoSP8Ur4U2Oh0vj4mhKdr0oGdljr6Z018B2cJI12iSNXoYtdNiUJCJxoEmojgAyr/@vger.kernel.org X-Gm-Message-State: AOJu0YxoSRvFoAEIR+z0ArIlM7IJxCabKuim4RdInJoH7vWN28RKUPq6 Debkv9358jW35cA7CPx05chSb/elKQhhfNzpmpUNlhCFa95a0Va/G+NmPOMzk3XyphBR7hDBS3V d8A== X-Google-Smtp-Source: AGHT+IH5y4h19aQ8D3UFZNHEjFs0l+IkJ/F+GwDOdY5CLNZ6WKY+eP+zfpO4DiXt2PfVXCIgSRJOtCb+AeE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:6f44:b0:71e:268b:845e with SMTP id d2e1a72fcca58-71e26e53c16mr11733b3a.1.1728584766802; Thu, 10 Oct 2024 11:26:06 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:38 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-37-seanjc@google.com> Subject: [PATCH v13 36/85] KVM: x86: Don't fault-in APIC access page during initial allocation From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop the gfn_to_page() lookup when installing KVM's internal memslot for the APIC access page, as KVM doesn't need to immediately fault-in the page now that the page isn't pinned. In the extremely unlikely event the kernel can't allocate a 4KiB page, KVM can just as easily return -EFAULT on the future page fault. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 20526e4d6c62..65412640cfc7 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2647,7 +2647,6 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu) int kvm_alloc_apic_access_page(struct kvm *kvm) { - struct page *page; void __user *hva; int ret = 0; @@ -2663,17 +2662,6 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) goto out; } - page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (!page) { - ret = -EFAULT; - goto out; - } - - /* - * Do not pin the page in memory, so that memory hot-unplug - * is able to migrate it. - */ - put_page(page); kvm->arch.apic_access_memslot_enabled = true; out: mutex_unlock(&kvm->slots_lock); From patchwork Thu Oct 10 18:23:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830702 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FDEC2010E5 for ; Thu, 10 Oct 2024 18:26:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584771; cv=none; b=ii9LbGdKPbfJ2uHAAJ22iccIXxp9G+VxFS/H9v8yRHX0uu5BM9kQoBaeJT0/z1rrwGLao9d6qmEiYDvLoyRb+fmeKko1c/Sx0tIwPqMHI1oiZMshBUO9w9TiZpuRnGTkjihaGfqdUjx+VUW3+P/Mpz8vIDVYgLYuJuzp/N37o7k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584771; c=relaxed/simple; bh=FOrzmk+4dQuoAh6BJ51EHAqwvBR4fq2vz5+jTHFfYUI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WQhzcmCd0MH6CyXGKdAU//lNi0d9oXRmJfICnlS5fgoDcu4JxGeZupgfi7yf2V911Fg7/ZCKBifq915oROSbjDZKtBYI5U4RaP5mftz6O/CvHlG/BgdEyPh/NJXeIUWtf7/7UjGsDvqNscSczGoPncOOUWi8FVFvoz+WaSKV+kM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n3ZB9Hl9; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n3ZB9Hl9" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20c8b0b0736so7708575ad.3 for ; Thu, 10 Oct 2024 11:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584769; x=1729189569; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=n3ZB9Hl9GSMKmEx/lUk3TOxxvmnE72q9Nb6QJN/j4oQK/ZZl7fEE+aU1iTmEyHusGF MvskZF7JItYNQUwaksK1w6E2W6vRxUpqG/GQskA591YhhwipexkEwz3mVDnMI35S5jsv Y4WSCAtt0cfpUgxFf5UIV/1sPoj+lv6W94JgPBO2jl9ksD7m1Suxh1wAL4H+VTW+S7Pw k0Dmcxek3i15MXvGlTyKrDmdYFW9aHLcUrFd1RXD7zqVAp9dLiiN4ZKuzQ+JtjjDKIk+ B6ODMMAUkUqPGWTVvDQlBjpM7hat/NMris04mBqZ5fLmXqjpagd/DFoKuVvenG/nokos h98A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584769; x=1729189569; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=Cegc58q8KJn+uDTXFISk8xGcZyiFJlUSz84getTLgbVOo0MhmyY2nr5+prno2zrPK4 O4ep7dy6+Do11Ie3HnbHhvTt7hO1AVLC6wfmR8l+awKaGR/UMxC1nsqqfFrT7x6mdoxR QgLyZZiIv6FoxTDXCor5ZoVXPU+iZ3kagdVLymjLGJ133SPYJY5ChrHwowO3TynfSrVS srC3/9ewkJo7m2eACzBCfDmjLOIIYXOQPNfvx16xTGGEOsP9ZNlj02g5ae1QFPUQ21M/ MXIAEperMCzgjjUP72d70LVzIQfZV1rVmkFGHPDAmxvusI2sL//9pbUThBxBRLlS/aKE fykA== X-Forwarded-Encrypted: i=1; AJvYcCVfuDovehgmWUXv9zgocIloBJMTr7yDBHctAIbFmXHYTU/8fSxqT7QfYAfvYtBY4yUnntt3G5TV40BH@vger.kernel.org X-Gm-Message-State: AOJu0YxY6XvI83qaWDSpAzIK7iOw0X0Xk+HFU8E3VHAhEAXO13dyS3EE K1DqUeGs1NfbScj5x5xIXNEgSxQHLiwTW878loXAaUm85/fe5sJcFcRHBTzep/dNkihQBXdhr1W XaA== X-Google-Smtp-Source: AGHT+IGKm7yq47izgr1FmkV+xWCQVhrzT+Qt86SXT8qqzZ7digIfsw6vfVxyVo72E4861BiCHpsXRIV5BzI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e80f:b0:1fa:2ae7:cc6a with SMTP id d9443c01a7336-20c63722d7fmr648865ad.4.1728584768900; Thu, 10 Oct 2024 11:26:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:39 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-38-seanjc@google.com> Subject: [PATCH v13 37/85] KVM: x86/mmu: Add "mmu" prefix fault-in helpers to free up generic names From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Prefix x86's faultin_pfn helpers with "mmu" so that the mmu-less names can be used by common KVM for similar APIs. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 19 ++++++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 28f2b842d6ca..e451e1b9a55a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4347,8 +4347,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, return max_level; } -static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, - struct kvm_page_fault *fault) +static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { int max_order, r; @@ -4371,10 +4371,11 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, return RET_PF_CONTINUE; } -static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) +static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { if (fault->is_private) - return kvm_faultin_pfn_private(vcpu, fault); + return kvm_mmu_faultin_pfn_private(vcpu, fault); fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable); @@ -4409,8 +4410,8 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return RET_PF_CONTINUE; } -static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, - unsigned int access) +static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, unsigned int access) { struct kvm_memory_slot *slot = fault->slot; int ret; @@ -4493,7 +4494,7 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn)) return RET_PF_RETRY; - ret = __kvm_faultin_pfn(vcpu, fault); + ret = __kvm_mmu_faultin_pfn(vcpu, fault); if (ret != RET_PF_CONTINUE) return ret; @@ -4570,7 +4571,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r = kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r != RET_PF_CONTINUE) return r; @@ -4661,7 +4662,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r = kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r != RET_PF_CONTINUE) return r; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 633aedec3c2e..59e600f6ff9d 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -235,7 +235,7 @@ struct kvm_page_fault { /* The memslot containing gfn. May be NULL. */ struct kvm_memory_slot *slot; - /* Outputs of kvm_faultin_pfn. */ + /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; bool map_writable; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 143b7e9f26dc..9bd3d6f5db91 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -812,7 +812,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, walker.pte_access); + r = kvm_mmu_faultin_pfn(vcpu, fault, walker.pte_access); if (r != RET_PF_CONTINUE) return r; From patchwork Thu Oct 10 18:23:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830703 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81E00201245 for ; Thu, 10 Oct 2024 18:26:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584774; cv=none; b=iMi63pUGzGAAHDxS9UCVhdJNWPSS+Jsu6nF1egtjPX5c44rEr1HwlyELXQ7lJOjPRqQynZ0VF8bfFcCpPXG0Ry+I0io2s8mXxxDNRxT5CG6krUAavGDWyUbvVf1JDL4UHB3gvLkejQnokdGh6bUkn+AEe2E0mai0SBxmyebatsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584774; c=relaxed/simple; bh=t32hhiDzmz/KF55Q/3vtmKLh+T++3xGW8zC+Y7mhAxY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WeTrhex4v0u1gPanCcL15a9ihylm50Ht28eSfd/mKFg7V5zgQielucb0DlS53qbWAyeXu1496fs0HaUZqpdXyZHsyqmrnJIPrdf864zqfsjUm9mZwZW7jDQI8ESWq3opKb70vXufSqGIvU3Oo5IQ/RZZuioEsT8xNBviZjmgRFQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bMjYV7ju; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bMjYV7ju" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7e6c40a5795so1040658a12.3 for ; Thu, 10 Oct 2024 11:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584772; x=1729189572; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=bMjYV7juBeFlVMGvI7NvnOECGx9NNcc/zEtv4XL/auuP5Ka903KuEIjSTMzcC1Eugg j8rtVp7Wpeix1cw/WDIwsZIU0wTWT5U751Nxt1Ns0Xa2Fr7UkWo6hT9YrDpr7YpS6OTw LU+YXtX5CGLiB6ZjvIB4nH+NtxFny5aWZnAqymn5BW+gfWLaiXeYGy+NAT0NJ5UGu00k 4dsKzgGcgqUs6EjEUOnGq7lwY8emCm7AxNltyvDSO0iq23nJjbJoWCfuddHB0rSEfkzM JWAWaFN301LUh25d47GCwzrcPcxjw1ki4q6ZHEZ7nWT/H19p2Slj5JxKjRL74P3AOnEm sJ0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584772; x=1729189572; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=UfdqIvQ1w59RWA+ybcQ8vyQqkxhgMMxtTG1Js7gov2dZmzh6nE0eJDH4babVpeAslv jt7p53BAJddCZY09FimD9fKsuaSVQ29H78Cyh5bB1Zhrexpaxvbz7W7u16FzWvM1zH8l Z2d9x10Xt2kyxH0OB1sogflarPi0iUm+OdwesPSIRnY3bSxdj/jMqiMnAxUd8IALynG6 qhiZeeTdFkhjqwS9vYmI3/whPNlJ1Cd5YiuU96cm4XbUlL0pp7nmz20vuctn7xh9yZc/ LvU70Gt+5m4CuMgvXKclmoH3IybdVkwURWhBmRASukE6xArXf65gojuENBubVENS962/ 5sdA== X-Forwarded-Encrypted: i=1; AJvYcCXDi+I2GZUeyaqdpDHSaG0gbgB1AI+p1NEj8igqD3kCZWMZBK07nNFMP7uDjJbs2/jU8wVfER0d+kAr@vger.kernel.org X-Gm-Message-State: AOJu0YxbEpW1v5XtLi4YuljHVbFK0ywDd904ws2EMkXgoThFdEVVeU2a mYlJZzUwXyrjZ7aCeXEdoyTu/YfkoXY16HgoVhT1m9yN3k2yvvfCn12YpQ3LjJ1uNNNnMbOOKR6 /Pg== X-Google-Smtp-Source: AGHT+IHq9hr54ORGaKAudrJMY0Hy9bRYjmUvRYz8zAm0RAj3x+bD9YD3f2uWGLgDblXeQYOnnP141duUJXA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:1d03:0:b0:7db:539:893c with SMTP id 41be03b00d2f7-7ea535a65c6mr17a12.9.1728584770873; Thu, 10 Oct 2024 11:26:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:40 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-39-seanjc@google.com> Subject: [PATCH v13 38/85] KVM: x86/mmu: Put direct prefetched pages via kvm_release_page_clean() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use kvm_release_page_clean() to put prefeteched pages instead of calling put_page() directly. This will allow de-duplicating the prefetch code between indirect and direct MMUs. Note, there's a small functional change as kvm_release_page_clean() marks the page/folio as accessed. While it's not strictly guaranteed that the guest will access the page, KVM won't intercept guest accesses, i.e. won't mark the page accessed if it _is_ accessed by the guest (unless A/D bits are disabled, but running without A/D bits is effectively limited to pre-HSW Intel CPUs). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e451e1b9a55a..62924f95a398 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2965,7 +2965,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, for (i = 0; i < ret; i++, gfn++, start++) { mmu_set_spte(vcpu, slot, start, access, gfn, page_to_pfn(pages[i]), NULL); - put_page(pages[i]); + kvm_release_page_clean(pages[i]); } return 0; From patchwork Thu Oct 10 18:23:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830704 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5325520125B for ; Thu, 10 Oct 2024 18:26:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584776; cv=none; b=W7Df++bo26/iu1IAtQ5RTcqfY3NcNb3ToPNgrWPJes6XgpFqwXMF6vrEr1e3aiDkp/6NcVOOth8MB/ejLKeV1quV3XzlGsarjevP1HrnAtqpFxaaCZc8li5qXQc8RAQHfG6R6lQrt2y4vdjxoJKaAITylv29ko08D0aFRnUxjhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584776; c=relaxed/simple; bh=AMgsQWUhFHfCzgS24A9rVoaxkKJ7Z500dfTCiVjIUZQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oBRAKdSuNaSKVQXWxm8mwcYHYM53oaKzfzPWoMLI4g+A91NDvVb5oUFNLJ1Q4Dh/p491pSYlet2hrsvXHPlHFdo3A/d1KA9cnu6v5B5hg1XvB/NJJU8lAYcgvJ09Hz1TWVANrpFutrlQJ773yXleAOckbV0u+LF7hQFWT+jIFKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uwFCrjqa; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uwFCrjqa" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20c61e4b890so14347675ad.1 for ; Thu, 10 Oct 2024 11:26:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584774; x=1729189574; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=uwFCrjqa7ghmvrDiWY1KS05snxhx368q6kuiBqFzKfDm/bDcUbsQJzABMEIxvC3s/D Slu5mrI9UMYuTQmhw0sA92WKhhiNVsMx9HKgola5QQUGUxYW6ooZJiq5CZ2pOxm9TWY3 Jpfh0H5DOQibZ6lLTjBLWEPcYgkVnXuwvNL/oqgFZ6aLSk/iPMXxJP8vFMUY4hwuYBQE OE1YgF94rIC52TMb4mM+yjZUo/IKJ+IQVB3ZvxV8RZ87he18StRjX489yhFk4pPyaGq+ 1nUyzh9EOKMTJ/MxSSmXIoFRbwC6Hq7zQw2bNbNqevOKWkDvKcgjZgEqXKxi0/s2g2wa tuXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584774; x=1729189574; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=u/PV5hwvTLbbq9ogSem9uxLBtHUAazIj31ojtL59A1FD/AwGK0VxMkT+Drwv8+/mtW UJspMnX0zk0vbRc8Vb5ldm7FuhvX8RwyYejsVvaWotd0k1nr7BlKKHYN1vqIxfXbjbkp x+2EUM30/eLXFZEbt37OX2lCBkA5CrrZS+tI6kAV9zicrdLRhmXWsJIWrlPCZyJIPYe8 EPR93OmPq2tbTfBrM76BIaLz4cBAQNFAoeTZWjzoTOeZi07RobIl1n0MCUFKM+/lbvF/ 7sf34+zMs7VCsMsTTuDxdjon+ghWLP4ZIefHRU3pEqXTv0G+bm+5SvV13mk+6SvAjr74 MROQ== X-Forwarded-Encrypted: i=1; AJvYcCUcblSAIIQHmHEegfEvwEheUbQpCmaXIBVAgU873K77x8H4a8k6/MFQCzcUCla3dcVdsoR4tqVNeWtz@vger.kernel.org X-Gm-Message-State: AOJu0Yx9SsIdvWr3AT9AgyPM+yLSU2OfLPMpBjZNoPkFX0vc1leySC9z pxa+f5DqGF5uBdNgjohMnOpf1F3P0u1tqvxIMm+Msk7slXENwRroLGlO6WtOm1w1BTRfGoQQzeO mjA== X-Google-Smtp-Source: AGHT+IHTeV3dFyfdACUyzit3PmgjtoD4THwZ77SaAB8Cv7KtK6sEvfc4IjBdPT+nDHGdhMcMXcKC7R1qaBw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:1c8:b0:20c:716c:5af with SMTP id d9443c01a7336-20c716c078dmr946605ad.3.1728584773496; Thu, 10 Oct 2024 11:26:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:41 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-40-seanjc@google.com> Subject: [PATCH v13 39/85] KVM: x86/mmu: Add common helper to handle prefetching SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Deduplicate the prefetching code for indirect and direct MMUs. The core logic is the same, the only difference is that indirect MMUs need to prefetch SPTEs one-at-a-time, as contiguous guest virtual addresses aren't guaranteed to yield contiguous guest physical addresses. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 40 +++++++++++++++++++++------------- arch/x86/kvm/mmu/paging_tmpl.h | 13 +---------- 2 files changed, 26 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 62924f95a398..65d3a602eb2c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2943,32 +2943,41 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, return ret; } -static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, - struct kvm_mmu_page *sp, - u64 *start, u64 *end) +static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep, + int nr_pages, unsigned int access) { struct page *pages[PTE_PREFETCH_NUM]; struct kvm_memory_slot *slot; - unsigned int access = sp->role.access; - int i, ret; - gfn_t gfn; + int i; + + if (WARN_ON_ONCE(nr_pages > PTE_PREFETCH_NUM)) + return false; - gfn = kvm_mmu_page_get_gfn(sp, spte_index(start)); slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, access & ACC_WRITE_MASK); if (!slot) - return -1; + return false; - ret = kvm_prefetch_pages(slot, gfn, pages, end - start); - if (ret <= 0) - return -1; + nr_pages = kvm_prefetch_pages(slot, gfn, pages, nr_pages); + if (nr_pages <= 0) + return false; - for (i = 0; i < ret; i++, gfn++, start++) { - mmu_set_spte(vcpu, slot, start, access, gfn, + for (i = 0; i < nr_pages; i++, gfn++, sptep++) { + mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); kvm_release_page_clean(pages[i]); } - return 0; + return true; +} + +static bool direct_pte_prefetch_many(struct kvm_vcpu *vcpu, + struct kvm_mmu_page *sp, + u64 *start, u64 *end) +{ + gfn_t gfn = kvm_mmu_page_get_gfn(sp, spte_index(start)); + unsigned int access = sp->role.access; + + return kvm_mmu_prefetch_sptes(vcpu, gfn, start, end - start, access); } static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, @@ -2986,8 +2995,9 @@ static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, if (is_shadow_present_pte(*spte) || spte == sptep) { if (!start) continue; - if (direct_pte_prefetch_many(vcpu, sp, start, spte) < 0) + if (!direct_pte_prefetch_many(vcpu, sp, start, spte)) return; + start = NULL; } else if (!start) start = spte; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 9bd3d6f5db91..a476a5428017 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -533,9 +533,7 @@ static bool FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, u64 *spte, pt_element_t gpte) { - struct kvm_memory_slot *slot; unsigned pte_access; - struct page *page; gfn_t gfn; if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) @@ -545,16 +543,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, pte_access = sp->role.access & FNAME(gpte_access)(gpte); FNAME(protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); - slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, pte_access & ACC_WRITE_MASK); - if (!slot) - return false; - - if (kvm_prefetch_pages(slot, gfn, &page, 1) != 1) - return false; - - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); - kvm_release_page_clean(page); - return true; + return kvm_mmu_prefetch_sptes(vcpu, gfn, spte, 1, pte_access); } static bool FNAME(gpte_changed)(struct kvm_vcpu *vcpu, From patchwork Thu Oct 10 18:23:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830705 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41C8B202F67 for ; Thu, 10 Oct 2024 18:26:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584777; cv=none; b=dn9XUfx9J0omZDL2g58jpyzeOMVxrw40p2z39l/5bsYI6wNf+a56b9EMKNLtamNNZOES41/eW9VLqwLBp7YcRKwHW8Hz+MCK/z+OyI4kbhcbqYiaQJ/YsxU0n9eNEByR2sSgGzd6c0UJEbgFc9q5ktUu0h9AzD9xF8/8L7IbOQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584777; c=relaxed/simple; bh=PQXxus6kr8nuokmnnM+Lbqq2T+wYMcW3c8L/2YrIVXU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MwZm3FNB58mXKYcj1TV5HX2/QdISqL0MLrRPKMdPtn4SiShpdG6fcoDhfpvAovrqU1kK0MDW98WmJje3EzldJzJ3tGHNvT3u1AY5S1zivUItci7lUQ2UDHbmQ4pp5smcPObfwC/6WeOH04dpqmgD50mJNq7Tz+eswHjfv53ep10= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gciGGDh5; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gciGGDh5" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589045b3a.2 for ; Thu, 10 Oct 2024 11:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584776; x=1729189576; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=gciGGDh5HFpYWLuKNpe3Ev4O1droYOafPNHG8yohFPMnZzjhRji78ghkqI9VqZhT+B Juf6rcXU/vrIP8E/Z1uAY9snG3nfyrtMdkveyBqYvx2MzjD4hY+IMWPMKq+/pXqVwVGX ea0byuwg6VXoi/3PVusUXLHHDb//D85kFO9B+vfoit+AWNu+8Ifoj/K1gWHZ1ayEP7bM 5W/mtnUCiPZ18zjIWT0zKiLM5FJtZBfhithRFXzW4f9XDSnPiQ81VGioPPD2PpwuXPhl XqHt3I/ZMv4OFkktHewNKir892D4AGcVInKoLbwo/PslHwbVt4L20OtMF8zjtlhzbESm QwcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584776; x=1729189576; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=tQZYr5A0pDzsmisrKmWLgOTlWl5u7XQyy8XC5145rNELcfi0z0YrnZoQKorfdkyZeP tAVOmmwMMy14bVkT+pJktK+IEUfP98A3AZWWPldlp+3zZ6aDxqqjSjuSYhcbv8S/6KJx Bz/xzn1tBHZvUFL0qeu0czkKr8PEdCB86mlRf8VADmLXcmlx+u/FUGHDSCK41bd7T16s kbYnQW8U4ETjjE8XeX107JEEfdATN9C5ezQVfAuIrx2obbRAS5IU/eR05DHmW5E8OgtT yfgllkR54g1eCnW7XjuwT+t9+GHjQq1Bu8dsXwU/cntwHnUWIlylWyksB7OH65B05aAh eRog== X-Forwarded-Encrypted: i=1; AJvYcCUFfyT0Cis9I7CzP+IbbMfnXEs5SscVTt5Jlj3ng74uIQeEzxQ59IplCWclWpnLwnwUd6YYMxCu+6NX@vger.kernel.org X-Gm-Message-State: AOJu0Yy/FoKsO4OKaqCNtUQOMVxy3OV8nJO4yfvXAGnpRoCQi56c7Dv5 K59yW1POVlFllqPI9G46zL1Dg6vpTNXRyH3l6YQvD2YRmMtFGCo/Wxl2185ZEOQwSQaGOfs3ctU gIQ== X-Google-Smtp-Source: AGHT+IEXxr/4vsCtgUsUHHkctm5xGRqb6tsNipBtkL7vgMjj75AzVZauqGsQqnffIf336Hhi8e9e6vZrhMQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7792b3a.5.1728584775458; Thu, 10 Oct 2024 11:26:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:42 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-41-seanjc@google.com> Subject: [PATCH v13 40/85] KVM: x86/mmu: Add helper to "finish" handling a guest page fault From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Add a helper to finish/complete the handling of a guest page, e.g. to mark the pages accessed and put any held references. In the near future, this will allow improving the logic without having to copy+paste changes into all page fault paths. And in the less near future, will allow sharing the "finish" API across all architectures. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 +++++++++--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 65d3a602eb2c..31a6ae41a6f4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4357,6 +4357,12 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, return max_level; } +static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, int r) +{ + kvm_release_pfn_clean(fault->pfn); +} + static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { @@ -4522,7 +4528,7 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * mmu_lock is acquired. */ if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn)) { - kvm_release_pfn_clean(fault->pfn); + kvm_mmu_finish_page_fault(vcpu, fault, RET_PF_RETRY); return RET_PF_RETRY; } @@ -4598,8 +4604,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r = direct_map(vcpu, fault); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } @@ -4685,8 +4691,8 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, r = kvm_tdp_mmu_map(vcpu, fault); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); read_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } #endif diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index a476a5428017..35d0c3f1a789 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -836,8 +836,8 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r = FNAME(fetch)(vcpu, fault, &walker); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } From patchwork Thu Oct 10 18:23:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830706 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63408202F8F for ; Thu, 10 Oct 2024 18:26:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584781; cv=none; b=NadI3wFVRQ7FpBAaaFbpBndYe2ZLW/Lc8gr+Ht6zd/Uir5ImHroTAiw3TxDZHCfYiOEQd4iPC2/m23aP48ERoeiyGxKT8AtqZ9VpP6A44trf1/pf1skdFlt+hzxVCjYrlbnE8n889XTCGMMCHNCm7E9PVfuJ9LmFwWUqTPbwqJE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584781; c=relaxed/simple; bh=wP8WMCipPwqkHPMKwQ03EbdALKWDvH1se8OSiGfXvd8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iF07FBc9SoCaKQF9CuazU2RYET/5xD3ZRYu+A8vogqAcJkLckBwRKVcyKjnzwiEc16wxmrU8hBP38vz0po0pbeNLYNmYhKjHU5rqcQ85N4CvEPWYob1GwEPRDzhGEjyIupc7eDLpgg1+mXGJD/Wz0lHxLrLkSjIh1461JGXDVAU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NPuFqHnM; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NPuFqHnM" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20b921fa133so13297385ad.1 for ; Thu, 10 Oct 2024 11:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584778; x=1729189578; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=NPuFqHnMcNqMW7akh4WWzd8zhX+rRXRzBI7zESBDBZBdwT0VEGK+uUaNUj3twhjoaK rcZEJ1pzRTQnu0rv1PaSI4+ObWewHv+EBTozp88TZjRIVplFldJLM+UkcY09Q1ua6RkR Dw9nQw0P48Ua8VAOW6e8QPalxUz5psROQe6Q4WAKd1T6oYcKBfrDyxTj1cLgmmtJXDMK f6TXqyy3Ji++fL8mOKm9O78UFQjHLpTNhmKisvw5USEpHdm6PhXanNcXX1Nq8NMsY8Gf nPwcM1Aql+6FmfWqD78E6aTWg5BuzTjuo501MWC6u3CV3+dNVtlksmD4K27Kqv/V+XkD gs3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584778; x=1729189578; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=c2mxbzRSTQcXEnrLTwdc8M8gt/xiqMpPxwYRL7OVEoaLlhg2wFfP9VuGOnOyzGOyAP o5sYSBaxcAm+YbemlL6Vvfr6i+O/iNvq5tck7/XLEHEmBH1z2mxDPHUJEQiZtFUrOWHz HjSHfgQhVGurOnGVArpfRnN/FsaDw9zNH+qtnSaeD5pdREWXx1/6UR1dVDBOQFue0cal +02YJfB77Wgf2OCYEmH4GEXua5sKmuQy7Uqil9rJJzQf7pG0FSxPdlg3ce8aI3MoHmnz ipvv44UpJKcmGNIDN81OwQC+Wg0qooeX8TwoWnh0Kg8gZ0gQu273uwIo8IDcMYsVX8b6 SatQ== X-Forwarded-Encrypted: i=1; AJvYcCUrpJh/me506TWDaLmmdR8YrWTkjm/zy+M7vCg4lHHkwq7ghM8RtpMzlOjYN0ozdzpLfYQlR6PUv5kx@vger.kernel.org X-Gm-Message-State: AOJu0Yzl5us+PSHqtPKBzQB3uQe+NlmsU0tY+hcbtfDx7LFl9lgxPJMT p9JZ4UOKbkK2zG51Lyhd8m4Xxf3zLT24ZuPKSyEmYI/cpihtRs6D1zv5QynxG28g6osQo8CNZYW uOw== X-Google-Smtp-Source: AGHT+IHgES02wxBmUBuw4MXImaMxOgBvLS6HzbS699NHinso/VI3SIlimd+hdhb5F/AgwxmTSyUtUzYKtBo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ec85:b0:20b:7bfa:ac0f with SMTP id d9443c01a7336-20ca037f212mr485ad.1.1728584777475; Thu, 10 Oct 2024 11:26:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:43 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-42-seanjc@google.com> Subject: [PATCH v13 41/85] KVM: x86/mmu: Mark pages/folios dirty at the origin of make_spte() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Move the marking of folios dirty from make_spte() out to its callers, which have access to the _struct page_, not just the underlying pfn. Once all architectures follow suit, this will allow removing KVM's ugly hack where KVM elevates the refcount of VM_MIXEDMAP pfns that happen to be struct page memory. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 30 ++++++++++++++++++++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 5 +++++ arch/x86/kvm/mmu/spte.c | 11 ----------- 3 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 31a6ae41a6f4..f730870887dd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2964,7 +2964,17 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep, for (i = 0; i < nr_pages; i++, gfn++, sptep++) { mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); - kvm_release_page_clean(pages[i]); + + /* + * KVM always prefetches writable pages from the primary MMU, + * and KVM can make its SPTE writable in the fast page handler, + * without notifying the primary MMU. Mark pages/folios dirty + * now to ensure file data is written back if it ends up being + * written by the guest. Because KVM's prefetching GUPs + * writable PTEs, the probability of unnecessary writeback is + * extremely low. + */ + kvm_release_page_dirty(pages[i]); } return true; @@ -4360,7 +4370,23 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - kvm_release_pfn_clean(fault->pfn); + lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || + r == RET_PF_RETRY); + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (!fault->map_writable || r == RET_PF_RETRY) + kvm_release_pfn_clean(fault->pfn); + else + kvm_release_pfn_dirty(fault->pfn); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 35d0c3f1a789..f4711674c47b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -954,6 +954,11 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int spte_to_pfn(spte), spte, true, true, host_writable, &spte); + /* + * There is no need to mark the pfn dirty, as the new protections must + * be a subset of the old protections, i.e. synchronizing a SPTE cannot + * change the SPTE from read-only to writable. + */ return mmu_spte_update(sptep, spte); } diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8e8d6ee79c8b..f1a50a78badb 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -277,17 +277,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } - /* - * If the page that KVM got from the primary MMU is writable, i.e. if - * it's host-writable, mark the page/folio dirty. As alluded to above, - * folios can't be safely marked dirty in the fast page fault handler, - * and so KVM must (somewhat) speculatively mark the folio dirty even - * though it isn't guaranteed to be written as KVM won't mark the folio - * dirty if/when the SPTE is made writable. - */ - if (host_writable) - kvm_set_pfn_dirty(pfn); - *new_spte = spte; return wrprot; } From patchwork Thu Oct 10 18:23:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830707 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A46C42038C0 for ; Thu, 10 Oct 2024 18:26:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584782; cv=none; b=ZCqcplN6fQPkUd2D9t8mCrAoZM6sOjG7OkwGomhNdElWuBswDKgETI/QaXOcWangZ+pKdFus6ziMOc2PlzgKC47fBkWg8HnPvBg32DyeKUqTcch61QFnrPhZbx18Kirtl6T22T2c5DzYV/dgTLmuHrHWmoHV3LIElg1sRl6U+xU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584782; c=relaxed/simple; bh=hSHnz2G1ntKemXewFOwXdDJXjhCTKhhhRrw1wSIGnaU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HtSxPI3ddHNRt1GNu1U/Xs/gjcn/yRICPrhW01Zucn+38LWBJl2T2u5NfT1vc7R5bpjjNsN7APhoTpuzqSlXCCm1N7PiC8bRErxAQpqOMIBGkoP23pMpZuZC/mS7zZpEUhWV5wEpT5qi2HfSJ/ajzx4t6W1+Ax8zggmyzrqB5kA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=E53iqmq8; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="E53iqmq8" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71dfeda9ac0so1339090b3a.1 for ; Thu, 10 Oct 2024 11:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584780; x=1729189580; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=E53iqmq8I7HHufRa9Nfa4QpSUD3mPUf0HdiC+gr3+ZJVA0PtJRMGFCy2RlpuApodID MHkYZafORHG4Fo5pdlmIa2Zu49fY42Dn8Mwbg8BEIZ/Pa8OJQiqqaoX3cE0NBkDUycJi 8knjazonwqf9LvKUPFmBk2crmB0MGGb2WbPM1i7pjkVZjFgAzrwTCr7oBWE4eumlOw58 NDuf4DR/sTZJYm1oq12ic4ny/HL0dHq+Txgeh165RAyYFWZt+w+um9fwU7jF7aG4NAnz uIpRYBDmcMn403NAN6nAehV9jbOFJtSN+/wsSbKYsr5xfV60UWvUZH56iYMQo+BvtQRs hZkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584780; x=1729189580; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=OqRDHtoDbqPiFjd742dQVrp45+F6KJ0+Bz1BzEl/cxUI9lGtk9tJJJAQpkvzZkzh4a fEajpL61M+acVuvF0KPxiEEjW6OaCyzcsLI15W5tbbsCyP6SireaZSc7Uxg0rOd4w9KG PnptUE48OqrwZq6BYFRwbrgPo9SE26tkEzJA+uuJAbZMWBbTCeeue9uNPnLWGK2P7P52 87V329E7Xzd20sUH3vthgnsULv5WmAAOwED9AFzCJyxvv1++NJE4F4o6vq67ubUiNWg/ sx2ThkhUm4XR7Ys+RSONHzfgh3WWZpMlQ0rJV6N/gYTAPvaBZuU6qYtcNCZVGIgzRlt/ /sNg== X-Forwarded-Encrypted: i=1; AJvYcCVdrUGFO3z9NxtpSnvqchMd04hw1n6cB9xt+rmCndaexjrDZyfQrDudMLyDF5LIM23F6gDlQYAqaKPQ@vger.kernel.org X-Gm-Message-State: AOJu0Yza7xEeSemPdSI9JuP752Mb1ovy8dWp8zKv9Zuu2BDBN+CcVxUn +n3LwV6C+n8n+LOhiAth4IhvcKxTG6Ym0WFa/tmhqqCaU3V8fGy4iPRgeZeP5aLGhR0Ms3YGzbZ qmw== X-Google-Smtp-Source: AGHT+IG3pPvsekcqYnotExeZVZUXEYlJLq+6e3a1eutYJX4gtVKcKQbhJKMNrxmpaRIGMBxtyW/kZ/ByL14= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:91a3:b0:71d:fb06:e79b with SMTP id d2e1a72fcca58-71e37c53abamr28b3a.0.1728584779496; Thu, 10 Oct 2024 11:26:19 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:44 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-43-seanjc@google.com> Subject: [PATCH v13 42/85] KVM: Move declarations of memslot accessors up in kvm_host.h From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Move the memslot lookup helpers further up in kvm_host.h so that they can be used by inlined "to pfn" wrappers. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9263375d0362..346bfef14e5a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1168,6 +1168,10 @@ static inline bool kvm_memslot_iter_is_valid(struct kvm_memslot_iter *iter, gfn_ kvm_memslot_iter_is_valid(iter, end); \ kvm_memslot_iter_next(iter)) +struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); +struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); +struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); + /* * KVM_SET_USER_MEMORY_REGION ioctl allows the following operations: * - create a new memory slot @@ -1303,15 +1307,13 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, }) int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len); -struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn); bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); -struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); -struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); + kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, From patchwork Thu Oct 10 18:23:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830708 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77DDF204083 for ; Thu, 10 Oct 2024 18:26:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584784; cv=none; b=QmD/PtXqoycKNyt0Kl94vdMVM3jmcZsXswWBbpaPzQ8cHKF0Bxue+f/f3ZU/swjfYBfB1S/Fi9MmHIayORfYurEp6skl9umqiEafB1LOJsuWvKp/688aR9dAhQpW/QsCydGfpeRJFR9mosdHr1wnvniihoxwe8PGtXAUgm+ZrLU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584784; c=relaxed/simple; bh=acp8IhS6KrAOGkkRIhpxPxZ31aINXkAYcQRqekRRiYM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=APjL5vScxqGjfRMwPg4XVpvbZu36gFvs5SePFrl93p3gg8Is4k3LJJZptGbMHiKg+UU12Hf2hx/C1dqHRqnrfhVrjW6b/EGYLbfRja8pAga7Z2jwkpZzk2l2v3VwURP2xP2TrQdfOrc3HXQXcCSidIxRAIFKZh7qUCxpDUFB43s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Xqt2Fs5n; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Xqt2Fs5n" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e29b4f8837so18944117b3.0 for ; Thu, 10 Oct 2024 11:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584782; x=1729189582; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=Xqt2Fs5nbw3a9rY6oVLT35/D0l+G/AD+Q49HX1QdXMfC2SfNz4O5O1dvtdhGzDWUDl JpqTDsMCsN6OxFww5+NH7ydMQnMFtfrmh34ZhDsUCf/uTkDGYmH6VlOYAcds86k0XnLu dP0P0MTgvn/Qf9H0M5XOt0K7URLrCKcwSGSrKXhaF7fEgRm8tzA0j54XzIeR4IkT7Ji5 AoZrpW3iiSMjwhmWTfZ64gNRi5aT38cY6iP8t9xd8k0HMzsVYBTUhBM1elfqfp5jqfzO ADF/8T0Dad0vzNIZJhArpIvd/mEd9y0HE3XU4hpM3jp77eHkpOjUClLqF4+0clJvSyS9 WkzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584782; x=1729189582; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=iSXYsm9PmpIyGzFIP/q9gY1OkzRGq/FBu/HubuY/auobL5frmDrH6TIo0OCdaxRWsm i0QJMDajAMW0RHF/ntl1EKi0RDWzvc3XlLuxYMnaDIYc+i0FeTZKrQDMZ4VvbyLVc9WF ElXSAGhT5z5FynZowihv2eJkjuUmC3xfnPP+AsYR0zcdm9bgiZcPEhXqieV2+05XOuDV vVIVgHsbgnrJceWFdV6rC3gD5IqbzwrbFnLzCLMe/g+E5SFOQAlzc1nr/Y58XTwWMiqU HAhAPawmFG+nYue91xmusAO98cl2Vuul86veBT1CHjBUvavIg15CUpc6BT2gwhYmg7VB 4DIA== X-Forwarded-Encrypted: i=1; AJvYcCW7+irpITqOZ5faOXHXo7EqeX5ZuYO/oJHNSwCPME/V0q45myy7wSMi8Dpbj1IK6o5iutwwmrUCN/vC@vger.kernel.org X-Gm-Message-State: AOJu0YwI50PUmz7z5w6UrLY7GfUX27QjFtOeskLcpG7pq+GmZ3zDub/N wEBifYZ/NEfdEa9rJ/aONzoBTq8pY4P0OGQm3YrsFdJhIkpLins13gNDmjUMwteL5MNAd7DbzYp 2HA== X-Google-Smtp-Source: AGHT+IEBXjEpYcXfPYRm3Nf4G9ZN1ucR55aqjscHr2p9QPsSrcRZYI0185ucK64AXquh+1QVcruWQv+kNDM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5605:b0:6be:523:af53 with SMTP id 00721157ae682-6e32f2f4e28mr919077b3.3.1728584781575; Thu, 10 Oct 2024 11:26:21 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:45 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-44-seanjc@google.com> Subject: [PATCH v13 43/85] KVM: Add kvm_faultin_pfn() to specifically service guest page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Add a new dedicated API, kvm_faultin_pfn(), for servicing guest page faults, i.e. for getting pages/pfns that will be mapped into the guest via an mmu_notifier-protected KVM MMU. Keep struct kvm_follow_pfn buried in internal code, as having __kvm_faultin_pfn() take "out" params is actually cleaner for several architectures, e.g. it allows the caller to have its own "page fault" structure without having to marshal data to/from kvm_follow_pfn. Long term, common KVM would ideally provide a kvm_page_fault structure, a la x86's struct of the same name. But all architectures need to be converted to a common API before that can happen. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 12 ++++++++++++ virt/kvm/kvm_main.c | 22 ++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 346bfef14e5a..3b9afb40e935 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,18 @@ static inline void kvm_release_page_unused(struct page *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page); + +static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, + bool write, bool *writable, + struct page **refcounted_page) +{ + return __kvm_faultin_pfn(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, + write ? FOLL_WRITE : 0, writable, refcounted_page); +} + kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6dcb4f0eed3e..696d5e429b3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3098,6 +3098,28 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page) +{ + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .flags = foll, + .map_writable = writable, + .refcounted_page = refcounted_page, + }; + + if (WARN_ON_ONCE(!writable || !refcounted_page)) + return KVM_PFN_ERR_FAULT; + + *writable = false; + *refcounted_page = NULL; + + return kvm_follow_pfn(&kfp); +} +EXPORT_SYMBOL_GPL(__kvm_faultin_pfn); + int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages) { From patchwork Thu Oct 10 18:23:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830709 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55C792040A4 for ; Thu, 10 Oct 2024 18:26:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584785; cv=none; b=b10bRbWVX+1b6oyP2mp0Wy5iLqfKJZu7UXPscdKmVNdE6qGaFKL4S1aUwDodNh1oOm85e9tjOtlaIW3nu0BpIiGiXAnJ5zc49IECMG1bCd9oTBvNgF9QmjQnDXUHWl4c3uZpRlBCwMaxQeQWvz0YbN5WIsW7F9T9bKkuz7aj9qQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584785; c=relaxed/simple; bh=Hkx6OqV7mFxqog7S2nXIkbHpJQToxM9/kCtdODEngY8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=T5mqv6aQ59A6AzTutDYi/pYDJVB5NW37nMWR0g+ziMW4JH7Xg6Pq49bs+FPKl9NkbL4Rs4+Ie/w8ADNNYNoBAE7yqCoK8/xh1/xALG+xevfPnqSi2CMemHhE+HyNEPR0KjixaU0HHhEVWQGufOKwYrUlSy+9pn0WyBg0Q0vi9Bg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bQ8YWjsA; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bQ8YWjsA" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7ea05b8ea21so1336010a12.2 for ; Thu, 10 Oct 2024 11:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584784; x=1729189584; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=bQ8YWjsAmSCtND2aP1+tnixLSReqcEMO5NgW5XaAIrBbL2LakVMBpTYBr79a6Ie4it yNo7Kq1xSA5MpJ+3LWpNyQ8FebCsfQePx6QL7MRRwa7HUCZ4EAISOfHABrJmf07r19dm vC7F1QJ2GzmgWC7lF+MFU+KbFzyvgv1xoyoqQgeGA7foloGcAJt5EuXt6aPoIbgmFg8d S+O2XvahdLb6BSZR7nlHR5Hzv2cmwMq+KpUVit9qdBBsIoaQJtbbi96Ot04RWV1vcYCh XiFcWmxVFgOdE1nBSgaaQ0SHKFiqMCAzj5V7S3qW2P8xyztxxmcbI6IW8VfCE4tAXTJs qhzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584784; x=1729189584; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=GE4DxHFOXIsVwP3v4GTh+dLu/z7lvNvt5z7cqmC0ztuOYhNPY6apiRS19q0Kvj9Pjm lSE/f5Zuu9bJOCT46ZtkvOwQpNdD7ZdvspbAtjgsFck0JjvnXgkC7lafj221HfynLVvo j8LPye/u8ZRaKIAM4N/OdV48l9N1sNU+OIk1TjbN5vsm7OIn+dzhqj3PC3BH85tZm9vT /Wi3W/Zf7zeJbAfhWRHMl2VA1kVX/R5mPaPmGOBY5ZcD4JqBLdIfyqi+khUicrdO4fBc 6B8gKLfaiS5AYUYWRaU/e2DlWq+TW24kFBf2SYhUQ0zTiUHp2Og7+ud9LLTXVC4gGxHD cMOA== X-Forwarded-Encrypted: i=1; AJvYcCUlwEENq6OfrCx2HWu5/vWh8k+CEQvDrMvAFTflpRI6yOUE+QoFMAI3QFTbjUZxEF71XwOkNXpkjUlr@vger.kernel.org X-Gm-Message-State: AOJu0YycyPlQAnPAIOAW/Ex5zMxXpwYaGcR3DS5FIZ1TLG44NSgB47vk Q1WFZgSX3i5Sred1HDTIavYXbhHpcQEneZE18WpKbdLuuqlyoin28yMnUm74Yhe5y7vVA2cnTkg 1vA== X-Google-Smtp-Source: AGHT+IGY3J0AFIsOlJbVDrqn6majLeEUOkcp3LAakwsbgSwDXT3SnAGgA9L20idMLdLGQvBeoRsHZDJxo50= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2445:0:b0:75d:16f9:c075 with SMTP id 41be03b00d2f7-7ea5359cd2amr19a12.9.1728584783356; Thu, 10 Oct 2024 11:26:23 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:46 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-45-seanjc@google.com> Subject: [PATCH v13 44/85] KVM: x86/mmu: Convert page fault paths to kvm_faultin_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert KVM x86 to use the recently introduced __kvm_faultin_pfn(). Opportunstically capture the refcounted_page grabbed by KVM for use in future changes. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++---- arch/x86/kvm/mmu/mmu_internal.h | 1 + 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f730870887dd..2e2076287aaf 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4416,11 +4416,14 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { + unsigned int foll = fault->write ? FOLL_WRITE : 0; + if (fault->is_private) return kvm_mmu_faultin_pfn_private(vcpu, fault); - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable); + foll |= FOLL_NOWAIT; + fault->pfn = __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4447,8 +4450,11 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable); + foll |= FOLL_INTERRUPTIBLE; + foll &= ~FOLL_NOWAIT; + fault->pfn = __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); + return RET_PF_CONTINUE; } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 59e600f6ff9d..fabbea504a69 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,6 +238,7 @@ struct kvm_page_fault { /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; + struct page *refcounted_page; bool map_writable; /* From patchwork Thu Oct 10 18:23:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830710 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F69F1E1A1F for ; Thu, 10 Oct 2024 18:26:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584789; cv=none; b=jmgtgDwDOHhWRwfY72wfTIRN5e7zwLbV+fTVSnCLXPkJTZZsGYD/qJJYoo2t6kP122t30Hh61fvctCulZ6ZeGiEoEJkFlaidhlN5G/p26U/rbDNzc14he7nZFbjYpQ4lj6lIUvxfOHWM5nVi5GtnkueTE+rS1rorcu65693Nxjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584789; c=relaxed/simple; bh=YHJlLBVBXana8wfsUYax0JzdySoal3WoOfu+2A9pl0c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qUI4vIQM46wtqJwT/Ru+CiCpjrbVjv1ivtpzkTWztkoYgIucQa8duWHwn4+VzPaV82Ctqu7oL6MlOZ5zjGvveghDM/tjS0HD7mJNMDjy7HInMM3NHzKLL2ezWfvi2sQx6YuBVWteUA0SnCGRnlmYFx5GltmvnPA3QheEqjnItxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dzGkvQXL; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dzGkvQXL" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e123d3d22so1429052b3a.3 for ; Thu, 10 Oct 2024 11:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584787; x=1729189587; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=dzGkvQXLwKnQ8kV49+00tqUsM+pb+EFzb+lv8vwXDt79FPnTmt4s8Md8nPZ0C2odAg BR9Oj/cN5OU7Z34cg4SiflaOOJ5uSAxjPR2L8QNFBM0aMzMUmmkaHfupKWrtFiM8kejf VixuAbaQpGtr0YhglX051laUICzExDICU+fAYRcbxNncGOKDgEvabPI3E8c8OvZZimsV 3a5isk8DF+25PgKctldAVdBofKibWUxqq0QyK7kk7WWlPWAEs6Px7f46IOU1Tcdum/We /OMbeEr7lpmmWWpGWaNU3NyZrQEJEuyYs8E+PUOurydDDM/bBLcqQeYJuELStICgdxYN kUHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584787; x=1729189587; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=ffZDSPvMj2ITpgMJnS8op5KzCIqsDP4BVLsnRy3LW7kyKcCAqm95ZEkJ+ipiKXV/bW INrkzDOGpSPGAHuwhbo2GHFFBlXDoAliegQk5bjA6EKrXQ2q3QmCNasxQeZ5f08mdAvX c5QgazN+8Z4jT+aVaV9stogSrIf7vMb9s0MfHjp1Dt/uypvDNdfmcjcqm/kxARHMDgJa qmKIB4Zezp8oYKotI/GyBtBQmn2tUbsA1A/j2O6nHkJw0do+SQsyWmKNf1al+U6kXMed Uw/qXvtPAmDY8ZObTj1wcCuCFs6/LhMbHq5O/RCiPLBqYMJI1vwHmRU9QNSvvYyAc7TW RG/g== X-Forwarded-Encrypted: i=1; AJvYcCWCaH9SxLUwD8K3eGcSFDIjfVnVU5s9B5g9UFT34gRXavBbSV74SQZFvuKAk5EZLS4IU+yM1kM/6s49@vger.kernel.org X-Gm-Message-State: AOJu0YzHPOe7HmbTLCmgmcUqace/QHtLSwIFV0nbk2lJtu8yV6aDcGeB O0anmw1ot1X222IOHKmWw6T5daymPgFYhGfyJ0COhGv79GOFK4VFh9rt9GTQQin8ZXeshijwhHt GxQ== X-Google-Smtp-Source: AGHT+IHljZntO7iRsDkdAW1llbbIMc26iesy/7PCA9rtG6130gQpp6naB/6JGeYrnZXaXdo41CPgD6F3FKs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7795b3a.5.1728584785442; Thu, 10 Oct 2024 11:26:25 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:47 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-46-seanjc@google.com> Subject: [PATCH v13 45/85] KVM: guest_memfd: Pass index, not gfn, to __kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Refactor guest_memfd usage of __kvm_gmem_get_pfn() to pass the index into the guest_memfd file instead of the gfn, i.e. resolve the index based on the slot+gfn in the caller instead of in __kvm_gmem_get_pfn(). This will allow kvm_gmem_get_pfn() to retrieve and return the specific "struct page", which requires the index into the folio, without a redoing the index calculation multiple times (which isn't costly, just hard to follow). Opportunistically add a kvm_gmem_get_index() helper to make the copy+pasted code easier to understand. Signed-off-by: Sean Christopherson --- virt/kvm/guest_memfd.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8f079a61a56d..8a878e57c5d4 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -302,6 +302,11 @@ static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot) return get_file_active(&slot->gmem.file); } +static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return gfn - slot->base_gfn + slot->gmem.pgoff; +} + static struct file_operations kvm_gmem_fops = { .open = generic_file_open, .release = kvm_gmem_release, @@ -551,12 +556,11 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot) } /* Returns a locked folio on success. */ -static struct folio * -__kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, bool *is_prepared, - int *max_order) +static struct folio *__kvm_gmem_get_pfn(struct file *file, + struct kvm_memory_slot *slot, + pgoff_t index, kvm_pfn_t *pfn, + bool *is_prepared, int *max_order) { - pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff; struct kvm_gmem *gmem = file->private_data; struct folio *folio; @@ -592,6 +596,7 @@ __kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *max_order) { + pgoff_t index = kvm_gmem_get_index(slot, gfn); struct file *file = kvm_gmem_get_file(slot); struct folio *folio; bool is_prepared = false; @@ -600,7 +605,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, if (!file) return -EFAULT; - folio = __kvm_gmem_get_pfn(file, slot, gfn, pfn, &is_prepared, max_order); + folio = __kvm_gmem_get_pfn(file, slot, index, pfn, &is_prepared, max_order); if (IS_ERR(folio)) { r = PTR_ERR(folio); goto out; @@ -648,6 +653,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long for (i = 0; i < npages; i += (1 << max_order)) { struct folio *folio; gfn_t gfn = start_gfn + i; + pgoff_t index = kvm_gmem_get_index(slot, gfn); bool is_prepared = false; kvm_pfn_t pfn; @@ -656,7 +662,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long break; } - folio = __kvm_gmem_get_pfn(file, slot, gfn, &pfn, &is_prepared, &max_order); + folio = __kvm_gmem_get_pfn(file, slot, index, &pfn, &is_prepared, &max_order); if (IS_ERR(folio)) { ret = PTR_ERR(folio); break; From patchwork Thu Oct 10 18:23:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830711 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9755E204953 for ; Thu, 10 Oct 2024 18:26:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584791; cv=none; b=L2HCiD4YXwShvvdSCyS6B/ryGdGgV1ojyMnVCeEfYWEW51EfSfnHA1SCa6OechmP4CQo0/me8+u6BIozJu+oD010fVVWPZ7JQaComWXVkcse5Wr7mE0xv2ViyKlTWj8RWBlUlEZwgCarkj1bsI3EPSUFnMy7CwpG/RQh5jPBtfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584791; c=relaxed/simple; bh=bZ2ermi+i+GlJV3rIdZ7irm2X1sw1usJ5ghB+vq7WnM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Jes9LIG15mftsdbWr/cNFjQCMZH1whZYhaX9AIePXzXPdSjmqILSmge5/dj0hqF+xG4BC4gwEja7H/FqMSQgUL2y4FUEqYQ1VbyOlkijBgHAue/javK96Ngf0VW8EI+Aw5GwcvPNEcaJD0yzXBGl71VXd2jNnb9kI7mcxUDciWM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fpNaofRE; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fpNaofRE" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e28d223794so23890077b3.0 for ; Thu, 10 Oct 2024 11:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584789; x=1729189589; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=fpNaofRERnQsVJWoNEKh/7Nowa8mnaYcBTqm4xvL96gZzqD0HTZnR8uv/7QUn44Mug vEiJev1UgG+mhZqqAgsDxOJvxhY9x3JDP2NQGvFiuEvxX24RgTLILDckgh/2qq6X3KtK 6Dc37IioCu1GchTrprfk2zvXU/JJTKtdXYop7F8ngmHmgB8Ds7A6py1mzGRTvBuh1e5v +tZ5qSEUmDv46Q7aijpywSEqaqKx6/E4e0UoBTUqX5sTdpf4Hd3zgayPbargNUbcalP/ ZD1NEvP7da5EwZ7MewAiUiFrQgpkfzQgUoU9SYS7aXy/Ax2P3Iv3RNqnAhJDoCKcHA5A WlSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584789; x=1729189589; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=aWoKjiJsVlG1JU23EQRnNwSItbXWbWZXgt+XCQfaP1gEUbwkfIbgtl8zHSIchcuMu6 qCyKi+eIpY+6Me/ofrmkgFCtNVgMXcZrxIC2umh4KYwrB6NRaCCzgD7EYFouhSN6ryzS te15KxcSnF1ymo79fNviNWByika5gATSsYJx5dwIZRLZppGBcjmeBNVnlDWK9ajAY3r5 Oxoxf9Yn0NB9xDZXOPO1ogxfxnwwrzATZZoKBjoES8EjxjTfvsT7L3ZMFpwVNk2YVeqR fOvC4zip/gm1bpyo1lnf+IxdhflNwZ/duktrYjkej+53Fiy4ldVioklBm/9PFCGzRYvl rVLw== X-Forwarded-Encrypted: i=1; AJvYcCUfcWaDZ7xooopiv3Euh3D8yIAP3I17tzN62d4pzNkxCvsTRkqwuSe3VMZzW6TYROEIQATNKUvbZZIN@vger.kernel.org X-Gm-Message-State: AOJu0YytuTQyWAz6rBIeLXpSdfuvtMxR/710QzQtz/pHFc5szAUKLIZ7 pDS7GUhfjRj5jC2lUafwYyCDceyvA2QL+Lg5MEOMsoKB8U548U6H9paE1udNl+n9dqWgisU1eRH +ig== X-Google-Smtp-Source: AGHT+IHLQFn8WLOIHu4PHZ3K+TrLKBMF9JlmWl30kTXK43TD90b+TNbg3ZepRFQQdmJeel5bU/EhDzobpdQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b191:0:b0:e1f:eaf1:2254 with SMTP id 3f1490d57ef6-e28fe41acaamr82536276.8.1728584788687; Thu, 10 Oct 2024 11:26:28 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:48 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-47-seanjc@google.com> Subject: [PATCH v13 46/85] KVM: guest_memfd: Provide "struct page" as output from kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Provide the "struct page" associated with a guest_memfd pfn as an output from __kvm_gmem_get_pfn() so that KVM guest page fault handlers can directly put the page instead of having to rely on kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/svm/sev.c | 10 ++++++---- include/linux/kvm_host.h | 6 ++++-- virt/kvm/guest_memfd.c | 8 ++++++-- 4 files changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2e2076287aaf..a038cde74f0d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4400,7 +4400,7 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, } r = kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn, - &max_order); + &fault->refcounted_page, &max_order); if (r) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return r; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 4557ff3804ae..c6c852485900 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3849,6 +3849,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) if (VALID_PAGE(svm->sev_es.snp_vmsa_gpa)) { gfn_t gfn = gpa_to_gfn(svm->sev_es.snp_vmsa_gpa); struct kvm_memory_slot *slot; + struct page *page; kvm_pfn_t pfn; slot = gfn_to_memslot(vcpu->kvm, gfn); @@ -3859,7 +3860,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) * The new VMSA will be private memory guest memory, so * retrieve the PFN from the gmem backend. */ - if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL)) + if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL)) return -EINVAL; /* @@ -3888,7 +3889,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) * changes then care should be taken to ensure * svm->sev_es.vmsa is pinned through some other means. */ - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } /* @@ -4688,6 +4689,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) struct kvm_memory_slot *slot; struct kvm *kvm = vcpu->kvm; int order, rmp_level, ret; + struct page *page; bool assigned; kvm_pfn_t pfn; gfn_t gfn; @@ -4714,7 +4716,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) return; } - ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order); + ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order); if (ret) { pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n", gpa); @@ -4772,7 +4774,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) out: trace_kvm_rmp_fault(vcpu, gpa, pfn, error_code, rmp_level, ret); out_no_trace: - put_page(pfn_to_page(pfn)); + kvm_release_page_unused(page); } static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3b9afb40e935..504483d35197 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2490,11 +2490,13 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) #ifdef CONFIG_KVM_PRIVATE_MEM int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order); + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order); #else static inline int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, - kvm_pfn_t *pfn, int *max_order) + kvm_pfn_t *pfn, struct page **page, + int *max_order) { KVM_BUG_ON(1, kvm); return -EIO; diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8a878e57c5d4..47a9f68f7b24 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -594,7 +594,8 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file, } int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order) + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order) { pgoff_t index = kvm_gmem_get_index(slot, gfn); struct file *file = kvm_gmem_get_file(slot); @@ -615,7 +616,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); folio_unlock(folio); - if (r < 0) + + if (!r) + *page = folio_file_page(folio, index); + else folio_put(folio); out: From patchwork Thu Oct 10 18:23:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830712 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5357E205E09 for ; Thu, 10 Oct 2024 18:26:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584793; cv=none; b=IbD6xbt33/DbVzXm+vR460D8AqcOER3ghZculRHMfDDE1kRgC8dTQKg5OrtTVqfjCwvnjShkAWGPlHZwvJE1H3oy5XRAOxttp/wgsj+Ak2B8F6KZ8V0x84tcoHertw6GkDY2BtguigWxtFqParTsv+t4R1tF4YZrmHQyLh1XMRs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584793; c=relaxed/simple; bh=P6yv0TC0koYnkFHtfd6W3SlzMlk4IripfIbi6+Lm/9A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=a2d/+odCzFeksHpyNQhqv38jZHmozDSH1rHR4NV+3KZlPbSrjx+Sgns7Eq4BV1FROU4Z+1T2GYv7djoFOfxgvAIJh1NL2Swcrw3ruKsTnQMe9nAotO2yaHdTpHtRPtSXex1OZh2kTlq4xTP/RkOHY+yPxNYN/d0dIrzGI4/Xe6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=a77PMqjI; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="a77PMqjI" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e0228d71dso1225578b3a.0 for ; Thu, 10 Oct 2024 11:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584792; x=1729189592; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=a77PMqjIq3vLWks/ShFXHpZbDnuBVQmai91zT69BbUwBg3+sNEzUevImv4vYPqB3SZ om8MeiFZmfuFyqTQa0KjSKJA9hlghMqeoCasv0rEqK8+1XvrBNkUsubtWjj9vD7aGzDK DWHK1/6pg+wg4oYoW9aC+R4ALvw4SAmWD1iz5nlxSjYqP9yc0vS3FrHLDcpYL5IfkSPZ ZjeNHX7Lj0IluQY7kzhgYqhHrmn/TxGtn+2w11P0tE/6zbCPcnROX6y1R0rOcOpCorWd xxlujfXtkPqmsR8pGzTSMnqlSfuQ2+mvEbtk+8vSE9cw0rUnXeF2Yz7zxJeQSu3EBkzH +QFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584792; x=1729189592; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=KoeA3rVOUmRwzMLxagyhtpxQ9VXs36t1BELyG1A2cqQWDwPAd+O7nHNRhOLMMWxRwi z1nt3hr9LGCK5MtI6Tfk06FCjWJoFpxB4nIQsrd0Dt5V+FVbIGbVM0bIJOza9k0ToSal Cvd9Sh/o0iqNJ2uqzZIET0lJgGfl/fhMJ5+MzBvd+KkYaaGz+/DYkY7mKT3OOF9ujloS gUoaZWoC6UCVmWjKUs4rLODmNlLnRk3NkZfIAyrnf8L0NWliJW6LdH3Ob1OCItkzkf2h tIBXYge5iyT5UKVzNvqcIR9joDSQd0494SnuZcz+7QcJteg50yGfF12rO90Fe/rr2PcJ l9ig== X-Forwarded-Encrypted: i=1; AJvYcCU7vFknUcRXCsfZMbfYnABb8W/MwmV/4tbz6j3fc3NWqaGeAVEuz1E1Z2bwoVcct1KESEkiS9fn5Fzz@vger.kernel.org X-Gm-Message-State: AOJu0YzzJ2ZFGfJRCqBlSyvKNOUdpVoc2Vc5ms2uSZS7etKJMFCBtkvU PqYkhFLFmpEQMYYPpwl7u8lgwR+07gA/V5BxO8d41KaE+/sILQiw4v4FswaDhQNXGjaTg51I3aC eIA== X-Google-Smtp-Source: AGHT+IFunsAHL0GYDEyxrwNnSExkBiXOv88aJkEgjVxw5ZDXOBsEr4Eha3ByEK4QIBQ4extfr7SnAz+Fl/s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7799b3a.5.1728584790633; Thu, 10 Oct 2024 11:26:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:49 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-48-seanjc@google.com> Subject: [PATCH v13 47/85] KVM: x86/mmu: Put refcounted pages instead of blindly releasing pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Now that all x86 page fault paths precisely track refcounted pages, use Use kvm_page_fault.refcounted_page to put references to struct page memory when finishing page faults. This is a baby step towards eliminating kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a038cde74f0d..f9b7e3a7370f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4373,6 +4373,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || r == RET_PF_RETRY); + if (!fault->refcounted_page) + return; + /* * If the page that KVM got from the *primary MMU* is writable, and KVM * installed or reused a SPTE, mark the page/folio dirty. Note, this @@ -4384,9 +4387,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, * folio dirty if KVM could locklessly make the SPTE writable. */ if (!fault->map_writable || r == RET_PF_RETRY) - kvm_release_pfn_clean(fault->pfn); + kvm_release_page_clean(fault->refcounted_page); else - kvm_release_pfn_dirty(fault->pfn); + kvm_release_page_dirty(fault->refcounted_page); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, From patchwork Thu Oct 10 18:23:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830713 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08D89206053 for ; Thu, 10 Oct 2024 18:26:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584796; cv=none; b=e1gu8oyHgkpHWlNsVULh8k3/x0Np8YyTbb2qqwHeyQaEx/PmPKGWL7Ub20lQhkbbz1c9kV6wxCTJlpOARN7Ws8HHU6yfbeJYjRmhh1t+smAReHgSaOa7npyrb3y648ge6cpkDWpnYARNcVu4fillfJWhKoEeS0GXVvpyYQJF014= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584796; c=relaxed/simple; bh=aOhxn0fpApYS19RFll+QfwBJLF7yF1QDDa2KZCpFRDw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RsU5S5YNwF12FoGxdndYHccUQBp09hiPMKSlFIb68BtFsjljlmg7wHnbUgMCz9w2UxE76IIyaabOot1Xez7U82VFB+D7nbuEvLP69I9gCYGI2138+sGP41wNeu0RgltqsO1ijHeArp6aTrN8mHWQ/nmFyn434yK9Waa9zi1gVqk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0FbCESn6; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0FbCESn6" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2e4874925so21251427b3.1 for ; Thu, 10 Oct 2024 11:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584794; x=1729189594; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=0FbCESn6n9wvwnedhP6g2o0aQ3uGz7LbaDMietMd23Zpw5H+WqY+ubJNtwyvC9nEJv FWG1G2wSjqX+D1kLYEKQ0FHw8fShSQ9I1S4nI3dc82vP/efq88a1zYT1Twk5N0zf9ScQ /53A6MT01CFUmrfBGbUUJJnvgQCVHLryQ3ee2m4qzskSNjfHdhLhZsMphQs5+zcS9Thx dQDoRI6PZz3brGVXYophhSUWOYJnUXXzqTn7D+S/653MDkAwi1Meo569GoTGnQl8jyTd gG9Yppau1gFkgT2RyuAm8WpGDImvPtucCOwbYKUzjX6gz+sxcdjTnHuWB3tqJHMrzQma e97A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584794; x=1729189594; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=Pl4pJRzNknZHDsb59k87i+ILAD20b/hb2VgMJv0VZRfM9IeBvWEDODuXzdSEX2hIEq A+vjs1vRj3vYsrmz4rWW60vKiCSj4r3vyzcMAEKSov8fStOwgIEXXZwP810W/N4/dP01 oBqkdnetcNaXvQHTrVh56OdApvUDtnOkZylB5bJJpR1hQs8T23FXchab3eLgXhi0qjBA gfX3xgs+ZjXy7orUD63EGxSH7P6Qu3zyQrcgqgjlSHMvScCpRq+Ims359Pqf6e6QkHdA U/KOKUl3ZM7+7fvitz2gSPAJY5aZT8d0Z2lwKCwakl2MMgJaN3smk68YjF1AlPShj0/X 3O/Q== X-Forwarded-Encrypted: i=1; AJvYcCVXDac8Wdfjms76PMNXo5CriNamoEZ0itGOe1tsvxpJiTjMW7cx7l+3WcrflgIbY0nbJmr06uL63nO3@vger.kernel.org X-Gm-Message-State: AOJu0YzS0PGRoHZCrt2bV9hAYgbX8whkNIQ4nx95EbXv+vF7+PgcA171 peIMdzcEYCMQyeBcMyFXhMGxy50oPs8H28RFvQihlfY8NJlFnCv9uNkOpHjsaEq45mPSzaXEdeN 4RA== X-Google-Smtp-Source: AGHT+IEjw5s9ln5Mmco0bHKHDYNNIWwRc71SIREcZZABOz/fcRYPRAU5+BeNEqASF/uZrX6+QTap8C7eoHE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6e2:3e17:1838 with SMTP id 00721157ae682-6e3221407b0mr440167b3.1.1728584793806; Thu, 10 Oct 2024 11:26:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:50 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-49-seanjc@google.com> Subject: [PATCH v13 48/85] KVM: x86/mmu: Don't mark unused faultin pages as accessed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones When finishing guest page faults, don't mark pages as accessed if KVM is resuming the guest _without_ installing a mapping, i.e. if the page isn't being used. While it's possible that marking the page accessed could avoid minor thrashing due to reclaiming a page that the guest is about to access, it's far more likely that the gfn=>pfn mapping was was invalidated, e.g. due a memslot change, or because the corresponding VMA is being modified. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f9b7e3a7370f..e14b84d2f55b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4386,7 +4386,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, * fault handler, and so KVM must (somewhat) speculatively mark the * folio dirty if KVM could locklessly make the SPTE writable. */ - if (!fault->map_writable || r == RET_PF_RETRY) + if (r == RET_PF_RETRY) + kvm_release_page_unused(fault->refcounted_page); + else if (!fault->map_writable) kvm_release_page_clean(fault->refcounted_page); else kvm_release_page_dirty(fault->refcounted_page); From patchwork Thu Oct 10 18:23:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830714 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D0CC206941 for ; Thu, 10 Oct 2024 18:26:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584798; cv=none; b=dI6EY/OtsvPnfHxpn+mFeTC8tstIgUxlygj7/ajOy1j1u9yg0ru1VCMGdbW7X7MxMta9A71m8LNGB70HfgiNRfvzsVsrxbu7hBnvFckCpfIi5eoY3HGknBjGBYg79iO8Wj4aZGPn2yQr7HJ8mls27pR8hxFa55eP5eqJfcRrXuA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584798; c=relaxed/simple; bh=7Japh8LNPjEnA4nandXvXnjfnzZjLu2hqYfo+wet/X8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GvHP/uRSbre5kyMZOu+0OE/9H3h35fms8iAxlzfujk41gcrkM4r/3cd3C2AIf2xYFbbTJ43Gh1ctH2IKNme9v3PTIJmTBQvfT0KI20cNDfSi78HziKYEsNxD2h2sTEOuL4By+dfGqnfyr1ZqxW2p6OrEU1V4/ss3J8ILRI/fQ1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xxvwKwOg; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xxvwKwOg" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7c6a9c1a9b8so928315a12.0 for ; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584796; x=1729189596; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=xxvwKwOgsGq9OHAeEg7ulS5M+xc0H7aGJvU84LLWDRvOqgU3TkZZQEPsrvINQkKuQh a0kSZ2m3MyVoMs8rT3NEB6ipyNHqf7wvINEndO31pndYJIj4JwdiGbpEIOldXIyZ4JUH m2Xwt+waP1WmBQ71gKCZlGDkYqz50bxSUom0dv824jufQ4rZn5keWyyQCE4jabUEcNHX 8IIDaN8klrGuagpek/bjhFComuRxJZEumPnJcNvLYqDKl0lJxlPsm74awlyYEz6E8Cx5 VMALXIP+FsaprgSWZvSmcNFCJoD0wV5yGOvsPvB1jxazYhU9yqzSqRKG2eEf1zu2lj+c BfPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584796; x=1729189596; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=fz4oA8I404f0whL+nXefhfJ/KZixO6+9no34P4ehebnfUBCgsI8gU56GoxwwojTq1R uhEqlZKE3lZDK+OnB7Y6AmRugzQrB/z8OBuA4H2aMgVr/staGr3TahAHclGuA6FO/32G 4a9yvgUCHDEQOho/9PQeibPtPcjjEawRy7F4WbYb9upTI2dqJib1tIkudEjGjYUxJcWK EFO+WHT0pXjeHobVJB2zkdRChp6m866gsCcrSglZ5vmSXRSLk/weHLPBLrtPfu6tnRZR 3LwDYHXCxIpd+aeCg1qqdaeSwz6risqLAMG+fFy6+VdI3qxGEKFxIpfrSZ1grNOa40Gg y60g== X-Forwarded-Encrypted: i=1; AJvYcCUxemyr4BmU1FThYNW7JQrMrpgxei6WDd4lC3qRMf7ZWwQtC7KJt4iDExAFEb0KviOUsnGReWgD3QSr@vger.kernel.org X-Gm-Message-State: AOJu0Yy31h3gReyvnk6It6jxJ1feUtdb6uCtacif0lQZCLNj8nnBBHe5 1e7tRRHRqafjK1Ab3iB1GoxBhiZ4hnmYwFjUYJzTPWHrfQvFVOdIXFEuRXkz2QqT1+hX/cTNnSi Heg== X-Google-Smtp-Source: AGHT+IGJc8hgBZq6lOovV4Dgly5ctIXEs7jDnJfEupYxVqninaLorgqTg5WNW0BPJZYnJnHdEtp0yWaf7UM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:450a:0:b0:717:a912:c302 with SMTP id 41be03b00d2f7-7ea53525e73mr49a12.1.1728584796119; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:51 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-50-seanjc@google.com> Subject: [PATCH v13 49/85] KVM: Move x86's API to release a faultin page to common KVM From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Move KVM x86's helper that "finishes" the faultin process to common KVM so that the logic can be shared across all architectures. Note, not all architectures implement a fast page fault path, but the gist of the comment applies to all architectures. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 24 ++---------------------- include/linux/kvm_host.h | 26 ++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e14b84d2f55b..5acdaf3b1007 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4370,28 +4370,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || - r == RET_PF_RETRY); - - if (!fault->refcounted_page) - return; - - /* - * If the page that KVM got from the *primary MMU* is writable, and KVM - * installed or reused a SPTE, mark the page/folio dirty. Note, this - * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if - * the GFN is write-protected. Folios can't be safely marked dirty - * outside of mmu_lock as doing so could race with writeback on the - * folio. As a result, KVM can't mark folios dirty in the fast page - * fault handler, and so KVM must (somewhat) speculatively mark the - * folio dirty if KVM could locklessly make the SPTE writable. - */ - if (r == RET_PF_RETRY) - kvm_release_page_unused(fault->refcounted_page); - else if (!fault->map_writable) - kvm_release_page_clean(fault->refcounted_page); - else - kvm_release_page_dirty(fault->refcounted_page); + kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page, + r == RET_PF_RETRY, fault->map_writable); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 504483d35197..9f7682ece4a1 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,32 @@ static inline void kvm_release_page_unused(struct page *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); +static inline void kvm_release_faultin_page(struct kvm *kvm, struct page *page, + bool unused, bool dirty) +{ + lockdep_assert_once(lockdep_is_held(&kvm->mmu_lock) || unused); + + if (!page) + return; + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (unused) + kvm_release_page_unused(page); + else if (dirty) + kvm_release_page_dirty(page); + else + kvm_release_page_clean(page); +} + kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page); From patchwork Thu Oct 10 18:23:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830715 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74E6E206E65 for ; Thu, 10 Oct 2024 18:26:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584801; cv=none; b=j7e06PoDQVovmuaJUmjA4aEVXDsSVB4SgxEBXO1BverB29fdNO/X9woBoEMFRG/zrjteo0LFzg7btQLPgfh8frUWVk3NZsg+r+7EGwbDGvqSHBKP3SAyX1pIiYuL1ccm/6vpuH6ZcDE0Gi3q+OaoHoRwjvdlFnycPguKcxI7g7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584801; c=relaxed/simple; bh=saU0+rag3xXF2/z4L9d0pyohkblMwsyLsKfPnspMwVQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aDpVRWd31IrG9cLUMqdZNu62Qdq3s8ClxiELrO0TN/z+o3mQZEFmBLowsF2X/dwKNFM9WBejWF2AxCeWrjKcg6BsQ2Cn8AARp7vPJmWITsNvq/T+6p/chddbTS06Z4MqNn/G5cydIVY8CGzy6yuoSc+psm3uASw++ZBpCG5Cjwk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n7BtqPXO; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n7BtqPXO" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e1346983c8so1278268a91.0 for ; Thu, 10 Oct 2024 11:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584800; x=1729189600; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=n7BtqPXOYdiWJ3X9kD0iG4C8w4FjCr57M/d0x7tEi5ZIPg/i+rqFcqjVa9Iu7dhTRm nS6nozj5Gx6r6aaboU/1PQ32h7s+9ziIKF4h+h1LWCrLRiUwW8rZY+NEeX29VkUMajfa kPp4Uu0Z6uHkKNXYVdvtCr+F14kxdarXnJ+OFvhdU/rGA6+vEjLsWSXDQnUXI1V6EUWB g/1qV+76T6dKYhGWKC3Y8+mwi7W/PFdRqfEjjmH1UO824LRmq6fMUl3qRoUJPp2+avqD GaAVk4tv/PFte/ebRcd7xjinGzyNHBzovubXDRuPTUDZ/OKUAHrM6Pk0WgOK4dH0HDWj 0krw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584800; x=1729189600; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=p5WaG2FYHmG+LtuGJJun2s49hlgThEDBNo6V6THJoNaS+SLhGUwF6QTXQWts2QM2dJ 6dCE4jvVJbToIWdjI99JL1QkqTIWlsk/1GP+ztj+tlgfjd2vpv4XootvM8aFIxFMwdrr /owEIT4e8krGeQlxnN7ff9tei5VwsijQCxKaGTqFa/3I+qDklUHLFs3pjd6V00Dv6wJq 5AFKl5zdz/D7A7C+aG2CYSLdYlNaDuvJuUEWCgD/3OkJ33VzBt3jsTPE/NxcfTUEE0Po PUm8A/qpuY7B9r4MZua0hDgf6NaEsikHPb1IEdjkrTJDV1ML0lo8EYN/BC8ST9tV5rid ttRA== X-Forwarded-Encrypted: i=1; AJvYcCUjUsWTnkNevsdttlN0aObOLw+7TEt7zjCUOz8hypE9RwHy3VOwKKOVF5Rk57+QvGTptp6lamXlvyBm@vger.kernel.org X-Gm-Message-State: AOJu0YxPtmR3x0ndbM1TMptJaMoARmLUVInovimzKWd2Xtk1ipnOEY9f PNXOWMef5Q5CaM/d1ga8/W28SuNOnNJwDFL3DUP2XZhRb57Owoe/BfsdWG2hYI90PQ1bhLooAnV +Ng== X-Google-Smtp-Source: AGHT+IEd4uxjZrUyEQQM0MU+jkiJqh3LQboji5/ZXtrCgOTVgrRv0yMO5ln1NKE0y5PwnoqkXxjnsp8xFDc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fe86:b0:2da:872e:9ea4 with SMTP id 98e67ed59e1d1-2e2f0d7e9d7mr54a91.3.1728584798284; Thu, 10 Oct 2024 11:26:38 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:52 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-51-seanjc@google.com> Subject: [PATCH v13 50/85] KVM: VMX: Hold mmu_lock until page is released when updating APIC access page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Hold mmu_lock across kvm_release_pfn_clean() when refreshing the APIC access page address to ensure that KVM doesn't mark a page/folio as accessed after it has been unmapped. Practically speaking marking a folio accesses is benign in this scenario, as KVM does hold a reference (it's really just marking folios dirty that is problematic), but there's no reason not to be paranoid (moving the APIC access page isn't a hot path), and no reason to be different from other mmu_notifier-protected flows in KVM. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1a4438358c5e..851be0820e04 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6832,25 +6832,22 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) return; read_lock(&vcpu->kvm->mmu_lock); - if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) { + if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); - read_unlock(&vcpu->kvm->mmu_lock); - goto out; - } + else + vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - read_unlock(&vcpu->kvm->mmu_lock); - - /* - * No need for a manual TLB flush at this point, KVM has already done a - * flush if there were SPTEs pointing at the previous page. - */ -out: /* * Do not pin apic access page in memory, the MMU notifier * will call us again if it is migrated or swapped out. */ kvm_release_pfn_clean(pfn); + + /* + * No need for a manual TLB flush at this point, KVM has already done a + * flush if there were SPTEs pointing at the previous page. + */ + read_unlock(&vcpu->kvm->mmu_lock); } void vmx_hwapic_isr_update(int max_isr) From patchwork Thu Oct 10 18:23:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830716 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69E5F206E7F for ; Thu, 10 Oct 2024 18:26:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584804; cv=none; b=q+9pjwjWbTcwWh6OWSaYnN4eSSn2zdb758IcSZj1wML8RpBOsCRqCR3GpgWmBAVDNPmMr1QjwTiTzs6e3tcMhP35fynoxXlfb0wChnW7BofEFj8vEjdG6ZypUge/L3mpsZQB8JeD8xg3jQSJ8/nfG0HwJZmhW9DJI2/jE4M2cZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584804; c=relaxed/simple; bh=ux21RVGWhz9JRdIoM/Dq30UYNcfog+x0iowAjNZ/4XU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=K3by/HccWZXHpEmxm7z5YiXAeyixVUIKYGUfjcBEtxMBOWeugOVXdALNWVPDXNF8P5LroKCPyYIooVUzQ5RWZrlLrAIFN0IuMvzpf7EfQjagu9So7fsAn0GA08c4SlbwHDhfdUx4WIzzSZBXaDIgusAe896TE+2858y6Ip9wAjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TkN0pgzM; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TkN0pgzM" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e31e5d1739so25048237b3.1 for ; Thu, 10 Oct 2024 11:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584801; x=1729189601; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=TkN0pgzMcbtXcehOT3G5lQY8rkiUBvkBcxM6gFvQBdXPdSHR+Bdw7nrzPweN+yVkaN uTb/gUN2mQ2Vg1tKjaOAVMtsZSWMylN5yuKIXvVBtG2VEPvienlnVdJRXRAHwEu0kkyK 9Ofhzp0jp3Ck9eoskAcxSdzp6PspHPEsug9npp829r5JsnbCN9OEKkWW2XlQrAVK1qDz HR2epCDdOmr5wBKAPBrQzUwIntkDyltheKd/ormx9zR0906kw75/PioCQuOEzlpeTg8X zE3iJHzZ2VWDViZQG9tcL7+JnaOlpG6gu6+8GDdZFhQ//6LMtrKiMkaqGEETsqG0K2s3 zyjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584801; x=1729189601; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=rVUZE7fZLg8CNcjgt4rEohn+GlL033RNOPH3EmIcrzR9x7DeYde02/Ny5p1a2MrQWE J6VpuaJ9xzzJt80h9OHaLPCposqRhdWqHn4iuKUX1n3LTB0D1YZUeA5O8woRWRnLXC3M gUhUbKwTpOidElSMxz0nEHULRzTZUszTvwYeARKFLNAaT6kCpOkNDQ87Y/COcwlnDzNC 1nKaSuQvYlp2+FAQ4YsWwWwSmkUycT8ctDvgwdhsL1HKB8HqtMGfbGI2P87Qdfnfh5Jq asUrmqDp+L9i7OLU45lAWaQOIyzAEkkvy8Qinea/K1+z2CHI037TOQJzXttfuoAGO3Al F0GQ== X-Forwarded-Encrypted: i=1; AJvYcCVXiu81kt8RE7hDx1HPV2d9OWXOkFbuhsNA4s8Uq3mfwYDF/whgx/k/UtqdVw7eKcoGAwi3kwdUMFLV@vger.kernel.org X-Gm-Message-State: AOJu0YwDN69n3djFha/L6Y1+7gEtG8vzG6OqMeL9M7JZw7JoPrH2dP7r Mz4w2sT9wGd8bt+QYERBwfjnnTmOQBImlvDAw+l+lqDA+NDFe7eV31y+nkzVnq7nDHMd2bi3pH7 sog== X-Google-Smtp-Source: AGHT+IGFUTZueI+Si3awRWj8jA+YGlhXMEzUcm/2nm7iA7dHF8070ZlvI1MSF0nUJUhFBhXPPz9v/uVg5Fg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2f08:b0:6e2:2c72:3abb with SMTP id 00721157ae682-6e3224db667mr816567b3.7.1728584801239; Thu, 10 Oct 2024 11:26:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:53 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-52-seanjc@google.com> Subject: [PATCH v13 51/85] KVM: VMX: Use __kvm_faultin_page() to get APIC access page/pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use __kvm_faultin_page() get the APIC access page so that KVM can precisely release the refcounted page, i.e. to remove yet another user of kvm_pfn_to_refcounted_page(). While the path isn't handling a guest page fault, the semantics are effectively the same; KVM just happens to be mapping the pfn into a VMCS field instead of a secondary MMU. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 851be0820e04..44cc25dfebba 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6790,8 +6790,10 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) struct kvm *kvm = vcpu->kvm; struct kvm_memslots *slots = kvm_memslots(kvm); struct kvm_memory_slot *slot; + struct page *refcounted_page; unsigned long mmu_seq; kvm_pfn_t pfn; + bool writable; /* Defer reload until vmcs01 is the current VMCS. */ if (is_guest_mode(vcpu)) { @@ -6827,7 +6829,7 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) * controls the APIC-access page memslot, and only deletes the memslot * if APICv is permanently inhibited, i.e. the memslot won't reappear. */ - pfn = gfn_to_pfn_memslot(slot, gfn); + pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, &writable, &refcounted_page); if (is_error_noslot_pfn(pfn)) return; @@ -6838,10 +6840,13 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); /* - * Do not pin apic access page in memory, the MMU notifier - * will call us again if it is migrated or swapped out. + * Do not pin the APIC access page in memory so that it can be freely + * migrated, the MMU notifier will call us again if it is migrated or + * swapped out. KVM backs the memslot with anonymous memory, the pfn + * should always point at a refcounted page (if the pfn is valid). */ - kvm_release_pfn_clean(pfn); + if (!WARN_ON_ONCE(!refcounted_page)) + kvm_release_page_clean(refcounted_page); /* * No need for a manual TLB flush at this point, KVM has already done a From patchwork Thu Oct 10 18:23:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830717 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F8D82071E1 for ; Thu, 10 Oct 2024 18:26:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584806; cv=none; b=Wb6hoN0K+yU0G0RTsTcweSz728IcJK4flMFUvhwWeMvqrK50G7rBPcXrmsY65GMo10Ba5vi5LNIswYtl3jaJieRcBxfiirBGyujaWIZS0Ivs+YrNw985UpAr8dgtZpY8lHiqAiNpZbwFkKbv0KcSSOvZwsatG+SXnLfOvnnMhG4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584806; c=relaxed/simple; bh=NP4IgcbBDVq5DZEeFtyiO8CdEZFJxxnULnDPMD13pKg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NfPeIrmkK+Tn1B/zP4Deht2jv2GzndEFPqUKrek8PqHa+fxG8LcL1DVTdo6WHdTCIj8zFE+9R0YjB4LexNjoZdUIJB73DyJ4uG9ve2qeOdD9uN0EElNBY8su9Jrc945IaAF9J2U8zigmaL1HxrQYfNOqwBItWjHlkspY/EK5Vf4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=g5hzo9jY; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="g5hzo9jY" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e29b4f8837so18952177b3.0 for ; Thu, 10 Oct 2024 11:26:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584803; x=1729189603; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=g5hzo9jYuMsLef5elH7BXii/Rz3O5NNAzg3BaELLMkoxekYgKvv3GlwOx0MeUswR5S u7D79PlxCqfi2Eui27GQ/wf2PDK/FtSU9HQpxYIEp6FYcUNcZ4RxRkWKsOxypmmScggd bXGvbabN+3GVR3u3Z5evpvBvV7hTapUH6bSv1xuzsGrxP42UL0bRjk5kLAIDqUd2xwc1 HxptH7WG5BO6Qq5eztg3t9rrh5agOngJ9zqfaGOzOhRlW/rHlxLQl7mVwlclH/5jU39z SiwFtCrrtukpmLM9KZ9QtNIJl9cGgY/kEqCKhzMSjtywjCUbz1pRhU34CDuRasgyTV4R S5uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584803; x=1729189603; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=HJi5nICiMY0jddgeGfXhzUs7hqIQ6S881OVyPxLh0J9KT5U+Tq19UWMLeiCr3Mfipb g5WyQfPcE5oaGxH3nRm9Ocuf9ADsDEB1r1D/cJZlhCtCSXwbp6kE7GAdT9cxz/d0VL2R lVIrh/Gtza239oNpWqHtexwUOhq7lI2geQcMrqQzAzZ/QFcBuAGx7qnYhTopWQfJGKR5 s0eRA9WxbGQbZfhPoEkVR9LTEleQYlzVkGwbm8EX+fstK7Ewm8tymUF/zJaAZArKjvnC QKYgX6kkQQtSd1mgHkBnSS3bOEkPtmAaDe9eBN5ySFrBcy7J5PGYhunGAsOmTYWHRynW 87cw== X-Forwarded-Encrypted: i=1; AJvYcCW6PNlP52c6Z/YRholYuxbTTzEnSRjr+bKs8OYYpnACSdXbX0WEh5jkc7fiW85EQoi0NXYOdWC9dYtQ@vger.kernel.org X-Gm-Message-State: AOJu0Yy969+3vlV6pFPcQOPIAbxQdI1Xv3b6KI9A5gqC++GpofZU5ma4 ZtWQGbeYbS4lsRDJIN0gWaPGlg8aH5SMBSDrJV2oOUKi0tAbvJymXDeDUno8icDJic9y7GSyNm4 dvw== X-Google-Smtp-Source: AGHT+IF0bQSeBV+k7XDKFBXYzNvC7N99ueMTlOEEH5KNk6JZHzu/00W1ndtLnT51eyBFoV94Bmz3yNyB+P4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:dd1:b0:6db:e464:addc with SMTP id 00721157ae682-6e344ce9308mr1337b3.4.1728584803335; Thu, 10 Oct 2024 11:26:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:54 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-53-seanjc@google.com> Subject: [PATCH v13 52/85] KVM: PPC: e500: Mark "struct page" dirty in kvmppc_e500_shadow_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark the underlying page as dirty in kvmppc_e500_ref_setup()'s sole caller, kvmppc_e500_shadow_map(), which will allow converting e500 to __kvm_faultin_pfn() + kvm_release_faultin_page() without having to do a weird dance between ref_setup() and shadow_map(). Opportunistically drop the redundant kvm_set_pfn_accessed(), as shadow_map() puts the page via kvm_release_pfn_clean(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index c664fdec75b1..5c2adfd19e12 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -242,7 +242,7 @@ static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); } -static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, +static inline bool kvmppc_e500_ref_setup(struct tlbe_ref *ref, struct kvm_book3e_206_tlb_entry *gtlbe, kvm_pfn_t pfn, unsigned int wimg) { @@ -252,11 +252,7 @@ static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, /* Use guest supplied MAS2_G and MAS2_E */ ref->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; - /* Mark the page accessed */ - kvm_set_pfn_accessed(pfn); - - if (tlbe_is_writable(gtlbe)) - kvm_set_pfn_dirty(pfn); + return tlbe_is_writable(gtlbe); } static inline void kvmppc_e500_ref_release(struct tlbe_ref *ref) @@ -337,6 +333,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, unsigned int wimg = 0; pgd_t *pgdir; unsigned long flags; + bool writable = false; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; @@ -490,7 +487,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, goto out; } } - kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + writable = kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + if (writable) + kvm_set_pfn_dirty(pfn); kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); From patchwork Thu Oct 10 18:23:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830718 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06A67207207 for ; Thu, 10 Oct 2024 18:26:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584807; cv=none; b=EA4mJCkFaUCgRyPfsIBc1YjfE3gIqryeXCjpQH+ZWLmfnQiRCreCRy5tEd27GT1KR1RKybKdwc5+D7WvwTVuZqPK3jI54O+WwREHan2gKQ3cLSAc5IhkCLmJSHgqnKzbO2hKWLfQH4Xi22BEWeE4Q1SoCl5LJ1veHZqAH72iRTk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584807; c=relaxed/simple; bh=kbjwflG2kz8fn2XlGwl/ZUkp2MoQm+/BMH1MpANV1Tw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PoozIGhyILTkiuy8nnO48iY8oKzUalat0ee0Ld2gSQ2zm336JYakyRyRu4JkzCljzkohGV/mbdDjT7O0po8LtdB/ykf06DC6uezM1wUZlz5cocK3dyme5t3Hu44sDUJ2nq7DgPSiDjRbsbcheZfGUg2comGhcvEmjfZ9TCaruf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lmtzsAHw; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lmtzsAHw" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e18b6cd304so1251730a91.0 for ; Thu, 10 Oct 2024 11:26:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584805; x=1729189605; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=lmtzsAHwQyNwRvm4YycGayWTaD3mMWa0KP6qhPQcjm6Xns70mqWPYJS1kLd7s22LTY /T7yWx+i0180z3+0c870icqFaPkxG/5jw4mabTqo8QyB0EcckP0wh91y7Mr5DVDEcAvZ hQO7SxtZXgjJ+5PNG6GYQ8lhMuxPsiaTv7bKN0qEUlt9rMOH5ix6Gd6u/08WvrX8BdhV sxBtnyqiiR8igp8kOk/PVonv0d4Scqtx+EB6UWy3fO105xlv1kZig9jJkbcTPFSX83n6 4g/v3dji6XZ61FVYO2EBHHk9ucMefuuWtbYJ3TSw4Isga9JWzpWIfkjpdmCzxAee0iT7 Q0eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584805; x=1729189605; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=g9A/n00GWHV+utatB8AyInR4HKO/Tj3TU4RWOSVsBxMSdLkyFpfL45MiX1XZ98qU/q Zjiw7gH00jRepFnz/33SX3OUneYI7h2dT+gW3NJHNpBMKyq+w/+m6SZ3OJzdB7uus5KL i0d4Wqn0PDBIKr8Wt5f+fxHicknU9mMuLPqr4XaSvVaZcPnq76+M0GCdhDMPaJeQwNKz f9VQF+yT9bg7zJEstHEl0QFnU1h6EG4SOVvSrCgQZ1uXfXAxo0J82HK+FTGLJyoCVyxE yDws9g+NWBwDeyWYuKsJFlCZYRebMVXJahzQP73kbcaHQPURZTvtB7Y1b9a39Lk/1XN4 VTxA== X-Forwarded-Encrypted: i=1; AJvYcCULVk1hIskDEzbldIvC66PgLgRh3IxZW3Y+LpmAZHkSYECEiR8GnkDomjWFxSbL/xwL833D64KRSeJ6@vger.kernel.org X-Gm-Message-State: AOJu0YwUxNLyXA96dB3c2CMmnxj9r9Ib6jA5w+qZO+ErItb93YFTOs6x JzeAxA6079Y+zyxkRuuoa9HGbi0TCCmGTFqeI/zzoQYi1xiRfT2b56NOye3qXfbaLac4H1uPxS2 yuw== X-Google-Smtp-Source: AGHT+IH3g+or1uGKI8zaqA7g6VjAId1uToioip8sNGZo/KyXggeZoODfUOK1q5z450FXHERnWYgwR7EwX2I= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr24a91.8.1728584805241; Thu, 10 Oct 2024 11:26:45 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:55 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-54-seanjc@google.com> Subject: [PATCH v13 53/85] KVM: PPC: e500: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages accessed before dropping mmu_lock when faulting in guest memory so that shadow_map() can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index 5c2adfd19e12..334dd96f8081 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -498,11 +498,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); out: - spin_unlock(&kvm->mmu_lock); - /* Drop refcount on page, so that mmu notifiers can clear it */ kvm_release_pfn_clean(pfn); - + spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830719 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99FB6207A03 for ; Thu, 10 Oct 2024 18:26:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584810; cv=none; b=g9kjVeoAtLGOcPBaj1HxN3eNb//WmaDef1wAAswWsK8PIwx+j28oQr/f0CZ1XM2OnunOoBque6R0Py3/pgwi3ta4HjKtjfSN66fWKAPTkcZLlgtQ7Po0ME1TMez5sZ/S3hYkwFVCDMEyjkKlKwe7PCMICy0sbtuaMe0pqYlQUYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584810; c=relaxed/simple; bh=ZWIlv2h6Cx1yrF7GGy3UqCKI1dfxdLpdeKJSlsj4QNk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Gnsa+35qelRudkHPnPBch6YqQ8pZjMF8Ww6MZH0V0BAEhHUzAQIAmM8AB0TGYVkys8fz/Kx6LAKu1nlcdfv/drJMOOR8jY36zwlFhymJt+zA+u6dbv2DkXZzSzr8LBeb8HGWmCrrTFPM8CBsl73gQrs1pwXX0sXN8+6UmDWoKKc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WnVYZ++e; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WnVYZ++e" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e28fdb4f35fso2111617276.0 for ; Thu, 10 Oct 2024 11:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584807; x=1729189607; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=WnVYZ++eloUHmaNH+PMyPBVaTNNQV7PpyzwKfQNf8E0TWCFXkHdgRSE8m9QZdbFqgl LUO9GmnO14pJlNUz/a9YQDP4sRdsmqE0I77QJPEJQLiNtELH3T55kxgvZXX+ZS04dduF HOubAFdITv4Os+cf8tlrjsJ28Wj+yEbPvdLMEALhRVl3zs/sWfw9mSXKeFLyukSkfF0t s159ZU6SoLfXQ7k/TJ51qYCX6bhgSQvRTMspjJT5LynWuHOAFbYEj7AgiVCpYczdtFzC uNelGCXQnKo8zvzdO7U5pHqlDSTPwijrS7Od2lq6i7bZd6sgWjEtE+f9igLiaozAMB78 DGDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584807; x=1729189607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=mrWdRtwNBUQF2Yr+aEVc22+8go8vpz6kToeELxawW3SKGqevMh+BAD9yIpS+Yywp+J 6KxXq09GfsB753nKod7ZAfWG2fQBHkSkmD4q8sjEU9DK7x4rZsvuR7Pdm/GmXiGp3Utc 7fLkEft16zt+lLF3sf5XNeNG1i6gJlkMRw2A682iq6sSeye4dByvRSS81i8J9esEl19Q LMmlipRmzAnkGaUByPTqQfDf+vry4VSj+VMMMLgo7ivDJ8qBYQa/5KXL82tQVL8VkNaV he2HoL8jSJXJvPcUnvmvfEXPLrIKU4rfuoFdaOMH4gv0xxam3loZg6iqh8CFc4+Sa3lT G+hg== X-Forwarded-Encrypted: i=1; AJvYcCXV1MG0fdu3EOXfaivT37Mwns5cbTakOur+hweV61mvEIKUzcYM2Ppmopgp/p/eSghvCQpnrlzej1pl@vger.kernel.org X-Gm-Message-State: AOJu0YzDUIYljype0lkC2+uR1i0ggcQjCDmarHdTLohdFtbMqXTC31k+ LYlCDFzg0tm8FzryR+hN1gG7o7HIk2nQV8KargSLjOydcgjf4ujSukb/ySNbxu8O6E81zxT3dcm KMg== X-Google-Smtp-Source: AGHT+IHQvsj0FcPHwm2v4Ks73fb0XY1uO3QmFFCKoUyfp7OKRYByAKtc8uXQ4anZ8QaSR/IH5oavOTtruEI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:b86:0:b0:e0e:8b26:484e with SMTP id 3f1490d57ef6-e28fe516b5amr5188276.8.1728584807403; Thu, 10 Oct 2024 11:26:47 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:56 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-55-seanjc@google.com> Subject: [PATCH v13 54/85] KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert PPC e500 to use __kvm_faultin_pfn()+kvm_release_faultin_page(), and continue the inexorable march towards the demise of kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index 334dd96f8081..e5a145b578a4 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -322,6 +322,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, { struct kvm_memory_slot *slot; unsigned long pfn = 0; /* silence GCC warning */ + struct page *page = NULL; unsigned long hva; int pfnmap = 0; int tsize = BOOK3E_PAGESZ_4K; @@ -443,7 +444,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, if (likely(!pfnmap)) { tsize_pages = 1UL << (tsize + 10 - PAGE_SHIFT); - pfn = gfn_to_pfn_memslot(slot, gfn); + pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, NULL, &page); if (is_error_noslot_pfn(pfn)) { if (printk_ratelimit()) pr_err("%s: real page not found for gfn %lx\n", @@ -488,8 +489,6 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, } } writable = kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); - if (writable) - kvm_set_pfn_dirty(pfn); kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); @@ -498,8 +497,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); out: - /* Drop refcount on page, so that mmu notifiers can clear it */ - kvm_release_pfn_clean(pfn); + kvm_release_faultin_page(kvm, page, !!ret, writable); spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:23:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830720 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B25AE208202 for ; Thu, 10 Oct 2024 18:26:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584812; cv=none; b=Eua3LGfl1jCTw7tN+bZkhb8oH6gADm+8TpLrV136ZGThbnKdGlSKqMIGwY5mXmLCGl/IEfzG0TLzwwRd9YD3Z/s3j8wz8JKPqCpQp5N48sSDezqrGVMUEBgHAajgdp4NZfXMPKpwd49uY7oUm3VlUdI6ntYHL1yvLGVto028WvA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584812; c=relaxed/simple; bh=lNDyXtKPmFS6sOILzqWSFd9L+nyXpdsd6JebnDHI5l8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lNVa+UG+95SR8Yx146PwdYufAjPRfhWvR/1/FeX6ZAoCxQ221uZea//b9LdRuCFy9FlKcMRsQTPaOmTEsidyyhVJg1wASpN0OXUUHI4t67t5iaYn/zz2D2pV+1dUZVemauGsiGCOs16XFixP28rI0e7fV+SZswSkhFdrLPYRI5M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dg9gcvn1; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dg9gcvn1" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589450b3a.2 for ; Thu, 10 Oct 2024 11:26:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584810; x=1729189610; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=dg9gcvn1EbsSMEFFWFfwFAC3/SO7ovIMxsDf+/JNqUPrNu+sX4BsLpvzKD94FQExHx ct/k5mewMmav7ktqHjn4oB9I4zkkRwXyn5w0tIoApPyDKIiQH7p6tHkpXFCPJTtIseQ3 VoNYFKhdpcHf9CbYy26zVGFEvyT2J19mE20vTiQrjL6CB9Vq2Zh4KkGD82MaCrPd6y9W UUtvrC92Pn2SaMErlsyPwQ0Db0yjSoKXkYH0TPgJINJmnOxNVVxCzUM3KVkk776W72n9 gwT6JZjKyO2gcBCYOp3KcN9F1c/eMnDgfxq3VKPhPILvMxpVO8cOkwe7VhMm+/YDQ2ay oHcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584810; x=1729189610; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=BAMl6QU7MaEBrh3Jrjhc34nOcyg2KBCDCMM1I6xnPTE6y0Q+MmTwwS0jHSEfbvffo8 dwj4qf1Nbx3R2o8Vr9Xmz+9a1C64XKSokGkAEVb5kYbl+LH3nwI5ShsZKZ7sRxEGPtfC UdR8Y5vcsIy/RQ/KRCOmskbq+fG55ZEpajGPvbduAo3MB3SpRtzsq5hTmgp8Zkq7/Ai1 h04xfJ4BI63zGdyjgMjQgkJ0p5hNIXLuSdPGDdIKYJm8BjS1Tc+dyLYSOoR7DSXs1ixG lrO+l7g0OjxyyaovPZCi7IaYecEryJu0jPZ6R2oeyyM59tyPCPk8YIKoB7xOVb7VtbYi m8Eg== X-Forwarded-Encrypted: i=1; AJvYcCXYyVv5mZaAPj9d194amA9OiqjwGcQNKKUvE8vdGNfosUjr0eLd0PKopwEI8O5HvMDTAk8AVS3RNCCA@vger.kernel.org X-Gm-Message-State: AOJu0YwbxAyg+rryxLc6djTfEily0Pe5EYvTk9CfarhamidCDno/Bc9d Qj/zSCIwzAIvBsRkcUEztJDc+c9+m7dDL7W7WKlX38QIeHwxpc5QoUDRTuM0S6l//LYUAgrkDyB /cw== X-Google-Smtp-Source: AGHT+IEO1ECBC98fGvB+GRqYHAQaBdxJcqvY9DmP3QI4gZm0KdOyij1fLmFl0bPXgfvra8JJdQe8JNFxBHM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9199:b0:71e:1e8:e337 with SMTP id d2e1a72fcca58-71e1dbe467fmr8496b3a.4.1728584809106; Thu, 10 Oct 2024 11:26:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:57 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-56-seanjc@google.com> Subject: [PATCH v13 55/85] KVM: arm64: Mark "struct page" pfns accessed/dirty before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages/folios accessed+dirty prior to dropping mmu_lock, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). While scary sounding, practically speaking the worst case scenario is that KVM would trigger this WARN in filemap_unaccount_folio(): /* * At this point folio must be either written or cleaned by * truncate. Dirty folio here signals a bug and loss of * unwritten data - on ordinary filesystems. * * But it's harmless on in-memory filesystems like tmpfs; and can * occur when a driver which did get_user_pages() sets page dirty * before putting it, while the inode is being finally evicted. * * Below fixes dirty accounting after removing the folio entirely * but leaves the dirty flag set: it has no effect for truncated * folio and anyway will be cleared before returning folio to * buddy allocator. */ if (WARN_ON_ONCE(folio_test_dirty(folio) && mapping_can_writeback(mapping))) folio_account_cleaned(folio, inode_to_wb(mapping->host)); KVM won't actually write memory because the stage-2 mappings are protected by the mmu_notifier, i.e. there is no risk of loss of data, even if the VM were backed by memory that needs writeback. See the link below for additional details. This will also allow converting arm64 to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index dd221587fcca..ecc6c2b56c43 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1692,15 +1692,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } out_unlock: + if (writable && !ret) + kvm_release_pfn_dirty(pfn); + else + kvm_release_pfn_clean(pfn); + read_unlock(&kvm->mmu_lock); /* Mark the page dirty only if the fault is handled successfully */ - if (writable && !ret) { - kvm_set_pfn_dirty(pfn); + if (writable && !ret) mark_page_dirty_in_slot(kvm, memslot, gfn); - } - kvm_release_pfn_clean(pfn); return ret != -EAGAIN ? ret : 0; } From patchwork Thu Oct 10 18:23:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830721 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23F29208228 for ; Thu, 10 Oct 2024 18:26:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584814; cv=none; b=BYr/LDwoPOPtCVXTdCpudjXZAKBS8Pk13WKtkgZnLzXZIyYyg4QFAtQO2uyDIw+CJAsD4A2T/MXQxQPsb7qgxL63UCVwacQTT34W5IFdbKH/BVhpbVe3GxiW5lEKWl4Wop6BsObH8RHRdWmOCjGgKCWcTUzS0F11Q2RjdkxZ8+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584814; c=relaxed/simple; bh=XrFS4rtFa/FgrThsq3EMvRUnjE+/w+4Wbq1cV5vxoHg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pssAHLKH5Gvd/aex1xoaB9ao0Y4QTy3KX7Gzl+3Xi2QeH3lPuLzSSVa7wQF1HhD8l8wNq0oAHIhWo/wdpWPloJ9cgjx3XN85ZKe/Rour6g7wX9dQC5nseFJqKQOrsv8FkXuvJKhXe00cXkDCgD02w+BQpD1QecYz4bS60NLXdKY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cMRKCJL2; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cMRKCJL2" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2c3338a9dso1524943a91.0 for ; Thu, 10 Oct 2024 11:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584812; x=1729189612; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=cMRKCJL2KTT4vd8DOQ2od82kSNUiso8aIqKq07Yvp3S6byddo0DPcg2b8Jr9ogLGAo BNTKkHdxlFw0l/CcZ5axwvEmJkR/6UB71Pb7jQUOYe4rhWAIBZzwBX4HH87G3XGLEtqO sSChKOxVtrZ3EpUVsVzWJqNdqZbLXzbA7bJvOxpoDZx9KYx9Xz26RYucJeLlSpK0us3L e20ClV/b8sQtSPdFHLtStwrnxtJUeQ5hHPWY0AHVXja2oHzcD5JA+7iFwMVByjbOQ2jD oC5gACr1JoJhr0hdDFfDr1ptb4PWqWzd1vvHuvzjVquTsBV+1uKxCjicP33d2mq+QoPY sXfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584812; x=1729189612; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=vXKPRcbNYC8nrQKbEIjI+nK0Nv+DK082ycXirhuG7y6Fk/+lXvLM4OGJ6RW/ZI9udV URnow9Yx9SFmou1zENyIAbqSNxTSyhmPatviD1r4H+cDZvhSz3QjzQp2WlBJgQ/6P6XB +W/vXziXvAdgtXHQ4lbmmvpvLbM+vDyll7eXrvxFFxVYR04n2S7y7Ojvm/CfD0rYyrxl SUYWxpvHixlXI64HjJu9KdMxQOj1/g1KrKuikj73qn6k8nU1AduBUBKWgW/1QqwqFNmt dmbac/xlEEhhBbKdejgkprtIbOHNMPsJ3xrUFxGub86rQKQkjCw2arKLghCTyKNJ1RQm 7r5w== X-Forwarded-Encrypted: i=1; AJvYcCUAqky0UmuovZS7kXrNy7RJ8YLxT4oySfyUw5+3dCg8ARSQxhj/2J8X7oXd/mFXDjUCJQTDJaZrJHT1@vger.kernel.org X-Gm-Message-State: AOJu0YyGYeN9+9fqAk5+vvi3sDIwoYZGcc0frN4X/WldNhaXMo+J9406 s2DkfJIkD6zKn2H7CHWATFcxWE3EVFEp0plBN18/TRkCAPDFDgfAKU7R3ucjLeZSGyhQsrm7xYt w/Q== X-Google-Smtp-Source: AGHT+IFNqFi+zU38KeaNCnXXdZA3xW66mqPsBP1LkkibZO//JxtU2HWKMaWXgt9tHH1pSDOFY+ZWY6Gk4GE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:4f4b:b0:2d8:bf47:947c with SMTP id 98e67ed59e1d1-2e2f0c456f8mr35a91.3.1728584812033; Thu, 10 Oct 2024 11:26:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:58 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-57-seanjc@google.com> Subject: [PATCH v13 56/85] KVM: arm64: Use __kvm_faultin_pfn() to handle memory aborts From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert arm64 to use __kvm_faultin_pfn()+kvm_release_faultin_page(). Three down, six to go. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index ecc6c2b56c43..4054356c9712 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1439,6 +1439,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, long vma_pagesize, fault_granule; enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; + struct page *page; if (fault_is_perm) fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu); @@ -1560,7 +1561,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or __gfn_to_pfn_memslot() become stale prior to + * vma_lookup() or __kvm_faultin_pfn() become stale prior to * acquiring kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -1569,8 +1570,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmu_seq = vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable); + pfn = __kvm_faultin_pfn(memslot, gfn, write_fault ? FOLL_WRITE : 0, + &writable, &page); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; @@ -1583,7 +1584,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * If the page was identified as device early by looking at * the VMA flags, vma_pagesize is already representing the * largest quantity we can map. If instead it was mapped - * via gfn_to_pfn_prot(), vma_pagesize is set to PAGE_SIZE + * via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE * and must not be upgraded. * * In both cases, we don't let transparent_hugepage_adjust() @@ -1692,11 +1693,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } out_unlock: - if (writable && !ret) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, !!ret, writable); read_unlock(&kvm->mmu_lock); /* Mark the page dirty only if the fault is handled successfully */ From patchwork Thu Oct 10 18:23:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830722 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F09C6208962 for ; Thu, 10 Oct 2024 18:26:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584816; cv=none; b=bCTyKG3Jxq1KyX2LVkJXh5kE5z1g+XnwPCUyKvu1Tx9KjTALc7G/jJEyctVxmOPliECw6+KO7Ad2m3YdcI4ftMWMMsgCrEL7Hkq9Y3swScNL4x9kNOQLcYUjAZeuKsNUUlSdw7ldBV7X7AcY8NAWiLy0Llm5snQmzt9K7dmJWj0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584816; c=relaxed/simple; bh=tUWxw/XFi5Jjw4MLGoQx4yzkrGK2c3j8OG0Ux9/pXvk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YSEZQ3TeJUwhlIk89XYp1FnJ+Yv7obOqEhsShzO1A1KA/I6kphGcxSLN1SySVHxfUzXCfM1UteHzPgshqNL4IhwM/PuvSausmVpF8gL2H690xbP8jv5KBrfiP9h34Mu5Z25BG3711/EKSqLKwMIYzOkjbwK5kx9DHUpXYzZ0Msw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XJ5G74vA; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XJ5G74vA" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2e4874925so21257147b3.1 for ; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584814; x=1729189614; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=XJ5G74vA5iAj8/LHLvdWEOyxWm1sfVqNq+jfgoUcBvu61bKrqTg0Rh5GOJLcTJf+Aq TL7FokBfV0aZSwz/KOejArh1F3p2bP0Yd44s9m4WX/lwCfKaHYxqgq5J8q0UCN1NNYda BH05WWr+Tayqa620VrfEhx9mjQLJkXQDcrwZGDsqxDGFZyd04Umi094470nxsXBb69Ur klYor8P+lwqKm5AFy3yGOCs8lPPDqjyRxQmoDgihuPO+cs/d0BqSVSdu7FrlZ3DPLEyd IdLqIfUnIN/c1Gy68qnUPzDojMCUIhUQknpkbfu0TTJvkhlb2kRKb2OlFqBihneknzCP bOKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584814; x=1729189614; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=KrOVkuAfnGgkOr4iPgYutZ5PyBm8h/ZSf05q33Zp6V19yH8mJUCX063/Q36w93gxiy yeGGCDJVbvC34zEC1lA5zwWv/ULwvQrAaCkbkB40zJeyKAtHNgZtmJ6rCQEl174pu9yu R1Db4E/OvL2q4/eaACWg8xmUCjEXG0iXbcrb0H3+3axE6U7BCPyyIFZxrtQLSoxURSFp E9OHQVS1UcBcNY8bsPkNKYpruYFSxvl9pAObrmi9n5BoQSQyxj9057sRShNvSbfbRkFg OtqTkul774SLcIU1Dqu++e1nsJnFOazVn8Q9mH1yP9wqc8Aja04pklIjLfh90TszaLoK Gpig== X-Forwarded-Encrypted: i=1; AJvYcCWcLHgRqmT60A+6/fsME82nsrAf2m6OP7k433CUBHNRH2zgJ1S3m099xvFdqqEfTiKe0f7rY6sdZUa8@vger.kernel.org X-Gm-Message-State: AOJu0YxAgOAja5zFTfg9EVv8Q9dTYJZdgRBzqNVNSh4ueHStM2G7ccWc gRwkWPpHjbfPJMD3vVKqVzOr6pOhdpKPCh/lXhh657DHDUUTtX4imDXh3Q+J8ww187fF8NhrGMK 4Ew== X-Google-Smtp-Source: AGHT+IFTz5/NgoL1lG2XSkf5pquU6CJl4M8l3x57mh4U7FbzzS84rW+oHIk+OXb87gKu92fZB4cgNqwOitE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:46c3:b0:6e3:21cf:a67f with SMTP id 00721157ae682-6e32242fb5emr1006557b3.7.1728584814138; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:59 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-58-seanjc@google.com> Subject: [PATCH v13 57/85] KVM: RISC-V: Mark "struct page" pfns dirty iff a stage-2 PTE is installed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Don't mark pages dirty if KVM bails from the page fault handler without installing a stage-2 mapping, i.e. if the page is guaranteed to not be written by the guest. In addition to being a (very) minor fix, this paves the way for converting RISC-V to use kvm_release_faultin_page(). Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index b63650f9b966..06aa5a0d056d 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -669,7 +669,6 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, goto out_unlock; if (writable) { - kvm_set_pfn_dirty(hfn); mark_page_dirty(kvm, gfn); ret = gstage_map_page(kvm, pcache, gpa, hfn << PAGE_SHIFT, vma_pagesize, false, true); @@ -682,6 +681,9 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); out_unlock: + if ((!ret || ret == -EEXIST) && writable) + kvm_set_pfn_dirty(hfn); + spin_unlock(&kvm->mmu_lock); kvm_set_pfn_accessed(hfn); kvm_release_pfn_clean(hfn); From patchwork Thu Oct 10 18:24:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830723 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 137E2208995 for ; Thu, 10 Oct 2024 18:26:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584818; cv=none; b=jMg2VkkZm1QO0zi0EJnY9fGfAfyMN30To6DjmHJpnHF9Wftohg+DZM72blYhY+BARnf+IqR0dsi7x496O8MFwRUkEIxiV/IzjX3BaEmfQoZhixR+VLZShBG95v2taJToIbDusfSnnXeoC8OBDtz9dR/aQau7COf3H38pwzcWrhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584818; c=relaxed/simple; bh=XU8sCtvYc/Uw+Iuw1o18evVVUc/CvxyvropaE1RipJE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LZIEbUQ1KYh4r0a11nHpZHWrjGkgZsIdVBxSb36/E9mJnx9XKpPuOo1OnPA/yf77lVdTDKLqy4t0toXPhBiQG1nG//H/8eRpSjZGUvIGS/apSsZe/HI1VJBshhomeM2FbiVjegFSKqJCGlB2sktct9pACBXwEv2RTAkGk4ZJt50= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=W+oxKjMU; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="W+oxKjMU" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e02b5792baaso1676936276.2 for ; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584816; x=1729189616; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=W+oxKjMUfyh+EqY/9j8uTawVGS+361nMyI2WTh0heIuHPPflaTW6hcERh//ZIPGINt hlypNXBr6qobPwsVdXmHX6BEqbZGa3FbwKmQ5Sd8OPggbYFVJFGhHU7MM+e8FbNNd40V 2sP5QAjfViExYotUYGsZOg6HLHcp9O01YQvah7hUFc9sAA6kvJYFAZeN1//luN4wfFDx PzSOdMsfcFKPsHiAosOR5+PBT3aMMCz3e61LQPme5dH6VbZADVeqVvCgX4TieT8OQL0u n2O7FIS+uh6/HFTeEZYPKIRdCh/EQxAPOAwyFhbsAZf74548oWBqEZcLrf0aGcNxzTXf 6CeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584816; x=1729189616; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=u6HxkxMokFmzStpo/VjJAS1c+JrRhVGBJs8biQAaNOdVapIG8YCcR4NaPRQ9vbfFPK KheE5cLaodE8ZuMBqE42DA844H+a1S38ksMKkzw6wJMoK5YK/xnvGV9SAzDjtHdz+np8 Ji5ohZZoT5DuPoQwfAcCax+Y1uY4/4n+lEbUXxzcg+fJ//QNQLpHy8nW7G5CwMZi6rqp Dx95/zz1HLY1E+cQ8bupuSRAV7zFQeICSgVlf4TCwxkh61YgJsg2GpVjS7sswrdJkcry t1MoJEBWpor7BTgAqr+0b82XNyXc7oEAyyKvldFBCsmolv3Mz3o1Yb0wQonbfwfqH5hJ yR4g== X-Forwarded-Encrypted: i=1; AJvYcCXoI2sNyRLZu0WGknFmmM2DgRmgmd19W7y/ozYYRof9n6h8udEW4q4C+5E9ZrTpCDlt1oKkQE0joZ44@vger.kernel.org X-Gm-Message-State: AOJu0YyVo5nPaowmKdcAP041z7A6TdJeFPwTqnnSrL7mUjUfmCsyPbUb wxXlXs1qHmNvqfxUWvSzXWOxhMFtCBd/qvZSYmtP4JN1sZWOgMpJ3N+32dMy0geHqtcgEOkQ291 4og== X-Google-Smtp-Source: AGHT+IER2ThMFqRrPlrH2YqE2Mm6yZZ3YtFM0VnOojG5BgkcM2HHrRnMjpl0ZvMfhTqewlD6nF2BR0rNKwA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:c45:0:b0:e0b:f69b:da30 with SMTP id 3f1490d57ef6-e28fe41d0e6mr89144276.9.1728584816073; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:00 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-59-seanjc@google.com> Subject: [PATCH v13 58/85] KVM: RISC-V: Mark "struct page" pfns accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages accessed before dropping mmu_lock when faulting in guest memory so that RISC-V can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy (see the link below). Do both under mmu_lock to minimize the chances of doing the wrong thing in the future. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 06aa5a0d056d..2e9aee518142 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -682,11 +682,11 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, out_unlock: if ((!ret || ret == -EEXIST) && writable) - kvm_set_pfn_dirty(hfn); + kvm_release_pfn_dirty(hfn); + else + kvm_release_pfn_clean(hfn); spin_unlock(&kvm->mmu_lock); - kvm_set_pfn_accessed(hfn); - kvm_release_pfn_clean(hfn); return ret; } From patchwork Thu Oct 10 18:24:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830724 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6F621E3DF6 for ; Thu, 10 Oct 2024 18:26:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584820; cv=none; b=MvQ+IMlEa74Ev7nxtndgWFWDs2Kk7ercJ51jhBMiuSX5NMcxGYoe7J+paXndF7uSMqDB4lA/gonZ2NxGRBfVabywArmSSG2v5uT7z0bTrtOQoveDLHaRvU2mkx4xf/6a5/B1GpyzhNnYABB6OUqA6EHAnhIb5ONd9F3zOojOu6U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584820; c=relaxed/simple; bh=h8Ty3DwV1uJmCoppjwTeaFXY32gukNMrofdCY+cnYhk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=t1Odvhv7V3WbwTIy1+1XLEE8+CK8b6rEmqVxDrGKjn4oHbUF3OQwfU7UdEak6XUBhq77tUzTtaySTtH/Cv5CASNL+VCCsPcQtckHgc0y1LU4MmpZWtUTA8J/DNbUfdYoPlTnrsagnImqeT5gEoeKyd36cV2anGUoU931Ma3HW4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JNfpTpqi; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JNfpTpqi" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-206f9b1bc52so11512335ad.0 for ; Thu, 10 Oct 2024 11:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584818; x=1729189618; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=JNfpTpqiBYBKFLYDya+q/+VqUsSM16GJP14qVcVJBPKXZq4oW86J7IKlLAOiNob6Px IEuSrspqaxeP+LwqTSlCTpUpi94WLf5XXZW/QoJCLOugk9sV3+x2J4yYTdzkt3tvnLQx nE5oE/+OqR1Yuki9kQowd4XCBCzHiNhnKLTalavt1s+dTpNB4LgxIrPIUAPsx+IFSoQe Dxl8Q0gJ0H3By5DV9zN8gz91BDltC3iw79l9V7QEzDzNavm6Mkl165TAaQtIf5TNC1uq y5ayftOuatk9cCD2wXS5Rbnu24zhxUq7JElq0/cz+LsUk+p71mBe6JOzZybeoA2rg/hT EvgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584818; x=1729189618; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=ef1hppGVCRjDWe62L8r/7C3TBxWezpgNZ2I7eUrRC3wFvJhzcQDKXGrWtzD8nKaOun ImazH++YRuEYXJheHn2Qvw5fqL+Nyit7612R84s2oKDaQbHiIzqSi/Ho4H+TuQ+OFaHY XtE7f8QkPKHuG6EIJtgmPuGUSm3vZ/1j3sOb6cK/udCnQf/NND3/xIGypBEL0cHTOC6d nz62iw6C/BW8ZtlR+Ntr0cN1NeIgI7dC0he74Hqk6nIvkDBBThiIzVvIVm4As9WfTDBU kdBfNaMhLIizE8x92/La4gnL63zlKNYXfPOtlPnUqay4+8brQpB/K1sA0ncr+gfYXyqi TwlQ== X-Forwarded-Encrypted: i=1; AJvYcCU0zYsrEIFmpBmzjXo8N/vdnEhwwfQvZln3PVRAfchnZ6qnnOnewQhEDeIk/zZpiCk4SKiyUQ2vi7Ys@vger.kernel.org X-Gm-Message-State: AOJu0YxIHXnhKQ6rbZp6seBkAtQK64qaOIxS8lzBA8cjykTei7cl9l8Q 1tFVqAUYBpYffnoo0Wlp4Hu3CAR0ZxL7QkwF1PzK3udYdr6li8iZWS6Kh2mAp/MGeBKb4NvMafg YCQ== X-Google-Smtp-Source: AGHT+IHVZkXcNH8eDb3GIQubykYaXKWHdrpGe/zHTLQHIjPMwQpAN0ra/QncFunLfkL/hgYZuPxk0qgLkH8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2343:b0:20c:747d:ec15 with SMTP id d9443c01a7336-20c747df618mr290175ad.1.1728584817934; Thu, 10 Oct 2024 11:26:57 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:01 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-60-seanjc@google.com> Subject: [PATCH v13 59/85] KVM: RISC-V: Use kvm_faultin_pfn() when mapping pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert RISC-V to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Opportunisticaly fix a s/priort/prior typo in the related comment. Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 2e9aee518142..e11ad1b616f3 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -601,6 +601,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, bool logging = (memslot->dirty_bitmap && !(memslot->flags & KVM_MEM_READONLY)) ? true : false; unsigned long vma_pagesize, mmu_seq; + struct page *page; /* We need minimum second+third level pages */ ret = kvm_mmu_topup_memory_cache(pcache, gstage_pgd_levels); @@ -631,7 +632,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or gfn_to_pfn_prot() become stale priort to acquiring + * vma_lookup() or __kvm_faultin_pfn() become stale prior to acquiring * kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -647,7 +648,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, return -EFAULT; } - hfn = gfn_to_pfn_prot(kvm, gfn, is_write, &writable); + hfn = kvm_faultin_pfn(vcpu, gfn, is_write, &writable, &page); if (hfn == KVM_PFN_ERR_HWPOISON) { send_sig_mceerr(BUS_MCEERR_AR, (void __user *)hva, vma_pageshift, current); @@ -681,11 +682,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); out_unlock: - if ((!ret || ret == -EEXIST) && writable) - kvm_release_pfn_dirty(hfn); - else - kvm_release_pfn_clean(hfn); - + kvm_release_faultin_page(kvm, page, ret && ret != -EEXIST, writable); spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:24:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830725 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2C08209F31 for ; Thu, 10 Oct 2024 18:27:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584822; cv=none; b=oKyM32EnEymfT4TjDto11aGOjxWX8OChJIBogEt7E3ZWDLfenQxZLOEngDN2xY8Roju5HZNMyLWY4ruZolloA84HvwjhabnPp3nwKHfQJyarIjO388Lqckz489riFrD296sVhydgsFw/0b5tP6keHVhzJy3L43Q41N6BWnHgiKo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584822; c=relaxed/simple; bh=dTkbtmFokyvOqUKSDyYRNtMSjRIKbeJnigSrLVN5f5s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TNkw4ukEQjNlXWze98zX2nAhmSxmm1U0VJ44KyCJ5kbB6xHW8qz4PFwpioZwUcz+H9oghoBF9UInIGYC/zsyA6rszZCvHsX4NQFkeYtFRoVZq5RSRD0nDaX+NquY0OAjDTHcS4B+UPU9SKQN2D9NNhFHu/aDxhtrsxExnD8ue3A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=E4/Nn2nf; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="E4/Nn2nf" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e29088a6942so2105694276.0 for ; Thu, 10 Oct 2024 11:27:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584820; x=1729189620; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=E4/Nn2nftJfFfMfz7XGdS1N64U1kTObHmK9jXjNfOx0RjrtgUP73dR5xAlk3+GMtGB VyKZy9lVNuNg6pbjDWdFmORwCovT3DjZBTB+WZ837kHeHvK2326FSbxxs8lJBmA+cqfJ AzLzFEr8HhjDktjfMFXgEDl8CY1fwEV6eCqYsTZqVWyTx1h236L085DUqUUmeS6vrvHB GSntK4yQbkZEuyozHwXiNZd4wO+6Z3lbwa+JRbDcIw4FDtwcrBqi9nXBbIffNp9qA3U0 zaS4plFaV8cXO7/+pyiIv0T2C+lmVigBUlVA+qtqRH+dtAzYEjWbWCUNPXbPKT5e2PWb 3B2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584820; x=1729189620; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=IdmjtuDulQNoUD3kN6kpV5kjbEQT+w1aFq8GAtvSg44z7By+7GFy/E0HIWOaRxpn0l 63Wikno21bFrFuNAdjubObvQ5i9rJu/Dc2mxBaXZ3E5nL5IMx6iOWqwkPgaRptadM8Ft f20ObUqb3oPphhy6ofQrwXbP2G61hsCb4KYLIXXMzMqjfi/IT5iCtaDsE/BqljxXO94f guE97tkRrgjIvSDRm0KyepdcbA9IqDuNI6DORS6zcnLI/jwXAkl4jxExsu7Rfe8n2RRU IuYURsgWkYrQ24sHHBEi1ZOnsSxiD45GIvvVFPO1crTgk5QL8itWVOL2V3793zszGOK9 HTHw== X-Forwarded-Encrypted: i=1; AJvYcCUriKztYZJ8QcAYEyJcj3+z8VnGGAZ1z9AtYGppw98Ia5YuKTEQQ0JB03oxK+KR8JRcDOdVONJWqUkV@vger.kernel.org X-Gm-Message-State: AOJu0Ywzz7m3jlqx+s17QfF2Qazvp996sO0taUS9etG2+wzlOqm6LYZw +OJBbRzRNoHhPrYDGFoIYLqRXPItkAblM8gHEtqge6oPyIdf1J1csxecIRh4HWqL+eDYIpg3Oon kjg== X-Google-Smtp-Source: AGHT+IGRQvxFZI4ySDHPZ8aV1dJi8vCYkQEt4dLiT8ndc3AvxflLIddldJBCv79vdNFr7/DaBU5iasgDdOk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ce11:0:b0:e28:fb96:d0f4 with SMTP id 3f1490d57ef6-e28fe33505emr40781276.2.1728584819828; Thu, 10 Oct 2024 11:26:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:02 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-61-seanjc@google.com> Subject: [PATCH v13 60/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s HV From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Replace Book3s HV's homebrewed fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_hv.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 2f1d58984b41..f305395cf26e 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -603,27 +603,10 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok = writing; hva = gfn_to_hva_memslot(memslot, gfn); - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ - if (get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - write_ok = true; - } else { - /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page = NULL; - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - if (PageReserved(page)) - page = NULL; - } - } + pfn = __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &write_ok, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; /* * Read the PTE from the process' radix tree and use that From patchwork Thu Oct 10 18:24:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830726 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3DF2209F54 for ; Thu, 10 Oct 2024 18:27:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584825; cv=none; b=hF7ynsEjS7gw7/AxRGvEbftbnYUltI8HEP8qCgK9QVMlJPdagPu+r16eAEcBrGedy2ilfgsbU12HTn4rwWgIDFTweVgCip1hUyFlCFvGFYSDup4V5BCWao2S4ENReidpdw3CXEb0jIcdAkfzmvGVmPJJS+C6yRo67UXkl71Nwnc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584825; c=relaxed/simple; bh=fHLEjIjpsClQjytVJZuAJLLMS5Q01wz8isn/MOvrB+A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RmMdFNWpqgdngaa24JM32ns0TeK7qubmgj1Y3rGw31aq/8+yJh9GsH8jf0tKYEBfSWbDJ6WJm1v9xlTC5R8N00pJpMgsQ1h9aYI4WSwYch/4A4N9JwbeXbOt21Ugqoxltx37MqS8VAtu3RozfwgSjA6U/sKeUGlU3bybA3XteZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=STJqucsL; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="STJqucsL" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2605ce4276so2182319276.3 for ; Thu, 10 Oct 2024 11:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584822; x=1729189622; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=STJqucsL1kdpv+t2aaRSW8Dsd/k9W8Xwx0z/3KChOk2f7SqOVYc8Fpu9FgNXp5fiLv U2zVArrxm+zFzt6S1ZnESa2Fakw84zykbu9yrPyt4txY5HGwhVwXFA7Q0M1ej5kOsfTF cOdyNhQsGU94JfmwxOT9eZyARvp1cSidtXQIDBSmdrHrJCdXoEDgKL9knbEsHRoX8+Q7 x+Vw3mSWTwWjREDYZH57XAFRL/3RValbk71y72Jep2YJ1nJzbIRF8212ST78cHKuXiCE RyAFsegZWGTdL8tf6s6VHlUs9C4I+kaJpsg8j16L/FE68+XPw0gkSDLsBwjx5xn1Kv6y vMIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584822; x=1729189622; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=CgXUbWWAYZM+6vfhDISgeaf3xNwYtqKI7ENlrLj2a8pg69EmSRpKPkHxRGyhctsFpy IomiS8oK2I9m43Ah887u9ot2ZR1tuhQ9L/sg2Fat5U/GcuY4FQt4/bTXLJlV2fmDiYPW YVVtsZUMRuZ6aIl2XJPW5F9RNQvj6eIkP963pAv4viI7Dcu7LRu25lpt/m0k7sl9+u9p GaUSkzIrcXGgrpV3kFIhntMJo8giB3JdmgmDn/kVfT1nja7Cf1NRdbuty2CLyaN8Jl5A FhFi2+Ztblh+EYFmAggCwuME1s1ab5A2SkBHr0t7FbM4vBo+/s1Bw15IdwWaDFDafFfG MJiQ== X-Forwarded-Encrypted: i=1; AJvYcCVOGC3ertu9KevqHj1Sbp7Kkx1nPpeLvuqHfTkk0jf6W45HnWb1FlCjy+0HrMgMwWEs6caQmLIU40Wl@vger.kernel.org X-Gm-Message-State: AOJu0Yzho1epU/4ziXmbpjlo+epIkdZCysHSVYmQiSq5CXVK/p5D3xUA t3z7jHuqGC0oCPE2R3rEjea2u6SjdVoPMUSVCx0EAHUzFNY4uzelZk9LP8UH0obNK1STgXmIzxV EFw== X-Google-Smtp-Source: AGHT+IFyzgxXPfejJoY0SZCQ3T+Vo8hzb1HJlB5l94FFDfZID39IeT9MHN/ZgR4beqIvxQ/gO8qC3c4Qafk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:4ac:b0:e28:fdfc:b788 with SMTP id 3f1490d57ef6-e28fe4426b8mr4979276.9.1728584821558; Thu, 10 Oct 2024 11:27:01 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-62-seanjc@google.com> Subject: [PATCH v13 61/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s Radix From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Replace Book3s Radix's homebrewed (read: copy+pasted) fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 29 +++++--------------------- 1 file changed, 5 insertions(+), 24 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 8304b6f8fe45..14891d0a3b73 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -829,40 +829,21 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long mmu_seq; unsigned long hva, gfn = gpa >> PAGE_SHIFT; bool upgrade_write = false; - bool *upgrade_p = &upgrade_write; pte_t pte, *ptep; unsigned int shift, level; int ret; bool large_enable; + kvm_pfn_t pfn; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; smp_rmb(); - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ hva = gfn_to_hva_memslot(memslot, gfn); - if (!kvm_ro && get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - upgrade_write = true; - } else { - unsigned long pfn; - - /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page = NULL; - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - if (PageReserved(page)) - page = NULL; - } - } + pfn = __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &upgrade_write, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; /* * Read the PTE from the process' radix tree and use that From patchwork Thu Oct 10 18:24:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830727 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75D4B20A5DB for ; Thu, 10 Oct 2024 18:27:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584826; cv=none; b=bG/0jnoL83w5ZqCgFRXfUPCLG3MtcfHzvkvwV1PVqeUekLugPEzrvn519chLgXnY7oAacOtHgX8ZDBpEF2c5+Ep5oVYrgfaDxGKYdFBWnjpPwpVmjEDQ9GeAeJyfauIMjedEdr+Bnu54ztq7f6ATX0g1fuaH0rCCE6pBsCeDhEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584826; c=relaxed/simple; bh=lnamoAhiaB2k3D/cv0luJANVeCRwGvcQfaabdaB/y4g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F78eOT8ujnaCgQzpftYkHkvWlBgzA0Wn3AbiM5C84oRiy5QcI1X8brapsvCavxMuUR4xYdxW/34mmk+CeIa6m3wrX8l0kVhDOUbjxNmdg8pj7zzt8/ueR0K/SEFDkl3naE5mm33zN1wN9Nar5uRwAnkeFDkuCHp6RrBr1Bhg5X8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bjTwwhuo; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bjTwwhuo" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1656078b3a.3 for ; Thu, 10 Oct 2024 11:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584824; x=1729189624; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=bjTwwhuoN0SpvNaeySCVuMnmyKMqY2y7VhEvpQdH8Tydsmp9oc3ImEzNIB7x2f58Vc LW75/zBKZ8Dtw2BE5RKmttfdjQIE7EcP0Et20wml0hzKLI3e8x/5FaIzoiMPPwoGie9t 02kIjtfFUWb/70njpoTggzuy3qcK/ZCuTPOg9Uy3fR3Pn5LLKSSXdxVywqZg7ZuphLmL dJqWYhncEpAB9sXQgmaXSJ7FaOaCiNrjHLf7jQsWXNhvERuNF0jIbJ84cEpV6XNBHYNv Z4H1YTKXoiAb/B1jipjTL2HYW90e2uWFVEIFwneMeDCmQziwMbpeaLSFC2sEZceI1UsL NTaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584824; x=1729189624; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=uC56ccow9WwFjIuc3TwyuCzGOC7PMLj5t7MhyMqY3eE3/CiyQqlJpiD0JWGUao1d79 7KsX/E8dCo2Y5XiTbpycrAa5MB1xlvDngp5tPlWZOnkscCIfeBULTkBhOaFvehWFn83I AMjVrp1SWEpnn4W3hLoAM2iLiZ08VmDougnjw5VsahwWLpeGTNfOazQBAhf3EdA7TAI4 Ect9gxPBh9VeDP+z6q7dKY2Hq52Qe3NIfpZT7g4LWJUYG4J/HP5zuyj+31tYPZF2guvb PwR06/26vcsXzva9BU2u+6K0P/o9YXzAPiDYx00PKIiBuWYHf+wECyOmxyWmfg6AM92I dJvQ== X-Forwarded-Encrypted: i=1; AJvYcCWW0WJS2rMeLFBnHnOhZAMCJwrJLGIgs3pt7Sgc3pudcbte/bk4MBnCjYWSumlijtJXSs4EB324X6dH@vger.kernel.org X-Gm-Message-State: AOJu0YxKCfxpBWjWWc5M0lAiycXMK2yMFDNwR1QEeQfZKMf5irpHzTl7 Rl6+bcxal02VawArLEf4+fptcIqfuJeznkfGYB2GPa3b3qXk2qq5kwzHyJNtn1zwTkZVp7Qu08O KmA== X-Google-Smtp-Source: AGHT+IFev0i08WnhWXOHYVCiDa8hDSrBgilu3q+XI+BXP7ns5qqyHbYBrSKz+aJVjoCmfpAuiTEXvCWN4KQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2daa:b0:71e:401:6580 with SMTP id d2e1a72fcca58-71e1dbf2587mr27202b3a.6.1728584823552; Thu, 10 Oct 2024 11:27:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-63-seanjc@google.com> Subject: [PATCH v13 62/85] KVM: PPC: Drop unused @kvm_ro param from kvmppc_book3s_instantiate_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop @kvm_ro from kvmppc_book3s_instantiate_page() as it is now only written, and never read. No functional change intended. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 ++---- arch/powerpc/kvm/book3s_hv_nested.c | 4 +--- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h index 10618622d7ef..3d289dbe3982 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -203,7 +203,7 @@ extern bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, extern int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp); extern int kvmppc_init_vm_radix(struct kvm *kvm); extern void kvmppc_free_radix(struct kvm *kvm); diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 14891d0a3b73..b3e6e73d6a08 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -821,7 +821,7 @@ bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, bool writing, int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp) { struct kvm *kvm = vcpu->kvm; @@ -931,7 +931,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot; long ret; bool writing = !!(dsisr & DSISR_ISSTORE); - bool kvm_ro = false; /* Check for unusual errors */ if (dsisr & DSISR_UNSUPP_MMU) { @@ -984,7 +983,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro = true; } /* Failed to set the reference/change bits */ @@ -1002,7 +1000,7 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, /* Try to insert a pte */ ret = kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, writing, - kvm_ro, NULL, NULL); + NULL, NULL); if (ret == 0 || ret == -EAGAIN) ret = RESUME_GUEST; diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_hv_nested.c index 05f5220960c6..771173509617 100644 --- a/arch/powerpc/kvm/book3s_hv_nested.c +++ b/arch/powerpc/kvm/book3s_hv_nested.c @@ -1527,7 +1527,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, unsigned long n_gpa, gpa, gfn, perm = 0UL; unsigned int shift, l1_shift, level; bool writing = !!(dsisr & DSISR_ISSTORE); - bool kvm_ro = false; long int ret; if (!gp->l1_gr_to_hr) { @@ -1607,7 +1606,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro = true; } /* 2. Find the host pte for this L1 guest real address */ @@ -1629,7 +1627,7 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, if (!pte_present(pte) || (writing && !(pte_val(pte) & _PAGE_WRITE))) { /* No suitable pte found -> try to insert a mapping */ ret = kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, - writing, kvm_ro, &pte, &level); + writing, &pte, &level); if (ret == -EAGAIN) return RESUME_GUEST; else if (ret) From patchwork Thu Oct 10 18:24:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830728 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 892A220ADD3 for ; Thu, 10 Oct 2024 18:27:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584828; cv=none; b=AzBUOg/0qydc1fw+uYRIMGR4u2tWZzarqXTVcW0+BDvSTB6dxYG2jKBmXiUspwizSrECi4FkzjEsK5WRlKQ2Ltrdv/R8qRdZyw6HTg4Xxn4T6+qWM4wGelK/kvz9h87IY6MWQKfZHcUV6FVKikcW8D8+JxmkQpgpbZhG5p4oRIo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584828; c=relaxed/simple; bh=PLrYiBlw82beEM6tLfodLLTPBglt9VZ+TVoJtuJPFlg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hUX+LgzqRRXazW2CA2ytp5M+tLt35iA5QnBkq23iRi8jMzMSZxtm1qb971ElyjptobiwtCJQdMvWPgpda0DZgKi14b9swzdAUuGuXmTxIsOALhIvbbalRlExYBXGZs9SRwqOegY+uLyxr7TMLLPl84UF48YBQOHxDIwZbO2WwiU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ooDlC/pc; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ooDlC/pc" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7ea0bf14523so1033549a12.1 for ; Thu, 10 Oct 2024 11:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584826; x=1729189626; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=ooDlC/pcgIiD9tsIsdI/XR8hhjsKnl/ubOYfWwqTcVLjG/9/GB7swjVJDrSqoLoWOw 7vkXEltPbHGSo6eNQSnfEV9KrL1bmyzVtt77RGsOKB4v9yqQdHB78U1BuZ/hIDzG2FZx cpC29Q8dE35bdHWwfpE6KKYut07InE+/YOzDqpcW4mYYCpTnffL1vL41QmzJeXlv5JSV Vy6UxPe1LGdKUb6imMlvnN/L7ACFiPOGAu0qVfgVxbFsbFAK29wlRNmnvUlIrLTo62bY 1kZmPlVjpXSazX1NzmqGpjSByYskWop3ZgGxIZFVYyv/jRa6XYdaRoFScRBxFJa6oghl JhXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584826; x=1729189626; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=qg0qOSnOB099Q8Bn7lYG52L/Mio9ojGd2fTuLZLkbI40B5M1W3FlyFKNMIFYA3p17w DegaHp/5InVUydxbUREluejjs3LgA5yGZIOMn4lL6kK+h0cewVL1GeeD80dhgvIyEOJR qHSllCEQFfYI7KT8BAQaBdwuNwkTzpgDQVuXxnUZjLoJx4Ev97i0wli3Bg9rL5wK4DbK CYmeMOfuvEaQpWuZiRm8zWapgB/zxfPa8r0mYpTUivG92cIYIRtGUmqA2p0hM53sNX0N ER3TgrD4Sb/zdcQ3rje+h3SkzMX23WyJ3k/vf4zzlSnT7LVG8GP8teqGzIiV9c3kVJoT 7yPQ== X-Forwarded-Encrypted: i=1; AJvYcCWy3hlGJwtmLb2Ur5i+A/eB76/EggFu3yvfxAonMcUI66EuX1XIpAQjPz8avugjqjnyeZN6vIkzJiyT@vger.kernel.org X-Gm-Message-State: AOJu0YweSpKJ4WkKij0NPszuCePUzm2usI5hT9G08oEb7WIdrMuGRqQD zUDZsCSl4gOwoSObuKG5aFXnS6JgGC/PpSefzx5hKgx8zFScL2KC0MJ3ybxifpC3zCsodcNQ8Og Ntg== X-Google-Smtp-Source: AGHT+IEBzxfSdV5URAWtipQSxApJSdWqJAcjcFibIBQs32g4G2eVPMYjVJgo5zFwBPuTEySHq4oumcdrCOE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e546:0:b0:684:6543:719 with SMTP id 41be03b00d2f7-7ea535307afmr40a12.4.1728584825522; Thu, 10 Oct 2024 11:27:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-64-seanjc@google.com> Subject: [PATCH v13 63/85] KVM: PPC: Book3S: Mark "struct page" pfns dirty/accessed after installing PTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages/folios dirty/accessed after installing a PTE, and more specifically after acquiring mmu_lock and checking for an mmu_notifier invalidation. Marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. This will also allow converting Book3S to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c index bc6a381b5346..d0e4f7bbdc3d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -121,13 +121,10 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, vpn = hpt_vpn(orig_pte->eaddr, map->host_vsid, MMU_SEGSIZE_256M); - kvm_set_pfn_accessed(pfn); if (!orig_pte->may_write || !writable) rflags |= PP_RXRX; - else { + else mark_page_dirty(vcpu->kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (!orig_pte->may_execute) rflags |= HPTE_R_N; @@ -202,8 +199,11 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, } out_unlock: + if (!orig_pte->may_write || !writable) + kvm_release_pfn_clean(pfn); + else + kvm_release_pfn_dirty(pfn); spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); From patchwork Thu Oct 10 18:24:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830729 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EF7520ADFE for ; Thu, 10 Oct 2024 18:27:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584831; cv=none; b=EDIZjpSu2sgptVsuVVm41q66Kp3toZQaUglOT87VRdBRTYfBuEVhbXulfe14iI70tXs2meZGZseO8PUV01vfl3maw3MBd7psO1RapxASgfPVO9CMAvKcMXr6T67tqCLitYBJiNfkFycadknIU6IOVGoiG5uh4e2C7UPbdxA/dX0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584831; c=relaxed/simple; bh=2hWIRT2fMV5KiR0O+VosZUqCujtj2BkFwotH6/Eh4cU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ap/4HRrq1kZ/8dRJ61Scpi/GjnU7HglNABHvn5JnRgeW38MpNC/rms8PZVJyXOO2FXrNyzbDU7eoN+6/UCTHra836mSliDFF3K7Uj/hTkmjcnx56Bs6LV73h8aN7VgaG+60BWA1ua0Ya0+A+QIZi3kDmUyt8TZnBo+tHCRJKbFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1pNJedDG; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1pNJedDG" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e29088a6942so2105912276.0 for ; Thu, 10 Oct 2024 11:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584828; x=1729189628; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=1pNJedDGxXfMA3UIXvlLk9yUgFqg1Xwn3494wFqV+9i756hIyBtpQeMuDK14Oc8dJ/ pfvqrjHQZGkwYSiQpe7VEIzjzFJBs4dSTwV/KaNCt5t+7W3SnJwkBgP8lzwASf4U8hX2 lWHShUHfEnMJxWGPlMeM6oT39oz82YA3W4dWND4IiJI0SR7DpM7qVYgr3EacLZKX+Ght 39PQ15sIXfArpkmZRyzDuN95/3br1T4ES3Q/qpqzwWVmjVIbR8OhZXDLtz+Q1DJ52QfE nOe3psdiZSnykO5KYiW4RM5eHmQ6AQ7kh4u/3+4CjkTQixhFThwX9QmI7yg626qLu3if LAWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584828; x=1729189628; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=C2VXEWKShvWQ+lXNG+h1c4i+QDlw8HljNPaIEN52mhWDi7QMi/zlK+kDHr87GffzEz rSxDe0cy+UsFuo8sjvf85+Ojq/miOFjmqK7rf9E7ZeciRqxXn4YV2tKKlbubSFoivoS1 FYFnuRTUdt0omwZh4+PAuizI4ZrtH5bzYlHp6YmhTvMVU8rWGR7w7DlQtqsUvqjB1CO8 Xyn/w+6lCMshTNCrZUL9ZWi1U8BqhJ/4Q60mw6a6V/12PoJMgZGA8xw6Aae3NThGPy0m bw42SwonlZflJ0FWbbEdzSba9UlsQ4m5VNc0pXV4GIWWRbBRXdwpSP19mDNt+y6kuMN9 krZA== X-Forwarded-Encrypted: i=1; AJvYcCUqUYsUwFm0kLdLnTGxqs9iN8Cb8hBs3g5F8+k+mXJZsHomXDBTeF95JTT7nJK190jTmGizeUEl4tvn@vger.kernel.org X-Gm-Message-State: AOJu0YzN3WXuSMU5ZpPY4a8N36ccDvjI+yqAYqk35vNA68HM2nn8Drrw /N4elGCSFeTimn3mkBO5gdTxBojgrYhBNM3HWSLy0tnfNOQuGdL+UpR3VybfbBOqs6Fzx1BkoKa LJA== X-Google-Smtp-Source: AGHT+IE3wQG+AmAWHFVOEkWtTHcZkQX/AbMzk25RszCXWWBD0mfEZbS2aMxfU1imj+f8RQ2cZBD1mpHhlQw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a07:0:b0:e28:ef8f:7423 with SMTP id 3f1490d57ef6-e28fe355de0mr48920276.4.1728584827639; Thu, 10 Oct 2024 11:27:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-65-seanjc@google.com> Subject: [PATCH v13 64/85] KVM: PPC: Use kvm_faultin_pfn() to handle page faults on Book3s PR From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert Book3S PR to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s.c | 7 ++++--- arch/powerpc/kvm/book3s_32_mmu_host.c | 7 ++++--- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h index 3d289dbe3982..e1ff291ba891 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -235,7 +235,7 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat, extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr); extern int kvmppc_emulate_paired_single(struct kvm_vcpu *vcpu); extern kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, - bool writing, bool *writable); + bool writing, bool *writable, struct page **page); extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev, unsigned long *rmap, long pte_index, int realmode); extern void kvmppc_update_dirty_map(const struct kvm_memory_slot *memslot, diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c index ff6c38373957..d79c5d1098c0 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -422,7 +422,7 @@ int kvmppc_core_prepare_to_enter(struct kvm_vcpu *vcpu) EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter); kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, - bool *writable) + bool *writable, struct page **page) { ulong mp_pa = vcpu->arch.magic_page_pa & KVM_PAM; gfn_t gfn = gpa >> PAGE_SHIFT; @@ -437,13 +437,14 @@ kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, kvm_pfn_t pfn; pfn = (kvm_pfn_t)virt_to_phys((void*)shared_page) >> PAGE_SHIFT; - get_page(pfn_to_page(pfn)); + *page = pfn_to_page(pfn); + get_page(*page); if (writable) *writable = true; return pfn; } - return gfn_to_pfn_prot(vcpu->kvm, gfn, writing, writable); + return kvm_faultin_pfn(vcpu, gfn, writing, writable, page); } EXPORT_SYMBOL_GPL(kvmppc_gpa_to_pfn); diff --git a/arch/powerpc/kvm/book3s_32_mmu_host.c b/arch/powerpc/kvm/book3s_32_mmu_host.c index 4b3a8d80cfa3..5b7212edbb13 100644 --- a/arch/powerpc/kvm/book3s_32_mmu_host.c +++ b/arch/powerpc/kvm/book3s_32_mmu_host.c @@ -130,6 +130,7 @@ extern char etext[]; int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool iswrite) { + struct page *page; kvm_pfn_t hpaddr; u64 vpn; u64 vsid; @@ -145,7 +146,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool writable; /* Get host physical address for gpa */ - hpaddr = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + hpaddr = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &page); if (is_error_noslot_pfn(hpaddr)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -232,7 +233,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, pte = kvmppc_mmu_hpte_cache_next(vcpu); if (!pte) { - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_unused(page); r = -EAGAIN; goto out; } @@ -250,7 +251,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, kvmppc_mmu_hpte_cache_map(vcpu, pte); - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_clean(page); out: return r; } diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c index d0e4f7bbdc3d..be20aee6fd7d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -88,13 +88,14 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, struct hpte_cache *cpte; unsigned long gfn = orig_pte->raddr >> PAGE_SHIFT; unsigned long pfn; + struct page *page; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; smp_rmb(); /* Get host physical address for gpa */ - pfn = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + pfn = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &page); if (is_error_noslot_pfn(pfn)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -199,10 +200,9 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, } out_unlock: - if (!orig_pte->may_write || !writable) - kvm_release_pfn_clean(pfn); - else - kvm_release_pfn_dirty(pfn); + /* FIXME: Don't unconditionally pass unused=false. */ + kvm_release_faultin_page(kvm, page, false, + orig_pte->may_write && writable); spin_unlock(&kvm->mmu_lock); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); From patchwork Thu Oct 10 18:24:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830730 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 526D71E47C2 for ; Thu, 10 Oct 2024 18:27:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584831; cv=none; b=F4OJ4sdiRk6OID9E2kARpC3IdepJNU9N30fu9LNDsC3WR1hgfjfeXM1fpV+sj5aVMD1N67oidE9tuKVBXjmkhsX+w8iTLiflrvY27twrIrKeBnc1Hk7Khu0/REiPMwg26lj17bzADh7VlasV4JOR53VavdjNRRcfQ3xSxT2/alc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584831; c=relaxed/simple; bh=ffJ+Y8PQp+slQQo0S2lzOgDI6zSQ/cN8IdVD7ZeuwkM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WkwvcP2eNh7t3B3wVFxHm91r2l/yg5xQZc3mrWjzjHrj7OvJc1q63kSi0eEJxUWCZEOeI7UKZKbKpi6U1Z94iwe36uQwXa5u4Befnwm2ePFbjk8hfmAijsflZZHd479IujQJrJi7uDhXTRSdZ9EbR3aaIYii208v2lHP5zspU2M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SNuiX9KG; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SNuiX9KG" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20c7ea6b060so15678405ad.1 for ; Thu, 10 Oct 2024 11:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584829; x=1729189629; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=SNuiX9KGV9gd57zGcX+khtxMQ/WZyfcI8IOEqbUSxhtcMIlWJDhUZzOYw3O/Jtp5tp 1NWu+kxo5Hi64N0d8Vocbim1DEDIN7pKYEMAT7CmUIRPux5M5gBhtwooiZa036V75h+k Ychu0Yrt+CY1DTpdXIkHOsKY0bGLGDYVFrglD82TfIlMDlvD77LbhWwQE2/GuTcCLoye BlO64+wQ9ljMZU5XEby944HMtp4M+bmKjEJdM7giaFb7OEfLYmaZJu6Xf8a4cSqRhdfW SsOXG0o6p9nmOEvzhsK039Qzqg4ApQUtgbICzcD04kyCCbohbJE5YCYQwAAz1ROtSzRt QapA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584829; x=1729189629; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=MU5xY38+fGxbcG75UnsbIDiSSmiujOk7WspML+bxJbjHgJw2EqAe2zow6fMSCFu6Is 3DfVO3XIFHlCuOUQ+c90KKCAJJNpGTJXj5yrA91fdOmZ/zwUhBUhLKXxAlSmIzOQL/5n CSgxG2tv0lU4AlWa0SniGxT10JIaSDBhV6jaILCnYNy9vAnzPNckoniUxz+L+RjaiMRE s+8iJEftfheA/7wEaUGsHGJIOHfjL31/+nF3AAyhKrax+YiuH6uZ73gIlHw80NrMpFFs D9W542KpYBqOdto5S7NtVDprdf2Q9BHnT1X3442e/I9h+afWlZEM6DS/sxwvuf6QTO/J NGsw== X-Forwarded-Encrypted: i=1; AJvYcCXYcUNsYmKOPYzdpub449xrIHyyrJKudjtnjGOmFk9jZzcaePZcGrZYtKCpXg4i05uzQWoBzIfaR65V@vger.kernel.org X-Gm-Message-State: AOJu0YwoOMoOHMah1mIbzSZ3tUDZgGBEjWR9cdhcUhpI835GmUcnJSuv ABlZ+NAtMBk1rFKpITWHcBN3g3r+y2hMEn6sEu4nOwXt47ic8rUxH0k2+VPV58aqC1J+gsjIzf4 cHA== X-Google-Smtp-Source: AGHT+IEVjHGN/ZoxQo+Z413816spxD1Vig97FmVkGQ57WYZx6k/7CY2I8TFVs2SPVm+Oq4UkGx7TgXf3+Fw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:192:b0:20b:bd8d:427a with SMTP id d9443c01a7336-20c6377ab7bmr1062205ad.5.1728584829203; Thu, 10 Oct 2024 11:27:09 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-66-seanjc@google.com> Subject: [PATCH v13 65/85] KVM: LoongArch: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 28681dfb4b85..cc2a5f289b14 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -608,13 +608,13 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ if (kvm_pte_young(changed)) kvm_set_pfn_accessed(pfn); - if (kvm_pte_dirty(changed)) { - mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (page) put_page(page); } + + if (kvm_pte_dirty(changed)) + mark_page_dirty(kvm, gfn); + return ret; out: spin_unlock(&kvm->mmu_lock); @@ -915,12 +915,14 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) else ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - spin_unlock(&kvm->mmu_lock); - if (prot_bits & _PAGE_DIRTY) { - mark_page_dirty_in_slot(kvm, memslot, gfn); + if (writeable) kvm_set_pfn_dirty(pfn); - } + + spin_unlock(&kvm->mmu_lock); + + if (prot_bits & _PAGE_DIRTY) + mark_page_dirty_in_slot(kvm, memslot, gfn); kvm_release_pfn_clean(pfn); out: From patchwork Thu Oct 10 18:24:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830731 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45E1420C46A for ; Thu, 10 Oct 2024 18:27:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584834; cv=none; b=IXNs/FwjC5M0mcw2zPZOuYzAh+rSYFpibhYdyoS0zzi3sTkrifNN9NCBm1GXYDcm/IxyJzNOmhIEgqDMGZXLwVhMjHCxoiIwufy2f1nq5SddsDBBELLTz5R3HtkrZqHqoou/lzN9kcEHbvDvAZkzEXVWbJwKXYFfnbr/UlRF0c0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584834; c=relaxed/simple; bh=Gq0ZEmsgjgD9l+p+SR2hvcEYSP8z96GfO+zPwvkDX5Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hWGaUeAcW4WqYi3A1QNYq8NOdLJohopRrX7qv29TwgbRq3oLVGmBYAicMgzi+BnnYuNxOlSK0vsaLYZElfwZHXMT2xow/Ed03laIq33MacTo+Gfdq6RyajL4EXl8NQsoxunhhs1JEaCx2JoJMG7BCvXyXLk1B3n6gl8JCWTpVEg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EfkfFSQF; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EfkfFSQF" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e28eea1074dso1984586276.0 for ; Thu, 10 Oct 2024 11:27:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584831; x=1729189631; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=EfkfFSQF3X+VfzLOUd+NMMcl3GFvR0O6RLsG8kW9/3abGsJmogSRIrtQpV+OWWRYNJ GPPNPPUNAXbsIX23X5G7HUZQZMe1WBRNFQm0FfhGz6XCl6mJGiQ4ZSYhxhO7CPQx+8rt Vkr+5ZJ+ns3A7/GgnaSxX6V/Azj+OACWU++fMoUbLZtZNL2ustwBKTJDWDypFdll23fy MW3rG+4rwyG1DZ5JU3s8eppFPrGqE5zTUyzexpo9UP6Hmnrdezl7ABJ1sGb0IFTAI+zz SNg9Wa77JEnsbl4/wjMJHG/BXG4AFPmPEmio8T5u6AeNKeqBwqQn2ivs9Vc9+kwrlQHL nxTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584831; x=1729189631; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=FvoO6+GuZjxwO5TQUUqcry7I928FtxorWGSx+k0LiYNZTBspOE9qaks0xrKTvfdz4k RjYcXzUz7gNrfNegszww1tfwpTwpR0oU2lfZTSNMaDyR1uPZrRAbbRyuQI2dUDDWpZhn JaEKXCgdqBwfXFNQiWbwepCNHEYkix3exs6HI23aYUyLnmuGm9PVOpn9WgBDVaCpDRsz soPGNysK5hLOzizveJn4EDUo7TT8l0li7ylMYUnpV/LbRQbRBO+QWJGHlv2BY+HdyScA JeUJqHLuKqntp5OXl+0TiKoHdiz/JA+mDNoKpNqnkLhwxFkXpQSziF+grh5/zhG6S7zN MNmg== X-Forwarded-Encrypted: i=1; AJvYcCWy7N8xeKVJyy3WKu3xyPzc1ZthrZORXSl27v4h+gsueDCu+GsYDVsaCZM5Lcvvq4bl5PO5Eeg9t9xr@vger.kernel.org X-Gm-Message-State: AOJu0Ywt6iQYwb+8R8/ynr+/kZK2soO66yTM0Je7jkMSLKqqXabyXQTE snsrp7cOZPc/mOs239h+IBLeg5nIgepHgpDAUeIQdzKidDjTa0HtXnwQ5WSUUTTurJ+LE+Jc/j1 EQA== X-Google-Smtp-Source: AGHT+IHdtHq1a5g5QmW1TlBMuTKUOZ7R4d6aOVSlzi7r2JK/pfzCqaKRixPsJw6iRhPElP6sWQ73LfqmPps= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d353:0:b0:e28:e74f:4cb with SMTP id 3f1490d57ef6-e28fe0df614mr96115276.0.1728584831124; Thu, 10 Oct 2024 11:27:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-67-seanjc@google.com> Subject: [PATCH v13 66/85] KVM: LoongArch: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages accessed only in the slow path, before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index cc2a5f289b14..ed43504c5c7e 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -552,12 +552,10 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) { int ret = 0; - kvm_pfn_t pfn = 0; kvm_pte_t *ptep, changed, new; gfn_t gfn = gpa >> PAGE_SHIFT; struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *slot; - struct page *page; spin_lock(&kvm->mmu_lock); @@ -570,8 +568,6 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ /* Track access to pages marked old */ new = kvm_pte_mkyoung(*ptep); - /* call kvm_set_pfn_accessed() after unlock */ - if (write && !kvm_pte_dirty(new)) { if (!kvm_pte_write(new)) { ret = -EFAULT; @@ -595,23 +591,11 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ } changed = new ^ (*ptep); - if (changed) { + if (changed) kvm_set_pte(ptep, new); - pfn = kvm_pte_pfn(new); - page = kvm_pfn_to_refcounted_page(pfn); - if (page) - get_page(page); - } + spin_unlock(&kvm->mmu_lock); - if (changed) { - if (kvm_pte_young(changed)) - kvm_set_pfn_accessed(pfn); - - if (page) - put_page(page); - } - if (kvm_pte_dirty(changed)) mark_page_dirty(kvm, gfn); From patchwork Thu Oct 10 18:24:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830732 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2151620C490 for ; Thu, 10 Oct 2024 18:27:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584836; cv=none; b=WOsw/thOYEHeh4eRfwNqSjV1Nc4BvDjyAYEAwYdT/4lwXTrQRb0i5PLdA+9GuACaAZZCs6GM/0aIIGhR9OhKemgVifeVvGYw1T5qTKnVEvufi7JcbcV9Q2tX+Jtt6BpS3aIFjCqRxmCCzzyndUFH5Zz3TBf2ihQAIWTr4UGG/h4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584836; c=relaxed/simple; bh=+TpPAbVVI1obWW7OdvHKpXsCMwy9+BV5v8TfW2u9vbU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HbZda94MjBlmb7PRTXTjg3CDNy7XcJ+9KBZ0Dxmv4X7SP+ed8Fv6Ea33bg38KIfKzlG+H5mKyTL49kCeqX3+Vh9MH5wjA8KH9UNwOQABCUJs070SvCD+rB0R+NLkAOu8ypboTkH5/4yG/z+tskcf6Z1EWVHq0SaawytfF1yuvfY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GNF92OQx; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GNF92OQx" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e2ca4fb175so1249366a91.3 for ; Thu, 10 Oct 2024 11:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584833; x=1729189633; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=GNF92OQxF0tq2Sy3NXgpLFxQxC22Wl9tw03n7HzL4JrVAJCkh+Hov4DvKeVAkbLbz6 guCgfWZA8nqZBp6XZ9pJ4Hawa9uVtgV3UErDEF6u0+OBm51ovt6v1pRCPWJe2lNSocFb 9IG5h2poJKL8NVnbhGa7Q5llq0RI6iyjaleF9SNYfF/o/fNFc2aKi+UykoVstbZ8OxwU d1wLhivyUM4mbES9K28ITC9CcdOVPnfsFw6ARgKHJiVEHnES4HjeBBPrQzXn/hChp5jb ncyckAUOmfR9C543A9BJWxeDUS9424yzgfnC3W2p+8MxlZym6ly+BGplBaNU9k30CJj/ w62Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584833; x=1729189633; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=IikyDk2GpcIZaOfmrfmbD3aggWMDUkUymk9pb4qKeucRUwnSLTH/tR6DV0LEZv8RbX LI36o4HzXJXMBX2cAmRrcWFuTKipTaogSynkbr90q5nBY2En9p68iaUOo45Ih6VBo/Jn DS4plHYHxVyjCL2s5S7lzE0IlmiZjnk8uuasFMDA3JTX/khXGxRSCQeDgJtdtDzoZRoD 1xGymGeDUDalOru2aO+pPWboi3uSMmH4H9CmhBTacepFZ2GiYB5mrj8G5M2iSQeKzwi+ 8L10XLX+7N9ScQiaTWFaTZXowG4wOS5677RwBv9HBU2kKzbrl07nrMH7gMAtzbYAPW88 ZGYg== X-Forwarded-Encrypted: i=1; AJvYcCUdTNrBVPXzHcbU4DjHgItCVQ3VkkH9cZMkZ7YgL8miNItiM6G0Fm0FK90sMFJ+TpjCQyy36Wo37iFw@vger.kernel.org X-Gm-Message-State: AOJu0Yx5OzsrnbLN9y22c1O4fH0PtY16pGOBs8/bPANvxR66l8D7lBme S7QZyIp8oTmYOAEcXoAEMWBy7TelYFD762HdYchuojbldgcuG/Fjn7zM4VYoAMoWq50+tQ8HZGo Ijg== X-Google-Smtp-Source: AGHT+IFpwqxWKAWfrUvKF1EatvnBv5g5CrzcpgT/ZTuiTODJnzoCoYK053S1MFwF0WyL9wI3LVhDYDAiv74= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:46d4:b0:2e2:e148:3d37 with SMTP id 98e67ed59e1d1-2e2f09f1a56mr94a91.2.1728584833102; Thu, 10 Oct 2024 11:27:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-68-seanjc@google.com> Subject: [PATCH v13 67/85] KVM: LoongArch: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages accessed before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index ed43504c5c7e..7066cafcce64 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -902,13 +902,13 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) if (writeable) kvm_set_pfn_dirty(pfn); + kvm_release_pfn_clean(pfn); spin_unlock(&kvm->mmu_lock); if (prot_bits & _PAGE_DIRTY) mark_page_dirty_in_slot(kvm, memslot, gfn); - kvm_release_pfn_clean(pfn); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; From patchwork Thu Oct 10 18:24:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830733 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 317A620CCF4 for ; Thu, 10 Oct 2024 18:27:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584837; cv=none; b=TvHlK14wBd8cJJoCMUWoikQgY0mL7u6NHm7Y0Px4LdDHFB8YRJ/OISLqPCAdrrBfmPWBQkXTZ7GW+d4XzKAazhtckRNpQoh46uuZuAFiBDXA9s3m+uEZ+e1VG/HW1zliOJ9epxEc0i1a4XX8uPAj5rBq6AkUyn/SjMianTyoehs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584837; c=relaxed/simple; bh=HhUA1PeKZEb/51LjtENBtdXbtQZ0o3j1vGZCLDD8mMg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=snPkYl0m5hrEi84c19blsgBdSI+Bkxckij27J0+ylbUNmF7FiMaR9QCgLIMCu1DO0Cy8OdgtlCEuH2qclNAXy5Z6veOZAUgfM1Bg4B2ddPyxAcBuK8ZCM11jMhUTTtLdCYkiBYGLPmv+Kl3xr6cqiCsXaKKaiGhQp68K1bG2v7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZfysK4Eb; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZfysK4Eb" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71dff575924so1543898b3a.1 for ; Thu, 10 Oct 2024 11:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584835; x=1729189635; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=ZfysK4EbyW9GP/xSuAfW3U3GxCRMKGJ1uQ0l+1a9fb8m3NTbOSK3zUlXUKjOSQ40T5 ULvpVNM76l5WMWoFdvGpGrBdcaqid0kWMM1s0kBGxkAqf4kwZrEANqHFgqecmt8eHYgx Ik8OK2Vg31lI7rB/rwW7MTUKastSn4QH1xeXn/WK8/jnpqBbiN9Zv5K+jntkQWm890PW KjLL2HC/A6kUxWwarJKMFJacXMwAT0kofIlFki+OA82WubYYtrJHX/t4g5yKgnDO+Ptq 4dA1ZW7S4NyO5r2DrnFx44K0GxT/uIbV+E5cT8JHXE51CWa5Cvp/710I6os174/V43Yl h/rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584835; x=1729189635; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=rtrdPsE7uecO6DveFnkvZl5IheFS3tmyEjmBzrabVXNflzQokd9PC5Y2YONcfhyPnV WhhvJOZyK3RcHpXVc+byitW6cewTfem33HEpjGHYfCibm7TYalrL/QHupY7E8PuBu0z/ b/DjZU6ZMu8g7B+HGOAU1BtiLx8npSS+79gO3eEYl+salm3Dr90NfLpgUgtIEZ/jpBow SMFCX5wxKvSimlSTgpqAARvHOM9NOG020fBY6fm4+IfNxtwvsA13TQOW3hOptFg/4a8m 9rrZW+ZnTO+57ifdQnnbonoYKS1AJzHphfNSgd7croTbKpElY2qQEwzKDt3jDf/WYL/8 pbrw== X-Forwarded-Encrypted: i=1; AJvYcCXEyG2h2NbTCt8IuxDGfFX1fjczT+Lv+Amk18zTrV6tt47ARU492b4AiKdJNdLf1ikyMV1FwxXPkC1r@vger.kernel.org X-Gm-Message-State: AOJu0YzMsrwzHI8XxLVUswFr2HTjMv7BtrTBbY1ZnJ2+tsQzENFgJMlU SCWn1OP9zV8vyt4YWRPcscUHVEcFxp6QcMVnlA5Hzs6m5TpC8qGhqYxhcmqOOGlJMjU1wF0GV1T nwg== X-Google-Smtp-Source: AGHT+IExt7obORK9hFSNzpsBdJKygVAVx/dPvrUuCl2bsyTFt9B1GPfq0Q1aHG/lNRl/yht5gsZcWFEXy1g= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8614:b0:70d:fba:c51c with SMTP id d2e1a72fcca58-71e1dbd032amr6898b3a.3.1728584835380; Thu, 10 Oct 2024 11:27:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-69-seanjc@google.com> Subject: [PATCH v13 68/85] KVM: LoongArch: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert LoongArch to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 7066cafcce64..4d203294767c 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -780,6 +780,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *memslot; struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache; + struct page *page; /* Try the fast path to handle old / clean pages */ srcu_idx = srcu_read_lock(&kvm->srcu); @@ -807,7 +808,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) mmu_seq = kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads in - * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -819,7 +820,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) smp_rmb(); /* Slow path - ask KVM core whether we can access this GPA */ - pfn = gfn_to_pfn_prot(kvm, gfn, write, &writeable); + pfn = kvm_faultin_pfn(vcpu, gfn, write, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err = -EFAULT; goto out; @@ -831,10 +832,10 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); if (retry_no > 100) { retry_no = 0; schedule(); @@ -900,10 +901,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); if (prot_bits & _PAGE_DIRTY) From patchwork Thu Oct 10 18:24:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830734 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 691C121018B for ; Thu, 10 Oct 2024 18:27:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584840; cv=none; b=kI3+ftIsIbsHNerWQaINeopYvZj09UI3+F3040ASKuovdrz21Py60gZyukT+3No11H2guOdElPo/Hq+W/BanGh4O1fIXrgF0hn0SoHb0y7P7pIDFSZmPpBsD7CveRfZm9WT9bnzUR2VBhhTIqxhB7qrrou8AR4avsiK/X4wLqlU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584840; c=relaxed/simple; bh=qdVWvBqKE29lnYElhxBDh75UWFXRti117FkDAjCdYI0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=e9QPt+HrgCSC6Hm5gZd4aBHkzjfvxeqicotGxrC8l17VQ8i8i60y8Mgi46hqH4P3y+/NpiGlCZjJpCvQQthHIDtyLGY9CGFsJiUptLLLQYng20ct+Sy7P1brmigak3H0GScK4ooR2QLU6l4LJiMAgxiJVUsoMmqY2hK/HLHig/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WEliYGH/; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WEliYGH/" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e31e5d1739so25060167b3.1 for ; Thu, 10 Oct 2024 11:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584837; x=1729189637; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=WEliYGH/EkhyOCm54H+/wesyw/aZ7v+0pWnBeiqQzyPlQ2Yj0TH4WvG+UvuO5i/tJP qqFT63mFFsNK8nCKRd2rdRSnBi6xiWs4I9YPkwQ/ORC/XHuM6kP3lGibolilYQnn8KkG 04jph3IKCWStNMMrOeDQO9d+9jis+VCzO92vxkbcdm50eGUrzmGw34nU9RjiIPyyw0Uh euFYKjXV0mCOLJBqh+xh9tHaXM4uh2RwdSp6qf7nlg/wQlMv1OOpW1BTqmQYcv2QbZgI qpCY90RHVgLxl5134r1g7+kqZbPFrbMCMGJiJssp7lQN4g6nR6yUrSuWdNXtIvq6MagM 3JOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584837; x=1729189637; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=s+Sl0C+dWq7sVgKfJO2QhwjMImBbFZvn+/aVoijXCH6QJAn7VuOBUDHW4lv/2y2JL4 uj8gdP8dMsqNUPmeEZfWB6CXYqXgJOZE6TTyX71tdjH3HrMW706cx+XkQ0UzU47tIDxg NZRoaYF4lTDUu9HN/J0fB+NVeBEO6t/vz5OJOudMKq6BpsjAGIL6XmwdEuncOdJW8Xgi DKyTSokbxiBXL6J6+LZhVdX9ZSy75ylS7U1c1QdGt+vAcZ2n4/dd6Y+i7gxbfTOKeeoz RtZav40inRHeYbdiEJqPSLrGQwnFctnvlpb2AjVZ5GysSDvMMmRZbAuxEkTtf63U43nh ZsFg== X-Forwarded-Encrypted: i=1; AJvYcCX5bKlEBRxS6Ru/XIULsxjb/uQKHfg0L3IZwdipkkBYbmgatk8MJk8FEVuJ7Hi2zXgPQBMbY5fiKkfu@vger.kernel.org X-Gm-Message-State: AOJu0YzJ1N2GK6MafvLaA8fmc0WRtf5IiIRjieDubqzI9wRiJSqOXW5t cFrVF5nVmuY0Q4vaTWIxrcqTlNfpe3W7T49ObbXlNqOlQzeiW5wMB1l3a4ZcFaJ0v3/YcKLlKn8 //A== X-Google-Smtp-Source: AGHT+IEjJriLMGICq9yty21Fx3Z3FQyTG0XCg9lS5qhc+MvTVoRASG8drmw0OxSoWZyAtrQmM3pJDo8jQYk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a4c:0:b0:e28:f454:7de5 with SMTP id 3f1490d57ef6-e28fe52740fmr75826276.6.1728584837458; Thu, 10 Oct 2024 11:27:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-70-seanjc@google.com> Subject: [PATCH v13 69/85] KVM: MIPS: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index c17157e700c0..4da9ce4eb54d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -514,7 +514,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, set_pte(ptep, pte_mkdirty(*ptep)); pfn = pte_pfn(*ptep); mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } if (out_entry) @@ -628,7 +627,6 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (write_fault) { prot_bits |= __WRITEABLE; mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } } entry = pfn_pte(pfn, __pgprot(prot_bits)); @@ -642,6 +640,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (out_buddy) *out_buddy = *ptep_buddy(ptep); + if (writeable) + kvm_set_pfn_dirty(pfn); + spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); kvm_set_pfn_accessed(pfn); From patchwork Thu Oct 10 18:24:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830735 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10E6E210C21 for ; Thu, 10 Oct 2024 18:27:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584843; cv=none; b=gmSJ+GFRaFFpg7ziJ2kLWFAgL2G6ysoWgjtOIOpsvd16R8KJxRvoB8fOdcq17Z2asy8pLVQc+bLMK2sFhrcTRI1kktxELsYrdrMsgY2a0ZLRup5cRMRfI08DBrO5VAj3IMkFXSJSjAmf6ozSlCr4xJoFmBs4rm5XBe47hOmN2xc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584843; c=relaxed/simple; bh=TVe3szAAU/4RHYDireJVG9Y8gE1GRTl7yNXWTsweF6A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ozn7fe+vSrxi3aI5HYUtBNJmzArnTRmS8JbbUeowwcIddUAo//ha21p/ZbOXpiPsq7FPWE4Jw+qAZ9AkjM+3A+DgDJJ7V8j+IfZ8eunoEwFGV5FDlTJpb/kpDrlpDQTfZVKqj1AoVdw0SsSDsuOtHM9y215VWHAK+x2qr1OczgA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WEyF0Eo2; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WEyF0Eo2" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e321d26b38so29286867b3.2 for ; Thu, 10 Oct 2024 11:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584841; x=1729189641; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=WEyF0Eo2XsJNPDFQCmldw4bvVns8foXPz6GhGq8QjDu8xJDPsgU2gDeg4tMkoGp7DG qpJiXq9ko9g1ZdHZfqLtcRYsvBPZ4zAKl/xQNiiKA25w/AQ8ybJ9Au1awx4vkn8oB+xt YHlrCluf/X2gZpoaDcL86BqHQoGLa8wJKT5byh7tuDXy8KhdxSaUh1uaieJMsHHoehat haDEfLKrjSNVFiwwnRXkhtFTP9jsOA1Q8XsG63RusobtgUY5tamyUlCGShrinJ6gi0cZ iOQEEssjZFClbHoFhivOJECpgHTXb0THhmZzFAYSyJ7PX3K8HOO54iWYJUfENuwbZtqd kRSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584841; x=1729189641; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=gIA4C2cxx7C+412fzOBxJtX9MpxxWB8V+gnKoVMwzzkP0vyqz2D9nvbXdOAbi1C85m dimC4UHuXOZNqDXH6hklSdbDXlqLgFHjc11teKfklormzYpOUjJhObsPfmDQ3EbbMNYE jyQnYDLFwuc3usytTRgIwBJo7ob2GPvkSXS0rvjhxUN1wvDFe94ZLLyg/g4kRZ5wcuke dRW0X8LvdQbUSA5ff/Vh6ITVg1bqKCfDG++SDz7RMXo7eSy+S1w3kVOGeAbbBmcTHddj yCkEZnKI3li12ygl2g+D/8fd44RbeWrDazp/9vn4L9G2yi5lHuLFKiSpzX3LdMj+TRQw v2lA== X-Forwarded-Encrypted: i=1; AJvYcCXpp9bfT9G7t653p1dG+qwbKj/jnQZY/inLS0IodAAw2CUYAKKD2PcFd/tThCwICyuIMab3Q7f15IrW@vger.kernel.org X-Gm-Message-State: AOJu0Yyrsk9jFHZiwzbBz9Lb1W/i4VGBzkk97Eao96cWoxUBkNwdoUyt 0AM75QdvfarSOnfbUaOrRwfPsT4YB0ohoKQOjpP0h5uNJAVYmFXiAbPpL6oKjLUbfnQx1wmeO8o bIQ== X-Google-Smtp-Source: AGHT+IEbamZE/KJDJ4kw63fZFZN0PThnG54xKd82AQ09AwFkx3KIodEPvEqsDBHZyT6DCHkY/GuTD4px7aQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6c05:0:b0:e24:c3eb:ad03 with SMTP id 3f1490d57ef6-e28fe540170mr1289276.10.1728584839383; Thu, 10 Oct 2024 11:27:19 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-71-seanjc@google.com> Subject: [PATCH v13 70/85] KVM: MIPS: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages accessed only in the slow page fault path in order to remove an unnecessary user of kvm_pfn_to_refcounted_page(). Marking pages accessed in the primary MMU during KVM page fault handling isn't harmful, but it's largely pointless and likely a waste of a cycles since the primary MMU will call into KVM via mmu_notifiers when aging pages. I.e. KVM participates in a "pull" model, so there's no need to also "push" updates. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 4da9ce4eb54d..f1e4b618ec6d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -484,8 +484,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm *kvm = vcpu->kvm; gfn_t gfn = gpa >> PAGE_SHIFT; pte_t *ptep; - kvm_pfn_t pfn = 0; /* silence bogus GCC warning */ - bool pfn_valid = false; int ret = 0; spin_lock(&kvm->mmu_lock); @@ -498,12 +496,9 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, } /* Track access to pages marked old */ - if (!pte_young(*ptep)) { + if (!pte_young(*ptep)) set_pte(ptep, pte_mkyoung(*ptep)); - pfn = pte_pfn(*ptep); - pfn_valid = true; - /* call kvm_set_pfn_accessed() after unlock */ - } + if (write_fault && !pte_dirty(*ptep)) { if (!pte_write(*ptep)) { ret = -EFAULT; @@ -512,7 +507,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, /* Track dirtying of writeable pages */ set_pte(ptep, pte_mkdirty(*ptep)); - pfn = pte_pfn(*ptep); mark_page_dirty(kvm, gfn); } @@ -523,8 +517,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, out: spin_unlock(&kvm->mmu_lock); - if (pfn_valid) - kvm_set_pfn_accessed(pfn); return ret; } From patchwork Thu Oct 10 18:24:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830736 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 416B0210C2C for ; Thu, 10 Oct 2024 18:27:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584844; cv=none; b=rLC0cTEcJl8DCRMklZLZZrb9RVgqB681r8V/D+4n91vnT2AtESzRTqm9qgFC6bWnWLULQr+PMmilVXsmkG/kDNH8O6FMgMr+NWsr0+geUvSPsT1rlmlLLud73bIRjEuLWwtu7DpsItW/vbpt0hon6TlSIWQQFomXYiU8LMN8PYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584844; c=relaxed/simple; bh=Q0WfZXD0AwljsCbiw/O0WdULhXfIX1X/By/5X/lP1LI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mQFdC3J/BOE+L/9wgsUTDvRN8TKy5gug4zOrQZ8b585mfEw5g4Bwr6yuxgDRZuya3P+5UZFFpWNECeZCWKg6eCKMX4nP+JkVatHBeJK3GX669XlhrqNzXz6Mz/FQ4EQgE1OUdtdQ7ISnrgsVfepfxJIK5sdp7WZjp+IIFfbe8lQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3qpUkcRq; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3qpUkcRq" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2906efc0eeso1521934276.3 for ; Thu, 10 Oct 2024 11:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584842; x=1729189642; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=3qpUkcRqOzUfCZcj+BZdHa8RALWFF7OtRiu+VQ3N3Sp/jKQU8vYZDxOavhvz5GzCPs 4wMtC27QPkYI4ZZHJTIhsyVCBkQHNDXjn80rQVg5sN8wuj71+RLQ2UFJPOQltkMKo9b0 qmkHu3LJr2xQyjv0+A4khnwPAAKx6T34tSkbj9Gi9Mhug6a0LgiqKq6ggRDbj1I13x3M Y3Gr8fNaFSD6n8ayAIf+4tHcbqgdwwrqOAUwKRd8eJ/bjgCPP4eKndQloqNzhdukKHBr xWjD7W7h8Vs/8C2ceSX6nGlC3uH/sYu6EzXd1UTRGjZgT5NEBX+d23twphqB7dwJBHG7 TZqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584842; x=1729189642; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=SZ5EjGNArDOA66qCP41a8JvRjb/1Hj3O1dQSl3tSIYT8PJO/DVFxK4PQXVXNLIOU5Z z+OmsfyexyFKFXxWdXlvYOq56rQ8AFg27rMjob+e9BIupI1gd43zFJz2KjcP+X+g3gt6 p4Ek05r0ljQK4SdYvHFzStB/Lc1mm38+3gVXloDNzgOLFPVWXqXN7FOE89bWfn99IFUg LqUxpQGBXoEx39bTX74o4qRzM1cGWnQAmjKS61fsGkJC4Ni/MXhrqw/0r8BNZLv7osvU LE0DCpIjrxu7HA+kAu0GsZPZE9Ec2yT/ofPQ+xa/ekL+bqcOKMqcIXXJzPJ3HOtKeyu8 O9mw== X-Forwarded-Encrypted: i=1; AJvYcCURybDxRCfh65vTbftKioYqxwBDsPrbYx9St+cEZsoaX6r/ZKHlzQYFKUlmKdSM1a5Xri/JxictrpGH@vger.kernel.org X-Gm-Message-State: AOJu0YxtJgKEwvwkFbxfCCeZVmPZ1qK48sSEeosZiAQf6+xRqbQv8ip7 sZqocM3FBpB8OBkCoZHeI2GmHiWWnseKOGqDSy8sml8ExlJx2KV31fkFWQHEfIPXxl95YPPdDyO STw== X-Google-Smtp-Source: AGHT+IHhD+Vk6TwFIS5v8fA24SNPRWU4vCcQM+1g+zVw6L+6pfrMpIdoNti+PehtHCeQDGmk8Jps/9lQ5Ek= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d30a:0:b0:e28:fe07:9cc1 with SMTP id 3f1490d57ef6-e28fe4a3a81mr68085276.3.1728584842321; Thu, 10 Oct 2024 11:27:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-72-seanjc@google.com> Subject: [PATCH v13 71/85] KVM: MIPS: Mark "struct page" pfns accessed prior to dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Mark pages accessed before dropping mmu_lock when faulting in guest memory so that MIPS can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index f1e4b618ec6d..69463ab24d97 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -634,10 +634,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (writeable) kvm_set_pfn_dirty(pfn); - - spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); - kvm_set_pfn_accessed(pfn); + + spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; From patchwork Thu Oct 10 18:24:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830737 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B92B021263B for ; Thu, 10 Oct 2024 18:27:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584847; cv=none; b=bxavMg6BIphw8ao/xaDI4u5ztUyAQYrxaHYmrnrEWwY7ex0oKbYH3FZL6nlN8Yd0okr9Kq5mvDlMrYmcB7i59YvwsPVd7TJhk1wVAgF8XPv3dKRWg72+jZEx4QW4wNtxXsfoBmCsdfYsFJ0P/VH5O+WhmC+qKHmQeFwc/tt/DsY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584847; c=relaxed/simple; bh=8dGNSdkZyds89DzrI//tIWXaU4bIHqe2OXPOvN+ci/o=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tZrzNKZmQbfgO+/8Zh6f3DZ6PmqsttLWw6tGTmtfKJB9My7iPkSAD5JHWiU8Q8xOGF3wlabNy1E1csZPzWhI5dlps2HjEOAzZvYCpJ6Pkij9h9ablrUYkeHVhII4sHikY+6LZ6GDB9DlGu7S7nyu+v/M+jbnc+M0ghXo9Aa4Nlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=k5VHsBRz; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="k5VHsBRz" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2913beeddaso635567276.0 for ; Thu, 10 Oct 2024 11:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584845; x=1729189645; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=k5VHsBRzBklUEEZpaBfm9HsLPu/fl+ZxEGgLgBVvUfF80DcEJzwU1QWieKAICkx++X HnbGTPjBrkvT9xzCQ3B93xKLHWRVdhB75JAuDs6CC9Pe6xaL4vhhFHEWoHH5yTLt0a1d oBf0+iPRkwknLrLLzAaQ5NCExSiQNmy4LG3tm9rA8tacQQKxVc665PiVzy5RmN+d1Yh/ UCLExXAcmMKNP3X6wfDNdtvlkBgntQ/wDrzkDIX5K72p0SkBLJWwlADKlJHdd65U++xh VGedCEpb09JKxtyzpX1AqdY2/ITlx8JQ9SeHr2ntFfUShozrMHutwXpY86gBXneT5nyL oVgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584845; x=1729189645; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=VSoj4mf4yPXcLzdAkICMnqCwxTFVqRVFj2YDICUcfYZLNEi8BiK6KQB4XbE3IoqsdA FeWteKo6E2uoTRo4el4yllzbTO+dPYZXqeFQAnnCl2V8cazE6L1skMXkJR7ArgrpnkUC 3iZAY5IJiZ8HTS0lpex+tdC+fqonIDfOKuMVnwVZfPEq32cfRPRSnoB5vXPm4KGPIf6e ezAxDNIwmC22ku4IGxFcPZONX4rjXh6SFpme4uScPNaI5rJY9VS5TOx6DQLB+KhxZrqQ NOaS7AAsVKIMR9VHSKCoQCSiKAUotiIuuul+691Fp1pItE5qtmfXHctol+f7F/H7vwgL ltSw== X-Forwarded-Encrypted: i=1; AJvYcCWn6lg/imuh5LP+o3FKV3zIywK3a96cGvL4tDPp6SYsnXhjx9jDCMwMata3i9wh4Zf5oqqqG3cbQvDA@vger.kernel.org X-Gm-Message-State: AOJu0YyarBY+w61PiEvwPESWU8r2zNyPYNLmV3uSNX+gHJVXDX9sWrHU mFsOvKZBFZw1PpZ0SUQKwnlD2KtA4q8sOh6LPcspBFtuUqJdeiBUhByXc9vRy4S9gJ51yzyexsB 4iA== X-Google-Smtp-Source: AGHT+IHEyKrdGSoNbMWcRNiKhgC/jPgVf/TjauaPCrq7yqhpXyuJQagleroMC0w40O7lZsOojHLr7EwrZqI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6805:0:b0:e20:cfc2:a326 with SMTP id 3f1490d57ef6-e29184333b9mr26276.6.1728584844184; Thu, 10 Oct 2024 11:27:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-73-seanjc@google.com> Subject: [PATCH v13 72/85] KVM: MIPS: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert MIPS to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 69463ab24d97..d2c3b6b41f18 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -557,6 +557,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool writeable; unsigned long prot_bits; unsigned long mmu_seq; + struct page *page; /* Try the fast path to handle old / clean pages */ srcu_idx = srcu_read_lock(&kvm->srcu); @@ -578,7 +579,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, mmu_seq = kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads - * in gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * in kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -590,7 +591,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, smp_rmb(); /* Slow path - ask KVM core whether we can access this GPA */ - pfn = gfn_to_pfn_prot(kvm, gfn, write_fault, &writeable); + pfn = kvm_faultin_pfn(vcpu, gfn, write_fault, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err = -EFAULT; goto out; @@ -602,10 +603,10 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } @@ -632,10 +633,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (out_buddy) *out_buddy = *ptep_buddy(ptep); - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); From patchwork Thu Oct 10 18:24:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830738 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DE091E5019 for ; Thu, 10 Oct 2024 18:27:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584850; cv=none; b=mz9U+7uo1QPrOg+XDLpi5mJLiM+8JfKxA4JjWUju+0ISQwpl4PSzVWkhfZqvlFiPurjesJWkr8dvhOLsv2nQxHGK8bURif6K9kBWCYkDXWbR2APohZq06twT3Eu8pPJPuwUVANrbcMNEoyVvdlkG+WgXNTO0vglhDThxS1uGfhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584850; c=relaxed/simple; bh=hdvy6TBStdDo47jCImbsHkCH8/fTSGPuTNA399ptBEY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OFLFupt/j1io6JF+rgrlFaef8OmOg4fhnfXNyODRfr8oFQLv30+evLYwRTnSwHrUfGB3Va5X0whYD0sdzE0W7QRpzDE2b9krh29BR8o7DG6PkFliBBZ7LBqsD8fHFM2g3mjf4/igjLGf/RxlXZ2i7BBUl6vK8SjXrnfO0SXKJmA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DuDHV6yB; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DuDHV6yB" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7c6a4d99dbdso298150a12.0 for ; Thu, 10 Oct 2024 11:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584848; x=1729189648; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=DuDHV6yBR6R4IF/Tk4tYF2+2idGq/vVgsnDPd/mRNzisey/lK2BmHd3cUo9vBvOANi zkoGepVCuNujuKVT0SvZmDE0XvYHerpurUIIJMAeWc+b57hH47kC0QJ5wHtQEMfJHmPh PpaE4aHKRT1h6TbKpgXWXqnsnto2OGNq0fH2Ws2KY8/MbS7jjFY3wbSKnu7MIzTUrs1G et9W4d1rTvNIgRtfcukOUMCi85rcuo5kTXGS1TQ9auZ4CcSXEEnywcZI9uI5KSy/FVdZ YPQRO6b5YAteoGfO1rbMctMWpre/nlO8BdIZjHegGDTKR2Jshft+Hp1edSWuOGnxTxrC JmMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584848; x=1729189648; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=l/OAZ/kEtjuIzHZuFRwzTEs4EOsiue+hmUeKZ4xMueERuR6OJxeZJ7xqFqNNNDwwM8 WgWUUVggRLqygQqqStq2xf62QyizFa3PNZxhOoESF6DbnbeJnkvP6QyUSVKnk/edXqsn SWN11ABfFXRrcQBbb0AFRlK8KnqUH1gb0B6GwLla+ZUVf/HMG1gxNMaw8RdDzK1/xD+X Gj3BNsAi349T02VhOZC3luDJZRKDXvG6i6tZ7ifhIGqB+gYeIjPLNBRaR4z7oJ+85HWy kdPjnAB1iVoOXZlx0j1mpsFSayBLBMBCkRpTpgAGK/L/EkhdFa0tNrPyGdQo/YHoqvCL dD/g== X-Forwarded-Encrypted: i=1; AJvYcCVHy+rGHoUULTltXlirc5j+v0TMN/rQG9KGDafdYoRh1kDQhglBNfCARORpwwFVzztCt6OsLREDT8vd@vger.kernel.org X-Gm-Message-State: AOJu0YyTMf5s8uqbJwy6PSVE2XIJryu7xCwK3yH8WzhPZnzFZ+DCjE/h 7RUiY3+Cq6cFtUZAl8jqnx5Vn9MXMYzUPWYbnveVkJn+cWhNp7UrbWxDKDjnI1VJjM+uYo3DGHw lmA== X-Google-Smtp-Source: AGHT+IHU1NaHt28BJAfE9+Eihx21gIRaHWD3wJUzNDiwOfeaS4l1n7K6hdxyKTp49hAFCJqVbhS/fDDFwRQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2545:0:b0:7e6:b4dd:fc0d with SMTP id 41be03b00d2f7-7ea535b0baemr20a12.7.1728584846518; Thu, 10 Oct 2024 11:27:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-74-seanjc@google.com> Subject: [PATCH v13 73/85] KVM: PPC: Remove extra get_page() to fix page refcount leak From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Don't manually do get_page() when patching dcbz, as gfn_to_page() gifts the caller a reference. I.e. doing get_page() will leak the page due to not putting all references. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index d7721297b9b6..cd7ab6d85090 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -652,7 +652,6 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) hpage_offset &= ~0xFFFULL; hpage_offset /= 4; - get_page(hpage); page = kmap_atomic(hpage); /* patch dcbz into reserved instruction, so we trap */ From patchwork Thu Oct 10 18:24:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830739 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7F8C1E571E for ; Thu, 10 Oct 2024 18:27:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584852; cv=none; b=crTmmNqJM0i+Lqw2cU209EtJXE0E5S+9KbXdZgWNjG69nivm8rKBzuc62/qE9Wc7IjF+4Z/Okd2Mh0R9639tz7cH/8Kdk9ki10Fyyujw/kg6aU5gngY0Kl7XxCr/uOo+y1Tg0Aof1MueyRDX7ZHdCHD0jbCQYS2nG5FZeqasiug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584852; c=relaxed/simple; bh=69Upu5+wnipQK1zG8Arq0ad1gowf7mjUzNjaillb+fY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iS6DicRl8CkE2NsrcSeA5QpTV82/0cKeUwV8zpMIXwtr/HJHN5CRCdo1QYSPk5llUtPZW/14F7+pSKFjHLUH302tYlqKkyU04XtkahEzHhBWuDS+yzc+c5BPS3ijunEpo0nuxPjqaIh/Ms7fQH80gR68tMnzAzYC2KUpptBbkA0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TYCOxp1c; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TYCOxp1c" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2270a147aso21797347b3.0 for ; Thu, 10 Oct 2024 11:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584850; x=1729189650; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=TYCOxp1cKgcq2S0piBT9YJkDFLpkMk1shhIozeymloTyKk4ZTR+gIawk/AuNbFbDQ4 J8TeUkJfKYxbM2AtDWZ02S3f2cAN7kjr7P+ogU4gcZwc5SJ2dw7hcKjGSiKZnDuWFVXD 0yacIImKBYGg/kOVQBDjgOI19vyelBez+dQ9CKKVnqCCN8KpSEniZAa2O+TAIbBYuk/8 u/EdPzuUpUAPgVZP64a88b75lAGbcbHuAV/J8cWjiNeHeXpsNY+rv2VipBNaNvpPRk0N 02Tdr0mvAbTRtwDHZcGpoGEUwZ3USAPeQZ3Kyha9LYFJVBqjMm+LcYDMRXO6L/g2oyJe dDcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584850; x=1729189650; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=UKLwtd/A30FU01ewZWPRmTuNAt1lQgpt95kgRp2lClyf4v1aSfMx5rXE7LrLUPdbbF V9HQMgSiirCT90srnEJJTji/i//qvKitK1hHBKzm87/haC93O3vSmPRbEeS23P0neMZW ZcwnsrSUb7GR7GF93Ce+UZiyByJEdk2GpzjN5feq5oJ8kmd+65cr2uTG/mv5Oywla7R5 gkbOtm1xwfiLRlMowQxvV6YeLQ2CxpfznodDMEhR4FSxvCGPfoA6WXAEsfddj4kkeIsd cTHEM0lvic/ElFt2vke7wqs5Cfqtpx6gnbFUbZmHvSzD4zjt9TFlbS1RxgQ8MCPCa1Hq cJaw== X-Forwarded-Encrypted: i=1; AJvYcCVqbakb3TyCCjLFYKJssbGf0WazlgSovOZQUZOfI1nFPn3QqEAUKLnoCBK6912WR3W7QTBwit5gMaO5@vger.kernel.org X-Gm-Message-State: AOJu0YzFJU/MpehJbDlQKZkY8/6e74N6wSYf0oKiwphjwnKnIu/7mBl+ XCjKJFvk/gLNlvytAOFiRwFyu4yudtVPhitnSdhaH0UvJJEbk//MCObAgTFkATT+m90jWU0YE0F jPg== X-Google-Smtp-Source: AGHT+IFYivPtvbNeJBK5QJ/MFgPKbd4XQ4GDAEMeTnfD7MuZnOh5ZwxKPEpX2RNQ0xKTZSwXcPzTQuagswA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:f043:0:b0:6db:c6eb:bae9 with SMTP id 00721157ae682-6e3220dbc31mr798327b3.2.1728584849896; Thu, 10 Oct 2024 11:27:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-75-seanjc@google.com> Subject: [PATCH v13 74/85] KVM: PPC: Use kvm_vcpu_map() to map guest memory to patch dcbz instructions From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use kvm_vcpu_map() when patching dcbz in guest memory, as a regular GUP isn't technically sufficient when writing to data in the target pages. As per Documentation/core-api/pin_user_pages.rst: Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() As a happy bonus, using kvm_vcpu_{,un}map() takes care of creating a mapping and marking the page dirty. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index cd7ab6d85090..83bcdc80ce51 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -639,28 +639,27 @@ static void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, u32 pvr) */ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) { - struct page *hpage; + struct kvm_host_map map; u64 hpage_offset; u32 *page; - int i; + int i, r; - hpage = gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (!hpage) + r = kvm_vcpu_map(vcpu, pte->raddr >> PAGE_SHIFT, &map); + if (r) return; hpage_offset = pte->raddr & ~PAGE_MASK; hpage_offset &= ~0xFFFULL; hpage_offset /= 4; - page = kmap_atomic(hpage); + page = map.hva; /* patch dcbz into reserved instruction, so we trap */ for (i=hpage_offset; i < hpage_offset + (HW_PAGE_SIZE / 4); i++) if ((be32_to_cpu(page[i]) & 0xff0007ff) == INS_DCBZ) page[i] &= cpu_to_be32(0xfffffff7); - kunmap_atomic(page); - put_page(hpage); + kvm_vcpu_unmap(vcpu, &map); } static bool kvmppc_visible_gpa(struct kvm_vcpu *vcpu, gpa_t gpa) From patchwork Thu Oct 10 18:24:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830740 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FF14217332 for ; Thu, 10 Oct 2024 18:27:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584853; cv=none; b=b0pYlCmbiZ2j3poNVmc26cKgnaZX8l5PC0kdFy5esvMIU5ok2zvCRS/jwvp7HGvIBe3RK1oKuzjAfqFCe72qMl5lK30TyN1+jFOygdsTonnhucqrQnmnv0s9MpSOtF5w2pGx4xEkK/dElxF4Y+cEztzBUkMATYpaTDVofg7rhzo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584853; c=relaxed/simple; bh=6VFC6ZQ3q9XjUbMWhh1JwxOVsy5FNZVXQFKK9OKSZCE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=raM/Wy5zWfZPTi/8+lTWw+kYLGtT+usVoxdh4KoRyaNvu+OUQgYdDi56rbYRiHQpTwhmka+mEqi7D8j7cKiXYqLT1oMmK8dyjOzUpIlLOPcRZNtuQX8rP5b5YJrjdGNNgFN28Dhrxi7lYPHyzYQarT0+XC5fRxto4sEh3lDsZdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kkhtwVA3; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kkhtwVA3" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e123d3d22so1429798b3a.3 for ; Thu, 10 Oct 2024 11:27:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584852; x=1729189652; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=kkhtwVA371S5ZHAeHS4k3XRilYdB/vGzQgltlsRLi0pp7zBjZNHvtNTHMuvtzL7CAF 6lNYOD3BDWvBSZ9L0rmgwIlueXDyhW/RA5LQ0k9HKGrI7aQrISXXF0UVGv1X7QGYXfmX nQAjpUZF9ow0QYSRFnzZERqDxBCGJILWB6bbaaxW+qU5NHSMCV90Jig9BhxHgjAoV1za TIvRad+uoWgS/qyZB0Ga3oI/+RuGbJNOcXT3YNU2DEx7phcpFbVb8mMgls/S1lDHVstd CsKALT5sXHk1JsYmc6mHfOyTLTtTo+ZkddLgoeZS9J57GTp1PUXVbgGiI+DyyF4UdFku wG3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584852; x=1729189652; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=ufyy4LJ+l8RY60ygY+UBMB1t7nnLPt/I+Zgst/w1o1yqPiFYzMbQlzZBmca8+Jo/X6 8ym5aqtT2Vwzrj7JadZGN5/L1M3taNIPwERriC6Kdb7rQAgRJhOOAoZ/jPBrJWWzhGvv iQPc4AUR9bc4v756xStFQ3Z958ymVGauoaLWBRhJ7kOKo0vylin2V7r0umn2MaMR5/5o Lc0BH3nwlP9ss3MfCOjxVJFeGlDGA7jqKdGfAI8v+vlHFvm0gA04lNbmpFBdvsdRT16W iV8CAVNoPWN+2hcYct/DWcOZ/kU9EU7VxqICRAkvK4Q9fricZB8TFIN57RBANk7EIGCp Aacw== X-Forwarded-Encrypted: i=1; AJvYcCXq+SLrvPjZgxvJZoJntNyUMk5kl3WnW6eYNx1SfIhODmXCSNNkxrF8WuiEhYN9FHz34BI8hDtL7vlf@vger.kernel.org X-Gm-Message-State: AOJu0YylluD2j0vbqK1RrnG+2M3InrVFemJsYwS0Jszbg/14I45hsba2 05ydqBHE0Grb62Tas7CeAnQTH3vyYONtHPYJJHWLlP+jGah34iywPM7U9g+nS+5EnBjYZYMJ0PX idw== X-Google-Smtp-Source: AGHT+IHVUYgAS8Gi1OfHaumBjOVs/68U//Xc3NGlK2wEye9l2w1SuCdDQ/WQfsMbBzqgGqsCESzcy3AUxv4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:93b4:0:b0:71d:f64d:712c with SMTP id d2e1a72fcca58-71e1db65a16mr7365b3a.1.1728584851459; Thu, 10 Oct 2024 11:27:31 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-76-seanjc@google.com> Subject: [PATCH v13 75/85] KVM: Convert gfn_to_page() to use kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Convert gfn_to_page() to the new kvm_follow_pfn() internal API, which will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 696d5e429b3e..1782242a4800 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3145,14 +3145,16 @@ EXPORT_SYMBOL_GPL(kvm_prefetch_pages); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - kvm_pfn_t pfn; + struct page *refcounted_page = NULL; + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(kvm, gfn), + .gfn = gfn, + .flags = FOLL_WRITE, + .refcounted_page = &refcounted_page, + }; - pfn = gfn_to_pfn(kvm, gfn); - - if (is_error_noslot_pfn(pfn)) - return NULL; - - return kvm_pfn_to_refcounted_page(pfn); + (void)kvm_follow_pfn(&kfp); + return refcounted_page; } EXPORT_SYMBOL_GPL(gfn_to_page); From patchwork Thu Oct 10 18:24:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830741 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3E64218D63 for ; Thu, 10 Oct 2024 18:27:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584856; cv=none; b=m5iiOgtYTGPjgvos6Xj3ccr2oUjHvJsLiIspoHUi3yYVHD5QTmwxTrU6eWS1mbYi3p36Opc5dJ54mvSVQCCiwPnLsFGh4i7wZz/nIt5AJNtyNge/mAdYi4m497M+1Ly+ph1W7Bmo3sdzVAD1eMJiCXrpGvot4fNciw3o+ougSVI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584856; c=relaxed/simple; bh=h3dpe04J9rq+dGEj5vV4pEUoAEcWlYwK7aXFrp+Hc5M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WmGbVW7yfDeQEUoxkhvINypdTO5nxZdmwkzofJ6M5BXZtp39bRFQkJ6zuEYsNL0KV0bZuu1cQBPb4/XfP8VUTe51BDqUlTaa/IQQs3cDTONWVqIjIzRTtEZ3qhWmBmGZAeCFP3hRGBQVskY4TEGiVOejpNOQGOaiVLY5wSfArEA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qpMT9mNV; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qpMT9mNV" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e28d223794so23915707b3.0 for ; Thu, 10 Oct 2024 11:27:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584854; x=1729189654; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=qpMT9mNVMlM3JYWe1f9UU6REj2+oc4kj/yK50HSO4pjHutnOBh7JNWul7tfhz3o4gh +vruI0VZ+Fq6kyfhQqfEBqSUimUY7mr4tcemk0pyokvcHPV3ClhMsBLLWdPlziIY/gZr qbn/UhT5Hr+z4AUeczlu7097wCvqY9n+PJbB4r96PtwCFdRgaEAiAjYU0lZuAW3qFAjq oJfimXxrRFyYcyH8hc0hpxmWiRZ23C/Pk0qrQf5Tx+NbiOGiHhn+vpApay6F/DbuQk85 z2HtIRTfoM/9gXAmhP050j35DEG0wsNyQRzI2Y5vZAgVPQ0yrJEQ+eN7pVysywTjL1vy 1DuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584854; x=1729189654; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=JMHWti5jJY9jj/2VrqtfYG6HjGzMbNTD/Ea77mVKNBvAeqUfc2SnFvSCDgkTKbw7gk DjfK8amf5JLT9P7S+LkIL2/ErLgHvoNumVEEgKpFgNdV4OrfUxEaStz7664TTKScC1WH jVuHT71l9SCkOkaxFyGNX3TZqgt0Xegk4EPbyvZWFlcHedYZkD9HphOIoigfrXvyvwhV YBF1eetFco5anS2z2+OsgLc8c/r7KG/fXJj54+9SHOjEJy+2/O7XDedM/03cnaqraD8I sZV098LCfFLKcLNdcZTl+ybRwuVjnDc84WfeZQTbrDSdtfg09qJCbbaOrkj+lUBrWg2c pcOg== X-Forwarded-Encrypted: i=1; AJvYcCVHNcF10T1xX2OG6/wF9ih2X4IYyAjlmfFH9ukzNWlarx6ohHvjvD7XkHbBtSNhf9PgjYqKW5rvyg3S@vger.kernel.org X-Gm-Message-State: AOJu0Yx+hV/JV6D13GY767D5V43VbuBMBG/+hx0Y0qQgKGeEULs+UoFk b4vQVpilpheX8krdv8x/ZiYCdCnBTcNw01D18VjI7rUCdQS5+EjXWzzypDZ/HeilcD6HXPPyWBS 2Sw== X-Google-Smtp-Source: AGHT+IGkMjnJcd2oWHVLgC5xfQJb4mHLQjnUvb6XgW9ZV8VH9lzrMuhcRfl3e0TBTVqXdJBO9V/Z2ttoh2Y= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4091:b0:64a:e220:bfb5 with SMTP id 00721157ae682-6e32213fbacmr192077b3.1.1728584853563; Thu, 10 Oct 2024 11:27:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-77-seanjc@google.com> Subject: [PATCH v13 76/85] KVM: Add support for read-only usage of gfn_to_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Rework gfn_to_page() to support read-only accesses so that it can be used by arm64 to get MTE tags out of guest memory. Opportunistically rewrite the comment to be even more stern about using gfn_to_page(), as there are very few scenarios where requiring a struct page is actually the right thing to do (though there are such scenarios). Add a FIXME to call out that KVM probably should be pinning pages, not just getting pages. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 7 ++++++- virt/kvm/kvm_main.c | 15 ++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9f7682ece4a1..af928b59b2ab 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1213,7 +1213,12 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm, int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages); -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write); +static inline struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +{ + return __gfn_to_page(kvm, gfn, true); +} + unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1782242a4800..8f8b2cd01189 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3138,25 +3138,26 @@ int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, EXPORT_SYMBOL_GPL(kvm_prefetch_pages); /* - * Do not use this helper unless you are absolutely certain the gfn _must_ be - * backed by 'struct page'. A valid example is if the backing memslot is - * controlled by KVM. Note, if the returned page is valid, it's refcount has - * been elevated by gfn_to_pfn(). + * Don't use this API unless you are absolutely, positively certain that KVM + * needs to get a struct page, e.g. to pin the page for firmware DMA. + * + * FIXME: Users of this API likely need to FOLL_PIN the page, not just elevate + * its refcount. */ -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write) { struct page *refcounted_page = NULL; struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(kvm, gfn), .gfn = gfn, - .flags = FOLL_WRITE, + .flags = write ? FOLL_WRITE : 0, .refcounted_page = &refcounted_page, }; (void)kvm_follow_pfn(&kfp); return refcounted_page; } -EXPORT_SYMBOL_GPL(gfn_to_page); +EXPORT_SYMBOL_GPL(__gfn_to_page); int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, bool writable) From patchwork Thu Oct 10 18:24:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830742 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53344218D8D for ; Thu, 10 Oct 2024 18:27:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584857; cv=none; b=qkiGD/+Slwp5INap0S7yA8Jb66f0n6TqcVyQaoAnwb5ZBxV+IwoSRIDJo19iD826N2Mibr7mKVTkOvOSAZo9yEnS1ztEqbvc+b2ItD2TLWPiRM7q7Rp9WKxWRYvbt+tjTUyG1JWS/JWot9LP6kluFyC1zER+5JenX3H3KnX5AKI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584857; c=relaxed/simple; bh=FErwCaq8NdcoMJPhpWM/50ZCDjLg7qNPTjMbO8/YuvM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ml2CcDCsxePFOj40sLvTuJo/I+EMBo2oPqxAj3L90dOb2oUs0j5U2RsaL+necUUJaIDULrJdA1FJuUCdlvG8mIO5rnUOidpVfRpDhBULjH3haX7I+e0eevm/t3xawCM9b2g/6H4niV/dKhEG0cCwO4/Uuzsud4yZk9vaAuJ7mZ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yE9BBZab; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yE9BBZab" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7e9fe0a21a7so1029416a12.3 for ; Thu, 10 Oct 2024 11:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584856; x=1729189656; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=yE9BBZabvoeIqzV8CVJHKuzxLfL2NYkYVtIdFaPw/BUSfPofvy1yEt9sV7egKYCr6h ECdzuEmt/3ij7vj98XqRqiNePbTZy3r1+KnoPq0Y9I3SWy9nfiSrfM3+HN5Tt1cuMsMe P5BqP8mEprGZtOyicY4QMcSjOoZuBJSucYO3xKIILjywU8h/g9JRWiGzdULPCY/h7RrW gs/lx0IgmUkEs0f46fCv0eA4nTg5+x1s07DF9AqCC/d91Tc45agqIYRUIW48yJgViOz+ 6RSp5Afamp+TyeZzsa3QwnWcskOkhHtKwtjtk2jBDyPhGpbSYoTMcCOFE9DIiSVIltMb h9yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584856; x=1729189656; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=WmwQPwKubqXXrASZKNCoQVnliM+FmTm0mJjfPbSYaxjkj+33lx4oKmCAokzlviqyoe X8eiTeM12RLFQ8XMKPCdAoFGrkJ3lXu3/qRTJ1oJ7hYLF8Z+UXHSaD0IrAXovNbEPCwm v99sdmWKvVV7SW33IxK8UC0Y6dW9QJiVEUG81Ws3QG6fCy7Z9FfLxbz04tYwRgQmt7vz iCusbIoj/1YUPUL2rJprWLdJQJu/VYVjrMrWQwF9+WKYooTT4nJB0JU5rvzgIDX/QCFv XF7H2Qz1mFcz9WPcvPVIwppNOEcEoAu/rIeO+DOT4+A3ouvlNTjCAcyKBoHWrCSrWeBp XnCg== X-Forwarded-Encrypted: i=1; AJvYcCVajwC7yGehds9sGuIlG8B31ZLXTsiW1p0LEOeLyz1H8jkU7cvzt5X4ojvAgeK8LnDvkwcuIgdeI6tW@vger.kernel.org X-Gm-Message-State: AOJu0Yz31AVB34KCToTDleCtGwNyn5zGkbsdSSauH1rszwCFHnwdrNLq WD20mZF2HFpgu5lAowEPrUytAk6RhnR+9LhxMQ43JjoStQEod/T7PjUzGXqYpl/c+DHJ4CInMAR 0SA== X-Google-Smtp-Source: AGHT+IHGiwNd3UbnZUzPre1tk98dNYoM10q7gFGG4lq0PldsJuDm1rxMHI+g8Yc3UsVnmkzWnOurUMFmqIQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:41c4:b0:20c:8f78:67c9 with SMTP id d9443c01a7336-20c8f786946mr26385ad.5.1728584855618; Thu, 10 Oct 2024 11:27:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-78-seanjc@google.com> Subject: [PATCH v13 77/85] KVM: arm64: Use __gfn_to_page() when copying MTE tags to/from userspace From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use __gfn_to_page() instead when copying MTE tags between guest and userspace. This will eventually allow removing gfn_to_pfn_prot(), gfn_to_pfn(), kvm_pfn_to_refcounted_page(), and related APIs. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/guest.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 962f985977c2..4cd7ffa76794 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -1051,20 +1051,18 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, } while (length > 0) { - kvm_pfn_t pfn = gfn_to_pfn_prot(kvm, gfn, write, NULL); + struct page *page = __gfn_to_page(kvm, gfn, write); void *maddr; unsigned long num_tags; - struct page *page; - if (is_error_noslot_pfn(pfn)) { - ret = -EFAULT; - goto out; - } - - page = pfn_to_online_page(pfn); if (!page) { + ret = -EFAULT; + goto out; + } + + if (!pfn_to_online_page(page_to_pfn(page))) { /* Reject ZONE_DEVICE memory */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); ret = -EFAULT; goto out; } @@ -1078,7 +1076,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, /* No tags in memory, so write zeros */ num_tags = MTE_GRANULES_PER_PAGE - clear_user(tags, MTE_GRANULES_PER_PAGE); - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } else { /* * Only locking to serialise with a concurrent @@ -1093,8 +1091,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, if (num_tags != MTE_GRANULES_PER_PAGE) mte_clear_page_tags(maddr); set_page_mte_tagged(page); - - kvm_release_pfn_dirty(pfn); + kvm_release_page_dirty(page); } if (num_tags != MTE_GRANULES_PER_PAGE) { From patchwork Thu Oct 10 18:24:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830743 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DBD021949C for ; Thu, 10 Oct 2024 18:27:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584859; cv=none; b=pvLUQ2KPNpF/cHCOAC6kr7gqUEKsgA5IEHxfhKVP3YQiNPGb9Fqrgs2+w6nrkarRnUrMUxMum1KxVQALERHtuO5JXyN/VSnpo5nlAFb+Qsp/GhKHXNa4SSF24+KJ41l13z13pQVSJhlZDif8pxx1HzdMQqhbV2iE66ZdCT9QyTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584859; c=relaxed/simple; bh=WU2bYG9iOpd2mUwv0Kp3QHVdfdxFDeft8vY+L5Knm2Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=o88o0jCWjO3OnwxBTwVKJgpyjWjXA7m5EweBvAT0tbIQtuTisjvsIztapnUS1uziy6IFsgR9eZTmREFy0bG5E5fH7KnObgoWrbmq2L+jEfiTh5kstDCD/4W4rxvxrBMAnZtWGQC43d1zOuPImJzAVeZZnioGYTe9n+QPsFnEwkI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rVwSkyMU; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rVwSkyMU" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-778702b9f8fso1040811a12.1 for ; Thu, 10 Oct 2024 11:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584858; x=1729189658; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=rVwSkyMURh+jnY5mhdfmEcQccS8IDp53PkuungLNC+QuxTGyFwJnXYuV4KpMgc91FZ bFvRWLz6gCAuL8I4pFdUmjoij8nlXt+RkgsWY/Ra7wXd/1Wau2HIby3umw7iSRAB2EmN ut+5qUUcerlx01e/0fA02Swap/VPizSrvEggtnZUOJ3WrTusH7oqshsyTmbh3f+dI20n uyRVKZBCTZawUdXzMhX2Ok6XvgFbIG0mVs+37d+uWqG7tQt6n6PSxhsbtHnD6bpftV5j rLI2JXT2gaMRXtSRAfpKWNB6WrSgK1uzbqCfQX+rH4CoQXpbwXxDDS5Cl51S6vCpTwBp n5Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584858; x=1729189658; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=ejkjH8jXkIsEPFRVGjFtEOOSaRQaUoGe7wDYdeJM9Vc/QlX1+vNAHnNr7APGuANMZX bWg0Zq/jXcToNC+lIkqS7qWfg2BCrsGi/U5ojBHX5qsr5TseQxA1Iv91n1ajmkb/f69a /jCEEMPyvHHZ6yXBTc8P6NbFwDqGtm7ouTXukUoNl1anMs+MCbqsvSJWBwQdpU9z8koS qUpv6FYTOzrTKaLC6BB2112c5UjI7wpNGxA7msCQTWYid9amsSuQJDCXJBT8h9G/YjN/ c6AbNpLVo8K0aIb+AxJJrSx03g2gf2z6gX4/Ch1+j6IoimhI9Yzqgg2oROc8z9YHojW6 DXgw== X-Forwarded-Encrypted: i=1; AJvYcCV6wY/yOUKuisB0DuTuktfsQQDnZsE8ehi4dR57u369Umk/x1k4blhcauFgy3gijOvLDwU+UaIzr+Qe@vger.kernel.org X-Gm-Message-State: AOJu0YzvCXqpdY7F6Eo765LhitAtUCF+BUmpxQ1ACrBafvkg/DjYwku6 aVpp4o4MCXKxb4Zi9HRacFMjqbyVUSo0S680zJews/9PCiX6PdKkSvIwH0bQvP1pYtcOH1X3v+e Y2Q== X-Google-Smtp-Source: AGHT+IGjs5y0lK7jvR8/zbNWJNDHfDWRUWSPa98vwOnU2bZ6sNQJgoH/6j7HDiYxyC+lHu033MhghQMujcY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:b208:0:b0:7e6:c3f2:24c2 with SMTP id 41be03b00d2f7-7ea535dce6cmr30a12.6.1728584857741; Thu, 10 Oct 2024 11:27:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-79-seanjc@google.com> Subject: [PATCH v13 78/85] KVM: PPC: Explicitly require struct page memory for Ultravisor sharing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Explicitly require "struct page" memory when sharing memory between guest and host via an Ultravisor. Given the number of pfn_to_page() calls in the code, it's safe to assume that KVM already requires that the pfn returned by gfn_to_pfn() is backed by struct page, i.e. this is likely a bug fix, not a reduction in KVM capabilities. Switching to gfn_to_page() will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_hv_uvmem.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c index 92f33115144b..3a6592a31a10 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -879,9 +879,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, { int ret = H_PARAMETER; - struct page *uvmem_page; + struct page *page, *uvmem_page; struct kvmppc_uvmem_page_pvt *pvt; - unsigned long pfn; unsigned long gfn = gpa >> page_shift; int srcu_idx; unsigned long uvmem_pfn; @@ -901,8 +900,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, retry: mutex_unlock(&kvm->arch.uvmem_lock); - pfn = gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page = gfn_to_page(kvm, gfn); + if (!page) goto out; mutex_lock(&kvm->arch.uvmem_lock); @@ -911,16 +910,16 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; pvt->remove_gfn = false; /* it continues to be a valid GFN */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } - if (!uv_page_in(kvm->arch.lpid, pfn << page_shift, gpa, 0, + if (!uv_page_in(kvm->arch.lpid, page_to_pfn(page) << page_shift, gpa, 0, page_shift)) { kvmppc_gfn_shared(gfn, kvm); ret = H_SUCCESS; } - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); @@ -1083,21 +1082,21 @@ kvmppc_h_svm_page_out(struct kvm *kvm, unsigned long gpa, int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) { - unsigned long pfn; + struct page *page; int ret = U_SUCCESS; - pfn = gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page = gfn_to_page(kvm, gfn); + if (!page) return -EFAULT; mutex_lock(&kvm->arch.uvmem_lock); if (kvmppc_gfn_is_uvmem_pfn(gfn, kvm, NULL)) goto out; - ret = uv_page_in(kvm->arch.lpid, pfn << PAGE_SHIFT, gfn << PAGE_SHIFT, - 0, PAGE_SHIFT); + ret = uv_page_in(kvm->arch.lpid, page_to_pfn(page) << PAGE_SHIFT, + gfn << PAGE_SHIFT, 0, PAGE_SHIFT); out: - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); return (ret == U_SUCCESS) ? RESUME_GUEST : -EFAULT; } From patchwork Thu Oct 10 18:24:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830744 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28A37219CAC for ; Thu, 10 Oct 2024 18:27:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584862; cv=none; b=cyeuYxzynZF8Wr2KlZYHL07MGESfmUq3qHWdOHiKrywFBnzqn0Yjnpuy3Bl74VwDMN2YjAmkXdB+WyrU07gHTY3++fGDzSbJyiJTZY8BW74Sfcx8yC76yT0iyRxf+f+UTw3hodeAKgFF8Wf0gc0VO9kXmv36SqM76AhbepBbT7c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584862; c=relaxed/simple; bh=5AMkvnsvbpt8lQB2/GwTipD8a+sP1HpfiWdIDM6I+Qg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=k0+uSYkcjIWBXBN9SBuQjErd8riRu8NAz4wwUuIM633rGPFLuT8VkSrSMvdlXAiH+le3u5wwtH1FcqBRK1sdoCbuSpjizws/XksJ30e4AtuaixOdjRhRnmO1EhlYYrJTZsc1C+9P2kH7RxLwiR3Qlcv3f76WJylKei7fVMB54OQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=N38MzbV0; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="N38MzbV0" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e28d223794so23918177b3.0 for ; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584860; x=1729189660; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=N38MzbV0Opg6YS04YGXSDcFzpDbMb+69BxLeF5I/m5HWLMmyQi0t6ZpoTakUKsxG+Y x+62Qe6FkM5MXIYUqzR7ZuT5vHKJWIzNuNGguoPNHzQzRheqSS6wmdrvwE+E+IZ22h0E s9iPV/7n2nG5SP0RkA+0+K0w1YN5FwNO8br3GqOTRvzCCC4t0JdSc920HAAUmDRsQA+Y PwuIw/p9M4ndYcmO76PWt2AyRJmaA5G/ms6keQLf2kLMT2JU3Pns4jgtRATr7ImtkXkA jP4B02S0wLHqIbb3wVDpdw/3eyHLn7HPlSOgctNgsdNUTRNyJWcnfufIx/x/sAAhixZ3 RTkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584860; x=1729189660; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=N1sOeqIKGIyKPAlGrK/O5atUCjYvSQXKlEH5DjDLbmpYinaagee6yKhRuE6IIYEhLQ aKAyNcwp7+Zi3ZuEXspLmuBvTLPrpDvZkHupa+Xr1l34vo1Tccb4AxjMlFPvS/5OxEB/ 6DCZ42kG1VQckcW1LgpY6kQtuW5QXhkxUxkNGEwubaBPOFdtijKjCjpXKFrhmk4TtSa1 RnfaZWAOjmvqf+N4QdoUz4Z79sr6aXk1lAPT4EnQKk0xhj3T9rNUzzgAeKxtdKfPGWKJ AXTZ3OPbdGP0Kh4ntby72Tr0lzqFSq5mtJTEhvIAz+58zN2DSvN0NGtikgd21cL7YkY1 sXRA== X-Forwarded-Encrypted: i=1; AJvYcCVVb6sxDKH5Vajt6ErO+8NBg4x3r7bpWS5jNSGW9nP3lyhSzT3wtq7NHkYHjQsmZpbkNaCC9j/9C71H@vger.kernel.org X-Gm-Message-State: AOJu0Yy+1LAZ4eUf68ZU/k7VD3XT7/JboWN7tBBu4RKGD5oNDDg8W2oN bK5+wGKZGUM3Cto0mhXMV05XPUNA1JUzeToFk/gGKIeotPxrtsFQ5lNr2HSz1Oi8w3R4exheicE 2Ow== X-Google-Smtp-Source: AGHT+IGvHgo281GUU728l2kpqfRNdoQ8M4AtWxtMdwK+Sbd/Nmo7aGxeBXrCmxWS+HWzNnhFIgaVqhxkocM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:f20e:0:b0:e11:5da7:337 with SMTP id 3f1490d57ef6-e28fe3505c2mr129032276.3.1728584859805; Thu, 10 Oct 2024 11:27:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-80-seanjc@google.com> Subject: [PATCH v13 79/85] KVM: Drop gfn_to_pfn() APIs now that all users are gone From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Drop gfn_to_pfn() and all its variants now that all users are gone. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 11 -------- virt/kvm/kvm_main.c | 59 ---------------------------------------- 2 files changed, 70 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index af928b59b2ab..4a1eaa40a215 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,14 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable); -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable); - void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); @@ -1356,9 +1348,6 @@ unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); - int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, bool writable); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8f8b2cd01189..b2c8d429442d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3039,65 +3039,6 @@ static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) return hva_to_pfn(kfp); } -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable) -{ - struct kvm_follow_pfn kfp = { - .slot = slot, - .gfn = gfn, - .map_writable = writable, - }; - - if (write_fault) - kfp.flags |= FOLL_WRITE; - if (no_wait) - kfp.flags |= FOLL_NOWAIT; - if (interruptible) - kfp.flags |= FOLL_INTERRUPTIBLE; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable) -{ - struct kvm_follow_pfn kfp = { - .slot = gfn_to_memslot(kvm, gfn), - .gfn = gfn, - .flags = write_fault ? FOLL_WRITE : 0, - .map_writable = writable, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); - -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - struct kvm_follow_pfn kfp = { - .slot = slot, - .gfn = gfn, - .flags = FOLL_WRITE, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) -{ - return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); - kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page) From patchwork Thu Oct 10 18:24:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830745 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCE5721A6E1 for ; Thu, 10 Oct 2024 18:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584864; cv=none; b=s6WwC5AXfnusGaD3kxhbsYazzMxWO44vbZfEkNe3yiC7UE+031M1mEgd6nd4A1QhU3NGUyNKSBIXKL2QkluRUTdUUim8q8mdd/g2pGa2h4Tu7jq02PtVVjCWZXVFT7jeuVMkRmANccW6Vf/Axqu84y/4K/2VKpROHJDYLdCjXIc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584864; c=relaxed/simple; bh=Xp30Q5aCXuVbCuroW5D/p68jGCOUTPuU5mMuuixoyAE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=puIlmGdGz8TejKviMl8c/ZAlzHZhOZ/dFXaf8XeQZOaHf/KbY8v10wXbWTW/1iMsyRk1PXWcnwzV6VlRg/x/slbt8p4NhsEDjFqIhPg4Iuxw2OqtRPH3bdR9YAyNrHhdJIazuDZFQFWVBRRgD+XAz6Fusfwb+enzvH30tA7+Fbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TC4r7pK0; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TC4r7pK0" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71dfeda9ac0so1340110b3a.1 for ; Thu, 10 Oct 2024 11:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584862; x=1729189662; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=TC4r7pK0g3B0mqcPz8ZXxRS8Rw8BfJUEjGW4S/zxeQ0UkyLNagAVzGBjTNdiDb2SpF nBffEhUIrykBpiBbRrDSiRgwT5seeVZBDRzQyYetSIdUzr6J3plmmVwFPQ6RY8jHlo2a /bzTNyfYCg7FOcuV8L8aRSCfF0OxPZH0J98kU8yMIJP4RIJQCk+K5i6C+TXRFdGZ6eYI vpEeOfusRR2SHfTSTJqQIXxoxJLlil/pU/N0igi2UlS4qu3p4+hgRBxyMsiC7c3vwh79 jWUX53zTsO2dJgzKEQNNHmt+hOKbbgUbybt44zCMmnLOdc8l2Lu5JrNAilAaQXl0kghC 1L/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584862; x=1729189662; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=JM9IWVzA6dub2vP984qk/drm4Vspx3P0K52NoLVb6PtSBRe1QaMU3uD/VeSFiClWhl ECjC/h6F5FxDQM10EJz3+BVR3xGmHFEqwLBwV9XexZLe7iAhN1DCJL6CSgzWuyVaMGVn qYFEcx1TKft2LbhM3Jl+/xfevEyYBde39T9s/iovhThwkMbdtNcu+YHkkg69qsyQKwzJ SEwjQAXDtRp/pHZj/XS2HSnzZ1XLdiimiowKSFxrTxSqnWOGZjLYZAZnkJ1JFX/RpJqS vYykuXQf3BNmntda9dbFHfJd4jeu38m2RgvYOAOKzax6mj7mlhjl3RVNhESRJvvI6qht hnfA== X-Forwarded-Encrypted: i=1; AJvYcCWMhhgYZZEnDmhnCDiiiMyogbFSud5dEG+dYO9Wg72RD7jFDpkWX/J+c5H/dxlfwJS2mTPcXDEsMZDW@vger.kernel.org X-Gm-Message-State: AOJu0YytIkjYJDa/RaEq93/EJW/tCjlEp64VJd3vKb1ptbXc47CYTxiw 0UiDU4iTEkEPQbt0HDf/eGL1t2j9FskoC8ZV63N4QAohZINkXwxfJz9yaWbrWqDL63h1hRI/gPt PmQ== X-Google-Smtp-Source: AGHT+IFRVsKTyvnlBUrQtNNg8zvYT0OU1qw5Av8a7Gq/P6vdrCxit4uLl8sEcLrSKXXTWA/vla3myJTtPpY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9443:b0:71d:ff10:7c4 with SMTP id d2e1a72fcca58-71e1dbd1d0fmr15196b3a.4.1728584861919; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-81-seanjc@google.com> Subject: [PATCH v13 80/85] KVM: s390: Use kvm_release_page_dirty() to unpin "struct page" memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Use kvm_release_page_dirty() when unpinning guest pages, as the pfn was retrieved via pin_guest_page(), i.e. is guaranteed to be backed by struct page memory. This will allow dropping kvm_release_pfn_dirty() and friends. Signed-off-by: Sean Christopherson --- arch/s390/kvm/vsie.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 763a070f5955..e1fdf83879cf 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -670,7 +670,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa) /* Unpins a page previously pinned via pin_guest_page, marking it as dirty. */ static void unpin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t hpa) { - kvm_release_pfn_dirty(hpa >> PAGE_SHIFT); + kvm_release_page_dirty(pfn_to_page(hpa >> PAGE_SHIFT)); /* mark the page always as dirty for migration */ mark_page_dirty(kvm, gpa_to_gfn(gpa)); } From patchwork Thu Oct 10 18:24:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830746 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E671321A715 for ; Thu, 10 Oct 2024 18:27:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584867; cv=none; b=CaYoQnzDMB7lBdJIdeJMkXkm5F0jPIDRRigNzOQACvqojLa4h85diSoakAu3TSWc6r/ZonTkLETG9KKHi7nghYniAaZNF5QIjUCmMzX6W48TcKDkA/PAH44hAhNzwozLvhzZSJULr2F+BM83iuv6TfQmlGeYOtLXfJ5TRVadIu0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584867; c=relaxed/simple; bh=Ioj1P42KohQLhw3WPnqB6EBTWlPlGqFgp6J41ckJwc0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LVWEEe16N6qm/EJ3ZXm9r3BhS3MfSgBo5jWcKtoFIYzN5/qpcJqbUsnjj120zgmcOBsXwBxpwK2lpqXkiIcSNJwallKB0jWiJC5Pbps6zvbDfYmFrm+7H+7GabKQYbUJRt1WrqaOIymG9EmKaIgdQY1I6xD/6wr4n+7GyTelt88= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4fg+oFS9; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4fg+oFS9" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e02b5792baaso1678028276.2 for ; Thu, 10 Oct 2024 11:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584864; x=1729189664; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=4fg+oFS9HQ6bkDg2/cCK4u+yUaY2HdLlUAKKyU3XiR5S4aFSzAjEWkOGXTGjmr5ePX vGkI99yd3NqFEQL5q+JkKGytg2VixnQYq4PGFlA34nG7QkyPIj6PJ0ZlrIWRk8AoR/X0 dosEXhI0H3WZr2UmtQAb75cyDhtq3eS64wMYjKmvudbvRsnzH7xd9K0hwQjEJYZEEdS5 S5ioD1i4hPY0tOXwvT+EQOL0urSQ1tchIC5V6FTIv7IWcFTlAadalixlcVZHj01gXoru OYG5pbvbDCYSnXyeBnRIHws5VAd7XO3tCVjhRsUtK87ywGe0CzqKMCGctDzPKDvNGGTs 7TYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584864; x=1729189664; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=P5uTeV0PscWqXOlJCsLbqzJLZ5ShfGgOZdCMP4m4Vdf7lEx0Bdww6/vwaqIcmQOv76 r73PmcdgKlOb2B9jTr4QoHPNZ/k1BZETAPky/DUigRwipY7MJHOo120NqcfvO0QbR5sa 5ExtHcVcQ/u63ZovV1tHyxsJgD888vg668HIFnoVRlE2sqK662SYp8TZDPZ6toEqxlEf GKy5rJg4FgddjnzTh/txAtGdh05xEd8+u/4TiAGHuN1GFSodA58u3t+S4KR0V/GS7FKg hcFLLqWEnkefJyYCx2Xy6nFXrAJxq5OJX/IbkiGxcPV6hudOF+wIx13VVxzvqPFJlCam fr5A== X-Forwarded-Encrypted: i=1; AJvYcCV8j+lqZbqO2UKOhMIvfC6bV9xtKf6f34mnGczFgcwXg/6oiMl+yYMEuyYT7Afbophs3sHx0hqu0Mkq@vger.kernel.org X-Gm-Message-State: AOJu0Yy75EZNbJDxfDXRKIqelKN+qxvIqhpjA5Sb4KLaIKukg88xZaKf 2I7jrOGt1MImP0Kf411j62lZgF/aZiE4doLlb2QOOwhYNJ17fwQu8WmgYFGF06U5Rk8ySLkVduV 9Qw== X-Google-Smtp-Source: AGHT+IEHmTL+CF0lwT1L0/srW8AumAgUZ3z+xXpT6s6F1mY7sAC4x3PdYh5uuuYsI/CKxaNz7ipzx99kqDg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ed0b:0:b0:e17:8e73:866c with SMTP id 3f1490d57ef6-e28fe421ddfmr136730276.10.1728584863963; Thu, 10 Oct 2024 11:27:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-82-seanjc@google.com> Subject: [PATCH v13 81/85] KVM: Make kvm_follow_pfn.refcounted_page a required field From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Now that the legacy gfn_to_pfn() APIs are gone, and all callers of hva_to_pfn() pass in a refcounted_page pointer, make it a required field to ensure all future usage in KVM plays nice. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b2c8d429442d..a483da96f4be 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2834,8 +2834,7 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, pfn = page_to_pfn(page); } - if (kfp->refcounted_page) - *kfp->refcounted_page = page; + *kfp->refcounted_page = page; return pfn; } @@ -2986,6 +2985,9 @@ kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) might_sleep(); + if (WARN_ON_ONCE(!kfp->refcounted_page)) + return KVM_PFN_ERR_FAULT; + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; From patchwork Thu Oct 10 18:24:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830747 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79B1B21B45C for ; Thu, 10 Oct 2024 18:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584869; cv=none; b=DCbsClYM1JyLOqJV5x4b2AZt+jvU0pthB5ulnQWx7ZF1H7A3TZ6A1avlAoqHVLHdQLVXGCcQ8GCF0M/6PI8kteuBEDvhxscH4rZiH0OkfoMOeK1n9VRATIKpW0gAkfv6/4m5X6o2Anf3Dn79HKT4WasQ8g6FCZfzsAzJ+jDpjl8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584869; c=relaxed/simple; bh=PnrPzaCfKEgiP1btOoqWv8SGAKP2OnxCXZvE+imIcuc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Vt9kouPJy7S6NYYDPehzF7xGccgoPmEyu75gaL2LXgF71vj0PjpMxJQnWe/mUh43Ht6h0Q3yci4OGIFBXhFbEPkQbiu92UvVTUNR7+ehfVX7tl57sm2IbEg3Zk4SOEqtGbQ4FocWl8f9RCHRnKIvU8JGCn/C7Z4uaKM8c7b+nps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=AchV74I+; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AchV74I+" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2acec0109so21932727b3.3 for ; Thu, 10 Oct 2024 11:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584866; x=1729189666; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=AchV74I+JHnLrzKw1CnNCcRL41wNv//gHQc4NBTbfpjxRDy9PpJE1B/SqMD9myx1R0 mVHR+P8bSOu3/Oz7lQ1hgUtN9RY36X8gOczJcSEzTD6A4XAuLIdAiocO1EWVeSzm/4NK iNG3yhnElAt6f+kb1qQgRYpbE90dKeTPD0ImYwKFpZwH+xttYCi4J3DbFBX+y8+g6m/J qSa67DktV6zF5OIY/4SFr42+hJi/UbFrb3E+c81xKI47QPmXP/2WartrnYvjKaDYHUd5 jRzI+lw3Ax3tYbLNUtXYTLdiL9GbDm/+hYmfVoG+Cx077QveQVOlUtcb0lxUp1GhD1cL Ji/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584866; x=1729189666; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=uVKxK/QD47JRq0XwJuj4kkErcmAQYKq7brWRJ7AeHpJuAkVwwdd44TbXzlLEbTqpKh 0KUbUV8Nwew27VoGyd9CA7NhtNQLlv0/Up1tTmVBc362Jy08UdbO6rxWj5Z9eYfDMC1m sKyevPtFLWxPsbNPzb6o3L1/lHwzEf6C/NX1p4mfYnnNNNHiqCKAg+tNTdRxMGOnBYro K3UBGB7V79hSp+obPmPqQGDxr3BYv+9YZvpjVvaXFOFvoMakC2/BJrPgIuiZWfq6aw1B NcAWNNT9UeB+pBTYwsLaZ4b6WMZ07xxf8tNKwc34oJBdMedvwz05EdlK9xaPDzVQ2EsM ujhw== X-Forwarded-Encrypted: i=1; AJvYcCXhv6QAtnm9xi2Ofy2/I2bXJWGyDKwcyNJ2iJUHR6ozmbPWQ08mhEsARM25Vt5dVs0/H2appURhEcp3@vger.kernel.org X-Gm-Message-State: AOJu0Yxcjxx7YNfpvWEwo6r1fGPrZG5F3ToHW+jkBCuPhmoMtIzXy+8c mKcowNp5eB1zhE91bB0KYFO/FRiL2Wm8SDLPi9h2G79e5647WxqT7+7K0a/dYyrEp6ITcG5QXPh AfQ== X-Google-Smtp-Source: AGHT+IE49fRlqLzz+ddXUJ4fW719WQNIEaF+exc+KxAZEodOOT6YKCh59uZdX+xzyPwnVHJ15Mb6QwFZ+jU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:470f:b0:6e3:39e5:f0e8 with SMTP id 00721157ae682-6e339e5f461mr119227b3.6.1728584866570; Thu, 10 Oct 2024 11:27:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-83-seanjc@google.com> Subject: [PATCH v13 82/85] KVM: x86/mmu: Don't mark "struct page" accessed when zapping SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Don't mark pages/folios as accessed in the primary MMU when zapping SPTEs, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. And if KVM zaps a SPTe in response to an mmu_notifier, marking it accessed _after_ the primary MMU has decided to zap the page is likely to go unnoticed, i.e. odds are good that, if the page is being zapped for reclaim, the page will be swapped out regardless of whether or not KVM marks the page accessed. Dropping x86's use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 17 ----------------- arch/x86/kvm/mmu/tdp_mmu.c | 3 --- 2 files changed, 20 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5acdaf3b1007..55eeca931e23 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -559,10 +559,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) */ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) { - kvm_pfn_t pfn; u64 old_spte = *sptep; int level = sptep_to_sp(sptep)->role.level; - struct page *page; if (!is_shadow_present_pte(old_spte) || !spte_has_volatile_bits(old_spte)) @@ -574,21 +572,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) return old_spte; kvm_update_page_stats(kvm, level, -1); - - pfn = spte_to_pfn(old_spte); - - /* - * KVM doesn't hold a reference to any pages mapped into the guest, and - * instead uses the mmu_notifier to ensure that KVM unmaps any pages - * before they are reclaimed. Sanity check that, if the pfn is backed - * by a refcounted page, the refcount is elevated. - */ - page = kvm_pfn_to_refcounted_page(pfn); - WARN_ON_ONCE(page && !page_count(page)); - - if (is_accessed_spte(old_spte)) - kvm_set_pfn_accessed(pfn); - return old_spte; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 8aa0d7a7602b..91caa73a905b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -861,9 +861,6 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); - if (is_accessed_spte(iter.old_spte)) - kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); - /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. From patchwork Thu Oct 10 18:24:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830748 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE05121BAF3 for ; Thu, 10 Oct 2024 18:27:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584871; cv=none; b=L3Vq5ve3ryLr1zxsgLl2I6TFK/mXeNGjVdYS5y67zSg4UjwCxRmlJG5/3YJpOoVvOJ5QsXq5UjQ064upOAKC07lxp4bbozbNHbt2FgtNvp6ejBDCGkRBhsgnlKzwopBuiyRgbJ4wvXdnTKn1QNU6y+hmyo+O9MRqLCp+4YV3Va4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584871; c=relaxed/simple; bh=4ce0ST1uTErGaf3Fu8uQPVTe9dXk8k7UaRtdo8iN0BA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jhROojNrhnLcYadwO8CAdtDqJ+AvnNN8plH5amTn9qNzDdD2k1NiDJt9FCM5gW5Yp8EHVfgRfLRqqGB0cAzHLe6k6GcQ7RqQDGuQAnLBYrPr7kaFPGp1HY8oanWXRFI3zMBH5vo/NOkEnd/ri5GkEFM1fAsIHJDkzfFZ4PzSrN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LR0bZlhg; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LR0bZlhg" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-204e310e050so14844495ad.0 for ; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584868; x=1729189668; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=LR0bZlhgBqqlK/AxVHUnYDgdPVeVfg62ritjsHW4icLlp2VXJMdBEQpqQ2DjaOB4Vo BMk5u0CdNiJU47xxEKEBNDUg1VIzyKQFA0lEenO//nWQFutF6qqNsjqYWpd0KkluabCG scoWgGmVwMXCV2bJcO3DxVbQdljLpb0S7K+9zHipw5iopVLZ5O/Et9tquTm7UfUH/R3W Tzhi8J9rL8rJ6Zw+R2IdaOJPRXvC3k33WFcoS6zAsYYyRLTyXGYrquN21MhPyyOeIlNt k76iPpUznp7HqqlLd6wYwP+ZU1DkIHHgVTgrkh+gxlQpcl0jRX8rnB7ytNb8WRs+Wq5s 9Kuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584868; x=1729189668; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=MvHUOGaBFgAmhP86700VKYz/6Xzs2nn9pw4O0goOKqh+j3Sxzz2x+uivMiURtu7PwH RZGCPWBNdp0Qqtn59hMT0k8B5Bn/FT5yaEohcuP2hQ+F/4l7b40oT541Lnie+ztb27Bg J+4nQPPbCqGIWlH+Sg9zZOMcKSdPLTr/oYSqMr4usubnCXdAoteFs9K/yZLBpY02DvhE lYXmxYUNPNGkveNyE6brlsB/InDJ/nONkmCupC5nJ5UouAHb6Waaoq2VwLZAxl6rDH+i AsKg37XQIgWxMgLPchZPQihdplWUEZZTZUkrpkopiLkmBy6JI9E8TV7uzWqUfGuhWWth ctig== X-Forwarded-Encrypted: i=1; AJvYcCU9OHmfbShHiayGekSxWyXHppJItV9ogJoB+1bEr28GiJKsvERAhvfaGEduibNP7wG/EP9dyJf/OWYw@vger.kernel.org X-Gm-Message-State: AOJu0YySkQlftrc5zsTL5HmLiL8w2OQQHGrtosfw0Uh81U3AsC7EJH6m 2vowKFj3e7a9YU7US5hFqq/KkTdt6gzG0HAqWRxCo0tqZjtyBRV0g4LZxE+k5GpyJxd1Me28kC6 4vA== X-Google-Smtp-Source: AGHT+IHfwq/leOSDpGNmElHsEevN2NLhktzxL5WWYsNV7s51yqV4fIBatX1NQsHzbv29AfLNcoyCuLVeClM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e745:b0:20c:857b:5dcb with SMTP id d9443c01a7336-20c857b62ffmr639275ad.4.1728584868183; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-84-seanjc@google.com> Subject: [PATCH v13 83/85] KVM: arm64: Don't mark "struct page" accessed when making SPTE young From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Don't mark pages/folios as accessed in the primary MMU when making a SPTE young in KVM's secondary MMU, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. Dropping use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/include/asm/kvm_pgtable.h | 4 +--- arch/arm64/kvm/hyp/pgtable.c | 7 ++----- arch/arm64/kvm/mmu.c | 6 +----- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 03f4c3d7839c..aab04097b505 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -674,10 +674,8 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size); * * If there is a valid, leaf page-table entry used to translate @addr, then * set the access flag in that entry. - * - * Return: The old page-table entry prior to setting the flag, 0 on failure. */ -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); /** * kvm_pgtable_stage2_test_clear_young() - Test and optionally clear the access diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b11bcebac908..40bd55966540 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1245,19 +1245,16 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size) NULL, NULL, 0); } -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) { - kvm_pte_t pte = 0; int ret; ret = stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, - &pte, NULL, + NULL, NULL, KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED); if (!ret) dsb(ishst); - - return pte; } struct stage2_age_data { diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 4054356c9712..e2ae9005e333 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1706,18 +1706,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* Resolve the access fault by making the page young again. */ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) { - kvm_pte_t pte; struct kvm_s2_mmu *mmu; trace_kvm_access_fault(fault_ipa); read_lock(&vcpu->kvm->mmu_lock); mmu = vcpu->arch.hw_mmu; - pte = kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); + kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); read_unlock(&vcpu->kvm->mmu_lock); - - if (kvm_pte_valid(pte)) - kvm_set_pfn_accessed(kvm_pte_to_pfn(pte)); } /** From patchwork Thu Oct 10 18:24:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830749 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3F4421C173 for ; Thu, 10 Oct 2024 18:27:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584873; cv=none; b=TBrt5AHvqP0zmGuC1+E0Nhq8948xOlNhoIrN0szFyS1jMvnBGvLQhB7Qm3rMUMb3OCU9Lpm+9Khl5NScZzzzJ9qthUYdZzCRyswBPteev7toJOoFx3wpmhvKH83SrKfcRZy5IiLz487sNhrjBiwPXZpTyFN376dzupV1J2ym9Z8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584873; c=relaxed/simple; bh=9yDh4yVl+Y5ECwKMlXFEDQ/uzUa/qYgKcVRj3mqEjek=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qArDXe8TKiSIw3aOv+aZYsrpYq266yd7oU9qar32DXndmMcSUTuJpcdeQSVSxlTdyAk69pV277AgZ8aUYhGEICPlJVgQKR8FZ5s/eEq4mGHuQYWThF2wPGerdIjw5utIazVVMymqzfM9XQQOG3o3b2F6BBQZKPBA/l1EgyTuU/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1AdnNrnI; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1AdnNrnI" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2e4874925so21273767b3.1 for ; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584870; x=1729189670; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=1AdnNrnIXLgNymQtK7DilLGthF1JsfkTOKsD8zYlozlgDyVnSyLY6rQs1ch4aUN54v fTlIRF6YDP2svUdnyjp88hSQcfOn6wvrAuXYoJPHVBwm+zBiaq88Wsg3ie8JPbOZVm6v +FdIhr7f29Y4owHrhAD9S/zkPO3/FI5JCSjOek0MkvkxKw6zlH8yk7tESmKbM59fwOxY truIpSu4JHQroLJXqzMSmcI2c+TlvXrk0XyyyXlQcN9BQPbIRRXz6PSG0liD/fpQxsWp 5Y2cwNpf+u5nZPuU2t9NEaeYfk9Jawe11Dvm4yVWD7vMf94BZ/9HtAVH1x+u2rygpxWQ 2Xsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584870; x=1729189670; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=XkSWoiBLLwhgiZ6IRcjIOaS1LE9cvXBJoMzhaCIT/BZN85XTF0FBmkzOGIaFBRmmXk jWZcCNb1UN6FQ41urS6wyBCBtv+laCxRTTC7oxHaMLfThi2/P6eryhLgcThwylReqlHn kzuftVBQtS2oj+OFHaOqA4ovwzWjZcgKO6B2l5bORrbpeVziXP6jTULxLVgIdQgkWqt3 P2QQDNZTdleDwomthMWpbU4BKF6B1k1XN93qpJKylH3hXBNHikolDDidR2BinMqX5Ha/ WGPC22JsQGrtgXxrN2P/HaBgdvAHmgZqyIJZKt4hyHqvynyvYq3u4GczycGHysNO6KSf SVyg== X-Forwarded-Encrypted: i=1; AJvYcCUGpSqtYlAvxagEF3XkhRkqLyyMYzoqS4yHhmYFwYX99Ju39sQHsj0erdRUQi9/ft0MluGzLIOI0hWX@vger.kernel.org X-Gm-Message-State: AOJu0YzrTr7X3gw4XY3BekES7axLWEKv1eLAyowXfrYPpbZIn0tyQrJQ M/VFTmhW2Tt1Ta5bl3DCx3wPjlSyVfci1X9WmG8F6985KV3N0WDZdO9Ogqz0qls0V7TwOAENH+1 9ow== X-Google-Smtp-Source: AGHT+IHhmEpfdjhYs/g96WpbaMl3fao5EtfU192MAf3yyGbJNoRiDL4cTPbNYR1fq37SA+x5liddjIzSKiU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2e86:b0:6dd:bcce:7cd4 with SMTP id 00721157ae682-6e3221404ddmr516647b3.2.1728584870148; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-85-seanjc@google.com> Subject: [PATCH v13 84/85] KVM: Drop APIs that manipulate "struct page" via pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Remove all kvm_{release,set}_pfn_*() APIs now that all users are gone. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 5 ---- virt/kvm/kvm_main.c | 55 ---------------------------------------- 2 files changed, 60 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4a1eaa40a215..d045f8310a48 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,11 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } -void kvm_release_pfn_clean(kvm_pfn_t pfn); -void kvm_release_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_accessed(kvm_pfn_t pfn); - int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index a483da96f4be..396ca14f18f3 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3164,61 +3164,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); -void kvm_release_pfn_clean(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); - -void kvm_release_pfn_dirty(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_dirty(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty); - -/* - * Note, checking for an error/noslot pfn is the caller's responsibility when - * directly marking a page dirty/accessed. Unlike the "release" helpers, the - * "set" helpers are not to be used when the pfn might point at garbage. - */ -void kvm_set_pfn_dirty(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_dirty(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); - -void kvm_set_pfn_accessed(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_accessed(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_accessed); - static int next_segment(unsigned long len, int offset) { if (len > PAGE_SIZE - offset) From patchwork Thu Oct 10 18:24:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830750 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4CF721D166 for ; Thu, 10 Oct 2024 18:27:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584875; cv=none; b=AeCF5P4/CyyVSWvf/WUs+3WVewSckdkaNTdXQ3K/jMNZNW3UMhGgj3wdlBeONl7+R7feGa1iXMXEvKST3sE/9XFEegGtrvzLnqh17YwH+ejhFAzH1BIvt4qiyb8+rHbwGXa1mUj1GP0s1mGvZnh1xrGaE+UxVduJOfk+2yFxFCA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584875; c=relaxed/simple; bh=aNI4I+LkKsxzC1YKs2vCoD7b+5doZynW6Blg4vQwwvc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dALTs2r53hC50+rZA5mcKfNJq6WtN4ApU7rv/sibouZhFYPoUtCXsO5iJ2Up6V1g/FObuPq5+qSXVVTHKuzfJH+CcUBhgHg3LUwZcY8XAD7k3SbxfjDA4PJZcZSvnL+a6Fd02WTKcaReEMb+a54uYuIWTkf4JcHm+TwrPjVygbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xyu1Nn/r; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xyu1Nn/r" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7d4f9974c64so900243a12.1 for ; Thu, 10 Oct 2024 11:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584873; x=1729189673; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=xyu1Nn/rp+FyjGRlvaySHpdLv16ogLwofUZlPSL88LrsPtUsdHxPdI4aMAZWpZ+LCb G47QycBP2achTmuGd1EXNbbjK4pjlf7ihMkj5vlGqDxw6OCVDaNJvwxpCB4vy6cQt+eC bPNyYrqL1IWM+djQgloaSwHlTl/m9PsGRY+UQiNbQLuX8csAdsfFUDyRbBcmemNyquwH u56PAc5JRwdiVO7tt7yODdJN/pEha8mrjjtisATQgI2/uYcwTl1QzP6LKGp+KqXDTb7X W3csPJXsIxIBK/YbHvH2KWvDbUJR0U0jHjWARSRlv5XaOYevRMa1tcC3LSfpUG2LIezf k0Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584873; x=1729189673; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=PwDflMX1QoQbLkvlGotQiVPSnS3jBwHeenOFUvPGCQjaRI3zIZAe7XLhHXQTrNvgr+ BNpA0A1UWNuERmTqssKbZC/cJBYSkU+JVd7FowSVG2cDD+XACh83tXZUjWqETBzVeLL4 GXqPLqhCQXkfp0rf3C65ti0CPbeQq95j6ECjlpswtsuSyKwV6zbhz8KG/MXvGFRazGbZ d8COrdizGJu0JovkXnb4FKYIaehuozmUTvN7dzV2/drW9ugZHaZvUwVbMs8gxM04LS/3 zFLdxq7wCp9Az78pdK39PXOcP7fdT8sa5f3DFK5tI77GQ4k6bCWCk3ssqccf5lQbXNnG NBEA== X-Forwarded-Encrypted: i=1; AJvYcCXJEmDG1L+57doDxW4WuB0htW/TUnq4bPhfMHRyl7oVifwOAhz7z+52ZD90lv9+bKY/TPapO7t6pBMF@vger.kernel.org X-Gm-Message-State: AOJu0YyAZ6+n167A9uht0MV9SMDSHP2MICm4cOlanM1wkl0j7mignaYd 4kE+8iEnjuc6yU2zjJP5XRqRQLAvSvk9fGFby1wDuLrim5gkpOQnPTlrm546EGJRz9q/Cy+JfN5 fQg== X-Google-Smtp-Source: AGHT+IHZp/jcJ6VF/U4r8Ds8cBqCm7oOG+WyWj8b167iLxGpwwfpynwEnbVm0Ou2T4AqluEw0PW/YyS/yKE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e715:0:b0:7d5:d088:57d4 with SMTP id 41be03b00d2f7-7ea535b6b4fmr26a12.9.1728584872016; Thu, 10 Oct 2024 11:27:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-86-seanjc@google.com> Subject: [PATCH v13 85/85] KVM: Don't grab reference on VM_MIXEDMAP pfns that have a "struct page" From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones Now that KVM no longer relies on an ugly heuristic to find its struct page references, i.e. now that KVM can't get false positives on VM_MIXEDMAP pfns, remove KVM's hack to elevate the refcount for pfns that happen to have a valid struct page. In addition to removing a long-standing wart in KVM, this allows KVM to map non-refcounted struct page memory into the guest, e.g. for exposing GPU TTM buffers to KVM guests. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 3 -- virt/kvm/kvm_main.c | 75 ++-------------------------------------- 2 files changed, 2 insertions(+), 76 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index d045f8310a48..02f0206fd2dc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1730,9 +1730,6 @@ void kvm_arch_sync_events(struct kvm *kvm); int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu); -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn); -bool kvm_is_zone_device_page(struct page *page); - struct kvm_irq_ack_notifier { struct hlist_node link; unsigned gsi; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 396ca14f18f3..b1b10dc408a0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -160,52 +160,6 @@ __weak void kvm_arch_guest_memory_reclaimed(struct kvm *kvm) { } -bool kvm_is_zone_device_page(struct page *page) -{ - /* - * The metadata used by is_zone_device_page() to determine whether or - * not a page is ZONE_DEVICE is guaranteed to be valid if and only if - * the device has been pinned, e.g. by get_user_pages(). WARN if the - * page_count() is zero to help detect bad usage of this helper. - */ - if (WARN_ON_ONCE(!page_count(page))) - return false; - - return is_zone_device_page(page); -} - -/* - * Returns a 'struct page' if the pfn is "valid" and backed by a refcounted - * page, NULL otherwise. Note, the list of refcounted PG_reserved page types - * is likely incomplete, it has been compiled purely through people wanting to - * back guest with a certain type of memory and encountering issues. - */ -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn) -{ - struct page *page; - - if (!pfn_valid(pfn)) - return NULL; - - page = pfn_to_page(pfn); - if (!PageReserved(page)) - return page; - - /* The ZERO_PAGE(s) is marked PG_reserved, but is refcounted. */ - if (is_zero_pfn(pfn)) - return page; - - /* - * ZONE_DEVICE pages currently set PG_reserved, but from a refcounting - * perspective they are "normal" pages, albeit with slightly different - * usage rules. - */ - if (kvm_is_zone_device_page(page)) - return page; - - return NULL; -} - /* * Switches to specified vcpu, until a matching vcpu_put() */ @@ -2804,35 +2758,10 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, if (kfp->map_writable) *kfp->map_writable = writable; - /* - * FIXME: Remove this once KVM no longer blindly calls put_page() on - * every pfn that points at a struct page. - * - * Get a reference for follow_pte() pfns if they happen to point at a - * struct page, as KVM will ultimately call kvm_release_pfn_clean() on - * the returned pfn, i.e. KVM expects to have a reference. - * - * Certain IO or PFNMAP mappings can be backed with valid struct pages, - * but be allocated without refcounting, e.g. tail pages of - * non-compound higher order allocations. Grabbing and putting a - * reference to such pages would cause KVM to prematurely free a page - * it doesn't own (KVM gets and puts the one and only reference). - * Don't allow those pages until the FIXME is resolved. - * - * Don't grab a reference for pins, callers that pin pages are required - * to check refcounted_page, i.e. must not blindly release the pfn. - */ - if (map) { + if (map) pfn = map->pfn; - - if (!kfp->pin) { - page = kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; - } - } else { + else pfn = page_to_pfn(page); - } *kfp->refcounted_page = page;